From noreply@sourceforge.net Mon Oct 1 09:10:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 01:10:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-466752 ] "deref" error in _PyBuffer_FromObject Message-ID: Bugs item #466752, was opened at 2001-10-01 01:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466752&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: "deref" error in _PyBuffer_FromObject Initial Comment: The "deref" code in function _PyBuffer_FromObject, file bufferobject.c is an error: ============== /* if the base object is another buffer, then "deref" it */ if ( PyBuffer_Check(base) ) base = ((PyBufferObject*)base)->b_base; return _PyBuffer_FromMemory(base, (char *)p + offset, size, readonly); ============== It is possible, that "deref"'ed base is NULL. In this case the new object will have no reference to any base object. Example (in C): ================ base_buffer=PyBuffer_New(100); derived_buffer=PyBuffer_FromObject(base_buffer,0,50); Py_DECREF(base_buffer); ================ And derived_buffer memory pointer points to nowhere. I think, the right code in _PyBuffer_FromObject would be: ============== /* if the base object is another buffer, then "deref" it */ if ( PyBuffer_Check(base) && (((PyBufferObject*)base)->b_base != NULL)) base = ((PyBufferObject*)base)->b_base; return _PyBuffer_FromMemory(base, (char *)p + offset, size, readonly); ============== Victor Sologoubov VSologoubov@ibs.ru ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466752&group_id=5470 From noreply@sourceforge.net Mon Oct 1 11:17:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 03:17:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-435455 ] Python 2.0.1c1 fails to build on RH7.1 Message-ID: Bugs item #435455, was opened at 2001-06-22 06:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=435455&group_id=5470 Category: Distutils Group: Platform-specific >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Ole H. Nielsen (ohnielse) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.0.1c1 fails to build on RH7.1 Initial Comment: Building Python 2.0.1c1 on a RedHat 7.1 (2.4.2-2 on i586) fails at this point: cd Modules; make OPT="-g -O2 -Wall -Wstrict-prototypes -fPIC" VERSION="2.0" \ prefix="/usr/local" exec_prefix="/usr/local" \ sharedmods make[1]: Entering directory `/scratch/ohnielse/Python-2.0.1/Modules' gcc -fpic -g -O2 -Wall -Wstrict-prototypes -fPIC -I./../Include -I.. -DHAVE_CONFIG_H -c ./bsddbmodule.c ./bsddbmodule.c: In function `newdbhashobject': ./bsddbmodule.c:55: `HASHINFO' undeclared (first use in this function) ./bsddbmodule.c:55: (Each undeclared identifier is reported only once ./bsddbmodule.c:55: for each function it appears in.) ./bsddbmodule.c:55: parse error before `info' ./bsddbmodule.c:60: `info' undeclared (first use in this function) ./bsddbmodule.c:71: warning: implicit declaration of function `dbopen' ./bsddbmodule.c:71: warning: assignment makes pointer from integer without a cast ./bsddbmodule.c: In function `newdbbtobject': ./bsddbmodule.c:100: `BTREEINFO' undeclared (first use in this function) ./bsddbmodule.c:100: parse error before `info' ./bsddbmodule.c:105: `info' undeclared (first use in this function) ./bsddbmodule.c:118: warning: assignment makes pointer from integer without a cast ./bsddbmodule.c: In function `newdbrnobject': ./bsddbmodule.c:147: `RECNOINFO' undeclared (first use in this function) ./bsddbmodule.c:147: parse error before `info' ./bsddbmodule.c:152: `info' undeclared (first use in this function) ./bsddbmodule.c:164: warning: assignment makes pointer from integer without a cast ./bsddbmodule.c: In function `bsddb_dealloc': ./bsddbmodule.c:202: too few arguments to function ./bsddbmodule.c: In function `bsddb_length': ./bsddbmodule.c:232: structure has no member named `seq' ./bsddbmodule.c:233: `R_FIRST' undeclared (first use in this function) ./bsddbmodule.c:235: structure has no member named `seq' ./bsddbmodule.c:236: `R_NEXT' undeclared (first use in this function) ./bsddbmodule.c:229: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_subscript': ./bsddbmodule.c:265: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:265: too few arguments to function ./bsddbmodule.c: In function `bsddb_ass_sub': ./bsddbmodule.c:307: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:307: too few arguments to function ./bsddbmodule.c:330: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:330: too few arguments to function ./bsddbmodule.c: In function `bsddb_close': ./bsddbmodule.c:357: too few arguments to function ./bsddbmodule.c: In function `bsddb_keys': ./bsddbmodule.c:386: structure has no member named `seq' ./bsddbmodule.c:386: `R_FIRST' undeclared (first use in this function) ./bsddbmodule.c:407: structure has no member named `seq' ./bsddbmodule.c:407: `R_NEXT' undeclared (first use in this function) ./bsddbmodule.c:376: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_has_key': ./bsddbmodule.c:440: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:440: too few arguments to function ./bsddbmodule.c: In function `bsddb_set_location': ./bsddbmodule.c:466: structure has no member named `seq' ./bsddbmodule.c:466: `R_CURSOR' undeclared (first use in this function) ./bsddbmodule.c:453: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_seq': ./bsddbmodule.c:503: structure has no member named `seq' ./bsddbmodule.c:489: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_next': ./bsddbmodule.c:531: `R_NEXT' undeclared (first use in this function) ./bsddbmodule.c: In function `bsddb_previous': ./bsddbmodule.c:536: `R_PREV' undeclared (first use in this function) ./bsddbmodule.c: In function `bsddb_first': ./bsddbmodule.c:541: `R_FIRST' undeclared (first use in this function) ./bsddbmodule.c: In function `bsddb_last': ./bsddbmodule.c:546: `R_LAST' undeclared (first use in this function) make[1]: *** [bsddbmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-01 03:17 Message: Logged In: YES user_id=21627 Since it is unlikely that we'll get further analysis on the cause of this problem, I'm closing the report now. Please try one of the 2.2 prereleases; if problems remain, please report them in a new bug report. ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-09-06 05:25 Message: Logged In: YES user_id=34209 Okay, unassigning. Why I had it ? unsure. It probably dates back from the time I still had a RedHat box. I swapped them all with Debian and I live a much happier life as a result :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:25 Message: Logged In: YES user_id=6380 Can someone see if this is still a problem with current CVS (or even with 2.2a2)? I believe we've added another layer of patchwork around the bsddb compilation in setup.py. Why does Thomas own this item? Thomas, if you don't want it, feel free to unassign it. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2001-07-10 01:30 Message: Logged In: YES user_id=43607 I have no trouble building the current CVS version on RedHat 7.1. However, when I try the gcc command given in the error message, I get much the same errors. The deciding difference was: setup.py (part of the standard current build process) adds -DHAVE_DB_185_H=1 to the compiler command line. ---------------------------------------------------------------------- Comment By: Ole H. Nielsen (ohnielse) Date: 2001-07-09 07:51 Message: Logged In: YES user_id=27232 twouters wrote: > I suspect the problem here is that you don't have the > appropriate -devel RPM installed. I think you need > 'db3-devel', but 'db2-devel', 'db1-devel' or 'db-devel' > might work too. Sorry, no. These RPMs seem to be installed by default on RedHat 7.1: # rpm -qa | grep devel | grep db db1-devel-1.85-5 db2-devel-2.4.14-5 gdbm-devel-1.8.0-5 db3-devel-3.1.17-7 It would be nice if a Python developer could gain access to a RedHat 7.1 machine to sort this problem out :-) ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-07-09 07:06 Message: Logged In: YES user_id=34209 I suspect the problem here is that you don't have the appropriate -devel RPM installed. I think you need 'db3-devel', but 'db2-devel', 'db1-devel' or 'db-devel' might work too. Can you confirm that you have none of those installed ? I think what happens is that distutils can't find an include header to include, so it assumes the DB implementation is included in libc, instead. (Changing catagory to distutils on that assumption.) ---------------------------------------------------------------------- Comment By: Ole H. Nielsen (ohnielse) Date: 2001-06-26 14:52 Message: Logged In: YES user_id=27232 loewis wrote: > I have asked you to report things, not to change things. > It is not easy to repeat for me, as I don't have Redhat 7.1. Sorry about the confusion. My report is for RedHat 7.1: 1. Untar the distribution 2.0.1c1 2. ./configure 3. make The errors reported occur. I have changed nothing. I don't know what bsddb is, nor why it's getting compiled. Hopefully, someone knowing Python 2.0.1 and having access to a RedHat 7.1 box could look into the problem. At this point Python building seems broken on RH7.1 :-( ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-06-26 00:02 Message: Logged In: YES user_id=21627 I have asked you to report things, not to change things. It is not easy to repeat for me, as I don't have Redhat 7.1. ---------------------------------------------------------------------- Comment By: Ole H. Nielsen (ohnielse) Date: 2001-06-25 15:20 Message: Logged In: YES user_id=27232 loewis wrote: > Please report the following things: > - the line in Setup that you activated to enable > compilation of bsddb > - the exact version of the bsddb RPM package that provides > db.h > - whether or not this packages includes a file db_185.h Sorry, I didn't change ANYTHING ! I was trying a vanilla build on RedHat 7.1 ! Should be easy to repeat... Maybe the build scripts make some incorrect choices on RedHat 7.1 ? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-06-23 13:58 Message: Logged In: YES user_id=21627 Please report the following things: - the line in Setup that you activated to enable compilation of bsddb - the exact version of the bsddb RPM package that provides db.h - whether or not this packages includes a file db_185.h ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=435455&group_id=5470 From noreply@sourceforge.net Mon Oct 1 13:17:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 05:17:37 -0700 Subject: [Python-bugs-list] [ python-Bugs-466804 ] "x.__type__" in object.__init__.__doc__ Message-ID: Bugs item #466804, was opened at 2001-10-01 05:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466804&group_id=5470 Category: Type/class unification Group: None Status: Open Resolution: None Priority: 5 Submitted By: Walter Dörwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: "x.__type__" in object.__init__.__doc__ Initial Comment: The docstring for object.__init__ says: "x.__init__(...) initializes x; see x.__type__.__doc__ for signature". This should probably be "x.__class__" instead of "x.__type__". ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466804&group_id=5470 From noreply@sourceforge.net Mon Oct 1 14:18:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 06:18:01 -0700 Subject: [Python-bugs-list] [ python-Bugs-466804 ] "x.__type__" in object.__init__.__doc__ Message-ID: Bugs item #466804, was opened at 2001-10-01 05:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466804&group_id=5470 Category: Type/class unification Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Walter Dörwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) >Summary: "x.__type__" in object.__init__.__doc__ Initial Comment: The docstring for object.__init__ says: "x.__init__(...) initializes x; see x.__type__.__doc__ for signature". This should probably be "x.__class__" instead of "x.__type__". ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-01 06:18 Message: Logged In: YES user_id=6380 Thanks! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466804&group_id=5470 From noreply@sourceforge.net Mon Oct 1 14:22:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 06:22:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-466752 ] "deref" error in _PyBuffer_FromObject Message-ID: Bugs item #466752, was opened at 2001-10-01 01:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466752&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) >Summary: "deref" error in _PyBuffer_FromObject Initial Comment: The "deref" code in function _PyBuffer_FromObject, file bufferobject.c is an error: ============== /* if the base object is another buffer, then "deref" it */ if ( PyBuffer_Check(base) ) base = ((PyBufferObject*)base)->b_base; return _PyBuffer_FromMemory(base, (char *)p + offset, size, readonly); ============== It is possible, that "deref"'ed base is NULL. In this case the new object will have no reference to any base object. Example (in C): ================ base_buffer=PyBuffer_New(100); derived_buffer=PyBuffer_FromObject(base_buffer,0,50); Py_DECREF(base_buffer); ================ And derived_buffer memory pointer points to nowhere. I think, the right code in _PyBuffer_FromObject would be: ============== /* if the base object is another buffer, then "deref" it */ if ( PyBuffer_Check(base) && (((PyBufferObject*)base)->b_base != NULL)) base = ((PyBufferObject*)base)->b_base; return _PyBuffer_FromMemory(base, (char *)p + offset, size, readonly); ============== Victor Sologoubov VSologoubov@ibs.ru ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-01 06:22 Message: Logged In: YES user_id=6380 Your example C code is simply wrong. This has been brought up before (is there an example on the web somewhere that does this???). Do not use the buffer API to allocate memory. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466752&group_id=5470 From noreply@sourceforge.net Mon Oct 1 17:33:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 09:33:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-420851 ] Missing docs for iteration support. Message-ID: Bugs item #420851, was opened at 2001-05-02 13:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=420851&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 6 Submitted By: Fred L. Drake, Jr. (fdrake) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Missing docs for iteration support. Initial Comment: Documentation is needed for the iteration support. At this time, the following things are missing docs: - iter() function - {}.iteritems(), .iterkeys(), .itervalues() - tp_iter/tp_iternext - __iter__() (ref. manual -- "Object Customization") If you can think of other aspects of this that I've missed, please note them as comments on this bug report so I can address these (or at least keep track of these) in a single place. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-01 09:33 Message: Logged In: YES user_id=3066 __iter__() documented in Doc/ref/ref3.tex revision 1.75. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-09-06 12:05 Message: Logged In: YES user_id=3066 Documented the iter() function in Doc/lib/libfuncs.tex revision 1.84. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-08-20 09:46 Message: Logged In: YES user_id=12800 Setting to priority 6 so that it won't hold up the 2.2a2 release. Although it would be good to have these docs, remember that priorities >= 7 must be fixed before the release can happen. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-05-25 06:52 Message: Logged In: YES user_id=3066 Docs for {}.iteritems(), {}.iterkeys(), and {}.itervalues() checked in as Doc/lib/libstdtypes.tex revision 1.59. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=420851&group_id=5470 From noreply@sourceforge.net Mon Oct 1 17:52:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 09:52:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-466894 ] property <-> PyGetSetDef Message-ID: Bugs item #466894, was opened at 2001-10-01 09:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466894&group_id=5470 Category: Type/class unification Group: None Status: Open Resolution: None Priority: 5 Submitted By: Walter Dörwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: property <-> PyGetSetDef Initial Comment: Now that "getset" has been renamed to "property" maybe "PyGetSetDef" should be renamed to "PyPropertyDef"? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466894&group_id=5470 From noreply@sourceforge.net Mon Oct 1 18:25:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 10:25:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-466907 ] OSX Python cannot run in console Message-ID: Bugs item #466907, was opened at 2001-10-01 10:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 Category: None Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Wight (schwa) Assigned to: Nobody/Anonymous (nobody) Summary: OSX Python cannot run in console Initial Comment: OS: Mac OS X 10.1 HW: Verified on G4/733 & PowerBook G4/400Version: 'Unix' Python 2.2a4 (also seen on 2.2a3) Configured: --with-suffix=.exe --without-toolbox-glue Python cannot run on a Mac OS X machine when the user is logged into console mode (using ">console" as login name). Download .tar.gz, configure python as shown, make, make install. Log out of Mac OS X. Log in as console user: use ">console" as login name. Run python interactively from the shell... Python will fail to run and emit these errors: KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorInvalidConnection: CGSGetEventPort: Invalid connection I believe Python is trying to connect to the CoreGraphics server. Why? I don't know (esp. as I've NOT compiled the toolbox glue). Perhaps Python is linking to a Carbon library and the library is automatically trying to open a connection. My makefile skills are not good enough to enable me to check. Python compiled on OS X without the toolbox glue should essentially be identical to any other 'unix' app compiled for Mac OS X. Doesn't seem to be that way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 From noreply@sourceforge.net Mon Oct 1 18:36:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 10:36:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-466894 ] property <-> PyGetSetDef Message-ID: Bugs item #466894, was opened at 2001-10-01 09:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466894&group_id=5470 Category: Type/class unification Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Walter Dörwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: property <-> PyGetSetDef Initial Comment: Now that "getset" has been renamed to "property" maybe "PyGetSetDef" should be renamed to "PyPropertyDef"? ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-01 10:36 Message: Logged In: YES user_id=6380 I don't think so -- this doesn't map directly onto properties. The property type takes up to three callables and a doc string. The tp_getset mechanism takes two C functions. While this is very similar, I think it's better to keep the C-level names different to avoid confusion. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466894&group_id=5470 From noreply@sourceforge.net Mon Oct 1 18:37:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 10:37:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-466907 ] OSX Python cannot run in console Message-ID: Bugs item #466907, was opened at 2001-10-01 10:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 >Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Wight (schwa) >Assigned to: Jack Jansen (jackjansen) Summary: OSX Python cannot run in console Initial Comment: OS: Mac OS X 10.1 HW: Verified on G4/733 & PowerBook G4/400Version: 'Unix' Python 2.2a4 (also seen on 2.2a3) Configured: --with-suffix=.exe --without-toolbox-glue Python cannot run on a Mac OS X machine when the user is logged into console mode (using ">console" as login name). Download .tar.gz, configure python as shown, make, make install. Log out of Mac OS X. Log in as console user: use ">console" as login name. Run python interactively from the shell... Python will fail to run and emit these errors: KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorInvalidConnection: CGSGetEventPort: Invalid connection I believe Python is trying to connect to the CoreGraphics server. Why? I don't know (esp. as I've NOT compiled the toolbox glue). Perhaps Python is linking to a Carbon library and the library is automatically trying to open a connection. My makefile skills are not good enough to enable me to check. Python compiled on OS X without the toolbox glue should essentially be identical to any other 'unix' app compiled for Mac OS X. Doesn't seem to be that way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 From noreply@sourceforge.net Mon Oct 1 18:42:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 10:42:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-465467 ] os.unsetenv not implemented Message-ID: Bugs item #465467, was opened at 2001-09-26 18:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465467&group_id=5470 Category: Python Library Group: Feature Request >Status: Deleted >Resolution: Duplicate Priority: 5 Submitted By: Naris Siamwalla (naris) Assigned to: Nobody/Anonymous (nobody) Summary: os.unsetenv not implemented Initial Comment: if os.putenv is implemented, os.unsetenv should be, too. ---------------------------------------------------------------------- >Comment By: Naris Siamwalla (naris) Date: 2001-10-01 10:42 Message: Logged In: YES user_id=67995 dup of 460805 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-30 14:43 Message: Logged In: YES user_id=21627 A patch for this bug is under development in http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=460805 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465467&group_id=5470 From noreply@sourceforge.net Mon Oct 1 20:56:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 12:56:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-466125 ] PyLong_AsLongLong works for any integer Message-ID: Bugs item #466125, was opened at 2001-09-28 09:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466125&group_id=5470 Category: Python Interpreter Core Group: Feature Request Status: Closed Resolution: Fixed Priority: 5 Submitted By: Tom Epperly (tepperly) Assigned to: Tim Peters (tim_one) Summary: PyLong_AsLongLong works for any integer Initial Comment: It would be nice if PyLong_AsLongLong worked more like PyInt_AsLong. When the value can be coerced into an integer, it should be. The current behavior is that PyLong_AsLongLong only works (without raising an exception) on instances of a PyLongObject. I would be far better for my application, http://www.llnl.gov/casc/components/, if PyLong_AsLongLong called PyInt_AsLong in cases where the argument is not a PyLongObject. The return value of PyInt_AsLong can safely be promoted to a long long. The end result that I would like is that an 'L' in a PyArg_ParseTuple call return any integer value (int or long) as a C long long. PyArg_ParseTuple ultimately uses PyLong_AsLongLong to handle the L format code. It would be nice to have this behavior in past versions too (i.e. 1.5.2, 2.0 and 2.1). Here is an example of how one might do this. This patch also changes the behavior of PyLong_AsLong to fall back on PyInt_AsLong. $ diff -c longobject.c~ longobject.c *** longobject.c~ Wed Jan 17 07:33:18 2001 --- longobject.c Fri Sep 28 09:42:45 2001 *************** *** 144,153 **** unsigned long x, prev; int i, sign; ! if (vv == NULL || !PyLong_Check(vv)) { PyErr_BadInternalCall(); return -1; } v = (PyLongObject *)vv; i = v->ob_size; sign = 1; --- 144,156 ---- unsigned long x, prev; int i, sign; ! if (vv == NULL) { PyErr_BadInternalCall(); return -1; } + if (!PyLong_Check(vv)) { + return PyInt_AsLong(vv); + } v = (PyLongObject *)vv; i = v->ob_size; sign = 1; *************** *** 387,395 **** LONG_LONG x, prev; int i, sign; ! if (vv == NULL || !PyLong_Check(vv)) { PyErr_BadInternalCall(); return -1; } v = (PyLongObject *)vv; --- 390,401 ---- LONG_LONG x, prev; int i, sign; ! if (vv == NULL) { PyErr_BadInternalCall(); return -1; + } + if (!PyLong_Check(vv)) { + return PyInt_AsLong(vv); } v = (PyLongObject *)vv; ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-01 12:56 Message: Logged In: YES user_id=31435 Ah -- that's unlikely. There's never been a bugfix release for 1.5.2, and there won't be any more for 2.0 or 2.1 either unless someone (unexpectedly) volunteers to produce them. If someone does, what goes in would be up to them, but you're right that the bugfix-release volunteers so far have taken a strict "no new features" line. I suppose that means *you* could volunteer to produce bugfix releases, though, and sneak in anything you felt like . ---------------------------------------------------------------------- Comment By: Tom Epperly (tepperly) Date: 2001-09-30 11:28 Message: Logged In: YES user_id=94539 Thanks to all for making the change. It will make my Python language bindings easier and better. When I spoke of changing it in past version, I did not mean to suggest time travel or some method of altering everyone's installations of previously released version. I was wondering if it could be included in future bug fix releases to previously released versions. However, this appears to be against your policy for bug fixes that I've gleaned from your WWW pages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-29 22:12 Message: Logged In: YES user_id=31435 PyLong_AsLongLong (and so also PyArg_ParseTuple 'L') now accepts longs and ints (but not types other than those and subclasses of those), for 2.2: Doc/ext/extending.tex; new revision: 1.3 Misc/ACKS; new revision: 1.113 Misc/NEWS; new revision: 1.260 Modules/_testcapimodule.c; new revision: 1.12 Objects/longobject.c; new revision: 1.110 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-29 21:36 Message: Logged In: YES user_id=31435 Oops! What Guido did in PyLong_AsLong is back off to PyInt_AsLong but only if PyInt_Check is true. What the suggested patch does is call PyInt_AsLong in *any* non-NULL case where PyLong_Check is false, and that includes dubious things like converting floats to ints. I don't think we want that, and it isn't needed for your stated goal ("'L' in a PyArg_ParseTuple call return any integer value (int or long) as a C long long"). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-29 21:20 Message: Logged In: YES user_id=31435 Hmm. The patch is against some older version of Python that doesn't bear much textual resemblance to current longobject.c (i.e., patch can't make head or tail of this). I believe Guido already changed PyLong_AsLong(), about 2 weeks ago, to back off to PyInt_AsLong if an int object is passed in. PyLong_AsLongLong() does not, though, and I agree it's a good idea. So I'll do that. Tom, I hope you didn't intend your "It would be nice to have this behavior in past versions too (i.e. 1.5.2, 2.0 and 2.1)" to have more actual content than "It would be nice if people never died before they were ready to go" <0.9 wink>. But if so, we obviously can't change the behavior of code that's already been released. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 10:05 Message: Logged In: YES user_id=6380 Makes sense to me. Tim, what do you think? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466125&group_id=5470 From noreply@sourceforge.net Mon Oct 1 22:33:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 14:33:35 -0700 Subject: [Python-bugs-list] [ python-Bugs-466907 ] OSX Python cannot run in console Message-ID: Bugs item #466907, was opened at 2001-10-01 10:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Wight (schwa) Assigned to: Jack Jansen (jackjansen) Summary: OSX Python cannot run in console Initial Comment: OS: Mac OS X 10.1 HW: Verified on G4/733 & PowerBook G4/400Version: 'Unix' Python 2.2a4 (also seen on 2.2a3) Configured: --with-suffix=.exe --without-toolbox-glue Python cannot run on a Mac OS X machine when the user is logged into console mode (using ">console" as login name). Download .tar.gz, configure python as shown, make, make install. Log out of Mac OS X. Log in as console user: use ">console" as login name. Run python interactively from the shell... Python will fail to run and emit these errors: KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorInvalidConnection: CGSGetEventPort: Invalid connection I believe Python is trying to connect to the CoreGraphics server. Why? I don't know (esp. as I've NOT compiled the toolbox glue). Perhaps Python is linking to a Carbon library and the library is automatically trying to open a connection. My makefile skills are not good enough to enable me to check. Python compiled on OS X without the toolbox glue should essentially be identical to any other 'unix' app compiled for Mac OS X. Doesn't seem to be that way. ---------------------------------------------------------------------- >Comment By: Jonathan Wight (schwa) Date: 2001-10-01 14:33 Message: Logged In: YES user_id=29309 FYI. Running 'otool -l' on the Python executable shows that it is being linked to both: /System/Library/Frameworks/Carbon.framework and /System/Library/Frameworks/Foundation.framework Carbon.framework links in /System/Library/Frameworks/ApplicationServices.framework/ which contains the CoreGraphics framework. I'm assuming that causes CoreGraphics to be initialized (guessing). Why does a plain 'unix' build of Python link to both Carbon and Foundation? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 From noreply@sourceforge.net Mon Oct 1 22:38:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 14:38:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-466981 ] item_cget() missing parm. in Tix/HList Message-ID: Bugs item #466981, was opened at 2001-10-01 14:38 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466981&group_id=5470 Category: Tkinter Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: item_cget() missing parm. in Tix/HList Initial Comment: This applies to Tix.py version 1.3, any OS. The item_cget method of HList in Tix.py is missing the first parameter (entryPath in the Tix documentation). The patch below fixes this: *************** *** 730,737 **** c = self.tk.call(self._w, 'info', 'selection') return self.tk.splitlist(c) ! def item_cget(self, entry, col, opt): ! return self.tk.call(self._w, 'item', 'cget', entry, col, opt) def item_configure(self, entry, col, cnf={}, **kw): if cnf is None: --- 730,737 ---- c = self.tk.call(self._w, 'info', 'selection') return self.tk.splitlist(c) ! def item_cget(self, col, opt): ! return self.tk.call(self._w, 'item', 'cget', col, opt) def item_configure(self, entry, col, cnf={}, **kw): if cnf is None: ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466981&group_id=5470 From noreply@sourceforge.net Mon Oct 1 23:17:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 15:17:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-466907 ] OSX Python cannot run in console Message-ID: Bugs item #466907, was opened at 2001-10-01 10:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Wight (schwa) Assigned to: Jack Jansen (jackjansen) Summary: OSX Python cannot run in console Initial Comment: OS: Mac OS X 10.1 HW: Verified on G4/733 & PowerBook G4/400Version: 'Unix' Python 2.2a4 (also seen on 2.2a3) Configured: --with-suffix=.exe --without-toolbox-glue Python cannot run on a Mac OS X machine when the user is logged into console mode (using ">console" as login name). Download .tar.gz, configure python as shown, make, make install. Log out of Mac OS X. Log in as console user: use ">console" as login name. Run python interactively from the shell... Python will fail to run and emit these errors: KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorInvalidConnection: CGSGetEventPort: Invalid connection I believe Python is trying to connect to the CoreGraphics server. Why? I don't know (esp. as I've NOT compiled the toolbox glue). Perhaps Python is linking to a Carbon library and the library is automatically trying to open a connection. My makefile skills are not good enough to enable me to check. Python compiled on OS X without the toolbox glue should essentially be identical to any other 'unix' app compiled for Mac OS X. Doesn't seem to be that way. ---------------------------------------------------------------------- >Comment By: Jonathan Wight (schwa) Date: 2001-10-01 15:17 Message: Logged In: YES user_id=29309 OK... Another Follow Up. Checked the configure.in file and there is no --without-toolbox-glue switch for configure. There is a --disable-toolbox-glue and --enable-toolbox-glue. The line in the README that states you should use --without-toolbox-glue is incorrect. This is a bug in the documentation! (easy to fix though). I've checked the newly built executable (using --disable-toolbox-glue) with 'otool -l' and the linkage dependencies are gone. This is good. Unfortunately when the 'make install' stage of the build fails with: skipping 'gestalt' extension (up-to-date) dyld: ./python.exe Undefined symbols: _Gestalt _PyMac_Error make: *** [sharedmods] Error 67 ---------------------------------------------------------------------- Comment By: Jonathan Wight (schwa) Date: 2001-10-01 14:33 Message: Logged In: YES user_id=29309 FYI. Running 'otool -l' on the Python executable shows that it is being linked to both: /System/Library/Frameworks/Carbon.framework and /System/Library/Frameworks/Foundation.framework Carbon.framework links in /System/Library/Frameworks/ApplicationServices.framework/ which contains the CoreGraphics framework. I'm assuming that causes CoreGraphics to be initialized (guessing). Why does a plain 'unix' build of Python link to both Carbon and Foundation? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 From noreply@sourceforge.net Tue Oct 2 01:20:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 17:20:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-467021 ] socket.inet_aton() incorrect results. Message-ID: Bugs item #467021, was opened at 2001-10-01 17:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467021&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket.inet_aton() incorrect results. Initial Comment: socket.inet_aton("1.2.3.8") gives '\x01\x02\x03\x08' socket.inet_aton("1.2.3.9") gives '\x01\x02\x03\t' ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467021&group_id=5470 From noreply@sourceforge.net Tue Oct 2 03:55:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 19:55:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-467055 ] -flat_namespace not in Makefile.pre.in for Darwin/MacOS Message-ID: Bugs item #467055, was opened at 2001-10-01 19:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 Category: Installation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Mitchell L Model (mlm) Assigned to: Nobody/Anonymous (nobody) Summary: -flat_namespace not in Makefile.pre.in for Darwin/MacOS Initial Comment: Python 2.2a4 fixed Python 2.1's inadequate LDSHARED setting for Darwin/OS X as far as building Python is concerned. However, it does not go quite far enough and the problem still exists for building other software that uses a similar configuration process, in particular Zope 2.4(.1). It turns out that the top-level Makefile and Makefile.pre both have -flat_namespace correctly set, but Makefile.pre.in does not. Since Makefile.pre.in gets installed in /usr/local/lib/python2.2/config, and other software using this build process reference Makefile.pre.in from there, -flat_namespace isn't set for the other software. Manually adding: LDSHARED= $(CC) $(LDFLAGS) -flat_namespace -undefined suppress fixed the problem, at least for building Zope 2.4. I don't know how much of that is actually needed other than -flat_namespace. Please fix this for the real release! The improvements for building on Mac OS X are wonderful -- it would be a shame for them to fail for the lack of one line in one file! (I'm using Mac OS 10.1, in case that matters, with the developer tools installed.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 From noreply@sourceforge.net Tue Oct 2 04:25:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 20:25:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-467059 ] htmllib broken Message-ID: Bugs item #467059, was opened at 2001-10-01 20:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: htmllib broken Initial Comment: Responding to a question in python-help about extracting links from web pages, I wrote a simple href printer (see attached file). When run using 2.2a4, it never prints anything. It outputs a list of hrefs when run with 2.1 or 1.6. Either there's a bug somewhere (in my code possibly, though it's pretty simple) or some semantics changed that I missed. I thought maybe the method resolution order change affected things, but htmllib.HTMLParser only uses single inheritance. When displaying help about htmllib.HTMLParser, pydoc.help does emit the method resolution order, which it doesn't generally seem to do: class HTMLParser(sgmllib.SGMLParser) | Method resolution order: | HTMLParser | sgmllib.SGMLParser | markupbase.ParserBase ... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 From noreply@sourceforge.net Tue Oct 2 04:28:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 20:28:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-467059 ] htmllib broken Message-ID: Bugs item #467059, was opened at 2001-10-01 20:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: htmllib broken Initial Comment: Responding to a question in python-help about extracting links from web pages, I wrote a simple href printer (see attached file). When run using 2.2a4, it never prints anything. It outputs a list of hrefs when run with 2.1 or 1.6. Either there's a bug somewhere (in my code possibly, though it's pretty simple) or some semantics changed that I missed. I thought maybe the method resolution order change affected things, but htmllib.HTMLParser only uses single inheritance. When displaying help about htmllib.HTMLParser, pydoc.help does emit the method resolution order, which it doesn't generally seem to do: class HTMLParser(sgmllib.SGMLParser) | Method resolution order: | HTMLParser | sgmllib.SGMLParser | markupbase.ParserBase ... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 From noreply@sourceforge.net Tue Oct 2 04:30:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 20:30:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-467059 ] htmllib broken Message-ID: Bugs item #467059, was opened at 2001-10-01 20:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: htmllib broken Initial Comment: Responding to a question in python-help about extracting links from web pages, I wrote a simple href printer (see attached file). When run using 2.2a4, it never prints anything. It outputs a list of hrefs when run with 2.1 or 1.6. Either there's a bug somewhere (in my code possibly, though it's pretty simple) or some semantics changed that I missed. I thought maybe the method resolution order change affected things, but htmllib.HTMLParser only uses single inheritance. When displaying help about htmllib.HTMLParser, pydoc.help does emit the method resolution order, which it doesn't generally seem to do: class HTMLParser(sgmllib.SGMLParser) | Method resolution order: | HTMLParser | sgmllib.SGMLParser | markupbase.ParserBase ... ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-01 20:30 Message: Logged In: YES user_id=44345 SF apparently doesn't like file uploads from Opera, so it's pasted here... import htmllib, formatter class MyParser(htmllib.HTMLParser): def anchor_bgn(self, href, name, type): print href fmt = formatter.NullFormatter() parser = MyParser(fmt, verbose=1) parser.feed(open("tour01.html").read()) parser.close() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 From noreply@sourceforge.net Tue Oct 2 07:27:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 01 Oct 2001 23:27:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-467021 ] socket.inet_aton() incorrect results. Message-ID: Bugs item #467021, was opened at 2001-10-01 17:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467021&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket.inet_aton() incorrect results. Initial Comment: socket.inet_aton("1.2.3.8") gives '\x01\x02\x03\x08' socket.inet_aton("1.2.3.9") gives '\x01\x02\x03\t' ---------------------------------------------------------------------- Comment By: James G. sack (jim) (jgsack) Date: 2001-10-01 23:27 Message: Logged In: YES user_id=31123 (Responding to my own post) It seems I just misread the function spec, and thought it was supposed to return an integer, whereas the docs clearly say 'string four characters in length'. So.. nevermind! . ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467021&group_id=5470 From noreply@sourceforge.net Tue Oct 2 13:10:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 05:10:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Tue Oct 2 14:02:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 06:02:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-467021 ] socket.inet_aton() incorrect results. Message-ID: Bugs item #467021, was opened at 2001-10-01 17:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467021&group_id=5470 Category: Extension Modules >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket.inet_aton() incorrect results. Initial Comment: socket.inet_aton("1.2.3.8") gives '\x01\x02\x03\x08' socket.inet_aton("1.2.3.9") gives '\x01\x02\x03\t' ---------------------------------------------------------------------- Comment By: James G. sack (jim) (jgsack) Date: 2001-10-01 23:27 Message: Logged In: YES user_id=31123 (Responding to my own post) It seems I just misread the function spec, and thought it was supposed to return an integer, whereas the docs clearly say 'string four characters in length'. So.. nevermind! . ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467021&group_id=5470 From noreply@sourceforge.net Tue Oct 2 14:09:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 06:09:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-467055 ] -flat_namespace not in Makefile.pre.in for Darwin/MacOS Message-ID: Bugs item #467055, was opened at 2001-10-01 19:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 >Category: Macintosh >Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitchell L Model (mlm) >Assigned to: Jack Jansen (jackjansen) Summary: -flat_namespace not in Makefile.pre.in for Darwin/MacOS Initial Comment: Python 2.2a4 fixed Python 2.1's inadequate LDSHARED setting for Darwin/OS X as far as building Python is concerned. However, it does not go quite far enough and the problem still exists for building other software that uses a similar configuration process, in particular Zope 2.4(.1). It turns out that the top-level Makefile and Makefile.pre both have -flat_namespace correctly set, but Makefile.pre.in does not. Since Makefile.pre.in gets installed in /usr/local/lib/python2.2/config, and other software using this build process reference Makefile.pre.in from there, -flat_namespace isn't set for the other software. Manually adding: LDSHARED= $(CC) $(LDFLAGS) -flat_namespace -undefined suppress fixed the problem, at least for building Zope 2.4. I don't know how much of that is actually needed other than -flat_namespace. Please fix this for the real release! The improvements for building on Mac OS X are wonderful -- it would be a shame for them to fail for the lack of one line in one file! (I'm using Mac OS 10.1, in case that matters, with the developer tools installed.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 From noreply@sourceforge.net Tue Oct 2 14:59:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 06:59:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-467055 ] -flat_namespace not in Makefile.pre.in for Darwin/MacOS Message-ID: Bugs item #467055, was opened at 2001-10-01 19:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitchell L Model (mlm) Assigned to: Jack Jansen (jackjansen) Summary: -flat_namespace not in Makefile.pre.in for Darwin/MacOS Initial Comment: Python 2.2a4 fixed Python 2.1's inadequate LDSHARED setting for Darwin/OS X as far as building Python is concerned. However, it does not go quite far enough and the problem still exists for building other software that uses a similar configuration process, in particular Zope 2.4(.1). It turns out that the top-level Makefile and Makefile.pre both have -flat_namespace correctly set, but Makefile.pre.in does not. Since Makefile.pre.in gets installed in /usr/local/lib/python2.2/config, and other software using this build process reference Makefile.pre.in from there, -flat_namespace isn't set for the other software. Manually adding: LDSHARED= $(CC) $(LDFLAGS) -flat_namespace -undefined suppress fixed the problem, at least for building Zope 2.4. I don't know how much of that is actually needed other than -flat_namespace. Please fix this for the real release! The improvements for building on Mac OS X are wonderful -- it would be a shame for them to fail for the lack of one line in one file! (I'm using Mac OS 10.1, in case that matters, with the developer tools installed.) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 06:59 Message: Logged In: YES user_id=6380 I'm not sure what you're suggesting we *do* here. We can't add -flat_namespace to Makefile.pre.in, because it's a platform-independent file. Also note that there's some name confusion: the Makefile.pre.in that's installed in .../config is not the same one that is used to build Python; it's an old (and deprecated) way to build 3rd party extensions, whose source is in Misc/Makefile.pre.in; the current party line is to use distutils. Maybe Zope should switch to using distutils too. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 From noreply@sourceforge.net Tue Oct 2 18:54:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 10:54:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-467253 ] license points to the wrong file Message-ID: Bugs item #467253, was opened at 2001-10-02 10:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467253&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: license points to the wrong file Initial Comment: This is Python 2.2a4, as in the RPM from www.python.org license() says: See http://www.pythonlabs.com/products/python2.0/license.html A part for being clearly the wrong document, it doesn't even exist! The right reference should be: http://www.python.org/2.2/license.html Kudos for the improvemnts, Ottavio ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467253&group_id=5470 From noreply@sourceforge.net Tue Oct 2 19:28:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 11:28:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-467253 ] license points to the wrong file Message-ID: Bugs item #467253, was opened at 2001-10-02 10:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467253&group_id=5470 >Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) Summary: license points to the wrong file Initial Comment: This is Python 2.2a4, as in the RPM from www.python.org license() says: See http://www.pythonlabs.com/products/python2.0/license.html A part for being clearly the wrong document, it doesn't even exist! The right reference should be: http://www.python.org/2.2/license.html Kudos for the improvemnts, Ottavio ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 11:28 Message: Logged In: YES user_id=6380 Thanks! Fixed in CVS. But you've got to tell me how you got this. It only shows this message when you don't have a LICENSE or LICENSE.txt file. Or did you just peek at the source? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467253&group_id=5470 From noreply@sourceforge.net Tue Oct 2 19:51:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 11:51:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-467265 ] Compile errors on SuSe Linux on IBM/s390 Message-ID: Bugs item #467265, was opened at 2001-10-02 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bernard Niset (bniset) Assigned to: Nobody/Anonymous (nobody) Summary: Compile errors on SuSe Linux on IBM/s390 Initial Comment: I requested a test account at IBM on one of their IBM/s390 machine. I tried to compile Python-2.2a4.tgz and got the following errors... I submit this as a bug because the version 2.1.1 compile and run fine on this machine. Cheers, Bernard Niset, bn@smartobjects.be gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/fileobject.o Objects/fileobject.c Objects/fileobject.c: In function `file_seek': Objects/fileobject.c:328: warning: `offset' might be used uninitialized in this function Objects/fileobject.c: In function `file_truncate': Objects/fileobject.c:365: warning: `newsize' might be used uninitialized in this function gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/stringobject.o Objects/stringobject.c Objects/stringobject.c: In function `PyString_FromFormatV': Objects/stringobject.c:153: invalid initializer make: *** [Objects/stringobject.o] Error 1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 From noreply@sourceforge.net Tue Oct 2 19:56:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 11:56:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-467267 ] isinstance depends on import form Message-ID: Bugs item #467267, was opened at 2001-10-02 11:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Luis P Caamano (lcaamano) Assigned to: Nobody/Anonymous (nobody) Summary: isinstance depends on import form Initial Comment: isinstance(obj, cls) fails if cls is in a package and the import line of the obj creator is different from the import line in the code calling isinstance. It seems to me that the problem is the address test in PyObject_IsInstance(), because different import forms produce objects with different addresses. A limited workaround would be to test for class name but that doesn't include subclasses or types. So, instead of: if isinstance(obj, class): use if obj.__class__.__name__ == 'classname': -------Sample-------------------- Python 2.1.1 (#2, Jul 25 2001, 17:54:11) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from spam.eggs import SpamAndEggs >>> import spam.eggs >>> '0x%x' % id(spam.eggs.SpamAndEggs) '0x8161ce4' >>> '0x%x' % id(SpamAndEggs) '0x814c29c' >>> sag = spam.eggs.SpamAndEggs() >>> isinstance(sag, SpamAndEggs) 0 >>> isinstance(sag, spam.eggs.SpamAndEggs) 1 >>> sag.__class__ >>> sag.__class__.__name__ 'SpamAndEggs' >>> SpamAndEggs.__name__ 'SpamAndEggs' >>> spam.eggs.SpamAndEggs.__name__ 'SpamAndEggs' >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 From noreply@sourceforge.net Tue Oct 2 20:26:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 12:26:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-467267 ] isinstance depends on import form Message-ID: Bugs item #467267, was opened at 2001-10-02 11:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Luis P Caamano (lcaamano) Assigned to: Nobody/Anonymous (nobody) Summary: isinstance depends on import form Initial Comment: isinstance(obj, cls) fails if cls is in a package and the import line of the obj creator is different from the import line in the code calling isinstance. It seems to me that the problem is the address test in PyObject_IsInstance(), because different import forms produce objects with different addresses. A limited workaround would be to test for class name but that doesn't include subclasses or types. So, instead of: if isinstance(obj, class): use if obj.__class__.__name__ == 'classname': -------Sample-------------------- Python 2.1.1 (#2, Jul 25 2001, 17:54:11) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from spam.eggs import SpamAndEggs >>> import spam.eggs >>> '0x%x' % id(spam.eggs.SpamAndEggs) '0x8161ce4' >>> '0x%x' % id(SpamAndEggs) '0x814c29c' >>> sag = spam.eggs.SpamAndEggs() >>> isinstance(sag, SpamAndEggs) 0 >>> isinstance(sag, spam.eggs.SpamAndEggs) 1 >>> sag.__class__ >>> sag.__class__.__name__ 'SpamAndEggs' >>> SpamAndEggs.__name__ 'SpamAndEggs' >>> spam.eggs.SpamAndEggs.__name__ 'SpamAndEggs' >>> ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 12:26 Message: Logged In: YES user_id=6380 We need more information: please show all code involved (use the file upload, please check the checkbox or the upload won't work; you must be logged in to SourceForge as yourself). >From the session you post, it seems that there are two distinct SpamAndEggs objects, but without seeing your code it's hard to understand why. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 From noreply@sourceforge.net Tue Oct 2 20:44:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 12:44:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-467253 ] license points to the wrong file Message-ID: Bugs item #467253, was opened at 2001-10-02 10:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467253&group_id=5470 Category: Python Library Group: Python 2.2 Status: Closed Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: license points to the wrong file Initial Comment: This is Python 2.2a4, as in the RPM from www.python.org license() says: See http://www.pythonlabs.com/products/python2.0/license.html A part for being clearly the wrong document, it doesn't even exist! The right reference should be: http://www.python.org/2.2/license.html Kudos for the improvemnts, Ottavio ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-02 12:44 Message: Logged In: NO As I wrote, I just installed the RPM made available by Sean Reifschneider: I suppose he forgot to include that file. I'm going to write him. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 11:28 Message: Logged In: YES user_id=6380 Thanks! Fixed in CVS. But you've got to tell me how you got this. It only shows this message when you don't have a LICENSE or LICENSE.txt file. Or did you just peek at the source? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467253&group_id=5470 From noreply@sourceforge.net Tue Oct 2 20:53:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 12:53:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-467265 ] Compile errors on SuSe Linux on IBM/s390 Message-ID: Bugs item #467265, was opened at 2001-10-02 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bernard Niset (bniset) Assigned to: Nobody/Anonymous (nobody) Summary: Compile errors on SuSe Linux on IBM/s390 Initial Comment: I requested a test account at IBM on one of their IBM/s390 machine. I tried to compile Python-2.2a4.tgz and got the following errors... I submit this as a bug because the version 2.1.1 compile and run fine on this machine. Cheers, Bernard Niset, bn@smartobjects.be gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/fileobject.o Objects/fileobject.c Objects/fileobject.c: In function `file_seek': Objects/fileobject.c:328: warning: `offset' might be used uninitialized in this function Objects/fileobject.c: In function `file_truncate': Objects/fileobject.c:365: warning: `newsize' might be used uninitialized in this function gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/stringobject.o Objects/stringobject.c Objects/stringobject.c: In function `PyString_FromFormatV': Objects/stringobject.c:153: invalid initializer make: *** [Objects/stringobject.o] Error 1 ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-02 12:53 Message: Logged In: YES user_id=31435 The two warnings look braindead: offset and newsize are obviously initialized before use. So not inclined to try to do anything about those. The error needs more information: an auto vrbl of type va_list is initialized to a vrbl of type va_list. Why is that invalid? What does va_list resolve to on this box? Is VA_LIST_IS_ARRAY #define'd by the config for this box? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 From noreply@sourceforge.net Tue Oct 2 21:33:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 13:33:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-467055 ] -flat_namespace not in Makefile.pre.in for Darwin/MacOS Message-ID: Bugs item #467055, was opened at 2001-10-01 19:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitchell L Model (mlm) Assigned to: Jack Jansen (jackjansen) Summary: -flat_namespace not in Makefile.pre.in for Darwin/MacOS Initial Comment: Python 2.2a4 fixed Python 2.1's inadequate LDSHARED setting for Darwin/OS X as far as building Python is concerned. However, it does not go quite far enough and the problem still exists for building other software that uses a similar configuration process, in particular Zope 2.4(.1). It turns out that the top-level Makefile and Makefile.pre both have -flat_namespace correctly set, but Makefile.pre.in does not. Since Makefile.pre.in gets installed in /usr/local/lib/python2.2/config, and other software using this build process reference Makefile.pre.in from there, -flat_namespace isn't set for the other software. Manually adding: LDSHARED= $(CC) $(LDFLAGS) -flat_namespace -undefined suppress fixed the problem, at least for building Zope 2.4. I don't know how much of that is actually needed other than -flat_namespace. Please fix this for the real release! The improvements for building on Mac OS X are wonderful -- it would be a shame for them to fail for the lack of one line in one file! (I'm using Mac OS 10.1, in case that matters, with the developer tools installed.) ---------------------------------------------------------------------- >Comment By: Mitchell L Model (mlm) Date: 2001-10-02 13:33 Message: Logged In: YES user_id=221340 OK, I understand. And I realized later that maybe it would have to be Zope that changes not Python. I was somewhat seduced by the improvement from Python 2.1 to 2.2a4 with respect to building on OS X that I assumed the intention was to fix things entirely and that given how early it is in the Python 2.2 and Mac OS 10.1 games it seemed likely that what I uncovered was a bug. If it was it would be great to have gotten it into the 2.2 release, so I shot out my message last night without completely thinking it through. Although I have an enormous amount of experience in a lot of different languages, systems, and build mechanisms, I'm just recently getting involved in the Open Source world, Python, and OS X, and there's such a huge amount to know about that I'm finding I'm often running on guesses and hopes. In this case, I was assuming that what Python and Zope were doing were both some kind of generally accepted installation procedure -- it certainly looked like Zope was using the Python mechanisms, since it was affected by Makefile.pre.in, etc. I followed enough of the configure and build mechanisms through to figure out where Python was getting -flat_namespace from for its Makefile and Makefile.pre. [This is a complex mechanism! Reminds me of the early days of the X Window System and its imake facility. There's a point at which even the most knowledgeable and experienced human can't follow enough levels of indirection and parameterization to make sense of things....] Since Python installed Makefile.pre.in, and Zope used it, I assumed that was all as intended. If Zope should use a different build process, then nothing needs to be done for Python. However, wouldn't it be easy enough for the Python installation process to modify the Makefile.pre.in that it installs in ../config? Or at least add a release note telling people using Mac OS 10.1 to add a line to the installed file after the installation is complete? Your call. I just wanted to provide a heads-up in case I had discovered something you would want to have fixed. [This is the first time I'm actually contributing anything t SourceForge, rather than just downloading stuff from it. I invite you to email me with suggestions about what I should be saying using the bug tracker and what I shouldn't, or anything else you want to suggest about community assumptions and conventions.] ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 06:59 Message: Logged In: YES user_id=6380 I'm not sure what you're suggesting we *do* here. We can't add -flat_namespace to Makefile.pre.in, because it's a platform-independent file. Also note that there's some name confusion: the Makefile.pre.in that's installed in .../config is not the same one that is used to build Python; it's an old (and deprecated) way to build 3rd party extensions, whose source is in Misc/Makefile.pre.in; the current party line is to use distutils. Maybe Zope should switch to using distutils too. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 From noreply@sourceforge.net Tue Oct 2 21:34:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 13:34:56 -0700 Subject: [Python-bugs-list] [ python-Bugs-467267 ] isinstance depends on import form Message-ID: Bugs item #467267, was opened at 2001-10-02 11:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Luis P Caamano (lcaamano) Assigned to: Nobody/Anonymous (nobody) Summary: isinstance depends on import form Initial Comment: isinstance(obj, cls) fails if cls is in a package and the import line of the obj creator is different from the import line in the code calling isinstance. It seems to me that the problem is the address test in PyObject_IsInstance(), because different import forms produce objects with different addresses. A limited workaround would be to test for class name but that doesn't include subclasses or types. So, instead of: if isinstance(obj, class): use if obj.__class__.__name__ == 'classname': -------Sample-------------------- Python 2.1.1 (#2, Jul 25 2001, 17:54:11) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from spam.eggs import SpamAndEggs >>> import spam.eggs >>> '0x%x' % id(spam.eggs.SpamAndEggs) '0x8161ce4' >>> '0x%x' % id(SpamAndEggs) '0x814c29c' >>> sag = spam.eggs.SpamAndEggs() >>> isinstance(sag, SpamAndEggs) 0 >>> isinstance(sag, spam.eggs.SpamAndEggs) 1 >>> sag.__class__ >>> sag.__class__.__name__ 'SpamAndEggs' >>> SpamAndEggs.__name__ 'SpamAndEggs' >>> spam.eggs.SpamAndEggs.__name__ 'SpamAndEggs' >>> ---------------------------------------------------------------------- >Comment By: Luis P Caamano (lcaamano) Date: 2001-10-02 13:34 Message: Logged In: YES user_id=279987 After getting your email I went and rechecked things in site-packages and the only suspicious thing was a .pth file. I removed the .pth file and now I'm getting the same address. I'm still investigating. At this moment I have no idea why I'm getting different objects. Let me investigate a little bit more and I'll get back to you. I apologize for not having verified this better. -- Luis Caamano ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 12:26 Message: Logged In: YES user_id=6380 We need more information: please show all code involved (use the file upload, please check the checkbox or the upload won't work; you must be logged in to SourceForge as yourself). >From the session you post, it seems that there are two distinct SpamAndEggs objects, but without seeing your code it's hard to understand why. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 From noreply@sourceforge.net Tue Oct 2 21:41:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 13:41:29 -0700 Subject: [Python-bugs-list] [ python-Bugs-467055 ] -flat_namespace not in Makefile.pre.in for Darwin/MacOS Message-ID: Bugs item #467055, was opened at 2001-10-01 19:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitchell L Model (mlm) Assigned to: Jack Jansen (jackjansen) Summary: -flat_namespace not in Makefile.pre.in for Darwin/MacOS Initial Comment: Python 2.2a4 fixed Python 2.1's inadequate LDSHARED setting for Darwin/OS X as far as building Python is concerned. However, it does not go quite far enough and the problem still exists for building other software that uses a similar configuration process, in particular Zope 2.4(.1). It turns out that the top-level Makefile and Makefile.pre both have -flat_namespace correctly set, but Makefile.pre.in does not. Since Makefile.pre.in gets installed in /usr/local/lib/python2.2/config, and other software using this build process reference Makefile.pre.in from there, -flat_namespace isn't set for the other software. Manually adding: LDSHARED= $(CC) $(LDFLAGS) -flat_namespace -undefined suppress fixed the problem, at least for building Zope 2.4. I don't know how much of that is actually needed other than -flat_namespace. Please fix this for the real release! The improvements for building on Mac OS X are wonderful -- it would be a shame for them to fail for the lack of one line in one file! (I'm using Mac OS 10.1, in case that matters, with the developer tools installed.) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 13:41 Message: Logged In: YES user_id=6380 Yes, the build procedure can be quite dazzling -- I don't claim complete knowledge myself any more. :-) Any platform-specific things needed for 3rd party builds *should* be picked up when the Modules/makesetup script runs and transforms the installed Makefile.pre.in into a Makefile. It would be interesting to find out why it doesn't pick up the LDSHARED setting; can you figure that out? When I try this on Linux (not with Zope), the LDSHARED setting chosen by the configure script doe get stuffed into Makefile correctly... ---------------------------------------------------------------------- Comment By: Mitchell L Model (mlm) Date: 2001-10-02 13:33 Message: Logged In: YES user_id=221340 OK, I understand. And I realized later that maybe it would have to be Zope that changes not Python. I was somewhat seduced by the improvement from Python 2.1 to 2.2a4 with respect to building on OS X that I assumed the intention was to fix things entirely and that given how early it is in the Python 2.2 and Mac OS 10.1 games it seemed likely that what I uncovered was a bug. If it was it would be great to have gotten it into the 2.2 release, so I shot out my message last night without completely thinking it through. Although I have an enormous amount of experience in a lot of different languages, systems, and build mechanisms, I'm just recently getting involved in the Open Source world, Python, and OS X, and there's such a huge amount to know about that I'm finding I'm often running on guesses and hopes. In this case, I was assuming that what Python and Zope were doing were both some kind of generally accepted installation procedure -- it certainly looked like Zope was using the Python mechanisms, since it was affected by Makefile.pre.in, etc. I followed enough of the configure and build mechanisms through to figure out where Python was getting -flat_namespace from for its Makefile and Makefile.pre. [This is a complex mechanism! Reminds me of the early days of the X Window System and its imake facility. There's a point at which even the most knowledgeable and experienced human can't follow enough levels of indirection and parameterization to make sense of things....] Since Python installed Makefile.pre.in, and Zope used it, I assumed that was all as intended. If Zope should use a different build process, then nothing needs to be done for Python. However, wouldn't it be easy enough for the Python installation process to modify the Makefile.pre.in that it installs in ../config? Or at least add a release note telling people using Mac OS 10.1 to add a line to the installed file after the installation is complete? Your call. I just wanted to provide a heads-up in case I had discovered something you would want to have fixed. [This is the first time I'm actually contributing anything t SourceForge, rather than just downloading stuff from it. I invite you to email me with suggestions about what I should be saying using the bug tracker and what I shouldn't, or anything else you want to suggest about community assumptions and conventions.] ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 06:59 Message: Logged In: YES user_id=6380 I'm not sure what you're suggesting we *do* here. We can't add -flat_namespace to Makefile.pre.in, because it's a platform-independent file. Also note that there's some name confusion: the Makefile.pre.in that's installed in .../config is not the same one that is used to build Python; it's an old (and deprecated) way to build 3rd party extensions, whose source is in Misc/Makefile.pre.in; the current party line is to use distutils. Maybe Zope should switch to using distutils too. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 From noreply@sourceforge.net Tue Oct 2 22:01:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 14:01:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-467267 ] isinstance depends on import form Message-ID: Bugs item #467267, was opened at 2001-10-02 11:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Luis P Caamano (lcaamano) Assigned to: Nobody/Anonymous (nobody) Summary: isinstance depends on import form Initial Comment: isinstance(obj, cls) fails if cls is in a package and the import line of the obj creator is different from the import line in the code calling isinstance. It seems to me that the problem is the address test in PyObject_IsInstance(), because different import forms produce objects with different addresses. A limited workaround would be to test for class name but that doesn't include subclasses or types. So, instead of: if isinstance(obj, class): use if obj.__class__.__name__ == 'classname': -------Sample-------------------- Python 2.1.1 (#2, Jul 25 2001, 17:54:11) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from spam.eggs import SpamAndEggs >>> import spam.eggs >>> '0x%x' % id(spam.eggs.SpamAndEggs) '0x8161ce4' >>> '0x%x' % id(SpamAndEggs) '0x814c29c' >>> sag = spam.eggs.SpamAndEggs() >>> isinstance(sag, SpamAndEggs) 0 >>> isinstance(sag, spam.eggs.SpamAndEggs) 1 >>> sag.__class__ >>> sag.__class__.__name__ 'SpamAndEggs' >>> SpamAndEggs.__name__ 'SpamAndEggs' >>> spam.eggs.SpamAndEggs.__name__ 'SpamAndEggs' >>> ---------------------------------------------------------------------- >Comment By: Luis P Caamano (lcaamano) Date: 2001-10-02 14:01 Message: Logged In: YES user_id=279987 With the site-packages/pkg1.pth file: >>> from pkg1.spam.eggs import SpamAndEggs >>> import spam.eggs >>> '0x%x' % id(SpamAndEggs) '0x814e174' >>> '0x%x' % id(spam.eggs.SpamAndEggs) '0x8168fac' >>> import pkg1.spam.eggs >>> '0x%x' % id(pkg1.spam.eggs.SpamAndEggs) '0x814e174' >>> It is starting to make sense. -- lpc ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2001-10-02 13:34 Message: Logged In: YES user_id=279987 After getting your email I went and rechecked things in site-packages and the only suspicious thing was a .pth file. I removed the .pth file and now I'm getting the same address. I'm still investigating. At this moment I have no idea why I'm getting different objects. Let me investigate a little bit more and I'll get back to you. I apologize for not having verified this better. -- Luis Caamano ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 12:26 Message: Logged In: YES user_id=6380 We need more information: please show all code involved (use the file upload, please check the checkbox or the upload won't work; you must be logged in to SourceForge as yourself). >From the session you post, it seems that there are two distinct SpamAndEggs objects, but without seeing your code it's hard to understand why. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 From noreply@sourceforge.net Tue Oct 2 22:09:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 14:09:14 -0700 Subject: [Python-bugs-list] [ python-Bugs-467267 ] isinstance depends on import form Message-ID: Bugs item #467267, was opened at 2001-10-02 11:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 Category: Python Interpreter Core >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Luis P Caamano (lcaamano) >Assigned to: Guido van Rossum (gvanrossum) Summary: isinstance depends on import form Initial Comment: isinstance(obj, cls) fails if cls is in a package and the import line of the obj creator is different from the import line in the code calling isinstance. It seems to me that the problem is the address test in PyObject_IsInstance(), because different import forms produce objects with different addresses. A limited workaround would be to test for class name but that doesn't include subclasses or types. So, instead of: if isinstance(obj, class): use if obj.__class__.__name__ == 'classname': -------Sample-------------------- Python 2.1.1 (#2, Jul 25 2001, 17:54:11) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from spam.eggs import SpamAndEggs >>> import spam.eggs >>> '0x%x' % id(spam.eggs.SpamAndEggs) '0x8161ce4' >>> '0x%x' % id(SpamAndEggs) '0x814c29c' >>> sag = spam.eggs.SpamAndEggs() >>> isinstance(sag, SpamAndEggs) 0 >>> isinstance(sag, spam.eggs.SpamAndEggs) 1 >>> sag.__class__ >>> sag.__class__.__name__ 'SpamAndEggs' >>> SpamAndEggs.__name__ 'SpamAndEggs' >>> spam.eggs.SpamAndEggs.__name__ 'SpamAndEggs' >>> ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 14:09 Message: Logged In: YES user_id=6380 OK, closing this as Invalid. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2001-10-02 14:01 Message: Logged In: YES user_id=279987 With the site-packages/pkg1.pth file: >>> from pkg1.spam.eggs import SpamAndEggs >>> import spam.eggs >>> '0x%x' % id(SpamAndEggs) '0x814e174' >>> '0x%x' % id(spam.eggs.SpamAndEggs) '0x8168fac' >>> import pkg1.spam.eggs >>> '0x%x' % id(pkg1.spam.eggs.SpamAndEggs) '0x814e174' >>> It is starting to make sense. -- lpc ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2001-10-02 13:34 Message: Logged In: YES user_id=279987 After getting your email I went and rechecked things in site-packages and the only suspicious thing was a .pth file. I removed the .pth file and now I'm getting the same address. I'm still investigating. At this moment I have no idea why I'm getting different objects. Let me investigate a little bit more and I'll get back to you. I apologize for not having verified this better. -- Luis Caamano ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 12:26 Message: Logged In: YES user_id=6380 We need more information: please show all code involved (use the file upload, please check the checkbox or the upload won't work; you must be logged in to SourceForge as yourself). >From the session you post, it seems that there are two distinct SpamAndEggs objects, but without seeing your code it's hard to understand why. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467267&group_id=5470 From noreply@sourceforge.net Tue Oct 2 22:35:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 14:35:51 -0700 Subject: [Python-bugs-list] [ python-Bugs-467265 ] Compile errors on SuSe Linux on IBM/s390 Message-ID: Bugs item #467265, was opened at 2001-10-02 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bernard Niset (bniset) >Assigned to: Tim Peters (tim_one) Summary: Compile errors on SuSe Linux on IBM/s390 Initial Comment: I requested a test account at IBM on one of their IBM/s390 machine. I tried to compile Python-2.2a4.tgz and got the following errors... I submit this as a bug because the version 2.1.1 compile and run fine on this machine. Cheers, Bernard Niset, bn@smartobjects.be gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/fileobject.o Objects/fileobject.c Objects/fileobject.c: In function `file_seek': Objects/fileobject.c:328: warning: `offset' might be used uninitialized in this function Objects/fileobject.c: In function `file_truncate': Objects/fileobject.c:365: warning: `newsize' might be used uninitialized in this function gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/stringobject.o Objects/stringobject.c Objects/stringobject.c: In function `PyString_FromFormatV': Objects/stringobject.c:153: invalid initializer make: *** [Objects/stringobject.o] Error 1 ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-02 14:35 Message: Logged In: YES user_id=31435 I made some changes that may have repaired the error (the "invalid initializer"). Or maybe not -- I don't have access to a box of this kind, so I guess the only way we'll know is if you try again with current CVS Python. Python/errors.c, rev 2.66 Objects/stringobject.c, rev 2.136 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-02 12:53 Message: Logged In: YES user_id=31435 The two warnings look braindead: offset and newsize are obviously initialized before use. So not inclined to try to do anything about those. The error needs more information: an auto vrbl of type va_list is initialized to a vrbl of type va_list. Why is that invalid? What does va_list resolve to on this box? Is VA_LIST_IS_ARRAY #define'd by the config for this box? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 From noreply@sourceforge.net Tue Oct 2 23:22:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 15:22:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-467331 ] ClassType.__doc__ always None Message-ID: Bugs item #467331, was opened at 2001-10-02 15:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467331&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: ClassType.__doc__ always None Initial Comment: The docstring for a new-style class is too hard to dig out: >>> class C(object): ... "Hi. I'm C's docstring." ... >>> C.__doc__ # no luck >>> getattr(C, "__doc__") # no luck >>> C.__dict__['__doc__'] # bingo "Hi. I'm C's docstring." >>> Note that in restricted mode, you can't eveb get at C.__dict__ (but we don't want to hide the docstring, right?). If the cause+cure for this isn't instantly known you, assign it back to me and I'll dig into it. Note that things like int.__doc__ and file.__doc__ work fine; it appears to be only user-defined types that suffer here. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467331&group_id=5470 From noreply@sourceforge.net Tue Oct 2 23:31:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 15:31:56 -0700 Subject: [Python-bugs-list] [ python-Bugs-467336 ] doctest failures w/ new-style classes Message-ID: Bugs item #467336, was opened at 2001-10-02 15:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467336&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: doctest failures w/ new-style classes Initial Comment: doctest doesn't find docstrings for static or class methods in new-style classes, or for properties. This is for the same reasons inspect.py/pydoc.py didn't understand these things (the old checks for types via types.py don't succeed). Also doesn't find new-style class docstrings (but opened a different bug about that, since C.__doc__ being None when there is a docstring just doesn't seem right). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467336&group_id=5470 From noreply@sourceforge.net Wed Oct 3 00:21:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 16:21:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-467331 ] ClassType.__doc__ always None Message-ID: Bugs item #467331, was opened at 2001-10-02 15:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467331&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) >Assigned to: Tim Peters (tim_one) Summary: ClassType.__doc__ always None Initial Comment: The docstring for a new-style class is too hard to dig out: >>> class C(object): ... "Hi. I'm C's docstring." ... >>> C.__doc__ # no luck >>> getattr(C, "__doc__") # no luck >>> C.__dict__['__doc__'] # bingo "Hi. I'm C's docstring." >>> Note that in restricted mode, you can't eveb get at C.__dict__ (but we don't want to hide the docstring, right?). If the cause+cure for this isn't instantly known you, assign it back to me and I'll dig into it. Note that things like int.__doc__ and file.__doc__ work fine; it appears to be only user-defined types that suffer here. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 16:21 Message: Logged In: YES user_id=6380 I'm guessing this is because there's a tp_doc slot in the type object but that's never assigned to in type_new. Enjoy learning about type_new! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467331&group_id=5470 From noreply@sourceforge.net Wed Oct 3 03:19:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 19:19:44 -0700 Subject: [Python-bugs-list] [ python-Bugs-467381 ] print statement and exception Message-ID: Bugs item #467381, was opened at 2001-10-02 19:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: print statement and exception Initial Comment: a = 3 b = 0 try: print "quotient is",a/b except: pass print "sum is", a+b prints something like "quotient is sum is 3". The first print statement prints "quotient is" before the exception is thrown, and then doesn't print the final newline. The final newline should be protected by the equivalent of a "finally" clause. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 From noreply@sourceforge.net Wed Oct 3 03:25:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 19:25:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Wed Oct 3 04:08:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 20:08:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-467381 ] print statement and exception Message-ID: Bugs item #467381, was opened at 2001-10-02 19:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 Category: Python Interpreter Core >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: paul rubin (phr) >Assigned to: Guido van Rossum (gvanrossum) Summary: print statement and exception Initial Comment: a = 3 b = 0 try: print "quotient is",a/b except: pass print "sum is", a+b prints something like "quotient is sum is 3". The first print statement prints "quotient is" before the exception is thrown, and then doesn't print the final newline. The final newline should be protected by the equivalent of a "finally" clause. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 20:08 Message: Logged In: YES user_id=6380 Sorry, this is how it's defined. The print statement does its things one at a time. If one of its things raises an exception, then the remaining things won't be printed -- including the newline. If that's not what you want, calculate the value and catch the exception before printing the value, or use a print with a trailing comma (suppresses the newline printing) and use an empty print statement separately (supplies the newline), like this: try: print "quotient is", a/b, except: pass print ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 From noreply@sourceforge.net Wed Oct 3 05:11:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 21:11:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-467336 ] doctest failures w/ new-style classes Message-ID: Bugs item #467336, was opened at 2001-10-02 15:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467336&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: doctest failures w/ new-style classes Initial Comment: doctest doesn't find docstrings for static or class methods in new-style classes, or for properties. This is for the same reasons inspect.py/pydoc.py didn't understand these things (the old checks for types via types.py don't succeed). Also doesn't find new-style class docstrings (but opened a different bug about that, since C.__doc__ being None when there is a docstring just doesn't seem right). ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-02 21:11 Message: Logged In: YES user_id=31435 doctest now finds and runs docstrings associated with new- style class static methods, class methods and properties: Lib/doctest.py; new revision: 1.20 Lib/test/test_doctest2.py; new file Lib/test/test_pyclbr.py; new revision: 1.6 Lib/test/test_support.py; new revision: 1.38 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467336&group_id=5470 From noreply@sourceforge.net Wed Oct 3 05:44:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 02 Oct 2001 21:44:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-467055 ] -flat_namespace not in Makefile.pre.in for Darwin/MacOS Message-ID: Bugs item #467055, was opened at 2001-10-01 19:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitchell L Model (mlm) Assigned to: Jack Jansen (jackjansen) Summary: -flat_namespace not in Makefile.pre.in for Darwin/MacOS Initial Comment: Python 2.2a4 fixed Python 2.1's inadequate LDSHARED setting for Darwin/OS X as far as building Python is concerned. However, it does not go quite far enough and the problem still exists for building other software that uses a similar configuration process, in particular Zope 2.4(.1). It turns out that the top-level Makefile and Makefile.pre both have -flat_namespace correctly set, but Makefile.pre.in does not. Since Makefile.pre.in gets installed in /usr/local/lib/python2.2/config, and other software using this build process reference Makefile.pre.in from there, -flat_namespace isn't set for the other software. Manually adding: LDSHARED= $(CC) $(LDFLAGS) -flat_namespace -undefined suppress fixed the problem, at least for building Zope 2.4. I don't know how much of that is actually needed other than -flat_namespace. Please fix this for the real release! The improvements for building on Mac OS X are wonderful -- it would be a shame for them to fail for the lack of one line in one file! (I'm using Mac OS 10.1, in case that matters, with the developer tools installed.) ---------------------------------------------------------------------- >Comment By: Mitchell L Model (mlm) Date: 2001-10-02 21:44 Message: Logged In: YES user_id=221340 I started working my way through the Zope build process to see what was going on. Things looked correct far enough into the process that I began to wonder what was going on. In desperation, I renamed /usr/local to something else, unpacked the Python 2.2a4 and Zope 2.4.1 tar files, and remade both. I didn't get the error I got last night. Ugh. I spent some time retracing my steps, but I couldn't figure out what happened. So, false alarm. But I learned a lot! Sorry to bother you with all this. By the way, with the updates for OS 10.1 in Python 2.2a4, all I had to do to make both Python and Zope was: cd Python2.2a4 configure --with-suffix=.exe make sudo make install cd Zope python wo_pcgi.py That's all. Just that one configure flag, and no global variables set. Very, very nice. What a delight to have a just released MacOS treated with respect by multi-platform software! ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 13:41 Message: Logged In: YES user_id=6380 Yes, the build procedure can be quite dazzling -- I don't claim complete knowledge myself any more. :-) Any platform-specific things needed for 3rd party builds *should* be picked up when the Modules/makesetup script runs and transforms the installed Makefile.pre.in into a Makefile. It would be interesting to find out why it doesn't pick up the LDSHARED setting; can you figure that out? When I try this on Linux (not with Zope), the LDSHARED setting chosen by the configure script doe get stuffed into Makefile correctly... ---------------------------------------------------------------------- Comment By: Mitchell L Model (mlm) Date: 2001-10-02 13:33 Message: Logged In: YES user_id=221340 OK, I understand. And I realized later that maybe it would have to be Zope that changes not Python. I was somewhat seduced by the improvement from Python 2.1 to 2.2a4 with respect to building on OS X that I assumed the intention was to fix things entirely and that given how early it is in the Python 2.2 and Mac OS 10.1 games it seemed likely that what I uncovered was a bug. If it was it would be great to have gotten it into the 2.2 release, so I shot out my message last night without completely thinking it through. Although I have an enormous amount of experience in a lot of different languages, systems, and build mechanisms, I'm just recently getting involved in the Open Source world, Python, and OS X, and there's such a huge amount to know about that I'm finding I'm often running on guesses and hopes. In this case, I was assuming that what Python and Zope were doing were both some kind of generally accepted installation procedure -- it certainly looked like Zope was using the Python mechanisms, since it was affected by Makefile.pre.in, etc. I followed enough of the configure and build mechanisms through to figure out where Python was getting -flat_namespace from for its Makefile and Makefile.pre. [This is a complex mechanism! Reminds me of the early days of the X Window System and its imake facility. There's a point at which even the most knowledgeable and experienced human can't follow enough levels of indirection and parameterization to make sense of things....] Since Python installed Makefile.pre.in, and Zope used it, I assumed that was all as intended. If Zope should use a different build process, then nothing needs to be done for Python. However, wouldn't it be easy enough for the Python installation process to modify the Makefile.pre.in that it installs in ../config? Or at least add a release note telling people using Mac OS 10.1 to add a line to the installed file after the installation is complete? Your call. I just wanted to provide a heads-up in case I had discovered something you would want to have fixed. [This is the first time I'm actually contributing anything t SourceForge, rather than just downloading stuff from it. I invite you to email me with suggestions about what I should be saying using the bug tracker and what I shouldn't, or anything else you want to suggest about community assumptions and conventions.] ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 06:59 Message: Logged In: YES user_id=6380 I'm not sure what you're suggesting we *do* here. We can't add -flat_namespace to Makefile.pre.in, because it's a platform-independent file. Also note that there's some name confusion: the Makefile.pre.in that's installed in .../config is not the same one that is used to build Python; it's an old (and deprecated) way to build 3rd party extensions, whose source is in Misc/Makefile.pre.in; the current party line is to use distutils. Maybe Zope should switch to using distutils too. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 From noreply@sourceforge.net Wed Oct 3 08:03:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 00:03:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-467265 ] Compile errors on SuSe Linux on IBM/s390 Message-ID: Bugs item #467265, was opened at 2001-10-02 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bernard Niset (bniset) Assigned to: Tim Peters (tim_one) Summary: Compile errors on SuSe Linux on IBM/s390 Initial Comment: I requested a test account at IBM on one of their IBM/s390 machine. I tried to compile Python-2.2a4.tgz and got the following errors... I submit this as a bug because the version 2.1.1 compile and run fine on this machine. Cheers, Bernard Niset, bn@smartobjects.be gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/fileobject.o Objects/fileobject.c Objects/fileobject.c: In function `file_seek': Objects/fileobject.c:328: warning: `offset' might be used uninitialized in this function Objects/fileobject.c: In function `file_truncate': Objects/fileobject.c:365: warning: `newsize' might be used uninitialized in this function gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/stringobject.o Objects/stringobject.c Objects/stringobject.c: In function `PyString_FromFormatV': Objects/stringobject.c:153: invalid initializer make: *** [Objects/stringobject.o] Error 1 ---------------------------------------------------------------------- >Comment By: Bernard Niset (bniset) Date: 2001-10-03 00:03 Message: Logged In: YES user_id=47338 OK.. I will test that tonight ... Cheers. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-02 14:35 Message: Logged In: YES user_id=31435 I made some changes that may have repaired the error (the "invalid initializer"). Or maybe not -- I don't have access to a box of this kind, so I guess the only way we'll know is if you try again with current CVS Python. Python/errors.c, rev 2.66 Objects/stringobject.c, rev 2.136 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-02 12:53 Message: Logged In: YES user_id=31435 The two warnings look braindead: offset and newsize are obviously initialized before use. So not inclined to try to do anything about those. The error needs more information: an auto vrbl of type va_list is initialized to a vrbl of type va_list. Why is that invalid? What does va_list resolve to on this box? Is VA_LIST_IS_ARRAY #define'd by the config for this box? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 From noreply@sourceforge.net Wed Oct 3 09:09:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 01:09:05 -0700 Subject: [Python-bugs-list] [ python-Bugs-467430 ] CGIHTTPServer - 2 issues and/or bugs Message-ID: Bugs item #467430, was opened at 2001-10-03 01:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467430&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: CGIHTTPServer - 2 issues and/or bugs Initial Comment: 1) First, the easy one. I believe the following self.error_message calls should be replaced with self.log_message calls ... since no error actually has occured. *** /opt/arseed/tfs-lib/lib/python2.1/CGIHTTPServer.py Tue Oct 2 20:35:04 2001 --- /home/norton/CGIHTTPServer.py Wed Oct 3 17:07:05 2001 *************** *** 227,233 **** cmdline = "%s %s" % (interp, cmdline) if '=' not in query and '"' not in query: cmdline = '%s "%s"' % (cmdline, query) ! self.log_error("command: %s", cmdline) try: nbytes = int(length) except: --- 227,233 ---- cmdline = "%s %s" % (interp, cmdline) if '=' not in query and '"' not in query: cmdline = '%s "%s"' % (cmdline, query) ! self.log_message("command: %s", cmdline) try: nbytes = int(length) except: *************** *** 242,248 **** if sts: self.log_error("CGI script exit status %#x", sts) else: ! self.log_error("CGI script exited OK") else: # Other O.S. -- execute script in this process --- 242,248 ---- if sts: self.log_error("CGI script exit status %#x", sts) else: ! self.log_message("CGI script exited OK") else: # Other O.S. -- execute script in this process *************** *** 267,273 **** except SystemExit, sts: self.log_error("CGI script exit status %s", str(sts)) else: ! self.log_error("CGI script exited OK") nobody = None --- 267,273 ---- except SystemExit, sts: self.log_error("CGI script exit status %s", str(sts)) else: ! self.log_message("CGI script exited OK") nobody = None 2) Secondly, I have created my own CGIHTTP handler class that re-uses the CGHITTPServer CGIHTTPRequestHandler class. I have modified Zope's ZPublisher.Client code to redirect file:// urls to this CGIHTTP handler so that external cgi executables can be proxied by zope. I have attached my handler to this request (only if you are curious). Anyway, I feel the following patch should be applied to the standard python lib. I faced troubles with os.fork, os.dup2, and reading from self.rfile in the child process if the self.rfile is not flushed as well prior to forking. *** Python-2.1.1/Lib/CGIHTTPServer.py Sat Sep 29 15:52:51 2001 --- Python-2.1.1/Lib/CGIHTTPServer.py Tue Oct 2 19:55:14 2001 *************** *** 192,197 **** --- 192,198 ---- if '=' not in decoded_query: args.append(decoded_query) nobody = nobody_uid() + self.rfile.flush() # Always flush before forking self.wfile.flush() # Always flush before forking pid = os.fork() if pid != 0: ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467430&group_id=5470 From noreply@sourceforge.net Wed Oct 3 09:13:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 01:13:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-467430 ] CGIHTTPServer - 2 issues and/or bugs Message-ID: Bugs item #467430, was opened at 2001-10-03 01:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467430&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: CGIHTTPServer - 2 issues and/or bugs Initial Comment: 1) First, the easy one. I believe the following self.error_message calls should be replaced with self.log_message calls ... since no error actually has occured. *** /opt/arseed/tfs-lib/lib/python2.1/CGIHTTPServer.py Tue Oct 2 20:35:04 2001 --- /home/norton/CGIHTTPServer.py Wed Oct 3 17:07:05 2001 *************** *** 227,233 **** cmdline = "%s %s" % (interp, cmdline) if '=' not in query and '"' not in query: cmdline = '%s "%s"' % (cmdline, query) ! self.log_error("command: %s", cmdline) try: nbytes = int(length) except: --- 227,233 ---- cmdline = "%s %s" % (interp, cmdline) if '=' not in query and '"' not in query: cmdline = '%s "%s"' % (cmdline, query) ! self.log_message("command: %s", cmdline) try: nbytes = int(length) except: *************** *** 242,248 **** if sts: self.log_error("CGI script exit status %#x", sts) else: ! self.log_error("CGI script exited OK") else: # Other O.S. -- execute script in this process --- 242,248 ---- if sts: self.log_error("CGI script exit status %#x", sts) else: ! self.log_message("CGI script exited OK") else: # Other O.S. -- execute script in this process *************** *** 267,273 **** except SystemExit, sts: self.log_error("CGI script exit status %s", str(sts)) else: ! self.log_error("CGI script exited OK") nobody = None --- 267,273 ---- except SystemExit, sts: self.log_error("CGI script exit status %s", str(sts)) else: ! self.log_message("CGI script exited OK") nobody = None 2) Secondly, I have created my own CGIHTTP handler class that re-uses the CGHITTPServer CGIHTTPRequestHandler class. I have modified Zope's ZPublisher.Client code to redirect file:// urls to this CGIHTTP handler so that external cgi executables can be proxied by zope. I have attached my handler to this request (only if you are curious). Anyway, I feel the following patch should be applied to the standard python lib. I faced troubles with os.fork, os.dup2, and reading from self.rfile in the child process if the self.rfile is not flushed as well prior to forking. *** Python-2.1.1/Lib/CGIHTTPServer.py Sat Sep 29 15:52:51 2001 --- Python-2.1.1/Lib/CGIHTTPServer.py Tue Oct 2 19:55:14 2001 *************** *** 192,197 **** --- 192,198 ---- if '=' not in decoded_query: args.append(decoded_query) nobody = nobody_uid() + self.rfile.flush() # Always flush before forking self.wfile.flush() # Always flush before forking pid = os.fork() if pid != 0: ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-03 01:13 Message: Logged In: NO The experimental handler attachment that I mentioned in the above message was rejected while adding the bug posting. If you would like to look at the code, please contact me (norton@alum.mit.edu) directly. thanks, - joe n. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467430&group_id=5470 From noreply@sourceforge.net Wed Oct 3 15:32:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 07:32:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Wed Oct 3 16:58:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 08:58:59 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Wed Oct 3 18:51:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 10:51:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Nobody/Anonymous (nobody) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Wed Oct 3 19:14:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 11:14:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Nobody/Anonymous (nobody) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Wed Oct 3 19:14:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 11:14:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Nobody/Anonymous (nobody) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Wed Oct 3 19:18:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 11:18:20 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Nobody/Anonymous (nobody) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:18 Message: Logged In: YES user_id=340060 Ok, this is ridiculous... I'll see if a different browser lets me attach a file. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Wed Oct 3 19:19:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 11:19:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Nobody/Anonymous (nobody) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:19 Message: Logged In: YES user_id=340060 last time I forgot the little checkbox... do I get an award for the most empty followups? ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:18 Message: Logged In: YES user_id=340060 Ok, this is ridiculous... I'll see if a different browser lets me attach a file. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Wed Oct 3 19:23:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 11:23:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Nobody/Anonymous (nobody) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:22 Message: Logged In: YES user_id=340060 This is my last attempt to attach the sample code... if it doesn't work, send me mail and I'll send the code. -- Steve (stephen.langer@nist.gov) ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:19 Message: Logged In: YES user_id=340060 last time I forgot the little checkbox... do I get an award for the most empty followups? ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:18 Message: Logged In: YES user_id=340060 Ok, this is ridiculous... I'll see if a different browser lets me attach a file. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Wed Oct 3 19:36:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 11:36:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-467583 ] Restricted Execution docs broken ref Message-ID: Bugs item #467583, was opened at 2001-10-03 11:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467583&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Restricted Execution docs broken ref Initial Comment: The restricted execution documention references an old and broken url to grail. It uses grail.cnri.reston.va.us/grail which no longer resolves. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467583&group_id=5470 From noreply@sourceforge.net Wed Oct 3 19:41:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 11:41:03 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:22 Message: Logged In: YES user_id=340060 This is my last attempt to attach the sample code... if it doesn't work, send me mail and I'll send the code. -- Steve (stephen.langer@nist.gov) ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:19 Message: Logged In: YES user_id=340060 last time I forgot the little checkbox... do I get an award for the most empty followups? ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:18 Message: Logged In: YES user_id=340060 Ok, this is ridiculous... I'll see if a different browser lets me attach a file. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Wed Oct 3 20:45:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 12:45:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-445980 ] _socket fails to build on IRIX 6.5 Message-ID: Bugs item #445980, was opened at 2001-07-30 08:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445980&group_id=5470 Category: Extension Modules Group: Platform-specific >Status: Closed >Resolution: Postponed Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: _socket fails to build on IRIX 6.5 Initial Comment: Building 2.2a1 on a 64-bit SGI box running IRIX 6.5. I used the MIPSPro 7.30 compiler, using the "-n32" flag ("new 32-bit object"). Building the interpreter and most extensions works, but creating _socket.so fails as follows: ld -shared -all build/temp.irix64-6.5-2.2/socketmodule.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.irix64-6.5-2.2/_socket.so ld32: FATAL 12 : Expecting n32 objects: /usr/local/ssl/lib/libssl.so is o32. WARNING: building of extension "_socket" failed: command 'ld' failed with exit status 4 I think this is a local configuration error, but the build process should probably be more resilient to it. At the very least, there should probably be a --without-ssl configure option so linking against OpenSSL can be disabled on systems with broken OpenSSL installations (like this one). ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-03 12:45 Message: Logged In: YES user_id=14422 This build failure happened because /usr/local is broken on the system where I tried to build it. (Briefly, /usr/local is full of 32-bit libs from IRIX 5, and I'm building on 64-bit IRIX 6.) The main Python problem here is lack of flexibility in setup.py: it always adds /usr/local/{lib,include} to the library/include search paths, and always looks for OpenSSL in /usr/local/lib. If /usr/local is broken or untrustworthy, this is the wrong thing to do. This has nothing to do with the _socket module per se, though, so I'm going to close this bug. I'll see if I can't come up with a patch to add options to setup.py, eg. "--without-openssl" or "--without-usrlocal". ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445980&group_id=5470 From noreply@sourceforge.net Wed Oct 3 20:46:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 12:46:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-445987 ] test_pty fails on IRIX 6.5 Message-ID: Bugs item #445987, was opened at 2001-07-30 08:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445987&group_id=5470 Category: Python Library Group: Platform-specific >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: test_pty fails on IRIX 6.5 Initial Comment: Built 2.2a1 on a 64-bit SGI box running IRIX 6.5. test_pty fails; here's the output from "make test": test_pty The actual stdout doesn't match the expected stdout. This much did match (between asterisk lines): ********************************************************************** test_pty ********************************************************************** Then ... We expected (repr): 'I wish to buy a fish license.\nFor my pet fish, Eric.\n' But instead we got: 'I wish to buy a fish license.\r\nFor my pet fish, Eric.\n' test test_pty failed -- Writing: 'I wish to buy a fish license.\r\nFor my pet fish, Eric.\n', expected: 'I wish to buy a fish license.\nFor my pet fish, Eric.\n' and here's what I get running the test script directly: ./python Lib/test/test_pty.py Calling master_open() Got master_fd '3', slave_name '/dev/ttyq7' Calling slave_open('/dev/ttyq7') Got slave_fd '4' Writing to slave_fd Writing chunked output I wish to buy a fish license. For my pet fish, Eric. calling pty.fork() Waiting for child (193150) to finish. Child (193150) exited with status 4 (1024). $ ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-03 12:46 Message: Logged In: YES user_id=14422 This doesn't seem to be happening any more with the latest CVS (2.2a4+). Closing the bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445987&group_id=5470 From noreply@sourceforge.net Wed Oct 3 20:46:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 12:46:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-445987 ] test_pty fails on IRIX 6.5 Message-ID: Bugs item #445987, was opened at 2001-07-30 08:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445987&group_id=5470 Category: Python Library Group: Platform-specific Status: Closed Resolution: Out of Date Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: test_pty fails on IRIX 6.5 Initial Comment: Built 2.2a1 on a 64-bit SGI box running IRIX 6.5. test_pty fails; here's the output from "make test": test_pty The actual stdout doesn't match the expected stdout. This much did match (between asterisk lines): ********************************************************************** test_pty ********************************************************************** Then ... We expected (repr): 'I wish to buy a fish license.\nFor my pet fish, Eric.\n' But instead we got: 'I wish to buy a fish license.\r\nFor my pet fish, Eric.\n' test test_pty failed -- Writing: 'I wish to buy a fish license.\r\nFor my pet fish, Eric.\n', expected: 'I wish to buy a fish license.\nFor my pet fish, Eric.\n' and here's what I get running the test script directly: ./python Lib/test/test_pty.py Calling master_open() Got master_fd '3', slave_name '/dev/ttyq7' Calling slave_open('/dev/ttyq7') Got slave_fd '4' Writing to slave_fd Writing chunked output I wish to buy a fish license. For my pet fish, Eric. calling pty.fork() Waiting for child (193150) to finish. Child (193150) exited with status 4 (1024). $ ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-03 12:46 Message: Logged In: YES user_id=14422 This doesn't seem to be happening any more with the latest CVS (2.2a4+). Closing the bug. ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2001-10-03 12:46 Message: Logged In: YES user_id=14422 This doesn't seem to be happening any more with the latest CVS (2.2a4+). Closing the bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445987&group_id=5470 From noreply@sourceforge.net Wed Oct 3 20:49:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 12:49:51 -0700 Subject: [Python-bugs-list] [ python-Bugs-445980 ] _socket fails to build on IRIX 6.5 Message-ID: Bugs item #445980, was opened at 2001-07-30 08:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445980&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Closed Resolution: Postponed Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: _socket fails to build on IRIX 6.5 Initial Comment: Building 2.2a1 on a 64-bit SGI box running IRIX 6.5. I used the MIPSPro 7.30 compiler, using the "-n32" flag ("new 32-bit object"). Building the interpreter and most extensions works, but creating _socket.so fails as follows: ld -shared -all build/temp.irix64-6.5-2.2/socketmodule.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.irix64-6.5-2.2/_socket.so ld32: FATAL 12 : Expecting n32 objects: /usr/local/ssl/lib/libssl.so is o32. WARNING: building of extension "_socket" failed: command 'ld' failed with exit status 4 I think this is a local configuration error, but the build process should probably be more resilient to it. At the very least, there should probably be a --without-ssl configure option so linking against OpenSSL can be disabled on systems with broken OpenSSL installations (like this one). ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-03 12:49 Message: Logged In: YES user_id=14422 Oh, I forgot to mention: I worked around the problem by editing setup.py. In PyBuildExt.detect_modules(), I just commented out all the code that looks for OpenSSL, so _socket.so is always built from socketmodule.c alone, with no other junk (eg. broken SSL libs). ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2001-10-03 12:45 Message: Logged In: YES user_id=14422 This build failure happened because /usr/local is broken on the system where I tried to build it. (Briefly, /usr/local is full of 32-bit libs from IRIX 5, and I'm building on 64-bit IRIX 6.) The main Python problem here is lack of flexibility in setup.py: it always adds /usr/local/{lib,include} to the library/include search paths, and always looks for OpenSSL in /usr/local/lib. If /usr/local is broken or untrustworthy, this is the wrong thing to do. This has nothing to do with the _socket module per se, though, so I'm going to close this bug. I'll see if I can't come up with a patch to add options to setup.py, eg. "--without-openssl" or "--without-usrlocal". ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445980&group_id=5470 From noreply@sourceforge.net Wed Oct 3 20:51:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 12:51:28 -0700 Subject: [Python-bugs-list] [ python-Bugs-445991 ] test_struct dumps core on IRIX 6.5 Message-ID: Bugs item #445991, was opened at 2001-07-30 08:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445991&group_id=5470 Category: Extension Modules Group: Platform-specific >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: test_struct dumps core on IRIX 6.5 Initial Comment: Built 2.2a1 on a 64-bit SGI box running IRIX 6.5. test_struct dumps core: $ /python Lib/test/test_struct.py trying: xcbhilfd trying: xcBHILfd trying: @xcbhilfd [...] 'f' 2.0 '@\x00\x00\x00' '\x00\x00\x00@' 'd' 2.0 '@\x00\x00\x00\x00\x00\x00\x00' '\x00\x00\x00\x00\x00\x00\x00@' 'f' -2.0 '\xc0\x00\x00\x00' '\x00\x00\x00\xc0' 'd' -2.0 '\xc0\x00\x00\x00\x00\x00\x00\x00' '\x00\x00\x00\x00\x00\x00\x00\xc0' zsh: 193500 bus error (core dumped) ./python Lib/test/test_struct.py Analyzing the coredump with dbx reveals: $ dbx ./python core dbx version 7.3 MR 55458_Apr30_MR Apr 30 1999 13:44:41 Core from signal SIGBUS: Bus error (dbx) where > 0 np_longlong(0x101aaf8c, 0x1012c354, 0x5fff88fc, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Modules/structmodule.c":703, 0x5ffe304c] 1 struct_pack(0x0, 0x1022ce6c, 0x0, 0xc388060, 0x101aaf8c, 0x5, 0x0, 0x1013d015) ["/tmp/Python-2.2a1/Modules/structmodule.c":1362, 0x5ffe50b0] 2 call_cfunction(0x101a75a4, 0x1022ce6c, 0x0, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/ceval.c":3064, 0x1005d37c] 3 eval_frame(0x101f55fc, 0x101f5748, 0x83, 0x101f5748, 0x10231b98, 0x10230ffa, 0x0, 0x0) ["/tmp/Python-2.2a1/Python/ceval.c":1913, 0x10058dec] 4 PyEval_EvalCodeEx(0x10231b98, 0x10139924, 0x10139924, 0x0, 0x0, 0x0, 0x0, 0x0) ["/tmp/Python-2.2a1/Python/ceval.c":2518, 0x1005b59c] 5 PyEval_EvalCode(0x10231b98, 0x10139924, 0x10139924, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/ceval.c":491, 0x10053590] 6 run_node(0x1019ac50, 0x7fff3009, 0x10139924, 0x10139924, 0x7fff2df4, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":1057, 0x10062704] 7 run_err_node(0x1019ac50, 0x7fff3009, 0x10139924, 0x10139924, 0x7fff2df4, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":1044, 0x10062660] 8 PyRun_FileExFlags(0xfb4e640, 0x7fff3009, 0x101, 0x10139924, 0x10139924, 0x1, 0x7fff2df4, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":1035, 0x100625e4] 9 PyRun_SimpleFileExFlags(0xfb4e640, 0x7fff3009, 0x1, 0x7fff2df4, 0x5, 0x5, 0x0, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":672, 0x10060dc4] 10 PyRun_AnyFileExFlags(0xfb4e640, 0x7fff3009, 0x1, 0x7fff2df4, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":482, 0x1006037c] 11 Py_Main(0x2, 0x7fff2ea4, 0x0, 0xfb4e640, 0x5, 0x0, 0x0, 0x0) ["/tmp/Python-2.2a1/Modules/main.c":320, 0x1000d378] 12 ::main(0x2, 0x7fff2ea4, 0xfb4e3d8, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Modules/ccpython.cc":10, 0x1000c7ec] 13 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M3/csu/crt1text.s":177, 0x1000c788] ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-03 12:51 Message: Logged In: YES user_id=14422 This no longer seems to happen with the latest CVS (2.2a4+). Closing this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445991&group_id=5470 From noreply@sourceforge.net Wed Oct 3 20:51:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 12:51:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-445991 ] test_struct dumps core on IRIX 6.5 Message-ID: Bugs item #445991, was opened at 2001-07-30 08:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445991&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Closed Resolution: Out of Date Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: test_struct dumps core on IRIX 6.5 Initial Comment: Built 2.2a1 on a 64-bit SGI box running IRIX 6.5. test_struct dumps core: $ /python Lib/test/test_struct.py trying: xcbhilfd trying: xcBHILfd trying: @xcbhilfd [...] 'f' 2.0 '@\x00\x00\x00' '\x00\x00\x00@' 'd' 2.0 '@\x00\x00\x00\x00\x00\x00\x00' '\x00\x00\x00\x00\x00\x00\x00@' 'f' -2.0 '\xc0\x00\x00\x00' '\x00\x00\x00\xc0' 'd' -2.0 '\xc0\x00\x00\x00\x00\x00\x00\x00' '\x00\x00\x00\x00\x00\x00\x00\xc0' zsh: 193500 bus error (core dumped) ./python Lib/test/test_struct.py Analyzing the coredump with dbx reveals: $ dbx ./python core dbx version 7.3 MR 55458_Apr30_MR Apr 30 1999 13:44:41 Core from signal SIGBUS: Bus error (dbx) where > 0 np_longlong(0x101aaf8c, 0x1012c354, 0x5fff88fc, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Modules/structmodule.c":703, 0x5ffe304c] 1 struct_pack(0x0, 0x1022ce6c, 0x0, 0xc388060, 0x101aaf8c, 0x5, 0x0, 0x1013d015) ["/tmp/Python-2.2a1/Modules/structmodule.c":1362, 0x5ffe50b0] 2 call_cfunction(0x101a75a4, 0x1022ce6c, 0x0, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/ceval.c":3064, 0x1005d37c] 3 eval_frame(0x101f55fc, 0x101f5748, 0x83, 0x101f5748, 0x10231b98, 0x10230ffa, 0x0, 0x0) ["/tmp/Python-2.2a1/Python/ceval.c":1913, 0x10058dec] 4 PyEval_EvalCodeEx(0x10231b98, 0x10139924, 0x10139924, 0x0, 0x0, 0x0, 0x0, 0x0) ["/tmp/Python-2.2a1/Python/ceval.c":2518, 0x1005b59c] 5 PyEval_EvalCode(0x10231b98, 0x10139924, 0x10139924, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/ceval.c":491, 0x10053590] 6 run_node(0x1019ac50, 0x7fff3009, 0x10139924, 0x10139924, 0x7fff2df4, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":1057, 0x10062704] 7 run_err_node(0x1019ac50, 0x7fff3009, 0x10139924, 0x10139924, 0x7fff2df4, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":1044, 0x10062660] 8 PyRun_FileExFlags(0xfb4e640, 0x7fff3009, 0x101, 0x10139924, 0x10139924, 0x1, 0x7fff2df4, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":1035, 0x100625e4] 9 PyRun_SimpleFileExFlags(0xfb4e640, 0x7fff3009, 0x1, 0x7fff2df4, 0x5, 0x5, 0x0, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":672, 0x10060dc4] 10 PyRun_AnyFileExFlags(0xfb4e640, 0x7fff3009, 0x1, 0x7fff2df4, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Python/pythonrun.c":482, 0x1006037c] 11 Py_Main(0x2, 0x7fff2ea4, 0x0, 0xfb4e640, 0x5, 0x0, 0x0, 0x0) ["/tmp/Python-2.2a1/Modules/main.c":320, 0x1000d378] 12 ::main(0x2, 0x7fff2ea4, 0xfb4e3d8, 0xc388060, 0x5, 0x5, 0x8, 0x7fff27f7) ["/tmp/Python-2.2a1/Modules/ccpython.cc":10, 0x1000c7ec] 13 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M3/csu/crt1text.s":177, 0x1000c788] ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-03 12:51 Message: Logged In: YES user_id=14422 This no longer seems to happen with the latest CVS (2.2a4+). Closing this bug. ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2001-10-03 12:51 Message: Logged In: YES user_id=14422 This no longer seems to happen with the latest CVS (2.2a4+). Closing this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445991&group_id=5470 From noreply@sourceforge.net Wed Oct 3 22:47:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 14:47:16 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-03 14:47 Message: Logged In: YES user_id=3066 The example code seems to be attached now; thanks! We're increasingly aware of the problems with the profiler. We expect to take care of this in the "right way" for Python 2.2 (by changing the events passed to the profiler) so that the stack management is reliable. We hope to also have a working profiler for Python 2.1.x as well, and sooner rather than later. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:22 Message: Logged In: YES user_id=340060 This is my last attempt to attach the sample code... if it doesn't work, send me mail and I'll send the code. -- Steve (stephen.langer@nist.gov) ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:19 Message: Logged In: YES user_id=340060 last time I forgot the little checkbox... do I get an award for the most empty followups? ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:18 Message: Logged In: YES user_id=340060 Ok, this is ridiculous... I'll see if a different browser lets me attach a file. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Thu Oct 4 02:11:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 18:11:01 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Stephen A. Langer (slanger) >Assigned to: Guido van Rossum (gvanrossum) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 18:11 Message: Logged In: YES user_id=6380 Steve, can you try again with the profile.py from current CVS? It works with Python 1.5.2 through 2.2. If I understand your test program well, this fixes both problems. (We don't always fix deep problems this quickly, but it so happens that this had just come up at Zope Corp. ;-) Closing this as fixed now (ever the optimist...). ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-03 14:47 Message: Logged In: YES user_id=3066 The example code seems to be attached now; thanks! We're increasingly aware of the problems with the profiler. We expect to take care of this in the "right way" for Python 2.2 (by changing the events passed to the profiler) so that the stack management is reliable. We hope to also have a working profiler for Python 2.1.x as well, and sooner rather than later. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:22 Message: Logged In: YES user_id=340060 This is my last attempt to attach the sample code... if it doesn't work, send me mail and I'll send the code. -- Steve (stephen.langer@nist.gov) ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:19 Message: Logged In: YES user_id=340060 last time I forgot the little checkbox... do I get an award for the most empty followups? ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:18 Message: Logged In: YES user_id=340060 Ok, this is ridiculous... I'll see if a different browser lets me attach a file. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Thu Oct 4 06:29:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 22:29:01 -0700 Subject: [Python-bugs-list] [ python-Bugs-467331 ] ClassType.__doc__ always None Message-ID: Bugs item #467331, was opened at 2001-10-02 15:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467331&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: ClassType.__doc__ always None Initial Comment: The docstring for a new-style class is too hard to dig out: >>> class C(object): ... "Hi. I'm C's docstring." ... >>> C.__doc__ # no luck >>> getattr(C, "__doc__") # no luck >>> C.__dict__['__doc__'] # bingo "Hi. I'm C's docstring." >>> Note that in restricted mode, you can't eveb get at C.__dict__ (but we don't want to hide the docstring, right?). If the cause+cure for this isn't instantly known you, assign it back to me and I'll dig into it. Note that things like int.__doc__ and file.__doc__ work fine; it appears to be only user-defined types that suffer here. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-03 22:29 Message: Logged In: YES user_id=31435 Fixed, in the way you thought it would be: Lib/test/test_descr.py; new revision: 1.82 Lib/test/test_doctest2.py; new revision: 1.3 Objects/typeobject.c; new revision: 2.87 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 16:21 Message: Logged In: YES user_id=6380 I'm guessing this is because there's a tp_doc slot in the type object but that's never assigned to in type_new. Enjoy learning about type_new! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467331&group_id=5470 From noreply@sourceforge.net Thu Oct 4 07:22:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 23:22:33 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: Later Priority: 3 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Thu Oct 4 07:33:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 23:33:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-467763 ] test_support's run_suite() is broken Message-ID: Bugs item #467763, was opened at 2001-10-03 23:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467763&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: test_support's run_suite() is broken Initial Comment: When test_support.run_suite() runs a failing test, in one of the failure branches it can reference vrbl "testclass" twice. But testclass is an argument to run_unittest(), and has no value in run_suite(). So run_suite() dies with a NameError, instead of raising TestFailed. Assigned to Fred, because I vaguely believe he made the change so has a good chance of guessing what he intended to do instead . ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467763&group_id=5470 From noreply@sourceforge.net Thu Oct 4 07:34:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 03 Oct 2001 23:34:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-467763 ] test_support's run_suite() is broken Message-ID: Bugs item #467763, was opened at 2001-10-03 23:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467763&group_id=5470 Category: Python Library >Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: test_support's run_suite() is broken Initial Comment: When test_support.run_suite() runs a failing test, in one of the failure branches it can reference vrbl "testclass" twice. But testclass is an argument to run_unittest(), and has no value in run_suite(). So run_suite() dies with a NameError, instead of raising TestFailed. Assigned to Fred, because I vaguely believe he made the change so has a good chance of guessing what he intended to do instead . ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-03 23:34 Message: Logged In: YES user_id=31435 Changed Group to Python 2.2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467763&group_id=5470 From noreply@sourceforge.net Thu Oct 4 09:34:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 01:34:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-467265 ] Compile errors on SuSe Linux on IBM/s390 Message-ID: Bugs item #467265, was opened at 2001-10-02 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bernard Niset (bniset) Assigned to: Tim Peters (tim_one) Summary: Compile errors on SuSe Linux on IBM/s390 Initial Comment: I requested a test account at IBM on one of their IBM/s390 machine. I tried to compile Python-2.2a4.tgz and got the following errors... I submit this as a bug because the version 2.1.1 compile and run fine on this machine. Cheers, Bernard Niset, bn@smartobjects.be gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/fileobject.o Objects/fileobject.c Objects/fileobject.c: In function `file_seek': Objects/fileobject.c:328: warning: `offset' might be used uninitialized in this function Objects/fileobject.c: In function `file_truncate': Objects/fileobject.c:365: warning: `newsize' might be used uninitialized in this function gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/stringobject.o Objects/stringobject.c Objects/stringobject.c: In function `PyString_FromFormatV': Objects/stringobject.c:153: invalid initializer make: *** [Objects/stringobject.o] Error 1 ---------------------------------------------------------------------- >Comment By: Bernard Niset (bniset) Date: 2001-10-04 01:34 Message: Logged In: YES user_id=47338 It compiles now ok... Here are the test results... I guess it is normal that linuxaudiodev fails on such a machine (Mainframe). Cheers, Bernard. 160 tests OK. 1 test failed: test_linuxaudiodev 14 tests skipped: test_al test_cd test_cl test_dl test_gl test_imgfile test_largefile test_nis test_ntpath test_socket_ssl test_socketserver test_sunaudiodev test_winreg test_winsound Those skips are all expected on linux2. ---------------------------------------------------------------------- Comment By: Bernard Niset (bniset) Date: 2001-10-03 00:03 Message: Logged In: YES user_id=47338 OK.. I will test that tonight ... Cheers. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-02 14:35 Message: Logged In: YES user_id=31435 I made some changes that may have repaired the error (the "invalid initializer"). Or maybe not -- I don't have access to a box of this kind, so I guess the only way we'll know is if you try again with current CVS Python. Python/errors.c, rev 2.66 Objects/stringobject.c, rev 2.136 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-02 12:53 Message: Logged In: YES user_id=31435 The two warnings look braindead: offset and newsize are obviously initialized before use. So not inclined to try to do anything about those. The error needs more information: an auto vrbl of type va_list is initialized to a vrbl of type va_list. Why is that invalid? What does va_list resolve to on this box? Is VA_LIST_IS_ARRAY #define'd by the config for this box? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 From noreply@sourceforge.net Thu Oct 4 11:08:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 03:08:13 -0700 Subject: [Python-bugs-list] [ python-Bugs-467811 ] Inability to "see" some module content Message-ID: Bugs item #467811, was opened at 2001-10-04 03:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467811&group_id=5470 Category: None Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tibs (Tony J. Ibbs) (tibs) Assigned to: Nobody/Anonymous (nobody) Summary: Inability to "see" some module content Initial Comment: I am not sure if this is new to 2.2a4 or not - the problem became evident via trying to use compiler.py, and this didn't work in 2.2a3 and earlier for different reasons. In Python 2.2a4, running on Debian Potato, if one imports the "dis" module, and attempts to access (for example) "dis.cmp_op", the result is an AttributeError. Some of the module content can be accessed, some cannot. It does not appear to be related to the __all__ list within the module. I append a file showing the problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467811&group_id=5470 From noreply@sourceforge.net Thu Oct 4 11:29:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 03:29:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-467811 ] Inability to "see" some module content Message-ID: Bugs item #467811, was opened at 2001-10-04 03:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467811&group_id=5470 Category: None Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Tibs (Tony J. Ibbs) (tibs) >Assigned to: Guido van Rossum (gvanrossum) >Summary: Inability to "see" some module content Initial Comment: I am not sure if this is new to 2.2a4 or not - the problem became evident via trying to use compiler.py, and this didn't work in 2.2a3 and earlier for different reasons. In Python 2.2a4, running on Debian Potato, if one imports the "dis" module, and attempts to access (for example) "dis.cmp_op", the result is an AttributeError. Some of the module content can be accessed, some cannot. It does not appear to be related to the __all__ list within the module. I append a file showing the problem. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 03:29 Message: Logged In: YES user_id=6380 Note the traceback: you're getting the compiler package from site-packages. That's not the version that comes with 2.2a4! Unfortunately there's a bug in the install target for the Makefile in 2.2a4 that forgets to install the new packages (compiler and email); get Makefile.pre.in rev 1.60 or later from CVS for a fix. Or you can manually copy the compiler package to the right place. And note that some of the problems with dis are typos: you typed dir instead of dis a few times. I'll close this unless you have a real bug to report that somehow didn't come out... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467811&group_id=5470 From noreply@sourceforge.net Thu Oct 4 11:56:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 03:56:03 -0700 Subject: [Python-bugs-list] [ python-Bugs-467811 ] Inability to "see" some module content Message-ID: Bugs item #467811, was opened at 2001-10-04 03:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467811&group_id=5470 Category: None Group: Python 2.2 Status: Closed Resolution: Fixed Priority: 5 Submitted By: Tibs (Tony J. Ibbs) (tibs) Assigned to: Guido van Rossum (gvanrossum) >Summary: Inability to "see" some module content Initial Comment: I am not sure if this is new to 2.2a4 or not - the problem became evident via trying to use compiler.py, and this didn't work in 2.2a3 and earlier for different reasons. In Python 2.2a4, running on Debian Potato, if one imports the "dis" module, and attempts to access (for example) "dis.cmp_op", the result is an AttributeError. Some of the module content can be accessed, some cannot. It does not appear to be related to the __all__ list within the module. I append a file showing the problem. ---------------------------------------------------------------------- >Comment By: Tibs (Tony J. Ibbs) (tibs) Date: 2001-10-04 03:56 Message: Logged In: YES user_id=297336 Drat - thanks, I hadn't spotted that it was pulling in the "old" version of compiler. I *did* notice the two uses of "dir", but only after posting - sorry. Please close this if you wish and if I find something "real" I'll resubmit a better report. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 03:29 Message: Logged In: YES user_id=6380 Note the traceback: you're getting the compiler package from site-packages. That's not the version that comes with 2.2a4! Unfortunately there's a bug in the install target for the Makefile in 2.2a4 that forgets to install the new packages (compiler and email); get Makefile.pre.in rev 1.60 or later from CVS for a fix. Or you can manually copy the compiler package to the right place. And note that some of the problems with dis are typos: you typed dir instead of dis a few times. I'll close this unless you have a real bug to report that somehow didn't come out... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467811&group_id=5470 From noreply@sourceforge.net Thu Oct 4 15:10:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 07:10:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-467571 ] profile.py mishandles exceptions Message-ID: Bugs item #467571, was opened at 2001-10-03 10:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Stephen A. Langer (slanger) Assigned to: Guido van Rossum (gvanrossum) Summary: profile.py mishandles exceptions Initial Comment: The profiler in profile.py does not correctly handle exceptions raised by Python code and caught in built-in functions. For example, if a class has a __getattr__ function that raises an AttributeError, and that exception is caught by hasattr, the profiler never notices that __getattr__ has returned. This is because it never receives an 'exception' event corresponding to the frame in which the exception was caught. The consequence is that subsequent function calls are attributed to the wrong callers, non-recursive functions may be seen as recursive, and the times reported for functions may be vastly incorrect. I have attached an example program that demonstrates this. I've checked it with Python 1.5.2, 2.0.1 and 2.1.1 on Intel/Linux2.4, and Python 2.1 on SGI/IRIX6.5. I have also written a replacement for profile.py and pstats.py, which I'd be glad to contribute once I've documented it a bit better. It runs a bit slower than the original, but it does get the right answers (I believe). How does one go about making submissions to the library? Thanks. Steve Langer National Institute of Standards and Technology stephen.langer@nist.gov ---------------------------------------------------------------------- >Comment By: Stephen A. Langer (slanger) Date: 2001-10-04 07:10 Message: Logged In: YES user_id=340060 Yes, profile.py from the current CVS fixes the problems. Thanks! -- Steve ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 18:11 Message: Logged In: YES user_id=6380 Steve, can you try again with the profile.py from current CVS? It works with Python 1.5.2 through 2.2. If I understand your test program well, this fixes both problems. (We don't always fix deep problems this quickly, but it so happens that this had just come up at Zope Corp. ;-) Closing this as fixed now (ever the optimist...). ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-03 14:47 Message: Logged In: YES user_id=3066 The example code seems to be attached now; thanks! We're increasingly aware of the problems with the profiler. We expect to take care of this in the "right way" for Python 2.2 (by changing the events passed to the profiler) so that the stack management is reliable. We hope to also have a working profiler for Python 2.1.x as well, and sooner rather than later. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:22 Message: Logged In: YES user_id=340060 This is my last attempt to attach the sample code... if it doesn't work, send me mail and I'll send the code. -- Steve (stephen.langer@nist.gov) ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:19 Message: Logged In: YES user_id=340060 last time I forgot the little checkbox... do I get an award for the most empty followups? ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:18 Message: Logged In: YES user_id=340060 Ok, this is ridiculous... I'll see if a different browser lets me attach a file. ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- Comment By: Stephen A. Langer (slanger) Date: 2001-10-03 11:14 Message: Logged In: YES user_id=340060 Second attempt to attach sample code.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467571&group_id=5470 From noreply@sourceforge.net Thu Oct 4 16:10:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 08:10:29 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Thu Oct 4 16:20:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 08:20:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-467055 ] -flat_namespace not in Makefile.pre.in for Darwin/MacOS Message-ID: Bugs item #467055, was opened at 2001-10-01 19:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 Category: Macintosh Group: Python 2.2 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Mitchell L Model (mlm) Assigned to: Jack Jansen (jackjansen) Summary: -flat_namespace not in Makefile.pre.in for Darwin/MacOS Initial Comment: Python 2.2a4 fixed Python 2.1's inadequate LDSHARED setting for Darwin/OS X as far as building Python is concerned. However, it does not go quite far enough and the problem still exists for building other software that uses a similar configuration process, in particular Zope 2.4(.1). It turns out that the top-level Makefile and Makefile.pre both have -flat_namespace correctly set, but Makefile.pre.in does not. Since Makefile.pre.in gets installed in /usr/local/lib/python2.2/config, and other software using this build process reference Makefile.pre.in from there, -flat_namespace isn't set for the other software. Manually adding: LDSHARED= $(CC) $(LDFLAGS) -flat_namespace -undefined suppress fixed the problem, at least for building Zope 2.4. I don't know how much of that is actually needed other than -flat_namespace. Please fix this for the real release! The improvements for building on Mac OS X are wonderful -- it would be a shame for them to fail for the lack of one line in one file! (I'm using Mac OS 10.1, in case that matters, with the developer tools installed.) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:20 Message: Logged In: YES user_id=6380 Thanks! Closing this bug now. ---------------------------------------------------------------------- Comment By: Mitchell L Model (mlm) Date: 2001-10-02 21:44 Message: Logged In: YES user_id=221340 I started working my way through the Zope build process to see what was going on. Things looked correct far enough into the process that I began to wonder what was going on. In desperation, I renamed /usr/local to something else, unpacked the Python 2.2a4 and Zope 2.4.1 tar files, and remade both. I didn't get the error I got last night. Ugh. I spent some time retracing my steps, but I couldn't figure out what happened. So, false alarm. But I learned a lot! Sorry to bother you with all this. By the way, with the updates for OS 10.1 in Python 2.2a4, all I had to do to make both Python and Zope was: cd Python2.2a4 configure --with-suffix=.exe make sudo make install cd Zope python wo_pcgi.py That's all. Just that one configure flag, and no global variables set. Very, very nice. What a delight to have a just released MacOS treated with respect by multi-platform software! ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 13:41 Message: Logged In: YES user_id=6380 Yes, the build procedure can be quite dazzling -- I don't claim complete knowledge myself any more. :-) Any platform-specific things needed for 3rd party builds *should* be picked up when the Modules/makesetup script runs and transforms the installed Makefile.pre.in into a Makefile. It would be interesting to find out why it doesn't pick up the LDSHARED setting; can you figure that out? When I try this on Linux (not with Zope), the LDSHARED setting chosen by the configure script doe get stuffed into Makefile correctly... ---------------------------------------------------------------------- Comment By: Mitchell L Model (mlm) Date: 2001-10-02 13:33 Message: Logged In: YES user_id=221340 OK, I understand. And I realized later that maybe it would have to be Zope that changes not Python. I was somewhat seduced by the improvement from Python 2.1 to 2.2a4 with respect to building on OS X that I assumed the intention was to fix things entirely and that given how early it is in the Python 2.2 and Mac OS 10.1 games it seemed likely that what I uncovered was a bug. If it was it would be great to have gotten it into the 2.2 release, so I shot out my message last night without completely thinking it through. Although I have an enormous amount of experience in a lot of different languages, systems, and build mechanisms, I'm just recently getting involved in the Open Source world, Python, and OS X, and there's such a huge amount to know about that I'm finding I'm often running on guesses and hopes. In this case, I was assuming that what Python and Zope were doing were both some kind of generally accepted installation procedure -- it certainly looked like Zope was using the Python mechanisms, since it was affected by Makefile.pre.in, etc. I followed enough of the configure and build mechanisms through to figure out where Python was getting -flat_namespace from for its Makefile and Makefile.pre. [This is a complex mechanism! Reminds me of the early days of the X Window System and its imake facility. There's a point at which even the most knowledgeable and experienced human can't follow enough levels of indirection and parameterization to make sense of things....] Since Python installed Makefile.pre.in, and Zope used it, I assumed that was all as intended. If Zope should use a different build process, then nothing needs to be done for Python. However, wouldn't it be easy enough for the Python installation process to modify the Makefile.pre.in that it installs in ../config? Or at least add a release note telling people using Mac OS 10.1 to add a line to the installed file after the installation is complete? Your call. I just wanted to provide a heads-up in case I had discovered something you would want to have fixed. [This is the first time I'm actually contributing anything t SourceForge, rather than just downloading stuff from it. I invite you to email me with suggestions about what I should be saying using the bug tracker and what I shouldn't, or anything else you want to suggest about community assumptions and conventions.] ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 06:59 Message: Logged In: YES user_id=6380 I'm not sure what you're suggesting we *do* here. We can't add -flat_namespace to Makefile.pre.in, because it's a platform-independent file. Also note that there's some name confusion: the Makefile.pre.in that's installed in .../config is not the same one that is used to build Python; it's an old (and deprecated) way to build 3rd party extensions, whose source is in Misc/Makefile.pre.in; the current party line is to use distutils. Maybe Zope should switch to using distutils too. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467055&group_id=5470 From noreply@sourceforge.net Thu Oct 4 16:23:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 08:23:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific Status: Open >Resolution: Remind >Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Martin v. Löwis (loewis) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:23 Message: Logged In: YES user_id=6380 I'm assigning this to Martin, since he checked in the code that tests for -Kpthread, mentioning Patch #418659: Fixes for UnixWare and ReliantUnix. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Thu Oct 4 16:54:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 08:54:14 -0700 Subject: [Python-bugs-list] [ python-Bugs-467924 ] Improve the ZipFile Interface Message-ID: Bugs item #467924, was opened at 2001-10-04 08:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467924&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Improve the ZipFile Interface Initial Comment: There exist two methods to write to a ZipFile write(self, filename, arcname=None, compress_type=None) writestr(self, zinfo, bytes) but only one to read from it read(self, name) Additionally, the two 'write's behave differently with respect to compression. --- (a) 'read' does not fit to 'write', since 'write' takes a file and adds it to a ZipFile, but 'read' is not the reverse operation. 'read' should be called 'readstr' since it much better matches to 'writestr'. (b) It is confusing what 'write' and 'read' actually mean. Does 'write' write a file, or into the ZipFile? It would be more obvious if ZipFile has 4 methods which pair-wise fit together: writestr (self, zinfo, bytes) # same as now readstr (self, name) # returns bytes (as string), currently called 'read' # 'read' could still live but should be deprecated add (self, filename, arcname=None, compress_type=None) # currently 'write' # 'write' could still live but should be deprecated extract (self, name, filename, arcname=None) # new, desired functionality (c) BOTH, 'writestr' and 'add' should by default use the 'compress_type' that was passed to the constructor of 'ZipFile'. Currently, 'write' does it, 'writestr' via zinfo does it not. 'ZipInfo' sets the compression strict to 'ZIP_STORED' :-( It should not do that! It rather should: - allow more parameters in the signature of the constructor to also pass the compression type (and some other attributes, too) - default to 'None', so that 'writestr' can see this, and then take the default from the 'ZipFile' instance. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467924&group_id=5470 From noreply@sourceforge.net Thu Oct 4 20:46:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 12:46:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-467763 ] test_support's run_suite() is broken Message-ID: Bugs item #467763, was opened at 2001-10-03 23:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467763&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: test_support's run_suite() is broken Initial Comment: When test_support.run_suite() runs a failing test, in one of the failure branches it can reference vrbl "testclass" twice. But testclass is an argument to run_unittest(), and has no value in run_suite(). So run_suite() dies with a NameError, instead of raising TestFailed. Assigned to Fred, because I vaguely believe he made the change so has a good chance of guessing what he intended to do instead . ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-04 12:46 Message: Logged In: YES user_id=3066 Sorry, run_suite() wasn't mine, though I've used it. Fixed in Lib/test/test_support.py revision 1.39. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-03 23:34 Message: Logged In: YES user_id=31435 Changed Group to Python 2.2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467763&group_id=5470 From noreply@sourceforge.net Thu Oct 4 21:05:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 13:05:27 -0700 Subject: [Python-bugs-list] [ python-Bugs-467059 ] htmllib broken Message-ID: Bugs item #467059, was opened at 2001-10-01 20:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 Category: Python Library >Group: Irreproducible >Status: Pending Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: htmllib broken Initial Comment: Responding to a question in python-help about extracting links from web pages, I wrote a simple href printer (see attached file). When run using 2.2a4, it never prints anything. It outputs a list of hrefs when run with 2.1 or 1.6. Either there's a bug somewhere (in my code possibly, though it's pretty simple) or some semantics changed that I missed. I thought maybe the method resolution order change affected things, but htmllib.HTMLParser only uses single inheritance. When displaying help about htmllib.HTMLParser, pydoc.help does emit the method resolution order, which it doesn't generally seem to do: class HTMLParser(sgmllib.SGMLParser) | Method resolution order: | HTMLParser | sgmllib.SGMLParser | markupbase.ParserBase ... ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-04 13:05 Message: Logged In: YES user_id=3066 Please attach the input for which this fails. A trivial test case does not fail (see Lib/test/test_htmllib.py). Set status to "pending". ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-01 20:30 Message: Logged In: YES user_id=44345 SF apparently doesn't like file uploads from Opera, so it's pasted here... import htmllib, formatter class MyParser(htmllib.HTMLParser): def anchor_bgn(self, href, name, type): print href fmt = formatter.NullFormatter() parser = MyParser(fmt, verbose=1) parser.feed(open("tour01.html").read()) parser.close() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 From noreply@sourceforge.net Thu Oct 4 21:40:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 13:40:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-467583 ] Restricted Execution docs broken ref Message-ID: Bugs item #467583, was opened at 2001-10-03 11:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467583&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Restricted Execution docs broken ref Initial Comment: The restricted execution documention references an old and broken url to grail. It uses grail.cnri.reston.va.us/grail which no longer resolves. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-04 13:40 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/librestricted.tex revision 1.8, along with other outdated references to CNRI addresses. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467583&group_id=5470 From noreply@sourceforge.net Thu Oct 4 21:52:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 13:52:35 -0700 Subject: [Python-bugs-list] [ python-Bugs-468061 ] __str__ ignored in str subclass Message-ID: Bugs item #468061, was opened at 2001-10-04 13:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: __str__ ignored in str subclass Initial Comment: >From c.l.py; looks messy (e.g., if PyObject_String didn't ignore octetstring.__str__, the call to str() in the latter would just recurse forever; and he's right that b2a_hex requires a genuine string; and e.g. return b2a_hex(self + "") would be mighty obscure). ----Original Message----- From: Andreas.Trawoeger Sent: Thursday, October 04, 2001 7:10 AM To: python-list Subject: Python 2.2a4: Problems with overloading __str__ in subclasses of str I would like to write a class that behaves like a string only with the difference that it displays the string in hex. In Python 2.2a4 this should be possible by subclassing str and overloading __repr__ and __str__ : from binascii import b2a_hex class octetstring(str): def __repr__(self): return b2a_hex(str(self)) def __str__(self): return b2a_hex(str(self)) o = octetstring('A') print "o.__repr__():",o.__repr__() print "o.__str__() :",o.__str__() print "str(o) :",str(o) print "o :",o The result is pretty strange: o.__repr__(): 41 o.__str__() : 41 str(o) : A o : A I'm not shure what's the exact reason for this problem. Either __str__ isn't used for str() and print. Or I'm trapped in an chicken egg problem: I have to convert self to an string (otherwise b2a_hex won't work) to do this Python should use __str__ (which is the method I'm changing). cu andreas ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 From noreply@sourceforge.net Thu Oct 4 21:57:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 13:57:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-468061 ] __str__ ignored in str subclass Message-ID: Bugs item #468061, was opened at 2001-10-04 13:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) >Assigned to: Tim Peters (tim_one) Summary: __str__ ignored in str subclass Initial Comment: >From c.l.py; looks messy (e.g., if PyObject_String didn't ignore octetstring.__str__, the call to str() in the latter would just recurse forever; and he's right that b2a_hex requires a genuine string; and e.g. return b2a_hex(self + "") would be mighty obscure). ----Original Message----- From: Andreas.Trawoeger Sent: Thursday, October 04, 2001 7:10 AM To: python-list Subject: Python 2.2a4: Problems with overloading __str__ in subclasses of str I would like to write a class that behaves like a string only with the difference that it displays the string in hex. In Python 2.2a4 this should be possible by subclassing str and overloading __repr__ and __str__ : from binascii import b2a_hex class octetstring(str): def __repr__(self): return b2a_hex(str(self)) def __str__(self): return b2a_hex(str(self)) o = octetstring('A') print "o.__repr__():",o.__repr__() print "o.__str__() :",o.__str__() print "str(o) :",str(o) print "o :",o The result is pretty strange: o.__repr__(): 41 o.__str__() : 41 str(o) : A o : A I'm not shure what's the exact reason for this problem. Either __str__ isn't used for str() and print. Or I'm trapped in an chicken egg problem: I have to convert self to an string (otherwise b2a_hex won't work) to do this Python should use __str__ (which is the method I'm changing). cu andreas ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-04 13:57 Message: Logged In: YES user_id=31435 Looks like I assigned this to Guido by mistake; back to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 From noreply@sourceforge.net Thu Oct 4 22:08:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 14:08:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-467265 ] Compile errors on SuSe Linux on IBM/s390 Message-ID: Bugs item #467265, was opened at 2001-10-02 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Bernard Niset (bniset) Assigned to: Tim Peters (tim_one) Summary: Compile errors on SuSe Linux on IBM/s390 Initial Comment: I requested a test account at IBM on one of their IBM/s390 machine. I tried to compile Python-2.2a4.tgz and got the following errors... I submit this as a bug because the version 2.1.1 compile and run fine on this machine. Cheers, Bernard Niset, bn@smartobjects.be gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/fileobject.o Objects/fileobject.c Objects/fileobject.c: In function `file_seek': Objects/fileobject.c:328: warning: `offset' might be used uninitialized in this function Objects/fileobject.c: In function `file_truncate': Objects/fileobject.c:365: warning: `newsize' might be used uninitialized in this function gcc -c -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DHAVE_CONFIG_H -o Objects/stringobject.o Objects/stringobject.c Objects/stringobject.c: In function `PyString_FromFormatV': Objects/stringobject.c:153: invalid initializer make: *** [Objects/stringobject.o] Error 1 ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-04 14:08 Message: Logged In: YES user_id=31435 Thanks for following up! Closed. As I mentioned, I'm not going to do anything about the warnings: they don't make sense. Can only imagine this compiler's flow analysis is confused by #if preprocessor statements. linuxaudiodev will fail if you don't have appropriate sound- card drivers, and I agree it's likely this box doesn't have any. ---------------------------------------------------------------------- Comment By: Bernard Niset (bniset) Date: 2001-10-04 01:34 Message: Logged In: YES user_id=47338 It compiles now ok... Here are the test results... I guess it is normal that linuxaudiodev fails on such a machine (Mainframe). Cheers, Bernard. 160 tests OK. 1 test failed: test_linuxaudiodev 14 tests skipped: test_al test_cd test_cl test_dl test_gl test_imgfile test_largefile test_nis test_ntpath test_socket_ssl test_socketserver test_sunaudiodev test_winreg test_winsound Those skips are all expected on linux2. ---------------------------------------------------------------------- Comment By: Bernard Niset (bniset) Date: 2001-10-03 00:03 Message: Logged In: YES user_id=47338 OK.. I will test that tonight ... Cheers. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-02 14:35 Message: Logged In: YES user_id=31435 I made some changes that may have repaired the error (the "invalid initializer"). Or maybe not -- I don't have access to a box of this kind, so I guess the only way we'll know is if you try again with current CVS Python. Python/errors.c, rev 2.66 Objects/stringobject.c, rev 2.136 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-02 12:53 Message: Logged In: YES user_id=31435 The two warnings look braindead: offset and newsize are obviously initialized before use. So not inclined to try to do anything about those. The error needs more information: an auto vrbl of type va_list is initialized to a vrbl of type va_list. Why is that invalid? What does va_list resolve to on this box? Is VA_LIST_IS_ARRAY #define'd by the config for this box? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467265&group_id=5470 From noreply@sourceforge.net Fri Oct 5 00:08:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 16:08:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: Remind Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-04 16:08 Message: Logged In: YES user_id=21627 Just taking out the -Kpthread test is not acceptable. It will mean that thread support will break systems that require it. Likewise, taking it as a fallback will fail on the very same system. Some of those systems (e.g. UnixWare) do provide a libpthread and a pthread_create function within, but the documentation explicitly says that you must not use -lpthread. So we *must* use -Kpthread were available. What exactly is the problem on HP/UX? If the compiler links the test program, why don't threads work then? Can you give a C program that shows that threads are not available when just passing -Kpthread? I agree that special-casing gcc is a hack, but what I need is a solution, not removal of the existing feature. Also, what exact problem is solved with your change "for pthread_create in -lpthread": Why do you need a test program; why is AC_CHECK_LIB not good enough? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:23 Message: Logged In: YES user_id=6380 I'm assigning this to Martin, since he checked in the code that tests for -Kpthread, mentioning Patch #418659: Fixes for UnixWare and ReliantUnix. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Fri Oct 5 00:23:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 16:23:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: Remind Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-10-04 16:23 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-04 16:08 Message: Logged In: YES user_id=21627 Just taking out the -Kpthread test is not acceptable. It will mean that thread support will break systems that require it. Likewise, taking it as a fallback will fail on the very same system. Some of those systems (e.g. UnixWare) do provide a libpthread and a pthread_create function within, but the documentation explicitly says that you must not use -lpthread. So we *must* use -Kpthread were available. What exactly is the problem on HP/UX? If the compiler links the test program, why don't threads work then? Can you give a C program that shows that threads are not available when just passing -Kpthread? I agree that special-casing gcc is a hack, but what I need is a solution, not removal of the existing feature. Also, what exact problem is solved with your change "for pthread_create in -lpthread": Why do you need a test program; why is AC_CHECK_LIB not good enough? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:23 Message: Logged In: YES user_id=6380 I'm assigning this to Martin, since he checked in the code that tests for -Kpthread, mentioning Patch #418659: Fixes for UnixWare and ReliantUnix. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin-A- (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Fri Oct 5 00:41:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 16:41:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-468118 ] gzipmodule.c has wrong URL in comment Message-ID: Bugs item #468118, was opened at 2001-10-04 16:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468118&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Richard Jones (richard) Assigned to: Nobody/Anonymous (nobody) Summary: gzipmodule.c has wrong URL in comment Initial Comment: The URL for the zlib library is incorrect at the top of zlibmodule.c It should be: http://www.gzip.org/zlib/ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468118&group_id=5470 From noreply@sourceforge.net Fri Oct 5 00:46:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 16:46:04 -0700 Subject: [Python-bugs-list] [ python-Bugs-468120 ] setup.py assumes existence of /usr/local Message-ID: Bugs item #468120, was opened at 2001-10-04 16:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Richard Jones (richard) Assigned to: Nobody/Anonymous (nobody) Summary: setup.py assumes existence of /usr/local Initial Comment: /usr/local doesn't exist on Max OS X, so the lines in setup.py at the start of detect_modules should propbably have: if os.path.isdir('/usr/local/lib') and '/usr/local/lib' not in self.compilier.library_dirs: etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 From noreply@sourceforge.net Fri Oct 5 05:16:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 21:16:59 -0700 Subject: [Python-bugs-list] [ python-Bugs-429357 ] non-greedy regexp duplicating match bug Message-ID: Bugs item #429357, was opened at 2001-06-01 09:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=429357&group_id=5470 Category: Regular Expressions Group: None Status: Open Resolution: None Priority: 5 Submitted By: Matthew Mueller (donut) Assigned to: Fredrik Lundh (effbot) Summary: non-greedy regexp duplicating match bug Initial Comment: I found some weird bug, where when a non-greedy match doesn't match anything, it will duplicate the rest of the string instead of being None. #pyrebug.py: import re urlrebug=re.compile(""" (.*?):// #scheme ( (.*?) #user (?: :(.*) #pass )? @)? (.*?) #addr (?::([0-9]+))? #port (/.*)?$ #path """, re.VERBOSE) testbad='foo://bah:81/pth' print urlrebug.match(testbad).groups() Bug Output: >python2.1 pyrebug.py ('foo', None, 'bah:81/pth', None, 'bah', '81', '/pth') >python-cvs pyrebug.py ('foo', None, 'bah:81/pth', None, 'bah', '81', '/pth') Good (expected) Output: >python1.5 pyrebug.py ('foo', None, None, None, 'bah', '81', '/pth') ---------------------------------------------------------------------- >Comment By: Matthew Mueller (donut) Date: 2001-10-04 21:16 Message: Logged In: YES user_id=65253 Ok, after poking and prodding the _sre.c code a bunch until I (hopefully) understand what is happening, I've created a patch. It passes all existing re tests as well as new ones I added for this bug. (I've also made a patch for the similar, but seperate, bug #448951 which I will post there shortly.) ---------------------------------------------------------------------- Comment By: Gregory Smith (gregsmith) Date: 2001-08-30 09:26 Message: Logged In: YES user_id=292741 This looks like the same bug I have reported (with a much simpler example) as #448951 (missed this one before because I was looking for 'group'). What I found is consistent with Jeffrey's comments - if you have a situation where an optional part is fully scanned before the state machine can tell if it should actually be matched, the contained tentative match(es) are stored in the group() even if the optional part turns out to fail. Presumably, such a case needs to be handled by going back and deleting these after the s.m. determines that the optional part was not matched. In my example, I mention a small modification to the test case where the failure of the optional ? is decided one character later (at the end of the () group, not beyond it); this is enough to make it start working again. ---------------------------------------------------------------------- Comment By: Matthew Mueller (donut) Date: 2001-06-14 00:59 Message: Logged In: YES user_id=65253 I think I understand what you are saying, and in the context of the test, it doesn't seem too bad. BUT, my original code (and what I'd like to have) did not have the surrounding group. So I'd just get: ('foo', 'bah:81/pth', None, 'bah', '81', '/pth') Knowing the general ease of messing up regexs when writing them, I'm sure you can image the pain I went through before actually realizing it was a python bug :) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-06-13 10:12 Message: Logged In: NO What's happening makes sense, on one level. When the regex engine gets to the user:pass@ part ((.*?)(?::(.*))?@)? which fill groups 2, 3, and 4, the .*? of group 3 has to try at every character in the rest of the string before admitting overall defeat. In doing that, the last time that group 3 successfully completely locally, it has the rest of the string matched. Of course, overall, group three is enclosed within group 2, and when group two couldn't complete successfully, the engine knows it can skip group two (due to the ? modifying it), so it totally bails on groups 2, 3 and 4 to continue with the rest of the expression. What you'd like to happen is when that "bailing" happens for group 2, the enclosing groups 3 and 4 would get zereoed out (since they didn't participate in the *final* overall match). That makes sense, and is what I would expect to happen. However, what *is* happening is that group 3 is keeping the string that *it* last matched (even thought that last match didn't contribute to the final, overall match). I'm not explaining this well -- I hope you can understand it despite that. Sorry. Jeffrey ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=429357&group_id=5470 From noreply@sourceforge.net Fri Oct 5 05:39:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 04 Oct 2001 21:39:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-448951 ] Bug in re group handling Message-ID: Bugs item #448951, was opened at 2001-08-07 17:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448951&group_id=5470 Category: Regular Expressions Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fredrik Lundh (effbot) Summary: Bug in re group handling Initial Comment: # # read it or run it! # import re,sys print sys.version # # Bug in 're' lib in Python 2.1 # # Consider this regexp : (?:([0-3]):)?0# # This will match one of # '0#', '0:0#', '1:0#', '2:0#', '3:0#' # # The matching itself works fine, but group(1) should # be None for the '0#' case, and 'x' for the 'x:0#' cases. # For '0#', the optional '([0-3]):' part of the # r.e. (enclosed in (?: )) does not match anything, and that # is what contains group 1. # # The actual result is, group(1) is '0' for both '0#' and '0:0#'. # Likely this happens because when '0' is seen, the state machine # cannot not yet determine whether the ([0-3]): should be matched, # but has already seen enough of it to know what group(1) is, assuming # it does match. The match needs to be deleted once the containing # ? fails. Indeed, if the group is expanded to include the ':', # as in '(?:([0-3]:))?0#', or just '([0-3]:)?0#', '0#' produces # group(1)=None as it should. # # Also, this is a good time to point out an error in the # docs. The docs say that group(n) returns -1 when the # group is in an unmatched part the of the r.e.; actually # it returns None, which is more sensible. # rexp = '(?:([0-3]):)?0#' mat1 = re.compile(rexp) print "Re = ", rexp for str in [ '2:0#', '0:0#', '0#', '0:#', ':0#']: print "\n-----<<", str, ">>-----" mat = mat1.match(str) if mat: print " group(0) = ", mat.group(0) print " group(1) = ", mat.group(1) else: print " no match" # # output is below # ################################# # Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32 # Re = (?:([0-3]):)?0# # # -----<< 2:0# >>----- # group(0) = 2:0# # group(1) = 2 # # -----<< 0:0# >>----- # group(0) = 0:0# # group(1) = 0 # # -----<< 0# >>----- # group(0) = 0# # group(1) = 0 # # -----<< 0:# >>----- # no match # # -----<< :0# >>----- # no match # ############################################ ---------------------------------------------------------------------- Comment By: Matthew Mueller (donut) Date: 2001-10-04 21:39 Message: Logged In: YES user_id=65253 I posted a fix as patch #468169 since I don't seem to have access to add it here. ---------------------------------------------------------------------- Comment By: Gregory Smith (gregsmith) Date: 2001-08-30 09:30 Message: Logged In: YES user_id=292741 This appears to be the same bug as #429357, albeit using a simpler test case. I have added a comment to that one. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448951&group_id=5470 From noreply@sourceforge.net Fri Oct 5 16:21:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 08:21:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: Remind Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 08:21 Message: Logged In: YES user_id=21627 I still don't see the breakage on AIX. The compiler will complain about the option; ok. If the binary is executable (and really uses threads), what is the problem? In any case, if you come up with an improved test, I can try it out. On the pthread_create change: +1. I'd like to see a comment on why you need to link there, though. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-04 16:23 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-04 16:08 Message: Logged In: YES user_id=21627 Just taking out the -Kpthread test is not acceptable. It will mean that thread support will break systems that require it. Likewise, taking it as a fallback will fail on the very same system. Some of those systems (e.g. UnixWare) do provide a libpthread and a pthread_create function within, but the documentation explicitly says that you must not use -lpthread. So we *must* use -Kpthread were available. What exactly is the problem on HP/UX? If the compiler links the test program, why don't threads work then? Can you give a C program that shows that threads are not available when just passing -Kpthread? I agree that special-casing gcc is a hack, but what I need is a solution, not removal of the existing feature. Also, what exact problem is solved with your change "for pthread_create in -lpthread": Why do you need a test program; why is AC_CHECK_LIB not good enough? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:23 Message: Logged In: YES user_id=6380 I'm assigning this to Martin, since he checked in the code that tests for -Kpthread, mentioning Patch #418659: Fixes for UnixWare and ReliantUnix. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Fri Oct 5 16:38:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 08:38:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 08:38 Message: Logged In: YES user_id=56897 I've checked the Monitor button while logged in, but I still do not receive email notification of updates. When I clicked it again, it said I was no longer monitoring, so I clicked it yet again back into monitoring mode. Apparently SF knows I'm monitoring it, but it still doesn't send me email. Mail to my @users.sf.net account does work, so I'm at a loss to explain why a) My login cookie doesn't stick around very long at all and b) Why I never get monitor email from SF Re: the patch, I have something that works well on Solaris. I'll try it on Linux today, but I don't have access to an HP-UX system. I'm a little concerned about the impact to HP-UX (pre 11.0 and post 11.0) and AIX, and I don't have access to those machines to check out those concerns. Hopefully I'll have the patch posted by today. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Fri Oct 5 18:09:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 10:09:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 10:09 Message: Logged In: YES user_id=56897 I've submitted the patch for pthread signal masking. My biggest concerns are the guesses I made for DCE threads and whether they will work for AIX which might need to use sigthreadmask(). Regarding reproducing this on Linux, I was able to get Linux to crash if I held down Ctrl-C (with fairly fast key repeat). After starting the spinning thread, Python would crash on Linux under a storm of SIGINTs within 30 seconds or so. Without the spinning thread, I couldn't get it to crash. With the patch applied, the spinning thread running during the storm of SIGINTs wouldn't crash it. So that implies the signal masking is doing something good even in the Linux case. Re: my SF problems, I submitted a few support requests. Hopefully something gets fixed. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 08:38 Message: Logged In: YES user_id=56897 I've checked the Monitor button while logged in, but I still do not receive email notification of updates. When I clicked it again, it said I was no longer monitoring, so I clicked it yet again back into monitoring mode. Apparently SF knows I'm monitoring it, but it still doesn't send me email. Mail to my @users.sf.net account does work, so I'm at a loss to explain why a) My login cookie doesn't stick around very long at all and b) Why I never get monitor email from SF Re: the patch, I have something that works well on Solaris. I'll try it on Linux today, but I don't have access to an HP-UX system. I'm a little concerned about the impact to HP-UX (pre 11.0 and post 11.0) and AIX, and I don't have access to those machines to check out those concerns. Hopefully I'll have the patch posted by today. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Fri Oct 5 18:13:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 10:13:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: Remind Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-05 10:13 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 08:21 Message: Logged In: YES user_id=21627 I still don't see the breakage on AIX. The compiler will complain about the option; ok. If the binary is executable (and really uses threads), what is the problem? In any case, if you come up with an improved test, I can try it out. On the pthread_create change: +1. I'd like to see a comment on why you need to link there, though. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-04 16:23 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-04 16:08 Message: Logged In: YES user_id=21627 Just taking out the -Kpthread test is not acceptable. It will mean that thread support will break systems that require it. Likewise, taking it as a fallback will fail on the very same system. Some of those systems (e.g. UnixWare) do provide a libpthread and a pthread_create function within, but the documentation explicitly says that you must not use -lpthread. So we *must* use -Kpthread were available. What exactly is the problem on HP/UX? If the compiler links the test program, why don't threads work then? Can you give a C program that shows that threads are not available when just passing -Kpthread? I agree that special-casing gcc is a hack, but what I need is a solution, not removal of the existing feature. Also, what exact problem is solved with your change "for pthread_create in -lpthread": Why do you need a test program; why is AC_CHECK_LIB not good enough? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:23 Message: Logged In: YES user_id=6380 I'm assigning this to Martin, since he checked in the code that tests for -Kpthread, mentioning Patch #418659: Fixes for UnixWare and ReliantUnix. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Fri Oct 5 20:37:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 12:37:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-468384 ] os.P_* constants poorly documented Message-ID: Bugs item #468384, was opened at 2001-10-05 12:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468384&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: os.P_* constants poorly documented Initial Comment: The P_* constants in the os module are not well documented: Possible values for the mode parameter to spawnv() and spawnve(). Since these constants originated in the Windows spawn* functions, they are not likely to be well-understood outside that environment. P_WAIT and P_NOWAIT have obvious meanings, but the others are unclear (at least to me). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468384&group_id=5470 From noreply@sourceforge.net Fri Oct 5 21:59:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 13:59:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-417176 ] MultiFile.read() includes CRLF boundary Message-ID: Bugs item #417176, was opened at 2001-04-18 15:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417176&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Martijn Pieters (mjpieters) Assigned to: Guido van Rossum (gvanrossum) Summary: MultiFile.read() includes CRLF boundary Initial Comment: multifile.MultiFile.readlines()and .read() will return a body of a multipart message including the line delimiter that is to be regarded part of the boundary. In a partial multipart message like: --BoundaryHere Content-Type: text/plain 1 2 3 4 --BoundaryHere the message within the delimiters does not include the final line delimiter (CRLF or LF or whatnot) after the line reading '4'; it is considered part of the boundary. MultiFile however, returns it as part of the body. See RFC2046 section 5.1.1. In the usual text formatting of the RFC, you'll find the definition and explanation in the first two paragraphs of page 19. ---------------------------------------------------------------------- >Comment By: Martijn Pieters (mjpieters) Date: 2001-10-05 13:59 Message: Logged In: YES user_id=116747 I just found again where I ran into this problem; in the Zope HTTP Range header test suite. The code generates RFC compliant multi-part mime responses and the test suite uses MessageFile to see if the correct parts are returned. See expectMultipleRanges in: http://cvs.zope.org/Zope/lib/python/OFS/tests/testRanges.py?rev=1.3&content-type=text/vnd.viewcvs-markup Right now there is code there that catches the extra that's part of the boundary and strips this off; this fails with Python 2.2a4 because now the \n is stripped but the \r is still attached! I am more and more convinced that MessageFile should not expect that the line endings have been normalized to UNIX only. Instead, it should handle at least the UNIX \n and the RFC-compliant \r\n situations. ---------------------------------------------------------------------- Comment By: Martijn Pieters (mjpieters) Date: 2001-09-18 09:16 Message: Logged In: YES user_id=116747 Okay, if all the code depends on line-endings being Unix-style, the patch has my blessings. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-18 08:40 Message: Logged In: YES user_id=6380 I think that CRLF support in this case isn't worth it. It's not done elsewhere in this module -- it assumes that line endings have already been converted to Unix style. Lone CR is definitely not supported -- none of the code would work. ---------------------------------------------------------------------- Comment By: Martijn Pieters (mjpieters) Date: 2001-09-18 08:09 Message: Logged In: YES user_id=116747 Your patch looks sound, apart from the fact it'll only remove a LF. The Spec says the CRLF is part of the boundary, and, to account for broken implementations, it should probably remove and of 'CRLF', 'LF', or 'CR' at the end. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-18 07:34 Message: Logged In: YES user_id=6380 I've checked in the patch now. Still waiting for Martijn's feedback before I close the report. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-13 12:59 Message: Logged In: YES user_id=6380 Martijn, here's a fix. Can you test this? The fix works (how else) by reading ahead one line and stripping the final newline if the next line is empty. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 10:54 Message: Logged In: YES user_id=6380 I wrote that code and I'm probably culpable. It's also always bothered me. Unassigning it from Barry (it has nothing to do with Barry). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417176&group_id=5470 From noreply@sourceforge.net Fri Oct 5 23:56:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 15:56:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-467381 ] print statement and exception Message-ID: Bugs item #467381, was opened at 2001-10-02 19:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 Category: Python Interpreter Core Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: paul rubin (phr) Assigned to: Guido van Rossum (gvanrossum) Summary: print statement and exception Initial Comment: a = 3 b = 0 try: print "quotient is",a/b except: pass print "sum is", a+b prints something like "quotient is sum is 3". The first print statement prints "quotient is" before the exception is thrown, and then doesn't print the final newline. The final newline should be protected by the equivalent of a "finally" clause. ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-05 15:56 Message: Logged In: YES user_id=72053 I believe that's inconsistent with the documentation, (http://www.python.org/doc/current/ref/print.html), which says 'A "\n" character is written at the end, unless the print statement ends with a comma.'. It also says "print evaluates each expression in turn and writes the resulting object to the standard output", which could be interpreted to mean it builds up a tuple of the value of all the expressions, then prints the tuple. In that situation, an exception evaluating one of the expressions could result in printing nothing at all, or just printing a blank line. Anyway, I never would have guessed the actual behavior by reading the documentation. At minimum, the doc should be clarified. Regards Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 20:08 Message: Logged In: YES user_id=6380 Sorry, this is how it's defined. The print statement does its things one at a time. If one of its things raises an exception, then the remaining things won't be printed -- including the newline. If that's not what you want, calculate the value and catch the exception before printing the value, or use a print with a trailing comma (suppresses the newline printing) and use an empty print statement separately (supplies the newline), like this: try: print "quotient is", a/b, except: pass print ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 From noreply@sourceforge.net Sat Oct 6 00:02:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 16:02:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-447945 ] time.time() is not non-decreasing Message-ID: Bugs item #447945, was opened at 2001-08-04 08:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=447945&group_id=5470 Category: Documentation Group: Not a Bug Status: Closed Resolution: Fixed Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: time.time() is not non-decreasing Initial Comment:

After spending many hours tracking down weird race conditions in Mojo Nation, I've finally realized that the problem is that we assumed that time.time() would return non-decreasing answers. In fact, successive calls to time.time() can return a smaller number than previous calls, as demonstrated by this test (Python 2.0 on debian woody):

import time

x = 0
while 1:
    ox = x
    x = time.time()
    if x < ox:
        print "this is WRONG: ox: %s, x: %s\n" % (ox,
x,)
    else:
        print ".",

In order to get the race conditions out of Mojo Nation, I'm writing a non_decreasing_time(), which does what we thought time() would do. In fact, I might just make it increasing_time(). In any case, the time.time() in the standard library should either be fixed (my preference) to be non-decreasing, or the documentation should be updated to warn about the surprise.

Regards,

Zooko

---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-05 16:02 Message: Logged In: YES user_id=72053 You're using poor system software if it's setting the clock back to adjust for clock skew. The correct way to handle that is to slow down the clock updates slightly, so the correct time "catches up" with the clock time. That's the right way to do it because of the precise problem you encountered. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-08-05 08:44 Message: Logged In: YES user_id=3066 Added a note about time() being non-decreasing except when the system clock is set back; see Doc/lib/libtime.tex revision 1.43. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-04 13:01 Message: Logged In: YES user_id=6380 Zooko writes: """ It turns out that it *is* because my system clock is getting reset, to adjust for clock skew. I'm testing my new `increasing_time()' function which I can rely on to always return higher values. I'll post about it to python-list. Sorry for the bug report which was really just a problem with a frequently misunderstood semantics. Perhaps the time.time() docs should warn you about the possibility of getting different results. """ Assigning to Fred for doco update. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-04 11:03 Message: Logged In: YES user_id=6380 I don't understand why this can happen. time.time() is a simple wrapper around either gettimeofday(), ftime(), or time(). So you should be able to reproduce this in C as well, and it should be a kernel bug, unless somehow the conversion to float is busted. Can you investigate? Of course, if the super-user resets the system clock, time.time() may also return non-monononous values -- I assume that that's not what you're seeing -- but that's a reason why Python can't guarantee a monotonously increasing time. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=447945&group_id=5470 From noreply@sourceforge.net Sat Oct 6 00:05:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 16:05:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-465045 ] base n integer to string conversion Message-ID: Bugs item #465045, was opened at 2001-09-25 18:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: base n integer to string conversion Initial Comment: I see that the C files for int and long arithmetic already contain format() functions to format the numbers in arbitrary bases. The Python-exported decimal and hex conversion routines simply call the format function. I have three suggestions/requests, in increasing order of controversy: 1) How about exporting the format routine to Python, so there's a builtin function for converting an integer object to an arbitrary base 2<=b<=36. The code is already there--may as well let people use it. Something like format(n,base) would be a simple interface. 2) Right now there's no simple way to convert an integer to a raw byte string. I end up doing something like a = binascii.unhexlify("%x"%n) which is a kludge. I'd like to propose that as a special case, format(n,256) give the raw bytes. 3) Finally, if it's not too weird, format(n,128) could possibly return the BER (base 128) encoded representation of n, similar to pack("w",n) in perl. Bit 8 (the high bit) is set in each byte except the last. That means you can read BER integers byte by byte and know where they end. They're used in various security objects like X509 certificates. If not here, then sooner or later this function should appear in some library module. Thanks Paul ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-05 16:05 Message: Logged In: YES user_id=72053 Re which bases are useful: I want base 256 all the time and find the lack of any clean way to convert ints to binary is a surprising deficiency in Python. I'm not too concerned about what the interface should be, but I hope one is added. I agree, BER is a bit obscure, but if you feel it's ok to put it in the binascii module, I might submit a patch. How about if I put base-256 there too? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-26 09:00 Message: Logged In: YES user_id=31435 1) Base 2 is semi-frequently requested. Note that Greg Wilson has open patch 455076 to add %b format and bin() function. No non-binary base has any fans. 2) I expect Paul means unbounded ints (based on previous requests). 3) Yup. Jeremy's Pisces may already deal w/ this (found via Google search). 3') Pickle already represents ints < 256 in one byte, and less than 2**16 in 2 (plus a 1-byte type code, of course). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-26 06:38 Message: Logged In: YES user_id=6380 1) Which bases besides 8, 10 and 16 do you plan to use? The C code strikes me as more general than needed. 2) You can do this with the struct module. No need for kludges. 3) BER seems pretty esoteric. The binascii module seems the right place. Feel free to submit a patch! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-09-25 18:45 Message: Logged In: YES user_id=72053 One minor comment: besides representing longs, BER encoded integers are also good for small integers. Numbers < 128 are represented in one byte. If short strings were marshalled with a BER-encoded length instead of a 4-byte length, that could save some space in .pyc files and pickled objects. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 From noreply@sourceforge.net Sat Oct 6 00:09:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 16:09:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-468432 ] add more object sizes to struct Message-ID: Bugs item #468432, was opened at 2001-10-05 16:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468432&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: add more object sizes to struct Initial Comment: It would be good if the struct module could deal with 64-bit, 32-bit, 4-bit, and 1-bit objects. It currently handles only 8 and 16 bits. There's an 'I' type which is the size of a C int, but that's 32 bits on some systems and 64 on others--maybe even 16 on some. I propose adding the format letters: t- one bit n (nibble) - 4 bits q (quadbyte) - 32 bits o (octbyte) - 64 bits q and o should support signed and unsigned variants. Signed bits and nibbles probably aren't that useful, though they could be supported for completeness sake. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468432&group_id=5470 From noreply@sourceforge.net Sat Oct 6 00:32:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 16:32:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-468432 ] add more object sizes to struct Message-ID: Bugs item #468432, was opened at 2001-10-05 16:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468432&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: add more object sizes to struct Initial Comment: It would be good if the struct module could deal with 64-bit, 32-bit, 4-bit, and 1-bit objects. It currently handles only 8 and 16 bits. There's an 'I' type which is the size of a C int, but that's 32 bits on some systems and 64 on others--maybe even 16 on some. I propose adding the format letters: t- one bit n (nibble) - 4 bits q (quadbyte) - 32 bits o (octbyte) - 64 bits q and o should support signed and unsigned variants. Signed bits and nibbles probably aren't that useful, though they could be supported for completeness sake. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-05 16:32 Message: Logged In: YES user_id=31435 struct has more than mode. Native mode follows the platform C in all respects (size, alignment and endianness). The standard modes are platform-independent, and 'i' and 'I' are 32 bits on all platforms in standard modes. 2.2 struct also has corresponding 'q' and 'Q' format codes for signed and unsigned 64-bit ints in standard modes, and whatever "long long" means to C in native mode (or __int64 in Windows native mode). Note that the purpose of struct is for passing data in/out to/from C structs, and so non-C things like nibbles and bits are going to be a hard sell. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468432&group_id=5470 From noreply@sourceforge.net Sat Oct 6 00:43:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 16:43:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-468443 ] Python download/install process Message-ID: Bugs item #468443, was opened at 2001-10-05 16:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468443&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Talin (talin) Assigned to: Nobody/Anonymous (nobody) Summary: Python download/install process Initial Comment: I just guided a friend through the process of downloading and installing Python 2.1.1. Here is the transcript of our conversation. See comments below. ---- Deanna: Talin, I want to d/l Python but I don't see where the download is. can you help me? Talin: What does the web page title say? Deanna: Python 2.1.1 - a bugfix release for Python 2.1 Talin: OK, scroll down to where it says "download locations" and click on the link that says HTTP Deanna: got it Talin: Click on the file ending with .exe and download it Talin: Then run that file Deanna: k Talin: That's an installer. Deanna: and the others? Talin: Just that one Deanna: thanks ---- Normally I wouldn't bother to mention this - people who use programming languages should know how to download and install things. However, since Python is supposed to be intended for novice computer users, I think it would make sense to improve the download process and make it more self-explanatory. In the above transcript, some things are apparent: 1) The download page is kind of terse - the "Download Locations" simply says "Python.org HTTP". Not everyone knows that "HTTP" is a way of downloading files, most people think it's just a way of viewing web pages. (Most people don't realize that web pages are files.) 2) The link leads you to a raw listing of files, with no explanation of which files to download. 3) Since my friend is a windows user, she doesn't know that ".tar.gz" files are typically only used on Unix systems. A similar logic can be made about the other files in that directory. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468443&group_id=5470 From noreply@sourceforge.net Sat Oct 6 01:10:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 17:10:53 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 6 01:13:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 17:13:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-468120 ] setup.py assumes existence of /usr/local Message-ID: Bugs item #468120, was opened at 2001-10-04 16:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Richard Jones (richard) Assigned to: Nobody/Anonymous (nobody) Summary: setup.py assumes existence of /usr/local Initial Comment: /usr/local doesn't exist on Max OS X, so the lines in setup.py at the start of detect_modules should propbably have: if os.path.isdir('/usr/local/lib') and '/usr/local/lib' not in self.compilier.library_dirs: etc. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:13 Message: Logged In: YES user_id=21627 Why is that a problem? What happens with the current code? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 From noreply@sourceforge.net Sat Oct 6 01:23:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 17:23:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-468443 ] Python download/install process Message-ID: Bugs item #468443, was opened at 2001-10-05 16:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468443&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Talin (talin) Assigned to: Nobody/Anonymous (nobody) Summary: Python download/install process Initial Comment: I just guided a friend through the process of downloading and installing Python 2.1.1. Here is the transcript of our conversation. See comments below. ---- Deanna: Talin, I want to d/l Python but I don't see where the download is. can you help me? Talin: What does the web page title say? Deanna: Python 2.1.1 - a bugfix release for Python 2.1 Talin: OK, scroll down to where it says "download locations" and click on the link that says HTTP Deanna: got it Talin: Click on the file ending with .exe and download it Talin: Then run that file Deanna: k Talin: That's an installer. Deanna: and the others? Talin: Just that one Deanna: thanks ---- Normally I wouldn't bother to mention this - people who use programming languages should know how to download and install things. However, since Python is supposed to be intended for novice computer users, I think it would make sense to improve the download process and make it more self-explanatory. In the above transcript, some things are apparent: 1) The download page is kind of terse - the "Download Locations" simply says "Python.org HTTP". Not everyone knows that "HTTP" is a way of downloading files, most people think it's just a way of viewing web pages. (Most people don't realize that web pages are files.) 2) The link leads you to a raw listing of files, with no explanation of which files to download. 3) Since my friend is a windows user, she doesn't know that ".tar.gz" files are typically only used on Unix systems. A similar logic can be made about the other files in that directory. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:23 Message: Logged In: YES user_id=21627 Your point 2) is not true. Above the download locations, it say "Windows users should download Python-2.1.1.exe, the Windows installer, from one of the download locations below, run it, and follow the friendly instructions on the screen to complete the installation." It also explains Python-2.1.1-Debug.zip. This also makes your point 3) irrelevant: the page clearly says that the .tgz is for "All others". On HTTP/FTP, somebody who doesn't know what to do would hopefully be smart enough to click on the link that follows the text "Download locations", perhaps assuming that these were different locations. So I propose to close this with no change. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468443&group_id=5470 From noreply@sourceforge.net Sat Oct 6 03:05:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 19:05:53 -0700 Subject: [Python-bugs-list] [ python-Bugs-467381 ] print statement and exception Message-ID: Bugs item #467381, was opened at 2001-10-02 19:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 Category: Python Interpreter Core Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: paul rubin (phr) Assigned to: Guido van Rossum (gvanrossum) Summary: print statement and exception Initial Comment: a = 3 b = 0 try: print "quotient is",a/b except: pass print "sum is", a+b prints something like "quotient is sum is 3". The first print statement prints "quotient is" before the exception is thrown, and then doesn't print the final newline. The final newline should be protected by the equivalent of a "finally" clause. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-05 19:05 Message: Logged In: YES user_id=6380 Well, the docs don't say at all what happens when an exception occurs. I don't want to argue over this any more, but I note that the use of the phrase "in turn" was meant to suggest that each value is written as soon as it's available. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-05 15:56 Message: Logged In: YES user_id=72053 I believe that's inconsistent with the documentation, (http://www.python.org/doc/current/ref/print.html), which says 'A "\n" character is written at the end, unless the print statement ends with a comma.'. It also says "print evaluates each expression in turn and writes the resulting object to the standard output", which could be interpreted to mean it builds up a tuple of the value of all the expressions, then prints the tuple. In that situation, an exception evaluating one of the expressions could result in printing nothing at all, or just printing a blank line. Anyway, I never would have guessed the actual behavior by reading the documentation. At minimum, the doc should be clarified. Regards Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-02 20:08 Message: Logged In: YES user_id=6380 Sorry, this is how it's defined. The print statement does its things one at a time. If one of its things raises an exception, then the remaining things won't be printed -- including the newline. If that's not what you want, calculate the value and catch the exception before printing the value, or use a print with a trailing comma (suppresses the newline printing) and use an empty print statement separately (supplies the newline), like this: try: print "quotient is", a/b, except: pass print ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467381&group_id=5470 From noreply@sourceforge.net Sat Oct 6 03:09:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 19:09:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-465045 ] base n integer to string conversion Message-ID: Bugs item #465045, was opened at 2001-09-25 18:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: base n integer to string conversion Initial Comment: I see that the C files for int and long arithmetic already contain format() functions to format the numbers in arbitrary bases. The Python-exported decimal and hex conversion routines simply call the format function. I have three suggestions/requests, in increasing order of controversy: 1) How about exporting the format routine to Python, so there's a builtin function for converting an integer object to an arbitrary base 2<=b<=36. The code is already there--may as well let people use it. Something like format(n,base) would be a simple interface. 2) Right now there's no simple way to convert an integer to a raw byte string. I end up doing something like a = binascii.unhexlify("%x"%n) which is a kludge. I'd like to propose that as a special case, format(n,256) give the raw bytes. 3) Finally, if it's not too weird, format(n,128) could possibly return the BER (base 128) encoded representation of n, similar to pack("w",n) in perl. Bit 8 (the high bit) is set in each byte except the last. That means you can read BER integers byte by byte and know where they end. They're used in various security objects like X509 certificates. If not here, then sooner or later this function should appear in some library module. Thanks Paul ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-05 19:09 Message: Logged In: YES user_id=6380 Various forms of fixed-length int to binary conversion are provided by the struct module. What's a "surprising deficiency" for you could simply be everybody else's lack of interest -- if nobody needs it, it won't be added to the language... :-) ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-05 16:05 Message: Logged In: YES user_id=72053 Re which bases are useful: I want base 256 all the time and find the lack of any clean way to convert ints to binary is a surprising deficiency in Python. I'm not too concerned about what the interface should be, but I hope one is added. I agree, BER is a bit obscure, but if you feel it's ok to put it in the binascii module, I might submit a patch. How about if I put base-256 there too? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-26 09:00 Message: Logged In: YES user_id=31435 1) Base 2 is semi-frequently requested. Note that Greg Wilson has open patch 455076 to add %b format and bin() function. No non-binary base has any fans. 2) I expect Paul means unbounded ints (based on previous requests). 3) Yup. Jeremy's Pisces may already deal w/ this (found via Google search). 3') Pickle already represents ints < 256 in one byte, and less than 2**16 in 2 (plus a 1-byte type code, of course). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-26 06:38 Message: Logged In: YES user_id=6380 1) Which bases besides 8, 10 and 16 do you plan to use? The C code strikes me as more general than needed. 2) You can do this with the struct module. No need for kludges. 3) BER seems pretty esoteric. The binascii module seems the right place. Feel free to submit a patch! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-09-25 18:45 Message: Logged In: YES user_id=72053 One minor comment: besides representing longs, BER encoded integers are also good for small integers. Numbers < 128 are represented in one byte. If short strings were marshalled with a BER-encoded length instead of a 4-byte length, that could save some space in .pyc files and pickled objects. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 From noreply@sourceforge.net Sat Oct 6 03:21:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 05 Oct 2001 19:21:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-468443 ] Python download/install process Message-ID: Bugs item #468443, was opened at 2001-10-05 16:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468443&group_id=5470 Category: Installation Group: Feature Request >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Talin (talin) Assigned to: Nobody/Anonymous (nobody) Summary: Python download/install process Initial Comment: I just guided a friend through the process of downloading and installing Python 2.1.1. Here is the transcript of our conversation. See comments below. ---- Deanna: Talin, I want to d/l Python but I don't see where the download is. can you help me? Talin: What does the web page title say? Deanna: Python 2.1.1 - a bugfix release for Python 2.1 Talin: OK, scroll down to where it says "download locations" and click on the link that says HTTP Deanna: got it Talin: Click on the file ending with .exe and download it Talin: Then run that file Deanna: k Talin: That's an installer. Deanna: and the others? Talin: Just that one Deanna: thanks ---- Normally I wouldn't bother to mention this - people who use programming languages should know how to download and install things. However, since Python is supposed to be intended for novice computer users, I think it would make sense to improve the download process and make it more self-explanatory. In the above transcript, some things are apparent: 1) The download page is kind of terse - the "Download Locations" simply says "Python.org HTTP". Not everyone knows that "HTTP" is a way of downloading files, most people think it's just a way of viewing web pages. (Most people don't realize that web pages are files.) 2) The link leads you to a raw listing of files, with no explanation of which files to download. 3) Since my friend is a windows user, she doesn't know that ".tar.gz" files are typically only used on Unix systems. A similar logic can be made about the other files in that directory. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:23 Message: Logged In: YES user_id=21627 Your point 2) is not true. Above the download locations, it say "Windows users should download Python-2.1.1.exe, the Windows installer, from one of the download locations below, run it, and follow the friendly instructions on the screen to complete the installation." It also explains Python-2.1.1-Debug.zip. This also makes your point 3) irrelevant: the page clearly says that the .tgz is for "All others". On HTTP/FTP, somebody who doesn't know what to do would hopefully be smart enough to click on the link that follows the text "Download locations", perhaps assuming that these were different locations. So I propose to close this with no change. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468443&group_id=5470 From noreply@sourceforge.net Sun Oct 7 08:38:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 07 Oct 2001 00:38:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-468120 ] setup.py assumes existence of /usr/local Message-ID: Bugs item #468120, was opened at 2001-10-04 16:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Richard Jones (richard) Assigned to: Nobody/Anonymous (nobody) Summary: setup.py assumes existence of /usr/local Initial Comment: /usr/local doesn't exist on Max OS X, so the lines in setup.py at the start of detect_modules should propbably have: if os.path.isdir('/usr/local/lib') and '/usr/local/lib' not in self.compilier.library_dirs: etc. ---------------------------------------------------------------------- >Comment By: Richard Jones (richard) Date: 2001-10-07 00:38 Message: Logged In: YES user_id=6405 It generates a warning, several lines long, for each compile command. Doesn't break, just makes the compile more verbose than necessary :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:13 Message: Logged In: YES user_id=21627 Why is that a problem? What happens with the current code? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 From noreply@sourceforge.net Sun Oct 7 20:57:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 07 Oct 2001 12:57:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-468120 ] setup.py assumes existence of /usr/local Message-ID: Bugs item #468120, was opened at 2001-10-04 16:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Richard Jones (richard) Assigned to: Nobody/Anonymous (nobody) Summary: setup.py assumes existence of /usr/local Initial Comment: /usr/local doesn't exist on Max OS X, so the lines in setup.py at the start of detect_modules should propbably have: if os.path.isdir('/usr/local/lib') and '/usr/local/lib' not in self.compilier.library_dirs: etc. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-07 12:57 Message: Logged In: YES user_id=21627 Please give as much detail as possible. What is the exact text of the warning message? Can you guess what program is generating it? When I add a non-existing directory in my copy of setup.py, no disturbing warnings are produced. ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-07 00:38 Message: Logged In: YES user_id=6405 It generates a warning, several lines long, for each compile command. Doesn't break, just makes the compile more verbose than necessary :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:13 Message: Logged In: YES user_id=21627 Why is that a problem? What happens with the current code? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468120&group_id=5470 From noreply@sourceforge.net Mon Oct 8 02:20:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 07 Oct 2001 18:20:53 -0700 Subject: [Python-bugs-list] [ python-Bugs-468887 ] type(self) not preserved on some methods Message-ID: Bugs item #468887, was opened at 2001-10-07 18:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468887&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark J (average) Assigned to: Guido van Rossum (gvanrossum) Summary: type(self) not preserved on some methods Initial Comment: Python-2.2a4 For user-defined types derived from built-in types, it appears that built-in methods which return copies are not preserving the type of self. >>> class Test(list): pass >>> t=Test() >>> map(type, [t, t[:], t+t]) [, , ] >>> t += t >>> type(t) Same with types int, dictionary, etc. Thanks, Mark ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468887&group_id=5470 From noreply@sourceforge.net Mon Oct 8 05:04:31 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 07 Oct 2001 21:04:31 -0700 Subject: [Python-bugs-list] [ python-Bugs-468887 ] type(self) not preserved on some methods Message-ID: Bugs item #468887, was opened at 2001-10-07 18:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468887&group_id=5470 Category: Type/class unification >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Mark J (average) >Assigned to: Tim Peters (tim_one) Summary: type(self) not preserved on some methods Initial Comment: Python-2.2a4 For user-defined types derived from built-in types, it appears that built-in methods which return copies are not preserving the type of self. >>> class Test(list): pass >>> t=Test() >>> map(type, [t, t[:], t+t]) [, , ] >>> t += t >>> type(t) Same with types int, dictionary, etc. Thanks, Mark ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-07 21:04 Message: Logged In: YES user_id=31435 This is deliberate. If, e.g., you don't override the slicing operator in Test, then t[:] is handled by the base class's slicing operator. The base class can't know what invariants Test needs to preserve, so does the best it can by constructing a list. If you want operators that return Test instances instead, you have to supply them. Note that base-class operators *sometimes* returned instances of subclasses in earlier 2.2 alphas, but unpredictably (depending on internal optimizations). This was properly reported as a bug, and was fixed for 2.2a4; see bug 460020 for details. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468887&group_id=5470 From noreply@sourceforge.net Mon Oct 8 05:19:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 07 Oct 2001 21:19:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-468948 ] urllib2, basic authentication, & 302 Message-ID: Bugs item #468948, was opened at 2001-10-07 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeffrey C. Ollie (jcollie) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2, basic authentication, & 302 Initial Comment: I've found a bug in how urllib2 handles authentication. The crux of the problem is that the AbstractBasicAuthHandler.__current_realm is basically a global variable. I discovered the bug because I needed to use HTTP basic authentication and the HTTP If-Modified-Since header. Since the HTTP server returns a 302 error if the requested URL has not changed the line of code that resets AbstractBasicAuthHandler.__current_realm back to None never gets executed because an exception will be raised when the retrieval is retried. I suspect that this bug would also cause problems in multi-threaded code. The digest authentication appears to have similar problems. The solution that I found is to get rid of the __current_realm attribute and prevent infinite retries by checking for the presence of an Authenticate: header in the request object that exactly matches the Authenticate: header that would be added. The bug exists in 2.1.1, 2.2a4 and the current CVS. Patch attached. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 From noreply@sourceforge.net Mon Oct 8 14:21:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 08 Oct 2001 06:21:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: Remind Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-08 06:21 Message: Logged In: YES user_id=21627 I have now made the -Kpthread more robust by running a program; this allows to remove the gcc check (atleast for Linux and Solaris). The same check should work on all systems that require special options for pthread support; please confirm whether it solves the AIX problem. I've also applied your patch to check pthread_create linkage through including pthread.h; please report whether configuration now succeeds on HP/UX. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-05 10:13 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 08:21 Message: Logged In: YES user_id=21627 I still don't see the breakage on AIX. The compiler will complain about the option; ok. If the binary is executable (and really uses threads), what is the problem? In any case, if you come up with an improved test, I can try it out. On the pthread_create change: +1. I'd like to see a comment on why you need to link there, though. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-04 16:23 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-04 16:08 Message: Logged In: YES user_id=21627 Just taking out the -Kpthread test is not acceptable. It will mean that thread support will break systems that require it. Likewise, taking it as a fallback will fail on the very same system. Some of those systems (e.g. UnixWare) do provide a libpthread and a pthread_create function within, but the documentation explicitly says that you must not use -lpthread. So we *must* use -Kpthread were available. What exactly is the problem on HP/UX? If the compiler links the test program, why don't threads work then? Can you give a C program that shows that threads are not available when just passing -Kpthread? I agree that special-casing gcc is a hack, but what I need is a solution, not removal of the existing feature. Also, what exact problem is solved with your change "for pthread_create in -lpthread": Why do you need a test program; why is AC_CHECK_LIB not good enough? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:23 Message: Logged In: YES user_id=6380 I'm assigning this to Martin, since he checked in the code that tests for -Kpthread, mentioning Patch #418659: Fixes for UnixWare and ReliantUnix. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Mon Oct 8 20:41:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 08 Oct 2001 12:41:16 -0700 Subject: [Python-bugs-list] [ python-Bugs-469250 ] xxmodule.c needs to be updated Message-ID: Bugs item #469250, was opened at 2001-10-08 12:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469250&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: xxmodule.c needs to be updated Initial Comment: Module/xxmodule.c is now out of date as it does not include the new fields for iterators in the sample Xxo_Type structure. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469250&group_id=5470 From noreply@sourceforge.net Tue Oct 9 11:47:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 03:47:27 -0700 Subject: [Python-bugs-list] [ python-Bugs-469250 ] xxmodule.c needs to be updated Message-ID: Bugs item #469250, was opened at 2001-10-08 12:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469250&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: xxmodule.c needs to be updated Initial Comment: Module/xxmodule.c is now out of date as it does not include the new fields for iterators in the sample Xxo_Type structure. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:47 Message: Logged In: YES user_id=21627 Fixed in xxmodule.c 2.23. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469250&group_id=5470 From noreply@sourceforge.net Tue Oct 9 11:55:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 03:55:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-468118 ] gzipmodule.c has wrong URL in comment Message-ID: Bugs item #468118, was opened at 2001-10-04 16:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468118&group_id=5470 Category: Extension Modules Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Richard Jones (richard) Assigned to: Nobody/Anonymous (nobody) Summary: gzipmodule.c has wrong URL in comment Initial Comment: The URL for the zlib library is incorrect at the top of zlibmodule.c It should be: http://www.gzip.org/zlib/ ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:55 Message: Logged In: YES user_id=21627 Thanks, fixed in zlibmodule.c 2.43. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468118&group_id=5470 From noreply@sourceforge.net Tue Oct 9 11:57:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 03:57:53 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Tue Oct 9 11:58:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 03:58:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Tue Oct 9 12:51:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 04:51:27 -0700 Subject: [Python-bugs-list] [ python-Bugs-466981 ] item_cget() missing parm. in Tix/HList Message-ID: Bugs item #466981, was opened at 2001-10-01 14:38 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466981&group_id=5470 Category: Tkinter Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: item_cget() missing parm. in Tix/HList Initial Comment: This applies to Tix.py version 1.3, any OS. The item_cget method of HList in Tix.py is missing the first parameter (entryPath in the Tix documentation). The patch below fixes this: *************** *** 730,737 **** c = self.tk.call(self._w, 'info', 'selection') return self.tk.splitlist(c) ! def item_cget(self, entry, col, opt): ! return self.tk.call(self._w, 'item', 'cget', entry, col, opt) def item_configure(self, entry, col, cnf={}, **kw): if cnf is None: --- 730,737 ---- c = self.tk.call(self._w, 'info', 'selection') return self.tk.splitlist(c) ! def item_cget(self, col, opt): ! return self.tk.call(self._w, 'item', 'cget', col, opt) def item_configure(self, entry, col, cnf={}, **kw): if cnf is None: ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 04:51 Message: Logged In: YES user_id=21627 Thanks for the report. This is fixed in Tix.py 1.4. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466981&group_id=5470 From noreply@sourceforge.net Tue Oct 9 14:56:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 06:56:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Tue Oct 9 17:19:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 09:19:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-465502 ] urllib2: urlopen unicode problem Message-ID: Bugs item #465502, was opened at 2001-09-26 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465502&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: John Quigley (jmquigs) Assigned to: Jeremy Hylton (jhylton) Summary: urllib2: urlopen unicode problem Initial Comment: It seems like urlopen doesn't like unicode strings for urls: Python 2.1.1 (#1, Jul 23 2001, 22:08:25) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import urllib2 >>> url = u"http://www.yahoo.com" >>> urllib2.urlopen(url) Traceback (most recent call last): File "", line 1, in ? File "/home/john/python-install/lib/python2.1/urllib2.py", line 135, in urlopen return _opener.open(url, data) File "/home/john/python-install/lib/python2.1/urllib2.py", line 310, in open assert isinstance(req, Request) # really only care about interface AssertionError One fix which seems to work is to add: or type(fullurl) == types.UnicodeType to the if on line 303 (Matrix!!!) of urllib2.py. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 09:19 Message: Logged In: YES user_id=31392 Fixed in rev. 1.22. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-28 00:28 Message: Logged In: YES user_id=21627 For cases like this, I'd propose to use types.StringTypes until there is a common base type for both. This allows the code to continue to work even if Unicode is configured out of the compiler; the idiom then is type(fullurl) in types.StringTypes ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-27 11:08 Message: Logged In: YES user_id=6380 He's right. Assigned to Jeremy. (This is trivial, but I'd like you to look it over.) (Note to Jeremy: I'm not quite comfortable with all the isinstance() checks in your code. I'm afraid that some of these are already broken in 2.2, e.g. is_callable(). It would be better to test for specific attributes to make the distinctions or tests you need to make.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465502&group_id=5470 From noreply@sourceforge.net Tue Oct 9 17:41:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 09:41:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Tue Oct 9 18:21:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 10:21:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-468948 ] urllib2, basic authentication, & 302 Message-ID: Bugs item #468948, was opened at 2001-10-07 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeffrey C. Ollie (jcollie) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2, basic authentication, & 302 Initial Comment: I've found a bug in how urllib2 handles authentication. The crux of the problem is that the AbstractBasicAuthHandler.__current_realm is basically a global variable. I discovered the bug because I needed to use HTTP basic authentication and the HTTP If-Modified-Since header. Since the HTTP server returns a 302 error if the requested URL has not changed the line of code that resets AbstractBasicAuthHandler.__current_realm back to None never gets executed because an exception will be raised when the retrieval is retried. I suspect that this bug would also cause problems in multi-threaded code. The digest authentication appears to have similar problems. The solution that I found is to get rid of the __current_realm attribute and prevent infinite retries by checking for the presence of an Authenticate: header in the request object that exactly matches the Authenticate: header that would be added. The bug exists in 2.1.1, 2.2a4 and the current CVS. Patch attached. ---------------------------------------------------------------------- >Comment By: Jeffrey C. Ollie (jcollie) Date: 2001-10-09 10:21 Message: Logged In: YES user_id=37310 Oops, that patch that I attached is reversed, use -R! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 From noreply@sourceforge.net Tue Oct 9 19:06:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 11:06:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-407783 ] urllib2: AbstractHTTPHandler limits flexible client implemen Message-ID: Bugs item #407783, was opened at 2001-03-11 16:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=407783&group_id=5470 Category: Python Library Group: None Status: Open Resolution: Postponed Priority: 4 Submitted By: Bill Bumgarner (bbum) Assigned to: Jeremy Hylton (jhylton) Summary: urllib2: AbstractHTTPHandler limits flexible client implemen Initial Comment: The implementation of the do_open() method on the AbstractHTTPHandler class contains a couple of "features" that could be considered to be "bugs". In any case, each time I have wanted to use urllib2 for relatively straightforward development of an HTTP client, I have had to effectively replace the HTTPHandler with one that reimplements do_open() (or http_open() in 2.0). Maybe my usage is not the norm-- in any case, the more information, the better... Specifics (all names in context of Python 2.1): - AbstractHTTPHandler does not allow for anything but GET or POST requests. GET is the default and POST happens anytime the request object contains data to be passed to the server. This limitation is the only thing that stands in the way of using the AbstractHTTPHandler *directly* to implement, say, a WebDAV client or to do something like a site sucker that uses the HEAD method to determine if content has changed. - [this is likely a bug] the method will throw an exception if *any* response is received from the server other than 200. However, HTTP defines that all 2XX responses should be treated as successful. In any case, there are *a lot* of contexts within which a non-200 response may be treated as a 'success' of some sort or another. Regardless, it is really outside of the scope of the AbstractHTTPHandler's implementation to make the success/failure decision-- it should simply return the same thing regardless of the response status. - [a bug?] Whenever an exception is raised (a non-200 code is received), the status code and reason (as provided by the server) are both lost. I see that moshez has been primarily responsible for recent changes surrounding this code. I would be happy to contribute to the evolution of the code; please feel free to contact me directly. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 11:06 Message: Logged In: YES user_id=31392 It's only six or seven months since there was an active discussion on this bug report. Anyone still interested in fixing it? I think it's reasonable to try and fix these issues for 2.2, but I don't have time to implement it all myself. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:07 Message: Logged In: YES user_id=6380 Unassigning from Moshe -- he doesn't seem to have time (Moshe, if you're still interested, just change the owner field back to you). ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-04-09 07:02 Message: Logged In: YES user_id=11645 I'm formally postponing it until the 2.1 release comes out -- clearly none of this can be considered a bug fix. ---------------------------------------------------------------------- Comment By: Bill Bumgarner (bbum) Date: 2001-03-19 16:37 Message: Logged In: YES user_id=103811 OK-- I can understand that logic (close to beta, etc). Given the prominence of Python in the WebDav community combined with the increasing use of 2xx (and 1xx) codes, it would be extremely useful to include-- at the least-- examples of handling such via the urllib2 modules. Beyond that, it would be quite helpful to the developers to expend some amount of engineering effort such that handling 2xx response codes doesn't require __getattr__ trickery! Similarly, breaking out the HTTP raw connection setup from the method that actually composes and sends the HTTP request would be helpful in that it would greatly reduce the amount of code that has to be duplicated when subclassing the handler to customize handling of 2xx or when specifying methods other than GET/POST. I.e. most developers will be confused to the point of being overwhelmed if "how do I customize responses such that they don't raise" or "how do I send an OPTIONS or HEAD request" requires figuring out how to deal with setting up and sending a request via the much-lower-level-than-urllib2 HTTP API. ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-03-18 01:22 Message: Logged In: YES user_id=11645 None of these can really be classified as "bugs" rather then functionality enhancement requests, and this is something I'm not sure I want to do this close to the second beta. BTW, one thing I'm sure I *don't* want to change -- handling of 20x codes. If you want to handle 201/206/whatever, then just handle them. With some __getattr__ trickery, you can have a class that handles all http_error_20x errors, so this is *easy* for 3rd party urllib2 extensions to add. Regarding explicitly determining the command: just put the command inside the request object, and use it in your own HTTPHandler/HTTPSHandler. This may be done in the next version of urllib2 (for 2.2). At that time I might also add the feature that other encodings (not just application/x-www-form-urlencoded, but also multipart/form-data) will be supported. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-03-16 10:43 Message: Logged In: YES user_id=31392 I haven't had any spare cycles to devote to urllib2 this year. Perhaps Moshe can be of more help in the near term. Following the 2.1 release, I may have more time. I never used urllib2 it a situation that produced anything other than vanilla responses -- 200, 401, etc. I'm not to surprised to hear that there are problems with 2XX cases. Can you post some examples of the sorts of things you want to do? It sounds reasonable in the abstract, but some code would help. If not in this patch archive, perhaps on comp.lang.python? ---------------------------------------------------------------------- Comment By: Bill Bumgarner (bbum) Date: 2001-03-11 19:59 Message: Logged In: YES user_id=103811 I realized that the exception throw behaviour is more fundamental to the underlying implementation than may have been indicated in the above description. In particular, throwing an HTTP exception when handling a 401 is key to making the various Authentication Handlers work. I still feel that the behaviour should be normalized across all requests such that the callee is responsible for determining error conditions or, at the lest, has access to the same data in a relatively similar format upon success or failure. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=407783&group_id=5470 From noreply@sourceforge.net Tue Oct 9 19:07:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 11:07:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-468384 ] os.P_* constants poorly documented Message-ID: Bugs item #468384, was opened at 2001-10-05 12:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468384&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: os.P_* constants poorly documented Initial Comment: The P_* constants in the os module are not well documented: Possible values for the mode parameter to spawnv() and spawnve(). Since these constants originated in the Windows spawn* functions, they are not likely to be well-understood outside that environment. P_WAIT and P_NOWAIT have obvious meanings, but the others are unclear (at least to me). ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-09 11:07 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/libos.tex revision 1.67. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468384&group_id=5470 From noreply@sourceforge.net Tue Oct 9 19:40:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 11:40:35 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-09 11:40 Message: Logged In: YES user_id=56897 Patch is #468347 [mask signals for non-main pthreads] ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 10:09 Message: Logged In: YES user_id=56897 I've submitted the patch for pthread signal masking. My biggest concerns are the guesses I made for DCE threads and whether they will work for AIX which might need to use sigthreadmask(). Regarding reproducing this on Linux, I was able to get Linux to crash if I held down Ctrl-C (with fairly fast key repeat). After starting the spinning thread, Python would crash on Linux under a storm of SIGINTs within 30 seconds or so. Without the spinning thread, I couldn't get it to crash. With the patch applied, the spinning thread running during the storm of SIGINTs wouldn't crash it. So that implies the signal masking is doing something good even in the Linux case. Re: my SF problems, I submitted a few support requests. Hopefully something gets fixed. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 08:38 Message: Logged In: YES user_id=56897 I've checked the Monitor button while logged in, but I still do not receive email notification of updates. When I clicked it again, it said I was no longer monitoring, so I clicked it yet again back into monitoring mode. Apparently SF knows I'm monitoring it, but it still doesn't send me email. Mail to my @users.sf.net account does work, so I'm at a loss to explain why a) My login cookie doesn't stick around very long at all and b) Why I never get monitor email from SF Re: the patch, I have something that works well on Solaris. I'll try it on Linux today, but I don't have access to an HP-UX system. I'm a little concerned about the impact to HP-UX (pre 11.0 and post 11.0) and AIX, and I don't have access to those machines to check out those concerns. Hopefully I'll have the patch posted by today. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Tue Oct 9 22:36:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 14:36:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-469681 ] Replace a whole string with empty string Message-ID: Bugs item #469681, was opened at 2001-10-09 14:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Replace a whole string with empty string Initial Comment: Replaceing a whole string with an empty string is behaving differently in Unix and windows environment. It raises a memory error in Unix as shown below: UNIX RUN: Python 2.1 (#18, May 24 2001, 14:56:32) [C] on osf1V4 Type "copyright", "credits" or "license" for more information. >>> x = 'test' >>> x.replace('test','') Traceback (most recent call last): File "", line 1, in ? MemoryError The same test has passed in Windows environment: PythonWin 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32. Portions Copyright 1994-2001 Mark Hammond (MarkH@ActiveState.com) - see 'Help/About PythonWin' for further copyright information. >>> x = 'test' >>> x.replace('test','') '' >>> x 'test' >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 From noreply@sourceforge.net Tue Oct 9 23:08:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 15:08:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-469681 ] Replace a whole string with empty string Message-ID: Bugs item #469681, was opened at 2001-10-09 14:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Replace a whole string with empty string Initial Comment: Replaceing a whole string with an empty string is behaving differently in Unix and windows environment. It raises a memory error in Unix as shown below: UNIX RUN: Python 2.1 (#18, May 24 2001, 14:56:32) [C] on osf1V4 Type "copyright", "credits" or "license" for more information. >>> x = 'test' >>> x.replace('test','') Traceback (most recent call last): File "", line 1, in ? MemoryError The same test has passed in Windows environment: PythonWin 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32. Portions Copyright 1994-2001 Mark Hammond (MarkH@ActiveState.com) - see 'Help/About PythonWin' for further copyright information. >>> x = 'test' >>> x.replace('test','') '' >>> x 'test' >>> ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-10-09 15:08 Message: Logged In: YES user_id=38388 I tried to reproduce this on Linux but did not succeed. Could this be a compiler optimization problem on the OSF system ? Does the same problem occur if you compile Python without optimization ? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 From noreply@sourceforge.net Wed Oct 10 00:20:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 16:20:02 -0700 Subject: [Python-bugs-list] [ python-Bugs-469681 ] Replace a whole string with empty string Message-ID: Bugs item #469681, was opened at 2001-10-09 14:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Replace a whole string with empty string Initial Comment: Replaceing a whole string with an empty string is behaving differently in Unix and windows environment. It raises a memory error in Unix as shown below: UNIX RUN: Python 2.1 (#18, May 24 2001, 14:56:32) [C] on osf1V4 Type "copyright", "credits" or "license" for more information. >>> x = 'test' >>> x.replace('test','') Traceback (most recent call last): File "", line 1, in ? MemoryError The same test has passed in Windows environment: PythonWin 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32. Portions Copyright 1994-2001 Mark Hammond (MarkH@ActiveState.com) - see 'Help/About PythonWin' for further copyright information. >>> x = 'test' >>> x.replace('test','') '' >>> x 'test' >>> ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-09 16:20 Message: Logged In: YES user_id=6380 I can't reproduce this either with Python 2.1.1. I believe it's been fixed there; can you try? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-10-09 15:08 Message: Logged In: YES user_id=38388 I tried to reproduce this on Linux but did not succeed. Could this be a compiler optimization problem on the OSF system ? Does the same problem occur if you compile Python without optimization ? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 From noreply@sourceforge.net Wed Oct 10 01:16:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 17:16:02 -0700 Subject: [Python-bugs-list] [ python-Bugs-469681 ] Replace a whole string with empty string Message-ID: Bugs item #469681, was opened at 2001-10-09 14:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: Replace a whole string with empty string Initial Comment: Replaceing a whole string with an empty string is behaving differently in Unix and windows environment. It raises a memory error in Unix as shown below: UNIX RUN: Python 2.1 (#18, May 24 2001, 14:56:32) [C] on osf1V4 Type "copyright", "credits" or "license" for more information. >>> x = 'test' >>> x.replace('test','') Traceback (most recent call last): File "", line 1, in ? MemoryError The same test has passed in Windows environment: PythonWin 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32. Portions Copyright 1994-2001 Mark Hammond (MarkH@ActiveState.com) - see 'Help/About PythonWin' for further copyright information. >>> x = 'test' >>> x.replace('test','') '' >>> x 'test' >>> ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-09 17:16 Message: Logged In: YES user_id=31435 Closed as Duplicate: this was the subject of bug 422088, and was indeed fixed in 2.1.1. It varied across platforms because it depended on what the platform malloc returned when passed 0. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-09 16:20 Message: Logged In: YES user_id=6380 I can't reproduce this either with Python 2.1.1. I believe it's been fixed there; can you try? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-10-09 15:08 Message: Logged In: YES user_id=38388 I tried to reproduce this on Linux but did not succeed. Could this be a compiler optimization problem on the OSF system ? Does the same problem occur if you compile Python without optimization ? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469681&group_id=5470 From noreply@sourceforge.net Wed Oct 10 02:36:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 18:36:37 -0700 Subject: [Python-bugs-list] [ python-Bugs-469732 ] os.path.walk docstring inconsistent Message-ID: Bugs item #469732, was opened at 2001-10-09 18:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469732&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Mark Hammond (mhammond) Assigned to: Tim Peters (tim_one) Summary: os.path.walk docstring inconsistent Initial Comment: On Windows, the docstring for os.path.walk says (in ntpath.py """Directory tree walk whth callback function. walk(top, func, arg) calls func(arg, d, files) for each directory d in the tree rooted at top (including top itself); files is a list of all the files and subdirs in directory d.""" (*** Note the "whth" in the first line ***) Comparing with posixpath.py, the entire first line of the docstring is missing. I would have simply corrected the spelling of "whth", but when I noticed that the 2 platforms' strings were different I decided to open this bug and allow Tim to make a quick decision about which string is the correct one. Feel free to punt back, but I am guessing it quicker for you to make the decision and check it in. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469732&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:07:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:07:59 -0700 Subject: [Python-bugs-list] [ python-Bugs-219707 ] urllib failure when return code not 200 Message-ID: Bugs item #219707, was opened at 2000-10-29 16:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=219707&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: urllib failure when return code not 200 Initial Comment: urllib fails sometimes. A traceback follows but the problem is that, on line 286, fp is used as a parameter when it has a value of None. Not sure why; HTTP.getreply seems like it always returns a file. Not time right now to look further into it... brian@sweetapp.com Traceback (most recent call last): File "/usr/home/sweetapp/public_html/zonewatcher/Rating", line 83, in ? UpdatePlayerRatings( database, zonePlayer ) File "/usr/home/sweetapp/public_html/zonewatcher/Rating", line 59, in UpdatePlayerRatings mainRatings = GetMainRatings( zonePlayer.GetZoneName( ), AOEIImainRating, AOEIIexpansionMainRating ) File "/usr/home/sweetapp/public_html/zonewatcher/Rating", line 20, in GetMainRatings ratingsPage = GetRatingsPage( zoneID ) File "/usr/home/sweetapp/public_html/zonewatcher/Rating", line 14, in GetRatingsPage url = urllib.urlopen( 'http://www.zone.com/Profile/RatingsPlayer.asp?PlayerID=' + zoneID ) File "/usr/home/sweetapp/Python-2.0/Lib/urllib.py", line 61, in urlopen return _urlopener.open(url) File "/usr/home/sweetapp/Python-2.0/Lib/urllib.py", line 166, in open return getattr(self, name)(url) File "/usr/home/sweetapp/Python-2.0/Lib/urllib.py", line 286, in open_http return self.http_error(url, fp, errcode, errmsg, headers) File "/usr/home/sweetapp/Python-2.0/Lib/urllib.py", line 303, in http_error return self.http_error_default(url, fp, errcode, errmsg, headers) File "/usr/home/sweetapp/Python-2.0/Lib/urllib.py", line 518, in http_error_default return addinfourl(fp, headers, "http:" + url) File "/usr/home/sweetapp/Python-2.0/Lib/urllib.py", line 772, in __init__ addbase.__init__(self, fp) File "/usr/home/sweetapp/Python-2.0/Lib/urllib.py", line 726, in __init__ self.read = self.fp.read AttributeError: 'None' object has no attribute 'read' ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 19:07 Message: Logged In: YES user_id=31392 I've taken long enough to look at this that I'll trust the comments from gaul: Neither he nor the submittor can reproduce the bug, so I assume it's not a bug. ---------------------------------------------------------------------- Comment By: Andrew Gaul (gaul) Date: 2001-07-14 21:18 Message: Logged In: YES user_id=139865 I've spoken with brian@sweetapp.com, and neither he (with 2.1) nor I (with 2.0) could reproduce this bug. Have you had any luck Jeremey? ---------------------------------------------------------------------- Comment By: Andrew Gaul (gaul) Date: 2001-07-14 21:13 Message: Logged In: YES user_id=139865 I've spoken with brian@sweetapp.com, and neither he (with 2.1) nor I (with 2.0) could reproduce this bug. Have you had any luck Jeremey? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-12-12 13:17 Message: Jeremy, can you see if this is a valid bug report? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=219707&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:11:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:11:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-468948 ] urllib2, basic authentication, & 302 Message-ID: Bugs item #468948, was opened at 2001-10-07 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeffrey C. Ollie (jcollie) >Assigned to: Jeremy Hylton (jhylton) Summary: urllib2, basic authentication, & 302 Initial Comment: I've found a bug in how urllib2 handles authentication. The crux of the problem is that the AbstractBasicAuthHandler.__current_realm is basically a global variable. I discovered the bug because I needed to use HTTP basic authentication and the HTTP If-Modified-Since header. Since the HTTP server returns a 302 error if the requested URL has not changed the line of code that resets AbstractBasicAuthHandler.__current_realm back to None never gets executed because an exception will be raised when the retrieval is retried. I suspect that this bug would also cause problems in multi-threaded code. The digest authentication appears to have similar problems. The solution that I found is to get rid of the __current_realm attribute and prevent infinite retries by checking for the presence of an Authenticate: header in the request object that exactly matches the Authenticate: header that would be added. The bug exists in 2.1.1, 2.2a4 and the current CVS. Patch attached. ---------------------------------------------------------------------- Comment By: Jeffrey C. Ollie (jcollie) Date: 2001-10-09 10:21 Message: Logged In: YES user_id=37310 Oops, that patch that I attached is reversed, use -R! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:17:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:17:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-468948 ] urllib2, basic authentication, & 302 Message-ID: Bugs item #468948, was opened at 2001-10-07 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeffrey C. Ollie (jcollie) Assigned to: Jeremy Hylton (jhylton) Summary: urllib2, basic authentication, & 302 Initial Comment: I've found a bug in how urllib2 handles authentication. The crux of the problem is that the AbstractBasicAuthHandler.__current_realm is basically a global variable. I discovered the bug because I needed to use HTTP basic authentication and the HTTP If-Modified-Since header. Since the HTTP server returns a 302 error if the requested URL has not changed the line of code that resets AbstractBasicAuthHandler.__current_realm back to None never gets executed because an exception will be raised when the retrieval is retried. I suspect that this bug would also cause problems in multi-threaded code. The digest authentication appears to have similar problems. The solution that I found is to get rid of the __current_realm attribute and prevent infinite retries by checking for the presence of an Authenticate: header in the request object that exactly matches the Authenticate: header that would be added. The bug exists in 2.1.1, 2.2a4 and the current CVS. Patch attached. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 19:17 Message: Logged In: YES user_id=31392 Does the patch in bug #451295 fix your problem? ---------------------------------------------------------------------- Comment By: Jeffrey C. Ollie (jcollie) Date: 2001-10-09 10:21 Message: Logged In: YES user_id=37310 Oops, that patch that I attached is reversed, use -R! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:39:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:39:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-461358 ] SSL constructor/destructor bugs Message-ID: Bugs item #461358, was opened at 2001-09-13 14:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461358&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: SSL constructor/destructor bugs Initial Comment: routine newSSLObject() in socketmodule.c does not initialize variables which SSL_dealloc() frees. SSL_dealloc() calls SSL_CTX_free() and then SSL_free (). However, SSL_free() also frees the CTX. in SSLObject, variable 'x_attr' seems to be unnecessary and maybe the source of a memory leak. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 19:39 Message: Logged In: YES user_id=31392 You're sure right about the memory leak! It calls PyDict_New(), stores the result in x_attr, and later sets x_attr to NULL. ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-09-16 21:00 Message: Logged In: YES user_id=163326 Hello! I'm pretty new to OpenSSL, but I'm trying my best to nail the recently reported SSL bugs down. I could confirm everything you state here. The docs didn't say anything about SSL_free() freeing the CTX, but the OpenSSL source says yes, it does. If you'd log in next time, that would ease communication :-) I'll soon (0-2 days) submit a patch that tries to fix some of the recently reported problems. I'll also try to bug a few people proficient in Python & OpenSSL to review my patch. You seem to be knowledgable in both, so another pair of eyes won't hurt. Btw. I can be reached at gerhard@bigfoot.de ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461358&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:42:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:42:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-461353 ] SSL write doesn't check return codes Message-ID: Bugs item #461353, was opened at 2001-09-13 14:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Jeremy Hylton (jhylton) Summary: SSL write doesn't check return codes Initial Comment: routine SSL_SSLwrite() in socketmodule.c does not check the SSL_write() return codes. As a consequence, SSL write errors do not throw exceptions. Something like the following needs to be added: len = SSL_write(self->ssl, data, len); res = SSL_get_error(self->ssl, len); switch (res) { case SSL_ERROR_NONE: assert(len > 0); break; case SSL_ERROR_ZERO_RETURN: /* connection closed */ assert(len == 0); #ifdef MS_WINDOWS ec = WSAENOTCONN; #elif defined(PYOS_OS2) ec = -1; #else ec = ENOTCONN; #endif v = Py_BuildValue("(is)", ec, "Connection closed"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; break; case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && len == 0) { v = Py_BuildValue("(is)", - 1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; default: return PySocket_Err(); } ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:42:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:42:38 -0700 Subject: [Python-bugs-list] [ python-Bugs-461358 ] SSL constructor/destructor bugs Message-ID: Bugs item #461358, was opened at 2001-09-13 14:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461358&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Jeremy Hylton (jhylton) Summary: SSL constructor/destructor bugs Initial Comment: routine newSSLObject() in socketmodule.c does not initialize variables which SSL_dealloc() frees. SSL_dealloc() calls SSL_CTX_free() and then SSL_free (). However, SSL_free() also frees the CTX. in SSLObject, variable 'x_attr' seems to be unnecessary and maybe the source of a memory leak. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 19:39 Message: Logged In: YES user_id=31392 You're sure right about the memory leak! It calls PyDict_New(), stores the result in x_attr, and later sets x_attr to NULL. ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-09-16 21:00 Message: Logged In: YES user_id=163326 Hello! I'm pretty new to OpenSSL, but I'm trying my best to nail the recently reported SSL bugs down. I could confirm everything you state here. The docs didn't say anything about SSL_free() freeing the CTX, but the OpenSSL source says yes, it does. If you'd log in next time, that would ease communication :-) I'll soon (0-2 days) submit a patch that tries to fix some of the recently reported problems. I'll also try to bug a few people proficient in Python & OpenSSL to review my patch. You seem to be knowledgable in both, so another pair of eyes won't hurt. Btw. I can be reached at gerhard@bigfoot.de ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461358&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:42:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:42:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-461350 ] SSL support crashes python Message-ID: Bugs item #461350, was opened at 2001-09-13 13:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461350&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Jeremy Hylton (jhylton) Summary: SSL support crashes python Initial Comment: with a debug version of python on Windows, try: >> import socket >> sk = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >> sl = socket.ssl(sk._sock) This is due to PyObject_Del() calls in socketmodule.c: if ((SSL_connect(self->ssl)) == -1) { /* Actually negotiate SSL connection */ PyErr_SetObject(SSLErrorObject, PyString_FromString ("SSL_connect error")); PyObject_Del(self); return NULL; } these need to be replaced by: Py_DECREF(self); ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461350&group_id=5470 From noreply@sourceforge.net Wed Oct 10 03:54:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 19:54:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-468948 ] urllib2, basic authentication, & 302 Message-ID: Bugs item #468948, was opened at 2001-10-07 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeffrey C. Ollie (jcollie) Assigned to: Jeremy Hylton (jhylton) Summary: urllib2, basic authentication, & 302 Initial Comment: I've found a bug in how urllib2 handles authentication. The crux of the problem is that the AbstractBasicAuthHandler.__current_realm is basically a global variable. I discovered the bug because I needed to use HTTP basic authentication and the HTTP If-Modified-Since header. Since the HTTP server returns a 302 error if the requested URL has not changed the line of code that resets AbstractBasicAuthHandler.__current_realm back to None never gets executed because an exception will be raised when the retrieval is retried. I suspect that this bug would also cause problems in multi-threaded code. The digest authentication appears to have similar problems. The solution that I found is to get rid of the __current_realm attribute and prevent infinite retries by checking for the presence of an Authenticate: header in the request object that exactly matches the Authenticate: header that would be added. The bug exists in 2.1.1, 2.2a4 and the current CVS. Patch attached. ---------------------------------------------------------------------- >Comment By: Jeffrey C. Ollie (jcollie) Date: 2001-10-09 19:54 Message: Logged In: YES user_id=37310 I haven't tried the patch yet but I think that it would fix the bug in the case of a single-threaded program. However, with a multi-threaded program I think that my patch for this problem is superior because it doesn't rely on a property shared by all of the threads that use the same opener object. Of course, my patch assumes that a request object is used only by one thread but I think that that's a safer assumption. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 19:17 Message: Logged In: YES user_id=31392 Does the patch in bug #451295 fix your problem? ---------------------------------------------------------------------- Comment By: Jeffrey C. Ollie (jcollie) Date: 2001-10-09 10:21 Message: Logged In: YES user_id=37310 Oops, that patch that I attached is reversed, use -R! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 From noreply@sourceforge.net Wed Oct 10 05:18:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 21:18:03 -0700 Subject: [Python-bugs-list] [ python-Bugs-469732 ] os.path.walk docstring inconsistent Message-ID: Bugs item #469732, was opened at 2001-10-09 18:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469732&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Mark Hammond (mhammond) Assigned to: Tim Peters (tim_one) Summary: os.path.walk docstring inconsistent Initial Comment: On Windows, the docstring for os.path.walk says (in ntpath.py """Directory tree walk whth callback function. walk(top, func, arg) calls func(arg, d, files) for each directory d in the tree rooted at top (including top itself); files is a list of all the files and subdirs in directory d.""" (*** Note the "whth" in the first line ***) Comparing with posixpath.py, the entire first line of the docstring is missing. I would have simply corrected the spelling of "whth", but when I noticed that the 2 platforms' strings were different I decided to open this bug and allow Tim to make a quick decision about which string is the correct one. Feel free to punt back, but I am guessing it quicker for you to make the decision and check it in. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-09 21:18 Message: Logged In: YES user_id=31435 After long thought, I decided "whth" was a typo . It's actually worse than you thought: we appear to have 5 implementations of walk(), with 5 different docstrings. I tried to combine the best of all of them, and then fiddled the result to tell the truth. Lib/dospath.py; new revision: 1.25 Lib/macpath.py; new revision: 1.31 Lib/ntpath.py; new revision: 1.43 Lib/posixpath.py; new revision: 1.46 Lib/plat-riscos/riscospath.py; new revision: 1.6 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469732&group_id=5470 From noreply@sourceforge.net Wed Oct 10 07:27:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 23:27:04 -0700 Subject: [Python-bugs-list] [ python-Bugs-469772 ] support additional LINK elements. Message-ID: Bugs item #469772, was opened at 2001-10-09 23:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: support additional LINK elements. Initial Comment: Now that mozilla supports the LINK element properly, it would be most excellent if the python docs were generated with more LINK elements. At the moment they're built with up, previous, and next. Some of the others are defined in http://www.w3.org/TR/REC-html40/types.html#type-links Ones that would be very nice are Top, Contents and Index. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 From noreply@sourceforge.net Wed Oct 10 07:28:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 09 Oct 2001 23:28:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-469773 ] docs should include man page Message-ID: Bugs item #469773, was opened at 2001-10-09 23:28 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: docs should include man page Initial Comment: Some stuff that's in the man page (e.g. the syntax for the -W switch) doesn't appear to be in the HTML documentation. If it is there, it's _very_ well hidden away. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 From noreply@sourceforge.net Wed Oct 10 10:38:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 02:38:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Wed Oct 10 14:04:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 06:04:37 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Wed Oct 10 14:08:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 06:08:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-469859 ] range function problem Message-ID: Bugs item #469859, was opened at 2001-10-10 06:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469859&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: zhaoyun (jilly) Assigned to: Nobody/Anonymous (nobody) Summary: range function problem Initial Comment: I am a new user of python language. I tested a small program which likes the following: ******************************************** * n = 19999999 * for x in range(2, n/2+1): * if n % x == 0: * print n, 'equals', x, '*', n/x * break * else: * print n, 'is a prime number' ********************************************** I found something interesting when the variable 'n' equals 19999999 that my computer seemed crazy because it began to eat my memory at a surprising speed ,the memory amount used just rose from 131M to 258M and the process had no response. But when the variable 'n' equals 1999999, the program works well. So I just ran the expression 'range(2, n/2+1)'(n=19999999),and i found the same situation like the front happened. So I want to know if it is a bug of the range function. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469859&group_id=5470 From noreply@sourceforge.net Wed Oct 10 14:12:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 06:12:51 -0700 Subject: [Python-bugs-list] [ python-Bugs-469859 ] range function problem Message-ID: Bugs item #469859, was opened at 2001-10-10 06:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469859&group_id=5470 Category: Python Library >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: zhaoyun (jilly) Assigned to: Nobody/Anonymous (nobody) Summary: range function problem Initial Comment: I am a new user of python language. I tested a small program which likes the following: ******************************************** * n = 19999999 * for x in range(2, n/2+1): * if n % x == 0: * print n, 'equals', x, '*', n/x * break * else: * print n, 'is a prime number' ********************************************** I found something interesting when the variable 'n' equals 19999999 that my computer seemed crazy because it began to eat my memory at a surprising speed ,the memory amount used just rose from 131M to 258M and the process had no response. But when the variable 'n' equals 1999999, the program works well. So I just ran the expression 'range(2, n/2+1)'(n=19999999),and i found the same situation like the front happened. So I want to know if it is a bug of the range function. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:12 Message: Logged In: YES user_id=6380 It's not a bug -- the range() function creates a list containing all those numbers, and that takes up a lot of memory. If you want to loop over a large range without consuming so much memory, use xrange() instead of range(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469859&group_id=5470 From noreply@sourceforge.net Wed Oct 10 16:06:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 08:06:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Wed Oct 10 16:47:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 08:47:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-467059 ] htmllib broken Message-ID: Bugs item #467059, was opened at 2001-10-01 20:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 Category: Python Library Group: Irreproducible >Status: Closed Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: htmllib broken Initial Comment: Responding to a question in python-help about extracting links from web pages, I wrote a simple href printer (see attached file). When run using 2.2a4, it never prints anything. It outputs a list of hrefs when run with 2.1 or 1.6. Either there's a bug somewhere (in my code possibly, though it's pretty simple) or some semantics changed that I missed. I thought maybe the method resolution order change affected things, but htmllib.HTMLParser only uses single inheritance. When displaying help about htmllib.HTMLParser, pydoc.help does emit the method resolution order, which it doesn't generally seem to do: class HTMLParser(sgmllib.SGMLParser) | Method resolution order: | HTMLParser | sgmllib.SGMLParser | markupbase.ParserBase ... ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-10 08:47 Message: Logged In: YES user_id=44345 I tried it again just now with the same input that was failing when I first submitted this bug. It worked this time (though the output was slightly different than we running against 2.1 - the href parameter is a tuple instead of a string), so I went ahead and closed the bug instead of just leaving it pending. Something apparently changed in the past 9 days. (Sorry for the delay responding. My procmail filters classed the message as spam...) Skip ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-04 13:05 Message: Logged In: YES user_id=3066 Please attach the input for which this fails. A trivial test case does not fail (see Lib/test/test_htmllib.py). Set status to "pending". ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-01 20:30 Message: Logged In: YES user_id=44345 SF apparently doesn't like file uploads from Opera, so it's pasted here... import htmllib, formatter class MyParser(htmllib.HTMLParser): def anchor_bgn(self, href, name, type): print href fmt = formatter.NullFormatter() parser = MyParser(fmt, verbose=1) parser.feed(open("tour01.html").read()) parser.close() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 From noreply@sourceforge.net Wed Oct 10 16:48:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 08:48:10 -0700 Subject: [Python-bugs-list] [ python-Bugs-467059 ] htmllib broken Message-ID: Bugs item #467059, was opened at 2001-10-01 20:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 Category: Python Library Group: Irreproducible Status: Closed Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: htmllib broken Initial Comment: Responding to a question in python-help about extracting links from web pages, I wrote a simple href printer (see attached file). When run using 2.2a4, it never prints anything. It outputs a list of hrefs when run with 2.1 or 1.6. Either there's a bug somewhere (in my code possibly, though it's pretty simple) or some semantics changed that I missed. I thought maybe the method resolution order change affected things, but htmllib.HTMLParser only uses single inheritance. When displaying help about htmllib.HTMLParser, pydoc.help does emit the method resolution order, which it doesn't generally seem to do: class HTMLParser(sgmllib.SGMLParser) | Method resolution order: | HTMLParser | sgmllib.SGMLParser | markupbase.ParserBase ... ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-10 08:47 Message: Logged In: YES user_id=44345 I tried it again just now with the same input that was failing when I first submitted this bug. It worked this time (though the output was slightly different than we running against 2.1 - the href parameter is a tuple instead of a string), so I went ahead and closed the bug instead of just leaving it pending. Something apparently changed in the past 9 days. (Sorry for the delay responding. My procmail filters classed the message as spam...) Skip ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-04 13:05 Message: Logged In: YES user_id=3066 Please attach the input for which this fails. A trivial test case does not fail (see Lib/test/test_htmllib.py). Set status to "pending". ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-01 20:30 Message: Logged In: YES user_id=44345 SF apparently doesn't like file uploads from Opera, so it's pasted here... import htmllib, formatter class MyParser(htmllib.HTMLParser): def anchor_bgn(self, href, name, type): print href fmt = formatter.NullFormatter() parser = MyParser(fmt, verbose=1) parser.feed(open("tour01.html").read()) parser.close() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467059&group_id=5470 From noreply@sourceforge.net Wed Oct 10 20:36:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 12:36:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-469972 ] xmlrpclib won't marshal new types Message-ID: Bugs item #469972, was opened at 2001-10-10 12:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469972&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: xmlrpclib won't marshal new types Initial Comment: Maybe xmlrpclib should be modified to allow it to marshal subclasses of builtin types (ints, strings, etc). Here's a simple example that demonstrates that it currently won't work with a subclass of str: >>> import xmlrpclib >>> class MyString(str): ... pass ... >>> s = MyString("sdfsdfsdf") >>> s 'sdfsdfsdf' >>> s.__class__ >>> xmlrpclib.dumps((s,)) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/xmlrpclib.py", line 752, in dumps data = m.dumps(params) File "/usr/local/lib/python2.2/xmlrpclib.py", line 448, in dumps self.__dump(v) File "/usr/local/lib/python2.2/xmlrpclib.py", line 459, in __dump raise TypeError, "cannot marshal %s objects" % type(value) TypeError: cannot marshal objects ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469972&group_id=5470 From noreply@sourceforge.net Wed Oct 10 20:50:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 12:50:53 -0700 Subject: [Python-bugs-list] [ python-Bugs-228685 ] popen on Win9x isnt smart enough about finding w9xpopen.exe Message-ID: Bugs item #228685, was opened at 2001-01-13 11:38 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=228685&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Open Resolution: Fixed Priority: 5 Submitted By: Mark Hammond (mhammond) Assigned to: Mark Hammond (mhammond) Summary: popen on Win9x isnt smart enough about finding w9xpopen.exe Initial Comment: The code in posixmodule that emulates popen*() isnt very smart about locating the .exe for the Win9x popen hacks. It assumes that w9xpopen.exe can be located in the directory of the main executable, which fails in may embedding situations. A reasonable fix would be to also attempt to locate the file in the "sys.prefix" directory - typically this will be set correctly. Im not at my main dev machine, hence I am submitting a bug rather than a patch. I will submit a patch ASAP. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-10 12:50 Message: Logged In: NO I discovered something related to this bug yesterday, while experimenting with popen using ActivePython 2.1.1.212 on a Win98SE box. The test code was this (I'm new to python, please forgive any silly things you can find in it :-)) from os import popen2 def getoutput(cmd): i,o=popen2(cmd) result=[] l=o.readline() while l: result.append(l) l=o.readline() return "\n".join(result) o.close() i.close() If I start python from a directory other than install dir (e:\python21\ on my box) then: - the test code hangs on "l=o.readline" - A short sound is played when the code is run - a w9xpopen task is shown in task list, after I quit the hanged python using CTRL-BREAK - the system hangs after trying to run test code several times (5 to 7) If I start python from its install dir, everything works fine. It must be noted that if I start python from its install dir, run the test code, quit python, change the working dir, restart python and run test code, then it works fine. Looks like w9xpopen remains in memory even if python is quit and then restarted from a different dir: well, I don't think it really stays in memory, but it looks like it stays there :-) ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-09-11 15:41 Message: Logged In: YES user_id=14198 You will see the problem if w9xpopen.exe can not be found either in: * the directory of the executable * the sys.prefix directory. In a build environment, you will have problems if you run an executable from *outside* the pcbuild directory (eg pythonwin.exe, or a copy of python.exe copied elsewhere) In a pre-installed Python, this scenario ('external' .exe) should work correctly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-11 09:22 Message: Logged In: YES user_id=31435 Mark, I haven't noticed any differences in the behavior of test_popen2 on Win98SE, whether from an installed Python or from PCBuild. Did you expect something to break in the latter case? It doesn't for me. You can test it yourself on Win2K by setting COMSPEC to point to command.com (in which case a recent patch makes Python use w9xpopen.exe even on NT/2000). ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-09-11 05:37 Message: Logged In: YES user_id=14198 oops - meant "now that sys.prefix is always set correctly." in the last entry. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-09-11 05:26 Message: Logged In: YES user_id=14198 I consider this bug fixed, now that sys.path is always set correctly. Code already in the popen() implementation will attempt to locate w9xopen.exe in sys.prefix, which will be correct in all "installed" Pythons. Note that this does not work in a build python - sys.prefix points to the top-level Python source directory, while the executables are in ".\pcbuild". I don't care enough to fix this, as I never test on win9x, and it can't bother Tim too much as he would have just screamed louder before now. Leaving open until I test some more on a on installed versions on my w98 box. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-09-11 05:25 Message: Logged In: YES user_id=14198 I consider this bug fixed, now that sys.path is always set correctly. Code already in the popen() implementation will attempt to locate w9xopen.exe in sys.prefix, which will be correct in all "installed" Pythons. Note that this does not work in a build python - sys.prefix points to the top-level Python source directory, while the executables are in ".\pcbuild". I don't care enough to fix this, as I never test on win9x, and it can't bother Tim too much as he would have just screamed louder before now. Leaving open until I test some more on a on installed versions on my w98 box. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-01-30 23:32 Message: Partial fix checked in Rev 2.183 of Modules/posixmodule.c As per the comments in that checkin, the code now tries sys.prefix, but in many embedded situations this is still NULL/empty. As soon as source-force is accepting new bugs I will close this and open a new one for sys.prefix being empty in some conditions. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-01-13 11:39 Message: Giving to me! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=228685&group_id=5470 From noreply@sourceforge.net Wed Oct 10 20:55:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 12:55:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-461350 ] SSL support crashes python Message-ID: Bugs item #461350, was opened at 2001-09-13 13:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461350&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL support crashes python Initial Comment: with a debug version of python on Windows, try: >> import socket >> sk = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >> sl = socket.ssl(sk._sock) This is due to PyObject_Del() calls in socketmodule.c: if ((SSL_connect(self->ssl)) == -1) { /* Actually negotiate SSL connection */ PyErr_SetObject(SSLErrorObject, PyString_FromString ("SSL_connect error")); PyObject_Del(self); return NULL; } these need to be replaced by: Py_DECREF(self); ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-10 12:55 Message: Logged In: YES user_id=31392 Fixed in rev 1.171 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461350&group_id=5470 From noreply@sourceforge.net Wed Oct 10 23:40:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 15:40:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-461358 ] SSL constructor/destructor bugs Message-ID: Bugs item #461358, was opened at 2001-09-13 14:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461358&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL constructor/destructor bugs Initial Comment: routine newSSLObject() in socketmodule.c does not initialize variables which SSL_dealloc() frees. SSL_dealloc() calls SSL_CTX_free() and then SSL_free (). However, SSL_free() also frees the CTX. in SSLObject, variable 'x_attr' seems to be unnecessary and maybe the source of a memory leak. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-10 15:40 Message: Logged In: YES user_id=31392 newSSLObject() now invitializes the variables it frees. It looks like SSL_dealloc() should call both SSL_CTX_free() and SSL_free(), because we have two separate references to the context that both need to be accounted for. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 19:39 Message: Logged In: YES user_id=31392 You're sure right about the memory leak! It calls PyDict_New(), stores the result in x_attr, and later sets x_attr to NULL. ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-09-16 21:00 Message: Logged In: YES user_id=163326 Hello! I'm pretty new to OpenSSL, but I'm trying my best to nail the recently reported SSL bugs down. I could confirm everything you state here. The docs didn't say anything about SSL_free() freeing the CTX, but the OpenSSL source says yes, it does. If you'd log in next time, that would ease communication :-) I'll soon (0-2 days) submit a patch that tries to fix some of the recently reported problems. I'll also try to bug a few people proficient in Python & OpenSSL to review my patch. You seem to be knowledgable in both, so another pair of eyes won't hurt. Btw. I can be reached at gerhard@bigfoot.de ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461358&group_id=5470 From noreply@sourceforge.net Wed Oct 10 23:51:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 15:51:16 -0700 Subject: [Python-bugs-list] [ python-Bugs-470040 ] ParseTuple t# vs subclasses Message-ID: Bugs item #470040, was opened at 2001-10-10 15:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470040&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: ParseTuple t# vs subclasses Initial Comment: Part of the problem in bug 468061 is that PyArg_ParseTuple's 't#' format rejects objects that are of a str subclass. Apparently that in turn is because the latter don't have Py_TPFLAGS_HAVE_GETCHARBUFFER. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470040&group_id=5470 From noreply@sourceforge.net Wed Oct 10 23:52:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 15:52:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-468061 ] __str__ ignored in str subclass Message-ID: Bugs item #468061, was opened at 2001-10-04 13:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: __str__ ignored in str subclass Initial Comment: >From c.l.py; looks messy (e.g., if PyObject_String didn't ignore octetstring.__str__, the call to str() in the latter would just recurse forever; and he's right that b2a_hex requires a genuine string; and e.g. return b2a_hex(self + "") would be mighty obscure). ----Original Message----- From: Andreas.Trawoeger Sent: Thursday, October 04, 2001 7:10 AM To: python-list Subject: Python 2.2a4: Problems with overloading __str__ in subclasses of str I would like to write a class that behaves like a string only with the difference that it displays the string in hex. In Python 2.2a4 this should be possible by subclassing str and overloading __repr__ and __str__ : from binascii import b2a_hex class octetstring(str): def __repr__(self): return b2a_hex(str(self)) def __str__(self): return b2a_hex(str(self)) o = octetstring('A') print "o.__repr__():",o.__repr__() print "o.__str__() :",o.__str__() print "str(o) :",str(o) print "o :",o The result is pretty strange: o.__repr__(): 41 o.__str__() : 41 str(o) : A o : A I'm not shure what's the exact reason for this problem. Either __str__ isn't used for str() and print. Or I'm trapped in an chicken egg problem: I have to convert self to an string (otherwise b2a_hex won't work) to do this Python should use __str__ (which is the method I'm changing). cu andreas ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-10 15:52 Message: Logged In: YES user_id=31435 The business about b2a_hex not accepting an object of a subclass of str is now the subject of bug 470040. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-04 13:57 Message: Logged In: YES user_id=31435 Looks like I assigned this to Guido by mistake; back to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 From noreply@sourceforge.net Thu Oct 11 00:48:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 16:48:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-461353 ] SSL write doesn't check return codes Message-ID: Bugs item #461353, was opened at 2001-09-13 14:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL write doesn't check return codes Initial Comment: routine SSL_SSLwrite() in socketmodule.c does not check the SSL_write() return codes. As a consequence, SSL write errors do not throw exceptions. Something like the following needs to be added: len = SSL_write(self->ssl, data, len); res = SSL_get_error(self->ssl, len); switch (res) { case SSL_ERROR_NONE: assert(len > 0); break; case SSL_ERROR_ZERO_RETURN: /* connection closed */ assert(len == 0); #ifdef MS_WINDOWS ec = WSAENOTCONN; #elif defined(PYOS_OS2) ec = -1; #else ec = ENOTCONN; #endif v = Py_BuildValue("(is)", ec, "Connection closed"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; break; case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && len == 0) { v = Py_BuildValue("(is)", - 1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; default: return PySocket_Err(); } ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-10-10 16:48 Message: Logged In: YES user_id=163326 My patch #462759 tries to fix this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 From noreply@sourceforge.net Thu Oct 11 00:48:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 16:48:59 -0700 Subject: [Python-bugs-list] [ python-Bugs-458839 ] SSL errno wrong under Windows Message-ID: Bugs item #458839, was opened at 2001-09-05 12:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: SSL errno wrong under Windows Initial Comment: in socketmodule.c, routine SSL_SSLread(), lines 2348 and 2355: return PyErr_SetFromErrno(PySocket_Error); does not work under Windows; as elsewhere: return PySocket_Err(); should be used instead. In addition, if SSL_read() returns SSL_ERROR_SYSCALL, errno will not be set correctly if an EOF is the cause of this return code. Add something like: case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && count == 0) { v = Py_BuildValue("(is)", -1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject(PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-10-10 16:48 Message: Logged In: YES user_id=163326 My patch #462759 tries to fix this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 From noreply@sourceforge.net Thu Oct 11 02:29:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 10 Oct 2001 18:29:35 -0700 Subject: [Python-bugs-list] [ python-Bugs-470072 ] Change in Unicode response to t# Message-ID: Bugs item #470072, was opened at 2001-10-10 18:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470072&group_id=5470 Category: Unicode Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: M.-A. Lemburg (lemburg) Summary: Change in Unicode response to t# Initial Comment: Test case: import binascii h = binascii.b2a_hex(u'a') print `h` Under 2.0.1 and 2.1.1, this produces '61'. Under current CVS, '6100'. I tried on Windows, and Barry confirmed the same on Linux. b2a_hex hasn't changed; instead it looks like Unicode objects have changed what they return to PyArg_ParseTuple in response to the 't#' code. Bug or feature? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470072&group_id=5470 From noreply@sourceforge.net Thu Oct 11 10:55:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 02:55:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Thu Oct 11 10:57:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 02:57:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Thu Oct 11 11:18:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 03:18:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Thu Oct 11 14:10:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 06:10:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Thu Oct 11 17:06:31 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 09:06:31 -0700 Subject: [Python-bugs-list] [ python-Bugs-470072 ] Change in Unicode response to t# Message-ID: Bugs item #470072, was opened at 2001-10-10 18:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470072&group_id=5470 Category: Unicode >Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) >Assigned to: Jeremy Hylton (jhylton) Summary: Change in Unicode response to t# Initial Comment: Test case: import binascii h = binascii.b2a_hex(u'a') print `h` Under 2.0.1 and 2.1.1, this produces '61'. Under current CVS, '6100'. I tried on Windows, and Barry confirmed the same on Linux. b2a_hex hasn't changed; instead it looks like Unicode objects have changed what they return to PyArg_ParseTuple in response to the 't#' code. Bug or feature? ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-11 09:06 Message: Logged In: YES user_id=31435 Reassigned to Jeremy, because I believe he fixed it. Jeremy, if you agree, please close this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470072&group_id=5470 From noreply@sourceforge.net Thu Oct 11 17:07:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 09:07:16 -0700 Subject: [Python-bugs-list] [ python-Bugs-458839 ] SSL errno wrong under Windows Message-ID: Bugs item #458839, was opened at 2001-09-05 12:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Jeremy Hylton (jhylton) Summary: SSL errno wrong under Windows Initial Comment: in socketmodule.c, routine SSL_SSLread(), lines 2348 and 2355: return PyErr_SetFromErrno(PySocket_Error); does not work under Windows; as elsewhere: return PySocket_Err(); should be used instead. In addition, if SSL_read() returns SSL_ERROR_SYSCALL, errno will not be set correctly if an EOF is the cause of this return code. Add something like: case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && count == 0) { v = Py_BuildValue("(is)", -1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject(PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-10-10 16:48 Message: Logged In: YES user_id=163326 My patch #462759 tries to fix this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 From noreply@sourceforge.net Thu Oct 11 17:07:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 09:07:16 -0700 Subject: [Python-bugs-list] [ python-Bugs-458835 ] SSL support in httplib causes exception Message-ID: Bugs item #458835, was opened at 2001-09-05 11:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458835&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Jeremy Hylton (jhylton) Summary: SSL support in httplib causes exception Initial Comment: in class 'FakeSocket', method 'makefile' line 593: except socket.sslerror,msg: doesn't work as the socketmodule.c code raises socket.error not socket.sslerror; perhaps: except (socket.error,socket.sslerror), msg: or except socket.error, msg: ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458835&group_id=5470 From noreply@sourceforge.net Thu Oct 11 17:07:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 09:07:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-232460 ] SSL Support (Apparently) Broken on Solaris Message-ID: Bugs item #232460, was opened at 2001-02-14 19:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Open Resolution: None >Priority: 5 Submitted By: David M. Beazley (beazley) >Assigned to: Jeremy Hylton (jhylton) Summary: SSL Support (Apparently) Broken on Solaris Initial Comment: I have spent about 10 hours banging on this with no luck. Python : Python-2.0 Platform : SPARC Solaris 2.8 Compiler : Sun Workshop Pro v5.0, gcc-2.95-2 OpenSSL : openssl-0.9.6 and 0.9.5. Problem - All attempts to open a SSL connection result in an "SSL_connect error". I have tried various combinations of keys and certificates. I have looked at the Python source code extensively and added debugging to try and get more information. I have run the system using the openssl s_client and s_server testing tools. I have recompiled everything in various configurations of versions and compilers. In all cases, the same error is generated. That said, has *ANYBODY* gotten this to work on Solaris? If so, do you have any details that can be shared? Cheers, Dave P.S. I will submit a patch if I can ever get this to actually work. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-07-28 10:39 Message: Logged In: YES user_id=21627 I still have the same problem, both on my own installation (both with 2.1.1, and with the CVS), as well as on cf.sf.net:/home/users/loewis/Python-2.1.1/python (you need to set LD_LIBRARY_PATH=/usr/local/ssl/lib to use this one). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-14 20:30 Message: Logged In: YES user_id=6380 David, is this still broken? On the SourceForge compile farm I can use SSL just fine. Check out my directory /home/users/gvanrossum/Python-2.1-solaris on cf.sourceforge.net. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-10 11:55 Message: Logged In: YES user_id=6380 So, David, you still interested in this? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-20 07:58 Message: Logged In: YES user_id=11375 Moshe checked in patch #405101 to the CVS tree. David, can you update and see if it fixed the problem for you? ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-03-18 02:30 Message: Logged In: YES user_id=11645 I just want to me-too Neil, since I submitted the original patch. I think it *did* fix problems on Solaris, as long as the user is running EGD and set up RANDFILE env. variable properly. Otherwise, you won't be getting connect errors, but a warning from Python about using a non-secure way to generate seed. (For the record, the problem is that Solaris doesn't have a /dev/urandom like Linux does) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-02 07:49 Message: Logged In: YES user_id=35752 It looks like patch "[ #405101 ] Add Random Seeding to OpenSSL" might fix this. Assigning to Andrew since he is assigned that patch too. ---------------------------------------------------------------------- Comment By: David M. Beazley (beazley) Date: 2001-02-15 06:12 Message: A followup on this bug.... It appears that Python is not properly seeding the OpenSSL random number generator when it creates a secure socket. This, in turn, causes the SSL_connect() function to fail due to improper random number seeding (although the error wasn't obvious--at least not to me). This is also due to an apparent "feature" of Solaris not providing a usable /dev/random device. A simple fix is to modify init_socket() in socketmodule.c to include a call to RAND_seed() like this: #ifdef USE_SSL SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); /* Sick hack for Solaris */ { char bogus[64]; /* Fill in bogus with some random noise */ ... RAND_seed(bogus, 64); } ... #endif Presumably one would need to think about the randomness actually passed to RAND_seed() (I have done nothing above). Here are related comments from the OpenSSL FAQ: "Cryptographic software needs a source of unpredictable data to work correctly. Many open source operating systems provide a "randomness device" that serves this purpose. On other systems, applications have to call the RAND_add() or RAND_seed() function with appropriate data before generating keys or performing public key encryption. Some broken applications do not do this. As of version 0.9.5, the OpenSSL functions that need randomness report an error if the random number generator has not been seeded with at least 128 bits of randomness. If this error occurs, please contact the author of the application you are using. It is likely that it never worked correctly. OpenSSL 0.9.5 and later make the error visible by refusing to perform potentially insecure encryption." Python-2.1a2 does not appear to include a fix, but I can't test it to find out (since 2.1a2 doesn't compile on my machine). Cheers, Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 From noreply@sourceforge.net Thu Oct 11 17:12:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 09:12:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-433775 ] module build dir first in test import Message-ID: Bugs item #433775, was opened at 2001-06-16 10:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=433775&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Sjoerd Mullender (sjoerd) >Assigned to: Nobody/Anonymous (nobody) Summary: module build dir first in test import Initial Comment: This problem was found on a Linux (RedHat 7.1) system, but applies to all Unix systems. In the step "python setup.py build", after a shared module is built, setup checks whether it can import the module. During this test, the build directory should be at the beginning of sys.path to make sure you get the module that was actually built. Currently, the build directory is at the end. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=433775&group_id=5470 From noreply@sourceforge.net Thu Oct 11 17:28:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 09:28:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-470072 ] Change in Unicode response to t# Message-ID: Bugs item #470072, was opened at 2001-10-10 18:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470072&group_id=5470 Category: Unicode Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Change in Unicode response to t# Initial Comment: Test case: import binascii h = binascii.b2a_hex(u'a') print `h` Under 2.0.1 and 2.1.1, this produces '61'. Under current CVS, '6100'. I tried on Windows, and Barry confirmed the same on Linux. b2a_hex hasn't changed; instead it looks like Unicode objects have changed what they return to PyArg_ParseTuple in response to the 't#' code. Bug or feature? ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 09:28 Message: Logged In: YES user_id=31392 Fixed in rev. 2.67 of getargs.c ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 09:06 Message: Logged In: YES user_id=31435 Reassigned to Jeremy, because I believe he fixed it. Jeremy, if you agree, please close this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470072&group_id=5470 From noreply@sourceforge.net Thu Oct 11 18:24:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 10:24:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-461353 ] SSL write doesn't check return codes Message-ID: Bugs item #461353, was opened at 2001-09-13 14:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL write doesn't check return codes Initial Comment: routine SSL_SSLwrite() in socketmodule.c does not check the SSL_write() return codes. As a consequence, SSL write errors do not throw exceptions. Something like the following needs to be added: len = SSL_write(self->ssl, data, len); res = SSL_get_error(self->ssl, len); switch (res) { case SSL_ERROR_NONE: assert(len > 0); break; case SSL_ERROR_ZERO_RETURN: /* connection closed */ assert(len == 0); #ifdef MS_WINDOWS ec = WSAENOTCONN; #elif defined(PYOS_OS2) ec = -1; #else ec = ENOTCONN; #endif v = Py_BuildValue("(is)", ec, "Connection closed"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; break; case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && len == 0) { v = Py_BuildValue("(is)", - 1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; default: return PySocket_Err(); } ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:24 Message: Logged In: YES user_id=31392 Fixed as of rev 1.177 of socketmodule.c. ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-10-10 16:48 Message: Logged In: YES user_id=163326 My patch #462759 tries to fix this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 From noreply@sourceforge.net Thu Oct 11 18:24:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 10:24:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-461353 ] SSL write doesn't check return codes Message-ID: Bugs item #461353, was opened at 2001-09-13 14:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL write doesn't check return codes Initial Comment: routine SSL_SSLwrite() in socketmodule.c does not check the SSL_write() return codes. As a consequence, SSL write errors do not throw exceptions. Something like the following needs to be added: len = SSL_write(self->ssl, data, len); res = SSL_get_error(self->ssl, len); switch (res) { case SSL_ERROR_NONE: assert(len > 0); break; case SSL_ERROR_ZERO_RETURN: /* connection closed */ assert(len == 0); #ifdef MS_WINDOWS ec = WSAENOTCONN; #elif defined(PYOS_OS2) ec = -1; #else ec = ENOTCONN; #endif v = Py_BuildValue("(is)", ec, "Connection closed"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; break; case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && len == 0) { v = Py_BuildValue("(is)", - 1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject (PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; default: return PySocket_Err(); } ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:24 Message: Logged In: YES user_id=31392 Fixed as of rev 1.177 of socketmodule.c. ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-10-10 16:48 Message: Logged In: YES user_id=163326 My patch #462759 tries to fix this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=461353&group_id=5470 From noreply@sourceforge.net Thu Oct 11 18:28:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 10:28:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-458839 ] SSL errno wrong under Windows Message-ID: Bugs item #458839, was opened at 2001-09-05 12:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL errno wrong under Windows Initial Comment: in socketmodule.c, routine SSL_SSLread(), lines 2348 and 2355: return PyErr_SetFromErrno(PySocket_Error); does not work under Windows; as elsewhere: return PySocket_Err(); should be used instead. In addition, if SSL_read() returns SSL_ERROR_SYSCALL, errno will not be set correctly if an EOF is the cause of this return code. Add something like: case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && count == 0) { v = Py_BuildValue("(is)", -1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject(PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:28 Message: Logged In: YES user_id=31392 Fixed in rev. 1.178 of socketmodule.c ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-10-10 16:48 Message: Logged In: YES user_id=163326 My patch #462759 tries to fix this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 From noreply@sourceforge.net Thu Oct 11 18:28:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 10:28:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-458839 ] SSL errno wrong under Windows Message-ID: Bugs item #458839, was opened at 2001-09-05 12:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL errno wrong under Windows Initial Comment: in socketmodule.c, routine SSL_SSLread(), lines 2348 and 2355: return PyErr_SetFromErrno(PySocket_Error); does not work under Windows; as elsewhere: return PySocket_Err(); should be used instead. In addition, if SSL_read() returns SSL_ERROR_SYSCALL, errno will not be set correctly if an EOF is the cause of this return code. Add something like: case SSL_ERROR_SYSCALL: if (ERR_get_error() == 0 && count == 0) { v = Py_BuildValue("(is)", -1, "Protocol violation: unexpected EOF"); if (v != NULL) { PyErr_SetObject(PySocket_Error, v); Py_DECREF(v); } return NULL; } else { return PySocket_Err(); } break; ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:28 Message: Logged In: YES user_id=31392 Fixed in rev. 1.178 of socketmodule.c ---------------------------------------------------------------------- Comment By: Gerhard Häring (ghaering) Date: 2001-10-10 16:48 Message: Logged In: YES user_id=163326 My patch #462759 tries to fix this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458839&group_id=5470 From noreply@sourceforge.net Thu Oct 11 18:34:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 10:34:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-232460 ] SSL Support (Apparently) Broken on Solaris Message-ID: Bugs item #232460, was opened at 2001-02-14 19:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Jeremy Hylton (jhylton) Summary: SSL Support (Apparently) Broken on Solaris Initial Comment: I have spent about 10 hours banging on this with no luck. Python : Python-2.0 Platform : SPARC Solaris 2.8 Compiler : Sun Workshop Pro v5.0, gcc-2.95-2 OpenSSL : openssl-0.9.6 and 0.9.5. Problem - All attempts to open a SSL connection result in an "SSL_connect error". I have tried various combinations of keys and certificates. I have looked at the Python source code extensively and added debugging to try and get more information. I have run the system using the openssl s_client and s_server testing tools. I have recompiled everything in various configurations of versions and compilers. In all cases, the same error is generated. That said, has *ANYBODY* gotten this to work on Solaris? If so, do you have any details that can be shared? Cheers, Dave P.S. I will submit a patch if I can ever get this to actually work. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:34 Message: Logged In: YES user_id=31392 I've made some recent changes that affect SSL_connect(), although the primary change is to improve the error messages. Dave and Martin, can you check the current code on Solaris? If it still fails, we'll at least get a clearer error message. BTW, does anyone have suggestions for how to test the code with key and cert files? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-07-28 10:39 Message: Logged In: YES user_id=21627 I still have the same problem, both on my own installation (both with 2.1.1, and with the CVS), as well as on cf.sf.net:/home/users/loewis/Python-2.1.1/python (you need to set LD_LIBRARY_PATH=/usr/local/ssl/lib to use this one). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-14 20:30 Message: Logged In: YES user_id=6380 David, is this still broken? On the SourceForge compile farm I can use SSL just fine. Check out my directory /home/users/gvanrossum/Python-2.1-solaris on cf.sourceforge.net. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-10 11:55 Message: Logged In: YES user_id=6380 So, David, you still interested in this? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-20 07:58 Message: Logged In: YES user_id=11375 Moshe checked in patch #405101 to the CVS tree. David, can you update and see if it fixed the problem for you? ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-03-18 02:30 Message: Logged In: YES user_id=11645 I just want to me-too Neil, since I submitted the original patch. I think it *did* fix problems on Solaris, as long as the user is running EGD and set up RANDFILE env. variable properly. Otherwise, you won't be getting connect errors, but a warning from Python about using a non-secure way to generate seed. (For the record, the problem is that Solaris doesn't have a /dev/urandom like Linux does) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-02 07:49 Message: Logged In: YES user_id=35752 It looks like patch "[ #405101 ] Add Random Seeding to OpenSSL" might fix this. Assigning to Andrew since he is assigned that patch too. ---------------------------------------------------------------------- Comment By: David M. Beazley (beazley) Date: 2001-02-15 06:12 Message: A followup on this bug.... It appears that Python is not properly seeding the OpenSSL random number generator when it creates a secure socket. This, in turn, causes the SSL_connect() function to fail due to improper random number seeding (although the error wasn't obvious--at least not to me). This is also due to an apparent "feature" of Solaris not providing a usable /dev/random device. A simple fix is to modify init_socket() in socketmodule.c to include a call to RAND_seed() like this: #ifdef USE_SSL SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); /* Sick hack for Solaris */ { char bogus[64]; /* Fill in bogus with some random noise */ ... RAND_seed(bogus, 64); } ... #endif Presumably one would need to think about the randomness actually passed to RAND_seed() (I have done nothing above). Here are related comments from the OpenSSL FAQ: "Cryptographic software needs a source of unpredictable data to work correctly. Many open source operating systems provide a "randomness device" that serves this purpose. On other systems, applications have to call the RAND_add() or RAND_seed() function with appropriate data before generating keys or performing public key encryption. Some broken applications do not do this. As of version 0.9.5, the OpenSSL functions that need randomness report an error if the random number generator has not been seeded with at least 128 bits of randomness. If this error occurs, please contact the author of the application you are using. It is likely that it never worked correctly. OpenSSL 0.9.5 and later make the error visible by refusing to perform potentially insecure encryption." Python-2.1a2 does not appear to include a fix, but I can't test it to find out (since 2.1a2 doesn't compile on my machine). Cheers, Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 From noreply@sourceforge.net Thu Oct 11 18:48:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 10:48:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-458835 ] SSL support in httplib causes exception Message-ID: Bugs item #458835, was opened at 2001-09-05 11:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458835&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL support in httplib causes exception Initial Comment: in class 'FakeSocket', method 'makefile' line 593: except socket.sslerror,msg: doesn't work as the socketmodule.c code raises socket.error not socket.sslerror; perhaps: except (socket.error,socket.sslerror), msg: or except socket.error, msg: ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:48 Message: Logged In: YES user_id=31392 Most likely fixed by rev. 1.41 of httplib.py. The original error report isn't clear enough, because most socket errors are legitimate errors and shouldn't be caught inside makefile(). I've added a handler for EINTR, but nothing else. Since the poster was anonymous, I'll just close this and assume it's fixed enough. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458835&group_id=5470 From noreply@sourceforge.net Thu Oct 11 18:57:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 10:57:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-232460 ] SSL Support (Apparently) Broken on Solaris Message-ID: Bugs item #232460, was opened at 2001-02-14 19:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Jeremy Hylton (jhylton) Summary: SSL Support (Apparently) Broken on Solaris Initial Comment: I have spent about 10 hours banging on this with no luck. Python : Python-2.0 Platform : SPARC Solaris 2.8 Compiler : Sun Workshop Pro v5.0, gcc-2.95-2 OpenSSL : openssl-0.9.6 and 0.9.5. Problem - All attempts to open a SSL connection result in an "SSL_connect error". I have tried various combinations of keys and certificates. I have looked at the Python source code extensively and added debugging to try and get more information. I have run the system using the openssl s_client and s_server testing tools. I have recompiled everything in various configurations of versions and compilers. In all cases, the same error is generated. That said, has *ANYBODY* gotten this to work on Solaris? If so, do you have any details that can be shared? Cheers, Dave P.S. I will submit a patch if I can ever get this to actually work. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 10:57 Message: Logged In: YES user_id=21627 Running httplib.py now crashes with Traceback (most recent call last): File "Lib/httplib.py", line 867, in ? test() File "Lib/httplib.py", line 853, in test hs.connect(host) File "Lib/httplib.py", line 698, in connect self._conn.connect() File "Lib/httplib.py", line 653, in connect ssl = socket.ssl(realsock, self.key_file, self.cert_file) socket.sslerror: (1, 'error:24064064:random number generator:SSLEAY_RAND_BYTES:PRNG not seeded') Not sure how this should be fixed, though. We use prngd here. To configure OpenSSH, I had to give prngd parameters to OpenSSH (i.e. that it should use prngd, and what the port number is). On testing certificates: It might be sufficient to generate a self-signed one, and distribute that as part of the test suite. Of course, you need to write server and client code to use the appropriate files. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:34 Message: Logged In: YES user_id=31392 I've made some recent changes that affect SSL_connect(), although the primary change is to improve the error messages. Dave and Martin, can you check the current code on Solaris? If it still fails, we'll at least get a clearer error message. BTW, does anyone have suggestions for how to test the code with key and cert files? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-07-28 10:39 Message: Logged In: YES user_id=21627 I still have the same problem, both on my own installation (both with 2.1.1, and with the CVS), as well as on cf.sf.net:/home/users/loewis/Python-2.1.1/python (you need to set LD_LIBRARY_PATH=/usr/local/ssl/lib to use this one). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-14 20:30 Message: Logged In: YES user_id=6380 David, is this still broken? On the SourceForge compile farm I can use SSL just fine. Check out my directory /home/users/gvanrossum/Python-2.1-solaris on cf.sourceforge.net. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-10 11:55 Message: Logged In: YES user_id=6380 So, David, you still interested in this? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-20 07:58 Message: Logged In: YES user_id=11375 Moshe checked in patch #405101 to the CVS tree. David, can you update and see if it fixed the problem for you? ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-03-18 02:30 Message: Logged In: YES user_id=11645 I just want to me-too Neil, since I submitted the original patch. I think it *did* fix problems on Solaris, as long as the user is running EGD and set up RANDFILE env. variable properly. Otherwise, you won't be getting connect errors, but a warning from Python about using a non-secure way to generate seed. (For the record, the problem is that Solaris doesn't have a /dev/urandom like Linux does) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-02 07:49 Message: Logged In: YES user_id=35752 It looks like patch "[ #405101 ] Add Random Seeding to OpenSSL" might fix this. Assigning to Andrew since he is assigned that patch too. ---------------------------------------------------------------------- Comment By: David M. Beazley (beazley) Date: 2001-02-15 06:12 Message: A followup on this bug.... It appears that Python is not properly seeding the OpenSSL random number generator when it creates a secure socket. This, in turn, causes the SSL_connect() function to fail due to improper random number seeding (although the error wasn't obvious--at least not to me). This is also due to an apparent "feature" of Solaris not providing a usable /dev/random device. A simple fix is to modify init_socket() in socketmodule.c to include a call to RAND_seed() like this: #ifdef USE_SSL SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); /* Sick hack for Solaris */ { char bogus[64]; /* Fill in bogus with some random noise */ ... RAND_seed(bogus, 64); } ... #endif Presumably one would need to think about the randomness actually passed to RAND_seed() (I have done nothing above). Here are related comments from the OpenSSL FAQ: "Cryptographic software needs a source of unpredictable data to work correctly. Many open source operating systems provide a "randomness device" that serves this purpose. On other systems, applications have to call the RAND_add() or RAND_seed() function with appropriate data before generating keys or performing public key encryption. Some broken applications do not do this. As of version 0.9.5, the OpenSSL functions that need randomness report an error if the random number generator has not been seeded with at least 128 bits of randomness. If this error occurs, please contact the author of the application you are using. It is likely that it never worked correctly. OpenSSL 0.9.5 and later make the error visible by refusing to perform potentially insecure encryption." Python-2.1a2 does not appear to include a fix, but I can't test it to find out (since 2.1a2 doesn't compile on my machine). Cheers, Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 From noreply@sourceforge.net Thu Oct 11 19:32:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 11:32:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-451607 ] SSL support in socket module needs tests Message-ID: Bugs item #451607, was opened at 2001-08-16 09:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Martin v. Löwis (loewis) Summary: SSL support in socket module needs tests Initial Comment: SSL support in the socket module should have a regression test, either in test_socket.py or in a separate test_ssl.py file. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 11:32 Message: Logged In: YES user_id=21627 On PRNG problems, it seems we have two options: - wait for, then require OpenSSL 0.9.7. It will look for a prngd socket in default locations (/var/run/egd-pool, /dev/egd-pool, /etc/egd-pool and /etc/entropy); then require administrators to set up OpenSSL that it indeed finds a prngd in these locations when needed. - expose RAND_add. Exposing any other of the interfaces is pointless; on our installation, prngd runs on localhost:708 instead of a Unix domain socket, and none of the other interfaces could use such a configuration. On top of RAND_add, we could communicate with prngd ourselves, e.g. by using the RANDFILE environment variable. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-03 08:12 Message: Logged In: YES user_id=21627 I'd suggest an approach similar to the one in test_socket, i.e. test only if can_fork. As for the certificates: I think they should be generated in advance and included into the test suite. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-08-28 06:53 Message: Logged In: YES user_id=12800 There is currently an extremely minimal test in test_socket_ssl.py, but it is fairly bogus because it requires an outside network connection. The only test there just tries to hit https://sf.net. Ideally, there would be a set of regression tests that only required local resources. I.e. it would set up a local SSL server, then do connects to it to test the various SSL features in socket module. Take a look at test_socket_ssl.py in the 2.2 cvs and see if you can improve matters. From what I understand, the tricky part is generating all the necessary certificates, etc. Also, if you intend to work on this, please log in to SF to continue this dialog so we know who we're talking to! :) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-27 22:58 Message: Logged In: NO I may be able to help with this. I'm fairly familiar with SSL in general, though not about the socket module. What's needed? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470 From noreply@sourceforge.net Thu Oct 11 19:33:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 11:33:13 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Tim Peters (tim_one) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Thu Oct 11 19:44:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 11:44:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-451607 ] SSL support in socket module needs tests Message-ID: Bugs item #451607, was opened at 2001-08-16 09:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Martin v. Löwis (loewis) Summary: SSL support in socket module needs tests Initial Comment: SSL support in the socket module should have a regression test, either in test_socket.py or in a separate test_ssl.py file. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 11:44 Message: Logged In: YES user_id=72053 1) I think it's dangerous to run a prngd on an IP socket instead of a Unix domain socket. It creates the possibility of (either accidentally or by ignorance) running OpenSSL on a separate host from the prngd, making the random numbers vulnerable to network sniffing. That's bad--the numbers are cryptographic secrets and should not be exposed. 2) It's simple to set up a local SSL server with the command line openssl s_server option. 3) I'm not crazy about the whole prngd concept. I haven't looked at the CHILL interface yet, but if it's possible to abandon prngd and get random numbers through CHILL, that might be best. On Linux, /dev/urandom should be used. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 11:32 Message: Logged In: YES user_id=21627 On PRNG problems, it seems we have two options: - wait for, then require OpenSSL 0.9.7. It will look for a prngd socket in default locations (/var/run/egd-pool, /dev/egd-pool, /etc/egd-pool and /etc/entropy); then require administrators to set up OpenSSL that it indeed finds a prngd in these locations when needed. - expose RAND_add. Exposing any other of the interfaces is pointless; on our installation, prngd runs on localhost:708 instead of a Unix domain socket, and none of the other interfaces could use such a configuration. On top of RAND_add, we could communicate with prngd ourselves, e.g. by using the RANDFILE environment variable. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-03 08:12 Message: Logged In: YES user_id=21627 I'd suggest an approach similar to the one in test_socket, i.e. test only if can_fork. As for the certificates: I think they should be generated in advance and included into the test suite. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-08-28 06:53 Message: Logged In: YES user_id=12800 There is currently an extremely minimal test in test_socket_ssl.py, but it is fairly bogus because it requires an outside network connection. The only test there just tries to hit https://sf.net. Ideally, there would be a set of regression tests that only required local resources. I.e. it would set up a local SSL server, then do connects to it to test the various SSL features in socket module. Take a look at test_socket_ssl.py in the 2.2 cvs and see if you can improve matters. From what I understand, the tricky part is generating all the necessary certificates, etc. Also, if you intend to work on this, please log in to SF to continue this dialog so we know who we're talking to! :) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-27 22:58 Message: Logged In: NO I may be able to help with this. I'm fairly familiar with SSL in general, though not about the socket module. What's needed? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470 From noreply@sourceforge.net Thu Oct 11 20:34:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 12:34:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-232460 ] SSL Support (Apparently) Broken on Solaris Message-ID: Bugs item #232460, was opened at 2001-02-14 19:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Jeremy Hylton (jhylton) Summary: SSL Support (Apparently) Broken on Solaris Initial Comment: I have spent about 10 hours banging on this with no luck. Python : Python-2.0 Platform : SPARC Solaris 2.8 Compiler : Sun Workshop Pro v5.0, gcc-2.95-2 OpenSSL : openssl-0.9.6 and 0.9.5. Problem - All attempts to open a SSL connection result in an "SSL_connect error". I have tried various combinations of keys and certificates. I have looked at the Python source code extensively and added debugging to try and get more information. I have run the system using the openssl s_client and s_server testing tools. I have recompiled everything in various configurations of versions and compilers. In all cases, the same error is generated. That said, has *ANYBODY* gotten this to work on Solaris? If so, do you have any details that can be shared? Cheers, Dave P.S. I will submit a patch if I can ever get this to actually work. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 12:34 Message: Logged In: YES user_id=21627 On PRNG problems, it seems we have two options: - wait for, then require OpenSSL 0.9.7. It will look for a prngd socket in default locations (/var/run/egd-pool, /dev/egd-pool, /etc/egd-pool and /etc/entropy); then require administrators to set up OpenSSL that it indeed finds a prngd in these locations when needed. - expose RAND_add. Exposing any other of the interfaces is pointless; on our installation, prngd runs on localhost:708 instead of a Unix domain socket, and none of the other interfaces could use such a configuration. On top of RAND_add, we could communicate with prngd ourselves, e.g. by using the RANDFILE environment variable. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 10:57 Message: Logged In: YES user_id=21627 Running httplib.py now crashes with Traceback (most recent call last): File "Lib/httplib.py", line 867, in ? test() File "Lib/httplib.py", line 853, in test hs.connect(host) File "Lib/httplib.py", line 698, in connect self._conn.connect() File "Lib/httplib.py", line 653, in connect ssl = socket.ssl(realsock, self.key_file, self.cert_file) socket.sslerror: (1, 'error:24064064:random number generator:SSLEAY_RAND_BYTES:PRNG not seeded') Not sure how this should be fixed, though. We use prngd here. To configure OpenSSH, I had to give prngd parameters to OpenSSH (i.e. that it should use prngd, and what the port number is). On testing certificates: It might be sufficient to generate a self-signed one, and distribute that as part of the test suite. Of course, you need to write server and client code to use the appropriate files. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:34 Message: Logged In: YES user_id=31392 I've made some recent changes that affect SSL_connect(), although the primary change is to improve the error messages. Dave and Martin, can you check the current code on Solaris? If it still fails, we'll at least get a clearer error message. BTW, does anyone have suggestions for how to test the code with key and cert files? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-07-28 10:39 Message: Logged In: YES user_id=21627 I still have the same problem, both on my own installation (both with 2.1.1, and with the CVS), as well as on cf.sf.net:/home/users/loewis/Python-2.1.1/python (you need to set LD_LIBRARY_PATH=/usr/local/ssl/lib to use this one). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-14 20:30 Message: Logged In: YES user_id=6380 David, is this still broken? On the SourceForge compile farm I can use SSL just fine. Check out my directory /home/users/gvanrossum/Python-2.1-solaris on cf.sourceforge.net. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-10 11:55 Message: Logged In: YES user_id=6380 So, David, you still interested in this? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-20 07:58 Message: Logged In: YES user_id=11375 Moshe checked in patch #405101 to the CVS tree. David, can you update and see if it fixed the problem for you? ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-03-18 02:30 Message: Logged In: YES user_id=11645 I just want to me-too Neil, since I submitted the original patch. I think it *did* fix problems on Solaris, as long as the user is running EGD and set up RANDFILE env. variable properly. Otherwise, you won't be getting connect errors, but a warning from Python about using a non-secure way to generate seed. (For the record, the problem is that Solaris doesn't have a /dev/urandom like Linux does) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-02 07:49 Message: Logged In: YES user_id=35752 It looks like patch "[ #405101 ] Add Random Seeding to OpenSSL" might fix this. Assigning to Andrew since he is assigned that patch too. ---------------------------------------------------------------------- Comment By: David M. Beazley (beazley) Date: 2001-02-15 06:12 Message: A followup on this bug.... It appears that Python is not properly seeding the OpenSSL random number generator when it creates a secure socket. This, in turn, causes the SSL_connect() function to fail due to improper random number seeding (although the error wasn't obvious--at least not to me). This is also due to an apparent "feature" of Solaris not providing a usable /dev/random device. A simple fix is to modify init_socket() in socketmodule.c to include a call to RAND_seed() like this: #ifdef USE_SSL SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); /* Sick hack for Solaris */ { char bogus[64]; /* Fill in bogus with some random noise */ ... RAND_seed(bogus, 64); } ... #endif Presumably one would need to think about the randomness actually passed to RAND_seed() (I have done nothing above). Here are related comments from the OpenSSL FAQ: "Cryptographic software needs a source of unpredictable data to work correctly. Many open source operating systems provide a "randomness device" that serves this purpose. On other systems, applications have to call the RAND_add() or RAND_seed() function with appropriate data before generating keys or performing public key encryption. Some broken applications do not do this. As of version 0.9.5, the OpenSSL functions that need randomness report an error if the random number generator has not been seeded with at least 128 bits of randomness. If this error occurs, please contact the author of the application you are using. It is likely that it never worked correctly. OpenSSL 0.9.5 and later make the error visible by refusing to perform potentially insecure encryption." Python-2.1a2 does not appear to include a fix, but I can't test it to find out (since 2.1a2 doesn't compile on my machine). Cheers, Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 From noreply@sourceforge.net Thu Oct 11 20:39:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 12:39:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-451607 ] SSL support in socket module needs tests Message-ID: Bugs item #451607, was opened at 2001-08-16 09:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Martin v. Löwis (loewis) Summary: SSL support in socket module needs tests Initial Comment: SSL support in the socket module should have a regression test, either in test_socket.py or in a separate test_ssl.py file. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 12:39 Message: Logged In: YES user_id=21627 1) When prngd runs on a TCP socket, it still won't accept connections from remote systems. 2) Sure, but would that simplify testing? 3) My comment was actually directed at bug #232460, which is Solaris specific: Solaris does not have a /dev/[u]random, so prngd is the only choice. I agree that using /dev/random is better if available. Furthermore, OpenSSL will already make this choice for you at installation time. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 11:44 Message: Logged In: YES user_id=72053 1) I think it's dangerous to run a prngd on an IP socket instead of a Unix domain socket. It creates the possibility of (either accidentally or by ignorance) running OpenSSL on a separate host from the prngd, making the random numbers vulnerable to network sniffing. That's bad--the numbers are cryptographic secrets and should not be exposed. 2) It's simple to set up a local SSL server with the command line openssl s_server option. 3) I'm not crazy about the whole prngd concept. I haven't looked at the CHILL interface yet, but if it's possible to abandon prngd and get random numbers through CHILL, that might be best. On Linux, /dev/urandom should be used. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 11:32 Message: Logged In: YES user_id=21627 On PRNG problems, it seems we have two options: - wait for, then require OpenSSL 0.9.7. It will look for a prngd socket in default locations (/var/run/egd-pool, /dev/egd-pool, /etc/egd-pool and /etc/entropy); then require administrators to set up OpenSSL that it indeed finds a prngd in these locations when needed. - expose RAND_add. Exposing any other of the interfaces is pointless; on our installation, prngd runs on localhost:708 instead of a Unix domain socket, and none of the other interfaces could use such a configuration. On top of RAND_add, we could communicate with prngd ourselves, e.g. by using the RANDFILE environment variable. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-03 08:12 Message: Logged In: YES user_id=21627 I'd suggest an approach similar to the one in test_socket, i.e. test only if can_fork. As for the certificates: I think they should be generated in advance and included into the test suite. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-08-28 06:53 Message: Logged In: YES user_id=12800 There is currently an extremely minimal test in test_socket_ssl.py, but it is fairly bogus because it requires an outside network connection. The only test there just tries to hit https://sf.net. Ideally, there would be a set of regression tests that only required local resources. I.e. it would set up a local SSL server, then do connects to it to test the various SSL features in socket module. Take a look at test_socket_ssl.py in the 2.2 cvs and see if you can improve matters. From what I understand, the tricky part is generating all the necessary certificates, etc. Also, if you intend to work on this, please log in to SF to continue this dialog so we know who we're talking to! :) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-27 22:58 Message: Logged In: NO I may be able to help with this. I'm fairly familiar with SSL in general, though not about the socket module. What's needed? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470 From noreply@sourceforge.net Fri Oct 12 00:31:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 16:31:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-470429 ] relax data representation spec Message-ID: Bugs item #470429, was opened at 2001-10-11 16:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470429&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: relax data representation spec Initial Comment: I'm not sure how to submit this--"feature request" isn't really right. A documentation bug may also be involved. The specification for integers is in section 2.1.4 "Numeric Types" of the Python 2.1.1 library docs, which makes it unclear whether integers are supposed to be part of the language spec, or part of the library--so alternative implementations can implement integers differently and still conform to the language spec. Anyway, the Numeric Types section says "Plain integers (also just called integers) are implemented using long in C, which gives them at least 32 bits of precision". I don't think a sentence like that should be in a specification, since it describes how something is implemented (using long in C) rather than how it should behave. For example, Jython uses java ints or longs rather than C longs, so the description that ints are implemented as C longs is simply incorrect. So if this section is supposed to be a specification, the reference to C longs should be removed, or clearly labelled as a description of how one particular implementation works. More substantively, I think the requirement that ints have 32 bits of precision should be relaxed to 30 bits. That just means changing the specification-- implementations are free to continue supporting 32 bit ints. But it makes it easier to implement Python using 32-bit words with tag bits, a traditional implementation method for dynamically typed languages. Numerous processors even offer hardware support for tagged integers--not just special processors like Lisp machines, but also general purpose RISC cpu's like the Sparc. Since Python offers arbitrary sized longs and already doesn't promise a fixed size for ints, I don't think it would suffer by offering implementers a little more flexibility. In a more philosophical vein, Python is a really cool language and the designers should begin to think of "success" in terms of more implementations being developed, native-code compilers being written, a possible ANSI standard some day, etc. So care should be taken to keep the formal specification as precise as possible, and free of requirements of things that are really just implementation hacks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470429&group_id=5470 From noreply@sourceforge.net Fri Oct 12 03:40:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 19:40:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-470040 ] ParseTuple t# vs subclasses Message-ID: Bugs item #470040, was opened at 2001-10-10 15:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470040&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: ParseTuple t# vs subclasses Initial Comment: Part of the problem in bug 468061 is that PyArg_ParseTuple's 't#' format rejects objects that are of a str subclass. Apparently that in turn is because the latter don't have Py_TPFLAGS_HAVE_GETCHARBUFFER. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-11 19:40 Message: Logged In: YES user_id=31435 Fixed. Inheritance of tp_as_buffer methods simply wasn't being done. Lib/test/test_descr.py; new revision: 1.88 Objects/typeobject.c; new revision: 2.100 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470040&group_id=5470 From noreply@sourceforge.net Fri Oct 12 03:44:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 19:44:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-468061 ] __str__ ignored in str subclass Message-ID: Bugs item #468061, was opened at 2001-10-04 13:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: __str__ ignored in str subclass Initial Comment: >From c.l.py; looks messy (e.g., if PyObject_String didn't ignore octetstring.__str__, the call to str() in the latter would just recurse forever; and he's right that b2a_hex requires a genuine string; and e.g. return b2a_hex(self + "") would be mighty obscure). ----Original Message----- From: Andreas.Trawoeger Sent: Thursday, October 04, 2001 7:10 AM To: python-list Subject: Python 2.2a4: Problems with overloading __str__ in subclasses of str I would like to write a class that behaves like a string only with the difference that it displays the string in hex. In Python 2.2a4 this should be possible by subclassing str and overloading __repr__ and __str__ : from binascii import b2a_hex class octetstring(str): def __repr__(self): return b2a_hex(str(self)) def __str__(self): return b2a_hex(str(self)) o = octetstring('A') print "o.__repr__():",o.__repr__() print "o.__str__() :",o.__str__() print "str(o) :",str(o) print "o :",o The result is pretty strange: o.__repr__(): 41 o.__str__() : 41 str(o) : A o : A I'm not shure what's the exact reason for this problem. Either __str__ isn't used for str() and print. Or I'm trapped in an chicken egg problem: I have to convert self to an string (otherwise b2a_hex won't work) to do this Python should use __str__ (which is the method I'm changing). cu andreas ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-11 19:44 Message: Logged In: YES user_id=31435 Bug 470040 has been fixed, so you'll be able to pass instances of str subclasses directly to b2a_hex (which uses the buffer interface to obtain the value; buffer methods simply weren't getting inherited; OTOH, I'm not sure this is such a great thing, since I'm not sure a str subclass in Python can override the buffer methods now inherited from str!). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-10 15:52 Message: Logged In: YES user_id=31435 The business about b2a_hex not accepting an object of a subclass of str is now the subject of bug 470040. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-04 13:57 Message: Logged In: YES user_id=31435 Looks like I assigned this to Guido by mistake; back to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 From noreply@sourceforge.net Fri Oct 12 06:12:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 11 Oct 2001 22:12:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 08:20:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 00:20:13 -0700 Subject: [Python-bugs-list] [ python-Bugs-465045 ] base n integer to string conversion Message-ID: Bugs item #465045, was opened at 2001-09-25 18:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: base n integer to string conversion Initial Comment: I see that the C files for int and long arithmetic already contain format() functions to format the numbers in arbitrary bases. The Python-exported decimal and hex conversion routines simply call the format function. I have three suggestions/requests, in increasing order of controversy: 1) How about exporting the format routine to Python, so there's a builtin function for converting an integer object to an arbitrary base 2<=b<=36. The code is already there--may as well let people use it. Something like format(n,base) would be a simple interface. 2) Right now there's no simple way to convert an integer to a raw byte string. I end up doing something like a = binascii.unhexlify("%x"%n) which is a kludge. I'd like to propose that as a special case, format(n,256) give the raw bytes. 3) Finally, if it's not too weird, format(n,128) could possibly return the BER (base 128) encoded representation of n, similar to pack("w",n) in perl. Bit 8 (the high bit) is set in each byte except the last. That means you can read BER integers byte by byte and know where they end. They're used in various security objects like X509 certificates. If not here, then sooner or later this function should appear in some library module. Thanks Paul ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-12 00:20 Message: Logged In: YES user_id=72053 The struct module doesn't give any way of converting arbitrary ints (meaning longs) to binary. Really, it's needed. Some people do it with gmpy, but if Python is going to support longs as a built-in type, one shouldn't have to resort to 3rd-party modules to read and write them in binary. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-05 19:09 Message: Logged In: YES user_id=6380 Various forms of fixed-length int to binary conversion are provided by the struct module. What's a "surprising deficiency" for you could simply be everybody else's lack of interest -- if nobody needs it, it won't be added to the language... :-) ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-05 16:05 Message: Logged In: YES user_id=72053 Re which bases are useful: I want base 256 all the time and find the lack of any clean way to convert ints to binary is a surprising deficiency in Python. I'm not too concerned about what the interface should be, but I hope one is added. I agree, BER is a bit obscure, but if you feel it's ok to put it in the binascii module, I might submit a patch. How about if I put base-256 there too? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-26 09:00 Message: Logged In: YES user_id=31435 1) Base 2 is semi-frequently requested. Note that Greg Wilson has open patch 455076 to add %b format and bin() function. No non-binary base has any fans. 2) I expect Paul means unbounded ints (based on previous requests). 3) Yup. Jeremy's Pisces may already deal w/ this (found via Google search). 3') Pickle already represents ints < 256 in one byte, and less than 2**16 in 2 (plus a 1-byte type code, of course). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-26 06:38 Message: Logged In: YES user_id=6380 1) Which bases besides 8, 10 and 16 do you plan to use? The C code strikes me as more general than needed. 2) You can do this with the struct module. No need for kludges. 3) BER seems pretty esoteric. The binascii module seems the right place. Feel free to submit a patch! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-09-25 18:45 Message: Logged In: YES user_id=72053 One minor comment: besides representing longs, BER encoded integers are also good for small integers. Numbers < 128 are represented in one byte. If short strings were marshalled with a BER-encoded length instead of a 4-byte length, that could save some space in .pyc files and pickled objects. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 From noreply@sourceforge.net Fri Oct 12 09:30:30 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 01:30:30 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Tim Peters (tim_one) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Fri Oct 12 10:39:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 02:39:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 10:44:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 02:44:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 >Status: Open Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Tim Peters (tim_one) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Fri Oct 12 12:53:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 04:53:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Tim Peters (tim_one) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-12 04:53 Message: Logged In: NO To use curses (if GPL'd ncurses is not available) you have to make a patch and you have to compile with: -DSTRICT_SYSV_CURSES (some ncurses features are not available in this case) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** ../patch/py_curses.h Fri Dec 22 22:51:10 2000 --- py_curses.h Wed Oct 10 15:06:40 2001 *************** *** 5,11 **** --- 5,15 ---- #ifdef HAVE_NCURSES_H #include #else + #if defined(__hpux) + #include + #else #include + #endif #endif #ifdef __cplusplus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use threads on has to compile with: -D_POSIX_C_SOURCE=199506L (reentrant fct. are defined then) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use ftello and fseeko (largefile) one has to compile with: -D_LARGEFILE_SOURCE a new largefile API: ftello64 and fseeko64 (instead of ftell64 and fseek64 tested by configure) is available with: -D_LARGEFILE64_SOURCE (at the moment there is no difference to -D_LARGEFILE_SOURCE) /usr/share/doc/lg_files.txt contains the HPUX large file white paper BTW in Makefile.pre.in if have replaced all occurences of: $(INSTALL) -d -m with: mkdir -p -m - cmkl (cmkleffner@gmx.de) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Fri Oct 12 14:23:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 06:23:13 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Tim Peters (tim_one) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 06:23 Message: Logged In: YES user_id=21627 cmkl, a few questions: - It appears that STRICT_SYSV_CURSES should only be used if HAVE_NCURSES is not defined. Correct? - http://devresource.hp.com/STK/partner/xcurses.html claims that cur_colr is targeted for obsolescence. Why should we use it then? The same paper claims that _XOPEN_SOURCE_EXTENDED must be defined on 10.20 if /usr/include/curses.h (Xcurses) is used, due to a bug. - What functions does Python use that are only available with _POSIX_C_SOURCE=199506L ? - Could you please check out the current CVS and determine whether Python correctly enables large file support? What is the problem if only ftello/fseeko are used? What is the output of 'getconf LFS_CFLAGS'? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-12 04:53 Message: Logged In: NO To use curses (if GPL'd ncurses is not available) you have to make a patch and you have to compile with: -DSTRICT_SYSV_CURSES (some ncurses features are not available in this case) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** ../patch/py_curses.h Fri Dec 22 22:51:10 2000 --- py_curses.h Wed Oct 10 15:06:40 2001 *************** *** 5,11 **** --- 5,15 ---- #ifdef HAVE_NCURSES_H #include #else + #if defined(__hpux) + #include + #else #include + #endif #endif #ifdef __cplusplus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use threads on has to compile with: -D_POSIX_C_SOURCE=199506L (reentrant fct. are defined then) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use ftello and fseeko (largefile) one has to compile with: -D_LARGEFILE_SOURCE a new largefile API: ftello64 and fseeko64 (instead of ftell64 and fseek64 tested by configure) is available with: -D_LARGEFILE64_SOURCE (at the moment there is no difference to -D_LARGEFILE_SOURCE) /usr/share/doc/lg_files.txt contains the HPUX large file white paper BTW in Makefile.pre.in if have replaced all occurences of: $(INSTALL) -d -m with: mkdir -p -m - cmkl (cmkleffner@gmx.de) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Fri Oct 12 14:46:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 06:46:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-470429 ] relax data representation spec Message-ID: Bugs item #470429, was opened at 2001-10-11 16:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470429&group_id=5470 Category: None Group: Feature Request >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: relax data representation spec Initial Comment: I'm not sure how to submit this--"feature request" isn't really right. A documentation bug may also be involved. The specification for integers is in section 2.1.4 "Numeric Types" of the Python 2.1.1 library docs, which makes it unclear whether integers are supposed to be part of the language spec, or part of the library--so alternative implementations can implement integers differently and still conform to the language spec. Anyway, the Numeric Types section says "Plain integers (also just called integers) are implemented using long in C, which gives them at least 32 bits of precision". I don't think a sentence like that should be in a specification, since it describes how something is implemented (using long in C) rather than how it should behave. For example, Jython uses java ints or longs rather than C longs, so the description that ints are implemented as C longs is simply incorrect. So if this section is supposed to be a specification, the reference to C longs should be removed, or clearly labelled as a description of how one particular implementation works. More substantively, I think the requirement that ints have 32 bits of precision should be relaxed to 30 bits. That just means changing the specification-- implementations are free to continue supporting 32 bit ints. But it makes it easier to implement Python using 32-bit words with tag bits, a traditional implementation method for dynamically typed languages. Numerous processors even offer hardware support for tagged integers--not just special processors like Lisp machines, but also general purpose RISC cpu's like the Sparc. Since Python offers arbitrary sized longs and already doesn't promise a fixed size for ints, I don't think it would suffer by offering implementers a little more flexibility. In a more philosophical vein, Python is a really cool language and the designers should begin to think of "success" in terms of more implementations being developed, native-code compilers being written, a possible ANSI standard some day, etc. So care should be taken to keep the formal specification as precise as possible, and free of requirements of things that are really just implementation hacks. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 06:46 Message: Logged In: YES user_id=6380 The documentation reference to C long ints is not intended to require any particular implementation; rather, it is intended to require that the range requirements for Python ints are the same as those for C longs -- basically, at least 32 bits precision. We could change the docs to clarify that, but I'm not sure what purpose it serves. Implementations should behave *as if* it were implemented using C longs on some hypothetical architecture, and that's what Jython does. Regarding the suggestion to reduce the limit to 30 bits: the trend is going the other way. Read PEP 237. Eventually there won't be any difference between plain and long ints, so I don't see a point in reducing the number of bits for plain ints. If you do an alternative implementation that limits plain ints to 30 bits and gracefully overflows to long ints, that may technically not be conforming, but in a few years it will be, so who cares. I personally define success in terms of number of users. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470429&group_id=5470 From noreply@sourceforge.net Fri Oct 12 15:23:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 07:23:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-470582 ] sre bug with nested groups Message-ID: Bugs item #470582, was opened at 2001-10-12 07:23 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470582&group_id=5470 Category: Regular Expressions Group: None Status: Open Resolution: None Priority: 5 Submitted By: Markus F.X.J. Oberhumer (mfx) Assigned to: Fredrik Lundh (effbot) Summary: sre bug with nested groups Initial Comment: sre incorrectly assigns non-matched inner groups. See m.group(2) in the example below. This bug seems to be present in all sre versions (tested with python 1.6.x, 2.0.x, 2.1.x and 2.2a4). Markus P.S. this seems not related to SF bug #468169 Script started on Tue Oct 9 05:09:28 2001 mfx@laetitia:~/tmp > cat sre_bug.py import sys, pre, sre def time2ms(re, time): m = re.search(r"^((\d)\:)?(\d\d)\.(\d\d\d)$", time) print re.__name__ if hasattr(re, "__version__"): print re.__version__ print m.groups() print sys.version time2ms(pre, "34.123") time2ms(sre, "34.123") mfx@laetitia:~/tmp > pyhon2.2a4 sre_bug.py 2.2a4 (#1, Sep 29 2001, 23:40:47) [GCC 2.95.4 20010902 (Debian prerelease)] pre (None, None, '34', '123') sre 2.1.1 (None, '3', '34', '123') Script done on Tue Oct 9 05:09:45 2001 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470582&group_id=5470 From noreply@sourceforge.net Fri Oct 12 15:27:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 07:27:27 -0700 Subject: [Python-bugs-list] [ python-Bugs-465045 ] base n integer to string conversion Message-ID: Bugs item #465045, was opened at 2001-09-25 18:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: base n integer to string conversion Initial Comment: I see that the C files for int and long arithmetic already contain format() functions to format the numbers in arbitrary bases. The Python-exported decimal and hex conversion routines simply call the format function. I have three suggestions/requests, in increasing order of controversy: 1) How about exporting the format routine to Python, so there's a builtin function for converting an integer object to an arbitrary base 2<=b<=36. The code is already there--may as well let people use it. Something like format(n,base) would be a simple interface. 2) Right now there's no simple way to convert an integer to a raw byte string. I end up doing something like a = binascii.unhexlify("%x"%n) which is a kludge. I'd like to propose that as a special case, format(n,256) give the raw bytes. 3) Finally, if it's not too weird, format(n,128) could possibly return the BER (base 128) encoded representation of n, similar to pack("w",n) in perl. Bit 8 (the high bit) is set in each byte except the last. That means you can read BER integers byte by byte and know where they end. They're used in various security objects like X509 certificates. If not here, then sooner or later this function should appear in some library module. Thanks Paul ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:27 Message: Logged In: YES user_id=6380 OK, I believe you. Can you submit a patch? ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 00:20 Message: Logged In: YES user_id=72053 The struct module doesn't give any way of converting arbitrary ints (meaning longs) to binary. Really, it's needed. Some people do it with gmpy, but if Python is going to support longs as a built-in type, one shouldn't have to resort to 3rd-party modules to read and write them in binary. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-05 19:09 Message: Logged In: YES user_id=6380 Various forms of fixed-length int to binary conversion are provided by the struct module. What's a "surprising deficiency" for you could simply be everybody else's lack of interest -- if nobody needs it, it won't be added to the language... :-) ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-05 16:05 Message: Logged In: YES user_id=72053 Re which bases are useful: I want base 256 all the time and find the lack of any clean way to convert ints to binary is a surprising deficiency in Python. I'm not too concerned about what the interface should be, but I hope one is added. I agree, BER is a bit obscure, but if you feel it's ok to put it in the binascii module, I might submit a patch. How about if I put base-256 there too? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-09-26 09:00 Message: Logged In: YES user_id=31435 1) Base 2 is semi-frequently requested. Note that Greg Wilson has open patch 455076 to add %b format and bin() function. No non-binary base has any fans. 2) I expect Paul means unbounded ints (based on previous requests). 3) Yup. Jeremy's Pisces may already deal w/ this (found via Google search). 3') Pickle already represents ints < 256 in one byte, and less than 2**16 in 2 (plus a 1-byte type code, of course). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-26 06:38 Message: Logged In: YES user_id=6380 1) Which bases besides 8, 10 and 16 do you plan to use? The C code strikes me as more general than needed. 2) You can do this with the struct module. No need for kludges. 3) BER seems pretty esoteric. The binascii module seems the right place. Feel free to submit a patch! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-09-25 18:45 Message: Logged In: YES user_id=72053 One minor comment: besides representing longs, BER encoded integers are also good for small integers. Numbers < 128 are represented in one byte. If short strings were marshalled with a BER-encoded length instead of a 4-byte length, that could save some space in .pyc files and pickled objects. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465045&group_id=5470 From noreply@sourceforge.net Fri Oct 12 15:33:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 07:33:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 15:43:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 07:43:03 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Tim Peters (tim_one) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:43 Message: Logged In: YES user_id=6380 To rptownsend: the warnings you get seem to come from assigning function pointers to void*. Could you try the following change? 3693c3693 < {NAME, offsetof(PyTypeObject, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(PyTypeObject, SLOT), (void *)FUNCTION, WRAPPER} 3695c3695 < {NAME, offsetof(etype, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(etype, SLOT), (void *)FUNCTION, WRAPPER} There's one more line with a warning; can you figure out where to put a (void*) cast there yourself and let me know? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 06:23 Message: Logged In: YES user_id=21627 cmkl, a few questions: - It appears that STRICT_SYSV_CURSES should only be used if HAVE_NCURSES is not defined. Correct? - http://devresource.hp.com/STK/partner/xcurses.html claims that cur_colr is targeted for obsolescence. Why should we use it then? The same paper claims that _XOPEN_SOURCE_EXTENDED must be defined on 10.20 if /usr/include/curses.h (Xcurses) is used, due to a bug. - What functions does Python use that are only available with _POSIX_C_SOURCE=199506L ? - Could you please check out the current CVS and determine whether Python correctly enables large file support? What is the problem if only ftello/fseeko are used? What is the output of 'getconf LFS_CFLAGS'? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-12 04:53 Message: Logged In: NO To use curses (if GPL'd ncurses is not available) you have to make a patch and you have to compile with: -DSTRICT_SYSV_CURSES (some ncurses features are not available in this case) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** ../patch/py_curses.h Fri Dec 22 22:51:10 2000 --- py_curses.h Wed Oct 10 15:06:40 2001 *************** *** 5,11 **** --- 5,15 ---- #ifdef HAVE_NCURSES_H #include #else + #if defined(__hpux) + #include + #else #include + #endif #endif #ifdef __cplusplus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use threads on has to compile with: -D_POSIX_C_SOURCE=199506L (reentrant fct. are defined then) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use ftello and fseeko (largefile) one has to compile with: -D_LARGEFILE_SOURCE a new largefile API: ftello64 and fseeko64 (instead of ftell64 and fseek64 tested by configure) is available with: -D_LARGEFILE64_SOURCE (at the moment there is no difference to -D_LARGEFILE_SOURCE) /usr/share/doc/lg_files.txt contains the HPUX large file white paper BTW in Makefile.pre.in if have replaced all occurences of: $(INSTALL) -d -m with: mkdir -p -m - cmkl (cmkleffner@gmx.de) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Fri Oct 12 18:04:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 10:04:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-470633 ] segfault with 2.2a4 Message-ID: Bugs item #470633, was opened at 2001-10-12 10:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470633&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: segfault with 2.2a4 Initial Comment: I've attached a simply script to periodically check a POP3 mailbox and print the number of messages available there. The first two checks are successful, then the third time through a segfault occurs (I think in the socket code.) I've attached the script which includes a gdb backtrace commented out at the end Python v2.2a4 Mandrake Linux 8.1, glibc v2.2.4 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470633&group_id=5470 From noreply@sourceforge.net Fri Oct 12 18:06:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 10:06:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-470634 ] readlines() on linux requires 2 ^D's Message-ID: Bugs item #470634, was opened at 2001-10-12 10:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470634&group_id=5470 Category: Python Interpreter Core Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: readlines() on linux requires 2 ^D's Initial Comment: Platform: This bug has been confirmed on Python 1.5.2, 2.0.1, and 2.1.1 running on Debian Linux 2.2 w/kernel 2.4.9 and libc6 2.2.4, as well as Python 1.5.2 running on Debian Linux 2.0 w/kernel 2.0.38 and libc6 2.0.7. It has been found *not* to happen with Python 2.1.1 on Solaris 8 and Cygwin. Bug Description: When using the standard idiom: for line in sys.stdin.readlines(): print line ...to loop through lines submitted to stdin, and providing that input interactively (i.e. typing at the console), it is necessary to press ^D *twice* at the start of line of input to terminate input. The behavior can be noticed both in python's interactive mode *and* when running a script. Redirected input does not show this problem, that is if the above lines are contained in myscript.py, and you execute: cat | myscript.py A single ^D is sufficient to terminate inputs. Also, the problem is unique to the "readlines()" method. If you code the loop as for line in sys.stdin.read().splitlines(): then a single ^D at the beginning of a line is sufficient to terminate input. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470634&group_id=5470 From noreply@sourceforge.net Fri Oct 12 18:07:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 10:07:28 -0700 Subject: [Python-bugs-list] [ python-Bugs-470635 ] simple script causes segfault Message-ID: Bugs item #470635, was opened at 2001-10-12 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Travis Hume (travishume) Assigned to: Nobody/Anonymous (nobody) Summary: simple script causes segfault Initial Comment: I've attached a simple script that uses the poplib to periodically check a mailbox and print the number of messages available. Seems to work twice, then the third time through a segfault occurs. The attached script contains a gdb backtrace at the end. Python v2.2a4 Linux Mandrake v8.1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 From noreply@sourceforge.net Fri Oct 12 18:16:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 10:16:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-470634 ] readlines() on linux requires 2 ^D's Message-ID: Bugs item #470634, was opened at 2001-10-12 10:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470634&group_id=5470 Category: Python Interpreter Core Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: readlines() on linux requires 2 ^D's Initial Comment: Platform: This bug has been confirmed on Python 1.5.2, 2.0.1, and 2.1.1 running on Debian Linux 2.2 w/kernel 2.4.9 and libc6 2.2.4, as well as Python 1.5.2 running on Debian Linux 2.0 w/kernel 2.0.38 and libc6 2.0.7. It has been found *not* to happen with Python 2.1.1 on Solaris 8 and Cygwin. Bug Description: When using the standard idiom: for line in sys.stdin.readlines(): print line ...to loop through lines submitted to stdin, and providing that input interactively (i.e. typing at the console), it is necessary to press ^D *twice* at the start of line of input to terminate input. The behavior can be noticed both in python's interactive mode *and* when running a script. Redirected input does not show this problem, that is if the above lines are contained in myscript.py, and you execute: cat | myscript.py A single ^D is sufficient to terminate inputs. Also, the problem is unique to the "readlines()" method. If you code the loop as for line in sys.stdin.read().splitlines(): then a single ^D at the beginning of a line is sufficient to terminate input. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 10:16 Message: Logged In: YES user_id=6380 Confirmed on Linux (RH 6.2) with 2.2 from CVS too. I think it's probably a logic error in readlines() -- maybe the first EOF ends it reading the current line, but then it needs another EOF to break out of the loop. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470634&group_id=5470 From noreply@sourceforge.net Fri Oct 12 18:19:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 10:19:01 -0700 Subject: [Python-bugs-list] [ python-Bugs-470633 ] segfault with 2.2a4 Message-ID: Bugs item #470633, was opened at 2001-10-12 10:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470633&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: segfault with 2.2a4 Initial Comment: I've attached a simply script to periodically check a POP3 mailbox and print the number of messages available there. The first two checks are successful, then the third time through a segfault occurs (I think in the socket code.) I've attached the script which includes a gdb backtrace commented out at the end Python v2.2a4 Mandrake Linux 8.1, glibc v2.2.4 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 10:19 Message: Logged In: YES user_id=6380 Duplicate of #470635 -- since that one has a name attached to it, I'm closing this anonymous one. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470633&group_id=5470 From noreply@sourceforge.net Fri Oct 12 18:46:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 10:46:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-470635 ] simple script causes segfault Message-ID: Bugs item #470635, was opened at 2001-10-12 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Travis Hume (travishume) >Assigned to: Guido van Rossum (gvanrossum) Summary: simple script causes segfault Initial Comment: I've attached a simple script that uses the poplib to periodically check a mailbox and print the number of messages available. Seems to work twice, then the third time through a segfault occurs. The attached script contains a gdb backtrace at the end. Python v2.2a4 Linux Mandrake v8.1 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 10:46 Message: Logged In: YES user_id=6380 I'm deleting your attachment since it contained a password. I'm investigating the crash now.O It's an assert atat Objects/typeobject.c:329 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 From noreply@sourceforge.net Fri Oct 12 19:39:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 11:39:44 -0700 Subject: [Python-bugs-list] [ python-Bugs-470635 ] simple script causes segfault Message-ID: Bugs item #470635, was opened at 2001-10-12 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Travis Hume (travishume) Assigned to: Guido van Rossum (gvanrossum) Summary: simple script causes segfault Initial Comment: I've attached a simple script that uses the poplib to periodically check a mailbox and print the number of messages available. Seems to work twice, then the third time through a segfault occurs. The attached script contains a gdb backtrace at the end. Python v2.2a4 Linux Mandrake v8.1 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 11:39 Message: Logged In: YES user_id=6380 I'm hot on the trail. The problem is most likely in getaddrinfo(). With a debug build, I get this: >>> import socket [... refs] >>> socket.getaddrinfo("mail.tenzing.com", 110, 0, socket.SOCK_STREAM) [(2, 1, 6, '', ('63.115.0.25', 110))] ../Objects/tupleobject.c:147 negative ref count -1 [... refs] >>> ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 10:46 Message: Logged In: YES user_id=6380 I'm deleting your attachment since it contained a password. I'm investigating the crash now.O It's an assert atat Objects/typeobject.c:329 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 From noreply@sourceforge.net Fri Oct 12 20:01:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 12:01:38 -0700 Subject: [Python-bugs-list] [ python-Bugs-470635 ] simple script causes segfault Message-ID: Bugs item #470635, was opened at 2001-10-12 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Travis Hume (travishume) Assigned to: Guido van Rossum (gvanrossum) Summary: simple script causes segfault Initial Comment: I've attached a simple script that uses the poplib to periodically check a mailbox and print the number of messages available. Seems to work twice, then the third time through a segfault occurs. The attached script contains a gdb backtrace at the end. Python v2.2a4 Linux Mandrake v8.1 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:01 Message: Logged In: YES user_id=6380 Fixed in current CVS, socketmodule.c rev 1.179. This was a refcount bug in the getaddrinfo() wrapper. If you don't have access to CVS, get rid of the two Py_XDECREF(pobj) calls in PySocket_getaddrinfo(). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 11:39 Message: Logged In: YES user_id=6380 I'm hot on the trail. The problem is most likely in getaddrinfo(). With a debug build, I get this: >>> import socket [... refs] >>> socket.getaddrinfo("mail.tenzing.com", 110, 0, socket.SOCK_STREAM) [(2, 1, 6, '', ('63.115.0.25', 110))] ../Objects/tupleobject.c:147 negative ref count -1 [... refs] >>> ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 10:46 Message: Logged In: YES user_id=6380 I'm deleting your attachment since it contained a password. I'm investigating the crash now.O It's an assert atat Objects/typeobject.c:329 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470635&group_id=5470 From noreply@sourceforge.net Fri Oct 12 20:29:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 12:29:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 20:37:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 12:37:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 21:02:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 13:02:29 -0700 Subject: [Python-bugs-list] [ python-Bugs-470634 ] readlines() on linux requires 2 ^D's Message-ID: Bugs item #470634, was opened at 2001-10-12 10:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470634&group_id=5470 Category: Python Interpreter Core >Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) Summary: readlines() on linux requires 2 ^D's Initial Comment: Platform: This bug has been confirmed on Python 1.5.2, 2.0.1, and 2.1.1 running on Debian Linux 2.2 w/kernel 2.4.9 and libc6 2.2.4, as well as Python 1.5.2 running on Debian Linux 2.0 w/kernel 2.0.38 and libc6 2.0.7. It has been found *not* to happen with Python 2.1.1 on Solaris 8 and Cygwin. Bug Description: When using the standard idiom: for line in sys.stdin.readlines(): print line ...to loop through lines submitted to stdin, and providing that input interactively (i.e. typing at the console), it is necessary to press ^D *twice* at the start of line of input to terminate input. The behavior can be noticed both in python's interactive mode *and* when running a script. Redirected input does not show this problem, that is if the above lines are contained in myscript.py, and you execute: cat | myscript.py A single ^D is sufficient to terminate inputs. Also, the problem is unique to the "readlines()" method. If you code the loop as for line in sys.stdin.read().splitlines(): then a single ^D at the beginning of a line is sufficient to terminate input. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:02 Message: Logged In: YES user_id=6380 Fixed in CVS. Try fileobject.c rev. 2.134. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 10:16 Message: Logged In: YES user_id=6380 Confirmed on Linux (RH 6.2) with 2.2 from CVS too. I think it's probably a logic error in readlines() -- maybe the first EOF ends it reading the current line, but then it needs another EOF to break out of the loop. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470634&group_id=5470 From noreply@sourceforge.net Fri Oct 12 21:16:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 13:16:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 21:24:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 13:24:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 21:56:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 13:56:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-416906 ] readlines() corrupts file opened for 'a' Message-ID: Bugs item #416906, was opened at 2001-04-17 18:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=416906&group_id=5470 Category: None Group: None Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: readlines() corrupts file opened for 'a' Initial Comment: Version 2.1 final release for Windows (using Idle) Windows 98 2nd edition f=open('c:\My Documents\textfile.txt', 'a') f.write('hello file\n') f.readlines() raises the following error - it probably should. Traceback (most recent call last): File "", line 1, in ? f.readlines() IOError: [Errno 9] Bad file descriptor However, the file gets a bunch of garbage appended to it. In some cases, the entire contents of the GUI window gets appended. In Redhat 6.2 (also with Version 2.1 Final)the same code just erases the file. ---------------------------------------------------------------------- Comment By: Martin D Katz, Ph.D. (drbits) Date: 2001-10-12 13:56 Message: Logged In: YES user_id=276840 Under MS Windows 2K, opening the file for 'w' and then reading (with read()) also causes this. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-04-18 11:24 Message: Logged In: YES user_id=31435 Note that the problem isn't *just* output followed by input without an intervening positioning operation: the file was opened in append mode ("a"), not append-update mode ("a+"), so input isn't "legal" period. MS can mangle part of the file even if there is a seek() in between (but doesn't if opened in append-update mode instead). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-18 07:00 Message: Logged In: YES user_id=6380 Writing after reading without an intervening seek() is illegal in C's stdio, but the Python file object (a thin wrapper around stdio) doesn't know how to check for this error condition. So, we have to say "then don't do this". Possible point of light: there's discussion on writing a new I/O library for Python that avoids relying on stdio. It could fix this issue, too. Closing the bug now. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-04-18 02:39 Message: Logged In: YES user_id=31435 Back to you, Guido: you're the one fond of arguing that MS is within its rights to do anything whatsoever when the rules for C stdio are broken . Seriously, MS won't "fix" this (they'll deny it's broken). Same kind of thing happens in straight C under MS. BTW, note that Anonymous did not claim it's an MS bug: two different bugs were claimed (Win98SE appends crap, RH 6.2 destroys the whole file). I don't see a realistic choice here but to say "hmm -- tough luck, don't do that". ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-17 21:17 Message: Logged In: YES user_id=6380 Assigned to Tim, because he believes that this is a Microsoft bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=416906&group_id=5470 From noreply@sourceforge.net Fri Oct 12 22:09:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 14:09:03 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 22:12:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 14:12:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-438517 ] tkSimpleDialog.askString option Message-ID: Bugs item #438517, was opened at 2001-07-04 05:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=438517&group_id=5470 Category: Tkinter Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: tkSimpleDialog.askString option Initial Comment: I read the documentation of tkSimpleDialog.askString and found that there are only two options: initialvalue and parent. However, it doesn't support the option, show, from the Entry widget option that avoids echoing the string I input. I found the tkSimpleDialog.askString is a perfect fit for dialogs asking for encryption key or password. Although it is easy to write my own, I don't see the need because of an option. Many thanks Joe ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 14:12 Message: Logged In: YES user_id=6380 If you can submit this as a patch to the existing code it'll breeze right through. If you wait for one of us to write it, it may be a loooooooong time before we get to it (like when we need this ourselves :-). Since you know how to do it, it shouldn't be too hard for you to prepare a patch. See http://python.sourceforge.net/sf-faq.html#patches ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=438517&group_id=5470 From noreply@sourceforge.net Fri Oct 12 22:13:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 14:13:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-452221 ] __dynamic__ should be on by default Message-ID: Bugs item #452221, was opened at 2001-08-17 14:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=452221&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 2 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: __dynamic__ should be on by default Initial Comment: Problem: it's twice as slow! :-( Static classes are about as fast as classic classes, currently. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 14:13 Message: Logged In: YES user_id=6380 This is fixed now -- and it's not any slower than static was. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=452221&group_id=5470 From noreply@sourceforge.net Fri Oct 12 22:41:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 14:41:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Fri Oct 12 22:51:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 14:51:04 -0700 Subject: [Python-bugs-list] [ python-Bugs-219772 ] Interactive InterPreter+ Thread -> core dump at exit Message-ID: Bugs item #219772, was opened at 2000-10-30 08:28 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=219772&group_id=5470 Category: Threads Group: None >Status: Closed >Resolution: Fixed Priority: 4 Submitted By: Ivan Munoz (imbunche) Assigned to: Guido van Rossum (gvanrossum) Summary: Interactive InterPreter+ Thread -> core dump at exit Initial Comment: The following code executes a thread that prints a message every second while the main thread shows a python shell. It will run fine and if one leaves the interpreter with python will exit cleanly. However if one press to kill the interactive interpreter and then exits python or press return a couple of time you get a core dump. My platform is: HP-UX te35 B.11.00 A 9000/785 2015352223 two-user license I am running Python 2.0 This is the code of the test program: #!/usr/local/bin/python import time, threading, whrandom, code, sys class MyThread(threading.Thread): """ reports once per second """ def run(self): while 1: print "alive ...." time.sleep(1) print self.getName(), "is DONE" if __name__ == '__main__': threadList = [] # Create MyThread() threads t = MyThread() t.setDaemon(1) t.start() # This is the main thread ... since there is only one other # thread and it is a 'daemon', killing the main thread kills # the whole thing. locals = sys.modules['__main__'].__dict__ code.interact(locals) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 14:51 Message: Logged In: YES user_id=6380 I'm tentatively closing this; thread_pthread.h rev 2.33 blocks signals during thread creation, causing threads to be started with all signals off. Please check that this in fact resolves the problem. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-10-30 08:47 Message: I can reproduce this on RH Linux, but not running Python under gdb. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=219772&group_id=5470 From noreply@sourceforge.net Fri Oct 12 22:52:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 14:52:14 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 14:52 Message: Logged In: YES user_id=6380 Since I've now applied your patch, I presume this is fixed, and I'm closing the bug report. Let me know if there are still problems. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-09 11:40 Message: Logged In: YES user_id=56897 Patch is #468347 [mask signals for non-main pthreads] ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 10:09 Message: Logged In: YES user_id=56897 I've submitted the patch for pthread signal masking. My biggest concerns are the guesses I made for DCE threads and whether they will work for AIX which might need to use sigthreadmask(). Regarding reproducing this on Linux, I was able to get Linux to crash if I held down Ctrl-C (with fairly fast key repeat). After starting the spinning thread, Python would crash on Linux under a storm of SIGINTs within 30 seconds or so. Without the spinning thread, I couldn't get it to crash. With the patch applied, the spinning thread running during the storm of SIGINTs wouldn't crash it. So that implies the signal masking is doing something good even in the Linux case. Re: my SF problems, I submitted a few support requests. Hopefully something gets fixed. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 08:38 Message: Logged In: YES user_id=56897 I've checked the Monitor button while logged in, but I still do not receive email notification of updates. When I clicked it again, it said I was no longer monitoring, so I clicked it yet again back into monitoring mode. Apparently SF knows I'm monitoring it, but it still doesn't send me email. Mail to my @users.sf.net account does work, so I'm at a loss to explain why a) My login cookie doesn't stick around very long at all and b) Why I never get monitor email from SF Re: the patch, I have something that works well on Solaris. I'll try it on Linux today, but I don't have access to an HP-UX system. I'm a little concerned about the impact to HP-UX (pre 11.0 and post 11.0) and AIX, and I don't have access to those machines to check out those concerns. Hopefully I'll have the patch posted by today. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Sat Oct 13 04:08:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 20:08:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 13 04:43:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 20:43:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 13 06:39:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 12 Oct 2001 22:39:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-470781 ] Build broken in Mac OS X Message-ID: Bugs item #470781, was opened at 2001-10-12 22:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 Category: Macintosh Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jack Jansen (jackjansen) Summary: Build broken in Mac OS X Initial Comment: ./configure --with-suffix=.exe make generates a warning for every file: cc: unrecognized option `-Kpthread' and then terminates with an error: cc -Kpthread -flat_namespace -u __dummy -u _PyMac_Error -framework System -framework Carbon -framework Foundation -o python.exe \ Modules/python.o \ libpython2.2.a cc: unrecognized option `-Kpthread' /usr/bin/ld: Undefined symbols: _pthread_sigmask make: *** [python.exe] Error 1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 From noreply@sourceforge.net Sat Oct 13 09:14:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 01:14:10 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 13 09:31:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 01:31:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-454383 ] make test failed in solaris_2.7 with gcc Message-ID: Bugs item #454383, was opened at 2001-08-22 16:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=454383&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: Thiep Duong (duongusa) Assigned to: A.M. Kuchling (akuchling) Summary: make test failed in solaris_2.7 with gcc Initial Comment: I tried to compile/build configure with no option, make gave no apparent error, but make test failed at the below messages. Any clue what to do next? I am new with Python .... Please help. skipping 'sunaudiodev' extension (up-to-date) running build_scripts not copying /fas2/eda2/duong/dev/source/UNIX/Python-2.1.1/Tools/scripts/pydoc (up-to-date) find ./Lib -name '*.py[co]' -print | xargs rm -f PYTHONPATH= ./python -tt ./Lib/test/regrtest.py -l Traceback (most recent call last): File "./Lib/test/regrtest.py", line 39, in ? import random File "/fas2/eda2/duong/dev/source/UNIX/Python-2.1.1/Lib/random.py", line 76, in ? from math import log as _log, exp as _exp, pi as _pi, e as _e ImportError: cannot import name log make: [test] Error 1 (ignored) PYTHONPATH= ./python -tt ./Lib/test/regrtest.py -l Traceback (most recent call last): File "./Lib/test/regrtest.py", line 39, in ? import random File "/fas2/eda2/duong/dev/source/UNIX/Python-2.1.1/Lib/random.py", line 76, in ? from math import log as _log, exp as _exp, pi as _pi, e as _e ImportError: cannot import name log Bus Error - core dumped make: *** [test] Error 138 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:31 Message: Logged In: YES user_id=21627 On the readline problem: If you only have a static library, then you should not integrate it into a shared extension module, unless it is PIC code. This results in the following alternatives: - enable readline in Modules/Setup as a non-shared module, or - recompile readline by giving -fPIC/-KPIC, or - build readline as a shared library. If neither of these is acceptable, then you can add the -mimpure-text as a gcc linker option. Since it seems that your original problem got solved, I'm closing this report. If you have any further problems, please create a new one. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-09-05 21:24 Message: Logged In: NO > Thanks for helping out. I got it fixed now, here what I did I updated > the readline-4.2 library and the coredump went away, but I got ton of > messages like: > > gcc -g -O2 -Wall -Wstrict-prototypes -fPIC -I. -I/fas2/eda2/duong/dev/source/UNIX/Python-2.1.1/./Include - > I/usr/local/include -IInclude/ -c /fas2/eda2/duong/dev/source/UNIX/Python-2.1.1/Modules/readline.c -o buil > d/temp.solaris-2.7-sun4u-2.1/readline.o > /fas2/eda2/duong/dev/source/UNIX/Python-2.1.1/Modules/readline.c: In function `set_completer_delims': > /fas2/eda2/duong/dev/source/UNIX/Python-2.1.1/Modules/readline.c:212: warning: passing arg 1 of `free' dis > cards qualifiers from pointer target type > gcc -shared build/temp.solaris-2.7-sun4u-2.1/readline.o -L/usr/lib/termcap -L/usr/local/lib -lreadline -lt > ermcap -o build/lib.solaris-2.7-sun4u-2.1/readline.so > Text relocation remains referenced > against symbol offset in file > sigdelset 0x64 /usr/local/lib/libreadline.a(signals.o) > rl_on_new_line_with_prompt 0x1e0 /usr/local/lib/libreadline.a(readline.o) > kill 0x74 /usr/local/lib/libreadline.a(signals.o) > rl_beginning_of_history 0x238 /usr/local/lib/libreadline.a(vi_mode.o) > rl_beginning_of_history 0x250 /usr/local/lib/libreadline.a(vi_mode.o) > > ...... many other modules too. > I also have a SUN C/C++ compiler (V5.0),and use this instead of > gcc, and the python seems to be much happier. > > Thanks for looking into this. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:31 Message: Logged In: YES user_id=6380 It looks like the math module didn't get compiled completely. Possibly the compiler invocation used by setup.py to build shared libraries doesn't work for your platform. Assigning this to Andrew, who knows about such things. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=454383&group_id=5470 From noreply@sourceforge.net Sat Oct 13 09:39:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 01:39:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 13 09:50:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 01:50:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-417240 ] [Solaris] warning during build Message-ID: Bugs item #417240, was opened at 2001-04-19 00:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417240&group_id=5470 Category: Build Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Dimitri Papadopoulos (papadopo) Assigned to: Eric S. Raymond (esr) Summary: [Solaris] warning during build Initial Comment: Hi, While building on Solaris with Sun's compiler I get this warning: $ ./configure --prefix=/usr/local/python-2.1 --with-signal-module --with-threads $make all [...] building '_curses_panel' extension cc -I. -I/tmp/Python-2.1/./Include -I/usr/local/include -IInclude/ -c /tmp/Python-2.1/Modules/_curses_panel.c -o build/temp.solaris-2.7-sun4u-2.1/_curses_panel.o -O -I. -I./Include -DHAVE_CONFIG_H "/tmp/Python-2.1/Modules/_curses_panel.c", line 307: warning: argument #2 is incompatible with prototype: prototype: pointer to char : "/usr/include/panel.h", line 51 argument : pointer to struct _object {int ob_refcnt, pointer to struct _typeobject {..} ob_type} [...] This is only a warning, not very important. Best Regards, Dimitri ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:50 Message: Logged In: YES user_id=21627 Thanks for the bug report. This is fixed in _curses_panel.c 1.7. ---------------------------------------------------------------------- Comment By: Dimitri Papadopoulos (papadopo) Date: 2001-04-19 02:01 Message: Logged In: YES user_id=52414 This is with Python 2.1 by the way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417240&group_id=5470 From noreply@sourceforge.net Sat Oct 13 10:03:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 02:03:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-417290 ] [Solaris] errors in _curses extension Message-ID: Bugs item #417290, was opened at 2001-04-19 01:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417290&group_id=5470 Category: Build Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Dimitri Papadopoulos (papadopo) Assigned to: Eric S. Raymond (esr) Summary: [Solaris] errors in _curses extension Initial Comment: Hi, Building with the Sun compiler on Solaris 7: $ make all [...] building '_curses' extension cc -I. -I/tmp/Python-2.1/./Include -I/usr/local/include -IInclude/ -c /tmp/Python-2.1/Modules/_cursesmodule.c -o build/temp.solaris-2.7-sun4u-2.1/_cursesmodule.o -O -I. -I./Include -DHAVE_CONFIG_H "/tmp/Python-2.1/Modules/_cursesmodule.c", line 313: undefined symbol: attr_t "/tmp/Python-2.1/Modules/_cursesmodule.c", line 313: syntax error before or at: arg1 "/tmp/Python-2.1/Modules/_cursesmodule.c", line 313: undefined symbol: arg1 "/tmp/Python-2.1/Modules/_cursesmodule.c", line 314: undefined symbol: attr_t "/tmp/Python-2.1/Modules/_cursesmodule.c", line 314: syntax error before or at: arg1 [...] $ Solaris 7 has only not and attr_t seems to be specific to . Regards, Dimitri ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 02:03 Message: Logged In: YES user_id=21627 Thanks for the report, this is fixed in _cursesmodule.c 2.56. ---------------------------------------------------------------------- Comment By: Dimitri Papadopoulos (papadopo) Date: 2001-04-19 01:59 Message: Logged In: YES user_id=52414 This is with Python 2.1 by the way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417290&group_id=5470 From noreply@sourceforge.net Sat Oct 13 10:20:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 02:20:29 -0700 Subject: [Python-bugs-list] [ python-Bugs-436130 ] solaris2.6 problems with readline Message-ID: Bugs item #436130, was opened at 2001-06-25 10:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=436130&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Fredrik Stenberg (fredriks) Assigned to: Nobody/Anonymous (nobody) Summary: solaris2.6 problems with readline Initial Comment: having problem with compiling python2.0.1 2.0 (i think i always had this problem after 1.5.2) on solaris 2.6 gcc -g -O2 -Wall -Wstrict-prototypes -fPIC -I./../Include -I.. -DHAVE_CONFIG_H -c ./readline.c ./readline.c: In function `setup_readline': ./readline.c:414: `CPPFunction' undeclared (first use in this function) ./readline.c:414: (Each undeclared identifier is reported only once ./readline.c:414: for each function it appears in.) ./readline.c:414: parse error before `)' *** Error code 1 I have always used to exchange Modules/readline.c with the old file from the 1.5.2 release. I finally got around to checking whats wrong, (or atleast browse around the code). readline.c Line 414 in void setup_readline states, rl_attempted_completion_function = (CPPFunction *)flex_complete; should this not be; rl_attempted_completion_function = (Function *)flex_complete; I have no problems if i change CPPfunction into Function, i'm no readline expert but i think this is the problem. *sysinfo* gcc 2.95.2 solaris 2.6 readline4.1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 02:20 Message: Logged In: YES user_id=21627 Closing it as fixed, then. ---------------------------------------------------------------------- Comment By: Fredrik Stenberg (fredriks) Date: 2001-07-05 06:34 Message: Logged In: YES user_id=5299 Had no problems after using the latest version of readline.c from the CVS... So the problem is fixed (in the future)....(not in the current dist) /fredriks ---------------------------------------------------------------------- Comment By: Fredrik Stenberg (fredriks) Date: 2001-06-27 04:10 Message: Logged In: YES user_id=5299 I tried it on solaris2.8 later that night, I installed readline 4.2 also (no problems with the installetion, all testprograms worked fine) But python Module/readline.c refused to compile once again. I could (as I always have) copy the old readline.c from 1.5.2 and get it to work..... Is it only me? /fredriks gcc -I/tmp/su96-fst/include -g -O2 -Wall -Wstrict-prototypes -fPIC -I./../Include -I.. -DHAVE_CONFIG_H -c ./readline.c In file included from /tmp/su96-fst/include/readline/keymaps.h:37, from /tmp/su96-fst/include/readline/readline.h:36, from ./readline.c:28: /tmp/su96-fst/include/readline/rltypedefs.h:35: warning: function declaration isn't a prototype /tmp/su96-fst/include/readline/rltypedefs.h:36: warning: function declaration isn't a prototype /tmp/su96-fst/include/readline/rltypedefs.h:37: warning: function declaration isn't a prototype /tmp/su96-fst/include/readline/rltypedefs.h:38: warning: function declaration isn't a prototype In file included from ./readline.c:28: /tmp/su96-fst/include/readline/readline.h:350: warning: function declaration isn't a prototype ./readline.c:31: conflicting types for `rl_read_init_file' /tmp/su96-fst/include/readline/readline.h:303: previous declaration of `rl_read_init_file' ./readline.c:32: conflicting types for `rl_insert_text' /tmp/su96-fst/include/readline/readline.h:363: previous declaration of `rl_insert_text' ./readline.c: In function `set_completer_delims': ./readline.c:227: warning: passing arg 1 of `free' discards qualifiers from pointer target type ./readline.c: In function `flex_complete': ./readline.c:399: warning: implicit declaration of function `completion_matches' ./readline.c:399: warning: return makes pointer from integer without a cast *** Error code 1 make: Fatal error: Command failed for target `readline.o' Current working directory /tmp/su96-fst/Python-2.0.1/Modules *** Error code 1 make: Fatal error: Command failed for target `Modules' ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-06-26 15:07 Message: Logged In: YES user_id=21627 CPPFunction is defined in readline 4.2, so one solution would be to update to 4.2. The real type of this variable is rl_completion_func_t. So if this typedef is already available in 4.1, we should probably change the cast to rl_completion_func_t. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=436130&group_id=5470 From noreply@sourceforge.net Sat Oct 13 10:35:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 02:35:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-426892 ] grid_location class at the wrong place Message-ID: Bugs item #426892, was opened at 2001-05-24 01:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=426892&group_id=5470 Category: Tkinter Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Martin v. Löwis (loewis) Summary: grid_location class at the wrong place Initial Comment: grid_location is a method for the master, and thus it should be in Misc instead of Grid. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 02:35 Message: Logged In: YES user_id=21627 Thanks for the report, fixed in Tkinter.py 1.160. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=426892&group_id=5470 From noreply@sourceforge.net Sat Oct 13 11:24:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 03:24:05 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 13 19:52:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 11:52:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-470781 ] Build broken in Mac OS X Message-ID: Bugs item #470781, was opened at 2001-10-12 22:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 Category: Macintosh Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Martin v. Löwis (loewis) Summary: Build broken in Mac OS X Initial Comment: ./configure --with-suffix=.exe make generates a warning for every file: cc: unrecognized option `-Kpthread' and then terminates with an error: cc -Kpthread -flat_namespace -u __dummy -u _PyMac_Error -framework System -framework Carbon -framework Foundation -o python.exe \ Modules/python.o \ libpython2.2.a cc: unrecognized option `-Kpthread' /usr/bin/ld: Undefined symbols: _pthread_sigmask make: *** [python.exe] Error 1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 From noreply@sourceforge.net Sat Oct 13 20:07:30 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 12:07:30 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 13 21:20:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 13:20:14 -0700 Subject: [Python-bugs-list] [ python-Bugs-470781 ] Build broken in Mac OS X Message-ID: Bugs item #470781, was opened at 2001-10-12 22:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 Category: Macintosh Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Build broken in Mac OS X Initial Comment: ./configure --with-suffix=.exe make generates a warning for every file: cc: unrecognized option `-Kpthread' and then terminates with an error: cc -Kpthread -flat_namespace -u __dummy -u _PyMac_Error -framework System -framework Carbon -framework Foundation -o python.exe \ Modules/python.o \ libpython2.2.a cc: unrecognized option `-Kpthread' /usr/bin/ld: Undefined symbols: _pthread_sigmask make: *** [python.exe] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 13:20 Message: Logged In: YES user_id=21627 What Python version? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 From noreply@sourceforge.net Sat Oct 13 22:04:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 14:04:44 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 14:04 Message: Logged In: YES user_id=21627 7-bit vs. 8-bit: You were confusing tag encoding and INTEGER value encoding. no way to encode a string with a length: suppose you want a 32 bit length, what's wrong with struct.pack("l",len(s))+s 15-bit representation: I believe the add and sub implementations make use of the guarantee that a short won't overflow if the input fits into 15 bits. Tim: BER = Basic Encoding Rules (as in the subtitle of X.690) Even after all this discussion, I still cannot see why the existing libraries (including those offered for free by third parties) are not sufficient. It appears that Paul wants, among other things, that marshal becomes documented; it also appears that this won't hapen. What the other things are that Paul wants, I cannot tell, so I recommend to close this report with "won't fix". Paul, if you have a specific change that you want to be made, or a specific problem that you want to point out, please submit a new report. This issue "provide a documented serialization func" really ought to be closed as "Fixed"; xmlrpclib is already part of standard library and fits the original problem description: - it is a library for serializing Python basic objects - it is documented in the sense that the protocol is specified precisely enough to write interoperating implementations in other languages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sat Oct 13 23:25:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 15:25:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-13 15:25 Message: Logged In: YES user_id=31435 Martin, you're right about add and sub, but it's a shallow assumption easy to relax (basically just declare carry/borrow as twodigits instead of digit). I'd be more worried about the stwodigits type, but since nothing is actually broken here I'm not keen to fritter away time proving bounds on the temps in bigint division. How we implement bigints internally is off topic anyway (provided we're not trying to hijack internal implementation formats for unintended purposes). About BER, yes, and the URL I included is to a freely downloadable copy of the X.690 std; section 8.3 spells out the INTEGER rules. They aren't at all the rules Paul sketched, hence "I don't really know what you [Paul] mean by 'BER'". For the rest, while xmlrcplib may meet the letter of what Paul asked for at first, it's clear to me that it doesn't meet what he really wants. My suggestion remains to add a new, efficient bigint format to pickle, which would meet everything except Paul's desire to have a special gimmick limited to his specific application and without having to write one himself. The internal API functions _PyLong_AsByteArray and _PyLong_FromByteArray already do the heavy lifting in both directions (to or from base 256, unsigned or complemented, big- or little-endian). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 14:04 Message: Logged In: YES user_id=21627 7-bit vs. 8-bit: You were confusing tag encoding and INTEGER value encoding. no way to encode a string with a length: suppose you want a 32 bit length, what's wrong with struct.pack("l",len(s))+s 15-bit representation: I believe the add and sub implementations make use of the guarantee that a short won't overflow if the input fits into 15 bits. Tim: BER = Basic Encoding Rules (as in the subtitle of X.690) Even after all this discussion, I still cannot see why the existing libraries (including those offered for free by third parties) are not sufficient. It appears that Paul wants, among other things, that marshal becomes documented; it also appears that this won't hapen. What the other things are that Paul wants, I cannot tell, so I recommend to close this report with "won't fix". Paul, if you have a specific change that you want to be made, or a specific problem that you want to point out, please submit a new report. This issue "provide a documented serialization func" really ought to be closed as "Fixed"; xmlrpclib is already part of standard library and fits the original problem description: - it is a library for serializing Python basic objects - it is documented in the sense that the protocol is specified precisely enough to write interoperating implementations in other languages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sun Oct 14 00:29:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 16:29:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-13 16:29 Message: Logged In: YES user_id=72053 A pickle subset ("gherkin"?) could possibly also fill this need, if it was documented, even though pickle format is considerably more complicated than marshal format (it uses marshal.dumps for binary output, actually taking apart the marshalled strings). It was obvious in seconds how marshal.c works but after 30 minutes of looking at pickle.py I'm still not sure I understand it. It looks like the unpickler can construct arbitrary class instances and import arbitrary modules, which makes a security hole if the pickled strings are potentially hostile, but I might not be reading it right. Also, the unpickler must implement constant folding (the memo scheme), which complicates it somewhat, though it's not that bad. The idea of leaving the marshal formats of some Python- specific objects undocumented isn't to get out of documenting stuff, but to leave those formats open to later change. Re BER/DER, Burt Kaliski's "Layman's Guide" is pretty readable (http://borg.isc.ucsb.edu/aka/Auth/ASN1layman.htm). You're right about using all 8 bits in BER integers--it looks like the 7 bit representation is only used for OID components (I didn't realize that til checking on it just now). BER might be ok for what I'm doing--I'm not sure right now since I don't understand ASN1 that well. It looks not in the spirit of marshal/pickle though: to encode a compound object it looks like you need an ASN1 spec of EXACTLY what you expect to find in the object. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 15:25 Message: Logged In: YES user_id=31435 Martin, you're right about add and sub, but it's a shallow assumption easy to relax (basically just declare carry/borrow as twodigits instead of digit). I'd be more worried about the stwodigits type, but since nothing is actually broken here I'm not keen to fritter away time proving bounds on the temps in bigint division. How we implement bigints internally is off topic anyway (provided we're not trying to hijack internal implementation formats for unintended purposes). About BER, yes, and the URL I included is to a freely downloadable copy of the X.690 std; section 8.3 spells out the INTEGER rules. They aren't at all the rules Paul sketched, hence "I don't really know what you [Paul] mean by 'BER'". For the rest, while xmlrcplib may meet the letter of what Paul asked for at first, it's clear to me that it doesn't meet what he really wants. My suggestion remains to add a new, efficient bigint format to pickle, which would meet everything except Paul's desire to have a special gimmick limited to his specific application and without having to write one himself. The internal API functions _PyLong_AsByteArray and _PyLong_FromByteArray already do the heavy lifting in both directions (to or from base 256, unsigned or complemented, big- or little-endian). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 14:04 Message: Logged In: YES user_id=21627 7-bit vs. 8-bit: You were confusing tag encoding and INTEGER value encoding. no way to encode a string with a length: suppose you want a 32 bit length, what's wrong with struct.pack("l",len(s))+s 15-bit representation: I believe the add and sub implementations make use of the guarantee that a short won't overflow if the input fits into 15 bits. Tim: BER = Basic Encoding Rules (as in the subtitle of X.690) Even after all this discussion, I still cannot see why the existing libraries (including those offered for free by third parties) are not sufficient. It appears that Paul wants, among other things, that marshal becomes documented; it also appears that this won't hapen. What the other things are that Paul wants, I cannot tell, so I recommend to close this report with "won't fix". Paul, if you have a specific change that you want to be made, or a specific problem that you want to point out, please submit a new report. This issue "provide a documented serialization func" really ought to be closed as "Fixed"; xmlrpclib is already part of standard library and fits the original problem description: - it is a library for serializing Python basic objects - it is documented in the sense that the protocol is specified precisely enough to write interoperating implementations in other languages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sun Oct 14 01:49:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 13 Oct 2001 17:49:56 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-13 17:49 Message: Logged In: YES user_id=72053 I agree with Tim that the internal implementation of long arithmetic isn't relevant to this--it was just surprising, and means the current marshal format isn't all that natural for external use. I don't have a particular agenda to get marshal documented, beyond that it would happen to solve my immediate problem. Alternatives are fine too. The ones suggested so far just don't seem to do the job, viz.: xmlrpc does NOT serialize basic Python objects--in particular it doesn't serialize integers longer than 32 bits. I can't consider using pickle until I've convinced myself that it doesn't make security holes, and so far it looks like the opposite. (Can someone tell me I'm not reading it right?). Yes, of course, it's not that difficult to write Python code to do everything I want. It's just surprising that I should need to do that. I mean, imagine if there was no integer addition function (no "+" operator) and the maintainers said "that's ok, to add a and b, just use 'a - (-b)'". It's not a showstopping obstacle, but I'm surprised to get so much grief for suggesting making the operation more convenient, since it's an obvious thing to want to do (as evidenced by there already being so many overlapping serialization functions: marshal, pickle, rpclib, the Serialization class from Vaults of Parnassus, three different ASN1 implementations you mentioned, etc). I can't see anywhere where I've requested a "special gimmick". Yes, an efficient bigint representation in pickle is nice and ought to be added, but I can live without it. I can NOT live with security holes, but wanting security shouldn't be considered a special gimmick! With binary bigints, a documented format, and a way to 100% stop the unpickler from ever calling eval or apply on untrusted data, I wouldn't mind using pickle despite its additional complexity compared to marshal. I don't want to depend on third party modules unless I bundle them with my application (again not a showstopper, but it's not in Python's "batteries included" spirit to need them at all). Telling a user "to run this app, first download modules vreeble from and frob from " where url1 and url2 usually turn out to be broken links by the time the user sees them is not the right way to distribute an app. (It happens I'm going to sometimes tell the user "to run this app, first destroy your handheld computer's OS by reflashing the firmware..." but it's the principle of the thing, you know). Anyway, the 15 bit bigint representation is reason enough to not want to freeze the current marshal format. Maybe a future marshaller can use a cleaner bigint format and at that point perhaps the issue can be revisited. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 16:29 Message: Logged In: YES user_id=72053 A pickle subset ("gherkin"?) could possibly also fill this need, if it was documented, even though pickle format is considerably more complicated than marshal format (it uses marshal.dumps for binary output, actually taking apart the marshalled strings). It was obvious in seconds how marshal.c works but after 30 minutes of looking at pickle.py I'm still not sure I understand it. It looks like the unpickler can construct arbitrary class instances and import arbitrary modules, which makes a security hole if the pickled strings are potentially hostile, but I might not be reading it right. Also, the unpickler must implement constant folding (the memo scheme), which complicates it somewhat, though it's not that bad. The idea of leaving the marshal formats of some Python- specific objects undocumented isn't to get out of documenting stuff, but to leave those formats open to later change. Re BER/DER, Burt Kaliski's "Layman's Guide" is pretty readable (http://borg.isc.ucsb.edu/aka/Auth/ASN1layman.htm). You're right about using all 8 bits in BER integers--it looks like the 7 bit representation is only used for OID components (I didn't realize that til checking on it just now). BER might be ok for what I'm doing--I'm not sure right now since I don't understand ASN1 that well. It looks not in the spirit of marshal/pickle though: to encode a compound object it looks like you need an ASN1 spec of EXACTLY what you expect to find in the object. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 15:25 Message: Logged In: YES user_id=31435 Martin, you're right about add and sub, but it's a shallow assumption easy to relax (basically just declare carry/borrow as twodigits instead of digit). I'd be more worried about the stwodigits type, but since nothing is actually broken here I'm not keen to fritter away time proving bounds on the temps in bigint division. How we implement bigints internally is off topic anyway (provided we're not trying to hijack internal implementation formats for unintended purposes). About BER, yes, and the URL I included is to a freely downloadable copy of the X.690 std; section 8.3 spells out the INTEGER rules. They aren't at all the rules Paul sketched, hence "I don't really know what you [Paul] mean by 'BER'". For the rest, while xmlrcplib may meet the letter of what Paul asked for at first, it's clear to me that it doesn't meet what he really wants. My suggestion remains to add a new, efficient bigint format to pickle, which would meet everything except Paul's desire to have a special gimmick limited to his specific application and without having to write one himself. The internal API functions _PyLong_AsByteArray and _PyLong_FromByteArray already do the heavy lifting in both directions (to or from base 256, unsigned or complemented, big- or little-endian). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 14:04 Message: Logged In: YES user_id=21627 7-bit vs. 8-bit: You were confusing tag encoding and INTEGER value encoding. no way to encode a string with a length: suppose you want a 32 bit length, what's wrong with struct.pack("l",len(s))+s 15-bit representation: I believe the add and sub implementations make use of the guarantee that a short won't overflow if the input fits into 15 bits. Tim: BER = Basic Encoding Rules (as in the subtitle of X.690) Even after all this discussion, I still cannot see why the existing libraries (including those offered for free by third parties) are not sufficient. It appears that Paul wants, among other things, that marshal becomes documented; it also appears that this won't hapen. What the other things are that Paul wants, I cannot tell, so I recommend to close this report with "won't fix". Paul, if you have a specific change that you want to be made, or a specific problem that you want to point out, please submit a new report. This issue "provide a documented serialization func" really ought to be closed as "Fixed"; xmlrpclib is already part of standard library and fits the original problem description: - it is a library for serializing Python basic objects - it is documented in the sense that the protocol is specified precisely enough to write interoperating implementations in other languages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sun Oct 14 21:36:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 14 Oct 2001 13:36:51 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-14 13:36 Message: Logged In: YES user_id=31435 Ack -- Paul, you add a new hitherto secret requirement with each reply. marshal isn't secure at all: because its *purpose* is to load .pyc files, marshal creates Python code objects out of any bytes you happen to feed it following a "code object" tag. That's a hole big enough to swallow the solar system. In 2.2, marshal refuses to unpack code objects in restricted execution mode, but not before 2.2, and it never refuses in unrestricted mode. In contrast, pickle doesn't know anything about code objects, so doesn't have this hole. The pickle docs are clear about this, too, spelling out that marshal's code- object abilities create "the possibility of smuggling Trojan horses into a program". When wondering about security, you should be looking at (and using) cPickle.c instead of pickle.py; cPickle doesn't use marshal at all, nor does it do eval()s etc. Yes, it can reconstruct pickled instances of classes that already exist, but it cannot create new classes. I haven't heard that characterized as an insecurity before, but to each his own level of discomfort. I want to go back to the start: if the question is whether Python is interested in documenting another data transmission format, my answer is no. There are many already (don't forget the ones from the CORBA and ILU worlds either) available from Python, and there's no reason to believe encoders/decoders for a Python-specific format would get implemented in any other language. pickle is Python's generic answer to the Python-specific serialization question. I'd be happy to see patches to improve it (whether for efficient longs, or some stricter notion of security, or even just docs). But I expect any additional Python-specific serialization scheme has an audience of one (if you disagree, fine, write a PEP and get some community consensus). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 17:49 Message: Logged In: YES user_id=72053 I agree with Tim that the internal implementation of long arithmetic isn't relevant to this--it was just surprising, and means the current marshal format isn't all that natural for external use. I don't have a particular agenda to get marshal documented, beyond that it would happen to solve my immediate problem. Alternatives are fine too. The ones suggested so far just don't seem to do the job, viz.: xmlrpc does NOT serialize basic Python objects--in particular it doesn't serialize integers longer than 32 bits. I can't consider using pickle until I've convinced myself that it doesn't make security holes, and so far it looks like the opposite. (Can someone tell me I'm not reading it right?). Yes, of course, it's not that difficult to write Python code to do everything I want. It's just surprising that I should need to do that. I mean, imagine if there was no integer addition function (no "+" operator) and the maintainers said "that's ok, to add a and b, just use 'a - (-b)'". It's not a showstopping obstacle, but I'm surprised to get so much grief for suggesting making the operation more convenient, since it's an obvious thing to want to do (as evidenced by there already being so many overlapping serialization functions: marshal, pickle, rpclib, the Serialization class from Vaults of Parnassus, three different ASN1 implementations you mentioned, etc). I can't see anywhere where I've requested a "special gimmick". Yes, an efficient bigint representation in pickle is nice and ought to be added, but I can live without it. I can NOT live with security holes, but wanting security shouldn't be considered a special gimmick! With binary bigints, a documented format, and a way to 100% stop the unpickler from ever calling eval or apply on untrusted data, I wouldn't mind using pickle despite its additional complexity compared to marshal. I don't want to depend on third party modules unless I bundle them with my application (again not a showstopper, but it's not in Python's "batteries included" spirit to need them at all). Telling a user "to run this app, first download modules vreeble from and frob from " where url1 and url2 usually turn out to be broken links by the time the user sees them is not the right way to distribute an app. (It happens I'm going to sometimes tell the user "to run this app, first destroy your handheld computer's OS by reflashing the firmware..." but it's the principle of the thing, you know). Anyway, the 15 bit bigint representation is reason enough to not want to freeze the current marshal format. Maybe a future marshaller can use a cleaner bigint format and at that point perhaps the issue can be revisited. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 16:29 Message: Logged In: YES user_id=72053 A pickle subset ("gherkin"?) could possibly also fill this need, if it was documented, even though pickle format is considerably more complicated than marshal format (it uses marshal.dumps for binary output, actually taking apart the marshalled strings). It was obvious in seconds how marshal.c works but after 30 minutes of looking at pickle.py I'm still not sure I understand it. It looks like the unpickler can construct arbitrary class instances and import arbitrary modules, which makes a security hole if the pickled strings are potentially hostile, but I might not be reading it right. Also, the unpickler must implement constant folding (the memo scheme), which complicates it somewhat, though it's not that bad. The idea of leaving the marshal formats of some Python- specific objects undocumented isn't to get out of documenting stuff, but to leave those formats open to later change. Re BER/DER, Burt Kaliski's "Layman's Guide" is pretty readable (http://borg.isc.ucsb.edu/aka/Auth/ASN1layman.htm). You're right about using all 8 bits in BER integers--it looks like the 7 bit representation is only used for OID components (I didn't realize that til checking on it just now). BER might be ok for what I'm doing--I'm not sure right now since I don't understand ASN1 that well. It looks not in the spirit of marshal/pickle though: to encode a compound object it looks like you need an ASN1 spec of EXACTLY what you expect to find in the object. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 15:25 Message: Logged In: YES user_id=31435 Martin, you're right about add and sub, but it's a shallow assumption easy to relax (basically just declare carry/borrow as twodigits instead of digit). I'd be more worried about the stwodigits type, but since nothing is actually broken here I'm not keen to fritter away time proving bounds on the temps in bigint division. How we implement bigints internally is off topic anyway (provided we're not trying to hijack internal implementation formats for unintended purposes). About BER, yes, and the URL I included is to a freely downloadable copy of the X.690 std; section 8.3 spells out the INTEGER rules. They aren't at all the rules Paul sketched, hence "I don't really know what you [Paul] mean by 'BER'". For the rest, while xmlrcplib may meet the letter of what Paul asked for at first, it's clear to me that it doesn't meet what he really wants. My suggestion remains to add a new, efficient bigint format to pickle, which would meet everything except Paul's desire to have a special gimmick limited to his specific application and without having to write one himself. The internal API functions _PyLong_AsByteArray and _PyLong_FromByteArray already do the heavy lifting in both directions (to or from base 256, unsigned or complemented, big- or little-endian). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 14:04 Message: Logged In: YES user_id=21627 7-bit vs. 8-bit: You were confusing tag encoding and INTEGER value encoding. no way to encode a string with a length: suppose you want a 32 bit length, what's wrong with struct.pack("l",len(s))+s 15-bit representation: I believe the add and sub implementations make use of the guarantee that a short won't overflow if the input fits into 15 bits. Tim: BER = Basic Encoding Rules (as in the subtitle of X.690) Even after all this discussion, I still cannot see why the existing libraries (including those offered for free by third parties) are not sufficient. It appears that Paul wants, among other things, that marshal becomes documented; it also appears that this won't hapen. What the other things are that Paul wants, I cannot tell, so I recommend to close this report with "won't fix". Paul, if you have a specific change that you want to be made, or a specific problem that you want to point out, please submit a new report. This issue "provide a documented serialization func" really ought to be closed as "Fixed"; xmlrpclib is already part of standard library and fits the original problem description: - it is a library for serializing Python basic objects - it is documented in the sense that the protocol is specified precisely enough to write interoperating implementations in other languages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Sun Oct 14 21:59:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 14 Oct 2001 13:59:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-471111 ] online docs out of sync? Message-ID: Bugs item #471111, was opened at 2001-10-14 13:59 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: online docs out of sync? Initial Comment: http://web.lfw.org/python/inspect.html doesn't include the getmodulename() function and http://python.sourceforge.net/devel-docs/lib/module-inspect.html doesn't include the getframeinfo() function, but: MAIL imp:~$ python Python 2.2a4+ (#2, Oct 11 2001, 10:21:08) [GCC 2.95.4 20011006 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import inspect >>> dir(inspect) ['BlockFinder', 'CO_NEWLOCALS', 'CO_OPTIMIZED', 'CO_VARARGS', 'CO_VARKEYWORDS', 'EndOfBlock', 'ListReader', '__author__', '__builtins__', '__date__', '__doc__', '__file__', '__name__', '_searchbases', 'classify_class_attrs', 'currentframe', 'dis', 'findsource', 'formatargspec', 'formatargvalues', 'getabsfile', 'getargs', 'getargspec', 'getargvalues', 'getblock', 'getclasstree', 'getcomments', 'getdoc', 'getfile', 'getframeinfo', 'getinnerframes', 'getlineno', 'getmembers', 'getmodule', 'getmoduleinfo', 'getmodulename', 'getmro', 'getouterframes', 'getsource', 'getsourcefile', 'getsourcelines', 'imp', 'indentsize', 'isbuiltin', 'isclass', 'iscode', 'isframe', 'isfunction', 'ismethod', 'ismethoddescriptor', 'ismodule', 'isroutine', 'istraceback', 'joinseq', 'modulesbyfile', 'os', 're', 'stack', 'string', 'strseq', 'sys', 'tokenize', 'trace', 'types', 'walktree'] >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 From noreply@sourceforge.net Sun Oct 14 23:04:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 14 Oct 2001 15:04:44 -0700 Subject: [Python-bugs-list] [ python-Bugs-471111 ] online docs out of sync? Message-ID: Bugs item #471111, was opened at 2001-10-14 13:59 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: online docs out of sync? Initial Comment: http://web.lfw.org/python/inspect.html doesn't include the getmodulename() function and http://python.sourceforge.net/devel-docs/lib/module-inspect.html doesn't include the getframeinfo() function, but: MAIL imp:~$ python Python 2.2a4+ (#2, Oct 11 2001, 10:21:08) [GCC 2.95.4 20011006 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import inspect >>> dir(inspect) ['BlockFinder', 'CO_NEWLOCALS', 'CO_OPTIMIZED', 'CO_VARARGS', 'CO_VARKEYWORDS', 'EndOfBlock', 'ListReader', '__author__', '__builtins__', '__date__', '__doc__', '__file__', '__name__', '_searchbases', 'classify_class_attrs', 'currentframe', 'dis', 'findsource', 'formatargspec', 'formatargvalues', 'getabsfile', 'getargs', 'getargspec', 'getargvalues', 'getblock', 'getclasstree', 'getcomments', 'getdoc', 'getfile', 'getframeinfo', 'getinnerframes', 'getlineno', 'getmembers', 'getmodule', 'getmoduleinfo', 'getmodulename', 'getmro', 'getouterframes', 'getsource', 'getsourcefile', 'getsourcelines', 'imp', 'indentsize', 'isbuiltin', 'isclass', 'iscode', 'isframe', 'isfunction', 'ismethod', 'ismethoddescriptor', 'ismodule', 'isroutine', 'istraceback', 'joinseq', 'modulesbyfile', 'os', 're', 'stack', 'string', 'strseq', 'sys', 'tokenize', 'trace', 'types', 'walktree'] >>> ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-14 15:04 Message: Logged In: YES user_id=21627 Please note that http://web.lfw.org/python/inspect.html is maintained independently, so you should complain to the maintainer of this page (Ka-Ping Yee) about inconsistencies. The official Python documentation for the current release is http://www.python.org/doc/current/lib/module-inspect.html and it does mention getmodulename. So it appears that the core of your report is that getframeinfo is not documented. Please understand that two sets of Python documentation are maintained: one is the doc strings, which are the foundation of Ping's pages (via pydoc). The other is the Latex documentation, which is converted automatically into the "online" (ie. HTML documentation), where inspect is documented in Doc/lib/libinspect.tex. If you find that the Tex/HTML documentation misses some pieces that are worth being mentioned, and if you have convenient access to CVS, please also consider providing patches for the documentation. In this specific case, it might be sufficient to copy the docstring into the tex documentation; having such patches available makes it easier to integrate them into the documentation. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 From noreply@sourceforge.net Sun Oct 14 23:27:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 14 Oct 2001 15:27:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-431886 ] listcomp syntax too confusing (tuples) Message-ID: Bugs item #431886, was opened at 2001-06-08 23:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=431886&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None >Priority: 7 Submitted By: Tim Peters (tim_one) >Assigned to: Guido van Rossum (gvanrossum) Summary: listcomp syntax too confusing (tuples) Initial Comment: We were careful to make sure that tuple targets in listcomps required parens, i.e. [x, x+1 for x in s] # rejected [(x, x+1) for x in s] # OK but we didn't anticipate other "surprise tuple" cases. Most recently from c.l.py, """ I tried the one-line command in a interaction mode: [x for x in [1, 2, 3], y for y in [4, 5, 6]] and the result surprised me, that is: [[1,2,3],[1,2,3],[1,2,3],9,9,9] Who can explain the behavior? Since I expected the result should be: [[1,4],[1,5],[1,6],[2,4],...] """ This is too surprising; we should require that the listcomp be spelled [x for x in ([1, 2, 3], y) for y in [4, 5, 6]] instead if that's really what they want (which it almost certainly isn't!). ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-14 15:27 Message: Logged In: YES user_id=31435 Guido, this is easy to fix: diff -u -r1.44 Grammar --- Grammar 2001/08/08 05:00:17 1.44 +++ Grammar 2001/10/14 21:57:12 @@ -97,5 +97,5 @@ argument: [test '='] test # Really [keyword '='] test list_iter: list_for | list_if -list_for: 'for' exprlist 'in' testlist [list_iter] +list_for: 'for' exprlist 'in' test [list_iter] list_if: 'if' test [list_iter] test_parser fails then, and parsermodule.c also needs a patch. A combined context diff is attached. I'm assigning this to you because it's not wholly backward compatible; e.g., [i**2 for i in 1, 2] is no longer accepted -- but then that's the point of the exercise. I don't know of any real code that breaks. Risk it? Forget it? PEP? YAFS (yet another future stmt)? Bumped the priority, because if we *are* going to change it, we should do so for 2.2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=431886&group_id=5470 From noreply@sourceforge.net Sun Oct 14 23:27:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 14 Oct 2001 15:27:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-14 15:27 Message: Logged In: YES user_id=72053 My understanding of marshal (I better check it, but I did mention the issue in the original request) is that it can create code objects but it doesn't actually execute the code in them. My implementation currently uses marshal but checks that the stuff marshal returns doesn't contain anything unexpected. Unpickle is different--it looks like it can execute hostile code before the loads call ever returns. By the time you have a chance to check the result, it's too late. cPickle.c appears to work exactly the same way (using eval and creating arbitrary instances, but maybe not calling marshal) as pickle.py. It never would have occured to me that the unpickler would work that way (and I'm still not convinced I understand it--I better try putting together a test to see if it's really like that). That's why I didn't notice the security issue til we started discussing pickle and I actually looked at the code. I'm sorry if that sounds like I'm adding requirements. I'd have thought it would go without saying that an important utility shouldn't have security holes. I'm ok with using pickle if the doc and security concerns are taken care of. More efficient longs would be helpful but they would break interoperability with old versions and I can probably live without them. It's really sad that longs were shrugged off when the pickle binary format was designed. Now in order to have efficient longs, yet another flag will have to be added to the constructor. Btw, if the unpickle security issue is real (I'm still not convinced!), I feel it should be treated as a major bug and that an announcement should be sent out. Unpickle already anticipates hostile pickled strings in the non-binary format and checks for them (see _is_secure_string) though I'd want want to spend an hour or two checking both the `...` code and the evaluator before believing that _is_secure_string is really safe-- and even if it is, it's brittle. But it looks like object creation security is an area they didn't think about. Basically I have nothing against pickle in principle, but it has these (fixable) problems, and while marshal is straightforwardly written, both pickle implementations are excessively clever and make me queasy. Anyway, I can go along with the idea that the right solution is to fix pickle--but at present, pickle looks like it's in worse shape than marshal. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-14 13:36 Message: Logged In: YES user_id=31435 Ack -- Paul, you add a new hitherto secret requirement with each reply. marshal isn't secure at all: because its *purpose* is to load .pyc files, marshal creates Python code objects out of any bytes you happen to feed it following a "code object" tag. That's a hole big enough to swallow the solar system. In 2.2, marshal refuses to unpack code objects in restricted execution mode, but not before 2.2, and it never refuses in unrestricted mode. In contrast, pickle doesn't know anything about code objects, so doesn't have this hole. The pickle docs are clear about this, too, spelling out that marshal's code- object abilities create "the possibility of smuggling Trojan horses into a program". When wondering about security, you should be looking at (and using) cPickle.c instead of pickle.py; cPickle doesn't use marshal at all, nor does it do eval()s etc. Yes, it can reconstruct pickled instances of classes that already exist, but it cannot create new classes. I haven't heard that characterized as an insecurity before, but to each his own level of discomfort. I want to go back to the start: if the question is whether Python is interested in documenting another data transmission format, my answer is no. There are many already (don't forget the ones from the CORBA and ILU worlds either) available from Python, and there's no reason to believe encoders/decoders for a Python-specific format would get implemented in any other language. pickle is Python's generic answer to the Python-specific serialization question. I'd be happy to see patches to improve it (whether for efficient longs, or some stricter notion of security, or even just docs). But I expect any additional Python-specific serialization scheme has an audience of one (if you disagree, fine, write a PEP and get some community consensus). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 17:49 Message: Logged In: YES user_id=72053 I agree with Tim that the internal implementation of long arithmetic isn't relevant to this--it was just surprising, and means the current marshal format isn't all that natural for external use. I don't have a particular agenda to get marshal documented, beyond that it would happen to solve my immediate problem. Alternatives are fine too. The ones suggested so far just don't seem to do the job, viz.: xmlrpc does NOT serialize basic Python objects--in particular it doesn't serialize integers longer than 32 bits. I can't consider using pickle until I've convinced myself that it doesn't make security holes, and so far it looks like the opposite. (Can someone tell me I'm not reading it right?). Yes, of course, it's not that difficult to write Python code to do everything I want. It's just surprising that I should need to do that. I mean, imagine if there was no integer addition function (no "+" operator) and the maintainers said "that's ok, to add a and b, just use 'a - (-b)'". It's not a showstopping obstacle, but I'm surprised to get so much grief for suggesting making the operation more convenient, since it's an obvious thing to want to do (as evidenced by there already being so many overlapping serialization functions: marshal, pickle, rpclib, the Serialization class from Vaults of Parnassus, three different ASN1 implementations you mentioned, etc). I can't see anywhere where I've requested a "special gimmick". Yes, an efficient bigint representation in pickle is nice and ought to be added, but I can live without it. I can NOT live with security holes, but wanting security shouldn't be considered a special gimmick! With binary bigints, a documented format, and a way to 100% stop the unpickler from ever calling eval or apply on untrusted data, I wouldn't mind using pickle despite its additional complexity compared to marshal. I don't want to depend on third party modules unless I bundle them with my application (again not a showstopper, but it's not in Python's "batteries included" spirit to need them at all). Telling a user "to run this app, first download modules vreeble from and frob from " where url1 and url2 usually turn out to be broken links by the time the user sees them is not the right way to distribute an app. (It happens I'm going to sometimes tell the user "to run this app, first destroy your handheld computer's OS by reflashing the firmware..." but it's the principle of the thing, you know). Anyway, the 15 bit bigint representation is reason enough to not want to freeze the current marshal format. Maybe a future marshaller can use a cleaner bigint format and at that point perhaps the issue can be revisited. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 16:29 Message: Logged In: YES user_id=72053 A pickle subset ("gherkin"?) could possibly also fill this need, if it was documented, even though pickle format is considerably more complicated than marshal format (it uses marshal.dumps for binary output, actually taking apart the marshalled strings). It was obvious in seconds how marshal.c works but after 30 minutes of looking at pickle.py I'm still not sure I understand it. It looks like the unpickler can construct arbitrary class instances and import arbitrary modules, which makes a security hole if the pickled strings are potentially hostile, but I might not be reading it right. Also, the unpickler must implement constant folding (the memo scheme), which complicates it somewhat, though it's not that bad. The idea of leaving the marshal formats of some Python- specific objects undocumented isn't to get out of documenting stuff, but to leave those formats open to later change. Re BER/DER, Burt Kaliski's "Layman's Guide" is pretty readable (http://borg.isc.ucsb.edu/aka/Auth/ASN1layman.htm). You're right about using all 8 bits in BER integers--it looks like the 7 bit representation is only used for OID components (I didn't realize that til checking on it just now). BER might be ok for what I'm doing--I'm not sure right now since I don't understand ASN1 that well. It looks not in the spirit of marshal/pickle though: to encode a compound object it looks like you need an ASN1 spec of EXACTLY what you expect to find in the object. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 15:25 Message: Logged In: YES user_id=31435 Martin, you're right about add and sub, but it's a shallow assumption easy to relax (basically just declare carry/borrow as twodigits instead of digit). I'd be more worried about the stwodigits type, but since nothing is actually broken here I'm not keen to fritter away time proving bounds on the temps in bigint division. How we implement bigints internally is off topic anyway (provided we're not trying to hijack internal implementation formats for unintended purposes). About BER, yes, and the URL I included is to a freely downloadable copy of the X.690 std; section 8.3 spells out the INTEGER rules. They aren't at all the rules Paul sketched, hence "I don't really know what you [Paul] mean by 'BER'". For the rest, while xmlrcplib may meet the letter of what Paul asked for at first, it's clear to me that it doesn't meet what he really wants. My suggestion remains to add a new, efficient bigint format to pickle, which would meet everything except Paul's desire to have a special gimmick limited to his specific application and without having to write one himself. The internal API functions _PyLong_AsByteArray and _PyLong_FromByteArray already do the heavy lifting in both directions (to or from base 256, unsigned or complemented, big- or little-endian). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 14:04 Message: Logged In: YES user_id=21627 7-bit vs. 8-bit: You were confusing tag encoding and INTEGER value encoding. no way to encode a string with a length: suppose you want a 32 bit length, what's wrong with struct.pack("l",len(s))+s 15-bit representation: I believe the add and sub implementations make use of the guarantee that a short won't overflow if the input fits into 15 bits. Tim: BER = Basic Encoding Rules (as in the subtitle of X.690) Even after all this discussion, I still cannot see why the existing libraries (including those offered for free by third parties) are not sufficient. It appears that Paul wants, among other things, that marshal becomes documented; it also appears that this won't hapen. What the other things are that Paul wants, I cannot tell, so I recommend to close this report with "won't fix". Paul, if you have a specific change that you want to be made, or a specific problem that you want to point out, please submit a new report. This issue "provide a documented serialization func" really ought to be closed as "Fixed"; xmlrpclib is already part of standard library and fits the original problem description: - it is a library for serializing Python basic objects - it is documented in the sense that the protocol is specified precisely enough to write interoperating implementations in other languages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Mon Oct 15 05:01:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 14 Oct 2001 21:01:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-471202 ] bit shift operator << precedence Message-ID: Bugs item #471202, was opened at 2001-10-14 21:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471202&group_id=5470 Category: Parser/Compiler Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bit shift operator << precedence Initial Comment: In 2.1.1, It appears bit shift operator << has lower precedence than operator +. Expression 1<<16 + 1 results in 131072 rather than the expected 65537. User has to use (1<<16) +1 to get the expected result. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471202&group_id=5470 From noreply@sourceforge.net Mon Oct 15 08:28:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 00:28:20 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Tim Peters (tim_one) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-15 00:28 Message: Logged In: YES user_id=200117 I made the changes you suggested and most of the warnings for typeobject.c disappeared. The other line you mentioned should be changed thus: 3448c3448 < PyObject_GenericGetAttr) --- > (void *)PyObject_GenericGetAttr) This fixes all the warnings for that source file. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:43 Message: Logged In: YES user_id=6380 To rptownsend: the warnings you get seem to come from assigning function pointers to void*. Could you try the following change? 3693c3693 < {NAME, offsetof(PyTypeObject, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(PyTypeObject, SLOT), (void *)FUNCTION, WRAPPER} 3695c3695 < {NAME, offsetof(etype, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(etype, SLOT), (void *)FUNCTION, WRAPPER} There's one more line with a warning; can you figure out where to put a (void*) cast there yourself and let me know? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 06:23 Message: Logged In: YES user_id=21627 cmkl, a few questions: - It appears that STRICT_SYSV_CURSES should only be used if HAVE_NCURSES is not defined. Correct? - http://devresource.hp.com/STK/partner/xcurses.html claims that cur_colr is targeted for obsolescence. Why should we use it then? The same paper claims that _XOPEN_SOURCE_EXTENDED must be defined on 10.20 if /usr/include/curses.h (Xcurses) is used, due to a bug. - What functions does Python use that are only available with _POSIX_C_SOURCE=199506L ? - Could you please check out the current CVS and determine whether Python correctly enables large file support? What is the problem if only ftello/fseeko are used? What is the output of 'getconf LFS_CFLAGS'? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-12 04:53 Message: Logged In: NO To use curses (if GPL'd ncurses is not available) you have to make a patch and you have to compile with: -DSTRICT_SYSV_CURSES (some ncurses features are not available in this case) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** ../patch/py_curses.h Fri Dec 22 22:51:10 2000 --- py_curses.h Wed Oct 10 15:06:40 2001 *************** *** 5,11 **** --- 5,15 ---- #ifdef HAVE_NCURSES_H #include #else + #if defined(__hpux) + #include + #else #include + #endif #endif #ifdef __cplusplus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use threads on has to compile with: -D_POSIX_C_SOURCE=199506L (reentrant fct. are defined then) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use ftello and fseeko (largefile) one has to compile with: -D_LARGEFILE_SOURCE a new largefile API: ftello64 and fseeko64 (instead of ftell64 and fseek64 tested by configure) is available with: -D_LARGEFILE64_SOURCE (at the moment there is no difference to -D_LARGEFILE_SOURCE) /usr/share/doc/lg_files.txt contains the HPUX large file white paper BTW in Makefile.pre.in if have replaced all occurences of: $(INSTALL) -d -m with: mkdir -p -m - cmkl (cmkleffner@gmx.de) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Mon Oct 15 09:07:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 01:07:05 -0700 Subject: [Python-bugs-list] [ python-Bugs-470781 ] Build broken in Mac OS X Message-ID: Bugs item #470781, was opened at 2001-10-12 22:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 Category: Macintosh Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Build broken in Mac OS X Initial Comment: ./configure --with-suffix=.exe make generates a warning for every file: cc: unrecognized option `-Kpthread' and then terminates with an error: cc -Kpthread -flat_namespace -u __dummy -u _PyMac_Error -framework System -framework Carbon -framework Foundation -o python.exe \ Modules/python.o \ libpython2.2.a cc: unrecognized option `-Kpthread' /usr/bin/ld: Undefined symbols: _pthread_sigmask make: *** [python.exe] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-15 01:07 Message: Logged In: YES user_id=21627 Fixed in configure.in 1.268; configure 1.260. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 13:20 Message: Logged In: YES user_id=21627 What Python version? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 From noreply@sourceforge.net Mon Oct 15 11:56:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 03:56:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-471270 ] SystemError: unknown opcode Message-ID: Bugs item #471270, was opened at 2001-10-15 03:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471270&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Niki Spahiev (nikis) Assigned to: Nobody/Anonymous (nobody) Summary: SystemError: unknown opcode Initial Comment: Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] on win32 ... XXX lineno: 63, opcode: 0 Traceback (most recent call last): File "\niki\pyfr\wxmain.py", line 259, in OnMenuCommand self.OnTreeCommand( item, event.GetId() ) File "\niki\pyfr\wxmain.py", line 250, in OnTreeCommand self.dd.Action( self, 'tree', self.tree.GetPyData(item), event ) File "D:\NIKI\RASKROY\SRC\MAIN2.PY", line 698, in Action cmd.Execute( view, self ) File "D:\NIKI\RASKROY\SRC\task.py", line 139, in Execute model.DelBlock( b ) File "D:\NIKI\RASKROY\SRC\raskroy.py", line 64, in DelBlock b.AdviseDel() File "D:\NIKI\RASKROY\SRC\meta.py", line 63, in AdviseDel break SystemError: unknown opcode ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471270&group_id=5470 From noreply@sourceforge.net Mon Oct 15 14:20:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 06:20:05 -0700 Subject: [Python-bugs-list] [ python-Bugs-471111 ] inspect.getframeinfo() needs docs Message-ID: Bugs item #471111, was opened at 2001-10-14 13:59 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) >Assigned to: Tim Peters (tim_one) >Summary: inspect.getframeinfo() needs docs Initial Comment: http://web.lfw.org/python/inspect.html doesn't include the getmodulename() function and http://python.sourceforge.net/devel-docs/lib/module-inspect.html doesn't include the getframeinfo() function, but: MAIL imp:~$ python Python 2.2a4+ (#2, Oct 11 2001, 10:21:08) [GCC 2.95.4 20011006 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import inspect >>> dir(inspect) ['BlockFinder', 'CO_NEWLOCALS', 'CO_OPTIMIZED', 'CO_VARARGS', 'CO_VARKEYWORDS', 'EndOfBlock', 'ListReader', '__author__', '__builtins__', '__date__', '__doc__', '__file__', '__name__', '_searchbases', 'classify_class_attrs', 'currentframe', 'dis', 'findsource', 'formatargspec', 'formatargvalues', 'getabsfile', 'getargs', 'getargspec', 'getargvalues', 'getblock', 'getclasstree', 'getcomments', 'getdoc', 'getfile', 'getframeinfo', 'getinnerframes', 'getlineno', 'getmembers', 'getmodule', 'getmoduleinfo', 'getmodulename', 'getmro', 'getouterframes', 'getsource', 'getsourcefile', 'getsourcelines', 'imp', 'indentsize', 'isbuiltin', 'isclass', 'iscode', 'isframe', 'isfunction', 'ismethod', 'ismethoddescriptor', 'ismodule', 'isroutine', 'istraceback', 'joinseq', 'modulesbyfile', 'os', 're', 'stack', 'string', 'strseq', 'sys', 'tokenize', 'trace', 'types', 'walktree'] >>> ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-15 06:20 Message: Logged In: YES user_id=3066 Assigned to inspect-maintainer-of-the-day for docs. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-14 15:04 Message: Logged In: YES user_id=21627 Please note that http://web.lfw.org/python/inspect.html is maintained independently, so you should complain to the maintainer of this page (Ka-Ping Yee) about inconsistencies. The official Python documentation for the current release is http://www.python.org/doc/current/lib/module-inspect.html and it does mention getmodulename. So it appears that the core of your report is that getframeinfo is not documented. Please understand that two sets of Python documentation are maintained: one is the doc strings, which are the foundation of Ping's pages (via pydoc). The other is the Latex documentation, which is converted automatically into the "online" (ie. HTML documentation), where inspect is documented in Doc/lib/libinspect.tex. If you find that the Tex/HTML documentation misses some pieces that are worth being mentioned, and if you have convenient access to CVS, please also consider providing patches for the documentation. In this specific case, it might be sufficient to copy the docstring into the tex documentation; having such patches available makes it easier to integrate them into the documentation. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 From noreply@sourceforge.net Mon Oct 15 14:36:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 06:36:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-471202 ] bit shift operator << precedence Message-ID: Bugs item #471202, was opened at 2001-10-14 21:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471202&group_id=5470 Category: Parser/Compiler Group: Python 2.1.1 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bit shift operator << precedence Initial Comment: In 2.1.1, It appears bit shift operator << has lower precedence than operator +. Expression 1<<16 + 1 results in 131072 rather than the expected 65537. User has to use (1<<16) +1 to get the expected result. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 06:36 Message: Logged In: YES user_id=6380 Alas, Python doesn't yet parse whitespace in expressions. :-) According to the Python reference manual, << has indeed lower precedence than +, so this is not a bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471202&group_id=5470 From noreply@sourceforge.net Mon Oct 15 14:54:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 06:54:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-471270 ] SystemError: unknown opcode Message-ID: Bugs item #471270, was opened at 2001-10-15 03:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471270&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Niki Spahiev (nikis) Assigned to: Nobody/Anonymous (nobody) Summary: SystemError: unknown opcode Initial Comment: Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] on win32 ... XXX lineno: 63, opcode: 0 Traceback (most recent call last): File "\niki\pyfr\wxmain.py", line 259, in OnMenuCommand self.OnTreeCommand( item, event.GetId() ) File "\niki\pyfr\wxmain.py", line 250, in OnTreeCommand self.dd.Action( self, 'tree', self.tree.GetPyData(item), event ) File "D:\NIKI\RASKROY\SRC\MAIN2.PY", line 698, in Action cmd.Execute( view, self ) File "D:\NIKI\RASKROY\SRC\task.py", line 139, in Execute model.DelBlock( b ) File "D:\NIKI\RASKROY\SRC\raskroy.py", line 64, in DelBlock b.AdviseDel() File "D:\NIKI\RASKROY\SRC\meta.py", line 63, in AdviseDel break SystemError: unknown opcode ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 06:54 Message: Logged In: YES user_id=6380 I'm guessing this is because of a bug in the continue-within-try code generation. This is fixed in 2.2. Can you verify that this doesn't break with Python 2.2a4? (python.org/2.2/). I don't have the proper input to provoke the bug in 2.1.1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471270&group_id=5470 From noreply@sourceforge.net Mon Oct 15 15:37:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 07:37:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-470781 ] Build broken in Mac OS X Message-ID: Bugs item #470781, was opened at 2001-10-12 22:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 Category: Macintosh Group: Platform-specific Status: Closed Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Build broken in Mac OS X Initial Comment: ./configure --with-suffix=.exe make generates a warning for every file: cc: unrecognized option `-Kpthread' and then terminates with an error: cc -Kpthread -flat_namespace -u __dummy -u _PyMac_Error -framework System -framework Carbon -framework Foundation -o python.exe \ Modules/python.o \ libpython2.2.a cc: unrecognized option `-Kpthread' /usr/bin/ld: Undefined symbols: _pthread_sigmask make: *** [python.exe] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-15 07:37 Message: Logged In: YES user_id=21627 The previous patch only arranged to not use -Kpthread anymore. The pthread_sigmask problem was unrelated; this is fixed with pyconfig.h.in 1.15 configure 1.261 configure.in 1.269 thread_pthread.h 2.34 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-15 01:07 Message: Logged In: YES user_id=21627 Fixed in configure.in 1.268; configure 1.260. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 13:20 Message: Logged In: YES user_id=21627 What Python version? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=470781&group_id=5470 From noreply@sourceforge.net Mon Oct 15 16:30:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 08:30:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-431886 ] listcomp syntax too confusing (tuples) Message-ID: Bugs item #431886, was opened at 2001-06-08 23:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=431886&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: listcomp syntax too confusing (tuples) Initial Comment: We were careful to make sure that tuple targets in listcomps required parens, i.e. [x, x+1 for x in s] # rejected [(x, x+1) for x in s] # OK but we didn't anticipate other "surprise tuple" cases. Most recently from c.l.py, """ I tried the one-line command in a interaction mode: [x for x in [1, 2, 3], y for y in [4, 5, 6]] and the result surprised me, that is: [[1,2,3],[1,2,3],[1,2,3],9,9,9] Who can explain the behavior? Since I expected the result should be: [[1,4],[1,5],[1,6],[2,4],...] """ This is too surprising; we should require that the listcomp be spelled [x for x in ([1, 2, 3], y) for y in [4, 5, 6]] instead if that's really what they want (which it almost certainly isn't!). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 08:30 Message: Logged In: YES user_id=6380 I'll fix it in a way that isn't quite so drastic -- you'll be able to have a testlist there and even one that ends in a comma, except that a singleton list cannot end in a comma. Expect checkins soon. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-14 15:27 Message: Logged In: YES user_id=31435 Guido, this is easy to fix: diff -u -r1.44 Grammar --- Grammar 2001/08/08 05:00:17 1.44 +++ Grammar 2001/10/14 21:57:12 @@ -97,5 +97,5 @@ argument: [test '='] test # Really [keyword '='] test list_iter: list_for | list_if -list_for: 'for' exprlist 'in' testlist [list_iter] +list_for: 'for' exprlist 'in' test [list_iter] list_if: 'if' test [list_iter] test_parser fails then, and parsermodule.c also needs a patch. A combined context diff is attached. I'm assigning this to you because it's not wholly backward compatible; e.g., [i**2 for i in 1, 2] is no longer accepted -- but then that's the point of the exercise. I don't know of any real code that breaks. Risk it? Forget it? PEP? YAFS (yet another future stmt)? Bumped the priority, because if we *are* going to change it, we should do so for 2.2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=431886&group_id=5470 From noreply@sourceforge.net Mon Oct 15 16:44:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 08:44:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-431886 ] listcomp syntax too confusing (tuples) Message-ID: Bugs item #431886, was opened at 2001-06-08 23:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=431886&group_id=5470 Category: Parser/Compiler Group: None >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: listcomp syntax too confusing (tuples) Initial Comment: We were careful to make sure that tuple targets in listcomps required parens, i.e. [x, x+1 for x in s] # rejected [(x, x+1) for x in s] # OK but we didn't anticipate other "surprise tuple" cases. Most recently from c.l.py, """ I tried the one-line command in a interaction mode: [x for x in [1, 2, 3], y for y in [4, 5, 6]] and the result surprised me, that is: [[1,2,3],[1,2,3],[1,2,3],9,9,9] Who can explain the behavior? Since I expected the result should be: [[1,4],[1,5],[1,6],[2,4],...] """ This is too surprising; we should require that the listcomp be spelled [x for x in ([1, 2, 3], y) for y in [4, 5, 6]] instead if that's really what they want (which it almost certainly isn't!). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 08:44 Message: Logged In: YES user_id=6380 Fixed, checked in. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 08:30 Message: Logged In: YES user_id=6380 I'll fix it in a way that isn't quite so drastic -- you'll be able to have a testlist there and even one that ends in a comma, except that a singleton list cannot end in a comma. Expect checkins soon. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-14 15:27 Message: Logged In: YES user_id=31435 Guido, this is easy to fix: diff -u -r1.44 Grammar --- Grammar 2001/08/08 05:00:17 1.44 +++ Grammar 2001/10/14 21:57:12 @@ -97,5 +97,5 @@ argument: [test '='] test # Really [keyword '='] test list_iter: list_for | list_if -list_for: 'for' exprlist 'in' testlist [list_iter] +list_for: 'for' exprlist 'in' test [list_iter] list_if: 'if' test [list_iter] test_parser fails then, and parsermodule.c also needs a patch. A combined context diff is attached. I'm assigning this to you because it's not wholly backward compatible; e.g., [i**2 for i in 1, 2] is no longer accepted -- but then that's the point of the exercise. I don't know of any real code that breaks. Risk it? Forget it? PEP? YAFS (yet another future stmt)? Bumped the priority, because if we *are* going to change it, we should do so for 2.2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=431886&group_id=5470 From noreply@sourceforge.net Mon Oct 15 20:19:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 12:19:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-471432 ] __cdecl / __stdcall unspecified in *.h Message-ID: Bugs item #471432, was opened at 2001-10-15 12:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Tim Peters (tim_one) Summary: __cdecl / __stdcall unspecified in *.h Initial Comment: Currently, the win32 function call convention (__stdcall vs __cdecl) is never specified in the function prototypes contained in the .h header files. The call convention is therefore left up to the configuration of the compiler incluing the header files (__cdecl by default on VC++). However, there are situations in which the compiler default call convention must be set to __stdcall (/Gz option on VC++'s cl.exe), and this makes the link fail; until __cdecl keywords are manually insterted in the Python.h function prototypes required for linking. [For instance, /Gz is required when using typedef definitions on function pointers, later passed as arguments on __stdcall functions (the default on Microsoft/commercial libraries)]. Besides, Microsoft recommands, it the VC++ documentation, that the __stdcall convention be used by default for function with fixed numbers of arguments (smaller code size and better performances); __cdecl being limited for use with functions with variable number of args ('...'). A solution would consist in defining: #ifdef _MS_VER #define FIXARGCALL __stdcall #define VARARGCALL __cdecl #else ... #endif and sticking FIXARGCALL / VARARGCALL in front of all DLL_IMPORT(), macro calls associated to a an exported function prototype, across all Python .h files. Frederic Giacometti ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 From noreply@sourceforge.net Mon Oct 15 20:34:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 12:34:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-471432 ] __cdecl / __stdcall unspecified in *.h Message-ID: Bugs item #471432, was opened at 2001-10-15 12:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) >Assigned to: Mark Hammond (mhammond) Summary: __cdecl / __stdcall unspecified in *.h Initial Comment: Currently, the win32 function call convention (__stdcall vs __cdecl) is never specified in the function prototypes contained in the .h header files. The call convention is therefore left up to the configuration of the compiler incluing the header files (__cdecl by default on VC++). However, there are situations in which the compiler default call convention must be set to __stdcall (/Gz option on VC++'s cl.exe), and this makes the link fail; until __cdecl keywords are manually insterted in the Python.h function prototypes required for linking. [For instance, /Gz is required when using typedef definitions on function pointers, later passed as arguments on __stdcall functions (the default on Microsoft/commercial libraries)]. Besides, Microsoft recommands, it the VC++ documentation, that the __stdcall convention be used by default for function with fixed numbers of arguments (smaller code size and better performances); __cdecl being limited for use with functions with variable number of args ('...'). A solution would consist in defining: #ifdef _MS_VER #define FIXARGCALL __stdcall #define VARARGCALL __cdecl #else ... #endif and sticking FIXARGCALL / VARARGCALL in front of all DLL_IMPORT(), macro calls associated to a an exported function prototype, across all Python .h files. Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:34 Message: Logged In: YES user_id=31435 Mark, does this make sense to you? Any notion that we *have* to add non-standard decorations seems inherently fishy to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 From noreply@sourceforge.net Mon Oct 15 20:35:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 12:35:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Guido van Rossum (gvanrossum) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:35 Message: Logged In: YES user_id=31435 Reassigned to Guido to match reality. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-15 00:28 Message: Logged In: YES user_id=200117 I made the changes you suggested and most of the warnings for typeobject.c disappeared. The other line you mentioned should be changed thus: 3448c3448 < PyObject_GenericGetAttr) --- > (void *)PyObject_GenericGetAttr) This fixes all the warnings for that source file. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:43 Message: Logged In: YES user_id=6380 To rptownsend: the warnings you get seem to come from assigning function pointers to void*. Could you try the following change? 3693c3693 < {NAME, offsetof(PyTypeObject, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(PyTypeObject, SLOT), (void *)FUNCTION, WRAPPER} 3695c3695 < {NAME, offsetof(etype, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(etype, SLOT), (void *)FUNCTION, WRAPPER} There's one more line with a warning; can you figure out where to put a (void*) cast there yourself and let me know? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 06:23 Message: Logged In: YES user_id=21627 cmkl, a few questions: - It appears that STRICT_SYSV_CURSES should only be used if HAVE_NCURSES is not defined. Correct? - http://devresource.hp.com/STK/partner/xcurses.html claims that cur_colr is targeted for obsolescence. Why should we use it then? The same paper claims that _XOPEN_SOURCE_EXTENDED must be defined on 10.20 if /usr/include/curses.h (Xcurses) is used, due to a bug. - What functions does Python use that are only available with _POSIX_C_SOURCE=199506L ? - Could you please check out the current CVS and determine whether Python correctly enables large file support? What is the problem if only ftello/fseeko are used? What is the output of 'getconf LFS_CFLAGS'? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-12 04:53 Message: Logged In: NO To use curses (if GPL'd ncurses is not available) you have to make a patch and you have to compile with: -DSTRICT_SYSV_CURSES (some ncurses features are not available in this case) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** ../patch/py_curses.h Fri Dec 22 22:51:10 2000 --- py_curses.h Wed Oct 10 15:06:40 2001 *************** *** 5,11 **** --- 5,15 ---- #ifdef HAVE_NCURSES_H #include #else + #if defined(__hpux) + #include + #else #include + #endif #endif #ifdef __cplusplus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use threads on has to compile with: -D_POSIX_C_SOURCE=199506L (reentrant fct. are defined then) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use ftello and fseeko (largefile) one has to compile with: -D_LARGEFILE_SOURCE a new largefile API: ftello64 and fseeko64 (instead of ftell64 and fseek64 tested by configure) is available with: -D_LARGEFILE64_SOURCE (at the moment there is no difference to -D_LARGEFILE_SOURCE) /usr/share/doc/lg_files.txt contains the HPUX large file white paper BTW in Makefile.pre.in if have replaced all occurences of: $(INSTALL) -d -m with: mkdir -p -m - cmkl (cmkleffner@gmx.de) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Mon Oct 15 20:45:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 12:45:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 12:45 Message: Logged In: YES user_id=6380 Thanks! You might want to check current CVS to make sure I did the right thing. Is there any reason for this to remain open? Over to MvL for the answer to that question... ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:35 Message: Logged In: YES user_id=31435 Reassigned to Guido to match reality. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-15 00:28 Message: Logged In: YES user_id=200117 I made the changes you suggested and most of the warnings for typeobject.c disappeared. The other line you mentioned should be changed thus: 3448c3448 < PyObject_GenericGetAttr) --- > (void *)PyObject_GenericGetAttr) This fixes all the warnings for that source file. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:43 Message: Logged In: YES user_id=6380 To rptownsend: the warnings you get seem to come from assigning function pointers to void*. Could you try the following change? 3693c3693 < {NAME, offsetof(PyTypeObject, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(PyTypeObject, SLOT), (void *)FUNCTION, WRAPPER} 3695c3695 < {NAME, offsetof(etype, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(etype, SLOT), (void *)FUNCTION, WRAPPER} There's one more line with a warning; can you figure out where to put a (void*) cast there yourself and let me know? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 06:23 Message: Logged In: YES user_id=21627 cmkl, a few questions: - It appears that STRICT_SYSV_CURSES should only be used if HAVE_NCURSES is not defined. Correct? - http://devresource.hp.com/STK/partner/xcurses.html claims that cur_colr is targeted for obsolescence. Why should we use it then? The same paper claims that _XOPEN_SOURCE_EXTENDED must be defined on 10.20 if /usr/include/curses.h (Xcurses) is used, due to a bug. - What functions does Python use that are only available with _POSIX_C_SOURCE=199506L ? - Could you please check out the current CVS and determine whether Python correctly enables large file support? What is the problem if only ftello/fseeko are used? What is the output of 'getconf LFS_CFLAGS'? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-12 04:53 Message: Logged In: NO To use curses (if GPL'd ncurses is not available) you have to make a patch and you have to compile with: -DSTRICT_SYSV_CURSES (some ncurses features are not available in this case) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** ../patch/py_curses.h Fri Dec 22 22:51:10 2000 --- py_curses.h Wed Oct 10 15:06:40 2001 *************** *** 5,11 **** --- 5,15 ---- #ifdef HAVE_NCURSES_H #include #else + #if defined(__hpux) + #include + #else #include + #endif #endif #ifdef __cplusplus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use threads on has to compile with: -D_POSIX_C_SOURCE=199506L (reentrant fct. are defined then) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use ftello and fseeko (largefile) one has to compile with: -D_LARGEFILE_SOURCE a new largefile API: ftello64 and fseeko64 (instead of ftell64 and fseek64 tested by configure) is available with: -D_LARGEFILE64_SOURCE (at the moment there is no difference to -D_LARGEFILE_SOURCE) /usr/share/doc/lg_files.txt contains the HPUX large file white paper BTW in Makefile.pre.in if have replaced all occurences of: $(INSTALL) -d -m with: mkdir -p -m - cmkl (cmkleffner@gmx.de) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Mon Oct 15 20:46:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 12:46:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-471432 ] __cdecl / __stdcall unspecified in *.h Message-ID: Bugs item #471432, was opened at 2001-10-15 12:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Mark Hammond (mhammond) Summary: __cdecl / __stdcall unspecified in *.h Initial Comment: Currently, the win32 function call convention (__stdcall vs __cdecl) is never specified in the function prototypes contained in the .h header files. The call convention is therefore left up to the configuration of the compiler incluing the header files (__cdecl by default on VC++). However, there are situations in which the compiler default call convention must be set to __stdcall (/Gz option on VC++'s cl.exe), and this makes the link fail; until __cdecl keywords are manually insterted in the Python.h function prototypes required for linking. [For instance, /Gz is required when using typedef definitions on function pointers, later passed as arguments on __stdcall functions (the default on Microsoft/commercial libraries)]. Besides, Microsoft recommands, it the VC++ documentation, that the __stdcall convention be used by default for function with fixed numbers of arguments (smaller code size and better performances); __cdecl being limited for use with functions with variable number of args ('...'). A solution would consist in defining: #ifdef _MS_VER #define FIXARGCALL __stdcall #define VARARGCALL __cdecl #else ... #endif and sticking FIXARGCALL / VARARGCALL in front of all DLL_IMPORT(), macro calls associated to a an exported function prototype, across all Python .h files. Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Frederic Giacometti (giacometti) Date: 2001-10-15 12:46 Message: Logged In: YES user_id=93657 As far as MS Windows / dll stuff is concerned, you're already swimming in non-C-ANSI fishtank anyway (e.g. DLL_IMPORT() macros...). Just try to keep the right fishes in the tank :)) FG ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:34 Message: Logged In: YES user_id=31435 Mark, does this make sense to you? Any notion that we *have* to add non-standard decorations seems inherently fishy to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 From noreply@sourceforge.net Mon Oct 15 22:48:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 14:48:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-466175 ] cPickle core dump Message-ID: Bugs item #466175, was opened at 2001-09-28 12:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466175&group_id=5470 Category: Extension Modules Group: None >Status: Closed >Resolution: Fixed Priority: 6 Submitted By: Jeremy Hylton (jhylton) Assigned to: Jeremy Hylton (jhylton) Summary: cPickle core dump Initial Comment: A cPickler Pickler() operating in fast mode, i.e. when Pickler().fast =1, dumps core on recursive objects. a = [] a.append(a) p = cPickle.Pickler() p.fast = 1 p.dump(a) [segfault] The fast flag indicates that pickle shouldn't keep track of object it has seen before, so it gets caught in an infinite loop if it hits a recursive object. Jim worries that fixing the segfault will slow down the fast mode, which is a big performance improvement. I think it can be solved using the same trick that we use for comparing and printing recursive objects like [[...]]. The approach is to use a simple counter to track recursive calls. Only track objects previously seen after the counter exceeds some threshhold. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-15 14:48 Message: Logged In: YES user_id=31392 Fixed in rev. 2.66 of cPickle.c ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-09-28 15:07 Message: Logged In: YES user_id=31392 No customer project. It's higher than 5 because it's a core dump rather than merely incorrect behavior. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 13:04 Message: Logged In: YES user_id=6380 Go for it. I take it this relates to a customer project? Else why the priority 6? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466175&group_id=5470 From noreply@sourceforge.net Tue Oct 16 02:45:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 18:45:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-471432 ] __cdecl / __stdcall unspecified in *.h Message-ID: Bugs item #471432, was opened at 2001-10-15 12:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Mark Hammond (mhammond) Summary: __cdecl / __stdcall unspecified in *.h Initial Comment: Currently, the win32 function call convention (__stdcall vs __cdecl) is never specified in the function prototypes contained in the .h header files. The call convention is therefore left up to the configuration of the compiler incluing the header files (__cdecl by default on VC++). However, there are situations in which the compiler default call convention must be set to __stdcall (/Gz option on VC++'s cl.exe), and this makes the link fail; until __cdecl keywords are manually insterted in the Python.h function prototypes required for linking. [For instance, /Gz is required when using typedef definitions on function pointers, later passed as arguments on __stdcall functions (the default on Microsoft/commercial libraries)]. Besides, Microsoft recommands, it the VC++ documentation, that the __stdcall convention be used by default for function with fixed numbers of arguments (smaller code size and better performances); __cdecl being limited for use with functions with variable number of args ('...'). A solution would consist in defining: #ifdef _MS_VER #define FIXARGCALL __stdcall #define VARARGCALL __cdecl #else ... #endif and sticking FIXARGCALL / VARARGCALL in front of all DLL_IMPORT(), macro calls associated to a an exported function prototype, across all Python .h files. Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Mark Hammond (mhammond) Date: 2001-10-15 18:45 Message: Logged In: YES user_id=14198 This does make sense to me, and I have even been bitten by it. There are some strange libraries out there that need to be compiled with a different calling convention than the default. Trying to extend Python with such a library can be difficult. Having the calling convention specified in the prototypes would solve the problem. I have never been brave enough to do it, given the small number of times it has been reported as a problem. However, I would be happy to help come up with a reasonable scheme. ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-15 12:46 Message: Logged In: YES user_id=93657 As far as MS Windows / dll stuff is concerned, you're already swimming in non-C-ANSI fishtank anyway (e.g. DLL_IMPORT() macros...). Just try to keep the right fishes in the tank :)) FG ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:34 Message: Logged In: YES user_id=31435 Mark, does this make sense to you? Any notion that we *have* to add non-standard decorations seems inherently fishy to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 From noreply@sourceforge.net Tue Oct 16 04:00:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 20:00:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-471432 ] __cdecl / __stdcall unspecified in *.h Message-ID: Bugs item #471432, was opened at 2001-10-15 12:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Mark Hammond (mhammond) Summary: __cdecl / __stdcall unspecified in *.h Initial Comment: Currently, the win32 function call convention (__stdcall vs __cdecl) is never specified in the function prototypes contained in the .h header files. The call convention is therefore left up to the configuration of the compiler incluing the header files (__cdecl by default on VC++). However, there are situations in which the compiler default call convention must be set to __stdcall (/Gz option on VC++'s cl.exe), and this makes the link fail; until __cdecl keywords are manually insterted in the Python.h function prototypes required for linking. [For instance, /Gz is required when using typedef definitions on function pointers, later passed as arguments on __stdcall functions (the default on Microsoft/commercial libraries)]. Besides, Microsoft recommands, it the VC++ documentation, that the __stdcall convention be used by default for function with fixed numbers of arguments (smaller code size and better performances); __cdecl being limited for use with functions with variable number of args ('...'). A solution would consist in defining: #ifdef _MS_VER #define FIXARGCALL __stdcall #define VARARGCALL __cdecl #else ... #endif and sticking FIXARGCALL / VARARGCALL in front of all DLL_IMPORT(), macro calls associated to a an exported function prototype, across all Python .h files. Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-15 20:00 Message: Logged In: YES user_id=31435 I guess my first question is, if library X needs something other than the default, why isn't it X's responsibility to force *their* non-default behavior where needed? After all, they're the oddballs. Some days I get real tired of cleaing up after everyone else's loose bowels <0.9 wink>. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-10-15 18:45 Message: Logged In: YES user_id=14198 This does make sense to me, and I have even been bitten by it. There are some strange libraries out there that need to be compiled with a different calling convention than the default. Trying to extend Python with such a library can be difficult. Having the calling convention specified in the prototypes would solve the problem. I have never been brave enough to do it, given the small number of times it has been reported as a problem. However, I would be happy to help come up with a reasonable scheme. ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-15 12:46 Message: Logged In: YES user_id=93657 As far as MS Windows / dll stuff is concerned, you're already swimming in non-C-ANSI fishtank anyway (e.g. DLL_IMPORT() macros...). Just try to keep the right fishes in the tank :)) FG ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:34 Message: Logged In: YES user_id=31435 Mark, does this make sense to you? Any notion that we *have* to add non-standard decorations seems inherently fishy to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 From noreply@sourceforge.net Tue Oct 16 07:34:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 15 Oct 2001 23:34:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-471615 ] Incorrepopen2 exit status misrepresented Message-ID: Bugs item #471615, was opened at 2001-10-15 23:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471615&group_id=5470 Category: Documentation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Incorrepopen2 exit status misrepresented Initial Comment: The docs for the os module describe popen2, popen3, and popen4 as though they behaved like popen. Specifically, the doc says that "The exit status of the command (encoded in the format specified for wait()) is available as the return value of the close() method of the file object ..." This is not true. popen2 and friends do not return process exit status this way. Instead, they provide a wait() method for fetching process status. The docs for the popen2 module more truthfully describe how exit status is returned. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471615&group_id=5470 From noreply@sourceforge.net Tue Oct 16 08:45:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 00:45:59 -0700 Subject: [Python-bugs-list] [ python-Bugs-467145 ] Python 2.2a4 build problem on HPUX 11.0 Message-ID: Bugs item #467145, was opened at 2001-10-02 05:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Martin v. Löwis (loewis) Summary: Python 2.2a4 build problem on HPUX 11.0 Initial Comment: When compiling Python 2.2a4 on HP-UX 11.0 using the HP ansi C compiler, each source file generates an error like this: cc -Ae -Kpthread -c +DAportable -I. - I./Include -DHAVE_CONFIG_H -o Parser/acceler.o Parser/acceler.c cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-16 00:45 Message: Logged In: YES user_id=21627 Closing it, since the original problem seems to be solved. Richard, if any further issues remain, please open a new report, since it is getting difficult to see in this report what the problems were and how they got solved. On the curses issue: Please mail me in private if you want to pursue creation of a patch. I have some ideas, but cannot test them due to lack of hardware. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 12:45 Message: Logged In: YES user_id=6380 Thanks! You might want to check current CVS to make sure I did the right thing. Is there any reason for this to remain open? Over to MvL for the answer to that question... ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:35 Message: Logged In: YES user_id=31435 Reassigned to Guido to match reality. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-15 00:28 Message: Logged In: YES user_id=200117 I made the changes you suggested and most of the warnings for typeobject.c disappeared. The other line you mentioned should be changed thus: 3448c3448 < PyObject_GenericGetAttr) --- > (void *)PyObject_GenericGetAttr) This fixes all the warnings for that source file. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:43 Message: Logged In: YES user_id=6380 To rptownsend: the warnings you get seem to come from assigning function pointers to void*. Could you try the following change? 3693c3693 < {NAME, offsetof(PyTypeObject, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(PyTypeObject, SLOT), (void *)FUNCTION, WRAPPER} 3695c3695 < {NAME, offsetof(etype, SLOT), FUNCTION, WRAPPER} --- > {NAME, offsetof(etype, SLOT), (void *)FUNCTION, WRAPPER} There's one more line with a warning; can you figure out where to put a (void*) cast there yourself and let me know? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 06:23 Message: Logged In: YES user_id=21627 cmkl, a few questions: - It appears that STRICT_SYSV_CURSES should only be used if HAVE_NCURSES is not defined. Correct? - http://devresource.hp.com/STK/partner/xcurses.html claims that cur_colr is targeted for obsolescence. Why should we use it then? The same paper claims that _XOPEN_SOURCE_EXTENDED must be defined on 10.20 if /usr/include/curses.h (Xcurses) is used, due to a bug. - What functions does Python use that are only available with _POSIX_C_SOURCE=199506L ? - Could you please check out the current CVS and determine whether Python correctly enables large file support? What is the problem if only ftello/fseeko are used? What is the output of 'getconf LFS_CFLAGS'? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-12 04:53 Message: Logged In: NO To use curses (if GPL'd ncurses is not available) you have to make a patch and you have to compile with: -DSTRICT_SYSV_CURSES (some ncurses features are not available in this case) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** ../patch/py_curses.h Fri Dec 22 22:51:10 2000 --- py_curses.h Wed Oct 10 15:06:40 2001 *************** *** 5,11 **** --- 5,15 ---- #ifdef HAVE_NCURSES_H #include #else + #if defined(__hpux) + #include + #else #include + #endif #endif #ifdef __cplusplus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use threads on has to compile with: -D_POSIX_C_SOURCE=199506L (reentrant fct. are defined then) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use ftello and fseeko (largefile) one has to compile with: -D_LARGEFILE_SOURCE a new largefile API: ftello64 and fseeko64 (instead of ftell64 and fseek64 tested by configure) is available with: -D_LARGEFILE64_SOURCE (at the moment there is no difference to -D_LARGEFILE_SOURCE) /usr/share/doc/lg_files.txt contains the HPUX large file white paper BTW in Makefile.pre.in if have replaced all occurences of: $(INSTALL) -d -m with: mkdir -p -m - cmkl (cmkleffner@gmx.de) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:44 Message: Logged In: YES user_id=21627 Re-opening; this probably needs to be investigated. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-12 01:30 Message: Logged In: YES user_id=200117 I have built from the current CVS and it builds OK and 'make test' does not produce any failures. Please note I'm now getting some new warnings for typeobject.c which I didn't get in Python-2.2a4 - see attached file 'Make.txt'. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 11:33 Message: Logged In: YES user_id=31435 Reassigned to me and Closed as Fixed, but it would be very helpful if you could build from current CVS to confirm the fix on your platform (I'm using a union trick to force the GC header to have no-looser-than-double alignment, which should also help on platforms that don't require 8-byte double alignment but run faster if doubles are 8-byte aligned). Include/objimpl.h; new revision: 2.43 Modules/gcmodule.c; new revision: 2.24 ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 06:10 Message: Logged In: YES user_id=200117 Yes that fixes it, test_descr.py now runs OK. >From our experiments, I can confirm sizeof(long) == 4 and a double does require 8-byte alignment on HP-UX 11.00. I also ran 'make test' again and now get no failures. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-11 03:18 Message: Logged In: YES user_id=6380 Thanks. It looks like it's not happy with the alignment of a pointer to double; in particular, it's in Number.__repr__ in function complexes(). I'm assuming that on your architecture: - sizeof(long) == 4 - a double requires 8-byte alignment Then I believe I have a fix. Can you try this: In Include/objimpl.h, in the structure PyGC_Head, after "int gc_refs;" (line 272), insert another -- dummy -- int field, e.g. "int dummy;". Then recompile and re-run the test. If this fixes the problem, I have some work to do, but it would confirm my theory (that the alignment is botched because the GC header moves the structure up 12 bytes). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:57 Message: Logged In: YES user_id=200117 Oops, forgot to check the box... ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-11 02:55 Message: Logged In: YES user_id=200117 After doing a CVS update, test_descr.py still core dumps. See attached file Gdb.txt for the output from gdb (hope it's what you want). ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 08:06 Message: Logged In: YES user_id=200117 I will try. However, there will be a delay as I do not have cvs access to the internet here at work. I will have to do the update from home and bring in the new files to build tomorrow :-( Also, I'm not sure if we have gdb on the HP machines... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-10 06:04 Message: Logged In: YES user_id=6380 Can you do a CVS update and run the test for test_descr.py again? If it still core dumps, can you provide a stack trace using gdb? ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-10 02:38 Message: Logged In: YES user_id=200117 Thanks for the pointer. I have downloaded the current CVS and it builds without the errors for '-Kpthread'. (It still fails to build the _curses extension by the way). When I ran 'make test' it failed in test_descr.py [see attached file test.txt for output]. I then manually ran the following tests: test_thread.py test_threaded_import.py test_threaded_tempfile.py test_threading.py and these all succeeded. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 09:41 Message: Logged In: YES user_id=21627 Please have a look at http://sourceforge.net/cvs/?group_id=5470 This has instructions for "Anonymous CVS Access". ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-09 06:56 Message: Logged In: YES user_id=200117 I would be happy to try - please can you advise how to download the current CVS, I can only see the option to browse it and download individual files. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-09 03:57 Message: Logged In: YES user_id=21627 Can you please try the current CVS and report whether it also fixes the problem? The fix you've used was integrated in a modified form. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-04 08:10 Message: Logged In: YES user_id=200117 I have now tested the patch ftp://ftp.thewrittenword.com/outgoing/pub/python- 210665.patch with Python-2.2a4 and it fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467145&group_id=5470 From noreply@sourceforge.net Tue Oct 16 15:04:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 07:04:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-471615 ] Incorrepopen2 exit status misrepresented Message-ID: Bugs item #471615, was opened at 2001-10-15 23:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471615&group_id=5470 Category: Documentation Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Incorrepopen2 exit status misrepresented Initial Comment: The docs for the os module describe popen2, popen3, and popen4 as though they behaved like popen. Specifically, the doc says that "The exit status of the command (encoded in the format specified for wait()) is available as the return value of the close() method of the file object ..." This is not true. popen2 and friends do not return process exit status this way. Instead, they provide a wait() method for fetching process status. The docs for the popen2 module more truthfully describe how exit status is returned. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-16 07:04 Message: Logged In: YES user_id=3066 This has already been fixed in the development version of the documentation. See: http://python.sourceforge.net/devel-docs/lib/module-os.html ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471615&group_id=5470 From noreply@sourceforge.net Tue Oct 16 15:04:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 07:04:53 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Nobody/Anonymous (nobody) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Tue Oct 16 15:25:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 07:25:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-471725 ] sys.settrace is thread-local state Message-ID: Bugs item #471725, was opened at 2001-10-16 07:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471725&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: sys.settrace is thread-local state Initial Comment: Apparently sys.settrace sets a thread-local hook not a global hook. Perhaps the documentation at http://python.sourceforge.net/devel-docs/lib/debugger-hooks.html should be changed to reflect that. BTW, the link from http://python.sourceforge.net/devel-docs/lib/module-sys.html to http://python.sourceforge.net/devel-docs/lib/debugger-hooks.html#debugger-hooks would seem to be better if it linked to http://python.sourceforge.net/devel-docs/lib/debugger-hooks.html ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471725&group_id=5470 From noreply@sourceforge.net Tue Oct 16 15:31:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 07:31:10 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 >Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Tue Oct 16 15:49:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 07:49:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Tue Oct 16 15:55:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 07:55:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-471725 ] sys.settrace is thread-local state Message-ID: Bugs item #471725, was opened at 2001-10-16 07:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471725&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: sys.settrace is thread-local state Initial Comment: Apparently sys.settrace sets a thread-local hook not a global hook. Perhaps the documentation at http://python.sourceforge.net/devel-docs/lib/debugger-hooks.html should be changed to reflect that. BTW, the link from http://python.sourceforge.net/devel-docs/lib/module-sys.html to http://python.sourceforge.net/devel-docs/lib/debugger-hooks.html#debugger-hooks would seem to be better if it linked to http://python.sourceforge.net/devel-docs/lib/debugger-hooks.html ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-16 07:55 Message: Logged In: YES user_id=3066 Added information in Doc/lib/libsys.tex revision 1.53. The weird link is a bug in the tools; this is just another reason to abandon the current toolchain. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471725&group_id=5470 From noreply@sourceforge.net Tue Oct 16 16:35:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 08:35:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Tue Oct 16 17:12:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 09:12:29 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Tue Oct 16 17:28:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 09:28:13 -0700 Subject: [Python-bugs-list] [ python-Bugs-232460 ] SSL Support (Apparently) Broken on Solaris Message-ID: Bugs item #232460, was opened at 2001-02-14 19:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 Category: Extension Modules Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Jeremy Hylton (jhylton) Summary: SSL Support (Apparently) Broken on Solaris Initial Comment: I have spent about 10 hours banging on this with no luck. Python : Python-2.0 Platform : SPARC Solaris 2.8 Compiler : Sun Workshop Pro v5.0, gcc-2.95-2 OpenSSL : openssl-0.9.6 and 0.9.5. Problem - All attempts to open a SSL connection result in an "SSL_connect error". I have tried various combinations of keys and certificates. I have looked at the Python source code extensively and added debugging to try and get more information. I have run the system using the openssl s_client and s_server testing tools. I have recompiled everything in various configurations of versions and compilers. In all cases, the same error is generated. That said, has *ANYBODY* gotten this to work on Solaris? If so, do you have any details that can be shared? Cheers, Dave P.S. I will submit a patch if I can ever get this to actually work. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-16 09:28 Message: Logged In: YES user_id=31392 I think we'll add some RAND_xxx() methods to the Python API to allow people to seed the prng themselves. I'd like to have some mechanism in place to do the seeding autmatically, but I don't think it makes a lot of sense for us to try and maintain this in Python. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 12:34 Message: Logged In: YES user_id=21627 On PRNG problems, it seems we have two options: - wait for, then require OpenSSL 0.9.7. It will look for a prngd socket in default locations (/var/run/egd-pool, /dev/egd-pool, /etc/egd-pool and /etc/entropy); then require administrators to set up OpenSSL that it indeed finds a prngd in these locations when needed. - expose RAND_add. Exposing any other of the interfaces is pointless; on our installation, prngd runs on localhost:708 instead of a Unix domain socket, and none of the other interfaces could use such a configuration. On top of RAND_add, we could communicate with prngd ourselves, e.g. by using the RANDFILE environment variable. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 10:57 Message: Logged In: YES user_id=21627 Running httplib.py now crashes with Traceback (most recent call last): File "Lib/httplib.py", line 867, in ? test() File "Lib/httplib.py", line 853, in test hs.connect(host) File "Lib/httplib.py", line 698, in connect self._conn.connect() File "Lib/httplib.py", line 653, in connect ssl = socket.ssl(realsock, self.key_file, self.cert_file) socket.sslerror: (1, 'error:24064064:random number generator:SSLEAY_RAND_BYTES:PRNG not seeded') Not sure how this should be fixed, though. We use prngd here. To configure OpenSSH, I had to give prngd parameters to OpenSSH (i.e. that it should use prngd, and what the port number is). On testing certificates: It might be sufficient to generate a self-signed one, and distribute that as part of the test suite. Of course, you need to write server and client code to use the appropriate files. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:34 Message: Logged In: YES user_id=31392 I've made some recent changes that affect SSL_connect(), although the primary change is to improve the error messages. Dave and Martin, can you check the current code on Solaris? If it still fails, we'll at least get a clearer error message. BTW, does anyone have suggestions for how to test the code with key and cert files? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-07-28 10:39 Message: Logged In: YES user_id=21627 I still have the same problem, both on my own installation (both with 2.1.1, and with the CVS), as well as on cf.sf.net:/home/users/loewis/Python-2.1.1/python (you need to set LD_LIBRARY_PATH=/usr/local/ssl/lib to use this one). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-14 20:30 Message: Logged In: YES user_id=6380 David, is this still broken? On the SourceForge compile farm I can use SSL just fine. Check out my directory /home/users/gvanrossum/Python-2.1-solaris on cf.sourceforge.net. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-10 11:55 Message: Logged In: YES user_id=6380 So, David, you still interested in this? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-20 07:58 Message: Logged In: YES user_id=11375 Moshe checked in patch #405101 to the CVS tree. David, can you update and see if it fixed the problem for you? ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-03-18 02:30 Message: Logged In: YES user_id=11645 I just want to me-too Neil, since I submitted the original patch. I think it *did* fix problems on Solaris, as long as the user is running EGD and set up RANDFILE env. variable properly. Otherwise, you won't be getting connect errors, but a warning from Python about using a non-secure way to generate seed. (For the record, the problem is that Solaris doesn't have a /dev/urandom like Linux does) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-02 07:49 Message: Logged In: YES user_id=35752 It looks like patch "[ #405101 ] Add Random Seeding to OpenSSL" might fix this. Assigning to Andrew since he is assigned that patch too. ---------------------------------------------------------------------- Comment By: David M. Beazley (beazley) Date: 2001-02-15 06:12 Message: A followup on this bug.... It appears that Python is not properly seeding the OpenSSL random number generator when it creates a secure socket. This, in turn, causes the SSL_connect() function to fail due to improper random number seeding (although the error wasn't obvious--at least not to me). This is also due to an apparent "feature" of Solaris not providing a usable /dev/random device. A simple fix is to modify init_socket() in socketmodule.c to include a call to RAND_seed() like this: #ifdef USE_SSL SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); /* Sick hack for Solaris */ { char bogus[64]; /* Fill in bogus with some random noise */ ... RAND_seed(bogus, 64); } ... #endif Presumably one would need to think about the randomness actually passed to RAND_seed() (I have done nothing above). Here are related comments from the OpenSSL FAQ: "Cryptographic software needs a source of unpredictable data to work correctly. Many open source operating systems provide a "randomness device" that serves this purpose. On other systems, applications have to call the RAND_add() or RAND_seed() function with appropriate data before generating keys or performing public key encryption. Some broken applications do not do this. As of version 0.9.5, the OpenSSL functions that need randomness report an error if the random number generator has not been seeded with at least 128 bits of randomness. If this error occurs, please contact the author of the application you are using. It is likely that it never worked correctly. OpenSSL 0.9.5 and later make the error visible by refusing to perform potentially insecure encryption." Python-2.1a2 does not appear to include a fix, but I can't test it to find out (since 2.1a2 doesn't compile on my machine). Cheers, Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 From noreply@sourceforge.net Tue Oct 16 17:59:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 09:59:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-16 09:59 Message: Logged In: YES user_id=56897 I'll grab the 2.2b1 release when it is available and test it on the Solaris and Linux configurations we have. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 14:52 Message: Logged In: YES user_id=6380 Since I've now applied your patch, I presume this is fixed, and I'm closing the bug report. Let me know if there are still problems. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-09 11:40 Message: Logged In: YES user_id=56897 Patch is #468347 [mask signals for non-main pthreads] ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 10:09 Message: Logged In: YES user_id=56897 I've submitted the patch for pthread signal masking. My biggest concerns are the guesses I made for DCE threads and whether they will work for AIX which might need to use sigthreadmask(). Regarding reproducing this on Linux, I was able to get Linux to crash if I held down Ctrl-C (with fairly fast key repeat). After starting the spinning thread, Python would crash on Linux under a storm of SIGINTs within 30 seconds or so. Without the spinning thread, I couldn't get it to crash. With the patch applied, the spinning thread running during the storm of SIGINTs wouldn't crash it. So that implies the signal masking is doing something good even in the Linux case. Re: my SF problems, I submitted a few support requests. Hopefully something gets fixed. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 08:38 Message: Logged In: YES user_id=56897 I've checked the Monitor button while logged in, but I still do not receive email notification of updates. When I clicked it again, it said I was no longer monitoring, so I clicked it yet again back into monitoring mode. Apparently SF knows I'm monitoring it, but it still doesn't send me email. Mail to my @users.sf.net account does work, so I'm at a loss to explain why a) My login cookie doesn't stick around very long at all and b) Why I never get monitor email from SF Re: the patch, I have something that works well on Solaris. I'll try it on Linux today, but I don't have access to an HP-UX system. I'm a little concerned about the impact to HP-UX (pre 11.0 and post 11.0) and AIX, and I don't have access to those machines to check out those concerns. Hopefully I'll have the patch posted by today. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Tue Oct 16 18:19:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 10:19:56 -0700 Subject: [Python-bugs-list] [ python-Bugs-471432 ] __cdecl / __stdcall unspecified in *.h Message-ID: Bugs item #471432, was opened at 2001-10-15 12:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Mark Hammond (mhammond) Summary: __cdecl / __stdcall unspecified in *.h Initial Comment: Currently, the win32 function call convention (__stdcall vs __cdecl) is never specified in the function prototypes contained in the .h header files. The call convention is therefore left up to the configuration of the compiler incluing the header files (__cdecl by default on VC++). However, there are situations in which the compiler default call convention must be set to __stdcall (/Gz option on VC++'s cl.exe), and this makes the link fail; until __cdecl keywords are manually insterted in the Python.h function prototypes required for linking. [For instance, /Gz is required when using typedef definitions on function pointers, later passed as arguments on __stdcall functions (the default on Microsoft/commercial libraries)]. Besides, Microsoft recommands, it the VC++ documentation, that the __stdcall convention be used by default for function with fixed numbers of arguments (smaller code size and better performances); __cdecl being limited for use with functions with variable number of args ('...'). A solution would consist in defining: #ifdef _MS_VER #define FIXARGCALL __stdcall #define VARARGCALL __cdecl #else ... #endif and sticking FIXARGCALL / VARARGCALL in front of all DLL_IMPORT(), macro calls associated to a an exported function prototype, across all Python .h files. Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Frederic Giacometti (giacometti) Date: 2001-10-16 10:19 Message: Logged In: YES user_id=93657 The fault is not in the other libraries. In MS Windows, 'well-behaved' libraries define explicitely the calling conventions for each function in their header file. This is all. There's nobody's guts loosely hanging out in this ;). FG ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-15 20:00 Message: Logged In: YES user_id=31435 I guess my first question is, if library X needs something other than the default, why isn't it X's responsibility to force *their* non-default behavior where needed? After all, they're the oddballs. Some days I get real tired of cleaing up after everyone else's loose bowels <0.9 wink>. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-10-15 18:45 Message: Logged In: YES user_id=14198 This does make sense to me, and I have even been bitten by it. There are some strange libraries out there that need to be compiled with a different calling convention than the default. Trying to extend Python with such a library can be difficult. Having the calling convention specified in the prototypes would solve the problem. I have never been brave enough to do it, given the small number of times it has been reported as a problem. However, I would be happy to help come up with a reasonable scheme. ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-15 12:46 Message: Logged In: YES user_id=93657 As far as MS Windows / dll stuff is concerned, you're already swimming in non-C-ANSI fishtank anyway (e.g. DLL_IMPORT() macros...). Just try to keep the right fishes in the tank :)) FG ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-15 12:34 Message: Logged In: YES user_id=31435 Mark, does this make sense to you? Any notion that we *have* to add non-standard decorations seems inherently fishy to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471432&group_id=5470 From noreply@sourceforge.net Tue Oct 16 19:03:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 11:03:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-459767 ] ftplib fails with files > 2GB Message-ID: Bugs item #459767, was opened at 2001-09-08 03:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 2 Submitted By: Martin Preishuber (preisl) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib fails with files > 2GB Initial Comment: The ftplib.py module is not ready for large files > 2GB. The funtions size and parse150 return integer values as file size which causes overflows on large files. ---------------------------------------------------------------------- >Comment By: Martin Preishuber (preisl) Date: 2001-10-16 11:03 Message: Logged In: YES user_id=29575 These changes are necessary: For Python 1.*: replace atoi by atol in size() and parse150() For Python 2.*: replace int by long in size() and parse150() ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-10 06:36 Message: Logged In: YES user_id=6380 Please upload your patches and I'll see... ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-09-08 04:14 Message: Logged In: YES user_id=29575 well, I've written emirror, some ftp mirroring software. There are people, who use it to make backups of such large files. I already use some modified ftp library, which fixes some bugs in the 1.52 library (those bugs are included in >2.0 versions) and I have made the necessary changes for large file support myself ... I just thought it'd be nice to have that support generally. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-08 04:08 Message: Logged In: YES user_id=6380 Do you have an actual use for this, or it this just a theoretical complaint? Show me an FTP server with a file > 2GB and I'll consider fixing it. In the future (Python 2.3), int() will auto-convert to long if necessary -- see PEP 237. So this will be fixed eventually anyway -- probably long before such large files will become an issue. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 From noreply@sourceforge.net Tue Oct 16 20:13:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 12:13:01 -0700 Subject: [Python-bugs-list] [ python-Bugs-459767 ] ftplib fails with files > 2GB Message-ID: Bugs item #459767, was opened at 2001-09-08 03:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 2 Submitted By: Martin Preishuber (preisl) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib fails with files > 2GB Initial Comment: The ftplib.py module is not ready for large files > 2GB. The funtions size and parse150 return integer values as file size which causes overflows on large files. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:13 Message: Logged In: YES user_id=6380 Can you verify that this patch works for you? (I don't have a server with files > 2GB lying around, and my network would take days to download them anyway. ;-( ) See the uploaded file "ftplib.patch.txt" ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-10-16 11:03 Message: Logged In: YES user_id=29575 These changes are necessary: For Python 1.*: replace atoi by atol in size() and parse150() For Python 2.*: replace int by long in size() and parse150() ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-10 06:36 Message: Logged In: YES user_id=6380 Please upload your patches and I'll see... ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-09-08 04:14 Message: Logged In: YES user_id=29575 well, I've written emirror, some ftp mirroring software. There are people, who use it to make backups of such large files. I already use some modified ftp library, which fixes some bugs in the 1.52 library (those bugs are included in >2.0 versions) and I have made the necessary changes for large file support myself ... I just thought it'd be nice to have that support generally. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-08 04:08 Message: Logged In: YES user_id=6380 Do you have an actual use for this, or it this just a theoretical complaint? Show me an FTP server with a file > 2GB and I'll consider fixing it. In the future (Python 2.3), int() will auto-convert to long if necessary -- see PEP 237. So this will be fixed eventually anyway -- probably long before such large files will become an issue. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 From noreply@sourceforge.net Tue Oct 16 20:25:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 12:25:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-471828 ] x< Bugs item #471828, was opened at 2001-10-16 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471828&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: x<>> 1<<30 1073741824 >>> 1<<31 -2147483648 >>> 1<<32 0 >>> The change would of course cause incompatibilities with existing programs relying on the old behaviour so I am not sure I want it very much myself. But just mentioning it, I suppose either the pep or Python needs updating. Sverker Nilsson ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471828&group_id=5470 From noreply@sourceforge.net Tue Oct 16 20:40:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 12:40:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-471834 ] Nomenclature in os module, exec*() Message-ID: Bugs item #471834, was opened at 2001-10-16 12:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471834&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 3 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Nomenclature in os module, exec*() Initial Comment: Both Stevens and the Linux manpages make a small but important distinction in the first argument to execvp() and execlp() verses the other exec*() functions. The `p' version takes a file name, which is used in a $PATH search if the argument does not contain a slash. The others take a path argument for which $PATH is not searched. The documentation for execvp() and execlp() in the os module should mimic this nomenclature. I.e. `file' for execvp(), execlp() -- and for good measure, our own hybrid execvpe(). It should be `path' for execl(), execv(), execle(), and execve(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471834&group_id=5470 From noreply@sourceforge.net Tue Oct 16 20:46:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 12:46:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-459767 ] ftplib fails with files > 2GB Message-ID: Bugs item #459767, was opened at 2001-09-08 03:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 2 Submitted By: Martin Preishuber (preisl) >Assigned to: Guido van Rossum (gvanrossum) Summary: ftplib fails with files > 2GB Initial Comment: The ftplib.py module is not ready for large files > 2GB. The funtions size and parse150 return integer values as file size which causes overflows on large files. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:46 Message: Logged In: YES user_id=6380 I've fixed this in CVS. ftplib.py rev 1.59. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:13 Message: Logged In: YES user_id=6380 Can you verify that this patch works for you? (I don't have a server with files > 2GB lying around, and my network would take days to download them anyway. ;-( ) See the uploaded file "ftplib.patch.txt" ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-10-16 11:03 Message: Logged In: YES user_id=29575 These changes are necessary: For Python 1.*: replace atoi by atol in size() and parse150() For Python 2.*: replace int by long in size() and parse150() ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-10 06:36 Message: Logged In: YES user_id=6380 Please upload your patches and I'll see... ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-09-08 04:14 Message: Logged In: YES user_id=29575 well, I've written emirror, some ftp mirroring software. There are people, who use it to make backups of such large files. I already use some modified ftp library, which fixes some bugs in the 1.52 library (those bugs are included in >2.0 versions) and I have made the necessary changes for large file support myself ... I just thought it'd be nice to have that support generally. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-08 04:08 Message: Logged In: YES user_id=6380 Do you have an actual use for this, or it this just a theoretical complaint? Show me an FTP server with a file > 2GB and I'll consider fixing it. In the future (Python 2.3), int() will auto-convert to long if necessary -- see PEP 237. So this will be fixed eventually anyway -- probably long before such large files will become an issue. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 From noreply@sourceforge.net Tue Oct 16 20:59:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 12:59:01 -0700 Subject: [Python-bugs-list] [ python-Bugs-471828 ] x< Bugs item #471828, was opened at 2001-10-16 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471828&group_id=5470 Category: Python Interpreter Core >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) Summary: x<>> 1<<30 1073741824 >>> 1<<31 -2147483648 >>> 1<<32 0 >>> The change would of course cause incompatibilities with existing programs relying on the old behaviour so I am not sure I want it very much myself. But just mentioning it, I suppose either the pep or Python needs updating. Sverker Nilsson ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:59 Message: Logged In: YES user_id=6380 SectionTransition label A explains this: this is one of the semantic aspects that will be addressed after 2.2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471828&group_id=5470 From noreply@sourceforge.net Tue Oct 16 21:21:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 13:21:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-468061 ] __str__ ignored in str subclass Message-ID: Bugs item #468061, was opened at 2001-10-04 13:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: __str__ ignored in str subclass Initial Comment: >From c.l.py; looks messy (e.g., if PyObject_String didn't ignore octetstring.__str__, the call to str() in the latter would just recurse forever; and he's right that b2a_hex requires a genuine string; and e.g. return b2a_hex(self + "") would be mighty obscure). ----Original Message----- From: Andreas.Trawoeger Sent: Thursday, October 04, 2001 7:10 AM To: python-list Subject: Python 2.2a4: Problems with overloading __str__ in subclasses of str I would like to write a class that behaves like a string only with the difference that it displays the string in hex. In Python 2.2a4 this should be possible by subclassing str and overloading __repr__ and __str__ : from binascii import b2a_hex class octetstring(str): def __repr__(self): return b2a_hex(str(self)) def __str__(self): return b2a_hex(str(self)) o = octetstring('A') print "o.__repr__():",o.__repr__() print "o.__str__() :",o.__str__() print "str(o) :",str(o) print "o :",o The result is pretty strange: o.__repr__(): 41 o.__str__() : 41 str(o) : A o : A I'm not shure what's the exact reason for this problem. Either __str__ isn't used for str() and print. Or I'm trapped in an chicken egg problem: I have to convert self to an string (otherwise b2a_hex won't work) to do this Python should use __str__ (which is the method I'm changing). cu andreas ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-16 13:21 Message: Logged In: YES user_id=31435 Closed. str() and print applied to subclasses of str now respect a __str__ method override (if any). Ditto for the corresponding C API functions (PyObject_Str() in particular). Lib/test/test_descr.py; new revision: 1.90 Objects/object.c; new revision: 2.157 Objects/stringobject.c; new revision: 2.139 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-11 19:44 Message: Logged In: YES user_id=31435 Bug 470040 has been fixed, so you'll be able to pass instances of str subclasses directly to b2a_hex (which uses the buffer interface to obtain the value; buffer methods simply weren't getting inherited; OTOH, I'm not sure this is such a great thing, since I'm not sure a str subclass in Python can override the buffer methods now inherited from str!). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-10 15:52 Message: Logged In: YES user_id=31435 The business about b2a_hex not accepting an object of a subclass of str is now the subject of bug 470040. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-04 13:57 Message: Logged In: YES user_id=31435 Looks like I assigned this to Guido by mistake; back to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468061&group_id=5470 From noreply@sourceforge.net Tue Oct 16 23:01:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 15:01:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Tue Oct 16 23:42:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 15:42:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Wed Oct 17 00:03:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 16:03:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-471111 ] inspect.getframeinfo() needs docs Message-ID: Bugs item #471111, was opened at 2001-10-14 13:59 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Tim Peters (tim_one) Summary: inspect.getframeinfo() needs docs Initial Comment: http://web.lfw.org/python/inspect.html doesn't include the getmodulename() function and http://python.sourceforge.net/devel-docs/lib/module-inspect.html doesn't include the getframeinfo() function, but: MAIL imp:~$ python Python 2.2a4+ (#2, Oct 11 2001, 10:21:08) [GCC 2.95.4 20011006 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import inspect >>> dir(inspect) ['BlockFinder', 'CO_NEWLOCALS', 'CO_OPTIMIZED', 'CO_VARARGS', 'CO_VARKEYWORDS', 'EndOfBlock', 'ListReader', '__author__', '__builtins__', '__date__', '__doc__', '__file__', '__name__', '_searchbases', 'classify_class_attrs', 'currentframe', 'dis', 'findsource', 'formatargspec', 'formatargvalues', 'getabsfile', 'getargs', 'getargspec', 'getargvalues', 'getblock', 'getclasstree', 'getcomments', 'getdoc', 'getfile', 'getframeinfo', 'getinnerframes', 'getlineno', 'getmembers', 'getmodule', 'getmoduleinfo', 'getmodulename', 'getmro', 'getouterframes', 'getsource', 'getsourcefile', 'getsourcelines', 'imp', 'indentsize', 'isbuiltin', 'isclass', 'iscode', 'isframe', 'isfunction', 'ismethod', 'ismethoddescriptor', 'ismodule', 'isroutine', 'istraceback', 'joinseq', 'modulesbyfile', 'os', 're', 'stack', 'string', 'strseq', 'sys', 'tokenize', 'trace', 'types', 'walktree'] >>> ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-16 16:03 Message: Logged In: YES user_id=31435 Added docs for getframeinfo, in Doc/lib/libinspect.tex; new revision: 1.6 Closed, as there doesn't appear to be any point to reassigning to Ping. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-15 06:20 Message: Logged In: YES user_id=3066 Assigned to inspect-maintainer-of-the-day for docs. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-14 15:04 Message: Logged In: YES user_id=21627 Please note that http://web.lfw.org/python/inspect.html is maintained independently, so you should complain to the maintainer of this page (Ka-Ping Yee) about inconsistencies. The official Python documentation for the current release is http://www.python.org/doc/current/lib/module-inspect.html and it does mention getmodulename. So it appears that the core of your report is that getframeinfo is not documented. Please understand that two sets of Python documentation are maintained: one is the doc strings, which are the foundation of Ping's pages (via pydoc). The other is the Latex documentation, which is converted automatically into the "online" (ie. HTML documentation), where inspect is documented in Doc/lib/libinspect.tex. If you find that the Tex/HTML documentation misses some pieces that are worth being mentioned, and if you have convenient access to CVS, please also consider providing patches for the documentation. In this specific case, it might be sufficient to copy the docstring into the tex documentation; having such patches available makes it easier to integrate them into the documentation. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471111&group_id=5470 From noreply@sourceforge.net Wed Oct 17 00:06:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 16:06:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Wed Oct 17 00:28:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 16:28:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-467384 ] provide a documented serialization func Message-ID: Bugs item #467384, was opened at 2001-10-02 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 Category: None Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: provide a documented serialization func Initial Comment: It would be nice if there was a documented library function for serializing Python basic objects (numbers, strings, dictionaries, and lists). By documented I mean the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages. Code-wise, the marshal.dumps and loads functions do what I want, but their data format is (according to the documentation) intentionally not specified, because the format might change in future Python versions. Maybe that doc was written long enough ago that it's ok to freeze the marshal format now, and document it? I just mean for the basic types listed above. Stuff like code objects don't have to be specified. In fact it would be nice if there was a flag to the loads and dumps functions to refuse to marshal/ unmarshal those objects. Pickle/cpickle aren't really appropriate for what I'm asking, since they're complicated (they try to handle class instances, circular structure, etc.) and anyway they're not documented either. The XDR library is sort of ok, but it's written in Python (i.e. slow) and it doesn't automatically handle compound objects. Thanks ---------------------------------------------------------------------- >Comment By: paul rubin (phr) Date: 2001-10-16 16:28 Message: Logged In: YES user_id=72053 Tim has opened a doc bug for pickle/marshal security issues as #471893. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-14 15:27 Message: Logged In: YES user_id=72053 My understanding of marshal (I better check it, but I did mention the issue in the original request) is that it can create code objects but it doesn't actually execute the code in them. My implementation currently uses marshal but checks that the stuff marshal returns doesn't contain anything unexpected. Unpickle is different--it looks like it can execute hostile code before the loads call ever returns. By the time you have a chance to check the result, it's too late. cPickle.c appears to work exactly the same way (using eval and creating arbitrary instances, but maybe not calling marshal) as pickle.py. It never would have occured to me that the unpickler would work that way (and I'm still not convinced I understand it--I better try putting together a test to see if it's really like that). That's why I didn't notice the security issue til we started discussing pickle and I actually looked at the code. I'm sorry if that sounds like I'm adding requirements. I'd have thought it would go without saying that an important utility shouldn't have security holes. I'm ok with using pickle if the doc and security concerns are taken care of. More efficient longs would be helpful but they would break interoperability with old versions and I can probably live without them. It's really sad that longs were shrugged off when the pickle binary format was designed. Now in order to have efficient longs, yet another flag will have to be added to the constructor. Btw, if the unpickle security issue is real (I'm still not convinced!), I feel it should be treated as a major bug and that an announcement should be sent out. Unpickle already anticipates hostile pickled strings in the non-binary format and checks for them (see _is_secure_string) though I'd want want to spend an hour or two checking both the `...` code and the evaluator before believing that _is_secure_string is really safe-- and even if it is, it's brittle. But it looks like object creation security is an area they didn't think about. Basically I have nothing against pickle in principle, but it has these (fixable) problems, and while marshal is straightforwardly written, both pickle implementations are excessively clever and make me queasy. Anyway, I can go along with the idea that the right solution is to fix pickle--but at present, pickle looks like it's in worse shape than marshal. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-14 13:36 Message: Logged In: YES user_id=31435 Ack -- Paul, you add a new hitherto secret requirement with each reply. marshal isn't secure at all: because its *purpose* is to load .pyc files, marshal creates Python code objects out of any bytes you happen to feed it following a "code object" tag. That's a hole big enough to swallow the solar system. In 2.2, marshal refuses to unpack code objects in restricted execution mode, but not before 2.2, and it never refuses in unrestricted mode. In contrast, pickle doesn't know anything about code objects, so doesn't have this hole. The pickle docs are clear about this, too, spelling out that marshal's code- object abilities create "the possibility of smuggling Trojan horses into a program". When wondering about security, you should be looking at (and using) cPickle.c instead of pickle.py; cPickle doesn't use marshal at all, nor does it do eval()s etc. Yes, it can reconstruct pickled instances of classes that already exist, but it cannot create new classes. I haven't heard that characterized as an insecurity before, but to each his own level of discomfort. I want to go back to the start: if the question is whether Python is interested in documenting another data transmission format, my answer is no. There are many already (don't forget the ones from the CORBA and ILU worlds either) available from Python, and there's no reason to believe encoders/decoders for a Python-specific format would get implemented in any other language. pickle is Python's generic answer to the Python-specific serialization question. I'd be happy to see patches to improve it (whether for efficient longs, or some stricter notion of security, or even just docs). But I expect any additional Python-specific serialization scheme has an audience of one (if you disagree, fine, write a PEP and get some community consensus). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 17:49 Message: Logged In: YES user_id=72053 I agree with Tim that the internal implementation of long arithmetic isn't relevant to this--it was just surprising, and means the current marshal format isn't all that natural for external use. I don't have a particular agenda to get marshal documented, beyond that it would happen to solve my immediate problem. Alternatives are fine too. The ones suggested so far just don't seem to do the job, viz.: xmlrpc does NOT serialize basic Python objects--in particular it doesn't serialize integers longer than 32 bits. I can't consider using pickle until I've convinced myself that it doesn't make security holes, and so far it looks like the opposite. (Can someone tell me I'm not reading it right?). Yes, of course, it's not that difficult to write Python code to do everything I want. It's just surprising that I should need to do that. I mean, imagine if there was no integer addition function (no "+" operator) and the maintainers said "that's ok, to add a and b, just use 'a - (-b)'". It's not a showstopping obstacle, but I'm surprised to get so much grief for suggesting making the operation more convenient, since it's an obvious thing to want to do (as evidenced by there already being so many overlapping serialization functions: marshal, pickle, rpclib, the Serialization class from Vaults of Parnassus, three different ASN1 implementations you mentioned, etc). I can't see anywhere where I've requested a "special gimmick". Yes, an efficient bigint representation in pickle is nice and ought to be added, but I can live without it. I can NOT live with security holes, but wanting security shouldn't be considered a special gimmick! With binary bigints, a documented format, and a way to 100% stop the unpickler from ever calling eval or apply on untrusted data, I wouldn't mind using pickle despite its additional complexity compared to marshal. I don't want to depend on third party modules unless I bundle them with my application (again not a showstopper, but it's not in Python's "batteries included" spirit to need them at all). Telling a user "to run this app, first download modules vreeble from and frob from " where url1 and url2 usually turn out to be broken links by the time the user sees them is not the right way to distribute an app. (It happens I'm going to sometimes tell the user "to run this app, first destroy your handheld computer's OS by reflashing the firmware..." but it's the principle of the thing, you know). Anyway, the 15 bit bigint representation is reason enough to not want to freeze the current marshal format. Maybe a future marshaller can use a cleaner bigint format and at that point perhaps the issue can be revisited. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 16:29 Message: Logged In: YES user_id=72053 A pickle subset ("gherkin"?) could possibly also fill this need, if it was documented, even though pickle format is considerably more complicated than marshal format (it uses marshal.dumps for binary output, actually taking apart the marshalled strings). It was obvious in seconds how marshal.c works but after 30 minutes of looking at pickle.py I'm still not sure I understand it. It looks like the unpickler can construct arbitrary class instances and import arbitrary modules, which makes a security hole if the pickled strings are potentially hostile, but I might not be reading it right. Also, the unpickler must implement constant folding (the memo scheme), which complicates it somewhat, though it's not that bad. The idea of leaving the marshal formats of some Python- specific objects undocumented isn't to get out of documenting stuff, but to leave those formats open to later change. Re BER/DER, Burt Kaliski's "Layman's Guide" is pretty readable (http://borg.isc.ucsb.edu/aka/Auth/ASN1layman.htm). You're right about using all 8 bits in BER integers--it looks like the 7 bit representation is only used for OID components (I didn't realize that til checking on it just now). BER might be ok for what I'm doing--I'm not sure right now since I don't understand ASN1 that well. It looks not in the spirit of marshal/pickle though: to encode a compound object it looks like you need an ASN1 spec of EXACTLY what you expect to find in the object. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 15:25 Message: Logged In: YES user_id=31435 Martin, you're right about add and sub, but it's a shallow assumption easy to relax (basically just declare carry/borrow as twodigits instead of digit). I'd be more worried about the stwodigits type, but since nothing is actually broken here I'm not keen to fritter away time proving bounds on the temps in bigint division. How we implement bigints internally is off topic anyway (provided we're not trying to hijack internal implementation formats for unintended purposes). About BER, yes, and the URL I included is to a freely downloadable copy of the X.690 std; section 8.3 spells out the INTEGER rules. They aren't at all the rules Paul sketched, hence "I don't really know what you [Paul] mean by 'BER'". For the rest, while xmlrcplib may meet the letter of what Paul asked for at first, it's clear to me that it doesn't meet what he really wants. My suggestion remains to add a new, efficient bigint format to pickle, which would meet everything except Paul's desire to have a special gimmick limited to his specific application and without having to write one himself. The internal API functions _PyLong_AsByteArray and _PyLong_FromByteArray already do the heavy lifting in both directions (to or from base 256, unsigned or complemented, big- or little-endian). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 14:04 Message: Logged In: YES user_id=21627 7-bit vs. 8-bit: You were confusing tag encoding and INTEGER value encoding. no way to encode a string with a length: suppose you want a 32 bit length, what's wrong with struct.pack("l",len(s))+s 15-bit representation: I believe the add and sub implementations make use of the guarantee that a short won't overflow if the input fits into 15 bits. Tim: BER = Basic Encoding Rules (as in the subtitle of X.690) Even after all this discussion, I still cannot see why the existing libraries (including those offered for free by third parties) are not sufficient. It appears that Paul wants, among other things, that marshal becomes documented; it also appears that this won't hapen. What the other things are that Paul wants, I cannot tell, so I recommend to close this report with "won't fix". Paul, if you have a specific change that you want to be made, or a specific problem that you want to point out, please submit a new report. This issue "provide a documented serialization func" really ought to be closed as "Fixed"; xmlrpclib is already part of standard library and fits the original problem description: - it is a library for serializing Python basic objects - it is documented in the sense that the protocol is specified precisely enough to write interoperating implementations in other languages. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 12:07 Message: Logged In: YES user_id=31435 I don't buy the argument that pickle is "complicated", as you weren't going to document the parts of the marshal format you didn't care about either. A subset of pickle is just as easy to document and implement across languages as a subset of marshal, but with the key benefit that the pickle format is stable across releases. So if you want a structure packer, pickle is the obvious choice; it just lacks an efficient (in time and space) scheme for storing longs now. And unlike marshal, it isn't a dead end when you decide your app needs something fancier -- pickle already handles just about everything that *can* be pickled, and is designed to be extensible to user-defined types too, so you can painlessly expand your view of what the "interesting" subset is as your ambitions grow. I don't really know what you mean by "BER". The ANS.1 std section 8.3 is quite clear that all 8 bits are used in each byte for integer representations -- it's a giant 2's-comp integer, with a variable-length length prefix, redundant sign bytes are forbidden, and there's nothing special about the last byte. I agree with Martin that ANS.1 BER is as compact a standardized bigint representation as there is. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-13 03:24 Message: Logged In: YES user_id=72053 1) if Python longs are currently implemented as vectors of 15-bit digits (yikes--why on earth would anyone do that) and marshalled like that, then I agree that THAT much weirdness doesn't need to be propagated to future versions. Wow! I never looked at the long int code closely, but the marshal code certainly didn't reflect that. It's still possible to freeze the current marshal format and let future versions define a new mechanism for loading .pyc's. >From my own self-interest (of wanting to distribute apps that work across versions) that idea attracts me, but it's probably not the right thing in the long run. Better may be to fix the long int format right away and THEN document/ freeze it. (Use a new format byte so the 2.2 demarshaller can still read 2.1 .pyc files). By "fix" I mean use a simple packed binary format, no 15 bit digits, no BER, and the length prefix should be a byte or bit count, not multibyte "digits". 2) Unfortunately it's not easy in portable C with 32 bit longs to use digits wider than 16 bits--multiplication becomes too complicated. If the compiler supports wide ints (long long int) then conditionalized code to use them might or might not be deemed worthwhile. Python's long int arithmetic (unlike Perl's Math::BigInt class) is fast enough to be useable for real applications and I don't expect it to go to the extremes that gmpy does (highly tuned algorithms for everything, asm code for many cpu's, etc). So currently I use gmpy when it's available and fall back on longs if gmpy won't import--this works pretty well so far. 3) I like the idea of a BER/DER library for Python but I don't feel like being the guy who writes it. I'd probably use it if it was available, though maybe not for this purpose. (I'd like to handle X509 certificates in Python). BER really isn't the most efficient way to store long ints, by the way, since it puts just 7 useful bits in a byte. 4) My suggestion of BER in 465045 was motivated slightly differently, which was to add a feature from Perl's pack/ unpack function that's missing from Python's struct.pack/ unpack. I understand a little better now what the struct module is for, so binascii may be a better place for such a thing. However, I believe Python really needs a pack/unpack module that does all the stuff that Perl's does. Data conversion like that is an area where Perl is still beating Python pretty badly. (Again, I don't feel like being the one who writes the module). 5) Sorry I didn't notice Guido's post of 20:24 earlier (several arrived at once). I guess I'm willing to submit a patch for binascii to read and write longs in binary. It's slightly humorous to put it in binascii since it's a binary-binary conversion with no ascii involved, but the function fits well there in any case. I'd still rather use marshal, since I want to write out more kinds of data than longs, and with a long->binary conversion function I'd still need to supply Python code to traverse dictionaries and lists, and encode strings. Btw, the struct module doesn't have any way to encode strings with a length, except the Pascal format which is limited to 256 bytes and is therefore useless for many things. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-13 01:39 Message: Logged In: YES user_id=31435 Martin, Paul suggested BER previously in 465045. I suspect he's going to suggest this for every module one by one, until somebody bites . I doubt he wants genuine ASN.1 BER, though, as that's a complicated beast, and he only cares about ints with a measly few hundred bits; regardless, a Python long can't have more digits than can be counted in a C int. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-13 01:14 Message: Logged In: YES user_id=21627 I would have never guessed that arbitrarily long ints are a requirement in your application... For that application, I'd recommend to use ASN.1 BER as a well-document, efficient, binary marshalling format. I don't think any other format marshals arbitrary large integers in a more compact form. You can find an implementation of that in http://www.cnri.reston.va.us/software/pisces/ or http://www.enteract.com/~asl2/software/PyZ3950/asn1.py or http://sourceforge.net/projects/pysnmp (ber.py) I'd be in favour of having a BER support library in the Python core, but somebody would have to contribute such a thing, of course. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 20:43 Message: Logged In: YES user_id=31435 The marshal long format actually uses 15-bit digits, each *stored* in 16 bits (the high bit of the high byte of which is always 0). That would be a PITA to preserve even if Python just moved to 16-bit digits. marshal's purpose is for efficient loading of .pyc files, where that odd format makes good sense; since it wasn't designed to be a general- purpose data transmission format (and has many shortcomings for such use), I don't want to see a tail wagging the dog here. Cross-release compatibility is taken seriously in pickle, and pickle handles many more cases than marshal, although pickle's author (as you've discovered) didn't give a hoot about efficient storage of longs. I'd rather add an efficient long format to pickle than hobble marshal (although because pickle does take x-release compatibility seriously, it has to continue accepting the "longs as decimal strings" format forever). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 20:08 Message: Logged In: YES user_id=72053 Skip - C has struct objects which are sort of like Python dictionaries. XMLRPC represents structs as name-value pairs, for example. And "other languages" doesn't necessarily mean C. The marshaller should be able to represent the non-Python-specific serializable objects, not just scalars. Basically this means strings, integers (of any length), dictionaries, lists, and floats (hmm--unicode?), but not necessarily stuff like code objects. Having an independent marshal library is ok, I guess, though I don't feel it's necessary to create more implementation work. And one the benefit of using the existing marshaller is that it's already available in most versions of Python that people are running (Red Hat 7.1 still comes with Python 1.5 for example). Tim - yes, I'm originally used a binascii.hexlify hack similar to yours and it worked ok, but it was ugly. I also had to handle strings (generate a length count followed by the contents) and then dictionaries (name-value pairs) and finally felt I shouldn't need to rewrite the marshaller like that. There's already a built-in library function that does everything I need, very efficiently in native code, in one call, and being able to use it is in the "batteries included" spirit. Also, the current long int marshalling format is just a digit count (16-bit digits) followed by the digits in binary. If the digit width changes, the marshalling format doesn't have to change--the marshalling code should still be able to use the same external representation without excessive contortions and without slowing down. (You'll see that it's already not a simple memory dump, but a structure read and written one byte at a time through layers of subroutines). Changing widths while keeping the old format means putting a minor kludge in the marshalling code, but no user will ever notice it. As for the speed of Python longs, my stuff's runtime is dominated by modular exponentiations and I'm already using gmpy for those when it's available (but I don't depend on it). The speedup with gmpy is substantial, but the speed with ordinary Python longs is quite acceptable on my PIII (the StrongARM is another story--probably the C compiler's fault). Examining Python/marshal.c, I don't see any objects of the types I've mentioned that are likely to need to change representations--do you? Btw I notice that the pickle module represents long ints as decimal strings even in "binary" mode, but I'll resist opening another bug for that, for now. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-12 14:41 Message: Logged In: YES user_id=44345 If you head in the direction of documenting marshal with the aim of potentially interoperating with other languages, I think it would be a good idea to create a Python-independent marshal library. This would facilitate incorporation into other languages. Such a library probably wouldn't be able to do everything marshal can (there isn't an obvious C equivalent of Python's dictionary object, for example), but would still help nail down compatibility issues for the basic scalar types. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-12 14:09 Message: Logged In: YES user_id=31435 I'm not sure this is making progress. Paul, if you want to use marshal, you already can: the pack and unpack routines are exposed in Python via the marshal module. Freezing the representation isn't a particularly appealing idea; e.g., if anyone is likely to complain about the speed of Python's longs, it's you , and the current marshal format for longs is just a raw dump of Python's internal long representation -- but the most obvious everything-benefits way to speed Python longs is to increase the size of the "digits" used in its internal representation. If that's ever done, the marshal format would want to change too. It's easy enough to code your own storage format for longs, e.g. >>> def tobin(i): ... import binascii ... ashex = hex(long(i))[2:-1] # chop '0x' and trailing 'L' ... if len(ashex) & 1: ... ashex = '0' + ashex ... return binascii.unhexlify(ashex) implements "base 256" for unsigned longs, and the runtime cannot be improved by rewriting in C except by a constant factor (the Python spelling has the right O() behavior). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 13:24 Message: Logged In: YES user_id=6380 This helps tremendously. I think that marshal is probably overkill. Rather, you need helper routines to convert longs to and from binary. You can do everything else using the struct module, and it's probably easier to write your own protocol using that and these helpers. I suggest that the best place to add these helpers is the binascii module, which already has a bunch of similar things (e.g. hexlify and crc32). Note the xmlrpc is bundled with Python 2.2. Looking forward to your patch (much simpler to get accepted than a PEP :-). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 13:16 Message: Logged In: YES user_id=72053 Decimal is bad not just because of the data expansion but because the arithmetic to convert a decimal string to binary can be expensive (all that multiplication). I'd rather use hex than decimal for that reason. One envisioned application is communicating a cryptography coprocessor: an 8-bit microcontroller (with a public key accelerator) connected to the host computer through a slow serial port. Most of the ints involved would be around 300 decimal digits. A simple binary format is a lot easier to deal with in that environment than something like xmlrpc. Also, the format would be used for data persistence, so again, unnecessary data expansion isn't desirable. I looked at XMLRPC and it's not designed for this purpose. It's intended as an RPC protocol over HTTP and isn't well suited for object persistence. Also, it doesn't support integers over 32 bits, and binary strings must be base64 encoded (more bloat). Finally, it's not included with Python, so I'd have to bundle an implementation written in Python (i.e. slow) with my application (I don't know whether Fred's implementation is Python or C). I think the marshal format hasn't changed since before Python 1.5, so basing serialization on marshal would mean applications could interoperate with older versions of Python as well as newer ones, which helps Python's maturity. (Maturity of a program means, among other things, that users rarely need to be told they need the latest version in order to use some feature). Really, the marshal functions are written the way they're written because that's the simplest and most natural way of doing this kind of thing. So the proposal is mainly to make them available for user applications, rather than only for system internals. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 12:37 Message: Logged In: YES user_id=6380 If the PEP makes a reasonable case for freezing the spec, yes. I wonder why you can't use decimal? Are you talking really large volumes? The PEP needs to motivate this with an example, preferably plucked from real life! ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-12 12:29 Message: Logged In: YES user_id=72053 I just want to be able to do convenient transfers of python data to other programs including over the network. XMLRPC is excessive bloat in my opinion. Sending a number like 12345678 should take at most 5 bytes (a type byte and a 4-byte int) instead of 12345678. For long ints (300 digits) it's even worse. The marshal format is fine, and writing a PEP would solve the doc problem, but the current marshal doc says the non-specification is intentional. Writing it in a PEP means not just documenting--it means asking the language maintainers to freeze the marshal format of certain types, instead of reserving the right to change the format in future versions. Writing the PEP only makes sense if you're willing to freeze the format for those types (the other types can stay undocumented). Is that ok with you? Thanks Paul ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 07:33 Message: Logged In: YES user_id=6380 Paul, I don't understand the application that you are envisioning. If you think that the marshal format is what you want, why don't you write a PEP that specifies the format? That would solve the documentation problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-12 02:39 Message: Logged In: YES user_id=21627 Well, then I guess you need to specify your requirements more clearly. XML-RPC was precisely developed to be something simple for primitive types and structures that is sufficiently well-specified to allow interoperation between various languages. I don't see why extending the data 'by an order of magnitude' would be a problem per se, nor do I see why 'requiring a complicated parser' is a problem if the implementation already does all the unpacking for you under the hoods. Furthermore, I believe it is simply not true that XML-RPC expands the representation by an order of magnitude. For example, the Python Integer object 1 takes 12 bytes in its internal representation (plus the overhead that malloc requires); the XML-RPC representation '1' also uses 12 bytes. In short, you need to say as precise as possible what it is that you want, or you won't get it. Also, it may be that you have conflicting requirements (e.g. 'compact, binary', and 'simple, easily processible in different languages'); then you won't get it either. For a marshalling format that is accessible from different languages, you better specify it first, and implement it then. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-11 22:12 Message: Logged In: YES user_id=72053 I haven't looked at xmlrpclib, but I'm looking for a simple, compact, binary representation, not something that needs a complicated parser and expands the data by an order of magnitude. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 17:10 Message: Logged In: YES user_id=21627 So what's wrong with xmlrpclib? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467384&group_id=5470 From noreply@sourceforge.net Wed Oct 17 01:26:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 17:26:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) >Assigned to: Guido van Rossum (gvanrossum) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 17:26 Message: Logged In: YES user_id=6380 OK. It would be a great help if you could upload a patch (a context diff) relative to the current state of CVS, or the 2.2a4 release if you must. When uploading, don't forget to check the file upload checkbox. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Wed Oct 17 02:35:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 18:35:14 -0700 Subject: [Python-bugs-list] [ python-Bugs-471928 ] global made w/nested list comprehensions Message-ID: Bugs item #471928, was opened at 2001-10-16 18:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: global made w/nested list comprehensions Initial Comment: This bug affects Python 2.1.1 & 2.2a4+. In nested list comprehension code like this: [bad for s in 'a b' for bad in s.split()] the variable bad becomes a global, but it should be local variable. The attached file shows the disassembled byte code, along with a test case. The original source of the error was found from cgi.py:209 in the standard library: pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')] Neal ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 From noreply@sourceforge.net Wed Oct 17 03:56:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 19:56:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Nobody/Anonymous (nobody) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Wed Oct 17 07:06:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 23:06:44 -0700 Subject: [Python-bugs-list] [ python-Bugs-471928 ] global made w/nested list comprehensions Message-ID: Bugs item #471928, was opened at 2001-10-16 18:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Thomas Wouters (twouters) Summary: global made w/nested list comprehensions Initial Comment: This bug affects Python 2.1.1 & 2.2a4+. In nested list comprehension code like this: [bad for s in 'a b' for bad in s.split()] the variable bad becomes a global, but it should be local variable. The attached file shows the disassembled byte code, along with a test case. The original source of the error was found from cgi.py:209 in the standard library: pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')] Neal ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 23:06 Message: Logged In: YES user_id=6380 Good catch! I don't know the list comprehension code generation, and I don't know who does. Maybe Thomas Wouters? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 From noreply@sourceforge.net Wed Oct 17 07:47:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 16 Oct 2001 23:47:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-467430 ] CGIHTTPServer - 2 issues and/or bugs Message-ID: Bugs item #467430, was opened at 2001-10-03 01:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467430&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) Summary: CGIHTTPServer - 2 issues and/or bugs Initial Comment: 1) First, the easy one. I believe the following self.error_message calls should be replaced with self.log_message calls ... since no error actually has occured. *** /opt/arseed/tfs-lib/lib/python2.1/CGIHTTPServer.py Tue Oct 2 20:35:04 2001 --- /home/norton/CGIHTTPServer.py Wed Oct 3 17:07:05 2001 *************** *** 227,233 **** cmdline = "%s %s" % (interp, cmdline) if '=' not in query and '"' not in query: cmdline = '%s "%s"' % (cmdline, query) ! self.log_error("command: %s", cmdline) try: nbytes = int(length) except: --- 227,233 ---- cmdline = "%s %s" % (interp, cmdline) if '=' not in query and '"' not in query: cmdline = '%s "%s"' % (cmdline, query) ! self.log_message("command: %s", cmdline) try: nbytes = int(length) except: *************** *** 242,248 **** if sts: self.log_error("CGI script exit status %#x", sts) else: ! self.log_error("CGI script exited OK") else: # Other O.S. -- execute script in this process --- 242,248 ---- if sts: self.log_error("CGI script exit status %#x", sts) else: ! self.log_message("CGI script exited OK") else: # Other O.S. -- execute script in this process *************** *** 267,273 **** except SystemExit, sts: self.log_error("CGI script exit status %s", str(sts)) else: ! self.log_error("CGI script exited OK") nobody = None --- 267,273 ---- except SystemExit, sts: self.log_error("CGI script exit status %s", str(sts)) else: ! self.log_message("CGI script exited OK") nobody = None 2) Secondly, I have created my own CGIHTTP handler class that re-uses the CGHITTPServer CGIHTTPRequestHandler class. I have modified Zope's ZPublisher.Client code to redirect file:// urls to this CGIHTTP handler so that external cgi executables can be proxied by zope. I have attached my handler to this request (only if you are curious). Anyway, I feel the following patch should be applied to the standard python lib. I faced troubles with os.fork, os.dup2, and reading from self.rfile in the child process if the self.rfile is not flushed as well prior to forking. *** Python-2.1.1/Lib/CGIHTTPServer.py Sat Sep 29 15:52:51 2001 --- Python-2.1.1/Lib/CGIHTTPServer.py Tue Oct 2 19:55:14 2001 *************** *** 192,197 **** --- 192,198 ---- if '=' not in decoded_query: args.append(decoded_query) nobody = nobody_uid() + self.rfile.flush() # Always flush before forking self.wfile.flush() # Always flush before forking pid = os.fork() if pid != 0: ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 23:47 Message: Logged In: YES user_id=6380 Thanks! Applied as rev 1.19. You need an SF account and log in to upload attachments. Also, next time please upload the patches as attachments instead of inlining them. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-03 01:13 Message: Logged In: NO The experimental handler attachment that I mentioned in the above message was rejected while adding the bug posting. If you would like to look at the code, please contact me (norton@alum.mit.edu) directly. thanks, - joe n. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=467430&group_id=5470 From noreply@sourceforge.net Wed Oct 17 08:14:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 00:14:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-417796 ] illegal version 1.0.2pre Message-ID: Bugs item #417796, was opened at 2001-04-20 21:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417796&group_id=5470 Category: Distutils Group: None >Status: Closed Resolution: Fixed Priority: 5 Submitted By: Garth T Kidd (gtk) Assigned to: Thomas Heller (theller) Summary: illegal version 1.0.2pre Initial Comment: Description: The version in distutils/__init__ doesn't conform to the strict versioning guidelines in distutils/version, resulting in failure when trying to use distutils to install 4Suite (and perhaps other modules). Workaround: Hand-edit distutils/__init__.py to remove "pre". Pointers to vital stuff in code: Lib/distutils/__init__.py: __version__ = "1.0.2pre" Lib/distutils/version.py: A version number consists of two or three dot- separated numeric components, with an optional "pre- release" tag on the end. The pre-release tag consists of the letter 'a' or 'b' followed by a number. Demonstration of problem: Python 2.1 (#1, Apr 18 2001, 10:49:25) [GCC 2.95.2 19991024 (release)] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import distutils >>> import distutils.version >>> v = distutils.version.StrictVersion (distutils.__version__) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/distutils/version.py", line 42, in __init__ self.parse(vstring) File "/usr/local/lib/python2.1/distutils/version.py", line 109, in parse raise ValueError, "invalid version number '%s'" % vstring ValueError: invalid version number '1.0.2pre' >>> ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2001-09-28 03:56 Message: Logged In: YES user_id=11105 Seems better to close this bug, since it is fixed. Iff a python version 2.1.2 is released, someone will have to remember this one... ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2001-09-05 12:00 Message: Logged In: YES user_id=11105 I didn't close it because it should have been fixed in Python 2.1.1, just in case there will be a 2.1.2 release, where it should be fixed. Shall I close it? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:14 Message: Logged In: YES user_id=6380 Why is this not closed? ---------------------------------------------------------------------- Comment By: Garth T Kidd (gtk) Date: 2001-07-29 23:15 Message: Logged In: YES user_id=59803 Fixed in distutils 1.20 tagged for release with Python 2.2, meaning 4Suite et al won't install cleanly with any out-of- the-box Python 2.1. :/ If there's ever a 2.1.2, anyone want to tag this for it? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=417796&group_id=5470 From noreply@sourceforge.net Wed Oct 17 09:34:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 01:34:02 -0700 Subject: [Python-bugs-list] [ python-Bugs-472004 ] socket.makefile() missing link to open() Message-ID: Bugs item #472004, was opened at 2001-10-17 01:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472004&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: socket.makefile() missing link to open() Initial Comment: the socket.makefile() docs refer to the arguments of "the builtin open() function", but there is no hyperlink. it would be convenient. -- erno@iki.fi ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472004&group_id=5470 From noreply@sourceforge.net Wed Oct 17 09:56:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 01:56:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-472007 ] Linker problem on HP-UX 11.00 Message-ID: Bugs item #472007, was opened at 2001-10-17 01:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Nobody/Anonymous (nobody) Summary: Linker problem on HP-UX 11.00 Initial Comment: I use the following command to run the configure script: CC=cc \ OPT="+DAportable" \ INSTALL="./install-sh -c" \ ./configure --with-threads --without-gcc \ --with-cxx=/opt/aCC/bin/aCC --without-libd \ --with-cycle-gc --prefix=/opt/python \ --exec-prefix=/opt/python In the Makefile I get the following lines: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CC) The build then fails with: /usr/ccs/bin/ld: Unsatisfied symbols: _main (first referenced in Modules/ccpython.o) (code) *** Error exit code 1 If I edit the Makefile, thus: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CXX) The builds succeeds. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 From noreply@sourceforge.net Wed Oct 17 10:10:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 02:10:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Guido van Rossum (gvanrossum) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-17 02:10 Message: Logged In: YES user_id=350896 This is a context diff of a proposal for a patch relative to the current state of CVS. I have to admit that I tested this only with 2.2a4 and not with the current CVS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 17:26 Message: Logged In: YES user_id=6380 OK. It would be a great help if you could upload a patch (a context diff) relative to the current state of CVS, or the 2.2a4 release if you must. When uploading, don't forget to check the file upload checkbox. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Wed Oct 17 13:36:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 05:36:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-472007 ] Linker problem on HP-UX 11.00 Message-ID: Bugs item #472007, was opened at 2001-10-17 01:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Martin v. Löwis (loewis) Summary: Linker problem on HP-UX 11.00 Initial Comment: I use the following command to run the configure script: CC=cc \ OPT="+DAportable" \ INSTALL="./install-sh -c" \ ./configure --with-threads --without-gcc \ --with-cxx=/opt/aCC/bin/aCC --without-libd \ --with-cycle-gc --prefix=/opt/python \ --exec-prefix=/opt/python In the Makefile I get the following lines: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CC) The build then fails with: /usr/ccs/bin/ld: Unsatisfied symbols: _main (first referenced in Modules/ccpython.o) (code) *** Error exit code 1 If I edit the Makefile, thus: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CXX) The builds succeeds. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-17 05:36 Message: Logged In: YES user_id=6380 Martin, do you understand the logic in configre.in enough to suggest a patch? If not, please unassign it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 From noreply@sourceforge.net Wed Oct 17 14:08:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 06:08:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Nobody/Anonymous (nobody) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Wed Oct 17 14:29:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 06:29:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-472007 ] Linker problem on HP-UX 11.00 Message-ID: Bugs item #472007, was opened at 2001-10-17 01:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Nobody/Anonymous (nobody) Summary: Linker problem on HP-UX 11.00 Initial Comment: I use the following command to run the configure script: CC=cc \ OPT="+DAportable" \ INSTALL="./install-sh -c" \ ./configure --with-threads --without-gcc \ --with-cxx=/opt/aCC/bin/aCC --without-libd \ --with-cycle-gc --prefix=/opt/python \ --exec-prefix=/opt/python In the Makefile I get the following lines: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CC) The build then fails with: /usr/ccs/bin/ld: Unsatisfied symbols: _main (first referenced in Modules/ccpython.o) (code) *** Error exit code 1 If I edit the Makefile, thus: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CXX) The builds succeeds. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 06:29 Message: Logged In: YES user_id=21627 LINKCC already was set to CXX before (configure.in 1.113); that was undone later (1.198). Unfortunately, none of these checkins indicate why they have been done; I recall that people where complaining that python requires libstdc++ on Linux. I've now designed a test that checks whether CXX must be used to link if main() was compiled with CXX. I cannot test it on a system that has this property, so I'll attach it to this report. When patching configure fails for you, please re-run autoconf. If you don't have autoconf, please fix the two line numbers in the patch to match your configure; the patch should apply then. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-17 05:36 Message: Logged In: YES user_id=6380 Martin, do you understand the logic in configre.in enough to suggest a patch? If not, please unassign it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 From noreply@sourceforge.net Wed Oct 17 14:31:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 06:31:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) >Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Wed Oct 17 15:03:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 07:03:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-459767 ] ftplib fails with files > 2GB Message-ID: Bugs item #459767, was opened at 2001-09-08 03:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 Category: Python Library Group: None >Status: Open Resolution: Fixed Priority: 2 Submitted By: Martin Preishuber (preisl) Assigned to: Guido van Rossum (gvanrossum) Summary: ftplib fails with files > 2GB Initial Comment: The ftplib.py module is not ready for large files > 2GB. The funtions size and parse150 return integer values as file size which causes overflows on large files. ---------------------------------------------------------------------- >Comment By: Martin Preishuber (preisl) Date: 2001-10-17 07:03 Message: Logged In: YES user_id=29575 I just tried it ... the patch is basically ok, but int(s) raises an ValueError, not an OverflowError with Python 2.1.1: ValueError: int() literal too large: 2720628736 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:46 Message: Logged In: YES user_id=6380 I've fixed this in CVS. ftplib.py rev 1.59. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:13 Message: Logged In: YES user_id=6380 Can you verify that this patch works for you? (I don't have a server with files > 2GB lying around, and my network would take days to download them anyway. ;-( ) See the uploaded file "ftplib.patch.txt" ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-10-16 11:03 Message: Logged In: YES user_id=29575 These changes are necessary: For Python 1.*: replace atoi by atol in size() and parse150() For Python 2.*: replace int by long in size() and parse150() ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-10 06:36 Message: Logged In: YES user_id=6380 Please upload your patches and I'll see... ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-09-08 04:14 Message: Logged In: YES user_id=29575 well, I've written emirror, some ftp mirroring software. There are people, who use it to make backups of such large files. I already use some modified ftp library, which fixes some bugs in the 1.52 library (those bugs are included in >2.0 versions) and I have made the necessary changes for large file support myself ... I just thought it'd be nice to have that support generally. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-08 04:08 Message: Logged In: YES user_id=6380 Do you have an actual use for this, or it this just a theoretical complaint? Show me an FTP server with a file > 2GB and I'll consider fixing it. In the future (Python 2.3), int() will auto-convert to long if necessary -- see PEP 237. So this will be fixed eventually anyway -- probably long before such large files will become an issue. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 From noreply@sourceforge.net Wed Oct 17 16:25:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 08:25:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-472007 ] Linker problem on HP-UX 11.00 Message-ID: Bugs item #472007, was opened at 2001-10-17 01:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Richard Townsend (rptownsend) Assigned to: Nobody/Anonymous (nobody) Summary: Linker problem on HP-UX 11.00 Initial Comment: I use the following command to run the configure script: CC=cc \ OPT="+DAportable" \ INSTALL="./install-sh -c" \ ./configure --with-threads --without-gcc \ --with-cxx=/opt/aCC/bin/aCC --without-libd \ --with-cycle-gc --prefix=/opt/python \ --exec-prefix=/opt/python In the Makefile I get the following lines: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CC) The build then fails with: /usr/ccs/bin/ld: Unsatisfied symbols: _main (first referenced in Modules/ccpython.o) (code) *** Error exit code 1 If I edit the Makefile, thus: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CXX) The builds succeeds. ---------------------------------------------------------------------- >Comment By: Richard Townsend (rptownsend) Date: 2001-10-17 08:25 Message: Logged In: YES user_id=200117 I applied your patch on HP-UX 11.00 and it modified configure and configure.in without errors. When I ran configure, Makefile and Makefile.pre still contain: LINKCC= $(PURIFY) $(CC) Just in case, I ran autoconf and then ran configure again, but got the same result. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 06:29 Message: Logged In: YES user_id=21627 LINKCC already was set to CXX before (configure.in 1.113); that was undone later (1.198). Unfortunately, none of these checkins indicate why they have been done; I recall that people where complaining that python requires libstdc++ on Linux. I've now designed a test that checks whether CXX must be used to link if main() was compiled with CXX. I cannot test it on a system that has this property, so I'll attach it to this report. When patching configure fails for you, please re-run autoconf. If you don't have autoconf, please fix the two line numbers in the patch to match your configure; the patch should apply then. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-17 05:36 Message: Logged In: YES user_id=6380 Martin, do you understand the logic in configre.in enough to suggest a patch? If not, please unassign it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 From noreply@sourceforge.net Wed Oct 17 18:23:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 10:23:37 -0700 Subject: [Python-bugs-list] [ python-Bugs-459767 ] ftplib fails with files > 2GB Message-ID: Bugs item #459767, was opened at 2001-09-08 03:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 Category: Python Library Group: None >Status: Closed Resolution: Fixed Priority: 2 Submitted By: Martin Preishuber (preisl) Assigned to: Guido van Rossum (gvanrossum) Summary: ftplib fails with files > 2GB Initial Comment: The ftplib.py module is not ready for large files > 2GB. The funtions size and parse150 return integer values as file size which causes overflows on large files. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-17 10:23 Message: Logged In: YES user_id=6380 Oops, you're right. I've changed OverflowError into (OverflowError, ValueError) in CVS now. Closing. ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-10-17 07:03 Message: Logged In: YES user_id=29575 I just tried it ... the patch is basically ok, but int(s) raises an ValueError, not an OverflowError with Python 2.1.1: ValueError: int() literal too large: 2720628736 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:46 Message: Logged In: YES user_id=6380 I've fixed this in CVS. ftplib.py rev 1.59. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 12:13 Message: Logged In: YES user_id=6380 Can you verify that this patch works for you? (I don't have a server with files > 2GB lying around, and my network would take days to download them anyway. ;-( ) See the uploaded file "ftplib.patch.txt" ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-10-16 11:03 Message: Logged In: YES user_id=29575 These changes are necessary: For Python 1.*: replace atoi by atol in size() and parse150() For Python 2.*: replace int by long in size() and parse150() ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-10 06:36 Message: Logged In: YES user_id=6380 Please upload your patches and I'll see... ---------------------------------------------------------------------- Comment By: Martin Preishuber (preisl) Date: 2001-09-08 04:14 Message: Logged In: YES user_id=29575 well, I've written emirror, some ftp mirroring software. There are people, who use it to make backups of such large files. I already use some modified ftp library, which fixes some bugs in the 1.52 library (those bugs are included in >2.0 versions) and I have made the necessary changes for large file support myself ... I just thought it'd be nice to have that support generally. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-08 04:08 Message: Logged In: YES user_id=6380 Do you have an actual use for this, or it this just a theoretical complaint? Show me an FTP server with a file > 2GB and I'll consider fixing it. In the future (Python 2.3), int() will auto-convert to long if necessary -- see PEP 237. So this will be fixed eventually anyway -- probably long before such large files will become an issue. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459767&group_id=5470 From noreply@sourceforge.net Wed Oct 17 20:25:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 12:25:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-472202 ] dlerror()-messages bogus on netbsd Message-ID: Bugs item #472202, was opened at 2001-10-17 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Audun Arnesen Nordal (audun) Assigned to: Nobody/Anonymous (nobody) Summary: dlerror()-messages bogus on netbsd Initial Comment: In Python/dynload_shlib.c, a preprocessor check is done to determine if the 'NetBSD' identifier predates some value (199712) on NetBSD platforms. However, this identifier is defined in the file which is not included, and dlerror() messages are thus mistakingly hardcoded to "error in dynamic linking" on all NetBSD platforms. Also, recent ELF-handling added targeted for OpenBSD applies to NetBSD as well. This bug has been handled in the NetBSD package tree for some time (patch committed there by Matthias Drochner), but I couldn't find it reported in Python's buglist and the bug is present in Python's most recent cvs. Patch enclosed. Best regards, Audun Nordal ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 From noreply@sourceforge.net Wed Oct 17 20:31:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 12:31:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-472202 ] dlerror()-messages bogus on netbsd Message-ID: Bugs item #472202, was opened at 2001-10-17 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Audun Arnesen Nordal (audun) Assigned to: Nobody/Anonymous (nobody) Summary: dlerror()-messages bogus on netbsd Initial Comment: In Python/dynload_shlib.c, a preprocessor check is done to determine if the 'NetBSD' identifier predates some value (199712) on NetBSD platforms. However, this identifier is defined in the file which is not included, and dlerror() messages are thus mistakingly hardcoded to "error in dynamic linking" on all NetBSD platforms. Also, recent ELF-handling added targeted for OpenBSD applies to NetBSD as well. This bug has been handled in the NetBSD package tree for some time (patch committed there by Matthias Drochner), but I couldn't find it reported in Python's buglist and the bug is present in Python's most recent cvs. Patch enclosed. Best regards, Audun Nordal ---------------------------------------------------------------------- >Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 12:31 Message: Logged In: YES user_id=47184 Grrr, my browser did something funny with the diff-file. In verbatim below. Audun --- Python/dynload_shlib.c Wed Oct 17 21:05:08 2001 +++ Python/dynload_shlib.c.orig Wed Oct 17 20:58:42 2001 @@ -6,9 +6,7 @@ #include #include -#if defined(__NetBSD__) -#include -#if (NetBSD < 199712) +#if defined(__NetBSD__) && (NetBSD < 199712) #include #include #define dlerror() "error in dynamic linking" @@ -17,9 +15,8 @@ #include #endif #endif -#endif /* NetBSD */ -#if ((defined(__OpenBSD__) || defined(__NetBSD__)) && !defined(__ELF__) +#if defined(__OpenBSD__) && !defined(__ELF__) #define LEAD_UNDERSCORE "_" #else #define LEAD_UNDERSCORE "" ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 From noreply@sourceforge.net Wed Oct 17 21:02:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 13:02:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-472202 ] dlerror()-messages bogus on netbsd Message-ID: Bugs item #472202, was opened at 2001-10-17 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Audun Arnesen Nordal (audun) Assigned to: Nobody/Anonymous (nobody) Summary: dlerror()-messages bogus on netbsd Initial Comment: In Python/dynload_shlib.c, a preprocessor check is done to determine if the 'NetBSD' identifier predates some value (199712) on NetBSD platforms. However, this identifier is defined in the file which is not included, and dlerror() messages are thus mistakingly hardcoded to "error in dynamic linking" on all NetBSD platforms. Also, recent ELF-handling added targeted for OpenBSD applies to NetBSD as well. This bug has been handled in the NetBSD package tree for some time (patch committed there by Matthias Drochner), but I couldn't find it reported in Python's buglist and the bug is present in Python's most recent cvs. Patch enclosed. Best regards, Audun Nordal ---------------------------------------------------------------------- >Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 13:02 Message: Logged In: YES user_id=47184 Right. I got the diff the wrong way around, plus there's a stray paranthesis *blush*. Sorry about that. Audun ---------------------------------------------------------------------- Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 12:31 Message: Logged In: YES user_id=47184 Grrr, my browser did something funny with the diff-file. In verbatim below. Audun --- Python/dynload_shlib.c Wed Oct 17 21:05:08 2001 +++ Python/dynload_shlib.c.orig Wed Oct 17 20:58:42 2001 @@ -6,9 +6,7 @@ #include #include -#if defined(__NetBSD__) -#include -#if (NetBSD < 199712) +#if defined(__NetBSD__) && (NetBSD < 199712) #include #include #define dlerror() "error in dynamic linking" @@ -17,9 +15,8 @@ #include #endif #endif -#endif /* NetBSD */ -#if ((defined(__OpenBSD__) || defined(__NetBSD__)) && !defined(__ELF__) +#if defined(__OpenBSD__) && !defined(__ELF__) #define LEAD_UNDERSCORE "_" #else #define LEAD_UNDERSCORE "" ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 From noreply@sourceforge.net Wed Oct 17 21:37:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 13:37:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-472202 ] dlerror()-messages bogus on netbsd Message-ID: Bugs item #472202, was opened at 2001-10-17 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Audun Arnesen Nordal (audun) >Assigned to: Martin v. Löwis (loewis) Summary: dlerror()-messages bogus on netbsd Initial Comment: In Python/dynload_shlib.c, a preprocessor check is done to determine if the 'NetBSD' identifier predates some value (199712) on NetBSD platforms. However, this identifier is defined in the file which is not included, and dlerror() messages are thus mistakingly hardcoded to "error in dynamic linking" on all NetBSD platforms. Also, recent ELF-handling added targeted for OpenBSD applies to NetBSD as well. This bug has been handled in the NetBSD package tree for some time (patch committed there by Matthias Drochner), but I couldn't find it reported in Python's buglist and the bug is present in Python's most recent cvs. Patch enclosed. Best regards, Audun Nordal ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:37 Message: Logged In: YES user_id=21627 The patch looks fine to me. ---------------------------------------------------------------------- Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 13:02 Message: Logged In: YES user_id=47184 Right. I got the diff the wrong way around, plus there's a stray paranthesis *blush*. Sorry about that. Audun ---------------------------------------------------------------------- Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 12:31 Message: Logged In: YES user_id=47184 Grrr, my browser did something funny with the diff-file. In verbatim below. Audun --- Python/dynload_shlib.c Wed Oct 17 21:05:08 2001 +++ Python/dynload_shlib.c.orig Wed Oct 17 20:58:42 2001 @@ -6,9 +6,7 @@ #include #include -#if defined(__NetBSD__) -#include -#if (NetBSD < 199712) +#if defined(__NetBSD__) && (NetBSD < 199712) #include #include #define dlerror() "error in dynamic linking" @@ -17,9 +15,8 @@ #include #endif #endif -#endif /* NetBSD */ -#if ((defined(__OpenBSD__) || defined(__NetBSD__)) && !defined(__ELF__) +#if defined(__OpenBSD__) && !defined(__ELF__) #define LEAD_UNDERSCORE "_" #else #define LEAD_UNDERSCORE "" ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 From noreply@sourceforge.net Wed Oct 17 21:44:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 13:44:01 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Wed Oct 17 21:45:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 13:45:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-472234 ] type(obj) calls type->tp_init Message-ID: Bugs item #472234, was opened at 2001-10-17 13:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472234&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Roeland Rengelink (rengelink) Assigned to: Guido van Rossum (gvanrossum) Summary: type(obj) calls type->tp_init Initial Comment: In 2.2a4 Consider: class Meta(type): def __init__(self, *args): print 'Meta.__init__ called' class Object(object): __metaclass__ = Meta obj = Object() print type(obj) will print Meta.__init__ called Meta.__init__ called The second call to Meta.__init__ results from the type(obj) call. This is because type(obj) is special cased in type_new(...) rather than type_call(...). Although it's rather useless to define an __init__ in a class derived from type (for metaclasses it's a better idea to override __new__). I do think the call of Meta.__init__ in type(obj) is a bug. Thanks, Roeland ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472234&group_id=5470 From noreply@sourceforge.net Thu Oct 18 02:22:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 18:22:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-471928 ] global made w/nested list comprehensions Message-ID: Bugs item #471928, was opened at 2001-10-16 18:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Jeremy Hylton (jhylton) Summary: global made w/nested list comprehensions Initial Comment: This bug affects Python 2.1.1 & 2.2a4+. In nested list comprehension code like this: [bad for s in 'a b' for bad in s.split()] the variable bad becomes a global, but it should be local variable. The attached file shows the disassembled byte code, along with a test case. The original source of the error was found from cgi.py:209 in the standard library: pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')] Neal ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-17 18:22 Message: Logged In: YES user_id=31435 Reassigned to Jeremy, as I expect he's got a better chance of understanding why the symbol-table code is calling one of these local and the other global (listcomp codegen treats them the same). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 23:06 Message: Logged In: YES user_id=6380 Good catch! I don't know the list comprehension code generation, and I don't know who does. Maybe Thomas Wouters? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 From noreply@sourceforge.net Thu Oct 18 05:58:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 21:58:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 21:58 Message: Logged In: YES user_id=29957 Ok, I have an intact core file, and a matching binary, no optimisations, nothing. This crash is showing the crash at line 166 of gcmodule.c traverse = PyObject_FROM_GC(gc)->ob_type->tp_traverse; PyObject_FROM_GC(gc)->ob_type in this case is $24 = {ob_refcnt = 1, ob_type = 0x0} To check my logic, I checked gc_next and gc_prev using the same GDB magic, and they correctly show up as a tuple and an instance method. Some fiddling around seems to rule out stack space as the problem, as well. We're going to try and see if purify helps here, but the problem looks to be a junk object - I have no idea how to track this down further. Help? Would taking the horrible horrible hack of removing the object from the gc linked list if ob_type is null help? Well, it'd stop the crashes, anyway. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Thu Oct 18 07:54:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 17 Oct 2001 23:54:05 -0700 Subject: [Python-bugs-list] [ python-Bugs-472347 ] pydoc and properties Message-ID: Bugs item #472347, was opened at 2001-10-17 23:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472347&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Roeland Rengelink (rengelink) Assigned to: Nobody/Anonymous (nobody) Summary: pydoc and properties Initial Comment: pydoc in server mode throws an error when parsing the attached module. From the command line documentation is shown correctly. The problem seems to be in HTMLDoc.spillproperties(). Replacing: base = self.document(func, name + tag, mod, funcs, classes, mdict, object) with: base = self.document(func, name+tag, mod) seems to solve this. This is also more in line with TextDoc.spillproperties() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472347&group_id=5470 From noreply@sourceforge.net Thu Oct 18 11:11:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 03:11:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:11 Message: Logged In: YES user_id=21627 There are two options: a) the object isn't really a GC object, i.e. has no GC header. In gdb, you can try to cast gc to PyObject*, then see if the resulting pointer has a better ob_type (this is unlikely, though, since the logic entering the object was already using fromgc/togc) b) somebody has cleared the ob_type field. Can you guarantee that all extension modules have been compiled with the 2.1.1 header files? Is the problem repeatable in the sense that gc will have the same pointer value on each crash? If so, it is relatively easy to track down: just set a gdb change watchpoint on the address on the ob_type field of that address (note that setting watchpoints is not possible until there is really mapped memory on that address). If you can't analyse it through change breakpoints, I recommend to annotate the interpreter in the following way: in pyobject_init, put a printf that prints the address and the tp_name of the type. In subtract_refs, if the ob_type slot is null, print the address of the object and abort. Then analyse the log to see whether a object really has been allocated on that address, and what its type was (make sure you consider the possibility that address are off by the delta that FROM_GC adds). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 21:58 Message: Logged In: YES user_id=29957 Ok, I have an intact core file, and a matching binary, no optimisations, nothing. This crash is showing the crash at line 166 of gcmodule.c traverse = PyObject_FROM_GC(gc)->ob_type->tp_traverse; PyObject_FROM_GC(gc)->ob_type in this case is $24 = {ob_refcnt = 1, ob_type = 0x0} To check my logic, I checked gc_next and gc_prev using the same GDB magic, and they correctly show up as a tuple and an instance method. Some fiddling around seems to rule out stack space as the problem, as well. We're going to try and see if purify helps here, but the problem looks to be a junk object - I have no idea how to track this down further. Help? Would taking the horrible horrible hack of removing the object from the gc linked list if ob_type is null help? Well, it'd stop the crashes, anyway. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Thu Oct 18 11:31:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 03:31:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:31 Message: Logged In: YES user_id=29957 It's not a GC object. I'm positive all the extension objects are correct - I just recompiled, without the 1.5/2.0 headers around. It's a different pointer each time round, unfortunately. It also takes anything from 5 minutes to 2 hours to reproduce. I've got about 4 copies of it running now, and I've got a bunch of different core files. I've grabbed purify and an eval license, and I'm feeding it the binary. The printf approach is probably not going to work - these are busy busy Zope servers. Instead, my plan, assuming that purify doesn't immediately spot a problem, is to change the code so that if it gets a dud GC object, it will just bust it out of the tree and let it leak, and print a message saying so. Then I can quit the program, and purify will tell me 'hey, you leaked!' and also tell me where it was allocated. More concerning, about half the segfaults are not from the GC at all, but from realloc in PyFrame_New (line 161 of frameobject). These are the only two I'm getting - it's split 50-50 amongst the 10 coredumps I have now. I'm not sure whether to open a seperate bug for this. Has python2.1.1 been purified? With Zope and zope's extensions? Wow - it's amazing how this SF bug thing is so painful for conversations :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:11 Message: Logged In: YES user_id=21627 There are two options: a) the object isn't really a GC object, i.e. has no GC header. In gdb, you can try to cast gc to PyObject*, then see if the resulting pointer has a better ob_type (this is unlikely, though, since the logic entering the object was already using fromgc/togc) b) somebody has cleared the ob_type field. Can you guarantee that all extension modules have been compiled with the 2.1.1 header files? Is the problem repeatable in the sense that gc will have the same pointer value on each crash? If so, it is relatively easy to track down: just set a gdb change watchpoint on the address on the ob_type field of that address (note that setting watchpoints is not possible until there is really mapped memory on that address). If you can't analyse it through change breakpoints, I recommend to annotate the interpreter in the following way: in pyobject_init, put a printf that prints the address and the tp_name of the type. In subtract_refs, if the ob_type slot is null, print the address of the object and abort. Then analyse the log to see whether a object really has been allocated on that address, and what its type was (make sure you consider the possibility that address are off by the delta that FROM_GC adds). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 21:58 Message: Logged In: YES user_id=29957 Ok, I have an intact core file, and a matching binary, no optimisations, nothing. This crash is showing the crash at line 166 of gcmodule.c traverse = PyObject_FROM_GC(gc)->ob_type->tp_traverse; PyObject_FROM_GC(gc)->ob_type in this case is $24 = {ob_refcnt = 1, ob_type = 0x0} To check my logic, I checked gc_next and gc_prev using the same GDB magic, and they correctly show up as a tuple and an instance method. Some fiddling around seems to rule out stack space as the problem, as well. We're going to try and see if purify helps here, but the problem looks to be a junk object - I have no idea how to track this down further. Help? Would taking the horrible horrible hack of removing the object from the gc linked list if ob_type is null help? Well, it'd stop the crashes, anyway. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Thu Oct 18 11:46:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 03:46:13 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:46 Message: Logged In: YES user_id=21627 This is something to ask the pythonlabs folks; I believe Python has been purified once in a while - perhaps even with Zope extensions. I'd still suspect a bug in the Zope extensions instead of a Python bug, though: if malloc crashes, chances are high that somebody was overwriting free memory. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:31 Message: Logged In: YES user_id=29957 It's not a GC object. I'm positive all the extension objects are correct - I just recompiled, without the 1.5/2.0 headers around. It's a different pointer each time round, unfortunately. It also takes anything from 5 minutes to 2 hours to reproduce. I've got about 4 copies of it running now, and I've got a bunch of different core files. I've grabbed purify and an eval license, and I'm feeding it the binary. The printf approach is probably not going to work - these are busy busy Zope servers. Instead, my plan, assuming that purify doesn't immediately spot a problem, is to change the code so that if it gets a dud GC object, it will just bust it out of the tree and let it leak, and print a message saying so. Then I can quit the program, and purify will tell me 'hey, you leaked!' and also tell me where it was allocated. More concerning, about half the segfaults are not from the GC at all, but from realloc in PyFrame_New (line 161 of frameobject). These are the only two I'm getting - it's split 50-50 amongst the 10 coredumps I have now. I'm not sure whether to open a seperate bug for this. Has python2.1.1 been purified? With Zope and zope's extensions? Wow - it's amazing how this SF bug thing is so painful for conversations :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:11 Message: Logged In: YES user_id=21627 There are two options: a) the object isn't really a GC object, i.e. has no GC header. In gdb, you can try to cast gc to PyObject*, then see if the resulting pointer has a better ob_type (this is unlikely, though, since the logic entering the object was already using fromgc/togc) b) somebody has cleared the ob_type field. Can you guarantee that all extension modules have been compiled with the 2.1.1 header files? Is the problem repeatable in the sense that gc will have the same pointer value on each crash? If so, it is relatively easy to track down: just set a gdb change watchpoint on the address on the ob_type field of that address (note that setting watchpoints is not possible until there is really mapped memory on that address). If you can't analyse it through change breakpoints, I recommend to annotate the interpreter in the following way: in pyobject_init, put a printf that prints the address and the tp_name of the type. In subtract_refs, if the ob_type slot is null, print the address of the object and abort. Then analyse the log to see whether a object really has been allocated on that address, and what its type was (make sure you consider the possibility that address are off by the delta that FROM_GC adds). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 21:58 Message: Logged In: YES user_id=29957 Ok, I have an intact core file, and a matching binary, no optimisations, nothing. This crash is showing the crash at line 166 of gcmodule.c traverse = PyObject_FROM_GC(gc)->ob_type->tp_traverse; PyObject_FROM_GC(gc)->ob_type in this case is $24 = {ob_refcnt = 1, ob_type = 0x0} To check my logic, I checked gc_next and gc_prev using the same GDB magic, and they correctly show up as a tuple and an instance method. Some fiddling around seems to rule out stack space as the problem, as well. We're going to try and see if purify helps here, but the problem looks to be a junk object - I have no idea how to track this down further. Help? Would taking the horrible horrible hack of removing the object from the gc linked list if ob_type is null help? Well, it'd stop the crashes, anyway. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Thu Oct 18 11:59:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 03:59:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:59 Message: Logged In: YES user_id=29957 What's the appropriate way to ask this? drop a line to python-dev? purify's showing a UMR in strop_expandtabs (from memory - don't have it on screen right now) when python starts up. Yeah, the realloc bug concerns me - I have to wonder if something's a little bit/lot screwy. I just don't know where. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:46 Message: Logged In: YES user_id=21627 This is something to ask the pythonlabs folks; I believe Python has been purified once in a while - perhaps even with Zope extensions. I'd still suspect a bug in the Zope extensions instead of a Python bug, though: if malloc crashes, chances are high that somebody was overwriting free memory. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:31 Message: Logged In: YES user_id=29957 It's not a GC object. I'm positive all the extension objects are correct - I just recompiled, without the 1.5/2.0 headers around. It's a different pointer each time round, unfortunately. It also takes anything from 5 minutes to 2 hours to reproduce. I've got about 4 copies of it running now, and I've got a bunch of different core files. I've grabbed purify and an eval license, and I'm feeding it the binary. The printf approach is probably not going to work - these are busy busy Zope servers. Instead, my plan, assuming that purify doesn't immediately spot a problem, is to change the code so that if it gets a dud GC object, it will just bust it out of the tree and let it leak, and print a message saying so. Then I can quit the program, and purify will tell me 'hey, you leaked!' and also tell me where it was allocated. More concerning, about half the segfaults are not from the GC at all, but from realloc in PyFrame_New (line 161 of frameobject). These are the only two I'm getting - it's split 50-50 amongst the 10 coredumps I have now. I'm not sure whether to open a seperate bug for this. Has python2.1.1 been purified? With Zope and zope's extensions? Wow - it's amazing how this SF bug thing is so painful for conversations :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:11 Message: Logged In: YES user_id=21627 There are two options: a) the object isn't really a GC object, i.e. has no GC header. In gdb, you can try to cast gc to PyObject*, then see if the resulting pointer has a better ob_type (this is unlikely, though, since the logic entering the object was already using fromgc/togc) b) somebody has cleared the ob_type field. Can you guarantee that all extension modules have been compiled with the 2.1.1 header files? Is the problem repeatable in the sense that gc will have the same pointer value on each crash? If so, it is relatively easy to track down: just set a gdb change watchpoint on the address on the ob_type field of that address (note that setting watchpoints is not possible until there is really mapped memory on that address). If you can't analyse it through change breakpoints, I recommend to annotate the interpreter in the following way: in pyobject_init, put a printf that prints the address and the tp_name of the type. In subtract_refs, if the ob_type slot is null, print the address of the object and abort. Then analyse the log to see whether a object really has been allocated on that address, and what its type was (make sure you consider the possibility that address are off by the delta that FROM_GC adds). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 21:58 Message: Logged In: YES user_id=29957 Ok, I have an intact core file, and a matching binary, no optimisations, nothing. This crash is showing the crash at line 166 of gcmodule.c traverse = PyObject_FROM_GC(gc)->ob_type->tp_traverse; PyObject_FROM_GC(gc)->ob_type in this case is $24 = {ob_refcnt = 1, ob_type = 0x0} To check my logic, I checked gc_next and gc_prev using the same GDB magic, and they correctly show up as a tuple and an instance method. Some fiddling around seems to rule out stack space as the problem, as well. We're going to try and see if purify helps here, but the problem looks to be a junk object - I have no idea how to track this down further. Help? Would taking the horrible horrible hack of removing the object from the gc linked list if ob_type is null help? Well, it'd stop the crashes, anyway. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Thu Oct 18 12:32:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 04:32:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 04:32 Message: Logged In: YES user_id=21627 Looking at the changes since 2.1.1, I can see one bugfix that might explain strange behaviour, namely ceval.c 2.277. If you want to try it out, you can get the change here http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Python/ceval.c.diff?r1=2.276&r2=2.277 ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:59 Message: Logged In: YES user_id=29957 What's the appropriate way to ask this? drop a line to python-dev? purify's showing a UMR in strop_expandtabs (from memory - don't have it on screen right now) when python starts up. Yeah, the realloc bug concerns me - I have to wonder if something's a little bit/lot screwy. I just don't know where. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:46 Message: Logged In: YES user_id=21627 This is something to ask the pythonlabs folks; I believe Python has been purified once in a while - perhaps even with Zope extensions. I'd still suspect a bug in the Zope extensions instead of a Python bug, though: if malloc crashes, chances are high that somebody was overwriting free memory. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:31 Message: Logged In: YES user_id=29957 It's not a GC object. I'm positive all the extension objects are correct - I just recompiled, without the 1.5/2.0 headers around. It's a different pointer each time round, unfortunately. It also takes anything from 5 minutes to 2 hours to reproduce. I've got about 4 copies of it running now, and I've got a bunch of different core files. I've grabbed purify and an eval license, and I'm feeding it the binary. The printf approach is probably not going to work - these are busy busy Zope servers. Instead, my plan, assuming that purify doesn't immediately spot a problem, is to change the code so that if it gets a dud GC object, it will just bust it out of the tree and let it leak, and print a message saying so. Then I can quit the program, and purify will tell me 'hey, you leaked!' and also tell me where it was allocated. More concerning, about half the segfaults are not from the GC at all, but from realloc in PyFrame_New (line 161 of frameobject). These are the only two I'm getting - it's split 50-50 amongst the 10 coredumps I have now. I'm not sure whether to open a seperate bug for this. Has python2.1.1 been purified? With Zope and zope's extensions? Wow - it's amazing how this SF bug thing is so painful for conversations :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:11 Message: Logged In: YES user_id=21627 There are two options: a) the object isn't really a GC object, i.e. has no GC header. In gdb, you can try to cast gc to PyObject*, then see if the resulting pointer has a better ob_type (this is unlikely, though, since the logic entering the object was already using fromgc/togc) b) somebody has cleared the ob_type field. Can you guarantee that all extension modules have been compiled with the 2.1.1 header files? Is the problem repeatable in the sense that gc will have the same pointer value on each crash? If so, it is relatively easy to track down: just set a gdb change watchpoint on the address on the ob_type field of that address (note that setting watchpoints is not possible until there is really mapped memory on that address). If you can't analyse it through change breakpoints, I recommend to annotate the interpreter in the following way: in pyobject_init, put a printf that prints the address and the tp_name of the type. In subtract_refs, if the ob_type slot is null, print the address of the object and abort. Then analyse the log to see whether a object really has been allocated on that address, and what its type was (make sure you consider the possibility that address are off by the delta that FROM_GC adds). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 21:58 Message: Logged In: YES user_id=29957 Ok, I have an intact core file, and a matching binary, no optimisations, nothing. This crash is showing the crash at line 166 of gcmodule.c traverse = PyObject_FROM_GC(gc)->ob_type->tp_traverse; PyObject_FROM_GC(gc)->ob_type in this case is $24 = {ob_refcnt = 1, ob_type = 0x0} To check my logic, I checked gc_next and gc_prev using the same GDB magic, and they correctly show up as a tuple and an instance method. Some fiddling around seems to rule out stack space as the problem, as well. We're going to try and see if purify helps here, but the problem looks to be a junk object - I have no idea how to track this down further. Help? Would taking the horrible horrible hack of removing the object from the gc linked list if ob_type is null help? Well, it'd stop the crashes, anyway. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Thu Oct 18 12:52:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 04:52:20 -0700 Subject: [Python-bugs-list] [ python-Bugs-472202 ] dlerror()-messages bogus on netbsd Message-ID: Bugs item #472202, was opened at 2001-10-17 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Audun Arnesen Nordal (audun) Assigned to: Martin v. Löwis (loewis) Summary: dlerror()-messages bogus on netbsd Initial Comment: In Python/dynload_shlib.c, a preprocessor check is done to determine if the 'NetBSD' identifier predates some value (199712) on NetBSD platforms. However, this identifier is defined in the file which is not included, and dlerror() messages are thus mistakingly hardcoded to "error in dynamic linking" on all NetBSD platforms. Also, recent ELF-handling added targeted for OpenBSD applies to NetBSD as well. This bug has been handled in the NetBSD package tree for some time (patch committed there by Matthias Drochner), but I couldn't find it reported in Python's buglist and the bug is present in Python's most recent cvs. Patch enclosed. Best regards, Audun Nordal ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 04:52 Message: Logged In: YES user_id=21627 Thanks for your report. I have committed the attached file (dynload_shlib.c.diff) as dynload_shlib.c 2.11. I could not spot anything funny in that file, nor could I see any stray parentheses anywhere. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:37 Message: Logged In: YES user_id=21627 The patch looks fine to me. ---------------------------------------------------------------------- Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 13:02 Message: Logged In: YES user_id=47184 Right. I got the diff the wrong way around, plus there's a stray paranthesis *blush*. Sorry about that. Audun ---------------------------------------------------------------------- Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 12:31 Message: Logged In: YES user_id=47184 Grrr, my browser did something funny with the diff-file. In verbatim below. Audun --- Python/dynload_shlib.c Wed Oct 17 21:05:08 2001 +++ Python/dynload_shlib.c.orig Wed Oct 17 20:58:42 2001 @@ -6,9 +6,7 @@ #include #include -#if defined(__NetBSD__) -#include -#if (NetBSD < 199712) +#if defined(__NetBSD__) && (NetBSD < 199712) #include #include #define dlerror() "error in dynamic linking" @@ -17,9 +15,8 @@ #include #endif #endif -#endif /* NetBSD */ -#if ((defined(__OpenBSD__) || defined(__NetBSD__)) && !defined(__ELF__) +#if defined(__OpenBSD__) && !defined(__ELF__) #define LEAD_UNDERSCORE "_" #else #define LEAD_UNDERSCORE "" ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 From noreply@sourceforge.net Thu Oct 18 13:00:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 05:00:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-472202 ] dlerror()-messages bogus on netbsd Message-ID: Bugs item #472202, was opened at 2001-10-17 12:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 Category: Build Group: Platform-specific Status: Closed Resolution: Accepted Priority: 5 Submitted By: Audun Arnesen Nordal (audun) Assigned to: Martin v. Löwis (loewis) Summary: dlerror()-messages bogus on netbsd Initial Comment: In Python/dynload_shlib.c, a preprocessor check is done to determine if the 'NetBSD' identifier predates some value (199712) on NetBSD platforms. However, this identifier is defined in the file which is not included, and dlerror() messages are thus mistakingly hardcoded to "error in dynamic linking" on all NetBSD platforms. Also, recent ELF-handling added targeted for OpenBSD applies to NetBSD as well. This bug has been handled in the NetBSD package tree for some time (patch committed there by Matthias Drochner), but I couldn't find it reported in Python's buglist and the bug is present in Python's most recent cvs. Patch enclosed. Best regards, Audun Nordal ---------------------------------------------------------------------- >Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-18 05:00 Message: Logged In: YES user_id=47184 Yes, the attached diff was fine. The one that I pasted into my second message was reversed and bogus. Sorry about the inclarity. Thank you for applying the patch. Audun ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 04:52 Message: Logged In: YES user_id=21627 Thanks for your report. I have committed the attached file (dynload_shlib.c.diff) as dynload_shlib.c 2.11. I could not spot anything funny in that file, nor could I see any stray parentheses anywhere. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:37 Message: Logged In: YES user_id=21627 The patch looks fine to me. ---------------------------------------------------------------------- Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 13:02 Message: Logged In: YES user_id=47184 Right. I got the diff the wrong way around, plus there's a stray paranthesis *blush*. Sorry about that. Audun ---------------------------------------------------------------------- Comment By: Audun Arnesen Nordal (audun) Date: 2001-10-17 12:31 Message: Logged In: YES user_id=47184 Grrr, my browser did something funny with the diff-file. In verbatim below. Audun --- Python/dynload_shlib.c Wed Oct 17 21:05:08 2001 +++ Python/dynload_shlib.c.orig Wed Oct 17 20:58:42 2001 @@ -6,9 +6,7 @@ #include #include -#if defined(__NetBSD__) -#include -#if (NetBSD < 199712) +#if defined(__NetBSD__) && (NetBSD < 199712) #include #include #define dlerror() "error in dynamic linking" @@ -17,9 +15,8 @@ #include #endif #endif -#endif /* NetBSD */ -#if ((defined(__OpenBSD__) || defined(__NetBSD__)) && !defined(__ELF__) +#if defined(__OpenBSD__) && !defined(__ELF__) #define LEAD_UNDERSCORE "_" #else #define LEAD_UNDERSCORE "" ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472202&group_id=5470 From noreply@sourceforge.net Thu Oct 18 16:36:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 08:36:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-472007 ] Linker problem on HP-UX 11.00 Message-ID: Bugs item #472007, was opened at 2001-10-17 01:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Martin v. Löwis (loewis) Summary: Linker problem on HP-UX 11.00 Initial Comment: I use the following command to run the configure script: CC=cc \ OPT="+DAportable" \ INSTALL="./install-sh -c" \ ./configure --with-threads --without-gcc \ --with-cxx=/opt/aCC/bin/aCC --without-libd \ --with-cycle-gc --prefix=/opt/python \ --exec-prefix=/opt/python In the Makefile I get the following lines: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CC) The build then fails with: /usr/ccs/bin/ld: Unsatisfied symbols: _main (first referenced in Modules/ccpython.o) (code) *** Error exit code 1 If I edit the Makefile, thus: CC= cc -Ae CXX= /opt/aCC/bin/aCC LINKCC= $(PURIFY) $(CXX) The builds succeeds. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 08:36 Message: Logged In: YES user_id=21627 Fixed in configure.in 1.270. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-10-17 08:25 Message: Logged In: YES user_id=200117 I applied your patch on HP-UX 11.00 and it modified configure and configure.in without errors. When I ran configure, Makefile and Makefile.pre still contain: LINKCC= $(PURIFY) $(CC) Just in case, I ran autoconf and then ran configure again, but got the same result. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 06:29 Message: Logged In: YES user_id=21627 LINKCC already was set to CXX before (configure.in 1.113); that was undone later (1.198). Unfortunately, none of these checkins indicate why they have been done; I recall that people where complaining that python requires libstdc++ on Linux. I've now designed a test that checks whether CXX must be used to link if main() was compiled with CXX. I cannot test it on a system that has this property, so I'll attach it to this report. When patching configure fails for you, please re-run autoconf. If you don't have autoconf, please fix the two line numbers in the patch to match your configure; the patch should apply then. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-17 05:36 Message: Logged In: YES user_id=6380 Martin, do you understand the logic in configre.in enough to suggest a patch? If not, please unassign it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472007&group_id=5470 From noreply@sourceforge.net Thu Oct 18 16:49:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 08:49:27 -0700 Subject: [Python-bugs-list] [ python-Bugs-472234 ] type(obj) calls type->tp_init Message-ID: Bugs item #472234, was opened at 2001-10-17 13:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472234&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Roeland Rengelink (rengelink) Assigned to: Guido van Rossum (gvanrossum) Summary: type(obj) calls type->tp_init Initial Comment: In 2.2a4 Consider: class Meta(type): def __init__(self, *args): print 'Meta.__init__ called' class Object(object): __metaclass__ = Meta obj = Object() print type(obj) will print Meta.__init__ called Meta.__init__ called The second call to Meta.__init__ results from the type(obj) call. This is because type(obj) is special cased in type_new(...) rather than type_call(...). Although it's rather useless to define an __init__ in a class derived from type (for metaclasses it's a better idea to override __new__). I do think the call of Meta.__init__ in type(obj) is a bug. Thanks, Roeland ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 08:49 Message: Logged In: YES user_id=6380 You're right. I've fixed this in CVS now, just in tome for 2.2b1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472234&group_id=5470 From noreply@sourceforge.net Thu Oct 18 16:46:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 08:46:56 -0700 Subject: [Python-bugs-list] [ python-Bugs-469772 ] support additional LINK elements. Message-ID: Bugs item #469772, was opened at 2001-10-09 23:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: support additional LINK elements. Initial Comment: Now that mozilla supports the LINK element properly, it would be most excellent if the python docs were generated with more LINK elements. At the moment they're built with up, previous, and next. Some of the others are defined in http://www.w3.org/TR/REC-html40/types.html#type-links Ones that would be very nice are Top, Contents and Index. ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 08:46 Message: Logged In: YES user_id=29957 There's a very nice list of what link relationships do what for mozilla 0.9.5 posted at: http://lists.w3.org/Archives/Public/www-html/2001Oct/0026.html (link stolen from mozillazine) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 From noreply@sourceforge.net Thu Oct 18 17:15:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 09:15:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-471928 ] global made w/nested list comprehensions Message-ID: Bugs item #471928, was opened at 2001-10-16 18:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Jeremy Hylton (jhylton) Summary: global made w/nested list comprehensions Initial Comment: This bug affects Python 2.1.1 & 2.2a4+. In nested list comprehension code like this: [bad for s in 'a b' for bad in s.split()] the variable bad becomes a global, but it should be local variable. The attached file shows the disassembled byte code, along with a test case. The original source of the error was found from cgi.py:209 in the standard library: pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')] Neal ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-18 09:15 Message: Logged In: YES user_id=31392 The symbol table pass wasn't visiting nested list comprehenesions. Fixed in rev. 2.227 of compile.c. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-17 18:22 Message: Logged In: YES user_id=31435 Reassigned to Jeremy, as I expect he's got a better chance of understanding why the symbol-table code is calling one of these local and the other global (listcomp codegen treats them the same). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 23:06 Message: Logged In: YES user_id=6380 Good catch! I don't know the list comprehension code generation, and I don't know who does. Maybe Thomas Wouters? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471928&group_id=5470 From noreply@sourceforge.net Thu Oct 18 17:18:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 09:18:29 -0700 Subject: [Python-bugs-list] [ python-Bugs-433625 ] bug in PyThread_release_lock() Message-ID: Bugs item #433625, was opened at 2001-06-15 19:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=433625&group_id=5470 Category: Threads Group: Platform-specific >Status: Deleted Resolution: Invalid Priority: 5 Submitted By: Shih-Hao Liu (shihao) Assigned to: Tim Peters (tim_one) Summary: bug in PyThread_release_lock() Initial Comment: Mutex should be hold when calling pthread_cond_signal(). This function should look like: PyThread_release_lock(PyThread_type_lock lock) { pthread_lock *thelock = (pthread_lock *)lock; int status, error = 0; dprintf(("PyThread_release_lock(%p) called\n", lock)); status = pthread_mutex_lock( &thelock->mut ); CHECK_STATUS("pthread_mutex_lock[3]"); thelock->locked = 0; /* ***** call pthread_cond_signal before unlock mutex */ status = pthread_cond_signal( &thelock->lock_released ); CHECK_STATUS("pthread_cond_signal"); status = pthread_mutex_unlock( &thelock->mut ); CHECK_STATUS("pthread_mutex_unlock[3]"); /* wake up someone (anyone, if any) waiting on the lock */ } ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-18 09:18 Message: Logged In: YES user_id=31435 Deleted this bug report, at Shih-Hao Liu's request. ---------------------------------------------------------------------- Comment By: Shih-Hao Liu (shihao) Date: 2001-06-20 00:17 Message: Logged In: YES user_id=246388 > I also browse through > LinuxThread implementation and can't find > pthread_cond_xxxx is implemented automically. I found they do call __pthread_lock when updating the priority queue after take a closer look. I guess I have to look at elsewhere to figure out why my Python script is spinning. ---------------------------------------------------------------------- Comment By: Shih-Hao Liu (shihao) Date: 2001-06-19 22:14 Message: Logged In: YES user_id=246388 > I understand both of those. Are you assuming that, e.g., > thread 3's PyThread_acquire_lock completes in whole during > this gap? I don't know what else you could mean, so let's > assume that. Yes, I assume thread 3 completed in this gap and it is possible. "Collide" means while thread 2's pthread_cond_wait was modifiying the internal data structure, thread 1 calls pthread_cond_signal. The point here is not missing signals, the problem is the possiblity that pthread_cond_signal preempt the execution of pthread_cond_wait. I can't find any document says pthread_cond_xxx functions must be automic operations. In pthread_cond_xxx man page, it only mentioned: pthread_cond_wait atomically unlocks the mutex (as per ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pthread_unlock_mutex) and waits for the condition variable ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cond to be signaled. The thread execution is suspended and ^^^^^^^^^^^^^^^^^^^ does not consume any CPU time until the condition variable is signaled. The mutex must be locked by the calling thread on entrance to pthread_cond_wait. Before returning to the calling thread, pthread_cond_wait re-acquires mutex (as per pthread_lock_mutex). Unlocking the mutex and suspending on the condition vari­ able is done atomically. Thus, if all threads always acquire the mutex before signaling the condition, this guarantees that the condition cannot be signaled (and thus ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ignored) between the time a thread locks the mutex and the ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ time it waits on the condition variable. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I take it that between the time the mutex is locked by pthread_mutex_lock and unlocked by pthread_cond_wait, you can't call pthread_cond_signal. I also browse through LinuxThread implementation and can't find pthread_cond_xxxx is implemented automically. I found there is another way to fix this without having to call the pthread_cond_signal while holding the mutex. If we do: if (!thelock->locked) status = pthread_cond_signal( &thelock->lock_released ); we guarantee that when pthread_cond_signal is called, the acquire_lock code will not be in the middle of pthread_cond_signal. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-06-19 15:06 Message: Logged In: YES user_id=31435 > The problem will be arised if there is a thread 3 > called PyThread_acquire_lock You never mentioned thread 3 again, so I don't know what it has to do with this. > after thread 1 set thelock->locked to 0 and before > thread 2 calling pthread_mutex_lock. I understand both of those. Are you assuming that, e.g., thread 3's PyThread_acquire_lock completes in whole during this gap? I don't know what else you could mean, so let's assume that. > "while (thelock->locked)" will success for thread 2 Sure. > and it will call pthread_cond_wait Yup. > and might collide with thread 1's pthread_cond_signal. What does "collide" mean to you? All the pthread_cond_xxx functions must be implemented as if atomic, so there's no meaningful sense (to me) in which they can collide -- unless they're implemented incorrectly. Assuming they are implemented correctly, it again doesn't matter that thread 2 misses thread 1's signal, because thread *3* exploited the information thread 1 was going to signal, by acquiring the lock. It's actually good that thread 2 isn't bothered with it: there's no real info in the signal anymore (at best, if thread 2 got it, it would wake up and go "oops! it's still locked; I'll wait again"). All that matters now is whether thread 2 gets a chance to see thread *3*'s signal, at the time thread 3 releases the lock. And it will, because thread 3 can't release the lock without acquiring the mutex first, and thread 2 holds the mutex at all times except when in its pthread_cond_wait call (so thread 3 can't release the lock except when thread 2 is in pthread_cond_wait). Note that I'm not at all concerned about "fairness" here, only about races. ---------------------------------------------------------------------- Comment By: Shih-Hao Liu (shihao) Date: 2001-06-19 14:41 Message: Logged In: YES user_id=246388 Oops. The problem will be arised if there is a thread 3 called PyThread_acquire_lock after thread 1 set thelock->locked to 0 and before thread 2 calling pthread_mutex_lock. "while (thelock->locked)" will success for thread 2 and it will call pthread_cond_wait and might collide with thread 1's pthread_cond_signal. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-06-19 13:52 Message: Logged In: YES user_id=31435 It appears you're concerned that the signal will "get lost" in this scenario. I agree that it may, but it doesn't matter: thread 2's "while (thelock->locked)" test fails because thread 1 already set thelock->locked to 0, so thread 2 doesn't execute its pthread_cond_wait, so it doesn't matter that nobody is *waiting* to see thread 1's pthread_cond_signal. IOW, the condition thread 1 will try to signal has *already* been detected by thread 2, and the signal is useless (because redundant) information. Indeed, it's a beauty of the condition protocol that signals can be sloppy. The linuxthread man page is worded strangely here, but note that it does not say the mutex *must* be locked. They can't, either, because POSIX doesn't require it; while POSIX stds aren't available freely online, some derived specs are, and are much clearer about this; e.g., see the Single Unix Specification, here: I'll tell you why I don't *want* to change this: in Python's use of the global interpreter lock, it's almost always the case that someone is waiting on the lock. By releasing the mutex before signaling, this gives a waiter a chance to run immediately upon calling pthread_cond_signal. Else, because pthread_cond_wait (which the waiters are executing) has to lock the mutex, if the signaler is holding the mutex during the signal, pthread_cond_signal can't finish the job -- it was to go back to the signaler and let it unlock the mutex first before a waiter can proceed. This makes the region of exclusion longer than it needs to be. So if there's not an actual race problem here (& I still don't see one), I don't want to change this. ---------------------------------------------------------------------- Comment By: Shih-Hao Liu (shihao) Date: 2001-06-17 23:01 Message: Logged In: YES user_id=246388 I closed it because I thought the thelock->locked variable will ensure that the PyThread_release_lock will help to protect the condition variable and I was wrong. The linuxthread man page on pthread_cond_signal: A condition variable must always be associated with a mutex, to avoid the race condition where a thread prepares to wait on a condition variable and another thread signals the condition just before the first thread actually waits on it. which means you can't call pthread_cond_signal & pthread_cond_wait on the same condition variable at the same time. And using a mutex to protect them is a good idea. Here is how thing might go wrong with current implementation: thread 1 thread 2 |int PyThread_acquire_lock _ |/** assume lock was acquired | by thread 1, hence locked=0 | & success would be 0 **/ |{ | ... | status = pthread_mutex_lo | CHECK_STATUS("pthread_mut | success = thelock->locked | if (success) thelock->loc | status = pthread_mutex_un | /** thread 2 suspended **/ void PyThread_release_lock _| { | ... | status = pthread_mutex_loc| CHECK_STATUS("pthread_mute| | thelock->locked = 0; | | status = pthread_mutex_unl| /** thread 1 suspend **/ | | CHECK_STATUS("pthread_mut | | if ( !success && waitflag | /* continue trying unti | | /* mut must be locked b | * protocol */ | status = pthread_mutex_ | CHECK_STATUS("pthread_m | while ( thelock->locked | status = pthread_cond |/** thread 2 suspended while | updating shared data ** CHECK_STATUS("pthread_mute| | /* wake up someone (anyone| status = pthread_cond_sign| /** thread 1 update shared | data and corrupt it. **/ | Not sure what the effect would be. It's wouldn't be nice anyway. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-06-17 16:02 Message: Logged In: YES user_id=31435 Closing this again, as it appears the original submitter deleted it. shihao, if you want to pursure this, open it again. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-06-17 15:01 Message: Logged In: YES user_id=31435 Ack, did I delete this?! I sure didn't intend to -- didn't even intend to close it. Reopened pending more info. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-06-17 14:51 Message: Logged In: YES user_id=6380 Set status to closed -- no need to delete it. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-06-17 14:16 Message: Logged In: YES user_id=31435 Why? It's allowed to signal the condition whether or not the mutex is held. Since changing this can have visible effects on thread scheduling, I'm reluctant to change it without a good reason. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=433625&group_id=5470 From noreply@sourceforge.net Thu Oct 18 17:25:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 09:25:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-472347 ] pydoc and properties Message-ID: Bugs item #472347, was opened at 2001-10-17 23:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472347&group_id=5470 >Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Roeland Rengelink (rengelink) >Assigned to: Tim Peters (tim_one) Summary: pydoc and properties Initial Comment: pydoc in server mode throws an error when parsing the attached module. From the command line documentation is shown correctly. The problem seems to be in HTMLDoc.spillproperties(). Replacing: base = self.document(func, name + tag, mod, funcs, classes, mdict, object) with: base = self.document(func, name+tag, mod) seems to solve this. This is also more in line with TextDoc.spillproperties() ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-18 09:25 Message: Logged In: YES user_id=31435 Assigned to me. The HTML flavor of .document() uses the extra arguments to figure out if any words in the generated docs should be turned into hyperlinks; that's irrelevant to the text-mode flavor; so, whatever the true cause, the correct cure probably isn't just to give up in HTML mode. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472347&group_id=5470 From noreply@sourceforge.net Thu Oct 18 19:01:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 11:01:59 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 >Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Guido van Rossum (gvanrossum) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 11:01 Message: Logged In: YES user_id=6380 OK. That looks good. I've checked this into CVS, in time for 2.2b1. Question: what should be done if an exception occurs in finish_request()? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-17 02:10 Message: Logged In: YES user_id=350896 This is a context diff of a proposal for a patch relative to the current state of CVS. I have to admit that I tested this only with 2.2a4 and not with the current CVS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 17:26 Message: Logged In: YES user_id=6380 OK. It would be a great help if you could upload a patch (a context diff) relative to the current state of CVS, or the 2.2a4 release if you must. When uploading, don't forget to check the file upload checkbox. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Thu Oct 18 19:52:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 11:52:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-404545 ] frozen package import uses wrong files Message-ID: Bugs item #404545, was opened at 2001-02-27 02:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=404545&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 2 Submitted By: Toby Dickenson (htrd) Assigned to: Guido van Rossum (gvanrossum) Summary: frozen package import uses wrong files Initial Comment: In a frozen package, importing a module from another package causes the import machinery to try to open some curiously named files, before finally finding the frozen data. It is possible to 'break' a frozen program by creating a file of that name. The frozen program will try to import from it rather than the frozen data. The following collection of modules demonstrates this (also in the attached zip): Directory of D:\Projects\import 2001-02-27 08:57 11 b.n.py 2001-02-27 08:49 10 x.py 2 File(s) 21 bytes Directory of D:\Projects\import\a 2001-02-27 08:57 27 m.py 2001-02-27 09:58 0 __init__.py 2 File(s) 27 bytes Directory of D:\Projects\import\b 2001-02-27 08:56 11 n.py 2001-02-27 09:58 0 __init__.py 2 File(s) 11 bytes Total Files Listed: 6 File(s) 59 bytes 0 Dir(s) 1,485,537,280 bytes free The 'real' program is made up of the three files with single character names plus the two __init__ files. b.n.py is a rogue file that breaks a frozen program. x.py contains "import a.m" a/m.py contains "import b.n". This is the import that goes wrong. When run as a normal script it imports b/n.py. However, a frozen binary appears to search for various a.b.* files over sys.path first. If it is run from the same directory as a.b.py then it will load that file instead. Note that this file is not included in the freeze. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 11:52 Message: Logged In: YES user_id=6380 This is now fixed by patch #416704 which is checked in. ---------------------------------------------------------------------- Comment By: Toby Dickenson (htrd) Date: 2001-04-17 07:30 Message: Logged In: YES user_id=46460 Fix for this (and several other ways to break a frozen program) are in patch #416704 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-03-23 10:09 Message: Logged In: YES user_id=6380 Note, I tried this on Linux, and I couldn't reproduce it. What Python version were you using? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-03-20 11:43 Message: Logged In: YES user_id=6380 I agree this is a bug. I think there are lots of other ways to break frozen programs, so I don't think this is a high priority security bug. I wish I had more time to research this, but I don't, so I'll give this a low priority. If someone submits a patch, I'd be grateful! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=404545&group_id=5470 From noreply@sourceforge.net Thu Oct 18 20:08:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 12:08:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-441409 ] re.compile of non-string misleading Message-ID: Bugs item #441409, was opened at 2001-07-15 00:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=441409&group_id=5470 Category: Regular Expressions Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Ben Escoto (bescoto) Assigned to: Fredrik Lundh (effbot) Summary: re.compile of non-string misleading Initial Comment: re.compile(foo) returns foo, instead of raising an error or returning a regular expression object, if foo isn't a string. For instance: >>> import re >>> re.compile(5) 5 Instead, it seems an exception should be raised. This is especially confusing when foo is a UserString or another class that subclasses it. ---------------------------------------------------------------------- >Comment By: Fredrik Lundh (effbot) Date: 2001-10-18 12:08 Message: Logged In: YES user_id=38376 you're supposed to be allowed to pass compiled patterns as the pattern. in 2.2b1 and later, the test has been changed to look for patterns instead of non-strings. thanks /F ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=441409&group_id=5470 From noreply@sourceforge.net Thu Oct 18 20:41:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 12:41:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-463572 ] Misleading documentation for gettext Message-ID: Bugs item #463572, was opened at 2001-09-21 07:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=463572&group_id=5470 Category: Documentation Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Barry Warsaw (bwarsaw) Summary: Misleading documentation for gettext Initial Comment: in location: http://www.python.org/doc/2.1.1/lib/node187.html there are docs for translation(), but there is no indication or link to mappings between the supplied language and the subdirectory actually searched. A link to the relevant information would help people considerably, or rather an NB mentioning that the 'languages' in the translation() docs is not the plural of the 'language' in the find() docs. Once I'd renamed my French resource directores fr -> fr_FR it all worked, of course, but I only discovered this from debugging gettext :( kind regards, -Simon Harrison. On a side note, I have an alternative pygettext.py that inserts the English as the translated text when the "-t" option is supplied (test). This has been fairly useful since it allows me to run the code, and test for correct operation prior to real translation. I can supply the new version if useful. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-18 12:41 Message: Logged In: YES user_id=12800 Hmm, I think there is are some misunderstandings. First, both find() and translation() take a list of strings as their languages argument. In fact, the first three arguments to translation() are passed straight through to find()! Second, the documentation for find() clearly describes how the arguments map to the search directories. I'm not sure how that could be any clearer. There is one mistake in find() though: it claims that all arguments are strings, but that's not the case for the languages argument, which in fact should be a list of strings. I will clarify that in the docs. Your pygettext.py addition sounds interesting. If you'd like me to look at it and perhaps add it to the tool, please send it directly to me: barry@zope.com ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-09-21 08:06 Message: Logged In: YES user_id=3066 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=463572&group_id=5470 From noreply@sourceforge.net Thu Oct 18 20:46:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 12:46:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-456504 ] Memory leak in PyString_FromString Message-ID: Bugs item #456504, was opened at 2001-08-29 06:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=456504&group_id=5470 Category: Python Interpreter Core >Group: Irreproducible >Status: Closed >Resolution: Invalid Priority: 4 Submitted By: Nobody/Anonymous (nobody) Assigned to: Barry Warsaw (bwarsaw) Summary: Memory leak in PyString_FromString Initial Comment: Purify (on WinNT) reports a memory leak: Memory leak of 567 bytes from 14 blocks allocated in PyString_FromString Distribution of leaked blocks Allocation location malloc [msvcrt.dll] PyString_FromString [python21.dll] ??? [python.exe ip=0x00405059] This fragments the heap and my long-running program eventually runs out of memory. I was running my sample program with wxPython and so could not run debug version for more explicit info without compiling wx myself. If this is not enough info we can probably get a more detailed output. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-18 12:46 Message: Logged In: YES user_id=12800 Closing as per Guido's recommendation. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 10:51 Message: Logged In: YES user_id=6380 With 99.9% probability, this is not a bug in PyString_FromString(), but a missing Py_DECREF() somewhere else. Because of the way reference counting works, missing DECREFs show up as memory leaks originating in one of the "new-object-creation" routines, but these routines are usually without blame. The trick is to find the missing DECREF! If you want to help us fund it, the best thing to do is boil your example program down to something as small as possible that still exhibits the leak -- then show us the code. With the Python code in hand we can look at what C code is being invoked that might have a missing DECREF. I've done this many times so I know it can be very effective -- and Purify or Insure are very little help, alas. HOWEVER, you mention wxPython. That's a HUGE body of 3rd party code. If you can only reproduce this when wxPython is involved, it's probably a bug in wxPython (3rd party code is always the most likely cause for refcount bugs, in my experience). I propose to lower the priority and close this bug a month from now if no new evidence is found that it's a core Python bug. (BTW the Python version would also be handy to know.) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-08-29 12:40 Message: Logged In: YES user_id=31435 I'm afraid we need better evidence of a "real leak" than this. PyString_FromString is called from 100s of locations, and some of the targets are file-static vrbls that indeed are never *expected* to get cleared (for example, abstract.c constructs the string "__bases__" once and then reuses it forever after). BTW, it's extremely unlikely that a "loss" of 567 bytes is causing you to run out of memory, yes? I'd look elsewhere. Assigning to Barry, since he runs Insure++ regularly. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=456504&group_id=5470 From noreply@sourceforge.net Thu Oct 18 20:48:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 12:48:38 -0700 Subject: [Python-bugs-list] [ python-Bugs-448351 ] coredump in selectmodule.c on Solaris 8 Message-ID: Bugs item #448351, was opened at 2001-08-06 01:23 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448351&group_id=5470 Category: Extension Modules Group: Platform-specific >Status: Closed Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Barry Warsaw (bwarsaw) Summary: coredump in selectmodule.c on Solaris 8 Initial Comment: I get coredump if I run a small script with Python 2.0 on Solaris 8, compiled with Sun CC Forte 6.1 compiler (64 bits). I suggest to run it more than once to produce the error. Purify showed me that there are reading and writings outside the stack boundary. The interesting part of the source: Modules/selectmodule.c . . static PyObject * select_select(PyObject *self, PyObject *args) { #ifdef MS_WINDOWS /* This would be an awful lot of stack space on Windows! */ pylist *rfd2obj, *wfd2obj, *efd2obj; #else pylist rfd2obj[FD_SETSIZE + 3]; pylist wfd2obj[FD_SETSIZE + 3]; pylist efd2obj[FD_SETSIZE + 3]; #endif . . . } In our environment FD_SETSIZE is 65536 as defined in sys/select.h (see below). The allocated stack space in select_select is 3*sizeof(rfd2obj)*(FD_SETSIZE+3). It is more than 3Mbytes. The difference between the addresses of the same variable in two seperate threads is about 2Mbytes. Lets suppose char *p1 = (char *)rfd2obj in thread N and char *p2 = (char *)rfd2obj in thread N + 1, abs(p1-p2) is about 2MB (dbx showed this). The stack is overwritten between the threads. Is it possible that the stack size is limited to 2 Mbytes per thread? We fixed it as solved on Windows allocating these variables on the heap. Select.h from Solaris 8. /usr/include/sys/select.h: . . #ifndef FD_SETSIZE #ifdef _LP64 #define FD_SETSIZE 65536 #else #define FD_SETSIZE 1024 #endif /* _LP64 */ . . ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-18 12:48 Message: Logged In: YES user_id=12800 Ah I forgot to close this bug report. This code's been in the source long enough to have uncovered problems if there were any. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-08-16 14:58 Message: Logged In: YES user_id=31435 Back to Barry for re-testing on Linux: I checked in another version that refuses to add 3 to FD_SETSIZE anymore. Fortune favors the bold. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-08-16 09:53 Message: Logged In: YES user_id=12800 I've just checked in a minimal patch, taking the easy way out (i.e. testing for FD_SETSIZE > 1024). This subtly changes the behavior on Windows (where Tim says the value is now 512) since Windows won't normally take the allocate-on-heap path now. selectmodule.c 2.54 has the change. The changes pass on Linux both for normal, small FD_SETSIZEs and for artificially cranked FD_SETSIZEs. Assigning back to Tim for verification on Windows. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-08-06 13:52 Message: Logged In: YES user_id=31435 Reassigned to Barry. I may a good choice to write the code, but not to test it (I know zilch about sockets, and can't provoke the problem on Windows anyway). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-06 05:36 Message: Logged In: YES user_id=6380 Dear Anonymous, a quick workaround is to change the three #ifdef MS_WINDOWS in selectmodule.c into #if FD_SETSIZE > 1024. A better idea is currently being discussed on python-dev. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-06 01:31 Message: Logged In: NO The attachment missed, this is the mentioned script: import threading import telnetlib def telnetToHost(): hostname = "my_hostname" username = "user_name" password = "password" tn = telnetlib.Telnet(hostname) tn.read_until("login: ") tn.write(username + '\n') tn.read_until("Password: ") tn.write(password + '\n') class MyThread(threading.Thread): def run(self): print "ThreadID", self.cnt, "started" telnetToHost() print "ThreadID", self.cnt, "finished" for i in range(0,4): m = MyThread() m.cnt = i m.start() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448351&group_id=5470 From noreply@sourceforge.net Thu Oct 18 20:59:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 12:59:28 -0700 Subject: [Python-bugs-list] [ python-Bugs-472347 ] pydoc and properties Message-ID: Bugs item #472347, was opened at 2001-10-17 23:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472347&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Roeland Rengelink (rengelink) Assigned to: Tim Peters (tim_one) Summary: pydoc and properties Initial Comment: pydoc in server mode throws an error when parsing the attached module. From the command line documentation is shown correctly. The problem seems to be in HTMLDoc.spillproperties(). Replacing: base = self.document(func, name + tag, mod, funcs, classes, mdict, object) with: base = self.document(func, name+tag, mod) seems to solve this. This is also more in line with TextDoc.spillproperties() ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-18 12:59 Message: Logged In: YES user_id=31435 Fixed in Lib/pydoc.py; new revision: 1.55 Lib/test/pydocfodder.py; new revision: 1.3 simply by making GUI-mode .docother() ignore excess arguments. Property functions are usually methods or functions, not arbitrary callables, so usually go thru .docroutine(), which wants all these arguments. .docother() has no use for them, but shouldn't blow up just because its caller N levels back couldn't guess that .docother() would eventually get called. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-18 09:25 Message: Logged In: YES user_id=31435 Assigned to me. The HTML flavor of .document() uses the extra arguments to figure out if any words in the generated docs should be turned into hyperlinks; that's irrelevant to the text-mode flavor; so, whatever the true cause, the correct cure probably isn't just to give up in HTML mode. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472347&group_id=5470 From noreply@sourceforge.net Thu Oct 18 21:12:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 13:12:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-460686 ] iter() over non-sequence message Message-ID: Bugs item #460686, was opened at 2001-09-11 05:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=460686&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Guido van Rossum (gvanrossum) >Assigned to: Tim Peters (tim_one) Summary: iter() over non-sequence message Initial Comment: Effbot writes: maybe it's just me, but I just spent five minutes trying to figure out why an innocent-looking line of code resulted in an "iter() of non-sequence" type error. I finally ran it under 2.1, and immediately realized what was wrong. is there any chance of getting the old, far more helpful "unpack non-sequence" and "loop over non-sequence" error messages back before 2.2 final? ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-18 13:12 Message: Logged In: YES user_id=31435 Closing as Fixed, because I restored "unpack non-sequence" was some time ago, and the generic msg was changed to "iteration over non-sequence" at the same time. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=460686&group_id=5470 From noreply@sourceforge.net Thu Oct 18 22:04:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 14:04:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-432373 ] [Windows] file.tell() gives wrong value Message-ID: Bugs item #432373, was opened at 2001-06-12 04:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=432373&group_id=5470 Category: Python Library Group: Platform-specific >Status: Closed >Resolution: Wont Fix Priority: 3 Submitted By: Elmar Sonnenschein (eso) >Assigned to: Tim Peters (tim_one) Summary: [Windows] file.tell() gives wrong value Initial Comment: Invoking tell() on a file object will return a wrong (arbitrary?) value if called before seeking. Example: The following script f = open('c:\test.xyz') print 'pos: ' + `f.tell()` print 'read: ' + f.read(3) print 'pos: ' + `f.tell()` f.seek(0) print 'pos: ' + `f.tell()` print 'read: ' + f.read(3) print 'pos: ' + `f.tell()` f.close() will yield the following result: pos: 0 read: XYZ pos: 3587 <-- wrong value pos: 0 read: XYZ pos: 3 Only the return value of tell is wrong, not the actual file position, i. e. a consecutive read() will return the correct bytes. It doesn't help to seek before reading, only seeking _after_ reading will set the return value of tell() correctly. File size of 'test.xyz' was 3.822.167 Bytes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-18 14:04 Message: Logged In: YES user_id=31435 Closing for lack of followup info requested 4 months ago. Here's a cute file: >>> f = file('ga', 'wb') >>> f.write(chr(26) * 10000) >>> f.write('a') >>> f.close() If I feed that into the test program, it displays pos: 0L read: pos: 4096L pos: 0L read: pos: 512L under 2.2a4. But this has nothing to do with Python -- don't ever open binary files in text mode -- or, if you must, complain about the results to Microsoft . ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-06-13 22:48 Message: Logged In: YES user_id=31435 Changed to Platform-specific (I'm sure this doesn't happen under Unix variants). What happens if you write this little program in C instead? My guess it will do the same thing. If so, it's a Microsoft library problem Python can't hide (Python .tell () and .seek() simply call the platform C library functions). Reduced the priority until there's evidence this is actually a Python (not mscvrt.dll) inelegance. ---------------------------------------------------------------------- Comment By: Elmar Sonnenschein (eso) Date: 2001-06-12 06:56 Message: Logged In: YES user_id=145214 Just found out that it only happens if it is a binary file which is opened without the 'b' mode flag. Therefore it is not severe but still strange behavior. Platform is Windows 2000. ---------------------------------------------------------------------- Comment By: Hans Nowak (zephyrfalcon) Date: 2001-06-12 06:52 Message: Logged In: YES user_id=173607 Works fine for me... I'm using Python 2.1 on Windows NT 4, sp 5. :-/ Maybe it's platform dependent? ---------------------------------------------------------------------- Comment By: Elmar Sonnenschein (eso) Date: 2001-06-12 04:15 Message: Logged In: YES user_id=145214 Checked on Python 2.0, 2.1, and ActivePython 2.1 - always the same. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=432373&group_id=5470 From noreply@sourceforge.net Thu Oct 18 22:31:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 14:31:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-472568 ] PyBuffer_New() memory not aligned Message-ID: Bugs item #472568, was opened at 2001-10-18 14:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: PyBuffer_New() memory not aligned Initial Comment: Memory buffer areas created by PyBuffer_New are missaligned on 32 bit machines, for doubles (64 bit). This typically generates a bus error crash on most RISC machines when a library function tries to write a double in a memory buffer allocated by PyBuffer_New(). When looking at the bufferobject.c code, this seems to come from the fact that the memory buffer points at 'malloc() + sizeof struct PyBufferObject'; [line: b->b_ptr = (void *)(b + 1); ] To the extent that struct PyBufferObject is not a multiple of sizeof (double), the largest value type; thus the misalignment, and the crashes on most RISC processors. A quick and temporary solution would consist in adding a dummy double field as the last field of the PyBufferObject structure: struct PyBufferObject { ... double dummy; } ; and setting b->b_ptr = (void*)&b->dummy; /*was (void*(b+1)*/ In doing so, b->b_ptr will always by aligned on sizeof (double), on either 32 and 64 bit architectures. Since I'm on the buffer type problem: It would be nice (and probably easy to do) to augment the buffer protocol interface with the ability to specify the basic value type stored in the buffer. A possible list of values (enum ...) would be: - undefined (backward compatibility) - char, unsigned char, short, ....int, ... long, long long, float, double, and void* (memory address). This would enable to check at runtime the type of values stored in a buffer (and prevent missalignement buserrors, as well as catching garbage situations when improper array types are passed by means of the buffer interface [e.g.: int/float/double/short...). Frederic Giacometti Frederic Giacometti ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 From noreply@sourceforge.net Thu Oct 18 22:35:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 14:35:33 -0700 Subject: [Python-bugs-list] [ python-Bugs-472568 ] PyBuffer_New() memory not aligned Message-ID: Bugs item #472568, was opened at 2001-10-18 14:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: PyBuffer_New() memory not aligned Initial Comment: Memory buffer areas created by PyBuffer_New are missaligned on 32 bit machines, for doubles (64 bit). This typically generates a bus error crash on most RISC machines when a library function tries to write a double in a memory buffer allocated by PyBuffer_New(). When looking at the bufferobject.c code, this seems to come from the fact that the memory buffer points at 'malloc() + sizeof struct PyBufferObject'; [line: b->b_ptr = (void *)(b + 1); ] To the extent that struct PyBufferObject is not a multiple of sizeof (double), the largest value type; thus the misalignment, and the crashes on most RISC processors. A quick and temporary solution would consist in adding a dummy double field as the last field of the PyBufferObject structure: struct PyBufferObject { ... double dummy; } ; and setting b->b_ptr = (void*)&b->dummy; /*was (void*(b+1)*/ In doing so, b->b_ptr will always by aligned on sizeof (double), on either 32 and 64 bit architectures. Since I'm on the buffer type problem: It would be nice (and probably easy to do) to augment the buffer protocol interface with the ability to specify the basic value type stored in the buffer. A possible list of values (enum ...) would be: - undefined (backward compatibility) - char, unsigned char, short, ....int, ... long, long long, float, double, and void* (memory address). This would enable to check at runtime the type of values stored in a buffer (and prevent missalignement buserrors, as well as catching garbage situations when improper array types are passed by means of the buffer interface [e.g.: int/float/double/short...). Frederic Giacometti Frederic Giacometti ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-18 14:35 Message: Logged In: YES user_id=93657 I wasn't looged in when I submitted the item. Don't think I'm becoming anonymous :)) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 From noreply@sourceforge.net Thu Oct 18 23:30:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 15:30:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-227699 ] Memory leaks using imp.load_module Message-ID: Bugs item #227699, was opened at 2001-01-05 12:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=227699&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Charles G Waldman (cgw) Assigned to: Barry Warsaw (bwarsaw) Summary: Memory leaks using imp.load_module Initial Comment: #!/usr/bin/env python import os import sys import imp print os.getpid() sys.path.append("/tmp") count = 0 while (count<1000): modname = "testmod%s" % count count = count + 1 filename = '/tmp/' + modname + '.py' f = open(filename, 'w') for x in range(10000): f.write('x%s=%s\n'%(x,x)) f.close() modinfo = imp.find_module(modname) print 'loading', modname m = apply(imp.load_module, ('testmod',) + modinfo) ## run "watch -n 1 ps up ## where is the pid printed out by this program ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-18 15:30 Message: Logged In: YES user_id=12800 I'm having a hard time tracking down the leak. I don't believe it's at the C layer since preliminary Insure runs don't indicate any leaked blocks (there are plenty of inuse blocks, but those generally don't count). I will note that if you comment out the imp.load_module call the memory usage remains constant. Loading each module though is going to naturally consume more memory, right? And those loaded modules aren't ever going to get freed, at least without mucking about in sys.modules. The memory consumption with ps is pretty meager, so I suspect we don't have a valid leak here, but just normal memory consumption due to importation of 10k modules. I'm running a more detailed insure run right now just to be sure, but I'm skeptical that there's really a bug here. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-02-23 11:53 Message: I can verify that memory grows with this process. I will run it under insure to get more information. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-01-18 19:43 Message: Barry...?!?! ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-01-07 15:32 Message: Assigned to Barry since he's the memory leak expert (& has the right tools). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=227699&group_id=5470 From noreply@sourceforge.net Thu Oct 18 23:43:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 15:43:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-445862 ] bsddb fails for larger amount of data Message-ID: Bugs item #445862, was opened at 2001-07-30 00:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445862&group_id=5470 Category: Extension Modules Group: None >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Barry Warsaw (bwarsaw) Summary: bsddb fails for larger amount of data Initial Comment: The attached script fails after approx. 72500 insert operations. If you vary the size of the keys and/or the values, the bug occurs earlier or later, but even with a value size of 1 the bug will occur. Probably, this explains also bug #408271 ("crash in shelve module"). Platform: W2K ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-18 15:43 Message: Logged In: YES user_id=12800 I'm finally closing this bug report. With my Mandrake 7.2-ish system I get bsddb 3 by default and there seems to be no problem inserting several hundred thousand items (I killed it after 205k+). So it's very likely that the reported problems were due to the extremely old db 1.85. Note though, that it should be okay license-wise if we wanted to distribute the latest Sleepycat library (currently 3.3.11) with Python. I've had personal conversations with the Sleepycat guys, and they've said that their read on their own license would allow this. We'd want to get it in writing first I think, and we'd have to ask about the binary-only Windows distros, but I think it would be okay. If we did this, we should also distribute Robin Dunn's excellent PyBSDDB3. No time for this in Python 2.2, but let's look at it again for Python 2.3. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-08-04 20:35 Message: Logged In: YES user_id=31435 Barry apparently forgot to assign this to himself; repaired his bug . ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-08-04 19:59 Message: Logged In: YES user_id=12800 if you can live with the licensing for sleepycat's db3, do yourself a huge favor and go to pybsddb.sf.net. robin dunn's got a very excellent, stable, new python binding, which i would like to integrate into the standard distro for the py2.2 release. it claims to support db1.85, although i've only tried it with a very recent v3.9.x. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-08-04 19:42 Message: Logged In: YES user_id=44345 I don't know anything about the history, present, or prospects for bsddb -- like, is there a more recent unencumbered version we could use? Ya got me. I've been using lib db 2 for quite awhile. They recently released lib db 3 (again, with file format incom- patibilities). I don't know the details of their license. It just comes with whatever version of Linux I happen to be running. Saw this on the Sleepycat website: The Berkeley DB 3.0 source code is available for download at no charge from Sleepycat Software's Web site, at www.sleepycat.com. It runs on all common versions of UNIX, and on Windows 95, Windows 98 and Windows NT. Berkeley DB is an Open Source product, and may be redistributed without charge in many circumstances. Licensing and pricing information are available from the company. My guess would be that you can distribute lib db 3 with the binary version of Python. I am, as they say, "not a lawyer", so YMMV. For a definitive answer I think you'll have to ask Sleepycat. Skip ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-08-04 17:02 Message: Logged In: YES user_id=31435 Skip, I reran the test after changing the open line to db = bsddb.btopen("test.dbm", "n") I killed it by hand at this point: Last i: 326577, last key:abcdef4387101.63608 because Win98SE gets mondo unstable when it starts thrashing madly to disk, and it became impossible to get any work done while this was running. I don't know anything about the history, present, or prospects for bsddb -- like, is there a more recent unencumbered version we could use? It looks like Sam's 1.85 Windows port is over 5 years old. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-04 15:59 Message: Logged In: NO According to www.sleepycat.com/historic.html, talking about bsd db: "we recommend that you avoid the following operations when using versions 1.85 and 1.86: o Btree cursor (seq and put using a cursor) operations. o Large numbers of btree duplicates (specifically, avoid migrating duplicate keys to internal pages). o Large numbers of btree deletes (you should periodically dump and rebuild the database if you delete large numbers of records). o Overwriting or deleting overflow hash key/data pairs (pairs with items larger than the page size). o Intermixing hash cursor operations with deletes. " My problem arises, I think, because I have been doing the fourth of these operations - i.e. overwriting long items in a hash. The problems others are experiencing perhaps have a similar cause, though the original problem summary says "even with a value size of 1 the bug will occur", so perhaps not. I'm now using a workaround which involves writing several shorter items, each containing a slice of the data formerly held in the one long item. For keys I use my old key with a subscript number appended. It isn't nice, but it seems to be working. Martin Gradwell. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-08-04 08:12 Message: Logged In: YES user_id=44345 Based upon the traceback Tim reported, my guess is that the exception is being raised near the end of bsddb_ass_sub. Tim, can you give it a try changing anydbm.open to bsddb.btopen? As I recall, the significant bug(s) in libdb were in the hash file implementation. It's unfortunate that anydbm has used the hash file all these years, but it's a bit late to spring that change on unsuspecting users now without going through a significant transition period. Skip ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-08-03 14:40 Message: Logged In: YES user_id=31435 Thanks for taking a look, Skip! On Win98SE it dies for me like so: ... 70000 71000 72000 Last i: 72758, last key:abcdef1691515.8934 Traceback (most recent call last): File "ka.py", line 15, in ? db[key] = val bsddb.error: (0, 'Error') test.dbm is 37,778,944 bytes at the end. I assume Anonymous has the same problem (if not, he/she should say so). On Windows we use the ancient db.1.85.win32.zip, from the "bsd db" (not "bsddb"!) link at http://www.nightmare.com/software.html I doubt Sam has done any maintenance on that in years; and afraid I don't know anything else about this. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-08-03 13:25 Message: Logged In: YES user_id=44345 What version of libdb are you using? I'm running your script on Linux at the moment. I had to change it slightly because the only machine I have available with the spare cojones to run that script is running 1.5.2 (so I call random.uniform instead of using a Random instance). On that machine I'm sort of ashamed to say I'm still running the known buggy libdb 1.85. So far I'm up to 680,000 keys with a db file of over 166MB with no problem. On my laptop running 2.1 and libdb3 (and a much more modestly performing disk drive) I gave up after about 287,000 keys. I then changed the db open call to bsddb.btopen and watched it march (slowly) up to 183,000 keys and a 32MB file on disk before I killed it. Aside from the grief it gives my disk drives, I don't see anything particularly bad happening. You didn't include a traceback with your bug report. What was printed? Perhaps it's something simple like running out of disk space. In any case, I think trying to create a libdb database of 1,000,000 sort of random keys is going to strain that package and most disk drives in any case, bugs or no bugs. My guess is that if there's a bug it's in libdb, not the bsddb module. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-03 00:50 Message: Logged In: NO Here it is: import anydbm import bsddb import random MAX = 1000000 r = random.Random(42) r.seed(1017) db = anydbm.open("test.dbm", "n") #db = bsddb.hashopen("test.dbm", "n") try: for i in xrange(0, MAX): if i % 1000 == 0: print i key = "abcdef" + str(r.uniform(0, 10 * MAX)) val = "a" * 80 + str(i) db[key] = val finally: db.close() print "Last i: %s, last key:%s" % (i,key) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-08-02 12:41 Message: Logged In: YES user_id=31435 Alas, there's no script attached -- please attach one, so we have something concrete to investigate. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-02 03:08 Message: Logged In: NO I was getting crashes in shelve module, Using NT4 (Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32). I've changed my program to re-read previously written keys fairly frequently, and I get keyerrors for keys that have definitely been written, and that gave no error a little earlier in the same program. The program doesn't contain any delete statements. The same program works when using dumbdbm instead of bsddb (but produces huge indexes), so there definitely appears to be a problem with bsddbm on windows NT. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445862&group_id=5470 From noreply@sourceforge.net Fri Oct 19 00:28:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 16:28:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-227699 ] Memory leaks using imp.load_module Message-ID: Bugs item #227699, was opened at 2001-01-05 12:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=227699&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Charles G Waldman (cgw) Assigned to: Barry Warsaw (bwarsaw) Summary: Memory leaks using imp.load_module Initial Comment: #!/usr/bin/env python import os import sys import imp print os.getpid() sys.path.append("/tmp") count = 0 while (count<1000): modname = "testmod%s" % count count = count + 1 filename = '/tmp/' + modname + '.py' f = open(filename, 'w') for x in range(10000): f.write('x%s=%s\n'%(x,x)) f.close() modinfo = imp.find_module(modname) print 'loading', modname m = apply(imp.load_module, ('testmod',) + modinfo) ## run "watch -n 1 ps up ## where is the pid printed out by this program ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 16:28 Message: Logged In: YES user_id=6380 Barry, note that he's cheating with the module name passed to load_module(): the module name is always set to 'testmod', so he's in fact overwriting the module on each load. (This acts the same way as reload(), so it's not creating new module objects -- it's loading the code in the same namespace over and over.) I see the process size grow very quickly to 15M, probably on account of the huge parse tree for those 10,000 assignment statements (much less for the 10,000 actual variables created). After that it gradually grows -- after 100 loads it's up to 27 M. Maybe Insure doesn't see this because the data is actually kept alive somewhere? ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-18 15:30 Message: Logged In: YES user_id=12800 I'm having a hard time tracking down the leak. I don't believe it's at the C layer since preliminary Insure runs don't indicate any leaked blocks (there are plenty of inuse blocks, but those generally don't count). I will note that if you comment out the imp.load_module call the memory usage remains constant. Loading each module though is going to naturally consume more memory, right? And those loaded modules aren't ever going to get freed, at least without mucking about in sys.modules. The memory consumption with ps is pretty meager, so I suspect we don't have a valid leak here, but just normal memory consumption due to importation of 10k modules. I'm running a more detailed insure run right now just to be sure, but I'm skeptical that there's really a bug here. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-02-23 11:53 Message: I can verify that memory grows with this process. I will run it under insure to get more information. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-01-18 19:43 Message: Barry...?!?! ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-01-07 15:32 Message: Assigned to Barry since he's the memory leak expert (& has the right tools). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=227699&group_id=5470 From noreply@sourceforge.net Fri Oct 19 01:42:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 17:42:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-227699 ] Memory leaks using imp.load_module Message-ID: Bugs item #227699, was opened at 2001-01-05 12:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=227699&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Charles G Waldman (cgw) Assigned to: Barry Warsaw (bwarsaw) Summary: Memory leaks using imp.load_module Initial Comment: #!/usr/bin/env python import os import sys import imp print os.getpid() sys.path.append("/tmp") count = 0 while (count<1000): modname = "testmod%s" % count count = count + 1 filename = '/tmp/' + modname + '.py' f = open(filename, 'w') for x in range(10000): f.write('x%s=%s\n'%(x,x)) f.close() modinfo = imp.find_module(modname) print 'loading', modname m = apply(imp.load_module, ('testmod',) + modinfo) ## run "watch -n 1 ps up ## where is the pid printed out by this program ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 17:42 Message: Logged In: YES user_id=6380 Using the PYTHONDUMPREFS=1 environment variable and the debug version of Python, I don't see this leaking objects except that all the file names ('/tmp/testmod0.py' etc.) seem to be saved as interned strings, which eternalizes them of course. Hm, I wonder why the filename is interned? (I ran it twice, with different counts (100 and 1000), and 1000 instead of 10,000 variables, saving the PYTHONDUMPREFS output to two different files; then diffed the output files.) So if it's leaking beyond this, it's not in the form of objects. And note that Insure++ (or any leak detection tool worth the name) doesn't consider interned strings leaks -- they are reachable via a global variable. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 16:28 Message: Logged In: YES user_id=6380 Barry, note that he's cheating with the module name passed to load_module(): the module name is always set to 'testmod', so he's in fact overwriting the module on each load. (This acts the same way as reload(), so it's not creating new module objects -- it's loading the code in the same namespace over and over.) I see the process size grow very quickly to 15M, probably on account of the huge parse tree for those 10,000 assignment statements (much less for the 10,000 actual variables created). After that it gradually grows -- after 100 loads it's up to 27 M. Maybe Insure doesn't see this because the data is actually kept alive somewhere? ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-18 15:30 Message: Logged In: YES user_id=12800 I'm having a hard time tracking down the leak. I don't believe it's at the C layer since preliminary Insure runs don't indicate any leaked blocks (there are plenty of inuse blocks, but those generally don't count). I will note that if you comment out the imp.load_module call the memory usage remains constant. Loading each module though is going to naturally consume more memory, right? And those loaded modules aren't ever going to get freed, at least without mucking about in sys.modules. The memory consumption with ps is pretty meager, so I suspect we don't have a valid leak here, but just normal memory consumption due to importation of 10k modules. I'm running a more detailed insure run right now just to be sure, but I'm skeptical that there's really a bug here. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-02-23 11:53 Message: I can verify that memory grows with this process. I will run it under insure to get more information. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-01-18 19:43 Message: Barry...?!?! ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-01-07 15:32 Message: Assigned to Barry since he's the memory leak expert (& has the right tools). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=227699&group_id=5470 From noreply@sourceforge.net Fri Oct 19 01:43:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 17:43:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-471270 ] SystemError: unknown opcode Message-ID: Bugs item #471270, was opened at 2001-10-15 03:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471270&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Niki Spahiev (nikis) >Assigned to: Guido van Rossum (gvanrossum) Summary: SystemError: unknown opcode Initial Comment: Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] on win32 ... XXX lineno: 63, opcode: 0 Traceback (most recent call last): File "\niki\pyfr\wxmain.py", line 259, in OnMenuCommand self.OnTreeCommand( item, event.GetId() ) File "\niki\pyfr\wxmain.py", line 250, in OnTreeCommand self.dd.Action( self, 'tree', self.tree.GetPyData(item), event ) File "D:\NIKI\RASKROY\SRC\MAIN2.PY", line 698, in Action cmd.Execute( view, self ) File "D:\NIKI\RASKROY\SRC\task.py", line 139, in Execute model.DelBlock( b ) File "D:\NIKI\RASKROY\SRC\raskroy.py", line 64, in DelBlock b.AdviseDel() File "D:\NIKI\RASKROY\SRC\meta.py", line 63, in AdviseDel break SystemError: unknown opcode ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 17:43 Message: Logged In: YES user_id=6380 Closed; the sample code does hyave a continue in a try block so that's the cause. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-15 06:54 Message: Logged In: YES user_id=6380 I'm guessing this is because of a bug in the continue-within-try code generation. This is fixed in 2.2. Can you verify that this doesn't break with Python 2.2a4? (python.org/2.2/). I don't have the proper input to provoke the bug in 2.1.1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471270&group_id=5470 From noreply@sourceforge.net Fri Oct 19 01:45:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 17:45:04 -0700 Subject: [Python-bugs-list] [ python-Bugs-462461 ] Some install scripts ignore ENV VARS Message-ID: Bugs item #462461, was opened at 2001-09-17 21:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462461&group_id=5470 Category: Installation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) Summary: Some install scripts ignore ENV VARS Initial Comment: I am trying to make a python 2.1.1 RPM for my linux distro. I am running into a problem during the python install. It seems that pydoc is trying to get installed into my systems /usr/bin (good thing I build rpms as a unprivledged user). From what I can tell, whatever installs pydoc seems to ignore $BINDIR and $prefix as I set them in my "make install" command. copying build/lib.linux-i686-2.1/zlib.so -> /home/miah/rpm_build/tmp/python- root/usr/lib/python2.1/lib-dynload copying build/lib.linux-i686-2.1/linuxaudiodev.so -> /home/miah/rpm_build/tmp/python- root/usr/lib/python2.1/lib-dynload running install_scripts copying build/scripts/pydoc -> /usr/bin error: /usr/bin/pydoc: Permission denied make: *** [sharedinstall] Error 1 error: Bad exit status from /home/miah/rpm_build/tmp/rpm-tmp.23763 (%install) My install command is make prefix=$RPM_BUILD_ROOT/usr \ BINDIR=$RPM_BUILD_ROOT/usr/bin \ install ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 17:45 Message: Logged In: YES user_id=6380 This is fixed in CVS for 2.2. Might be a good idea to fix this in the 2.1 branch as well. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-18 15:52 Message: Logged In: YES user_id=6380 That's a reasonable request. Since you know your needs, we might make more progress if you figured out a patch (I imagine changes to setup.py or to the Makefile are needed) and uploaded it, so we know exactly what you'd like to see changed. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-09-18 14:27 Message: Logged In: NO The reason I cannot use --prefix with configure is that programs will get that path embedded into them as they are compiled and expect for libs/modules/whatever to be in that path. The way I get all RPMS to build proper (redhat, and everybody else pretty much does this) is to build it with normal parameters (ie, configure --prefix=/usr --whatever) and then when you do your make install you simply pass a different path to the Makefile. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-18 13:05 Message: Logged In: YES user_id=21627 It appears that passing --prefix to setup.py might solve this problem. Would you like to verify this theory, and produce a patch if it solves your problem? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462461&group_id=5470 From noreply@sourceforge.net Fri Oct 19 01:48:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 17:48:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-459423 ] setup.py ignores OPT make variable Message-ID: Bugs item #459423, was opened at 2001-09-06 23:23 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459423&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: setup.py ignores OPT make variable Initial Comment: If you build Python using something like make OPT=-O6 when it gets around to building the extension modules it's forgotten (or it ignores) OPT: $ make OPT=-O3 PYTHONPATH= ./python ../setup.py build running build running build_ext building 'struct' extension gcc -g -O2 -Wall -Wstrict-prototypes -fPIC -I. -I/home/skip/src/Python-2.1.1/./Include -I/usr/local/include -IInclude/ -c /home/skip/src/Python-2.1.1/Modules/structmodule.c -o build/temp.linux-i686-2.1/structmodule.o gcc -shared build/temp.linux-i686-2.1/structmodule.o -L/usr/local/lib -o build/lib.linux-i686-2.1/struct.so ... Shouldn't setup.py recognize and use the value of OPT? ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 17:48 Message: Logged In: YES user_id=6380 Apparently setup.py doesn't pull OPT out of the environment; I added OPT='$(OPT)' to the command that invokes setup.py build, but it still built with the default OPT values. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-07 17:04 Message: Logged In: YES user_id=21627 GNU make does indeed pass all make variables to subprocesses. According to the recent POSIX drafts, this behaviour is in clear violation to POSIX make behaviour, see #438786 for details. So if anything is done to get OPT from the environment, you also need to pass it from the make variable to the environment of the subprocess, for it to work on all systems. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-09-07 09:32 Message: Logged In: YES user_id=44345 I recall that GNU make at least passes variables to sub-makes in the environment. Perhaps it does that for all programs it forks. If so, then setup.py should be able to read OPT and other interesting bits from the environment. Also, perhaps configure can be made to set OPT in the Makefile if it's found when it runs. I don't have time to look into this today but if it's not taken care of by the weekend I'll try to fix it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-07 05:34 Message: Logged In: YES user_id=6380 The setup.py script reads the Makefile but has no access to the options you pass dynamically to Make. I'm not sure it's worth fixing. You can edit the Makefile instead. But I'm not setup.py's maintainer, so I'll shut up now. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459423&group_id=5470 From noreply@sourceforge.net Fri Oct 19 02:03:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 18:03:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-459423 ] setup.py ignores OPT make variable Message-ID: Bugs item #459423, was opened at 2001-09-06 23:23 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459423&group_id=5470 Category: None >Group: Not a Bug >Status: Closed Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: setup.py ignores OPT make variable Initial Comment: If you build Python using something like make OPT=-O6 when it gets around to building the extension modules it's forgotten (or it ignores) OPT: $ make OPT=-O3 PYTHONPATH= ./python ../setup.py build running build running build_ext building 'struct' extension gcc -g -O2 -Wall -Wstrict-prototypes -fPIC -I. -I/home/skip/src/Python-2.1.1/./Include -I/usr/local/include -IInclude/ -c /home/skip/src/Python-2.1.1/Modules/structmodule.c -o build/temp.linux-i686-2.1/structmodule.o gcc -shared build/temp.linux-i686-2.1/structmodule.o -L/usr/local/lib -o build/lib.linux-i686-2.1/struct.so ... Shouldn't setup.py recognize and use the value of OPT? ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-18 18:03 Message: Logged In: YES user_id=44345 I think I finally concluded (probably after getting beat up about something similar by the Gtk folks) that the correct thing to do is to pass the desired OPT setting to the configure script. That way the Makefile is generated with the correct settings that setup.py then extracts. Marking as not-a-bug and closing... (Can you add "it's a feature" to the list of possible groups? ;-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 17:48 Message: Logged In: YES user_id=6380 Apparently setup.py doesn't pull OPT out of the environment; I added OPT='$(OPT)' to the command that invokes setup.py build, but it still built with the default OPT values. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-07 17:04 Message: Logged In: YES user_id=21627 GNU make does indeed pass all make variables to subprocesses. According to the recent POSIX drafts, this behaviour is in clear violation to POSIX make behaviour, see #438786 for details. So if anything is done to get OPT from the environment, you also need to pass it from the make variable to the environment of the subprocess, for it to work on all systems. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-09-07 09:32 Message: Logged In: YES user_id=44345 I recall that GNU make at least passes variables to sub-makes in the environment. Perhaps it does that for all programs it forks. If so, then setup.py should be able to read OPT and other interesting bits from the environment. Also, perhaps configure can be made to set OPT in the Makefile if it's found when it runs. I don't have time to look into this today but if it's not taken care of by the weekend I'll try to fix it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-07 05:34 Message: Logged In: YES user_id=6380 The setup.py script reads the Makefile but has no access to the options you pass dynamically to Make. I'm not sure it's worth fixing. You can edit the Makefile instead. But I'm not setup.py's maintainer, so I'll shut up now. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=459423&group_id=5470 From noreply@sourceforge.net Fri Oct 19 04:42:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 20:42:10 -0700 Subject: [Python-bugs-list] [ python-Bugs-472642 ] interpreter crash when import .so fails Message-ID: Bugs item #472642, was opened at 2001-10-18 20:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472642&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when import .so fails Initial Comment: The python interpreter crashes when import _socket fails (.so). This only happens when running purify (on solaris 2.8). The problem is that dlerror() returns NULL, which is then passed to strlen(). Attached is a file with the stack trace which caused the problem, and a patch to fix the problem. Neal ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472642&group_id=5470 From noreply@sourceforge.net Fri Oct 19 05:18:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 21:18:27 -0700 Subject: [Python-bugs-list] [ python-Bugs-472646 ] Cookie.py and : in key Message-ID: Bugs item #472646, was opened at 2001-10-18 21:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472646&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Cookie.py and : in key Initial Comment: An older version of Cookie.py that I used in Mailman allowed : (colon) in the keys. The version that's in Python right now does not. The question comes down to whether Cookie.py should be a strict interpretation of RFCs 2109 and 2068 (which appears to mandate such keys be enclosed in quotes) or a looser interpretation. On the one hand, it shouldn't allow you to break the standards, but OTOH most browsers allow this, and besides, it's entirely possible that you'll get ill-conformant cookie data. Be liberal in what you accept and strict in what you generate. Here's the thread in python-dev: http://mail.python.org/pipermail/python-dev/2001-October/017722.html Mostly I'm submitting this bug so it doesn't get buried in my inbox. Posted patch is attached. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472646&group_id=5470 From noreply@sourceforge.net Fri Oct 19 06:57:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 22:57:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-211710 ] socket.send() can do partial writes on some systems. Message-ID: Bugs item #211710, was opened at 2000-08-11 13:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 Category: Python Library Group: Platform-specific Status: Closed Resolution: None Priority: 5 Submitted By: scott cotton (scottc) Assigned to: Jeremy Hylton (jhylton) Summary: socket.send() can do partial writes on some systems. Initial Comment: 0811 15:57 chronis:~% uname -a FreeBSD chronis.pobox.com 4.0-STABLE FreeBSD 4.0-STABLE #0: Tue Mar 21 01:05:14 EST 2000 root@chronis.pobox.com:/usr/src/sys/compile/MYBSD i386 0811 15:57 chronis:~% 0811 15:57 chronis:~% cat scratch/sendtstsrv.py import socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind("chronis.pobox.com", 8010) while 1: s.listen(1) conn, addr = s.accept() print "connected from", addr while 1: data = conn.recv(1024) if not data: break print "done" conn.close() 0811 15:58 chronis:~% python scratch/sendtstsrv.py & [2] 76562 0811 15:58 chronis:~% cat scratch/sendtst.py #!/usr/bin/env python s = "0" * 10000000 import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect("chronis.pobox.com", 8010) sent = sock.send(s) if sent != len(s): print "sent %d/%d chars" % (sent, len(s)) else: print "sent all chars" 0811 15:58 chronis:~% python !$ python scratch/sendtst.py connected from ('208.210.124.49', 1258) sent 17520/10000000 chars done 0811 15:58 chronis:~% NOTE: there is nothing in any man page for send() under linux (RH 6.1), Solaris (SunOS 5.5.1), OSF1 V4.0, or FreeBSD{3,4} that states that send() must not perform a partial write, though of those systems, it only seems to reproducibly do partial writes under FreeBSD4.0 Stable. Additionally, W.Richard Stevens' Unix Network Programming Vol 1, second edition states that """ A read or write on a stream socket might input or output fewer bytes than requested, but this is not an error condition.""" (page 77). Later, Stevens says of send() and recv() "These two functions are similar to the standard read and write functions, but one additional argument is required". (page 354). scott ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 22:57 Message: Logged In: YES user_id=6405 Why is this bug closed? The standard library uses send() in a number of places - we've just run up against it in httplib. Again, on FreeBSD, we're seeing that send() is only sending a part of the data we want it to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-15 11:50 Message: scott-- Is there a bug here? It seems clear that the send system call is not required to send all the bytes you asked it to send. It returns the number of bytes it did send. If you want to make sure all your data is sent, you can wrap send in a while loop that calls send until all the data is consumed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:01 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 From noreply@sourceforge.net Fri Oct 19 07:04:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 23:04:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-211710 ] socket.send() can do partial writes on some systems. Message-ID: Bugs item #211710, was opened at 2000-08-11 13:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 Category: Python Library Group: Platform-specific Status: Closed Resolution: None Priority: 5 Submitted By: scott cotton (scottc) Assigned to: Jeremy Hylton (jhylton) Summary: socket.send() can do partial writes on some systems. Initial Comment: 0811 15:57 chronis:~% uname -a FreeBSD chronis.pobox.com 4.0-STABLE FreeBSD 4.0-STABLE #0: Tue Mar 21 01:05:14 EST 2000 root@chronis.pobox.com:/usr/src/sys/compile/MYBSD i386 0811 15:57 chronis:~% 0811 15:57 chronis:~% cat scratch/sendtstsrv.py import socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind("chronis.pobox.com", 8010) while 1: s.listen(1) conn, addr = s.accept() print "connected from", addr while 1: data = conn.recv(1024) if not data: break print "done" conn.close() 0811 15:58 chronis:~% python scratch/sendtstsrv.py & [2] 76562 0811 15:58 chronis:~% cat scratch/sendtst.py #!/usr/bin/env python s = "0" * 10000000 import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect("chronis.pobox.com", 8010) sent = sock.send(s) if sent != len(s): print "sent %d/%d chars" % (sent, len(s)) else: print "sent all chars" 0811 15:58 chronis:~% python !$ python scratch/sendtst.py connected from ('208.210.124.49', 1258) sent 17520/10000000 chars done 0811 15:58 chronis:~% NOTE: there is nothing in any man page for send() under linux (RH 6.1), Solaris (SunOS 5.5.1), OSF1 V4.0, or FreeBSD{3,4} that states that send() must not perform a partial write, though of those systems, it only seems to reproducibly do partial writes under FreeBSD4.0 Stable. Additionally, W.Richard Stevens' Unix Network Programming Vol 1, second edition states that """ A read or write on a stream socket might input or output fewer bytes than requested, but this is not an error condition.""" (page 77). Later, Stevens says of send() and recv() "These two functions are similar to the standard read and write functions, but one additional argument is required". (page 354). scott ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:04 Message: Logged In: YES user_id=6405 We have patched httplib.HTTPConnection.send() so it does the following: sent = 0 while sent < len(str): sent = sent + self.sock.send(str[sent:]) Could this be the default behaviour in the socket module perhaps? ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 22:57 Message: Logged In: YES user_id=6405 Why is this bug closed? The standard library uses send() in a number of places - we've just run up against it in httplib. Again, on FreeBSD, we're seeing that send() is only sending a part of the data we want it to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-15 11:50 Message: scott-- Is there a bug here? It seems clear that the send system call is not required to send all the bytes you asked it to send. It returns the number of bytes it did send. If you want to make sure all your data is sent, you can wrap send in a while loop that calls send until all the data is consumed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:01 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 From noreply@sourceforge.net Fri Oct 19 07:33:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 23:33:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-472642 ] interpreter crash when import .so fails Message-ID: Bugs item #472642, was opened at 2001-10-18 20:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472642&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when import .so fails Initial Comment: The python interpreter crashes when import _socket fails (.so). This only happens when running purify (on solaris 2.8). The problem is that dlerror() returns NULL, which is then passed to strlen(). Attached is a file with the stack trace which caused the problem, and a patch to fix the problem. Neal ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 23:33 Message: Logged In: YES user_id=21627 I find it surprising that dlerror returns NULL. According to dlerror(3DL), it will only return NULL if there was no error since the last call to dlerror. Since handle is NULL, there certainly was an error very recently (in the dlopen), and I cannot see any dlerror call in-between. In any case, your patch seems to be wrong: the bug is that dlerror returns NULL when it shouldn't. If we really need to work around this bug (which I'm not convinced that we should), then the work-around is to pass a non-null constant string to PyErr_SetString if dlerror returned null. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472642&group_id=5470 From noreply@sourceforge.net Fri Oct 19 07:43:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 18 Oct 2001 23:43:16 -0700 Subject: [Python-bugs-list] [ python-Bugs-211710 ] socket.send() can do partial writes on some systems. Message-ID: Bugs item #211710, was opened at 2000-08-11 13:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 Category: Python Library Group: Platform-specific Status: Closed Resolution: None Priority: 5 Submitted By: scott cotton (scottc) Assigned to: Jeremy Hylton (jhylton) Summary: socket.send() can do partial writes on some systems. Initial Comment: 0811 15:57 chronis:~% uname -a FreeBSD chronis.pobox.com 4.0-STABLE FreeBSD 4.0-STABLE #0: Tue Mar 21 01:05:14 EST 2000 root@chronis.pobox.com:/usr/src/sys/compile/MYBSD i386 0811 15:57 chronis:~% 0811 15:57 chronis:~% cat scratch/sendtstsrv.py import socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind("chronis.pobox.com", 8010) while 1: s.listen(1) conn, addr = s.accept() print "connected from", addr while 1: data = conn.recv(1024) if not data: break print "done" conn.close() 0811 15:58 chronis:~% python scratch/sendtstsrv.py & [2] 76562 0811 15:58 chronis:~% cat scratch/sendtst.py #!/usr/bin/env python s = "0" * 10000000 import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect("chronis.pobox.com", 8010) sent = sock.send(s) if sent != len(s): print "sent %d/%d chars" % (sent, len(s)) else: print "sent all chars" 0811 15:58 chronis:~% python !$ python scratch/sendtst.py connected from ('208.210.124.49', 1258) sent 17520/10000000 chars done 0811 15:58 chronis:~% NOTE: there is nothing in any man page for send() under linux (RH 6.1), Solaris (SunOS 5.5.1), OSF1 V4.0, or FreeBSD{3,4} that states that send() must not perform a partial write, though of those systems, it only seems to reproducibly do partial writes under FreeBSD4.0 Stable. Additionally, W.Richard Stevens' Unix Network Programming Vol 1, second edition states that """ A read or write on a stream socket might input or output fewer bytes than requested, but this is not an error condition.""" (page 77). Later, Stevens says of send() and recv() "These two functions are similar to the standard read and write functions, but one additional argument is required". (page 354). scott ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:43 Message: Logged In: YES user_id=6405 This problem is only seen on FreeBSD, it seems. Not even OSX, the BSD derivative, has this problem. However, my FreeBSD (4.3-RELEASE) socket(2) man page states: If no messages space is available at the socket to hold the message to be transmitted, then send() normally blocks, unless the socket has been placed in non-blocking I/O mode. The select(2) call may be used to de- termine when it is possible to send more data. Linux (2.4.7) has a similar message: When the message does not fit into the send buffer of the socket, send normally blocks, unless the socket has been placed in non-blocking I/O mode. In non-blocking mode it would return EAGAIN in this case. The select(2) call may be used to determine when it is possible to send more data. So "normally" is being interpreted loosly by FreeBSD in this case, it seems. A solution might be to have socket.send() try to send all the data. At a minimum, the standard library should be fixed so it works on FreeBSD. >From the Lib directory of 2.1.1: [richard@ike /usr/local/src/Python-2.1.1]% grep -r '\.send(' Lib Lib/ftplib.py: self.sock.send(line) Lib/ftplib.py: self.sock.send(line, MSG_OOB) Lib/ftplib.py: conn.send(buf) Lib/ftplib.py: conn.send(buf) Lib/gopherlib.py: s.send(selector + CRLF) Lib/httplib.py: self.sock.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send('\r\n') Lib/httplib.py: self.send(body) Lib/imaplib.py: self.sock.send('%s%s' % (data, CRLF)) Lib/imaplib.py: self.sock.send(literal) Lib/imaplib.py: self.sock.send(CRLF) Lib/nntplib.py: self.sock.send(line) Lib/poplib.py: self.sock.send('%s%s' % (line, CRLF)) Lib/smtplib.py: sendptr = sendptr + self.sock.send(str[sendptr:]) Lib/smtplib.py: self.send(str) Lib/smtplib.py: self.send(q) Lib/socket.py: self._sock.send(self._wbuf) Lib/telnetlib.py: self.sock.send(buffer) Lib/telnetlib.py: self.sock.send(IAC + WONT + opt) Lib/telnetlib.py: self.sock.send(IAC + DONT + opt) Lib/urllib.py: h.send(data) Lib/urllib.py: h.send(data) Lib/urllib2.py: h.send(data) Lib/webbrowser.py: s.send(action) Lib/test/test_asynchat.py: n = conn.send(buffer) Lib/test/test_socket.py: conn.send(data) Lib/test/test_socket.py: s.send(msg) Lib/test/test_socketserver.py: s.send(teststring) Note that smtplib handles send()'s return, where nothing else does. Note also that the above grep does not include our patch. ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:04 Message: Logged In: YES user_id=6405 We have patched httplib.HTTPConnection.send() so it does the following: sent = 0 while sent < len(str): sent = sent + self.sock.send(str[sent:]) Could this be the default behaviour in the socket module perhaps? ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 22:57 Message: Logged In: YES user_id=6405 Why is this bug closed? The standard library uses send() in a number of places - we've just run up against it in httplib. Again, on FreeBSD, we're seeing that send() is only sending a part of the data we want it to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-15 11:50 Message: scott-- Is there a bug here? It seems clear that the send system call is not required to send all the bytes you asked it to send. It returns the number of bytes it did send. If you want to make sure all your data is sent, you can wrap send in a while loop that calls send until all the data is consumed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:01 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 From noreply@sourceforge.net Fri Oct 19 08:22:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 00:22:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-472675 ] CVS socketmodule now doesn't compile Message-ID: Bugs item #472675, was opened at 2001-10-19 00:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark Favas (mfavas) Assigned to: Nobody/Anonymous (nobody) Summary: CVS socketmodule now doesn't compile Initial Comment: As of 17 October, the CVS version of socketmodule.c fails to compile on Tru64 Unix, V4.0F with Compaq's C compiler. Results: building '_socket' extension cc: Error: Modules/socketmodule.c, line 2820: An unexpected newline character is present in a string literal. (nlstring) "RAND_egd(path) -> bytes ^ cc: Warning: Modules/socketmodule.c, line 2821: Invalid token discarded. (invaltoken) \n\ ^ cc: Warning: Modules/socketmodule.c, line 2822: Invalid token discarded. (invaltoken) Queries the entropy gather daemon (EGD) on socket path. Returns number\n\ -----------------------------------------------------------------------^ cc: Warning: Modules/socketmodule.c, line 2823: Invalid token discarded. (invaltoken) of bytes read. Raises socket.sslerror if connection to EGD fails or\n\ --------------------------------------------------------------------^ cc: Error: Modules/socketmodule.c, line 2824: An unexpected newline character is present in a string literal. (nlstring) if it does provide enough data to seed PRNG."; --------------------------------------------^ cc: Error: Modules/socketmodule.c, line 3037: In this statement, "PySocket_methods" is not declared. (undeclared) m = Py_InitModule3("_socket", PySocket_methods, module_doc); ------------^ /bin/cc -O -Olimit 1500 -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I./Include -I/usr/local/include -IInclude/ -c Modules/socketmodule.c -o build/temp.osf1-V4.0-alpha-2.2/socketmodule.o WARNING: building of extension "_socket" failed: command '/bin/cc' failed with exit status 1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 From noreply@sourceforge.net Fri Oct 19 12:19:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 04:19:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-472727 ] Tkinter.mainloop() confuses IDLE Message-ID: Bugs item #472727, was opened at 2001-10-19 04:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472727&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Edward Loper (edloper) Assigned to: Guido van Rossum (gvanrossum) Summary: Tkinter.mainloop() confuses IDLE Initial Comment: Cause: 1. create a Tk window 2. call Tkinter.mainloop() 3. close the Tk window created in (1) Result: 1. IDLE doesn't return to a Python prompt. 2. IDLE won't close, unless you kill it from the task manager. Code sample: >>> import Tkinter >>> Tkinter.tk() >>> Tkinter.mainloop() Presumably, this happens because IDLE is using Tk itself, and is not isolating the Python interactive shell from itself well enough. It's possible that solving this problem would require changes to kinter? But at the very least, it should be ocumented. A workaround (posted by stefan kusch) is to use the following test to see if you're running in IDLE with: _inidle = type(sys.stdin) == types.InstanceType and \ sys.stdin.__class__.__name__ == 'PyShell' and not call mainloop() if you're in IDLE. But, of course, a real fix is preferable to a workaround :) A web search turned up several mentions of this, but I didn't see a bug report for it, so I figured I'd submit one. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472727&group_id=5470 From noreply@sourceforge.net Fri Oct 19 13:20:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 05:20:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-211710 ] socket.send() can do partial writes on some systems. Message-ID: Bugs item #211710, was opened at 2000-08-11 13:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 Category: Python Library Group: Platform-specific >Status: Open Resolution: None Priority: 5 Submitted By: scott cotton (scottc) Assigned to: Jeremy Hylton (jhylton) Summary: socket.send() can do partial writes on some systems. Initial Comment: 0811 15:57 chronis:~% uname -a FreeBSD chronis.pobox.com 4.0-STABLE FreeBSD 4.0-STABLE #0: Tue Mar 21 01:05:14 EST 2000 root@chronis.pobox.com:/usr/src/sys/compile/MYBSD i386 0811 15:57 chronis:~% 0811 15:57 chronis:~% cat scratch/sendtstsrv.py import socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind("chronis.pobox.com", 8010) while 1: s.listen(1) conn, addr = s.accept() print "connected from", addr while 1: data = conn.recv(1024) if not data: break print "done" conn.close() 0811 15:58 chronis:~% python scratch/sendtstsrv.py & [2] 76562 0811 15:58 chronis:~% cat scratch/sendtst.py #!/usr/bin/env python s = "0" * 10000000 import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect("chronis.pobox.com", 8010) sent = sock.send(s) if sent != len(s): print "sent %d/%d chars" % (sent, len(s)) else: print "sent all chars" 0811 15:58 chronis:~% python !$ python scratch/sendtst.py connected from ('208.210.124.49', 1258) sent 17520/10000000 chars done 0811 15:58 chronis:~% NOTE: there is nothing in any man page for send() under linux (RH 6.1), Solaris (SunOS 5.5.1), OSF1 V4.0, or FreeBSD{3,4} that states that send() must not perform a partial write, though of those systems, it only seems to reproducibly do partial writes under FreeBSD4.0 Stable. Additionally, W.Richard Stevens' Unix Network Programming Vol 1, second edition states that """ A read or write on a stream socket might input or output fewer bytes than requested, but this is not an error condition.""" (page 77). Later, Stevens says of send() and recv() "These two functions are similar to the standard read and write functions, but one additional argument is required". (page 354). scott ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:20 Message: Logged In: YES user_id=6380 Reopened. You have a point. (I think we closed it becase we couldn't reproduce it. Our fault for not having access to FreeBSD. :-) ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:43 Message: Logged In: YES user_id=6405 This problem is only seen on FreeBSD, it seems. Not even OSX, the BSD derivative, has this problem. However, my FreeBSD (4.3-RELEASE) socket(2) man page states: If no messages space is available at the socket to hold the message to be transmitted, then send() normally blocks, unless the socket has been placed in non-blocking I/O mode. The select(2) call may be used to de- termine when it is possible to send more data. Linux (2.4.7) has a similar message: When the message does not fit into the send buffer of the socket, send normally blocks, unless the socket has been placed in non-blocking I/O mode. In non-blocking mode it would return EAGAIN in this case. The select(2) call may be used to determine when it is possible to send more data. So "normally" is being interpreted loosly by FreeBSD in this case, it seems. A solution might be to have socket.send() try to send all the data. At a minimum, the standard library should be fixed so it works on FreeBSD. >From the Lib directory of 2.1.1: [richard@ike /usr/local/src/Python-2.1.1]% grep -r '\.send(' Lib Lib/ftplib.py: self.sock.send(line) Lib/ftplib.py: self.sock.send(line, MSG_OOB) Lib/ftplib.py: conn.send(buf) Lib/ftplib.py: conn.send(buf) Lib/gopherlib.py: s.send(selector + CRLF) Lib/httplib.py: self.sock.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send('\r\n') Lib/httplib.py: self.send(body) Lib/imaplib.py: self.sock.send('%s%s' % (data, CRLF)) Lib/imaplib.py: self.sock.send(literal) Lib/imaplib.py: self.sock.send(CRLF) Lib/nntplib.py: self.sock.send(line) Lib/poplib.py: self.sock.send('%s%s' % (line, CRLF)) Lib/smtplib.py: sendptr = sendptr + self.sock.send(str[sendptr:]) Lib/smtplib.py: self.send(str) Lib/smtplib.py: self.send(q) Lib/socket.py: self._sock.send(self._wbuf) Lib/telnetlib.py: self.sock.send(buffer) Lib/telnetlib.py: self.sock.send(IAC + WONT + opt) Lib/telnetlib.py: self.sock.send(IAC + DONT + opt) Lib/urllib.py: h.send(data) Lib/urllib.py: h.send(data) Lib/urllib2.py: h.send(data) Lib/webbrowser.py: s.send(action) Lib/test/test_asynchat.py: n = conn.send(buffer) Lib/test/test_socket.py: conn.send(data) Lib/test/test_socket.py: s.send(msg) Lib/test/test_socketserver.py: s.send(teststring) Note that smtplib handles send()'s return, where nothing else does. Note also that the above grep does not include our patch. ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:04 Message: Logged In: YES user_id=6405 We have patched httplib.HTTPConnection.send() so it does the following: sent = 0 while sent < len(str): sent = sent + self.sock.send(str[sent:]) Could this be the default behaviour in the socket module perhaps? ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 22:57 Message: Logged In: YES user_id=6405 Why is this bug closed? The standard library uses send() in a number of places - we've just run up against it in httplib. Again, on FreeBSD, we're seeing that send() is only sending a part of the data we want it to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-15 11:50 Message: scott-- Is there a bug here? It seems clear that the send system call is not required to send all the bytes you asked it to send. It returns the number of bytes it did send. If you want to make sure all your data is sent, you can wrap send in a while loop that calls send until all the data is consumed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:01 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 From noreply@sourceforge.net Fri Oct 19 13:32:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 05:32:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-472675 ] CVS socketmodule now doesn't compile Message-ID: Bugs item #472675, was opened at 2001-10-19 00:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark Favas (mfavas) >Assigned to: Guido van Rossum (gvanrossum) Summary: CVS socketmodule now doesn't compile Initial Comment: As of 17 October, the CVS version of socketmodule.c fails to compile on Tru64 Unix, V4.0F with Compaq's C compiler. Results: building '_socket' extension cc: Error: Modules/socketmodule.c, line 2820: An unexpected newline character is present in a string literal. (nlstring) "RAND_egd(path) -> bytes ^ cc: Warning: Modules/socketmodule.c, line 2821: Invalid token discarded. (invaltoken) \n\ ^ cc: Warning: Modules/socketmodule.c, line 2822: Invalid token discarded. (invaltoken) Queries the entropy gather daemon (EGD) on socket path. Returns number\n\ -----------------------------------------------------------------------^ cc: Warning: Modules/socketmodule.c, line 2823: Invalid token discarded. (invaltoken) of bytes read. Raises socket.sslerror if connection to EGD fails or\n\ --------------------------------------------------------------------^ cc: Error: Modules/socketmodule.c, line 2824: An unexpected newline character is present in a string literal. (nlstring) if it does provide enough data to seed PRNG."; --------------------------------------------^ cc: Error: Modules/socketmodule.c, line 3037: In this statement, "PySocket_methods" is not declared. (undeclared) m = Py_InitModule3("_socket", PySocket_methods, module_doc); ------------^ /bin/cc -O -Olimit 1500 -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I./Include -I/usr/local/include -IInclude/ -c Modules/socketmodule.c -o build/temp.osf1-V4.0-alpha-2.2/socketmodule.o WARNING: building of extension "_socket" failed: command '/bin/cc' failed with exit status 1 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:32 Message: Logged In: YES user_id=6380 Mark, I believe this can be fixed by adding \n\ at the end of the first line of the docstring for RAND_egd() -- line 2821. Can you test that? (Is this something you could have figured out yourself? We'd have appreciated a patch. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 From noreply@sourceforge.net Fri Oct 19 14:42:31 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 06:42:31 -0700 Subject: [Python-bugs-list] [ python-Bugs-472727 ] Tkinter.mainloop() confuses IDLE Message-ID: Bugs item #472727, was opened at 2001-10-19 04:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472727&group_id=5470 Category: IDLE Group: None >Status: Closed >Resolution: Later Priority: 5 Submitted By: Edward Loper (edloper) Assigned to: Guido van Rossum (gvanrossum) Summary: Tkinter.mainloop() confuses IDLE Initial Comment: Cause: 1. create a Tk window 2. call Tkinter.mainloop() 3. close the Tk window created in (1) Result: 1. IDLE doesn't return to a Python prompt. 2. IDLE won't close, unless you kill it from the task manager. Code sample: >>> import Tkinter >>> Tkinter.tk() >>> Tkinter.mainloop() Presumably, this happens because IDLE is using Tk itself, and is not isolating the Python interactive shell from itself well enough. It's possible that solving this problem would require changes to kinter? But at the very least, it should be ocumented. A workaround (posted by stefan kusch) is to use the following test to see if you're running in IDLE with: _inidle = type(sys.stdin) == types.InstanceType and \ sys.stdin.__class__.__name__ == 'PyShell' and not call mainloop() if you're in IDLE. But, of course, a real fix is preferable to a workaround :) A web search turned up several mentions of this, but I didn't see a bug report for it, so I figured I'd submit one. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 06:42 Message: Logged In: YES user_id=6380 There's nothing you can do about this without a serious restructuring of IDLE, to run the Python code in a separate process. We're thinking about that, and for scripts (as opposed to the >>> shell window) it's alread implemented in the "idlefork" project on SF, where most of the development on IDLE is going on. I'm going to close this bug report with a "Later" annotation. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472727&group_id=5470 From noreply@sourceforge.net Fri Oct 19 14:43:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 06:43:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-416696 ] --with-threads fail for 2.1c2 on HPUX 11 Message-ID: Bugs item #416696, was opened at 2001-04-17 07:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=416696&group_id=5470 Category: None Group: Platform-specific >Status: Closed Resolution: Out of Date Priority: 4 Submitted By: Harri Pasanen (harripasanen) Assigned to: Guido van Rossum (gvanrossum) Summary: --with-threads fail for 2.1c2 on HPUX 11 Initial Comment: ./configure --with-thread --without-gcc . . . checking for --with-threads... yes checking for mach/cthreads.h... no checking for --with-pth... no checking for pthread_create in -lpthread... no checking for pthread_detach... yes . . . make test . . . test_thread test test_thread crashed -- thread.error: can't start new thread . . . 4 tests failed: test_asynchat test_fork1 test_thread test_threadedtempfile -- The cause here is that at link time -lpthread is not included. If I manually relink with -lpthread, then no tests fail tests pass. 113 tests OK. -Harri ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 06:43 Message: Logged In: YES user_id=6380 Closed for lack of response. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-10 12:08 Message: Logged In: YES user_id=6380 Sorry, that patch no longer works. Would you mind submitting a new patch that only does a minimal change (rather than re-indenting a large block of code)? ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. This is a duplicate of bug #210665 (PR#360). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-04-19 15:16 Message: Logged In: YES user_id=31435 Assigned to Guido. Harri, can you supply a patch? You seem to know what you're doing on this box, and nobody else does: every release *something* about threads is broken on HPUX, and no patch submitted to date has ever cured this. No active Python developer runs HPUX, and I think we've all given up trying to sort out the conflicting claims of HP-UX users. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=416696&group_id=5470 From noreply@sourceforge.net Fri Oct 19 14:45:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 06:45:05 -0700 Subject: [Python-bugs-list] [ python-Bugs-231377 ] [HP-UX] Python chokes on pthread_mutex_init Message-ID: Bugs item #231377, was opened at 2001-02-07 01:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=231377&group_id=5470 Category: Threads Group: Platform-specific >Status: Closed Resolution: None Priority: 3 Submitted By: Thomas Wallgram (twallgram) Assigned to: Guido van Rossum (gvanrossum) Summary: [HP-UX] Python chokes on pthread_mutex_init Initial Comment: Hello, I need python 1.5.2 (only this version) on a HP-UX-System with threads. I started configure with the --with-threads-Option. Then I did the 'make'-command (No problems during the compilation). Then I started make test. See the output bellow: # make test (cd Modules; make -f Makefile.pre Makefile) `Makefile' is up to date. making Makefile in subdirectory . `Makefile' is up to date. making Makefile in subdirectory Parser `Makefile' is up to date. making Makefile in subdirectory Objects `Makefile' is up to date. making Makefile in subdirectory Python `Makefile' is up to date. making Makefile in subdirectory Modules (rm -f Modules/hassignal; cd Modules; make hassignal) rm -f hassignal for i in threadmodule.o regexmodule.o regexpr.o pcremodule.o pypcre.o posixmodule.o signalmodule.o arraymodule.o cmathmodule.o mathmodule.o stropmodule.o structmodule.o timemodule.o operator.o fcntlmodule.o pwdmodule.o grpmodule.o selectmodule.o socketmodule.o errnomodule.o md5module.o md5c.o shamodule.o rotormodule.o newmodule.o binascii.o parsermodule.o cStringIO.o cPickle.o config.o getpath.o main.o getbuildinfo.o; do \ if test "$i" = "signalmodule.o"; then \ echo yes >hassignal; break; \ fi; \ done cd Parser ; make OPT="-g -O2" VERSION="1.5" \ prefix="/usr/local" exec_prefix="/usr/local" all cd Objects ; make OPT="-g -O2" VERSION="1.5" \ prefix="/usr/local" exec_prefix="/usr/local" all cd Python ; make OPT="-g -O2" VERSION="1.5" \ prefix="/usr/local" exec_prefix="/usr/local" all cd Modules ; make OPT="-g -O2" VERSION="1.5" \ prefix="/usr/local" exec_prefix="/usr/local" all if test ! -f libpython1.5.a; \ then for i in Parser Objects Python Modules; do rm -f $i/add2lib; done; true; \ else true; fi for i in Parser Objects Python Modules; do \ (cd $i; make VERSION="1.5" add2lib); done `add2lib' is up to date. `add2lib' is up to date. `add2lib' is up to date. `add2lib' is up to date. rm -f ./Lib/test/*.py[co] PYTHONPATH= ./python ./Lib/test/regrtest.py pthread_mutex_init: Invalid argument sh: 11423 Memory fault(coredump) *** Error exit code 139 (ignored) PYTHONPATH= ./python ./Lib/test/regrtest.py pthread_mutex_init: Invalid argument sh: 11425 Memory fault(coredump) *** Error exit code 139 Stop. I need python 1.5.2 for compiling Zope 2.0 (which only needs python in this version with threads). It seems the same error as bug # 130029, but I can submit a coredump. Can anybody help me? You can use this HP-UX-machine for checking the compilation :-) bye Tom ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 06:45 Message: Logged In: YES user_id=6380 Closing without solution. If Python 2.2b1 still doesn't compile with threads on HP-UX, somebody should come up with a patch and submit it as a new patch to SourceForge. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-15 23:48 Message: Logged In: NO I've got the same problem. The problem was solved by changes in makefiles. You should specify some additional D-flags, and one or more libraries. If anybody need more details, I'l be responsible on gpu@swisslog-service.de regards, gpu ---------------------------------------------------------------------- Comment By: T Farrell (glory_2_god) Date: 2001-04-05 11:27 Message: Logged In: YES user_id=11441 harripasanen has the solution nailed. Unfortunately, just how to do this escapes many people. I will explain a bit more thoroughly, so people who come across this by a google search (as I did) will have more info. This is for HP-UX 11.00, Python-2.0, compiling with GNU tools: ./configure --with-threads make rm python cd Modules gcc -Wl,-E -Wl,+s -Wl,+b/lib/python2.0/lib-dynload \ python.o ../libpython2.0.a -lnsl -ldld \ -lpthread -lm -o python mv python .. make install Happy Zoping :) PS. If you built zope before, don't forget to python wo_pcgi.py ---------------------------------------------------------------------- Comment By: Harri Pasanen (harripasanen) Date: 2001-04-04 07:19 Message: Logged In: YES user_id=77088 If it is HP-UX 11.00, just perform the final link of python manually, replacing -lcma with -lpthread -Harri ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-02-09 15:50 Message: Another for the eternal HP-UX pile. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=231377&group_id=5470 From noreply@sourceforge.net Fri Oct 19 14:49:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 06:49:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-472642 ] interpreter crash when import .so fails w/ Purify Message-ID: Bugs item #472642, was opened at 2001-10-18 20:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472642&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) >Summary: interpreter crash when import .so fails w/ Purify Initial Comment: The python interpreter crashes when import _socket fails (.so). This only happens when running purify (on solaris 2.8). The problem is that dlerror() returns NULL, which is then passed to strlen(). Attached is a file with the stack trace which caused the problem, and a patch to fix the problem. Neal ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 06:49 Message: Logged In: YES user_id=6380 Note that Neal mentions Purify as a condition. I've added that to the subject. Could Purify somehow be screwing dlerror()? Not much we can do about that... On the one hand it seems wrong that PyErr_SetString() should be patched; on the other hand there's nothing wrong with making that function a bit more robust. I give the patch a +0. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 23:33 Message: Logged In: YES user_id=21627 I find it surprising that dlerror returns NULL. According to dlerror(3DL), it will only return NULL if there was no error since the last call to dlerror. Since handle is NULL, there certainly was an error very recently (in the dlopen), and I cannot see any dlerror call in-between. In any case, your patch seems to be wrong: the bug is that dlerror returns NULL when it shouldn't. If we really need to work around this bug (which I'm not convinced that we should), then the work-around is to pass a non-null constant string to PyErr_SetString if dlerror returned null. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472642&group_id=5470 From noreply@sourceforge.net Fri Oct 19 14:53:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 06:53:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-216405 ] Bug in buffer interface Message-ID: Bugs item #216405, was opened at 2000-10-09 02:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=216405&group_id=5470 Category: Python Interpreter Core Group: Feature Request Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Thomas Heller (theller) Assigned to: Guido van Rossum (gvanrossum) Summary: Bug in buffer interface Initial Comment: Consider the following code: PyObject *base = PyBuffer_New(100); PyObject *buffer = PyBuffer_FromObject(base); Py_DECREF(base); After this code is executed, buffer points to deallocated memory (because buffer does not hold a reference to base anymore). ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2001-10-19 06:53 Message: Logged In: YES user_id=11105 Fixed in Python 2.2b2 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-12 09:17 Message: Added to PEP-42. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-09 07:13 Message: Reopened. In private mail, Thomas explained things better. The missing arguments to PyBuffer_FromObject() were a typo in the bug report. The real problem is that the base is already a buffer object! Thomas writes: The problem is the following piece of code in bufferobject.c:, function _PyBuffer_FromObject: /* if the base object is another buffer, then "deref" it */ if ( PyBuffer_Check(base) ) base = ((PyBufferObject *)base)->b_base; return _PyBuffer_FromMemory(base, (char *)p + offset, size, readonly); } which should be changed to (IMO) /* if the base object is another buffer, then "deref" it */ if ( PyBuffer_Check(base) && ((PyBufferObject *)base->b_base) base = ((PyBufferObject *)base)->b_base; return _PyBuffer_FromMemory(base, (char *)p + offset, size, readonly); } If base is an object which had been created by PyBuffer_New(), then its b_base is NULL, and in this case the newly created object MUST keep the reference to base itself, and not base->b_base. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-09 06:23 Message: Make sure the base stays alive as long as the buffer. The buffer is for advanced uses -- I have a feeling you don't know what it is for and are trying to use it to solve something it isn't intended to solve. In any case this is not a topic for a bug report. ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2000-10-09 05:57 Message: I know this joke, but it really won't help me. Are we NOT going to fix this? How can I use the buffer interface? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-09 05:48 Message: Do you know the old joke that begins with "Doctor, it hurts if I do this..." ? That code is broken. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=216405&group_id=5470 From noreply@sourceforge.net Fri Oct 19 15:16:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 07:16:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-469972 ] xmlrpclib won't marshal new types Message-ID: Bugs item #469972, was opened at 2001-10-10 12:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469972&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: xmlrpclib won't marshal new types Initial Comment: Maybe xmlrpclib should be modified to allow it to marshal subclasses of builtin types (ints, strings, etc). Here's a simple example that demonstrates that it currently won't work with a subclass of str: >>> import xmlrpclib >>> class MyString(str): ... pass ... >>> s = MyString("sdfsdfsdf") >>> s 'sdfsdfsdf' >>> s.__class__ >>> xmlrpclib.dumps((s,)) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/xmlrpclib.py", line 752, in dumps data = m.dumps(params) File "/usr/local/lib/python2.2/xmlrpclib.py", line 448, in dumps self.__dump(v) File "/usr/local/lib/python2.2/xmlrpclib.py", line 459, in __dump raise TypeError, "cannot marshal %s objects" % type(value) TypeError: cannot marshal objects ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-19 07:16 Message: Logged In: YES user_id=72053 1) anything but the base types aren't part of the xmlrpc spec 2) be very very careful about unmarshalling any but basic objects, because of possible security attacks. See item 471893 for some more info. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469972&group_id=5470 From noreply@sourceforge.net Fri Oct 19 15:26:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 07:26:33 -0700 Subject: [Python-bugs-list] [ python-Bugs-472798 ] SSL in non-blocking mode Message-ID: Bugs item #472798, was opened at 2001-10-19 07:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: SSL in non-blocking mode Initial Comment: There is a memory leak when you use SSL sockets in Non-blocking mode. The fix that works for me: In socketmodule.c, in function SSL_SSLRead() Add a Py_DECREF(buf); to the default in the switch statement. Below is the code to change... Change the switch statement from: switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: return PyErr_SetFromErrno(SSLErrorObject); } To switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: Py_DECREF(buf); return PyErr_SetFromErrno(SSLErrorObject); } Note the Py_DECREF(buf); in the default case ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 From noreply@sourceforge.net Fri Oct 19 15:36:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 07:36:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-472801 ] Python Advocacy HOWTO R0.03 URLs broken Message-ID: Bugs item #472801, was opened at 2001-10-19 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472801&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ronny De Winter (rdwi) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Python Advocacy HOWTO R0.03 URLs broken Initial Comment: Python Advocacy HOWTO Release 0.03 A.M.Kuchling Some links in "3.Useful Resources" are broken, ie. http://www.scriptics.com/people/john.ousterhout/scripting.html http://www.pythonjournal.com/volume1/art-interview/ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472801&group_id=5470 From noreply@sourceforge.net Fri Oct 19 15:40:31 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 07:40:31 -0700 Subject: [Python-bugs-list] [ python-Bugs-472798 ] SSL in non-blocking mode Message-ID: Bugs item #472798, was opened at 2001-10-19 07:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Jeremy Hylton (jhylton) Summary: SSL in non-blocking mode Initial Comment: There is a memory leak when you use SSL sockets in Non-blocking mode. The fix that works for me: In socketmodule.c, in function SSL_SSLRead() Add a Py_DECREF(buf); to the default in the switch statement. Below is the code to change... Change the switch statement from: switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: return PyErr_SetFromErrno(SSLErrorObject); } To switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: Py_DECREF(buf); return PyErr_SetFromErrno(SSLErrorObject); } Note the Py_DECREF(buf); in the default case ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 07:40 Message: Logged In: YES user_id=6380 Jeremy, this is an easy one. I believe this has been fixed in 2.2b1; can you confirm that? Also, maybe this is a 2.1.2 bugfix candidate. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 From noreply@sourceforge.net Fri Oct 19 16:05:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 08:05:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-472798 ] SSL in non-blocking mode Message-ID: Bugs item #472798, was opened at 2001-10-19 07:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL in non-blocking mode Initial Comment: There is a memory leak when you use SSL sockets in Non-blocking mode. The fix that works for me: In socketmodule.c, in function SSL_SSLRead() Add a Py_DECREF(buf); to the default in the switch statement. Below is the code to change... Change the switch statement from: switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: return PyErr_SetFromErrno(SSLErrorObject); } To switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: Py_DECREF(buf); return PyErr_SetFromErrno(SSLErrorObject); } Note the Py_DECREF(buf); in the default case ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-19 08:05 Message: Logged In: YES user_id=31392 The read() function is radically different in 2.2b1. I expect the specific problem is fixed. Please file a new bug report if it still exists. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 07:40 Message: Logged In: YES user_id=6380 Jeremy, this is an easy one. I believe this has been fixed in 2.2b1; can you confirm that? Also, maybe this is a 2.1.2 bugfix candidate. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 From noreply@sourceforge.net Fri Oct 19 16:08:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 08:08:13 -0700 Subject: [Python-bugs-list] [ python-Bugs-232460 ] SSL Support (Apparently) Broken on Solaris Message-ID: Bugs item #232460, was opened at 2001-02-14 19:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 Category: Extension Modules Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Jeremy Hylton (jhylton) Summary: SSL Support (Apparently) Broken on Solaris Initial Comment: I have spent about 10 hours banging on this with no luck. Python : Python-2.0 Platform : SPARC Solaris 2.8 Compiler : Sun Workshop Pro v5.0, gcc-2.95-2 OpenSSL : openssl-0.9.6 and 0.9.5. Problem - All attempts to open a SSL connection result in an "SSL_connect error". I have tried various combinations of keys and certificates. I have looked at the Python source code extensively and added debugging to try and get more information. I have run the system using the openssl s_client and s_server testing tools. I have recompiled everything in various configurations of versions and compilers. In all cases, the same error is generated. That said, has *ANYBODY* gotten this to work on Solaris? If so, do you have any details that can be shared? Cheers, Dave P.S. I will submit a patch if I can ever get this to actually work. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-19 08:08 Message: Logged In: YES user_id=31392 RAND_add(), RAND_egd(), and RAND_status() are all available from the socket module in Python 2.2b1. So this bug should be fixed; specifically, applications can seed the PRNG if they need to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-16 09:28 Message: Logged In: YES user_id=31392 I think we'll add some RAND_xxx() methods to the Python API to allow people to seed the prng themselves. I'd like to have some mechanism in place to do the seeding autmatically, but I don't think it makes a lot of sense for us to try and maintain this in Python. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 12:34 Message: Logged In: YES user_id=21627 On PRNG problems, it seems we have two options: - wait for, then require OpenSSL 0.9.7. It will look for a prngd socket in default locations (/var/run/egd-pool, /dev/egd-pool, /etc/egd-pool and /etc/entropy); then require administrators to set up OpenSSL that it indeed finds a prngd in these locations when needed. - expose RAND_add. Exposing any other of the interfaces is pointless; on our installation, prngd runs on localhost:708 instead of a Unix domain socket, and none of the other interfaces could use such a configuration. On top of RAND_add, we could communicate with prngd ourselves, e.g. by using the RANDFILE environment variable. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-11 10:57 Message: Logged In: YES user_id=21627 Running httplib.py now crashes with Traceback (most recent call last): File "Lib/httplib.py", line 867, in ? test() File "Lib/httplib.py", line 853, in test hs.connect(host) File "Lib/httplib.py", line 698, in connect self._conn.connect() File "Lib/httplib.py", line 653, in connect ssl = socket.ssl(realsock, self.key_file, self.cert_file) socket.sslerror: (1, 'error:24064064:random number generator:SSLEAY_RAND_BYTES:PRNG not seeded') Not sure how this should be fixed, though. We use prngd here. To configure OpenSSH, I had to give prngd parameters to OpenSSH (i.e. that it should use prngd, and what the port number is). On testing certificates: It might be sufficient to generate a self-signed one, and distribute that as part of the test suite. Of course, you need to write server and client code to use the appropriate files. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-11 10:34 Message: Logged In: YES user_id=31392 I've made some recent changes that affect SSL_connect(), although the primary change is to improve the error messages. Dave and Martin, can you check the current code on Solaris? If it still fails, we'll at least get a clearer error message. BTW, does anyone have suggestions for how to test the code with key and cert files? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-07-28 10:39 Message: Logged In: YES user_id=21627 I still have the same problem, both on my own installation (both with 2.1.1, and with the CVS), as well as on cf.sf.net:/home/users/loewis/Python-2.1.1/python (you need to set LD_LIBRARY_PATH=/usr/local/ssl/lib to use this one). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-14 20:30 Message: Logged In: YES user_id=6380 David, is this still broken? On the SourceForge compile farm I can use SSL just fine. Check out my directory /home/users/gvanrossum/Python-2.1-solaris on cf.sourceforge.net. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-04-10 11:55 Message: Logged In: YES user_id=6380 So, David, you still interested in this? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-20 07:58 Message: Logged In: YES user_id=11375 Moshe checked in patch #405101 to the CVS tree. David, can you update and see if it fixed the problem for you? ---------------------------------------------------------------------- Comment By: Moshe Zadka (moshez) Date: 2001-03-18 02:30 Message: Logged In: YES user_id=11645 I just want to me-too Neil, since I submitted the original patch. I think it *did* fix problems on Solaris, as long as the user is running EGD and set up RANDFILE env. variable properly. Otherwise, you won't be getting connect errors, but a warning from Python about using a non-secure way to generate seed. (For the record, the problem is that Solaris doesn't have a /dev/urandom like Linux does) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-02 07:49 Message: Logged In: YES user_id=35752 It looks like patch "[ #405101 ] Add Random Seeding to OpenSSL" might fix this. Assigning to Andrew since he is assigned that patch too. ---------------------------------------------------------------------- Comment By: David M. Beazley (beazley) Date: 2001-02-15 06:12 Message: A followup on this bug.... It appears that Python is not properly seeding the OpenSSL random number generator when it creates a secure socket. This, in turn, causes the SSL_connect() function to fail due to improper random number seeding (although the error wasn't obvious--at least not to me). This is also due to an apparent "feature" of Solaris not providing a usable /dev/random device. A simple fix is to modify init_socket() in socketmodule.c to include a call to RAND_seed() like this: #ifdef USE_SSL SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); /* Sick hack for Solaris */ { char bogus[64]; /* Fill in bogus with some random noise */ ... RAND_seed(bogus, 64); } ... #endif Presumably one would need to think about the randomness actually passed to RAND_seed() (I have done nothing above). Here are related comments from the OpenSSL FAQ: "Cryptographic software needs a source of unpredictable data to work correctly. Many open source operating systems provide a "randomness device" that serves this purpose. On other systems, applications have to call the RAND_add() or RAND_seed() function with appropriate data before generating keys or performing public key encryption. Some broken applications do not do this. As of version 0.9.5, the OpenSSL functions that need randomness report an error if the random number generator has not been seeded with at least 128 bits of randomness. If this error occurs, please contact the author of the application you are using. It is likely that it never worked correctly. OpenSSL 0.9.5 and later make the error visible by refusing to perform potentially insecure encryption." Python-2.1a2 does not appear to include a fix, but I can't test it to find out (since 2.1a2 doesn't compile on my machine). Cheers, Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232460&group_id=5470 From noreply@sourceforge.net Fri Oct 19 16:18:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 08:18:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-472798 ] SSL in non-blocking mode Message-ID: Bugs item #472798, was opened at 2001-10-19 07:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Closed Resolution: Out of Date Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL in non-blocking mode Initial Comment: There is a memory leak when you use SSL sockets in Non-blocking mode. The fix that works for me: In socketmodule.c, in function SSL_SSLRead() Add a Py_DECREF(buf); to the default in the switch statement. Below is the code to change... Change the switch statement from: switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: return PyErr_SetFromErrno(SSLErrorObject); } To switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: Py_DECREF(buf); return PyErr_SetFromErrno(SSLErrorObject); } Note the Py_DECREF(buf); in the default case ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 08:18 Message: Logged In: YES user_id=6380 I've plugged this leak in the 2.1.2 branch in the way suggested by the bug report -- it is obviously correct. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-19 08:05 Message: Logged In: YES user_id=31392 The read() function is radically different in 2.2b1. I expect the specific problem is fixed. Please file a new bug report if it still exists. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 07:40 Message: Logged In: YES user_id=6380 Jeremy, this is an easy one. I believe this has been fixed in 2.2b1; can you confirm that? Also, maybe this is a 2.1.2 bugfix candidate. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 From noreply@sourceforge.net Fri Oct 19 16:23:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 08:23:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-472798 ] SSL in non-blocking mode Message-ID: Bugs item #472798, was opened at 2001-10-19 07:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Closed Resolution: Out of Date Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL in non-blocking mode Initial Comment: There is a memory leak when you use SSL sockets in Non-blocking mode. The fix that works for me: In socketmodule.c, in function SSL_SSLRead() Add a Py_DECREF(buf); to the default in the switch statement. Below is the code to change... Change the switch statement from: switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: return PyErr_SetFromErrno(SSLErrorObject); } To switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: Py_DECREF(buf); return PyErr_SetFromErrno(SSLErrorObject); } Note the Py_DECREF(buf); in the default case ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-19 08:23 Message: Logged In: YES user_id=31392 Actually, the new SSL code should all be merged into the 2.1.2 branch. The old code is mostly broken. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 08:18 Message: Logged In: YES user_id=6380 I've plugged this leak in the 2.1.2 branch in the way suggested by the bug report -- it is obviously correct. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-19 08:05 Message: Logged In: YES user_id=31392 The read() function is radically different in 2.2b1. I expect the specific problem is fixed. Please file a new bug report if it still exists. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 07:40 Message: Logged In: YES user_id=6380 Jeremy, this is an easy one. I believe this has been fixed in 2.2b1; can you confirm that? Also, maybe this is a 2.1.2 bugfix candidate. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 From noreply@sourceforge.net Fri Oct 19 16:26:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 08:26:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-472798 ] SSL in non-blocking mode Message-ID: Bugs item #472798, was opened at 2001-10-19 07:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Closed Resolution: Out of Date Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: SSL in non-blocking mode Initial Comment: There is a memory leak when you use SSL sockets in Non-blocking mode. The fix that works for me: In socketmodule.c, in function SSL_SSLRead() Add a Py_DECREF(buf); to the default in the switch statement. Below is the code to change... Change the switch statement from: switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: return PyErr_SetFromErrno(SSLErrorObject); } To switch (res) { case SSL_ERROR_NONE: assert(count > 0); break; case SSL_ERROR_ZERO_RETURN: /* normal EOF */ assert(count == 0); break; default: Py_DECREF(buf); return PyErr_SetFromErrno(SSLErrorObject); } Note the Py_DECREF(buf); in the default case ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 08:26 Message: Logged In: YES user_id=6380 If you say so. The RAND stuff is a new API though, isn't it? I'll leave this for the 2.1.2 release manager (NOT ME!) to decide. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-19 08:23 Message: Logged In: YES user_id=31392 Actually, the new SSL code should all be merged into the 2.1.2 branch. The old code is mostly broken. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 08:18 Message: Logged In: YES user_id=6380 I've plugged this leak in the 2.1.2 branch in the way suggested by the bug report -- it is obviously correct. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-19 08:05 Message: Logged In: YES user_id=31392 The read() function is radically different in 2.2b1. I expect the specific problem is fixed. Please file a new bug report if it still exists. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 07:40 Message: Logged In: YES user_id=6380 Jeremy, this is an easy one. I believe this has been fixed in 2.2b1; can you confirm that? Also, maybe this is a 2.1.2 bugfix candidate. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472798&group_id=5470 From noreply@sourceforge.net Fri Oct 19 16:47:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 08:47:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-471834 ] Nomenclature in os module, exec*() Message-ID: Bugs item #471834, was opened at 2001-10-16 12:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471834&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 3 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Nomenclature in os module, exec*() Initial Comment: Both Stevens and the Linux manpages make a small but important distinction in the first argument to execvp() and execlp() verses the other exec*() functions. The `p' version takes a file name, which is used in a $PATH search if the argument does not contain a slash. The others take a path argument for which $PATH is not searched. The documentation for execvp() and execlp() in the os module should mimic this nomenclature. I.e. `file' for execvp(), execlp() -- and for good measure, our own hybrid execvpe(). It should be `path' for execl(), execv(), execle(), and execve(). ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-19 08:47 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/libos.tex revision 1.70. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471834&group_id=5470 From noreply@sourceforge.net Fri Oct 19 17:00:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 09:00:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-469972 ] xmlrpclib won't marshal new types Message-ID: Bugs item #469972, was opened at 2001-10-10 12:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469972&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: xmlrpclib won't marshal new types Initial Comment: Maybe xmlrpclib should be modified to allow it to marshal subclasses of builtin types (ints, strings, etc). Here's a simple example that demonstrates that it currently won't work with a subclass of str: >>> import xmlrpclib >>> class MyString(str): ... pass ... >>> s = MyString("sdfsdfsdf") >>> s 'sdfsdfsdf' >>> s.__class__ >>> xmlrpclib.dumps((s,)) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/xmlrpclib.py", line 752, in dumps data = m.dumps(params) File "/usr/local/lib/python2.2/xmlrpclib.py", line 448, in dumps self.__dump(v) File "/usr/local/lib/python2.2/xmlrpclib.py", line 459, in __dump raise TypeError, "cannot marshal %s objects" % type(value) TypeError: cannot marshal objects ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-19 09:00 Message: Logged In: YES user_id=44345 I don't think #471893 applies here. While xmlrpclib will marshal the dict of an instance object, it is seen as a marshalled dict at the other end and would be unmarshalled as a dict without special effort on the part of the programmer. XML-RPC only supports a small, fixed set of types: ints, booleans, floats, date-time strings, lists and dicts. My concern with allowing XML-RPC marshalling of a subclass of strings (for example), is that the programmer might be led to believe their extra instance attributes would be marshalled as well. While I submitted this bug report, I was simply recording someone else's observation. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-19 07:16 Message: Logged In: YES user_id=72053 1) anything but the base types aren't part of the xmlrpc spec 2) be very very careful about unmarshalling any but basic objects, because of possible security attacks. See item 471893 for some more info. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469972&group_id=5470 From noreply@sourceforge.net Fri Oct 19 18:26:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 10:26:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-472004 ] socket.makefile() missing link to open() Message-ID: Bugs item #472004, was opened at 2001-10-17 01:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472004&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: socket.makefile() missing link to open() Initial Comment: the socket.makefile() docs refer to the arguments of "the builtin open() function", but there is no hyperlink. it would be convenient. -- erno@iki.fi ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-19 10:26 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/libsocket.tex revision 1.56. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472004&group_id=5470 From noreply@sourceforge.net Fri Oct 19 18:30:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 10:30:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-472801 ] Python Advocacy HOWTO R0.03 URLs broken Message-ID: Bugs item #472801, was opened at 2001-10-19 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472801&group_id=5470 Category: Documentation >Group: 3rd Party >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Ronny De Winter (rdwi) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Python Advocacy HOWTO R0.03 URLs broken Initial Comment: Python Advocacy HOWTO Release 0.03 A.M.Kuchling Some links in "3.Useful Resources" are broken, ie. http://www.scriptics.com/people/john.ousterhout/scripting.html http://www.pythonjournal.com/volume1/art-interview/ ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-19 10:30 Message: Logged In: YES user_id=3066 This bug was filed with the wrong project, and there does not appear to be a way to migrate bugs to other projects any more. Please file the bug with the py-howto projects on SourceForge: http://sourceforge.net/projects/py-howto/ Thanks! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472801&group_id=5470 From noreply@sourceforge.net Fri Oct 19 18:32:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 10:32:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-472646 ] Cookie.py and : in key Message-ID: Bugs item #472646, was opened at 2001-10-18 21:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472646&group_id=5470 Category: Python Library Group: None Status: Open >Resolution: Accepted Priority: 5 Submitted By: Barry Warsaw (bwarsaw) >Assigned to: Barry Warsaw (bwarsaw) Summary: Cookie.py and : in key Initial Comment: An older version of Cookie.py that I used in Mailman allowed : (colon) in the keys. The version that's in Python right now does not. The question comes down to whether Cookie.py should be a strict interpretation of RFCs 2109 and 2068 (which appears to mandate such keys be enclosed in quotes) or a looser interpretation. On the one hand, it shouldn't allow you to break the standards, but OTOH most browsers allow this, and besides, it's entirely possible that you'll get ill-conformant cookie data. Be liberal in what you accept and strict in what you generate. Here's the thread in python-dev: http://mail.python.org/pipermail/python-dev/2001-October/017722.html Mostly I'm submitting this bug so it doesn't get buried in my inbox. Posted patch is attached. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-19 10:32 Message: Logged In: YES user_id=3066 This looks good to me. Please check it in and close the bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472646&group_id=5470 From noreply@sourceforge.net Fri Oct 19 19:49:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 11:49:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-472881 ] distutils does not deduce dependencies Message-ID: Bugs item #472881, was opened at 2001-10-19 11:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472881&group_id=5470 Category: Distutils Group: None Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: A.M. Kuchling (akuchling) Summary: distutils does not deduce dependencies Initial Comment: I just "cvs up"d my Python tree and executed "make". Nothing much changed, except patchlevel.h. Make got this right and rebuilt everything (patchlevel.h is included from Python.h, so changing patchlevel.h should cause make to rebuild just about everything). Distutils failed to notice this, however, so it didn't rebuild any modules. In this case, I think the failure to rebuild is probably innocuous, but I'm not at all confident it will do the right thing if I modify some other file. For example, I've noticed that it's not sufficient to delete a module's .o file when you want to rebuild it. Distutils still thinks the .so file is okay. If distutils is going to take the place of make I think it's going to need to do a much better job deducing file dependencies or provide a robust way for a person to tell it what those dependencies are. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472881&group_id=5470 From noreply@sourceforge.net Fri Oct 19 23:31:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 15:31:56 -0700 Subject: [Python-bugs-list] [ python-Bugs-472940 ] can't getattr() attribute shown by dir() Message-ID: Bugs item #472940, was opened at 2001-10-19 15:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeremy Hylton (jhylton) Assigned to: Nobody/Anonymous (nobody) Summary: can't getattr() attribute shown by dir() Initial Comment: I occasionally write introspective code that does something like: for attr in dir(obj): print attr, getattr(obj, attr) I ran into a problem with it, though: >>> ArithmeticError.__init__ >>> dir(ArithmeticError.__init__) ['__call__', '__class__', '__cmp__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__name__', '__new__', '__reduce__', '__repr__', '__setattr__', '__str__', 'im_class', 'im_func', 'im_self'] >>> '__dict__' in dir(ArithmeticError.__init__) 1 >>> ArithmeticError.__init__.__dict__ Traceback (most recent call last): File "", line 1, in ? AttributeError: 'builtin_function_or_method' object has no attribute '__dict__' ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 From noreply@sourceforge.net Sat Oct 20 00:29:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 16:29:20 -0700 Subject: [Python-bugs-list] [ python-Bugs-472956 ] UMR when there is a syntax error Message-ID: Bugs item #472956, was opened at 2001-10-19 16:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472956&group_id=5470 Category: Parser/Compiler Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: UMR when there is a syntax error Initial Comment: perrdetail.token is unitialized when there is a syntax error in a file. Call chain during the error is: err_input [pythonrun.c:1207] PyParser_SimpleParseFile [pythonrun.c:1158] load_source_module [import.c:634] load_module [import.c:1318] imp_load_module [import.c:2310] PyCFunction_Call [methodobject.c:101] The attached patch fixes the problem. Neal ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472956&group_id=5470 From noreply@sourceforge.net Sat Oct 20 05:42:30 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 19 Oct 2001 21:42:30 -0700 Subject: [Python-bugs-list] [ python-Bugs-473009 ] binascii_b2a_base64() improper str limit Message-ID: Bugs item #473009, was opened at 2001-10-19 21:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Dave Cinege (dcinege) Assigned to: Nobody/Anonymous (nobody) Summary: binascii_b2a_base64() improper str limit Initial Comment: Modules/binascii.c binascii_b2a_base64() contains the following restrictive code: if ( bin_len > BASE64_MAXBIN ) { PyErr_SetString(Error, "Too much data for base64 line"); return NULL; } This is an error. The base64 method of encoding data has no length limitation. The MIME message RCF has such a limitation of base64 encoded data. The function should not assume it's only input must be MIME compatible. The base64 python module itself is designed for MIME I/O only, and properly limits itself. The binascii function should be left raw. binascii_a2b_base64() properly accepts input of any size. How I came across this bug: I use base64 to ascii armor binary data in log entries in a distributed network monitoring system. For the sake of ease of parsing (human and machine) all log entries are delimited by a single line. I commonly have unbroken base64 encoded fields of 64KB in size or greater. Unfortunatly I am unable to encode this data like this: result64 = binascii.b2a_base64(s) I must do this: result64 = re.sub('[ |\n]','',base64.encodestring(s)) Which is *much* slower. : < I feel this is an outright bug and should be corrected. If their is some argument for backward compatibly an optional function argument should be present to allow bypassing this limitation. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 From noreply@sourceforge.net Sat Oct 20 09:56:30 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 01:56:30 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Python Library Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Guido van Rossum (gvanrossum) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 01:56 Message: Logged In: YES user_id=350896 You are right. In the current version exceptions are not properly handled: The new thread for a request does not catch exceptions. However, the new thread should handle the exception. Therefore: I suggest to exactly copy the behaviour in "handle_request" in "process_request_thread" (see patch): All exceptions are caught, in case of an exception in either finish_request or close_request the "handle_error" method is called and then "close_request" is done. This means, that when "close_request" causes the exception, it is tried again after "handle_error". Another possibility would be to move "close_request" out of the "try"-clause... During testing this I found another problem: In case of an exception the files "rfile" and "wfile" in the BaseRequestHandler instance are not closed. Because they contain dup'ed file descriptors of the socket, the connection remains open, probably until this instance is garbage collected. Therefore I suggest the following to resolve this problem: Move the call to the "finish" method in "__init__" of BaseRequestHandler into the "finally" clause. Then the files are properly closed and the exception is propagated further up. This is also in the patch. This however changes the behaviour of this class also in the non-Threading case from the user's point of view (if the finish method is overloaded)! I do not know whether you want to introduce such a change into the code between alpha and beta releases... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 11:01 Message: Logged In: YES user_id=6380 OK. That looks good. I've checked this into CVS, in time for 2.2b1. Question: what should be done if an exception occurs in finish_request()? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-17 02:10 Message: Logged In: YES user_id=350896 This is a context diff of a proposal for a patch relative to the current state of CVS. I have to admit that I tested this only with 2.2a4 and not with the current CVS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 17:26 Message: Logged In: YES user_id=6380 OK. It would be a great help if you could upload a patch (a context diff) relative to the current state of CVS, or the 2.2a4 release if you must. When uploading, don't forget to check the file upload checkbox. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Sat Oct 20 09:57:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 01:57:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Python Library Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Guido van Rossum (gvanrossum) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 01:57 Message: Logged In: YES user_id=350896 You are right. In the current version exceptions are not properly handled: The new thread for a request does not catch exceptions. However, the new thread should handle the exception. Therefore: I suggest to exactly copy the behaviour in "handle_request" in "process_request_thread" (see patch): All exceptions are caught, in case of an exception in either finish_request or close_request the "handle_error" method is called and then "close_request" is done. This means, that when "close_request" causes the exception, it is tried again after "handle_error". Another possibility would be to move "close_request" out of the "try"-clause... During testing this I found another problem: In case of an exception the files "rfile" and "wfile" in the BaseRequestHandler instance are not closed. Because they contain dup'ed file descriptors of the socket, the connection remains open, probably until this instance is garbage collected. Therefore I suggest the following to resolve this problem: Move the call to the "finish" method in "__init__" of BaseRequestHandler into the "finally" clause. Then the files are properly closed and the exception is propagated further up. This is also in the patch. This however changes the behaviour of this class also in the non-Threading case from the user's point of view (if the finish method is overloaded)! I do not know whether you want to introduce such a change into the code between alpha and beta releases... ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 01:56 Message: Logged In: YES user_id=350896 You are right. In the current version exceptions are not properly handled: The new thread for a request does not catch exceptions. However, the new thread should handle the exception. Therefore: I suggest to exactly copy the behaviour in "handle_request" in "process_request_thread" (see patch): All exceptions are caught, in case of an exception in either finish_request or close_request the "handle_error" method is called and then "close_request" is done. This means, that when "close_request" causes the exception, it is tried again after "handle_error". Another possibility would be to move "close_request" out of the "try"-clause... During testing this I found another problem: In case of an exception the files "rfile" and "wfile" in the BaseRequestHandler instance are not closed. Because they contain dup'ed file descriptors of the socket, the connection remains open, probably until this instance is garbage collected. Therefore I suggest the following to resolve this problem: Move the call to the "finish" method in "__init__" of BaseRequestHandler into the "finally" clause. Then the files are properly closed and the exception is propagated further up. This is also in the patch. This however changes the behaviour of this class also in the non-Threading case from the user's point of view (if the finish method is overloaded)! I do not know whether you want to introduce such a change into the code between alpha and beta releases... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 11:01 Message: Logged In: YES user_id=6380 OK. That looks good. I've checked this into CVS, in time for 2.2b1. Question: what should be done if an exception occurs in finish_request()? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-17 02:10 Message: Logged In: YES user_id=350896 This is a context diff of a proposal for a patch relative to the current state of CVS. I have to admit that I tested this only with 2.2a4 and not with the current CVS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 17:26 Message: Logged In: YES user_id=6380 OK. It would be a great help if you could upload a patch (a context diff) relative to the current state of CVS, or the 2.2a4 release if you must. When uploading, don't forget to check the file upload checkbox. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Sat Oct 20 10:02:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 02:02:33 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Python Library Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Guido van Rossum (gvanrossum) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 02:02 Message: Logged In: YES user_id=350896 Sorry for accidentally submitting my last message twice. In the first there was an error with the patch. The comments are identical. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 01:57 Message: Logged In: YES user_id=350896 You are right. In the current version exceptions are not properly handled: The new thread for a request does not catch exceptions. However, the new thread should handle the exception. Therefore: I suggest to exactly copy the behaviour in "handle_request" in "process_request_thread" (see patch): All exceptions are caught, in case of an exception in either finish_request or close_request the "handle_error" method is called and then "close_request" is done. This means, that when "close_request" causes the exception, it is tried again after "handle_error". Another possibility would be to move "close_request" out of the "try"-clause... During testing this I found another problem: In case of an exception the files "rfile" and "wfile" in the BaseRequestHandler instance are not closed. Because they contain dup'ed file descriptors of the socket, the connection remains open, probably until this instance is garbage collected. Therefore I suggest the following to resolve this problem: Move the call to the "finish" method in "__init__" of BaseRequestHandler into the "finally" clause. Then the files are properly closed and the exception is propagated further up. This is also in the patch. This however changes the behaviour of this class also in the non-Threading case from the user's point of view (if the finish method is overloaded)! I do not know whether you want to introduce such a change into the code between alpha and beta releases... ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 01:56 Message: Logged In: YES user_id=350896 You are right. In the current version exceptions are not properly handled: The new thread for a request does not catch exceptions. However, the new thread should handle the exception. Therefore: I suggest to exactly copy the behaviour in "handle_request" in "process_request_thread" (see patch): All exceptions are caught, in case of an exception in either finish_request or close_request the "handle_error" method is called and then "close_request" is done. This means, that when "close_request" causes the exception, it is tried again after "handle_error". Another possibility would be to move "close_request" out of the "try"-clause... During testing this I found another problem: In case of an exception the files "rfile" and "wfile" in the BaseRequestHandler instance are not closed. Because they contain dup'ed file descriptors of the socket, the connection remains open, probably until this instance is garbage collected. Therefore I suggest the following to resolve this problem: Move the call to the "finish" method in "__init__" of BaseRequestHandler into the "finally" clause. Then the files are properly closed and the exception is propagated further up. This is also in the patch. This however changes the behaviour of this class also in the non-Threading case from the user's point of view (if the finish method is overloaded)! I do not know whether you want to introduce such a change into the code between alpha and beta releases... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 11:01 Message: Logged In: YES user_id=6380 OK. That looks good. I've checked this into CVS, in time for 2.2b1. Question: what should be done if an exception occurs in finish_request()? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-17 02:10 Message: Logged In: YES user_id=350896 This is a context diff of a proposal for a patch relative to the current state of CVS. I have to admit that I tested this only with 2.2a4 and not with the current CVS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 17:26 Message: Logged In: YES user_id=6380 OK. It would be a great help if you could upload a patch (a context diff) relative to the current state of CVS, or the 2.2a4 release if you must. When uploading, don't forget to check the file upload checkbox. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Sat Oct 20 10:22:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 02:22:28 -0700 Subject: [Python-bugs-list] [ python-Bugs-473060 ] Doc/ doesn't compile in 2.2b1 Message-ID: Bugs item #473060, was opened at 2001-10-20 02:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 Category: Build Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Masanori Omote (omote) Assigned to: Nobody/Anonymous (nobody) Summary: Doc/ doesn't compile in 2.2b1 Initial Comment: Documentations in the distribution tar file of 2.2b1 doesn't compile. Doc/ of a snapshot tar ball on 10/19/01 compiles correctly. Materials under Doc/ in both tar balls are the same according to diff -r check. Containing a dot (.) in the name of the top directory of 2.2b1 leads to a LaTeX2HTML error, according to its error message. It wouldn't be a bug because the README file in Doc directory says that a user should "twiddle" Doc/Makefile. But I hope all the documentation can be easily compiled from the next distribution files. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 From noreply@sourceforge.net Sat Oct 20 14:01:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 06:01:51 -0700 Subject: [Python-bugs-list] [ python-Bugs-472940 ] can't getattr() attribute shown by dir() Message-ID: Bugs item #472940, was opened at 2001-10-19 15:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeremy Hylton (jhylton) >Assigned to: Tim Peters (tim_one) Summary: can't getattr() attribute shown by dir() Initial Comment: I occasionally write introspective code that does something like: for attr in dir(obj): print attr, getattr(obj, attr) I ran into a problem with it, though: >>> ArithmeticError.__init__ >>> dir(ArithmeticError.__init__) ['__call__', '__class__', '__cmp__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__name__', '__new__', '__reduce__', '__repr__', '__setattr__', '__str__', 'im_class', 'im_func', 'im_self'] >>> '__dict__' in dir(ArithmeticError.__init__) 1 >>> ArithmeticError.__init__.__dict__ Traceback (most recent call last): File "", line 1, in ? AttributeError: 'builtin_function_or_method' object has no attribute '__dict__' ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 06:01 Message: Logged In: YES user_id=6380 For Tim. Is it possible that the __dict__ comes from looking at all the attributes of foo.__class__? ArithmeticError.__init__.__class__ has a __dict__ attribute. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 From noreply@sourceforge.net Sat Oct 20 14:30:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 06:30:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-473009 ] binascii_b2a_base64() improper str limit Message-ID: Bugs item #473009, was opened at 2001-10-19 21:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Dave Cinege (dcinege) Assigned to: Nobody/Anonymous (nobody) Summary: binascii_b2a_base64() improper str limit Initial Comment: Modules/binascii.c binascii_b2a_base64() contains the following restrictive code: if ( bin_len > BASE64_MAXBIN ) { PyErr_SetString(Error, "Too much data for base64 line"); return NULL; } This is an error. The base64 method of encoding data has no length limitation. The MIME message RCF has such a limitation of base64 encoded data. The function should not assume it's only input must be MIME compatible. The base64 python module itself is designed for MIME I/O only, and properly limits itself. The binascii function should be left raw. binascii_a2b_base64() properly accepts input of any size. How I came across this bug: I use base64 to ascii armor binary data in log entries in a distributed network monitoring system. For the sake of ease of parsing (human and machine) all log entries are delimited by a single line. I commonly have unbroken base64 encoded fields of 64KB in size or greater. Unfortunatly I am unable to encode this data like this: result64 = binascii.b2a_base64(s) I must do this: result64 = re.sub('[ |\n]','',base64.encodestring(s)) Which is *much* slower. : < I feel this is an outright bug and should be corrected. If their is some argument for backward compatibly an optional function argument should be present to allow bypassing this limitation. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:30 Message: Logged In: YES user_id=21627 Can you cite any relevant standard that defines base64 to work in that way? Base64 is defined in RFC 2045 section 6.8., which clearly says The encoded output stream must be represented in lines of no more than 76 characters each. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 From noreply@sourceforge.net Sat Oct 20 14:31:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 06:31:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-473060 ] Doc/ doesn't compile in 2.2b1 Message-ID: Bugs item #473060, was opened at 2001-10-20 02:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 Category: Build Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Masanori Omote (omote) Assigned to: Nobody/Anonymous (nobody) Summary: Doc/ doesn't compile in 2.2b1 Initial Comment: Documentations in the distribution tar file of 2.2b1 doesn't compile. Doc/ of a snapshot tar ball on 10/19/01 compiles correctly. Materials under Doc/ in both tar balls are the same according to diff -r check. Containing a dot (.) in the name of the top directory of 2.2b1 leads to a LaTeX2HTML error, according to its error message. It wouldn't be a bug because the README file in Doc directory says that a user should "twiddle" Doc/Makefile. But I hope all the documentation can be easily compiled from the next distribution files. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:31 Message: Logged In: YES user_id=21627 Can you give more details as to how exactly compilation fails? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 From noreply@sourceforge.net Sat Oct 20 14:39:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 06:39:34 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Nobody/Anonymous (nobody) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Sat Oct 20 15:23:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 07:23:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) >Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Sat Oct 20 15:28:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 07:28:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-472956 ] UMR when there is a syntax error Message-ID: Bugs item #472956, was opened at 2001-10-19 16:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472956&group_id=5470 Category: Parser/Compiler Group: Python 2.2 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Guido van Rossum (gvanrossum) Summary: UMR when there is a syntax error Initial Comment: perrdetail.token is unitialized when there is a syntax error in a file. Call chain during the error is: err_input [pythonrun.c:1207] PyParser_SimpleParseFile [pythonrun.c:1158] load_source_module [import.c:634] load_module [import.c:1318] imp_load_module [import.c:2310] PyCFunction_Call [methodobject.c:101] The attached patch fixes the problem. Neal ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:28 Message: Logged In: YES user_id=6380 Thanks -- good job! Applied to CVS. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472956&group_id=5470 From noreply@sourceforge.net Sat Oct 20 16:41:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 08:41:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Michael Piotrowski (mxp) Date: 2001-10-20 08:41 Message: Logged In: YES user_id=354556 Thist patch has no effect on threads. However, threads do work (i.e., are recognized by configure) on HP-UX 11 since 2.2b1 (I just wanted to submit a patch to the 2.2a4 configure.in when 2.2b1 was released). I am willing to support you regarding HP-UX, if necessary (there still seem to be some open issues). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Sat Oct 20 17:17:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 09:17:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-473060 ] Doc/ doesn't compile in 2.2b1 Message-ID: Bugs item #473060, was opened at 2001-10-20 02:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 Category: Build Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Masanori Omote (omote) Assigned to: Nobody/Anonymous (nobody) Summary: Doc/ doesn't compile in 2.2b1 Initial Comment: Documentations in the distribution tar file of 2.2b1 doesn't compile. Doc/ of a snapshot tar ball on 10/19/01 compiles correctly. Materials under Doc/ in both tar balls are the same according to diff -r check. Containing a dot (.) in the name of the top directory of 2.2b1 leads to a LaTeX2HTML error, according to its error message. It wouldn't be a bug because the README file in Doc directory says that a user should "twiddle" Doc/Makefile. But I hope all the documentation can be easily compiled from the next distribution files. ---------------------------------------------------------------------- >Comment By: Masanori Omote (omote) Date: 2001-10-20 09:17 Message: Logged In: YES user_id=232907 Additional info of omote's case: System: RH 7.1 Python: 2.1.1 (2.2b1 not yet installed) LaTeX2HTML: 2K.1beta (1.47) When I compile Doc/ in the extracted tree of 2.2b1 distribution tar file, it fails. It doesn't fail when I move Doc/ directory to any other directory which path doesn't contain a dot. Following is a failure example to compile 'doc'. % pwd /home/masa/ftp/python/Python-2.2b1/Doc % make doc python tools/mkhowto --html --about html/stdabout.dat --address "See About this document... for information on suggesting changes." --up-link ../index.html --up-title "Python Documentation Index" --global-module-index "../modindex.html" --dir html/doc doc/doc.tex +++ TEXINPUTS=/home/masa/ftp/python/Python-2.2b1/Doc/doc:/home/masa/ftp/python/Python-2.2b1/Doc/paper-letter:/home/masa/ftp/python/Python-2.2b1/Doc/texinputs: +++ latex doc +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. The relevant lines from the transcript are: ------------------------------------------------------------------------ +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex Note: Initialising with file: doc.l2h This is LaTeX2HTML Version 2K.1beta (1.47) by Nikos Drakos, Computer Based Learning Unit, University of Leeds. Revised and extended by: Marcus Hennecke, Ross Moore, Herb Swan and others ...producing markup for HTML version 4.0 Loading /usr/share/latex2html/versions/html4_0.pl *** processing declarations *** Loading /usr/share/latex2html/versions/latin1.pl *** Fatal Error --- but easy to fix *** Cannot have '.' in file-name prefix, else dvips fails on images Change the name from /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex and try again. *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:31 Message: Logged In: YES user_id=21627 Can you give more details as to how exactly compilation fails? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 From noreply@sourceforge.net Sat Oct 20 21:16:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 13:16:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-232637 ] can't compile modules on AIX 4.2.1 (for real this time) Message-ID: Bugs item #232637, was opened at 2001-02-15 15:59 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232637&group_id=5470 Category: Build Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Benjamin Collar (bcollar) Assigned to: Neil Schemenauer (nascheme) Summary: can't compile modules on AIX 4.2.1 (for real this time) Initial Comment: Hi, CC=cc_r CFLAGS="-O2 -qmaxmem=6000" ./configure --prefix=/development/utils --without-gcc make CC=cc_r OPT="-O2 -qmaxmem=6000". I'm building 2.1a2 with patch 103679 applied (necessary for makexp_aix and ld_so_aix to be found earlier in the process). Here's some output (it's the same for all modules): building '_tkinter' extension /development/utils/lib/python2.1/config/ld_so_aix cc -bI:/development/utils/lib/python2.1/config/python.exp build/temp.aix-2-000310094C00-2.1/_tkinter.o build/temp.aix-2-000310094C00-2.1/tkappinit.o -L/usr/X11/lib -L/usr/local/lib -ltk8.0 -ltcl8.0 -lld -lX11 -o build/lib.aix-2-000310094C00-2.1/_tkinter.so unable to execute /development/utils/lib/python2.1/config/ld_so_aix: No such file or directory WARNING: building of extension "_tkinter" failed: command '/development/utils/lib/python2.1/config/ld_so_aix' failed with exit status 1 There are TWO things I notice here: 1) ld_so_aix is in Modules, not in /development/utils/lib/python2.1/config. In fact, there is no directory called /development/utils/lib/python2.1. 2) (copied from above) "/development/utils/lib/python2.1/config/ld_so_aix cc" Note it says cc, not cc_r, which is how I configured and ran make. cc_r is darn important, since python will blow up if it's configured with threads but you don't run cc_r. Previously this problem was mentioned in bug #129991, which was closed when I submitted some patches I thought solved the problem...well they were incomplete. I don't know exactly what to patch for the above problems... ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2001-10-20 13:16 Message: Logged In: YES user_id=35752 Closed as per Guido's request. Resolution is "none". ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:08 Message: Logged In: YES user_id=6380 Neil, it may make sense to just close this one -- there's little chance we'll ever make progress given the last message from bcollar. ---------------------------------------------------------------------- Comment By: Benjamin Collar (bcollar) Date: 2001-04-04 12:53 Message: Logged In: YES user_id=7170 Hi, I have a little bad news...I'm at another job now, so I don't currently have access to the machines I was building on. I am trying to get permission to use the machine in order to follow up on this bug; I won't find out if I can until next week. Sorry Ben ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-21 15:58 Message: Logged In: YES user_id=35752 What are the values of CC and LINKCC in Makefile? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-17 09:17 Message: Logged In: YES user_id=11375 Reassigning to Neil; I don't know why the value of CC changes depending on the version of make. setup.py looks at the CC environment variable; perhaps AIX make doesn't pass its variables to subprocesses correctly? ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-02-28 20:52 Message: Logged In: YES user_id=3066 This relates to the move to distutils, but may have already been fixed. Andrew? ---------------------------------------------------------------------- Comment By: Benjamin Collar (bcollar) Date: 2001-02-22 11:47 Message: I got today's snapshot and ran the same configure and make I've been doing. Lo and behold everything worked GREAT. No problems building modules at all. NOTE: this is using GNU make. Good job Neil! However, when I used AIX's make, there were a few errors while building Modules: ./Modules/ld_so_aix cc -bI:Modules/python.exp build/temp.aix-2-000310094C00-2.1/dlmodule.o -L/usr/local/lib -o build/lib.aix-2-000310094C00-2.1/dl.so ld: 0711-317 ERROR: Undefined symbol: .dlopen ld: 0711-317 ERROR: Undefined symbol: .dlerror ld: 0711-317 ERROR: Undefined symbol: .dlsym ld: 0711-317 ERROR: Undefined symbol: .dlclose ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. ./Modules/ld_so_aix cc -bI:Modules/python.exp build/temp.aix-2-000310094C00-2.1/_cursesmodule.o -L/usr/local/lib -lcurses -ltermcap -o build/lib.aix-2-000310094C00-2.1/_curses.so ld: 0706-006 Cannot find or open library file: -l termcap ld:open(): A file or directory in the path name does not exist. While GNU make during the Module builds used "ld_so_aix cc_r" ... you'll note that AIX's make used cc. Shall we continue with this? Or is it sufficient to say "Please use GNU make if you're on AIX"? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-02-16 01:51 Message: Just reading through the checkins today I fonud that Neil has been fiddling with that code. Perhaps you ought to grab the latest CVS snapshot and rerun the install ?! ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-02-16 01:45 Message: Ok, here we go again :-) 1) distutils assumes Python to be already installed on the machine and thus it looks for the AIX tools in the config dir -- unfortunately, setup.py is run before these files are installed, so it cannot find them. Perhaps we ought to add a special case to distutils which allows finding them anyway ?! 2) It seems as if your make doesn't copy the command line variables into the OS environment. The setup.py file contains explicit code which uses the OS environment variables to choose a compiler and linker: # When you run "make CC=altcc" or something similar, you really want # those environment variables passed into the setup.py phase. Here's # a small set of useful ones. compiler = os.environ.get('CC') linker_so = os.environ.get('LDSHARED') Not sure what to do about this. It hints at another problem though: distutils uses the settings from the Makefile per default. It seems that in your case it is having trouble parsing that file. BTW, why does sys.platform return for you ? (There is a switch in distutils.sysconfig which switches on 'aix4' -- could be that this causes the problem) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=232637&group_id=5470 From noreply@sourceforge.net Sat Oct 20 21:45:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 13:45:06 -0700 Subject: [Python-bugs-list] [ python-Bugs-230075 ] dbmmodule build fails on Debian GNU/Linux unstable (Sid) Message-ID: Bugs item #230075, was opened at 2001-01-25 12:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: Wont Fix Priority: 5 Submitted By: Kalle Svensson (krftkndl) >Assigned to: A.M. Kuchling (akuchling) Summary: dbmmodule build fails on Debian GNU/Linux unstable (Sid) Initial Comment: When building on Debian GNU/Linux unstable, the build fails with: /home/kalle/src/python/dist/src/Modules/dbmmodule.c:24: #error "No ndbm.h available!" error: command 'gcc' failed with exit status 1 make: *** [sharedmods] Error 1 The *dbm.h files available are /usr/include/dbm.h /usr/include/gdbm.h /usr/include/gdbm-ndbm.h All are part of the package libgdbmg1-dev. ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2001-10-20 13:45 Message: Logged In: YES user_id=35752 Attached is a patch that fixes things for me. It's pretty simple so it could probably get into 2.2 yet. I don't know what happened to Thomas's patch. :-( ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-06 06:37 Message: Logged In: YES user_id=6380 Well hurry up then! ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-09-06 05:38 Message: Logged In: YES user_id=34209 No, I didn't include it in 2.1.1 because I decided I'd rather keep it broken on Debian than possibly break it on an unknown number of other systems, for a bugfix-release. I see no problem with breaking an unknown number of systems in 2.2aX though :-) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-17 08:36 Message: Logged In: YES user_id=11375 What does the patch look like? You can probably go ahead and check it in. I'd be leery of removing the 'not in "cygwin"' check; ask Jason Tishler if it's still needed, since he's the Cygwin guru. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 22:26 Message: Logged In: YES user_id=35752 db1/ndbm.h from libc6_2.2.2-1 still includes db.h instead of db1/db.h. I think I understand the reason now. If you want to use db1 or gdbm as a dbm compatible database you have to link with the approprate library. Therefore, if you should add both -ldb1 and -Idb1. I've created a patch to setup.py which does this. Andrew? BTW, I'm not sure if the "if platform not in ['cygwin']" is needed anymore. I bet it could be removed. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 21:50 Message: Logged In: YES user_id=35752 Python is looking for gdbm/ndbm.h. I could add more logic to look for gdbm-ndbm.h but I don't think its worth it. libc6 includes db1 which can be used by dbmmodule. configure is not finding db1/ndbm.h because ndbm.h includes db.h. I think it should include db1/db.h instead. So, it looks like a Debian unstable bug. I'm downloading Debian libc6_2.2.2-1 right now. Perhaps the bug has been fixed. ---------------------------------------------------------------------- Comment By: Kalle Svensson (krftkndl) Date: 2001-01-25 12:21 Message: Forgot to say that it's Python out of CVS, updated Thu Jan 25 21:23:52 CET 2001. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 From noreply@sourceforge.net Sat Oct 20 22:39:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 14:39:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-230075 ] dbmmodule build fails on Debian GNU/Linux unstable (Sid) Message-ID: Bugs item #230075, was opened at 2001-01-25 12:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: Wont Fix Priority: 5 Submitted By: Kalle Svensson (krftkndl) Assigned to: A.M. Kuchling (akuchling) Summary: dbmmodule build fails on Debian GNU/Linux unstable (Sid) Initial Comment: When building on Debian GNU/Linux unstable, the build fails with: /home/kalle/src/python/dist/src/Modules/dbmmodule.c:24: #error "No ndbm.h available!" error: command 'gcc' failed with exit status 1 make: *** [sharedmods] Error 1 The *dbm.h files available are /usr/include/dbm.h /usr/include/gdbm.h /usr/include/gdbm-ndbm.h All are part of the package libgdbmg1-dev. ---------------------------------------------------------------------- >Comment By: Thomas Wouters (twouters) Date: 2001-10-20 14:39 Message: Logged In: YES user_id=34209 Just to clarify: I _had_ no patch of my own. I tested Neil's patch, assigned the bug to Neil and was waiting for him to respond to AMK's questions. Miscommunication somewhere ? ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-10-20 13:45 Message: Logged In: YES user_id=35752 Attached is a patch that fixes things for me. It's pretty simple so it could probably get into 2.2 yet. I don't know what happened to Thomas's patch. :-( ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-06 06:37 Message: Logged In: YES user_id=6380 Well hurry up then! ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-09-06 05:38 Message: Logged In: YES user_id=34209 No, I didn't include it in 2.1.1 because I decided I'd rather keep it broken on Debian than possibly break it on an unknown number of other systems, for a bugfix-release. I see no problem with breaking an unknown number of systems in 2.2aX though :-) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-17 08:36 Message: Logged In: YES user_id=11375 What does the patch look like? You can probably go ahead and check it in. I'd be leery of removing the 'not in "cygwin"' check; ask Jason Tishler if it's still needed, since he's the Cygwin guru. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 22:26 Message: Logged In: YES user_id=35752 db1/ndbm.h from libc6_2.2.2-1 still includes db.h instead of db1/db.h. I think I understand the reason now. If you want to use db1 or gdbm as a dbm compatible database you have to link with the approprate library. Therefore, if you should add both -ldb1 and -Idb1. I've created a patch to setup.py which does this. Andrew? BTW, I'm not sure if the "if platform not in ['cygwin']" is needed anymore. I bet it could be removed. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 21:50 Message: Logged In: YES user_id=35752 Python is looking for gdbm/ndbm.h. I could add more logic to look for gdbm-ndbm.h but I don't think its worth it. libc6 includes db1 which can be used by dbmmodule. configure is not finding db1/ndbm.h because ndbm.h includes db.h. I think it should include db1/db.h instead. So, it looks like a Debian unstable bug. I'm downloading Debian libc6_2.2.2-1 right now. Perhaps the bug has been fixed. ---------------------------------------------------------------------- Comment By: Kalle Svensson (krftkndl) Date: 2001-01-25 12:21 Message: Forgot to say that it's Python out of CVS, updated Thu Jan 25 21:23:52 CET 2001. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 From noreply@sourceforge.net Sat Oct 20 23:20:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 15:20:21 -0700 Subject: [Python-bugs-list] [ python-Bugs-473274 ] Tut. 6.3: __builtins__ misspelled Message-ID: Bugs item #473274, was opened at 2001-10-20 15:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Tut. 6.3: __builtins__ misspelled Initial Comment: There's a small error in section 6.3 of the Tutorial: It says "__builtin__" in several places, which should be "__builtins__". By the way: Great tutorial, it's very helpful! Best regards Oliver Fromme ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 From noreply@sourceforge.net Sun Oct 21 01:50:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 17:50:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-230075 ] dbmmodule build fails on Debian GNU/Linux unstable (Sid) Message-ID: Bugs item #230075, was opened at 2001-01-25 12:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: Wont Fix Priority: 5 Submitted By: Kalle Svensson (krftkndl) Assigned to: A.M. Kuchling (akuchling) Summary: dbmmodule build fails on Debian GNU/Linux unstable (Sid) Initial Comment: When building on Debian GNU/Linux unstable, the build fails with: /home/kalle/src/python/dist/src/Modules/dbmmodule.c:24: #error "No ndbm.h available!" error: command 'gcc' failed with exit status 1 make: *** [sharedmods] Error 1 The *dbm.h files available are /usr/include/dbm.h /usr/include/gdbm.h /usr/include/gdbm-ndbm.h All are part of the package libgdbmg1-dev. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 17:50 Message: Logged In: YES user_id=6380 Neil, are you sure it's a good idea to assign this to Andrew? I've noticed that he's collected quite a backlog of bugs & patches. Maybe he needs some relief? ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-10-20 14:39 Message: Logged In: YES user_id=34209 Just to clarify: I _had_ no patch of my own. I tested Neil's patch, assigned the bug to Neil and was waiting for him to respond to AMK's questions. Miscommunication somewhere ? ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-10-20 13:45 Message: Logged In: YES user_id=35752 Attached is a patch that fixes things for me. It's pretty simple so it could probably get into 2.2 yet. I don't know what happened to Thomas's patch. :-( ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-06 06:37 Message: Logged In: YES user_id=6380 Well hurry up then! ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-09-06 05:38 Message: Logged In: YES user_id=34209 No, I didn't include it in 2.1.1 because I decided I'd rather keep it broken on Debian than possibly break it on an unknown number of other systems, for a bugfix-release. I see no problem with breaking an unknown number of systems in 2.2aX though :-) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-17 08:36 Message: Logged In: YES user_id=11375 What does the patch look like? You can probably go ahead and check it in. I'd be leery of removing the 'not in "cygwin"' check; ask Jason Tishler if it's still needed, since he's the Cygwin guru. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 22:26 Message: Logged In: YES user_id=35752 db1/ndbm.h from libc6_2.2.2-1 still includes db.h instead of db1/db.h. I think I understand the reason now. If you want to use db1 or gdbm as a dbm compatible database you have to link with the approprate library. Therefore, if you should add both -ldb1 and -Idb1. I've created a patch to setup.py which does this. Andrew? BTW, I'm not sure if the "if platform not in ['cygwin']" is needed anymore. I bet it could be removed. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 21:50 Message: Logged In: YES user_id=35752 Python is looking for gdbm/ndbm.h. I could add more logic to look for gdbm-ndbm.h but I don't think its worth it. libc6 includes db1 which can be used by dbmmodule. configure is not finding db1/ndbm.h because ndbm.h includes db.h. I think it should include db1/db.h instead. So, it looks like a Debian unstable bug. I'm downloading Debian libc6_2.2.2-1 right now. Perhaps the bug has been fixed. ---------------------------------------------------------------------- Comment By: Kalle Svensson (krftkndl) Date: 2001-01-25 12:21 Message: Forgot to say that it's Python out of CVS, updated Thu Jan 25 21:23:52 CET 2001. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 From noreply@sourceforge.net Sun Oct 21 01:55:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 17:55:20 -0700 Subject: [Python-bugs-list] [ python-Bugs-473274 ] Tut. 6.3: __builtins__ misspelled Message-ID: Bugs item #473274, was opened at 2001-10-20 15:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 Category: Documentation >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Tut. 6.3: __builtins__ misspelled Initial Comment: There's a small error in section 6.3 of the Tutorial: It says "__builtin__" in several places, which should be "__builtins__". By the way: Great tutorial, it's very helpful! Best regards Oliver Fromme ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 17:55 Message: Logged In: YES user_id=6380 Nonsense. __builtin__ is right. __builtins__ is a different thing altogether! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 From noreply@sourceforge.net Sun Oct 21 02:02:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 18:02:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 18:02 Message: Logged In: YES user_id=6380 Do the tests for threads pass? (test_thread.py, test_threading.py, test_threaded_import.py, test_threadedtempfile.py) That would be great news! If there are any other problems on HP-UX, it would be great if you could help out. Do you know what those problems are? ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-20 08:41 Message: Logged In: YES user_id=354556 Thist patch has no effect on threads. However, threads do work (i.e., are recognized by configure) on HP-UX 11 since 2.2b1 (I just wanted to submit a patch to the 2.2a4 configure.in when 2.2b1 was released). I am willing to support you regarding HP-UX, if necessary (there still seem to be some open issues). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Sun Oct 21 04:05:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 20:05:14 -0700 Subject: [Python-bugs-list] [ python-Bugs-473274 ] Tut. 6.3: __builtins__ misspelled Message-ID: Bugs item #473274, was opened at 2001-10-20 15:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 Category: Documentation Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Tut. 6.3: __builtins__ misspelled Initial Comment: There's a small error in section 6.3 of the Tutorial: It says "__builtin__" in several places, which should be "__builtins__". By the way: Great tutorial, it's very helpful! Best regards Oliver Fromme ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-20 20:05 Message: Logged In: NO OK, I'm not a Python expert, I'm just reading the Tutorial and trying the examples. One of the examples in section 6.3 is "dir(__builtin__)". When I try this with Python 2.1.1, this is what I get: >>> dir(__builtin__) Traceback (most recent call last): File "", line 1, in ? NameError: name '__builtin__' is not defined However, when I add an "s", it works as described: >>> dir(__builtins__) ['ArithmeticError', 'AssertionError', 'AttributeError', ......] So, it very much looks like there is a typo in the Tutorial. I'm sorry if I haven't made myself clear enough in my first message. Best regards Oliver Fromme ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 17:55 Message: Logged In: YES user_id=6380 Nonsense. __builtin__ is right. __builtins__ is a different thing altogether! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 From noreply@sourceforge.net Sun Oct 21 05:34:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 21:34:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-473009 ] binascii_b2a_base64() improper str limit Message-ID: Bugs item #473009, was opened at 2001-10-19 21:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Dave Cinege (dcinege) Assigned to: Nobody/Anonymous (nobody) Summary: binascii_b2a_base64() improper str limit Initial Comment: Modules/binascii.c binascii_b2a_base64() contains the following restrictive code: if ( bin_len > BASE64_MAXBIN ) { PyErr_SetString(Error, "Too much data for base64 line"); return NULL; } This is an error. The base64 method of encoding data has no length limitation. The MIME message RCF has such a limitation of base64 encoded data. The function should not assume it's only input must be MIME compatible. The base64 python module itself is designed for MIME I/O only, and properly limits itself. The binascii function should be left raw. binascii_a2b_base64() properly accepts input of any size. How I came across this bug: I use base64 to ascii armor binary data in log entries in a distributed network monitoring system. For the sake of ease of parsing (human and machine) all log entries are delimited by a single line. I commonly have unbroken base64 encoded fields of 64KB in size or greater. Unfortunatly I am unable to encode this data like this: result64 = binascii.b2a_base64(s) I must do this: result64 = re.sub('[ |\n]','',base64.encodestring(s)) Which is *much* slower. : < I feel this is an outright bug and should be corrected. If their is some argument for backward compatibly an optional function argument should be present to allow bypassing this limitation. ---------------------------------------------------------------------- >Comment By: Dave Cinege (dcinege) Date: 2001-10-20 21:34 Message: Logged In: YES user_id=314434 >Can you cite any relevant standard that defines base64 to >work in that way? Base64 is defined in RFC 2045 section >6.8., which clearly says >The encoded output stream must be represented in lines >of no more than 76 characters each. This is difficult to do because base64 itself has not (yet) been seperatly defined in it's own RFC. It should be and this issue has been brought up recently on the W3 lists. IE: http://lists.w3.org/Archives/Public/w3c-ietf-xmldsig/2001AprJun/0212.html http://lists.w3.org/Archives/Public/w3c-ietf-xmldsig/2001AprJun/0210.html The part of the RFC you have quoted is relevent to the use of base64 encoding in the context of MIME, the purpose clearly being to ensure compatibly with email (SMTP, POP3, MUA, etc) standards. However this 76 character line length rule is irrelevent when dealing with arbitary binary data, not meant for MIME encapulated transmission. This is clearly seen the describtion of the actual base64 algorithms itself: The encoding process represents 24-bit groups of input bits as output strings of 4 encoded characters. Proceeding from left to right, a 24-bit input group is formed by concatenating 3 8bit input groups. These 24 bits are then treated as 4 concatenated 6-bit groups, each of which is translated into a single digit in the base64 alphabet. When encoding a bit stream via the base64 encoding, the bit stream must be presumed to be ordered with the most-significant-bit first. That is, the first bit in the stream will be the high-order bit in the first 8bit byte, and the eighth bit will be the low-order bit in the first 8bit byte, and so on. ... In base64 data, characters other than those in Table 1, line breaks, and other white space probably indicate a transmission error, about which a warning message or even a message rejection might be appropriate under some circumstances. Additionally the use of 'unlimited length' base64 encoding of binary data has reached critical mass. For braod based example HTTP based authorization 'encrypts' the username:password in base64. However no length limit can be used, else it would arbiltarily limit the amount of data that could be passed without interfering with the HTTP protocol itself. IE: (Lines should not appear wrapped) 'Logging in' to a webserver with Username: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXY Z0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUV WXYZ0123456789 Password: test Will have the web broswer send the AUTH request header as follows: Authorization: Basic YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0NTY3ODlhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0RFRkdISUpLTE1OT1BRUlNUVVZXWFlaMDEyMzQ1Njc4OTp0ZXN The latter field is an 'unlimited' length base64 encoding. (Testing done with KDE Konqueror, other browsers may vary) Due to it's simple application you will find many a reference stating: ''The Base64 algorithm has become "the standard" for encoding binary data.'' Clearly line length limitation are counter productive to such use. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:30 Message: Logged In: YES user_id=21627 Can you cite any relevant standard that defines base64 to work in that way? Base64 is defined in RFC 2045 section 6.8., which clearly says The encoded output stream must be represented in lines of no more than 76 characters each. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 From noreply@sourceforge.net Sun Oct 21 05:35:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 21:35:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-473274 ] Tut. 6.3: __builtins__ misspelled Message-ID: Bugs item #473274, was opened at 2001-10-20 15:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 Category: Documentation Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Tut. 6.3: __builtins__ misspelled Initial Comment: There's a small error in section 6.3 of the Tutorial: It says "__builtin__" in several places, which should be "__builtins__". By the way: Great tutorial, it's very helpful! Best regards Oliver Fromme ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-20 21:35 Message: Logged In: YES user_id=31435 Oliver, it looks like you missed reading the line >>> import __builtin__ right before the >>> dir(__builtin__) you're talking about in section 6.3. The import is necessary! Trust me when I tell you that you really don't want to know about the differences between __builtin__ and __builtins__ yet . ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-20 20:05 Message: Logged In: NO OK, I'm not a Python expert, I'm just reading the Tutorial and trying the examples. One of the examples in section 6.3 is "dir(__builtin__)". When I try this with Python 2.1.1, this is what I get: >>> dir(__builtin__) Traceback (most recent call last): File "", line 1, in ? NameError: name '__builtin__' is not defined However, when I add an "s", it works as described: >>> dir(__builtins__) ['ArithmeticError', 'AssertionError', 'AttributeError', ......] So, it very much looks like there is a typo in the Tutorial. I'm sorry if I haven't made myself clear enough in my first message. Best regards Oliver Fromme ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 17:55 Message: Logged In: YES user_id=6380 Nonsense. __builtin__ is right. __builtins__ is a different thing altogether! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473274&group_id=5470 From noreply@sourceforge.net Sun Oct 21 07:22:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 20 Oct 2001 23:22:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-472940 ] can't getattr() attribute shown by dir() Message-ID: Bugs item #472940, was opened at 2001-10-19 15:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeremy Hylton (jhylton) >Assigned to: Guido van Rossum (gvanrossum) Summary: can't getattr() attribute shown by dir() Initial Comment: I occasionally write introspective code that does something like: for attr in dir(obj): print attr, getattr(obj, attr) I ran into a problem with it, though: >>> ArithmeticError.__init__ >>> dir(ArithmeticError.__init__) ['__call__', '__class__', '__cmp__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__name__', '__new__', '__reduce__', '__repr__', '__setattr__', '__str__', 'im_class', 'im_func', 'im_self'] >>> '__dict__' in dir(ArithmeticError.__init__) 1 >>> ArithmeticError.__init__.__dict__ Traceback (most recent call last): File "", line 1, in ? AttributeError: 'builtin_function_or_method' object has no attribute '__dict__' ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-20 23:22 Message: Logged In: YES user_id=31435 Guido, yes, the string '__dict__' is a key in ArithmeticError.__init__.__class__.__dict__ so merge_class_dict() (object.c) merges it into the result dict for ArithmeticError.__init__. Perhaps PyObject_Dir should try PyObject_HasAttr on every key it finds and weed out the ones that fail? Surely a hack, but would fix Jeremy's problem. simply by trying it before he does . ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 06:01 Message: Logged In: YES user_id=6380 For Tim. Is it possible that the __dict__ comes from looking at all the attributes of foo.__class__? ArithmeticError.__init__.__class__ has a __dict__ attribute. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 From noreply@sourceforge.net Sun Oct 21 08:07:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 00:07:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-211710 ] socket.send() can do partial writes on some systems. Message-ID: Bugs item #211710, was opened at 2000-08-11 13:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 Category: Python Library Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: scott cotton (scottc) Assigned to: Jeremy Hylton (jhylton) Summary: socket.send() can do partial writes on some systems. Initial Comment: 0811 15:57 chronis:~% uname -a FreeBSD chronis.pobox.com 4.0-STABLE FreeBSD 4.0-STABLE #0: Tue Mar 21 01:05:14 EST 2000 root@chronis.pobox.com:/usr/src/sys/compile/MYBSD i386 0811 15:57 chronis:~% 0811 15:57 chronis:~% cat scratch/sendtstsrv.py import socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind("chronis.pobox.com", 8010) while 1: s.listen(1) conn, addr = s.accept() print "connected from", addr while 1: data = conn.recv(1024) if not data: break print "done" conn.close() 0811 15:58 chronis:~% python scratch/sendtstsrv.py & [2] 76562 0811 15:58 chronis:~% cat scratch/sendtst.py #!/usr/bin/env python s = "0" * 10000000 import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect("chronis.pobox.com", 8010) sent = sock.send(s) if sent != len(s): print "sent %d/%d chars" % (sent, len(s)) else: print "sent all chars" 0811 15:58 chronis:~% python !$ python scratch/sendtst.py connected from ('208.210.124.49', 1258) sent 17520/10000000 chars done 0811 15:58 chronis:~% NOTE: there is nothing in any man page for send() under linux (RH 6.1), Solaris (SunOS 5.5.1), OSF1 V4.0, or FreeBSD{3,4} that states that send() must not perform a partial write, though of those systems, it only seems to reproducibly do partial writes under FreeBSD4.0 Stable. Additionally, W.Richard Stevens' Unix Network Programming Vol 1, second edition states that """ A read or write on a stream socket might input or output fewer bytes than requested, but this is not an error condition.""" (page 77). Later, Stevens says of send() and recv() "These two functions are similar to the standard read and write functions, but one additional argument is required". (page 354). scott ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-21 00:07 Message: Logged In: YES user_id=29957 The two approaches would seem to be: patch the socketmodule to make sure send() actually sends all the bytes. or patch the offending std library calls. The latter preserves the std unix semantics, but possibly violates the principle of Least Suprise. For 2.1.2, I'm patching the std library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:20 Message: Logged In: YES user_id=6380 Reopened. You have a point. (I think we closed it becase we couldn't reproduce it. Our fault for not having access to FreeBSD. :-) ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:43 Message: Logged In: YES user_id=6405 This problem is only seen on FreeBSD, it seems. Not even OSX, the BSD derivative, has this problem. However, my FreeBSD (4.3-RELEASE) socket(2) man page states: If no messages space is available at the socket to hold the message to be transmitted, then send() normally blocks, unless the socket has been placed in non-blocking I/O mode. The select(2) call may be used to de- termine when it is possible to send more data. Linux (2.4.7) has a similar message: When the message does not fit into the send buffer of the socket, send normally blocks, unless the socket has been placed in non-blocking I/O mode. In non-blocking mode it would return EAGAIN in this case. The select(2) call may be used to determine when it is possible to send more data. So "normally" is being interpreted loosly by FreeBSD in this case, it seems. A solution might be to have socket.send() try to send all the data. At a minimum, the standard library should be fixed so it works on FreeBSD. >From the Lib directory of 2.1.1: [richard@ike /usr/local/src/Python-2.1.1]% grep -r '\.send(' Lib Lib/ftplib.py: self.sock.send(line) Lib/ftplib.py: self.sock.send(line, MSG_OOB) Lib/ftplib.py: conn.send(buf) Lib/ftplib.py: conn.send(buf) Lib/gopherlib.py: s.send(selector + CRLF) Lib/httplib.py: self.sock.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send('\r\n') Lib/httplib.py: self.send(body) Lib/imaplib.py: self.sock.send('%s%s' % (data, CRLF)) Lib/imaplib.py: self.sock.send(literal) Lib/imaplib.py: self.sock.send(CRLF) Lib/nntplib.py: self.sock.send(line) Lib/poplib.py: self.sock.send('%s%s' % (line, CRLF)) Lib/smtplib.py: sendptr = sendptr + self.sock.send(str[sendptr:]) Lib/smtplib.py: self.send(str) Lib/smtplib.py: self.send(q) Lib/socket.py: self._sock.send(self._wbuf) Lib/telnetlib.py: self.sock.send(buffer) Lib/telnetlib.py: self.sock.send(IAC + WONT + opt) Lib/telnetlib.py: self.sock.send(IAC + DONT + opt) Lib/urllib.py: h.send(data) Lib/urllib.py: h.send(data) Lib/urllib2.py: h.send(data) Lib/webbrowser.py: s.send(action) Lib/test/test_asynchat.py: n = conn.send(buffer) Lib/test/test_socket.py: conn.send(data) Lib/test/test_socket.py: s.send(msg) Lib/test/test_socketserver.py: s.send(teststring) Note that smtplib handles send()'s return, where nothing else does. Note also that the above grep does not include our patch. ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:04 Message: Logged In: YES user_id=6405 We have patched httplib.HTTPConnection.send() so it does the following: sent = 0 while sent < len(str): sent = sent + self.sock.send(str[sent:]) Could this be the default behaviour in the socket module perhaps? ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 22:57 Message: Logged In: YES user_id=6405 Why is this bug closed? The standard library uses send() in a number of places - we've just run up against it in httplib. Again, on FreeBSD, we're seeing that send() is only sending a part of the data we want it to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-15 11:50 Message: scott-- Is there a bug here? It seems clear that the send system call is not required to send all the bytes you asked it to send. It returns the number of bytes it did send. If you want to make sure all your data is sent, you can wrap send in a while loop that calls send until all the data is consumed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:01 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 From noreply@sourceforge.net Sun Oct 21 08:24:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 00:24:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-472675 ] CVS socketmodule now doesn't compile Message-ID: Bugs item #472675, was opened at 2001-10-19 00:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark Favas (mfavas) Assigned to: Guido van Rossum (gvanrossum) Summary: CVS socketmodule now doesn't compile Initial Comment: As of 17 October, the CVS version of socketmodule.c fails to compile on Tru64 Unix, V4.0F with Compaq's C compiler. Results: building '_socket' extension cc: Error: Modules/socketmodule.c, line 2820: An unexpected newline character is present in a string literal. (nlstring) "RAND_egd(path) -> bytes ^ cc: Warning: Modules/socketmodule.c, line 2821: Invalid token discarded. (invaltoken) \n\ ^ cc: Warning: Modules/socketmodule.c, line 2822: Invalid token discarded. (invaltoken) Queries the entropy gather daemon (EGD) on socket path. Returns number\n\ -----------------------------------------------------------------------^ cc: Warning: Modules/socketmodule.c, line 2823: Invalid token discarded. (invaltoken) of bytes read. Raises socket.sslerror if connection to EGD fails or\n\ --------------------------------------------------------------------^ cc: Error: Modules/socketmodule.c, line 2824: An unexpected newline character is present in a string literal. (nlstring) if it does provide enough data to seed PRNG."; --------------------------------------------^ cc: Error: Modules/socketmodule.c, line 3037: In this statement, "PySocket_methods" is not declared. (undeclared) m = Py_InitModule3("_socket", PySocket_methods, module_doc); ------------^ /bin/cc -O -Olimit 1500 -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I./Include -I/usr/local/include -IInclude/ -c Modules/socketmodule.c -o build/temp.osf1-V4.0-alpha-2.2/socketmodule.o WARNING: building of extension "_socket" failed: command '/bin/cc' failed with exit status 1 ---------------------------------------------------------------------- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2001-10-21 00:24 Message: Logged In: YES user_id=71407 The Python-2.2b1 socketmodule also fails to compile under Tru64 5.1. My configure command: ./configure --prefix=/usr/local/Python-2.2b1 --disable-ipv6 The error messages: cc -O -Olimit 1500 -I. -I/net/ringneck/scratch1/rwgk/Python- 2.2b1/./Include -I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1-alpha- 2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:32 Message: Logged In: YES user_id=6380 Mark, I believe this can be fixed by adding \n\ at the end of the first line of the docstring for RAND_egd() -- line 2821. Can you test that? (Is this something you could have figured out yourself? We'd have appreciated a patch. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 From noreply@sourceforge.net Sun Oct 21 08:46:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 00:46:24 -0700 Subject: [Python-bugs-list] [ python-Bugs-211710 ] socket.send() can do partial writes on some systems. Message-ID: Bugs item #211710, was opened at 2000-08-11 13:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 Category: Python Library Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: scott cotton (scottc) Assigned to: Jeremy Hylton (jhylton) Summary: socket.send() can do partial writes on some systems. Initial Comment: 0811 15:57 chronis:~% uname -a FreeBSD chronis.pobox.com 4.0-STABLE FreeBSD 4.0-STABLE #0: Tue Mar 21 01:05:14 EST 2000 root@chronis.pobox.com:/usr/src/sys/compile/MYBSD i386 0811 15:57 chronis:~% 0811 15:57 chronis:~% cat scratch/sendtstsrv.py import socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind("chronis.pobox.com", 8010) while 1: s.listen(1) conn, addr = s.accept() print "connected from", addr while 1: data = conn.recv(1024) if not data: break print "done" conn.close() 0811 15:58 chronis:~% python scratch/sendtstsrv.py & [2] 76562 0811 15:58 chronis:~% cat scratch/sendtst.py #!/usr/bin/env python s = "0" * 10000000 import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect("chronis.pobox.com", 8010) sent = sock.send(s) if sent != len(s): print "sent %d/%d chars" % (sent, len(s)) else: print "sent all chars" 0811 15:58 chronis:~% python !$ python scratch/sendtst.py connected from ('208.210.124.49', 1258) sent 17520/10000000 chars done 0811 15:58 chronis:~% NOTE: there is nothing in any man page for send() under linux (RH 6.1), Solaris (SunOS 5.5.1), OSF1 V4.0, or FreeBSD{3,4} that states that send() must not perform a partial write, though of those systems, it only seems to reproducibly do partial writes under FreeBSD4.0 Stable. Additionally, W.Richard Stevens' Unix Network Programming Vol 1, second edition states that """ A read or write on a stream socket might input or output fewer bytes than requested, but this is not an error condition.""" (page 77). Later, Stevens says of send() and recv() "These two functions are similar to the standard read and write functions, but one additional argument is required". (page 354). scott ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-21 00:46 Message: Logged In: YES user_id=29957 On second (or is this third?) thoughts, patching socket.py to make it wrap the _socketmodule send code is an alternative approach that may be safer. Pity about the _fileobject, tho - it needs considerably more help. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-21 00:07 Message: Logged In: YES user_id=29957 The two approaches would seem to be: patch the socketmodule to make sure send() actually sends all the bytes. or patch the offending std library calls. The latter preserves the std unix semantics, but possibly violates the principle of Least Suprise. For 2.1.2, I'm patching the std library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:20 Message: Logged In: YES user_id=6380 Reopened. You have a point. (I think we closed it becase we couldn't reproduce it. Our fault for not having access to FreeBSD. :-) ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:43 Message: Logged In: YES user_id=6405 This problem is only seen on FreeBSD, it seems. Not even OSX, the BSD derivative, has this problem. However, my FreeBSD (4.3-RELEASE) socket(2) man page states: If no messages space is available at the socket to hold the message to be transmitted, then send() normally blocks, unless the socket has been placed in non-blocking I/O mode. The select(2) call may be used to de- termine when it is possible to send more data. Linux (2.4.7) has a similar message: When the message does not fit into the send buffer of the socket, send normally blocks, unless the socket has been placed in non-blocking I/O mode. In non-blocking mode it would return EAGAIN in this case. The select(2) call may be used to determine when it is possible to send more data. So "normally" is being interpreted loosly by FreeBSD in this case, it seems. A solution might be to have socket.send() try to send all the data. At a minimum, the standard library should be fixed so it works on FreeBSD. >From the Lib directory of 2.1.1: [richard@ike /usr/local/src/Python-2.1.1]% grep -r '\.send(' Lib Lib/ftplib.py: self.sock.send(line) Lib/ftplib.py: self.sock.send(line, MSG_OOB) Lib/ftplib.py: conn.send(buf) Lib/ftplib.py: conn.send(buf) Lib/gopherlib.py: s.send(selector + CRLF) Lib/httplib.py: self.sock.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send('\r\n') Lib/httplib.py: self.send(body) Lib/imaplib.py: self.sock.send('%s%s' % (data, CRLF)) Lib/imaplib.py: self.sock.send(literal) Lib/imaplib.py: self.sock.send(CRLF) Lib/nntplib.py: self.sock.send(line) Lib/poplib.py: self.sock.send('%s%s' % (line, CRLF)) Lib/smtplib.py: sendptr = sendptr + self.sock.send(str[sendptr:]) Lib/smtplib.py: self.send(str) Lib/smtplib.py: self.send(q) Lib/socket.py: self._sock.send(self._wbuf) Lib/telnetlib.py: self.sock.send(buffer) Lib/telnetlib.py: self.sock.send(IAC + WONT + opt) Lib/telnetlib.py: self.sock.send(IAC + DONT + opt) Lib/urllib.py: h.send(data) Lib/urllib.py: h.send(data) Lib/urllib2.py: h.send(data) Lib/webbrowser.py: s.send(action) Lib/test/test_asynchat.py: n = conn.send(buffer) Lib/test/test_socket.py: conn.send(data) Lib/test/test_socket.py: s.send(msg) Lib/test/test_socketserver.py: s.send(teststring) Note that smtplib handles send()'s return, where nothing else does. Note also that the above grep does not include our patch. ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:04 Message: Logged In: YES user_id=6405 We have patched httplib.HTTPConnection.send() so it does the following: sent = 0 while sent < len(str): sent = sent + self.sock.send(str[sent:]) Could this be the default behaviour in the socket module perhaps? ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 22:57 Message: Logged In: YES user_id=6405 Why is this bug closed? The standard library uses send() in a number of places - we've just run up against it in httplib. Again, on FreeBSD, we're seeing that send() is only sending a part of the data we want it to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-15 11:50 Message: scott-- Is there a bug here? It seems clear that the send system call is not required to send all the bytes you asked it to send. It returns the number of bytes it did send. If you want to make sure all your data is sent, you can wrap send in a while loop that calls send until all the data is consumed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:01 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 From noreply@sourceforge.net Sun Oct 21 15:48:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 07:48:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:48 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_??? and test_??? fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 18:02 Message: Logged In: YES user_id=6380 Do the tests for threads pass? (test_thread.py, test_threading.py, test_threaded_import.py, test_threadedtempfile.py) That would be great news! If there are any other problems on HP-UX, it would be great if you could help out. Do you know what those problems are? ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-20 08:41 Message: Logged In: YES user_id=354556 Thist patch has no effect on threads. However, threads do work (i.e., are recognized by configure) on HP-UX 11 since 2.2b1 (I just wanted to submit a patch to the 2.2a4 configure.in when 2.2b1 was released). I am willing to support you regarding HP-UX, if necessary (there still seem to be some open issues). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Sun Oct 21 15:50:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 07:50:27 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:50 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_socket and test_time fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:48 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_??? and test_??? fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 18:02 Message: Logged In: YES user_id=6380 Do the tests for threads pass? (test_thread.py, test_threading.py, test_threaded_import.py, test_threadedtempfile.py) That would be great news! If there are any other problems on HP-UX, it would be great if you could help out. Do you know what those problems are? ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-20 08:41 Message: Logged In: YES user_id=354556 Thist patch has no effect on threads. However, threads do work (i.e., are recognized by configure) on HP-UX 11 since 2.2b1 (I just wanted to submit a patch to the 2.2a4 configure.in when 2.2b1 was released). I am willing to support you regarding HP-UX, if necessary (there still seem to be some open issues). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Sun Oct 21 18:07:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 10:07:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-473433 ] resource.RLIMIT_VMEM Message-ID: Bugs item #473433, was opened at 2001-10-21 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473433&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: resource.RLIMIT_VMEM Initial Comment: resource.RLIMIT_VMEM doesn't seem to exist although its mentioned in the documentation. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473433&group_id=5470 From noreply@sourceforge.net Sun Oct 21 18:25:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 10:25:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-449000 ] re.sub(r'\n', ...) broke Message-ID: Bugs item #449000, was opened at 2001-08-07 21:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=449000&group_id=5470 Category: Regular Expressions Group: Python 2.2 >Status: Closed >Resolution: Out of Date Priority: 6 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Fredrik Lundh (effbot) Summary: re.sub(r'\n', ...) broke Initial Comment: Fredrik, I believe the latest changes to re.sub() broke an endcase in re.sub(). I used to be able to write r'\n' as a pattern and it would do the same thing as '\n'. Now this doesn't work any more; '\n' works but r'\n' doesn't seem to do anything. I'm guessing this is an endcase handled wrong in the most recent checkin ("map re.sub() to string.replace(), when possible" sounds suspicious to me). I haven't verified this hypothesis, but it still works in 2.1.1 and is broken in current CVS. ---------------------------------------------------------------------- >Comment By: Fredrik Lundh (effbot) Date: 2001-10-21 10:25 Message: Logged In: YES user_id=38376 the string.replace hack is no longer relevant in SRE 2.2.1 (where sub/subn is implemented in C) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 10:58 Message: Logged In: YES user_id=6380 Fredrik, when will you have time to fix this? There is currently a large amount of code checked in that is disabled because it's buggy... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-20 20:47 Message: Logged In: YES user_id=6380 Lowered priority -- this won't need to hold up the 2.2a2 release. (But /F, if you fix it before Wednesday, it may still make it into 2.2a2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=449000&group_id=5470 From noreply@sourceforge.net Sun Oct 21 18:31:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 10:31:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-441851 ] SystemError in sre scanner Message-ID: Bugs item #441851, was opened at 2001-07-16 17:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=441851&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fredrik Lundh (effbot) Summary: SystemError in sre scanner Initial Comment: File "mbr_id.py", line 91, in set_up_dict akf = non_an_ripper("", akf).upper() File "c:\python21\lib\sre.py", line 164, in _sub return _subn(pattern, template, string, count)[0] File "c:\python21\lib\sre.py", line 177, in _subn c = pattern.scanner(string) SystemError: NULL object passed to PyObject_Init where: non_an_ripper = re.compile("[^A-Za-z0-9]").sub and akf contains typically a short string like 'A123456(7)' or 'XG123456-7' This happens with Python 2.1 running under Windows 95B. The primary problem is mine (out of memory) : (a) if I wrap a try/except around my invocation of non_an_ripper (), the effect is to trigger a MemoryError exception somewhere else in my code (b) if I give Python enough memory (e.g. by shutting down Excel, IE5.5, etc), then there is no exception raised at all, and execution completes as expected. However there seems to be a secondary problem: maybe sre is not checking the return value from some .*alloc () function. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=441851&group_id=5470 From noreply@sourceforge.net Sun Oct 21 19:30:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 11:30:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-473456 ] float round up on decimal numbers Message-ID: Bugs item #473456, was opened at 2001-10-21 11:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473456&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Nobody/Anonymous (nobody) Summary: float round up on decimal numbers Initial Comment: We used to get this right on python 1.5.2; and this breaks since Python 2.0: >>> for i in range( 10): x = 0.1 * i; print x, str( x), repr( x) ... 0.0 0.0 0.0 0.1 0.1 0.10000000000000001 0.2 0.2 0.20000000000000001 0.3 0.3 0.30000000000000004 0.4 0.4 0.40000000000000002 0.5 0.5 0.5 0.6 0.6 0.60000000000000009 0.7 0.7 0.70000000000000007 0.8 0.8 0.80000000000000004 0.9 0.9 0.90000000000000002 >>> print 0.6 == 0.60000000000000009 0 >>> >>> 0.3 0.29999999999999999 >>> 0.1 0.10000000000000001 >>> 1.0 1.0 >>> 1.01 1.01 >>> 0.8 0.80000000000000004 >>> 0.5 0.5 >>> 1.3 1.3 >>> We get the same problem on Unix and Windows; and python1.5.2 gives the exact decimal values everywhere. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473456&group_id=5470 From noreply@sourceforge.net Sun Oct 21 20:50:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 12:50:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-473456 ] float round up on decimal numbers Message-ID: Bugs item #473456, was opened at 2001-10-21 11:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473456&group_id=5470 Category: Python Interpreter Core >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Frederic Giacometti (giacometti) >Assigned to: Tim Peters (tim_one) Summary: float round up on decimal numbers Initial Comment: We used to get this right on python 1.5.2; and this breaks since Python 2.0: >>> for i in range( 10): x = 0.1 * i; print x, str( x), repr( x) ... 0.0 0.0 0.0 0.1 0.1 0.10000000000000001 0.2 0.2 0.20000000000000001 0.3 0.3 0.30000000000000004 0.4 0.4 0.40000000000000002 0.5 0.5 0.5 0.6 0.6 0.60000000000000009 0.7 0.7 0.70000000000000007 0.8 0.8 0.80000000000000004 0.9 0.9 0.90000000000000002 >>> print 0.6 == 0.60000000000000009 0 >>> >>> 0.3 0.29999999999999999 >>> 0.1 0.10000000000000001 >>> 1.0 1.0 >>> 1.01 1.01 >>> 0.8 0.80000000000000004 >>> 0.5 0.5 >>> 1.3 1.3 >>> We get the same problem on Unix and Windows; and python1.5.2 gives the exact decimal values everywhere. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-21 12:50 Message: Logged In: YES user_id=31435 This is not a bug. Binary floating point cannot represent decimal fractions exactly, so some rounding always occurs (even in Python 1.5.2). What changed is that Python 2.0 shows more precision than before in certain circumstances (repr() and the interactive prompt). You can use str() or print to get the old, rounded output: >>> print 0.1+0.1 0.2 >>> Follow the link for more information: http://python.sourceforge.net/devel-docs/tut/node14.html ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473456&group_id=5470 From noreply@sourceforge.net Sun Oct 21 21:01:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 13:01:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-473456 ] float round up on decimal numbers Message-ID: Bugs item #473456, was opened at 2001-10-21 11:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473456&group_id=5470 Category: Python Interpreter Core Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Tim Peters (tim_one) Summary: float round up on decimal numbers Initial Comment: We used to get this right on python 1.5.2; and this breaks since Python 2.0: >>> for i in range( 10): x = 0.1 * i; print x, str( x), repr( x) ... 0.0 0.0 0.0 0.1 0.1 0.10000000000000001 0.2 0.2 0.20000000000000001 0.3 0.3 0.30000000000000004 0.4 0.4 0.40000000000000002 0.5 0.5 0.5 0.6 0.6 0.60000000000000009 0.7 0.7 0.70000000000000007 0.8 0.8 0.80000000000000004 0.9 0.9 0.90000000000000002 >>> print 0.6 == 0.60000000000000009 0 >>> >>> 0.3 0.29999999999999999 >>> 0.1 0.10000000000000001 >>> 1.0 1.0 >>> 1.01 1.01 >>> 0.8 0.80000000000000004 >>> 0.5 0.5 >>> 1.3 1.3 >>> We get the same problem on Unix and Windows; and python1.5.2 gives the exact decimal values everywhere. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-21 13:01 Message: Logged In: YES user_id=31435 BTW, that was the canned "float precision" response (this is a FAQ). Your last example illustrates why Python made this change: the best possible 754 double approximation to 0.6 is indeed not equal to the best possible 754 double approximation to the product of 6 with the best possible 754 double approximation to 0.1. So if you expected 0.6 to equal 0.1 * 6 exactly, you probably shouldn't be using floating-point at all, in Python or any other language . ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-21 12:50 Message: Logged In: YES user_id=31435 This is not a bug. Binary floating point cannot represent decimal fractions exactly, so some rounding always occurs (even in Python 1.5.2). What changed is that Python 2.0 shows more precision than before in certain circumstances (repr() and the interactive prompt). You can use str() or print to get the old, rounded output: >>> print 0.1+0.1 0.2 >>> Follow the link for more information: http://python.sourceforge.net/devel-docs/tut/node14.html ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473456&group_id=5470 From noreply@sourceforge.net Sun Oct 21 21:19:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 13:19:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-473491 ] "install -d" doesn't work on HP-UX Message-ID: Bugs item #473491, was opened at 2001-10-21 13:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473491&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Nobody/Anonymous (nobody) Summary: "install -d" doesn't work on HP-UX Initial Comment: "make install" in 2.2b1 uses "install -d" to create directories. However, the install command on HP-UX (/opt/imake/bin/install) doesn't have this option, resulting in an incomplete or no installation at all. Python 2.1 used "mkdir" to create directories, which was probably more portable. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473491&group_id=5470 From noreply@sourceforge.net Sun Oct 21 23:17:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 15:17:25 -0700 Subject: [Python-bugs-list] [ python-Bugs-230075 ] dbmmodule build fails on Debian GNU/Linux unstable (Sid) Message-ID: Bugs item #230075, was opened at 2001-01-25 12:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Kalle Svensson (krftkndl) >Assigned to: Nobody/Anonymous (nobody) Summary: dbmmodule build fails on Debian GNU/Linux unstable (Sid) Initial Comment: When building on Debian GNU/Linux unstable, the build fails with: /home/kalle/src/python/dist/src/Modules/dbmmodule.c:24: #error "No ndbm.h available!" error: command 'gcc' failed with exit status 1 make: *** [sharedmods] Error 1 The *dbm.h files available are /usr/include/dbm.h /usr/include/gdbm.h /usr/include/gdbm-ndbm.h All are part of the package libgdbmg1-dev. ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2001-10-21 15:17 Message: Logged In: YES user_id=35752 Yes, Andrew is doomed. :-) I've commited the patch. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 17:50 Message: Logged In: YES user_id=6380 Neil, are you sure it's a good idea to assign this to Andrew? I've noticed that he's collected quite a backlog of bugs & patches. Maybe he needs some relief? ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-10-20 14:39 Message: Logged In: YES user_id=34209 Just to clarify: I _had_ no patch of my own. I tested Neil's patch, assigned the bug to Neil and was waiting for him to respond to AMK's questions. Miscommunication somewhere ? ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-10-20 13:45 Message: Logged In: YES user_id=35752 Attached is a patch that fixes things for me. It's pretty simple so it could probably get into 2.2 yet. I don't know what happened to Thomas's patch. :-( ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-06 06:37 Message: Logged In: YES user_id=6380 Well hurry up then! ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-09-06 05:38 Message: Logged In: YES user_id=34209 No, I didn't include it in 2.1.1 because I decided I'd rather keep it broken on Debian than possibly break it on an unknown number of other systems, for a bugfix-release. I see no problem with breaking an unknown number of systems in 2.2aX though :-) ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-07-19 07:28 Message: Logged In: YES user_id=35752 Have you fixed this one in 2.1.1 Thomas? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-03-17 08:36 Message: Logged In: YES user_id=11375 What does the patch look like? You can probably go ahead and check it in. I'd be leery of removing the 'not in "cygwin"' check; ask Jason Tishler if it's still needed, since he's the Cygwin guru. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 22:26 Message: Logged In: YES user_id=35752 db1/ndbm.h from libc6_2.2.2-1 still includes db.h instead of db1/db.h. I think I understand the reason now. If you want to use db1 or gdbm as a dbm compatible database you have to link with the approprate library. Therefore, if you should add both -ldb1 and -Idb1. I've created a patch to setup.py which does this. Andrew? BTW, I'm not sure if the "if platform not in ['cygwin']" is needed anymore. I bet it could be removed. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-03-03 21:50 Message: Logged In: YES user_id=35752 Python is looking for gdbm/ndbm.h. I could add more logic to look for gdbm-ndbm.h but I don't think its worth it. libc6 includes db1 which can be used by dbmmodule. configure is not finding db1/ndbm.h because ndbm.h includes db.h. I think it should include db1/db.h instead. So, it looks like a Debian unstable bug. I'm downloading Debian libc6_2.2.2-1 right now. Perhaps the bug has been fixed. ---------------------------------------------------------------------- Comment By: Kalle Svensson (krftkndl) Date: 2001-01-25 12:21 Message: Forgot to say that it's Python out of CVS, updated Thu Jan 25 21:23:52 CET 2001. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=230075&group_id=5470 From noreply@sourceforge.net Sun Oct 21 23:33:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 15:33:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-473491 ] "install -d" doesn't work on HP-UX Message-ID: Bugs item #473491, was opened at 2001-10-21 13:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473491&group_id=5470 Category: Installation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Michael Piotrowski (mxp) >Assigned to: Neil Schemenauer (nascheme) >Summary: "install -d" doesn't work on HP-UX Initial Comment: "make install" in 2.2b1 uses "install -d" to create directories. However, the install command on HP-UX (/opt/imake/bin/install) doesn't have this option, resulting in an incomplete or no installation at all. Python 2.1 used "mkdir" to create directories, which was probably more portable. ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2001-10-21 15:33 Message: Logged In: YES user_id=35752 The install-sh script will now be used on HP-UX. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473491&group_id=5470 From noreply@sourceforge.net Mon Oct 22 00:00:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 16:00:28 -0700 Subject: [Python-bugs-list] [ python-Bugs-473525 ] ClassBrowser broken? Message-ID: Bugs item #473525, was opened at 2001-10-21 16:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: ClassBrowser broken? Initial Comment: classbrowser on pydoc.py does not show class entries reordering of classdefinitions shows class entries but listing breaks lateron idle 0.8 Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] on win32 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 From noreply@sourceforge.net Mon Oct 22 00:46:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 16:46:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-448100 ] imaplib.py test code does not work with Message-ID: Bugs item #448100, was opened at 2001-08-05 00:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448100&group_id=5470 Category: Python Library Group: Platform-specific Status: Closed Resolution: Fixed Priority: 1 Submitted By: Piers Lauder (pierslauder) Assigned to: Piers Lauder (pierslauder) Summary: imaplib.py test code does not work with Initial Comment: The test code at the end of the imaplib module contains a test message that uses NL as line terminators, instead of the IMAP4-mandated CRNL standard. As a result, the test code will not work with input sensitive IMAP4 servers, such as `NTmail'. This bug was reported by: Hung Jung Lu ---------------------------------------------------------------------- >Comment By: Piers Lauder (pierslauder) Date: 2001-10-21 16:46 Message: Logged In: YES user_id=196212 My apologies. I just wanted to ensure this change was tracked. In future I'll just check in small changes. ---------------------------------------------------------------------- Comment By: Piers Lauder (pierslauder) Date: 2001-08-05 03:53 Message: Logged In: YES user_id=196212 The test code has been modified to use CRLF. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448100&group_id=5470 From noreply@sourceforge.net Mon Oct 22 02:01:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 18:01:03 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:01 Message: Logged In: YES user_id=6380 Thanks for the list. Can you submit separate patches or bug reports for the things you know need work? That seems easier than holding this conversation in an unrelated closed bug report. :-) OPT is *not* the right place for a -I option, but there's currently no "right" place. I've seen them added to "CC", which makes more sense to me. Is HP-UX 10.20 still relevant? If so, it might be nice if the configure script defaulted to --without-threads. (Is the problem that -lcma has pthread_create but in fact doesn't support the rest of the PTHREAD API?) ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:50 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_socket and test_time fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:48 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_??? and test_??? fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 18:02 Message: Logged In: YES user_id=6380 Do the tests for threads pass? (test_thread.py, test_threading.py, test_threaded_import.py, test_threadedtempfile.py) That would be great news! If there are any other problems on HP-UX, it would be great if you could help out. Do you know what those problems are? ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-20 08:41 Message: Logged In: YES user_id=354556 Thist patch has no effect on threads. However, threads do work (i.e., are recognized by configure) on HP-UX 11 since 2.2b1 (I just wanted to submit a patch to the 2.2a4 configure.in when 2.2b1 was released). I am willing to support you regarding HP-UX, if necessary (there still seem to be some open issues). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Mon Oct 22 02:02:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 18:02:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-211710 ] socket.send() can do partial writes on some systems. Message-ID: Bugs item #211710, was opened at 2000-08-11 13:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 Category: Python Library Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: scott cotton (scottc) Assigned to: Jeremy Hylton (jhylton) Summary: socket.send() can do partial writes on some systems. Initial Comment: 0811 15:57 chronis:~% uname -a FreeBSD chronis.pobox.com 4.0-STABLE FreeBSD 4.0-STABLE #0: Tue Mar 21 01:05:14 EST 2000 root@chronis.pobox.com:/usr/src/sys/compile/MYBSD i386 0811 15:57 chronis:~% 0811 15:57 chronis:~% cat scratch/sendtstsrv.py import socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind("chronis.pobox.com", 8010) while 1: s.listen(1) conn, addr = s.accept() print "connected from", addr while 1: data = conn.recv(1024) if not data: break print "done" conn.close() 0811 15:58 chronis:~% python scratch/sendtstsrv.py & [2] 76562 0811 15:58 chronis:~% cat scratch/sendtst.py #!/usr/bin/env python s = "0" * 10000000 import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect("chronis.pobox.com", 8010) sent = sock.send(s) if sent != len(s): print "sent %d/%d chars" % (sent, len(s)) else: print "sent all chars" 0811 15:58 chronis:~% python !$ python scratch/sendtst.py connected from ('208.210.124.49', 1258) sent 17520/10000000 chars done 0811 15:58 chronis:~% NOTE: there is nothing in any man page for send() under linux (RH 6.1), Solaris (SunOS 5.5.1), OSF1 V4.0, or FreeBSD{3,4} that states that send() must not perform a partial write, though of those systems, it only seems to reproducibly do partial writes under FreeBSD4.0 Stable. Additionally, W.Richard Stevens' Unix Network Programming Vol 1, second edition states that """ A read or write on a stream socket might input or output fewer bytes than requested, but this is not an error condition.""" (page 77). Later, Stevens says of send() and recv() "These two functions are similar to the standard read and write functions, but one additional argument is required". (page 354). scott ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:02 Message: Logged In: YES user_id=6380 Please go slowly here. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-21 00:46 Message: Logged In: YES user_id=29957 On second (or is this third?) thoughts, patching socket.py to make it wrap the _socketmodule send code is an alternative approach that may be safer. Pity about the _fileobject, tho - it needs considerably more help. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-21 00:07 Message: Logged In: YES user_id=29957 The two approaches would seem to be: patch the socketmodule to make sure send() actually sends all the bytes. or patch the offending std library calls. The latter preserves the std unix semantics, but possibly violates the principle of Least Suprise. For 2.1.2, I'm patching the std library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:20 Message: Logged In: YES user_id=6380 Reopened. You have a point. (I think we closed it becase we couldn't reproduce it. Our fault for not having access to FreeBSD. :-) ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:43 Message: Logged In: YES user_id=6405 This problem is only seen on FreeBSD, it seems. Not even OSX, the BSD derivative, has this problem. However, my FreeBSD (4.3-RELEASE) socket(2) man page states: If no messages space is available at the socket to hold the message to be transmitted, then send() normally blocks, unless the socket has been placed in non-blocking I/O mode. The select(2) call may be used to de- termine when it is possible to send more data. Linux (2.4.7) has a similar message: When the message does not fit into the send buffer of the socket, send normally blocks, unless the socket has been placed in non-blocking I/O mode. In non-blocking mode it would return EAGAIN in this case. The select(2) call may be used to determine when it is possible to send more data. So "normally" is being interpreted loosly by FreeBSD in this case, it seems. A solution might be to have socket.send() try to send all the data. At a minimum, the standard library should be fixed so it works on FreeBSD. >From the Lib directory of 2.1.1: [richard@ike /usr/local/src/Python-2.1.1]% grep -r '\.send(' Lib Lib/ftplib.py: self.sock.send(line) Lib/ftplib.py: self.sock.send(line, MSG_OOB) Lib/ftplib.py: conn.send(buf) Lib/ftplib.py: conn.send(buf) Lib/gopherlib.py: s.send(selector + CRLF) Lib/httplib.py: self.sock.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send(str) Lib/httplib.py: self.send('\r\n') Lib/httplib.py: self.send(body) Lib/imaplib.py: self.sock.send('%s%s' % (data, CRLF)) Lib/imaplib.py: self.sock.send(literal) Lib/imaplib.py: self.sock.send(CRLF) Lib/nntplib.py: self.sock.send(line) Lib/poplib.py: self.sock.send('%s%s' % (line, CRLF)) Lib/smtplib.py: sendptr = sendptr + self.sock.send(str[sendptr:]) Lib/smtplib.py: self.send(str) Lib/smtplib.py: self.send(q) Lib/socket.py: self._sock.send(self._wbuf) Lib/telnetlib.py: self.sock.send(buffer) Lib/telnetlib.py: self.sock.send(IAC + WONT + opt) Lib/telnetlib.py: self.sock.send(IAC + DONT + opt) Lib/urllib.py: h.send(data) Lib/urllib.py: h.send(data) Lib/urllib2.py: h.send(data) Lib/webbrowser.py: s.send(action) Lib/test/test_asynchat.py: n = conn.send(buffer) Lib/test/test_socket.py: conn.send(data) Lib/test/test_socket.py: s.send(msg) Lib/test/test_socketserver.py: s.send(teststring) Note that smtplib handles send()'s return, where nothing else does. Note also that the above grep does not include our patch. ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 23:04 Message: Logged In: YES user_id=6405 We have patched httplib.HTTPConnection.send() so it does the following: sent = 0 while sent < len(str): sent = sent + self.sock.send(str[sent:]) Could this be the default behaviour in the socket module perhaps? ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2001-10-18 22:57 Message: Logged In: YES user_id=6405 Why is this bug closed? The standard library uses send() in a number of places - we've just run up against it in httplib. Again, on FreeBSD, we're seeing that send() is only sending a part of the data we want it to. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-15 11:50 Message: scott-- Is there a bug here? It seems clear that the send system call is not required to send all the bytes you asked it to send. It returns the number of bytes it did send. If you want to make sure all your data is sent, you can wrap send in a while loop that calls send until all the data is consumed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:01 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=211710&group_id=5470 From noreply@sourceforge.net Mon Oct 22 02:04:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 18:04:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-472675 ] CVS socketmodule now doesn't compile Message-ID: Bugs item #472675, was opened at 2001-10-19 00:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark Favas (mfavas) >Assigned to: Martin v. Löwis (loewis) Summary: CVS socketmodule now doesn't compile Initial Comment: As of 17 October, the CVS version of socketmodule.c fails to compile on Tru64 Unix, V4.0F with Compaq's C compiler. Results: building '_socket' extension cc: Error: Modules/socketmodule.c, line 2820: An unexpected newline character is present in a string literal. (nlstring) "RAND_egd(path) -> bytes ^ cc: Warning: Modules/socketmodule.c, line 2821: Invalid token discarded. (invaltoken) \n\ ^ cc: Warning: Modules/socketmodule.c, line 2822: Invalid token discarded. (invaltoken) Queries the entropy gather daemon (EGD) on socket path. Returns number\n\ -----------------------------------------------------------------------^ cc: Warning: Modules/socketmodule.c, line 2823: Invalid token discarded. (invaltoken) of bytes read. Raises socket.sslerror if connection to EGD fails or\n\ --------------------------------------------------------------------^ cc: Error: Modules/socketmodule.c, line 2824: An unexpected newline character is present in a string literal. (nlstring) if it does provide enough data to seed PRNG."; --------------------------------------------^ cc: Error: Modules/socketmodule.c, line 3037: In this statement, "PySocket_methods" is not declared. (undeclared) m = Py_InitModule3("_socket", PySocket_methods, module_doc); ------------^ /bin/cc -O -Olimit 1500 -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I./Include -I/usr/local/include -IInclude/ -c Modules/socketmodule.c -o build/temp.osf1-V4.0-alpha-2.2/socketmodule.o WARNING: building of extension "_socket" failed: command '/bin/cc' failed with exit status 1 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:04 Message: Logged In: YES user_id=6380 Mark's original problem was indeed fixed by adding the \n\. I'm keeping this open because of the new bug report. As it is IPv6-related, I've assigned this bug report to Martin. (rwgk, next time please open a new bug report rather than attaching a message to an existing report of a different bug!) ---------------------------------------------------------------------- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2001-10-21 00:24 Message: Logged In: YES user_id=71407 The Python-2.2b1 socketmodule also fails to compile under Tru64 5.1. My configure command: ./configure --prefix=/usr/local/Python-2.2b1 --disable-ipv6 The error messages: cc -O -Olimit 1500 -I. -I/net/ringneck/scratch1/rwgk/Python- 2.2b1/./Include -I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1-alpha- 2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:32 Message: Logged In: YES user_id=6380 Mark, I believe this can be fixed by adding \n\ at the end of the first line of the docstring for RAND_egd() -- line 2821. Can you test that? (Is this something you could have figured out yourself? We'd have appreciated a patch. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 From noreply@sourceforge.net Mon Oct 22 02:20:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 18:20:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-472940 ] can't getattr() attribute shown by dir() Message-ID: Bugs item #472940, was opened at 2001-10-19 15:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeremy Hylton (jhylton) Assigned to: Guido van Rossum (gvanrossum) Summary: can't getattr() attribute shown by dir() Initial Comment: I occasionally write introspective code that does something like: for attr in dir(obj): print attr, getattr(obj, attr) I ran into a problem with it, though: >>> ArithmeticError.__init__ >>> dir(ArithmeticError.__init__) ['__call__', '__class__', '__cmp__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__name__', '__new__', '__reduce__', '__repr__', '__setattr__', '__str__', 'im_class', 'im_func', 'im_self'] >>> '__dict__' in dir(ArithmeticError.__init__) 1 >>> ArithmeticError.__init__.__dict__ Traceback (most recent call last): File "", line 1, in ? AttributeError: 'builtin_function_or_method' object has no attribute '__dict__' ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:20 Message: Logged In: YES user_id=6380 Aha. No, I don't think such a hack is needed. We can simply get rid of the instancemethod_getsetlist -- it defines specific getset functions to get the __dict__, __doc__ and __name__ of the underlying function, but that's not necessary -- the normal mechanism that passes on getattr requests already takes care of this. I'll check this in shortly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-20 23:22 Message: Logged In: YES user_id=31435 Guido, yes, the string '__dict__' is a key in ArithmeticError.__init__.__class__.__dict__ so merge_class_dict() (object.c) merges it into the result dict for ArithmeticError.__init__. Perhaps PyObject_Dir should try PyObject_HasAttr on every key it finds and weed out the ones that fail? Surely a hack, but would fix Jeremy's problem. simply by trying it before he does . ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 06:01 Message: Logged In: YES user_id=6380 For Tim. Is it possible that the __dict__ comes from looking at all the attributes of foo.__class__? ArithmeticError.__init__.__class__ has a __dict__ attribute. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 From noreply@sourceforge.net Mon Oct 22 02:26:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 18:26:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-473009 ] binascii_b2a_base64() improper str limit Message-ID: Bugs item #473009, was opened at 2001-10-19 21:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Dave Cinege (dcinege) Assigned to: Nobody/Anonymous (nobody) Summary: binascii_b2a_base64() improper str limit Initial Comment: Modules/binascii.c binascii_b2a_base64() contains the following restrictive code: if ( bin_len > BASE64_MAXBIN ) { PyErr_SetString(Error, "Too much data for base64 line"); return NULL; } This is an error. The base64 method of encoding data has no length limitation. The MIME message RCF has such a limitation of base64 encoded data. The function should not assume it's only input must be MIME compatible. The base64 python module itself is designed for MIME I/O only, and properly limits itself. The binascii function should be left raw. binascii_a2b_base64() properly accepts input of any size. How I came across this bug: I use base64 to ascii armor binary data in log entries in a distributed network monitoring system. For the sake of ease of parsing (human and machine) all log entries are delimited by a single line. I commonly have unbroken base64 encoded fields of 64KB in size or greater. Unfortunatly I am unable to encode this data like this: result64 = binascii.b2a_base64(s) I must do this: result64 = re.sub('[ |\n]','',base64.encodestring(s)) Which is *much* slower. : < I feel this is an outright bug and should be corrected. If their is some argument for backward compatibly an optional function argument should be present to allow bypassing this limitation. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:26 Message: Logged In: YES user_id=6380 I'm with David. It's up to the higher level code (e.g. the base64 module) to avoid writing lines longer than 76 characters; the underlying function in binascii doesn't have to act as a policeman here. There may be other applications of the same encoding where the 76-char limit does not apply. ---------------------------------------------------------------------- Comment By: Dave Cinege (dcinege) Date: 2001-10-20 21:34 Message: Logged In: YES user_id=314434 >Can you cite any relevant standard that defines base64 to >work in that way? Base64 is defined in RFC 2045 section >6.8., which clearly says >The encoded output stream must be represented in lines >of no more than 76 characters each. This is difficult to do because base64 itself has not (yet) been seperatly defined in it's own RFC. It should be and this issue has been brought up recently on the W3 lists. IE: http://lists.w3.org/Archives/Public/w3c-ietf-xmldsig/2001AprJun/0212.html http://lists.w3.org/Archives/Public/w3c-ietf-xmldsig/2001AprJun/0210.html The part of the RFC you have quoted is relevent to the use of base64 encoding in the context of MIME, the purpose clearly being to ensure compatibly with email (SMTP, POP3, MUA, etc) standards. However this 76 character line length rule is irrelevent when dealing with arbitary binary data, not meant for MIME encapulated transmission. This is clearly seen the describtion of the actual base64 algorithms itself: The encoding process represents 24-bit groups of input bits as output strings of 4 encoded characters. Proceeding from left to right, a 24-bit input group is formed by concatenating 3 8bit input groups. These 24 bits are then treated as 4 concatenated 6-bit groups, each of which is translated into a single digit in the base64 alphabet. When encoding a bit stream via the base64 encoding, the bit stream must be presumed to be ordered with the most-significant-bit first. That is, the first bit in the stream will be the high-order bit in the first 8bit byte, and the eighth bit will be the low-order bit in the first 8bit byte, and so on. ... In base64 data, characters other than those in Table 1, line breaks, and other white space probably indicate a transmission error, about which a warning message or even a message rejection might be appropriate under some circumstances. Additionally the use of 'unlimited length' base64 encoding of binary data has reached critical mass. For braod based example HTTP based authorization 'encrypts' the username:password in base64. However no length limit can be used, else it would arbiltarily limit the amount of data that could be passed without interfering with the HTTP protocol itself. IE: (Lines should not appear wrapped) 'Logging in' to a webserver with Username: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXY Z0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUV WXYZ0123456789 Password: test Will have the web broswer send the AUTH request header as follows: Authorization: Basic YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0NTY3ODlhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0RFRkdISUpLTE1OT1BRUlNUVVZXWFlaMDEyMzQ1Njc4OTp0ZXN The latter field is an 'unlimited' length base64 encoding. (Testing done with KDE Konqueror, other browsers may vary) Due to it's simple application you will find many a reference stating: ''The Base64 algorithm has become "the standard" for encoding binary data.'' Clearly line length limitation are counter productive to such use. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:30 Message: Logged In: YES user_id=21627 Can you cite any relevant standard that defines base64 to work in that way? Base64 is defined in RFC 2045 section 6.8., which clearly says The encoded output stream must be represented in lines of no more than 76 characters each. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 From noreply@sourceforge.net Mon Oct 22 03:00:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 21 Oct 2001 19:00:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-472940 ] can't getattr() attribute shown by dir() Message-ID: Bugs item #472940, was opened at 2001-10-19 15:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Jeremy Hylton (jhylton) Assigned to: Guido van Rossum (gvanrossum) Summary: can't getattr() attribute shown by dir() Initial Comment: I occasionally write introspective code that does something like: for attr in dir(obj): print attr, getattr(obj, attr) I ran into a problem with it, though: >>> ArithmeticError.__init__ >>> dir(ArithmeticError.__init__) ['__call__', '__class__', '__cmp__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__name__', '__new__', '__reduce__', '__repr__', '__setattr__', '__str__', 'im_class', 'im_func', 'im_self'] >>> '__dict__' in dir(ArithmeticError.__init__) 1 >>> ArithmeticError.__init__.__dict__ Traceback (most recent call last): File "", line 1, in ? AttributeError: 'builtin_function_or_method' object has no attribute '__dict__' ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 19:00 Message: Logged In: YES user_id=6380 Fixed in CVS. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:20 Message: Logged In: YES user_id=6380 Aha. No, I don't think such a hack is needed. We can simply get rid of the instancemethod_getsetlist -- it defines specific getset functions to get the __dict__, __doc__ and __name__ of the underlying function, but that's not necessary -- the normal mechanism that passes on getattr requests already takes care of this. I'll check this in shortly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-20 23:22 Message: Logged In: YES user_id=31435 Guido, yes, the string '__dict__' is a key in ArithmeticError.__init__.__class__.__dict__ so merge_class_dict() (object.c) merges it into the result dict for ArithmeticError.__init__. Perhaps PyObject_Dir should try PyObject_HasAttr on every key it finds and weed out the ones that fail? Surely a hack, but would fix Jeremy's problem. simply by trying it before he does . ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 06:01 Message: Logged In: YES user_id=6380 For Tim. Is it possible that the __dict__ comes from looking at all the attributes of foo.__class__? ArithmeticError.__init__.__class__ has a __dict__ attribute. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472940&group_id=5470 From noreply@sourceforge.net Mon Oct 22 11:21:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 03:21:02 -0700 Subject: [Python-bugs-list] [ python-Bugs-473623 ] Emacs: python-mode.el Message-ID: Bugs item #473623, was opened at 2001-10-22 03:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473623&group_id=5470 Category: None Group: Not a Bug Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Emacs: python-mode.el Initial Comment: Please explain in python-mode.el how to install the mode in emacs. On Windows you need to copy python-mode.el into $EMACS_HOME/lisp and add "(require 'python-mode)" into your startupscript (E.g. $HOME/.emacs). Thank you ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473623&group_id=5470 From noreply@sourceforge.net Mon Oct 22 15:20:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 07:20:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-473433 ] resource.RLIMIT_VMEM not always defined Message-ID: Bugs item #473433, was opened at 2001-10-21 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473433&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: resource.RLIMIT_VMEM not always defined Initial Comment: resource.RLIMIT_VMEM doesn't seem to exist although its mentioned in the documentation. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 07:20 Message: Logged In: YES user_id=3066 Clarified the comments about platform-dependent symbol definition in Doc/lib/libresource.tex revision 1.16. Symbols will not be defined when the platform headers do not define them. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473433&group_id=5470 From noreply@sourceforge.net Mon Oct 22 16:08:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 08:08:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-469772 ] support additional LINK elements. Message-ID: Bugs item #469772, was opened at 2001-10-09 23:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: support additional LINK elements. Initial Comment: Now that mozilla supports the LINK element properly, it would be most excellent if the python docs were generated with more LINK elements. At the moment they're built with up, previous, and next. Some of the others are defined in http://www.w3.org/TR/REC-html40/types.html#type-links Ones that would be very nice are Top, Contents and Index. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 08:08 Message: Logged In: YES user_id=3066 Added some support for the "up" link in Doc/tools/support.py revision 1.4, but this only affects a couple of pages. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 08:46 Message: Logged In: YES user_id=29957 There's a very nice list of what link relationships do what for mozilla 0.9.5 posted at: http://lists.w3.org/Archives/Public/www-html/2001Oct/0026.html (link stolen from mozillazine) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 From noreply@sourceforge.net Mon Oct 22 18:05:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 10:05:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-469772 ] support additional LINK elements. Message-ID: Bugs item #469772, was opened at 2001-10-09 23:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: support additional LINK elements. Initial Comment: Now that mozilla supports the LINK element properly, it would be most excellent if the python docs were generated with more LINK elements. At the moment they're built with up, previous, and next. Some of the others are defined in http://www.w3.org/TR/REC-html40/types.html#type-links Ones that would be very nice are Top, Contents and Index. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 10:05 Message: Logged In: YES user_id=3066 OK, I've played with this a bit more, and I've modified the transformation to support more elements (Doc/perl/l2hinit.perl revision 1.56), but the intended semantics are still pretty fuzzy. In particular, the links described seem to work within a tree which can share a single definition of these links throughout. Given that "top", "start", and "first" are not well defined, it's not clear how to apply these. The Python documentation consists of a set of documents which each have specific identities (the Library Reference, Python/C API manual, etc.). Does "start" refer to the front of an individual manual or the collection? How about "top"? The use of "document" to mean "page" in the W3C documents doesn't help. What I've done uses the title page of each document as the "first" link, but doesn't provide the "top" link at all. Mozilla also doesn't seem to understand that multiple indexes are entirely valid; it should probably convert the "Index" entry to a menu when there are multiple indexes -- I'll file a Mozilla bug report for this one. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 08:08 Message: Logged In: YES user_id=3066 Added some support for the "up" link in Doc/tools/support.py revision 1.4, but this only affects a couple of pages. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 08:46 Message: Logged In: YES user_id=29957 There's a very nice list of what link relationships do what for mozilla 0.9.5 posted at: http://lists.w3.org/Archives/Public/www-html/2001Oct/0026.html (link stolen from mozillazine) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 From noreply@sourceforge.net Mon Oct 22 19:49:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 11:49:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-473623 ] Emacs: python-mode.el Message-ID: Bugs item #473623, was opened at 2001-10-22 03:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473623&group_id=5470 Category: None Group: Not a Bug Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Emacs: python-mode.el Initial Comment: Please explain in python-mode.el how to install the mode in emacs. On Windows you need to copy python-mode.el into $EMACS_HOME/lisp and add "(require 'python-mode)" into your startupscript (E.g. $HOME/.emacs). Thank you ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-22 11:49 Message: Logged In: YES user_id=21627 There is a link to http://www.python.org/emacs/python-mode/ in python-mode.el, which in turn offers a link "Installation Notes". Is this not enough? I think these instructions are much beter than putting a require into .emacs, since they only load python-mode when needed. OTOH, they are also somewhat more complex, since they work through auto-mode-alist and interpreter-mode-alist. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473623&group_id=5470 From noreply@sourceforge.net Mon Oct 22 20:01:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 12:01:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-472675 ] CVS socketmodule now doesn't compile Message-ID: Bugs item #472675, was opened at 2001-10-19 00:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark Favas (mfavas) Assigned to: Martin v. Löwis (loewis) Summary: CVS socketmodule now doesn't compile Initial Comment: As of 17 October, the CVS version of socketmodule.c fails to compile on Tru64 Unix, V4.0F with Compaq's C compiler. Results: building '_socket' extension cc: Error: Modules/socketmodule.c, line 2820: An unexpected newline character is present in a string literal. (nlstring) "RAND_egd(path) -> bytes ^ cc: Warning: Modules/socketmodule.c, line 2821: Invalid token discarded. (invaltoken) \n\ ^ cc: Warning: Modules/socketmodule.c, line 2822: Invalid token discarded. (invaltoken) Queries the entropy gather daemon (EGD) on socket path. Returns number\n\ -----------------------------------------------------------------------^ cc: Warning: Modules/socketmodule.c, line 2823: Invalid token discarded. (invaltoken) of bytes read. Raises socket.sslerror if connection to EGD fails or\n\ --------------------------------------------------------------------^ cc: Error: Modules/socketmodule.c, line 2824: An unexpected newline character is present in a string literal. (nlstring) if it does provide enough data to seed PRNG."; --------------------------------------------^ cc: Error: Modules/socketmodule.c, line 3037: In this statement, "PySocket_methods" is not declared. (undeclared) m = Py_InitModule3("_socket", PySocket_methods, module_doc); ------------^ /bin/cc -O -Olimit 1500 -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I./Include -I/usr/local/include -IInclude/ -c Modules/socketmodule.c -o build/temp.osf1-V4.0-alpha-2.2/socketmodule.o WARNING: building of extension "_socket" failed: command '/bin/cc' failed with exit status 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-22 12:01 Message: Logged In: YES user_id=21627 Ralf, Please do open a new bug report for this. I will need the pyconfig.h of your system, since I cannot understand why EAI_MAX isn't defined. Since you probably cannot attach files to this report, we'll need to open a new one; please assign this to me as well. EAI_MAX ought to be defined in addrinfo.h, unless HAVE_GETADDRINFO is defined, in which case getaddrinfo.c should not get included. The only option I can see is that __APPLE__ is defined, which would be very strange... Mark, can you please comment on whether socketmodule.c compiles for you now? It appears that you have a similar system. Besten Dank im voraus! ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:04 Message: Logged In: YES user_id=6380 Mark's original problem was indeed fixed by adding the \n\. I'm keeping this open because of the new bug report. As it is IPv6-related, I've assigned this bug report to Martin. (rwgk, next time please open a new bug report rather than attaching a message to an existing report of a different bug!) ---------------------------------------------------------------------- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2001-10-21 00:24 Message: Logged In: YES user_id=71407 The Python-2.2b1 socketmodule also fails to compile under Tru64 5.1. My configure command: ./configure --prefix=/usr/local/Python-2.2b1 --disable-ipv6 The error messages: cc -O -Olimit 1500 -I. -I/net/ringneck/scratch1/rwgk/Python- 2.2b1/./Include -I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1-alpha- 2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:32 Message: Logged In: YES user_id=6380 Mark, I believe this can be fixed by adding \n\ at the end of the first line of the docstring for RAND_egd() -- line 2821. Can you test that? (Is this something you could have figured out yourself? We'd have appreciated a patch. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 From noreply@sourceforge.net Mon Oct 22 20:12:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 12:12:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-473753 ] Plug sys.exc_info() leaks Message-ID: Bugs item #473753, was opened at 2001-10-22 12:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473753&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: Plug sys.exc_info() leaks Initial Comment: As the Library manual has warned for years, this kind of program leaks bigtime: import sys def f(): . try: . 1 / 0 . except ZeroDivisionError: . err = sys.exc_info() while 1: . f() because err[-1] is a traceback object, from which f's frame can be reached, from which we can get back to 'err'. 2.2 added frames to gc (thanks to NeilS), but this example still leaks. That's at best an embarrassing trap, and it just popped up again in Zope's testing framework. Best guess is that a cure amounts to adding traceback objects to gc too, although I haven't tried that so am not sure. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473753&group_id=5470 From noreply@sourceforge.net Mon Oct 22 21:19:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 13:19:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-472568 ] PyBuffer_New() memory not aligned Message-ID: Bugs item #472568, was opened at 2001-10-18 14:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: PyBuffer_New() memory not aligned Initial Comment: Memory buffer areas created by PyBuffer_New are missaligned on 32 bit machines, for doubles (64 bit). This typically generates a bus error crash on most RISC machines when a library function tries to write a double in a memory buffer allocated by PyBuffer_New(). When looking at the bufferobject.c code, this seems to come from the fact that the memory buffer points at 'malloc() + sizeof struct PyBufferObject'; [line: b->b_ptr = (void *)(b + 1); ] To the extent that struct PyBufferObject is not a multiple of sizeof (double), the largest value type; thus the misalignment, and the crashes on most RISC processors. A quick and temporary solution would consist in adding a dummy double field as the last field of the PyBufferObject structure: struct PyBufferObject { ... double dummy; } ; and setting b->b_ptr = (void*)&b->dummy; /*was (void*(b+1)*/ In doing so, b->b_ptr will always by aligned on sizeof (double), on either 32 and 64 bit architectures. Since I'm on the buffer type problem: It would be nice (and probably easy to do) to augment the buffer protocol interface with the ability to specify the basic value type stored in the buffer. A possible list of values (enum ...) would be: - undefined (backward compatibility) - char, unsigned char, short, ....int, ... long, long long, float, double, and void* (memory address). This would enable to check at runtime the type of values stored in a buffer (and prevent missalignement buserrors, as well as catching garbage situations when improper array types are passed by means of the buffer interface [e.g.: int/float/double/short...). Frederic Giacometti Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-22 13:19 Message: Logged In: YES user_id=31392 Note to whomever take this bug: PyBuffer_New() is not called anywhere in the Python source tree; nor are there any tests for buffer objects that I'm aware of. A few simple test cases would have caught this bug already. (And for the case of the builtin buffer() call, it might be good if it used PyBuffer_New().) ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-18 14:35 Message: Logged In: YES user_id=93657 I wasn't looged in when I submitted the item. Don't think I'm becoming anonymous :)) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 From noreply@sourceforge.net Mon Oct 22 21:33:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 13:33:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-473788 ] CVS socket module does not compile Message-ID: Bugs item #473788, was opened at 2001-10-22 13:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473788&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Nobody/Anonymous (nobody) Summary: CVS socket module does not compile Initial Comment: Platform: OSF1 ringneck.lbl.gov V5.1 732 alpha Compaq C V6.3-028 on Compaq Tru64 UNIX V5.1 (Rev. 732) Compiler Driver V6.3-026 (sys) cc Driver Python 2.2b1 (#1, Oct 20 2001, 23:40:31) [C] on osf1V5 configure --disable-ipv6 cc -O -Olimit 1500 -I. - I/net/ringneck/scratch1/rwgk/Python-2.2b1/./Include - I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1- alpha-2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473788&group_id=5470 From noreply@sourceforge.net Mon Oct 22 21:52:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 13:52:36 -0700 Subject: [Python-bugs-list] [ python-Bugs-473753 ] Plug sys.exc_info() leaks Message-ID: Bugs item #473753, was opened at 2001-10-22 12:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473753&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) >Assigned to: Jeremy Hylton (jhylton) Summary: Plug sys.exc_info() leaks Initial Comment: As the Library manual has warned for years, this kind of program leaks bigtime: import sys def f(): . try: . 1 / 0 . except ZeroDivisionError: . err = sys.exc_info() while 1: . f() because err[-1] is a traceback object, from which f's frame can be reached, from which we can get back to 'err'. 2.2 added frames to gc (thanks to NeilS), but this example still leaks. That's at best an embarrassing trap, and it just popped up again in Zope's testing framework. Best guess is that a cure amounts to adding traceback objects to gc too, although I haven't tried that so am not sure. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473753&group_id=5470 From noreply@sourceforge.net Mon Oct 22 22:27:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 14:27:10 -0700 Subject: [Python-bugs-list] [ python-Bugs-469772 ] support additional LINK elements. Message-ID: Bugs item #469772, was opened at 2001-10-09 23:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: support additional LINK elements. Initial Comment: Now that mozilla supports the LINK element properly, it would be most excellent if the python docs were generated with more LINK elements. At the moment they're built with up, previous, and next. Some of the others are defined in http://www.w3.org/TR/REC-html40/types.html#type-links Ones that would be very nice are Top, Contents and Index. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 14:27 Message: Logged In: YES user_id=3066 The Mozilla bug I filed (see my previous comment) was declared a duplicate of this bug: http://bugzilla.mozilla.org/show_bug.cgi?id=102915 ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 10:05 Message: Logged In: YES user_id=3066 OK, I've played with this a bit more, and I've modified the transformation to support more elements (Doc/perl/l2hinit.perl revision 1.56), but the intended semantics are still pretty fuzzy. In particular, the links described seem to work within a tree which can share a single definition of these links throughout. Given that "top", "start", and "first" are not well defined, it's not clear how to apply these. The Python documentation consists of a set of documents which each have specific identities (the Library Reference, Python/C API manual, etc.). Does "start" refer to the front of an individual manual or the collection? How about "top"? The use of "document" to mean "page" in the W3C documents doesn't help. What I've done uses the title page of each document as the "first" link, but doesn't provide the "top" link at all. Mozilla also doesn't seem to understand that multiple indexes are entirely valid; it should probably convert the "Index" entry to a menu when there are multiple indexes -- I'll file a Mozilla bug report for this one. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 08:08 Message: Logged In: YES user_id=3066 Added some support for the "up" link in Doc/tools/support.py revision 1.4, but this only affects a couple of pages. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 08:46 Message: Logged In: YES user_id=29957 There's a very nice list of what link relationships do what for mozilla 0.9.5 posted at: http://lists.w3.org/Archives/Public/www-html/2001Oct/0026.html (link stolen from mozillazine) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 From noreply@sourceforge.net Tue Oct 23 00:37:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 16:37:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-462783 ] mmap bus error on linux Message-ID: Bugs item #462783, was opened at 2001-09-19 00:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462783&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Andrew Dalke (dalke) Assigned to: Nobody/Anonymous (nobody) Summary: mmap bus error on linux Initial Comment: Not using the mmap module correctly can cause a bus error under Linux. This is a DEC, err, Compaq, err, HP Alpha. Here's a reproducible. The problem appears to be trying to mmap past the end of the file, since putting in a "seek(0)" at the indicated spot fixes things. [dalke@pw600a src]$ ./python Python 2.2a3+ (#6, Sep 17 2001, 05:05:24) [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> file = open("test.dat", "w+") >>> file.write("12345") >>> import mmap >>> # Forgot to do "file.seek(0)" here >>> s = mmap.mmap(file.fileno(), 5) >>> s[:5] Bus error [dalke@pw600a src]$ gdb ./python core GNU gdb 19991004 Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "alpha-redhat-linux"... warning: "/home/dalke/cvses/python/dist/src/core": ambiguous core format, 2 handle rs match warning: core file may not match specified executable file. Core was generated by `perl'. Program terminated with signal 8, Floating point exception. #0 0x12004d7b8 in com_assert_stmt (c=0x0, n=0x0) at Python/compile.c:717 717 PyString_AsString(c->c_code)[c- >c_nexti++] = byte; (gdb) where #0 0x12004d7b8 in com_assert_stmt (c=0x0, n=0x0) at Python/compile.c:717 (gdb) Andrew dalke@dalkescientific.com ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-22 16:37 Message: Logged In: YES user_id=31392 I can reproduce the problem just fine on Intel Linux, but I'm not really sure what we can do about it. When I run the test, I get a bus error when it tries to dereference self->data in mmap_slice(). I don't see how we can prevent the bus error short of checking the length of the file before we map it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462783&group_id=5470 From noreply@sourceforge.net Tue Oct 23 02:36:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 18:36:47 -0700 Subject: [Python-bugs-list] [ python-Bugs-473864 ] doctest expects spurios space Message-ID: Bugs item #473864, was opened at 2001-10-22 18:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473864&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: doctest expects spurios space Initial Comment: When trying something like this: >>> for k in ((1, 2), (3, 4)): ... print k, ... (1, 2) (3, 4) doctest expects an extra leading space in the output. See the attached file for some more details. Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473864&group_id=5470 From noreply@sourceforge.net Tue Oct 23 02:47:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 18:47:52 -0700 Subject: [Python-bugs-list] [ python-Bugs-473864 ] doctest expects spurios space Message-ID: Bugs item #473864, was opened at 2001-10-22 18:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473864&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: doctest expects spurios space Initial Comment: When trying something like this: >>> for k in ((1, 2), (3, 4)): ... print k, ... (1, 2) (3, 4) doctest expects an extra leading space in the output. See the attached file for some more details. Sverker ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-22 18:47 Message: Logged In: NO I forgot my email before when I sent in the bug report. Interested to know if it is a real bug or if it is just me and my system.. sverker.is@home.se ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473864&group_id=5470 From noreply@sourceforge.net Tue Oct 23 03:25:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 19:25:26 -0700 Subject: [Python-bugs-list] [ python-Bugs-473864 ] doctest expects spurios space Message-ID: Bugs item #473864, was opened at 2001-10-22 18:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473864&group_id=5470 Category: Python Library >Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: doctest expects spurios space Initial Comment: When trying something like this: >>> for k in ((1, 2), (3, 4)): ... print k, ... (1, 2) (3, 4) doctest expects an extra leading space in the output. See the attached file for some more details. Sverker ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-22 19:25 Message: Logged In: YES user_id=31435 You're screwing yourself with the trailing comma in the *preceding* example. This sets the magical "softspace" attribute on doctest's stdout proxy, which causes Python to generate a leading space on the *next* example's first line of output. This is repaired now in current CVS doctest Lib/doctest.py; new revision: 1.21 via destroying the softspace attr after every example. Before 2.2, don't write examples like this. If you *have* to use a trailing comma in a "print" example, use an empty print statement afterwards to ensure the last output line is terminated correctly. So, e.g., >>> for i in range(2): ... print i, ... print 0 1 instead of >>> for i range(2): ... print i, 0 1 Then it will work correctly under all versions of doctest. At heart, doctest can't tell whether or not example output lines actually terminated with a newline (since the examples doctest sees are taken from Python files, it *always* look like they have a terminating newline). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-22 18:47 Message: Logged In: NO I forgot my email before when I sent in the bug report. Interested to know if it is a real bug or if it is just me and my system.. sverker.is@home.se ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473864&group_id=5470 From noreply@sourceforge.net Tue Oct 23 05:48:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 22 Oct 2001 21:48:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-472675 ] CVS socketmodule now doesn't compile Message-ID: Bugs item #472675, was opened at 2001-10-19 00:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark Favas (mfavas) Assigned to: Martin v. Löwis (loewis) Summary: CVS socketmodule now doesn't compile Initial Comment: As of 17 October, the CVS version of socketmodule.c fails to compile on Tru64 Unix, V4.0F with Compaq's C compiler. Results: building '_socket' extension cc: Error: Modules/socketmodule.c, line 2820: An unexpected newline character is present in a string literal. (nlstring) "RAND_egd(path) -> bytes ^ cc: Warning: Modules/socketmodule.c, line 2821: Invalid token discarded. (invaltoken) \n\ ^ cc: Warning: Modules/socketmodule.c, line 2822: Invalid token discarded. (invaltoken) Queries the entropy gather daemon (EGD) on socket path. Returns number\n\ -----------------------------------------------------------------------^ cc: Warning: Modules/socketmodule.c, line 2823: Invalid token discarded. (invaltoken) of bytes read. Raises socket.sslerror if connection to EGD fails or\n\ --------------------------------------------------------------------^ cc: Error: Modules/socketmodule.c, line 2824: An unexpected newline character is present in a string literal. (nlstring) if it does provide enough data to seed PRNG."; --------------------------------------------^ cc: Error: Modules/socketmodule.c, line 3037: In this statement, "PySocket_methods" is not declared. (undeclared) m = Py_InitModule3("_socket", PySocket_methods, module_doc); ------------^ /bin/cc -O -Olimit 1500 -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I./Include -I/usr/local/include -IInclude/ -c Modules/socketmodule.c -o build/temp.osf1-V4.0-alpha-2.2/socketmodule.o WARNING: building of extension "_socket" failed: command '/bin/cc' failed with exit status 1 ---------------------------------------------------------------------- >Comment By: Mark Favas (mfavas) Date: 2001-10-22 21:48 Message: Logged In: YES user_id=44979 Martin, as Guido noted, socketmodule does now compile for me. There are a number of differences between Version 4 and Version 5 of Tru64 Unix, although Ralf's problems do appear strange. I don't have access to a Version 5 system to test it... ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-22 12:01 Message: Logged In: YES user_id=21627 Ralf, Please do open a new bug report for this. I will need the pyconfig.h of your system, since I cannot understand why EAI_MAX isn't defined. Since you probably cannot attach files to this report, we'll need to open a new one; please assign this to me as well. EAI_MAX ought to be defined in addrinfo.h, unless HAVE_GETADDRINFO is defined, in which case getaddrinfo.c should not get included. The only option I can see is that __APPLE__ is defined, which would be very strange... Mark, can you please comment on whether socketmodule.c compiles for you now? It appears that you have a similar system. Besten Dank im voraus! ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:04 Message: Logged In: YES user_id=6380 Mark's original problem was indeed fixed by adding the \n\. I'm keeping this open because of the new bug report. As it is IPv6-related, I've assigned this bug report to Martin. (rwgk, next time please open a new bug report rather than attaching a message to an existing report of a different bug!) ---------------------------------------------------------------------- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2001-10-21 00:24 Message: Logged In: YES user_id=71407 The Python-2.2b1 socketmodule also fails to compile under Tru64 5.1. My configure command: ./configure --prefix=/usr/local/Python-2.2b1 --disable-ipv6 The error messages: cc -O -Olimit 1500 -I. -I/net/ringneck/scratch1/rwgk/Python- 2.2b1/./Include -I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1-alpha- 2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:32 Message: Logged In: YES user_id=6380 Mark, I believe this can be fixed by adding \n\ at the end of the first line of the docstring for RAND_egd() -- line 2821. Can you test that? (Is this something you could have figured out yourself? We'd have appreciated a patch. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 From noreply@sourceforge.net Tue Oct 23 11:42:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 03:42:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-473986 ] __all__ undocumented for modules Message-ID: Bugs item #473986, was opened at 2001-10-23 03:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473986&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Richard Brodie (leogah) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: __all__ undocumented for modules Initial Comment: The effect of defining __all__ in module scope is undocumented. In 6.11 of the language reference, the statement beginning: "If the list of identifiers is replaced by a star ("*")," needs to be qualified. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473986&group_id=5470 From noreply@sourceforge.net Tue Oct 23 14:48:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 06:48:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-473985 ] str, __getitem__ and slices Message-ID: Bugs item #473985, was opened at 2001-10-23 03:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473985&group_id=5470 >Category: None Group: None Status: Open >Resolution: Invalid Priority: 5 Submitted By: Walter Dörwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: str, __getitem__ and slices Initial Comment: Using slices with __getitem__ doesn't work with classes derived from str: --- class S(str): def __getitem__(self, index): print "getitem", index return str.__getitem__(self, index) def __setitem__(self, index, value): print "setitem", index, value s = S("foo") print s[0] print s[0:2] s[0] = "b" s[0:2] = "bar" --- This prints: --- getitem 0 f fo setitem 0 b setitem slice(0, 2, None) bar --- instead of --- getitem 0 f getitem slice(0, 2, None) fo setitem 0 b setitem slice(0, 2, None) bar --- i.e. when no __getslice__ is defined s[0:2] will not be forwarded to __getitem__, but it seems to work for __setitem__. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-23 06:48 Message: Logged In: YES user_id=6380 I don't think this is a bug. S inherits __getslice__ from the str base class, but since str is immutable, it doesn't define a __setslice__ to inherit for S. BTW I'm reclassifying this as a bug, not a patch. This will unfortunately invalidate the old URL you had for this entry (SF doesn't really like reclassification that much.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473985&group_id=5470 From noreply@sourceforge.net Tue Oct 23 15:05:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 07:05:03 -0700 Subject: [Python-bugs-list] [ python-Bugs-473985 ] str, __getitem__ and slices Message-ID: Bugs item #473985, was opened at 2001-10-23 03:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473985&group_id=5470 Category: None Group: None Status: Open Resolution: Invalid Priority: 5 Submitted By: Walter Dörwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: str, __getitem__ and slices Initial Comment: Using slices with __getitem__ doesn't work with classes derived from str: --- class S(str): def __getitem__(self, index): print "getitem", index return str.__getitem__(self, index) def __setitem__(self, index, value): print "setitem", index, value s = S("foo") print s[0] print s[0:2] s[0] = "b" s[0:2] = "bar" --- This prints: --- getitem 0 f fo setitem 0 b setitem slice(0, 2, None) bar --- instead of --- getitem 0 f getitem slice(0, 2, None) fo setitem 0 b setitem slice(0, 2, None) bar --- i.e. when no __getslice__ is defined s[0:2] will not be forwarded to __getitem__, but it seems to work for __setitem__. ---------------------------------------------------------------------- >Comment By: Walter Dörwald (doerwalter) Date: 2001-10-23 07:05 Message: Logged In: YES user_id=89016 Oops, this wasn't meant to be a patch. Should have gone into the bugs category. The problems is that http://python.sourceforge.net/devel- docs/ref/sequence-methods.html says, that __getslice__ is "Deprecated since release 2.0", so the user will want to implement __getslice__ via __getitem__. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-23 06:48 Message: Logged In: YES user_id=6380 I don't think this is a bug. S inherits __getslice__ from the str base class, but since str is immutable, it doesn't define a __setslice__ to inherit for S. BTW I'm reclassifying this as a bug, not a patch. This will unfortunately invalidate the old URL you had for this entry (SF doesn't really like reclassification that much.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473985&group_id=5470 From noreply@sourceforge.net Tue Oct 23 15:31:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 07:31:32 -0700 Subject: [Python-bugs-list] [ python-Bugs-474077 ] 2.2b1: Error compiling extns with BCC Message-ID: Bugs item #474077, was opened at 2001-10-23 07:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474077&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Luigi Ballabio (lballabio) Assigned to: Nobody/Anonymous (nobody) Summary: 2.2b1: Error compiling extns with BCC Initial Comment: Borland C++ fails to compile extensions for Python 2.2b1. The error message is: D:\Python22\include\pyport.h 350: Only one of a set of overloaded functions can be "C" The solution is to remove line 183 of pyconfig.h, namely, #undef HAVE_HYPOT ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474077&group_id=5470 From noreply@sourceforge.net Tue Oct 23 15:50:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 07:50:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-473985 ] str, __getitem__ and slices Message-ID: Bugs item #473985, was opened at 2001-10-23 03:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473985&group_id=5470 >Category: Type/class unification >Group: Python 2.2 Status: Open >Resolution: Remind Priority: 5 Submitted By: Walter Dörwald (doerwalter) >Assigned to: Guido van Rossum (gvanrossum) Summary: str, __getitem__ and slices Initial Comment: Using slices with __getitem__ doesn't work with classes derived from str: --- class S(str): def __getitem__(self, index): print "getitem", index return str.__getitem__(self, index) def __setitem__(self, index, value): print "setitem", index, value s = S("foo") print s[0] print s[0:2] s[0] = "b" s[0:2] = "bar" --- This prints: --- getitem 0 f fo setitem 0 b setitem slice(0, 2, None) bar --- instead of --- getitem 0 f getitem slice(0, 2, None) fo setitem 0 b setitem slice(0, 2, None) bar --- i.e. when no __getslice__ is defined s[0:2] will not be forwarded to __getitem__, but it seems to work for __setitem__. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-23 07:50 Message: Logged In: YES user_id=6380 I've thought about this some more. It should be fixed, but the fix is complex, and think I'd rather not make this change in 2.2 -- (1) I have little time, and (2) we've already issued a beta release and a declared a feature freeze. So I'd like to put this on hold until I can start working on 2.3. This is what would have to be done: - Change all built-in sequence types to get rid of the sq_slice and sq_ass_slice dispatch functions; instead, add a mapping extension to these types and add mp_subscript and mp_ass_subscript dispatch functions that recognize slice objects. - We'd probably also have to get rid of the sq_item and sq_ass_item dispatch functions, to roll them into mp_subscript and mp_ass_subscript. - Possibly, the whole "PySequenceMethods" structure should be deprecated; all the operations there are overloading either numerical or mapping operations. Extension types that define a sequence methods structure should continue to work, but the recommendation would be to migrate these out. I wish I could do this in 2.2, but it's really too much work to start now, *and* it would be a serious incompatibility between 2.2b1 and 2.2b2. ---------------------------------------------------------------------- Comment By: Walter Dörwald (doerwalter) Date: 2001-10-23 07:05 Message: Logged In: YES user_id=89016 Oops, this wasn't meant to be a patch. Should have gone into the bugs category. The problems is that http://python.sourceforge.net/devel- docs/ref/sequence-methods.html says, that __getslice__ is "Deprecated since release 2.0", so the user will want to implement __getslice__ via __getitem__. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-23 06:48 Message: Logged In: YES user_id=6380 I don't think this is a bug. S inherits __getslice__ from the str base class, but since str is immutable, it doesn't define a __setslice__ to inherit for S. BTW I'm reclassifying this as a bug, not a patch. This will unfortunately invalidate the old URL you had for this entry (SF doesn't really like reclassification that much.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473985&group_id=5470 From noreply@sourceforge.net Tue Oct 23 21:54:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 13:54:23 -0700 Subject: [Python-bugs-list] [ python-Bugs-474238 ] 2.2b1 src rpm doesn't include Lib/email/ Message-ID: Bugs item #474238, was opened at 2001-10-23 13:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474238&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: 2.2b1 src rpm doesn't include Lib/email/ Initial Comment: Using the python2.2-2.2b1-1.src.rpm to create binary rpms, I found that the installed RPM did not contain the new (post-2.2a3, anyway) Lib/email/ module. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474238&group_id=5470 From noreply@sourceforge.net Tue Oct 23 22:48:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 14:48:33 -0700 Subject: [Python-bugs-list] [ python-Bugs-471720 ] ThreadingMixIn/TCPServer forgets close Message-ID: Bugs item #471720, was opened at 2001-10-16 07:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 Category: Python Library Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Max Neunhöffer (neunhoef) Assigned to: Guido van Rossum (gvanrossum) Summary: ThreadingMixIn/TCPServer forgets close Initial Comment: When using the SocketServer.TCPServer class in connection with the SocketServer.ThreadingMixIn class every request produces an open socket, because the overloaded process_request method only calls the finish_request method without calling close_request afterwards. The test in test_socketserver.py does not notice this. This is true for Python 2.1.1 and Python 2.2a4. Documentation for ThreadingMixIn (and ForkingMixIn) is a little meager. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-23 14:48 Message: Logged In: YES user_id=6380 Thanks. I've applied the first chunk of your second patch now. The second chunk I think is a semantic change that would be wrong -- the overloaded finish() in DataagramRequestHandler would write partial data when handle() raises an exception, and that's not right. If you want rfile and wfile to be closed, you'll have to catch the exception in handle() yourself. Maybe the StreamRequestHandler class should override __init__ to change the try/finally statement? Sigh... ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 02:02 Message: Logged In: YES user_id=350896 Sorry for accidentally submitting my last message twice. In the first there was an error with the patch. The comments are identical. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 01:57 Message: Logged In: YES user_id=350896 You are right. In the current version exceptions are not properly handled: The new thread for a request does not catch exceptions. However, the new thread should handle the exception. Therefore: I suggest to exactly copy the behaviour in "handle_request" in "process_request_thread" (see patch): All exceptions are caught, in case of an exception in either finish_request or close_request the "handle_error" method is called and then "close_request" is done. This means, that when "close_request" causes the exception, it is tried again after "handle_error". Another possibility would be to move "close_request" out of the "try"-clause... During testing this I found another problem: In case of an exception the files "rfile" and "wfile" in the BaseRequestHandler instance are not closed. Because they contain dup'ed file descriptors of the socket, the connection remains open, probably until this instance is garbage collected. Therefore I suggest the following to resolve this problem: Move the call to the "finish" method in "__init__" of BaseRequestHandler into the "finally" clause. Then the files are properly closed and the exception is propagated further up. This is also in the patch. This however changes the behaviour of this class also in the non-Threading case from the user's point of view (if the finish method is overloaded)! I do not know whether you want to introduce such a change into the code between alpha and beta releases... ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-20 01:56 Message: Logged In: YES user_id=350896 You are right. In the current version exceptions are not properly handled: The new thread for a request does not catch exceptions. However, the new thread should handle the exception. Therefore: I suggest to exactly copy the behaviour in "handle_request" in "process_request_thread" (see patch): All exceptions are caught, in case of an exception in either finish_request or close_request the "handle_error" method is called and then "close_request" is done. This means, that when "close_request" causes the exception, it is tried again after "handle_error". Another possibility would be to move "close_request" out of the "try"-clause... During testing this I found another problem: In case of an exception the files "rfile" and "wfile" in the BaseRequestHandler instance are not closed. Because they contain dup'ed file descriptors of the socket, the connection remains open, probably until this instance is garbage collected. Therefore I suggest the following to resolve this problem: Move the call to the "finish" method in "__init__" of BaseRequestHandler into the "finally" clause. Then the files are properly closed and the exception is propagated further up. This is also in the patch. This however changes the behaviour of this class also in the non-Threading case from the user's point of view (if the finish method is overloaded)! I do not know whether you want to introduce such a change into the code between alpha and beta releases... ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-18 11:01 Message: Logged In: YES user_id=6380 OK. That looks good. I've checked this into CVS, in time for 2.2b1. Question: what should be done if an exception occurs in finish_request()? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-17 02:10 Message: Logged In: YES user_id=350896 This is a context diff of a proposal for a patch relative to the current state of CVS. I have to admit that I tested this only with 2.2a4 and not with the current CVS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 17:26 Message: Logged In: YES user_id=6380 OK. It would be a great help if you could upload a patch (a context diff) relative to the current state of CVS, or the 2.2a4 release if you must. When uploading, don't forget to check the file upload checkbox. ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 15:01 Message: Logged In: YES user_id=350896 I suggest changing only the ThreadingMixIn class in the following way: (1) Add a method process_request_thread which does exactly the same as the process_request method of the BaseServer class: call finish_request call close_request (2) The overloaded process_request method does not launch the finish_request method as a new thread but instead the process_request_thread method. Another possibility would be to somehow access the process_request method of the corresponding base class generically. However I do not know a way to access this easily and cleanly. Still the documentation to ThreadingMixIn and ForkingMixIn could be improved. I do not know the sourceforge bug tracker well enough to merge these "threads" as montenaro suggests. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-16 09:12 Message: Logged In: YES user_id=44345 I am getting a sense of deja vu. Before we march down this path, can we revisit (and perhaps collapse this into) the previous "thread" on this subject: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=417845 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 08:35 Message: Logged In: YES user_id=6380 Can you suggest a fix then? ---------------------------------------------------------------------- Comment By: Max Neunhöffer (neunhoef) Date: 2001-10-16 07:49 Message: Logged In: YES user_id=350896 I would not like this, because then the correct code for a handler of a ThreadingMixIn/TCPServer would be different from a standard TCPServer. It is also not very intuitive from a programmers point of view, who only wants to use the server library. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-16 07:31 Message: Logged In: YES user_id=6380 Let's make this a documentation issue. When using threading, the thread is responsible for closing the socket. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471720&group_id=5470 From noreply@sourceforge.net Wed Oct 24 07:30:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 23 Oct 2001 23:30:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-464423 ] _socket problem on solaris Message-ID: Bugs item #464423, was opened at 2001-09-24 07:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=464423&group_id=5470 Category: Build Group: Python 2.1.1 >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: Bent Nagstrup Terp (bentterp) Assigned to: Martin v. Löwis (loewis) Summary: _socket problem on solaris Initial Comment: I can't get the _socket module to build on solaris 8 with gcc 2.95.3 The compilation doesn't produce any errors, only lots of "warning: function declaration isn't a prototype", mainly from openssl header files, but also some from socketmodule.c itself: /root/Python-2.1.1/Modules/socketmodule.c: In function `PySocket_socket': /root/Python-2.1.1/Modules/socketmodule.c:1921: warning: function declaration isn't a prototype /root/Python-2.1.1/Modules/socketmodule.c: In function `PySocket_fromfd': /root/Python-2.1.1/Modules/socketmodule.c:1959: warning: function declaration isn't a prototype But as they are only warnings and not errors, compilation succeeds: building '_socket' extension gcc -g -O2 -Wall -Wstrict-prototypes -fPIC -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I/root/Python-2.1.1/./Include -I/usr/local/include -IInclude/ -c /root/Python-2.1.1/Modules/socketmodule.c -o build/temp.solaris-2.8-sun4u-2.1/socketmodule.o gcc -shared build/temp.solaris-2.8-sun4u-2.1/socketmodule.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.solaris-2.8-sun4u-2.1/_socket.so The socket library is being referenced in the toplevel Makefile: LIBS= -lpthread -lsocket -lnsl -ldl -lthread The errors only start to show up during 'make test': test_asynchat Exception in thread Thread-1: Traceback (most recent call last): File "/root/Python-2.1.1/Lib/threading.py", line 378, in __bootstrap self.run() File "./Lib/test/test_asynchat.py", line 12, in run sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) AttributeError: 'socket' module has no attribute 'socket' test test_asynchat crashed -- exceptions.AttributeError: 'socket' module has no attribute 'AF_INET' test test_socket crashed -- exceptions.AttributeError: 'socket' module has no attribute 'error' Is there any other information that I could provide to help? with kind regards, Bent Nagstrup Terp Systemadministrator Center for Genomics and Bioinformatics Karolinska Instituttet von Eulers Väg 8 171 77 Stockholm, Sweden ---------------------------------------------------------------------- >Comment By: Bent Nagstrup Terp (bentterp) Date: 2001-10-23 23:30 Message: Logged In: YES user_id=329353 Unable to reproduce error after installing libs for gcc3.0.1 I know, that has nothing to do with it, but that's just Solaris for you ;-) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-09-26 10:44 Message: Logged In: YES user_id=21627 Try 'import _socket' and report what that does. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=464423&group_id=5470 From noreply@sourceforge.net Wed Oct 24 09:52:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 01:52:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows) Message-ID: Bugs item #216289, was opened at 2000-10-06 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470 Category: Windows Group: 3rd Party Status: Closed Resolution: Wont Fix Priority: 3 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: Programs using Tkinter sometimes can't shut down (Windows) Initial Comment: The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS. In my case, I was running IDLE via python ../tool/idle/idle.pyw from a DOS box in my PCbuild directory. Win98SE. *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging. As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap. -----Original Message----- From: Joseph Stubenrauch Sent: Friday, October 06, 2000 9:23 PM To: tutor@python.org Subject: Re: [Tutor] Python 1.6 BUG Strange, I have been experiencing the same bug myself. Here's the low down for me: Python 1.6 with win95 I am running a little Tkinter program The command line I use is simply: "python foo.py" About 25-35% of the time, when I close the Tkinter window, DOS seems to "freeze" and never returns to the c:\ command prompt. I have to ctrl-alt-delete repeatedly and shut down "winoldapp" to get rid of the window and then shell back into DOS and keep working. It's a bit of a pain, since I have the habit of testing EVERYTHING in tiny little stages, so I change one little thing, test it ... freeze ... ARGH! Change one more tiny thing, test it ... freeze ... ARGH! However, sometimes it seems to behave and doesn't bother me for an entire several hour session of python work. That's my report on the problem. Cheers, Joe ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-24 01:52 Message: Logged In: NO For those who are still trapped in this bug's hell, I will gladly share the one thing that saved my sanity: WingIDE's 'Kill' command will shut down the program with apparent 100% certainty and no fear of lockups. WingIDE has its own issues, so its not a perfect solution, but if you are like me and Joe (above) who test in small iterations, then using 'Kill' to quit out of your app while testing is a workaround that may preserve your sanity. Perhaps the python gods and the Wing guys can get together and tell us how to replicate 'kill' into our code. For now, I'll use WingIDE to edit, and pythonw.exe for my final client's delivery. ---------------------------------------------------------------------- Comment By: Howard Lightstone (hlightstone) Date: 2001-09-05 10:43 Message: Logged In: YES user_id=66570 I sometimes get bunches of these.... A tool I use (Taskinfo2000) reports that (after killing winoldap): python.exe is blocked on a mutex named OLESCELOCKMUTEX. The reported state is "Console Terminating". There appears to be only one (os) thread running. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-04-02 13:06 Message: Logged In: YES user_id=31435 No sign of progress on this presumed Tk/Tcl Windows bug in over 3 months, so closing it for lack of hope. ---------------------------------------------------------------------- Comment By: Doug Henderson (djhender) Date: 2001-02-05 21:13 Message: This was a symptom I saw while tracking down the essence of the problem reported in #131207. Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2000-12-12 18:00 Message: Just reproduced w/ current CVS, but didn't hang until the 8th try. http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-12-12 12:58 Message: Tim, can you still reproduce this with the current CVS version? There's been one critical patch to _tkinter since the 2.0 release. An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-10-15 09:47 Message: Same as I've reported earlier; it hangs in the call to Tcl_Finalize (which is called by the DLL finalization code). It's less likely to hang if I call Tcl_Finalize from the _tkinter DLL (from user code). Note that the problem isn't really Python-related -- I have stand-alone samples (based on wish) that hangs in the same way. More later. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:40 Message: Back to Tim since I have no clue what to do here. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-12 10:25 Message: The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either. As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again. Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2000-10-07 00:37 Message: More info (none good, but some worse so boosted priority): + Happens under release and debug builds. + Have not been able to provoke when starting in the debugger. + Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up. There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show. + This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot. + These processes cannot be killed! Wintop and Process Viewer both fail to get the job done. PrcView (a freeware process viewer) itself locks up if I try to kill them using it. Process Viewer freezes for several seconds before giving up. + Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program". + The processes are not accumulating cycles. + Smells like deadlock. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470 From noreply@sourceforge.net Wed Oct 24 15:41:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 07:41:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-24 07:41 Message: Logged In: YES user_id=56897 I've verified Python 2.2b1 fixes the thread-signal interaction on Solaris 6, Solaris 8, and RedHat Linux 7.1. Thanks for the quick patch application! ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-16 09:59 Message: Logged In: YES user_id=56897 I'll grab the 2.2b1 release when it is available and test it on the Solaris and Linux configurations we have. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 14:52 Message: Logged In: YES user_id=6380 Since I've now applied your patch, I presume this is fixed, and I'm closing the bug report. Let me know if there are still problems. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-09 11:40 Message: Logged In: YES user_id=56897 Patch is #468347 [mask signals for non-main pthreads] ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 10:09 Message: Logged In: YES user_id=56897 I've submitted the patch for pthread signal masking. My biggest concerns are the guesses I made for DCE threads and whether they will work for AIX which might need to use sigthreadmask(). Regarding reproducing this on Linux, I was able to get Linux to crash if I held down Ctrl-C (with fairly fast key repeat). After starting the spinning thread, Python would crash on Linux under a storm of SIGINTs within 30 seconds or so. Without the spinning thread, I couldn't get it to crash. With the patch applied, the spinning thread running during the storm of SIGINTs wouldn't crash it. So that implies the signal masking is doing something good even in the Linux case. Re: my SF problems, I submitted a few support requests. Hopefully something gets fixed. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 08:38 Message: Logged In: YES user_id=56897 I've checked the Monitor button while logged in, but I still do not receive email notification of updates. When I clicked it again, it said I was no longer monitoring, so I clicked it yet again back into monitoring mode. Apparently SF knows I'm monitoring it, but it still doesn't send me email. Mail to my @users.sf.net account does work, so I'm at a loss to explain why a) My login cookie doesn't stick around very long at all and b) Why I never get monitor email from SF Re: the patch, I have something that works well on Solaris. I'll try it on Linux today, but I don't have access to an HP-UX system. I'm a little concerned about the impact to HP-UX (pre 11.0 and post 11.0) and AIX, and I don't have access to those machines to check out those concerns. Hopefully I'll have the patch posted by today. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Wed Oct 24 15:45:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 07:45:07 -0700 Subject: [Python-bugs-list] [ python-Bugs-472675 ] CVS socketmodule now doesn't compile Message-ID: Bugs item #472675, was opened at 2001-10-19 00:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Mark Favas (mfavas) Assigned to: Martin v. Löwis (loewis) Summary: CVS socketmodule now doesn't compile Initial Comment: As of 17 October, the CVS version of socketmodule.c fails to compile on Tru64 Unix, V4.0F with Compaq's C compiler. Results: building '_socket' extension cc: Error: Modules/socketmodule.c, line 2820: An unexpected newline character is present in a string literal. (nlstring) "RAND_egd(path) -> bytes ^ cc: Warning: Modules/socketmodule.c, line 2821: Invalid token discarded. (invaltoken) \n\ ^ cc: Warning: Modules/socketmodule.c, line 2822: Invalid token discarded. (invaltoken) Queries the entropy gather daemon (EGD) on socket path. Returns number\n\ -----------------------------------------------------------------------^ cc: Warning: Modules/socketmodule.c, line 2823: Invalid token discarded. (invaltoken) of bytes read. Raises socket.sslerror if connection to EGD fails or\n\ --------------------------------------------------------------------^ cc: Error: Modules/socketmodule.c, line 2824: An unexpected newline character is present in a string literal. (nlstring) if it does provide enough data to seed PRNG."; --------------------------------------------^ cc: Error: Modules/socketmodule.c, line 3037: In this statement, "PySocket_methods" is not declared. (undeclared) m = Py_InitModule3("_socket", PySocket_methods, module_doc); ------------^ /bin/cc -O -Olimit 1500 -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I./Include -I/usr/local/include -IInclude/ -c Modules/socketmodule.c -o build/temp.osf1-V4.0-alpha-2.2/socketmodule.o WARNING: building of extension "_socket" failed: command '/bin/cc' failed with exit status 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-24 07:45 Message: Logged In: YES user_id=21627 On OSF/1 V5.1, getaddrinfo does exist (although it appears to be slightly broken). The socket problems on this system are fixed with configure.in 1.277, configure 1.269, socketmodule.c 1.184. Ralf, thanks for the account. ---------------------------------------------------------------------- Comment By: Mark Favas (mfavas) Date: 2001-10-22 21:48 Message: Logged In: YES user_id=44979 Martin, as Guido noted, socketmodule does now compile for me. There are a number of differences between Version 4 and Version 5 of Tru64 Unix, although Ralf's problems do appear strange. I don't have access to a Version 5 system to test it... ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-22 12:01 Message: Logged In: YES user_id=21627 Ralf, Please do open a new bug report for this. I will need the pyconfig.h of your system, since I cannot understand why EAI_MAX isn't defined. Since you probably cannot attach files to this report, we'll need to open a new one; please assign this to me as well. EAI_MAX ought to be defined in addrinfo.h, unless HAVE_GETADDRINFO is defined, in which case getaddrinfo.c should not get included. The only option I can see is that __APPLE__ is defined, which would be very strange... Mark, can you please comment on whether socketmodule.c compiles for you now? It appears that you have a similar system. Besten Dank im voraus! ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:04 Message: Logged In: YES user_id=6380 Mark's original problem was indeed fixed by adding the \n\. I'm keeping this open because of the new bug report. As it is IPv6-related, I've assigned this bug report to Martin. (rwgk, next time please open a new bug report rather than attaching a message to an existing report of a different bug!) ---------------------------------------------------------------------- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2001-10-21 00:24 Message: Logged In: YES user_id=71407 The Python-2.2b1 socketmodule also fails to compile under Tru64 5.1. My configure command: ./configure --prefix=/usr/local/Python-2.2b1 --disable-ipv6 The error messages: cc -O -Olimit 1500 -I. -I/net/ringneck/scratch1/rwgk/Python- 2.2b1/./Include -I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1-alpha- 2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-19 05:32 Message: Logged In: YES user_id=6380 Mark, I believe this can be fixed by adding \n\ at the end of the first line of the docstring for RAND_egd() -- line 2821. Can you test that? (Is this something you could have figured out yourself? We'd have appreciated a patch. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472675&group_id=5470 From noreply@sourceforge.net Wed Oct 24 15:46:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 07:46:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-465673 ] pthreads need signal protection Message-ID: Bugs item #465673, was opened at 2001-09-27 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 Category: Threads Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: pthreads need signal protection Initial Comment: I've been playing around with Python and threads, and I've noticed some odd and often unstable behavior. In particular, on my Solaris 8 box I can get Python 1.5.2, 1.6, 2.0, or 2.1 to core dump every time with the following sequence. I've also seen this happen on Solaris 6 (all UltraSPARC based): 1. Enter the following code into the interactive interpreter: -- import threading def loopingfunc(): while 1: pass threading.Thread(target=loopingfunc).start() -- 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may vary). "Keyboard Interrupt" is displayed and so far everything looks fine. 3. Now simply press the key to enter a blank line in the interpreter. For my Solaris 8 box with the GNU readline 2.2 module present, this always ends up in a core dump. It may take a while, since at this point the readline signal handler is being re-entered recursively until the stack overflows. I've described this problem in the past on Usenet, but didn't get much response. For a more complete discussion of the problem and a possible solution, see http://groups.google.com/groups?hl=en&threadm=98osml%24sul%241%40newshost.mot.com&rnum=1&prev=/groups%3Fas_ugroup%3Dcomp.lang.python%26as_uauthors%3DJason%2520Lowe (If the URL doesn't work, search groups.google.com for posts by "Jason Lowe" in comp.lang.python and view the entire thread of the result.) Upon investigation of the problem, it looks like the problem is caused by an interaction with pthreads and signals. The SIGINT signal is delivered to the thread that is performing the spin loop, NOT the thread that is in the readline() module. Because the readline module uses setjmp()/longjmp() for its signal handling, the longjmp() ends up being executed by the wrong thread with dire results. Pthreads and signals don't mix very well, so one has to be very careful to make sure everything works properly. A typical solution is to ensure signals are only delivered to one thread by masking all signals in all other threads. I believe this will be the same root cause of bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I was able to solve the problem by modifying Python/thread_pthread.h's PyThread_start_new_thread() to block all signals with pthread_sigmask() after the new thread was started. This causes all threads created by Python except the initial thread to have all signals masked. This forces signals to be delivered to the main thread. I don't believe anyone is depending on the current behavior that signals will be delivered to an indeterminate thread, so this change seems safe. However I haven't run many other Python applications that deal with threads and signals. I propose that on platforms that implement Python threads with pthreads, the code masks all signals in all threads except the initial, main thread. This will resolve the problem of signals being delivered to threads indeterminately. I think I can dig up my initial code deltas if desired, or I can always recreate them. It's just a few lines to mask signals in the thread before thread creation, then restore them afterwards. (This causes only the main thread to have signals preserved.) A side question from this is whether the thread module (or posix module?) should expose the pthread_sigmask() functionality to Python threads on a platform that uses pthreads. This would allow developers to manipulate the signal masks of the Python threads so that a particular signal can be routed to a particular thread. (They would mask this signal in all other threads except the desired thread.) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-24 07:46 Message: Logged In: YES user_id=6380 Thanks for the followup! ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-24 07:41 Message: Logged In: YES user_id=56897 I've verified Python 2.2b1 fixes the thread-signal interaction on Solaris 6, Solaris 8, and RedHat Linux 7.1. Thanks for the quick patch application! ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-16 09:59 Message: Logged In: YES user_id=56897 I'll grab the 2.2b1 release when it is available and test it on the Solaris and Linux configurations we have. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-12 14:52 Message: Logged In: YES user_id=6380 Since I've now applied your patch, I presume this is fixed, and I'm closing the bug report. Let me know if there are still problems. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-09 11:40 Message: Logged In: YES user_id=56897 Patch is #468347 [mask signals for non-main pthreads] ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 10:09 Message: Logged In: YES user_id=56897 I've submitted the patch for pthread signal masking. My biggest concerns are the guesses I made for DCE threads and whether they will work for AIX which might need to use sigthreadmask(). Regarding reproducing this on Linux, I was able to get Linux to crash if I held down Ctrl-C (with fairly fast key repeat). After starting the spinning thread, Python would crash on Linux under a storm of SIGINTs within 30 seconds or so. Without the spinning thread, I couldn't get it to crash. With the patch applied, the spinning thread running during the storm of SIGINTs wouldn't crash it. So that implies the signal masking is doing something good even in the Linux case. Re: my SF problems, I submitted a few support requests. Hopefully something gets fixed. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-05 08:38 Message: Logged In: YES user_id=56897 I've checked the Monitor button while logged in, but I still do not receive email notification of updates. When I clicked it again, it said I was no longer monitoring, so I clicked it yet again back into monitoring mode. Apparently SF knows I'm monitoring it, but it still doesn't send me email. Mail to my @users.sf.net account does work, so I'm at a loss to explain why a) My login cookie doesn't stick around very long at all and b) Why I never get monitor email from SF Re: the patch, I have something that works well on Solaris. I'll try it on Linux today, but I don't have access to an HP-UX system. I'm a little concerned about the impact to HP-UX (pre 11.0 and post 11.0) and AIX, and I don't have access to those machines to check out those concerns. Hopefully I'll have the patch posted by today. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-03 08:58 Message: Logged In: YES user_id=6380 To get email (to your @users.sf.net account), click on the Monitor button that appears at the top of the bug entry when you're logged in to SF. ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-10-03 07:32 Message: Logged In: YES user_id=56897 I'm working on a patch now. Unfortunately, I only have access to Solaris and Linux right now, but I'll test the patch on those. I might be able to scrounge up an HPUX machine as well. I'll post more info as I get it. Unfortunately, it appears I have to poll this issue for updates, so I might not respond right away to comments. The 'monitor' feature doesn't seem to work for me, among many other SourceForge things. If I wait about 3 minutes, SourceForge wants me to log back in if I click anything and I never seem to get any email notifications (but my email address listed for my account is correct). Weird. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 09:08 Message: Logged In: YES user_id=6380 I don't have Solaris access, and I can't get this to break on Linux. But I agree with your suggestion that posix threads should block signals. Are you capable of coming up with a patch that does that, in a way that is independent of the specific platform (as long as it has PTHREADS)? You may have to open a new issue in the patch manager, since SF doesn't allow after-the-fact attachments to anonymous entries. (Maybe SF logs you out whenever you quit your browser? That's what it does for me. :-) ---------------------------------------------------------------------- Comment By: Jason Lowe (jasonlowe) Date: 2001-09-27 07:40 Message: Logged In: YES user_id=56897 Ack. SourceForge wants to log me out every few minutes, so I wasn't logged in when I submitted this. Sorry 'bout that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=465673&group_id=5470 From noreply@sourceforge.net Wed Oct 24 15:47:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 07:47:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-474485 ] pydoc generates some bad html Message-ID: Bugs item #474485, was opened at 2001-10-24 07:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474485&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Rich Salz (rsalz) Assigned to: Nobody/Anonymous (nobody) Summary: pydoc generates some bad html Initial Comment: One bug, two nits. When documenting lambda's, pydoc outputs which is interepreted as HTML markup. I partially fixed that; I didn't add support for A/HREF, which I think is what's really intended. The nits: HTMLtidy says tables should have a summary attribute, and

directly before a is an empty element which should be removed. Attached patch addresses these. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474485&group_id=5470 From noreply@sourceforge.net Wed Oct 24 18:00:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 10:00:35 -0700 Subject: [Python-bugs-list] [ python-Bugs-474538 ] Memory (reference) leak in poller.regis Message-ID: Bugs item #474538, was opened at 2001-10-24 10:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474538&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Dave Brueck (dbrueck) Assigned to: Nobody/Anonymous (nobody) Summary: Memory (reference) leak in poller.regis Initial Comment: A reference leak occurs if you call the register (and possibly unregister) functions of poller objects (obtained by calling select.poll). The problem does not manifest itself on file descriptors with a value <= 100, so it may be that register() is leaking references to integer objects and those less than 101 are simply leaked references to cached integer objects. I'm running Python 2.1.1 on Linux Mandrake 8.1 and Linux 2.1b1 on RedHat 7.0. This code reproduces the problem: import select poller = select.poll() while 1: poller.register(101, 1) (now go peek at the memory usage for this process - it skyrockets) If you replace 101 above with 100 or less, the memory usage remains flat. I found this problem because my program routinely has to handle more than 100 socket connections. I believe the reference to the flag parameter is also being leaked, but you don't usually see the problem because flag values are small and are therefore references to cached integers. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474538&group_id=5470 From noreply@sourceforge.net Wed Oct 24 18:14:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 10:14:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-432497 ] curses module doesn't build on HP-UX Message-ID: Bugs item #432497, was opened at 2001-06-12 11:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=432497&group_id=5470 Category: Extension Modules Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: M.-A. Lemburg (lemburg) >Assigned to: Martin v. Löwis (loewis) Summary: curses module doesn't build on HP-UX Initial Comment: The curses module does not build on HP-UX 11.00 (don't know about other versions). The reason according to Peter Stoldt (peter_stoldt@hp.com) who provided the fix below is to look for the curses header file in a different directory. Here is his fix: In py_curses.h exchange the line with #include with #include This would have to be done in a platform specific way of course. Perhaps all it takes is adding the curses_colr/ dir to the compiler call as -I option... not sure. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-24 10:14 Message: Logged In: YES user_id=21627 Using curses_colr/curses.h is actually wrong, since colr Curses is deprecated, and provided only for backwards compatibity. The long-term curses for HPUX is Xcurses from . This is supported through acconfig.h 1.57 configure.in 1.278 pyconfig.h.in 1.20 py_curses.h 1.3 _cursesmodule.c 2.58 ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2001-07-11 12:28 Message: Logged In: YES user_id=11375 It seems reasonable to check for that directory in the configure script, and use it if present. I'll do that. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=432497&group_id=5470 From noreply@sourceforge.net Wed Oct 24 18:16:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 10:16:45 -0700 Subject: [Python-bugs-list] [ python-Bugs-422265 ] Python 2.1 curses build failure on HPUX Message-ID: Bugs item #422265, was opened at 2001-05-08 05:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=422265&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Richard Townsend (rptownsend) >Assigned to: Martin v. Löwis (loewis) Summary: Python 2.1 curses build failure on HPUX Initial Comment: Builing the curses module from Python 2.1 on HPUX 11.0 fails with the following errors: building '_curses' extension cc -Ae -O +z -I. -I/home/richardt/python/Python- 2.1/./Include -I/usr/local/include -IInclude/ - c /home/richardt/python/Python- 2.1/Modules/_cursesmodule.c -o build/temp.hp-ux- B.11.00-9000/770-2.1/_cursesmodule.o cc: line 309: warning 5004: Uninitialized variable "arg1" in function "PyCursesWindow_getbegyx" (5004) cc: line 309: warning 5004: Uninitialized variable "arg2" in function "PyCursesWindow_getbegyx" (5004) cc: line 310: warning 5004: Uninitialized variable "arg1" in function "PyCursesWindow_getmaxyx" (5004) cc: line 310: warning 5004: Uninitialized variable "arg2" in function "PyCursesWindow_getmaxyx" (5004) cc: line 311: warning 5004: Uninitialized variable "arg1" in function "PyCursesWindow_getparyx" (5004) cc: line 311: warning 5004: Uninitialized variable "arg2" in function "PyCursesWindow_getparyx" (5004) For full error listing see attached file curses.txt ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-24 10:16 Message: Logged In: YES user_id=21627 Curses compilation is now fixed in acconfig.h 1.57 configure.in 1.278 pyconfig.h.in 1.20 py_curses.h 1.3 _cursesmodule.c 2.58 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=422265&group_id=5470 From noreply@sourceforge.net Wed Oct 24 18:20:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 10:20:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-210665 ] Compiling python on hpux 11.00 with threads (PR#360) Message-ID: Bugs item #210665, was opened at 2000-07-31 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 Category: Threads Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: Compiling python on hpux 11.00 with threads (PR#360) Initial Comment: Jitterbug-Id: 360 Submitted-By: philipp.jocham@salomon.at Date: Fri, 16 Jun 2000 08:47:06 -0400 (EDT) Version: 1.5.2 OS: HP-UX 11.00 There are two missing details in the configure process to make this work out of the box. First: The function pthread_create isn't found in library libpthread.a but in libcma.a, because pthread_create is just a macro in sys/pthread.h pointing to __pthread_create_system After patching ./configure directly and running ./configure --with-thread (now detecting the correct library /usr/lib/libpthread.a) I also added -lcl to Modules/Makefile at LIBS= -lnet -lnsl -ldld -lpthread -lcl otherwise importing of modules with threads didn't work (in this case oci_.sl from DCOracle). I'm not sure about the correct syntax or wether it's the correct place and method, but I would suggest a solution like the following code snippet. [...] AC_MSG_CHECKING(for --with-thread) [...] AC_DEFINE(_POSIX_THREADS) LIBS="$LIBS -lpthread -lcl" LIBOBJS="$LIBOBJS thread.o"], [ AC_CHECK_LIB(pthread, __pthread_create_system, [AC_DEFINE(WITH_THREAD) [...] I hope this helps to make installation process smoother. Fell free to contact me, if there are further questions. Philipp -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ==================================================================== Audit trail: Tue Jul 11 08:26:01 2000 guido moved from incoming to open ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-24 10:20 Message: Logged In: YES user_id=21627 According to Richard Townsend, the core of this problem has been fixed in the CVS. The last related check-in was configure.in 1.268. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-08 06:21 Message: Logged In: YES user_id=21627 I have now made the -Kpthread more robust by running a program; this allows to remove the gcc check (atleast for Linux and Solaris). The same check should work on all systems that require special options for pthread support; please confirm whether it solves the AIX problem. I've also applied your patch to check pthread_create linkage through including pthread.h; please report whether configuration now succeeds on HP/UX. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-05 10:13 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-05 08:21 Message: Logged In: YES user_id=21627 I still don't see the breakage on AIX. The compiler will complain about the option; ok. If the binary is executable (and really uses threads), what is the problem? In any case, if you come up with an improved test, I can try it out. On the pthread_create change: +1. I'd like to see a comment on why you need to link there, though. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-04 16:23 Message: Logged In: YES user_id=119770 Well, when -Kpthread was introduced, it should not have caused a regression on an existing solution. It breaks HP-UX 11.00 (not that it worked before though). And, it will break AIX. The problem is that: $ cc foo.c -Kpthread *will* generate an a.out that is executable. The compiler just complains about invalid options: cc: warning 422: Unknown option "K" ignored. cc: error 1400: Option t usage: -t c,name where c may be 1 or more of pc0al. Ditto for the IBM xlc compiler: $ xlc foo.c -Kpthread xlc: 1501-210 command option t contains an incorrect subargument So, the test for -Kpthread has to be more robust. Regarding AC_CHECK_LIB for pthread_create, the following will not work: AC_CHECK_LIB(pthread, pthread_create) By doing this, you mistakenly assume that pthread_create() equates to an entry in libpthread named pthread_create. This is not true on HP-UX 11.00 (nor Tru64 UNIX). The header file on HP-UX 11.00 includes which has pthread_create() equating to __pthread_create_system. By using the method I posted, we workaround whatever semantics the system uses to provide pthread_create. I will try to work up a solution for -Kpthread. Can I get access to a UnixWare system to test? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-04 16:08 Message: Logged In: YES user_id=21627 Just taking out the -Kpthread test is not acceptable. It will mean that thread support will break systems that require it. Likewise, taking it as a fallback will fail on the very same system. Some of those systems (e.g. UnixWare) do provide a libpthread and a pthread_create function within, but the documentation explicitly says that you must not use -lpthread. So we *must* use -Kpthread were available. What exactly is the problem on HP/UX? If the compiler links the test program, why don't threads work then? Can you give a C program that shows that threads are not available when just passing -Kpthread? I agree that special-casing gcc is a hack, but what I need is a solution, not removal of the existing feature. Also, what exact problem is solved with your change "for pthread_create in -lpthread": Why do you need a test program; why is AC_CHECK_LIB not good enough? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-04 08:23 Message: Logged In: YES user_id=6380 I'm assigning this to Martin, since he checked in the code that tests for -Kpthread, mentioning Patch #418659: Fixes for UnixWare and ReliantUnix. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-10-03 23:22 Message: Logged In: YES user_id=119770 Ok, new patch against HEAD (2.2) at ftp://ftp.thewrittenword.com/outgoing/pub/python-210665.patch. BTW, I've axed the addition of determining if "cc -Kpthread" works. This is gross. Worst case, add it after all of the other pthread checks. The HP-UX C compiler will link a test program with -Kpthread. It'll just issue a warning. I see you've made an exception to setting ac_cv_kpthread if the compiler is GCC. Ick! The IBM C compiler also issues a warning about -Kpthread but compiles the test program. The Sun, Compaq, and SGI compilers do error out though. With this patch, test_thread succeeds. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-08-08 08:11 Message: Logged In: YES user_id=119770 I'd rather see a new configure option, --with-cma-threads, similar to --with-dec-threads, to enable the CMA thread support for HP-UX 10.20. We'll try to work on a configure.in patch against CVS for HP-UX 11.00. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-08 07:56 Message: Logged In: YES user_id=6380 Actually, now that you mention it, yes, there's a reason not to accept that patch (which I hadn't seen before -- sorry). It appears to be patching tons of unrelated things, and some of the diffs in it appear to be diffs between different Python versions. Also, it patches configure instead of configure.in. configure is a generated file. Ditto for config.h.in. In other words, that patch may fix your problem, but I can't check it in. Perhaps you could work on a *minimal* patch that solves the problem, and which works with the current CVS (or at least the 2.2a1 release). Then I'll look again. Since you show interest in doing this, I've reopened the bug report. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-08-08 05:44 Message: Logged In: YES user_id=200117 On the 19th June I posted a comment here ref the patch file: 'python-2.1.patch' from ftp://ftp.thewrittenword.com/outgoing/pub After applying this patch to Python 2.1 source, I successfully built a Python interpreter on HP-UX 11.0 with threads enabled. (I have tested it with several examples from Mark Lutz's book using the thread and threading modules and it works fine). Is there some reason why you do not accept this patch ? It would be really useful if this fix could be included for Python 2.2 :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-07 12:26 Message: Logged In: YES user_id=6380 Closing this for lack of progress. If someone figures out what to do with threads on HPUX-11, please submit a fresh *patch*. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 10:31 Message: Logged In: YES user_id=6380 Alas, not. Those binaries don't have threads enabled. HP doesn't know how to do it either. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-07-27 08:37 Message: Logged In: YES user_id=6380 For all of those in search for an easy solution: HP makes binaries available. See http://hpux.connect.org.uk/hppd/hpux/Languages/python-2.1/ ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-19 03:07 Message: Logged In: YES user_id=200117 I have now applied the new patch file 'python-2.1.patch' available at: ftp://ftp.thewrittenword.com/outgoing/pub I can now successfully build Python 2.1 with threads enabled, on HP-UX 11. ---------------------------------------------------------------------- Comment By: Richard Townsend (rptownsend) Date: 2001-06-14 01:20 Message: Logged In: YES user_id=200117 I applied the patch from thewrittenword's site, but when I ran autoconf it generated a corrupt configure script. There problem occurs around lines 3895-3906: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1 fi LDLAST=-threads;; esac fi fi fi The case statement has been trashed by the extra 'fi' token. I tried manually editing it like this: if test "$USE_THREAD_MODULE" != "#" then # If the above checks didn't disable threads, (at least) OSF1 # needs this '-threads' argument during linking. case $ac_sys_system in OSF1) LDLAST=-threads;; esac fi fi fi But it still fails with an 'else' not matched at line 3422. I can't see where the extra 'fi' should go. ---------------------------------------------------------------------- Comment By: The Written Word (Albert Chin) (tww-china) Date: 2001-05-06 22:36 Message: Logged In: YES user_id=119770 You can find a patch to fix this against python 2.1 at: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.1-416696.patch You'll need to rerun autoconf to test. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-11-02 08:38 Message: Reopened because there's a dissenting opinion. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-02 07:25 Message: Ok, check out the configure.in patch I created against Python 2.0: ftp://ftp.thewrittenword.com/outgoing/pub/python-2.0.patch I tested it under HP-UX 11.00 and it works just fine. The thread test worked too. -- albert chin (china@thewrittenword.com) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-11-01 09:18 Message: Ick! Why check for anything with __ prepended to the name? Isn't that like checking for a "hidden" function, which might not be there in a followup version? On HP-UX 11.00, pthread_create is in /usr/lib/libc.sl anyway. The proper way to check for pthread_create is: AC_TRY_LINK([#include void * start_routine (void *arg) { exit (0); }], [ pthread_create (NULL, NULL, start_routine, NULL)], [ AC_MSG_RESULT(yes)], [ AC_MSG_RESULT(no)]) I modified configure.in in 2.0 to remove the patch you included in CVS 1.175 and added a test to include similar to the above (linked without -lpthread and with -lpthread). I'm testing now. Will provide a patch when things are tested. Also, I don't think threads on HP-UX 10.20 will work unless you have the DCE libraries installed. Anyhow, I'd probably avoid threads on 10.20. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-30 09:48 Message: Philipp submitted a patch to configure.in that fixes the problem for him and doesn't look like it would break things for others. configure.in, CVS revision 1.175. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:45 Message: OK, so the correct thing to do seems to be to somehow add #include to the tests for thread libraries. I'm afraid I won't be able to fix this in 2.0final, but I'll think about fixing it in 2.1 (or 2.0.1, whichever comes first :-). ---------------------------------------------------------------------- Comment By: Eddy De Greef (edg) Date: 2000-10-10 04:55 Message: I can confirm that the bug still exists in 2.0c1. It works fine on HP-UX 10.20 (which only has libcma), but not on HP-UX 11 (which both has libcma and libpthread). The pthread_create function is not defined as a macro though, but as a static function: static int pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { return(__pthread_create_system(tid, attr, start_routine, arg)); } I don't see an easy way to work around this. I'm not a configure expert, but perhaps the script should first check whether this code compiles and links: #include int main() { pthread_create(0,0,0,0); return 0; } and if not, fall back on the other tests ? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-06 10:40 Message: I have two reports from people for whom configure, build and run with threads now works on HP-UX 10 and 11. I'm not sure what to do about this report... What's different on Philipp's system??? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-25 06:10 Message: I'm hoping that this was fixed by recent changes. Sent an email to the original submittor to verify. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-09-22 02:56 Message: Taking this because I'm considering to redesign the thread configuration section in configure.in anyway -- there's a similar bug report for Alpha OSF1. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2000-09-07 15:05 Message: Please do triage on this bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210665&group_id=5470 From noreply@sourceforge.net Wed Oct 24 18:22:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 10:22:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-24 10:22 Message: Logged In: YES user_id=21627 On the curses problems, please check out the latest CVS again; it is supposed to be fixed on HP/UX 11. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:01 Message: Logged In: YES user_id=6380 Thanks for the list. Can you submit separate patches or bug reports for the things you know need work? That seems easier than holding this conversation in an unrelated closed bug report. :-) OPT is *not* the right place for a -I option, but there's currently no "right" place. I've seen them added to "CC", which makes more sense to me. Is HP-UX 10.20 still relevant? If so, it might be nice if the configure script defaulted to --without-threads. (Is the problem that -lcma has pthread_create but in fact doesn't support the rest of the PTHREAD API?) ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:50 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_socket and test_time fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:48 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_??? and test_??? fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 18:02 Message: Logged In: YES user_id=6380 Do the tests for threads pass? (test_thread.py, test_threading.py, test_threaded_import.py, test_threadedtempfile.py) That would be great news! If there are any other problems on HP-UX, it would be great if you could help out. Do you know what those problems are? ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-20 08:41 Message: Logged In: YES user_id=354556 Thist patch has no effect on threads. However, threads do work (i.e., are recognized by configure) on HP-UX 11 since 2.2b1 (I just wanted to submit a patch to the 2.2a4 configure.in when 2.2b1 was released). I am willing to support you regarding HP-UX, if necessary (there still seem to be some open issues). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Wed Oct 24 18:26:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 10:26:00 -0700 Subject: [Python-bugs-list] [ python-Bugs-416688 ] Python 2.1c2 build using --with-cxx Message-ID: Bugs item #416688, was opened at 2001-04-17 06:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=416688&group_id=5470 Category: Build Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Harri Pasanen (harripasanen) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.1c2 build using --with-cxx Initial Comment: After configure + build with: ./configure --with-thread --without-gcc --with-cxx=KCC make The build fails as it tries to perform the link with cc, not with KCC. It has previously build correctly ccpython using KCC. Manually performing the linking using KCC then works: cc -Ae -Wl,-E -Wl,+s -Wl,+b/usr/local/lib/python2.1/lib-dynload -o python \ Modules/ccpython.o \ libpython2.1.a -lnsl -ldld -lm /usr/ccs/bin/ld: (Warning) At least one PA 2.0 object file (libpython2.1.a(main.o)) was detected. The linked output may not run on a PA 1.x system. /usr/ccs/bin/ld: Unsatisfied symbols: _main (first referenced in Modules/ccpython.o) (code) The above problem seems to be present on Linux and HP-UX, (probably all platforms using autoconf). -Harri ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-24 10:26 Message: Logged In: YES user_id=21627 This should be fixed with configure.in 1.271 (to appear in Python 2.2b2), atleast for aCC. If the relevant configure test still fails for KCC, please submit a new report. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=416688&group_id=5470 From noreply@sourceforge.net Wed Oct 24 19:57:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 11:57:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-474583 ] non-standard types in mimetypes.py Message-ID: Bugs item #474583, was opened at 2001-10-24 11:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474583&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Martin v. Löwis (loewis) Summary: non-standard types in mimetypes.py Initial Comment: In the spirit of "be liberal in what you accept", I think mimetypes should have an option which searches non-standard but common types. E.g. I just received an email message which had a content type of image/jpg and I wanted to store that in a file with the proper extension. But mimetypes couldn't give that to me because image/jpeg is the official type. Attached is a patch to add an optional "strict" flag to guess_type() and guess_extension(). strict defaults to 1 to keep the current behavior. Set it to 0 and some additional types are searched. I added back the non-standard types that Martin removed in revision 1.19 of the file. This patch also contains a small improvement to the command line options when this is run as a script. I can split the patch into two parts if necessary, but I don't think it is. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474583&group_id=5470 From noreply@sourceforge.net Wed Oct 24 20:04:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 12:04:10 -0700 Subject: [Python-bugs-list] [ python-Bugs-474585 ] Deprecate modules Message-ID: Bugs item #474585, was opened at 2001-10-24 12:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Deprecate modules Initial Comment: The following modules should be marked as deprecated because the new email package provides all the functionality (using different APIs perhaps, and it steals some implementation from rfc822.py, but still...) rfc822.py mimetools.py MIMEWriter.py Technically mimify.py should be deprecated too, however the email package doesn't provide the run-as-script functionality. That would be easy to provide as a Tools/script, so I still think it should be deprecated. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 From noreply@sourceforge.net Wed Oct 24 20:24:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 12:24:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-473525 ] pyclbr broken Message-ID: Bugs item #473525, was opened at 2001-10-21 16:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) >Summary: pyclbr broken Initial Comment: classbrowser on pydoc.py does not show class entries reordering of classdefinitions shows class entries but listing breaks lateron idle 0.8 Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] on win32 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-24 12:24 Message: Logged In: YES user_id=6380 Yup, you're right. The three string literals in the synopsis() function of the form '"""' or 'r"""' apparently throw off the pyclbr parser. I've changed the subject to match this. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 From noreply@sourceforge.net Wed Oct 24 20:52:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 12:52:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-473986 ] __all__ undocumented for modules Message-ID: Bugs item #473986, was opened at 2001-10-23 03:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473986&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Richard Brodie (leogah) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: __all__ undocumented for modules Initial Comment: The effect of defining __all__ in module scope is undocumented. In 6.11 of the language reference, the statement beginning: "If the list of identifiers is replaced by a star ("*")," needs to be qualified. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 12:52 Message: Logged In: YES user_id=3066 Fixed in Doc/ref/ref6.tex revisions 1.42 and 1.37.2.1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473986&group_id=5470 From noreply@sourceforge.net Wed Oct 24 20:55:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 12:55:49 -0700 Subject: [Python-bugs-list] [ python-Bugs-473525 ] pyclbr broken Message-ID: Bugs item #473525, was opened at 2001-10-21 16:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: pyclbr broken Initial Comment: classbrowser on pydoc.py does not show class entries reordering of classdefinitions shows class entries but listing breaks lateron idle 0.8 Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] on win32 ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-24 12:55 Message: Logged In: YES user_id=31435 Reassigned to me -- pyclbr has always taken shortcuts in the presence of single-quoted strings, and fixing that is overdue. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-24 12:24 Message: Logged In: YES user_id=6380 Yup, you're right. The three string literals in the synopsis() function of the form '"""' or 'r"""' apparently throw off the pyclbr parser. I've changed the subject to match this. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 From noreply@sourceforge.net Wed Oct 24 21:24:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 13:24:55 -0700 Subject: [Python-bugs-list] [ python-Bugs-473525 ] pyclbr broken Message-ID: Bugs item #473525, was opened at 2001-10-21 16:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 Category: IDLE Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: pyclbr broken Initial Comment: classbrowser on pydoc.py does not show class entries reordering of classdefinitions shows class entries but listing breaks lateron idle 0.8 Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] on win32 ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-24 13:24 Message: Logged In: YES user_id=31435 Fixed, in Lib/pyclbr.py; new revision: 1.23 This teaches pyclbr all about single-quoted strings. If you try IDLE's class browser on the 2.2 pydoc.py, though, you'll find other flakiness due to pyclbr's and IDLE's limited understanding of nested classes and functions. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-24 12:55 Message: Logged In: YES user_id=31435 Reassigned to me -- pyclbr has always taken shortcuts in the presence of single-quoted strings, and fixing that is overdue. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-24 12:24 Message: Logged In: YES user_id=6380 Yup, you're right. The three string literals in the synopsis() function of the form '"""' or 'r"""' apparently throw off the pyclbr parser. I've changed the subject to match this. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473525&group_id=5470 From noreply@sourceforge.net Wed Oct 24 22:32:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 14:32:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-445960 ] Compiler warnings on IRIX 6.5 Message-ID: Bugs item #445960, was opened at 2001-07-30 07:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445960&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: Compiler warnings on IRIX 6.5 Initial Comment: Building 2.2a1 on IRIX 6.5 with the MIPSPro 7.30 C compiler results in a handful of minor compiler warnings. Some of them are legit, some are compiler stupidity. I've just included the warnings here that look legit to me, ie. it looks like the MIPSPro compiler is catching an error. cc-1552 cc: WARNING File = Python/import.c, Line = 857 The variable "f" is set but never used. struct _frozen *f; ^ cc-1552 cc: WARNING File = Python/import.c, Line = 1379 The variable "mod" is set but never used. PyObject *mod; ^ [GW: both variables are only used in "(f = (...)) != NULL)" constructs, so it looks like the warning is legit and these variables are unneeded] cc-1552 cc: WARNING File = /tmp/Python-2.2a1/Modules/xreadlinesmodule.c, Line = 174 The variable "m" is set but never used. PyObject *m; ^ cc-1552 cc: WARNING File = /tmp/Python-2.2a1/Modules/selectmodule.c, Line = 335 The variable "j" is set but never used. int i, j, pos; ^ ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-24 14:32 Message: Logged In: YES user_id=14422 Looks like all the legit warnings caught by the IRIX 6.5 compiler have been fixed. Said compiler still emits some warnings, but they're either a) compiler brain-damage or b) macro complications where there's no obvious "fix" (and it's not obvious whether the compiler is right or not). Thus, closing this bug and marking it fixed. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445960&group_id=5470 From noreply@sourceforge.net Wed Oct 24 22:42:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 14:42:38 -0700 Subject: [Python-bugs-list] [ python-Bugs-474585 ] Deprecate modules Message-ID: Bugs item #474585, was opened at 2001-10-24 12:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Deprecate modules Initial Comment: The following modules should be marked as deprecated because the new email package provides all the functionality (using different APIs perhaps, and it steals some implementation from rfc822.py, but still...) rfc822.py mimetools.py MIMEWriter.py Technically mimify.py should be deprecated too, however the email package doesn't provide the run-as-script functionality. That would be easy to provide as a Tools/script, so I still think it should be deprecated. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 14:42 Message: Logged In: YES user_id=3066 After discussion with Guido, this appears to be too agressive a deprecation schedule for these modules, especially rfc822. A good first step would be to demonstrate that all uses of these modules can use the email package. Perhaps implementations of these modules should be made using the facilities of the email package? (Don't check them into the library yet; perhaps add them to nondist/sandbox/Lib/ first.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 From noreply@sourceforge.net Wed Oct 24 22:45:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 14:45:10 -0700 Subject: [Python-bugs-list] [ python-Bugs-445986 ] test_locale fails on IRIX 6.5 Message-ID: Bugs item #445986, was opened at 2001-07-30 08:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445986&group_id=5470 Category: Python Library Group: Platform-specific >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Greg Ward (gward) Assigned to: Greg Ward (gward) Summary: test_locale fails on IRIX 6.5 Initial Comment: Built Python 2.2a1 on a 64-bit SGI box running IRIX 6.5. test_locale fails as follows: $ ./python Lib/test/test_locale.py '%f' % 1024 =? '1,024.000000' ... no '%f' % 1024 == '1024.000000' != '1,024.000000' '%f' % 102 =? '102.000000' ... yes '%f' % -42 =? '-42.000000' ... yes '%+f' % -42 =? '-42.000000' ... yes '%20.f' % -42 =? ' -42' ... yes '%+10.f' % -4200 =? ' -4,200' ... no '%+10.f' % -4200 == ' -4200' != ' -4,200' '%-10.f' % 4200 =? '4,200 ' ... no '%-10.f' % 4200 == '4200 ' != '4,200 ' ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2001-10-24 14:45 Message: Logged In: YES user_id=14422 I'm happy to take Martin's word on this -- closing this bug with "Won't fix". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-08-04 01:21 Message: Logged In: YES user_id=21627 This is a bug in the system C library, which fails to report proper separators in the en_US locale. Please complain to your operating system vendor; I propose to close this as "Won't fix". ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445986&group_id=5470 From noreply@sourceforge.net Wed Oct 24 23:06:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 15:06:12 -0700 Subject: [Python-bugs-list] [ python-Bugs-474585 ] Deprecate modules Message-ID: Bugs item #474585, was opened at 2001-10-24 12:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Open Resolution: Rejected Priority: 5 Submitted By: Barry Warsaw (bwarsaw) >Assigned to: Barry Warsaw (bwarsaw) Summary: Deprecate modules Initial Comment: The following modules should be marked as deprecated because the new email package provides all the functionality (using different APIs perhaps, and it steals some implementation from rfc822.py, but still...) rfc822.py mimetools.py MIMEWriter.py Technically mimify.py should be deprecated too, however the email package doesn't provide the run-as-script functionality. That would be easy to provide as a Tools/script, so I still think it should be deprecated. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-24 15:06 Message: Logged In: YES user_id=12800 Assigning back to me and re-opening. Not sure it'll be worth it, but this way it'll keep bugging me. :) ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 14:42 Message: Logged In: YES user_id=3066 After discussion with Guido, this appears to be too agressive a deprecation schedule for these modules, especially rfc822. A good first step would be to demonstrate that all uses of these modules can use the email package. Perhaps implementations of these modules should be made using the facilities of the email package? (Don't check them into the library yet; perhaps add them to nondist/sandbox/Lib/ first.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 From noreply@sourceforge.net Wed Oct 24 23:10:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 15:10:33 -0700 Subject: [Python-bugs-list] [ python-Bugs-469772 ] support additional LINK elements. Message-ID: Bugs item #469772, was opened at 2001-10-09 23:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 Category: Documentation Group: Feature Request Status: Open >Resolution: Fixed Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) >Assigned to: Anthony Baxter (anthonybaxter) Summary: support additional LINK elements. Initial Comment: Now that mozilla supports the LINK element properly, it would be most excellent if the python docs were generated with more LINK elements. At the moment they're built with up, previous, and next. Some of the others are defined in http://www.w3.org/TR/REC-html40/types.html#type-links Ones that would be very nice are Top, Contents and Index. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 15:10 Message: Logged In: YES user_id=3066 Anthony, what do you think of the current status on this? Do we need to worry about this further in the near future? I've marked it fixed but left it open & assigned to you; please comment and either close or re-assign to me as needed. Thanks! ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 14:27 Message: Logged In: YES user_id=3066 The Mozilla bug I filed (see my previous comment) was declared a duplicate of this bug: http://bugzilla.mozilla.org/show_bug.cgi?id=102915 ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 10:05 Message: Logged In: YES user_id=3066 OK, I've played with this a bit more, and I've modified the transformation to support more elements (Doc/perl/l2hinit.perl revision 1.56), but the intended semantics are still pretty fuzzy. In particular, the links described seem to work within a tree which can share a single definition of these links throughout. Given that "top", "start", and "first" are not well defined, it's not clear how to apply these. The Python documentation consists of a set of documents which each have specific identities (the Library Reference, Python/C API manual, etc.). Does "start" refer to the front of an individual manual or the collection? How about "top"? The use of "document" to mean "page" in the W3C documents doesn't help. What I've done uses the title page of each document as the "first" link, but doesn't provide the "top" link at all. Mozilla also doesn't seem to understand that multiple indexes are entirely valid; it should probably convert the "Index" entry to a menu when there are multiple indexes -- I'll file a Mozilla bug report for this one. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-22 08:08 Message: Logged In: YES user_id=3066 Added some support for the "up" link in Doc/tools/support.py revision 1.4, but this only affects a couple of pages. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 08:46 Message: Logged In: YES user_id=29957 There's a very nice list of what link relationships do what for mozilla 0.9.5 posted at: http://lists.w3.org/Archives/Public/www-html/2001Oct/0026.html (link stolen from mozillazine) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469772&group_id=5470 From noreply@sourceforge.net Wed Oct 24 23:27:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 15:27:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows) Message-ID: Bugs item #216289, was opened at 2000-10-06 19:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470 Category: Windows Group: 3rd Party Status: Closed Resolution: Wont Fix Priority: 3 Submitted By: Tim Peters (tim_one) Assigned to: Tim Peters (tim_one) Summary: Programs using Tkinter sometimes can't shut down (Windows) Initial Comment: The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS. In my case, I was running IDLE via python ../tool/idle/idle.pyw from a DOS box in my PCbuild directory. Win98SE. *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging. As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap. -----Original Message----- From: Joseph Stubenrauch Sent: Friday, October 06, 2000 9:23 PM To: tutor@python.org Subject: Re: [Tutor] Python 1.6 BUG Strange, I have been experiencing the same bug myself. Here's the low down for me: Python 1.6 with win95 I am running a little Tkinter program The command line I use is simply: "python foo.py" About 25-35% of the time, when I close the Tkinter window, DOS seems to "freeze" and never returns to the c:\ command prompt. I have to ctrl-alt-delete repeatedly and shut down "winoldapp" to get rid of the window and then shell back into DOS and keep working. It's a bit of a pain, since I have the habit of testing EVERYTHING in tiny little stages, so I change one little thing, test it ... freeze ... ARGH! Change one more tiny thing, test it ... freeze ... ARGH! However, sometimes it seems to behave and doesn't bother me for an entire several hour session of python work. That's my report on the problem. Cheers, Joe ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-24 15:27 Message: Logged In: YES user_id=31435 FYI, you don't need an IDE to do this -- in Win9x, hit Ctrl+Alt+Del and kill the process directly. A saner solution is to develop under Win2K, which doesn't appear to suffer this problem (the only reports I've seen, and experienced myself, came from Win9x boxes). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-10-24 01:52 Message: Logged In: NO For those who are still trapped in this bug's hell, I will gladly share the one thing that saved my sanity: WingIDE's 'Kill' command will shut down the program with apparent 100% certainty and no fear of lockups. WingIDE has its own issues, so its not a perfect solution, but if you are like me and Joe (above) who test in small iterations, then using 'Kill' to quit out of your app while testing is a workaround that may preserve your sanity. Perhaps the python gods and the Wing guys can get together and tell us how to replicate 'kill' into our code. For now, I'll use WingIDE to edit, and pythonw.exe for my final client's delivery. ---------------------------------------------------------------------- Comment By: Howard Lightstone (hlightstone) Date: 2001-09-05 10:43 Message: Logged In: YES user_id=66570 I sometimes get bunches of these.... A tool I use (Taskinfo2000) reports that (after killing winoldap): python.exe is blocked on a mutex named OLESCELOCKMUTEX. The reported state is "Console Terminating". There appears to be only one (os) thread running. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-04-02 13:06 Message: Logged In: YES user_id=31435 No sign of progress on this presumed Tk/Tcl Windows bug in over 3 months, so closing it for lack of hope. ---------------------------------------------------------------------- Comment By: Doug Henderson (djhender) Date: 2001-02-05 21:13 Message: This was a symptom I saw while tracking down the essence of the problem reported in #131207. Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2000-12-12 18:00 Message: Just reproduced w/ current CVS, but didn't hang until the 8th try. http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-12-12 12:58 Message: Tim, can you still reproduce this with the current CVS version? There's been one critical patch to _tkinter since the 2.0 release. An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2000-10-15 09:47 Message: Same as I've reported earlier; it hangs in the call to Tcl_Finalize (which is called by the DLL finalization code). It's less likely to hang if I call Tcl_Finalize from the _tkinter DLL (from user code). Note that the problem isn't really Python-related -- I have stand-alone samples (based on wish) that hangs in the same way. More later. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-13 07:40 Message: Back to Tim since I have no clue what to do here. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-10-12 10:25 Message: The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either. As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again. Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2000-10-07 00:37 Message: More info (none good, but some worse so boosted priority): + Happens under release and debug builds. + Have not been able to provoke when starting in the debugger. + Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up. There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show. + This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot. + These processes cannot be killed! Wintop and Process Viewer both fail to get the job done. PrcView (a freeware process viewer) itself locks up if I try to kill them using it. Process Viewer freezes for several seconds before giving up. + Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program". + The processes are not accumulating cycles. + Smells like deadlock. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470 From noreply@sourceforge.net Thu Oct 25 03:11:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 19:11:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-464900 ] Include TKInter Docs Message-ID: Bugs item #464900, was opened at 2001-09-25 11:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=464900&group_id=5470 Category: Documentation Group: None Status: Open >Resolution: Later >Priority: 4 Submitted By: paul rubin (phr) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Include TKInter Docs Initial Comment: The standard Python distributions include the TKinter library but no docs for it. Arggh! I hope this can be remedied. The docs I've seen aren't so great, but they're enough to get started. Please consider tossing them into the Python distribution even if there's nobody available to edit them further. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 19:11 Message: Logged In: YES user_id=3066 I'm not ignoring this, and one person is working on some docs for the Tk-related modules, but the existing published docs are not available to be tossed-in -- someone else wrote them and owns the copyright. Rest assured, this will be remedied, but it won't be quick given the number of documentation issues that need to be handled at this time. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=464900&group_id=5470 From noreply@sourceforge.net Thu Oct 25 03:12:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 19:12:20 -0700 Subject: [Python-bugs-list] [ python-Bugs-469773 ] docs should include man page Message-ID: Bugs item #469773, was opened at 2001-10-09 23:28 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 Category: Documentation Group: Feature Request >Status: Pending Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: docs should include man page Initial Comment: Some stuff that's in the man page (e.g. the syntax for the -W switch) doesn't appear to be in the HTML documentation. If it is there, it's _very_ well hidden away. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 19:12 Message: Logged In: YES user_id=3066 I agree. Where would you suggest the information be added? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 From noreply@sourceforge.net Thu Oct 25 04:23:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 20:23:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-474238 ] 2.2b1 src rpm doesn't include Lib/email/ Message-ID: Bugs item #474238, was opened at 2001-10-23 13:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474238&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: 2.2b1 src rpm doesn't include Lib/email/ Initial Comment: Using the python2.2-2.2b1-1.src.rpm to create binary rpms, I found that the installed RPM did not contain the new (post-2.2a3, anyway) Lib/email/ module. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2001-10-24 20:23 Message: Logged In: YES user_id=81797 Doh! I've also added the "compiler" module. 2.2b1-2 RPMs are on ftp.tummy.com:/pub/tummy/RPMS/SRPMS and RPMS/binaries-KRUD-7.2-i386 -- Guido should be moving them to the python.org site when he gets a chance. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474238&group_id=5470 From noreply@sourceforge.net Thu Oct 25 05:48:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 21:48:16 -0700 Subject: [Python-bugs-list] [ python-Bugs-474737 ] Problem overriding int.__init__() Message-ID: Bugs item #474737, was opened at 2001-10-24 21:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474737&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mark J (average) Assigned to: Guido van Rossum (gvanrossum) Summary: Problem overriding int.__init__() Initial Comment: Given: class I(int): def __init__(self, init_value, extra_param="test"): print extra_param int.__init__(self, init_value) Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 >>> i = I(1) #fine test #yes, new constructor used >>> i 1 #value set correctly >>> i = I(2, "oops") Traceback (most recent call last): File "", line 1, in ? TypeError: an integer is required >>> i = I(3, dummy="oops") Traceback (most recent call last): File "", line 1, in ? TypeError: dummy is an invalid keyword argument for this function >>> Thanks, Mark ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474737&group_id=5470 From noreply@sourceforge.net Thu Oct 25 07:17:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 24 Oct 2001 23:17:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-469773 ] docs should include man page Message-ID: Bugs item #469773, was opened at 2001-10-09 23:28 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 Category: Documentation Group: Feature Request >Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: docs should include man page Initial Comment: Some stuff that's in the man page (e.g. the syntax for the -W switch) doesn't appear to be in the HTML documentation. If it is there, it's _very_ well hidden away. ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-24 23:17 Message: Logged In: YES user_id=29957 See, that's the problem :) It's not clear where this might go - it's almost a "Using Python" section, which, as far as I can see, doesn't really exist. This might also include stuff about the Windows GUI and other interfaces for using python. If a shorter-term "just get the -W stuff in" approach is wanted, then extended the lib/module-warnings might be the approach... but it really does seem like it needs to be somewhere central. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 19:12 Message: Logged In: YES user_id=3066 I agree. Where would you suggest the information be added? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 From noreply@sourceforge.net Thu Oct 25 09:46:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 01:46:59 -0700 Subject: [Python-bugs-list] [ python-Bugs-474583 ] non-standard types in mimetypes.py Message-ID: Bugs item #474583, was opened at 2001-10-24 11:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474583&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open >Resolution: Accepted Priority: 5 Submitted By: Barry Warsaw (bwarsaw) >Assigned to: Barry Warsaw (bwarsaw) Summary: non-standard types in mimetypes.py Initial Comment: In the spirit of "be liberal in what you accept", I think mimetypes should have an option which searches non-standard but common types. E.g. I just received an email message which had a content type of image/jpg and I wanted to store that in a file with the proper extension. But mimetypes couldn't give that to me because image/jpeg is the official type. Attached is a patch to add an optional "strict" flag to guess_type() and guess_extension(). strict defaults to 1 to keep the current behavior. Set it to 0 and some additional types are searched. I added back the non-standard types that Martin removed in revision 1.19 of the file. This patch also contains a small improvement to the command line options when this is run as a script. I can split the patch into two parts if necessary, but I don't think it is. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-25 01:46 Message: Logged In: YES user_id=21627 Sounds good to me, assuming it is also accompanied by a documentation change. You may also consider adding the types I rejected from #438790, in particular image/pict, text/xsl and text/xul (not sure whether any others were rejected). Back to Barry for checkin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474583&group_id=5470 From noreply@sourceforge.net Thu Oct 25 12:15:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 04:15:35 -0700 Subject: [Python-bugs-list] [ python-Bugs-474831 ] Command history doesn't work on Mandrake Message-ID: Bugs item #474831, was opened at 2001-10-25 04:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: peter nordlund (qnorpet) Assigned to: Nobody/Anonymous (nobody) Summary: Command history doesn't work on Mandrake Initial Comment: When I build version 2.2b1 and 2.2a4 on my Linux Mandrake 8.1 pc the command history doesn't work. The python that is included in Mandrake 8.1 works ok in that regard. I can't use arrow keys or ctrl-p to go backwards. No ctrl- or arrow key-strokes seems to work. When I build 2.2a4 for Solaris it works ok. I have Swedish keyboards, in case that could matter. Best regards, Peter Nordlund ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 From noreply@sourceforge.net Thu Oct 25 12:22:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 04:22:17 -0700 Subject: [Python-bugs-list] [ python-Bugs-474836 ] Tix not included in windows distribution Message-ID: Bugs item #474836, was opened at 2001-10-25 04:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474836&group_id=5470 Category: Tkinter Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Tix not included in windows distribution Initial Comment: Although there is a Tix.py available, there is no Tix support in the precomiled Python-distribution for windows. So import Tix works fine, but root = Tix.Tk() results in TclError: package not found. It is possible to circumvent this problem by installing a regular Tcl/Tk distribution (e.g. in c:\programme\tcl) and installing Tix in the regular Tcl-path (i.e. tcl\lib). Mathias ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474836&group_id=5470 From noreply@sourceforge.net Thu Oct 25 13:43:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 05:43:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-474737 ] Problem overriding int.__init__() Message-ID: Bugs item #474737, was opened at 2001-10-24 21:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474737&group_id=5470 Category: Type/class unification >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Mark J (average) Assigned to: Guido van Rossum (gvanrossum) Summary: Problem overriding int.__init__() Initial Comment: Given: class I(int): def __init__(self, init_value, extra_param="test"): print extra_param int.__init__(self, init_value) Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 >>> i = I(1) #fine test #yes, new constructor used >>> i 1 #value set correctly >>> i = I(2, "oops") Traceback (most recent call last): File "", line 1, in ? TypeError: an integer is required >>> i = I(3, dummy="oops") Traceback (most recent call last): File "", line 1, in ? TypeError: dummy is an invalid keyword argument for this function >>> Thanks, Mark ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 05:43 Message: Logged In: YES user_id=6380 The constructor signature is shared by __new__ and __init__; you have to override both to accept the new argument. Add def __new__(cls, i=0, *args): return int.__new__(cls, i) (I promise we'll document all this in enough detail eventually!) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474737&group_id=5470 From noreply@sourceforge.net Thu Oct 25 13:47:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 05:47:54 -0700 Subject: [Python-bugs-list] [ python-Bugs-474238 ] 2.2b1 src rpm doesn't include Lib/email/ Message-ID: Bugs item #474238, was opened at 2001-10-23 13:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474238&group_id=5470 Category: Installation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) Summary: 2.2b1 src rpm doesn't include Lib/email/ Initial Comment: Using the python2.2-2.2b1-1.src.rpm to create binary rpms, I found that the installed RPM did not contain the new (post-2.2a3, anyway) Lib/email/ module. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 05:47 Message: Logged In: YES user_id=6380 Check out the site now -- the new RPMs are there! ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2001-10-24 20:23 Message: Logged In: YES user_id=81797 Doh! I've also added the "compiler" module. 2.2b1-2 RPMs are on ftp.tummy.com:/pub/tummy/RPMS/SRPMS and RPMS/binaries-KRUD-7.2-i386 -- Guido should be moving them to the python.org site when he gets a chance. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474238&group_id=5470 From noreply@sourceforge.net Thu Oct 25 13:56:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 05:56:19 -0700 Subject: [Python-bugs-list] [ python-Bugs-474831 ] Command history doesn't work on Mandrake Message-ID: Bugs item #474831, was opened at 2001-10-25 04:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 Category: Python Interpreter Core >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: peter nordlund (qnorpet) >Assigned to: Guido van Rossum (gvanrossum) Summary: Command history doesn't work on Mandrake Initial Comment: When I build version 2.2b1 and 2.2a4 on my Linux Mandrake 8.1 pc the command history doesn't work. The python that is included in Mandrake 8.1 works ok in that regard. I can't use arrow keys or ctrl-p to go backwards. No ctrl- or arrow key-strokes seems to work. When I build 2.2a4 for Solaris it works ok. I have Swedish keyboards, in case that could matter. Best regards, Peter Nordlund ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 05:56 Message: Logged In: YES user_id=6380 You probably don't have the GNU readline library installed in a place where Python looks for it. Download its source from a GNU site and build and install it. (If you *do* have GNU readline installed, it would be interesting to find out why setup.py doesn't find it and/or why it doesn't build the readline extension module.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 From noreply@sourceforge.net Thu Oct 25 16:15:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 08:15:05 -0700 Subject: [Python-bugs-list] [ python-Bugs-474831 ] Command history doesn't work on Mandrake Message-ID: Bugs item #474831, was opened at 2001-10-25 04:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 Category: Python Interpreter Core Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: peter nordlund (qnorpet) Assigned to: Guido van Rossum (gvanrossum) Summary: Command history doesn't work on Mandrake Initial Comment: When I build version 2.2b1 and 2.2a4 on my Linux Mandrake 8.1 pc the command history doesn't work. The python that is included in Mandrake 8.1 works ok in that regard. I can't use arrow keys or ctrl-p to go backwards. No ctrl- or arrow key-strokes seems to work. When I build 2.2a4 for Solaris it works ok. I have Swedish keyboards, in case that could matter. Best regards, Peter Nordlund ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-25 08:15 Message: Logged In: YES user_id=44345 Actually, you may well have the readline RPM installed, but not the readline-devel RPM. To compile programs that include support for a given package, you need to install the "-devel" RPM, which includes the various .h files needed to compile the program. Python's setup.py script is probably looking for, but not finding /usr/include/readline/readline.h. Skip ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 05:56 Message: Logged In: YES user_id=6380 You probably don't have the GNU readline library installed in a place where Python looks for it. Download its source from a GNU site and build and install it. (If you *do* have GNU readline installed, it would be interesting to find out why setup.py doesn't find it and/or why it doesn't build the readline extension module.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 From noreply@sourceforge.net Thu Oct 25 16:48:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 08:48:56 -0700 Subject: [Python-bugs-list] [ python-Bugs-474913 ] 2.2b1 won't start in remote shell Message-ID: Bugs item #474913, was opened at 2001-10-25 08:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474913&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jack Jansen (jackjansen) Summary: 2.2b1 won't start in remote shell Initial Comment: Seems possibly related to #466907 This does not seem to occur with 2.2a4, but does with 2.2b1. I tried this on three different Macs, each/all OS X 10.1 (relatively clean installs) with the 10-19-2001 security (and MS IE) patch. All are G4 machines w/ lots of memory. 2.2b1seems to build (configure --with-suffix=.exe), install, and run just fine on all machines when working from a local terminal window. However I see strange behavior when trying to run remotely. 1. If I connect to one of the machines remotely using ssh (e.g. ssh to my dual G4, which is faster than my home machine) and try and start python, I see the following errors: [mac-welch:~/proj/python] welch% python kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorInvalidConnection : CGSGetEventPort: Invalid connection [mac-welch:~/proj/python] welch% 2. If I su to root in the same remote session, python starts just fine! If I revert to 2.2a4 it seems to work fine. Thanks (in advance) folks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474913&group_id=5470 From noreply@sourceforge.net Thu Oct 25 16:52:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 08:52:33 -0700 Subject: [Python-bugs-list] [ python-Bugs-474919 ] 2.2b1 cannot start remotely (ssh) Message-ID: Bugs item #474919, was opened at 2001-10-25 08:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Greg Welch (gregwelch) Assigned to: Jack Jansen (jackjansen) Summary: 2.2b1 cannot start remotely (ssh) Initial Comment: Seems possibly related to #466907 This does not seem to occur with 2.2a4, but does with 2.2b1. I tried this on three different Macs, each/all OS X 10.1 (relatively clean installs) with the 10-19-2001 security (and MS IE) patch. All are G4 machines w/ lots of memory. 2.2b1seems to build (configure --with-suffix=.exe), install, and run just fine on all machines when working from a local terminal window. However I see strange behavior when trying to run remotely. 1. If I connect to one of the machines remotely using ssh (e.g. ssh to my dual G4, which is faster than my home machine) and try and start python, I see the following errors: [mac-welch:~/proj/python] welch% python kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorInvalidConnection : CGSGetEventPort: Invalid connection [mac-welch:~/proj/python] welch% 2. If I su to root in the same remote session, python starts just fine! If I revert to 2.2a4 it seems to work fine. Thanks (in advance) folks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 From noreply@sourceforge.net Thu Oct 25 19:22:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 11:22:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-474986 ] minidom doesn't implement importNode() Message-ID: Bugs item #474986, was opened at 2001-10-25 11:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474986&group_id=5470 Category: XML Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Fred L. Drake, Jr. (fdrake) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: minidom doesn't implement importNode() Initial Comment: The Document.importNode() method has not been implemented; this would be very useful for many applications. See: http://mail.python.org/pipermail/xml-sig/2001-October/006252.html ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474986&group_id=5470 From noreply@sourceforge.net Thu Oct 25 19:26:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 11:26:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-474836 ] Tix not included in windows distribution Message-ID: Bugs item #474836, was opened at 2001-10-25 04:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474836&group_id=5470 Category: Tkinter Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Tix not included in windows distribution Initial Comment: Although there is a Tix.py available, there is no Tix support in the precomiled Python-distribution for windows. So import Tix works fine, but root = Tix.Tk() results in TclError: package not found. It is possible to circumvent this problem by installing a regular Tcl/Tk distribution (e.g. in c:\programme\tcl) and installing Tix in the regular Tcl-path (i.e. tcl\lib). Mathias ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-25 11:26 Message: Logged In: YES user_id=31435 I don't know anything about Tix, so if somebody wants this in the Windows installer, they're going to have to explain exactly (by which I mean exactly <0.5 wink>) what's needed. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474836&group_id=5470 From noreply@sourceforge.net Thu Oct 25 19:45:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 11:45:41 -0700 Subject: [Python-bugs-list] [ python-Bugs-474992 ] python version benchmark Message-ID: Bugs item #474992, was opened at 2001-10-25 11:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474992&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Nobody/Anonymous (nobody) Summary: python version benchmark Initial Comment: This is not a bug per se, but following instructions by Tim Peters the One :), I'm submitting here the benchmark script. I'm getting currently the following final output (bench.txt): version bench1 nogc bench2 1.5.2 0 % 2.0 48 % 33 % 21 % 2.1.1 53 % 35 % 25 % 2.2 51 % 35 % 23 % where: bench1 = original python 1.5.2 script nogc = bench1 when disabling gc bench2 = bench1 script 'rewritten' for python 2.0 and latter (string methods, sre -> pre...) These numbers are high... Frederic Giacometti ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474992&group_id=5470 From noreply@sourceforge.net Thu Oct 25 21:19:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 13:19:11 -0700 Subject: [Python-bugs-list] [ python-Bugs-474538 ] Memory (reference) leak in poller.regis Message-ID: Bugs item #474538, was opened at 2001-10-24 10:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474538&group_id=5470 Category: Extension Modules Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Dave Brueck (dbrueck) >Assigned to: Guido van Rossum (gvanrossum) Summary: Memory (reference) leak in poller.regis Initial Comment: A reference leak occurs if you call the register (and possibly unregister) functions of poller objects (obtained by calling select.poll). The problem does not manifest itself on file descriptors with a value <= 100, so it may be that register() is leaking references to integer objects and those less than 101 are simply leaked references to cached integer objects. I'm running Python 2.1.1 on Linux Mandrake 8.1 and Linux 2.1b1 on RedHat 7.0. This code reproduces the problem: import select poller = select.poll() while 1: poller.register(101, 1) (now go peek at the memory usage for this process - it skyrockets) If you replace 101 above with 100 or less, the memory usage remains flat. I found this problem because my program routinely has to handle more than 100 socket connections. I believe the reference to the flag parameter is also being leaked, but you don't usually see the problem because flag values are small and are therefore references to cached integers. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 13:19 Message: Logged In: YES user_id=6380 Thanks! Fixed in CVS, will be in 2.2b2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474538&group_id=5470 From noreply@sourceforge.net Thu Oct 25 21:36:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 13:36:43 -0700 Subject: [Python-bugs-list] [ python-Bugs-475049 ] Slight documentation error in XML DOM Message-ID: Bugs item #475049, was opened at 2001-10-25 13:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475049&group_id=5470 Category: Documentation Group: Not a Bug Status: Open Resolution: None Priority: 5 Submitted By: Alan Meyer (vrmeyer) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Slight documentation error in XML DOM Initial Comment: This note applies to the documentation for xml.dom. I'm using ActiveState's build #15 of Python 2.1, with the Winhelp documentation module. In the documentation for section 13.5.2.2 Node Objects, it says that "attributes" identifies a "NamedNodeList", whereas "13.5.2.8 NamedNodeMap Objects" uses the term NamedNodeMap. I presume these are the same thing and one of the names is in error, though it's possible I'm just confused (happens often.) Thanks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475049&group_id=5470 From noreply@sourceforge.net Thu Oct 25 21:45:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 13:45:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-475049 ] Slight documentation error in XML DOM Message-ID: Bugs item #475049, was opened at 2001-10-25 13:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475049&group_id=5470 Category: Documentation Group: Not a Bug >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Alan Meyer (vrmeyer) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Slight documentation error in XML DOM Initial Comment: This note applies to the documentation for xml.dom. I'm using ActiveState's build #15 of Python 2.1, with the Winhelp documentation module. In the documentation for section 13.5.2.2 Node Objects, it says that "attributes" identifies a "NamedNodeList", whereas "13.5.2.8 NamedNodeMap Objects" uses the term NamedNodeMap. I presume these are the same thing and one of the names is in error, though it's possible I'm just confused (happens often.) Thanks. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-25 13:45 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/xmldom.tex revision 1.16 and 1.13.2.1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475049&group_id=5470 From noreply@sourceforge.net Thu Oct 25 22:41:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 14:41:57 -0700 Subject: [Python-bugs-list] [ python-Bugs-474583 ] non-standard types in mimetypes.py Message-ID: Bugs item #474583, was opened at 2001-10-24 11:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474583&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: non-standard types in mimetypes.py Initial Comment: In the spirit of "be liberal in what you accept", I think mimetypes should have an option which searches non-standard but common types. E.g. I just received an email message which had a content type of image/jpg and I wanted to store that in a file with the proper extension. But mimetypes couldn't give that to me because image/jpeg is the official type. Attached is a patch to add an optional "strict" flag to guess_type() and guess_extension(). strict defaults to 1 to keep the current behavior. Set it to 0 and some additional types are searched. I added back the non-standard types that Martin removed in revision 1.19 of the file. This patch also contains a small improvement to the command line options when this is run as a script. I can split the patch into two parts if necessary, but I don't think it is. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-25 14:41 Message: Logged In: YES user_id=12800 Thanks, I'll check this in, as well as a unit test and documentation updates. I've also integrated the rejected types from patch #438790. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-25 01:46 Message: Logged In: YES user_id=21627 Sounds good to me, assuming it is also accompanied by a documentation change. You may also consider adding the types I rejected from #438790, in particular image/pict, text/xsl and text/xul (not sure whether any others were rejected). Back to Barry for checkin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474583&group_id=5470 From noreply@sourceforge.net Fri Oct 26 04:18:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 20:18:51 -0700 Subject: [Python-bugs-list] [ python-Bugs-475166 ] Confusing code (typos?) in CGIHTTPServer Message-ID: Bugs item #475166, was opened at 2001-10-25 20:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475166&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Noah Spurrier (noah) Assigned to: Nobody/Anonymous (nobody) Summary: Confusing code (typos?) in CGIHTTPServer Initial Comment: I found the comment on line 224 of CGIHTTPServer.py to be confusing: # On Windows, use python.exe, not python.exe Also, since do_POST() does not call send_head(), then why do you need to handle the following condition in send_head() line 63???: if self.is_cgi(): return self.run_cgi() Note that do_POST() already calls self.run_cgi() on line 57, so why would send_head() call it? I'm sure I'm just being a dweeb and I'm missing something, but if not then I hope this clears up the code. I'm still confused why my custom do_POST() is not working... Yours, ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475166&group_id=5470 From noreply@sourceforge.net Fri Oct 26 04:40:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 25 Oct 2001 20:40:18 -0700 Subject: [Python-bugs-list] [ python-Bugs-475166 ] Confusing code (typos?) in CGIHTTPServer Message-ID: Bugs item #475166, was opened at 2001-10-25 20:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475166&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Noah Spurrier (noah) Assigned to: Nobody/Anonymous (nobody) Summary: Confusing code (typos?) in CGIHTTPServer Initial Comment: I found the comment on line 224 of CGIHTTPServer.py to be confusing: # On Windows, use python.exe, not python.exe Also, since do_POST() does not call send_head(), then why do you need to handle the following condition in send_head() line 63???: if self.is_cgi(): return self.run_cgi() Note that do_POST() already calls self.run_cgi() on line 57, so why would send_head() call it? I'm sure I'm just being a dweeb and I'm missing something, but if not then I hope this clears up the code. I'm still confused why my custom do_POST() is not working... Yours, ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 20:40 Message: Logged In: YES user_id=6380 The comment should have said "python.exe, not pythonw.exe". I've fixed this in CVS, as well as a typo on the next line. The is_cgi() check in send_head() is for when send_head() is called from do_GET() -- an alternative way to invoke CGI scripts. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475166&group_id=5470 From noreply@sourceforge.net Fri Oct 26 11:44:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 03:44:02 -0700 Subject: [Python-bugs-list] [ python-Bugs-475253 ] [OSX]ConfigPython fails 4 non-admin user Message-ID: Bugs item #475253, was opened at 2001-10-26 03:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475253&group_id=5470 Category: Macintosh Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Ryan Wilcox (ryanwilcox) Assigned to: Jack Jansen (jackjansen) Summary: [OSX]ConfigPython fails 4 non-admin user Initial Comment: ConfigurePythonCarbon fails on users that don't have admin privs. (It tries towrite PythonCore to /Library/ and fails, for obvious reasons). Having this fail means the IDE fails to build, and, well... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475253&group_id=5470 From noreply@sourceforge.net Fri Oct 26 11:52:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 03:52:38 -0700 Subject: [Python-bugs-list] [ python-Bugs-474831 ] Command history doesn't work on Mandrake Message-ID: Bugs item #474831, was opened at 2001-10-25 04:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 Category: Python Interpreter Core Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: peter nordlund (qnorpet) Assigned to: Guido van Rossum (gvanrossum) Summary: Command history doesn't work on Mandrake Initial Comment: When I build version 2.2b1 and 2.2a4 on my Linux Mandrake 8.1 pc the command history doesn't work. The python that is included in Mandrake 8.1 works ok in that regard. I can't use arrow keys or ctrl-p to go backwards. No ctrl- or arrow key-strokes seems to work. When I build 2.2a4 for Solaris it works ok. I have Swedish keyboards, in case that could matter. Best regards, Peter Nordlund ---------------------------------------------------------------------- >Comment By: peter nordlund (qnorpet) Date: 2001-10-26 03:52 Message: Logged In: YES user_id=358954 Ok, before I saw montanaros suggestion I built readline4.2 (I think it was) with --with-termcap, and edited Setup as follows: readline readline.c -I/work1/pkg/utils/readline/4.2/include -L/work1/pkg/utils/readline/4.2/lib -L/lib -lreadline - ltermcap I think I also had mo make a symlink from /lib/libtermcap.so.2.0.8 to /lib/termcap.so I could be wrong in some details, since I'm not sitting right at the Linux-box right now. Then it worked as it should!!! I will try monataros suggestion and will report back after the weekend. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-25 08:15 Message: Logged In: YES user_id=44345 Actually, you may well have the readline RPM installed, but not the readline-devel RPM. To compile programs that include support for a given package, you need to install the "-devel" RPM, which includes the various .h files needed to compile the program. Python's setup.py script is probably looking for, but not finding /usr/include/readline/readline.h. Skip ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 05:56 Message: Logged In: YES user_id=6380 You probably don't have the GNU readline library installed in a place where Python looks for it. Download its source from a GNU site and build and install it. (If you *do* have GNU readline installed, it would be interesting to find out why setup.py doesn't find it and/or why it doesn't build the readline extension module.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 From noreply@sourceforge.net Fri Oct 26 13:37:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 05:37:50 -0700 Subject: [Python-bugs-list] [ python-Bugs-474919 ] 2.2b1 cannot start remotely (ssh) Message-ID: Bugs item #474919, was opened at 2001-10-25 08:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 Category: Macintosh Group: Python 2.2 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Greg Welch (gregwelch) Assigned to: Jack Jansen (jackjansen) Summary: 2.2b1 cannot start remotely (ssh) Initial Comment: Seems possibly related to #466907 This does not seem to occur with 2.2a4, but does with 2.2b1. I tried this on three different Macs, each/all OS X 10.1 (relatively clean installs) with the 10-19-2001 security (and MS IE) patch. All are G4 machines w/ lots of memory. 2.2b1seems to build (configure --with-suffix=.exe), install, and run just fine on all machines when working from a local terminal window. However I see strange behavior when trying to run remotely. 1. If I connect to one of the machines remotely using ssh (e.g. ssh to my dual G4, which is faster than my home machine) and try and start python, I see the following errors: [mac-welch:~/proj/python] welch% python kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorInvalidConnection : CGSGetEventPort: Invalid connection [mac-welch:~/proj/python] welch% 2. If I su to root in the same remote session, python starts just fine! If I revert to 2.2a4 it seems to work fine. Thanks (in advance) folks. ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-10-26 05:37 Message: Logged In: YES user_id=45365 This is indeed a duplicate of #466907. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 From noreply@sourceforge.net Fri Oct 26 13:38:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 05:38:58 -0700 Subject: [Python-bugs-list] [ python-Bugs-474913 ] 2.2b1 won't start in remote shell Message-ID: Bugs item #474913, was opened at 2001-10-25 08:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474913&group_id=5470 Category: Macintosh Group: Python 2.2 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jack Jansen (jackjansen) Summary: 2.2b1 won't start in remote shell Initial Comment: Seems possibly related to #466907 This does not seem to occur with 2.2a4, but does with 2.2b1. I tried this on three different Macs, each/all OS X 10.1 (relatively clean installs) with the 10-19-2001 security (and MS IE) patch. All are G4 machines w/ lots of memory. 2.2b1seems to build (configure --with-suffix=.exe), install, and run just fine on all machines when working from a local terminal window. However I see strange behavior when trying to run remotely. 1. If I connect to one of the machines remotely using ssh (e.g. ssh to my dual G4, which is faster than my home machine) and try and start python, I see the following errors: [mac-welch:~/proj/python] welch% python kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorInvalidConnection : CGSGetEventPort: Invalid connection [mac-welch:~/proj/python] welch% 2. If I su to root in the same remote session, python starts just fine! If I revert to 2.2a4 it seems to work fine. Thanks (in advance) folks. ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-10-26 05:38 Message: Logged In: YES user_id=45365 Another duplicate of #466907 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474913&group_id=5470 From noreply@sourceforge.net Fri Oct 26 13:49:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 05:49:22 -0700 Subject: [Python-bugs-list] [ python-Bugs-466907 ] OSX Python cannot run in console Message-ID: Bugs item #466907, was opened at 2001-10-01 10:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Wight (schwa) Assigned to: Jack Jansen (jackjansen) Summary: OSX Python cannot run in console Initial Comment: OS: Mac OS X 10.1 HW: Verified on G4/733 & PowerBook G4/400Version: 'Unix' Python 2.2a4 (also seen on 2.2a3) Configured: --with-suffix=.exe --without-toolbox-glue Python cannot run on a Mac OS X machine when the user is logged into console mode (using ">console" as login name). Download .tar.gz, configure python as shown, make, make install. Log out of Mac OS X. Log in as console user: use ">console" as login name. Run python interactively from the shell... Python will fail to run and emit these errors: KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorInvalidConnection: CGSGetEventPort: Invalid connection I believe Python is trying to connect to the CoreGraphics server. Why? I don't know (esp. as I've NOT compiled the toolbox glue). Perhaps Python is linking to a Carbon library and the library is automatically trying to open a connection. My makefile skills are not good enough to enable me to check. Python compiled on OS X without the toolbox glue should essentially be identical to any other 'unix' app compiled for Mac OS X. Doesn't seem to be that way. ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-10-26 05:49 Message: Logged In: YES user_id=45365 Just a quick update so people know I haven't forgotten this bug. It turns out that Apple changed the Carbon.framework initialization: it used to be lazy, initializing carbon portions as needed, but since 10.1 a lot more is initialized up front, and therefore the window server is contacted. Therefore, as of 10.1, if you are not a windowing program you cannot link against Carbon.framework anymore but must link against some of the subordinate frameworks. This was always true theoretically (it seems), but now it has suddenly become practically true as well. But: it seems more people are bitten by this, so Apple suggested they may revert on this decision. And on the other hand I'm going to have a look at not linking with Carbon.framework but with CoreServices and/or ApplicationServices in stead. ---------------------------------------------------------------------- Comment By: Jonathan Wight (schwa) Date: 2001-10-01 15:17 Message: Logged In: YES user_id=29309 OK... Another Follow Up. Checked the configure.in file and there is no --without-toolbox-glue switch for configure. There is a --disable-toolbox-glue and --enable-toolbox-glue. The line in the README that states you should use --without-toolbox-glue is incorrect. This is a bug in the documentation! (easy to fix though). I've checked the newly built executable (using --disable-toolbox-glue) with 'otool -l' and the linkage dependencies are gone. This is good. Unfortunately when the 'make install' stage of the build fails with: skipping 'gestalt' extension (up-to-date) dyld: ./python.exe Undefined symbols: _Gestalt _PyMac_Error make: *** [sharedmods] Error 67 ---------------------------------------------------------------------- Comment By: Jonathan Wight (schwa) Date: 2001-10-01 14:33 Message: Logged In: YES user_id=29309 FYI. Running 'otool -l' on the Python executable shows that it is being linked to both: /System/Library/Frameworks/Carbon.framework and /System/Library/Frameworks/Foundation.framework Carbon.framework links in /System/Library/Frameworks/ApplicationServices.framework/ which contains the CoreGraphics framework. I'm assuming that causes CoreGraphics to be initialized (guessing). Why does a plain 'unix' build of Python link to both Carbon and Foundation? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 From noreply@sourceforge.net Fri Oct 26 15:24:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 07:24:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-473753 ] Plug sys.exc_info() leaks Message-ID: Bugs item #473753, was opened at 2001-10-22 12:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473753&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Plug sys.exc_info() leaks Initial Comment: As the Library manual has warned for years, this kind of program leaks bigtime: import sys def f(): . try: . 1 / 0 . except ZeroDivisionError: . err = sys.exc_info() while 1: . f() because err[-1] is a traceback object, from which f's frame can be reached, from which we can get back to 'err'. 2.2 added frames to gc (thanks to NeilS), but this example still leaks. That's at best an embarrassing trap, and it just popped up again in Zope's testing framework. Best guess is that a cure amounts to adding traceback objects to gc too, although I haven't tried that so am not sure. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-26 07:24 Message: Logged In: YES user_id=31392 Fixed in rev 2.34 of traceback.c. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473753&group_id=5470 From noreply@sourceforge.net Fri Oct 26 16:08:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 08:08:39 -0700 Subject: [Python-bugs-list] [ python-Bugs-475325 ] Frozen packages broken on the Mac Message-ID: Bugs item #475325, was opened at 2001-10-26 08:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475325&group_id=5470 Category: Macintosh Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Jack Jansen (jackjansen) Summary: Frozen packages broken on the Mac Initial Comment: Patch #416704 broke the freezing of packages on the Mac. The patch requires that submodules of frozen packages are also frozen, but disregards the fact that freezing is done completely different on the Mac. The code in import.c:find_module() now takes an early exit if we're importing from a package and find_frozen() returns false, so the code that locates MacPython PYC and PYD resource modules never gets executed. I need to investigate whether the correct solution is to duplicate that code after find_module() returns (but it's rather a big lump) or not take the exit in MacPython (but that would leave MacPython open to the problems the patch tries to address). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475325&group_id=5470 From noreply@sourceforge.net Fri Oct 26 16:13:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 08:13:40 -0700 Subject: [Python-bugs-list] [ python-Bugs-475327 ] type() produces incorrect error msg Message-ID: Bugs item #475327, was opened at 2001-10-26 08:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475327&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: type() produces incorrect error msg Initial Comment: type() takes 1 or 3 arguments, but the error msg is wrong: >>> type(1) >>> type('1', (), {}) >>> type('1', ()) Traceback (most recent call last): File "", line 1, in ? TypeError: type() takes exactly 3 arguments (2 given) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475327&group_id=5470 From noreply@sourceforge.net Sat Oct 27 00:20:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 26 Oct 2001 16:20:42 -0700 Subject: [Python-bugs-list] [ python-Bugs-473150 ] configure weaknesses on HP-UX Message-ID: Bugs item #473150, was opened at 2001-10-20 06:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 Category: Build Group: Python 2.2 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Michael Piotrowski (mxp) Assigned to: Guido van Rossum (gvanrossum) Summary: configure weaknesses on HP-UX Initial Comment: 1. configure doesn't handle HP-UX release numbers (e.g., B.11.00), resulting in MACHDEP = "hpuxB". 2. After checking for wchar.h, configure doesn't include it when checking the size of wchar_t. (Python 2.2b1 on HP-UX 11.00) ---------------------------------------------------------------------- >Comment By: Michael Piotrowski (mxp) Date: 2001-10-26 16:20 Message: Logged In: YES user_id=354556 Great, CVS version builds out of the box on HP-UX 11. Will open separate bugs for remaining problems, when I find time. I listed them here as just a summary, since you asked ;-) HP-UX 10.20 is still relevant, will have to do some testing re: configure options. Problem with threads on 10.20 seems to be that they're DCE threads, i.e., a draft version of pthreads. OK, that's my last word on this closed report... ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-24 10:22 Message: Logged In: YES user_id=21627 On the curses problems, please check out the latest CVS again; it is supposed to be fixed on HP/UX 11. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:01 Message: Logged In: YES user_id=6380 Thanks for the list. Can you submit separate patches or bug reports for the things you know need work? That seems easier than holding this conversation in an unrelated closed bug report. :-) OPT is *not* the right place for a -I option, but there's currently no "right" place. I've seen them added to "CC", which makes more sense to me. Is HP-UX 10.20 still relevant? If so, it might be nice if the configure script defaulted to --without-threads. (Is the problem that -lcma has pthread_create but in fact doesn't support the rest of the PTHREAD API?) ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:50 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_socket and test_time fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-21 07:48 Message: Logged In: YES user_id=354556 Yes, it passes the thread tests. Other things that I'm aware of (if not noted otherwise, I haven't done further investigations yet): On HP-UX 10.20 and 11: - Building the curses module (Bug #432497): I've got a patch for configure.in, but I'm not sure if OPT is the right place to prepend a -I option--shall I attach it to bug #432497? - test_locale is skipped: This is only because HP-UX uses en_US.iso88591 instead of en_US patch for test_locale.py attached. - configure detects large-file support, but test_largefile is skipped On HP-UX 11, when building a 64-bit (LP64) executable (don't know if anybody wants to do this): - curses can't be built. This *might* be because there's no 64-bit libtermcap - test_??? and test_??? fail. On HP-UX 10.20: - It is not clear which threads work. DCE threads seem not to work (tested), GNU pth should work (not yet tested). However, configure detects pthread_create in -lcma, so that the build fails if --without-threads is not specified (this is acceptable, I think, if it's documented). - Even if building --without-threads, 7 tests failed: test_fpformat test_mimetypes test_pyclbr test_socket test_sundry test_urllib test_urllib2. (I'm currently doing all builds with HP C) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 18:02 Message: Logged In: YES user_id=6380 Do the tests for threads pass? (test_thread.py, test_threading.py, test_threaded_import.py, test_threadedtempfile.py) That would be great news! If there are any other problems on HP-UX, it would be great if you could help out. Do you know what those problems are? ---------------------------------------------------------------------- Comment By: Michael Piotrowski (mxp) Date: 2001-10-20 08:41 Message: Logged In: YES user_id=354556 Thist patch has no effect on threads. However, threads do work (i.e., are recognized by configure) on HP-UX 11 since 2.2b1 (I just wanted to submit a patch to the 2.2a4 configure.in when 2.2b1 was released). I am willing to support you regarding HP-UX, if necessary (there still seem to be some open issues). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-20 07:23 Message: Logged In: YES user_id=6380 Thanks! Checked into CVS. With this patch, do threads work on your system? (We've had long-standing complaints that Python can't be built with threads on HP-UX 11, but these are hard to verify for us. It would be great if we could say with confidence that this is fixed in 2.2b2.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473150&group_id=5470 From noreply@sourceforge.net Sat Oct 27 14:31:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 27 Oct 2001 06:31:51 -0700 Subject: [Python-bugs-list] [ python-Bugs-474919 ] 2.2b1 cannot start remotely (ssh) Message-ID: Bugs item #474919, was opened at 2001-10-25 08:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Closed Resolution: Duplicate Priority: 5 Submitted By: Greg Welch (gregwelch) Assigned to: Jack Jansen (jackjansen) Summary: 2.2b1 cannot start remotely (ssh) Initial Comment: Seems possibly related to #466907 This does not seem to occur with 2.2a4, but does with 2.2b1. I tried this on three different Macs, each/all OS X 10.1 (relatively clean installs) with the 10-19-2001 security (and MS IE) patch. All are G4 machines w/ lots of memory. 2.2b1seems to build (configure --with-suffix=.exe), install, and run just fine on all machines when working from a local terminal window. However I see strange behavior when trying to run remotely. 1. If I connect to one of the machines remotely using ssh (e.g. ssh to my dual G4, which is faster than my home machine) and try and start python, I see the following errors: [mac-welch:~/proj/python] welch% python kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorInvalidConnection : CGSGetEventPort: Invalid connection [mac-welch:~/proj/python] welch% 2. If I su to root in the same remote session, python starts just fine! If I revert to 2.2a4 it seems to work fine. Thanks (in advance) folks. ---------------------------------------------------------------------- >Comment By: Greg Welch (gregwelch) Date: 2001-10-27 06:31 Message: Logged In: YES user_id=324637 I was wrong - apparently this happens w/ 2.2a4 also. There must be some other variable/condition I am missing. While perhaps a "duplicate" of #466907 (same root cause perhaps) running python remotely is probably more likely than running in console mode. I.e. perhaps a slightly higher priority. ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-10-26 05:37 Message: Logged In: YES user_id=45365 This is indeed a duplicate of #466907. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 From noreply@sourceforge.net Sat Oct 27 14:43:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 27 Oct 2001 06:43:09 -0700 Subject: [Python-bugs-list] [ python-Bugs-475538 ] os.system() and cgi Message-ID: Bugs item #475538, was opened at 2001-10-27 06:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475538&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: M. Ibaraki (redmaple) Assigned to: Nobody/Anonymous (nobody) Summary: os.system() and cgi Initial Comment: This script (test.py) works fine and creates a file 'outputfile' when I run this in test mode (python test.py). --- test.py ---- #!/usr/bin/python import cgi, os print "Content-type: text/html\r\n" print "" print "" ret = os.system('/usr/X11R6/bin/xmgrace -help > outputfile') # ret = os.system('/bin/ls > outputfile') print "

ret = ", ret ---------------- It shows: Content-type: text/html

ret = 0 However, when this is executed as a cgi it shows: ret = 256 and no 'outputfile' file is created. Permission of /usr/X11R6/bin/xmgrace is: -rwxr-xr-x 1 root root 450 May 20 18:48 /usr/X11R6/bin/xmgrace xmgrace is a 2D plotting tool (http://plasma-gate.weizmann.ac.il/Grace/) I also tried '/bin/ls > outputfile' instead of '/usr/X11R6/bin/xmgrace -help > outputfile'. This 'ls' command is executed and it creates 'outputfile' in both cgi and test modes. Permission of /bin/ls is: -rwxr-xr-x 1 root root 48316 Sep 15 12:54 /bin/ls Any comments would be greatly appreciated. M. Ibaraki ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475538&group_id=5470 From noreply@sourceforge.net Sat Oct 27 16:53:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 27 Oct 2001 08:53:15 -0700 Subject: [Python-bugs-list] [ python-Bugs-475538 ] os.system() and cgi Message-ID: Bugs item #475538, was opened at 2001-10-27 06:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475538&group_id=5470 Category: Extension Modules Group: Python 2.1.1 >Status: Closed Resolution: None Priority: 5 Submitted By: M. Ibaraki (redmaple) >Assigned to: Guido van Rossum (gvanrossum) Summary: os.system() and cgi Initial Comment: This script (test.py) works fine and creates a file 'outputfile' when I run this in test mode (python test.py). --- test.py ---- #!/usr/bin/python import cgi, os print "Content-type: text/html\r\n" print "" print "" ret = os.system('/usr/X11R6/bin/xmgrace -help > outputfile') # ret = os.system('/bin/ls > outputfile') print "

ret = ", ret ---------------- It shows: Content-type: text/html

ret = 0 However, when this is executed as a cgi it shows: ret = 256 and no 'outputfile' file is created. Permission of /usr/X11R6/bin/xmgrace is: -rwxr-xr-x 1 root root 450 May 20 18:48 /usr/X11R6/bin/xmgrace xmgrace is a 2D plotting tool (http://plasma-gate.weizmann.ac.il/Grace/) I also tried '/bin/ls > outputfile' instead of '/usr/X11R6/bin/xmgrace -help > outputfile'. This 'ls' command is executed and it creates 'outputfile' in both cgi and test modes. Permission of /bin/ls is: -rwxr-xr-x 1 root root 48316 Sep 15 12:54 /bin/ls Any comments would be greatly appreciated. M. Ibaraki ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-27 08:53 Message: Logged In: YES user_id=6380 I can't help you with this. This really doesn't look like a Python bug; more some kind of configuration problem. Try posting on comp.lang.python. If you hadn't said that outputfile is not created, I would've guessed that xmgrace, being an X11 tool, doesn't like not having a DISPLAY environment variable. But then it would still have created an (empty) outputfile. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475538&group_id=5470 From noreply@sourceforge.net Sat Oct 27 20:38:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 27 Oct 2001 12:38:37 -0700 Subject: [Python-bugs-list] [ python-Bugs-475327 ] type() produces incorrect error msg Message-ID: Bugs item #475327, was opened at 2001-10-26 08:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475327&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Tim Peters (tim_one) Summary: type() produces incorrect error msg Initial Comment: type() takes 1 or 3 arguments, but the error msg is wrong: >>> type(1) >>> type('1', (), {}) >>> type('1', ()) Traceback (most recent call last): File "", line 1, in ? TypeError: type() takes exactly 3 arguments (2 given) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-27 12:38 Message: Logged In: YES user_id=31435 The msg in the example is now TypeError: type() takes 1 or 3 arguments Include/object.h; new revision: 2.98 Objects/typeobject.c; new revision: 2.114 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475327&group_id=5470 From noreply@sourceforge.net Sun Oct 28 03:22:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 27 Oct 2001 20:22:48 -0700 Subject: [Python-bugs-list] [ python-Bugs-475678 ] y += x versus y = y+x surprise Message-ID: Bugs item #475678, was opened at 2001-10-27 20:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475678&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: y += x versus y = y+x surprise Initial Comment: This report from c.l.py. At a PythonLabs release mtg we agreed it was "a bug". Offhand, though, I don't recall which part was considered buggy . """ From: Dale Strickland-Clark Sent: Monday, October 22, 2001 6:20 PM To: python-list@python.org Subject: Augmented augmented assignment? Can someone explain this? It doesn't seem obvious to me but it's quite handy. I assume it's something to do with the in-place operation of augmented assignment but I can't seem to reproduce it using normal assignment. >>> x = 'spam' >>> y = [] >>> y = y + x Traceback (most recent call last): File "", line 1, in ? TypeError: can only concatenate list (not "string") to list >>> y += x >>> y ['s', 'p', 'a', 'm'] >>> """ Note that both behaviors were in 2.0, so a. This isn't due to "iterable object" generalization (it's due to list += mapping to list.extend). and b. There are backward-compatibility concerns. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475678&group_id=5470 From noreply@sourceforge.net Sun Oct 28 03:24:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 27 Oct 2001 20:24:08 -0700 Subject: [Python-bugs-list] [ python-Bugs-474992 ] python version benchmark Message-ID: Bugs item #474992, was opened at 2001-10-25 11:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474992&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Nobody/Anonymous (nobody) Summary: python version benchmark Initial Comment: This is not a bug per se, but following instructions by Tim Peters the One :), I'm submitting here the benchmark script. I'm getting currently the following final output (bench.txt): version bench1 nogc bench2 1.5.2 0 % 2.0 48 % 33 % 21 % 2.1.1 53 % 35 % 25 % 2.2 51 % 35 % 23 % where: bench1 = original python 1.5.2 script nogc = bench1 when disabling gc bench2 = bench1 script 'rewritten' for python 2.0 and latter (string methods, sre -> pre...) These numbers are high... Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-27 20:24 Message: Logged In: YES user_id=31435 Useful info from the effbot: """ fwiw, here's the timings I get on a slow windows box (using PythonWare builds, and time.clock() instead of os.times()): python 1.5: 2.01 python 2.0: 3.02 python 2.1: 3.00 python 2.2: 3.04 after playing some more, I ended up with this little benchmark: def dosomething(item): return [None, None, None] data = [None] * 100000 import time t1 = time.clock(); x = map(dosomething, data) t2 = time.clock() print t2 - t1 python 1.5: 1.02 seconds python 2.0: 1.95 seconds python 2.1: 1.85 seconds python 2.2: 1.90 seconds """ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474992&group_id=5470 From noreply@sourceforge.net Sun Oct 28 03:25:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 27 Oct 2001 20:25:46 -0700 Subject: [Python-bugs-list] [ python-Bugs-474992 ] python version benchmark Message-ID: Bugs item #474992, was opened at 2001-10-25 11:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474992&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Frederic Giacometti (giacometti) Assigned to: Nobody/Anonymous (nobody) Summary: python version benchmark Initial Comment: This is not a bug per se, but following instructions by Tim Peters the One :), I'm submitting here the benchmark script. I'm getting currently the following final output (bench.txt): version bench1 nogc bench2 1.5.2 0 % 2.0 48 % 33 % 21 % 2.1.1 53 % 35 % 25 % 2.2 51 % 35 % 23 % where: bench1 = original python 1.5.2 script nogc = bench1 when disabling gc bench2 = bench1 script 'rewritten' for python 2.0 and latter (string methods, sre -> pre...) These numbers are high... Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-27 20:25 Message: Logged In: YES user_id=31435 And more from effbot: """ doh. forgot to disable GC for 2.0 and later. with GC switched off, the difference is a bit smaller. python 1.5: 1.02 seconds python 2.0: 1.30 seconds python 2.1: 1.20 seconds python 2.2: 1.05 seconds now replace [None, None, None] with [item, item, item]: python 1.5: 0.58 seconds python 2.0: 0.85 seconds python 2.1: 0.69 seconds python 2.2: 0.71 seconds playing with other dosomething bodies (and with GC switched off), I've noticed that the following things has gotten slower from 1.5.2 to 2.2: - concatenating strings (item+item+item is 20% slower) - calling the float builtin using global lookup (20% slower) - list and tuple forming using locals (10-20% slower) """ ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-27 20:24 Message: Logged In: YES user_id=31435 Useful info from the effbot: """ fwiw, here's the timings I get on a slow windows box (using PythonWare builds, and time.clock() instead of os.times()): python 1.5: 2.01 python 2.0: 3.02 python 2.1: 3.00 python 2.2: 3.04 after playing some more, I ended up with this little benchmark: def dosomething(item): return [None, None, None] data = [None] * 100000 import time t1 = time.clock(); x = map(dosomething, data) t2 = time.clock() print t2 - t1 python 1.5: 1.02 seconds python 2.0: 1.95 seconds python 2.1: 1.85 seconds python 2.2: 1.90 seconds """ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474992&group_id=5470 From noreply@sourceforge.net Mon Oct 29 03:24:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 28 Oct 2001 19:24:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-475877 ] Mutable subtype instances are hashable Message-ID: Bugs item #475877, was opened at 2001-10-28 19:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: Mutable subtype instances are hashable Initial Comment: >>> class D(dictionary): pass ... >>> d = D() >>> hash(d) 7920544 >>> id(d) 7920544 >>> Ditto for instances of list subclasses: >>> class L(list): pass ... >>> x = L(range(100)) >>> hash(x) 7928992 >>> id(x) 7928992 >>> Among other nasties, this lets them get used as mutable dict keys. Reported by Mark J on c.l.py: """ From: Mark J Sent: Sunday, October 28, 2001 10:03 PM To: python-list@python.org Subject: Python 2.2b1 hashable dictionary bug? Since I haven't had a good hit rate at detecting bugs vs. features, I thought I'd post here before filing a bug report at SourceForge. Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class D(dictionary): pass ... >>> d = {} >>> d2 = D() >>> d[d2] = "dictionary used as key" >>> d {{}: 'dictionary used as key'} >>> d[D()]="now have two keys that look the same" >>> d {{}: 'now have two keys that look the same', {}: 'dictionary used as key'} >>> d2["key"] = "mutable key in d" >>> d {{}: 'now have two keys that look the same', {'key': 'mutable key in d'}: 'dictionary used as key'} >>> d[d] = "plain dictionary not allowed as key" Traceback (most recent call last): File "", line 1, in ? TypeError: unhashable type >>> d2[d2] = "subclassed dictionary allowed though" >>> d2 {{...}: 'subclassed dictionary allowed though', 'key': 'mutable keys'} >>> #whoa: what just happened: {...} ?? Interesting.... So is this a feature or a bug? """ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 From noreply@sourceforge.net Mon Oct 29 03:28:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 28 Oct 2001 19:28:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-475877 ] Mutable subtype instances are hashable Message-ID: Bugs item #475877, was opened at 2001-10-28 19:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: Mutable subtype instances are hashable Initial Comment: >>> class D(dictionary): pass ... >>> d = D() >>> hash(d) 7920544 >>> id(d) 7920544 >>> Ditto for instances of list subclasses: >>> class L(list): pass ... >>> x = L(range(100)) >>> hash(x) 7928992 >>> id(x) 7928992 >>> Among other nasties, this lets them get used as mutable dict keys. Reported by Mark J on c.l.py: """ From: Mark J Sent: Sunday, October 28, 2001 10:03 PM To: python-list@python.org Subject: Python 2.2b1 hashable dictionary bug? Since I haven't had a good hit rate at detecting bugs vs. features, I thought I'd post here before filing a bug report at SourceForge. Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class D(dictionary): pass ... >>> d = {} >>> d2 = D() >>> d[d2] = "dictionary used as key" >>> d {{}: 'dictionary used as key'} >>> d[D()]="now have two keys that look the same" >>> d {{}: 'now have two keys that look the same', {}: 'dictionary used as key'} >>> d2["key"] = "mutable key in d" >>> d {{}: 'now have two keys that look the same', {'key': 'mutable key in d'}: 'dictionary used as key'} >>> d[d] = "plain dictionary not allowed as key" Traceback (most recent call last): File "", line 1, in ? TypeError: unhashable type >>> d2[d2] = "subclassed dictionary allowed though" >>> d2 {{...}: 'subclassed dictionary allowed though', 'key': 'mutable keys'} >>> #whoa: what just happened: {...} ?? Interesting.... So is this a feature or a bug? """ ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:28 Message: Logged In: YES user_id=31435 Jeez, hash() has gone off the deep end: >>> import sys >>> hash(sys.stdin) 7690032 >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 From noreply@sourceforge.net Mon Oct 29 03:33:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 28 Oct 2001 19:33:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-475877 ] Mutable subtype instances are hashable Message-ID: Bugs item #475877, was opened at 2001-10-28 19:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) >Assigned to: Nobody/Anonymous (nobody) Summary: Mutable subtype instances are hashable Initial Comment: >>> class D(dictionary): pass ... >>> d = D() >>> hash(d) 7920544 >>> id(d) 7920544 >>> Ditto for instances of list subclasses: >>> class L(list): pass ... >>> x = L(range(100)) >>> hash(x) 7928992 >>> id(x) 7928992 >>> Among other nasties, this lets them get used as mutable dict keys. Reported by Mark J on c.l.py: """ From: Mark J Sent: Sunday, October 28, 2001 10:03 PM To: python-list@python.org Subject: Python 2.2b1 hashable dictionary bug? Since I haven't had a good hit rate at detecting bugs vs. features, I thought I'd post here before filing a bug report at SourceForge. Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class D(dictionary): pass ... >>> d = {} >>> d2 = D() >>> d[d2] = "dictionary used as key" >>> d {{}: 'dictionary used as key'} >>> d[D()]="now have two keys that look the same" >>> d {{}: 'now have two keys that look the same', {}: 'dictionary used as key'} >>> d2["key"] = "mutable key in d" >>> d {{}: 'now have two keys that look the same', {'key': 'mutable key in d'}: 'dictionary used as key'} >>> d[d] = "plain dictionary not allowed as key" Traceback (most recent call last): File "", line 1, in ? TypeError: unhashable type >>> d2[d2] = "subclassed dictionary allowed though" >>> d2 {{...}: 'subclassed dictionary allowed though', 'key': 'mutable keys'} >>> #whoa: what just happened: {...} ?? Interesting.... So is this a feature or a bug? """ ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:33 Message: Logged In: YES user_id=31435 I swear I didn't assign this to Guido -- I intended to leave this unassigned for now. That's the second time in two weeks I believe SF made up an assignment on one of my reports. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:28 Message: Logged In: YES user_id=31435 Jeez, hash() has gone off the deep end: >>> import sys >>> hash(sys.stdin) 7690032 >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 From noreply@sourceforge.net Mon Oct 29 07:36:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 28 Oct 2001 23:36:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-475877 ] Mutable subtype instances are hashable Message-ID: Bugs item #475877, was opened at 2001-10-28 19:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: Mutable subtype instances are hashable Initial Comment: >>> class D(dictionary): pass ... >>> d = D() >>> hash(d) 7920544 >>> id(d) 7920544 >>> Ditto for instances of list subclasses: >>> class L(list): pass ... >>> x = L(range(100)) >>> hash(x) 7928992 >>> id(x) 7928992 >>> Among other nasties, this lets them get used as mutable dict keys. Reported by Mark J on c.l.py: """ From: Mark J Sent: Sunday, October 28, 2001 10:03 PM To: python-list@python.org Subject: Python 2.2b1 hashable dictionary bug? Since I haven't had a good hit rate at detecting bugs vs. features, I thought I'd post here before filing a bug report at SourceForge. Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class D(dictionary): pass ... >>> d = {} >>> d2 = D() >>> d[d2] = "dictionary used as key" >>> d {{}: 'dictionary used as key'} >>> d[D()]="now have two keys that look the same" >>> d {{}: 'now have two keys that look the same', {}: 'dictionary used as key'} >>> d2["key"] = "mutable key in d" >>> d {{}: 'now have two keys that look the same', {'key': 'mutable key in d'}: 'dictionary used as key'} >>> d[d] = "plain dictionary not allowed as key" Traceback (most recent call last): File "", line 1, in ? TypeError: unhashable type >>> d2[d2] = "subclassed dictionary allowed though" >>> d2 {{...}: 'subclassed dictionary allowed though', 'key': 'mutable keys'} >>> #whoa: what just happened: {...} ?? Interesting.... So is this a feature or a bug? """ ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-28 23:36 Message: Logged In: YES user_id=6380 There's an admin setting that auto-assigns certain categories. Documentation gets auto-assugned to Fred; type/class to me; Regular Expressions to Effbot I believe. The hash(sys.stdin) result is expected; files are compared by address and so their hash() is derived from their address too. I'll think about the real issue; this has to do with the way the hash stub gets set. I'm not sure yet whether this is best fixed by adding more code to slot_tp_hash or to the code that sticks slot_to_hash in the tp_hash slot. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:33 Message: Logged In: YES user_id=31435 I swear I didn't assign this to Guido -- I intended to leave this unassigned for now. That's the second time in two weeks I believe SF made up an assignment on one of my reports. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:28 Message: Logged In: YES user_id=31435 Jeez, hash() has gone off the deep end: >>> import sys >>> hash(sys.stdin) 7690032 >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 From noreply@sourceforge.net Mon Oct 29 10:15:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 02:15:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Mon Oct 29 11:53:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 03:53:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-474836 ] Tix not included in windows distribution Message-ID: Bugs item #474836, was opened at 2001-10-25 04:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474836&group_id=5470 Category: Tkinter Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Tix not included in windows distribution Initial Comment: Although there is a Tix.py available, there is no Tix support in the precomiled Python-distribution for windows. So import Tix works fine, but root = Tix.Tk() results in TclError: package not found. It is possible to circumvent this problem by installing a regular Tcl/Tk distribution (e.g. in c:\programme\tcl) and installing Tix in the regular Tcl-path (i.e. tcl\lib). Mathias ---------------------------------------------------------------------- Comment By: Mathias Palm (monos) Date: 2001-10-29 03:53 Message: Logged In: YES user_id=361926 As mentioned in the mail above (by me, Mathias), Tix is a package belonging to Tcl/Tk (to be found on sourceforge: tix.sourceforge.net, or via the Python home page - tkinter link). Everything needed can be found there, just read about it (and dont forget about the winking, eyes might be getting dry) Mathias ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-25 11:26 Message: Logged In: YES user_id=31435 I don't know anything about Tix, so if somebody wants this in the Windows installer, they're going to have to explain exactly (by which I mean exactly <0.5 wink>) what's needed. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474836&group_id=5470 From noreply@sourceforge.net Mon Oct 29 12:59:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 04:59:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 04:59 Message: Logged In: YES user_id=6380 Please try again with Python 2.2b1 (http://python.org/2.2/). We've done a lot of work and we've got at least one report that it now works out of the box on HP-UX 11. I'd like to hear if that solves your problems (and if 2.2b1 builds with either compiler). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Mon Oct 29 16:33:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 08:33:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-453099 ] select in asyncore not defensive Message-ID: Bugs item #453099, was opened at 2001-08-19 15:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=453099&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: select in asyncore not defensive Initial Comment: The poll method in asyncore.py does not wrap the select.select call on line 86 in a try/except block to test for EINTR/EWOULDBLOCK and then try the select call again. This was brought to my attention during the perusal of the Zope-dev list. Seems like the person who originally reported the symptom did not open a bug so I thought I would. Platform: SunOS 5.8/Sparc Python version: 2.1.1 Zope version: 2.4.0 Thanks. Here is the mail from a person on the Zope-dev mailing list: I am moving this thread over from the zope list in the hope that someone here may have more insight into the nature of this problem, and perhaps how to go about better diagnosing/fixing the problem. To sum up: Under apparently random circumstances and rather mild amounts of traffic, Zope is crashing and printing the Traceback below to the console. This is Zope-2.4.0, Python-2.1.1 on SunOS 5.8/Sparc. Traceback (most recent call last): File "/u01/zope/Zope-2.4.0-src/z2.py", line 774, in ? asyncore.loop() File "/u01/zope//lib/python2.1/asyncore.py", line 194, in loop poll_fun (timeout, map) File "/u01/zope//lib/python2.1/asyncore.py", line 86, in poll r,w,e = select.select (r,w,e, timeout) select.error: (4, 'Interrupted system call') ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-10-29 08:33 Message: Logged In: YES user_id=31392 Fixed in rev 1 .25 of asyncore.py ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-09-11 10:45 Message: Logged In: YES user_id=31392 I thkn the select() and poll() calls should be defensive and be resumed as long as a timeout was specified and not yet exceeded. I'll check with Sam and see what he things. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-08-20 09:57 Message: Logged In: YES user_id=12800 Assigning to Jeremy. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-08-19 15:47 Message: Logged In: NO Sorry, the user's email is jziniti@speakeasy.org. EWOULDBLOCK was reported by that user for a different method in asyncore called accept though it seems that EWOULDBLOCK is being handled correctly in accept. It seems that the user did add the try/except block around the select call on line 86 which resulted in later call to accept() throwing EWOULDBLOCK. The trace is as follows: system.error 4((4, 'Interrupted system call')) ------ 2001-08-16T13:16:01 ERROR(200) ZServer uncaptured python exception, closing channel <__repr__ (self) failed for object at dee7bc (addr='/tmp/zope.soc')> (exceptions.TypeError:unpack non-sequence [/u01/zope/lib/python2.1/asyncore.py|poll|101] [/u01/zope/lib/python2.1/asyncore.py|handle_read_event|383] [/u01/zope/Zope-2.4.0-src/ZServer/FCGIServer.py|handle_accept|697]) ------ 2001-08-16T13:16:01 ERROR(200) ZServer uncaptured python exception, closing channel <__repr__ (self) failed for object at dec4bc (addr='/u01/zope/Zope-2.4.0-src/var/pcgi.soc')> (exceptions.TypeError:unpack non-sequence [/u01/zope/lib/python2.1/asyncore.py|poll|101] [/u01/zope/lib/python2.1/asyncore.py|handle_read_event|383] [/u01/zope/Zope-2.4.0-src/ZServer/PCGIServer.py|handle_accept|380]) ------ 2001-08-16T13:16:01 ERROR(200) ZServer uncaptured python exception, closing channel (exceptions.TypeError:unpack non-sequence [/u01/zope/lib/python2.1/asyncore.py|poll|101] [/u01/zope/lib/python2.1/asyncore.py|handle_read_event|383] [/u01/zope/Zope-2.4.0-src/ZServer/FTPServer.py|handle_accept|694]) ------ 2001-08-16T13:16:01 PROBLEM(100) ZServer warning: server accept() threw EWOULDBLOCK ------ 2001-08-16T13:16:01 ERROR(200) ZServer uncaptured python exception, closing channel (exceptions.OSError:[Errno 11] Resource temporarily unavailable [/u01/zope/lib/python2.1/asyncore.py|poll|101] [/u01/zope/lib/python2.1/asyncore.py|handle_read_event|389] [/u01/zope/Zope-2.4.0-src/ZServer/medusa/thread/select_trigger.py|handle_read|77] [/u01/zope/lib/python2.1/asyncore.py|recv|341] [/u01/zope/lib/python2.1/asyncore.py|recv|523]) Thanks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=453099&group_id=5470 From noreply@sourceforge.net Mon Oct 29 16:52:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 08:52:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-476054 ] small improvement to random Message-ID: Bugs item #476054, was opened at 2001-10-29 08:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476054&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: small improvement to random Initial Comment: there is this code in the random module: # Initialize from current time import time a = long(time.time() * 256) iwouldn't it be better to use multiply by 1000000 since the microseconds are probably more random than the seconds? -- erno@iki.fi ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476054&group_id=5470 From noreply@sourceforge.net Mon Oct 29 17:07:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 09:07:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-476054 ] small improvement to random Message-ID: Bugs item #476054, was opened at 2001-10-29 08:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476054&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: small improvement to random Initial Comment: there is this code in the random module: # Initialize from current time import time a = long(time.time() * 256) iwouldn't it be better to use multiply by 1000000 since the microseconds are probably more random than the seconds? -- erno@iki.fi ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476054&group_id=5470 From noreply@sourceforge.net Mon Oct 29 18:00:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 10:00:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-476054 ] small improvement to random Message-ID: Bugs item #476054, was opened at 2001-10-29 08:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476054&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: small improvement to random Initial Comment: there is this code in the random module: # Initialize from current time import time a = long(time.time() * 256) iwouldn't it be better to use multiply by 1000000 since the microseconds are probably more random than the seconds? -- erno@iki.fi ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-29 10:00 Message: Logged In: YES user_id=31435 The code has always done this, and I always assumed it was a good x-platform compromise. For example, on Windows time.time() % 1.0 can only take on 100 distinct values (it has only 1/100th second resolution), and is updated only about 18.2 times per second (it has only 0.055 second precision). The upshoot is that multiplying by 1e6 *there* would yield a long whose last 4 bits are always 0: >>> from time import time >>> d = {} >>> for i in xrange(1000000): ... t = time() * 1e6 ... t = int(long(t) & 0xff) ... d[t] = 1 ... >>> len(d) 16 >>> k = d.keys() >>> k.sort() >>> k >>> map(hex, k) ['0x0', '0x10', '0x20', '0x30', '0x40', '0x50', '0x60', '0x70', '0x80', '0x90','0xa0', '0xb0', '0xc0', '0xd0', '0xe0', '0xf0'] >>> That's worse than what the code does now (in short, ya, maybe microseconds are "more random", but only on boxes that have microsecond time() precision). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476054&group_id=5470 From noreply@sourceforge.net Mon Oct 29 19:39:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 11:39:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-475877 ] Mutable subtype instances are hashable Message-ID: Bugs item #475877, was opened at 2001-10-28 19:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: Mutable subtype instances are hashable Initial Comment: >>> class D(dictionary): pass ... >>> d = D() >>> hash(d) 7920544 >>> id(d) 7920544 >>> Ditto for instances of list subclasses: >>> class L(list): pass ... >>> x = L(range(100)) >>> hash(x) 7928992 >>> id(x) 7928992 >>> Among other nasties, this lets them get used as mutable dict keys. Reported by Mark J on c.l.py: """ From: Mark J Sent: Sunday, October 28, 2001 10:03 PM To: python-list@python.org Subject: Python 2.2b1 hashable dictionary bug? Since I haven't had a good hit rate at detecting bugs vs. features, I thought I'd post here before filing a bug report at SourceForge. Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class D(dictionary): pass ... >>> d = {} >>> d2 = D() >>> d[d2] = "dictionary used as key" >>> d {{}: 'dictionary used as key'} >>> d[D()]="now have two keys that look the same" >>> d {{}: 'now have two keys that look the same', {}: 'dictionary used as key'} >>> d2["key"] = "mutable key in d" >>> d {{}: 'now have two keys that look the same', {'key': 'mutable key in d'}: 'dictionary used as key'} >>> d[d] = "plain dictionary not allowed as key" Traceback (most recent call last): File "", line 1, in ? TypeError: unhashable type >>> d2[d2] = "subclassed dictionary allowed though" >>> d2 {{...}: 'subclassed dictionary allowed though', 'key': 'mutable keys'} >>> #whoa: what just happened: {...} ?? Interesting.... So is this a feature or a bug? """ ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 11:39 Message: Logged In: YES user_id=6380 Interestingly, in Python list.__hash__ is the same as object.__hash__, but in C, list.tp_hash is NULL while object.tp_hash is not. I think that the best solution is to somehow program an exception into add_operators that adds a dummy __hash__ wrapper (which always raises an exception) when the tp_hash field is found to be NULL. (Note that inherit_slots already contains special-casing for tp_hash.) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-28 23:36 Message: Logged In: YES user_id=6380 There's an admin setting that auto-assigns certain categories. Documentation gets auto-assugned to Fred; type/class to me; Regular Expressions to Effbot I believe. The hash(sys.stdin) result is expected; files are compared by address and so their hash() is derived from their address too. I'll think about the real issue; this has to do with the way the hash stub gets set. I'm not sure yet whether this is best fixed by adding more code to slot_tp_hash or to the code that sticks slot_to_hash in the tp_hash slot. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:33 Message: Logged In: YES user_id=31435 I swear I didn't assign this to Guido -- I intended to leave this unassigned for now. That's the second time in two weeks I believe SF made up an assignment on one of my reports. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:28 Message: Logged In: YES user_id=31435 Jeez, hash() has gone off the deep end: >>> import sys >>> hash(sys.stdin) 7690032 >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 From noreply@sourceforge.net Mon Oct 29 20:40:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 12:40:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-476129 ] gc.collect sometimes hangs Message-ID: Bugs item #476129, was opened at 2001-10-29 12:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitch Chapman (mitchchapman) Assigned to: Nobody/Anonymous (nobody) Summary: gc.collect sometimes hangs Initial Comment: When run under Python 2.1 on Mandrake Linux 7.2 (kernel 2.2.17) the attached script terminates normally. When run on the same machine under Python 2.2b1 it runs forever, printing an infinite sequence of "No such attribute __del__" messages to stdout. If you comment out the call to gc.disable in Container.refresh, the script terminates normally under both versions of the interpreter. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470 From noreply@sourceforge.net Mon Oct 29 21:02:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 13:02:12 -0800 Subject: [Python-bugs-list] [ python-Bugs-476129 ] gc.collect sometimes hangs Message-ID: Bugs item #476129, was opened at 2001-10-29 12:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitch Chapman (mitchchapman) >Assigned to: Neil Schemenauer (nascheme) Summary: gc.collect sometimes hangs Initial Comment: When run under Python 2.1 on Mandrake Linux 7.2 (kernel 2.2.17) the attached script terminates normally. When run on the same machine under Python 2.2b1 it runs forever, printing an infinite sequence of "No such attribute __del__" messages to stdout. If you comment out the call to gc.disable in Container.refresh, the script terminates normally under both versions of the interpreter. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 13:02 Message: Logged In: YES user_id=6380 Assigning to Neil Schemenauer, our GC expert. Note that in 2.1, I get a whole bunch of messages "No such attribute __del__" to stdout too -- but it stops. In 2.2, the *first* call to c=Container() and c.delegate() does the same thing. But the *second* call to Container() causes an infinite number of messages to print (even if it's not deleting the first c instance). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470 From noreply@sourceforge.net Mon Oct 29 21:14:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 13:14:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-476138 ] tempfile behavior across platforms Message-ID: Bugs item #476138, was opened at 2001-10-29 13:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeremy Hylton (jhylton) Assigned to: Nobody/Anonymous (nobody) Summary: tempfile behavior across platforms Initial Comment: The object returned by TemporaryFile.tempfile() has a close() method. On Unix, this is a builtin file object's close method. On Windows, it is a TemporaryFileWrapper object's close method. The two have different semantics, which limits the utility of the module in code that should run on both platforms. There are two differences: - The close() method on Windows unlinks the file after closing it. The close() method on Unix does not. - Calling close() on a closed tempfile raises an exception on Windows, but has no effect on Unix. (This is a consequence of the previous difference. The second attempt to unlink the file fails.) I think the sensible fix is to catch and ignore failed unlink calls on Windows. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 From noreply@sourceforge.net Mon Oct 29 21:21:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 13:21:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-476138 ] tempfile behavior across platforms Message-ID: Bugs item #476138, was opened at 2001-10-29 13:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeremy Hylton (jhylton) Assigned to: Nobody/Anonymous (nobody) Summary: tempfile behavior across platforms Initial Comment: The object returned by TemporaryFile.tempfile() has a close() method. On Unix, this is a builtin file object's close method. On Windows, it is a TemporaryFileWrapper object's close method. The two have different semantics, which limits the utility of the module in code that should run on both platforms. There are two differences: - The close() method on Windows unlinks the file after closing it. The close() method on Unix does not. - Calling close() on a closed tempfile raises an exception on Windows, but has no effect on Unix. (This is a consequence of the previous difference. The second attempt to unlink the file fails.) I think the sensible fix is to catch and ignore failed unlink calls on Windows. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 13:21 Message: Logged In: YES user_id=6380 The first difference doesn't really bother you, does it? On Unix, the file has already been unlinked when it was first created. It sounds like the real difference you're after is that the Unix version allows calling close() more than once, because open files can be closed more than once. I think this is a useful semantic to retain, and propose to fix it as you suggest. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 From noreply@sourceforge.net Mon Oct 29 21:31:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 13:31:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-476138 ] tempfile behavior across platforms Message-ID: Bugs item #476138, was opened at 2001-10-29 13:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jeremy Hylton (jhylton) >Assigned to: Tim Peters (tim_one) Summary: tempfile behavior across platforms Initial Comment: The object returned by TemporaryFile.tempfile() has a close() method. On Unix, this is a builtin file object's close method. On Windows, it is a TemporaryFileWrapper object's close method. The two have different semantics, which limits the utility of the module in code that should run on both platforms. There are two differences: - The close() method on Windows unlinks the file after closing it. The close() method on Unix does not. - Calling close() on a closed tempfile raises an exception on Windows, but has no effect on Unix. (This is a consequence of the previous difference. The second attempt to unlink the file fails.) I think the sensible fix is to catch and ignore failed unlink calls on Windows. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-29 13:31 Message: Logged In: YES user_id=31435 Well, on Unix the file is unlinked before the tempfile *create* returns. That isn't possible on Windows. I agree a tempfile should-- like a Python file object --be closeable any number of times, although it should raise an exception if the *first* attempt to unlink it fails. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 13:21 Message: Logged In: YES user_id=6380 The first difference doesn't really bother you, does it? On Unix, the file has already been unlinked when it was first created. It sounds like the real difference you're after is that the Unix version allows calling close() more than once, because open files can be closed more than once. I think this is a useful semantic to retain, and propose to fix it as you suggest. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 From noreply@sourceforge.net Mon Oct 29 21:48:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 13:48:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-476138 ] tempfile behavior across platforms Message-ID: Bugs item #476138, was opened at 2001-10-29 13:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Jeremy Hylton (jhylton) Assigned to: Tim Peters (tim_one) Summary: tempfile behavior across platforms Initial Comment: The object returned by TemporaryFile.tempfile() has a close() method. On Unix, this is a builtin file object's close method. On Windows, it is a TemporaryFileWrapper object's close method. The two have different semantics, which limits the utility of the module in code that should run on both platforms. There are two differences: - The close() method on Windows unlinks the file after closing it. The close() method on Unix does not. - Calling close() on a closed tempfile raises an exception on Windows, but has no effect on Unix. (This is a consequence of the previous difference. The second attempt to unlink the file fails.) I think the sensible fix is to catch and ignore failed unlink calls on Windows. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-29 13:48 Message: Logged In: YES user_id=31435 tempfiles can now be closed any number of times on Windows, although if the unlink fails on Windows something is wrong, so I don't suppress that: Lib/tempfile.py; new revision: 1.32 Lib/test/test_tempfile.py; initial revision: 1.1 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-29 13:31 Message: Logged In: YES user_id=31435 Well, on Unix the file is unlinked before the tempfile *create* returns. That isn't possible on Windows. I agree a tempfile should-- like a Python file object --be closeable any number of times, although it should raise an exception if the *first* attempt to unlink it fails. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 13:21 Message: Logged In: YES user_id=6380 The first difference doesn't really bother you, does it? On Unix, the file has already been unlinked when it was first created. It sounds like the real difference you're after is that the Unix version allows calling close() more than once, because open files can be closed more than once. I think this is a useful semantic to retain, and propose to fix it as you suggest. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476138&group_id=5470 From noreply@sourceforge.net Mon Oct 29 23:45:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 15:45:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-476129 ] gc.collect sometimes hangs Message-ID: Bugs item #476129, was opened at 2001-10-29 12:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Mitch Chapman (mitchchapman) Assigned to: Neil Schemenauer (nascheme) Summary: gc.collect sometimes hangs Initial Comment: When run under Python 2.1 on Mandrake Linux 7.2 (kernel 2.2.17) the attached script terminates normally. When run on the same machine under Python 2.2b1 it runs forever, printing an infinite sequence of "No such attribute __del__" messages to stdout. If you comment out the call to gc.disable in Container.refresh, the script terminates normally under both versions of the interpreter. ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2001-10-29 15:45 Message: Logged In: YES user_id=35752 I think the bug is that gc.collect() does not do locking. The collector can start running twice. That can corrupt the GC linked lists. Please try the attached patch with 2.2. I think this bug is in 2.1 as well. I'd check the patch in now but SF CVS seems to be broken. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 13:02 Message: Logged In: YES user_id=6380 Assigning to Neil Schemenauer, our GC expert. Note that in 2.1, I get a whole bunch of messages "No such attribute __del__" to stdout too -- but it stops. In 2.2, the *first* call to c=Container() and c.delegate() does the same thing. But the *second* call to Container() causes an infinite number of messages to print (even if it's not deleting the first c instance). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470 From noreply@sourceforge.net Tue Oct 30 00:00:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 16:00:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-476194 ] dir(Ellipsis) Message-ID: Bugs item #476194, was opened at 2001-10-29 16:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476194&group_id=5470 Category: Type/class unification Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: dir(Ellipsis) Initial Comment: Seems like dir(Ellipsis) in 2.2 should return a non- empty list that includes __class__, etc. just like the other builtin types C:\>python Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> >>> dir(None) ['__class__', '__delattr__', '__getattribute__', '__has h__', '__init__', '__new__', '__reduce__', '_ _repr__', '__setattr__', '__str__'] >>> Ellipsis Ellipsis >>> dir(Ellipsis) [] >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476194&group_id=5470 From noreply@sourceforge.net Tue Oct 30 02:42:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 18:42:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-476194 ] dir(Ellipsis) Message-ID: Bugs item #476194, was opened at 2001-10-29 16:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476194&group_id=5470 Category: Type/class unification >Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: dir(Ellipsis) Initial Comment: Seems like dir(Ellipsis) in 2.2 should return a non- empty list that includes __class__, etc. just like the other builtin types C:\>python Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> >>> dir(None) ['__class__', '__delattr__', '__getattribute__', '__has h__', '__init__', '__new__', '__reduce__', '_ _repr__', '__setattr__', '__str__'] >>> Ellipsis Ellipsis >>> dir(Ellipsis) [] >>> ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 18:42 Message: Logged In: YES user_id=6380 Thanks for noticing. I've fixed this now in CVS, sliceobject.c 2.10. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476194&group_id=5470 From noreply@sourceforge.net Tue Oct 30 02:49:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 18:49:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-473788 ] CVS socket module does not compile on Tru64 Message-ID: Bugs item #473788, was opened at 2001-10-22 13:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473788&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Ralf W. Grosse-Kunstleve (rwgk) >Assigned to: Martin v. Löwis (loewis) >Summary: CVS socket module does not compile on Tru64 Initial Comment: Platform: OSF1 ringneck.lbl.gov V5.1 732 alpha Compaq C V6.3-028 on Compaq Tru64 UNIX V5.1 (Rev. 732) Compiler Driver V6.3-026 (sys) cc Driver Python 2.2b1 (#1, Oct 20 2001, 23:40:31) [C] on osf1V5 configure --disable-ipv6 cc -O -Olimit 1500 -I. - I/net/ringneck/scratch1/rwgk/Python-2.2b1/./Include - I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1- alpha-2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473788&group_id=5470 From noreply@sourceforge.net Tue Oct 30 02:50:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 18:50:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-474077 ] 2.2b1: Error compiling extns with BCC Message-ID: Bugs item #474077, was opened at 2001-10-23 07:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474077&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Luigi Ballabio (lballabio) >Assigned to: Tim Peters (tim_one) Summary: 2.2b1: Error compiling extns with BCC Initial Comment: Borland C++ fails to compile extensions for Python 2.2b1. The error message is: D:\Python22\include\pyport.h 350: Only one of a set of overloaded functions can be "C" The solution is to remove line 183 of pyconfig.h, namely, #undef HAVE_HYPOT ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474077&group_id=5470 From noreply@sourceforge.net Tue Oct 30 02:50:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 18:50:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-474485 ] pydoc generates some bad html Message-ID: Bugs item #474485, was opened at 2001-10-24 07:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474485&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Rich Salz (rsalz) >Assigned to: Tim Peters (tim_one) Summary: pydoc generates some bad html Initial Comment: One bug, two nits. When documenting lambda's, pydoc outputs which is interepreted as HTML markup. I partially fixed that; I didn't add support for A/HREF, which I think is what's really intended. The nits: HTMLtidy says tables should have a summary attribute, and

directly before a

is an empty element which should be removed. Attached patch addresses these. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474485&group_id=5470 From noreply@sourceforge.net Tue Oct 30 02:51:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 18:51:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-473060 ] Doc/ doesn't compile in 2.2b1 Message-ID: Bugs item #473060, was opened at 2001-10-20 02:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 Category: Build Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Masanori Omote (omote) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Doc/ doesn't compile in 2.2b1 Initial Comment: Documentations in the distribution tar file of 2.2b1 doesn't compile. Doc/ of a snapshot tar ball on 10/19/01 compiles correctly. Materials under Doc/ in both tar balls are the same according to diff -r check. Containing a dot (.) in the name of the top directory of 2.2b1 leads to a LaTeX2HTML error, according to its error message. It wouldn't be a bug because the README file in Doc directory says that a user should "twiddle" Doc/Makefile. But I hope all the documentation can be easily compiled from the next distribution files. ---------------------------------------------------------------------- Comment By: Masanori Omote (omote) Date: 2001-10-20 09:17 Message: Logged In: YES user_id=232907 Additional info of omote's case: System: RH 7.1 Python: 2.1.1 (2.2b1 not yet installed) LaTeX2HTML: 2K.1beta (1.47) When I compile Doc/ in the extracted tree of 2.2b1 distribution tar file, it fails. It doesn't fail when I move Doc/ directory to any other directory which path doesn't contain a dot. Following is a failure example to compile 'doc'. % pwd /home/masa/ftp/python/Python-2.2b1/Doc % make doc python tools/mkhowto --html --about html/stdabout.dat --address "See About this document... for information on suggesting changes." --up-link ../index.html --up-title "Python Documentation Index" --global-module-index "../modindex.html" --dir html/doc doc/doc.tex +++ TEXINPUTS=/home/masa/ftp/python/Python-2.2b1/Doc/doc:/home/masa/ftp/python/Python-2.2b1/Doc/paper-letter:/home/masa/ftp/python/Python-2.2b1/Doc/texinputs: +++ latex doc +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. The relevant lines from the transcript are: ------------------------------------------------------------------------ +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex Note: Initialising with file: doc.l2h This is LaTeX2HTML Version 2K.1beta (1.47) by Nikos Drakos, Computer Based Learning Unit, University of Leeds. Revised and extended by: Marcus Hennecke, Ross Moore, Herb Swan and others ...producing markup for HTML version 4.0 Loading /usr/share/latex2html/versions/html4_0.pl *** processing declarations *** Loading /usr/share/latex2html/versions/latin1.pl *** Fatal Error --- but easy to fix *** Cannot have '.' in file-name prefix, else dvips fails on images Change the name from /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex and try again. *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:31 Message: Logged In: YES user_id=21627 Can you give more details as to how exactly compilation fails? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 From noreply@sourceforge.net Tue Oct 30 02:51:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 18:51:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-473623 ] Emacs: python-mode.el Message-ID: Bugs item #473623, was opened at 2001-10-22 03:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473623&group_id=5470 >Category: Documentation Group: Not a Bug Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Barry Warsaw (bwarsaw) Summary: Emacs: python-mode.el Initial Comment: Please explain in python-mode.el how to install the mode in emacs. On Windows you need to copy python-mode.el into $EMACS_HOME/lisp and add "(require 'python-mode)" into your startupscript (E.g. $HOME/.emacs). Thank you ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-22 11:49 Message: Logged In: YES user_id=21627 There is a link to http://www.python.org/emacs/python-mode/ in python-mode.el, which in turn offers a link "Installation Notes". Is this not enough? I think these instructions are much beter than putting a require into .emacs, since they only load python-mode when needed. OTOH, they are also somewhat more complex, since they work through auto-mode-alist and interpreter-mode-alist. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473623&group_id=5470 From noreply@sourceforge.net Tue Oct 30 03:01:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 19:01:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-473009 ] binascii_b2a_base64() improper str limit Message-ID: Bugs item #473009, was opened at 2001-10-19 21:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 Category: Python Library >Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Dave Cinege (dcinege) >Assigned to: Guido van Rossum (gvanrossum) Summary: binascii_b2a_base64() improper str limit Initial Comment: Modules/binascii.c binascii_b2a_base64() contains the following restrictive code: if ( bin_len > BASE64_MAXBIN ) { PyErr_SetString(Error, "Too much data for base64 line"); return NULL; } This is an error. The base64 method of encoding data has no length limitation. The MIME message RCF has such a limitation of base64 encoded data. The function should not assume it's only input must be MIME compatible. The base64 python module itself is designed for MIME I/O only, and properly limits itself. The binascii function should be left raw. binascii_a2b_base64() properly accepts input of any size. How I came across this bug: I use base64 to ascii armor binary data in log entries in a distributed network monitoring system. For the sake of ease of parsing (human and machine) all log entries are delimited by a single line. I commonly have unbroken base64 encoded fields of 64KB in size or greater. Unfortunatly I am unable to encode this data like this: result64 = binascii.b2a_base64(s) I must do this: result64 = re.sub('[ |\n]','',base64.encodestring(s)) Which is *much* slower. : < I feel this is an outright bug and should be corrected. If their is some argument for backward compatibly an optional function argument should be present to allow bypassing this limitation. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 19:01 Message: Logged In: YES user_id=6380 Fixed in CVS, will be in 2.2b2. (binascii.c 2.32). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-21 18:26 Message: Logged In: YES user_id=6380 I'm with David. It's up to the higher level code (e.g. the base64 module) to avoid writing lines longer than 76 characters; the underlying function in binascii doesn't have to act as a policeman here. There may be other applications of the same encoding where the 76-char limit does not apply. ---------------------------------------------------------------------- Comment By: Dave Cinege (dcinege) Date: 2001-10-20 21:34 Message: Logged In: YES user_id=314434 >Can you cite any relevant standard that defines base64 to >work in that way? Base64 is defined in RFC 2045 section >6.8., which clearly says >The encoded output stream must be represented in lines >of no more than 76 characters each. This is difficult to do because base64 itself has not (yet) been seperatly defined in it's own RFC. It should be and this issue has been brought up recently on the W3 lists. IE: http://lists.w3.org/Archives/Public/w3c-ietf-xmldsig/2001AprJun/0212.html http://lists.w3.org/Archives/Public/w3c-ietf-xmldsig/2001AprJun/0210.html The part of the RFC you have quoted is relevent to the use of base64 encoding in the context of MIME, the purpose clearly being to ensure compatibly with email (SMTP, POP3, MUA, etc) standards. However this 76 character line length rule is irrelevent when dealing with arbitary binary data, not meant for MIME encapulated transmission. This is clearly seen the describtion of the actual base64 algorithms itself: The encoding process represents 24-bit groups of input bits as output strings of 4 encoded characters. Proceeding from left to right, a 24-bit input group is formed by concatenating 3 8bit input groups. These 24 bits are then treated as 4 concatenated 6-bit groups, each of which is translated into a single digit in the base64 alphabet. When encoding a bit stream via the base64 encoding, the bit stream must be presumed to be ordered with the most-significant-bit first. That is, the first bit in the stream will be the high-order bit in the first 8bit byte, and the eighth bit will be the low-order bit in the first 8bit byte, and so on. ... In base64 data, characters other than those in Table 1, line breaks, and other white space probably indicate a transmission error, about which a warning message or even a message rejection might be appropriate under some circumstances. Additionally the use of 'unlimited length' base64 encoding of binary data has reached critical mass. For braod based example HTTP based authorization 'encrypts' the username:password in base64. However no length limit can be used, else it would arbiltarily limit the amount of data that could be passed without interfering with the HTTP protocol itself. IE: (Lines should not appear wrapped) 'Logging in' to a webserver with Username: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXY Z0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUV WXYZ0123456789 Password: test Will have the web broswer send the AUTH request header as follows: Authorization: Basic YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0NTY3ODlhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ekFCQ0RFRkdISUpLTE1OT1BRUlNUVVZXWFlaMDEyMzQ1Njc4OTp0ZXN The latter field is an 'unlimited' length base64 encoding. (Testing done with KDE Konqueror, other browsers may vary) Due to it's simple application you will find many a reference stating: ''The Base64 algorithm has become "the standard" for encoding binary data.'' Clearly line length limitation are counter productive to such use. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:30 Message: Logged In: YES user_id=21627 Can you cite any relevant standard that defines base64 to work in that way? Base64 is defined in RFC 2045 section 6.8., which clearly says The encoded output stream must be represented in lines of no more than 76 characters each. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473009&group_id=5470 From noreply@sourceforge.net Tue Oct 30 03:13:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 19:13:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-462274 ] Add "l2h-arg" to mkhowto Message-ID: Bugs item #462274, was opened at 2001-09-17 09:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 Category: Demos and Tools Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Rich Salz (rsalz) >Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Add "l2h-arg" to mkhowto Initial Comment: It would be nice to be able to pass arbitrary latex2html options from the mkhowto command line. I know I can achieve (most of) the same effect by studying latex2html.pl and creating my own init file, but I'd rather stick to the published interface. :) The attached patch adds --l2h-arg= flag which lets you pass args to latex2html. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 From noreply@sourceforge.net Tue Oct 30 03:18:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 19:18:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-456386 ] test_commands regression failure Message-ID: Bugs item #456386, was opened at 2001-08-28 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=456386&group_id=5470 >Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Andrew Dalke (dalke) >Assigned to: Guido van Rossum (gvanrossum) Summary: test_commands regression failure Initial Comment: test_commands does not work on IRIX It assumes the output of "ls /bin/ls" is a line that starts with a '-'. On IRIX that file is a symbolic link, so the first character is an l. This causes test_getstatus to fail. >>> import commands >>> commands.getstatus("/bin/ls") 'lrwxr-xr-x 1 root sys 13 Feb 23 1997 /bin/ls -> ../../sbin/ls' >>> pat = r'''-..x..x..x # It is executable. ... \s+\d+ # It has some number of links. ... \s+\w+\s+\w+ # It has a user and group, which may ... # be named anything. ... [^/]* # Skip the date. ... /bin/ls # and end with the name of the file. ... ''' >>> import re >>> re.match(pat, commands.getstatus("/bin/ls"), re.VERBOSE) >>> Here's a context diff for a patch. (I hate how narrow this entry textbox is.) Feel free to adjust the '#'s so they all line up. *** Lib/test/test_commands.py 2001/07/23 04:08:01 1.1 --- Lib/test/test_commands.py 2001/08/29 04:18:36 *************** *** 32,38 **** def test_getstatus(self): # This pattern should match 'ls -ld /bin/ls' on any posix # system, however perversely configured. ! pat = r'''-..x..x..x # It is executable. \s+\d+ # It has some number of links. \s+\w+\s+\w+ # It has a user and group, which may # be named anything. --- 32,38 ---- def test_getstatus(self): # This pattern should match 'ls -ld /bin/ls' on any posix # system, however perversely configured. ! pat = r'''[l-]..x..x..x # It is executable. (May be a symlink.) \s+\d+ # It has some number of links. \s+\w+\s+\w+ # It has a user and group, which may # be named anything. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 19:18 Message: Logged In: YES user_id=6380 Thanks. Fixed in CVS, test_commands.py 1.3, exactly as you suggest. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=456386&group_id=5470 From noreply@sourceforge.net Tue Oct 30 05:25:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 21:25:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-473060 ] Doc/ doesn't compile in 2.2b1 Message-ID: Bugs item #473060, was opened at 2001-10-20 02:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 Category: Build >Group: None Status: Open Resolution: None >Priority: 4 Submitted By: Masanori Omote (omote) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Doc/ doesn't compile in 2.2b1 Initial Comment: Documentations in the distribution tar file of 2.2b1 doesn't compile. Doc/ of a snapshot tar ball on 10/19/01 compiles correctly. Materials under Doc/ in both tar balls are the same according to diff -r check. Containing a dot (.) in the name of the top directory of 2.2b1 leads to a LaTeX2HTML error, according to its error message. It wouldn't be a bug because the README file in Doc directory says that a user should "twiddle" Doc/Makefile. But I hope all the documentation can be easily compiled from the next distribution files. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-29 21:25 Message: Logged In: YES user_id=3066 As noted in the error message, this is because there's a dot in the path to the documentation sources. While I consider this to be primarily a tools problem, it's sufficiently annoying that it deserves some attention. ---------------------------------------------------------------------- Comment By: Masanori Omote (omote) Date: 2001-10-20 09:17 Message: Logged In: YES user_id=232907 Additional info of omote's case: System: RH 7.1 Python: 2.1.1 (2.2b1 not yet installed) LaTeX2HTML: 2K.1beta (1.47) When I compile Doc/ in the extracted tree of 2.2b1 distribution tar file, it fails. It doesn't fail when I move Doc/ directory to any other directory which path doesn't contain a dot. Following is a failure example to compile 'doc'. % pwd /home/masa/ftp/python/Python-2.2b1/Doc % make doc python tools/mkhowto --html --about html/stdabout.dat --address "See About this document... for information on suggesting changes." --up-link ../index.html --up-title "Python Documentation Index" --global-module-index "../modindex.html" --dir html/doc doc/doc.tex +++ TEXINPUTS=/home/masa/ftp/python/Python-2.2b1/Doc/doc:/home/masa/ftp/python/Python-2.2b1/Doc/paper-letter:/home/masa/ftp/python/Python-2.2b1/Doc/texinputs: +++ latex doc +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. The relevant lines from the transcript are: ------------------------------------------------------------------------ +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex Note: Initialising with file: doc.l2h This is LaTeX2HTML Version 2K.1beta (1.47) by Nikos Drakos, Computer Based Learning Unit, University of Leeds. Revised and extended by: Marcus Hennecke, Ross Moore, Herb Swan and others ...producing markup for HTML version 4.0 Loading /usr/share/latex2html/versions/html4_0.pl *** processing declarations *** Loading /usr/share/latex2html/versions/latin1.pl *** Fatal Error --- but easy to fix *** Cannot have '.' in file-name prefix, else dvips fails on images Change the name from /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex and try again. *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:31 Message: Logged In: YES user_id=21627 Can you give more details as to how exactly compilation fails? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 From noreply@sourceforge.net Tue Oct 30 05:29:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 29 Oct 2001 21:29:28 -0800 Subject: [Python-bugs-list] [ python-Bugs-462274 ] Add "l2h-arg" to mkhowto Message-ID: Bugs item #462274, was opened at 2001-09-17 09:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 Category: Demos and Tools Group: Feature Request >Status: Pending Resolution: None Priority: 5 Submitted By: Rich Salz (rsalz) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Add "l2h-arg" to mkhowto Initial Comment: It would be nice to be able to pass arbitrary latex2html options from the mkhowto command line. I know I can achieve (most of) the same effect by studying latex2html.pl and creating my own init file, but I'd rather stick to the published interface. :) The attached patch adds --l2h-arg= flag which lets you pass args to latex2html. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-29 21:29 Message: Logged In: YES user_id=3066 Before I accept this, can you describe what you were trying to achieve that required this? It may be that there's a better way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 From noreply@sourceforge.net Tue Oct 30 08:56:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 00:56:41 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-30 00:56 Message: Logged In: YES user_id=21627 It would be helpful if you would provide all details in a report also. When you say "it crashes", did you really mean "it crashes" (i.e. with a core dump, machine reboot, or the like)? If the compiler merely rejected the code, it would be good to see what the error message was. Please also report version numbers: HP-UX version, gcc version, etc. Are you sure you are using a gcc built for your OS version? If the gcc doesn't get prototypes right, this often comes from having fixincluded header files of a different OS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 04:59 Message: Logged In: YES user_id=6380 Please try again with Python 2.2b1 (http://python.org/2.2/). We've done a lot of work and we've got at least one report that it now works out of the box on HP-UX 11. I'd like to hear if that solves your problems (and if 2.2b1 builds with either compiler). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Tue Oct 30 11:25:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 03:25:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-476326 ] Unicode and imp.find_module Message-ID: Bugs item #476326, was opened at 2001-10-30 03:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476326&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Unicode and imp.find_module Initial Comment: When a Unicode string is passed as the module name to imp.find_module, the function fails to import the named module even when it exists in the specified path, returning the error message "No module named ..." as a result. The problem in Python 2.0 can be traced to line 922 of Python/import.c which ensures that any strings involved in the find_module function must be standard Python strings and not Unicode strings, since it tests the type of path components against &PyString_Type explicitly. Interestingly, the __import__ built-in function seems to work with Unicode strings. Either way, it would be great if this could be documented or even fixed, but I don't know what the policy is on Unicode module names (even when they only contain ASCII-compatible characters). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476326&group_id=5470 From noreply@sourceforge.net Tue Oct 30 11:31:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 03:31:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-473788 ] CVS socket module does not compile on Tru64 Message-ID: Bugs item #473788, was opened at 2001-10-22 13:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473788&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Martin v. Löwis (loewis) Summary: CVS socket module does not compile on Tru64 Initial Comment: Platform: OSF1 ringneck.lbl.gov V5.1 732 alpha Compaq C V6.3-028 on Compaq Tru64 UNIX V5.1 (Rev. 732) Compiler Driver V6.3-026 (sys) cc Driver Python 2.2b1 (#1, Oct 20 2001, 23:40:31) [C] on osf1V5 configure --disable-ipv6 cc -O -Olimit 1500 -I. - I/net/ringneck/scratch1/rwgk/Python-2.2b1/./Include - I/usr/local/include -IInclude/ - c /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/socketmodule.c -o build/temp.osf1-V5.1- alpha-2.2/socketmodule.o cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 205: In this statement, "EAI_MAX" is not declared. (undeclared) if (ecode < 0 || ecode > EAI_MAX) ---------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 285: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR(EAI_BADHINTS); /* xxx */ ------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 286: In this statement, "AI_MASK" is not declared. (undeclared) if (hints->ai_flags & ~AI_MASK) ---------------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 320: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 326: In this statement, "EAI_BADHINTS" is not declared. (undeclared) ERR (EAI_BADHINTS); /*xxx*/ --------------------------------^ cc: Error: /net/ringneck/scratch1/rwgk/Python- 2.2b1/Modules/getaddrinfo.c, line 376: In this statement, "EAI_PROTOCOL" is not declared. (undeclared) ERR (EAI_PROTOCOL); /*xxx*/ ----------------------------------------^ WARNING: building of extension "_socket" failed: command 'cc' failed with exit status 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-10-30 03:31 Message: Logged In: YES user_id=21627 Fixed with configure.in 1.277. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473788&group_id=5470 From noreply@sourceforge.net Tue Oct 30 14:30:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 06:30:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-462274 ] Add "l2h-arg" to mkhowto Message-ID: Bugs item #462274, was opened at 2001-09-17 09:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 Category: Demos and Tools Group: Feature Request >Status: Open Resolution: None Priority: 5 Submitted By: Rich Salz (rsalz) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Add "l2h-arg" to mkhowto Initial Comment: It would be nice to be able to pass arbitrary latex2html options from the mkhowto command line. I know I can achieve (most of) the same effect by studying latex2html.pl and creating my own init file, but I'd rather stick to the published interface. :) The attached patch adds --l2h-arg= flag which lets you pass args to latex2html. ---------------------------------------------------------------------- >Comment By: Rich Salz (rsalz) Date: 2001-10-30 06:30 Message: Logged In: YES user_id=36737 I think was trying to change the navigation links, or even suppress them altogether. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-29 21:29 Message: Logged In: YES user_id=3066 Before I accept this, can you describe what you were trying to achieve that required this? It may be that there's a better way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 From noreply@sourceforge.net Tue Oct 30 14:53:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 06:53:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-462274 ] Add "l2h-arg" to mkhowto Message-ID: Bugs item #462274, was opened at 2001-09-17 09:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 >Category: Documentation Group: Feature Request >Status: Pending >Resolution: Later >Priority: 4 Submitted By: Rich Salz (rsalz) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Add "l2h-arg" to mkhowto Initial Comment: It would be nice to be able to pass arbitrary latex2html options from the mkhowto command line. I know I can achieve (most of) the same effect by studying latex2html.pl and creating my own init file, but I'd rather stick to the published interface. :) The attached patch adds --l2h-arg= flag which lets you pass args to latex2html. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-30 06:53 Message: Logged In: YES user_id=3066 Take a look in the CVS version of Python at how the iSilo version is generated. You'll want to look in Doc/Makefile and Doc/perl/isilo.perl. I'm dropping the priority & marking this pending/later since there's not a clear immediate need. If the iSilo version doesn't show how to do what you need, please explain in detail what the real effect is that you're looking for. (If it does show you what you need, let me know so I can close this report.) ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-10-30 06:30 Message: Logged In: YES user_id=36737 I think was trying to change the navigation links, or even suppress them altogether. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-29 21:29 Message: Logged In: YES user_id=3066 Before I accept this, can you describe what you were trying to achieve that required this? It may be that there's a better way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 From noreply@sourceforge.net Tue Oct 30 15:41:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 07:41:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-462274 ] Add "l2h-arg" to mkhowto Message-ID: Bugs item #462274, was opened at 2001-09-17 09:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 Category: Documentation Group: Feature Request >Status: Open Resolution: Later Priority: 4 Submitted By: Rich Salz (rsalz) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Add "l2h-arg" to mkhowto Initial Comment: It would be nice to be able to pass arbitrary latex2html options from the mkhowto command line. I know I can achieve (most of) the same effect by studying latex2html.pl and creating my own init file, but I'd rather stick to the published interface. :) The attached patch adds --l2h-arg= flag which lets you pass args to latex2html. ---------------------------------------------------------------------- >Comment By: Rich Salz (rsalz) Date: 2001-10-30 07:41 Message: Logged In: YES user_id=36737 okay, fine. :) ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-30 06:53 Message: Logged In: YES user_id=3066 Take a look in the CVS version of Python at how the iSilo version is generated. You'll want to look in Doc/Makefile and Doc/perl/isilo.perl. I'm dropping the priority & marking this pending/later since there's not a clear immediate need. If the iSilo version doesn't show how to do what you need, please explain in detail what the real effect is that you're looking for. (If it does show you what you need, let me know so I can close this report.) ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-10-30 06:30 Message: Logged In: YES user_id=36737 I think was trying to change the navigation links, or even suppress them altogether. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-29 21:29 Message: Logged In: YES user_id=3066 Before I accept this, can you describe what you were trying to achieve that required this? It may be that there's a better way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 From noreply@sourceforge.net Tue Oct 30 17:49:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 09:49:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-476419 ] Turtle documentation not included. Message-ID: Bugs item #476419, was opened at 2001-10-30 09:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476419&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Josh Cogliati (jjc) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Turtle documentation not included. Initial Comment: Problem: The file libturtle.tex has been written. However it is not included in lib.tex Also in libundoc.tex the turtle library is listed as undocumented, dispite the fact that documentation has been written. Solution: Include libturtle.tex in lib.tex and remove the references to turtle being undocumented in libundoc.tex ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476419&group_id=5470 From noreply@sourceforge.net Tue Oct 30 18:52:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 10:52:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-476435 ] inconsistant behavior with float and str Message-ID: Bugs item #476435, was opened at 2001-10-30 10:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476435&group_id=5470 Category: None Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Brian Hoenig (rambozo) Assigned to: Nobody/Anonymous (nobody) Summary: inconsistant behavior with float and str Initial Comment: >>> 1.1 1.1000000000000001 >>> str(1.1) '1.1' >>> str(1.1000000000000001) '1.1' >>> This happens on 2.1.1 on Linux and win32. There seem to be several issues here. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476435&group_id=5470 From noreply@sourceforge.net Tue Oct 30 19:58:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 11:58:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-476435 ] inconsistant behavior with float and str Message-ID: Bugs item #476435, was opened at 2001-10-30 10:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476435&group_id=5470 >Category: Python Interpreter Core >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Brian Hoenig (rambozo) >Assigned to: Tim Peters (tim_one) Summary: inconsistant behavior with float and str Initial Comment: >>> 1.1 1.1000000000000001 >>> str(1.1) '1.1' >>> str(1.1000000000000001) '1.1' >>> This happens on 2.1.1 on Linux and win32. There seem to be several issues here. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-30 11:58 Message: Logged In: YES user_id=31435 This is not a bug. Binary floating point cannot represent decimal fractions exactly, so some rounding always occurs (even in Python 1.5.2). What changed is that Python 2.0 shows more precision than before in certain circumstances (repr() and the interactive prompt). You can use str() or print to get the old, rounded output: >>> print 0.1+0.1 0.2 >>> Follow the link for more information: http://python.sourceforge.net/devel-docs/tut/node14.html ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476435&group_id=5470 From noreply@sourceforge.net Tue Oct 30 20:35:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 12:35:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-462274 ] Add "l2h-arg" to mkhowto Message-ID: Bugs item #462274, was opened at 2001-09-17 09:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 Category: Documentation Group: Feature Request >Status: Closed >Resolution: Rejected Priority: 4 Submitted By: Rich Salz (rsalz) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Add "l2h-arg" to mkhowto Initial Comment: It would be nice to be able to pass arbitrary latex2html options from the mkhowto command line. I know I can achieve (most of) the same effect by studying latex2html.pl and creating my own init file, but I'd rather stick to the published interface. :) The attached patch adds --l2h-arg= flag which lets you pass args to latex2html. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-30 12:35 Message: Logged In: YES user_id=3066 Closed due to lack of a clear requirement. If a clear requirement surfaces this can be re-visited. ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-10-30 07:41 Message: Logged In: YES user_id=36737 okay, fine. :) ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-30 06:53 Message: Logged In: YES user_id=3066 Take a look in the CVS version of Python at how the iSilo version is generated. You'll want to look in Doc/Makefile and Doc/perl/isilo.perl. I'm dropping the priority & marking this pending/later since there's not a clear immediate need. If the iSilo version doesn't show how to do what you need, please explain in detail what the real effect is that you're looking for. (If it does show you what you need, let me know so I can close this report.) ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-10-30 06:30 Message: Logged In: YES user_id=36737 I think was trying to change the navigation links, or even suppress them altogether. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-29 21:29 Message: Logged In: YES user_id=3066 Before I accept this, can you describe what you were trying to achieve that required this? It may be that there's a better way. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=462274&group_id=5470 From noreply@sourceforge.net Tue Oct 30 21:10:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 13:10:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-474077 ] 2.2b1: Error compiling extns with BCC Message-ID: Bugs item #474077, was opened at 2001-10-23 07:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474077&group_id=5470 >Category: Build >Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Luigi Ballabio (lballabio) Assigned to: Tim Peters (tim_one) Summary: 2.2b1: Error compiling extns with BCC Initial Comment: Borland C++ fails to compile extensions for Python 2.2b1. The error message is: D:\Python22\include\pyport.h 350: Only one of a set of overloaded functions can be "C" The solution is to remove line 183 of pyconfig.h, namely, #undef HAVE_HYPOT ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-30 13:10 Message: Logged In: YES user_id=31435 Well, I've removed the line, in Misc/ACKS; new revision: 1.134 PC/pyconfig.h; new revision: 1.5 but can't test it. Do you have any idea why it was there before? Somebody went out of their way to add it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474077&group_id=5470 From noreply@sourceforge.net Tue Oct 30 22:46:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 14:46:07 -0800 Subject: [Python-bugs-list] [ python-Bugs-476557 ] Wrong error message for file.write(a, b) Message-ID: Bugs item #476557, was opened at 2001-10-30 14:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Wrong error message for file.write(a, b) Initial Comment: Passing too many arguments to a file's write() method produces an error message that complains about paasing a tuple rather than more accurately complaining about too many arguments: TypeError: argument must be string or read-only character buffer, not tuple Hamish Lawson ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 From noreply@sourceforge.net Wed Oct 31 01:27:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 17:27:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-476593 ] Bad mode for opening file gives bad msg. Message-ID: Bugs item #476593, was opened at 2001-10-30 17:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476593&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Gregory H. Ball (greg_ball) Assigned to: Nobody/Anonymous (nobody) Summary: Bad mode for opening file gives bad msg. Initial Comment: Calling the open (or file) builtin with an incorrect mode (for example, 'bw' or 's') causes an exception which claims that the filename is invalid. >>> open('tmp','s') Traceback (innermost last): File "", line 1, in ? IOError: [Errno 22] Invalid argument: 'tmp' (double spaced in hopes of preserving line breaks) It comes down to this call: PyErr_SetFromErrnoWithFilename(PyExc_IOError, name); which assumes that the filename is relevant to any IOError that pops up. man fopen on linux says: EINVAL The mode provided to fopen, fdopen, or freopen was invalid. so it seems possible to catch this case, but not without making the code more complex. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476593&group_id=5470 From noreply@sourceforge.net Wed Oct 31 02:57:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 18:57:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-476616 ] UserDict.copy() error Message-ID: Bugs item #476616, was opened at 2001-10-30 18:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476616&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: UserDict.copy() error Initial Comment: The copy() method applied to subclasses of UserDict copies the object, but does not copy the contained dictionary. The dictionary is shared between the original and the copy. The copy method applied to UserDict (not a subclass) works okay. This error goes back to at least 2.0, and still exists in 2.2b1. This little test program demonstrates the error: ---------------------------------------------- from UserDict import UserDict class MyDict (UserDict): pass d = MyDict() d[1] = 11 c = d.copy() c[2] = 22 # this should affect c only, but also affects d print "c:", c print "d:", d --------------------------------------- BTW, subclassing 'dictionary' works fine in 2.2 (of course). For running under 2.0, I worked around this by using copy.deepcopy(). I'm in no hurry for a fix for this. Bob Alexander balexander@rsv.ricoh.com ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476616&group_id=5470 From noreply@sourceforge.net Wed Oct 31 03:32:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 19:32:54 -0800 Subject: [Python-bugs-list] [ python-Bugs-476616 ] UserDict.copy() error Message-ID: Bugs item #476616, was opened at 2001-10-30 18:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476616&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: UserDict.copy() error Initial Comment: The copy() method applied to subclasses of UserDict copies the object, but does not copy the contained dictionary. The dictionary is shared between the original and the copy. The copy method applied to UserDict (not a subclass) works okay. This error goes back to at least 2.0, and still exists in 2.2b1. This little test program demonstrates the error: ---------------------------------------------- from UserDict import UserDict class MyDict (UserDict): pass d = MyDict() d[1] = 11 c = d.copy() c[2] = 22 # this should affect c only, but also affects d print "c:", c print "d:", d --------------------------------------- BTW, subclassing 'dictionary' works fine in 2.2 (of course). For running under 2.0, I worked around this by using copy.deepcopy(). I'm in no hurry for a fix for this. Bob Alexander balexander@rsv.ricoh.com ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-30 19:32 Message: Logged In: YES user_id=6380 Hm, you're right. Sigh. This has been there since 1.5.2. I'm not sure how to fix this -- using deepcopy() is too much (and not what happens if self.__class__ is UserDict either). Maybe this: c = copy.copy(self) c.data = self.data.copy() return c ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476616&group_id=5470 From noreply@sourceforge.net Wed Oct 31 04:22:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 20:22:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-474485 ] pydoc generates some bad html Message-ID: Bugs item #474485, was opened at 2001-10-24 07:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474485&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Rich Salz (rsalz) Assigned to: Tim Peters (tim_one) Summary: pydoc generates some bad html Initial Comment: One bug, two nits. When documenting lambda's, pydoc outputs which is interepreted as HTML markup. I partially fixed that; I didn't add support for A/HREF, which I think is what's really intended. The nits: HTMLtidy says tables should have a summary attribute, and

directly before a

is an empty element which should be removed. Attached patch addresses these. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-30 20:22 Message: Logged In: YES user_id=31435 Thanks, Rich! The patch was applied, in Lib/pydoc.py; new revision: 1.56 Misc/ACKS; new revision: 1.136 If you have more changes you want to make to this, it would be a real help if you could make them against the current CVS version of pydoc; some of the lines I believe you wanted to change were over 100 lines away from where this patch thought they were, and that it made it harder and more error-prone than it should be. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474485&group_id=5470 From noreply@sourceforge.net Wed Oct 31 04:58:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 20:58:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Mark Rowe (icep17) Assigned to: Nobody/Anonymous (nobody) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Wed Oct 31 07:54:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 30 Oct 2001 23:54:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Mark Rowe (icep17) >Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Wed Oct 31 10:31:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 02:31:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-476557 ] Wrong error message for file.write(a, b) Message-ID: Bugs item #476557, was opened at 2001-10-30 14:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Wrong error message for file.write(a, b) Initial Comment: Passing too many arguments to a file's write() method produces an error message that complains about paasing a tuple rather than more accurately complaining about too many arguments: TypeError: argument must be string or read-only character buffer, not tuple Hamish Lawson ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-10-31 02:31 Message: Logged In: YES user_id=6656 I think this is easy. Can someone have a look at the attached patch while make test is running? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 From noreply@sourceforge.net Wed Oct 31 12:15:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 04:15:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-466907 ] OSX Python cannot run in console Message-ID: Bugs item #466907, was opened at 2001-10-01 10:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 Category: Macintosh Group: Python 2.2 >Status: Closed Resolution: None Priority: 5 Submitted By: Jonathan Wight (schwa) Assigned to: Jack Jansen (jackjansen) Summary: OSX Python cannot run in console Initial Comment: OS: Mac OS X 10.1 HW: Verified on G4/733 & PowerBook G4/400Version: 'Unix' Python 2.2a4 (also seen on 2.2a3) Configured: --with-suffix=.exe --without-toolbox-glue Python cannot run on a Mac OS X machine when the user is logged into console mode (using ">console" as login name). Download .tar.gz, configure python as shown, make, make install. Log out of Mac OS X. Log in as console user: use ">console" as login name. Run python interactively from the shell... Python will fail to run and emit these errors: KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorFailure: CGSNewConnection cannot connect to server KCGErrorInvalidConnection: CGSGetEventPort: Invalid connection I believe Python is trying to connect to the CoreGraphics server. Why? I don't know (esp. as I've NOT compiled the toolbox glue). Perhaps Python is linking to a Carbon library and the library is automatically trying to open a connection. My makefile skills are not good enough to enable me to check. Python compiled on OS X without the toolbox glue should essentially be identical to any other 'unix' app compiled for Mac OS X. Doesn't seem to be that way. ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-10-31 04:15 Message: Logged In: YES user_id=45365 A fix has been checked in to the repository for this. The Python core library now links against CoreServices in stead of Carbon. This should fix four related problems: - Python doesn't run in ">console" logins - Python doesn't run on OS X Server - Python doesn't run in ssh logins - Python crashes when run in two terminal windows at the same time. I haven't been able to verify that all these cases are now okay, but I'm 99% sure. Please reopen the bug if there are still problems. ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-10-26 05:49 Message: Logged In: YES user_id=45365 Just a quick update so people know I haven't forgotten this bug. It turns out that Apple changed the Carbon.framework initialization: it used to be lazy, initializing carbon portions as needed, but since 10.1 a lot more is initialized up front, and therefore the window server is contacted. Therefore, as of 10.1, if you are not a windowing program you cannot link against Carbon.framework anymore but must link against some of the subordinate frameworks. This was always true theoretically (it seems), but now it has suddenly become practically true as well. But: it seems more people are bitten by this, so Apple suggested they may revert on this decision. And on the other hand I'm going to have a look at not linking with Carbon.framework but with CoreServices and/or ApplicationServices in stead. ---------------------------------------------------------------------- Comment By: Jonathan Wight (schwa) Date: 2001-10-01 15:17 Message: Logged In: YES user_id=29309 OK... Another Follow Up. Checked the configure.in file and there is no --without-toolbox-glue switch for configure. There is a --disable-toolbox-glue and --enable-toolbox-glue. The line in the README that states you should use --without-toolbox-glue is incorrect. This is a bug in the documentation! (easy to fix though). I've checked the newly built executable (using --disable-toolbox-glue) with 'otool -l' and the linkage dependencies are gone. This is good. Unfortunately when the 'make install' stage of the build fails with: skipping 'gestalt' extension (up-to-date) dyld: ./python.exe Undefined symbols: _Gestalt _PyMac_Error make: *** [sharedmods] Error 67 ---------------------------------------------------------------------- Comment By: Jonathan Wight (schwa) Date: 2001-10-01 14:33 Message: Logged In: YES user_id=29309 FYI. Running 'otool -l' on the Python executable shows that it is being linked to both: /System/Library/Frameworks/Carbon.framework and /System/Library/Frameworks/Foundation.framework Carbon.framework links in /System/Library/Frameworks/ApplicationServices.framework/ which contains the CoreGraphics framework. I'm assuming that causes CoreGraphics to be initialized (guessing). Why does a plain 'unix' build of Python link to both Carbon and Foundation? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=466907&group_id=5470 From noreply@sourceforge.net Wed Oct 31 12:16:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 04:16:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-475325 ] Frozen packages broken on the Mac Message-ID: Bugs item #475325, was opened at 2001-10-26 08:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475325&group_id=5470 Category: Macintosh Group: None >Status: Closed Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Jack Jansen (jackjansen) Summary: Frozen packages broken on the Mac Initial Comment: Patch #416704 broke the freezing of packages on the Mac. The patch requires that submodules of frozen packages are also frozen, but disregards the fact that freezing is done completely different on the Mac. The code in import.c:find_module() now takes an early exit if we're importing from a package and find_frozen() returns false, so the code that locates MacPython PYC and PYD resource modules never gets executed. I need to investigate whether the correct solution is to duplicate that code after find_module() returns (but it's rather a big lump) or not take the exit in MacPython (but that would leave MacPython open to the problems the patch tries to address). ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-10-31 04:16 Message: Logged In: YES user_id=45365 Fixed in the repository, by using the old code flow on the Mac. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475325&group_id=5470 From noreply@sourceforge.net Wed Oct 31 17:43:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 09:43:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-476557 ] Wrong error message for file.write(a, b) Message-ID: Bugs item #476557, was opened at 2001-10-30 14:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 Category: Python Interpreter Core Group: None Status: Open >Resolution: Accepted Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Michael Hudson (mwh) Summary: Wrong error message for file.write(a, b) Initial Comment: Passing too many arguments to a file's write() method produces an error message that complains about paasing a tuple rather than more accurately complaining about too many arguments: TypeError: argument must be string or read-only character buffer, not tuple Hamish Lawson ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-31 09:43 Message: Logged In: YES user_id=6380 Michael, the patch looks good, you can apply it yourself. (How did you manage to get the patch space-filled where the source code has only tabs?) ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-10-31 02:31 Message: Logged In: YES user_id=6656 I think this is easy. Can someone have a look at the attached patch while make test is running? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 From noreply@sourceforge.net Wed Oct 31 18:01:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 10:01:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-476852 ] Some bad macros in abstract.h Message-ID: Bugs item #476852, was opened at 2001-10-31 10:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476852&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: Some bad macros in abstract.h Initial Comment: Greg Chapman reported in c.l.py: abstract.h defines the following: #define PyMapping_DelItemString(O,K) PyDict_DelItemString((O),(K)) #define PyMapping_DelItem(O,K) PyDict_DelItem((O),(K)) It seems to me they should delegate to PyObject_DelItem (and PyObject_DelItemString, which will have to be added). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476852&group_id=5470 From noreply@sourceforge.net Wed Oct 31 18:07:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 10:07:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-476858 ] Assignment to () should be legal Message-ID: Bugs item #476858, was opened at 2001-10-31 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476858&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: Assignment to () should be legal Initial Comment: >From c.l.py: Currently, () = x gives a compile-time error. This should really be allowed (and require that x is an empty sequence, of course) as an end case of (a,b,c) = x # x must be a 3-sequence (a,b) = x # x must be a 2-sequence (a,) = x # x must be a 1-sequence () = x # why can't x be z 0-sequence? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476858&group_id=5470 From noreply@sourceforge.net Wed Oct 31 18:37:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 10:37:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-476858 ] Assignment to () should be legal Message-ID: Bugs item #476858, was opened at 2001-10-31 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476858&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: Assignment to () should be legal Initial Comment: >From c.l.py: Currently, () = x gives a compile-time error. This should really be allowed (and require that x is an empty sequence, of course) as an end case of (a,b,c) = x # x must be a 3-sequence (a,b) = x # x must be a 2-sequence (a,) = x # x must be a 1-sequence () = x # why can't x be z 0-sequence? ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-31 10:37 Message: Logged In: YES user_id=31435 -1. "Assignment statements are used to (re)bind names to values and to modify attributes or items of mutable objects" (from the Ref Man). Since the degenerate cases (don't forget "[] = x" too) don't do that, they're not "assignment statements" in a meanignful sense; they would just be a surprising way to spell if tuple(x): raise ValueError That isn't a frequent enough need to deserve special syntax. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476858&group_id=5470 From noreply@sourceforge.net Wed Oct 31 18:51:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 10:51:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-476557 ] Wrong error message for file.write(a, b) Message-ID: Bugs item #476557, was opened at 2001-10-30 14:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Michael Hudson (mwh) Summary: Wrong error message for file.write(a, b) Initial Comment: Passing too many arguments to a file's write() method produces an error message that complains about paasing a tuple rather than more accurately complaining about too many arguments: TypeError: argument must be string or read-only character buffer, not tuple Hamish Lawson ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-10-31 10:51 Message: Logged In: YES user_id=6656 OK, done in fileobject.c revision 2.136. Re space/tabs: submitting patches is a multi-OS multi-machine adventure for me at the moment. You probably don't want to know the details. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-31 09:43 Message: Logged In: YES user_id=6380 Michael, the patch looks good, you can apply it yourself. (How did you manage to get the patch space-filled where the source code has only tabs?) ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-10-31 02:31 Message: Logged In: YES user_id=6656 I think this is easy. Can someone have a look at the attached patch while make test is running? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476557&group_id=5470 From noreply@sourceforge.net Wed Oct 31 18:56:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 10:56:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-476884 ] Configparser line continuations fail Message-ID: Bugs item #476884, was opened at 2001-10-31 10:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476884&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Configparser line continuations fail Initial Comment: Configparser line continuations fail with a key error when the option is of mixed case. Configparser maps all options to lowercase (presumably for Win compatibility) but it does the mapping through a method call (optionxform) which leaves the option name (optname) untransformed. This causes a dictionary key error when continuation lines are encountered since the untransformed optname is used to build the continued line. There are at least three possible fixes: 1. Make options case sensitive by eliminating the optionxform method. (My workaround is to overload the optionsxform method...) 2. Replace optname by it's lower case equivalent when it is constructed and use the lower case form throughout. 3. Insert calls to optionxform where the continuation lines are built. Messy and inefficient, IMHO. For greatest generality, there ought to be parameters which manage case sensitivity for section names and option names. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476884&group_id=5470 From noreply@sourceforge.net Wed Oct 31 19:09:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 11:09:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-435455 ] Python 2.0.1c1 fails to build on RH7.1 Message-ID: Bugs item #435455, was opened at 2001-06-22 06:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=435455&group_id=5470 Category: Distutils Group: Platform-specific Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Ole H. Nielsen (ohnielse) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.0.1c1 fails to build on RH7.1 Initial Comment: Building Python 2.0.1c1 on a RedHat 7.1 (2.4.2-2 on i586) fails at this point: cd Modules; make OPT="-g -O2 -Wall -Wstrict-prototypes -fPIC" VERSION="2.0" \ prefix="/usr/local" exec_prefix="/usr/local" \ sharedmods make[1]: Entering directory `/scratch/ohnielse/Python-2.0.1/Modules' gcc -fpic -g -O2 -Wall -Wstrict-prototypes -fPIC -I./../Include -I.. -DHAVE_CONFIG_H -c ./bsddbmodule.c ./bsddbmodule.c: In function `newdbhashobject': ./bsddbmodule.c:55: `HASHINFO' undeclared (first use in this function) ./bsddbmodule.c:55: (Each undeclared identifier is reported only once ./bsddbmodule.c:55: for each function it appears in.) ./bsddbmodule.c:55: parse error before `info' ./bsddbmodule.c:60: `info' undeclared (first use in this function) ./bsddbmodule.c:71: warning: implicit declaration of function `dbopen' ./bsddbmodule.c:71: warning: assignment makes pointer from integer without a cast ./bsddbmodule.c: In function `newdbbtobject': ./bsddbmodule.c:100: `BTREEINFO' undeclared (first use in this function) ./bsddbmodule.c:100: parse error before `info' ./bsddbmodule.c:105: `info' undeclared (first use in this function) ./bsddbmodule.c:118: warning: assignment makes pointer from integer without a cast ./bsddbmodule.c: In function `newdbrnobject': ./bsddbmodule.c:147: `RECNOINFO' undeclared (first use in this function) ./bsddbmodule.c:147: parse error before `info' ./bsddbmodule.c:152: `info' undeclared (first use in this function) ./bsddbmodule.c:164: warning: assignment makes pointer from integer without a cast ./bsddbmodule.c: In function `bsddb_dealloc': ./bsddbmodule.c:202: too few arguments to function ./bsddbmodule.c: In function `bsddb_length': ./bsddbmodule.c:232: structure has no member named `seq' ./bsddbmodule.c:233: `R_FIRST' undeclared (first use in this function) ./bsddbmodule.c:235: structure has no member named `seq' ./bsddbmodule.c:236: `R_NEXT' undeclared (first use in this function) ./bsddbmodule.c:229: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_subscript': ./bsddbmodule.c:265: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:265: too few arguments to function ./bsddbmodule.c: In function `bsddb_ass_sub': ./bsddbmodule.c:307: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:307: too few arguments to function ./bsddbmodule.c:330: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:330: too few arguments to function ./bsddbmodule.c: In function `bsddb_close': ./bsddbmodule.c:357: too few arguments to function ./bsddbmodule.c: In function `bsddb_keys': ./bsddbmodule.c:386: structure has no member named `seq' ./bsddbmodule.c:386: `R_FIRST' undeclared (first use in this function) ./bsddbmodule.c:407: structure has no member named `seq' ./bsddbmodule.c:407: `R_NEXT' undeclared (first use in this function) ./bsddbmodule.c:376: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_has_key': ./bsddbmodule.c:440: warning: passing arg 2 of pointer to function from incompatible pointer type ./bsddbmodule.c:440: too few arguments to function ./bsddbmodule.c: In function `bsddb_set_location': ./bsddbmodule.c:466: structure has no member named `seq' ./bsddbmodule.c:466: `R_CURSOR' undeclared (first use in this function) ./bsddbmodule.c:453: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_seq': ./bsddbmodule.c:503: structure has no member named `seq' ./bsddbmodule.c:489: warning: `status' might be used uninitialized in this function ./bsddbmodule.c: In function `bsddb_next': ./bsddbmodule.c:531: `R_NEXT' undeclared (first use in this function) ./bsddbmodule.c: In function `bsddb_previous': ./bsddbmodule.c:536: `R_PREV' undeclared (first use in this function) ./bsddbmodule.c: In function `bsddb_first': ./bsddbmodule.c:541: `R_FIRST' undeclared (first use in this function) ./bsddbmodule.c: In function `bsddb_last': ./bsddbmodule.c:546: `R_LAST' undeclared (first use in this function) make[1]: *** [bsddbmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Erik Andersén (erik_andersen) Date: 2001-10-31 11:09 Message: Logged In: YES user_id=364358 In Modules/bsddbmodule.c change #ifdef HAVE_DB_185_H #include #else #include #endif to #include Erik Andersen ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-01 03:17 Message: Logged In: YES user_id=21627 Since it is unlikely that we'll get further analysis on the cause of this problem, I'm closing the report now. Please try one of the 2.2 prereleases; if problems remain, please report them in a new bug report. ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-09-06 05:25 Message: Logged In: YES user_id=34209 Okay, unassigning. Why I had it ? unsure. It probably dates back from the time I still had a RedHat box. I swapped them all with Debian and I live a much happier life as a result :-) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:25 Message: Logged In: YES user_id=6380 Can someone see if this is still a problem with current CVS (or even with 2.2a2)? I believe we've added another layer of patchwork around the bsddb compilation in setup.py. Why does Thomas own this item? Thomas, if you don't want it, feel free to unassign it. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2001-07-10 01:30 Message: Logged In: YES user_id=43607 I have no trouble building the current CVS version on RedHat 7.1. However, when I try the gcc command given in the error message, I get much the same errors. The deciding difference was: setup.py (part of the standard current build process) adds -DHAVE_DB_185_H=1 to the compiler command line. ---------------------------------------------------------------------- Comment By: Ole H. Nielsen (ohnielse) Date: 2001-07-09 07:51 Message: Logged In: YES user_id=27232 twouters wrote: > I suspect the problem here is that you don't have the > appropriate -devel RPM installed. I think you need > 'db3-devel', but 'db2-devel', 'db1-devel' or 'db-devel' > might work too. Sorry, no. These RPMs seem to be installed by default on RedHat 7.1: # rpm -qa | grep devel | grep db db1-devel-1.85-5 db2-devel-2.4.14-5 gdbm-devel-1.8.0-5 db3-devel-3.1.17-7 It would be nice if a Python developer could gain access to a RedHat 7.1 machine to sort this problem out :-) ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2001-07-09 07:06 Message: Logged In: YES user_id=34209 I suspect the problem here is that you don't have the appropriate -devel RPM installed. I think you need 'db3-devel', but 'db2-devel', 'db1-devel' or 'db-devel' might work too. Can you confirm that you have none of those installed ? I think what happens is that distutils can't find an include header to include, so it assumes the DB implementation is included in libc, instead. (Changing catagory to distutils on that assumption.) ---------------------------------------------------------------------- Comment By: Ole H. Nielsen (ohnielse) Date: 2001-06-26 14:52 Message: Logged In: YES user_id=27232 loewis wrote: > I have asked you to report things, not to change things. > It is not easy to repeat for me, as I don't have Redhat 7.1. Sorry about the confusion. My report is for RedHat 7.1: 1. Untar the distribution 2.0.1c1 2. ./configure 3. make The errors reported occur. I have changed nothing. I don't know what bsddb is, nor why it's getting compiled. Hopefully, someone knowing Python 2.0.1 and having access to a RedHat 7.1 box could look into the problem. At this point Python building seems broken on RH7.1 :-( ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-06-26 00:02 Message: Logged In: YES user_id=21627 I have asked you to report things, not to change things. It is not easy to repeat for me, as I don't have Redhat 7.1. ---------------------------------------------------------------------- Comment By: Ole H. Nielsen (ohnielse) Date: 2001-06-25 15:20 Message: Logged In: YES user_id=27232 loewis wrote: > Please report the following things: > - the line in Setup that you activated to enable > compilation of bsddb > - the exact version of the bsddb RPM package that provides > db.h > - whether or not this packages includes a file db_185.h Sorry, I didn't change ANYTHING ! I was trying a vanilla build on RedHat 7.1 ! Should be easy to repeat... Maybe the build scripts make some incorrect choices on RedHat 7.1 ? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-06-23 13:58 Message: Logged In: YES user_id=21627 Please report the following things: - the line in Setup that you activated to enable compilation of bsddb - the exact version of the bsddb RPM package that provides db.h - whether or not this packages includes a file db_185.h ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=435455&group_id=5470 From noreply@sourceforge.net Wed Oct 31 19:32:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 11:32:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-476898 ] Word "separate" spelled incorrectly Message-ID: Bugs item #476898, was opened at 2001-10-31 11:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: David J. Ritchie (r124c41) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Word "separate" spelled incorrectly Initial Comment: The word "separate" is spelled incorrectly in Section 1.6.1 of "Macintosh Library Modules". See "file in a seperate script window" in: http://www.python.org/doc/current/mac/node15.html I have not looked at other sections of the document. It may be elsewhere in the document as well. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 From noreply@sourceforge.net Wed Oct 31 19:35:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 11:35:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-476898 ] Word "separate" spelled incorrectly Message-ID: Bugs item #476898, was opened at 2001-10-31 11:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None >Priority: 1 Submitted By: David J. Ritchie (r124c41) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Word "separate" spelled incorrectly Initial Comment: The word "separate" is spelled incorrectly in Section 1.6.1 of "Macintosh Library Modules". See "file in a seperate script window" in: http://www.python.org/doc/current/mac/node15.html I have not looked at other sections of the document. It may be elsewhere in the document as well. ---------------------------------------------------------------------- >Comment By: David J. Ritchie (r124c41) Date: 2001-10-31 11:35 Message: Logged In: YES user_id=364368 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 From noreply@sourceforge.net Wed Oct 31 19:43:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 11:43:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-476904 ] Unquittable applets Message-ID: Bugs item #476904, was opened at 2001-10-31 11:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476904&group_id=5470 Category: Macintosh Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Just van Rossum (jvr) Assigned to: Jack Jansen (jackjansen) Summary: Unquittable applets Initial Comment: MacPython applets which have the "disable console until needed" option set cannot quit upon error exit because there is no quit menu (not even a file menu...). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476904&group_id=5470 From noreply@sourceforge.net Wed Oct 31 20:17:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 12:17:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-476912 ] regex annoyance Message-ID: Bugs item #476912, was opened at 2001-10-31 12:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Bill Bumgarner (bbum) Assigned to: Nobody/Anonymous (nobody) Summary: regex annoyance Initial Comment: (this may be a feature request-- but it is annoying enough that I filed it as a bug) Python's named sub expressions within regular expressions are an incredibly valuable feature; between it and the ability to automatically collapse multiline regex's w/comments leads to very readable regex's. However, there is an annoyance in named subexpressions that has bitten me several times. Namely, if you have a situation where a particular token must be parsed out of the input through the use of one of two (or more) expressions in a fashion that cannot be expressed without multiple possible means of matching any given subexpression, then the named subexpression will only be non-None intermittently (depending on expression order and what was matched). That is, given: (?:(?[a-z]+)\s(?[a-z]+))|(?:(? [a-z]+)\t(?[a-z]+)) In this case, Tok1 and Tok2 will be None if the first expression matches... (Yes, this is a contrived example that could be refactored to not use multiple / references-- however, more complex expressions do not always enable easy refactoring.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 From noreply@sourceforge.net Wed Oct 31 20:39:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 12:39:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-476858 ] Assignment to () should be legal Message-ID: Bugs item #476858, was opened at 2001-10-31 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476858&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: Assignment to () should be legal Initial Comment: >From c.l.py: Currently, () = x gives a compile-time error. This should really be allowed (and require that x is an empty sequence, of course) as an end case of (a,b,c) = x # x must be a 3-sequence (a,b) = x # x must be a 2-sequence (a,) = x # x must be a 1-sequence () = x # why can't x be z 0-sequence? ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-10-31 12:39 Message: Logged In: YES user_id=44345 I won't try and counter Tim's vote (besides, he's already got Guido's 51% vote to contend with), but just provide an example used in c.l.py. You might have a function that can return a possibly-empty tuple as part of a larger sequence, e.g.: def contrived(a, *args): return (len(args), args*a) (foo, (bar,)) = contrived(1,2,3) (foo, ()) = contrived(0,1,2) Some folks view this as useful. I reserve judgement on that. ;-) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 10:37 Message: Logged In: YES user_id=31435 -1. "Assignment statements are used to (re)bind names to values and to modify attributes or items of mutable objects" (from the Ref Man). Since the degenerate cases (don't forget "[] = x" too) don't do that, they're not "assignment statements" in a meanignful sense; they would just be a surprising way to spell if tuple(x): raise ValueError That isn't a frequent enough need to deserve special syntax. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476858&group_id=5470 From noreply@sourceforge.net Wed Oct 31 21:09:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 13:09:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-476884 ] Configparser line continuations fail Message-ID: Bugs item #476884, was opened at 2001-10-31 10:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476884&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Configparser line continuations fail Initial Comment: Configparser line continuations fail with a key error when the option is of mixed case. Configparser maps all options to lowercase (presumably for Win compatibility) but it does the mapping through a method call (optionxform) which leaves the option name (optname) untransformed. This causes a dictionary key error when continuation lines are encountered since the untransformed optname is used to build the continued line. There are at least three possible fixes: 1. Make options case sensitive by eliminating the optionxform method. (My workaround is to overload the optionsxform method...) 2. Replace optname by it's lower case equivalent when it is constructed and use the lower case form throughout. 3. Insert calls to optionxform where the continuation lines are built. Messy and inefficient, IMHO. For greatest generality, there ought to be parameters which manage case sensitivity for section names and option names. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-31 13:09 Message: Logged In: YES user_id=6380 Assigned to Fred. Fred, didn't you dabble some in ConfigParser? You might want to have a look at some of the other open ConfigParser bugs too. Or you can pass this off to someone who knows more about it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476884&group_id=5470 From noreply@sourceforge.net Wed Oct 31 23:10:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 15:10:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-476129 ] gc.collect sometimes hangs Message-ID: Bugs item #476129, was opened at 2001-10-29 12:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Mitch Chapman (mitchchapman) Assigned to: Neil Schemenauer (nascheme) Summary: gc.collect sometimes hangs Initial Comment: When run under Python 2.1 on Mandrake Linux 7.2 (kernel 2.2.17) the attached script terminates normally. When run on the same machine under Python 2.2b1 it runs forever, printing an infinite sequence of "No such attribute __del__" messages to stdout. If you comment out the call to gc.disable in Container.refresh, the script terminates normally under both versions of the interpreter. ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2001-10-31 15:10 Message: Logged In: YES user_id=35752 Closing because I'm quite sure the 2.26 change to gcmodule.c fixed this problem. ---------------------------------------------------------------------- Comment By: Neil Schemenauer (nascheme) Date: 2001-10-29 15:45 Message: Logged In: YES user_id=35752 I think the bug is that gc.collect() does not do locking. The collector can start running twice. That can corrupt the GC linked lists. Please try the attached patch with 2.2. I think this bug is in 2.1 as well. I'd check the patch in now but SF CVS seems to be broken. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 13:02 Message: Logged In: YES user_id=6380 Assigning to Neil Schemenauer, our GC expert. Note that in 2.1, I get a whole bunch of messages "No such attribute __del__" to stdout too -- but it stops. In 2.2, the *first* call to c=Container() and c.delegate() does the same thing. But the *second* call to Container() causes an infinite number of messages to print (even if it's not deleting the first c instance). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476129&group_id=5470