From patches@python.org Fri Feb 4 20:14:18 2000 From: patches@python.org (Guido van Rossum) Date: Fri, 04 Feb 2000 15:14:18 -0500 Subject: [Patches] PATCH: win32 dll support for modules In-Reply-To: Your message of "Fri, 04 Feb 2000 22:05:17 +0200." <1920.000204@technologist.com> References: <1920.000204@technologist.com> Message-ID: <200002042014.PAA15127@eric.cnri.reston.va.us> Hi Paul, Sorry to be a pest. I'm glad to see a patch so soon! I have two requests: (1) please resend your patches with the disclaimer from http://www.python.org/1.5/bugrelease.html (2) please use patches@python.org -- you couldn't know this yet, but we're just starting to use a different process for submitting patches --Guido van Rossum (home page: http://www.python.org/~guido/) From Paul Sokolovsky Fri Feb 4 20:17:44 2000 From: Paul Sokolovsky (Paul Sokolovsky) Date: Fri, 4 Feb 2000 22:17:44 +0200 Subject: [Patches] PATCH: win32 dll support for modules Message-ID: <14928.000204@technologist.com> ------------6F140246218DD9 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hello patches, Attached is patch (against 1.5.2 release) to allow some modules to be buildable as pyd's (usual &PyType_Type stuff). Other little is to make little more room for compiler's name as returned by Py_GetVersion(): latest egcs/gcc have quite long names and I guess it's better have it long than cut in the middle. DISCLAIMER I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Best regards, Paul mailto:Paul.Sokolovsky@technologist.com ------------6F140246218DD9 Content-Type: application/octet-stream; name="pymodules.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="pymodules.diff" ZGlmZiAtYyAtciAtTiBQeXRob24tMS41LjItb3JnL01vZHVsZXMvYXJyYXltb2R1bGUuYyBQeXRo b24tMS41LjIvTW9kdWxlcy9hcnJheW1vZHVsZS5jDQoqKiogUHl0aG9uLTEuNS4yLW9yZy9Nb2R1 bGVzL2FycmF5bW9kdWxlLmMJU2F0IE1hciAyMCAwMDo1MDoxMCAxOTk5DQotLS0gUHl0aG9uLTEu NS4yL01vZHVsZXMvYXJyYXltb2R1bGUuYwlXZWQgSnVsIDIxIDEzOjU5OjAwIDE5OTkNCioqKioq KioqKioqKioqKg0KKioqIDE0NDEsMTQ0NyAqKioqDQogICI7DQogIA0KICBzdGF0aWNoZXJlIFB5 VHlwZU9iamVjdCBBcnJheXR5cGUgPSB7DQohIAlQeU9iamVjdF9IRUFEX0lOSVQoJlB5VHlwZV9U eXBlKQ0KICAJMCwNCiAgCSJhcnJheSIsDQogIAlzaXplb2YoYXJyYXlvYmplY3QpLA0KLS0tIDE0 NDEsMTQ0NyAtLS0tDQogICI7DQogIA0KICBzdGF0aWNoZXJlIFB5VHlwZU9iamVjdCBBcnJheXR5 cGUgPSB7DQohIAlQeU9iamVjdF9IRUFEX0lOSVQoTlVMTCkNCiAgCTAsDQogIAkiYXJyYXkiLA0K ICAJc2l6ZW9mKGFycmF5b2JqZWN0KSwNCioqKioqKioqKioqKioqKg0KKioqIDE0NjksMTQ3NCAq KioqDQotLS0gMTQ2OSwxNDc1IC0tLS0NCiAgaW5pdGFycmF5KCkNCiAgew0KICAJUHlPYmplY3Qg Km0sICpkOw0KKyAgICAgICAgIEFycmF5dHlwZS5vYl90eXBlPSZQeVR5cGVfVHlwZTsNCiAgCW0g PSBQeV9Jbml0TW9kdWxlMygiYXJyYXkiLCBhX21ldGhvZHMsIG1vZHVsZV9kb2MpOw0KICAJZCA9 IFB5TW9kdWxlX0dldERpY3QobSk7DQogIAlQeURpY3RfU2V0SXRlbVN0cmluZyhkLCAiQXJyYXlU eXBlIiwgKFB5T2JqZWN0ICopJkFycmF5dHlwZSk7DQpkaWZmIC1jIC1yIC1OIFB5dGhvbi0xLjUu Mi1vcmcvTW9kdWxlcy9tZDVtb2R1bGUuYyBQeXRob24tMS41LjIvTW9kdWxlcy9tZDVtb2R1bGUu Yw0KKioqIFB5dGhvbi0xLjUuMi1vcmcvTW9kdWxlcy9tZDVtb2R1bGUuYwlGcmkgRGVjICA0IDIx OjQ5OjUyIDE5OTgNCi0tLSBQeXRob24tMS41LjIvTW9kdWxlcy9tZDVtb2R1bGUuYwlXZWQgSnVs IDIxIDE0OjQ3OjUwIDE5OTkNCioqKioqKioqKioqKioqKg0KKioqIDIwMCwyMDYgKioqKg0KICAi Ow0KICANCiAgc3RhdGljaGVyZSBQeVR5cGVPYmplY3QgTUQ1dHlwZSA9IHsNCiEgCVB5T2JqZWN0 X0hFQURfSU5JVCgmUHlUeXBlX1R5cGUpDQogIAkwLAkJCSAgLypvYl9zaXplKi8NCiAgCSJtZDUi LAkJCSAgLyp0cF9uYW1lKi8NCiAgCXNpemVvZihtZDVvYmplY3QpLAkgIC8qdHBfc2l6ZSovDQot LS0gMjAwLDIwNiAtLS0tDQogICI7DQogIA0KICBzdGF0aWNoZXJlIFB5VHlwZU9iamVjdCBNRDV0 eXBlID0gew0KISAJUHlPYmplY3RfSEVBRF9JTklUKE5VTEwpDQogIAkwLAkJCSAgLypvYl9zaXpl Ki8NCiAgCSJtZDUiLAkJCSAgLyp0cF9uYW1lKi8NCiAgCXNpemVvZihtZDVvYmplY3QpLAkgIC8q dHBfc2l6ZSovDQoqKioqKioqKioqKioqKioNCioqKiAyNzEsMjc2ICoqKioNCi0tLSAyNzEsMjc3 IC0tLS0NCiAgaW5pdG1kNSgpDQogIHsNCiAgCVB5T2JqZWN0ICptLCAqZDsNCisgICAgICAgICBN RDV0eXBlLm9iX3R5cGU9JlB5VHlwZV9UeXBlOw0KICAJbSA9IFB5X0luaXRNb2R1bGUzKCJtZDUi LCBtZDVfZnVuY3Rpb25zLCBtb2R1bGVfZG9jKTsNCiAgCWQgPSBQeU1vZHVsZV9HZXREaWN0KG0p Ow0KICAJUHlEaWN0X1NldEl0ZW1TdHJpbmcoZCwgIk1ENVR5cGUiLCAoUHlPYmplY3QgKikmTUQ1 dHlwZSk7DQpkaWZmIC1jIC1yIC1OIFB5dGhvbi0xLjUuMi1vcmcvTW9kdWxlcy9wY3JlbW9kdWxl LmMgUHl0aG9uLTEuNS4yL01vZHVsZXMvcGNyZW1vZHVsZS5jDQoqKiogUHl0aG9uLTEuNS4yLW9y Zy9Nb2R1bGVzL3BjcmVtb2R1bGUuYwlNb24gRmViICAxIDIwOjA5OjAwIDE5OTkNCi0tLSBQeXRo b24tMS41LjIvTW9kdWxlcy9wY3JlbW9kdWxlLmMJV2VkIEp1bCAyMSAxNDozMjoyNiAxOTk5DQoq KioqKioqKioqKioqKioNCioqKiAxNjUsMTcxICoqKioNCiAgDQogIA0KICBzdGF0aWNmb3J3YXJk IFB5VHlwZU9iamVjdCBQY3JlX1R5cGUgPSB7DQohIAlQeU9iamVjdF9IRUFEX0lOSVQoJlB5VHlw ZV9UeXBlKQ0KICAJMCwJCQkvKm9iX3NpemUqLw0KICAJIlBjcmUiLAkJCS8qdHBfbmFtZSovDQog IAlzaXplb2YoUGNyZU9iamVjdCksCS8qdHBfYmFzaWNzaXplKi8NCi0tLSAxNjUsMTcxIC0tLS0N CiAgDQogIA0KICBzdGF0aWNmb3J3YXJkIFB5VHlwZU9iamVjdCBQY3JlX1R5cGUgPSB7DQohIAlQ eU9iamVjdF9IRUFEX0lOSVQoTlVMTCkNCiAgCTAsCQkJLypvYl9zaXplKi8NCiAgCSJQY3JlIiwJ CQkvKnRwX25hbWUqLw0KICAJc2l6ZW9mKFBjcmVPYmplY3QpLAkvKnRwX2Jhc2ljc2l6ZSovDQoq KioqKioqKioqKioqKioNCioqKiA2NjQsNjcwICoqKioNCiAgew0KICAJUHlPYmplY3QgKm0sICpk Ow0KICANCiEgCS8qIENyZWF0ZSB0aGUgbW9kdWxlIGFuZCBhZGQgdGhlIGZ1bmN0aW9ucyAqLw0K ICAJbSA9IFB5X0luaXRNb2R1bGUoInBjcmUiLCBwY3JlX21ldGhvZHMpOw0KICANCiAgCS8qIEFk ZCBzb21lIHN5bWJvbGljIGNvbnN0YW50cyB0byB0aGUgbW9kdWxlICovDQotLS0gNjY0LDY3MiAt LS0tDQogIHsNCiAgCVB5T2JqZWN0ICptLCAqZDsNCiAgDQohICAgICAgICAgUGNyZV9UeXBlLm9i X3R5cGU9JlB5VHlwZV9UeXBlOw0KISANCiEgICAgICAgICAvKiBDcmVhdGUgdGhlIG1vZHVsZSBh bmQgYWRkIHRoZSBmdW5jdGlvbnMgKi8NCiAgCW0gPSBQeV9Jbml0TW9kdWxlKCJwY3JlIiwgcGNy ZV9tZXRob2RzKTsNCiAgDQogIAkvKiBBZGQgc29tZSBzeW1ib2xpYyBjb25zdGFudHMgdG8gdGhl IG1vZHVsZSAqLw0KZGlmZiAtYyAtciAtTiBQeXRob24tMS41LjItb3JnL1B5dGhvbi9nZXR2ZXJz aW9uLmMgUHl0aG9uLTEuNS4yL1B5dGhvbi9nZXR2ZXJzaW9uLmMNCioqKiBQeXRob24tMS41LjIt b3JnL1B5dGhvbi9nZXR2ZXJzaW9uLmMJV2VkIEphbiAyNyAyMDo1MzowOCAxOTk5DQotLS0gUHl0 aG9uLTEuNS4yL1B5dGhvbi9nZXR2ZXJzaW9uLmMJV2VkIEp1bCAyMSAxNTo0Njo1MCAxOTk5DQoq KioqKioqKioqKioqKioNCioqKiAzOSw0NSAqKioqDQogIFB5X0dldFZlcnNpb24oKQ0KICB7DQog IAlzdGF0aWMgY2hhciB2ZXJzaW9uWzEwMF07DQohIAlzcHJpbnRmKHZlcnNpb24sICIlLjEwcyAo JS40MHMpICUuNDBzIiwgUFlfVkVSU0lPTiwNCiAgCQlQeV9HZXRCdWlsZEluZm8oKSwgUHlfR2V0 Q29tcGlsZXIoKSk7DQogIAlyZXR1cm4gdmVyc2lvbjsNCiAgfQ0KLS0tIDM5LDQ1IC0tLS0NCiAg UHlfR2V0VmVyc2lvbigpDQogIHsNCiAgCXN0YXRpYyBjaGFyIHZlcnNpb25bMTAwXTsNCiEgCXNw cmludGYodmVyc2lvbiwgIiUuMTBzICglLjQwcykgJS42MHMiLCBQWV9WRVJTSU9OLA0KICAJCVB5 X0dldEJ1aWxkSW5mbygpLCBQeV9HZXRDb21waWxlcigpKTsNCiAgCXJldHVybiB2ZXJzaW9uOw0K ICB9DQo= ------------6F140246218DD9-- From guido@python.org Fri Feb 4 20:34:42 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 04 Feb 2000 15:34:42 -0500 Subject: [Patches] Re: [Python-checkins] CVS: python/dist/src/Modules md5module.c,2.12,2.13 In-Reply-To: Your message of "Fri, 04 Feb 2000 15:32:37 EST." <200002042032.PAA12317@weyr.cnri.reston.va.us> References: <200002042032.PAA12317@weyr.cnri.reston.va.us> Message-ID: <200002042034.PAA15279@eric.cnri.reston.va.us> Fred, notice this little indent hiccup in the diff? *************** *** 272,275 **** --- 272,277 ---- { PyObject *m, *d; + + MD5type.ob_type = &PyType_Type; m = Py_InitModule3("md5", md5_functions, module_doc); d = PyModule_GetDict(m); That's because he indented with spaces, not tabs. No need to fix this now (a waste of CVS revisions!), but in general I like to fix this kind of thing before committing. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Fri Feb 4 20:38:27 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 4 Feb 2000 15:38:27 -0500 (EST) Subject: [Patches] Re: [Python-checkins] CVS: python/dist/src/Modules md5module.c,2.12,2.13 In-Reply-To: <200002042034.PAA15279@eric.cnri.reston.va.us> References: <200002042032.PAA12317@weyr.cnri.reston.va.us> <200002042034.PAA15279@eric.cnri.reston.va.us> Message-ID: <14491.14531.706005.178728@weyr.cnri.reston.va.us> Guido van Rossum writes: > Fred, notice this little indent hiccup in the diff? ... > That's because he indented with spaces, not tabs. No need to fix this > now (a waste of CVS revisions!), but in general I like to fix this > kind of thing before committing. Ugh. And to think, I fixed it in one place where a comment had be reindented (to the same depth!). Picky, picky.... -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From dan@cgsoftware.com Fri Feb 4 21:59:26 2000 From: dan@cgsoftware.com (Daniel Berlin) Date: Fri, 4 Feb 2000 13:59:26 -0800 (PST) Subject: [Patches] Quoting patches Message-ID: Just to warn you guys, i noticed that someone (I believe it was guido) already quoted the text of part of a patch. If you do this, please indent it or something. Diff uses a very stupid marker, the only way to determine if something is a patch is to look for "^(\*\*\*|\+\+\+|\-\-\-|Index:)", so quoting a patch without indenting or doing anything will match the regex for a patch. So at some point in the future, if we have something that is generating lists of patches or something, it would think that message contained a patch, which is bad (TM). --Dan From akuchlin@mems-exchange.org Fri Feb 4 22:06:06 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Fri, 4 Feb 2000 17:06:06 -0500 (EST) Subject: [Patches] Multiply sequences by longs Message-ID: <200002042206.RAA11080@amarok.cnri.reston.va.us> This patch makes 5L * 'b' legal; with the large-file patches, some people have run into bugs in modules such as dumbdbm that don't expect f.tell() to return a long int.. One inconsistency: 1231231232312341414L * 'a' gets an OverflowError (long int too long to convert), while 312341414L * 'a' raises a MemoryError. I think this wouldn't be a problem in practice. Documentation: a patch to the Reference and LibRef are required; there may be other bits of the docs that need changing. If this gets accepted, I'll make patches for the docs. Question: should other bits of the core be changed to accept long ints? Slicing, for instance, as in mylist[ 0L:5L ]? (python-dev topic, maybe?) Index: Objects/abstract.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Objects/abstract.c,v retrieving revision 2.28 diff -C2 -r2.28 abstract.c *** abstract.c 1999/10/12 19:54:47 2.28 --- abstract.c 2000/02/04 20:38:22 *************** *** 385,392 **** m = tp->tp_as_sequence; if (m && m->sq_repeat) { ! if (!PyInt_Check(w)) return type_error( "can't multiply sequence with non-int"); ! return (*m->sq_repeat)(v, (int)PyInt_AsLong(w)); } return type_error("bad operand type(s) for *"); --- 385,403 ---- m = tp->tp_as_sequence; if (m && m->sq_repeat) { ! long mul_value; ! ! if (PyInt_Check(w)) { ! mul_value = PyInt_AsLong(w); ! } ! else if (PyLong_Check(w)) { ! mul_value = PyLong_AsLong(w); ! if (PyErr_Occurred()) ! return NULL; ! } ! else { return type_error( "can't multiply sequence with non-int"); ! } ! return (*m->sq_repeat)(v, (int)mul_value); } return type_error("bad operand type(s) for *"); From Gerrit Fri Feb 4 21:55:45 2000 From: Gerrit (Gerrit Holl) Date: Fri, 4 Feb 2000 22:55:45 +0100 Subject: [Patches] Patch for Tools/scripts/crlf.py, lfcr.py, logmerge.py Message-ID: <20000204225544.A18820@stopcontact.palga.uucp> --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Hello, Here is a patch for some scripts in the Tools/scripts directory: * In crlf.py and lfcr.py: regsub -> re * In logmerge.py: added '-r' flag to show the oldest checkin first instead of the newest, and getopt.getopt was used wrong. Output of "diff -u . --exclude=CVS /tmp/py" is attached, where /tmp/py is the dir. with the original scripts and . is the dir. with the changed scripts (in the CVS repository). I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. regards, Gerrit. --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=diff diff -u --exclude=CVS ./crlf.py /tmp/py/crlf.py --- ./crlf.py Fri Feb 4 18:24:23 2000 +++ /tmp/py/crlf.py Fri Feb 4 18:12:49 2000 @@ -2,7 +2,7 @@ "Replace CRLF with LF in argument files. Print names of changed files." -import sys, re, os +import sys, regsub, os for file in sys.argv[1:]: if os.path.isdir(file): print file, "Directory!" @@ -11,7 +11,7 @@ if '\0' in data: print file, "Binary!" continue - newdata = re.sub("\r\n", "\n", data) + newdata = regsub.gsub("\r\n", "\n", data) if newdata != data: print file f = open(file, "wb") diff -u --exclude=CVS ./lfcr.py /tmp/py/lfcr.py --- ./lfcr.py Fri Feb 4 18:24:31 2000 +++ /tmp/py/lfcr.py Fri Feb 4 18:12:49 2000 @@ -2,7 +2,7 @@ "Replace LF with CRLF in argument files. Print names of changed files." -import sys, re, os +import sys, regsub, os for file in sys.argv[1:]: if os.path.isdir(file): print file, "Directory!" @@ -11,7 +11,7 @@ if '\0' in data: print file, "Binary!" continue - newdata = re.sub("\r?\n", "\r\n", data) + newdata = regsub.gsub("\r?\n", "\r\n", data) if newdata != data: print file f = open(file, "wb") diff -u --exclude=CVS ./logmerge.py /tmp/py/logmerge.py --- ./logmerge.py Fri Feb 4 18:20:56 2000 +++ /tmp/py/logmerge.py Fri Feb 4 18:12:49 2000 @@ -32,13 +32,10 @@ def main(): """Main program""" truncate_last = 0 - reverse = 0 - opts, args = getopt.getopt(sys.argv[1:], "tr") + opts, args = getopt.getopt(sys.argv[1:], "-t") for o, a in opts: if o == '-t': truncate_last = 1 - elif o == '-r': - reverse = 1 database = [] while 1: chunk = read_chunk(sys.stdin) @@ -49,8 +46,7 @@ del records[-1] database[len(database):] = records database.sort() - if not reverse: - database.reverse() + database.reverse() format_output(database) def read_chunk(fp): --EVF5PPMfhYS0aIcm-- From Moshe Zadka Sat Feb 5 00:09:05 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 5 Feb 2000 02:09:05 +0200 (IST) Subject: [Patches] __contains__ hook, done right Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. ---559023410-1804928587-949709345=:6357 Content-Type: TEXT/PLAIN; charset=US-ASCII This patch adds a slot to the SequenceMethods structure for function to test membership in a container. It also adds a method to the SequenceMethods structure of PyInstanceType, which calls the magic method __contains__. Yucky lawyer stuff: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. ---559023410-1804928587-949709345=:6357 Content-Type: TEXT/PLAIN; charset=US-ASCII; name=python-diff Content-ID: Content-Description: Content-Transfer-Encoding: BASE64 ZGlmZiAtciAtYyAuLi8uLi8uLi9weXRob24vZGlzdC9zcmMvSW5jbHVkZS9v YmplY3QuaCAuL0luY2x1ZGUvb2JqZWN0LmgNCioqKiAuLi8uLi8uLi9weXRo b24vZGlzdC9zcmMvSW5jbHVkZS9vYmplY3QuaAlXZWQgRmViICAyIDE2OjAy OjUxIDIwMDANCi0tLSAuL0luY2x1ZGUvb2JqZWN0LmgJRnJpIEZlYiAgNCAx MzozOToxMyAyMDAwDQoqKioqKioqKioqKioqKioNCioqKiAxNTEsMTU2ICoq KioNCi0tLSAxNTEsMTYwIC0tLS0NCiAgdHlwZWRlZiBpbnQgKCpnZXRzZWdj b3VudHByb2MpIFB5X1BST1RPKChQeU9iamVjdCAqLCBpbnQgKikpOw0KICB0 eXBlZGVmIGludCAoKmdldGNoYXJidWZmZXJwcm9jKSBQeV9QUk9UTygoUHlP YmplY3QgKiwgaW50LCBjb25zdCBjaGFyICoqKSk7DQogIA0KKyAvKiB0ZW50 YXRpdmUgKGJ5IE1vc2hlKSAqLw0KKyB0eXBlZGVmIGludCgqb2Jqb2JqcHJv YykgUHlfUFJPVE8oKFB5T2JqZWN0ICosIFB5T2JqZWN0ICopKTsNCisgDQor IA0KICB0eXBlZGVmIHN0cnVjdCB7DQogIAliaW5hcnlmdW5jIG5iX2FkZDsN CiAgCWJpbmFyeWZ1bmMgbmJfc3VidHJhY3Q7DQoqKioqKioqKioqKioqKioN CioqKiAxODUsMTkwICoqKioNCi0tLSAxODksMTk1IC0tLS0NCiAgCWludGlu dGFyZ2Z1bmMgc3Ffc2xpY2U7DQogIAlpbnRvYmphcmdwcm9jIHNxX2Fzc19p dGVtOw0KICAJaW50aW50b2JqYXJncHJvYyBzcV9hc3Nfc2xpY2U7DQorIAlv YmpvYmpwcm9jIHNxX2NvbnRhaW5zOw0KICB9IFB5U2VxdWVuY2VNZXRob2Rz Ow0KICANCiAgdHlwZWRlZiBzdHJ1Y3Qgew0KKioqKioqKioqKioqKioqDQoq KiogMzE2LDMyMSAqKioqDQotLS0gMzIxLDMyNyAtLS0tDQogIA0KICAvKiBQ eUJ1ZmZlclByb2NzIGNvbnRhaW5zIGJmX2dldGNoYXJidWZmZXIgKi8NCiAg I2RlZmluZSBQeV9UUEZMQUdTX0hBVkVfR0VUQ0hBUkJVRkZFUiAgKDFMPDww KQ0KKyAjZGVmaW5lIFB5X1RQRkxBR1NfSEFWRV9TRVFVRU5DRV9JTiAgICAo MUw8PDEpDQogIA0KICAjZGVmaW5lIFB5X1RQRkxBR1NfREVGQVVMVCAgKFB5 X1RQRkxBR1NfSEFWRV9HRVRDSEFSQlVGRkVSKQ0KICANCmRpZmYgLXIgLWMg Li4vLi4vLi4vcHl0aG9uL2Rpc3Qvc3JjL09iamVjdHMvYWJzdHJhY3QuYyAu L09iamVjdHMvYWJzdHJhY3QuYw0KKioqIC4uLy4uLy4uL3B5dGhvbi9kaXN0 L3NyYy9PYmplY3RzL2Fic3RyYWN0LmMJRnJpIE9jdCAxNSAxNDowOTowMiAx OTk5DQotLS0gLi9PYmplY3RzL2Fic3RyYWN0LmMJRnJpIEZlYiAgNCAxNDow MDo0NSAyMDAwDQoqKioqKioqKioqKioqKioNCioqKiAxMTEwLDExMTYgKioq Kg0KLS0tIDExMTAsMTEyMSAtLS0tDQogIAkJfQ0KICAJCXJldHVybiAwOw0K ICAJfQ0KKyAJaWYoUHlUeXBlX0hhc0ZlYXR1cmUody0+b2JfdHlwZSwgUHlf VFBGTEFHU19IQVZFX1NFUVVFTkNFX0lOKSkgew0KKyAJCXNxID0gdy0+b2Jf dHlwZS0+dHBfYXNfc2VxdWVuY2U7DQorIAkJaWYoc3EtPnNxX2NvbnRhaW5z ICE9IE5VTEwpDQorIAkJCXJldHVybiAoKnNxLT5zcV9jb250YWlucykodywg dik7DQogIA0KKyAJfQ0KICAJc3EgPSB3LT5vYl90eXBlLT50cF9hc19zZXF1 ZW5jZTsNCiAgCWlmIChzcSA9PSBOVUxMIHx8IHNxLT5zcV9pdGVtID09IE5V TEwpIHsNCiAgCQlQeUVycl9TZXRTdHJpbmcoUHlFeGNfVHlwZUVycm9yLA0K T25seSBpbiAuL09iamVjdHM6IGFic3RyYWN0Lm8NCk9ubHkgaW4gLi9PYmpl Y3RzOiBidWZmZXJvYmplY3Qubw0KZGlmZiAtciAtYyAuLi8uLi8uLi9weXRo b24vZGlzdC9zcmMvT2JqZWN0cy9jbGFzc29iamVjdC5jIC4vT2JqZWN0cy9j bGFzc29iamVjdC5jDQoqKiogLi4vLi4vLi4vcHl0aG9uL2Rpc3Qvc3JjL09i amVjdHMvY2xhc3NvYmplY3QuYwlXZWQgRmViICAyIDE2OjAzOjE5IDIwMDAN Ci0tLSAuL09iamVjdHMvY2xhc3NvYmplY3QuYwlGcmkgRmViICA0IDEzOjQx OjAzIDIwMDANCioqKioqKioqKioqKioqKg0KKioqIDEwNjUsMTA3MCAqKioq DQotLS0gMTA2NSwxMDk2IC0tLS0NCiAgCXJldHVybiAwOw0KICB9DQogIA0K KyBzdGF0aWMgaW50IGluc3RhbmNlX2NvbnRhaW5zKFB5SW5zdGFuY2VPYmpl Y3QgKmluc3QsIFB5T2JqZWN0ICptZW1iZXIpDQorIHsNCisgCXN0YXRpYyBQ eU9iamVjdCAqX19jb250YWluc19fOw0KKyAJUHlPYmplY3QgKmZ1bmMsICph cmcsICpyZXM7DQorIAlpbnQgcmV0Ow0KKyANCisgCWlmKF9fY29udGFpbnNf XyA9PSBOVUxMKQ0KKyAJCV9fY29udGFpbnNfXyA9IFB5U3RyaW5nX0ludGVy bkZyb21TdHJpbmcoIl9fY29udGFpbnNfXyIpOw0KKyAJZnVuYyA9IGluc3Rh bmNlX2dldGF0dHIoaW5zdCwgX19jb250YWluc19fKTsNCisgCWlmKGZ1bmMg PT0gTlVMTCkNCisgCQlyZXR1cm4gLTE7DQorIAlhcmcgPSBQeV9CdWlsZFZh bHVlKCIoTykiLCBtZW1iZXIpOw0KKyAJaWYoYXJnID09IE5VTEwpIHsNCisg CQlQeV9ERUNSRUYoZnVuYyk7DQorIAkJcmV0dXJuIC0xOw0KKyAJfQ0KKyAJ cmVzID0gUHlFdmFsX0NhbGxPYmplY3QoZnVuYywgYXJnKTsNCisgCVB5X0RF Q1JFRihmdW5jKTsNCisgCVB5X0RFQ1JFRihhcmcpOw0KKyAJaWYocmVzID09 IE5VTEwpDQorIAkJcmV0dXJuIC0xOw0KKyAJcmV0ID0gUHlPYmplY3RfSXNU cnVlKHJlcyk7DQorIAlQeV9ERUNSRUYocmVzKTsNCisgCXJldHVybiByZXQ7 DQorIH0NCisgDQogIHN0YXRpYyBQeVNlcXVlbmNlTWV0aG9kcyBpbnN0YW5j ZV9hc19zZXF1ZW5jZSA9IHsNCiAgCShpbnF1aXJ5KWluc3RhbmNlX2xlbmd0 aCwgLypzcV9sZW5ndGgqLw0KICAJMCwgLypzcV9jb25jYXQqLw0KKioqKioq KioqKioqKioqDQoqKiogMTA3MywxMDc4ICoqKioNCi0tLSAxMDk5LDExMDUg LS0tLQ0KICAJKGludGludGFyZ2Z1bmMpaW5zdGFuY2Vfc2xpY2UsIC8qc3Ff c2xpY2UqLw0KICAJKGludG9iamFyZ3Byb2MpaW5zdGFuY2VfYXNzX2l0ZW0s IC8qc3FfYXNzX2l0ZW0qLw0KICAJKGludGludG9iamFyZ3Byb2MpaW5zdGFu Y2VfYXNzX3NsaWNlLCAvKnNxX2Fzc19zbGljZSovDQorIAkob2Jqb2JqcHJv YylpbnN0YW5jZV9jb250YWlucywgLyogc3FfY29udGFpbnMgKi8NCiAgfTsN CiAgDQogIHN0YXRpYyBQeU9iamVjdCAqDQoqKioqKioqKioqKioqKioNCioq KiAxNDA1LDE0MTAgKioqKg0KLS0tIDE0MzIsMTQzOSAtLS0tDQogIAkwLAkJ CS8qdHBfc3RyKi8NCiAgCShnZXRhdHRyb2Z1bmMpaW5zdGFuY2VfZ2V0YXR0 ciwgLyp0cF9nZXRhdHRybyovDQogIAkoc2V0YXR0cm9mdW5jKWluc3RhbmNl X3NldGF0dHIsIC8qdHBfc2V0YXR0cm8qLw0KKyAJMCwgICAgICAgICAgICAg ICAgICAgICAvKiB0cF9hc19idWZmZXIgKi8NCisgCVB5X1RQRkxBR1NfSEFW RV9TRVFVRU5DRV9JTiwgLyogdHBfZmxhZ3MgKi8NCiAgfTsNCiAgDQogIA0K ---559023410-1804928587-949709345=:6357-- From gstein@lyra.org Sat Feb 5 12:13:59 2000 From: gstein@lyra.org (Greg Stein) Date: Sat, 5 Feb 2000 04:13:59 -0800 (PST) Subject: [Patches] Quoting patches In-Reply-To: Message-ID: On Fri, 4 Feb 2000, Daniel Berlin wrote: > Just to warn you guys, i noticed that someone (I believe it was guido) > already quoted the text of part of a patch. > If you do this, please indent it or something. > Diff uses a very stupid marker, the only way to determine if something is > a patch is to look for "^(\*\*\*|\+\+\+|\-\-\-|Index:)", so quoting a > patch without indenting or doing anything will match the regex for a > patch. > So at some point in the future, if we have something that is generating > lists of patches or something, it would think that message contained a > patch, which is bad (TM). Sounds like an intractible problem. If the patch arrives as an attachment in one message, then the message is replied-to without deleting the attachment, then you have a problem. Same for forwarding. Maybe you can extract the patch from the message, compute an MD5 or SHA hash on it, then use the hash to determine whether you've seen the patch before. Cheers, -g -- Greg Stein, http://www.lyra.org/ From Vladimir.Marangozov@inrialpes.fr Sat Feb 5 14:00:59 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 5 Feb 2000 15:00:59 +0100 (CET) Subject: [Patches] Garbage collection patches for Python Message-ID: <200002051400.PAA13946@python.inrialpes.fr> Hello, Some time ago, you (Toby Kelsey, 'xr' and Neil Schemenauer) have submitted a patch for adding GC behavior to Python, that Guido was kind enough to share with some Python developers. Needless to say, on one hand, adding GC would be a radical core change of the interpreter and on the other hand, the interest in this topic has resulted in many useful discussion on c.l.py, as well as in experiments targetting a suitable implementation of the ideas that have beed proposed (in particular, the ones reflected by your submissions). I doubt that we've reached concensus on the subject so far, despite the experience that has been gained from past experiments. Therefore, I presume that incorporating any kind of GC implementation in the standard distribution, for now, is out of question (at least, not for the upcoming 1.6 release). However, as you may have noticed, there's one thing that needs to be done for any "third party" mem manager (including GC): replace the calls to malloc & friends in the core distribution by PyMem_MALLOC & friends (macros, defined in mymalloc.h). Thus, all malloc/free calls could be further redirected to a specific allocator/gc interface. I think that this is a pre-condition for any future work on mem management for Python, which can certainly be done for 1.6. Therefore, I suggest that we prepare a patch and sumbit it to the newly created patches@python.org mailing list. The patch would introduce the PyMem_MALLOC interface into the core, thus decoupling it from the standard libc malloc interface. For now, I think the PyMem_* macros in mymalloc.h should default to the libc malloc family. Neil Schemenauer has already submitted such a patch, but since his submission, the rules for submitting contributions have changed -- they should be accompanied by a short legal disclaimer. (see http://www.python.org/patches/bugrelease.html) Neil, could you please resubmit your 1st GC patch with the disclaimer, so that we can focus on this 1st step, test the patch and fold it into CVS? Thanks. PS: You guys should talk to each other and speak up (once again)! Each of you have had a long way towards implementing GC for Python. For info, this msg is cc'd patches@python.org and Guido. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From dan@cgsoftware.com Sat Feb 5 14:55:39 2000 From: dan@cgsoftware.com (Daniel Berlin) Date: Sat, 5 Feb 2000 06:55:39 -0800 (PST) Subject: [Patches] Quoting patches In-Reply-To: Message-ID: > > Sounds like an intractible problem. If the patch arrives as an attachment > in one message, then the message is replied-to without deleting the > attachment, then you have a problem. Same for forwarding. No, not really. I don't know many mailers that will keep the attachment. I remember having fun with this this summer. Neither OE nor Netscape mail nor Pine will keep the attachment on replies/forwards. You'd have to specifically reattach it. > > Maybe you can extract the patch from the message, compute an MD5 or SHA > hash on it, then use the hash to determine whether you've seen the patch > before. I was thinking about that, but the problem with this is that if their wasn't a disclaimer, and their is now, we want to accept it. The disclaimer might be in the text of th emessage, the actual patch might be exactly the same. So i'd have to track whether it was missing a disclaimer before or not as well. > > Cheers, > -g > > -- > Greg Stein, http://www.lyra.org/ > From nascheme@enme.ucalgary.ca Sat Feb 5 16:28:23 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Sat, 5 Feb 2000 09:28:23 -0700 Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <200002051400.PAA13946@python.inrialpes.fr>; from marangoz@python.inrialpes.fr on Sat, Feb 05, 2000 at 03:00:59PM +0100 References: <200002051400.PAA13946@python.inrialpes.fr> Message-ID: <20000205092822.A2204@acs.ucalgary.ca> --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii On Sat, Feb 05, 2000 at 03:00:59PM +0100, Vladimir Marangozov wrote: > Neil, could you please resubmit your 1st GC patch with the disclaimer, > so that we can focus on this 1st step, test the patch and fold it into > CVS? No problem. I've included my Boehm-Demers-Weiser garbage collector patch too as I have improved it to use autoconfig. The gc patch is small compared to the malloc cleanup. It should work on most Unix machines. I haven't got it to work properly on Windows yet. I also added support for classes and instances in Toby's very cool GC patch. If anyone is interested I can make a new patch for that. I am planning to change it to use a hash table instead of splay trees. Please let me know if you have any comments about that approach. Neil I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="malloc-cleanup.diff" diff -cr Python-cvs/Include/mymalloc.h Python-gc/Include/mymalloc.h *** Python-cvs/Include/mymalloc.h Sat Jul 17 12:13:10 1999 --- Python-gc/Include/mymalloc.h Sat Feb 5 09:07:54 2000 *************** *** 87,92 **** --- 87,94 ---- #define _PyMem_EXTRA 0 #endif + #define PyMem_MALLOC(n) malloc(n) + #define PyMem_REALLOC(p, n) realloc(p, n) #define PyMem_NEW(type, n) \ ( (type *) malloc(_PyMem_EXTRA + (n) * sizeof(type)) ) #define PyMem_RESIZE(p, type, n) \ *************** *** 97,103 **** _PyMem_EXTRA + (n) * sizeof(type)) #define PyMem_DEL(p) free((ANY *)p) #define PyMem_XDEL(p) if ((p) == NULL) ; else PyMem_DEL(p) - /* Two sets of function wrappers around malloc and friends; useful if you need to be sure that you are using the same memory allocator as --- 99,104 ---- diff -cr Python-cvs/Modules/pypcre.c Python-gc/Modules/pypcre.c *** Python-cvs/Modules/pypcre.c Sat Jul 17 12:13:10 1999 --- Python-gc/Modules/pypcre.c Sat Feb 5 09:00:47 2000 *************** *** 3057,3068 **** static int free_stack(match_data *md) { /* Free any stack space that was allocated by the call to match(). */ ! if (md->off_num) free(md->off_num); ! if (md->offset_top) free(md->offset_top); ! if (md->r1) free(md->r1); ! if (md->r2) free(md->r2); ! if (md->eptr) free((char *)md->eptr); ! if (md->ecode) free((char *)md->ecode); return 0; } --- 3057,3068 ---- static int free_stack(match_data *md) { /* Free any stack space that was allocated by the call to match(). */ ! if (md->off_num) PyMem_DEL(md->off_num); ! if (md->offset_top) PyMem_DEL(md->offset_top); ! if (md->r1) PyMem_DEL(md->r1); ! if (md->r2) PyMem_DEL(md->r2); ! if (md->eptr) PyMem_DEL((char *)md->eptr); ! if (md->ecode) PyMem_DEL((char *)md->ecode); return 0; } diff -cr Python-cvs/Modules/readline.c Python-gc/Modules/readline.c *** Python-cvs/Modules/readline.c Sat Feb 5 09:00:14 2000 --- Python-gc/Modules/readline.c Sat Feb 5 09:00:47 2000 *************** *** 41,46 **** --- 41,63 ---- extern int (*PyOS_InputHook)(); extern char *(*PyOS_ReadlineFunctionPointer) Py_PROTO((char *)); + /* convert a string allocated by malloc to one allocated by PyMem_MALLOC */ + static char * + malloc2PyMem(char *p) + { + char *py; + int n; + if (p == NULL) { + return p; + } else { + n = strlen(p); + py = PyMem_MALLOC(n); + strncpy(py, p, n+1); + free(p); + return py; + } + } + /* Exported function to send one line to readline's init file parser */ *************** *** 392,398 **** p = readline(prompt); signal(SIGINT, old_inthandler); if (p == NULL) { ! p = malloc(1); if (p != NULL) *p = '\0'; return p; --- 409,415 ---- p = readline(prompt); signal(SIGINT, old_inthandler); if (p == NULL) { ! p = PyMem_MALLOC(1); if (p != NULL) *p = '\0'; return p; *************** *** 404,410 **** p[n] = '\n'; p[n+1] = '\0'; } ! return p; } --- 421,427 ---- p[n] = '\n'; p[n+1] = '\0'; } ! return malloc2PyMem(p); } diff -cr Python-cvs/Modules/regexpr.c Python-gc/Modules/regexpr.c *** Python-cvs/Modules/regexpr.c Sat Jul 17 12:13:10 1999 --- Python-gc/Modules/regexpr.c Sat Feb 5 09:00:47 2000 *************** *** 175,181 **** { \ state.stack.current = state.stack.first.next; \ state.stack.first.next = state.stack.current->next; \ ! free(state.stack.current); \ } /* Discard the top 'count' stack items. */ --- 175,181 ---- { \ state.stack.current = state.stack.first.next; \ state.stack.first.next = state.stack.current->next; \ ! PyMem_DEL(state.stack.current); \ } /* Discard the top 'count' stack items. */ *************** *** 215,221 **** { \ if (stack.current->next == NULL) \ { \ ! stack.current->next = (item_page_t *)malloc(sizeof(item_page_t)); \ if (stack.current->next == NULL) \ on_error; \ stack.current->next->prev = stack.current; \ --- 215,221 ---- { \ if (stack.current->next == NULL) \ { \ ! stack.current->next = (item_page_t *)PyMem_MALLOC(sizeof(item_page_t)); \ if (stack.current->next == NULL) \ on_error; \ stack.current->next->prev = stack.current; \ *************** *** 723,729 **** visited = small_visited; else { ! visited = malloc(used); if (!visited) return 0; } --- 723,729 ---- visited = small_visited; else { ! visited = PyMem_MALLOC(used); if (!visited) return 0; } *************** *** 732,738 **** memset(visited, 0, used); re_compile_fastmap_aux(buffer, pos, visited, can_be_null, fastmap); if (visited != small_visited) ! free(visited); return 1; } --- 732,738 ---- memset(visited, 0, used); re_compile_fastmap_aux(buffer, pos, visited, can_be_null, fastmap); if (visited != small_visited) ! PyMem_DEL(visited); return 1; } *************** *** 1025,1031 **** if (pattern_offset+(amount) > alloc) \ { \ alloc += 256 + (amount); \ ! pattern = realloc(pattern, alloc); \ if (!pattern) \ goto out_of_memory; \ } \ --- 1025,1031 ---- if (pattern_offset+(amount) > alloc) \ { \ alloc += 256 + (amount); \ ! pattern = PyMem_REALLOC(pattern, alloc); \ if (!pattern) \ goto out_of_memory; \ } \ *************** *** 1184,1190 **** if (alloc == 0 || pattern == NULL) { alloc = 256; ! pattern = malloc(alloc); if (!pattern) goto out_of_memory; } --- 1184,1190 ---- if (alloc == 0 || pattern == NULL) { alloc = 256; ! pattern = PyMem_MALLOC(alloc); if (!pattern) goto out_of_memory; } diff -cr Python-cvs/Objects/bufferobject.c Python-gc/Objects/bufferobject.c *** Python-cvs/Objects/bufferobject.c Sat Feb 5 09:00:14 2000 --- Python-gc/Objects/bufferobject.c Sat Feb 5 09:01:35 2000 *************** *** 188,194 **** "size must be zero or positive"); return NULL; } ! b = (PyBufferObject *)malloc(sizeof(*b) + size); if ( b == NULL ) return PyErr_NoMemory(); b->ob_type = &PyBuffer_Type; --- 188,194 ---- "size must be zero or positive"); return NULL; } ! b = (PyBufferObject *)PyMem_MALLOC(sizeof(*b) + size); if ( b == NULL ) return PyErr_NoMemory(); b->ob_type = &PyBuffer_Type; *************** *** 212,218 **** PyBufferObject *self; { Py_XDECREF(self->b_base); ! free((void *)self); } static int --- 212,218 ---- PyBufferObject *self; { Py_XDECREF(self->b_base); ! PyMem_DEL((void *)self); } static int diff -cr Python-cvs/Objects/classobject.c Python-gc/Objects/classobject.c *** Python-cvs/Objects/classobject.c Sat Feb 5 09:00:14 2000 --- Python-gc/Objects/classobject.c Sat Feb 5 09:00:47 2000 *************** *** 146,152 **** Py_XDECREF(op->cl_getattr); Py_XDECREF(op->cl_setattr); Py_XDECREF(op->cl_delattr); ! free((ANY *)op); } static PyObject * --- 146,152 ---- Py_XDECREF(op->cl_getattr); Py_XDECREF(op->cl_setattr); Py_XDECREF(op->cl_delattr); ! PyMem_DEL((ANY *)op); } static PyObject * *************** *** 561,567 **** #endif /* Py_TRACE_REFS */ Py_DECREF(inst->in_class); Py_XDECREF(inst->in_dict); ! free((ANY *)inst); } static PyObject * --- 561,567 ---- #endif /* Py_TRACE_REFS */ Py_DECREF(inst->in_class); Py_XDECREF(inst->in_dict); ! PyMem_DEL((ANY *)inst); } static PyObject * diff -cr Python-cvs/Objects/complexobject.c Python-gc/Objects/complexobject.c *** Python-cvs/Objects/complexobject.c Sat Feb 5 09:00:14 2000 --- Python-gc/Objects/complexobject.c Sat Feb 5 09:00:47 2000 *************** *** 167,173 **** Py_complex cval; { register PyComplexObject *op = ! (PyComplexObject *) malloc(sizeof(PyComplexObject)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyComplex_Type; --- 167,173 ---- Py_complex cval; { register PyComplexObject *op = ! (PyComplexObject *) PyMem_MALLOC(sizeof(PyComplexObject)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyComplex_Type; diff -cr Python-cvs/Objects/dictobject.c Python-gc/Objects/dictobject.c *** Python-cvs/Objects/dictobject.c Sat Jul 17 12:13:10 1999 --- Python-gc/Objects/dictobject.c Sat Feb 5 09:00:47 2000 *************** *** 277,283 **** break; } } ! newtable = (dictentry *) malloc(sizeof(dictentry) * newsize); if (newtable == NULL) { PyErr_NoMemory(); return -1; --- 277,283 ---- break; } } ! newtable = (dictentry *) PyMem_MALLOC(sizeof(dictentry) * newsize); if (newtable == NULL) { PyErr_NoMemory(); return -1; diff -cr Python-cvs/Objects/fileobject.c Python-gc/Objects/fileobject.c *** Python-cvs/Objects/fileobject.c Sat Feb 5 09:00:14 2000 --- Python-gc/Objects/fileobject.c Sat Feb 5 09:00:47 2000 *************** *** 209,215 **** if (f->f_mode != NULL) { Py_DECREF(f->f_mode); } ! free((char *)f); } static PyObject * --- 209,215 ---- if (f->f_mode != NULL) { Py_DECREF(f->f_mode); } ! PyMem_DEL((char *)f); } static PyObject * diff -cr Python-cvs/Objects/floatobject.c Python-gc/Objects/floatobject.c *** Python-cvs/Objects/floatobject.c Sat Feb 5 09:00:15 2000 --- Python-gc/Objects/floatobject.c Sat Feb 5 09:00:47 2000 *************** *** 98,106 **** #define BHEAD_SIZE 8 /* Enough for a 64-bit pointer */ #define N_FLOATOBJECTS ((BLOCK_SIZE - BHEAD_SIZE) / sizeof(PyFloatObject)) - #define PyMem_MALLOC malloc - #define PyMem_FREE free - struct _floatblock { struct _floatblock *next; PyFloatObject objects[N_FLOATOBJECTS]; --- 98,103 ---- *************** *** 756,762 **** } } else { ! PyMem_FREE(list); bf++; } fsum += frem; --- 753,759 ---- } } else { ! PyMem_DEL(list); bf++; } fsum += frem; diff -cr Python-cvs/Objects/frameobject.c Python-gc/Objects/frameobject.c *** Python-cvs/Objects/frameobject.c Sat Feb 5 09:00:15 2000 --- Python-gc/Objects/frameobject.c Sat Feb 5 09:00:47 2000 *************** *** 179,185 **** builtins = NULL; if (free_list == NULL) { f = (PyFrameObject *) ! malloc(sizeof(PyFrameObject) + extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); --- 179,185 ---- builtins = NULL; if (free_list == NULL) { f = (PyFrameObject *) ! PyMem_MALLOC(sizeof(PyFrameObject) + extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); *************** *** 191,197 **** free_list = free_list->f_back; if (f->f_nlocals + f->f_stacksize < extras) { f = (PyFrameObject *) ! realloc(f, sizeof(PyFrameObject) + extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); --- 191,197 ---- free_list = free_list->f_back; if (f->f_nlocals + f->f_stacksize < extras) { f = (PyFrameObject *) ! PyMem_REALLOC(f, sizeof(PyFrameObject) + extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); diff -cr Python-cvs/Objects/intobject.c Python-gc/Objects/intobject.c *** Python-cvs/Objects/intobject.c Sat Feb 5 09:00:15 2000 --- Python-gc/Objects/intobject.c Sat Feb 5 09:00:47 2000 *************** *** 94,102 **** #define BHEAD_SIZE 8 /* Enough for a 64-bit pointer */ #define N_INTOBJECTS ((BLOCK_SIZE - BHEAD_SIZE) / sizeof(PyIntObject)) - #define PyMem_MALLOC malloc - #define PyMem_FREE free - struct _intblock { struct _intblock *next; PyIntObject objects[N_INTOBJECTS]; --- 94,99 ---- *************** *** 915,921 **** } } else { ! PyMem_FREE(list); bf++; } isum += irem; --- 912,918 ---- } } else { ! PyMem_DEL(list); bf++; } isum += irem; diff -cr Python-cvs/Objects/listobject.c Python-gc/Objects/listobject.c *** Python-cvs/Objects/listobject.c Sat Feb 5 09:00:15 2000 --- Python-gc/Objects/listobject.c Sat Feb 5 09:00:47 2000 *************** *** 70,76 **** if (nbytes / sizeof(PyObject *) != (size_t)size) { return PyErr_NoMemory(); } ! op = (PyListObject *) malloc(sizeof(PyListObject)); if (op == NULL) { return PyErr_NoMemory(); } --- 70,76 ---- if (nbytes / sizeof(PyObject *) != (size_t)size) { return PyErr_NoMemory(); } ! op = (PyListObject *) PyMem_MALLOC(sizeof(PyListObject)); if (op == NULL) { return PyErr_NoMemory(); } *************** *** 78,86 **** op->ob_item = NULL; } else { ! op->ob_item = (PyObject **) malloc(nbytes); if (op->ob_item == NULL) { ! free((ANY *)op); return PyErr_NoMemory(); } } --- 78,86 ---- op->ob_item = NULL; } else { ! op->ob_item = (PyObject **) PyMem_MALLOC(nbytes); if (op->ob_item == NULL) { ! PyMem_DEL((ANY *)op); return PyErr_NoMemory(); } } *************** *** 224,232 **** while (--i >= 0) { Py_XDECREF(op->ob_item[i]); } ! free((ANY *)op->ob_item); } ! free((ANY *)op); } static int --- 224,232 ---- while (--i >= 0) { Py_XDECREF(op->ob_item[i]); } ! PyMem_DEL((ANY *)op->ob_item); } ! PyMem_DEL((ANY *)op); } static int diff -cr Python-cvs/Objects/moduleobject.c Python-gc/Objects/moduleobject.c *** Python-cvs/Objects/moduleobject.c Sat Jul 17 12:13:10 1999 --- Python-gc/Objects/moduleobject.c Sat Feb 5 09:00:47 2000 *************** *** 170,176 **** _PyModule_Clear((PyObject *)m); Py_DECREF(m->md_dict); } ! free((char *)m); } static PyObject * --- 170,176 ---- _PyModule_Clear((PyObject *)m); Py_DECREF(m->md_dict); } ! PyMem_DEL((char *)m); } static PyObject * diff -cr Python-cvs/Objects/object.c Python-gc/Objects/object.c *** Python-cvs/Objects/object.c Sat Feb 5 09:00:18 2000 --- Python-gc/Objects/object.c Sat Feb 5 09:07:54 2000 *************** *** 33,38 **** --- 33,39 ---- #include "Python.h" + #if defined( Py_TRACE_REFS ) || defined( Py_REF_DEBUG ) DL_IMPORT(long) _Py_RefTotal; #endif *************** *** 119,125 **** #endif { #ifndef MS_COREDLL ! PyObject *op = (PyObject *) malloc(tp->tp_basicsize); #endif if (op == NULL) return PyErr_NoMemory(); --- 120,126 ---- #endif { #ifndef MS_COREDLL ! PyObject *op = (PyObject *) PyMem_MALLOC(tp->tp_basicsize); #endif if (op == NULL) return PyErr_NoMemory(); *************** *** 143,149 **** { #ifndef MS_COREDLL PyVarObject *op = (PyVarObject *) ! malloc(tp->tp_basicsize + size * tp->tp_itemsize); #endif if (op == NULL) return (PyVarObject *)PyErr_NoMemory(); --- 144,150 ---- { #ifndef MS_COREDLL PyVarObject *op = (PyVarObject *) ! PyMem_MALLOC(tp->tp_basicsize + size * tp->tp_itemsize); #endif if (op == NULL) return (PyVarObject *)PyErr_NoMemory(); *************** *** 754,760 **** if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = malloc(nbytes); if (p != NULL) return p; else { --- 755,761 ---- if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = PyMem_MALLOC(nbytes); if (p != NULL) return p; else { *************** *** 772,778 **** if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = realloc(p, nbytes); if (p != NULL) return p; else { --- 773,779 ---- if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = PyMem_REALLOC(p, nbytes); if (p != NULL) return p; else { *************** *** 785,791 **** Py_Free(p) ANY *p; { ! free(p); } /* The PyMem_{Malloc,Realloc} wrappers don't call anything on failure */ --- 786,792 ---- Py_Free(p) ANY *p; { ! PyMem_DEL(p); } /* The PyMem_{Malloc,Realloc} wrappers don't call anything on failure */ *************** *** 798,804 **** if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! return malloc(nbytes); } ANY * --- 799,805 ---- if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! return PyMem_MALLOC(nbytes); } ANY * *************** *** 810,823 **** if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! return realloc(p, nbytes); } void PyMem_Free(p) ANY *p; { ! free(p); } --- 811,824 ---- if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! return PyMem_REALLOC(p, nbytes); } void PyMem_Free(p) ANY *p; { ! PyMem_DEL(p); } diff -cr Python-cvs/Objects/stringobject.c Python-gc/Objects/stringobject.c *** Python-cvs/Objects/stringobject.c Sat Feb 5 09:00:18 2000 --- Python-gc/Objects/stringobject.c Sat Feb 5 09:00:47 2000 *************** *** 93,99 **** } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 93,99 ---- } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 143,149 **** } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 143,149 ---- } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 306,312 **** } size = a->ob_size + b->ob_size; op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 306,312 ---- } size = a->ob_size + b->ob_size; op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 341,347 **** return (PyObject *)a; } op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 341,347 ---- return (PyObject *)a; } op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 1646,1652 **** #endif _Py_ForgetReference(v); *pv = (PyObject *) ! realloc((char *)v, sizeof(PyStringObject) + newsize * sizeof(char)); if (*pv == NULL) { PyMem_DEL(v); --- 1646,1652 ---- #endif _Py_ForgetReference(v); *pv = (PyObject *) ! PyMem_REALLOC((char *)v, sizeof(PyStringObject) + newsize * sizeof(char)); if (*pv == NULL) { PyMem_DEL(v); diff -cr Python-cvs/Objects/tupleobject.c Python-gc/Objects/tupleobject.c *** Python-cvs/Objects/tupleobject.c Sat Feb 5 09:00:18 2000 --- Python-gc/Objects/tupleobject.c Sat Feb 5 09:02:17 2000 *************** *** 91,97 **** return PyErr_NoMemory(); } ; ! op = (PyTupleObject *) malloc(nbytes); if (op == NULL) return PyErr_NoMemory(); --- 91,97 ---- return PyErr_NoMemory(); } ; ! op = (PyTupleObject *) PyMem_MALLOC(nbytes); if (op == NULL) return PyErr_NoMemory(); *************** *** 184,190 **** } #endif } ! free((ANY *)op); } static int --- 184,190 ---- } #endif } ! PyMem_DEL(op); } static int *************** *** 467,473 **** v->ob_item[i] = NULL; } sv = (PyTupleObject *) ! realloc((char *)v, sizeof(PyTupleObject) + newsize * sizeof(PyObject *)); *pv = (PyObject *) sv; if (sv == NULL) { --- 467,473 ---- v->ob_item[i] = NULL; } sv = (PyTupleObject *) ! PyMem_REALLOC((char *)v, sizeof(PyTupleObject) + newsize * sizeof(PyObject *)); *pv = (PyObject *) sv; if (sv == NULL) { diff -cr Python-cvs/Parser/myreadline.c Python-gc/Parser/myreadline.c *** Python-cvs/Parser/myreadline.c Sat Jul 17 12:13:10 1999 --- Python-gc/Parser/myreadline.c Sat Feb 5 09:00:47 2000 *************** *** 89,95 **** int n; char *p; n = 100; ! if ((p = malloc(n)) == NULL) return NULL; fflush(stdout); if (prompt) --- 89,95 ---- int n; char *p; n = 100; ! if ((p = PyMem_MALLOC(n)) == NULL) return NULL; fflush(stdout); if (prompt) *************** *** 99,105 **** case 0: /* Normal case */ break; case 1: /* Interrupt */ ! free(p); return NULL; case -1: /* EOF */ case -2: /* Error */ --- 99,105 ---- case 0: /* Normal case */ break; case 1: /* Interrupt */ ! PyMem_DEL(p); return NULL; case -1: /* EOF */ case -2: /* Error */ *************** *** 117,130 **** n = strlen(p); while (n > 0 && p[n-1] != '\n') { int incr = n+2; ! p = realloc(p, n + incr); if (p == NULL) return NULL; if (my_fgets(p+n, incr, stdin) != 0) break; n += strlen(p+n); } ! return realloc(p, n+1); } --- 117,130 ---- n = strlen(p); while (n > 0 && p[n-1] != '\n') { int incr = n+2; ! p = PyMem_REALLOC(p, n + incr); if (p == NULL) return NULL; if (my_fgets(p+n, incr, stdin) != 0) break; n += strlen(p+n); } ! return PyMem_REALLOC(p, n+1); } diff -cr Python-cvs/Parser/tokenizer.c Python-gc/Parser/tokenizer.c *** Python-cvs/Parser/tokenizer.c Sat Jul 17 12:13:10 1999 --- Python-gc/Parser/tokenizer.c Sat Feb 5 09:00:47 2000 *************** *** 219,225 **** if (new == NULL) tok->done = E_INTR; else if (*new == '\0') { ! free(new); tok->done = E_EOF; } else if (tok->start != NULL) { --- 219,225 ---- if (new == NULL) tok->done = E_INTR; else if (*new == '\0') { ! PyMem_DEL(new); tok->done = E_EOF; } else if (tok->start != NULL) { *************** *** 229,244 **** char *buf = realloc(tok->buf, newlen+1); tok->lineno++; if (buf == NULL) { ! free(tok->buf); tok->buf = NULL; ! free(new); tok->done = E_NOMEM; return EOF; } tok->buf = buf; tok->cur = tok->buf + oldlen; strcpy(tok->buf + oldlen, new); ! free(new); tok->inp = tok->buf + newlen; tok->end = tok->inp + 1; tok->start = tok->buf + start; --- 229,244 ---- char *buf = realloc(tok->buf, newlen+1); tok->lineno++; if (buf == NULL) { ! PyMem_DEL(tok->buf); tok->buf = NULL; ! PyMem_DEL(new); tok->done = E_NOMEM; return EOF; } tok->buf = buf; tok->cur = tok->buf + oldlen; strcpy(tok->buf + oldlen, new); ! PyMem_DEL(new); tok->inp = tok->buf + newlen; tok->end = tok->inp + 1; tok->start = tok->buf + start; *************** *** 246,252 **** else { tok->lineno++; if (tok->buf != NULL) ! free(tok->buf); tok->buf = new; tok->cur = tok->buf; tok->inp = strchr(tok->buf, '\0'); --- 246,252 ---- else { tok->lineno++; if (tok->buf != NULL) ! PyMem_DEL(tok->buf); tok->buf = new; tok->cur = tok->buf; tok->inp = strchr(tok->buf, '\0'); --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="gc-autoconfig.diff" diff -cr Python-malloc/Include/mymalloc.h Python-gc/Include/mymalloc.h *** Python-malloc/Include/mymalloc.h Sat Feb 5 09:11:36 2000 --- Python-gc/Include/mymalloc.h Sat Feb 5 09:11:54 2000 *************** *** 87,92 **** --- 87,110 ---- #define _PyMem_EXTRA 0 #endif + #ifdef WITH_GC + + #include + #define PyMem_MALLOC(n) GC_MALLOC(n) + #define PyMem_REALLOC(p, n) GC_REALLOC(p, n) + #define PyMem_NEW(type, n) \ + ( (type *) GC_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)) ) + #define PyMem_RESIZE(p, type, n) \ + if ((p) == NULL) \ + (p) = (type *) GC_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)); \ + else \ + (p) = (type *) GC_REALLOC((ANY *)(p), \ + _PyMem_EXTRA + (n) * sizeof(type)) + #define PyMem_DEL(p) GC_FREE((ANY *)p) + #define PyMem_XDEL(p) if ((p) == NULL) ; else PyMem_DEL(p) + + #else /* ! WITH_GC */ + #define PyMem_MALLOC(n) malloc(n) #define PyMem_REALLOC(p, n) realloc(p, n) #define PyMem_NEW(type, n) \ *************** *** 99,104 **** --- 117,124 ---- _PyMem_EXTRA + (n) * sizeof(type)) #define PyMem_DEL(p) free((ANY *)p) #define PyMem_XDEL(p) if ((p) == NULL) ; else PyMem_DEL(p) + + #endif /* WITH_GC */ /* Two sets of function wrappers around malloc and friends; useful if you need to be sure that you are using the same memory allocator as diff -cr Python-malloc/Modules/_tkinter.c Python-gc/Modules/_tkinter.c *** Python-malloc/Modules/_tkinter.c Sat Feb 5 09:11:36 2000 --- Python-gc/Modules/_tkinter.c Sat Feb 5 09:11:54 2000 *************** *** 1268,1274 **** --- 1268,1281 ---- return NULL; } + #ifdef WITH_GC + /* If we use GC_malloc then the gc thinks that there are no more + * references to the callback and collects it */ + data = (PythonCmd_ClientData*) + GC_malloc_uncollectable(sizeof(PythonCmd_ClientData)); + #else data = PyMem_NEW(PythonCmd_ClientData, 1); + #endif if (!data) return NULL; Py_XINCREF(self); *************** *** 1334,1340 **** --- 1341,1354 ---- int id; { FileHandler_ClientData *p; + #ifdef WITH_GC + /* If we use GC_malloc then the gc thinks that there are no more + * references to the callback and collects it */ + p = (FileHandler_ClientData*) + GC_malloc_uncollectable(sizeof(FileHandler_ClientData)); + #else p = PyMem_NEW(FileHandler_ClientData, 1); + #endif if (p != NULL) { Py_XINCREF(func); Py_XINCREF(file); diff -cr Python-malloc/Objects/object.c Python-gc/Objects/object.c *** Python-malloc/Objects/object.c Sat Feb 5 09:11:36 2000 --- Python-gc/Objects/object.c Sat Feb 5 09:11:54 2000 *************** *** 108,113 **** --- 108,124 ---- } #endif + /* + #ifdef WITH_GC + static void + _PyObject_Finalizer(GC_PTR *op, GC_PTR *client_data) + { + destructor dealloc = ((PyObject*)op)->ob_type->tp_dealloc; + (*dealloc)(op); + } + #endif + */ + #ifndef MS_COREDLL PyObject * _PyObject_New(tp) *************** *** 124,129 **** --- 135,145 ---- #endif if (op == NULL) return PyErr_NoMemory(); + /* + #ifdef WITH_GC + GC_register_finalizer(op, _PyObject_Finalizer, 0, 0, 0); + #endif + */ op->ob_type = tp; _Py_NewReference(op); return op; *************** *** 148,153 **** --- 164,174 ---- #endif if (op == NULL) return (PyVarObject *)PyErr_NoMemory(); + /* + #ifdef WITH_GC + GC_register_finalizer(op, _PyObject_Finalizer, 0, 0, 0); + #endif + */ op->ob_type = tp; op->ob_size = size; _Py_NewReference((PyObject *)op); diff -cr Python-malloc/acconfig.h Python-gc/acconfig.h *** Python-malloc/acconfig.h Sat Feb 5 09:11:36 2000 --- Python-gc/acconfig.h Sat Feb 5 09:11:54 2000 *************** *** 126,131 **** --- 126,134 ---- /* Define if you want to compile in rudimentary thread support */ #undef WITH_THREAD + /* Define if you want to use the garbage collector */ + #undef WITH_GC + /* Define if you want to produce an OpenStep/Rhapsody framework (shared library plus accessory files). */ #undef WITH_NEXT_FRAMEWORK diff -cr Python-malloc/config.h.in Python-gc/config.h.in *** Python-malloc/config.h.in Sat Feb 5 09:11:36 2000 --- Python-gc/config.h.in Sat Feb 5 09:11:54 2000 *************** *** 181,186 **** --- 181,189 ---- /* Define if you want to compile in rudimentary thread support */ #undef WITH_THREAD + /* Define if you want to use the garbage collector */ + #undef WITH_GC + /* Define if you want to produce an OpenStep/Rhapsody framework (shared library plus accessory files). */ #undef WITH_NEXT_FRAMEWORK diff -cr Python-malloc/configure.in Python-gc/configure.in *** Python-malloc/configure.in Sat Feb 5 09:11:36 2000 --- Python-gc/configure.in Sat Feb 5 09:11:54 2000 *************** *** 687,692 **** --- 687,704 ---- AC_SUBST(DLINCLDIR) DLINCLDIR=/ + AC_MSG_CHECKING(for --with-gc) + AC_SUBST(LDLAST) + AC_ARG_WITH(gc, + [--with-gc use the Boehm-Demers-Weiser garbage collector], + [AC_MSG_RESULT($withval) + AC_DEFINE(WITH_GC) + LIBS="$LIBS -lgc" + if test "${with_gc+set}" != set; then + with_gc="$withval"; + fi], + AC_MSG_RESULT(no)) + AC_MSG_CHECKING(for --with-sgi-dl) AC_ARG_WITH(sgi-dl, [--with-sgi-dl=DIRECTORY IRIX 4 dynamic linking], [ AC_MSG_RESULT($withval) --TB36FDmn/VVEgNH/-- From dan@cgsoftware.com Sat Feb 5 17:50:05 2000 From: dan@cgsoftware.com (Daniel Berlin) Date: Sat, 5 Feb 2000 09:50:05 -0800 (PST) Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <20000205092822.A2204@acs.ucalgary.ca> Message-ID: > > No problem. I've included my Boehm-Demers-Weiser garbage > collector patch too as I have improved it to use autoconfig. The > gc patch is small compared to the malloc cleanup. It should work > on most Unix machines. I haven't got it to work properly on > Windows yet. Well, Boehm has trouble with shared libraries on quite a few platforms. It also has trouble with threads on a few platforms. I remember all this from porting 5.0alpha3 (and previous versions) to beos. > > I also added support for classes and instances in Toby's very > cool GC patch. If anyone is interested I can make a new patch > for that. I am planning to change it to use a hash table instead > of splay trees. Please let me know if you have any comments > about that approach. > > > Neil > > > > I confirm that, to the best of my knowledge and belief, this > contribution is free of any claims of third parties under > copyright, patent or other rights or interests ("claims"). To > the extent that I have any such claims, I hereby grant to CNRI a > nonexclusive, irrevocable, royalty-free, worldwide license to > reproduce, distribute, perform and/or display publicly, prepare > derivative versions, and otherwise use this contribution as part > of the Python software and its related documentation, or any > derivative versions thereof, at no cost to CNRI or its licensed > users, and to authorize others to do so. > > I acknowledge that CNRI may, at its sole discretion, decide > whether or not to incorporate this contribution in the Python > software and its related documentation. I further grant CNRI > permission to use my name and other identifying information > provided to CNRI by me for use in connection with the Python > software and its related documentation. > > > From pf@artcom-gmbh.de Sun Feb 6 22:17:13 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Sun, 6 Feb 2000 23:17:13 +0100 (MET) Subject: [Patches] Patch Guide-Lines for Documentation Message-ID: Hi! 'http://www.python.org/patches' contains detailed information how to submit patches. However it is unclear, how to submit patches against the Python Documentation and whether these guidelines also apply on submissions to python-docs@python.org. This should be made clearer. Especially, whether every small patch must indeed contain the standard disclaimer. Regards, Peter -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 From guido@python.org Mon Feb 7 16:11:21 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 07 Feb 2000 11:11:21 -0500 Subject: [Patches] segmentation fault when trying to read from a close (fwd) ()d gdbm database (PR#198) Message-ID: <200002071611.LAA17932@eric.cnri.reston.va.us> Not sure if this is appropriate to forward here; but seems easy to fix and worth it. --Guido van Rossum (home page: http://www.python.org/~guido/) ------- Forwarded Message Date: Mon, 07 Feb 2000 05:13:08 -0500 From: bhaddow@orbism.com To: python-bugs-list@python.org cc: bugs-py@python.org Subject: [Python-bugs-list] segmentation fault when trying to read from a close ()d gdbm database (PR#198) Full_Name: Barry Haddow Version: 1.52 OS: Linux (RedHat 6.0) Submission from: p868.as1.adl.dublin.eircom.net (159.134.231.100) Hi Running the following causes a segmentation fault. #/usr/bin/env python import gdbm db = gdbm.open('db','n') db['name'] = 'barry' db.close() print db['name'] The problem can be fixed by editing the function dbm_subscript() in Modules/gdbmmodule.c , line 148. A check should be added to ensure that the database is not already closed, as in lines 182-184 of dbm_ass_sub(). cheers Barry _______________________________________________ Python-bugs-list maillist - Python-bugs-list@python.org http://www.python.org/mailman/listinfo/python-bugs-list ------- End of Forwarded Message From thomas@xs4all.net Mon Feb 7 17:31:18 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 18:31:18 +0100 Subject: [Patches] Subject: nntplib.py patch for 'mode reader' command Message-ID: <20000207183118.Q23471@xs4all.nl> --pAwQNkOnpTn9IO2O Content-Type: text/plain; charset=us-ascii Attached you'll find a patch to nntplib.py, the NNTP class, to allow for a 'readermode' argument during construction. Some newsservers require a command, 'mode reader', to be sent before certain commands can be executed (inn uses it to spawn nnrpd and pass the connection to it. i am not sure if other, non-inn-based readers use it too.) The patch is based on a posting by Jim Tittsler , to the mailman-users list, in October last year: http://www.python.org/pipermail/mailman-users/1999-October/002371.html As he points out, this can be solved by wrapping the first group command around a try/except, and send the 'mode reader' command if it fails with an error 500, as long as authentication is not used. Mailman does not use authentication, and I've already posted a patch to the mailman-developpers list to fix it like that, but I think the obvious fix is the one attached. The patch also expands NNTP.__init__'s docstring some. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. (The patch is unified diff. I prefer unified, sue me and i'll send a context one ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --pAwQNkOnpTn9IO2O Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="nntplib.diff" Index: dist/src/Lib/nntplib.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/nntplib.py,v retrieving revision 1.17 diff -u -r1.17 nntplib.py --- nntplib.py 2000/02/04 15:10:33 1.17 +++ nntplib.py 2000/02/07 16:02:44 @@ -58,10 +58,13 @@ class NNTP: - def __init__(self, host, port = NNTP_PORT, user=None, password=None): + def __init__(self, host, port = NNTP_PORT, user=None, password=None, readermode=0): """Initialize an instance. Arguments: - host: hostname to connect to - - port: port to connect to (default the standard NNTP port)""" + - port: port to connect to (default the standard NNTP port) + - user: username to authenticate with + - password: password to use with username + - readermode: send 'mode reader' command after connecting""" self.host = host self.port = port @@ -70,6 +73,11 @@ self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if readermode: + try: + self.welcome = self.shortcmd('mode reader') + except error_perm: # error 500, probably 'not implemented' + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': --pAwQNkOnpTn9IO2O-- From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 17:53:17 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 12:53:17 -0500 (EST) Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> Message-ID: <14495.1677.982940.194724@anthem.cnri.reston.va.us> I'm cutting and pasting a message that originated on the mailman-users mailing list. I think this message is fairly self-explanatory. Can other NNTP experts look this over and see if it makes sense? I'll apply it if there are no objections. Note: the bare except is bothersome. For reference, the thread is here: http://www.python.org/pipermail/mailman-users/1999-October/002371.html -Barry -------------------- snip snip -------------------- On Sun, Oct 10, 1999 at 06:23:41PM -0700, Michael Ghens wrote: > Tried a news gateway test. This is the error: [...] > nntplib.error_perm: 500 "GROUP" not implemented; try "help". Were you running the mailman gateway on the same machine as the news server? (Or on another machine that is an NNTP peer of the news server?) Python's standard nntplib assumes that when it connects it will default to (NNRP) reader mode. If the connection ends up in NNTP/news feed mode, many commands, including "GROUP" will not be recognized. A quick hack is to add this to your nntplib.py: diff -u nntplib.py.dist nntplib.py --- nntplib.py.dist Tue Apr 28 17:43:35 1998 +++ nntplib.py Tue Jun 23 22:06:00 1998 @@ -71,6 +71,10 @@ self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + try: + self.welcome = self.shortcmd('mode reader') + except: + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': This makes nntplib use 'mode reader' for all connections. As Harald Meland pointed out, this should probably be controlled by adding another optional argument to nntplib.NNTP(), but if you only use the library with mailman or in other "client" applications this will suffice. Jim P.S. You could also do the 'mode reader' stanza in GatewayManager, but then you also need to repeat the 'authinfo' code there if your news server needs authentication, since that must follow the 'mode reader' command. -- Jim Tittsler, Tokyo From toby@puckish.demon.co.uk Mon Feb 7 16:35:44 2000 From: toby@puckish.demon.co.uk (Toby Kelsey) Date: Mon, 7 Feb 2000 16:35:44 +0000 Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <20000205092822.A2204@acs.ucalgary.ca> Message-ID: In message <20000205092822.A2204@acs.ucalgary.ca>, nascheme@enme.ucalgary.ca writes >I also added support for classes and instances in Toby's very >cool GC patch. If anyone is interested I can make a new patch >for that. I am planning to change it to use a hash table instead >of splay trees. Please let me know if you have any comments >about that approach. I was intending to make those changes anyway, but my round tuit is still on order :-). I sent two versions of the patch out, one to Guido and the second (tidier) one to Tim, hopefully you've seen the later one. I also discussed with Tim modifying type objects and involving pcrl at object creation time. I dont know whether these have also been discussed elsewhere. Toby -- Toby Kelsey From thomas@xs4all.net Mon Feb 7 18:17:13 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 19:17:13 +0100 Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <14495.1677.982940.194724@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Mon, Feb 07, 2000 at 12:53:17PM -0500 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> Message-ID: <20000207191713.L19265@xs4all.nl> On Mon, Feb 07, 2000 at 12:53:17PM -0500, Barry A. Warsaw wrote: > I'm cutting and pasting a message that originated on the mailman-users > mailing list. I think this message is fairly self-explanatory. Can > other NNTP experts look this over and see if it makes sense? I'll > apply it if there are no objections. > Note: the bare except is bothersome. I've discussed this with two of my colleagues, who do and have done a fair lot with INN, and the general concensus is that it should work just fine. Servers who dont need 'mode reader' but recognize it should ignore it, Servers who dont understand it should give a 500 error, 'error_perm', with 'unimplemented' or 'what ?' or some such as message text. Note, though, that i've already posted a small patch to mailman to fix this. Since the 'group' command is executed by gate_news, which doesn't need authentication, it's easy enough to try the group, and set the connection to readermode if it fails. Also note that I've already sent a patch to patches@python.org ;) with a pointer to the mailman-users discussion in question. http://www.python.org/pipermail/patches/2000-February/000015.html My patch uses an extra argument to NNTP.__init__, and avoids the bare except. To get the same behaviour as the patch below, sending 'mode reader' by default, the default value of readermode can be set to 1. I'm not sure if that's a wise thing, however, as some users of nntplib.NNTP might depend on the connection being able to do things that you cant do in readermode. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From jwt@dskk.co.jp Tue Feb 8 02:47:06 2000 From: jwt@dskk.co.jp (Jim Tittsler) Date: Tue, 8 Feb 2000 11:47:06 +0900 Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207191713.L19265@xs4all.nl>; from thomas@xs4all.net on Mon, Feb 07, 2000 at 07:17:13PM +0100 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> Message-ID: <20000208114706.C2908@mail.dskk.co.jp> --FCuugMFkClbJLl1L Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 07:17:13PM +0100, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 12:53:17PM -0500, Barry A. Warsaw wrote: > > > I'm cutting and pasting a message that originated on the mailman-users > > mailing list. I think this message is fairly self-explanatory. Can > > other NNTP experts look this over and see if it makes sense? I'll > > apply it if there are no objections. > > Note: the bare except is bothersome. I confess I wasn't sure if a news server that didn't recognize the 'mode reader' could be counted on to return a 500 series error. It seemed logical, but I couldn't be sure. (All of the servers I have access to accept the 'mode reader'.) > Note, though, that i've already posted a small patch to mailman to fix this. > Since the 'group' command is executed by gate_news, which doesn't need > authentication, it's easy enough to try the group, and set the connection to > readermode if it fails. I dislike your proposed solution for two reasons: 1. It uses the nntplib's shortcmd outside of nntplib, even though it is documented as "internal". 2. It doesn't work when you *do* need authentication. The group command will claim 'No such group' for groups that are hidden by authentication. (One of my mailman run on such a machine.) You need to do the 'mode reader' before authenticating. I still think it better to change nntplib. > Also note that I've already sent a patch to patches@python.org ;) with a > pointer to the mailman-users discussion in question. > > http://www.python.org/pipermail/patches/2000-February/000015.html > > My patch uses an extra argument to NNTP.__init__, and avoids the bare > except. To get the same behaviour as the patch below, sending 'mode > reader' by default, the default value of readermode can be set to 1. I'm not > sure if that's a wise thing, however, as some users of nntplib.NNTP might > depend on the connection being able to do things that you cant do in > readermode. I sent a similar nntplib patch to mailman-users last week, although I defaulted to the current behavior and required the additional argument (or a user specified for authentication) to get the "newsreader" behavior. http://www.python.org/pipermail/mailman-users/2000-February/003935.html Jim --FCuugMFkClbJLl1L Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="nntplib.diff" --- /home/jwt/nntplib.py.orig Sat Sep 18 09:16:08 1999 +++ nntplib.py Sat Sep 18 11:39:00 1999 @@ -61,17 +61,24 @@ # Initialize an instance. Arguments: # - host: hostname to connect to # - port: port to connect to (default the standard NNTP port) + # - mode: if not None (or user is specified), issue 'mode reader' - def __init__(self, host, port = NNTP_PORT, user=None, password=None): + def __init__(self, host, port = NNTP_PORT, user=None, password=None, + mode=None): self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect(self.host, self.port) self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if user or mode: + try: + self.welcome = self.shortcmd('mode reader') + except: + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: @@ -466,9 +473,9 @@ # Minimal test function def _test(): - s = NNTP('news') + s = NNTP('news', mode='reader') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last resp, subs = s.xhdr('subject', first + '-' + last) --FCuugMFkClbJLl1L-- From bwarsaw@cnri.reston.va.us Tue Feb 8 17:08:09 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Tue, 8 Feb 2000 12:08:09 -0500 (EST) Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <14496.19833.341973.55133@anthem.cnri.reston.va.us> >>>>> "JT" == Jim Tittsler writes: JT> I still think it better to change nntplib. I agree. I'd still like to get rid of your bare except though; I'm guessing that error_perm is probably correct because of this: >>> import nntplib >>> x = nntplib.NNTP('ournews') >>> x.shortcmd('mode foobar') Traceback (innermost last): File "", line 1, in ? File "/depot/sundry/lib/python1.5/nntplib.py", line 158, in shortcmd return self.getresp() File "/depot/sundry/lib/python1.5/nntplib.py", line 134, in getresp raise error_perm, resp nntplib.error_perm: 500 Syntax error or bad command The Right Thing To Do is to recast the nntplib errors as exception classes and perhaps use it's base class in the except. I'd be willing to make this change, and merge in your patch, but you'll need to resend your patch with the standard email disclaimer appended. See http://www.python.org/patches/ for details. Would you be willing to do that? Thanks, -Barry From skip@mojam.com (Skip Montanaro) Tue Feb 8 21:01:33 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 8 Feb 2000 15:01:33 -0600 Subject: [Patches] Re: urllib suggestion Message-ID: <200002082101.PAA19047@beluga.mojam.com> A request to the python-help mailing list by Markus Demleitner suggested exposing URLOpener.server_version at the module level so that applications can override it as appropriate. The appended patch adds a __client_name__ variable to the module level which is initialized to "Python-urllib" and renames __version__ to __client_version__. These variables are used to construct URLopener.server_version. I renamed __version__ to __client_version__ to make it clear that the version number is sent from the client to the server, is not there simply to document the version of urllib.py. To affect the headers sent as a result of a call to urlopen or urlretrieve, they must be set before either function is called. An alternative is to document that the undocumented FancyURLopener class can be subclassed to override server_version and this class then used as the value of urllib._urlopener for use by urlopen and urlretrieve. I'm not certain which alternative is better. Both mechanisms require the programmer to read the code. I'll let someone else argue the pros and cons of the two approaches (though URLopener and FancyURLopener should probably be documented). A minor nit: I also changed the triple-quoted module doc string to use ''' instead of """ as the string delimiter. This helps Emacs' python-mode behave better when colorizing the file's contents since the doc string contains " characters. Keep or toss. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ 847-971-7098 "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. *** /tmp/urllib.py.~1.91~ Tue Feb 8 14:45:13 2000 --- /tmp/urllib.py Tue Feb 8 14:45:13 2000 *************** *** 1,4 **** ! """Open an arbitrary URL. See the following document for more info on URLs: "Names and Addresses, URIs, URLs, URNs, URCs", at --- 1,4 ---- ! '''Open an arbitrary URL. See the following document for more info on URLs: "Names and Addresses, URIs, URLs, URNs, URCs", at *************** *** 20,26 **** The info() method returns a mimetools.Message object which can be used to query various info about the object, if available. (mimetools.Message objects are queried with the getheader() method.) ! """ import string import socket --- 20,26 ---- The info() method returns a mimetools.Message object which can be used to query various info about the object, if available. (mimetools.Message objects are queried with the getheader() method.) ! ''' import string import socket *************** *** 28,34 **** import sys ! __version__ = '1.12' # XXX This version is not always updated :-( MAXFTPCACHE = 10 # Trim the ftp cache beyond this size --- 28,37 ---- import sys ! # applications can override the following two variables to affect the ! # User-agent header that is sent to the client ! __client_version__ = '1.12' # XXX This version is not always updated :-( ! __client_name__ = 'Python-urllib' MAXFTPCACHE = 10 # Trim the ftp cache beyond this size *************** *** 89,95 **** self.proxies = proxies self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') ! server_version = "Python-urllib/%s" % __version__ self.addheaders = [('User-agent', server_version)] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() --- 92,98 ---- self.proxies = proxies self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') ! server_version = "%s/%s" % (__client_name__, __client_version__) self.addheaders = [('User-agent', server_version)] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() From bwarsaw@cnri.reston.va.us Tue Feb 8 23:18:14 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Tue, 8 Feb 2000 18:18:14 -0500 (EST) Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <14496.42038.761976.368014@anthem.cnri.reston.va.us> Okay, here's my proposed patch to the current CVS version of nntplib.py. It merges Jim's and Thomas's patches, and re-implements the module's exceptions as class-based (with backwards compatibility). I'm not sure what to do about the email disclaimers, but I'd say to be safe that both Jim and Thomas should re-submit their patches with the disclaimer attached (you can just to this to patches@python.org). Everyone else on patches@python.org: please let me know if you have any objections to this change. Otherwise, I'll commit it (and make the necessary documentation updates). -Barry -------------------- snip snip -------------------- Index: nntplib.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/nntplib.py,v retrieving revision 1.17 diff -c -r1.17 nntplib.py *** nntplib.py 2000/02/04 15:10:33 1.17 --- nntplib.py 2000/02/08 23:11:44 *************** *** 34,47 **** import string ! # Exception raised when an error or invalid response is received ! error_reply = 'nntplib.error_reply' # unexpected [123]xx reply ! error_temp = 'nntplib.error_temp' # 4xx errors ! error_perm = 'nntplib.error_perm' # 5xx errors ! error_proto = 'nntplib.error_proto' # response does not begin with [1-5] ! error_data = 'nntplib.error_data' # error in response data # Standard port used by NNTP servers NNTP_PORT = 119 --- 34,79 ---- import string ! ! # Exceptions raised when an error or invalid response is received ! class NNTPError(Exception): ! """Base class for all nntplib exceptions""" ! def __init__(self, *args): ! apply(Exception.__init__, (self,)+args) ! try: ! self.response = args[0] ! except IndexError: ! self.response = 'No response given' ! ! class NNTPReplyError(NNTPError): ! """Unexpected [123]xx reply""" ! pass ! ! class NNTPTemporaryError(NNTPError): ! """4xx errors""" ! pass ! ! class NNTPPermanentError(NNTPError): ! """5xx errors""" ! pass ! ! class NNTPProtocolError(NNTPError): ! """Response does not begin with [1-5]""" ! pass ! ! class NNTPDataError(NNTPError): ! """Error in response data""" ! pass ! ! # for backwards compatibility ! error_reply = NNTPReplyError ! error_temp = NNTPTemporaryError ! error_perm = NNTPPermanentError ! error_proto = NNTPProtocolError ! error_data = NNTPDataError + # Standard port used by NNTP servers NNTP_PORT = 119 *************** *** 54,68 **** CRLF = '\r\n' # The class itself - class NNTP: ! ! def __init__(self, host, port = NNTP_PORT, user=None, password=None): """Initialize an instance. Arguments: - host: hostname to connect to ! - port: port to connect to (default the standard NNTP port)""" ! self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) --- 86,108 ---- CRLF = '\r\n' + # The class itself class NNTP: ! def __init__(self, host, port = NNTP_PORT, user=None, password=None, ! readermode=None): """Initialize an instance. Arguments: - host: hostname to connect to ! - port: port to connect to (default the standard NNTP port) ! - user: username to authenticate with ! - password: password to use with username ! - readermode: if true, send 'mode reader' command after ! connecting. ! ! readermode is sometimes necessary if you are connecting to an ! NNTP server on your local machine and intend to call the ! group() method. ! """ self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) *************** *** 70,85 **** self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: ! raise error_reply, resp else: resp = self.shortcmd( 'authinfo pass '+password) if resp[:3] != '281': ! raise error_perm, resp def getwelcome(self): """Get the welcome message from the server --- 110,136 ---- self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if readermode: + try: + self.welcome = self.shortcmd('mode reader') + except NNTPPermanentError: + # error 500, probably 'not implemented' + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: ! raise NNTPReplyError(resp) else: resp = self.shortcmd( 'authinfo pass '+password) if resp[:3] != '281': ! raise NNTPPermanentError(resp) ! ! # Get the welcome message from the server ! # (this is read and squirreled away by __init__()). ! # If the response code is 200, posting is allowed; ! # if it 201, posting is not allowed def getwelcome(self): """Get the welcome message from the server *************** *** 128,138 **** if self.debugging: print '*resp*', `resp` c = resp[:1] if c == '4': ! raise error_temp, resp if c == '5': ! raise error_perm, resp if c not in '123': ! raise error_proto, resp return resp def getlongresp(self): --- 179,189 ---- if self.debugging: print '*resp*', `resp` c = resp[:1] if c == '4': ! raise NNTPTemporaryError(resp) if c == '5': ! raise NNTPPermanentError(resp) if c not in '123': ! raise NNTPProtocolError(resp) return resp def getlongresp(self): *************** *** 140,146 **** Raise various errors if the response indicates an error.""" resp = self.getresp() if resp[:3] not in LONGRESP: ! raise error_reply, resp list = [] while 1: line = self.getline() --- 191,197 ---- Raise various errors if the response indicates an error.""" resp = self.getresp() if resp[:3] not in LONGRESP: ! raise NNTPReplyError(resp) list = [] while 1: line = self.getline() *************** *** 206,212 **** resp = self.shortcmd('GROUP ' + name) if resp[:3] <> '211': ! raise error_reply, resp words = string.split(resp) count = first = last = 0 n = len(words) --- 257,263 ---- resp = self.shortcmd('GROUP ' + name) if resp[:3] <> '211': ! raise NNTPReplyError(resp) words = string.split(resp) count = first = last = 0 n = len(words) *************** *** 230,236 **** def statparse(self, resp): """Internal: parse the response of a STAT, NEXT or LAST command.""" if resp[:2] <> '22': ! raise error_reply, resp words = string.split(resp) nr = 0 id = '' --- 281,287 ---- def statparse(self, resp): """Internal: parse the response of a STAT, NEXT or LAST command.""" if resp[:2] <> '22': ! raise NNTPReplyError(resp) words = string.split(resp) nr = 0 id = '' *************** *** 349,355 **** elem[6], elem[7])) except IndexError: ! raise error_data,line return resp,xover_lines def xgtitle(self, group): --- 400,406 ---- elem[6], elem[7])) except IndexError: ! raise NNTPDataError(line) return resp,xover_lines def xgtitle(self, group): *************** *** 377,387 **** resp = self.shortcmd("XPATH " + id) if resp[:3] <> '223': ! raise error_reply, resp try: [resp_num, path] = string.split(resp) except ValueError: ! raise error_reply, resp else: return resp, path --- 428,438 ---- resp = self.shortcmd("XPATH " + id) if resp[:3] <> '223': ! raise NNTPReplyError(resp) try: [resp_num, path] = string.split(resp) except ValueError: ! raise NNTPReplyError(resp) else: return resp, path *************** *** 395,408 **** resp = self.shortcmd("DATE") if resp[:3] <> '111': ! raise error_reply, resp elem = string.split(resp) if len(elem) != 2: ! raise error_data, resp date = elem[1][2:8] time = elem[1][-6:] if len(date) != 6 or len(time) != 6: ! raise error_data, resp return resp, date, time --- 446,459 ---- resp = self.shortcmd("DATE") if resp[:3] <> '111': ! raise NNTPReplyError(resp) elem = string.split(resp) if len(elem) != 2: ! raise NNTPDataError(resp) date = elem[1][2:8] time = elem[1][-6:] if len(date) != 6 or len(time) != 6: ! raise NNTPDataError(resp) return resp, date, time *************** *** 415,421 **** resp = self.shortcmd('POST') # Raises error_??? if posting is not allowed if resp[0] <> '3': ! raise error_reply, resp while 1: line = f.readline() if not line: --- 466,472 ---- resp = self.shortcmd('POST') # Raises error_??? if posting is not allowed if resp[0] <> '3': ! raise NNTPReplyError(resp) while 1: line = f.readline() if not line: *************** *** 439,445 **** resp = self.shortcmd('IHAVE ' + id) # Raises error_??? if the server already has it if resp[0] <> '3': ! raise error_reply, resp while 1: line = f.readline() if not line: --- 490,496 ---- resp = self.shortcmd('IHAVE ' + id) # Raises error_??? if the server already has it if resp[0] <> '3': ! raise NNTPReplyError(resp) while 1: line = f.readline() if not line: *************** *** 465,471 **** def _test(): """Minimal test function.""" ! s = NNTP('news') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last --- 516,522 ---- def _test(): """Minimal test function.""" ! s = NNTP('news', readermode='reader') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last From thomas@xs4all.net Wed Feb 9 07:51:51 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Feb 2000 08:51:51 +0100 Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <14496.42038.761976.368014@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Tue, Feb 08, 2000 at 06:18:14PM -0500 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> <14496.42038.761976.368014@anthem.cnri.reston.va.us> Message-ID: <20000209085151.Z403@xs4all.nl> On Tue, Feb 08, 2000 at 06:18:14PM -0500, bwarsaw@cnri.reston.va.us wrote: > I'm not sure what to do about the email disclaimers, but I'd say to be > safe that both Jim and Thomas should re-submit their patches with the > disclaimer attached (you can just to this to patches@python.org). Mine already had the disclaimer attached. Let me know if i did that wrong ;) I'd like to make one little suggestion for your docstring, though. """ readermode is sometimes necessary if you are connecting to an NNTP server on your local machine and intend to call the group() method. """ Group isn't the only reader-specific command. Commands such as 'listgroup ' and 'xover ' are nnrpd-specific, as well. (at least on the only newsserver I can currently test with.) I'd suggest this: """ readermode is sometimes necessary if you are connecting to an NNTP server on your local machine and intend to call reader-specific commands, such as 'group'. """ And dont forget to mention something like 'if you get unexpected NNTPPermanentErrors, you might need to set readermode.' -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From jwt@dskk.co.jp Wed Feb 9 10:12:04 2000 From: jwt@dskk.co.jp (Jim Tittsler) Date: Wed, 9 Feb 2000 19:12:04 +0900 Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) Message-ID: <20000209191204.A4539@mail.dskk.co.jp> --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii [At Barry Warsaw's request, this note is provided for historical purposes. He has implemented similar functionality in a cleaner way.] When nntplib is used on a machine that is an NNTP peer of the news server, it defaults to "feeder" mode. It is useful to have an optional keyword argument to __init__ to select "mode reader" to get NNRP functionality prior to authentication or other commands. Default operation of NNTP is unchanged. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Jim Tittsler, Tokyo jwt@dskk.co.jp --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="nntplib.diff" --- /home/jwt/nntplib.py.orig Sat Sep 18 09:16:08 1999 +++ nntplib.py Sat Sep 18 11:39:00 1999 @@ -61,17 +61,24 @@ # Initialize an instance. Arguments: # - host: hostname to connect to # - port: port to connect to (default the standard NNTP port) + # - mode: if not None (or user is specified), issue 'mode reader' - def __init__(self, host, port = NNTP_PORT, user=None, password=None): + def __init__(self, host, port = NNTP_PORT, user=None, password=None, + mode=None): self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect(self.host, self.port) self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if user or mode: + try: + self.welcome = self.shortcmd('mode reader') + except: + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: @@ -466,9 +473,9 @@ # Minimal test function def _test(): - s = NNTP('news') + s = NNTP('news', mode='reader') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last resp, subs = s.xhdr('subject', first + '-' + last) --yrj/dFKFPuw6o+aM-- From Vladimir.Marangozov@inrialpes.fr Wed Feb 9 14:59:48 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 9 Feb 2000 15:59:48 +0100 (CET) Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <20000205092822.A2204@acs.ucalgary.ca> from "nascheme@enme.ucalgary.ca" at Feb 05, 2000 09:28:23 AM Message-ID: <200002091459.PAA02594@python.inrialpes.fr> nascheme@enme.ucalgary.ca wrote: > > On Sat, Feb 05, 2000 at 03:00:59PM +0100, Vladimir Marangozov wrote: > > Neil, could you please resubmit your 1st GC patch with the disclaimer, > > so that we can focus on this 1st step, test the patch and fold it into > > CVS? > > No problem. I've included my Boehm-Demers-Weiser garbage > collector patch too as I have improved it to use autoconfig. The > gc patch is small compared to the malloc cleanup. Thanks. After a closer look at the patch, I see a couple of problems with it. In particular, it mixes different API families. Let me start from the start: The goal is to remove Python's dependency on the standard POSIX interface (malloc/realloc/free) so that we can cleanly and easily plug in the future a "proprietary" mem manager, other than the one in the C library. For this purpose, the Python core should be patched and "cleaned" to use one or more of the following APIs: 1) PyMem_MALLOC 2) PyMem_NEW PyMem_REALLOC ==> PyMem_RESIZE PyMem_FREE PyMem_DEL PyMem_XDEL The proposed Guido's augmented version of 1) which is raw mem interface. more "Pythonic" and contains some safety additions, like _PyMem_EXTRA). This one (in mymalloc.h) should be defined in terms of 1). 3) Py_Malloc 4) PyMem_Malloc 5) _PyObject_New Py_Realloc PyMem_Realloc PyObject_NEW Py_Free PyMem_Free PyObject_NEW_VAR, ... These are implemented using 1) and/or 2) It seems to me that presently nobody uses the wrappers 3) and 4), (except one single occurence of Py_Malloc, lost in _localemodule.c), because these wrappers cause an additional func call overhead... All these APIs are certainly redundant and the added value of each of them has to be weighted one again, but for now, let's assume that we have them all (without more argumentation - we could throw away some of them later). The rule of thumb in this situation is: Every chunk of memory must be manupulated via the same malloc family. That is, if one gets some piece of mem through PyMem_MALLOC (1), s/he must release that memory with PyMem_FREE (1). Accordingly, if one gets a chunk via PyMem_MALLOC (1), that chunk *should not* be released with PyMem_DEL (2). (which is what Neil's patch does, not to mention that (2) is not defined in terms of (1). So we must be careful here. In particular, when patching a file, we must figure out which family (-ies) is (are) used in order to perform the "malloc cleanup" the right way. That is, we have to inspect every file one by one, once again. ----------- With the above theory in mind, I see that there are a couple of "problematic" files in the current snapshot. I remember that I have identified them with my allocator (pymalloc) either. (I'll use some excerpts of Neil's patch to illustrate the problems I'm talking about) 1) pypcre.c This one is "buggy" for sure. The PCRE code allocates memory through the proprietary functions "pcre_malloc/pcre_free" (which default to malloc/free), so I really don't see why there's code inside mentioning "free" and not "pcre_free". a) Would this code be dropped with the inclusion of /F's re engine in 1.6? b) If the answer to a) is negative, I hope AMK & the String-SIG will help us on this (it's a hairy code :-). > diff -cr Python-cvs/Modules/pypcre.c Python-gc/Modules/pypcre.c > *** Python-cvs/Modules/pypcre.c Sat Jul 17 12:13:10 1999 > --- Python-gc/Modules/pypcre.c Sat Feb 5 09:00:47 2000 > *************** > *** 3057,3068 **** > static int free_stack(match_data *md) > { > /* Free any stack space that was allocated by the call to match(). */ > ! if (md->off_num) free(md->off_num); > ! if (md->offset_top) free(md->offset_top); > ! if (md->r1) free(md->r1); > ! if (md->r2) free(md->r2); > ! if (md->eptr) free((char *)md->eptr); > ! if (md->ecode) free((char *)md->ecode); > return 0; > } > > --- 3057,3068 ---- > static int free_stack(match_data *md) > { > /* Free any stack space that was allocated by the call to match(). */ > ! if (md->off_num) PyMem_DEL(md->off_num); > ! if (md->offset_top) PyMem_DEL(md->offset_top); > ! if (md->r1) PyMem_DEL(md->r1); > ! if (md->r2) PyMem_DEL(md->r2); > ! if (md->eptr) PyMem_DEL((char *)md->eptr); > ! if (md->ecode) PyMem_DEL((char *)md->ecode); > return 0; > } 2) _tkinter.c I remember from my experiments that this one was really a mess from a malloc point of view because it caused lots of mixed API calls (once the core allocator was changed). I have to look at it carefully once again... 3) readline.c Neil, what's this? Could you elaborate on this one? > > diff -cr Python-cvs/Modules/readline.c Python-gc/Modules/readline.c > *** Python-cvs/Modules/readline.c Sat Feb 5 09:00:14 2000 > --- Python-gc/Modules/readline.c Sat Feb 5 09:00:47 2000 > *************** > *** 41,46 **** > --- 41,63 ---- > extern int (*PyOS_InputHook)(); > extern char *(*PyOS_ReadlineFunctionPointer) Py_PROTO((char *)); > > + /* convert a string allocated by malloc to one allocated by PyMem_MALLOC */ > + static char * > + malloc2PyMem(char *p) > + { > + char *py; > + int n; > + if (p == NULL) { > + return p; > + } else { > + n = strlen(p); > + py = PyMem_MALLOC(n); > + strncpy(py, p, n+1); > + free(p); > + return py; > + } > + } > + All in all, this patch isn't acceptable for me and I suggest that we prepare another one which respects "the rule", after demystifying some of the issues mentioned above. Note that introducing PyMem_MALLOC would constitute an additional constraint for C Python coders, who're used to malloc/free. In order to pave the way for alternatives to libc's malloc, we must propose a clean solution. Hence it would be good to collect as much opinions/reactions on the subject as possible and settle on an interface which would be convenient for everybody. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From bwarsaw@cnri.reston.va.us Wed Feb 9 19:57:58 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 9 Feb 2000 14:57:58 -0500 (EST) Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> <14496.42038.761976.368014@anthem.cnri.reston.va.us> <20000209085151.Z403@xs4all.nl> Message-ID: <14497.50886.90275.843869@anthem.cnri.reston.va.us> >>>>> "TW" == Thomas Wouters writes: TW> Mine already had the disclaimer attached. Let me know if i did TW> that wrong ;) I'd like to make one little suggestion for your TW> docstring, though. Oops, yes I see that now. I had to check the patches archive to be sure (I probably deleted the message with that text). Thanks for the docstring fix -- I will apply that. -Barry From nascheme@enme.ucalgary.ca Thu Feb 10 00:14:19 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Wed, 9 Feb 2000 17:14:19 -0700 Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <200002091459.PAA02594@python.inrialpes.fr>; from marangoz@python.inrialpes.fr on Wed, Feb 09, 2000 at 03:59:48PM +0100 References: <20000205092822.A2204@acs.ucalgary.ca> <200002091459.PAA02594@python.inrialpes.fr> Message-ID: <20000209171419.A10163@acs.ucalgary.ca> On Wed, Feb 09, 2000 at 03:59:48PM +0100, Vladimir Marangozov wrote: > The goal is to remove Python's dependency on the standard POSIX interface > (malloc/realloc/free) so that we can cleanly and easily plug in the future > a "proprietary" mem manager, other than the one in the C library. For this > purpose, the Python core should be patched and "cleaned" to use one or more > of the following APIs: [...] Shouldn't all these be based on the same malloc? We could define everything in terms of PyMem_MALLOC, PyMem_REALLOC, and PyMem_FREE if that makes things clearer. > Every chunk of memory must be manupulated via the same malloc family. Yes, and this is where things get tricky. Extension modules can use malloc to allocate objects and pass them to the Python core. Python uses PyMem_FREE or similar and *boom*, memory corruption (if they are different mallocs). > That is, if one gets some piece of mem through PyMem_MALLOC (1), > s/he must release that memory with PyMem_FREE (1). Accordingly, if one > gets a chunk via PyMem_MALLOC (1), that chunk *should not* be released > with PyMem_DEL (2). (which is what Neil's patch does, not to mention > that (2) is not defined in terms of (1). I don't think we want more than one malloc within Python. IMHO, it would be impossible to keep the calls straight. Why would we want PyMem_MALLOC to use a different malloc than PyMem_NEW? > 1) pypcre.c > > This one is "buggy" for sure. The PCRE code allocates memory through > the proprietary functions "pcre_malloc/pcre_free" (which default to > malloc/free), so I really don't see why there's code inside mentioning > "free" and not "pcre_free". No, I think it is okay. All the memory allocated with pcre_malloc is deallocated with pcre_free. The places that use free haved been changed to PyMem_FREE because that memory has come from the Python interpeter. I spent some time tracking this one down. The fact that you are sure this is buggy shows how tricky this business is. Of course, I could always be wrong too. :) > 3) readline.c > > Neil, what's this? Could you elaborate on this one? Its very ugly. Readline returns memory allocated by malloc. That memory is eventually freed from within the interpreter by PyMem_FREE. We can't change the PyMem_FREE call to free because it also frees memory allocated by PyMem_MALLOC. > Note that introducing PyMem_MALLOC would constitute an additional > constraint for C Python coders, who're used to malloc/free. It has to be assumed that C Python coders use PyMem_MALLOC when passing memory to Python and PyMem_FREE when deleting memory from the interpeter. Not a very good assumption I know. That is why real garbage collection is so hard to add to Python. I have already tested this patch with a different PyMem_MALLOC and malloc (the libc malloc and the Boehm GC malloc). I spent a long time looking a core dumps and straightening out the memory calls. It is possible that I missed some "API mixing" that my tests did not cover however. Note that I am working on the garbage collection scheme proposed by Toby Kelsey and Tim Peters. Right now things are looking promising. The cost seems to be quite low. It can find almost all reference cycles that would occur in real programs. Finally, it should be completely portable and does not require the use of a different malloc. I will be posting the code to the Python patches list shortly. Neil From Moshe Zadka Thu Feb 10 05:47:04 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 10 Feb 2000 07:47:04 +0200 (IST) Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <20000209171419.A10163@acs.ucalgary.ca> Message-ID: On Wed, 9 Feb 2000 nascheme@enme.ucalgary.ca wrote: > > This one is "buggy" for sure. The PCRE code allocates memory through > > the proprietary functions "pcre_malloc/pcre_free" (which default to > > malloc/free), so I really don't see why there's code inside mentioning > > "free" and not "pcre_free". > > No, I think it is okay. All the memory allocated with > pcre_malloc is deallocated with pcre_free. The places that use > free haved been changed to PyMem_FREE because that memory has > come from the Python interpeter. I spent some time tracking this > one down. Well, actually pcre should be a no brainer: it was written with "weird" allocators in mind: IMHO, Python should just set pcre_malloc/pcre_free to PyMem_Malloc and PyMem_FREE. Things should be easier then. > Its very ugly. Readline returns memory allocated by malloc. > That memory is eventually freed from within the interpreter by > PyMem_FREE. We can't change the PyMem_FREE call to free because > it also frees memory allocated by PyMem_MALLOC. Ummm....isn't there some way to override readline's allocator? if not, then it might not be a bad idea to submit a patch to the readline maintainer allowing that. Guile has GC and IIRC uses readline, so I'll check and see what's happening over there. > Note that I am working on the garbage collection scheme proposed > by Toby Kelsey and Tim Peters. Right now things are looking > promising. The cost seems to be quite low. It can find almost > all reference cycles that would occur in real programs. Finally, > it should be completely portable and does not require the use of > a different malloc. I will be posting the code to the Python > patches list shortly. Way cool! waiting-for-python-1.7-with-the-ultra-snazzy-garbage-collector-ly y'rs, Z. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From Vladimir.Marangozov@inrialpes.fr Thu Feb 10 11:06:05 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 10 Feb 2000 12:06:05 +0100 (CET) Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <20000209171419.A10163@acs.ucalgary.ca> from "nascheme@enme.ucalgary.ca" at Feb 09, 2000 05:14:19 PM Message-ID: <200002101106.MAA03993@python.inrialpes.fr> nascheme@enme.ucalgary.ca wrote: > > On Wed, Feb 09, 2000 at 03:59:48PM +0100, Vladimir Marangozov wrote: > > The goal is to remove Python's dependency on the standard POSIX interface > > (malloc/realloc/free) so that we can cleanly and easily plug in the future > > a "proprietary" mem manager, other than the one in the C library. For this > > purpose, the Python core should be patched and "cleaned" to use one or more > > of the following APIs: > [...] > > Shouldn't all these be based on the same malloc? We could > define everything in terms of PyMem_MALLOC, PyMem_REALLOC, and > PyMem_FREE if that makes things clearer. Yes, everything should be built on top/in terms of PyMem_MALLOC, including the PyMem_NEW family. That's exactly what I was trying to say. Thus switching to another malloc would equal changing 3 macros. > > > Every chunk of memory must be manupulated via the same malloc family. > > Yes, and this is where things get tricky. Extension modules can > use malloc to allocate objects and pass them to the Python core. > Python uses PyMem_FREE or similar and *boom*, memory corruption > (if they are different mallocs). No! This should not happen if the extension writer knows what she does! Two cases: a) If Python allocates a chunk, it's Python's job to release it. b) If a library allocates a chunk, it's the library's job to release it. Corrolaries: c) If a library allocates a chunk for Python, it's not Python's job to release it. d) If Python allocates a chunk for a library, it's not the library's job to release it. IOW, modules must manage *only* their own kids. If c) and d) happen to be false, this is a sure sign of bad design. This means that if Python gets some new mem from an extension, e.g. through "tree *t = new_tree()", and this tree has to be released, the extension must export a "del_tree(t)" function that Python should call. If it does not, then "it is implicitely assumed that the extension uses libc's malloc() for allocating any new mem, and the exposed function for releasing it by others is libc's free()". I agree that this may be tricky sometimes, but the principles remain the same. BTW, you'll note that for all Python objects, Guido has respected this rule: the code for allocating and releasing an object is always in the same file (e.g: PyInt_New and int_dealloc) and this is done systematically for all objects. i.e every object implementation exports it's own malloc/free. > > > That is, if one gets some piece of mem through PyMem_MALLOC (1), > > s/he must release that memory with PyMem_FREE (1). Accordingly, if one > > gets a chunk via PyMem_MALLOC (1), that chunk *should not* be released > > with PyMem_DEL (2). (which is what Neil's patch does, not to mention > > that (2) is not defined in terms of (1). > > I don't think we want more than one malloc within Python. I don't think we need that either. > IMHO, it would be impossible to keep the calls straight. Why would we > want PyMem_MALLOC to use a different malloc than PyMem_NEW? We don't want that. Do we? I don't think I've said anything like that. I insisted on the principle of not mixing the APIs. But all API's are defined in terms of PyMem_MALLOC. > > > 1) pypcre.c > > > > The fact that you are sure this is buggy shows how tricky this > business is. Of course, I could always be wrong too. :) Ok. I admit that I never insisted on fully understanding this code. So free() should be PyMem_FREE, right? > > > 3) readline.c > > > > Neil, what's this? Could you elaborate on this one? > > Its very ugly. Readline returns memory allocated by malloc. > That memory is eventually freed from within the interpreter by > PyMem_FREE. How so? This shouldn't happen! At least not with PyMem_FREE! See my plea above. If Python has to free mem allocated by readline (i.e. by malloc() implicitely), we have to use free(). > We can't change the PyMem_FREE call to free because > it also frees memory allocated by PyMem_MALLOC. Ah! This is another story. Can't we manage to separate the two things when freeing? If we managed to mix them (malloc() + PyMem_MALLOC), we should be able to separate them and use free() + PyMem_FREE. About GC: let's discuss that later and stay focused on the patch. It's not very exciting as a task, but it needs to be done and it will pay later. FWIW, your collective progress on GC sounds really good! -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From nascheme@enme.ucalgary.ca Thu Feb 10 14:05:04 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Thu, 10 Feb 2000 07:05:04 -0700 Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <200002101106.MAA03993@python.inrialpes.fr>; from marangoz@python.inrialpes.fr on Thu, Feb 10, 2000 at 12:06:05PM +0100 References: <20000209171419.A10163@acs.ucalgary.ca> <200002101106.MAA03993@python.inrialpes.fr> Message-ID: <20000210070504.A20339@acs.ucalgary.ca> On Thu, Feb 10, 2000 at 12:06:05PM +0100, Vladimir Marangozov wrote: > Yes, everything should be built on top/in terms of PyMem_MALLOC, > including the PyMem_NEW family. That's exactly what I was trying to say. Okay. We agree on this. Why are you making a big deal out of using PyMem_DEL vs. PyMem_FREE then? They do the same thing. Is there any advantage to keeping them separate? > I insisted on the principle of not mixing the APIs. Why? Is there any reason PyMem_DEL would ever do something different than PyMem_FREE? Perhaps we should just get rid of (depreciate?) one set of APIs. [on the readline hack] > Ah! This is another story. Can't we manage to separate the two things > when freeing? If we managed to mix them (malloc() + PyMem_MALLOC), we > should be able to separate them and use free() + PyMem_FREE. I guess so. Readline is passing strings to the parser. The parser also gets strings from other places. It is tricky. I don't think I want to sit down and figure it out. The best solution is probably to get readline to use Python's malloc. It is a deficiency of the readline library and should be fixed. Neil From skip@mojam.com (Skip Montanaro) Thu Feb 10 14:51:36 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 10 Feb 2000 08:51:36 -0600 Subject: [Patches] Add -t flag to Python executable for test and install targets Message-ID: <200002101451.IAA28249@beluga.mojam.com> In python-dev Paul Prescod raised the question whether -t (tab warning) should be the default behavior in future versions of Python. In the ensuing discussion someone commented that the standard for the Python distribution is to indent using all spaces. I decided to add -t to the test and install targets to catch problems (caught a few - I'll try and send patches shortly). The attached diff for the toplevel Makefile.in seems to do the trick. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. *** /tmp/Makefile.in.~1.81~ Thu Feb 10 08:41:53 2000 --- /tmp/Makefile.in Thu Feb 10 08:41:53 2000 *************** *** 234,240 **** # Test the interpreter (twice, once without .pyc files, once with) TESTOPTS= TESTPROG= $(srcdir)/Lib/test/regrtest.py ! TESTPYTHON= ./python$(EXE) test: all -rm -f $(srcdir)/Lib/test/*.py[co] -PYTHONPATH= $(TESTPYTHON) $(TESTPROG) $(TESTOPTS) --- 234,240 ---- # Test the interpreter (twice, once without .pyc files, once with) TESTOPTS= TESTPROG= $(srcdir)/Lib/test/regrtest.py ! TESTPYTHON= ./python$(EXE) -t test: all -rm -f $(srcdir)/Lib/test/*.py[co] -PYTHONPATH= $(TESTPYTHON) $(TESTPROG) $(TESTOPTS) *************** *** 346,352 **** done; \ done PYTHONPATH=$(LIBDEST) \ ! ./python$(EXE) $(LIBDEST)/compileall.py $(LIBDEST) PYTHONPATH=$(LIBDEST) \ ./python$(EXE) -O $(LIBDEST)/compileall.py $(LIBDEST) --- 346,352 ---- done; \ done PYTHONPATH=$(LIBDEST) \ ! ./python$(EXE) -t $(LIBDEST)/compileall.py $(LIBDEST) PYTHONPATH=$(LIBDEST) \ ./python$(EXE) -O $(LIBDEST)/compileall.py $(LIBDEST) From skip@mojam.com (Skip Montanaro) Thu Feb 10 14:55:57 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 10 Feb 2000 08:55:57 -0600 Subject: [Patches] string.py indentation patch Message-ID: <200002101455.IAA28516@beluga.mojam.com> It appears that the only file in the standard distribution that suffers from the tab devil is the mostly unused string.py. Attached is a context diff. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. --[[text/plain; type=patch Content-Disposition: attachment; filename="string.diff"][quoted-printable]] *** /tmp/string.py.~1.46~UNAUoC Thu Feb 10 08:54:00 2000 --- /tmp/string.pyUNAhyI Thu Feb 10 08:54:00 2000 *************** *** 197,206 **** = """ if type(s) =3D=3D _StringType: ! return _float(s) else: ! raise TypeError('argument 1: expected string, %s found' % ! type(s).__name__) = # Convert string to integer def atoi(*args): --- 197,206 ---- = """ if type(s) =3D=3D _StringType: ! return _float(s) else: ! raise TypeError('argument 1: expected string, %s found' % ! type(s).__name__) = # Convert string to integer def atoi(*args): *************** *** 215,232 **** = """ try: ! s =3D args[0] except IndexError: ! raise TypeError('function requires at least 1 argument: %d given' % ! len(args)) # Don't catch type error resulting from too many arguments to int()= =2E The # error message isn't compatible but the error type is, and this fu= nction # is complicated enough already. if type(s) =3D=3D _StringType: ! return _apply(_int, args) else: ! raise TypeError('argument 1: expected string, %s found' % ! type(s).__name__) = = # Convert string to long integer --- 215,232 ---- = """ try: ! s =3D args[0] except IndexError: ! raise TypeError('function requires at least 1 argument: %d give= n' % ! len(args)) # Don't catch type error resulting from too many arguments to int()= =2E The # error message isn't compatible but the error type is, and this fu= nction # is complicated enough already. if type(s) =3D=3D _StringType: ! return _apply(_int, args) else: ! raise TypeError('argument 1: expected string, %s found' % ! type(s).__name__) = = # Convert string to long integer *************** *** 243,260 **** = """ try: ! s =3D args[0] except IndexError: ! raise TypeError('function requires at least 1 argument: %d given' % ! len(args)) # Don't catch type error resulting from too many arguments to long(= ). The # error message isn't compatible but the error type is, and this fu= nction # is complicated enough already. if type(s) =3D=3D _StringType: ! return _apply(_long, args) else: ! raise TypeError('argument 1: expected string, %s found' % ! type(s).__name__) = = # Left-justify a string --- 243,260 ---- = """ try: ! s =3D args[0] except IndexError: ! raise TypeError('function requires at least 1 argument: %d give= n' % ! len(args)) # Don't catch type error resulting from too many arguments to long(= ). The # error message isn't compatible but the error type is, and this fu= nction # is complicated enough already. if type(s) =3D=3D _StringType: ! return _apply(_long, args) else: ! raise TypeError('argument 1: expected string, %s found' % ! type(s).__name__) = = # Left-justify a string *************** *** 296,303 **** if n <=3D 0: return s half =3D n/2 if n%2 and width%2: ! # This ensures that center(center(s, i), j) =3D center(s, j) ! half =3D half+1 return ' '*half + s + ' '*(n-half) = # Zero-fill a number, e.g., (12, 3) --> '012' and (-3, 3) --> '-03' --- 296,303 ---- if n <=3D 0: return s half =3D n/2 if n%2 and width%2: ! # This ensures that center(center(s, i), j) =3D center(s, j) ! half =3D half+1 return ' '*half + s + ' '*(n-half) = # Zero-fill a number, e.g., (12, 3) --> '012' and (-3, 3) --> '-03' *************** *** 316,322 **** if n >=3D width: return s sign =3D '' if s[0] in ('-', '+'): ! sign, s =3D s[0], s[1:] return sign + '0'*(width-n) + s = # Expand tabs in a string. --- 316,322 ---- if n >=3D width: return s sign =3D '' if s[0] in ('-', '+'): ! sign, s =3D s[0], s[1:] return sign + '0'*(width-n) + s = # Expand tabs in a string. *************** *** 331,342 **** """ res =3D line =3D '' for c in s: ! if c =3D=3D '\t': ! c =3D ' '*(tabsize - len(line) % tabsize) ! line =3D line + c ! if c =3D=3D '\n': ! res =3D res + line ! line =3D '' return res + line = # Character translation through look-up table. --- 331,342 ---- """ res =3D line =3D '' for c in s: ! if c =3D=3D '\t': ! c =3D ' '*(tabsize - len(line) % tabsize) ! line =3D line + c ! if c =3D=3D '\n': ! res =3D res + line ! line =3D '' return res + line = # Character translation through look-up table. *************** *** 385,398 **** = """ if len(fromstr) !=3D len(tostr): ! raise ValueError, "maketrans arguments must have same length" global _idmapL if not _idmapL: ! _idmapL =3D map(None, _idmap) L =3D _idmapL[:] fromstr =3D map(ord, fromstr) for i in range(len(fromstr)): ! L[fromstr[i]] =3D tostr[i] return joinfields(L, "") = # Substring replacement (global) --- 385,398 ---- = """ if len(fromstr) !=3D len(tostr): ! raise ValueError, "maketrans arguments must have same length" global _idmapL if not _idmapL: ! _idmapL =3D map(None, _idmap) L =3D _idmapL[:] fromstr =3D map(ord, fromstr) for i in range(len(fromstr)): ! L[fromstr[i]] =3D tostr[i] return joinfields(L, "") = # Substring replacement (global) *************** *** 426,429 **** from strop import maketrans, lowercase, uppercase, whitespace letters =3D lowercase + uppercase except ImportError: ! pass # Use the original versions --- 426,429 ---- from strop import maketrans, lowercase, uppercase, whitespace letters =3D lowercase + uppercase except ImportError: ! pass # Use the original ve= rsions From Vladimir.Marangozov@inrialpes.fr Thu Feb 10 15:42:16 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 10 Feb 2000 16:42:16 +0100 (CET) Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <20000210070504.A20339@acs.ucalgary.ca> from "nascheme@enme.ucalgary.ca" at Feb 10, 2000 07:05:04 AM Message-ID: <200002101542.QAA04645@python.inrialpes.fr> nascheme@enme.ucalgary.ca wrote: > > On Thu, Feb 10, 2000 at 12:06:05PM +0100, Vladimir Marangozov wrote: > > > I insisted on the principle of not mixing the APIs. > > Why? Because it's a principle ;-) that makes our life easier. > Is there any reason PyMem_DEL would ever do something > different than PyMem_FREE? They happen to do the same, but belong to different APIs. The 1st one deals with raw memory, the 2nd deals with per-type sized memory. And in the future, nobody says that the PyMem_NEW family won't evolve into something else which could require the inlining of additional processing in PyMem_DEL. That's why I'd prefer to keep things separate from each other. (We're cleaning things up after all) If we're MALLOCing, we have to be FREEing. If we're NEWing, we have to be DELing. > > Perhaps we should just get rid of (depreciate?) one set of APIs. > Perhaps, but actually they have different semantics. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From jim@interet.com Fri Feb 11 14:52:59 2000 From: jim@interet.com (James C. Ahlstrom) Date: Fri, 11 Feb 2000 09:52:59 -0500 Subject: [Patches] Add crc32() method to binascii Message-ID: <38A4224B.1A63F825@interet.com> This is a multi-part message in MIME format. --------------293ACB605291E3AB9ADB1807 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit This patch adds a new method to module binascii. It calculates a CRC-32 checksum. This method is generally useful, and is required for ZIP file checksums. Use as follows: print binascii.crc32("hello world") # Or, in two pieces: crc = binascii.crc32("hello") crc = binascii.crc32(" world", crc) print crc See below for the documentation. LEGAL ===== I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. DOCS (add to binascii docs) =========================== crc32(data, crc = 0) Compute CRC-32, the 32-bit checksum of data, starting with an initial crc. This is consistent with the ZIP file checksum. PATCH against 1.5.2 release (but just an addition) ================================================== See attached. JimA --------------293ACB605291E3AB9ADB1807 Content-Type: text/plain; charset=us-ascii; name="binascii.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="binascii.patch" *** binascii.c.orig Mon Feb 7 15:50:26 2000 --- binascii.c Fri Feb 11 09:36:06 2000 *************** *** 719,724 **** --- 719,844 ---- return Py_BuildValue("i", crc); } + static char doc_crc32[] = + "(data, oldcrc = 0) -> newcrc. Compute CRC-32 incrementally"; + + /* Crc - 32 BIT ANSI X3.66 CRC checksum files + Also known as: ISO 3307 + **********************************************************************| + * *| + * Demonstration program to compute the 32-bit CRC used as the frame *| + * check sequence in ADCCP (ANSI X3.66, also known as FIPS PUB 71 *| + * and FED-STD-1003, the U.S. versions of CCITT's X.25 link-level *| + * protocol). The 32-bit FCS was added via the Federal Register, *| + * 1 June 1982, p.23798. I presume but don't know for certain that *| + * this polynomial is or will be included in CCITT V.41, which *| + * defines the 16-bit CRC (often called CRC-CCITT) polynomial. FIPS *| + * PUB 78 says that the 32-bit FCS reduces otherwise undetected *| + * errors by a factor of 10^-5 over 16-bit FCS. *| + * *| + **********************************************************************| + + Copyright (C) 1986 Gary S. Brown. You may use this program, or + code or tables extracted from it, as desired without restriction. + + First, the polynomial itself and its table of feedback terms. The + polynomial is + X^32+X^26+X^23+X^22+X^16+X^12+X^11+X^10+X^8+X^7+X^5+X^4+X^2+X^1+X^0 + Note that we take it "backwards" and put the highest-order term in + the lowest-order bit. The X^32 term is "implied"; the LSB is the + X^31 term, etc. The X^0 term (usually shown as "+1") results in + the MSB being 1. + + Note that the usual hardware shift register implementation, which + is what we're using (we're merely optimizing it by doing eight-bit + chunks at a time) shifts bits into the lowest-order term. In our + implementation, that means shifting towards the right. Why do we + do it this way? Because the calculated CRC must be transmitted in + order from highest-order term to lowest-order term. UARTs transmit + characters in order from LSB to MSB. By storing the CRC this way, + we hand it to the UART in the order low-byte to high-byte; the UART + sends each low-bit to hight-bit; and the result is transmission bit + by bit from highest- to lowest-order term without requiring any bit + shuffling on our part. Reception works similarly. + + The feedback terms table consists of 256, 32-bit entries. Notes: + + 1. The table can be generated at runtime if desired; code to do so + is shown later. It might not be obvious, but the feedback + terms simply represent the results of eight shift/xor opera- + tions for all combinations of data and CRC register values. + + 2. The CRC accumulation logic is the same for all CRC polynomials, + be they sixteen or thirty-two bits wide. You simply choose the + appropriate table. Alternatively, because the table can be + generated at runtime, you can start by generating the table for + the polynomial in question and use exactly the same "updcrc", + if your application needn't simultaneously handle two CRC + polynomials. (Note, however, that XMODEM is strange.) + + 3. For 16-bit CRCs, the table entries need be only 16 bits wide; + of course, 32-bit entries work OK if the high 16 bits are zero. + + 4. The values must be right-shifted by eight bits by the "updcrc" + logic; the shift must be unsigned (bring in zeroes). On some + hardware you could probably optimize the shift in assembler by + using byte-swap instructions. + ********************************************************************/ + + static unsigned long crc_32_tab[256] = { + 0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3, + 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91, + 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, + 0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5, + 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, + 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59, + 0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f, + 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, + 0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433, + 0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, + 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457, + 0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65, + 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, + 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9, + 0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, + 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, + 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683, + 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, + 0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7, + 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, + 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b, + 0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79, + 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, + 0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d, + 0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, + 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, + 0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777, + 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, + 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db, + 0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, + 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf, + 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d + }; + + static PyObject * + binascii_crc32(self, args) + PyObject *self; + PyObject *args; + { /* By Jim Ahlstrom; All rights transferred to CNRI */ + unsigned char *bin_data; + unsigned long crc = 0; /* initial value of CRC */ + int len; + + if ( !PyArg_ParseTuple(args, "s#|l", &bin_data, &len, &crc) ) + return NULL; + + crc = crc ^ 0xFFFFFFFF; + while(len--) + crc = crc_32_tab[(crc ^ *bin_data++) & 0xff] ^ (crc >> 8); + /* Note: (crc >> 8) MUST zero fill on left */ + return Py_BuildValue("l", crc ^ 0xFFFFFFFF); + } + /* List of functions defined in the module */ static struct PyMethodDef binascii_module_methods[] = { *************** *** 735,740 **** --- 855,861 ---- {"rledecode_hqx", binascii_rledecode_hqx, 1, doc_rledecode_hqx}, {"crc_hqx", binascii_crc_hqx, 1, doc_crc_hqx}, + {"crc32", binascii_crc32, 1, doc_crc32}, {NULL, NULL} /* sentinel */ }; --------------293ACB605291E3AB9ADB1807-- From Fredrik Lundh" from the newsgroup: Oleg Broytmann wrote: > What am I doing wrong? And how to use urllib to get files by FTP? > > fname, headers =3D = urllib.urlretrieve("ftp://koobera.math.uic.edu/www/djb.html") > > IOError: [Errno ftp error] 426 Transmission failure: not a directory background: urllib issues an NLST command to make sure the file exists, before starting the download. however, RFC959 doesn't necessarily allow this: [NLST] causes a directory listing to be sent from server to user site. The pathname should specify a directory or other system-specific file group descriptor; a null argument implies the current directory. most FTP servers seem to treat NLST commands as an ordinary "ls -l", but this one doesn't. I'm pretty sure the only reasonable solution is to remove the offending code from urllib... -- end of posting and here's a patch that does exactly that. cannot see any problems with it, but on the other hand, I'm pretty sure the NLST call was added for a reason. please discuss before applying. *** urllib.py.bak Wed Feb 02 00:36:53 2000 --- urllib.py Fri Feb 11 15:53:51 2000 *************** *** 643,655 **** self.ftp.voidcmd(cmd) conn =3D None if file and not isdir: - # Use nlst to see if the file exists at all - try: - self.ftp.nlst(file) - except ftplib.error_perm, reason: - raise IOError, ('ftp error', reason), = sys.exc_info()[2] - # Restore the transfer mode! - self.ftp.voidcmd(cmd) # Try to retrieve as a file try: cmd =3D 'RETR ' + file --- 643,648 ---- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. From gstein@lyra.org Fri Feb 11 15:05:59 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 11 Feb 2000 07:05:59 -0800 (PST) Subject: [Patches] urllib NLST patch In-Reply-To: <021f01bf74a1$112a3a40$34aab5d4@hagrid> Message-ID: On Fri, 11 Feb 2000, Fredrik Lundh wrote: ... > background: urllib issues an NLST command to make sure > the file exists, before starting the download. however, > RFC959 doesn't necessarily allow this: > > [NLST] causes a directory listing to be sent from > server to user site. The pathname should specify a > directory or other system-specific file group descriptor; > a null argument implies the current directory. > > most FTP servers seem to treat NLST commands as > an ordinary "ls -l", but this one doesn't. > > I'm pretty sure the only reasonable solution is to remove > the offending code from urllib... > > -- end of posting > > and here's a patch that does exactly that. cannot see > any problems with it, but on the other hand, I'm pretty > sure the NLST call was added for a reason. > > please discuss before applying. Sure! FTP is bogus!! WebDAV is the Right Way!! :-) -- Greg Stein, http://www.lyra.org/ From jim@interet.com Fri Feb 11 16:23:38 2000 From: jim@interet.com (James C. Ahlstrom) Date: Fri, 11 Feb 2000 11:23:38 -0500 Subject: [Patches] New module zipfile.py Message-ID: <38A4378A.CD4BE4B3@interet.com> This is a multi-part message in MIME format. --------------D45D15E3F3D3AD5E54347B06 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit This new module consists of a single Python file, and it reads and writes ZIP files. It requires binascii.crc32, which is a separate patch. See docs below. LEGAL ===== I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. DOCS ==== zipfile -- Read and write files in ZIP format The ZIP file format is a common archive and compression standard. This module provides tools to create, read, write, append, and list a ZIP file. The available attributes of this module are: error The error raised for bad ZIP files. _debug Level of printing, default 1. class ZipFile The class for reading and writing ZIP files. is_zipfile(path) Return 1/0 if "path" is/is not a valid ZIP file based on its magic number. This module does not currently handle ZIP files which have appended comments. zip2date(zdate) Return (year, month, day) for a zip date code. zip2time(ztime) Return (hour, minute, second) for a zip time code. date2zip(year, month, day) Return a zip date code. time2zip(hour, minute, second) Return a zip time code. ZIP_STORED The numeric constant (zero) for an uncompressed archive. ZIP_DEFLATED The numeric constant for the usual ZIP compression method. This requires the zlib module. No other compression methods are currently supported. An instance of class ZipFile has self.TOC, a read-only dictionary whose keys are the names in the archive, and whose values are tuples as follows: 0: File data seek offset. 1: Zip file "extra" data as a string. 2: Zip file bit flags. 3: Zip file compression type. 4: File modification time in DOS format. 5: File modification date in DOS format. 6: The CRC-32 of the uncompressed data. 7: The compressed size of the file. 8: The uncompressed size of the file. The class ZipFile has these methods: __init__(self, filename, mode = "r", compression = 0) Open a ZIP file named "filename". Mode is "r" to read an existing file, "w" to truncate and write a new file, or "a" to append. For mode "a", if filename is a ZIP file, then additional files are added to it. If filename is not a ZIP file, then a new ZIP file is appended to the file. This is meant for adding a ZIP archive to another file such as python.exe. But "cat myzip.zip >> python.exe" also works, and at least WinZip can read such files. The "compression" is the ZIP compression method to use when writing the archive. listdir(self) Return a list of names in the archive. Equivalent to self.TOC.keys(). printdir(self) Print a table of contents for the archive to stdout. read(self, name) Return the bytes of the file in the archive. The archive must be open for read or append. writestr(self, bytes, arcname, year, month, day, hour, minute, second, extra = "") Write the string "bytes" and the other data to the archive, and give it the name "arcname". The "extra" is the ZIP extra data string. The archive must be open with mode "w" or "a". write(self, filename, arcname, extra = "") Write the file named "filename" to the archive, and give it the archive name "arcname". The "extra" is the ZIP extra data string. The archive must be open with mode "w" or "a". writepy(self, pathname, basename = "") Search for files *.py and add the corresponding file to the archive. The corresponding file is a *.pyo file if available, else a *.pyc file, compiling if necessary. If the pathname is a file, the file must end with ".py", and just the (corresponding *.py[oc]) file is added at the top level (no path information). If it is a directory, and the directory is not a package directory, then all the files *.py[oc] are added at the top level. If the directory is a package directory, then all *.py[oc] are added under the package name as a file path, and if any subdirectories are package directories, all of these are added recursively. The "basename" is intended for internal use only. The writepy() method makes archives with file names like this: string.pyc # Top level name test/__init__.pyc # Package directory test/testall.pyc # Package "test.testall" file test/bogus/__init__.pyc # Subpackage directory test/bogus/myfile.pyc # Subpackage "test.bogus.myfile" file close(self) Close the archive file. You must call close() before exiting your program or essential records will not be written. PATCH ===== See attached JimA --------------D45D15E3F3D3AD5E54347B06 Content-Type: text/plain; charset=us-ascii; name="zipfile.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="zipfile.py" "Read and write ZIP files" # Written by James C. Ahlstrom jim@interet.com # All rights transferred to CNRI pursuant to the Python contribution agreement import struct, os, time import binascii, py_compile try: import zlib # We may need its compression method except: pass class _BadZipfile(Exception): pass error = _BadZipfile # The exception raised by this module _debug = 1 # constants for Zip file compression methods ZIP_STORED = 0 ZIP_DEFLATED = 8 # other ZIP compression methods not supported def is_zipfile(filename): """Quickly see if file is a ZIP file by checking the magic number. Will not accept a ZIP archive with an ending comment.""" try: fpin = open(filename, "rb") fpin.seek(-22, 2) # Seek to end-of-file record endrec = fpin.read() fpin.close() if endrec[0:4] == "PK\005\006" and endrec[-2:] == "\000\000": return 1 # file has correct magic number except: pass def zip2date(d): "Return (year, month, day) for a date in zip format" return (d>>9)+1980, (d>>5)&0xF, d&0x1F def zip2time(t): "Return (hour, minute, second) for a time in zip format" return t>>11, (t>>5)&0x3F, t&0x1F * 2 def date2zip(year, month, day): "Return 16-bit zip date for year, month, day" return (year - 1980) << 9 | month << 5 | day def time2zip(hour, minute, second): "Return 16-bit zip time for hour, minute, second" return hour << 11 | minute << 5 | second / 2 class ZipFile: "Class with methods to open, read, write, close, list zip files" # Here are some struct module formats for reading headers structEndArchive = "<4s4H2lH" # 9 items, end of archive, 22 bytes stringEndArchive = "PK\005\006" # magic number for end of archive record structCentralDir = "<4s4B4H3i5H2i" # 19 items, central directory, 46 bytes stringCentralDir = "PK\001\002" # magic number for central directory structFileHeader = "<4s2B4H3i2H" # 12 items, file header record, 30 bytes stringFileHeader = "PK\003\004" # magic number for file header def __init__(self, filename, mode = "r", compression = 0): """Construct a ZipFile instance and open the ZIP file named "filename" with mode read "r", write "w" or append "a".""" if compression == ZIP_STORED: pass elif compression == ZIP_DEFLATED: try: import zlib except: raise RuntimeError, "Compression requires the (missing) zlib module" else: raise RuntimeError, "Compression method must be 0 or 8" self.TOC = {} # Table of contents for the archive self.compression = compression # Method of compression self.filename = filename self.mode = key = mode[0] if key == 'r': self.fp = open(filename, "rb") self._getTOC() elif key == 'w': self.fp = open(filename, "wb") elif key == 'a': fp = self.fp = open(filename, "r+b") fp.seek(-22, 2) # Seek to end-of-file record endrec = fp.read() if endrec[0:4] == self.stringEndArchive and endrec[-2:] == "\000\000": self._getTOC() # file is a zip file fp.seek(self.start_dir, 0) # seek to start of directory and overwrite else: # file is not a zip file, just append fp.seek(0, 2) else: raise RuntimeError, 'Mode must be "r", "w" or "a"' def _getTOC(self): "Read in the table of contents for the zip file" fp = self.fp fp.seek(-22, 2) # Start of end-of-archive record filesize = fp.tell() + 22 # Get file size endrec = fp.read(22) # Archive must not end with a comment! if endrec[0:4] != self.stringEndArchive or endrec[-2:] != "\000\000": raise BadZipfile, "File is not a zip file, or ends with a comment" endrec = struct.unpack(self.structEndArchive, endrec) if _debug > 2: print endrec size_cd = endrec[5] # bytes in central directory offset_cd = endrec[6] # offset of central directory x = filesize - 22 - size_cd concat = x - offset_cd # zero, unless zip was concatenated to another file if _debug > 2: print "given, inferred, offset", offset_cd, x, concat self.start_dir = offset_cd + concat # Position of start of central directory fp.seek(self.start_dir, 0) total = 0 flist = [] # List of file header offsets while total < size_cd: centdir = fp.read(46) total = total + 46 if centdir[0:4] != self.stringCentralDir: raise BadZipfile, "Bad magic number for central directory" centdir = struct.unpack(self.structCentralDir, centdir) if _debug > 2: print centdir fname = fp.read(centdir[12]) extra = fp.read(centdir[13]) comment = fp.read(centdir[14]) total = total + centdir[12] + centdir[13] + centdir[14] if _debug > 2: print "total", total flist.append(centdir[18]) # Offset of file header record if _debug > 2: print flist toc = self.TOC # Table of contents for offset in flist: fp.seek(offset + concat, 0) fheader = fp.read(30) if fheader[0:4] != self.stringFileHeader: raise BadZipfile, "Bad magic number for file header" fheader = struct.unpack(self.structFileHeader, fheader) fname = fp.read(fheader[10]) if _debug > 1: print "File", fname, fheader extra = fp.read(fheader[11]) toc[fname] = (fp.tell(), extra) + fheader[3:10] # toc key is the file name, value is: # 0:file offset, 1:extra data as string, 2:bit flags, 3:compression type, # 4:file time, 5:file date, 6:CRC-32, 7:compressed size, # 8:uncompressed size. def listdir(self): return self.TOC.keys() def printdir(self): "Print table of contents for zip file" toc = self.TOC if _debug > 2: print toc print "%-30s %19s %12s" % ("File Name", "Modified ", "Size") for name, data in toc.items(): bytes = self.read(name) # Just to check CRC-32 d = data[5] # Date t = data[4] # Time date = "%d-%02d-%02d %02d:%02d:%02d" % (zip2date(d) + zip2time(t)) print "%-30s %s %12d" % (name, date, data[8]) def read(self, name): "Return file bytes (as a string) for name" if self.mode not in ("r", "a"): raise RuntimeError, 'read() requires mode "r" or "a"' if not self.fp: raise RuntimeError, "Attempt to read ZIP archive that was already closed" data = self.TOC[name] filepos = self.fp.tell() self.fp.seek(data[0], 0) bytes = self.fp.read(data[7]) self.fp.seek(filepos, 0) if data[3] == ZIP_STORED: # Compression method: none pass elif data[3] == ZIP_DEFLATED: # Compression method: deflation # zlib compress/decompress code by Jeremy Hylton of CNRI dc = zlib.decompressobj(-15) bytes = dc.decompress(bytes) # need to feed in unused pad byte so that zlib won't choke ex = dc.decompress('Z') + dc.flush() if ex: bytes = bytes + ex else: raise BadZipfile, "Unsupported compression method %d for file %s" % (data[3], name) crc = binascii.crc32(bytes) if crc != data[6]: raise BadZipfile, "Bad CRC-32 for file %s" % name return bytes def write(self, filename, arcname, extra = ""): """Put the bytes from filename into the archive under the name arcname. The "extra" is the extra data string.""" mtime = os.stat(filename)[8] mtime = time.localtime(mtime) year, month, day, hour, minute, second = mtime[0:6] fp = open(filename, "rb") bytes = fp.read() fp.close() self.writestr(bytes, arcname, year, month, day, hour, minute, second, extra) def writestr(self, bytes, arcname, year, month, day, hour, minute, second, extra = ""): """Write bytes and other data into the archive under the name arcname. The "extra" is the extra data string.""" if self.TOC.has_key(arcname): # Warning for duplicate names if _debug: print "Duplicate name:", arcname if self.mode not in ("w", "a"): raise RuntimeError, 'write() requires mode "w" or "a"' if not self.fp: raise RuntimeError, "Attempt to write ZIP archive that was already closed" dosdate = date2zip(year, month, day) dostime = time2zip(hour, minute, second) compression = self.compression # Method of compression u_size = len(bytes) # Uncompressed size crc = binascii.crc32(bytes) # CRC-32 checksum if compression == ZIP_DEFLATED: co = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -15) bytes = co.compress(bytes) + co.flush() c_size = len(bytes) # Compressed size else: c_size = u_size header = struct.pack(self.structFileHeader, self.stringFileHeader, 10, 0, 0, compression, dostime, dosdate, crc, c_size, u_size, len(arcname), len(extra)) self.fp.write(header) self.fp.write(arcname) self.fp.write(extra) offset = self.fp.tell() # Start of file bytes self.fp.write(bytes) self.TOC[arcname] = (offset, extra, 0, compression, dostime, dosdate, crc, c_size, u_size) def writepy(self, pathname, basename = ""): """If pathname is a package directory, search the directory and all package subdirectories recursively for all *.py and enter the modules into the archive. If pathname is a plain directory, listdir *.py and enter all modules. Else, pathname must be a Python *.py file and the module will be put into the archive. Added modules are always module.pyo or module.pyc. This method will compile the module.py into module.pyc if necessary.""" dir, name = os.path.split(pathname) if os.path.isdir(pathname): initname = os.path.join(pathname, "__init__.py") if os.path.isfile(initname): # This is a package directory, add it if basename: basename = "%s/%s" % (basename, name) else: basename = name if _debug: print "Adding package in", pathname, "as", basename fname, arcname = self._get_codename(initname[0:-3], basename) if _debug: print "Adding", arcname self.write(fname, arcname) dirlist = os.listdir(pathname) dirlist.remove("__init__.py") # Add all *.py files and package subdirectories in the directory for filename in dirlist: path = os.path.join(pathname, filename) root, ext = os.path.splitext(filename) if os.path.isdir(path): if os.path.isfile(os.path.join(path, "__init__.py")): # This is a package directory, add it self.writepy(path, basename) # Recursive call elif ext == ".py": fname, arcname = self._get_codename(path[0:-3], basename) if _debug: print "Adding", arcname self.write(fname, arcname) else: # This is NOT a package directory, add its files at top level if _debug: print "Adding files from directory", pathname for filename in os.listdir(pathname): path = os.path.join(pathname, filename) root, ext = os.path.splitext(filename) if ext == ".py": fname, arcname = self._get_codename(path[0:-3], basename) if _debug: print "Adding", arcname self.write(fname, arcname) else: if pathname[-3:] != ".py": raise RuntimeError, 'Files added with writepy() must end with ".py"' fname, arcname = self._get_codename(pathname[0:-3], basename) if _debug: print "Adding file", arcname self.write(fname, arcname) def _get_codename(self, pathname, basename): """Given a module name path, return the correct file path and archive name, compiling if necessary. For example, for /python/lib/string, return (/python/lib/string.pyc, string)""" file_py = pathname + ".py" file_pyc = pathname + ".pyc" file_pyo = pathname + ".pyo" if os.path.isfile(file_pyo) and os.stat(file_pyo)[8] >= os.stat(file_py)[8]: fname = file_pyo # Use .pyo file elif not os.path.isfile(file_pyc) or os.stat(file_pyc)[8] < os.stat(file_py)[8]: if _debug: print "Compiling", file_py py_compile.compile(file_py, file_pyc) fname = file_pyc else: fname = file_pyc archivename = os.path.split(fname)[1] if basename: archivename = "%s/%s" % (basename, archivename) return (fname, archivename) def __del__(self): # User should have called close(), but just in case if self.fp: self.fp.close() self.fp = None def close(self): if self.mode in ("w", "a"): # write ending records attrib = 0666 << 16 # file attributes count = 0 pos1 = self.fp.tell() for name, data in self.TOC.items(): # write central directory count = count + 1 namesize = len(name) extrasize = len(data[1]) centdir = struct.pack(self.structCentralDir, self.stringCentralDir, 20, 3, 10, 0, data[2], data[3], data[4], data[5], data[6], data[7], data[8], namesize, extrasize, 0, 0, 0, attrib, data[0] - namesize - extrasize - 30) self.fp.write(centdir) self.fp.write(name) self.fp.write(data[1]) pos2 = self.fp.tell() endrec = struct.pack(self.structEndArchive, self.stringEndArchive, 0, 0, count, count, pos2 - pos1, pos1, 0) self.fp.write(endrec) self.fp.close() self.fp = None def test(): if 1: # Run from Python home directory z = ZipFile("temp.zip", "w") z.writepy("Lib") # Directory ./Lib must exist z.close() z = ZipFile("temp.zip", "r") z.printdir() z.close() if 0: z = ZipFile("jim1.zip", "w") z.write("jim.1", "jim.1") z.write("jim.2", "users/jim/jim.2") z.close() if 0: z = ZipFile("jim2.zip", "w") z.writepy("N:/prd/winlease/vest") z.write("jim.1", "jim.1") z.writepy("N:/python/Python-1.5.2/Lib/string.py") z.writepy("N:/python/Python-1.5.2/Lib/test") z.close() if 0: fp = open("python.exe", "rb") bytes = fp.read() fp.close() fp = open("python2.exe", "wb") fp.write(bytes) fp.close() z = ZipFile("python2.exe", "a") z.write("jim.2", "append/users/jim/jim.2") z.close() if 0: z = ZipFile("python2.exe", "r") z.printdir() z.close() if 0: z = ZipFile("jimcomp.zip", "w", ZIP_DEFLATED) z.write("jim.1", "jim.1") z.write("jim.2", "compress/jim.2") z.close() if 0: z = ZipFile("winzip.zip", "r") z.printdir() z.close() --------------D45D15E3F3D3AD5E54347B06-- From Gerrit Fri Feb 11 20:50:56 2000 From: Gerrit (Gerrit Holl) Date: Fri, 11 Feb 2000 21:50:56 +0100 Subject: [Patches] Complaints new patches list. Message-ID: <20000211215056.A7555@stopcontact.palga.uucp> Hello, I'm sorry to say it, but I have some complaints about the new patches list. More than a week ago, I sent a patch for some scripts in the Tools/ directory. Before the patches list existed, I sent a patch to nntplib.py to Guido, who forwarded it to a lietenant. The former added a 'reverse' option to logmerge.py and moved regex module usage to re module usage, the latter changes string based exceptions to class based exceptions. I did not receive any response. I know my email reached the list, because I can see the message in the archive. nntplib class exceptions *are* implemented now, but by Barry (was my code too bad?). That my code is refused is OK (but I personally don't like it). But I do want a personal response which tells me why my patch wasn't incorparated. I did send the legal notice; even if I didn't, I should have got a response telling me to do so. I don't like this. regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From fdrake@acm.org Fri Feb 11 21:49:19 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 11 Feb 2000 16:49:19 -0500 (EST) Subject: [Patches] Complaints new patches list. In-Reply-To: <20000211215056.A7555@stopcontact.palga.uucp> References: <20000211215056.A7555@stopcontact.palga.uucp> Message-ID: <14500.33759.874636.73931@weyr.cnri.reston.va.us> Gerrit Holl writes: > I'm sorry to say it, but I have some complaints about the new patches > list. ... > I don't like this. Gerrit, I think it's up to the individual "lietenants" to determine if they are sufficiently compentent on a topic to accept specific patches. I've ignored a number of them because I either don't know enough to determine whether the patch is the right thing to do, or because I don't feel comfortable channelling Guido. I *have* dealt with patches which I thought I could handle appropriately. Other patches, not to the patches list but to python-docs, have sat around because I've been too busy (working on Python is not part of my job). I suspect that's the case for several of us. I think what it comes down to is that the new arrangement *helps* Guido, but doesn't replace Guido. Since he's been out for most of this week, he hasn't been doing anything about patches. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Fredrik Lundh" Message-ID: <007f01bf74dd$0b9d0ee0$34aab5d4@hagrid> Gerrit Holl wrote: > I did send the legal notice; even if I didn't, I should have got a = response > telling me to do so. did you notice that the legal notice says: I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. > I don't like this. and I don't like your attitude. if someone sent me a patch for PIL, got it incorporated, and then behaved like this, I'd *plonk* him right away. lucky for you, I'm not the python patch pumpkin. From gstein@lyra.org Fri Feb 11 23:20:16 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 11 Feb 2000 15:20:16 -0800 (PST) Subject: [Patches] Complaints new patches list. In-Reply-To: <20000211215056.A7555@stopcontact.palga.uucp> Message-ID: On Fri, 11 Feb 2000, Gerrit Holl wrote: >... > More than a week ago, I sent a patch for some scripts in the Tools/ > directory. I saw it but ignored it. If it hasn't been applied by (approx) April when I truly start to review patches, then I'll take a close look at it and recommend it for inclusion or let you know what else needs to happen. Until then, it will sit there (unless somebody else looks at it). There is *NO* guaranteed response time. If you believe so, then go rethink things. > Before the patches list existed, I sent a patch to nntplib.py > to Guido, who forwarded it to a lietenant. Which lieutenant would this be? How do you know he did this? Before the patches list existed, Guido did not *have* "lieutenants." >... > I did not receive any response. We don't owe you one. Sure, it would be nice. Sure, it would be polite. But we have limited time. We are *volunteers*. If you want paid people responding to your email about language changes, then start sending patches to Microsoft Visual C++. I'm sure they'll respond in a timely fashion. > I know my email reached the list, because > I can see the message in the archive. nntplib class exceptions *are* > implemented now, but by Barry (was my code too bad?). That my code is > refused is OK (but I personally don't like it). There was another issue that came up with nntplib, which was fixed recently. As part of that fix, Barry also changed the exceptions. I believe it was done completely independent of your patch. I doubt anybody but Guido saw your patch. Ergo, nobody refused your patch... it just happened to be done a bit differently by a different person. > But I do want a personal > response which tells me why my patch wasn't incorparated. Most of us don't really even have time to process this stuff. I know that I don't. In a couple months? Maybe, but I'll make time then. I've got other stuff that I like to do, too. > I did send the > legal notice; even if I didn't, I should have got a response telling me > to do so. If you sent it to Guido, it probably went to a portion of his mailbox for pending patches and he didn't give it a second look. People have sent patches to this list without the notice and been asked to send one. So, we do respond to people at least that much. > I don't like this. Too bad. I don't think you have the right to complain. We're VOLUNTEERS. We do the best we can. We get to things when we happen to have time. Your complaint comes on the heels of calling people "liars" because they make a mistake on the www.list.org page. I don't think I like your attitude in anything you do. You are a very unpleasant person. Regards, -g -- Greg Stein, http://www.lyra.org/ From aahz@searchbutton.com Fri Feb 11 23:58:26 2000 From: aahz@searchbutton.com (Aahz) Date: Fri, 11 Feb 2000 15:58:26 -0800 Subject: [Patches] httplib.py timeout Message-ID: <6035C3D0EB76D31183B800902762824E040920@adsl-63-196-9-132.dsl.snfc21.pacbell.net> This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01BF74EB.E343222C Content-Type: text/plain; charset="iso-8859-1" This patch allows the user of httplib.py to specify a timeout in milliseconds. If a timeout is not specified, a default of sixty seconds is assumed. The user may specify no timeout by passing a timeout of zero. This patch is intended to be especially useful to urllib.py; there was discussion of this patch in comp.lang.python at the end of December, with general agreement that this patch is appropriate. I'm submitting this patch against CVS, but the patch has only been tested against 1.5.2. Let me know if you have any problems. Disclaimer (approved by my boss at Searchbutton.com): I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ------_=_NextPart_000_01BF74EB.E343222C Content-Type: application/octet-stream; name="httplib.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="httplib.diff" *** httplib.py.CVS Fri Feb 11 15:41:58 2000=0A= --- httplib.py Fri Feb 11 15:51:39 2000=0A= ***************=0A= *** 41,46 ****=0A= --- 41,47 ----=0A= HTTP_VERSION =3D 'HTTP/1.0'=0A= HTTP_PORT =3D 80=0A= HTTPS_PORT =3D 443=0A= + SOCKET_TIMEOUT =3D 60 * 1000 # Sixty seconds=0A= =0A= class FakeSocket:=0A= def __init__(self, sock, ssl):=0A= ***************=0A= *** 69,87 ****=0A= class HTTP:=0A= """This class manages a connection to an HTTP server."""=0A= =0A= ! def __init__(self, host =3D '', port =3D 0, **x509):=0A= """Initialize a new instance.=0A= =0A= If specified, `host' is the name of the remote host to = which=0A= to connect. If specified, `port' specifies the port to = which=0A= ! to connect. By default, httplib.HTTP_PORT is used.=0A= =0A= """=0A= self.key_file =3D x509.get('key_file')=0A= self.cert_file =3D x509.get('cert_file')=0A= self.debuglevel =3D 0=0A= self.file =3D None=0A= ! if host: self.connect(host, port)=0A= =0A= def set_debuglevel(self, debuglevel):=0A= """Set the debug output level.=0A= --- 70,91 ----=0A= class HTTP:=0A= """This class manages a connection to an HTTP server."""=0A= =0A= ! def __init__(self, host =3D '', port =3D 0, timeout =3D None, = **x509):=0A= """Initialize a new instance.=0A= =0A= If specified, `host' is the name of the remote host to = which=0A= to connect. If specified, `port' specifies the port to = which=0A= ! to connect. By default, httplib.HTTP_PORT is used. If = specified,=0A= ! timeout is the socket timeout in milliseconds. By = default,=0A= ! httplib.SOCKET_TIMEOUT is used. If timeout =3D=3D 0, then no = timeout=0A= ! is used.=0A= =0A= """=0A= self.key_file =3D x509.get('key_file')=0A= self.cert_file =3D x509.get('cert_file')=0A= self.debuglevel =3D 0=0A= self.file =3D None=0A= ! if host: self.connect(host, port, timeout)=0A= =0A= def set_debuglevel(self, debuglevel):=0A= """Set the debug output level.=0A= ***************=0A= *** 92,99 ****=0A= """=0A= self.debuglevel =3D debuglevel=0A= =0A= ! def connect(self, host, port =3D 0):=0A= ! """Connect to a host on a given port.=0A= =0A= Note: This method is automatically invoked by __init__,=0A= if a host is specified during instantiation.=0A= --- 96,104 ----=0A= """=0A= self.debuglevel =3D debuglevel=0A= =0A= ! def connect(self, host, port =3D 0, timeout =3D None):=0A= ! """Connect to a host on a given port with timeout in = milliseconds.=0A= ! No timeout if timeout =3D=3D 0.=0A= =0A= Note: This method is automatically invoked by __init__,=0A= if a host is specified during instantiation.=0A= ***************=0A= *** 108,113 ****=0A= --- 113,131 ----=0A= raise socket.error, "nonnumeric port"=0A= if not port: port =3D HTTP_PORT=0A= self.sock =3D socket.socket(socket.AF_INET, = socket.SOCK_STREAM)=0A= + # Use try/except to catch platforms where socket options = unavailable=0A= + if timeout is None:=0A= + try:=0A= + self.sock.setsockopt(socket.SOL_SOCKET, =0A= + socket.SO_RCVTIMEO, SOCKET_TIMEOUT)=0A= + except:=0A= + pass=0A= + elif timeout > 0:=0A= + try:=0A= + self.sock.setsockopt(socket.SOL_SOCKET, =0A= + socket.SO_RCVTIMEO, timeout)=0A= + except:=0A= + pass=0A= if self.debuglevel > 0: print 'connect:', (host, port)=0A= self.sock.connect(host, port)=0A= =0A= ***************=0A= *** 192,199 ****=0A= class HTTPS(HTTP):=0A= """This class allows communication via SSL."""=0A= =0A= ! def connect(self, host, port =3D 0):=0A= ! """Connect to a host on a given port.=0A= =0A= Note: This method is automatically invoked by = __init__,=0A= if a host is specified during instantiation.=0A= --- 210,218 ----=0A= class HTTPS(HTTP):=0A= """This class allows communication via SSL."""=0A= =0A= ! def connect(self, host, port =3D 0, timeout =3D None):=0A= ! """Connect to a host on a given port with timeout in = milliseconds.=0A= ! No timeout if timeout =3D=3D 0.=0A= =0A= Note: This method is automatically invoked by = __init__,=0A= if a host is specified during instantiation.=0A= ***************=0A= *** 208,213 ****=0A= --- 227,246 ----=0A= raise socket.error, "nonnumeric port"=0A= if not port: port =3D HTTPS_PORT=0A= sock =3D socket.socket(socket.AF_INET, = socket.SOCK_STREAM)=0A= + # Use try/except to catch platforms where socket options = =0A= + # unavailable=0A= + if timeout is None:=0A= + try:=0A= + self.sock.setsockopt(socket.SOL_SOCKET, =0A= + socket.SO_RCVTIMEO, SOCKET_TIMEOUT)=0A= + except:=0A= + pass=0A= + elif timeout > 0:=0A= + try:=0A= + self.sock.setsockopt(socket.SOL_SOCKET, =0A= + socket.SO_RCVTIMEO, timeout)=0A= + except:=0A= + pass=0A= if self.debuglevel > 0: print 'connect:', (host, port)=0A= sock.connect(host, port)=0A= ssl =3D socket.ssl(sock, self.key_file, = self.cert_file)=0A= ------_=_NextPart_000_01BF74EB.E343222C-- From Moshe Zadka Sat Feb 12 05:45:02 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 12 Feb 2000 07:45:02 +0200 (IST) Subject: [Patches] httplib.py timeout In-Reply-To: <6035C3D0EB76D31183B800902762824E040920@adsl-63-196-9-132.dsl.snfc21.pacbell.net> Message-ID: On Fri, 11 Feb 2000, Aahz wrote: > This patch allows the user of httplib.py to specify a timeout in > milliseconds. If a timeout is not specified, a default of sixty seconds is > assumed. The patch sounds great, except the default. The default should be no timeout -- don't force views about how software "should" behave down everybody's throats. Just my 0.02$ -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From dascher@mindspring.com Sat Feb 12 06:01:02 2000 From: dascher@mindspring.com (David Ascher) Date: Fri, 11 Feb 2000 22:01:02 -0800 Subject: [Patches] httplib.py timeout In-Reply-To: Message-ID: > On Fri, 11 Feb 2000, Aahz wrote: > > > This patch allows the user of httplib.py to specify a timeout in > > milliseconds. If a timeout is not specified, a default of > sixty seconds is > > assumed. > > The patch sounds great, except the default. The default should be no > timeout -- don't force views about how software "should" behave down > everybody's throats. Geez, folks, can we all chill out? Why is the patches list so young and yet so bitter already?! Moshe, having 'no timeout' is just as much an assumption about how software should behave as having a sixty second timeout. You assume that the software 'should' just hang for as long as it takes. Aahz believes that anything over 60 seconds is likely to be a bug. Both views are plausible. FWIW, most internet clients to my knowledge do use timeouts, and I personally am grateful for that... --david ascher From Fredrik Lundh" Message-ID: <000001bf753a$dcd89e40$34aab5d4@hagrid> > Moshe, having 'no timeout' is just as much an assumption about how = software > should behave as having a sixty second timeout. You assume that the > software 'should' just hang for as long as it takes. Aahz believes = that > anything over 60 seconds is likely to be a bug. Both views are = plausible. changing from no timeout to a given default timeout may break working programs (been there, done that, can be really confusing). how do standard socket implementations behave today? is 60 seconds a good approximation of how the most impatient implementations behave today? From tim_one@email.msn.com Sat Feb 12 20:19:08 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Feb 2000 15:19:08 -0500 Subject: [Patches] httplib.py timeout In-Reply-To: Message-ID: <000401bf7596$6aac8aa0$7c2d153f@tim> [David Ascher] > Geez, folks, can we all chill out? Why is the patches list so > young and yet so bitter already?! Good news! I wrote a Python program to analyze that. Turns out it's just a temporary startup glitch, and that you wrote exactly the right thing here to nudge it back to professional standards. > Moshe, having 'no timeout' is just as much an assumption about > how software should behave as having a sixty second timeout. Moshe meant to say that a quiet *change* in default behavior is going to cause real trouble, no matter which of the old or new way "is better". And Gerrit meant to say that the whiny tone of his unrealistic complaint was due to a flawed translation from the original Dutch . mucking-with-peoples'-code-is-even-more-contentious-than- mucking-with-their-opinions-ly y'rs - tim From tim_one@email.msn.com Sat Feb 12 20:19:21 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Feb 2000 15:19:21 -0500 Subject: [Patches] Add crc32() method to binascii In-Reply-To: <38A4224B.1A63F825@interet.com> Message-ID: <000b01bf7596$723b5b20$7c2d153f@tim> > This patch adds a new method to module binascii. It calculates > a CRC-32 checksum. ... Notes on this patch: + Format: The crc_32_tab initializer extends thru column 96; it shouldn't go beyond column 80. + Semantics: "UL" should be appended to all unsigned long literals, to prevent surprises on 64-bit machines (untagged literals are ints in C, and 0xFFF... is -1 on most machines, even the 64-bit ones). Note that this includes all the literals in the crc_32_tab initializer. + Test: "If it's not tested, it's broken." A test of this function needs to be added to the std test suite. > This method is generally useful, and is > required for ZIP file checksums. Use as follows: > > print binascii.crc32("hello world") > # Or, in two pieces: > crc = binascii.crc32("hello") > crc = binascii.crc32(" world", crc) > print crc This example should be in the docs, not in checkin mail commentary. I couldn't guess from the docs that this is supported usage (although I can reverse-engineer it from the code). pickily y'rs - tim From tim_one@email.msn.com Sun Feb 13 07:45:03 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 13 Feb 2000 02:45:03 -0500 Subject: [Patches] Garbage collection patches for Python In-Reply-To: <200002051400.PAA13946@python.inrialpes.fr> Message-ID: <000901bf75f6$3d12a560$962d153f@tim> [Vladimir Marangozov] > Some time ago, you (Toby Kelsey, 'xr' and Neil Schemenauer) have > submitted a patch for adding GC behavior to Python, that Guido was > kind enough to share with some Python developers. > ... > I doubt that we've reached concensus on the subject so far, despite > the experience that has been gained from past experiments. > > Therefore, I presume that incorporating any kind of GC implementation > in the standard distribution, for now, is out of question (at least, > not for the upcoming 1.6 release). I have since recommended to Guido (in pvt) that he give very serious consideration to folding Neil's rework of Toby's approach (actually, it's now closer to a scheme Guido once suggested) into 1.6. > However, as you may have noticed, there's one thing that needs to be > done for any "third party" mem manager (including GC): replace the > calls to malloc & friends in the core distribution by PyMem_MALLOC & > friends (macros, defined in mymalloc.h). I'm in favor of cleaning up malloc/free usage too, but that's actually independent of Neil's latest gc work: it doesn't hook malloc, or free, or even DECREF anymore. From Fredrik Lundh" Message-ID: <010701bf760e$70e46820$34aab5d4@hagrid> Tim wrote: > I have since recommended to Guido (in pvt) that he give very serious > consideration to folding Neil's rework of Toby's approach (actually, = it's > now closer to a scheme Guido once suggested) into 1.6. is this scheme perhaps documented somewhere? just-curious-ly y'rs /F From nascheme@enme.ucalgary.ca Sun Feb 13 11:09:08 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Sun, 13 Feb 2000 04:09:08 -0700 Subject: [Patches] Garbage collection patches for Python In-Reply-To: <010701bf760e$70e46820$34aab5d4@hagrid>; from effbot@telia.com on Sun, Feb 13, 2000 at 11:38:12AM +0100 References: <000901bf75f6$3d12a560$962d153f@tim> <010701bf760e$70e46820$34aab5d4@hagrid> Message-ID: <20000213040908.A293@acs.ucalgary.ca> On Sun, Feb 13, 2000 at 11:38:12AM +0100, Fredrik Lundh wrote: > is this scheme perhaps documented somewhere? Here is my (hopefully accurate) explanation: First, we keep track of objects that can be involved in reference cycles. To collect reference cycles, we find all the objects reachable from these tracked objects. For each object in this "reachable set", we determine if the object is referred to by an object not in the set (the reference counts tell us this). If an object is referred to from outside the set, it and all objects reachable from it are removed from the set. Once we finish doing this for each object, we know that the objects remaining in the set are unreachable and can be collected. This scheme will find all reference cycles that contain a least one of the tracked objects and are composed of objects that the implementation knows how to "follow". The current implementation only tracks dictionaries and knows how to follow lists, tuples, dictionaries, classes and instances. There is a bit more explaination in the comments in the implementation. You can find it here: http://www.acs.ucalgary.ca/~nascheme/python/gc-cycle.diff I have recently updated patch to include doc strings so you might want to reload it if you already have it. Neil From Fredrik Lundh" <010701bf760e$70e46820$34aab5d4@hagrid> <20000213040908.A293@acs.ucalgary.ca> Message-ID: <005201bf761e$414b17c0$34aab5d4@hagrid> Neil wrote: > This scheme will find all reference cycles that contain a least > one of the tracked objects and are composed of objects that the > implementation knows how to "follow". The current implementation > only tracks dictionaries and knows how to follow lists, tuples, > dictionaries, classes and instances. awesome! just two questions: -- what's involved in making this work with threads? -- looks like it would be fairly trivial to move the recurse stuff into the type implementations (so that user-defined types can support gc too). how about a tp_gc_recurse slot? what else would be needed? From tim_one@email.msn.com Sun Feb 13 21:23:08 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 13 Feb 2000 16:23:08 -0500 Subject: [Patches] Garbage collection patches for Python In-Reply-To: <005201bf761e$414b17c0$34aab5d4@hagrid> Message-ID: <000f01bf7668$863415e0$572d153f@tim> [/F] > -- what's involved in making this work with > threads? Unclear at present, but the global lock is one nuclear strength hammer . > -- looks like it would be fairly trivial to move the > recurse stuff into the type implementations (so > that user-defined types can support gc too). how > about a tp_gc_recurse slot? what else would be > needed? This should certainly be done. Guido already pronounced on that too (in pvt email among him, Toby & me, which got fwd'ed to Neil later but perhaps got overlooked in the shuffle). Leave "gc" out of the name. Should be a general-purpose traverser that accepts a procedure to be applied "at the leaves"; etc -- more like os.path.walk. There will be other good uses for recursive traversal, and user types must be given a clean way to participate. From nascheme@enme.ucalgary.ca Sun Feb 13 21:35:19 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Sun, 13 Feb 2000 14:35:19 -0700 Subject: [Patches] Garbage collection patches for Python In-Reply-To: <005201bf761e$414b17c0$34aab5d4@hagrid>; from effbot@telia.com on Sun, Feb 13, 2000 at 01:31:28PM +0100 References: <000901bf75f6$3d12a560$962d153f@tim> <010701bf760e$70e46820$34aab5d4@hagrid> <20000213040908.A293@acs.ucalgary.ca> <005201bf761e$414b17c0$34aab5d4@hagrid> Message-ID: <20000213143519.A9145@acs.ucalgary.ca> On Sun, Feb 13, 2000 at 01:31:28PM +0100, Fredrik Lundh wrote: > -- what's involved in making this work with > threads? I'm pretty ignorant of threads but I guess you would need a lock for changing the lists of dictionaries and maybe a lock for when you are doing collection. > -- looks like it would be fairly trivial to move the > recurse stuff into the type implementations (so > that user-defined types can support gc too). how > about a tp_gc_recurse slot? what else would be > needed? That should be it that is needed. I didn't do it yet because I wanted the patch to have minimal impact on the rest of the Python code. Neil -- "Applying computer technology is simply finding the right wrench to pound in the correct screw." -- Tim Wright, Sequent Computer Systems. From guido@python.org Mon Feb 14 00:23:02 2000 From: guido@python.org (Guido van Rossum) Date: Sun, 13 Feb 2000 19:23:02 -0500 Subject: [Patches] Ronald Hiller: Re: [Python-bugs-list] urljoin() bug with odd no of '..' (PR#194) Message-ID: <200002140023.TAA20705@eric.cnri.reston.va.us> ------- Forwarded Message Date: Sun, 13 Feb 2000 18:57:05 -0500 From: Ronald Hiller To: DrMalte@ddd.de, Guido van Rossum Subject: Re: [Python-bugs-list] urljoin() bug with odd no of '..' (PR#194) I noticed your work with the urljoin function. I ran into the same thing. My patch is slightly different. - -=-=-=-=-=-=-= I've been having some problems with the urljoin function. When I try and join URLs that have '..' components that make the path above the root, they aren't joined properly. For example: goof> python Python 1.5.2 (#1, Oct 24 1999, 20:24:11) [GCC 2.8.1] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import urlparse >>> urlparse.urljoin("http://www.xyz.com", "../x/y/z.gif") 'http://www.xyz.com/../x/y/z.gif' >>> # Now with the changes: goof> python Python 1.5.2 (#1, Oct 24 1999, 20:24:11) [GCC 2.8.1] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import urlparse >>> urlparse.urljoin("http://www.xyz.com", "../x/y/z.gif") 'http://www.xyz.com/x/y/z.gif' >>> My patch for urlparse is included below...do they look reasonable? Thanks, Ron I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. *** Lib/urlparse.py Thu Mar 18 10:10:44 1999 - --- urlparse.py Sun Feb 13 16:51:36 2000 *************** *** 166,171 **** - --- 166,175 ---- i = i+1 else: break + while segments[0] == '': + del segments[0] + while segments[0] == '..': + del segments[0] if len(segments) == 2 and segments[1] == '..' and segments[0] == '': segments[-1] = '' elif len(segments) >= 2 and segments[-1] == '..': ------- End of Forwarded Message From Gerrit Mon Feb 14 16:40:04 2000 From: Gerrit (Gerrit Holl) Date: Mon, 14 Feb 2000 17:40:04 +0100 Subject: [Patches] UserDict/UserList: raise TypeError when trying to set attrs Message-ID: <20000214174004.A2068@stopcontact.palga.uucp> --BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=us-ascii Hello Python gurus, I added a __setattr__ to UserDict and UserList. When trying to set attributes to dictionairies and lists, an error is raised. In UserDict and UserList, that's not true: >>> l=[] >>> l.a=1 Traceback (innermost last): File "", line 1, in ? TypeError: object has read-only attributes >>> from UserList import UserList >>> l=UserList() >>> l.a=1 This patch fixes it: >>> from UserList import UserList >>> l=UserList() >>> l.a=1 Traceback (innermost last): File "", line 1, in ? File "UserList.py", line 57, in __setattr__ raise TypeError("object has read-only attributes") TypeError: object has read-only attributes The 'data' attribute can be set though; it's not possible to prohibit this. I hope you've found some time to read this mail and I hope you'll find some more time either to include my patch or notify me why not doing so. (was this sentence correct English?) I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth --BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="UserList.py.diff" --- UserList.py Mon Feb 14 17:31:21 2000 +++ /tmp/UserList.py Mon Feb 14 17:28:21 2000 @@ -51,10 +51,6 @@ def __mul__(self, n): return self.__class__(self.data*n) __rmul__ = __mul__ - def __setattr__(self, key, val): - import types - if key != 'data' or type(val) is not types.ListType: - raise TypeError("object has read-only attributes") def append(self, item): self.data.append(item) def insert(self, i, item): self.data.insert(i, item) def pop(self, i=-1): return self.data.pop(i) --BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="UserDict.py.diff" --- UserDict.py Mon Feb 14 17:30:46 2000 +++ /tmp/UserDict.py.diff Mon Feb 14 17:31:41 2000 @@ -1,40 +0,0 @@ -"""A more or less complete user-defined wrapper around dictionary objects.""" - -class UserDict: - def __init__(self, dict=None): - self.data = {} - if dict is not None: self.update(dict) - def __repr__(self): return repr(self.data) - def __cmp__(self, dict): - if isinstance(dict, UserDict): - return cmp(self.data, dict.data) - else: - return cmp(self.data, dict) - def __len__(self): return len(self.data) - def __getitem__(self, key): return self.data[key] - def __setitem__(self, key, item): self.data[key] = item - def __delitem__(self, key): del self.data[key] - def __setattr__(self, key, val): - import types - if key != 'data' or type(val) is not types.DictType: - raise TypeError("object has read-only attributes") - def clear(self): self.data.clear() - def copy(self): - if self.__class__ is UserDict: - return UserDict(self.data) - import copy - return copy.copy(self) - def keys(self): return self.data.keys() - def items(self): return self.data.items() - def values(self): return self.data.values() - def has_key(self, key): return self.data.has_key(key) - def update(self, dict): - if isinstance(dict, UserDict): - self.data.update(dict.data) - elif isinstance(dict, type(self.data)): - self.data.update(dict) - else: - for k, v in dict.items(): - self.data[k] = v - def get(self, key, failobj=None): - return self.data.get(key, failobj) --BXVAT5kNtrzKuDFl-- From Fredrik Lundh" Message-ID: <00ef01bf770e$0cfb3360$34aab5d4@hagrid> > I added a __setattr__ to UserDict and UserList. When trying to set > attributes to dictionairies and lists, an error is raised. import UserList class MyList(UserList.UserList): def __init__(self): UserList.UserList.__init__(self) self.mydata =3D [] list =3D MyList() Traceback (innermost last): File "bzzt.py", line 8, in ? File "bzzt.py", line 6, in __init__ File "UserList.py", line 14, in __setattr__ raise TypeError("object has read-only attributes") TypeError: object has read-only attributes From aahz@searchbutton.com Mon Feb 14 17:38:12 2000 From: aahz@searchbutton.com (Aahz) Date: Mon, 14 Feb 2000 09:38:12 -0800 Subject: [Patches] httplib.py timeout Message-ID: <6035C3D0EB76D31183B800902762824E040926@adsl-63-196-9-132.dsl.snfc21.pacbell.net> > how do standard socket implementations behave > today? is 60 seconds a good approximation of how > the most impatient implementations behave today? IME, yes. Fredrik, you probably recall from our discussion last December that the main driving force behind this is the fact that urllib has no way to control HTTP() directly and that urllib probably should not be modified to control HTTP.socket() (eww!). I think that nobody uses urllib with the *intent* of having no timeout, and it's bitten a lot of people. I also think very few people use HTTP() with the *intent* of having no timeout, and those are the kind of people who probably already have their exceptions constructed correctly to deal with it. From Moshe Zadka Mon Feb 14 17:37:34 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 14 Feb 2000 19:37:34 +0200 (IST) Subject: [Patches] UserDict/UserList: raise TypeError when trying to set attrs In-Reply-To: <20000214174004.A2068@stopcontact.palga.uucp> Message-ID: On Mon, 14 Feb 2000, Gerrit Holl wrote: > Hello Python gurus, > > I added a __setattr__ to UserDict and UserList. When trying to set > attributes to dictionairies and lists, an error is raised. In > UserDict and UserList, that's not true: I'm certain I'm against this patch: it would make deriving from UserDict (e.g.) that much harder, by needing to override __setattr__ for any new attributes you'd want to add. My 0.02$ -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From Fredrik Lundh" Message-ID: <021001bf771b$ab1ba040$34aab5d4@hagrid> Aahz wrote: > IME, yes. +1, in other words. From fdrake@acm.org Mon Feb 14 19:52:07 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 14 Feb 2000 14:52:07 -0500 (EST) Subject: [Patches] Documentation patches Message-ID: <14504.23783.932818.209084@weyr.cnri.reston.va.us> I've just updated the page at python.org/patches/ to ask that documentation patches be sent to python-docs@python.org, with specific bug reports to be submitted through the bugs database (which I monitor). Hopefully this will clarify what should happen for documentation issues. It also means that the published documentation on the Web site isn't out of date! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Gerrit Mon Feb 14 19:40:05 2000 From: Gerrit (Gerrit Holl) Date: Mon, 14 Feb 2000 20:40:05 +0100 Subject: [Patches] UserDict/UserList: raise TypeError when trying to set attrs In-Reply-To: ; from moshez@math.huji.ac.il on Mon, Feb 14, 2000 at 07:37:34PM +0200 References: <20000214174004.A2068@stopcontact.palga.uucp> Message-ID: <20000214204005.B3438@stopcontact.palga.uucp> Hello, Moshe Zadka wrote on 950553454: > On Mon, 14 Feb 2000, Gerrit Holl wrote: > > > Hello Python gurus, > > > > I added a __setattr__ to UserDict and UserList. When trying to set > > attributes to dictionairies and lists, an error is raised. In > > UserDict and UserList, that's not true: > > I'm certain I'm against this patch: it would make deriving from UserDict > (e.g.) that much harder, by needing to override __setattr__ for any > new attributes you'd want to add. > > My 0.02$ I don't know why one would do that but... OK, thanks for answering! Hmm, if subclassing standard objects will eventually be implemented, I wonder how this would be done... regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From Fredrik Lundh" <20000214204005.B3438@stopcontact.palga.uucp> Message-ID: <029001bf7727$b25afac0$34aab5d4@hagrid> Gerrit Holl wrote: > I don't know why one would do that but... afaik, the whole idea behind UserDict and UserList is to support subclassing: http://www.python.org/doc/current/lib/module-UserDict.html http://www.python.org/doc/current/lib/module-UserList.html From jim@interet.com Mon Feb 14 20:15:59 2000 From: jim@interet.com (James C. Ahlstrom) Date: Mon, 14 Feb 2000 15:15:59 -0500 Subject: [Patches] Re: Add crc32() method to binascii Message-ID: <38A8627F.E93B9517@interet.com> This is a multi-part message in MIME format. --------------C69D42EDB13F3066394F4AA7 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit This is a corrected patch. Thanks to Tim Peters for helpful comments. This patch adds a new method to module binascii. It calculates a CRC-32 checksum. This method is generally useful, and is required for ZIP file checksums. LEGAL ===== I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. DOCS (add to binascii docs) =========================== crc32(data, crc = 0) Compute CRC-32, the 32-bit checksum of data, starting with an initial crc. This is consistent with the ZIP file checksum. Use as follows: print binascii.crc32("hello world") # Or, in two pieces: crc = binascii.crc32("hello") crc = binascii.crc32(" world", crc) print crc TEST (add to ./test/test_binascii.py) ===================================== *** test_binascii.py.orig Mon Feb 14 09:41:24 2000 --- test_binascii.py Mon Feb 14 09:58:36 2000 *************** *** 5,14 **** --- 5,19 ---- Roger E. Masse """ import binhex + import binascii import tempfile from test_support import verbose def test(): + crc = binascii.crc32("Test the CRC-32 of") + crc = binascii.crc32(" this string.", crc) + if crc != 1571220330: + print "binascii.crc32() failed." try: fname1 = tempfile.mktemp() PATCH against 1.5.2 release (but just an addition) ================================================== See attached. JimA --------------C69D42EDB13F3066394F4AA7 Content-Type: text/plain; charset=us-ascii; name="binascii.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="binascii.diff" *** binascii.c.orig Mon Feb 7 15:50:26 2000 --- binascii.c Mon Feb 14 09:37:56 2000 *************** *** 719,724 **** --- 719,864 ---- return Py_BuildValue("i", crc); } + static char doc_crc32[] = + "(data, oldcrc = 0) -> newcrc. Compute CRC-32 incrementally"; + + /* Crc - 32 BIT ANSI X3.66 CRC checksum files + Also known as: ISO 3307 + **********************************************************************| + * *| + * Demonstration program to compute the 32-bit CRC used as the frame *| + * check sequence in ADCCP (ANSI X3.66, also known as FIPS PUB 71 *| + * and FED-STD-1003, the U.S. versions of CCITT's X.25 link-level *| + * protocol). The 32-bit FCS was added via the Federal Register, *| + * 1 June 1982, p.23798. I presume but don't know for certain that *| + * this polynomial is or will be included in CCITT V.41, which *| + * defines the 16-bit CRC (often called CRC-CCITT) polynomial. FIPS *| + * PUB 78 says that the 32-bit FCS reduces otherwise undetected *| + * errors by a factor of 10^-5 over 16-bit FCS. *| + * *| + **********************************************************************| + + Copyright (C) 1986 Gary S. Brown. You may use this program, or + code or tables extracted from it, as desired without restriction. + + First, the polynomial itself and its table of feedback terms. The + polynomial is + X^32+X^26+X^23+X^22+X^16+X^12+X^11+X^10+X^8+X^7+X^5+X^4+X^2+X^1+X^0 + Note that we take it "backwards" and put the highest-order term in + the lowest-order bit. The X^32 term is "implied"; the LSB is the + X^31 term, etc. The X^0 term (usually shown as "+1") results in + the MSB being 1. + + Note that the usual hardware shift register implementation, which + is what we're using (we're merely optimizing it by doing eight-bit + chunks at a time) shifts bits into the lowest-order term. In our + implementation, that means shifting towards the right. Why do we + do it this way? Because the calculated CRC must be transmitted in + order from highest-order term to lowest-order term. UARTs transmit + characters in order from LSB to MSB. By storing the CRC this way, + we hand it to the UART in the order low-byte to high-byte; the UART + sends each low-bit to hight-bit; and the result is transmission bit + by bit from highest- to lowest-order term without requiring any bit + shuffling on our part. Reception works similarly. + + The feedback terms table consists of 256, 32-bit entries. Notes: + + 1. The table can be generated at runtime if desired; code to do so + is shown later. It might not be obvious, but the feedback + terms simply represent the results of eight shift/xor opera- + tions for all combinations of data and CRC register values. + + 2. The CRC accumulation logic is the same for all CRC polynomials, + be they sixteen or thirty-two bits wide. You simply choose the + appropriate table. Alternatively, because the table can be + generated at runtime, you can start by generating the table for + the polynomial in question and use exactly the same "updcrc", + if your application needn't simultaneously handle two CRC + polynomials. (Note, however, that XMODEM is strange.) + + 3. For 16-bit CRCs, the table entries need be only 16 bits wide; + of course, 32-bit entries work OK if the high 16 bits are zero. + + 4. The values must be right-shifted by eight bits by the "updcrc" + logic; the shift must be unsigned (bring in zeroes). On some + hardware you could probably optimize the shift in assembler by + using byte-swap instructions. + ********************************************************************/ + + static unsigned long crc_32_tab[256] = { + 0x00000000UL, 0x77073096UL, 0xee0e612cUL, 0x990951baUL, 0x076dc419UL, + 0x706af48fUL, 0xe963a535UL, 0x9e6495a3UL, 0x0edb8832UL, 0x79dcb8a4UL, + 0xe0d5e91eUL, 0x97d2d988UL, 0x09b64c2bUL, 0x7eb17cbdUL, 0xe7b82d07UL, + 0x90bf1d91UL, 0x1db71064UL, 0x6ab020f2UL, 0xf3b97148UL, 0x84be41deUL, + 0x1adad47dUL, 0x6ddde4ebUL, 0xf4d4b551UL, 0x83d385c7UL, 0x136c9856UL, + 0x646ba8c0UL, 0xfd62f97aUL, 0x8a65c9ecUL, 0x14015c4fUL, 0x63066cd9UL, + 0xfa0f3d63UL, 0x8d080df5UL, 0x3b6e20c8UL, 0x4c69105eUL, 0xd56041e4UL, + 0xa2677172UL, 0x3c03e4d1UL, 0x4b04d447UL, 0xd20d85fdUL, 0xa50ab56bUL, + 0x35b5a8faUL, 0x42b2986cUL, 0xdbbbc9d6UL, 0xacbcf940UL, 0x32d86ce3UL, + 0x45df5c75UL, 0xdcd60dcfUL, 0xabd13d59UL, 0x26d930acUL, 0x51de003aUL, + 0xc8d75180UL, 0xbfd06116UL, 0x21b4f4b5UL, 0x56b3c423UL, 0xcfba9599UL, + 0xb8bda50fUL, 0x2802b89eUL, 0x5f058808UL, 0xc60cd9b2UL, 0xb10be924UL, + 0x2f6f7c87UL, 0x58684c11UL, 0xc1611dabUL, 0xb6662d3dUL, 0x76dc4190UL, + 0x01db7106UL, 0x98d220bcUL, 0xefd5102aUL, 0x71b18589UL, 0x06b6b51fUL, + 0x9fbfe4a5UL, 0xe8b8d433UL, 0x7807c9a2UL, 0x0f00f934UL, 0x9609a88eUL, + 0xe10e9818UL, 0x7f6a0dbbUL, 0x086d3d2dUL, 0x91646c97UL, 0xe6635c01UL, + 0x6b6b51f4UL, 0x1c6c6162UL, 0x856530d8UL, 0xf262004eUL, 0x6c0695edUL, + 0x1b01a57bUL, 0x8208f4c1UL, 0xf50fc457UL, 0x65b0d9c6UL, 0x12b7e950UL, + 0x8bbeb8eaUL, 0xfcb9887cUL, 0x62dd1ddfUL, 0x15da2d49UL, 0x8cd37cf3UL, + 0xfbd44c65UL, 0x4db26158UL, 0x3ab551ceUL, 0xa3bc0074UL, 0xd4bb30e2UL, + 0x4adfa541UL, 0x3dd895d7UL, 0xa4d1c46dUL, 0xd3d6f4fbUL, 0x4369e96aUL, + 0x346ed9fcUL, 0xad678846UL, 0xda60b8d0UL, 0x44042d73UL, 0x33031de5UL, + 0xaa0a4c5fUL, 0xdd0d7cc9UL, 0x5005713cUL, 0x270241aaUL, 0xbe0b1010UL, + 0xc90c2086UL, 0x5768b525UL, 0x206f85b3UL, 0xb966d409UL, 0xce61e49fUL, + 0x5edef90eUL, 0x29d9c998UL, 0xb0d09822UL, 0xc7d7a8b4UL, 0x59b33d17UL, + 0x2eb40d81UL, 0xb7bd5c3bUL, 0xc0ba6cadUL, 0xedb88320UL, 0x9abfb3b6UL, + 0x03b6e20cUL, 0x74b1d29aUL, 0xead54739UL, 0x9dd277afUL, 0x04db2615UL, + 0x73dc1683UL, 0xe3630b12UL, 0x94643b84UL, 0x0d6d6a3eUL, 0x7a6a5aa8UL, + 0xe40ecf0bUL, 0x9309ff9dUL, 0x0a00ae27UL, 0x7d079eb1UL, 0xf00f9344UL, + 0x8708a3d2UL, 0x1e01f268UL, 0x6906c2feUL, 0xf762575dUL, 0x806567cbUL, + 0x196c3671UL, 0x6e6b06e7UL, 0xfed41b76UL, 0x89d32be0UL, 0x10da7a5aUL, + 0x67dd4accUL, 0xf9b9df6fUL, 0x8ebeeff9UL, 0x17b7be43UL, 0x60b08ed5UL, + 0xd6d6a3e8UL, 0xa1d1937eUL, 0x38d8c2c4UL, 0x4fdff252UL, 0xd1bb67f1UL, + 0xa6bc5767UL, 0x3fb506ddUL, 0x48b2364bUL, 0xd80d2bdaUL, 0xaf0a1b4cUL, + 0x36034af6UL, 0x41047a60UL, 0xdf60efc3UL, 0xa867df55UL, 0x316e8eefUL, + 0x4669be79UL, 0xcb61b38cUL, 0xbc66831aUL, 0x256fd2a0UL, 0x5268e236UL, + 0xcc0c7795UL, 0xbb0b4703UL, 0x220216b9UL, 0x5505262fUL, 0xc5ba3bbeUL, + 0xb2bd0b28UL, 0x2bb45a92UL, 0x5cb36a04UL, 0xc2d7ffa7UL, 0xb5d0cf31UL, + 0x2cd99e8bUL, 0x5bdeae1dUL, 0x9b64c2b0UL, 0xec63f226UL, 0x756aa39cUL, + 0x026d930aUL, 0x9c0906a9UL, 0xeb0e363fUL, 0x72076785UL, 0x05005713UL, + 0x95bf4a82UL, 0xe2b87a14UL, 0x7bb12baeUL, 0x0cb61b38UL, 0x92d28e9bUL, + 0xe5d5be0dUL, 0x7cdcefb7UL, 0x0bdbdf21UL, 0x86d3d2d4UL, 0xf1d4e242UL, + 0x68ddb3f8UL, 0x1fda836eUL, 0x81be16cdUL, 0xf6b9265bUL, 0x6fb077e1UL, + 0x18b74777UL, 0x88085ae6UL, 0xff0f6a70UL, 0x66063bcaUL, 0x11010b5cUL, + 0x8f659effUL, 0xf862ae69UL, 0x616bffd3UL, 0x166ccf45UL, 0xa00ae278UL, + 0xd70dd2eeUL, 0x4e048354UL, 0x3903b3c2UL, 0xa7672661UL, 0xd06016f7UL, + 0x4969474dUL, 0x3e6e77dbUL, 0xaed16a4aUL, 0xd9d65adcUL, 0x40df0b66UL, + 0x37d83bf0UL, 0xa9bcae53UL, 0xdebb9ec5UL, 0x47b2cf7fUL, 0x30b5ffe9UL, + 0xbdbdf21cUL, 0xcabac28aUL, 0x53b39330UL, 0x24b4a3a6UL, 0xbad03605UL, + 0xcdd70693UL, 0x54de5729UL, 0x23d967bfUL, 0xb3667a2eUL, 0xc4614ab8UL, + 0x5d681b02UL, 0x2a6f2b94UL, 0xb40bbe37UL, 0xc30c8ea1UL, 0x5a05df1bUL, + 0x2d02ef8dUL + }; + + static PyObject * + binascii_crc32(self, args) + PyObject *self; + PyObject *args; + { /* By Jim Ahlstrom; All rights transferred to CNRI */ + unsigned char *bin_data; + unsigned long crc = 0UL; /* initial value of CRC */ + int len; + + if ( !PyArg_ParseTuple(args, "s#|l", &bin_data, &len, &crc) ) + return NULL; + + crc = crc ^ 0xFFFFFFFFUL; + while(len--) + crc = crc_32_tab[(crc ^ *bin_data++) & 0xffUL] ^ (crc >> 8); + /* Note: (crc >> 8) MUST zero fill on left */ + return Py_BuildValue("l", crc ^ 0xFFFFFFFFUL); + } + /* List of functions defined in the module */ static struct PyMethodDef binascii_module_methods[] = { *************** *** 735,740 **** --- 875,881 ---- {"rledecode_hqx", binascii_rledecode_hqx, 1, doc_rledecode_hqx}, {"crc_hqx", binascii_crc_hqx, 1, doc_crc_hqx}, + {"crc32", binascii_crc32, 1, doc_crc32}, {NULL, NULL} /* sentinel */ }; --------------C69D42EDB13F3066394F4AA7-- From gward@cnri.reston.va.us Mon Feb 14 20:16:18 2000 From: gward@cnri.reston.va.us (Greg Ward) Date: Mon, 14 Feb 2000 15:16:18 -0500 Subject: [Patches] UserDict/UserList: raise TypeError when trying to set attrs In-Reply-To: <20000214204005.B3438@stopcontact.palga.uucp>; from gerrit.holl@pobox.com on Mon, Feb 14, 2000 at 08:40:05PM +0100 References: <20000214174004.A2068@stopcontact.palga.uucp> <20000214204005.B3438@stopcontact.palga.uucp> Message-ID: <20000214151617.D8785@cnri.reston.va.us> On 14 February 2000, Gerrit Holl said: > Moshe Zadka wrote on 950553454: > > I'm certain I'm against this patch: it would make deriving from UserDict > > (e.g.) that much harder, by needing to override __setattr__ for any > > new attributes you'd want to add. > > I don't know why one would do that but... OK, thanks for answering! Because the whole *point* of UserList and UserDict is to subclass them. If all you need is the functionality provided by UserList or UserDict, then you should just be using a raw list or dictionary. If nothing else, it's faster. > Hmm, if subclassing standard objects will eventually be implemented, I > wonder how this would be done... I believe the reason that UserList and UserDict exist at all is because you *can't* derive from standard Python types like "list" and "dictionary". This is the infamous "type/class dichotomy", and I think the best place to talk about that is on the types-sig. (I.e. it's not going to be fixed with a 20-line patch to patches@python.org.) Greg From guido@python.org Mon Feb 14 21:36:29 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 14 Feb 2000 16:36:29 -0500 Subject: [Patches] Multiply sequences by longs In-Reply-To: Your message of "Fri, 04 Feb 2000 17:06:06 EST." <200002042206.RAA11080@amarok.cnri.reston.va.us> References: <200002042206.RAA11080@amarok.cnri.reston.va.us> Message-ID: <200002142136.QAA22319@eric.cnri.reston.va.us> > This patch makes 5L * 'b' legal; with the large-file patches, some > people have run into bugs in modules such as dumbdbm that don't expect > f.tell() to return a long int.. Andrew, go ahead and check this in yourself. > One inconsistency: 1231231232312341414L * 'a' gets an > OverflowError (long int too long to convert), while 312341414L * 'a' > raises a MemoryError. I think this wouldn't be a problem in practice. No problem. > Documentation: a patch to the Reference and LibRef are required; there > may be other bits of the docs that need changing. If this gets > accepted, I'll make patches for the docs. Please go ahead -- you can check them in directly (unless Fred disagrees :-). > Question: should other bits of the core be changed to accept long > ints? Slicing, for instance, as in mylist[ 0L:5L ]? (python-dev > topic, maybe?) Yes, that's been a longstanding desire. I think it's okay to do this one case at a time. Note that even before slicing comes indexing! There are other, similar wishes: accept all sequences rather than just tuples or just lists, in many places. Also, we could be more generous in accepting mappings rather than just requiring dictionaries. However, there are some problems with this. There's an old patch that I never applied for various reasons; look for Ian Castleden in http://www.python.org/~guido/patch/. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Feb 14 21:44:17 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 14 Feb 2000 16:44:17 -0500 Subject: [Patches] Patch for Tools/scripts/crlf.py, lfcr.py, logmerge.py In-Reply-To: Your message of "Fri, 04 Feb 2000 22:55:45 +0100." <20000204225544.A18820@stopcontact.palga.uucp> References: <20000204225544.A18820@stopcontact.palga.uucp> Message-ID: <200002142144.QAA22417@eric.cnri.reston.va.us> Gerrit, Thanks for the patches. I've personally applied them. Please next time be a little more patient and kind with our overworked volunteers. They deserve kindness, not attitude. (And then, so do you. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Mon Feb 14 22:16:00 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Mon, 14 Feb 2000 17:16:00 -0500 (EST) Subject: [Patches] Multiply sequences by longs In-Reply-To: <200002142136.QAA22319@eric.cnri.reston.va.us> References: <200002042206.RAA11080@amarok.cnri.reston.va.us> <200002142136.QAA22319@eric.cnri.reston.va.us> Message-ID: <14504.32416.694611.757508@amarok.cnri.reston.va.us> Guido van Rossum writes: >> Question: should other bits of the core be changed to accept long >> ints? Slicing, for instance, as in mylist[ 0L:5L ]? (python-dev >> topic, maybe?) > >Yes, that's been a longstanding desire. I think it's okay to do this >one case at a time. Note that even before slicing comes indexing! OK; if I have a mandate to fixing slicing and indexing, too, then I'll hold off on checking in any documentation changes until that's done. (No point in writing text that says "* can also take a long integer" when it'll have to be changed to say "* and slicing..." in the next pass.) I'll check in the sequence multiplying patch now, and will send patches@python.org the remaining changes to abstract.c sometime this evening. -- A.M. Kuchling http://starship.python.net/crew/amk/ Among the most graceful of birds, they have the ugliest faces; in the countenance of a seagull we observe all the bitter hatred and malignance which we usually associate with the faces of money-lenders or book censors. -- Robertson Davies, _The Table Talk of Samuel Marchbanks_ From guido@python.org Mon Feb 14 23:44:29 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 14 Feb 2000 18:44:29 -0500 Subject: [Patches] Multiply sequences by longs In-Reply-To: Your message of "Mon, 14 Feb 2000 17:16:00 EST." <14504.32416.694611.757508@amarok.cnri.reston.va.us> References: <200002042206.RAA11080@amarok.cnri.reston.va.us> <200002142136.QAA22319@eric.cnri.reston.va.us> <14504.32416.694611.757508@amarok.cnri.reston.va.us> Message-ID: <200002142344.SAA22529@eric.cnri.reston.va.us> > OK; if I have a mandate to fixing slicing and indexing, too, then I'll > hold off on checking in any documentation changes until that's done. > (No point in writing text that says "* can also take a long integer" > when it'll have to be changed to say "* and slicing..." in the next > pass.) > > I'll check in the sequence multiplying patch now, and will send > patches@python.org the remaining changes to abstract.c sometime this > evening. Cool! --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 15 00:15:47 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 14 Feb 2000 19:15:47 -0500 (EST) Subject: [Patches] Complaints new patches list. References: <20000211215056.A7555@stopcontact.palga.uucp> Message-ID: <14504.39603.85034.723363@anthem.cnri.reston.va.us> >>>>> "GS" == Greg Stein writes: GS> There was another issue that came up with nntplib, which was GS> fixed recently. As part of that fix, Barry also changed the GS> exceptions. I believe it was done completely independent of GS> your patch. I doubt anybody but Guido saw your patch. Ergo, GS> nobody refused your patch... it just happened to be done a bit GS> differently by a different person. Gerrit, I'm pretty sure I never saw your patch. There's no match against `nntplib' in my inbox from you, nor is there a match in the Python bug list. I definitely posted a complete proposed diff against nntplib.py which included the exceptions to patches@python.org http://www.python.org/pipermail/patches/2000-February/000022.html I never saw a follow up by you to that thread. My checkin probably just beat your message to the public forums. patches@python.org is a new list, and there's undoubtedly some shaking out that has to happen. -Barry From piers@cs.su.oz.au Tue Feb 15 01:46:44 2000 From: piers@cs.su.oz.au (Piers Lauder) Date: Tue, 15 Feb 2000 12:46:44 +1100 Subject: [Patches] imaplib patches Message-ID: <950582175.149.433994435@cs.usyd.edu.au> --129.78.10.43.126.17167.950582175.165.27135 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi! The following patches to imaplib.py are wrt. the current Python CVS tree (as of 1/2 hour ago) and are generated via "diff -c". They include a patch I sent directly to Guido on the 28th Jan (before I read the latest instructions on python.org - apologies to Guido!) Standard disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Reasons for patches: 1st patch (15,21): version change 2nd patch (66,72): This is a patch I found in a Zope product release (quite by accident!). It relaxes the conditions for matching a literal. I've looked over the logic, and tested it, and it seems sensible. 3rd patch (117,123): It appears the quoting matcher was too general, and that the IMAP4 protocol requires characters like ':' in commands to be unquoted. (This is the patch already sent to Guido.) 4th patch (699,705): Spelling correction in comment. 5th patch (753,761): Another patch from the Zope product. It seems that some IMAP4 servers produce unexpected responses in the middle of valid command/response sequences. This patch ignores the unexpected responses in this situation. (How I wish users would send me bug reports with examples!). last 2 patches: (1015,1028) (1038,1044): Minor improvements to test code. --129.78.10.43.126.17167.950582175.165.27135 Content-Type: application/octet-stream; name=imaplib.diffs Content-Transfer-Encoding: 7bit *** python/dist/src/Lib/imaplib.py Tue Feb 15 13:02:15 2000 --- /usr/staff/piers/lib/html/python/imap4.py Tue Feb 15 12:42:47 2000 *************** *** 15,21 **** Time2Internaldate """ ! __version__ = "2.16" import binascii, re, socket, string, time, random, sys --- 15,21 ---- Time2Internaldate """ ! __version__ = "2.30" import binascii, re, socket, string, time, random, sys *************** *** 66,72 **** r' (?P[0-9][0-9]):(?P[0-9][0-9]):(?P[0-9][0-9])' r' (?P[-+])(?P[0-9][0-9])(?P[0-9][0-9])' r'"') ! Literal = re.compile(r'(?P.*) {(?P\d+)}$') Response_code = re.compile(r'\[(?P[A-Z-]+)( (?P[^\]]*))?\]') Untagged_response = re.compile(r'\* (?P[A-Z-]+)( (?P.*))?') Untagged_status = re.compile(r'\* (?P\d+) (?P[A-Z-]+)( (?P.*))?') --- 66,72 ---- r' (?P[0-9][0-9]):(?P[0-9][0-9]):(?P[0-9][0-9])' r' (?P[-+])(?P[0-9][0-9])(?P[0-9][0-9])' r'"') ! Literal = re.compile(r'.*{(?P\d+)}$') Response_code = re.compile(r'\[(?P[A-Z-]+)( (?P[^\]]*))?\]') Untagged_response = re.compile(r'\* (?P[A-Z-]+)( (?P.*))?') Untagged_status = re.compile(r'\* (?P\d+) (?P[A-Z-]+)( (?P.*))?') *************** *** 117,123 **** class abort(error): pass # Service errors - close and retry class readonly(abort): pass # Mailbox status changed to READ-ONLY ! mustquote = re.compile(r'\W') # Match any non-alphanumeric character def __init__(self, host = '', port = IMAP4_PORT): self.host = host --- 117,123 ---- class abort(error): pass # Service errors - close and retry class readonly(abort): pass # Mailbox status changed to READ-ONLY ! mustquote = re.compile(r"[^\w!#$%&'*+,.:;<=>?^`|~-]") def __init__(self, host = '', port = IMAP4_PORT): self.host = host *************** *** 699,705 **** dat2 = self.mo.group('data2') if self.mo is None: ! # Only other possibility is '+' (continuation) rsponse... if self._match(Continuation, resp): self.continuation_response = self.mo.group('data') --- 699,705 ---- dat2 = self.mo.group('data2') if self.mo is None: ! # Only other possibility is '+' (continuation) response... if self._match(Continuation, resp): self.continuation_response = self.mo.group('data') *************** *** 753,761 **** if result is not None: del self.tagged_commands[tag] return result - self._get_response() def _get_line(self): line = self.file.readline() --- 753,773 ---- if result is not None: del self.tagged_commands[tag] return result + # Some have reported "unexpected response" exceptions. + # (Isn't this non-IMAP4-compliant behaviour? + # Please mail me details printed below!) + # Anyway, ignore them here. + try: + self._get_response() + except self.abort, val: + if __debug__: + if self.debug >= 1: + _mesg('abort exception ignored: %s' % val) + print_log() + + def _get_line(self): line = self.file.readline() *************** *** 1015,1028 **** if sys.argv[1:]: host = sys.argv[1] USER = getpass.getuser() ! PASSWD = getpass.getpass("IMAP password for %s: " % (host or "localhost")) test_seq1 = ( ('login', (USER, PASSWD)), ('create', ('/tmp/xxx 1',)), ('rename', ('/tmp/xxx 1', '/tmp/yyy')), ('CREATE', ('/tmp/yyz 2',)), ! ('append', ('/tmp/yyz 2', None, None, 'From: anon@x.y.z\n\ndata...')), ('list', ('/tmp', 'yy*')), ('select', ('/tmp/yyz 2',)), ('search', (None, '(TO zork)')), --- 1027,1041 ---- if sys.argv[1:]: host = sys.argv[1] USER = getpass.getuser() ! PASSWD = getpass.getpass("IMAP password for %s on %s" % (USER, host or "localhost")) + test_mesg = 'From: %s@localhost\nSubject: IMAP4 test\n\ndata...\n' % USER test_seq1 = ( ('login', (USER, PASSWD)), ('create', ('/tmp/xxx 1',)), ('rename', ('/tmp/xxx 1', '/tmp/yyy')), ('CREATE', ('/tmp/yyz 2',)), ! ('append', ('/tmp/yyz 2', None, None, test_mesg)), ('list', ('/tmp', 'yy*')), ('select', ('/tmp/yyz 2',)), ('search', (None, '(TO zork)')), *************** *** 1038,1044 **** ('response',('UIDVALIDITY',)), ('uid', ('SEARCH', 'ALL')), ('response', ('EXISTS',)), ! ('append', (None, None, None, 'From: anon@x.y.z\n\ndata...')), ('recent', ()), ('logout', ()), ) --- 1051,1057 ---- ('response',('UIDVALIDITY',)), ('uid', ('SEARCH', 'ALL')), ('response', ('EXISTS',)), ! ('append', (None, None, None, test_mesg)), ('recent', ()), ('logout', ()), ) --129.78.10.43.126.17167.950582175.165.27135-- From Gerrit Tue Feb 15 06:27:42 2000 From: Gerrit (Gerrit Holl) Date: Tue, 15 Feb 2000 07:27:42 +0100 Subject: [Patches] Patch for Tools/scripts/crlf.py, lfcr.py, logmerge.py In-Reply-To: <200002142144.QAA22417@eric.cnri.reston.va.us>; from guido@python.org on Mon, Feb 14, 2000 at 04:44:17PM -0500 References: <20000204225544.A18820@stopcontact.palga.uucp> <200002142144.QAA22417@eric.cnri.reston.va.us> Message-ID: <20000215072742.A671@stopcontact.palga.uucp> Guido van Rossum wrote on 950543057: > Gerrit, > > Thanks for the patches. I've personally applied them. Thanks! > Please next time be a little more patient and kind with our overworked > volunteers. They deserve kindness, not attitude. (And then, so do > you. :-) I apologize for my impatience! Sorry all, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From tim_one@email.msn.com Tue Feb 15 08:42:13 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Feb 2000 03:42:13 -0500 Subject: [Patches] Re: Add crc32() method to binascii In-Reply-To: <38A8627F.E93B9517@interet.com> Message-ID: <001f01bf7790$8e4bee00$66a2143f@tim> [James C. Ahlstrom] > This is a corrected patch. Thanks to Tim Peters for helpful comments. Jim addressed every complaint I had, so ship it . If anyone has a 64-bit machine, please try it there (I ran it "in my head" on two different 64-bit architectures and *believe* it will work, even if sizeof(long) > 4 -- but no substitute for actually trying it. Note that there's a test case in the new patch, so trying it should be easy.). From akuchlin@mems-exchange.org Tue Feb 15 15:27:58 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Tue, 15 Feb 2000 10:27:58 -0500 (EST) Subject: [Patches] Long ints for getitem,setitem,slices Message-ID: <200002151527.KAA20518@amarok.cnri.reston.va.us> This patch allows long integers for getitem, setitem, and slicing. I think it's not complete yet, though, because sequence[-5L : pow(2,700L) ] raises an OverflowError, and it should really treat a too-large longint the same as a too-large integer. How should I fix this in Python/ceval.c? Compare the long int to sys.maxint? Take the longint's value as a "long long" instead, and check if it's too big to fit into a "long"? Ignore the error if OverflowError was raised? A little test script I've been using: L = range(0,21) # Test getitem print L[ 4L ] # Test setitem L[4L] = 17.92 # test slicing print L[0L : 5] print L[0L : 5L] print L[0 : 5L] # Test slicing w/ a -1 index (since it's also an error value) print L[-5 : -1L] print L[-5L : -1L] print L[-5L : -1] # Test overflows; both raise OverflowError #print L[-5L : pow(2,700L) ] #print L[ pow(2,700L) ] -- A.M. Kuchling http://starship.python.net/crew/amk/ Saying "Imminent death of the net predicted!": Imminent execution of poster predicted. -- Kibo, in the Happynet Manifesto Index: Objects//abstract.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Objects/abstract.c,v retrieving revision 2.29 diff -C2 -r2.29 abstract.c *** abstract.c 2000/02/14 22:22:04 2.29 --- abstract.c 2000/02/15 15:14:27 *************** *** 123,126 **** --- 123,132 ---- if (PyInt_Check(key)) return PySequence_GetItem(o, PyInt_AsLong(key)); + else if (PyLong_Check(key)) { + long key_value = PyLong_AsLong(key); + if (key_value == -1 && PyErr_Occurred()) + return NULL; + return PySequence_GetItem(o, key_value); + } return type_error("sequence index must be integer"); } *************** *** 148,151 **** --- 154,163 ---- if (PyInt_Check(key)) return PySequence_SetItem(o, PyInt_AsLong(key), value); + else if (PyLong_Check(key)) { + long key_value = PyLong_AsLong(key); + if (key_value == -1 && PyErr_Occurred()) + return -1; + return PySequence_SetItem(o, key_value, value); + } type_error("sequence index must be integer"); return -1; *************** *** 174,177 **** --- 186,195 ---- if (PyInt_Check(key)) return PySequence_DelItem(o, PyInt_AsLong(key)); + else if (PyLong_Check(key)) { + long key_value = PyLong_AsLong(key); + if (key_value == -1 && PyErr_Occurred()) + return -1; + return PySequence_DelItem(o, key_value); + } type_error("sequence index must be integer"); return -1; *************** *** 392,396 **** else if (PyLong_Check(w)) { mul_value = PyLong_AsLong(w); ! if (PyErr_Occurred()) return NULL; } --- 410,414 ---- else if (PyLong_Check(w)) { mul_value = PyLong_AsLong(w); ! if (mul_value == -1 && PyErr_Occurred()) return NULL; } Index: Python//ceval.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Python/ceval.c,v retrieving revision 2.166 diff -C2 -r2.166 ceval.c *** ceval.c 2000/01/12 22:45:54 2.166 --- ceval.c 2000/02/15 15:05:19 *************** *** 2538,2547 **** if (v != NULL) { long x; ! if (!PyInt_Check(v)) { PyErr_SetString(PyExc_TypeError, "slice index must be int"); return -1; } - x = PyInt_AsLong(v); /* Truncate -- very long indices are truncated anyway */ if (x > INT_MAX) --- 2538,2552 ---- if (v != NULL) { long x; ! if (PyInt_Check(v)) { ! x = PyInt_AsLong(v); ! } else if (PyLong_Check(v)) { ! x = PyLong_AsLong(v); ! if (x==-1 && PyErr_Occurred()) ! return -1; ! } else { PyErr_SetString(PyExc_TypeError, "slice index must be int"); return -1; } /* Truncate -- very long indices are truncated anyway */ if (x > INT_MAX) From guido@python.org Tue Feb 15 15:48:18 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 15 Feb 2000 10:48:18 -0500 Subject: [Patches] Long ints for getitem,setitem,slices In-Reply-To: Your message of "Tue, 15 Feb 2000 10:27:58 EST." <200002151527.KAA20518@amarok.cnri.reston.va.us> References: <200002151527.KAA20518@amarok.cnri.reston.va.us> Message-ID: <200002151548.KAA23472@eric.cnri.reston.va.us> > This patch allows long integers for getitem, setitem, and slicing. > > I think it's not complete yet, though, because sequence[-5L : pow(2,700L) ] > raises an OverflowError, and it should really treat a too-large > longint the same as a too-large integer. How should I fix this in Python/ceval.c? > Compare the long int to sys.maxint? Take the longint's value as a > "long long" instead, and check if it's too big to fit into a "long"? > Ignore the error if OverflowError was raised? You can't ignore the error, because the OverflowError could mean that the number is negative (and then it should be treated as -sys.maxint, which gets treated the same as 0). I personally think it's no big deal to raise the OverflowError. If you really want to be strict, catch the OverflowError, and then inspect the sign of the original long int to decide whether it should mean sys.maxint or 0. (This saves time in the common case -- no overflow.) --Guido van Rossum (home page: http://www.python.org/~guido/) From da@ski.org Tue Feb 15 16:52:05 2000 From: da@ski.org (David Ascher) Date: Tue, 15 Feb 2000 08:52:05 -0800 Subject: [Patches] Long ints for getitem,setitem,slices References: <200002151527.KAA20518@amarok.cnri.reston.va.us> Message-ID: <00c301bf77d5$01dfd950$0100000a@ski.org> From: Andrew M. Kuchling > This patch allows long integers for getitem, setitem, and slicing. [...] > A little test script I've been using: [...] I'm going to suggest that the guidelines for patches which add functionality should include a regression test patch against the test/ directory. I'm not worried about Andrew forgetting it, but I think that having such a patch not only solidifies the test suite, but also helps understand the purpose and scope of a patch. --david From guido@python.org Tue Feb 15 16:54:31 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 15 Feb 2000 11:54:31 -0500 Subject: [Patches] Long ints for getitem,setitem,slices In-Reply-To: Your message of "Tue, 15 Feb 2000 08:52:05 PST." <00c301bf77d5$01dfd950$0100000a@ski.org> References: <200002151527.KAA20518@amarok.cnri.reston.va.us> <00c301bf77d5$01dfd950$0100000a@ski.org> Message-ID: <200002151654.LAA23774@eric.cnri.reston.va.us> > I'm going to suggest that the guidelines for patches which add > functionality should include a regression test patch against the > test/ directory. Excellent. I'll add this and the requirement to include documentation in the patch guidelines. --Guido van Rossum (home page: http://www.python.org/~guido/) From Gerrit Tue Feb 15 20:46:53 2000 From: Gerrit (Gerrit Holl) Date: Tue, 15 Feb 2000 21:46:53 +0100 Subject: [Patches] ftp.python.org/pub/python/src/README Message-ID: <20000215214653.A2366@stopcontact.palga.uucp> Hello, I'm not sure wheter to tell it here or somewhere else, but in ftp://ftp.python.org/pub/python/src/README, the user it told to send patches to Guido@python.org. This address should - of course - be changed to patches@python.org. regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From guido@python.org Tue Feb 15 21:14:27 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 15 Feb 2000 16:14:27 -0500 Subject: [Patches] ftp.python.org/pub/python/src/README In-Reply-To: Your message of "Tue, 15 Feb 2000 21:46:53 +0100." <20000215214653.A2366@stopcontact.palga.uucp> References: <20000215214653.A2366@stopcontact.palga.uucp> Message-ID: <200002152114.QAA25963@eric.cnri.reston.va.us> Thanks, Gerrit. Fixed. If you find more of these, the proper address is really ftpmaster@python.org. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Feb 16 20:06:47 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 15:06:47 -0500 Subject: [Patches] __contains__ hook, done right In-Reply-To: Your message of "Sat, 05 Feb 2000 02:09:05 +0200." References: Message-ID: <200002162006.PAA28892@eric.cnri.reston.va.us> Moshe, I've finally had some time to review your patch. I think it's close. > diff -r -c ../../../python/dist/src/Include/object.h ./Include/object.h > *** ../../../python/dist/src/Include/object.h Wed Feb 2 16:02:51 2000 > --- ./Include/object.h Fri Feb 4 13:39:13 2000 > *************** > *** 151,156 **** > --- 151,160 ---- > typedef int (*getsegcountproc) Py_PROTO((PyObject *, int *)); > typedef int (*getcharbufferproc) Py_PROTO((PyObject *, int, const char **)); > > + /* tentative (by Moshe) */ Please don't put comments like this in :-) (See the patches style guide: http://www.python.org/patches/style.html). > + typedef int(*objobjproc) Py_PROTO((PyObject *, PyObject *)); > + > + > typedef struct { > binaryfunc nb_add; > binaryfunc nb_subtract; > *************** > *** 185,190 **** > --- 189,195 ---- > intintargfunc sq_slice; > intobjargproc sq_ass_item; > intintobjargproc sq_ass_slice; > + objobjproc sq_contains; > } PySequenceMethods; > > typedef struct { > *************** > *** 316,321 **** > --- 321,327 ---- > > /* PyBufferProcs contains bf_getcharbuffer */ > #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) > + #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) > > #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) I would modify this to include Py_TPFLAGS_HAVE_SEQUENCE_IN by default. The flag means that the code knows that the sq_contains field exists; not that this particular object has a non-NULL value in it. So it can always be on in code compiled with this version of the header file. > diff -r -c ../../../python/dist/src/Objects/abstract.c ./Objects/abstract.c > *** ../../../python/dist/src/Objects/abstract.c Fri Oct 15 14:09:02 1999 > --- ./Objects/abstract.c Fri Feb 4 14:00:45 2000 > *************** > *** 1110,1116 **** > --- 1110,1121 ---- > } > return 0; > } > + if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { > + sq = w->ob_type->tp_as_sequence; Here you have to check that sq != NULL first. (This is because I changed the meaning of the flag!) > + if(sq->sq_contains != NULL) > + return (*sq->sq_contains)(w, v); > > + } I would put the blank line *after* the brace :-) > sq = w->ob_type->tp_as_sequence; > if (sq == NULL || sq->sq_item == NULL) { > PyErr_SetString(PyExc_TypeError, > diff -r -c ../../../python/dist/src/Objects/classobject.c ./Objects/classobject.c > *** ../../../python/dist/src/Objects/classobject.c Wed Feb 2 16:03:19 2000 > --- ./Objects/classobject.c Fri Feb 4 13:41:03 2000 > *************** > *** 1065,1070 **** > --- 1065,1096 ---- > return 0; > } > > + static int instance_contains(PyInstanceObject *inst, PyObject *member) > + { > + static PyObject *__contains__; > + PyObject *func, *arg, *res; > + int ret; > + > + if(__contains__ == NULL) > + __contains__ = PyString_InternFromString("__contains__"); Wot? No error check? What if we run out of memory? (See examples using PyString_InternFromString() elsewhere in classobject.c.) > + func = instance_getattr(inst, __contains__); > + if(func == NULL) > + return -1; > + arg = Py_BuildValue("(O)", member); > + if(arg == NULL) { > + Py_DECREF(func); > + return -1; > + } > + res = PyEval_CallObject(func, arg); > + Py_DECREF(func); > + Py_DECREF(arg); > + if(res == NULL) > + return -1; > + ret = PyObject_IsTrue(res); > + Py_DECREF(res); > + return ret; > + } > + Good! > static PySequenceMethods instance_as_sequence = { > (inquiry)instance_length, /*sq_length*/ > 0, /*sq_concat*/ > *************** > *** 1073,1078 **** > --- 1099,1105 ---- > (intintargfunc)instance_slice, /*sq_slice*/ > (intobjargproc)instance_ass_item, /*sq_ass_item*/ > (intintobjargproc)instance_ass_slice, /*sq_ass_slice*/ > + (objobjproc)instance_contains, /* sq_contains */ > }; > > static PyObject * > *************** > *** 1405,1410 **** > --- 1432,1439 ---- > 0, /*tp_str*/ > (getattrofunc)instance_getattr, /*tp_getattro*/ > (setattrofunc)instance_setattr, /*tp_setattro*/ > + 0, /* tp_as_buffer */ > + Py_TPFLAGS_HAVE_SEQUENCE_IN, /* tp_flags */ This could be Py_TPFLAGS_DEFAULT. > }; > > > I was hoping that the special casing for strings, lists and tuples would also be moved to those respective files, from the code in abstract.c. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Feb 16 20:22:06 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 15:22:06 -0500 Subject: [Patches] Re: urllib suggestion In-Reply-To: Your message of "Tue, 08 Feb 2000 15:01:33 CST." <200002082101.PAA19047@beluga.mojam.com> References: <200002082101.PAA19047@beluga.mojam.com> Message-ID: <200002162022.PAA28954@eric.cnri.reston.va.us> > A request to the python-help mailing list by Markus Demleitner suggested > exposing URLOpener.server_version at the module level so that applications > can override it as appropriate. > > The appended patch adds a __client_name__ variable to the module level which > is initialized to "Python-urllib" and renames __version__ to > __client_version__. These variables are used to construct > URLopener.server_version. I renamed __version__ to __client_version__ to > make it clear that the version number is sent from the client to the server, > is not there simply to document the version of urllib.py. To affect the > headers sent as a result of a call to urlopen or urlretrieve, they must be > set before either function is called. I'm not real keen on this particular change. First of all, if we're not going to use __version__, there's little need to use variables with a __spam__ format. Second, I was using __version__ precisely because it documents the version of the urllib.py file (assuming it gets updated). > An alternative is to document that the undocumented FancyURLopener class can > be subclassed to override server_version and this class then used as the > value of urllib._urlopener for use by urlopen and urlretrieve. I'm not > certain which alternative is better. Both mechanisms require the programmer > to read the code. I'll let someone else argue the pros and cons of the two > approaches (though URLopener and FancyURLopener should probably be > documented). I would prefer that FancyURLopener was documented anyway! How about this patch? Index: urllib.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/urllib.py,v retrieving revision 1.91 diff -c -r1.91 urllib.py *** urllib.py 2000/02/04 15:28:41 1.91 --- urllib.py 2000/02/16 20:19:20 *************** *** 28,34 **** import sys ! __version__ = '1.12' # XXX This version is not always updated :-( MAXFTPCACHE = 10 # Trim the ftp cache beyond this size --- 28,34 ---- import sys ! __version__ = '1.13' # XXX This version is not always updated :-( MAXFTPCACHE = 10 # Trim the ftp cache beyond this size *************** *** 81,86 **** --- 81,88 ---- __tempfiles = None + server_version = "Python-urllib/%s" % __version__ + # Constructor def __init__(self, proxies=None, **x509): if proxies is None: *************** *** 89,96 **** self.proxies = proxies self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') ! server_version = "Python-urllib/%s" % __version__ ! self.addheaders = [('User-agent', server_version)] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() self.tempcache = None --- 91,97 ---- self.proxies = proxies self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') ! self.addheaders = [('User-agent', self.server_version)] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() self.tempcache = None > A minor nit: I also changed the triple-quoted module doc string to use ''' > instead of """ as the string delimiter. This helps Emacs' python-mode > behave better when colorizing the file's contents since the doc string > contains " characters. Keep or toss. Yes, sigh. --Guido van Rossum (home page: http://www.python.org/~guido/) From Fredrik Lundh" <200002162022.PAA28954@eric.cnri.reston.va.us> Message-ID: <010f01bf78be$4fae3700$34aab5d4@hagrid> talking about urllib, here's an idea: - what about adding a "content_type" argument to urlopen (etc) so users can override the default content type? (among others, xmlrpclib users would love this). ... or perhaps better, provide a Body wrapper class which can be used to control the method, type, and perhaps also other fields. e.g: class Body: def __init__(self, data, type=3DNone, method=3DNone): self.data =3D data self.type =3D type or "application/x-www-form-urlencoded") self.method =3D method or "POST" and use isinstance(data, Body) inside URLOpener.open_http... could also provide a FormBody subclass which takes a dictionary or 2-tuple list, and encodes it properly. class FormBody(Body): def __init__(self, dict, method=3DNone): ... ... what do you think? if anyone says "+1", I'll post patches. From guido@python.org Wed Feb 16 20:57:48 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 15:57:48 -0500 Subject: [Patches] urllib NLST patch In-Reply-To: Your message of "Fri, 11 Feb 2000 16:02:49 +0100." <021f01bf74a1$112a3a40$34aab5d4@hagrid> References: <021f01bf74a1$112a3a40$34aab5d4@hagrid> Message-ID: <200002162057.PAA29086@eric.cnri.reston.va.us> > from the newsgroup: > > Oleg Broytmann wrote: > > What am I doing wrong? And how to use urllib to get files by FTP? > > > > fname, headers = urllib.urlretrieve("ftp://koobera.math.uic.edu/www/djb.html") > > > > IOError: [Errno ftp error] 426 Transmission failure: not a directory > > background: urllib issues an NLST command to make sure > the file exists, before starting the download. however, > RFC959 doesn't necessarily allow this: > > [NLST] causes a directory listing to be sent from > server to user site. The pathname should specify a > directory or other system-specific file group descriptor; > a null argument implies the current directory. > > most FTP servers seem to treat NLST commands as > an ordinary "ls -l", but this one doesn't. > > I'm pretty sure the only reasonable solution is to remove > the offending code from urllib... > > -- end of posting > [effbot] > and here's a patch that does exactly that. cannot see > any problems with it, but on the other hand, I'm pretty > sure the NLST call was added for a reason. > > please discuss before applying. The NLST call was added because otherwise giving a bad filename will look like getting an empty file. Try it: after applying your patch, import urllib f = urllib.urlopen("ftp://ftp.python.org/this-file-doesnt-exist") print `f.read()` will succeed, and print ''. The question is, why is the login in retrfile() so convoluted? The answer lies in the behavior of the LIST and RETR commands, and the desire to guess whether the given URL refers to a file or a directory. For reasons that escape me (and hear I have to concur with Greg that FTP sucks :-), it's really in FTP hard without parsing unspecified output or error messages to determine whether a given name is (1) a plain file (2) a directory (3) non-existent The RETR command gives the same error code for cases (2) and (3). The LIST command never gives an error in a typical situation, it just returns no output plus a 226 success code when you list a non-existing directory, but that's the same thing it gives when you list an existing but empty directory. Thus, we can't distinguish between the three cases with just the LIST and RETR commands. The existing logic tries to rely on the experimental fact that NLST gives an error in case (3) but not in cases (1) and (2). I suppose to rework the logic slightly, so that the NLST is only tried after the RETR fails with a code 550 error. Comments? Index: urllib.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/urllib.py,v retrieving revision 1.91 diff -c -r1.91 urllib.py *** urllib.py 2000/02/04 15:28:41 1.91 --- urllib.py 2000/02/16 20:55:52 *************** *** 646,658 **** self.ftp.voidcmd(cmd) conn = None if file and not isdir: - # Use nlst to see if the file exists at all - try: - self.ftp.nlst(file) - except ftplib.error_perm, reason: - raise IOError, ('ftp error', reason), sys.exc_info()[2] - # Restore the transfer mode! - self.ftp.voidcmd(cmd) # Try to retrieve as a file try: cmd = 'RETR ' + file --- 646,651 ---- *************** *** 661,666 **** --- 654,665 ---- if reason[:3] != '550': raise IOError, ('ftp error', reason), sys.exc_info()[2] if not conn: + # Use nlst to see if the file exists at all + if file: + try: + self.ftp.nlst(file) + except ftplib.error_perm, reason: + raise IOError, ('ftp error', reason), sys.exc_info()[2] # Set transfer mode to ASCII! self.ftp.voidcmd('TYPE A') # Try a directory listing --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Feb 16 21:05:29 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 16:05:29 -0500 Subject: [Patches] Re: urllib suggestion In-Reply-To: Your message of "Wed, 16 Feb 2000 21:42:12 +0100." <010f01bf78be$4fae3700$34aab5d4@hagrid> References: <200002082101.PAA19047@beluga.mojam.com> <200002162022.PAA28954@eric.cnri.reston.va.us> <010f01bf78be$4fae3700$34aab5d4@hagrid> Message-ID: <200002162105.QAA29205@eric.cnri.reston.va.us> I'm not thrilled of discussion about half-formed features here. Patch or nothing! :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Feb 16 21:31:29 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 16:31:29 -0500 Subject: [Patches] New module zipfile.py In-Reply-To: Your message of "Fri, 11 Feb 2000 11:23:38 EST." <38A4378A.CD4BE4B3@interet.com> References: <38A4378A.CD4BE4B3@interet.com> Message-ID: <200002162131.QAA29480@eric.cnri.reston.va.us> Has anybody here played with Jim's zipfile.py module? My own response is that the code looks a bit dense, with 2-space indents, very few blank lines, and some lines longer than 78 chars, but that it is otherwise decent; however I don't have the time to stress-test it. I noticed that when I run the self-test (which should be moved into a module test/test_zipfile.py) I get an error: & ../sparc/python Python 1.5.2+ (#1017, Feb 16 2000, 16:03:45) [GCC 2.8.1] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import zipfile >>> zipfile.test() Traceback (innermost last): File "", line 1, in ? File "zipfile.py", line 335, in test z.writepy("Lib") # Directory ./Lib must exist File "zipfile.py", line 279, in writepy raise RuntimeError, 'Files added with writepy() must end with ".py"' RuntimeError: Files added with writepy() must end with ".py" >>> --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Feb 16 22:11:31 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 17:11:31 -0500 Subject: [Patches] httplib.py timeout In-Reply-To: Your message of "Fri, 11 Feb 2000 15:58:26 PST." <6035C3D0EB76D31183B800902762824E040920@adsl-63-196-9-132.dsl.snfc21.pacbell.net> References: <6035C3D0EB76D31183B800902762824E040920@adsl-63-196-9-132.dsl.snfc21.pacbell.net> Message-ID: <200002162211.RAA29632@eric.cnri.reston.va.us> > This patch allows the user of httplib.py to specify a timeout in > milliseconds. If a timeout is not specified, a default of sixty seconds is > assumed. The user may specify no timeout by passing a timeout of zero. > This patch is intended to be especially useful to urllib.py; there was > discussion of this patch in comp.lang.python at the end of December, with > general agreement that this patch is appropriate. On what systems is this supported? The patch contains an unqualified except. When I tried it, it didn't seem to make a difference; when I added print statements to those except clauses, I found that I got an error. On Solaris 2.7, I get: error: (99, 'Option not supported by protocol') (that's socket.error) On Linux: $ uname -a Linux loony.cnri.reston.va.us 2.0.34 #1 Fri May 8 16:05:57 EDT 1998 i586 unknown $ ./python ~/to.py -t 200 10.27.54.45 AttributeError: SO_RCVTIMEO On Windows NT, I seem to get neither error, but it doesn't seem to honor any timeout, either... This way, it doesn't seem very useful, and the unqualified except clause doesn't give me a lot of trust in the code. (Also, it seems wrong in Python to specify a timeout in milliseconds -- generally, times are specified in seconds as floating point numbers.) --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Wed Feb 16 22:21:40 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 17 Feb 2000 00:21:40 +0200 (IST) Subject: [Patches] __contains__ hook, done right In-Reply-To: <200002162006.PAA28892@eric.cnri.reston.va.us> Message-ID: On Wed, 16 Feb 2000, Guido van Rossum wrote: > I've finally had some time to review your patch. I think it's close. Thanks. > I would modify this to include Py_TPFLAGS_HAVE_SEQUENCE_IN by default. > The flag means that the code knows that the sq_contains field exists; > not that this particular object has a non-NULL value in it. So it can > always be on in code compiled with this version of the header file. You're right: I wasn't thinking straight. I'll change this. > I was hoping that the special casing for strings, lists and tuples > would also be moved to those respective files, from the code in > abstract.c. Well, special casing for strings *will* be moved, but I'd rather have it in two distinct patches: one (this one, basically, after I clean it up a little) introduces the mechanism and changes InstanceType, and a second one removes the special casing for strings into StringType. Special casing for sequences which don't know about the new method, which basically does the "for" loop will remain: sometimes, it *is* the best way, and it should be coded generically. Thanks for the comments! -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gstein@lyra.org Wed Feb 16 23:41:57 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 16 Feb 2000 15:41:57 -0800 (PST) Subject: [Patches] tp_flags (was: __contains__ hook, done right) In-Reply-To: <200002162006.PAA28892@eric.cnri.reston.va.us> Message-ID: On Wed, 16 Feb 2000, Guido van Rossum wrote: >... > > > > /* PyBufferProcs contains bf_getcharbuffer */ > > #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) > > + #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) If this flag is going to be defined, then it needs a comment about it. The above code seems to imply that HAVE_SEQUENCE_IN is related to the PyBufferProces. > > > > #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) > > I would modify this to include Py_TPFLAGS_HAVE_SEQUENCE_IN by default. > The flag means that the code knows that the sq_contains field exists; > not that this particular object has a non-NULL value in it. So it can > always be on in code compiled with this version of the header file. Guido: adding new flags is *only* necessary when you want to avoid changes in the PYTHON_API_VERSION. If the API has already changed between 1.5 and 1.6, then PYTHON_API_VERSION should be bumped, and this new tp_flags value is not necessary. In fact, when you bump the VERSION, it can even be argued that these specific flags get obsoleted (since an extension must be compiled with the new VERSION to be properly loaded, which makes it pick up the new slot). So... I'd pose this question to you, Guido: will the API version be bumped for Python 1.6? If so, then we have some potential cleanup that can occur. (note: tp_flags is not *reserved* for slot extensions; it is simply that we haven't discovered any other flags to put in there yet) > > *************** > > *** 1405,1410 **** > > --- 1432,1439 ---- > > 0, /*tp_str*/ > > (getattrofunc)instance_getattr, /*tp_getattro*/ > > (setattrofunc)instance_setattr, /*tp_setattro*/ > > + 0, /* tp_as_buffer */ > > + Py_TPFLAGS_HAVE_SEQUENCE_IN, /* tp_flags */ > > This could be Py_TPFLAGS_DEFAULT. I'd rephrase as *should*. Remember: the flag bits (as used today) are to determine whether a slot exists -- in lieu of changing the PYTHON_API_VERSION. Once you compile under the new definition of PyBufferProcs or PySequenceMethods, then the slots will definitely exist; therefore, Py_TPFLAGS_DEFAULT should be used. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@python.org Thu Feb 17 00:55:54 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 19:55:54 -0500 Subject: [Patches] tp_flags (was: __contains__ hook, done right) In-Reply-To: Your message of "Wed, 16 Feb 2000 15:41:57 PST." References: Message-ID: <200002170055.TAA29872@eric.cnri.reston.va.us> > > > /* PyBufferProcs contains bf_getcharbuffer */ > > > #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) > > > + #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) > > If this flag is going to be defined, then it needs a comment about it. The > above code seems to imply that HAVE_SEQUENCE_IN is related to the > PyBufferProces. Good catch! > > > #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) > > > > I would modify this to include Py_TPFLAGS_HAVE_SEQUENCE_IN by default. > > The flag means that the code knows that the sq_contains field exists; > > not that this particular object has a non-NULL value in it. So it can > > always be on in code compiled with this version of the header file. > > Guido: adding new flags is *only* necessary when you want to avoid changes > in the PYTHON_API_VERSION. If the API has already changed between 1.5 and > 1.6, then PYTHON_API_VERSION should be bumped, and this new tp_flags value > is not necessary. > > In fact, when you bump the VERSION, it can even be argued that these > specific flags get obsoleted (since an extension must be compiled with the > new VERSION to be properly loaded, which makes it pick up the new slot). > > So... I'd pose this question to you, Guido: will the API version be bumped > for Python 1.6? If so, then we have some potential cleanup that can occur. I'd like to keep the API version unchanged from 1.5.2, if at all possible. That will depend on how we end up doing new coercions and rich comparisons. > (note: tp_flags is not *reserved* for slot extensions; it is simply that > we haven't discovered any other flags to put in there yet) Indeed. > > > + 0, /* tp_as_buffer */ > > > + Py_TPFLAGS_HAVE_SEQUENCE_IN, /* tp_flags */ > > > > This could be Py_TPFLAGS_DEFAULT. > > I'd rephrase as *should*. > > Remember: the flag bits (as used today) are to determine whether a slot > exists -- in lieu of changing the PYTHON_API_VERSION. Once you compile > under the new definition of PyBufferProcs or PySequenceMethods, then the > slots will definitely exist; therefore, Py_TPFLAGS_DEFAULT should be used. Yeah, sure. --Guido van Rossum (home page: http://www.python.org/~guido/) From mhammond@skippinet.com.au Thu Feb 17 08:03:26 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 17 Feb 2000 19:03:26 +1100 Subject: [Patches] Windows specific exception. Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_006A_01BF7979.ABA479F0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit This is a set of patches as discussed on python-dev to add a new standard Windows error to the code. Most of the changes should be obvious, but the following are worth noting: * No support for string exceptions (as OK'd by Guido) * Changes to a recent patch by Chris Tismer to errors.c. Chris' patch always used FormatMessage() to get the error message passing the error code from errno - but errno and FormatMessage use a different numbering scheme. The main reason the patch looked OK was that ENOFILE==ERROR_FILE_NOT_FOUND - but that is about the only shared error code :-). The MS CRT docs tell you to use _sys_errlist()/_sys_nerr. My patch does also this, and adds a very similar function specifically for win32 error codes. For example, under Windows the existing CVS version of Python yields: >>> os.unlink(sys.executable) ... OSError: [Errno 13] The data is invalid: 'L:\\src\\...' Wheras the new code correctly reports: OSError: [Errno 13] Permission denied: '...' When I hear back about this I will submit the win32reg module (which depends on these patches) Thanks, Mark. Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ------=_NextPart_000_006A_01BF7979.ABA479F0 Content-Type: text/plain; name="diff.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="diff.txt" Index: errors.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /projects/cvsroot/python/dist/src/Python/errors.c,v retrieving revision 2.41 diff -c -r2.41 errors.c *** errors.c 1999/04/21 15:27:31 2.41 --- errors.c 2000/02/17 07:59:05 *************** *** 289,294 **** --- 289,297 ---- PyObject *v; char *s; int i =3D errno; + #ifdef MS_WIN32 + char *s_buf =3D NULL; + #endif #ifdef EINTR if (i =3D=3D EINTR && PyErr_CheckSignals()) return NULL; *************** *** 300,319 **** s =3D strerror(i); #else { ! int len =3D FormatMessage( ! FORMAT_MESSAGE_ALLOCATE_BUFFER | ! FORMAT_MESSAGE_FROM_SYSTEM | ! FORMAT_MESSAGE_IGNORE_INSERTS, ! NULL, /* no message source */ ! i, ! MAKELANGID(LANG_NEUTRAL, ! SUBLANG_DEFAULT), /* Default language */ ! (LPTSTR) &s, ! 0, /* size not used */ ! NULL); /* no args */ ! /* remove trailing cr/lf and dots */ ! while (len > 0 && s[len-1] <=3D '.') ! s[--len] =3D '\0'; } #endif if (filename !=3D NULL && Py_UseClassExceptionsFlag) --- 303,333 ---- s =3D strerror(i); #else { ! /* Note that the Win32 errors do not lineup with the ! errno error. So if the error is in the MSVC error ! table, we use it, otherwise we assume it really _is_=20 ! a Win32 error code ! */ ! if (i < _sys_nerr) { ! s =3D _sys_errlist[i]; ! } ! else { ! int len =3D FormatMessage( ! FORMAT_MESSAGE_ALLOCATE_BUFFER | ! FORMAT_MESSAGE_FROM_SYSTEM | ! FORMAT_MESSAGE_IGNORE_INSERTS, ! NULL, /* no message source */ ! i, ! MAKELANGID(LANG_NEUTRAL, ! SUBLANG_DEFAULT), /* Default language */ ! (LPTSTR) &s_buf, ! 0, /* size not used */ ! NULL); /* no args */ ! s =3D s_buf; ! /* remove trailing cr/lf and dots */ ! while (len > 0 && s[len-1] < '.') ! s[--len] =3D '\0'; ! } } #endif if (filename !=3D NULL && Py_UseClassExceptionsFlag) *************** *** 325,331 **** Py_DECREF(v); } #ifdef MS_WIN32 ! LocalFree(s); #endif return NULL; } --- 339,345 ---- Py_DECREF(v); } #ifdef MS_WIN32 ! LocalFree(s_buf); #endif return NULL; } *************** *** 337,342 **** --- 351,401 ---- { return PyErr_SetFromErrnoWithFilename(exc, NULL); } +=20 + #ifdef MS_WINDOWS=20 + /* Windows specific error code handling */ + PyObject *PyErr_SetFromWindowsErrWithFilename( + int ierr,=20 + const char *filename) + { + int len; + char *s; + PyObject *v; + DWORD err =3D (DWORD)ierr; + if (err=3D=3D0) err =3D GetLastError(); + len =3D FormatMessage( + /* Error API error */ + FORMAT_MESSAGE_ALLOCATE_BUFFER | + FORMAT_MESSAGE_FROM_SYSTEM | + FORMAT_MESSAGE_IGNORE_INSERTS, + NULL, /* no message source */ + err, + MAKELANGID(LANG_NEUTRAL, + SUBLANG_DEFAULT), /* Default language */ + (LPTSTR) &s, + 0, /* size not used */ + NULL); /* no args */ + /* remove trailing cr/lf */ + while (len > 0 && s[len-1] < '.') + s[--len] =3D '\0'; + if (filename !=3D NULL && Py_UseClassExceptionsFlag) + v =3D Py_BuildValue("(iss)", err, s, filename); + else + v =3D Py_BuildValue("(is)", err, s); + if (v !=3D NULL) { + PyErr_SetObject(PyExc_EnvironmentError, v); + Py_DECREF(v); + } + LocalFree(s); + return NULL; + } +=20 + PyObject *PyErr_SetFromWindowsErr(int ierr) + { + return PyErr_SetFromWindowsErrWithFilename(ierr, NULL); +=20 + } + #endif /* MS_WINDOWS */ =20 void PyErr_BadInternalCall() Index: bltinmodule.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /projects/cvsroot/python/dist/src/Python/bltinmodule.c,v retrieving revision 2.147 diff -c -r2.147 bltinmodule.c *** bltinmodule.c 1999/12/23 14:16:55 2.147 --- bltinmodule.c 2000/02/17 07:59:15 *************** *** 2262,2267 **** --- 2262,2270 ---- PyObject *PyExc_TypeError; PyObject *PyExc_ValueError; PyObject *PyExc_ZeroDivisionError; + #ifdef MS_WINDOWS + PyObject *PyExc_WindowsError; + #endif =20 PyObject *PyExc_MemoryErrorInst; =20 *************** *** 2303,2308 **** --- 2306,2314 ---- {"UnboundLocalError", &PyExc_UnboundLocalError, 1}, {"TypeError", &PyExc_TypeError, 1}, {"ValueError", &PyExc_ValueError, 1}, + #ifdef MS_WINDOWS + {"WindowsError", &PyExc_WindowsError, 1}, + #endif {"ZeroDivisionError", &PyExc_ZeroDivisionError, 1}, {NULL, NULL} }; Index: pyerrors.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /projects/cvsroot/python/dist/src/Include/pyerrors.h,v retrieving revision 2.33 diff -c -r2.33 pyerrors.h *** pyerrors.h 1999/06/22 14:46:42 2.33 --- pyerrors.h 2000/02/17 07:59:23 *************** *** 81,89 **** extern DL_IMPORT(PyObject *) PyExc_UnboundLocalError; extern DL_IMPORT(PyObject *) PyExc_ValueError; extern DL_IMPORT(PyObject *) PyExc_ZeroDivisionError; =20 - extern DL_IMPORT(PyObject *) PyExc_MemoryErrorInst; =20 =20 /* Convenience functions */ =20 --- 81,92 ---- extern DL_IMPORT(PyObject *) PyExc_UnboundLocalError; extern DL_IMPORT(PyObject *) PyExc_ValueError; extern DL_IMPORT(PyObject *) PyExc_ZeroDivisionError; + #ifdef MS_WINDOWS + extern DL_IMPORT(PyObject *) PyExc_WindowsError; + #endif =20 =20 + extern DL_IMPORT(PyObject *) PyExc_MemoryErrorInst; =20 /* Convenience functions */ =20 *************** *** 92,97 **** --- 95,104 ---- extern DL_IMPORT(PyObject *) PyErr_SetFromErrno Py_PROTO((PyObject = *)); extern DL_IMPORT(PyObject *) PyErr_SetFromErrnoWithFilename = Py_PROTO((PyObject *, char *)); extern DL_IMPORT(PyObject *) PyErr_Format Py_PROTO((PyObject *, const = char *, ...)); + #ifdef MS_WINDOWS + extern DL_IMPORT(PyObject *) PyErr_SetFromWindowsErrWithFilename(int, = const char *); + extern DL_IMPORT(PyObject *) PyErr_SetFromWindowsErr(int); + #endif =20 extern DL_IMPORT(void) PyErr_BadInternalCall Py_PROTO((void)); =20 Index: exceptions.py =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /projects/cvsroot/python/dist/src/Lib/exceptions.py,v retrieving revision 1.16 diff -c -r1.16 exceptions.py *** exceptions.py 1999/08/19 21:17:08 1.16 --- exceptions.py 2000/02/17 07:59:31 *************** *** 139,144 **** --- 139,148 ---- """OS system call failed.""" pass =20 + class WindowsError(OSError): + """MS-Windows OS system call failed.""" + pass +=20 class RuntimeError(StandardError): """Unspecified run-time error.""" pass ------=_NextPart_000_006A_01BF7979.ABA479F0-- From Fredrik Lundh" <200002162057.PAA29086@eric.cnri.reston.va.us> Message-ID: <003b01bf7946$92e647a0$34aab5d4@hagrid> [Guido] > I [propose] to rework the logic slightly, so that the NLST is only = tried > after the RETR fails with a code 550 error. Comments? looks fine to me. From skip@mojam.com (Skip Montanaro) Thu Feb 17 21:46:15 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 17 Feb 2000 15:46:15 -0600 (CST) Subject: [Patches] Re: urllib suggestion In-Reply-To: <200002162022.PAA28954@eric.cnri.reston.va.us> References: <200002082101.PAA19047@beluga.mojam.com> <200002162022.PAA28954@eric.cnri.reston.va.us> Message-ID: <14508.27687.949751.570938@beluga.mojam.com> >> An alternative is to document that the undocumented FancyURLopener >> class can be subclassed to override server_version and this class >> then used as the value of urllib._urlopener for use by urlopen and >> urlretrieve. I'm not certain which alternative is better. Both >> mechanisms require the programmer to read the code. I'll let someone >> else argue the pros and cons of the two approaches (though URLopener >> and FancyURLopener should probably be documented). Guido> I would prefer that FancyURLopener was documented anyway! Guido> How about this patch? ... Looks good to me. I'll try to send Fred some FancyURLopener documentation shortly. Skip From akuchlin@mems-exchange.org Fri Feb 18 00:35:45 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Thu, 17 Feb 2000 19:35:45 -0500 (EST) Subject: [Patches] Long ints for getitem,setitem,slices In-Reply-To: <200002151548.KAA23472@eric.cnri.reston.va.us> References: <200002151527.KAA20518@amarok.cnri.reston.va.us> <200002151548.KAA23472@eric.cnri.reston.va.us> Message-ID: <14508.37857.880073.23896@amarok.cnri.reston.va.us> Guido van Rossum writes: >If you really want to be strict, catch the OverflowError, and then >inspect the sign of the original long int to decide whether it should >mean sys.maxint or 0. (This saves time in the common case -- no >overflow.) Done. Here's a revised version of the patch, which handles overflowing long integers, and also includes a patch to the test suite that exercises the changes, though only for lists. (Guess I should look into implementing this for JPython now...) --amk Index: Python//ceval.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Python/ceval.c,v retrieving revision 2.166 diff -C2 -r2.166 ceval.c *** ceval.c 2000/01/12 22:45:54 2.166 --- ceval.c 2000/02/17 23:42:47 *************** *** 2538,2547 **** if (v != NULL) { long x; ! if (!PyInt_Check(v)) { PyErr_SetString(PyExc_TypeError, "slice index must be int"); return -1; } - x = PyInt_AsLong(v); /* Truncate -- very long indices are truncated anyway */ if (x > INT_MAX) --- 2538,2579 ---- if (v != NULL) { long x; ! if (PyInt_Check(v)) { ! x = PyInt_AsLong(v); ! } else if (PyLong_Check(v)) { ! x = PyLong_AsLong(v); ! if (x==-1 && PyErr_Occurred()) { ! PyObject *long_zero; ! ! if (!PyErr_ExceptionMatches( PyExc_OverflowError ) ) { ! /* It's not an overflow error, so just ! signal an error */ ! return -1; ! } ! ! /* It's an overflow error, so we need to ! check the sign of the long integer, ! set the value to INT_MAX or 0, and clear ! the error. */ ! ! /* Create a long integer with a value of 0 */ ! long_zero = PyLong_FromLong( 0L ); ! if (long_zero == NULL) return -1; ! ! /* Check sign */ ! if (PyObject_Compare(long_zero, v) < 0) ! x = INT_MAX; ! else ! x = 0; ! ! /* Free the long integer we created, and clear the ! OverflowError */ ! Py_DECREF(long_zero); ! PyErr_Clear(); ! } ! } else { PyErr_SetString(PyExc_TypeError, "slice index must be int"); return -1; } /* Truncate -- very long indices are truncated anyway */ if (x > INT_MAX) Index: Objects//abstract.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Objects/abstract.c,v retrieving revision 2.29 diff -C2 -r2.29 abstract.c *** abstract.c 2000/02/14 22:22:04 2.29 --- abstract.c 2000/02/15 15:14:27 *************** *** 123,126 **** --- 123,132 ---- if (PyInt_Check(key)) return PySequence_GetItem(o, PyInt_AsLong(key)); + else if (PyLong_Check(key)) { + long key_value = PyLong_AsLong(key); + if (key_value == -1 && PyErr_Occurred()) + return NULL; + return PySequence_GetItem(o, key_value); + } return type_error("sequence index must be integer"); } *************** *** 148,151 **** --- 154,163 ---- if (PyInt_Check(key)) return PySequence_SetItem(o, PyInt_AsLong(key), value); + else if (PyLong_Check(key)) { + long key_value = PyLong_AsLong(key); + if (key_value == -1 && PyErr_Occurred()) + return -1; + return PySequence_SetItem(o, key_value, value); + } type_error("sequence index must be integer"); return -1; *************** *** 174,177 **** --- 186,195 ---- if (PyInt_Check(key)) return PySequence_DelItem(o, PyInt_AsLong(key)); + else if (PyLong_Check(key)) { + long key_value = PyLong_AsLong(key); + if (key_value == -1 && PyErr_Occurred()) + return -1; + return PySequence_DelItem(o, key_value); + } type_error("sequence index must be integer"); return -1; *************** *** 392,396 **** else if (PyLong_Check(w)) { mul_value = PyLong_AsLong(w); ! if (PyErr_Occurred()) return NULL; } --- 410,414 ---- else if (PyLong_Check(w)) { mul_value = PyLong_AsLong(w); ! if (mul_value == -1 && PyErr_Occurred()) return NULL; } Index: Lib/test/test_types.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/test/test_types.py,v retrieving revision 1.12 diff -C2 -r1.12 test_types.py *** test_types.py 1998/07/16 15:29:06 1.12 --- test_types.py 2000/02/18 00:18:08 *************** *** 131,135 **** --- 131,137 ---- if [1,2]+[3,4] <> [1,2,3,4]: raise TestFailed, 'list concatenation' if [1,2]*3 <> [1,2,1,2,1,2]: raise TestFailed, 'list repetition *3' + if [1,2]*3L <> [1,2,1,2,1,2]: raise TestFailed, 'list repetition *3L' if 0*[1,2,3] <> []: raise TestFailed, 'list repetition 0*' + if 0L*[1,2,3] <> []: raise TestFailed, 'list repetition 0L*' if min([1,2]) <> 1 or max([1,2]) <> 2: raise TestFailed, 'min/max list' if 0 in [0,1,2] and 1 in [0,1,2] and 2 in [0,1,2] and 3 not in [0,1,2]: pass *************** *** 151,158 **** --- 153,167 ---- print '6.5.3a Additional list operations' a = [0,1,2,3,4] + a[0L] = 1 + a[1L] = 2 + a[2L] = 3 + if a <> [1,2,3,3,4]: raise TestFailed, 'list item assignment [0L], [1L], [2L]' a[0] = 5 a[1] = 6 a[2] = 7 if a <> [5,6,7,3,4]: raise TestFailed, 'list item assignment [0], [1], [2]' + a[-2L] = 88 + a[-1L] = 99 + if a <> [5,6,7,88,99]: raise TestFailed, 'list item assignment [-2L], [-1L]' a[-2] = 8 a[-1] = 9 *************** *** 162,165 **** --- 171,176 ---- a[1:1] = [1,2,3] if a <> [0,1,2,3,4]: raise TestFailed, 'list slice assignment' + a[ 1L : 4L] = [7,8,9] + if a <> [0,7,8,9,4]: raise TestFailed, 'list slice assignment using long ints' del a[1:4] if a <> [0,4]: raise TestFailed, 'list slice deletion' *************** *** 168,171 **** --- 179,189 ---- del a[-1] if a <> []: raise TestFailed, 'list item deletion [-1]' + a=range(0,5) + del a[1L:4L] + if a <> [0,4]: raise TestFailed, 'list slice deletion' + del a[0L] + if a <> [4]: raise TestFailed, 'list item deletion [0]' + del a[-1L] + if a <> []: raise TestFailed, 'list item deletion [-1]' a.append(0) a.append(1) *************** *** 192,195 **** --- 210,220 ---- z = range(12) z.sort(myComparison) + + # Test extreme cases with long ints + a = [0,1,2,3,4] + if a[ -pow(2,128L): 3 ] != [0,1,2]: + raise TestFailed, "list slicing with too-small long integer" + if a[ 3: pow(2,145L) ] != [3,4]: + raise TestFailed, "list slicing with too-large long integer" print '6.6 Mappings == Dictionaries' From Vladimir.Marangozov@inrialpes.fr Fri Feb 18 18:00:51 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 18 Feb 2000 19:00:51 +0100 (CET) Subject: [Patches] pcre object finalization Message-ID: <200002181800.TAA24501@python.inrialpes.fr> A long standing glitch in the pcre object finalization code. The buffers self->regex and self->regex_extra are allocated in pcre_compile() and pcre_study() via pcre_malloc, but are released via free() instead of pcre_free. (This caused us, Neil Schemenauer & me & possibly other enthousiasts willing to experiment with proprietary mallocs, troubles to run test_re, IDLE & other code using regexps). The fix is immediate, as soon as we realize the buglet, and makes all problems with the pcre code go away. (I suspected this code for two years but have never persisted enough to track down the problem.) -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- diff -cr Python-1.5.2/Modules/pcremodule.c Python-1.5.2_PyMem/Modules/pcremodule.c *** Python-1.5.2/Modules/pcremodule.c Fri Feb 11 13:34:57 2000 --- Python-1.5.2_PyMem/Modules/pcremodule.c Sun Feb 13 15:27:03 2000 *************** *** 93,102 **** PyPcre_dealloc(self) PcreObject *self; { ! if (self->regex) free(self->regex); ! if (self->regex_extra) free(self->regex_extra); ! self->regex=NULL; ! self->regex_extra=NULL; PyMem_DEL(self); } --- 93,100 ---- PyPcre_dealloc(self) PcreObject *self; { ! if (self->regex) (pcre_free)(self->regex); ! if (self->regex_extra) (pcre_free)(self->regex_extra); PyMem_DEL(self); } From akuchlin@mems-exchange.org Fri Feb 18 18:19:33 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Fri, 18 Feb 2000 13:19:33 -0500 (EST) Subject: [Patches] pcre object finalization In-Reply-To: <200002181800.TAA24501@python.inrialpes.fr> References: <200002181800.TAA24501@python.inrialpes.fr> Message-ID: <14509.36149.921024.418554@amarok.cnri.reston.va.us> Vladimir Marangozov writes: >The buffers self->regex and self->regex_extra are allocated in >pcre_compile() and pcre_study() via pcre_malloc, but are released >via free() instead of pcre_free. This analysis is correct, and the patch looks fine to me; I'd approve adding it. --amk From fdrake@acm.org Fri Feb 18 18:22:18 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 18 Feb 2000 13:22:18 -0500 (EST) Subject: [Patches] pcre object finalization In-Reply-To: <14509.36149.921024.418554@amarok.cnri.reston.va.us> References: <200002181800.TAA24501@python.inrialpes.fr> <14509.36149.921024.418554@amarok.cnri.reston.va.us> Message-ID: <14509.36314.598095.164732@weyr.cnri.reston.va.us> Andrew M. Kuchling writes: > This analysis is correct, and the patch looks fine to me; I'd approve > adding it. Check it in! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Vladimir.Marangozov@inrialpes.fr Fri Feb 18 19:04:36 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 18 Feb 2000 20:04:36 +0100 (CET) Subject: [Patches] pcre stack finalization Message-ID: <200002181904.UAA28338@python.inrialpes.fr> The same problem (mixed mallocs) exists for the pcre stack. The buffers md->... are allocated via PyMem_RESIZE in grow_stack(), while in free_stack() they are released with free() instead of PyMem_DEL. This patch fixes it. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- diff -cr Python-1.5.2/Modules/pypcre.c Python-1.5.2_PyMem/Modules/pypcre.c *** Python-1.5.2/Modules/pypcre.c Fri Feb 11 13:34:57 2000 --- Python-1.5.2_PyMem/Modules/pypcre.c Sun Feb 13 15:23:47 2000 *************** *** 3057,3068 **** static int free_stack(match_data *md) { /* Free any stack space that was allocated by the call to match(). */ ! if (md->off_num) free(md->off_num); ! if (md->offset_top) free(md->offset_top); ! if (md->r1) free(md->r1); ! if (md->r2) free(md->r2); ! if (md->eptr) free((char *)md->eptr); ! if (md->ecode) free((char *)md->ecode); return 0; } --- 3058,3069 ---- static int free_stack(match_data *md) { /* Free any stack space that was allocated by the call to match(). */ ! if (md->off_num) PyMem_DEL(md->off_num); ! if (md->offset_top) PyMem_DEL(md->offset_top); ! if (md->r1) PyMem_DEL(md->r1); ! if (md->r2) PyMem_DEL(md->r2); ! if (md->eptr) PyMem_DEL((char *)md->eptr); ! if (md->ecode) PyMem_DEL((char *)md->ecode); return 0; } From Moshe Zadka Fri Feb 18 20:36:00 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 18 Feb 2000 22:36:00 +0200 (IST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. ---559023410-1804928587-950906160=:6879 Content-Type: TEXT/PLAIN; charset=US-ASCII I've re-written the in overloading patch to comply with Guido's comments. The only comment I haven't taken care of is to remove the special casing for strings. This could be done in a seperate patch, and I don't want to burden this patch with too much. Please send me any comments! Icky lawerism: -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. ---559023410-1804928587-950906160=:6879 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="in_diff.txt" Content-ID: Content-Description: Content-Disposition: attachment; filename="in_diff.txt" Content-Transfer-Encoding: BASE64 ZGlmZiAtYyAtciBweXRob24vZGlzdC9zcmMvSW5jbHVkZS9vYmplY3QuaCBi dWlsZC9kaXN0L3NyYy9JbmNsdWRlL29iamVjdC5oDQoqKiogcHl0aG9uL2Rp c3Qvc3JjL0luY2x1ZGUvb2JqZWN0LmgJV2VkIEZlYiAgMiAxNjowMjo1MSAy MDAwDQotLS0gYnVpbGQvZGlzdC9zcmMvSW5jbHVkZS9vYmplY3QuaAlGcmkg RmViIDE4IDA5OjI2OjEyIDIwMDANCioqKioqKioqKioqKioqKg0KKioqIDE1 MCwxNTUgKioqKg0KLS0tIDE1MCwxNTYgLS0tLQ0KICB0eXBlZGVmIGludCAo KmdldHdyaXRlYnVmZmVycHJvYykgUHlfUFJPVE8oKFB5T2JqZWN0ICosIGlu dCwgdm9pZCAqKikpOw0KICB0eXBlZGVmIGludCAoKmdldHNlZ2NvdW50cHJv YykgUHlfUFJPVE8oKFB5T2JqZWN0ICosIGludCAqKSk7DQogIHR5cGVkZWYg aW50ICgqZ2V0Y2hhcmJ1ZmZlcnByb2MpIFB5X1BST1RPKChQeU9iamVjdCAq LCBpbnQsIGNvbnN0IGNoYXIgKiopKTsNCisgdHlwZWRlZiBpbnQgKCpvYmpv Ympwcm9jKSBQeV9QUk9UTygoUHlPYmplY3QgKiwgUHlPYmplY3QgKikpOw0K ICANCiAgdHlwZWRlZiBzdHJ1Y3Qgew0KICAJYmluYXJ5ZnVuYyBuYl9hZGQ7 DQoqKioqKioqKioqKioqKioNCioqKiAxODUsMTkwICoqKioNCi0tLSAxODYs MTkyIC0tLS0NCiAgCWludGludGFyZ2Z1bmMgc3Ffc2xpY2U7DQogIAlpbnRv Ymphcmdwcm9jIHNxX2Fzc19pdGVtOw0KICAJaW50aW50b2JqYXJncHJvYyBz cV9hc3Nfc2xpY2U7DQorICAgICAgICAgb2Jqb2JqcHJvYyBzcV9jb250YWlu czsNCiAgfSBQeVNlcXVlbmNlTWV0aG9kczsNCiAgDQogIHR5cGVkZWYgc3Ry dWN0IHsNCioqKioqKioqKioqKioqKg0KKioqIDMxNywzMjMgKioqKg0KICAv KiBQeUJ1ZmZlclByb2NzIGNvbnRhaW5zIGJmX2dldGNoYXJidWZmZXIgKi8N CiAgI2RlZmluZSBQeV9UUEZMQUdTX0hBVkVfR0VUQ0hBUkJVRkZFUiAgKDFM PDwwKQ0KICANCiEgI2RlZmluZSBQeV9UUEZMQUdTX0RFRkFVTFQgIChQeV9U UEZMQUdTX0hBVkVfR0VUQ0hBUkJVRkZFUikNCiAgDQogICNkZWZpbmUgUHlU eXBlX0hhc0ZlYXR1cmUodCxmKSAgKCgodCktPnRwX2ZsYWdzICYgKGYpKSAh PSAwKQ0KICANCi0tLSAzMTksMzI5IC0tLS0NCiAgLyogUHlCdWZmZXJQcm9j cyBjb250YWlucyBiZl9nZXRjaGFyYnVmZmVyICovDQogICNkZWZpbmUgUHlf VFBGTEFHU19IQVZFX0dFVENIQVJCVUZGRVIgICgxTDw8MCkNCiAgDQohIC8q IFB5U2VxdWVuY2VNZXRob2RzIGNvbnRhaW5zIHNxX2NvbnRhaW5zICovDQoh ICNkZWZpbmUgUHlfVFBGTEFHU19IQVZFX1NFUVVFTkNFX0lOICgxTDw8MSkN CiEgDQohICNkZWZpbmUgUHlfVFBGTEFHU19ERUZBVUxUICAoUHlfVFBGTEFH U19IQVZFX0dFVENIQVJCVUZGRVIgfCBcDQohICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgUHlfVFBGTEFHU19IQVZFX1NFUVVFTkNFX0lOKQ0KICAN CiAgI2RlZmluZSBQeVR5cGVfSGFzRmVhdHVyZSh0LGYpICAoKCh0KS0+dHBf ZmxhZ3MgJiAoZikpICE9IDApDQogIA0KZGlmZiAtYyAtciBweXRob24vZGlz dC9zcmMvT2JqZWN0cy9hYnN0cmFjdC5jIGJ1aWxkL2Rpc3Qvc3JjL09iamVj dHMvYWJzdHJhY3QuYw0KKioqIHB5dGhvbi9kaXN0L3NyYy9PYmplY3RzL2Fi c3RyYWN0LmMJRnJpIEZlYiAxOCAwOTowNjo0NSAyMDAwDQotLS0gYnVpbGQv ZGlzdC9zcmMvT2JqZWN0cy9hYnN0cmFjdC5jCUZyaSBGZWIgMTggMDk6MzA6 NDQgMjAwMA0KKioqKioqKioqKioqKioqDQoqKiogMTEyMSwxMTI3ICoqKioN CiAgCQl9DQogIAkJcmV0dXJuIDA7DQogIAl9DQohIA0KICAJc3EgPSB3LT5v Yl90eXBlLT50cF9hc19zZXF1ZW5jZTsNCiAgCWlmIChzcSA9PSBOVUxMIHx8 IHNxLT5zcV9pdGVtID09IE5VTEwpIHsNCiAgCQlQeUVycl9TZXRTdHJpbmco UHlFeGNfVHlwZUVycm9yLA0KLS0tIDExMjEsMTEzNSAtLS0tDQogIAkJfQ0K ICAJCXJldHVybiAwOw0KICAJfQ0KISAJaWYoUHlUeXBlX0hhc0ZlYXR1cmUo dy0+b2JfdHlwZSwgUHlfVFBGTEFHU19IQVZFX1NFUVVFTkNFX0lOKSkgew0K ISAJCXNxID0gdy0+b2JfdHlwZS0+dHBfYXNfc2VxdWVuY2U7DQohIAkgICAg ICAgIGlmKHNxICE9IE5VTEwpDQohIAkJCWlmKHNxLT5zcV9jb250YWlucyAh PSBOVUxMKQ0KISAJCQkJcmV0dXJuICgqc3EtPnNxX2NvbnRhaW5zKSh3LCB2 KTsNCiEgCX0NCiEgCQ0KISAJLyogSWYgdGhlcmUgaXMgbm8gYmV0dGVyIHdh eSB0byBjaGVjayB3aGV0aGVyIGFuIGl0ZW0gaXMgaXMgY29udGFpbmVkLA0K ISAJICAgZG8gaXQgdGhlIGhhcmQgd2F5ICovDQogIAlzcSA9IHctPm9iX3R5 cGUtPnRwX2FzX3NlcXVlbmNlOw0KICAJaWYgKHNxID09IE5VTEwgfHwgc3Et PnNxX2l0ZW0gPT0gTlVMTCkgew0KICAJCVB5RXJyX1NldFN0cmluZyhQeUV4 Y19UeXBlRXJyb3IsDQpkaWZmIC1jIC1yIHB5dGhvbi9kaXN0L3NyYy9PYmpl Y3RzL2NsYXNzb2JqZWN0LmMgYnVpbGQvZGlzdC9zcmMvT2JqZWN0cy9jbGFz c29iamVjdC5jDQoqKiogcHl0aG9uL2Rpc3Qvc3JjL09iamVjdHMvY2xhc3Nv YmplY3QuYwlGcmkgRmViIDE4IDA5OjA2OjQ5IDIwMDANCi0tLSBidWlsZC9k aXN0L3NyYy9PYmplY3RzL2NsYXNzb2JqZWN0LmMJRnJpIEZlYiAxOCAxMDow NjozMCAyMDAwDQoqKioqKioqKioqKioqKioNCioqKiAxMDY1LDEwNzAgKioq Kg0KLS0tIDEwNjUsMTA5OSAtLS0tDQogIAlyZXR1cm4gMDsNCiAgfQ0KICAN Cisgc3RhdGljIGludCBpbnN0YW5jZV9jb250YWlucyhQeUluc3RhbmNlT2Jq ZWN0ICppbnN0LCBQeU9iamVjdCAqbWVtYmVyKQ0KKyB7DQorIAlzdGF0aWMg UHlPYmplY3QgKl9fY29udGFpbnNfXzsNCisgCVB5T2JqZWN0ICpmdW5jLCAq YXJnLCAqcmVzOw0KKyAJaW50IHJldDsNCisgDQorIAlpZihfX2NvbnRhaW5z X18gPT0gTlVMTCkgew0KKyAJCV9fY29udGFpbnNfXyA9IFB5U3RyaW5nX0lu dGVybkZyb21TdHJpbmcoIl9fY29udGFpbnNfXyIpOw0KKyAJCWlmKF9fY29u dGFpbnNfXyA9PSBOVUxMKQ0KKyAJCQlyZXR1cm4gLTE7DQorIAl9DQorIAlm dW5jID0gaW5zdGFuY2VfZ2V0YXR0cihpbnN0LCBfX2NvbnRhaW5zX18pOw0K KyAJaWYoZnVuYyA9PSBOVUxMKQ0KKyAJCXJldHVybiAtMTsNCisgCWFyZyA9 IFB5X0J1aWxkVmFsdWUoIihPKSIsIG1lbWJlcik7DQorIAlpZihhcmcgPT0g TlVMTCkgew0KKyAJCVB5X0RFQ1JFRihmdW5jKTsNCisgCQlyZXR1cm4gLTE7 DQorIAl9DQorIAlyZXMgPSBQeUV2YWxfQ2FsbE9iamVjdChmdW5jLCBhcmcp Ow0KKyAJUHlfREVDUkVGKGZ1bmMpOw0KKyAJUHlfREVDUkVGKGFyZyk7DQor IAlpZihyZXMgPT0gTlVMTCkgDQorIAkJcmV0dXJuIC0xOw0KKyAJcmV0ID0g UHlPYmplY3RfSXNUcnVlKHJlcyk7DQorIAlQeV9ERUNSRUYocmVzKTsNCisg CXJldHVybiByZXQ7DQorIH0NCisgDQogIHN0YXRpYyBQeVNlcXVlbmNlTWV0 aG9kcyBpbnN0YW5jZV9hc19zZXF1ZW5jZSA9IHsNCiAgCShpbnF1aXJ5KWlu c3RhbmNlX2xlbmd0aCwgLypzcV9sZW5ndGgqLw0KICAJMCwgLypzcV9jb25j YXQqLw0KKioqKioqKioqKioqKioqDQoqKiogMTA3MywxMDc4ICoqKioNCi0t LSAxMTAyLDExMDggLS0tLQ0KICAJKGludGludGFyZ2Z1bmMpaW5zdGFuY2Vf c2xpY2UsIC8qc3Ffc2xpY2UqLw0KICAJKGludG9iamFyZ3Byb2MpaW5zdGFu Y2VfYXNzX2l0ZW0sIC8qc3FfYXNzX2l0ZW0qLw0KICAJKGludGludG9iamFy Z3Byb2MpaW5zdGFuY2VfYXNzX3NsaWNlLCAvKnNxX2Fzc19zbGljZSovDQor IAkob2Jqb2JqcHJvYylpbnN0YW5jZV9jb250YWlucywgLyogc3FfY29udGFp bnMgKi8NCiAgfTsNCiAgDQogIHN0YXRpYyBQeU9iamVjdCAqDQoqKioqKioq KioqKioqKioNCioqKiAxNDA1LDE0MTAgKioqKg0KLS0tIDE0MzUsMTQ0MiAt LS0tDQogIAkwLAkJCS8qdHBfc3RyKi8NCiAgCShnZXRhdHRyb2Z1bmMpaW5z dGFuY2VfZ2V0YXR0ciwgLyp0cF9nZXRhdHRybyovDQogIAkoc2V0YXR0cm9m dW5jKWluc3RhbmNlX3NldGF0dHIsIC8qdHBfc2V0YXR0cm8qLw0KKyAgICAg ICAgIDAsIC8qIHRwX2FzX2J1ZmZlciAqLw0KKyAJUHlfVFBGTEFHU19IQVZF X1NFUVVFTkNFX0lOLCAvKnRwX2ZsYWdzICovDQogIH07DQogIA0KICANCg== ---559023410-1804928587-950906160=:6879-- From trentm@ActiveState.com Sat Feb 19 09:49:37 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 19 Feb 2000 09:49:37 -0000 Subject: [Patches] FW: array.buffer_info() on LLP64 system Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_0011_01BF7ABE.A2568FA0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit ***** background: As was mentioned in the "64-bit port" thread on python-dev, the buffer_info() function in the array module improperly presumes that sizeof(long)==sizeof(void*) which is not true on LLP64 system (i.e. WIN64). This patch implements the fix as suggested by Greg Stein: http://www.python.org/pipermail/python-dev/2000-February/003624.html Aside: Does the test suite crash on the Win32 debug build of the latest CVS code for anyone else? Is does for me. I get an "python_d.exe - Application Error". It fails on the 're' test: [preceeding tests...] test_re Fatal Python error: UNREF invalid object ***** log message: Correct sizeof(void*)==sizeof(long) assumption in array.buffer_info() implementation. ***** disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Trent Mick trentm@ActiveState.com ------=_NextPart_000_0011_01BF7ABE.A2568FA0 Content-Type: text/plain; name="bufferinfo.patch.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="bufferinfo.patch.txt" *** ./dist/src/Modules/arraymodule.c Fri Feb 11 17:42:52 2000 --- ../../Apps/Perlium/Python/dist/src/Modules/arraymodule.c Wed Feb 16 = 15:21:50 2000 *************** *** 724,731 **** arrayobject *self; PyObject *args; { ! return Py_BuildValue("ll", ! (long)(self->ob_item), (long)(self->ob_size)); } =20 static char buffer_info_doc [] =3D --- 724,736 ---- arrayobject *self; PyObject *args; { ! PyObject* retval =3D PyTuple_New(2); ! if (!retval) return NULL; !=20 ! PyTuple_SET_ITEM(retval, 0, PyLong_FromVoidPtr(self->ob_item)); ! PyTuple_SET_ITEM(retval, 1, PyInt_FromLong((long)(self->ob_size))); !=20 ! return retval; } =20 static char buffer_info_doc [] =3D ------=_NextPart_000_0011_01BF7ABE.A2568FA0-- From trentm@ActiveState.com Sat Feb 19 09:49:37 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 19 Feb 2000 09:49:37 -0000 Subject: [Patches] FW: win64 configuration Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_000E_01BF7ABE.A232DB00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit ***** log message: Add WIN64 configuration information (guarded via _WIN64). Specialized COMPILER, PLATFORM, and SIZEOF_VOID_P. ***** disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Thanks, Trent Trent Mick trentm@ActiveState.com ------=_NextPart_000_000E_01BF7ABE.A232DB00 Content-Type: text/plain; name="win64config.patch.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="win64config.patch.txt" *** dist/src/PC/config.h Fri Feb 11 17:42:59 2000 --- ../../Apps/Perlium/Python/dist/src/PC/config.h Wed Feb 16 16:10:57 2000 *************** *** 60,66 **** #endif /* !MS_NO_COREDLL */ #ifdef _M_IX86 ! #define COMPILER "[MSC 32 bit (Intel)]" #elif defined(_M_ALPHA) #define COMPILER "[MSC 32 bit (Alpha)]" #else --- 60,70 ---- #endif /* !MS_NO_COREDLL */ #ifdef _M_IX86 ! # if defined(_WIN64) ! # define COMPILER "[MSC 64 bit (Intel)]" ! # else ! # define COMPILER "[MSC 32 bit (Intel)]" ! # endif #elif defined(_M_ALPHA) #define COMPILER "[MSC 32 bit (Alpha)]" #else *************** *** 201,207 **** /* End of compilers - finish up */ #ifdef MS_WIN32 ! #define PLATFORM "win32" #else #ifdef MS_WIN16 #define PLATFORM "win16" --- 205,215 ---- /* End of compilers - finish up */ #ifdef MS_WIN32 ! # ifdef _WIN64 ! # define PLATFORM "win64" ! # else ! # define PLATFORM "win32" ! # endif #else #ifdef MS_WIN16 #define PLATFORM "win16" *************** *** 230,235 **** --- 238,245 ---- #define SIZEOF_LONG_LONG 8 #ifdef _M_ALPHA + #define SIZEOF_VOID_P 8 + #elif defined(_WIN64) #define SIZEOF_VOID_P 8 #else #define SIZEOF_VOID_P 4 ------=_NextPart_000_000E_01BF7ABE.A232DB00-- From trentm@ActiveState.com Fri Feb 18 17:54:52 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 18 Feb 2000 17:54:52 -0000 Subject: [Patches] win64 configuration Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_0013_01BF7A39.41F4F3C0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit ***** log message: Add WIN64 configuration information (guarded via _WIN64). Specialized COMPILER, PLATFORM, and SIZEOF_VOID_P. ***** disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Thanks, Trent Trent Mick trentm@ActiveState.com ------=_NextPart_000_0013_01BF7A39.41F4F3C0 Content-Type: text/plain; name="win64config.patch.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="win64config.patch.txt" *** dist/src/PC/config.h Fri Feb 11 17:42:59 2000 --- ../../Apps/Perlium/Python/dist/src/PC/config.h Wed Feb 16 16:10:57 2000 *************** *** 60,66 **** #endif /* !MS_NO_COREDLL */ #ifdef _M_IX86 ! #define COMPILER "[MSC 32 bit (Intel)]" #elif defined(_M_ALPHA) #define COMPILER "[MSC 32 bit (Alpha)]" #else --- 60,70 ---- #endif /* !MS_NO_COREDLL */ #ifdef _M_IX86 ! # if defined(_WIN64) ! # define COMPILER "[MSC 64 bit (Intel)]" ! # else ! # define COMPILER "[MSC 32 bit (Intel)]" ! # endif #elif defined(_M_ALPHA) #define COMPILER "[MSC 32 bit (Alpha)]" #else *************** *** 201,207 **** /* End of compilers - finish up */ #ifdef MS_WIN32 ! #define PLATFORM "win32" #else #ifdef MS_WIN16 #define PLATFORM "win16" --- 205,215 ---- /* End of compilers - finish up */ #ifdef MS_WIN32 ! # ifdef _WIN64 ! # define PLATFORM "win64" ! # else ! # define PLATFORM "win32" ! # endif #else #ifdef MS_WIN16 #define PLATFORM "win16" *************** *** 230,235 **** --- 238,245 ---- #define SIZEOF_LONG_LONG 8 #ifdef _M_ALPHA + #define SIZEOF_VOID_P 8 + #elif defined(_WIN64) #define SIZEOF_VOID_P 8 #else #define SIZEOF_VOID_P 4 ------=_NextPart_000_0013_01BF7A39.41F4F3C0-- From trentm@ActiveState.com Fri Feb 18 17:16:13 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 18 Feb 2000 17:16:13 -0000 Subject: [Patches] array.buffer_info() on LLP64 system Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_0010_01BF7A33.DB473160 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit ***** background: As was mentioned in the "64-bit port" thread on python-dev, the buffer_info() function in the array module improperly presumes that sizeof(long)==sizeof(void*) which is not true on LLP64 system (i.e. WIN64). This patch implements the fix as suggested by Greg Stein: http://www.python.org/pipermail/python-dev/2000-February/003624.html Aside: Does the test suite crash on the Win32 debug build of the latest CVS code for anyone else? Is does for me. I get an "python_d.exe - Application Error". It fails on the 're' test: [preceeding tests...] test_re Fatal Python error: UNREF invalid object ***** log message: Correct sizeof(void*)==sizeof(long) assumption in array.buffer_info() implementation. ***** disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Trent Mick trentm@ActiveState.com ------=_NextPart_000_0010_01BF7A33.DB473160 Content-Type: text/plain; name="bufferinfo.patch.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="bufferinfo.patch.txt" *** ./dist/src/Modules/arraymodule.c Fri Feb 11 17:42:52 2000 --- ../../Apps/Perlium/Python/dist/src/Modules/arraymodule.c Wed Feb 16 = 15:21:50 2000 *************** *** 724,731 **** arrayobject *self; PyObject *args; { ! return Py_BuildValue("ll", ! (long)(self->ob_item), (long)(self->ob_size)); } =20 static char buffer_info_doc [] =3D --- 724,736 ---- arrayobject *self; PyObject *args; { ! PyObject* retval =3D PyTuple_New(2); ! if (!retval) return NULL; !=20 ! PyTuple_SET_ITEM(retval, 0, PyLong_FromVoidPtr(self->ob_item)); ! PyTuple_SET_ITEM(retval, 1, PyInt_FromLong((long)(self->ob_size))); !=20 ! return retval; } =20 static char buffer_info_doc [] =3D ------=_NextPart_000_0010_01BF7A33.DB473160-- From Moshe Zadka Sat Feb 19 13:09:45 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 19 Feb 2000 15:09:45 +0200 (IST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: OK, here's another version: (Same checkin message from last e-mail, the only change is a bug fix) -- diff -c -r python/dist/src/Include/object.h build/dist/src/Include/object.h *** python/dist/src/Include/object.h Wed Feb 2 16:02:51 2000 --- build/dist/src/Include/object.h Fri Feb 18 18:46:46 2000 *************** *** 150,155 **** --- 150,156 ---- typedef int (*getwritebufferproc) Py_PROTO((PyObject *, int, void **)); typedef int (*getsegcountproc) Py_PROTO((PyObject *, int *)); typedef int (*getcharbufferproc) Py_PROTO((PyObject *, int, const char **)); + typedef int (*objobjproc) Py_PROTO((PyObject *, PyObject *)); typedef struct { binaryfunc nb_add; *************** *** 185,190 **** --- 186,192 ---- intintargfunc sq_slice; intobjargproc sq_ass_item; intintobjargproc sq_ass_slice; + objobjproc sq_contains; } PySequenceMethods; typedef struct { *************** *** 317,323 **** /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) --- 319,329 ---- /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! /* PySequenceMethods contains sq_contains */ ! #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) ! ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER | \ ! Py_TPFLAGS_HAVE_SEQUENCE_IN) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) diff -c -r python/dist/src/Objects/abstract.c build/dist/src/Objects/abstract.c *** python/dist/src/Objects/abstract.c Fri Feb 18 09:06:45 2000 --- build/dist/src/Objects/abstract.c Fri Feb 18 18:47:23 2000 *************** *** 1121,1127 **** } return 0; } ! sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, --- 1121,1134 ---- } return 0; } ! if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { ! sq = w->ob_type->tp_as_sequence; ! if(sq != NULL && sq->sq_contains != NULL) ! return (*sq->sq_contains)(w, v); ! } ! ! /* If there is no better way to check whether an item is is contained, ! do it the hard way */ sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, diff -c -r python/dist/src/Objects/classobject.c build/dist/src/Objects/classobject.c *** python/dist/src/Objects/classobject.c Fri Feb 18 09:06:49 2000 --- build/dist/src/Objects/classobject.c Sat Feb 19 03:17:46 2000 *************** *** 1065,1070 **** --- 1065,1123 ---- return 0; } + static int instance_contains(PyInstanceObject *inst, PyObject *member) + { + static PyObject *__contains__; + PyObject *func, *arg, *res; + int ret; + + if(__contains__ == NULL) { + __contains__ = PyString_InternFromString("__contains__"); + if(__contains__ == NULL) + return -1; + } + func = instance_getattr(inst, __contains__); + if(func == NULL) { + /* fall back to previous behaviour */ + int i, cmp_res; + + if(!PyErr_ExceptionMatches(PyExc_AttributeError)) + return -1; + PyErr_Clear(); + for(i=0;;i++) { + PyObject *obj = instance_item(inst, i); + int ret = 0; + + if(obj == NULL) { + if(!PyErr_ExceptionMatches(PyExc_IndexError)) + return -1; + PyErr_Clear(); + return 0; + } + if(PyObject_Cmp(obj, member, &cmp_res) == -1) + ret = -1; + if(cmp_res == 0) + ret = 1; + Py_DECREF(obj); + if(ret) + return ret; + } + } + arg = Py_BuildValue("(O)", member); + if(arg == NULL) { + Py_DECREF(func); + return -1; + } + res = PyEval_CallObject(func, arg); + Py_DECREF(func); + Py_DECREF(arg); + if(res == NULL) + return -1; + ret = PyObject_IsTrue(res); + Py_DECREF(res); + return ret; + } + static PySequenceMethods instance_as_sequence = { (inquiry)instance_length, /*sq_length*/ 0, /*sq_concat*/ *************** *** 1073,1078 **** --- 1126,1132 ---- (intintargfunc)instance_slice, /*sq_slice*/ (intobjargproc)instance_ass_item, /*sq_ass_item*/ (intintobjargproc)instance_ass_slice, /*sq_ass_slice*/ + (objobjproc)instance_contains, /* sq_contains */ }; static PyObject * *************** *** 1405,1410 **** --- 1459,1466 ---- 0, /*tp_str*/ (getattrofunc)instance_getattr, /*tp_getattro*/ (setattrofunc)instance_setattr, /*tp_setattro*/ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /*tp_flags */ }; --- -- Legalese: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From Moshe Zadka Sat Feb 19 11:59:35 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 19 Feb 2000 13:59:35 +0200 (IST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: (Greg, a favour: my connection to Python.Org seems flaky. If you don't see it on the list within reasonable time, please forward this to the patch list. Thanks a lot) On Sat, 19 Feb 2000, Greg Stein wrote to say, I would break a lot of code by not falling back to the old behaviour on instances. Checking message: -- Allow both built-in type and class instances to define special behaviour for the "in" operator. The default is still to go through all the elements until IndexError or one compares equal. -- So, here it is, the new & improved patch: -- diff -c -r python/dist/src/Include/object.h build/dist/src/Include/object.h *** python/dist/src/Include/object.h Wed Feb 2 16:02:51 2000 --- build/dist/src/Include/object.h Fri Feb 18 18:46:46 2000 *************** *** 150,155 **** --- 150,156 ---- typedef int (*getwritebufferproc) Py_PROTO((PyObject *, int, void **)); typedef int (*getsegcountproc) Py_PROTO((PyObject *, int *)); typedef int (*getcharbufferproc) Py_PROTO((PyObject *, int, const char **)); + typedef int (*objobjproc) Py_PROTO((PyObject *, PyObject *)); typedef struct { binaryfunc nb_add; *************** *** 185,190 **** --- 186,192 ---- intintargfunc sq_slice; intobjargproc sq_ass_item; intintobjargproc sq_ass_slice; + objobjproc sq_contains; } PySequenceMethods; typedef struct { *************** *** 317,323 **** /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) --- 319,329 ---- /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! /* PySequenceMethods contains sq_contains */ ! #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) ! ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER | \ ! Py_TPFLAGS_HAVE_SEQUENCE_IN) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) diff -c -r python/dist/src/Objects/abstract.c build/dist/src/Objects/abstract.c *** python/dist/src/Objects/abstract.c Fri Feb 18 09:06:45 2000 --- build/dist/src/Objects/abstract.c Fri Feb 18 18:47:23 2000 *************** *** 1121,1127 **** } return 0; } ! sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, --- 1121,1134 ---- } return 0; } ! if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { ! sq = w->ob_type->tp_as_sequence; ! if(sq != NULL && sq->sq_contains != NULL) ! return (*sq->sq_contains)(w, v); ! } ! ! /* If there is no better way to check whether an item is is contained, ! do it the hard way */ sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, diff -c -r python/dist/src/Objects/classobject.c build/dist/src/Objects/classobject.c *** python/dist/src/Objects/classobject.c Fri Feb 18 09:06:49 2000 --- build/dist/src/Objects/classobject.c Sat Feb 19 02:14:37 2000 *************** *** 1065,1070 **** --- 1065,1122 ---- return 0; } + static int instance_contains(PyInstanceObject *inst, PyObject *member) + { + static PyObject *__contains__; + PyObject *func, *arg, *res; + int ret; + + if(__contains__ == NULL) { + __contains__ = PyString_InternFromString("__contains__"); + if(__contains__ == NULL) + return -1; + } + func = instance_getattr(inst, __contains__); + if(func == NULL) { + /* fall back to previous behaviour */ + int i, cmp_res; + + if(!PyErr_ExceptionMatches(PyExc_AttributeError)) + return -1; + PyErr_Clear(); + for(i=0;;i++) { + PyObject *obj = instance_item(inst, i); + if(obj == NULL) { + if(!PyErr_ExceptionMatches(PyExc_IndexError)) + return -1; + PyErr_Clear(); + return 0; + } + if(PyObject_Cmp(obj, member, &cmp_res) == -1) { + Py_DECREF(obj); + return -1; + } + if(cmp_res == 0) { + Py_DECREF(obj); + return 1; + } + } + } + arg = Py_BuildValue("(O)", member); + if(arg == NULL) { + Py_DECREF(func); + return -1; + } + res = PyEval_CallObject(func, arg); + Py_DECREF(func); + Py_DECREF(arg); + if(res == NULL) + return -1; + ret = PyObject_IsTrue(res); + Py_DECREF(res); + return ret; + } + static PySequenceMethods instance_as_sequence = { (inquiry)instance_length, /*sq_length*/ 0, /*sq_concat*/ *************** *** 1073,1078 **** --- 1125,1131 ---- (intintargfunc)instance_slice, /*sq_slice*/ (intobjargproc)instance_ass_item, /*sq_ass_item*/ (intintobjargproc)instance_ass_slice, /*sq_ass_slice*/ + (objobjproc)instance_contains, /* sq_contains */ }; static PyObject * *************** *** 1405,1410 **** --- 1458,1465 ---- 0, /*tp_str*/ (getattrofunc)instance_getattr, /*tp_getattro*/ (setattrofunc)instance_setattr, /*tp_setattro*/ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /*tp_flags */ }; ---- Legalese: -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From Moshe Zadka Sat Feb 19 11:11:22 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 19 Feb 2000 13:11:22 +0200 (IST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: On Sat, 19 Feb 2000, Greg Stein wrote: > Eek! I just noticed something... if an instance does NOT define a > __contains__, then it never looks for __getitem__. In other words, it > isn't falling back to the previous behavior. Yes, I know. I'm not really sure if it's worth it, but I guess I can do that. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From Moshe Zadka Sat Feb 19 04:31:51 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 19 Feb 2000 06:31:51 +0200 (IST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: On Fri, 18 Feb 2000, Greg Stein wrote: > My mailer isn't good with attachments, so I can't directly quote :-( No problems: I'll inline it this time. > * it looks like the indentation/tabbing in object.h for sq_contains is > different than the preceding lines. (in the diff, it shows at a > different indent level) You're right, I had an extra whitespace there. Taken care of. > * in abstract.c, you've got cascading "if" statements that could more > easily be written: > > if (sq != NULL && sq->sq_contains != NULL) Right again. Changed. > * in PyInstance_Type, you should use Py_TPFLAGS_DEFAULT. the slot > structures *do* have those new slots once you compile them in the > presence of the updated object.h file. Done. Thanks a lot Greg. Here's the new and updated version: ------------------- cut here ---------------- diff -c -r python/dist/src/Include/object.h build/dist/src/Include/object.h *** python/dist/src/Include/object.h Wed Feb 2 16:02:51 2000 --- build/dist/src/Include/object.h Fri Feb 18 18:46:46 2000 *************** *** 150,155 **** --- 150,156 ---- typedef int (*getwritebufferproc) Py_PROTO((PyObject *, int, void **)); typedef int (*getsegcountproc) Py_PROTO((PyObject *, int *)); typedef int (*getcharbufferproc) Py_PROTO((PyObject *, int, const char **)); + typedef int (*objobjproc) Py_PROTO((PyObject *, PyObject *)); typedef struct { binaryfunc nb_add; *************** *** 185,190 **** --- 186,192 ---- intintargfunc sq_slice; intobjargproc sq_ass_item; intintobjargproc sq_ass_slice; + objobjproc sq_contains; } PySequenceMethods; typedef struct { *************** *** 317,323 **** /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) --- 319,329 ---- /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! /* PySequenceMethods contains sq_contains */ ! #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) ! ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER | \ ! Py_TPFLAGS_HAVE_SEQUENCE_IN) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) diff -c -r python/dist/src/Objects/abstract.c build/dist/src/Objects/abstract.c *** python/dist/src/Objects/abstract.c Fri Feb 18 09:06:45 2000 --- build/dist/src/Objects/abstract.c Fri Feb 18 18:47:23 2000 *************** *** 1121,1127 **** } return 0; } ! sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, --- 1121,1134 ---- } return 0; } ! if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { ! sq = w->ob_type->tp_as_sequence; ! if(sq != NULL && sq->sq_contains != NULL) ! return (*sq->sq_contains)(w, v); ! } ! ! /* If there is no better way to check whether an item is is contained, ! do it the hard way */ sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, diff -c -r python/dist/src/Objects/classobject.c build/dist/src/Objects/classobject.c *** python/dist/src/Objects/classobject.c Fri Feb 18 09:06:49 2000 --- build/dist/src/Objects/classobject.c Fri Feb 18 18:47:55 2000 *************** *** 1065,1070 **** --- 1065,1099 ---- return 0; } + static int instance_contains(PyInstanceObject *inst, PyObject *member) + { + static PyObject *__contains__; + PyObject *func, *arg, *res; + int ret; + + if(__contains__ == NULL) { + __contains__ = PyString_InternFromString("__contains__"); + if(__contains__ == NULL) + return -1; + } + func = instance_getattr(inst, __contains__); + if(func == NULL) + return -1; + arg = Py_BuildValue("(O)", member); + if(arg == NULL) { + Py_DECREF(func); + return -1; + } + res = PyEval_CallObject(func, arg); + Py_DECREF(func); + Py_DECREF(arg); + if(res == NULL) + return -1; + ret = PyObject_IsTrue(res); + Py_DECREF(res); + return ret; + } + static PySequenceMethods instance_as_sequence = { (inquiry)instance_length, /*sq_length*/ 0, /*sq_concat*/ *************** *** 1073,1078 **** --- 1102,1108 ---- (intintargfunc)instance_slice, /*sq_slice*/ (intobjargproc)instance_ass_item, /*sq_ass_item*/ (intintobjargproc)instance_ass_slice, /*sq_ass_slice*/ + (objobjproc)instance_contains, /* sq_contains */ }; static PyObject * *************** *** 1405,1410 **** --- 1435,1442 ---- 0, /*tp_str*/ (getattrofunc)instance_getattr, /*tp_getattro*/ (setattrofunc)instance_setattr, /*tp_setattro*/ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /*tp_flags */ }; --------------- cut here ------------------------------------ Legal: -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gstein@lyra.org Sat Feb 19 12:25:49 2000 From: gstein@lyra.org (Greg Stein) Date: Sat, 19 Feb 2000 04:25:49 -0800 (PST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: On Sat, 19 Feb 2000, Moshe Zadka wrote: > (Greg, a favour: my connection to Python.Org seems flaky. If you don't see > it on the list within reasonable time, please forward this to the patch > list. Thanks a lot) I'm seeing it, too. The python.org mail handler (dinsdale.python.org) appears to be down. This mail thread is waiting in my outbound queue for the machine to reappear :-) (I presume the same on your end) >... > ! if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { > ! sq = w->ob_type->tp_as_sequence; > ! if(sq != NULL && sq->sq_contains != NULL) > ! return (*sq->sq_contains)(w, v); > ! } > ! > ! /* If there is no better way to check whether an item is is contained, > ! do it the hard way */ > sq = w->ob_type->tp_as_sequence; > if (sq == NULL || sq->sq_item == NULL) { > PyErr_SetString(PyExc_TypeError, It would be nice to fall through that "return" statement, but I see that we aren't really able to do that. Bummer :-( Well, a bit of code replication can't hurt too bad, I guess... >... > + func = instance_getattr(inst, __contains__); > + if(func == NULL) { > + /* fall back to previous behaviour */ > + int i, cmp_res; > + > + if(!PyErr_ExceptionMatches(PyExc_AttributeError)) > + return -1; > + PyErr_Clear(); Ah! Very good catch on "eating up" the AttributeError! > + for(i=0;;i++) { > + PyObject *obj = instance_item(inst, i); > + if(obj == NULL) { > + if(!PyErr_ExceptionMatches(PyExc_IndexError)) > + return -1; > + PyErr_Clear(); > + return 0; > + } > + if(PyObject_Cmp(obj, member, &cmp_res) == -1) { The PyObject_Compare() used in abstract.c is a teeny bit faster. > + Py_DECREF(obj); > + return -1; > + } > + if(cmp_res == 0) { > + Py_DECREF(obj); > + return 1; > + } > + } obj is never DECREF'd if the "if" statements never match. The style used in abstract.c, would work well here (I'd recommend following that one). Note that abstract.c uses Py_XDECREF even though we know x != NULL... your code can simply Py_DECREF. And your direct use of instance_item() is good! Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Sat Feb 19 11:39:20 2000 From: gstein@lyra.org (Greg Stein) Date: Sat, 19 Feb 2000 03:39:20 -0800 (PST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: On Sat, 19 Feb 2000, Moshe Zadka wrote: > On Sat, 19 Feb 2000, Greg Stein wrote: > > Eek! I just noticed something... if an instance does NOT define a > > __contains__, then it never looks for __getitem__. In other words, it > > isn't falling back to the previous behavior. > > Yes, I know. I'm not really sure if it's worth it, but I guess I can do > that. You would be breaking a lot of code that depends upon the old behavior. I consider it "worth it" to prevent that from happening(!) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Sat Feb 19 09:13:18 2000 From: gstein@lyra.org (Greg Stein) Date: Sat, 19 Feb 2000 01:13:18 -0800 (PST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: Eek! I just noticed something... if an instance does NOT define a __contains__, then it never looks for __getitem__. In other words, it isn't falling back to the previous behavior. Sorry for not seeing that before you rebuilt the patch. :-( Cheers, -g On Sat, 19 Feb 2000, Moshe Zadka wrote: >... > Here's the new and updated version: >... -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Sat Feb 19 03:34:16 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 18 Feb 2000 19:34:16 -0800 (PST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in In-Reply-To: Message-ID: On Fri, 18 Feb 2000, Moshe Zadka wrote: > > I've re-written the in overloading patch to comply with Guido's comments. > The only comment I haven't taken care of is to remove the special casing > for strings. This could be done in a seperate patch, and I don't want to > burden this patch with too much. > > Please send me any comments! My mailer isn't good with attachments, so I can't directly quote :-( * it looks like the indentation/tabbing in object.h for sq_contains is different than the preceding lines. (in the diff, it shows at a different indent level) * in abstract.c, you've got cascading "if" statements that could more easily be written: if (sq != NULL && sq->sq_contains != NULL) * in PyInstance_Type, you should use Py_TPFLAGS_DEFAULT. the slot structures *do* have those new slots once you compile them in the presence of the updated object.h file. [ technically, all types in the Python distribution should be updated to include Py_TPFLAGS_DEFAULT in their type structures ] Cheers, -g -- Greg Stein, http://www.lyra.org/ From reilly@tiac.net Mon Feb 21 07:45:30 2000 From: reilly@tiac.net (Matthew Reilly) Date: Mon, 21 Feb 2000 03:45:30 -0400 Subject: [Patches] Patches for Compaq Alpha(64 bit)/Linux to fix test_struct and other problems Message-ID: <38B0ED1A.E11F51F0@tiac.net> This is a multi-part message in MIME format. --------------48E65A9B2D2B1624A5ECAE36 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit -- Linux Rules! Linux on Alpha Rules Bigtime! --------------48E65A9B2D2B1624A5ECAE36 Content-Type: text/plain; charset=us-ascii; name="patch-struct-time" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch-struct-time" TWIMC, Enclosed are two patches to fix problems with the structmodule.c and timemodule.c code when compiling on an Alpha Linux platform, RedHat version 6.1, egcs-2.91.66. The timemodule.c patch fixes a problem with the call to strptime. The stock RedHat 5.2, 6.0 and 6.1 distributions have the /usr/include/time.h file setup so that strptime's definition is only exported under limited circumstances. This patch works around that problem. Without the patch, the compiler complains that strptime's return value is being coerced from an int (32 bits) to a pointer (64 bits). This is bad. ./timemodule.c: In function `time_strptime': ./timemodule.c:429: warning: assignment makes pointer from integer without a cast The structmodule.c patch fixes two problems: First, the int/float conversion routines really should use unsigned longs rather than signed longs. I've only tested the patch on an Alpha/Linux system, so I encapsulated the change in an #ifdef __alpha__ wrapper. I suspect the change is appropriate for all platforms. Second, for reasons that I can't possibly figure out the memcpy call that is meant to copy the float value isn't working properly under Alpha/Linux. I have no idea why. Memcpy really does work, I use it all the time elsewhere. In any case, the memcpy is a little excessive for what y'all are trying to do, the patch offers a simpler (and, on an alpha, a faster) solution. I've tested both patches with "make test". Before applying the patch to structmodule.c, test_struct will fail with a floating point exception. Now it passes. The two patches follow the disclaimer: ---------------------------------disclaimer-------------------------------- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. #----------------------timemodule.c.diff---------------------------------- Index: timemodule.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/timemodule.c,v retrieving revision 2.78 diff -C2 -r2.78 timemodule.c *** timemodule.c 2000/01/12 16:38:20 2.78 --- timemodule.c 2000/02/21 02:24:35 *************** *** 410,415 **** #ifdef HAVE_STRPTIME ! /* extern char *strptime(); /* Enable this if it's not declared in */ ! static PyObject * time_strptime(self, args) --- 410,419 ---- #ifdef HAVE_STRPTIME ! #ifdef __alpha__ ! #ifdef __linux__ ! extern char *strptime(); /* Enable this if it's not declared in */ ! #endif /* __linux__ */ ! #endif /* __alpha__ */ ! static PyObject * time_strptime(self, args) #------------------------------------------structmodule.c.diff---------------------- Index: structmodule.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/structmodule.c,v retrieving revision 2.29 diff -C2 -r2.29 structmodule.c *** structmodule.c 1998/12/04 18:50:11 2.29 --- structmodule.c 2000/02/21 02:24:05 *************** *** 62,66 **** #include "Python.h" #include "mymath.h" ! #include #include --- 62,66 ---- #include "Python.h" #include "mymath.h" ! #include #include #include *************** *** 231,235 **** --- 231,239 ---- int e; double f; + #ifdef __alpha__ + unsigned long fhi, flo; + #else long fhi, flo; + #endif if (x < 0) { *************** *** 366,370 **** int s; int e; ! long fhi, flo; double x; --- 370,378 ---- int s; int e; ! #ifdef __alpha__ ! unsigned long fhi, flo; ! #else ! long fhi, flo; ! #endif double x; *************** *** 651,654 **** --- 659,663 ---- const formatdef *f; { + double *dp; double x = PyFloat_AsDouble(v); if (x == -1 && PyErr_Occurred()) { *************** *** 657,661 **** return -1; } ! memcpy(p, (char *)&x, sizeof(double)); return 0; } --- 666,674 ---- return -1; } ! ! dp = (double *) p; ! ! *dp = x; ! return 0; } --------------48E65A9B2D2B1624A5ECAE36-- From jch@bsdi.com Mon Feb 21 19:21:16 2000 From: jch@bsdi.com (Jeffrey C Honig) Date: Mon, 21 Feb 2000 14:21:16 -0500 Subject: [Patches] BSD/OS 4.0 modification for Python 1.5.2 dynamic objects Message-ID: <200002211921.OAA20048@kismet.honig.net> ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <19939.951160711.1@kismet.honig.net> The following changes to Python's 1.5.2's configure allow the optional modules to be built as shared objects on BSD/OS 4.0 and later. Thanks. Jeff ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <19939.951160711.2@kismet.honig.net> Content-Description: Diffs Index: configure =================================================================== RCS file: /master/contrib/Python-1.5.2/configure,v retrieving revision 1.1.1.1 diff -c -r1.1.1.1 configure *** configure 1999/10/07 01:07:11 1.1.1.1 --- configure 2000/02/17 20:27:51 *************** *** 2246,2251 **** --- 2246,2252 ---- fi ;; Linux*) LDSHARED="gcc -shared";; dgux*) LDSHARED="ld -G";; + BSD/OS/4*) LDSHARED="ld -shared";; FreeBSD*/3*) LDSHARED="gcc -shared";; FreeBSD*|OpenBSD*) LDSHARED="ld -Bshareable";; NetBSD*) *************** *** 2272,2277 **** --- 2273,2279 ---- else CCSHARED="+z"; fi;; Linux*) CCSHARED="-fpic";; + BSD/OS/4*) CCSHARED="-fPIC";; FreeBSD*|OpenBSD*) CCSHARED="-fpic";; NetBSD*) CCSHARED="-fPIC";; SCO_SV*) CCSHARED="-KPIC -dy -Bdynamic";; *************** *** 2292,2297 **** --- 2294,2300 ---- AIX*) LINKFORSHARED='-Wl,-bE:python.exp -lld';; hp*|HP*) LINKFORSHARED="-Wl,-E -Wl,+s -Wl,+b\$(BINLIBDEST)/lib-dynload";; + BSD/OS/4*) LINKFORSHARED="-Xlinker -export-dynamic";; FreeBSD/3*) LINKFORSHARED="-Xlinker -export-dynamic";; Linux*) LINKFORSHARED="-Xlinker -export-dynamic";; # -u libsys_s pulls in all symbols in libsys ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <19939.951160711.3@kismet.honig.net> Content-Description: Disclaimer I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ------- =_aaaaaaaaaa0-- From jim@interet.com Tue Feb 22 19:51:48 2000 From: jim@interet.com (James C. Ahlstrom) Date: Tue, 22 Feb 2000 14:51:48 -0500 Subject: [Patches] Re: New module zipfile.py References: <38A4378A.CD4BE4B3@interet.com> Message-ID: <38B2E8D4.75E142@interet.com> I didn't hear anything about this patch. Has it been accepted or rejected? JimA From guido@python.org Tue Feb 22 19:58:28 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 22 Feb 2000 14:58:28 -0500 Subject: [Patches] Re: New module zipfile.py In-Reply-To: Your message of "Tue, 22 Feb 2000 14:51:48 EST." <38B2E8D4.75E142@interet.com> References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> Message-ID: <200002221958.OAA05758@eric.cnri.reston.va.us> > I didn't hear anything about this patch. Has it > been accepted or rejected? Neither yet. I'm waiting for some discussion. I am unfortunately in load-shedding mode, so I don't expect I'll be able to do something about it myself. :-( I wrote about it earlier (but forgot to cc you): | My own response is that the code looks a bit dense, with 2-space | indents, very few blank lines, and some lines longer than 78 chars, | but that it is otherwise decent; however I don't have the time to | stress-test it. | | I noticed that when I run the self-test (which should be moved into a | module test/test_zipfile.py) I get an error: | | & ../sparc/python | Python 1.5.2+ (#1017, Feb 16 2000, 16:03:45) [GCC 2.8.1] on sunos5 | Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam | >>> import zipfile | >>> zipfile.test() | Traceback (innermost last): | File "", line 1, in ? | File "zipfile.py", line 335, in test | z.writepy("Lib") # Directory ./Lib must exist | File "zipfile.py", line 279, in writepy | raise RuntimeError, 'Files added with writepy() must end with ".py"' | RuntimeError: Files added with writepy() must end with ".py" | >>> --Guido van Rossum (home page: http://www.python.org/~guido/) From jim@interet.com Wed Feb 23 16:31:23 2000 From: jim@interet.com (James C. Ahlstrom) Date: Wed, 23 Feb 2000 11:31:23 -0500 Subject: [Patches] Re: New module zipfile.py References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> Message-ID: <38B40B5B.D1619CCE@interet.com> This is a multi-part message in MIME format. --------------D110250949A2B8223A02F0DB Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Guido van Rossum wrote: > | My own response is that the code looks a bit dense, with 2-space > | indents, very few blank lines, and some lines longer than 78 chars, > | but that it is otherwise decent; however I don't have the time to > | stress-test it. I corrected long lines. > | I noticed that when I run the self-test (which should be moved into a > | module test/test_zipfile.py) I get an error: This "test" only worked from the Python home. I removed it and created a new test/test_zipfile.py which is attached along with the new zipfile.py. JimA --------------D110250949A2B8223A02F0DB Content-Type: text/plain; charset=us-ascii; name="zipfile.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="zipfile.py" "Read and write ZIP files" # Written by James C. Ahlstrom jim@interet.com # All rights transferred to CNRI pursuant to the Python contribution agreement import struct, os, time import binascii, py_compile try: import zlib # We may need its compression method except: pass class _BadZipfile(Exception): pass error = _BadZipfile # The exception raised by this module _debug = 1 # constants for Zip file compression methods ZIP_STORED = 0 ZIP_DEFLATED = 8 # other ZIP compression methods not supported def is_zipfile(filename): """Quickly see if file is a ZIP file by checking the magic number. Will not accept a ZIP archive with an ending comment.""" try: fpin = open(filename, "rb") fpin.seek(-22, 2) # Seek to end-of-file record endrec = fpin.read() fpin.close() if endrec[0:4] == "PK\005\006" and endrec[-2:] == "\000\000": return 1 # file has correct magic number except: pass def zip2date(d): "Return (year, month, day) for a date in zip format" return (d>>9)+1980, (d>>5)&0xF, d&0x1F def zip2time(t): "Return (hour, minute, second) for a time in zip format" return t>>11, (t>>5)&0x3F, t&0x1F * 2 def date2zip(year, month, day): "Return 16-bit zip date for year, month, day" return (year - 1980) << 9 | month << 5 | day def time2zip(hour, minute, second): "Return 16-bit zip time for hour, minute, second" return hour << 11 | minute << 5 | second / 2 class ZipFile: "Class with methods to open, read, write, close, list zip files" # Here are some struct module formats for reading headers structEndArchive = "<4s4H2lH" # 9 items, end of archive, 22 bytes stringEndArchive = "PK\005\006" # magic number for end of archive record structCentralDir = "<4s4B4H3i5H2i" # 19 items, central directory, 46 bytes stringCentralDir = "PK\001\002" # magic number for central directory structFileHeader = "<4s2B4H3i2H" # 12 items, file header record, 30 bytes stringFileHeader = "PK\003\004" # magic number for file header def __init__(self, filename, mode = "r", compression = 0): """Construct a ZipFile instance and open the ZIP file named "filename" with mode read "r", write "w" or append "a".""" if compression == ZIP_STORED: pass elif compression == ZIP_DEFLATED: try: import zlib except: raise RuntimeError, "Compression requires the (missing) zlib module" else: raise RuntimeError, "Compression method must be 0 or 8" self.TOC = {} # Table of contents for the archive self.compression = compression # Method of compression self.filename = filename self.mode = key = mode[0] if key == 'r': self.fp = open(filename, "rb") self._getTOC() elif key == 'w': self.fp = open(filename, "wb") elif key == 'a': fp = self.fp = open(filename, "r+b") fp.seek(-22, 2) # Seek to end-of-file record endrec = fp.read() if endrec[0:4] == self.stringEndArchive and endrec[-2:] == "\000\000": self._getTOC() # file is a zip file fp.seek(self.start_dir, 0) # seek to start of directory and overwrite else: # file is not a zip file, just append fp.seek(0, 2) else: raise RuntimeError, 'Mode must be "r", "w" or "a"' def _getTOC(self): "Read in the table of contents for the zip file" fp = self.fp fp.seek(-22, 2) # Start of end-of-archive record filesize = fp.tell() + 22 # Get file size endrec = fp.read(22) # Archive must not end with a comment! if endrec[0:4] != self.stringEndArchive or endrec[-2:] != "\000\000": raise BadZipfile, "File is not a zip file, or ends with a comment" endrec = struct.unpack(self.structEndArchive, endrec) if _debug > 2: print endrec size_cd = endrec[5] # bytes in central directory offset_cd = endrec[6] # offset of central directory x = filesize - 22 - size_cd # "concat" is zero, unless zip was concatenated to another file concat = x - offset_cd if _debug > 2: print "given, inferred, offset", offset_cd, x, concat # self.start_dir: Position of start of central directory self.start_dir = offset_cd + concat fp.seek(self.start_dir, 0) total = 0 flist = [] # List of file header offsets while total < size_cd: centdir = fp.read(46) total = total + 46 if centdir[0:4] != self.stringCentralDir: raise BadZipfile, "Bad magic number for central directory" centdir = struct.unpack(self.structCentralDir, centdir) if _debug > 2: print centdir fname = fp.read(centdir[12]) extra = fp.read(centdir[13]) comment = fp.read(centdir[14]) total = total + centdir[12] + centdir[13] + centdir[14] if _debug > 2: print "total", total flist.append(centdir[18]) # Offset of file header record if _debug > 2: print flist toc = self.TOC # Table of contents for offset in flist: fp.seek(offset + concat, 0) fheader = fp.read(30) if fheader[0:4] != self.stringFileHeader: raise BadZipfile, "Bad magic number for file header" fheader = struct.unpack(self.structFileHeader, fheader) fname = fp.read(fheader[10]) if _debug > 1: print "File", fname, fheader extra = fp.read(fheader[11]) toc[fname] = (fp.tell(), extra) + fheader[3:10] # toc key is the file name, and the corresponding value is the tuple: # 0:file offset, 1:extra data as string, 2:bit flags, # 3:compression type, 4:file time, 5:file date, 6:CRC-32, # 7:compressed size, 8:uncompressed size. def listdir(self): return self.TOC.keys() def printdir(self): "Print table of contents for zip file" toc = self.TOC if _debug > 2: print toc print "%-30s %19s %12s" % ("File Name", "Modified ", "Size") for name, data in toc.items(): bytes = self.read(name) # Just to check CRC-32 d = data[5] # Date t = data[4] # Time date = "%d-%02d-%02d %02d:%02d:%02d" % (zip2date(d) + zip2time(t)) print "%-30s %s %12d" % (name, date, data[8]) def read(self, name): "Return file bytes (as a string) for name" if self.mode not in ("r", "a"): raise RuntimeError, 'read() requires mode "r" or "a"' if not self.fp: raise RuntimeError, \ "Attempt to read ZIP archive that was already closed" data = self.TOC[name] filepos = self.fp.tell() self.fp.seek(data[0], 0) bytes = self.fp.read(data[7]) self.fp.seek(filepos, 0) if data[3] == ZIP_STORED: # Compression method: none pass elif data[3] == ZIP_DEFLATED: # Compression method: deflation # zlib compress/decompress code by Jeremy Hylton of CNRI dc = zlib.decompressobj(-15) bytes = dc.decompress(bytes) # need to feed in unused pad byte so that zlib won't choke ex = dc.decompress('Z') + dc.flush() if ex: bytes = bytes + ex else: raise BadZipfile, \ "Unsupported compression method %d for file %s" % (data[3], name) crc = binascii.crc32(bytes) if crc != data[6]: raise BadZipfile, "Bad CRC-32 for file %s" % name return bytes def write(self, filename, arcname, extra = ""): """Put the bytes from filename into the archive under the name arcname. The "extra" is the extra data string.""" mtime = os.stat(filename)[8] mtime = time.localtime(mtime) year, month, day, hour, minute, second = mtime[0:6] fp = open(filename, "rb") bytes = fp.read() fp.close() self.writestr(bytes, arcname, year, month, day, hour, minute, second, extra) def writestr(self, bytes, arcname, year, month, day, hour, minute, second, extra = ""): """Write bytes and other data into the archive under the name arcname. The "extra" is the extra data string.""" if self.TOC.has_key(arcname): # Warning for duplicate names if _debug: print "Duplicate name:", arcname if self.mode not in ("w", "a"): raise RuntimeError, 'write() requires mode "w" or "a"' if not self.fp: raise RuntimeError, \ "Attempt to write ZIP archive that was already closed" dosdate = date2zip(year, month, day) dostime = time2zip(hour, minute, second) compression = self.compression # Method of compression u_size = len(bytes) # Uncompressed size crc = binascii.crc32(bytes) # CRC-32 checksum if compression == ZIP_DEFLATED: co = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -15) bytes = co.compress(bytes) + co.flush() c_size = len(bytes) # Compressed size else: c_size = u_size header = struct.pack(self.structFileHeader, self.stringFileHeader, 10, 0, 0, compression, dostime, dosdate, crc, c_size, u_size, len(arcname), len(extra)) self.fp.write(header) self.fp.write(arcname) self.fp.write(extra) offset = self.fp.tell() # Start of file bytes self.fp.write(bytes) self.TOC[arcname] = (offset, extra, 0, compression, dostime, dosdate, crc, c_size, u_size) def writepy(self, pathname, basename = ""): """If pathname is a package directory, search the directory and all package subdirectories recursively for all *.py and enter the modules into the archive. If pathname is a plain directory, listdir *.py and enter all modules. Else, pathname must be a Python *.py file and the module will be put into the archive. Added modules are always module.pyo or module.pyc. This method will compile the module.py into module.pyc if necessary.""" dir, name = os.path.split(pathname) if os.path.isdir(pathname): initname = os.path.join(pathname, "__init__.py") if os.path.isfile(initname): # This is a package directory, add it if basename: basename = "%s/%s" % (basename, name) else: basename = name if _debug: print "Adding package in", pathname, "as", basename fname, arcname = self._get_codename(initname[0:-3], basename) if _debug: print "Adding", arcname self.write(fname, arcname) dirlist = os.listdir(pathname) dirlist.remove("__init__.py") # Add all *.py files and package subdirectories in the directory for filename in dirlist: path = os.path.join(pathname, filename) root, ext = os.path.splitext(filename) if os.path.isdir(path): if os.path.isfile(os.path.join(path, "__init__.py")): # This is a package directory, add it self.writepy(path, basename) # Recursive call elif ext == ".py": fname, arcname = self._get_codename(path[0:-3], basename) if _debug: print "Adding", arcname self.write(fname, arcname) else: # This is NOT a package directory, add its files at top level if _debug: print "Adding files from directory", pathname for filename in os.listdir(pathname): path = os.path.join(pathname, filename) root, ext = os.path.splitext(filename) if ext == ".py": fname, arcname = self._get_codename(path[0:-3], basename) if _debug: print "Adding", arcname self.write(fname, arcname) else: if pathname[-3:] != ".py": raise RuntimeError, 'Files added with writepy() must end with ".py"' fname, arcname = self._get_codename(pathname[0:-3], basename) if _debug: print "Adding file", arcname self.write(fname, arcname) def _get_codename(self, pathname, basename): """Given a module name path, return the correct file path and archive name, compiling if necessary. For example, given /python/lib/string, return (/python/lib/string.pyc, string)""" file_py = pathname + ".py" file_pyc = pathname + ".pyc" file_pyo = pathname + ".pyo" if os.path.isfile(file_pyo) and \ os.stat(file_pyo)[8] >= os.stat(file_py)[8]: fname = file_pyo # Use .pyo file elif not os.path.isfile(file_pyc) or \ os.stat(file_pyc)[8] < os.stat(file_py)[8]: if _debug: print "Compiling", file_py py_compile.compile(file_py, file_pyc) fname = file_pyc else: fname = file_pyc archivename = os.path.split(fname)[1] if basename: archivename = "%s/%s" % (basename, archivename) return (fname, archivename) def __del__(self): # User should have called close(), but just in case if self.fp: self.fp.close() self.fp = None def close(self): if self.mode in ("w", "a"): # write ending records attrib = 0666 << 16 # file attributes count = 0 pos1 = self.fp.tell() for name, data in self.TOC.items(): # write central directory count = count + 1 namesize = len(name) extrasize = len(data[1]) centdir = struct.pack(self.structCentralDir, self.stringCentralDir, 20, 3, 10, 0, data[2], data[3], data[4], data[5], data[6], data[7], data[8], namesize, extrasize, 0, 0, 0, attrib, data[0] - namesize - extrasize - 30) self.fp.write(centdir) self.fp.write(name) self.fp.write(data[1]) pos2 = self.fp.tell() endrec = struct.pack(self.structEndArchive, self.stringEndArchive, 0, 0, count, count, pos2 - pos1, pos1, 0) self.fp.write(endrec) self.fp.close() self.fp = None --------------D110250949A2B8223A02F0DB Content-Type: text/plain; charset=us-ascii; name="test_zipfile.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="test_zipfile.py" import zipfile, os srcname = "junk9630.tmp" zipname = "junk9708.tmp" try: fp = open(srcname, "w") # Make a source file with some lines for i in range(0, 1000): fp.write("Test of zipfile line %d.\n" % i) fp.close() zip = zipfile.ZipFile(zipname, "w") # Create the ZIP archive zip.write(srcname, srcname) zip.write(srcname, "another.name") zip.close() zip = zipfile.ZipFile(zipname, "r") # Read the ZIP archive zip.read("another.name") zip.read(srcname) zip.close() finally: if os.path.isfile(srcname): # Remove temporary files os.unlink(srcname) if os.path.isfile(zipname): os.unlink(zipname) --------------D110250949A2B8223A02F0DB-- From fdrake@acm.org Wed Feb 23 17:14:42 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 23 Feb 2000 12:14:42 -0500 (EST) Subject: [Patches] Re: New module zipfile.py In-Reply-To: <38B40B5B.D1619CCE@interet.com> References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> Message-ID: <14516.5507.2552.852735@weyr.cnri.reston.va.us> Jim, Are you planning to write a section for the Library Reference for this module? I don't need it right now since I'm really working on the last 1.5.2 release, but will need it before 1.6 final is release (preferably before the beta stage). Thanks! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From jim@interet.com Wed Feb 23 18:02:01 2000 From: jim@interet.com (James C. Ahlstrom) Date: Wed, 23 Feb 2000 13:02:01 -0500 Subject: [Patches] Re: New module zipfile.py References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> <14516.5507.2552.852735@weyr.cnri.reston.va.us> Message-ID: <38B42099.ADFA9BD0@interet.com> "Fred L. Drake, Jr." wrote: > > Jim, > Are you planning to write a section for the Library Reference for > this module? I already wrote the docs, and sent it with the original patch. If you can't find it, let me know and I will send it again. JimA From fdrake@acm.org Wed Feb 23 18:05:09 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 23 Feb 2000 13:05:09 -0500 (EST) Subject: [Patches] Re: New module zipfile.py In-Reply-To: <38B42099.ADFA9BD0@interet.com> References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> <14516.5507.2552.852735@weyr.cnri.reston.va.us> <38B42099.ADFA9BD0@interet.com> Message-ID: <14516.8533.276579.670427@weyr.cnri.reston.va.us> James C. Ahlstrom writes: > I already wrote the docs, and sent it with the > original patch. If you can't find it, let me > know and I will send it again. I'll try to dig it up & let you know if I can't find it. Thanks! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From guido@python.org Wed Feb 23 22:45:55 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 23 Feb 2000 17:45:55 -0500 Subject: [Patches] Re: New module zipfile.py In-Reply-To: Your message of "Wed, 23 Feb 2000 11:31:23 EST." <38B40B5B.D1619CCE@interet.com> References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> Message-ID: <200002232245.RAA07934@eric.cnri.reston.va.us> > I corrected long lines. That leaves the 2-space indents, lack of vertical whitespace (can we please have a blank line before each method?), and your curious habit of not indenting doc strings and not leaving a blank line between the "summary line" of the doc string (its first line) and the rest. None of that is a showstopper, and a separate coding style conformance pass can be made to fix these issues. More code nits: Instead of leaving zlib undefined when its import fails, it's more customary to set zlib to None in the except clause. Then later you can test for "zlib is None" rather than having to try the whole import again. I read through the docs (after reformatting them slightly) and played a bit with the code. Some comments on the docs and some comments on the API or on the implemented functionality follow here: > zipfile -- Read and write files in ZIP format > > The ZIP file format is a common archive and compression standard. This > module provides tools to create, read, write, append, and list a ZIP > file. > > > The available attributes of this module are: > > error > The error raised for bad ZIP files. > _debug > Level of printing, default 1. This would be better of being an instance variable (maybe with a class or global default). > class ZipFile > The class for reading and writing ZIP files. > is_zipfile(path) > Return 1/0 if "path" is/is not a valid ZIP file based on its magic > number. This module does not currently handle ZIP files which have > appended comments. > zip2date(zdate) > Return (year, month, day) for a zip date code. > zip2time(ztime) > Return (hour, minute, second) for a zip time code. > date2zip(year, month, day) > Return a zip date code. > time2zip(hour, minute, second) > Return a zip time code. Are these four really relevant in the module interface? It seems they are only needed internally. > ZIP_STORED > The numeric constant (zero) for an uncompressed archive. > ZIP_DEFLATED > The numeric constant for the usual ZIP compression > method. This requires the zlib module. No other > compression methods are currently supported. > > An instance of class ZipFile has self.TOC, a read-only dictionary > whose keys are the names in the archive, and whose values are tuples > as follows: (It took me a bit to realize that the numerical key was the tuple item index.) > 0: File data seek offset. > 1: Zip file "extra" data as a string. > 2: Zip file bit flags. > 3: Zip file compression type. > 4: File modification time in DOS format. > 5: File modification date in DOS format. > 6: The CRC-32 of the uncompressed data. > 7: The compressed size of the file. > 8: The uncompressed size of the file. Is it proper to document this? I can easily see that a later version of the module would store this information differently (e.g. as an instance of a "zipinfo" class). I can also easily see that the record would grow as we need to keep track of more information. On the other hand, there's some useful information available from the zip directory that I don't know how to extract from the TOC; in particular, the file's type and mode are often of interest when extracting. > The class ZipFile has these methods: > > __init__(self, filename, mode = "r", compression = 0) (Style nit: in function declarations I tend not to place spaces around the '=' sign in default arguments; also not in function calls with keyword arguments.) The default would be better written as ZIP_STORED. But... Shouldn't the compression be a per-file option? It certainly is recorded per file. > Open a ZIP file named "filename". Mode is "r" to read an existing > file, "w" to truncate and write a new file, or "a" to append. For > mode "a", if filename is a ZIP file, then additional files are added > to it. If filename is not a ZIP file, then a new ZIP file is appended > to the file. This is meant for adding a ZIP archive to another file > such as python.exe. But "cat myzip.zip >> python.exe" also works, and > at least WinZip can read such files. The "compression" is the ZIP > compression method to use when writing the archive. Explain that possible values for compression are ZIP_STORED and ZIP_DEFLATED. > listdir(self) > > Return a list of names in the archive. Equivalent to self.TOC.keys(). In which order are they returned? Wouldn't it be handy to return them in the same order in which the files occur in the archive? > printdir(self) > > Print a table of contents for the archive to stdout. I tried printdir() on a zipfile open for writing; it failed. If this is intentional, it should be documented; otherwise, fixed. :-) The printdir() function shows information that isn't available otherwise except by peeking in the TOC (which I think is an objectionable interface). > read(self, name) > > Return the bytes of the file in the archive. The archive must be open > for read or append. > > writestr(self, bytes, arcname, year, month, day, hour, minute, second, > extra = "") > > Write the string "bytes" and the other data to the archive, and give > it the name "arcname". The "extra" is the ZIP extra data string. The > archive must be open with mode "w" or "a". It would make more sense if the date/time arguments were grouped in a tuple such as returned by time.localtime(). Also, it would make sense if it was specified whether the time is supposed to be given in the local timezone. Also, I noticed that this emits a warning to stdout if the name already exists. The idea that the module prints to stdout for various reasons is a little scary -- as you often repeat yourself, in GUI apps, there may be no stdout, or writing to it may be frowned upon. (Maybe _debug should just be set to 0 by default? That would shut up everything except writepy() and printdir(), I believe.) > write(self, filename, arcname, extra = "") > > Write the file named "filename" to the archive, and give it the > archive name "arcname". The "extra" is the ZIP extra data string. > The archive must be open with mode "w" or "a". Should note that the file is opened in binary mode. > writepy(self, pathname, basename = "") > > Search for files *.py and add the corresponding file to the archive. > The corresponding file is a *.pyo file if available, else a *.pyc > file, compiling if necessary. If the pathname is a file, the file > must end with ".py", and just the (corresponding *.py[oc]) file is > added at the top level (no path information). If it is a directory, > and the directory is not a package directory, then all the files > *.py[oc] are added at the top level. If the directory is a package > directory, then all *.py[oc] are added under the package name as a > file path, and if any subdirectories are package directories, all of > these are added recursively. The "basename" is intended for internal > use only. The writepy() method makes archives with file names like > this: > > string.pyc # Top level name > test/__init__.pyc # Package directory > test/testall.pyc # Package "test.testall" file > test/bogus/__init__.pyc # Subpackage directory > test/bogus/myfile.pyc # Subpackage "test.bogus.myfile" file Argh! This method exists for a very specific purpose. I think it ought to be placed in a subclass (PyZipFile?) whose stated purpose it is to build Python zipfile. > close(self) > > Close the archive file. You must call close() before exiting your > program or essential records will not be written. Another comment: when I use the Unix zipinfo utility to list the contents of an archive created with this module, the first character of the file mode is listed as '?'. It should be '-', indicating a plain file. No idea which flag you should set tohow get this effect, but the zip documentation should tell you. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Feb 23 23:19:26 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 23 Feb 2000 18:19:26 -0500 Subject: [Patches] Re: New module zipfile.py In-Reply-To: Your message of "Wed, 23 Feb 2000 17:45:55 EST." <200002232245.RAA07934@eric.cnri.reston.va.us> References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> <200002232245.RAA07934@eric.cnri.reston.va.us> Message-ID: <200002232319.SAA08193@eric.cnri.reston.va.us> I discovered another problem. I get all sorts of weird dates in the future (e.g. 2005). When I list the same zipfile with zipinfo I get kosher dates (e.g. 1996). The rest of the date and time are also nonsense; could it be that you're not recognizing certain header format variations? The zip2date() function seems simple enough... I've placed a sample file that exhibits this behavior on ftp.python.org in pub/tmp/kiczales.zip. For this file, zipinfo says: Archive: kiczales.zip 361208 bytes 1 file -rw-a-- 2.0 fat 1698308 t- defN 9-Sep-96 13:01 tutorial2.ps 1 file, 1698308 bytes uncompressed, 361086 bytes compressed: 78.7% Your code finds a date field (TOC tuple item 4) of 0x6820 which decodes to 2032/1/0. --Guido van Rossum (home page: http://www.python.org/~guido/) From piers@cs.su.oz.au Thu Feb 24 01:18:50 2000 From: piers@cs.su.oz.au (Piers Lauder) Date: Thu, 24 Feb 2000 12:18:50 +1100 Subject: [Patches] imaplib.py patch Message-ID: <951355725.347.613520860@cs.usyd.edu.au> --129.78.10.43.126.5360.951355725.350.14600 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi! The following patch to imaplib.py is wrt. the current Python CVS tree (as of 1/2 hour ago) and was generated via "diff -c". Standard disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Reason for patch: A change in my last patch could, under certain circumstances, cause a loop if the connection to the server dropped while waiting for a command completion. I've changed the code to re-raise the error after possible debugging output. Piers Lauder. --129.78.10.43.126.5360.951355725.350.14600 Content-Type: application/octet-stream; name=imaplib-diff Content-Transfer-Encoding: 7bit *** dist/src/Lib/imaplib.py Thu Feb 24 11:03:13 2000 --- /usr/staff/piers/lib/html/python/imaplib.py Thu Feb 24 10:54:49 2000 *************** *** 15,21 **** Time2Internaldate """ ! __version__ = "2.30" import binascii, re, socket, string, time, random, sys --- 15,21 ---- Time2Internaldate """ ! __version__ = "2.32" import binascii, re, socket, string, time, random, sys *************** *** 755,763 **** return result # Some have reported "unexpected response" exceptions. ! # (Isn't this non-IMAP4-compliant behaviour? ! # Please mail me details printed below!) ! # Anyway, ignore them here. try: self._get_response() --- 755,763 ---- return result # Some have reported "unexpected response" exceptions. ! # Note that ignoring them here causes loops. ! # Instead, send me details of the unexpected response and ! # I'll update the code in `_get_response()'. try: self._get_response() *************** *** 764,771 **** except self.abort, val: if __debug__: if self.debug >= 1: - _mesg('abort exception ignored: %s' % val) print_log() def _get_line(self): --- 764,771 ---- except self.abort, val: if __debug__: if self.debug >= 1: print_log() + raise def _get_line(self): --129.78.10.43.126.5360.951355725.350.14600-- From tg@melaten.rwth-aachen.de Thu Feb 24 10:48:22 2000 From: tg@melaten.rwth-aachen.de (Thomas Gellekum) Date: 24 Feb 2000 11:48:22 +0100 Subject: [Patches] New function in socketmodule: getservbyport Message-ID: --=-=-= Moin, The first attachment contains diffs for Modules/socketmodule.c which implement an interface to getservbyport(3). I chose not to worry about aliases for the service name. The second attachment updates the documentation. Both patches are relative to Python-1.5.2. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. tg --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=socketmodule.c.diff Content-Description: Patch for socketmodule.c *** Modules/socketmodule.c.orig Tue Apr 13 06:07:32 1999 --- Modules/socketmodule.c Wed Feb 23 18:07:50 2000 *************** *** 48,53 **** --- 48,54 ---- - socket.gethostname() --> host name (string: 'spam' or 'spam.domain.com') - socket.getprotobyname(protocolname) --> protocol number - socket.getservbyname(servicename, protocolname) --> port number + - socket.getservbyport(portnumber, protocolname) --> service name - socket.socket(family, type [, proto]) --> new socket object - socket.ntohs(16 bit value) --> new int object - socket.ntohl(32 bit value) --> new int object *************** *** 1596,1601 **** --- 1597,1631 ---- The protocol name should be 'tcp' or 'udp'."; + /* Python interface to getservbyport(port). + This only returns the service name for now. */ + + /*ARGSUSED*/ + static PyObject * + BUILD_FUNC_DEF_2(PySocket_getservbyport,PyObject *,self, PyObject *,args) + { + int port; + char *proto; + struct servent *sp; + if (!PyArg_Parse(args, "(is)", &port, &proto)) + return NULL; + Py_BEGIN_ALLOW_THREADS + sp = getservbyport(ntohs(port), proto); + Py_END_ALLOW_THREADS + if (sp == NULL) { + PyErr_SetString(PySocket_Error, "port/proto not found"); + return NULL; + } + return PyString_FromString(sp->s_name); + } + + static char getservbyport_doc[] = + "getservbyport(port, protocolname) -> string\n\ + \n\ + Return a service name from a port number and protocol name.\n\ + The protocol name should be 'tcp' or 'udp'."; + + /* Python interface to getprotobyname(name). This only returns the protocol number, since the other info is already known or not useful (like the list of aliases). */ *************** *** 1794,1799 **** --- 1824,1830 ---- {"gethostbyaddr", PySocket_gethostbyaddr, 0, gethostbyaddr_doc}, {"gethostname", PySocket_gethostname, 0, gethostname_doc}, {"getservbyname", PySocket_getservbyname, 0, getservbyname_doc}, + {"getservbyport", PySocket_getservbyport, 0, getservbyport_doc}, {"getprotobyname", PySocket_getprotobyname, 0,getprotobyname_doc}, {"socket", PySocket_socket, 1, socket_doc}, #ifndef NO_DUP *************** *** 1924,1929 **** --- 1955,1961 ---- gethostbyname() -- map a hostname to its IP number\n\ gethostbyaddr() -- map an IP number or hostname to DNS info\n\ getservbyname() -- map a service name and a protocol name to a port number\n\ + getservbyport() -- map a port number and a protocol name to a service name\n\ getprotobyname() -- mape a protocol name (e.g. 'tcp') to a number\n\ ntohs(), ntohl() -- convert 16, 32 bit int from network to host byte order\n\ htons(), htonl() -- convert 16, 32 bit int from host to network byte order\n\ --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=libsocket.tex.diff Content-Description: Patch for libsocket.tex *** Doc/lib/libsocket.tex.orig Wed Apr 21 19:29:14 1999 --- Doc/lib/libsocket.tex Thu Feb 24 11:38:34 2000 *************** *** 150,155 **** --- 150,161 ---- \code{'udp'}. \end{funcdesc} + \begin{funcdesc}{getservbyport}{portnumber, protocolname} + Translate a port number and protocol name to an Internet service name + for that port. The protocol name should be \code{'tcp'} or + \code{'udp'}. + \end{funcdesc} + \begin{funcdesc}{socket}{family, type\optional{, proto}} Create a new socket using the given address family, socket type and protocol number. The address family should be \constant{AF_INET} or --=-=-=-- From Moshe Zadka Thu Feb 24 14:55:39 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 24 Feb 2000 16:55:39 +0200 (IST) Subject: [Patches] tkSimpleDialog docstrings Message-ID: *** /home/moshe/work/src/python/python/dist/src/Lib/lib-tk/tkSimpleDialog.py Fri Jun 25 17:53:54 1999 --- tkSimpleDialog.py Thu Feb 24 16:43:11 2000 *************** *** 11,23 **** --- 11,50 ---- # -------------------------------------------------------------------- # dialog base class + '''Dialog boxes + + This module handles dialog boxes. It contains the following + public symbols: + + Dialog -- a base class for dialogs + + askinteger -- get an integer from the user + + askfloat -- get a float from the user + + askstring -- get a string from the user + ''' + from Tkinter import * import os class Dialog(Toplevel): + '''Class to open dialogs. + + This class is intended as a base class for custom dialogs + ''' + def __init__(self, parent, title = None): + '''Initialize a dialog. + + Arguments: + + parent -- a parent window (the application window) + + title -- the dialog title + ''' Toplevel.__init__(self, parent) self.transient(parent) *************** *** 49,54 **** --- 76,82 ---- self.wait_window(self) def destroy(self): + '''Destroy the window''' self.initial_focus = None Toplevel.destroy(self) *************** *** 56,69 **** # construction hooks def body(self, master): ! # create dialog body. return widget that should have ! # initial focus. this method should be overridden pass def buttonbox(self): ! # add standard button box. override if you don't want the ! # standard buttons box = Frame(self) --- 84,102 ---- # construction hooks def body(self, master): ! '''create dialog body. + return widget that should have initial focus. + This method should be overridden, and is called + by the __init__ method. + ''' pass def buttonbox(self): ! '''add standard button box. ! ! override if you don't want the standard buttons ! ''' box = Frame(self) *************** *** 103,113 **** --- 136,156 ---- # command hooks def validate(self): + '''validate the data + This method is called automatically to validate the data before the + dialog is destroyed. By default, it always validates OK. + ''' + return 1 # override def apply(self): + '''process the data + This method is called automatically to process the data, *after* + the dialog is destroyed. By default, it does nothing. + ''' + pass # override *************** *** 196,201 **** --- 239,254 ---- return string.atoi(self.entry.get()) def askinteger(title, prompt, **kw): + '''get an integer from the user + + Arguments: + + title -- the dialog title + prompt -- the label text + **kw -- see SimpleDialog class + + Return value is an integer + ''' d = apply(_QueryInteger, (title, prompt), kw) return d.result *************** *** 205,210 **** --- 258,273 ---- return string.atof(self.entry.get()) def askfloat(title, prompt, **kw): + '''get a float from the user + + Arguments: + + title -- the dialog title + prompt -- the label text + **kw -- see SimpleDialog class + + Return value is a float + ''' d = apply(_QueryFloat, (title, prompt), kw) return d.result *************** *** 213,218 **** --- 276,291 ---- return self.entry.get() def askstring(title, prompt, **kw): + '''get a string from the user + + Arguments: + + title -- the dialog title + prompt -- the label text + **kw -- see SimpleDialog class + + Return value is a string + ''' d = apply(_QueryString, (title, prompt), kw) return d.result ------------ cut here ----- Checkin message: Added docstrings to tkSimpleDialog.py Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From pf@artcom-gmbh.de Thu Feb 24 14:49:42 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Thu, 24 Feb 2000 15:49:42 +0100 (MET) Subject: [Patches] Patch to src/Modules/mpzmodule.c Message-ID: Dear Python patcher! Please apply the patch appended below and commit into the CVS tree. --- argument: --- This patch allows building the Python 'mpzmodule' under SuSE Linux without having to install the source package of the GMP-libary. The gmp-mparam.h seems to be an internal header file. The patch shouldn't hurt any other platforms. --- obligatory disclaimer: --- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. --- dry signature: --- Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen) --- patch: --- *** ../../Python-CVS_10_02_00-orig/dist/src/Modules/mpzmodule.c Mon Dec 14 20:36:14 1998 --- src/Modules/mpzmodule.c Wed Feb 23 22:52:02 2000 *************** *** 92,98 **** --- 92,103 ---- */ #include "gmp.h" + + #if defined __GLIBC__ && __GLIBC__ >= 2 /*aid building on Linux distributions*/ + #define BITS_PER_MP_LIMB mp_bits_per_limb + #else #include "gmp-mparam.h" + #endif #if __GNU_MP__ + 0 == 2 #define GMP2 From akuchlin@mems-exchange.org Thu Feb 24 15:32:18 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Thu, 24 Feb 2000 10:32:18 -0500 (EST) Subject: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: References: Message-ID: <14517.20226.74800.504053@amarok.cnri.reston.va.us> Peter Funk writes: >This patch allows building the Python 'mpzmodule' under SuSE Linux >without having to install the source package of the GMP-libary. >The gmp-mparam.h seems to be an internal header file. The patch >shouldn't hurt any other platforms. >+ #if defined __GLIBC__ && __GLIBC__ >= 2 /*aid building on Linux distributions*/ >+ #define BITS_PER_MP_LIMB mp_bits_per_limb >+ #else > #include "gmp-mparam.h" >+ #endif This seems like the wrong fix, tying GMP to the version of glibc! (They haven't rolled GMP into glibc, have they?) If GMP 2.0 exposes an mp_bits_per_limb constant, then the correct patch may just be changing BITS_PER_MP_LIMB to mp_bits_per_limb, inside an #ifdef GMP2 if necessary. -- A.M. Kuchling http://starship.python.net/crew/amk/ "I'm your new assistant." "Oh, *no*." -- Jo Grant and the Doctor, in "Terror of the Autons" From guido@python.org Thu Feb 24 15:34:18 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Feb 2000 10:34:18 -0500 Subject: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: Your message of "Thu, 24 Feb 2000 10:32:18 EST." <14517.20226.74800.504053@amarok.cnri.reston.va.us> References: <14517.20226.74800.504053@amarok.cnri.reston.va.us> Message-ID: <200002241534.KAA09725@eric.cnri.reston.va.us> > Peter Funk writes: > >This patch allows building the Python 'mpzmodule' under SuSE Linux > >without having to install the source package of the GMP-libary. > >The gmp-mparam.h seems to be an internal header file. The patch > >shouldn't hurt any other platforms. > >+ #if defined __GLIBC__ && __GLIBC__ >= 2 /*aid building on Linux distributions*/ > >+ #define BITS_PER_MP_LIMB mp_bits_per_limb > >+ #else > > #include "gmp-mparam.h" > >+ #endif Andrew replies: > This seems like the wrong fix, tying GMP to the version of glibc! > (They haven't rolled GMP into glibc, have they?) If GMP 2.0 exposes > an mp_bits_per_limb constant, then the correct patch may just be > changing BITS_PER_MP_LIMB to mp_bits_per_limb, inside an #ifdef GMP2 > if necessary. Darn. I just checked it in. Well, if something better comes up, I'll check that in. I personally don't care much about this module (have never bothered to download gmp even so I can't even compile it). --Guido van Rossum (home page: http://www.python.org/~guido/) From pf@artcom-gmbh.de Thu Feb 24 15:17:34 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Thu, 24 Feb 2000 16:17:34 +0100 (MET) Subject: [Patches] Replace #!/usr/local/bin/python with portable /usr/bin/env python Message-ID: Dear Python patcher! Please apply the patch appended below and commit into the CVS tree. --- argument: --- On SuSE Linux and probably most other Linux Distributions the Python interpreter binary usually lives at /usr/bin/python. The exec-magic form '#!/usr/bin/env python' should work on all Unix compatible platforms. The files 'src/Lib/cgi.py' and 'src/Tools/faqwiz/faqw.py' contained the old depreceated form #!/usr/local/bin/python. --- obligatory disclaimer: --- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. --- dry signature: --- Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen) --- patch: --- *** ../../Python-CVS_10_02_00-orig/dist/src/Lib/cgi.py Fri Jun 11 20:26:09 1999 --- src/Lib/cgi.py Thu Feb 24 16:08:53 2000 *************** *** 1,4 **** ! #! /usr/local/bin/python """Support module for CGI (Common Gateway Interface) scripts. --- 1,4 ---- ! #! /usr/bin/env python """Support module for CGI (Common Gateway Interface) scripts. *** ../../Python-CVS_10_02_00-orig/dist/src/Tools/faqwiz/faqw.py Sat Apr 4 00:27:04 1998 --- src/Tools/faqwiz/faqw.py Thu Feb 24 16:08:04 2000 *************** *** 1,4 **** ! #! /usr/local/bin/python """FAQ wizard bootstrap.""" --- 1,4 ---- ! #! /usr/bin/env python """FAQ wizard bootstrap.""" From guido@python.org Thu Feb 24 15:48:50 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Feb 2000 10:48:50 -0500 Subject: [Patches] Replace #!/usr/local/bin/python with portable /usr/bin/env python In-Reply-To: Your message of "Thu, 24 Feb 2000 16:17:34 +0100." References: Message-ID: <200002241548.KAA09784@eric.cnri.reston.va.us> > On SuSE Linux and probably most other Linux Distributions the Python > interpreter binary usually lives at /usr/bin/python. The exec-magic > form '#!/usr/bin/env python' should work on all Unix compatible platforms. > The files 'src/Lib/cgi.py' and 'src/Tools/faqwiz/faqw.py' contained the > old depreceated form #!/usr/local/bin/python. No, no, no! The /usr/bin/env form *fails* on most platforms (except Linux) because Python is not installed in any of the default "bin" directories -- /usr/local/bin is the standard install place and that's not in the default path. I'm sorry, but Linux users (and everyone else who doesn't install Python in /usr/local, where it feels most at home) will simply have to edit their CGI scripts. That's no big deal; the cgi.py file is not realistically intended to be used as a script by itself anyway; the faqw.py wrapper needs to be edited anyway to point to the data directories. --Guido van Rossum (home page: http://www.python.org/~guido/) From pf@artcom-gmbh.de Thu Feb 24 15:33:24 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Thu, 24 Feb 2000 16:33:24 +0100 (MET) Subject: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: <14517.20226.74800.504053@amarok.cnri.reston.va.us> from "Andrew M. Kuchling" at "Feb 24, 2000 10:32:18 am" Message-ID: Hi! Andrew M. Kuchling schrieb: > Peter Funk writes: > >This patch allows building the Python 'mpzmodule' under SuSE Linux > >without having to install the source package of the GMP-libary. > >The gmp-mparam.h seems to be an internal header file. The patch > >shouldn't hurt any other platforms. > >+ #if defined __GLIBC__ && __GLIBC__ >= 2 /*aid building on Linux distributions*/ > >+ #define BITS_PER_MP_LIMB mp_bits_per_limb > >+ #else > > #include "gmp-mparam.h" > >+ #endif > > This seems like the wrong fix, tying GMP to the version of glibc! > (They haven't rolled GMP into glibc, have they?) No GMP and GLIBC are still distinct packages. May be this was too pragmatic patch just to be able to build the module without having to install the source-rpm previously. On RPM-based Linux distributions like Red Hat, SuSE and others you usually don't have the source tree unpacked during the build process of another package. Packages are unpacked and build one at a time on demand. > If GMP 2.0 exposes > an mp_bits_per_limb constant, then the correct patch may just be > changing BITS_PER_MP_LIMB to mp_bits_per_limb, inside an #ifdef GMP2 > if necessary. I've used the following version of GMP during my build: Name : gmp Distribution: SuSE Linux 6.3 (i386) Version : 2.0.2 Vendor: SuSE GmbH, Nuernberg, Germany [...] /usr/include/gmp.h [...] /usr/lib/libgmp.a As you can see, there is no 'gmp-param.h' exposed by the package. The package builder at SuSE used the same patch for building the Python 1.5.2 RPM distributed with SuSE Linux in 6.x. I have simply stolen his idea while building the CVS snapshot. Regards, Peter -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 From guido@python.org Thu Feb 24 16:05:12 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Feb 2000 11:05:12 -0500 Subject: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: Your message of "Thu, 24 Feb 2000 16:33:24 +0100." References: Message-ID: <200002241605.LAA09851@eric.cnri.reston.va.us> The question now arises: how is one supposed to use GMP without knowing the value of mp_bits_per_limb? Or, if this is truly a detail of the implementation, how come mpzmodule.c needs to know the value? These questions need to be answered before we can know the true fix for this problem! --Guido van Rossum (home page: http://www.python.org/~guido/) From jim@interet.com Thu Feb 24 16:44:36 2000 From: jim@interet.com (James C. Ahlstrom) Date: Thu, 24 Feb 2000 11:44:36 -0500 Subject: [Patches] Re: New module zipfile.py References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> <200002232245.RAA07934@eric.cnri.reston.va.us> <200002232319.SAA08193@eric.cnri.reston.va.us> Message-ID: <38B55FF4.743C9CB6@interet.com> Guido van Rossum wrote: > > I discovered another problem. I get all sorts of weird dates in the > future (e.g. 2005). When I list the same zipfile with zipinfo I get > kosher dates (e.g. 1996). The rest of the date and time are also > nonsense; could it be that you're not recognizing certain header > format variations? The zip2date() function seems simple enough... Try the z.printdir() method. It shows the correct dates. > I've placed a sample file that exhibits this behavior on > ftp.python.org in pub/tmp/kiczales.zip. > For this file, zipinfo says: > > Archive: kiczales.zip 361208 bytes 1 file > -rw-a-- 2.0 fat 1698308 t- defN 9-Sep-96 13:01 tutorial2.ps > 1 file, 1698308 bytes uncompressed, 361086 bytes compressed: 78.7% > > Your code finds a date field (TOC tuple item 4) of 0x6820 which > decodes to 2032/1/0. TOC tuple item 4 is the time, 5 is the date. This is the order of the fields in the file. JimA From guido@python.org Thu Feb 24 16:47:52 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Feb 2000 11:47:52 -0500 Subject: [Patches] Re: New module zipfile.py In-Reply-To: Your message of "Thu, 24 Feb 2000 11:44:36 EST." <38B55FF4.743C9CB6@interet.com> References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> <200002232245.RAA07934@eric.cnri.reston.va.us> <200002232319.SAA08193@eric.cnri.reston.va.us> <38B55FF4.743C9CB6@interet.com> Message-ID: <200002241647.LAA09959@eric.cnri.reston.va.us> > TOC tuple item 4 is the time, 5 is the date. This is the > order of the fields in the file. Argh! My sincere apologies. --Guido van Rossum (home page: http://www.python.org/~guido/) From pf@artcom-gmbh.de Thu Feb 24 18:05:45 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Thu, 24 Feb 2000 19:05:45 +0100 (MET) Subject: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: <200002241605.LAA09851@eric.cnri.reston.va.us> from Guido van Rossum at "Feb 24, 2000 11: 5:12 am" Message-ID: Hi! Guido van Rossum wrote: > The question now arises: how is one supposed to use GMP without > knowing the value of mp_bits_per_limb? Or, if this is truly a detail > of the implementation, how come mpzmodule.c needs to know the value? > > These questions need to be answered before we can know the true fix > for this problem! It has taken me some time to dig through a bunch of old CD-Roms. GMP 2.0.2 seems to be around rather long (at least since June 1996). The only older version I could find, was a May 21 1993 gmp-1.3.2.tar.gz, which didn't contain an 'gmp-mparam.h' file either. At least from 2.0.2 on the 'gmp.h' file contains the following lines: #define __GNU_MP__ 2 [...] extern __gmp_const int mp_bits_per_limb; [...] #define __GNU_MP_VERSION_MINOR 0 So Andrew was right and IMHO the best way to change mpzmodule.c is: #if defined __GNU_MP__ && __GNU_MP__ >= 2 #define BITS_PER_MP_LIMB mp_bits_per_limb #else #include "gmp-mparam.h" #endif sorry again for this confusion and best regards, Peter. -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 From akuchlin@mems-exchange.org Thu Feb 24 19:00:38 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Thu, 24 Feb 2000 14:00:38 -0500 (EST) Subject: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: References: <200002241605.LAA09851@eric.cnri.reston.va.us> Message-ID: <14517.32726.162776.374880@amarok.cnri.reston.va.us> Peter Funk writes: >Hi! >1996). The only older version I could find, was a May 21 1993 >gmp-1.3.2.tar.gz, which didn't contain an 'gmp-mparam.h' file either. Verified. I expect mpzmodule.c is very old; the copy in Python 1.2 from 1995 is substantially the same as the one in 1.5, the changes being mostly API renamings (INCREF -> Py_INCREF, object -> PyObject, etc.). Presumably it was written when GMP 1.3.2 was still widely deployed, and therefore it contains compatibility code for both versions. Now that it's 2000, we can safely assume everyone will use GMP 2, and the compatibility code could be dropped. (Though GvR might be inclined to just drop the module...) > #if defined __GNU_MP__ && __GNU_MP__ >= 2 > #define BITS_PER_MP_LIMB mp_bits_per_limb > #else > #include "gmp-mparam.h" > #endif Guido, want me to check this fix in, after I've actually tried it out? -- A.M. Kuchling http://starship.python.net/crew/amk/ "I thought you could foretell the future?" "I don't need to know the future. When the future's over, then it's me..." -- Orpheus and Death, in SANDMAN: "The Song of Orpheus" From guido@python.org Thu Feb 24 19:39:21 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Feb 2000 14:39:21 -0500 Subject: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: Your message of "Thu, 24 Feb 2000 14:00:38 EST." <14517.32726.162776.374880@amarok.cnri.reston.va.us> References: <200002241605.LAA09851@eric.cnri.reston.va.us> <14517.32726.162776.374880@amarok.cnri.reston.va.us> Message-ID: <200002241939.OAA10216@eric.cnri.reston.va.us> > > #if defined __GNU_MP__ && __GNU_MP__ >= 2 > > #define BITS_PER_MP_LIMB mp_bits_per_limb > > #else > > #include "gmp-mparam.h" > > #endif > > Guido, want me to check this fix in, after I've actually tried it out? Go ahead, Andrew! --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 24 20:13:03 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 24 Feb 2000 15:13:03 -0500 (EST) Subject: [Patches] Tkinter OptionMenu callback Message-ID: <14517.37071.520026.337312@anthem.cnri.reston.va.us> On 02-Apr-1999, Randal Hooper posted a question to the python-list about hooking a callback into a Tkinter OptionMenu, so that when some choice is selected, the callback is called. I've found that we need the same functionality. In this URL, Randall posts his patch: http://www.deja.com/[ST_rn=ps]/getdoc.xp?AN=462053022 I've slightly modified this so that the callback gets the value as the first argument; here's an example usage: -------------------- snip snip -------------------- from Tkinter import * import sys def quit(value, args=None): if value == 'quit': sys.exit(0) tk = Tk() svar = StringVar() svar.set('5x') f = Frame(tk) om = OptionMenu(f, svar, '5x', '10x', '20x', 'quit', command=quit) om.pack() f.pack() tk.mainloop() -------------------- snip snip -------------------- And here is my version of Randall's patch. What say ye, Tkinter experts? -Barry Index: Tkinter.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/lib-tk/Tkinter.py,v retrieving revision 1.134 diff -c -r1.134 Tkinter.py *** Tkinter.py 1999/10/20 12:29:56 1.134 --- Tkinter.py 2000/02/24 20:07:13 *************** *** 1790,1813 **** self.tk.call((self._w, 'yview', '-pickplace') + what) class _setit: ! def __init__(self, var, value): self.__value = value self.__var = var def __call__(self, *args): self.__var.set(self.__value) class OptionMenu(Menubutton): ! def __init__(self, master, variable, value, *values): ! kw = {"borderwidth": 2, "textvariable": variable, ! "indicatoron": 1, "relief": RAISED, "anchor": "c", ! "highlightthickness": 2} ! Widget.__init__(self, master, "menubutton", kw) self.widgetName = 'tk_optionMenu' menu = self.__menu = Menu(self, name="menu", tearoff=0) self.menuname = menu._w ! menu.add_command(label=value, command=_setit(variable, value)) for v in values: ! menu.add_command(label=v, command=_setit(variable, v)) self["menu"] = menu def __getitem__(self, name): --- 1790,1819 ---- self.tk.call((self._w, 'yview', '-pickplace') + what) class _setit: ! def __init__(self, var, value, callback=None): self.__value = value self.__var = var + self.__callback = callback def __call__(self, *args): + if self.__callback: + apply(self.__callback, (self.__value,)+args) self.__var.set(self.__value) class OptionMenu(Menubutton): ! def __init__(self, master, variable, value, *values, **kw): ! basekw = {"borderwidth": 2, "textvariable": variable, ! "indicatoron": 1, "relief": RAISED, "anchor": "c", ! "highlightthickness": 2} ! Widget.__init__(self, master, "menubutton", basekw) self.widgetName = 'tk_optionMenu' menu = self.__menu = Menu(self, name="menu", tearoff=0) self.menuname = menu._w ! callback = kw.get('command') ! menu.add_command(label=value, ! command=_setit(variable, value, callback)) for v in values: ! menu.add_command(label=v, ! command=_setit(variable, v, callback)) self["menu"] = menu def __getitem__(self, name): From guido@python.org Thu Feb 24 20:30:27 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Feb 2000 15:30:27 -0500 Subject: [Patches] Tkinter OptionMenu callback In-Reply-To: Your message of "Thu, 24 Feb 2000 15:13:03 EST." <14517.37071.520026.337312@anthem.cnri.reston.va.us> References: <14517.37071.520026.337312@anthem.cnri.reston.va.us> Message-ID: <200002242030.PAA10317@eric.cnri.reston.va.us> Nice, it seems that Pmw's OptionMenu has the same functionality. Two comments on the patch: (1) Shouldn't the _setit() class set the variable before calling the callback? I think that makes more sense; the callback might want to call something else that gets the new value from the Tcl variable rather than using the callback argument. (2) You should check that the only kw arg accepted is 'command'. The current code silently ignores other kw args, which is very un-Tk-ish. (I would also undo the rename of the 'kw' dict -- now it took me a while to figure out why the first 4 lines of the body of the __init__ were changed. Give the ** argument a different name, since it's totally new; e.g. **kwargs. I noticed that you didn't include the legal boilerplate <2.0 wink>. --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@CNRI.Reston.VA.US Thu Feb 24 20:48:07 2000 From: bwarsaw@CNRI.Reston.VA.US (bwarsaw@CNRI.Reston.VA.US) Date: Thu, 24 Feb 2000 15:48:07 -0500 (EST) Subject: [Patches] Tkinter OptionMenu callback References: <14517.37071.520026.337312@anthem.cnri.reston.va.us> <200002242030.PAA10317@eric.cnri.reston.va.us> Message-ID: <14517.39175.686685.428102@anthem.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> Nice, it seems that Pmw's OptionMenu has the same GvR> functionality. Exactly my motivation! I didn't want to pull in all of Pmw just for this useful feature (well, I didn't want Andrew to have to do that :). GvR> (1) Shouldn't the _setit() class set the variable before GvR> calling the callback? Yes. GvR> (2) You should check that the only kw arg accepted is GvR> 'command'. The current code silently ignores other kw args, GvR> which is very un-Tk-ish. Done, but in a slightly kludgy way, since Tk never sees this option. See what you think about this hack. GvR> (I would also undo the rename of the 'kw' dict Agreed. One other thing. I might argue that there should be a `configure' method that allows you to change the callback, but I on the other hand, it's probably not important enough to add.a GvR> I noticed that you didn't include the legal boilerplate <2.0 GvR> wink>. Well /I/ shouldn't need to do that, and Randall didn't include the boilerplate on his posted patch (from which mine was derived). Hmm, maybe we should ask Randall to add the boilerplate to his original patch, and submit that to patches@python.org? On the other hand, maybe the patch below looks different enough from Randall's that his disclaimer isn't necessary (although we'll still give due credit of course)? -Barry -------------------- snip snip -------------------- Index: Tkinter.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/lib-tk/Tkinter.py,v retrieving revision 1.134 diff -c -r1.134 Tkinter.py *** Tkinter.py 1999/10/20 12:29:56 1.134 --- Tkinter.py 2000/02/24 20:39:27 *************** *** 1790,1803 **** self.tk.call((self._w, 'yview', '-pickplace') + what) class _setit: ! def __init__(self, var, value): self.__value = value self.__var = var def __call__(self, *args): self.__var.set(self.__value) class OptionMenu(Menubutton): ! def __init__(self, master, variable, value, *values): kw = {"borderwidth": 2, "textvariable": variable, "indicatoron": 1, "relief": RAISED, "anchor": "c", "highlightthickness": 2} --- 1790,1806 ---- self.tk.call((self._w, 'yview', '-pickplace') + what) class _setit: ! def __init__(self, var, value, callback=None): self.__value = value self.__var = var + self.__callback = callback def __call__(self, *args): self.__var.set(self.__value) + if self.__callback: + apply(self.__callback, (self.__value,)+args) class OptionMenu(Menubutton): ! def __init__(self, master, variable, value, *values, **kwargs): kw = {"borderwidth": 2, "textvariable": variable, "indicatoron": 1, "relief": RAISED, "anchor": "c", "highlightthickness": 2} *************** *** 1805,1813 **** self.widgetName = 'tk_optionMenu' menu = self.__menu = Menu(self, name="menu", tearoff=0) self.menuname = menu._w ! menu.add_command(label=value, command=_setit(variable, value)) for v in values: ! menu.add_command(label=v, command=_setit(variable, v)) self["menu"] = menu def __getitem__(self, name): --- 1808,1824 ---- self.widgetName = 'tk_optionMenu' menu = self.__menu = Menu(self, name="menu", tearoff=0) self.menuname = menu._w ! # 'command' is the only supported keyword ! callback = kwargs.get('command') ! if callback: ! del kwargs['command'] ! if kwargs: ! raise TclError, 'unknown option -'+kwargs.keys()[0] ! menu.add_command(label=value, ! command=_setit(variable, value, callback)) for v in values: ! menu.add_command(label=v, ! command=_setit(variable, v, callback)) self["menu"] = menu def __getitem__(self, name): From guido@python.org Thu Feb 24 20:54:20 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Feb 2000 15:54:20 -0500 Subject: [Patches] Tkinter OptionMenu callback In-Reply-To: Your message of "Thu, 24 Feb 2000 15:48:07 EST." <14517.39175.686685.428102@anthem.cnri.reston.va.us> References: <14517.37071.520026.337312@anthem.cnri.reston.va.us> <200002242030.PAA10317@eric.cnri.reston.va.us> <14517.39175.686685.428102@anthem.cnri.reston.va.us> Message-ID: <200002242054.PAA10389@eric.cnri.reston.va.us> > Done, but in a slightly kludgy way, since Tk never sees this option. > See what you think about this hack. I would use has_key(). Try what happens when you pass command=0 or command=""! > One other thing. I might argue that there should be a `configure' > method that allows you to change the callback, but I on the other > hand, it's probably not important enough to add. No need. > GvR> I noticed that you didn't include the legal boilerplate <2.0 > GvR> wink>. > > Well /I/ shouldn't need to do that, and Randall didn't include the > boilerplate on his posted patch (from which mine was derived). Hmm, > maybe we should ask Randall to add the boilerplate to his original > patch, and submit that to patches@python.org? On the other hand, > maybe the patch below looks different enough from Randall's that his > disclaimer isn't necessary (although we'll still give due credit of > course)? I haven't seen Randall's patch -- is it different enough? --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@CNRI.Reston.VA.US Thu Feb 24 21:06:20 2000 From: bwarsaw@CNRI.Reston.VA.US (bwarsaw@CNRI.Reston.VA.US) Date: Thu, 24 Feb 2000 16:06:20 -0500 (EST) Subject: [Patches] Tkinter OptionMenu callback References: <14517.37071.520026.337312@anthem.cnri.reston.va.us> <200002242030.PAA10317@eric.cnri.reston.va.us> <14517.39175.686685.428102@anthem.cnri.reston.va.us> <200002242054.PAA10389@eric.cnri.reston.va.us> Message-ID: <14517.40268.274547.341419@anthem.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> I would use has_key(). Try what happens when you pass GvR> command=0 or command=""! That's what I get for rushing :) You're right of course. BAW> > One other thing. I might argue that there should be a BAW> `configure' > method that allows you to change the callback, BAW> but I on the other > hand, it's probably not important enough BAW> to add. GvR> No need. Cool. GvR> I haven't seen Randall's patch -- is it different enough? Define "enough" :) Clearly the spirit is all there. The patch is fairly small anyway, so if I had my GNU hat on and was adding this to a GNU project, it would probably fall under the disclaimer radar. YMMV. -Barry From pat@ktgroup.co.uk Fri Feb 25 11:22:47 2000 From: pat@ktgroup.co.uk (Pat Knight) Date: Fri, 25 Feb 2000 11:22:47 +0000 Subject: [Patches] configure.in patch to use POSIX threads on Solaris Message-ID: <4.2.2.20000225110217.00cbc1c0@vcsmail> --=====================_7950082==_ Content-Type: text/plain; charset="us-ascii"; format=flowed Purpose of patch. Solaris 2 has stub implementations of the POSIX thread functions such as pthread_detach in libc. This means that configure tries to use them without -lpthread, then the test of pthread_create fails and the configuration falls back to the Solaris thread library. This patch moves the test for pthread_create in -lpthread ahead of the test for pthread_detach in libc. The patch also ensures that -lpthread is at the start of the library list when linking, to pick up POSIX thread semantics for fork (see below). Justification. Use of POSIX threads on Solaris ensures that the fork() call only runs the thread that called fork() in the child. This is desirable to prevent (for example) parent server or database threads running in the child. Sun's -lthread library uses a traditional fork() which replicates all the parent's threads in the child. I find this undesirable. If the "patch cabal" :-) decide that they want to continue the current thread/fork() semantics on Solaris, I'll be happy to produce an optional module to expose the Solaris fork1() system call. fork1() gives the POSIX fork() semantics when using -lthread. I ran a cvs update immediately prior to producing this patch so it's against the state of the source tree at around 9:40 GMT today. The patch assumes the current directory is ...python/dist/src. Digression. The configure.in seems to always test for -lthread even if a POSIX library is found. I'm not enough of a configure.in wizard to decide whether this is desirable or how to fix it. It is also irrelevant to this patch - I just spotted it while testing. End of Digression. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. --=====================_7950082==_ Content-Type: application/octet-stream; name="thread-patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="thread-patch" KioqIGNvbmZpZ3VyZS5pbn4JRnJpIEZlYiAyNSAxMDowMjoxNyAyMDAwCi0tLSBjb25maWd1cmUu aW4JRnJpIEZlYiAyNSAxMDowMzo0NCAyMDAwCioqKioqKioqKioqKioqKgoqKiogNjQ3LDY1MiAq KioqCi0tLSA2NDcsNjU2IC0tLS0KICBBQ19DSEVDS19IRUFERVIobWFjaC9jdGhyZWFkcy5oLCBb QUNfREVGSU5FKFdJVEhfVEhSRUFEKQogIEFDX0RFRklORShDX1RIUkVBRFMpCiAgTElCT0JKUz0i JExJQk9CSlMgdGhyZWFkLm8iXSxbCisgQUNfQ0hFQ0tfTElCKHB0aHJlYWQsIHB0aHJlYWRfY3Jl YXRlLCBbQUNfREVGSU5FKFdJVEhfVEhSRUFEKQorIEFDX0RFRklORShfUE9TSVhfVEhSRUFEUykK KyBMSUJTPSItbHB0aHJlYWQgJExJQlMiCisgTElCT0JKUz0iJExJQk9CSlMgdGhyZWFkLm8iXSxb CiAgQUNfQ0hFQ0tfRlVOQyhwdGhyZWFkX2RldGFjaCwgW0FDX0RFRklORShXSVRIX1RIUkVBRCkK ICBBQ19ERUZJTkUoX1BPU0lYX1RIUkVBRFMpCiAgTElCT0JKUz0iJExJQk9CSlMgdGhyZWFkLm8i XSxbCioqKioqKioqKioqKioqKgoqKiogNjU2LDY2NSAqKioqCiAgQUNfQ0hFQ0tfTElCKHB0aHJl YWRzLCBwdGhyZWFkX2NyZWF0ZSwgW0FDX0RFRklORShXSVRIX1RIUkVBRCkKICBBQ19ERUZJTkUo X1BPU0lYX1RIUkVBRFMpCiAgTElCUz0iJExJQlMgLWxwdGhyZWFkcyIKLSBMSUJPQkpTPSIkTElC T0JKUyB0aHJlYWQubyJdLCBbCi0gQUNfQ0hFQ0tfTElCKHB0aHJlYWQsIHB0aHJlYWRfY3JlYXRl LCBbQUNfREVGSU5FKFdJVEhfVEhSRUFEKQotIEFDX0RFRklORShfUE9TSVhfVEhSRUFEUykKLSBM SUJTPSIkTElCUyAtbHB0aHJlYWQiCiAgTElCT0JKUz0iJExJQk9CSlMgdGhyZWFkLm8iXSwgWwog IEFDX0NIRUNLX0xJQihjX3IsIHB0aHJlYWRfY3JlYXRlLCBbQUNfREVGSU5FKFdJVEhfVEhSRUFE KQogIEFDX0RFRklORShfUE9TSVhfVEhSRUFEUykKLS0tIDY2MCw2NjUgLS0tLQo= --=====================_7950082==_ Content-Type: text/plain; charset="us-ascii"; format=flowed Pat Knight VCS Development Manager, The Knowledge Group, Bristol, UK +44-1454-281265 Corporate Web: http://www.ktgroup.co.uk, http://www.wideawake.co.uk --=====================_7950082==_-- From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:23:09 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:23:09 +0100 (CET) Subject: [Patches] PyMem [1/8] - intro Message-ID: <200002251223.NAA20710@python.inrialpes.fr> I'm about to send a couple of patches relative to the malloc cleanup discussed on python-dev. Since it affects many files in the distribution, I'll send a patch per directory, which will facilitate their review. Please examine them carefully. Although I'm pretty confident in their correctness, some details may have escaped my eyes due to the volume of the modified code. Summary: ======== Exported (public) interfaces ---------------------------- - PyMem_{Malloc, Realloc, Free} -- don't call anything on failure - Py_{Malloc, Realloc, Free} -- call PyErr_NoMemory() on failure - _PyObject_{New, NewVar, Del} -- object constructors/destructor Macros: - PyMem_{NEW, NEW_VAR, DEL} -- handy, use PyMem_{Malloc...} above - PyObject_{NEW, NEW_VAR, DEL} -- handy, use _PyObject_{New...} above Private core interfaces ----------------------- In addition to the public ones: - PyMem_{MALLOC, REALLOC, FREE} -- Python core allocator Macros: - PyMem_{NEW, NEW_VAR, DEL} -- handy, use PyMem_{MALLOC...} above - PyObject_DEL is inlined (PyMem_FREE) I'll comment further on every submission. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:25:39 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:25:39 +0100 (CET) Subject: [Patches] PyMem [2/8] - Include/* Message-ID: <200002251225.NAA20734@python.inrialpes.fr> A new header (pycore.h) defines the private interfaces and is included only by the files belonging to the core (basically *.c in Parser/, Python/ and Objects/). It redefines the macros NEW/DEL defined in the public interface (mymalloc.h) so that they call the core allocator directly. A new public function _PyObject_Del and a macro PyObject_DEL are exported by objimpl.h (the missing halves of _PyObject_New, PyObject_NEW). Subsequently, all objects allocated with PyObject_NEW are released with PyObject_DEL (even when _PyObject_New is inlined in the object's constructor for speed -- common situation for the core objects implementations). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- diff -cr PyCVS/Include/mymalloc.h PyMem/Include/mymalloc.h *** PyCVS/Include/mymalloc.h Thu Feb 24 14:49:33 2000 --- PyMem/Include/mymalloc.h Thu Feb 24 17:29:41 2000 *************** *** 67,109 **** #pragma lib_export on #endif - /* The following should never be necessary */ - #ifdef NEED_TO_DECLARE_MALLOC_AND_FRIEND - extern ANY *malloc Py_PROTO((size_t)); - extern ANY *calloc Py_PROTO((size_t, size_t)); - extern ANY *realloc Py_PROTO((ANY *, size_t)); - extern void free Py_PROTO((ANY *)); /* XXX sometimes int on Unix old systems */ - #endif - #ifndef NULL #define NULL ((ANY *)0) #endif - #ifdef MALLOC_ZERO_RETURNS_NULL - /* XXX Always allocate one extra byte, since some malloc's return NULL - XXX for malloc(0) or realloc(p, 0). */ - #define _PyMem_EXTRA 1 - #else - #define _PyMem_EXTRA 0 - #endif - - #define PyMem_NEW(type, n) \ - ( (type *) malloc(_PyMem_EXTRA + (n) * sizeof(type)) ) - #define PyMem_RESIZE(p, type, n) \ - if ((p) == NULL) \ - (p) = (type *) malloc(_PyMem_EXTRA + (n) * sizeof(type)); \ - else \ - (p) = (type *) realloc((ANY *)(p), \ - _PyMem_EXTRA + (n) * sizeof(type)) - #define PyMem_DEL(p) free((ANY *)p) - #define PyMem_XDEL(p) if ((p) == NULL) ; else PyMem_DEL(p) - - /* Two sets of function wrappers around malloc and friends; useful if you need to be sure that you are using the same memory allocator as Python. Note that the wrappers make sure that allocating 0 bytes ! returns a non-NULL pointer, even if the underlying malloc doesn't. ! The Python interpreter continues to use PyMem_NEW etc. */ /* These wrappers around malloc call PyErr_NoMemory() on failure */ extern DL_IMPORT(ANY *) Py_Malloc Py_PROTO((size_t)); --- 67,80 ---- #pragma lib_export on #endif #ifndef NULL #define NULL ((ANY *)0) #endif /* Two sets of function wrappers around malloc and friends; useful if you need to be sure that you are using the same memory allocator as Python. Note that the wrappers make sure that allocating 0 bytes ! returns a non-NULL pointer, even if the underlying malloc doesn't. */ /* These wrappers around malloc call PyErr_NoMemory() on failure */ extern DL_IMPORT(ANY *) Py_Malloc Py_PROTO((size_t)); *************** *** 114,119 **** --- 85,108 ---- extern DL_IMPORT(ANY *) PyMem_Malloc Py_PROTO((size_t)); extern DL_IMPORT(ANY *) PyMem_Realloc Py_PROTO((ANY *, size_t)); extern DL_IMPORT(void) PyMem_Free Py_PROTO((ANY *)); + + + /* Type-oriented memory interface. */ + + #define PyMem_NEW(type, n) \ + ( (type *) PyMem_Malloc((n) * sizeof(type)) ) + #define PyMem_RESIZE(p, type, n) \ + if ((p) == NULL) \ + (p) = (type *) PyMem_Malloc((n) * sizeof(type)); \ + else \ + (p) = (type *) PyMem_Realloc((ANY *)(p), (n) * sizeof(type)) + #define PyMem_DEL(p) PyMem_Free((ANY *)(p)) + + /* PyMem_XDEL is deprecated. + If you want to avoid the call to PyMem_DEL(p) when p is NULL, + it's recommended to write the test explicitly in the code. */ + + #define PyMem_XDEL(p) if ((p) == NULL) ; else PyMem_DEL(p) #ifdef __cplusplus } diff -cr PyCVS/Include/objimpl.h PyMem/Include/objimpl.h *** PyCVS/Include/objimpl.h Thu Feb 24 14:49:33 2000 --- PyMem/Include/objimpl.h Thu Feb 24 18:27:35 2000 *************** *** 74,79 **** --- 74,82 ---- #endif /* MS_COREDLL */ + extern DL_IMPORT(void) _PyObject_Del Py_PROTO((PyObject *)); + #define PyObject_DEL(op) _PyObject_Del((PyObject *)(op)) + #ifdef __cplusplus } #endif Only in PyMem/Include/: pycore.h From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:26:46 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:26:46 +0100 (CET) Subject: [Patches] PyMem [3/8] - Include/pycore.h Message-ID: <200002251226.NAA20760@python.inrialpes.fr> The new header pycore.h I'm not sure I fully understand the business with MS_COREDLL for Windows. Could somebody explain it to me? When & why exactly is this symbol defined? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- #ifndef Py_PYCORE_H #define Py_PYCORE_H /*********************************************************** Copyright 1991-1995 by Stichting Mathematisch Centrum, Amsterdam, The Netherlands. All Rights Reserved Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the names of Stichting Mathematisch Centrum or CWI or Corporation for National Research Initiatives or CNRI not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. While CWI is the initial source for this software, a modified version is made available by the Corporation for National Research Initiatives (CNRI) at the Internet address ftp://ftp.python.org. STICHTING MATHEMATISCH CENTRUM AND CNRI DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM OR CNRI BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ******************************************************************/ /* Private core interfaces. BEWARE: pycore.h should be included only in files which belong to the Python core implementation. Never include this file in an extension module! */ #include "patchlevel.h" #include "config.h" /* config.h may or may not define DL_IMPORT */ #ifndef DL_IMPORT /* declarations for DLL import/export */ #define DL_IMPORT(RTYPE) RTYPE #endif #ifndef DL_EXPORT /* declarations for DLL import/export */ #define DL_EXPORT(RTYPE) RTYPE #endif #ifdef SYMANTEC__CFM68K__ #define UsingSharedLibs #endif #if defined(__sgi) && defined(WITH_THREAD) && !defined(_SGI_MP_SOURCE) #define _SGI_MP_SOURCE #endif #include #include #include #ifdef HAVE_STDLIB_H #include #endif #include "myproto.h" /* Core memory allocator ===================== To make sure the interpreter is user-malloc friendly, all other memory or object core APIs are implemented on top of this one. These can be changed to make the interpreter use another allocator. The easiest way to change the core allocator *and* to avoid the wrapper overhead for extension modules that call PyMem_{Malloc, Realloc, Free} is to proceed as follows: 1) Replace the implementation of PyMem_{Malloc, Realloc, Free} in mymalloc.c with the new allocator. 2) Redefine hereafter PyCore_{MALLOC, REALLOC, FREE} as PyMem_{Malloc, Realloc, Free}. XXX These should probably move to configure. */ #define PyCore_MALLOC malloc #define PyCore_REALLOC realloc #define PyCore_FREE free /* We hereby redefine the type-oriented macros in mymalloc.h so that the Python core can use its own malloc functions. */ #include "mymalloc.h" /* for ANY and the public interfaces */ #undef PyMem_NEW #undef PyMem_RESIZE #undef PyMem_DEL #undef PyMem_XDEL /* deprecated */ /* Raw memory interface. */ #ifdef __cplusplus extern "C" { #endif #ifdef NEED_TO_DECLARE_MALLOC_AND_FRIEND extern ANY *PyCore_MALLOC Py_PROTO((size_t)); extern ANY *PyCore_REALLOC Py_PROTO((ANY *, size_t)); extern void PyCore_FREE Py_PROTO((ANY *)); #endif #ifdef __cplusplus } #endif #define PyMem_MALLOC(n) PyCore_MALLOC(n) #define PyMem_REALLOC(p, n) PyCore_REALLOC((ANY *)(p), (n)) #define PyMem_FREE(p) PyCore_FREE((ANY *)(p)) /* Type-oriented memory interface. */ #ifdef MALLOC_ZERO_RETURNS_NULL /* XXX Always allocate one extra byte, since some malloc's return NULL XXX for malloc(0) or realloc(p, 0). */ #define _PyMem_EXTRA 1 #else #define _PyMem_EXTRA 0 #endif #define PyMem_NEW(type, n) \ ( (type *) PyMem_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)) ) #define PyMem_RESIZE(p, type, n) \ if ((p) == NULL) \ (p) = (type *) PyMem_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)); \ else \ (p) = (type *) PyMem_REALLOC((p), \ _PyMem_EXTRA + (n) * sizeof(type)) #define PyMem_DEL(p) PyMem_FREE(p) /* Core object allocator ===================== PyObject_NEW/NEW_VAR are left unchanged. They call _PyObject_New/NewVar. PyObject_DEL is inlined to avoid the overhead of calling _PyObject_Del. */ #include "object.h" #include "objimpl.h" #undef PyObject_DEL #ifndef MS_COREDLL #define PyObject_DEL(op) PyMem_FREE(op) #else #define PyObject_DEL(op) free((ANY *)(op)) #endif #endif /* !Py_PYCORE_H */ From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:28:38 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:28:38 +0100 (CET) Subject: [Patches] PyMem [4/8] - Parser/* Message-ID: <200002251228.NAA20782@python.inrialpes.fr> - PyMem_{Malloc, Realloc, Free} are put in a new separate file "mymalloc.c" which *is* the Python core malloc implementation. It's in the directory Parser/ since it is used by pgen. - The files which import "pycore.h" are essentially those that belong to the Python library (libpython.a) -- i.e. AROBJS in Makefile.in - Cleanup w.r.t. various mallocs. - Added a comment about PyOS_Readline. If the input comes from somebody else (via the readline hook), we expect a Python memory buffer, because this buffer is freed subsequently by Python. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- diff -cr PyCVS/Parser/Makefile.in PyMem/Parser/Makefile.in *** PyCVS/Parser/Makefile.in Thu Feb 24 14:49:47 2000 --- PyMem/Parser/Makefile.in Thu Feb 24 17:19:17 2000 *************** *** 32,38 **** POBJS= acceler.o grammar1.o \ listnode.o node.o parser.o \ parsetok.o tokenizer.o bitset.o \ ! metagrammar.o AROBJS= $(POBJS) myreadline.o OBJS= $(AROBJS) intrcheck.o --- 32,38 ---- POBJS= acceler.o grammar1.o \ listnode.o node.o parser.o \ parsetok.o tokenizer.o bitset.o \ ! metagrammar.o mymalloc.o AROBJS= $(POBJS) myreadline.o OBJS= $(AROBJS) intrcheck.o *************** *** 85,90 **** --- 85,91 ---- grammar1.o: grammar1.c intrcheck.o: intrcheck.c listnode.o: listnode.c + mymalloc.o: mymalloc.c myreadline.o: myreadline.c node.o: node.c parser.o: parser.c diff -cr PyCVS/Parser/acceler.c PyMem/Parser/acceler.c *** PyCVS/Parser/acceler.c Thu Feb 24 14:49:47 2000 --- PyMem/Parser/acceler.c Thu Feb 24 16:31:20 2000 *************** *** 40,45 **** --- 40,46 ---- are not part of the static data structure written on graminit.[ch] by the parser generator. */ + #include "pycore.h" #include "pgenheaders.h" #include "grammar.h" #include "node.h" diff -cr PyCVS/Parser/bitset.c PyMem/Parser/bitset.c *** PyCVS/Parser/bitset.c Thu Feb 24 14:49:47 2000 --- PyMem/Parser/bitset.c Thu Feb 24 16:34:49 2000 *************** *** 31,36 **** --- 31,37 ---- /* Bitset primitives used by the parser generator */ + #include "pycore.h" #include "pgenheaders.h" #include "bitset.h" diff -cr PyCVS/Parser/grammar1.c PyMem/Parser/grammar1.c *** PyCVS/Parser/grammar1.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/grammar1.c Fri Feb 25 08:46:17 2000 *************** *** 31,36 **** --- 31,37 ---- /* Grammar subroutines needed by parser */ + #include "pycore.h" #include "pgenheaders.h" #include "assert.h" #include "grammar.h" diff -cr PyCVS/Parser/listnode.c PyMem/Parser/listnode.c *** PyCVS/Parser/listnode.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/listnode.c Fri Feb 25 08:46:46 2000 *************** *** 31,36 **** --- 31,37 ---- /* List a node on a file */ + #include "pycore.h" #include "pgenheaders.h" #include "token.h" #include "node.h" Only in PyMem/Parser/: mymalloc.c diff -cr PyCVS/Parser/myreadline.c PyMem/Parser/myreadline.c *** PyCVS/Parser/myreadline.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/myreadline.c Thu Feb 24 18:37:41 2000 *************** *** 40,45 **** --- 40,46 ---- */ #include "Python.h" + #include "pycore.h" int (*PyOS_InputHook)() = NULL; *************** *** 89,95 **** int n; char *p; n = 100; ! if ((p = malloc(n)) == NULL) return NULL; fflush(stdout); if (prompt) --- 90,96 ---- int n; char *p; n = 100; ! if ((p = PyMem_MALLOC(n)) == NULL) return NULL; fflush(stdout); if (prompt) *************** *** 99,105 **** case 0: /* Normal case */ break; case 1: /* Interrupt */ ! free(p); return NULL; case -1: /* EOF */ case -2: /* Error */ --- 100,106 ---- case 0: /* Normal case */ break; case 1: /* Interrupt */ ! PyMem_FREE(p); return NULL; case -1: /* EOF */ case -2: /* Error */ *************** *** 117,135 **** n = strlen(p); while (n > 0 && p[n-1] != '\n') { int incr = n+2; ! p = realloc(p, n + incr); if (p == NULL) return NULL; if (my_fgets(p+n, incr, stdin) != 0) break; n += strlen(p+n); } ! return realloc(p, n+1); } /* By initializing this function pointer, systems embedding Python can ! override the readline function. */ char *(*PyOS_ReadlineFunctionPointer) Py_PROTO((char *)); --- 118,138 ---- n = strlen(p); while (n > 0 && p[n-1] != '\n') { int incr = n+2; ! p = PyMem_REALLOC(p, n + incr); if (p == NULL) return NULL; if (my_fgets(p+n, incr, stdin) != 0) break; n += strlen(p+n); } ! return PyMem_REALLOC(p, n+1); } /* By initializing this function pointer, systems embedding Python can ! override the readline function. ! ! Note: Python expects in return a buffer allocated with PyMem_Malloc. */ char *(*PyOS_ReadlineFunctionPointer) Py_PROTO((char *)); diff -cr PyCVS/Parser/node.c PyMem/Parser/node.c *** PyCVS/Parser/node.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/node.c Thu Feb 24 16:31:59 2000 *************** *** 31,36 **** --- 31,37 ---- /* Parse tree node implementation */ + #include "pycore.h" #include "pgenheaders.h" #include "node.h" diff -cr PyCVS/Parser/parser.c PyMem/Parser/parser.c *** PyCVS/Parser/parser.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/parser.c Thu Feb 24 16:32:44 2000 *************** *** 35,40 **** --- 35,41 ---- /* XXX To do: error recovery */ + #include "pycore.h" #include "pgenheaders.h" #include "assert.h" #include "token.h" diff -cr PyCVS/Parser/parsetok.c PyMem/Parser/parsetok.c *** PyCVS/Parser/parsetok.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/parsetok.c Fri Feb 25 07:40:36 2000 *************** *** 31,36 **** --- 31,37 ---- /* Parser-tokenizer link implementation */ + #include "pycore.h" #include "pgenheaders.h" #include "tokenizer.h" #include "node.h" *************** *** 192,198 **** err_ret->offset = tok->cur - tok->buf; if (tok->buf != NULL) { int len = tok->inp - tok->buf; ! err_ret->text = malloc(len + 1); if (err_ret->text != NULL) { if (len > 0) strncpy(err_ret->text, tok->buf, len); --- 193,199 ---- err_ret->offset = tok->cur - tok->buf; if (tok->buf != NULL) { int len = tok->inp - tok->buf; ! err_ret->text = PyMem_NEW(char, len + 1); if (err_ret->text != NULL) { if (len > 0) strncpy(err_ret->text, tok->buf, len); diff -cr PyCVS/Parser/pgenmain.c PyMem/Parser/pgenmain.c *** PyCVS/Parser/pgenmain.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/pgenmain.c Fri Feb 25 07:41:32 2000 *************** *** 139,145 **** putc(' ', stderr); } fprintf(stderr, "^\n"); ! free(err.text); } Py_Exit(1); } --- 139,145 ---- putc(' ', stderr); } fprintf(stderr, "^\n"); ! PyMem_DEL(err.text); } Py_Exit(1); } *************** *** 196,202 **** char *prompt; { int n = 1000; ! char *p = malloc(n); char *q; if (p == NULL) return NULL; --- 196,202 ---- char *prompt; { int n = 1000; ! char *p = PyMem_Malloc(n); char *q; if (p == NULL) return NULL; *************** *** 209,215 **** n = strlen(p); if (n > 0 && p[n-1] != '\n') p[n-1] = '\n'; ! return realloc(p, n+1); } #ifdef HAVE_STDARG_PROTOTYPES --- 209,215 ---- n = strlen(p); if (n > 0 && p[n-1] != '\n') p[n-1] = '\n'; ! return PyMem_Realloc(p, n+1); } #ifdef HAVE_STDARG_PROTOTYPES diff -cr PyCVS/Parser/tokenizer.c PyMem/Parser/tokenizer.c *** PyCVS/Parser/tokenizer.c Thu Feb 24 14:49:48 2000 --- PyMem/Parser/tokenizer.c Thu Feb 24 18:52:41 2000 *************** *** 31,36 **** --- 31,37 ---- /* Tokenizer implementation */ + #include "pycore.h" #include "pgenheaders.h" #include *************** *** 219,244 **** if (new == NULL) tok->done = E_INTR; else if (*new == '\0') { ! free(new); tok->done = E_EOF; } else if (tok->start != NULL) { int start = tok->start - tok->buf; int oldlen = tok->cur - tok->buf; int newlen = oldlen + strlen(new); ! char *buf = realloc(tok->buf, newlen+1); tok->lineno++; if (buf == NULL) { ! free(tok->buf); tok->buf = NULL; ! free(new); tok->done = E_NOMEM; return EOF; } tok->buf = buf; tok->cur = tok->buf + oldlen; strcpy(tok->buf + oldlen, new); ! free(new); tok->inp = tok->buf + newlen; tok->end = tok->inp + 1; tok->start = tok->buf + start; --- 220,246 ---- if (new == NULL) tok->done = E_INTR; else if (*new == '\0') { ! PyMem_FREE(new); tok->done = E_EOF; } else if (tok->start != NULL) { int start = tok->start - tok->buf; int oldlen = tok->cur - tok->buf; int newlen = oldlen + strlen(new); ! char *buf = tok->buf; ! PyMem_RESIZE(buf, char, newlen+1); tok->lineno++; if (buf == NULL) { ! PyMem_DEL(tok->buf); tok->buf = NULL; ! PyMem_FREE(new); tok->done = E_NOMEM; return EOF; } tok->buf = buf; tok->cur = tok->buf + oldlen; strcpy(tok->buf + oldlen, new); ! PyMem_FREE(new); tok->inp = tok->buf + newlen; tok->end = tok->inp + 1; tok->start = tok->buf + start; *************** *** 246,252 **** else { tok->lineno++; if (tok->buf != NULL) ! free(tok->buf); tok->buf = new; tok->cur = tok->buf; tok->inp = strchr(tok->buf, '\0'); --- 248,254 ---- else { tok->lineno++; if (tok->buf != NULL) ! PyMem_DEL(tok->buf); tok->buf = new; tok->cur = tok->buf; tok->inp = strchr(tok->buf, '\0'); From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:29:49 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:29:49 +0100 (CET) Subject: [Patches] PyMem [5/8] - Parser/mymalloc.c Message-ID: <200002251229.NAA20804@python.inrialpes.fr> The new file Parser/mymalloc.c For now, Python's default core allocator is libc malloc (which means that you shouldn't notice anything with these patches... if everything goes well) -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- /*********************************************************** Copyright 1991-1995 by Stichting Mathematisch Centrum, Amsterdam, The Netherlands. All Rights Reserved Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the names of Stichting Mathematisch Centrum or CWI or Corporation for National Research Initiatives or CNRI not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. While CWI is the initial source for this software, a modified version is made available by the Corporation for National Research Initiatives (CNRI) at the Internet address ftp://ftp.python.org. STICHTING MATHEMATISCH CENTRUM AND CNRI DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM OR CNRI BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ******************************************************************/ /* Python's malloc implementation */ #include "pycore.h" ANY * PyMem_Malloc(nbytes) size_t nbytes; { #if _PyMem_EXTRA > 0 if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif return PyMem_MALLOC(nbytes); } ANY * PyMem_Realloc(p, nbytes) ANY *p; size_t nbytes; { #if _PyMem_EXTRA > 0 if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif return PyMem_REALLOC(p, nbytes); } void PyMem_Free(p) ANY *p; { PyMem_FREE(p); } From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:31:23 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:31:23 +0100 (CET) Subject: [Patches] PyMem [6/8] - Python/* Message-ID: <200002251231.NAA20827@python.inrialpes.fr> - Cleanup w.r.t. various mallocs - Bits of Greg's import.c are slightly rewritten for simplicity and beauty - What's left out of scope: a) The dynload_.c files b) The locks in thread_.h These can rest in piece; besides, to be on the safe side, it's probably better to use libc malloc when we deal with system locks, etc. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- diff -cr PyCVS/Python/bltinmodule.c PyMem/Python/bltinmodule.c *** PyCVS/Python/bltinmodule.c Thu Feb 24 14:49:50 2000 --- PyMem/Python/bltinmodule.c Thu Feb 24 19:10:20 2000 *************** *** 32,37 **** --- 32,38 ---- /* Built-in functions */ #include "Python.h" + #include "pycore.h" #include "node.h" #include "compile.h" *************** *** 1753,1759 **** else { /* strip trailing '\n' */ result = PyString_FromStringAndSize(s, strlen(s)-1); } ! free(s); return result; } if (v != NULL) { --- 1754,1760 ---- else { /* strip trailing '\n' */ result = PyString_FromStringAndSize(s, strlen(s)-1); } ! PyMem_FREE(s); return result; } if (v != NULL) { diff -cr PyCVS/Python/ceval.c PyMem/Python/ceval.c *** PyCVS/Python/ceval.c Thu Feb 24 14:49:50 2000 --- PyMem/Python/ceval.c Thu Feb 24 19:32:23 2000 *************** *** 38,43 **** --- 38,44 ---- */ #include "Python.h" + #include "pycore.h" #include "compile.h" #include "frameobject.h" *************** *** 2501,2507 **** class); Py_DECREF(arg); ! PyMem_XDEL(k); return result; } --- 2502,2509 ---- class); Py_DECREF(arg); ! if (k != NULL) ! PyMem_DEL(k); return result; } diff -cr PyCVS/Python/compile.c PyMem/Python/compile.c *** PyCVS/Python/compile.c Thu Feb 24 14:49:50 2000 --- PyMem/Python/compile.c Thu Feb 24 19:30:44 2000 *************** *** 45,50 **** --- 45,51 ---- #endif #include "Python.h" + #include "pycore.h" #include "node.h" #include "token.h" *************** *** 112,118 **** Py_XDECREF(co->co_filename); Py_XDECREF(co->co_name); Py_XDECREF(co->co_lnotab); ! PyMem_DEL(co); } static PyObject * --- 113,119 ---- Py_XDECREF(co->co_filename); Py_XDECREF(co->co_name); Py_XDECREF(co->co_lnotab); ! PyObject_DEL(co); } static PyObject * diff -cr PyCVS/Python/import.c PyMem/Python/import.c *** PyCVS/Python/import.c Thu Feb 24 14:49:50 2000 --- PyMem/Python/import.c Thu Feb 24 19:13:54 2000 *************** *** 32,37 **** --- 32,38 ---- /* Module definition and import implementation */ #include "Python.h" + #include "pycore.h" #include "node.h" #include "token.h" *************** *** 119,125 **** ++countD; for (scan = _PyImport_StandardFiletab; scan->suffix != NULL; ++scan) ++countS; ! filetab = malloc((countD + countS + 1) * sizeof(struct filedescr)); memcpy(filetab, _PyImport_DynLoadFiletab, countD * sizeof(struct filedescr)); memcpy(filetab + countD, _PyImport_StandardFiletab, --- 120,126 ---- ++countD; for (scan = _PyImport_StandardFiletab; scan->suffix != NULL; ++scan) ++countS; ! filetab = PyMem_NEW(struct filedescr, countD + countS + 1); memcpy(filetab, _PyImport_DynLoadFiletab, countD * sizeof(struct filedescr)); memcpy(filetab + countD, _PyImport_StandardFiletab, *************** *** 2410,2419 **** ; /* Allocate new memory for the combined table */ ! if (our_copy == NULL) ! p = malloc((i+n+1) * sizeof(struct _inittab)); ! else ! p = realloc(our_copy, (i+n+1) * sizeof(struct _inittab)); if (p == NULL) return -1; --- 2411,2418 ---- ; /* Allocate new memory for the combined table */ ! p = our_copy; ! PyMem_RESIZE(p, struct _inittab, i+n+1); if (p == NULL) return -1; diff -cr PyCVS/Python/pystate.c PyMem/Python/pystate.c *** PyCVS/Python/pystate.c Thu Feb 24 14:49:50 2000 --- PyMem/Python/pystate.c Thu Feb 24 19:24:57 2000 *************** *** 32,37 **** --- 32,38 ---- /* Thread and interpreter state structures and their interfaces */ #include "Python.h" + #include "pycore.h" #define ZAP(x) { \ PyObject *tmp = (PyObject *)(x); \ diff -cr PyCVS/Python/pythonrun.c PyMem/Python/pythonrun.c *** PyCVS/Python/pythonrun.c Thu Feb 24 14:49:50 2000 --- PyMem/Python/pythonrun.c Fri Feb 25 07:42:21 2000 *************** *** 32,37 **** --- 32,38 ---- /* Python interpreter top-level routines, including init/exit */ #include "Python.h" + #include "pycore.h" #include "grammar.h" #include "node.h" *************** *** 518,524 **** if (n == NULL) { if (err.error == E_EOF) { if (err.text) ! free(err.text); return E_EOF; } err_input(&err); --- 519,525 ---- if (n == NULL) { if (err.error == E_EOF) { if (err.text) ! PyMem_DEL(err.text); return E_EOF; } err_input(&err); *************** *** 984,990 **** v = Py_BuildValue("(ziiz)", err->filename, err->lineno, err->offset, err->text); if (err->text != NULL) { ! free(err->text); err->text = NULL; } switch (err->error) { --- 985,991 ---- v = Py_BuildValue("(ziiz)", err->filename, err->lineno, err->offset, err->text); if (err->text != NULL) { ! PyMem_DEL(err->text); err->text = NULL; } switch (err->error) { diff -cr PyCVS/Python/traceback.c PyMem/Python/traceback.c *** PyCVS/Python/traceback.c Thu Feb 24 14:49:50 2000 --- PyMem/Python/traceback.c Thu Feb 24 19:28:23 2000 *************** *** 32,37 **** --- 32,38 ---- /* Traceback implementation */ #include "Python.h" + #include "pycore.h" #include "compile.h" #include "frameobject.h" *************** *** 70,76 **** { Py_XDECREF(tb->tb_next); Py_XDECREF(tb->tb_frame); ! PyMem_DEL(tb); } #define Tracebacktype PyTraceBack_Type --- 71,77 ---- { Py_XDECREF(tb->tb_next); Py_XDECREF(tb->tb_frame); ! PyObject_DEL(tb); } #define Tracebacktype PyTraceBack_Type From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:33:04 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:33:04 +0100 (CET) Subject: [Patches] PyMem [7/8] - Objects/* Message-ID: <200002251233.NAA20851@python.inrialpes.fr> - General cleanup w.r.t various mallocs - In object.c, added _PyObject_Del and nuked PyMem_{Malloc, Realloc, Free} (MS_COREDLL is still a mystery...) -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- diff -cr PyCVS/Objects/bufferobject.c PyMem/Objects/bufferobject.c *** PyCVS/Objects/bufferobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/bufferobject.c Fri Feb 25 09:10:47 2000 *************** *** 32,38 **** /* Buffer object implementation */ #include "Python.h" ! typedef struct { PyObject_HEAD --- 32,38 ---- /* Buffer object implementation */ #include "Python.h" ! #include "pycore.h" typedef struct { PyObject_HEAD *************** *** 188,194 **** "size must be zero or positive"); return NULL; } ! b = (PyBufferObject *)malloc(sizeof(*b) + size); if ( b == NULL ) return PyErr_NoMemory(); b->ob_type = &PyBuffer_Type; --- 188,194 ---- "size must be zero or positive"); return NULL; } ! b = (PyBufferObject *) PyMem_MALLOC(sizeof(*b) + size); if ( b == NULL ) return PyErr_NoMemory(); b->ob_type = &PyBuffer_Type; *************** *** 212,218 **** PyBufferObject *self; { Py_XDECREF(self->b_base); ! free((void *)self); } static int --- 212,218 ---- PyBufferObject *self; { Py_XDECREF(self->b_base); ! PyObject_DEL(self); } static int diff -cr PyCVS/Objects/classobject.c PyMem/Objects/classobject.c *** PyCVS/Objects/classobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/classobject.c Thu Feb 24 20:18:41 2000 *************** *** 32,37 **** --- 32,39 ---- /* Class object implementation */ #include "Python.h" + #include "pycore.h" + #include "structmember.h" /* Forward */ *************** *** 146,152 **** Py_XDECREF(op->cl_getattr); Py_XDECREF(op->cl_setattr); Py_XDECREF(op->cl_delattr); ! free((ANY *)op); } static PyObject * --- 148,154 ---- Py_XDECREF(op->cl_getattr); Py_XDECREF(op->cl_setattr); Py_XDECREF(op->cl_delattr); ! PyObject_DEL((ANY *)op); } static PyObject * *************** *** 561,567 **** #endif /* Py_TRACE_REFS */ Py_DECREF(inst->in_class); Py_XDECREF(inst->in_dict); ! free((ANY *)inst); } static PyObject * --- 563,569 ---- #endif /* Py_TRACE_REFS */ Py_DECREF(inst->in_class); Py_XDECREF(inst->in_dict); ! PyObject_DEL(inst); } static PyObject * *************** *** 1623,1630 **** PyMethod_Fini() { while (free_list) { ! PyMethodObject *v = free_list; ! free_list = (PyMethodObject *)(v->im_self); ! PyMem_DEL(v); } } --- 1625,1632 ---- PyMethod_Fini() { while (free_list) { ! PyMethodObject *im = free_list; ! free_list = (PyMethodObject *)(im->im_self); ! PyObject_DEL(im); } } diff -cr PyCVS/Objects/cobject.c PyMem/Objects/cobject.c *** PyCVS/Objects/cobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/cobject.c Thu Feb 24 23:49:18 2000 *************** *** 32,38 **** /* Wrap void* pointers to be passed between C modules */ #include "Python.h" ! /* Declarations for objects of type PyCObject */ --- 32,38 ---- /* Wrap void* pointers to be passed between C modules */ #include "Python.h" ! #include "pycore.h" /* Declarations for objects of type PyCObject */ *************** *** 151,157 **** else (self->destructor)(self->cobject); } ! PyMem_DEL(self); } --- 151,157 ---- else (self->destructor)(self->cobject); } ! PyObject_DEL(self); } diff -cr PyCVS/Objects/complexobject.c PyMem/Objects/complexobject.c *** PyCVS/Objects/complexobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/complexobject.c Fri Feb 25 09:07:02 2000 *************** *** 38,43 **** --- 38,45 ---- #ifndef WITHOUT_COMPLEX #include "Python.h" + #include "pycore.h" + #include "mymath.h" #ifdef HAVE_LIMITS_H *************** *** 167,173 **** Py_complex cval; { register PyComplexObject *op = ! (PyComplexObject *) malloc(sizeof(PyComplexObject)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyComplex_Type; --- 169,175 ---- Py_complex cval; { register PyComplexObject *op = ! (PyComplexObject *) PyMem_MALLOC(sizeof(PyComplexObject));; if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyComplex_Type; *************** *** 226,232 **** complex_dealloc(op) PyObject *op; { ! PyMem_DEL(op); } --- 228,234 ---- complex_dealloc(op) PyObject *op; { ! PyObject_DEL(op); } diff -cr PyCVS/Objects/dictobject.c PyMem/Objects/dictobject.c *** PyCVS/Objects/dictobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/dictobject.c Fri Feb 25 00:03:10 2000 *************** *** 32,38 **** /* Dictionary object implementation using a hash table */ #include "Python.h" ! /* * MINSIZE is the minimum size of a dictionary. --- 32,38 ---- /* Dictionary object implementation using a hash table */ #include "Python.h" ! #include "pycore.h" /* * MINSIZE is the minimum size of a dictionary. *************** *** 277,283 **** break; } } ! newtable = (dictentry *) malloc(sizeof(dictentry) * newsize); if (newtable == NULL) { PyErr_NoMemory(); return -1; --- 277,283 ---- break; } } ! newtable = PyMem_NEW(dictentry, newsize); if (newtable == NULL) { PyErr_NoMemory(); return -1; *************** *** 301,307 **** } } ! PyMem_XDEL(oldtable); return 0; } --- 301,308 ---- } } ! if (oldtable != NULL) ! PyMem_DEL(oldtable); return 0; } *************** *** 487,494 **** Py_DECREF(ep->me_value); } } ! PyMem_XDEL(mp->ma_table); ! PyMem_DEL(mp); } static int --- 488,496 ---- Py_DECREF(ep->me_value); } } ! if (mp->ma_table != NULL) ! PyMem_DEL(mp->ma_table); ! PyObject_DEL(mp); } static int diff -cr PyCVS/Objects/fileobject.c PyMem/Objects/fileobject.c *** PyCVS/Objects/fileobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/fileobject.c Fri Feb 25 00:06:33 2000 *************** *** 32,37 **** --- 32,39 ---- /* File object implementation */ #include "Python.h" + #include "pycore.h" + #include "structmember.h" #ifndef DONT_HAVE_SYS_TYPES_H *************** *** 209,215 **** if (f->f_mode != NULL) { Py_DECREF(f->f_mode); } ! free((char *)f); } static PyObject * --- 211,217 ---- if (f->f_mode != NULL) { Py_DECREF(f->f_mode); } ! PyObject_DEL(f); } static PyObject * diff -cr PyCVS/Objects/floatobject.c PyMem/Objects/floatobject.c *** PyCVS/Objects/floatobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/floatobject.c Fri Feb 25 00:10:10 2000 *************** *** 35,40 **** --- 35,41 ---- for any kind of float exception without losing portability. */ #include "Python.h" + #include "pycore.h" #include #include "mymath.h" *************** *** 97,105 **** #define BLOCK_SIZE 1000 /* 1K less typical malloc overhead */ #define BHEAD_SIZE 8 /* Enough for a 64-bit pointer */ #define N_FLOATOBJECTS ((BLOCK_SIZE - BHEAD_SIZE) / sizeof(PyFloatObject)) - - #define PyMem_MALLOC malloc - #define PyMem_FREE free struct _floatblock { struct _floatblock *next; --- 98,103 ---- diff -cr PyCVS/Objects/frameobject.c PyMem/Objects/frameobject.c *** PyCVS/Objects/frameobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/frameobject.c Fri Feb 25 00:15:10 2000 *************** *** 32,37 **** --- 32,38 ---- /* Frame object implementation */ #include "Python.h" + #include "pycore.h" #include "compile.h" #include "frameobject.h" *************** *** 179,186 **** builtins = NULL; if (free_list == NULL) { f = (PyFrameObject *) ! malloc(sizeof(PyFrameObject) + ! extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); f->ob_type = &PyFrame_Type; --- 180,187 ---- builtins = NULL; if (free_list == NULL) { f = (PyFrameObject *) ! PyMem_MALLOC(sizeof(PyFrameObject) + ! extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); f->ob_type = &PyFrame_Type; *************** *** 191,198 **** free_list = free_list->f_back; if (f->f_nlocals + f->f_stacksize < extras) { f = (PyFrameObject *) ! realloc(f, sizeof(PyFrameObject) + ! extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); } --- 192,199 ---- free_list = free_list->f_back; if (f->f_nlocals + f->f_stacksize < extras) { f = (PyFrameObject *) ! PyMem_REALLOC(f, sizeof(PyFrameObject) + ! extras*sizeof(PyObject *)); if (f == NULL) return (PyFrameObject *)PyErr_NoMemory(); } *************** *** 374,379 **** while (free_list != NULL) { PyFrameObject *f = free_list; free_list = free_list->f_back; ! PyMem_DEL(f); } } --- 375,380 ---- while (free_list != NULL) { PyFrameObject *f = free_list; free_list = free_list->f_back; ! PyObject_DEL(f); } } diff -cr PyCVS/Objects/funcobject.c PyMem/Objects/funcobject.c *** PyCVS/Objects/funcobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/funcobject.c Fri Feb 25 00:16:28 2000 *************** *** 32,37 **** --- 32,39 ---- /* Function object implementation */ #include "Python.h" + #include "pycore.h" + #include "compile.h" #include "structmember.h" *************** *** 191,197 **** Py_DECREF(op->func_name); Py_XDECREF(op->func_defaults); Py_XDECREF(op->func_doc); ! PyMem_DEL(op); } static PyObject* --- 193,199 ---- Py_DECREF(op->func_name); Py_XDECREF(op->func_defaults); Py_XDECREF(op->func_doc); ! PyObject_DEL(op); } static PyObject* diff -cr PyCVS/Objects/intobject.c PyMem/Objects/intobject.c *** PyCVS/Objects/intobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/intobject.c Fri Feb 25 00:24:34 2000 *************** *** 32,37 **** --- 32,39 ---- /* Integer object implementation */ #include "Python.h" + #include "pycore.h" + #include #ifdef HAVE_LIMITS_H *************** *** 93,101 **** #define BLOCK_SIZE 1000 /* 1K less typical malloc overhead */ #define BHEAD_SIZE 8 /* Enough for a 64-bit pointer */ #define N_INTOBJECTS ((BLOCK_SIZE - BHEAD_SIZE) / sizeof(PyIntObject)) - - #define PyMem_MALLOC malloc - #define PyMem_FREE free struct _intblock { struct _intblock *next; --- 95,100 ---- diff -cr PyCVS/Objects/listobject.c PyMem/Objects/listobject.c *** PyCVS/Objects/listobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/listobject.c Fri Feb 25 09:17:05 2000 *************** *** 32,37 **** --- 32,38 ---- /* List object implementation */ #include "Python.h" + #include "pycore.h" #ifdef STDC_HEADERS #include *************** *** 70,76 **** if (nbytes / sizeof(PyObject *) != (size_t)size) { return PyErr_NoMemory(); } ! op = (PyListObject *) malloc(sizeof(PyListObject)); if (op == NULL) { return PyErr_NoMemory(); } --- 71,77 ---- if (nbytes / sizeof(PyObject *) != (size_t)size) { return PyErr_NoMemory(); } ! op = (PyListObject *) PyMem_MALLOC(sizeof(PyListObject)); if (op == NULL) { return PyErr_NoMemory(); } *************** *** 78,86 **** op->ob_item = NULL; } else { ! op->ob_item = (PyObject **) malloc(nbytes); if (op->ob_item == NULL) { ! free((ANY *)op); return PyErr_NoMemory(); } } --- 79,87 ---- op->ob_item = NULL; } else { ! op->ob_item = PyMem_NEW(PyObject *, size); if (op->ob_item == NULL) { ! PyMem_FREE(op); return PyErr_NoMemory(); } } *************** *** 224,232 **** while (--i >= 0) { Py_XDECREF(op->ob_item[i]); } ! free((ANY *)op->ob_item); } ! free((ANY *)op); } static int --- 225,233 ---- while (--i >= 0) { Py_XDECREF(op->ob_item[i]); } ! PyMem_DEL(op->ob_item); } ! PyObject_DEL(op); } static int *************** *** 479,485 **** else { /* Insert d items; recycle ihigh-ilow items */ NRESIZE(item, PyObject *, a->ob_size + d); if (item == NULL) { ! PyMem_XDEL(recycle); PyErr_NoMemory(); return -1; } --- 480,487 ---- else { /* Insert d items; recycle ihigh-ilow items */ NRESIZE(item, PyObject *, a->ob_size + d); if (item == NULL) { ! if (recycle != NULL) ! PyMem_DEL(recycle); PyErr_NoMemory(); return -1; } diff -cr PyCVS/Objects/longobject.c PyMem/Objects/longobject.c *** PyCVS/Objects/longobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/longobject.c Fri Feb 25 00:40:44 2000 *************** *** 34,39 **** --- 34,41 ---- /* XXX The functional organization of this file is terrible */ #include "Python.h" + #include "pycore.h" + #include "longintrepr.h" #include "mymath.h" *************** *** 968,974 **** long_dealloc(v) PyObject *v; { ! PyMem_DEL(v); } static PyObject * --- 970,976 ---- long_dealloc(v) PyObject *v; { ! PyObject_DEL(v); } static PyObject * diff -cr PyCVS/Objects/methodobject.c PyMem/Objects/methodobject.c *** PyCVS/Objects/methodobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/methodobject.c Fri Feb 25 00:42:41 2000 *************** *** 32,37 **** --- 32,38 ---- /* Method object implementation */ #include "Python.h" + #include "pycore.h" #include "token.h" *************** *** 288,293 **** while (free_list) { PyCFunctionObject *v = free_list; free_list = (PyCFunctionObject *)(v->m_self); ! PyMem_DEL(v); } } --- 289,294 ---- while (free_list) { PyCFunctionObject *v = free_list; free_list = (PyCFunctionObject *)(v->m_self); ! PyObject_DEL(v); } } diff -cr PyCVS/Objects/moduleobject.c PyMem/Objects/moduleobject.c *** PyCVS/Objects/moduleobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/moduleobject.c Fri Feb 25 00:44:23 2000 *************** *** 32,37 **** --- 32,38 ---- /* Module object implementation */ #include "Python.h" + #include "pycore.h" typedef struct { PyObject_HEAD *************** *** 170,176 **** _PyModule_Clear((PyObject *)m); Py_DECREF(m->md_dict); } ! free((char *)m); } static PyObject * --- 171,177 ---- _PyModule_Clear((PyObject *)m); Py_DECREF(m->md_dict); } ! PyObject_DEL(m); } static PyObject * diff -cr PyCVS/Objects/object.c PyMem/Objects/object.c *** PyCVS/Objects/object.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/object.c Fri Feb 25 00:53:10 2000 *************** *** 32,37 **** --- 32,38 ---- /* Generic object operations; and implementation of None (NoObject) */ #include "Python.h" + #include "pycore.h" #if defined( Py_TRACE_REFS ) || defined( Py_REF_DEBUG ) DL_IMPORT(long) _Py_RefTotal; *************** *** 119,125 **** #endif { #ifndef MS_COREDLL ! PyObject *op = (PyObject *) malloc(tp->tp_basicsize); #endif if (op == NULL) return PyErr_NoMemory(); --- 120,126 ---- #endif { #ifndef MS_COREDLL ! PyObject *op = (PyObject *) PyMem_MALLOC(tp->tp_basicsize); #endif if (op == NULL) return PyErr_NoMemory(); *************** *** 143,149 **** { #ifndef MS_COREDLL PyVarObject *op = (PyVarObject *) ! malloc(tp->tp_basicsize + size * tp->tp_itemsize); #endif if (op == NULL) return (PyVarObject *)PyErr_NoMemory(); --- 144,150 ---- { #ifndef MS_COREDLL PyVarObject *op = (PyVarObject *) ! PyMem_MALLOC(tp->tp_basicsize + size * tp->tp_itemsize); #endif if (op == NULL) return (PyVarObject *)PyErr_NoMemory(); *************** *** 153,158 **** --- 154,170 ---- return op; } + void + _PyObject_Del(op) + PyObject *op; + { + #ifndef MS_COREDLL + PyMem_FREE(op); + #else + free((ANY *)(op)); + #endif + } + int PyObject_Print(op, fp, flags) PyObject *op; *************** *** 754,760 **** if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = malloc(nbytes); if (p != NULL) return p; else { --- 766,772 ---- if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = PyMem_MALLOC(nbytes); if (p != NULL) return p; else { *************** *** 772,778 **** if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = realloc(p, nbytes); if (p != NULL) return p; else { --- 784,790 ---- if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif ! p = PyMem_REALLOC(p, nbytes); if (p != NULL) return p; else { *************** *** 785,823 **** Py_Free(p) ANY *p; { ! free(p); ! } ! ! /* The PyMem_{Malloc,Realloc} wrappers don't call anything on failure */ ! ! ANY * ! PyMem_Malloc(nbytes) ! size_t nbytes; ! { ! #if _PyMem_EXTRA > 0 ! if (nbytes == 0) ! nbytes = _PyMem_EXTRA; ! #endif ! return malloc(nbytes); ! } ! ! ANY * ! PyMem_Realloc(p, nbytes) ! ANY *p; ! size_t nbytes; ! { ! #if _PyMem_EXTRA > 0 ! if (nbytes == 0) ! nbytes = _PyMem_EXTRA; ! #endif ! return realloc(p, nbytes); ! } ! ! void ! PyMem_Free(p) ! ANY *p; ! { ! free(p); } --- 797,803 ---- Py_Free(p) ANY *p; { ! PyMem_FREE(p); } diff -cr PyCVS/Objects/rangeobject.c PyMem/Objects/rangeobject.c *** PyCVS/Objects/rangeobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/rangeobject.c Fri Feb 25 00:54:48 2000 *************** *** 32,37 **** --- 32,38 ---- /* Range object implementation */ #include "Python.h" + #include "pycore.h" typedef struct { PyObject_HEAD *************** *** 61,67 **** range_dealloc(r) rangeobject *r; { ! PyMem_DEL(r); } static PyObject * --- 62,68 ---- range_dealloc(r) rangeobject *r; { ! PyObject_DEL(r); } static PyObject * diff -cr PyCVS/Objects/sliceobject.c PyMem/Objects/sliceobject.c *** PyCVS/Objects/sliceobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/sliceobject.c Fri Feb 25 00:57:04 2000 *************** *** 14,19 **** --- 14,20 ---- */ #include "Python.h" + #include "pycore.h" static PyObject * ellipsis_repr(op) *************** *** 57,64 **** PyObject *stop; PyObject *step; { ! PySliceObject *obj = ! (PySliceObject *) PyObject_NEW(PySliceObject, &PySlice_Type); if (step == NULL) step = Py_None; Py_INCREF(step); --- 58,64 ---- PyObject *stop; PyObject *step; { ! PySliceObject *obj = PyObject_NEW(PySliceObject, &PySlice_Type); if (step == NULL) step = Py_None; Py_INCREF(step); *************** *** 115,121 **** Py_DECREF(r->step); Py_DECREF(r->start); Py_DECREF(r->stop); ! PyMem_DEL(r); } static PyObject * --- 115,121 ---- Py_DECREF(r->step); Py_DECREF(r->start); Py_DECREF(r->stop); ! PyObject_DEL(r); } static PyObject * diff -cr PyCVS/Objects/stringobject.c PyMem/Objects/stringobject.c *** PyCVS/Objects/stringobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/stringobject.c Fri Feb 25 09:22:58 2000 *************** *** 32,37 **** --- 32,38 ---- /* String object implementation */ #include "Python.h" + #include "pycore.h" #include "mymath.h" #include *************** *** 93,99 **** } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 94,100 ---- } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 143,149 **** } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 144,150 ---- } #endif /* DONT_SHARE_SHORT_STRINGS */ op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 172,178 **** string_dealloc(op) PyObject *op; { ! PyMem_DEL(op); } int --- 173,179 ---- string_dealloc(op) PyObject *op; { ! PyObject_DEL(op); } int *************** *** 306,312 **** } size = a->ob_size + b->ob_size; op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 307,313 ---- } size = a->ob_size + b->ob_size; op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 341,347 **** return (PyObject *)a; } op = (PyStringObject *) ! malloc(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; --- 342,348 ---- return (PyObject *)a; } op = (PyStringObject *) ! PyMem_MALLOC(sizeof(PyStringObject) + size * sizeof(char)); if (op == NULL) return PyErr_NoMemory(); op->ob_type = &PyString_Type; *************** *** 1369,1375 **** goto return_same; new_len = len + nfound*(sub_len - pat_len); ! new_s = (char *)malloc(new_len); if (new_s == NULL) return NULL; *out_len = new_len; --- 1370,1376 ---- goto return_same; new_len = len + nfound*(sub_len - pat_len); ! new_s = PyMem_NEW(char, new_len); if (new_s == NULL) return NULL; *out_len = new_len; *************** *** 1441,1447 **** } else { new = PyString_FromStringAndSize(new_s, out_len); ! free(new_s); } return new; } --- 1442,1448 ---- } else { new = PyString_FromStringAndSize(new_s, out_len); ! PyMem_DEL(new_s); } return new; } *************** *** 1646,1655 **** #endif _Py_ForgetReference(v); *pv = (PyObject *) ! realloc((char *)v, ! sizeof(PyStringObject) + newsize * sizeof(char)); if (*pv == NULL) { ! PyMem_DEL(v); PyErr_NoMemory(); return -1; } --- 1647,1656 ---- #endif _Py_ForgetReference(v); *pv = (PyObject *) ! PyMem_REALLOC((char *)v, ! sizeof(PyStringObject) + newsize*sizeof(char)); if (*pv == NULL) { ! PyObject_DEL(v); PyErr_NoMemory(); return -1; } diff -cr PyCVS/Objects/tupleobject.c PyMem/Objects/tupleobject.c *** PyCVS/Objects/tupleobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/tupleobject.c Fri Feb 25 01:21:47 2000 *************** *** 32,37 **** --- 32,38 ---- /* Tuple object implementation */ #include "Python.h" + #include "pycore.h" #ifndef MAXSAVESIZE #define MAXSAVESIZE 20 *************** *** 91,97 **** return PyErr_NoMemory(); } ; ! op = (PyTupleObject *) malloc(nbytes); if (op == NULL) return PyErr_NoMemory(); --- 92,98 ---- return PyErr_NoMemory(); } ; ! op = (PyTupleObject *) PyMem_MALLOC(nbytes); if (op == NULL) return PyErr_NoMemory(); *************** *** 184,190 **** } #endif } ! free((ANY *)op); } static int --- 185,191 ---- } #endif } ! PyObject_DEL(op); } static int *************** *** 467,477 **** v->ob_item[i] = NULL; } sv = (PyTupleObject *) ! realloc((char *)v, sizeof(PyTupleObject) + newsize * sizeof(PyObject *)); *pv = (PyObject *) sv; if (sv == NULL) { ! PyMem_DEL(v); PyErr_NoMemory(); return -1; } --- 468,478 ---- v->ob_item[i] = NULL; } sv = (PyTupleObject *) ! PyMem_REALLOC((char *)v, sizeof(PyTupleObject) + newsize * sizeof(PyObject *)); *pv = (PyObject *) sv; if (sv == NULL) { ! PyObject_DEL(v); PyErr_NoMemory(); return -1; } *************** *** 505,511 **** while (p) { q = p; p = (PyTupleObject *)(p->ob_item[0]); ! PyMem_DEL(q); } } #endif --- 506,512 ---- while (p) { q = p; p = (PyTupleObject *)(p->ob_item[0]); ! PyObject_DEL(q); } } #endif diff -cr PyCVS/Objects/typeobject.c PyMem/Objects/typeobject.c *** PyCVS/Objects/typeobject.c Thu Feb 24 14:49:47 2000 --- PyMem/Objects/typeobject.c Fri Feb 25 01:24:08 2000 *************** *** 32,37 **** --- 32,38 ---- /* Type object implementation */ #include "Python.h" + #include "pycore.h" /* Type object implementation */ From Vladimir.Marangozov@inrialpes.fr Fri Feb 25 12:35:08 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 25 Feb 2000 13:35:08 +0100 (CET) Subject: [Patches] PyMem [8/8] - Modules/* Message-ID: <200002251235.NAA20874@python.inrialpes.fr> - The extensions continue to use they're malloc of choice for internal purposes, but were rewritten to mention PyObject_NEW/DEL explicitely when they deal with Python objects. - The readline module takes care to feed PyOS_Readline with a buffer allocated via PyMem_Malloc. - Another buglet in the regexmodule obj finalization (the regex code uses primarily malloc for its buffers -- the latter need to be released via free()) - Overall niceties and cleanups... When and what gets in, please test thoroughly. Linux (RH6.1) and AIX seem to do fine with it. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -------------------------------[ cut here ]--------------------------- Only in PyMem/Modules/: Setup.local diff -cr PyCVS/Modules/_tkinter.c PyMem/Modules/_tkinter.c *** PyCVS/Modules/_tkinter.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/_tkinter.c Fri Feb 25 05:38:33 2000 *************** *** 1562,1568 **** Py_XDECREF(func); ! PyMem_DEL(self); } static PyObject * --- 1562,1568 ---- Py_XDECREF(func); ! PyObject_DEL(self); } static PyObject * *************** *** 1810,1816 **** ENTER_TCL Tcl_DeleteInterp(Tkapp_Interp(self)); LEAVE_TCL ! PyMem_DEL(self); DisableEventHook(); } --- 1810,1816 ---- ENTER_TCL Tcl_DeleteInterp(Tkapp_Interp(self)); LEAVE_TCL ! PyObject_DEL(self); DisableEventHook(); } diff -cr PyCVS/Modules/almodule.c PyMem/Modules/almodule.c *** PyCVS/Modules/almodule.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/almodule.c Fri Feb 25 02:45:35 2000 *************** *** 667,673 **** #else (void) ALfreeconfig(self->config); /* ignore errors */ #endif ! PyMem_DEL(self); } static PyObject * --- 667,673 ---- #else (void) ALfreeconfig(self->config); /* ignore errors */ #endif ! PyObject_DEL(self); } static PyObject * diff -cr PyCVS/Modules/arraymodule.c PyMem/Modules/arraymodule.c *** PyCVS/Modules/arraymodule.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/arraymodule.c Fri Feb 25 02:50:34 2000 *************** *** 346,354 **** if (nbytes / descr->itemsize != (size_t)size) { return PyErr_NoMemory(); } ! op = PyMem_NEW(arrayobject, 1); if (op == NULL) { ! return PyErr_NoMemory(); } if (size <= 0) { op->ob_item = NULL; --- 346,354 ---- if (nbytes / descr->itemsize != (size_t)size) { return PyErr_NoMemory(); } ! op = PyObject_NEW(arrayobject, &Arraytype); if (op == NULL) { ! return NULL; } if (size <= 0) { op->ob_item = NULL; *************** *** 356,369 **** else { op->ob_item = PyMem_NEW(char, nbytes); if (op->ob_item == NULL) { ! PyMem_DEL(op); return PyErr_NoMemory(); } } - op->ob_type = &Arraytype; op->ob_size = size; op->ob_descr = descr; - _Py_NewReference((PyObject *)op); return (PyObject *) op; } --- 356,367 ---- else { op->ob_item = PyMem_NEW(char, nbytes); if (op->ob_item == NULL) { ! PyObject_DEL(op); return PyErr_NoMemory(); } } op->ob_size = size; op->ob_descr = descr; return (PyObject *) op; } *************** *** 466,472 **** { if (op->ob_item != NULL) PyMem_DEL(op->ob_item); ! PyMem_DEL(op); } static int --- 464,470 ---- { if (op->ob_item != NULL) PyMem_DEL(op->ob_item); ! PyObject_DEL(op); } static int diff -cr PyCVS/Modules/audioop.c PyMem/Modules/audioop.c *** PyCVS/Modules/audioop.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/audioop.c Fri Feb 25 02:56:40 2000 *************** *** 1010,1017 **** inrate /= d; outrate /= d; ! prev_i = (int *) malloc(nchannels * sizeof(int)); ! cur_i = (int *) malloc(nchannels * sizeof(int)); len /= size * nchannels; /* # of frames */ if (prev_i == NULL || cur_i == NULL) { (void) PyErr_NoMemory(); --- 1010,1017 ---- inrate /= d; outrate /= d; ! prev_i = PyMem_NEW(int, nchannels); ! cur_i = PyMem_NEW(int, nchannels); len /= size * nchannels; /* # of frames */ if (prev_i == NULL || cur_i == NULL) { (void) PyErr_NoMemory(); *************** *** 1104,1112 **** } exit: if (prev_i != NULL) ! free(prev_i); if (cur_i != NULL) ! free(cur_i); return rv; } --- 1104,1112 ---- } exit: if (prev_i != NULL) ! PyMem_DEL(prev_i); if (cur_i != NULL) ! PyMem_DEL(cur_i); return rv; } diff -cr PyCVS/Modules/bsddbmodule.c PyMem/Modules/bsddbmodule.c *** PyCVS/Modules/bsddbmodule.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/bsddbmodule.c Fri Feb 25 03:01:32 2000 *************** *** 261,267 **** "Python bsddb: close errno %d in dealloc\n", errno); } ! PyMem_DEL(dp); } #ifdef WITH_THREAD --- 261,267 ---- "Python bsddb: close errno %d in dealloc\n", errno); } ! PyObject_DEL(dp); } #ifdef WITH_THREAD diff -cr PyCVS/Modules/cPickle.c PyMem/Modules/cPickle.c *** PyCVS/Modules/cPickle.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/cPickle.c Fri Feb 25 03:19:40 2000 *************** *** 169,175 **** if (self->data) free(self->data); ! PyMem_DEL(self); } static PyTypeObject PdataType = { --- 169,175 ---- if (self->data) free(self->data); ! PyObject_DEL(self); } static PyTypeObject PdataType = { *************** *** 2156,2162 **** free(self->write_buf); } ! PyMem_DEL(self); } --- 2156,2162 ---- free(self->write_buf); } ! PyObject_DEL(self); } *************** *** 3995,4001 **** free(self->buf); } ! PyMem_DEL(self); } --- 3995,4001 ---- free(self->buf); } ! PyObject_DEL(self); } diff -cr PyCVS/Modules/cStringIO.c PyMem/Modules/cStringIO.c *** PyCVS/Modules/cStringIO.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/cStringIO.c Fri Feb 25 03:23:41 2000 *************** *** 56,62 **** "\n" "This module provides a simple useful replacement for\n" "the StringIO module that is written in C. It does not provide the\n" ! "full generality if StringIO, but it provides anough for most\n" "applications and is especially useful in conjuction with the\n" "pickle module.\n" "\n" --- 56,62 ---- "\n" "This module provides a simple useful replacement for\n" "the StringIO module that is written in C. It does not provide the\n" ! "full generality of StringIO, but it provides enough for most\n" "applications and is especially useful in conjuction with the\n" "pickle module.\n" "\n" *************** *** 407,413 **** O_dealloc(Oobject *self) { if (self->buf != NULL) free(self->buf); ! PyMem_DEL(self); } static PyObject * --- 407,413 ---- O_dealloc(Oobject *self) { if (self->buf != NULL) free(self->buf); ! PyObject_DEL(self); } static PyObject * *************** *** 536,542 **** static void I_dealloc(Iobject *self) { Py_XDECREF(self->pbuf); ! PyMem_DEL(self); } static PyObject * --- 536,542 ---- static void I_dealloc(Iobject *self) { Py_XDECREF(self->pbuf); ! PyObject_DEL(self); } static PyObject * diff -cr PyCVS/Modules/cdmodule.c PyMem/Modules/cdmodule.c *** PyCVS/Modules/cdmodule.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/cdmodule.c Fri Feb 25 03:04:22 2000 *************** *** 447,453 **** { if (self->ob_cdplayer != NULL) CDclose(self->ob_cdplayer); ! PyMem_DEL(self); } static PyObject * --- 447,453 ---- { if (self->ob_cdplayer != NULL) CDclose(self->ob_cdplayer); ! PyObject_DEL(self); } static PyObject * *************** *** 761,767 **** self->ob_cdcallbacks[i].ob_cdcallbackarg = NULL; } CDdeleteparser(self->ob_cdparser); ! PyMem_DEL(self); } static PyObject * --- 761,767 ---- self->ob_cdcallbacks[i].ob_cdcallbackarg = NULL; } CDdeleteparser(self->ob_cdparser); ! PyObject_DEL(self); } static PyObject * diff -cr PyCVS/Modules/clmodule.c PyMem/Modules/clmodule.c *** PyCVS/Modules/clmodule.c Thu Feb 24 14:49:43 2000 --- PyMem/Modules/clmodule.c Fri Feb 25 03:09:02 2000 *************** *** 673,679 **** else clCloseDecompressor(SELF->ob_compressorHdl); } ! PyMem_DEL(self); } static PyObject * --- 673,679 ---- else clCloseDecompressor(SELF->ob_compressorHdl); } ! PyObject_DEL(self); } static PyObject * diff -cr PyCVS/Modules/cursesmodule.c PyMem/Modules/cursesmodule.c *** PyCVS/Modules/cursesmodule.c Thu Feb 24 14:49:43 2000 --- PyMem/Modules/cursesmodule.c Fri Feb 25 03:29:29 2000 *************** *** 303,309 **** { if (wo->win != stdscr) delwin(wo->win); ! PyMem_DEL(wo); } static PyObject * --- 303,309 ---- { if (wo->win != stdscr) delwin(wo->win); ! PyObject_DEL(wo); } static PyObject * diff -cr PyCVS/Modules/dbmmodule.c PyMem/Modules/dbmmodule.c *** PyCVS/Modules/dbmmodule.c Thu Feb 24 14:49:43 2000 --- PyMem/Modules/dbmmodule.c Fri Feb 25 03:30:58 2000 *************** *** 82,88 **** { if ( dp->di_dbm ) dbm_close(dp->di_dbm); ! PyMem_DEL(dp); } static int --- 82,88 ---- { if ( dp->di_dbm ) dbm_close(dp->di_dbm); ! PyObject_DEL(dp); } static int diff -cr PyCVS/Modules/dlmodule.c PyMem/Modules/dlmodule.c *** PyCVS/Modules/dlmodule.c Thu Feb 24 14:49:43 2000 --- PyMem/Modules/dlmodule.c Fri Feb 25 03:31:55 2000 *************** *** 67,73 **** { if (xp->dl_handle != NULL) dlclose(xp->dl_handle); ! PyMem_DEL(xp); } static PyObject * --- 67,73 ---- { if (xp->dl_handle != NULL) dlclose(xp->dl_handle); ! PyObject_DEL(xp); } static PyObject * diff -cr PyCVS/Modules/flmodule.c PyMem/Modules/flmodule.c *** PyCVS/Modules/flmodule.c Thu Feb 24 14:49:44 2000 --- PyMem/Modules/flmodule.c Fri Feb 25 03:36:11 2000 *************** *** 332,338 **** fl_free_object(g->ob_generic); Py_XDECREF(g->ob_callback); Py_XDECREF(g->ob_callback_arg); ! PyMem_DEL(g); } #define OFF(x) offsetof(FL_OBJECT, x) --- 332,338 ---- fl_free_object(g->ob_generic); Py_XDECREF(g->ob_callback); Py_XDECREF(g->ob_callback_arg); ! PyObject_DEL(g); } #define OFF(x) offsetof(FL_OBJECT, x) *************** *** 1852,1858 **** if (f->ob_form->visible) fl_hide_form(f->ob_form); fl_free_form(f->ob_form); ! PyMem_DEL(f); } #define OFF(x) offsetof(FL_FORM, x) --- 1852,1858 ---- if (f->ob_form->visible) fl_hide_form(f->ob_form); fl_free_form(f->ob_form); ! PyObject_DEL(f); } #define OFF(x) offsetof(FL_FORM, x) diff -cr PyCVS/Modules/fmmodule.c PyMem/Modules/fmmodule.c *** PyCVS/Modules/fmmodule.c Thu Feb 24 14:49:44 2000 --- PyMem/Modules/fmmodule.c Fri Feb 25 03:38:01 2000 *************** *** 196,202 **** fhobject *fhp; { fmfreefont(fhp->fh_fh); ! PyMem_DEL(fhp); } static PyTypeObject Fhtype = { --- 196,202 ---- fhobject *fhp; { fmfreefont(fhp->fh_fh); ! PyObject_DEL(fhp); } static PyTypeObject Fhtype = { diff -cr PyCVS/Modules/gdbmmodule.c PyMem/Modules/gdbmmodule.c *** PyCVS/Modules/gdbmmodule.c Thu Feb 24 14:49:44 2000 --- PyMem/Modules/gdbmmodule.c Fri Feb 25 03:40:41 2000 *************** *** 117,123 **** { if ( dp->di_dbm ) gdbm_close(dp->di_dbm); ! PyMem_DEL(dp); } static int --- 117,123 ---- { if ( dp->di_dbm ) gdbm_close(dp->di_dbm); ! PyObject_DEL(dp); } static int diff -cr PyCVS/Modules/getpath.c PyMem/Modules/getpath.c *** PyCVS/Modules/getpath.c Thu Feb 24 14:49:44 2000 --- PyMem/Modules/getpath.c Fri Feb 25 03:45:34 2000 *************** *** 529,535 **** bufsz += strlen(exec_prefix) + 1; /* This is the only malloc call in this file */ ! buf = malloc(bufsz); if (buf == NULL) { /* We can't exit, so print a warning and limp along */ --- 529,535 ---- bufsz += strlen(exec_prefix) + 1; /* This is the only malloc call in this file */ ! buf = PyMem_Malloc(bufsz); if (buf == NULL) { /* We can't exit, so print a warning and limp along */ diff -cr PyCVS/Modules/main.c PyMem/Modules/main.c *** PyCVS/Modules/main.c Thu Feb 24 14:49:45 2000 --- PyMem/Modules/main.c Fri Feb 25 02:34:56 2000 *************** *** 124,130 **** /* -c is the last option; following arguments that look like options are left for the the command to interpret. */ ! command = malloc(strlen(optarg) + 2); if (command == NULL) Py_FatalError( "not enough memory to copy -c argument"); --- 124,130 ---- /* -c is the last option; following arguments that look like options are left for the the command to interpret. */ ! command = PyMem_Malloc(strlen(optarg) + 2); if (command == NULL) Py_FatalError( "not enough memory to copy -c argument"); *************** *** 277,283 **** if (command) { sts = PyRun_SimpleString(command) != 0; ! free(command); } else { if (filename == NULL && stdin_is_interactive) { --- 277,283 ---- if (command) { sts = PyRun_SimpleString(command) != 0; ! PyMem_Free(command); } else { if (filename == NULL && stdin_is_interactive) { diff -cr PyCVS/Modules/md5module.c PyMem/Modules/md5module.c *** PyCVS/Modules/md5module.c Thu Feb 24 14:49:45 2000 --- PyMem/Modules/md5module.c Fri Feb 25 03:52:09 2000 *************** *** 71,77 **** md5_dealloc(md5p) md5object *md5p; { ! PyMem_DEL(md5p); } --- 71,77 ---- md5_dealloc(md5p) md5object *md5p; { ! PyObject_DEL(md5p); } diff -cr PyCVS/Modules/mpzmodule.c PyMem/Modules/mpzmodule.c *** PyCVS/Modules/mpzmodule.c Thu Feb 24 14:49:45 2000 --- PyMem/Modules/mpzmodule.c Fri Feb 25 03:56:42 2000 *************** *** 284,290 **** fputs( "mpz_dealloc() called...\n", stderr ); #endif /* def MPZ_DEBUG */ mpz_clear(&mpzp->mpz); ! PyMem_DEL(mpzp); } /* mpz_dealloc() */ --- 284,290 ---- fputs( "mpz_dealloc() called...\n", stderr ); #endif /* def MPZ_DEBUG */ mpz_clear(&mpzp->mpz); ! PyObject_DEL(mpzp); } /* mpz_dealloc() */ diff -cr PyCVS/Modules/parsermodule.c PyMem/Modules/parsermodule.c *** PyCVS/Modules/parsermodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/parsermodule.c Fri Feb 25 02:29:57 2000 *************** *** 332,338 **** PyAST_Object *ast; { PyNode_Free(ast->ast_node); ! PyMem_DEL(ast); } /* parser_free() */ --- 332,338 ---- PyAST_Object *ast; { PyNode_Free(ast->ast_node); ! PyObject_DEL(ast); } /* parser_free() */ *************** *** 850,856 **** PyObject *temp = PySequence_GetItem(elem, 1); /* check_terminal_tuple() already verified it's a string */ ! strn = (char *)malloc(PyString_GET_SIZE(temp) + 1); if (strn != NULL) (void) strcpy(strn, PyString_AS_STRING(temp)); Py_XDECREF(temp); --- 850,856 ---- PyObject *temp = PySequence_GetItem(elem, 1); /* check_terminal_tuple() already verified it's a string */ ! strn = PyMem_NEW(char, PyString_GET_SIZE(temp) + 1); if (strn != NULL) (void) strcpy(strn, PyString_AS_STRING(temp)); Py_XDECREF(temp); diff -cr PyCVS/Modules/pcremodule.c PyMem/Modules/pcremodule.c *** PyCVS/Modules/pcremodule.c Thu Feb 24 14:49:42 2000 --- PyMem/Modules/pcremodule.c Fri Feb 25 04:07:45 2000 *************** *** 95,101 **** { if (self->regex) (pcre_free)(self->regex); if (self->regex_extra) (pcre_free)(self->regex_extra); ! PyMem_DEL(self); } --- 95,101 ---- { if (self->regex) (pcre_free)(self->regex); if (self->regex_extra) (pcre_free)(self->regex_extra); ! PyObject_DEL(self); } diff -cr PyCVS/Modules/readline.c PyMem/Modules/readline.c *** PyCVS/Modules/readline.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/readline.c Fri Feb 25 02:25:39 2000 *************** *** 377,383 **** char *prompt; { int n; ! char *p; RETSIGTYPE (*old_inthandler)(); old_inthandler = signal(SIGINT, onintr); if (setjmp(jbuf)) { --- 377,383 ---- char *prompt; { int n; ! char *p, *q; RETSIGTYPE (*old_inthandler)(); old_inthandler = signal(SIGINT, onintr); if (setjmp(jbuf)) { *************** *** 391,398 **** rl_event_hook = PyOS_InputHook; p = readline(prompt); signal(SIGINT, old_inthandler); if (p == NULL) { ! p = malloc(1); if (p != NULL) *p = '\0'; return p; --- 391,400 ---- rl_event_hook = PyOS_InputHook; p = readline(prompt); signal(SIGINT, old_inthandler); + + /* We must return a buffer allocated with PyMem_Malloc. */ if (p == NULL) { ! p = PyMem_Malloc(1); if (p != NULL) *p = '\0'; return p; *************** *** 400,409 **** n = strlen(p); if (n > 0) add_history(p); ! if ((p = realloc(p, n+2)) != NULL) { p[n] = '\n'; p[n+1] = '\0'; } return p; } --- 402,417 ---- n = strlen(p); if (n > 0) add_history(p); ! /* Copy the malloc'ed buffer into a PyMem_Malloc'ed one and ! release the original. */ ! q = p; ! p = PyMem_Malloc(n+2); ! if (p != NULL) { ! strncpy(p, q, n); p[n] = '\n'; p[n+1] = '\0'; } + free(q); return p; } diff -cr PyCVS/Modules/regexmodule.c PyMem/Modules/regexmodule.c *** PyCVS/Modules/regexmodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/regexmodule.c Fri Feb 25 08:24:30 2000 *************** *** 64,76 **** reg_dealloc(re) regexobject *re; { ! PyMem_XDEL(re->re_patbuf.buffer); Py_XDECREF(re->re_translate); Py_XDECREF(re->re_lastok); Py_XDECREF(re->re_groupindex); Py_XDECREF(re->re_givenpat); Py_XDECREF(re->re_realpat); ! PyMem_DEL(re); } static PyObject * --- 64,77 ---- reg_dealloc(re) regexobject *re; { ! if (re->re_patbuf.buffer != NULL) ! free(re->re_patbuf.buffer); Py_XDECREF(re->re_translate); Py_XDECREF(re->re_lastok); Py_XDECREF(re->re_groupindex); Py_XDECREF(re->re_givenpat); Py_XDECREF(re->re_realpat); ! PyObject_DEL(re); } static PyObject * diff -cr PyCVS/Modules/rotormodule.c PyMem/Modules/rotormodule.c *** PyCVS/Modules/rotormodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/rotormodule.c Fri Feb 25 04:38:58 2000 *************** *** 477,483 **** PyMem_XDEL(xp->d_rotor); PyMem_XDEL(xp->positions); PyMem_XDEL(xp->advances); ! PyMem_DEL(xp); } static PyObject * --- 477,483 ---- PyMem_XDEL(xp->d_rotor); PyMem_XDEL(xp->positions); PyMem_XDEL(xp->advances); ! PyObject_DEL(xp); } static PyObject * diff -cr PyCVS/Modules/shamodule.c PyMem/Modules/shamodule.c *** PyCVS/Modules/shamodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/shamodule.c Fri Feb 25 04:42:23 2000 *************** *** 389,395 **** SHA_dealloc(ptr) PyObject *ptr; { ! PyMem_DEL(ptr); } --- 389,395 ---- SHA_dealloc(ptr) PyObject *ptr; { ! PyObject_DEL(ptr); } diff -cr PyCVS/Modules/socketmodule.c PyMem/Modules/socketmodule.c *** PyCVS/Modules/socketmodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/socketmodule.c Fri Feb 25 04:47:59 2000 *************** *** 1350,1356 **** BUILD_FUNC_DEF_1(PySocketSock_dealloc,PySocketSockObject *,s) { (void) close(s->sock_fd); ! PyMem_DEL(s); } --- 1350,1356 ---- BUILD_FUNC_DEF_1(PySocketSock_dealloc,PySocketSockObject *,s) { (void) close(s->sock_fd); ! PyObject_DEL(s); } *************** *** 1943,1949 **** if (self->ctx == NULL) { PyErr_SetObject(SSLErrorObject, PyString_FromString("SSL_CTX_new error")); ! PyMem_DEL(self); return NULL; } --- 1943,1949 ---- if (self->ctx == NULL) { PyErr_SetObject(SSLErrorObject, PyString_FromString("SSL_CTX_new error")); ! PyObject_DEL(self); return NULL; } *************** *** 1952,1958 **** PyErr_SetObject(SSLErrorObject, PyString_FromString( "Both the key & certificate files must be specified")); ! PyMem_DEL(self); return NULL; } --- 1952,1958 ---- PyErr_SetObject(SSLErrorObject, PyString_FromString( "Both the key & certificate files must be specified")); ! PyObject_DEL(self); return NULL; } *************** *** 1964,1970 **** PyErr_SetObject(SSLErrorObject, PyString_FromString( "SSL_CTX_use_PrivateKey_file error")); ! PyMem_DEL(self); return NULL; } --- 1964,1970 ---- PyErr_SetObject(SSLErrorObject, PyString_FromString( "SSL_CTX_use_PrivateKey_file error")); ! PyObject_DEL(self); return NULL; } *************** *** 1974,1980 **** PyErr_SetObject(SSLErrorObject, PyString_FromString( "SSL_CTX_use_certificate_chain_file error")); ! PyMem_DEL(self); return NULL; } } --- 1974,1980 ---- PyErr_SetObject(SSLErrorObject, PyString_FromString( "SSL_CTX_use_certificate_chain_file error")); ! PyObject_DEL(self); return NULL; } } *************** *** 1989,1995 **** /* Actually negotiate SSL connection */ PyErr_SetObject(SSLErrorObject, PyString_FromString("SSL_connect error")); ! PyMem_DEL(self); return NULL; } self->ssl->debug = 1; --- 1989,1995 ---- /* Actually negotiate SSL connection */ PyErr_SetObject(SSLErrorObject, PyString_FromString("SSL_connect error")); ! PyObject_DEL(self); return NULL; } self->ssl->debug = 1; *************** *** 2060,2066 **** SSL_free(self->ssl); Py_XDECREF(self->x_attr); Py_XDECREF(self->Socket); ! PyMem_DEL(self); } static PyObject *SSL_getattr(SSLObject *self, char *name) --- 2060,2066 ---- SSL_free(self->ssl); Py_XDECREF(self->x_attr); Py_XDECREF(self->Socket); ! PyObject_DEL(self); } static PyObject *SSL_getattr(SSLObject *self, char *name) diff -cr PyCVS/Modules/stdwinmodule.c PyMem/Modules/stdwinmodule.c *** PyCVS/Modules/stdwinmodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/stdwinmodule.c Fri Feb 25 05:06:33 2000 *************** *** 271,277 **** Py_DECREF(dp->d_ref); dp->d_ref = NULL; } ! free((char *)dp); } static PyObject * --- 271,277 ---- Py_DECREF(dp->d_ref); dp->d_ref = NULL; } ! PyObject_DEL(dp); } static PyObject * *************** *** 853,859 **** tefree(tp->t_text); Py_XDECREF(tp->t_attr); Py_XDECREF(tp->t_ref); ! PyMem_DEL(tp); } static PyObject * --- 853,859 ---- tefree(tp->t_text); Py_XDECREF(tp->t_attr); Py_XDECREF(tp->t_ref); ! PyObject_DEL(tp); } static PyObject * *************** *** 1216,1222 **** if (mp->m_menu != NULL) wmenudelete(mp->m_menu); Py_XDECREF(mp->m_attr); ! PyMem_DEL(mp); } static PyObject * --- 1216,1222 ---- if (mp->m_menu != NULL) wmenudelete(mp->m_menu); Py_XDECREF(mp->m_attr); ! PyObject_DEL(mp); } static PyObject * *************** *** 1405,1411 **** if (bp->b_bitmap != NULL) wfreebitmap(bp->b_bitmap); Py_XDECREF(bp->b_attr); ! PyMem_DEL(bp); } static PyObject * --- 1405,1411 ---- if (bp->b_bitmap != NULL) wfreebitmap(bp->b_bitmap); Py_XDECREF(bp->b_attr); ! PyObject_DEL(bp); } static PyObject * *************** *** 1554,1560 **** Py_DECREF(wp->w_title); if (wp->w_attr != NULL) Py_DECREF(wp->w_attr); ! free((char *)wp); } static PyObject * --- 1554,1560 ---- Py_DECREF(wp->w_title); if (wp->w_attr != NULL) Py_DECREF(wp->w_attr); ! PyObject_DEL(wp); } static PyObject * diff -cr PyCVS/Modules/stropmodule.c PyMem/Modules/stropmodule.c *** PyCVS/Modules/stropmodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/stropmodule.c Fri Feb 25 05:17:39 2000 *************** *** 1152,1158 **** goto return_same; new_len = len + nfound*(sub_len - pat_len); ! new_s = (char *)malloc(new_len); if (new_s == NULL) return NULL; *out_len = new_len; --- 1152,1158 ---- goto return_same; new_len = len + nfound*(sub_len - pat_len); ! new_s = PyMem_NEW(char, new_len); if (new_s == NULL) return NULL; *out_len = new_len; *************** *** 1225,1231 **** } else { new = PyString_FromStringAndSize(new_s, out_len); ! free(new_s); } return new; } --- 1225,1231 ---- } else { new = PyString_FromStringAndSize(new_s, out_len); ! PyMem_DEL(new_s); } return new; } diff -cr PyCVS/Modules/sunaudiodev.c PyMem/Modules/sunaudiodev.c *** PyCVS/Modules/sunaudiodev.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/sunaudiodev.c Fri Feb 25 05:23:08 2000 *************** *** 158,164 **** sadobject *xp; { close(xp->x_fd); ! PyMem_DEL(xp); } static PyObject * --- 158,164 ---- sadobject *xp; { close(xp->x_fd); ! PyObject_DEL(xp); } static PyObject * *************** *** 419,425 **** sads_dealloc(xp) sadstatusobject *xp; { ! PyMem_DEL(xp); } #define OFF(x) offsetof(audio_info_t,x) --- 419,425 ---- sads_dealloc(xp) sadstatusobject *xp; { ! PyObject_DEL(xp); } #define OFF(x) offsetof(audio_info_t,x) diff -cr PyCVS/Modules/svmodule.c PyMem/Modules/svmodule.c *** PyCVS/Modules/svmodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/svmodule.c Fri Feb 25 05:25:46 2000 *************** *** 340,346 **** Py_DECREF(self->ob_svideo); self->ob_svideo = NULL; } ! PyMem_DEL(self); } static PyObject * --- 340,346 ---- Py_DECREF(self->ob_svideo); self->ob_svideo = NULL; } ! PyObject_DEL(self); } static PyObject * *************** *** 994,1000 **** { if (self->ob_svideo != NULL) (void) svCloseVideo(self->ob_svideo); ! PyMem_DEL(self); } static PyObject * --- 994,1000 ---- { if (self->ob_svideo != NULL) (void) svCloseVideo(self->ob_svideo); ! PyObject_DEL(self); } static PyObject * diff -cr PyCVS/Modules/threadmodule.c PyMem/Modules/threadmodule.c *** PyCVS/Modules/threadmodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/threadmodule.c Fri Feb 25 05:30:42 2000 *************** *** 63,69 **** return NULL; self->lock_lock = PyThread_allocate_lock(); if (self->lock_lock == NULL) { ! PyMem_DEL(self); self = NULL; PyErr_SetString(ThreadError, "can't allocate lock"); } --- 63,69 ---- return NULL; self->lock_lock = PyThread_allocate_lock(); if (self->lock_lock == NULL) { ! PyObject_DEL(self); self = NULL; PyErr_SetString(ThreadError, "can't allocate lock"); } *************** *** 79,85 **** PyThread_release_lock(self->lock_lock); PyThread_free_lock(self->lock_lock); ! PyMem_DEL(self); } static PyObject * --- 79,85 ---- PyThread_release_lock(self->lock_lock); PyThread_free_lock(self->lock_lock); ! PyObject_DEL(self); } static PyObject * diff -cr PyCVS/Modules/xxmodule.c PyMem/Modules/xxmodule.c *** PyCVS/Modules/xxmodule.c Thu Feb 24 14:49:46 2000 --- PyMem/Modules/xxmodule.c Fri Feb 25 05:42:09 2000 *************** *** 76,82 **** XxoObject *self; { Py_XDECREF(self->x_attr); ! PyMem_DEL(self); } static PyObject * --- 76,82 ---- XxoObject *self; { Py_XDECREF(self->x_attr); ! PyObject_DEL(self); } static PyObject * diff -cr PyCVS/Modules/zlibmodule.c PyMem/Modules/zlibmodule.c *** PyCVS/Modules/zlibmodule.c Thu Feb 24 14:49:47 2000 --- PyMem/Modules/zlibmodule.c Fri Feb 25 05:46:11 2000 *************** *** 372,378 **** if (self->is_initialised) deflateEnd(&self->zst); Py_XDECREF(self->unused_data); ! PyMem_DEL(self); } static void --- 372,378 ---- if (self->is_initialised) deflateEnd(&self->zst); Py_XDECREF(self->unused_data); ! PyObject_DEL(self); } static void *************** *** 381,387 **** { inflateEnd(&self->zst); Py_XDECREF(self->unused_data); ! PyMem_DEL(self); } static char comp_compress__doc__[] = --- 381,387 ---- { inflateEnd(&self->zst); Py_XDECREF(self->unused_data); ! PyObject_DEL(self); } static char comp_compress__doc__[] = From Gerrit Fri Feb 25 15:41:20 2000 From: Gerrit (Gerrit Holl) Date: Fri, 25 Feb 2000 16:41:20 +0100 Subject: [Patches] moving acknowledgements from docstrings to comments Message-ID: <20000225164120.A19099@stopcontact.palga.uucp> --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Hello, I think acknowledgements do not belong in docstrings. This patch only changes this for getopt.py; I wouldn't like to do it for every module and find out you don't agree afterwards. Do you agree in changing this? It also changes string.index(opt, '=') to opt.index('='). DISCLAIMER I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. regards, Gerrit. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="getopt.py.diff" *** getopt.py Fri Feb 25 16:25:59 2000 --- /tmp/getopt.py Fri Feb 25 16:24:48 2000 *************** *** 7,12 **** --- 7,15 ---- may be used as well via an optional third argument. This module provides a single function and an exception: + Gerrit Holl moved the string-based exceptions + to class-based exceptions. + getopt() -- Parse command line options GetoptError -- exception (class) raised with 'opt' attribute, which is the option involved with the exception. *************** *** 14,22 **** # Long option support added by Lars Wirzenius . - # Gerrit Holl moved the string-based exceptions - # to class-based exceptions. - import string class GetoptError(Exception): --- 17,22 ---- *************** *** 81,87 **** def do_longs(opts, opt, longopts, args): try: ! i = opt.index('=') opt, optarg = opt[:i], opt[i+1:] except ValueError: optarg = None --- 81,87 ---- def do_longs(opts, opt, longopts, args): try: ! i = string.index(opt, '=') opt, optarg = opt[:i], opt[i+1:] except ValueError: optarg = None --G4iJoqBmSsgzjUCe-- From fdrake@acm.org Fri Feb 25 16:14:46 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 25 Feb 2000 11:14:46 -0500 (EST) Subject: [Patches] moving acknowledgements from docstrings to comments In-Reply-To: <20000225164120.A19099@stopcontact.palga.uucp> References: <20000225164120.A19099@stopcontact.palga.uucp> Message-ID: <14518.43638.699044.609025@weyr.cnri.reston.va.us> Gerrit Holl writes: > I think acknowledgements do not belong in docstrings. This patch only > changes this for getopt.py; I wouldn't like to do it for every module > and find out you don't agree afterwards. Do you agree in changing this? Gerrit, I agree; the documentation provides specific markup for identifying the author of a section as well as the author of the related module. These do not generate any output but serve to record the contribution. This is appropriate for module source as well. I've always thought a few magic vars are useful: __version__ = "version..." # I like letting CVS fill in # $Revision$ here, others don't want # the extra baggage. __author__ = "Fred L. Drake, Jr. " # just in case anyone cares... Finer-grained contribution information *can* go in comments, but *must* go into the CVS check-in messages! I notice that your patch is reversed -- use diff -c oldfile newfile You'll probably see the checkin message before this message. ;) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Fri Feb 25 16:27:16 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 25 Feb 2000 11:27:16 -0500 (EST) Subject: [Patches] moving acknowledgements from docstrings to comments References: <20000225164120.A19099@stopcontact.palga.uucp> Message-ID: <14518.44388.214657.775918@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: GH> It also changes string.index(opt, '=') to opt.index('='). Actually, this is an interesting point. How zealous should we be in using string meths? I wouldn't propose gobnourmous patches to convert the entire standard library, but when I'm hacking on a module anyway, can I do the conversion where it makes sense? -Barry From guido@python.org Fri Feb 25 16:32:11 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Feb 2000 11:32:11 -0500 Subject: [Patches] moving acknowledgements from docstrings to comments In-Reply-To: Your message of "Fri, 25 Feb 2000 11:14:46 EST." <14518.43638.699044.609025@weyr.cnri.reston.va.us> References: <20000225164120.A19099@stopcontact.palga.uucp> <14518.43638.699044.609025@weyr.cnri.reston.va.us> Message-ID: <200002251632.LAA16431@eric.cnri.reston.va.us> > This is appropriate for module source as well. I've always thought > a few magic vars are useful: > > __version__ = "version..." # I like letting CVS fill in > # $Revision$ here, others don't want > # the extra baggage. > > __author__ = "Fred L. Drake, Jr. " > # just in case anyone cares... Hm, the point of making __version__ a variable instead of a comment is that it is useful sometimes to check the version at run time. I see no such use for __author__, so I think a comment is sufficient. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Feb 25 16:30:37 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Feb 2000 11:30:37 -0500 Subject: [Patches] moving acknowledgements from docstrings to comments In-Reply-To: Your message of "Fri, 25 Feb 2000 16:41:20 +0100." <20000225164120.A19099@stopcontact.palga.uucp> References: <20000225164120.A19099@stopcontact.palga.uucp> Message-ID: <200002251630.LAA16418@eric.cnri.reston.va.us> Thanks. I agree. One nit: next time, do diff -c oldfile newfile; you somehow did a reverse diff, which is slightly disturbing to read :-) No need to resend, of course. --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Fri Feb 25 16:32:50 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 25 Feb 2000 18:32:50 +0200 (IST) Subject: [Patches] moving acknowledgements from docstrings to comments In-Reply-To: <14518.44388.214657.775918@anthem.cnri.reston.va.us> Message-ID: On Fri, 25 Feb 2000, Barry A. Warsaw wrote: > > >>>>> "GH" == Gerrit Holl writes: > > GH> It also changes string.index(opt, '=') to opt.index('='). > > Actually, this is an interesting point. How zealous should we be in > using string meths? I wouldn't propose gobnourmous patches to convert > the entire standard library, but when I'm hacking on a module anyway, > can I do the conversion where it makes sense? I think so. Anyone using an older version of Python with a newer version of the library is just begging for trouble, so you might as well give him some . If they feel more natural in context, the standard library should use them -- that's what they're there for. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@python.org Fri Feb 25 16:36:07 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Feb 2000 11:36:07 -0500 Subject: [Patches] moving acknowledgements from docstrings to comments In-Reply-To: Your message of "Fri, 25 Feb 2000 11:27:16 EST." <14518.44388.214657.775918@anthem.cnri.reston.va.us> References: <20000225164120.A19099@stopcontact.palga.uucp> <14518.44388.214657.775918@anthem.cnri.reston.va.us> Message-ID: <200002251636.LAA16487@eric.cnri.reston.va.us> > >>>>> "GH" == Gerrit Holl writes: > > GH> It also changes string.index(opt, '=') to opt.index('='). > > Actually, this is an interesting point. How zealous should we be in > using string meths? I wouldn't propose gobnourmous patches to convert > the entire standard library, but when I'm hacking on a module anyway, > can I do the conversion where it makes sense? That sounds reasonable. I think that in the sake of stability, there's little point in going around to remove all string-module references. (I'm sure that if we do this on too massive a scale, we'll end up with at least one accident -- I don't think that's worth it.) Conservatively yours, --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Feb 25 17:00:17 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Feb 2000 12:00:17 -0500 Subject: [Patches] Problem with _SC_AIO_LIST_MAX in posixmodule.c on Linux Message-ID: <200002251700.MAA19078@eric.cnri.reston.va.us> On my Linux box (uname -a: Linux loony.cnri.reston.va.us 2.0.34 #1 Fri May 8 16:05:57 EDT 1998 i586 unknown) I am unable to compile posixmodule.c, because somehow the _SC_AIO_LIST_MAX symbol, while defined, appears to have no value. I get the following errors: gcc -g -O2 -I../../Modules/../Include -I.. -DHAVE_CONFIG_H -c ../../Modules/posixmodule.c ../../Modules/posixmodule.c:3789: `_SC_AIO_LIST_MAX' undeclared here (not in a function) ../../Modules/posixmodule.c:3789: initializer element for `posix_constants_sysconf[10].value' is not constant This seems a bug in the confname.h header file, which contains: enum { ... _SC_SHARED_MEMORY_OBJECTS, #define _SC_SHARED_MEMORY_OBJECTS _SC_SHARED_MEMORY_OBJECTS _SC_AIO_LISTIO_MAX, #define _SC_AIO_LIST_MAX _SC_AIO_LIST_MAX _SC_AIO_MAX, #define _SC_AIO_MAX _SC_AIO_MAX ... }; Note that the general pattern is X, #define X X but here the enum value uses LISTIO_MAX while the #define uses LIST_MAX. Can anybody here suggest a better fix than this patch? Index: posixmodule.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/posixmodule.c,v retrieving revision 2.123 diff -c -r2.123 posixmodule.c *** posixmodule.c 2000/01/31 18:41:26 2.123 --- posixmodule.c 2000/02/25 16:50:02 *************** *** 3785,3791 **** #ifdef _SC_AIO_LISTIO_MAX {"SC_AIO_LISTIO_MAX", _SC_AIO_LISTIO_MAX}, #endif ! #ifdef _SC_AIO_LIST_MAX {"SC_AIO_LIST_MAX", _SC_AIO_LIST_MAX}, #endif #ifdef _SC_AIO_MAX --- 3785,3791 ---- #ifdef _SC_AIO_LISTIO_MAX {"SC_AIO_LISTIO_MAX", _SC_AIO_LISTIO_MAX}, #endif ! #if defined(_SC_AIO_LIST_MAX) && !defined(linux) {"SC_AIO_LIST_MAX", _SC_AIO_LIST_MAX}, #endif #ifdef _SC_AIO_MAX --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Fri Feb 25 17:10:48 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 25 Feb 2000 09:10:48 -0800 (PST) Subject: [Patches] Problem with _SC_AIO_LIST_MAX in posixmodule.c on Linux In-Reply-To: <200002251700.MAA19078@eric.cnri.reston.va.us> Message-ID: On Fri, 25 Feb 2000, Guido van Rossum wrote: >... > This seems a bug in the confname.h header file, which contains: > > enum > { > ... > _SC_SHARED_MEMORY_OBJECTS, > #define _SC_SHARED_MEMORY_OBJECTS _SC_SHARED_MEMORY_OBJECTS > _SC_AIO_LISTIO_MAX, > #define _SC_AIO_LIST_MAX _SC_AIO_LIST_MAX > _SC_AIO_MAX, > #define _SC_AIO_MAX _SC_AIO_MAX > ... > }; ... > but here the enum value uses LISTIO_MAX while the #define uses > LIST_MAX. On my Linux machine, I've got glic 2.1.2, and the above bug has been corrected. Specifically, it looks like: ... #define _SC_SHARED_MEMORY_OBJECTS _SC_SHARED_MEMORY_OBJECTS _SC_AIO_LISTIO_MAX, #define _SC_AIO_LISTIO_MAX _SC_AIO_LISTIO_MAX _SC_AIO_MAX, ... > Can anybody here suggest a better fix than this patch? I would suggest keeping the SC_AIO_LISTIO_MAX stuff, and tossing SC_AIO_LIST_MAX. > Index: posixmodule.c > =================================================================== > RCS file: /projects/cvsroot/python/dist/src/Modules/posixmodule.c,v > retrieving revision 2.123 > diff -c -r2.123 posixmodule.c > *** posixmodule.c 2000/01/31 18:41:26 2.123 > --- posixmodule.c 2000/02/25 16:50:02 > *************** > *** 3785,3791 **** > #ifdef _SC_AIO_LISTIO_MAX > {"SC_AIO_LISTIO_MAX", _SC_AIO_LISTIO_MAX}, > #endif > ! #ifdef _SC_AIO_LIST_MAX > {"SC_AIO_LIST_MAX", _SC_AIO_LIST_MAX}, > #endif In other words: toss the latter three lines, keep the first three. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gward@cnri.reston.va.us Fri Feb 25 17:12:05 2000 From: gward@cnri.reston.va.us (Greg Ward) Date: Fri, 25 Feb 2000 12:12:05 -0500 Subject: [Patches] Problem with _SC_AIO_LIST_MAX in posixmodule.c on Linux In-Reply-To: <200002251700.MAA19078@eric.cnri.reston.va.us>; from guido@python.org on Fri, Feb 25, 2000 at 12:00:17PM -0500 References: <200002251700.MAA19078@eric.cnri.reston.va.us> Message-ID: <20000225121204.F2441@cnri.reston.va.us> On 25 February 2000, Guido van Rossum said: > This seems a bug in the confname.h header file, which contains: [...] > Can anybody here suggest a better fix than this patch? Wouldn't it be better to test for the specific version of glibc that has this bug, rather than tarring all Linuces with it? Eg. change your #if defined(_SC_AIO_LIST_MAX) && !defined(linux) to #if defined(_SC_AIO_LIST_MAX) && \ !(defined (__GLIBC__) && __GLIBC__ == 2 && \ defined (__GLIBC_MINOR) && __GLIBC_MINOR == 1) ...or something like that. This assumes the bug is confined to glibc 2.1, which is the version of glibc present included with (at least) Red Hat 6.1. Greg From bwarsaw@cnri.reston.va.us Fri Feb 25 17:12:50 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Fri, 25 Feb 2000 12:12:50 -0500 (EST) Subject: [Patches] moving acknowledgements from docstrings to comments References: <20000225164120.A19099@stopcontact.palga.uucp> <14518.44388.214657.775918@anthem.cnri.reston.va.us> <200002251636.LAA16487@eric.cnri.reston.va.us> Message-ID: <14518.47122.551539.80397@anthem.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> I think that in the sake of stability, there's little point GvR> in going around to remove all string-module references. (I'm GvR> sure that if we do this on too massive a scale, we'll end up GvR> with at least one accident -- I don't think that's worth it.) Agreed completely. I think the approach should be that if you're reworking, fixing, or improving modules anyway, you have the liberty (but not the requirement) to use string methods. I'd vote for consistency though: if you're going to move toward string methods for some part of the module, you should probably do so for the entire module (hopefully allowing you to remove the import of string). >>>>> "MZ" == Moshe Zadka writes: MZ> I think so. Anyone using an older version of Python with a MZ> newer version of the library is just begging for trouble, so MZ> you might as well give him some . Just one little thing to cloud the issue. Some parts of the Python library and/or Tools and such /are/ useful outside the context of Python 1.6. E.g. I distribute the latest smtplib.py with Mailman, so converting that module to string methods would cause me pain. That's my problem though, not yours... just something to think about. -Barry From gstein@lyra.org Fri Feb 25 17:20:06 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 25 Feb 2000 09:20:06 -0800 (PST) Subject: [Patches] Problem with _SC_AIO_LIST_MAX in posixmodule.c on Linux In-Reply-To: <20000225121204.F2441@cnri.reston.va.us> Message-ID: On Fri, 25 Feb 2000, Greg Ward wrote: >... > Wouldn't it be better to test for the specific version of glibc that has > this bug, rather than tarring all Linuces with it? Eg. change your It would be, but the fix in glibc was to torch _SC_AIO_LIST_MAX altogether. No reason for us to keep it, is there? Does another OS define that one? If so, then you'll need to check for something even more refined that glibc 2.1, since it is fixed on 2.1.2. Guido: what version of glibc do you have? "rpm -q glibc-devel" will tell you. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Fri Feb 25 17:30:19 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 25 Feb 2000 09:30:19 -0800 (PST) Subject: [Patches] configure.in patch to use POSIX threads on Solaris In-Reply-To: <4.2.2.20000225110217.00cbc1c0@vcsmail> Message-ID: On Fri, 25 Feb 2000, Pat Knight wrote: >... > Justification. > Use of POSIX threads on Solaris ensures that the fork() call only runs the > thread that called fork() in the child. This is desirable to prevent (for > example) parent server or database threads running in the child. Sun's > -lthread library uses a traditional fork() which replicates all the > parent's threads in the child. I find this undesirable. +1 fyi: Apache ran into this same kind of issue (I think on AIX) for Apache 2.0 (which is threaded). The fork() used for running CGIs caused a huge overhead. As a result, Apache 2.0 actually has a "CGI daemon" process that the main server relies on to fork/exec CGI scripts. The daemon has no threads to duplicate at fork() time, so the fork-efficiency is much better. My point is that the fork() behavior Pat mentions is quite bothersome; it would be nice for Python to avoid it. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@python.org Fri Feb 25 17:47:38 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Feb 2000 12:47:38 -0500 Subject: [Patches] Problem with _SC_AIO_LIST_MAX in posixmodule.c on Linux In-Reply-To: Your message of "Fri, 25 Feb 2000 09:20:06 PST." References: Message-ID: <200002251747.MAA19342@eric.cnri.reston.va.us> > It would be, but the fix in glibc was to torch _SC_AIO_LIST_MAX > altogether. No reason for us to keep it, is there? Does another OS define > that one? Hm, it doesn't exist on Solaris either. Could it be that the symbol just shouldn't exist and that we have it only because we used the Linux confname.h to generate a list of names we should know about? I agree that the simplest fix then is to simply torch those three lines. > If so, then you'll need to check for something even more refined that > glibc 2.1, since it is fixed on 2.1.2. > > Guido: what version of glibc do you have? "rpm -q glibc-devel" will tell > you. Shouldn't matter any more, but it's glibc-devel-2.0.7-13. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Feb 25 17:49:32 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Feb 2000 12:49:32 -0500 Subject: [Patches] configure.in patch to use POSIX threads on Solaris In-Reply-To: Your message of "Fri, 25 Feb 2000 09:30:19 PST." References: Message-ID: <200002251749.MAA19353@eric.cnri.reston.va.us> > On Fri, 25 Feb 2000, Pat Knight wrote: > >... > > Justification. > > Use of POSIX threads on Solaris ensures that the fork() call only runs the > > thread that called fork() in the child. This is desirable to prevent (for > > example) parent server or database threads running in the child. Sun's > > -lthread library uses a traditional fork() which replicates all the > > parent's threads in the child. I find this undesirable. > > +1 > > fyi: Apache ran into this same kind of issue (I think on AIX) for Apache > 2.0 (which is threaded). The fork() used for running CGIs caused a huge > overhead. As a result, Apache 2.0 actually has a "CGI daemon" process that > the main server relies on to fork/exec CGI scripts. The daemon has no > threads to duplicate at fork() time, so the fork-efficiency is much > better. > > My point is that the fork() behavior Pat mentions is quite bothersome; it > would be nice for Python to avoid it. Agreed. We still can't promise this on all platforms (who will write a test program in C to determine whether fork() behaves like fork1()?) but where we can, we should promote fork==fork1. --Guido van Rossum (home page: http://www.python.org/~guido/) From davidv@elisra.com Sat Feb 26 10:39:16 2000 From: davidv@elisra.com (David Vainapel) Date: Sat, 26 Feb 2000 12:39:16 +0200 Subject: [Patches] Readline replacement under QNX in myreadline.c Message-ID: <38B7AD54.57498D82@elisra.com> Disclaimer I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. This patch for Python 1.52 , /Parser/myreadline.c on QNX using Watcom C++. Readline does not work properly . Using QNX input_line function instead of Linux readline. ------------------------------------------------------------ 30,38d29 < < #ifdef __QNX__ < p = input_line( fp, buf, len ); < if( p ) { < int n = strlen(p); < p[n] = '\n'; < p[n+1] = 0; < } < #else 40d30 < #endif ------------------------------------------------------------- Result file: /* Readline interface for tokenizer.c and [raw_]input() in bltinmodule.c. By default, or when stdin is not a tty device, we have a super simple my_readline function using fgets. Optionally, we can use the GNU readline library. my_readline() has a different return value from GNU readline(): - NULL if an interrupt occurred or if an error occurred - a malloc'ed empty string if EOF was read - a malloc'ed string ending in \n normally */ #include "Python.h" int (*PyOS_InputHook)() = NULL; /* This function restarts a fgets() after an EINTR error occurred except if PyOS_InterruptOccurred() returns true. */ static int my_fgets(buf, len, fp) char *buf; int len; FILE *fp; { char *p; for (;;) { if (PyOS_InputHook != NULL) (void)(PyOS_InputHook)(); errno = 0; #ifdef __QNX__ p = input_line( fp, buf, len ); if( p ) { int n = strlen(p); p[n] = '\n'; p[n+1] = 0; } #else p = fgets(buf, len, fp); #endif if (p != NULL) return 0; /* No error */ if (feof(fp)) { return -1; /* EOF */ } #ifdef EINTR if (errno == EINTR) { if (PyOS_InterruptOccurred()) { return 1; /* Interrupt */ } continue; } #endif if (PyOS_InterruptOccurred()) { return 1; /* Interrupt */ } return -2; /* Error */ } /* NOTREACHED */ } /* Readline implementation using fgets() */ char * PyOS_StdioReadline(prompt) char *prompt; { int n; char *p; n = 100; if ((p = malloc(n)) == NULL) return NULL; fflush(stdout); if (prompt) fprintf(stderr, "%s", prompt); fflush(stderr); switch (my_fgets(p, n, stdin)) { case 0: /* Normal case */ break; case 1: /* Interrupt */ free(p); return NULL; case -1: /* EOF */ case -2: /* Error */ default: /* Shouldn't happen */ *p = '\0'; break; } #ifdef MPW /* Hack for MPW C where the prompt comes right back in the input */ /* XXX (Actually this would be rather nice on most systems...) */ n = strlen(prompt); if (strncmp(p, prompt, n) == 0) memmove(p, p + n, strlen(p) - n + 1); #endif n = strlen(p); while (n > 0 && p[n-1] != '\n') { int incr = n+2; p = realloc(p, n + incr); if (p == NULL) return NULL; if (my_fgets(p+n, incr, stdin) != 0) break; n += strlen(p+n); } return realloc(p, n+1); } /* By initializing this function pointer, systems embedding Python can override the readline function. */ char *(*PyOS_ReadlineFunctionPointer) Py_PROTO((char *)); /* Interface used by tokenizer.c and bltinmodule.c */ char * PyOS_Readline(prompt) char *prompt; { char *rv; if (PyOS_ReadlineFunctionPointer == NULL) { PyOS_ReadlineFunctionPointer = PyOS_StdioReadline; } Py_BEGIN_ALLOW_THREADS rv = (*PyOS_ReadlineFunctionPointer)(prompt); Py_END_ALLOW_THREADS return rv; } D. Vainapel From Moshe Zadka Sat Feb 26 13:00:47 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 26 Feb 2000 15:00:47 +0200 (IST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in (fwd) Message-ID: Due to the recent mail-congealing problem on python.org, me and Greg's hashing of my patch came out in the wrong order, and probably no one managed to get any of it. Here's a short summary: I wrote a small patch so that Python types would be able to override the default "in" behaviour (going through every element and checking for equality). I've also added to the instance-type an override to check if a __contains__ method exists, and if so, use that as the function to check for "in". Greg Stein noted that if __contains__ is not found, it should fall back to the previous behaviour in that case too, and I added that. Greg was helpful enough to find several more buglets, which I corrected. Checkin message --------------- Added "in" overriding hooks to both built-in types and user-defined classes, falling back to the old behaviour (checking each item for equality) if the hooks are not used. --------------- Patch and legal disclaimer forwarded from original method follow. Any comments, questions and flames are welcome! (Well, any of the above except flames ) -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. ---------- Forwarded message ---------- Date: Sat, 19 Feb 2000 15:09:45 +0200 (IST) From: Moshe Zadka Reply-To: Moshe Zadka To: Greg Stein Cc: patches@python.org Subject: Re: [Patches] in-overloading: done right, with Guido's comments folded in OK, here's another version: (Same checkin message from last e-mail, the only change is a bug fix) -- diff -c -r python/dist/src/Include/object.h build/dist/src/Include/object.h *** python/dist/src/Include/object.h Wed Feb 2 16:02:51 2000 --- build/dist/src/Include/object.h Fri Feb 18 18:46:46 2000 *************** *** 150,155 **** --- 150,156 ---- typedef int (*getwritebufferproc) Py_PROTO((PyObject *, int, void **)); typedef int (*getsegcountproc) Py_PROTO((PyObject *, int *)); typedef int (*getcharbufferproc) Py_PROTO((PyObject *, int, const char **)); + typedef int (*objobjproc) Py_PROTO((PyObject *, PyObject *)); typedef struct { binaryfunc nb_add; *************** *** 185,190 **** --- 186,192 ---- intintargfunc sq_slice; intobjargproc sq_ass_item; intintobjargproc sq_ass_slice; + objobjproc sq_contains; } PySequenceMethods; typedef struct { *************** *** 317,323 **** /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) --- 319,329 ---- /* PyBufferProcs contains bf_getcharbuffer */ #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) ! /* PySequenceMethods contains sq_contains */ ! #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) ! ! #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER | \ ! Py_TPFLAGS_HAVE_SEQUENCE_IN) #define PyType_HasFeature(t,f) (((t)->tp_flags & (f)) != 0) diff -c -r python/dist/src/Objects/abstract.c build/dist/src/Objects/abstract.c *** python/dist/src/Objects/abstract.c Fri Feb 18 09:06:45 2000 --- build/dist/src/Objects/abstract.c Fri Feb 18 18:47:23 2000 *************** *** 1121,1127 **** } return 0; } ! sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, --- 1121,1134 ---- } return 0; } ! if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { ! sq = w->ob_type->tp_as_sequence; ! if(sq != NULL && sq->sq_contains != NULL) ! return (*sq->sq_contains)(w, v); ! } ! ! /* If there is no better way to check whether an item is is contained, ! do it the hard way */ sq = w->ob_type->tp_as_sequence; if (sq == NULL || sq->sq_item == NULL) { PyErr_SetString(PyExc_TypeError, diff -c -r python/dist/src/Objects/classobject.c build/dist/src/Objects/classobject.c *** python/dist/src/Objects/classobject.c Fri Feb 18 09:06:49 2000 --- build/dist/src/Objects/classobject.c Sat Feb 19 03:17:46 2000 *************** *** 1065,1070 **** --- 1065,1123 ---- return 0; } + static int instance_contains(PyInstanceObject *inst, PyObject *member) + { + static PyObject *__contains__; + PyObject *func, *arg, *res; + int ret; + + if(__contains__ == NULL) { + __contains__ = PyString_InternFromString("__contains__"); + if(__contains__ == NULL) + return -1; + } + func = instance_getattr(inst, __contains__); + if(func == NULL) { + /* fall back to previous behaviour */ + int i, cmp_res; + + if(!PyErr_ExceptionMatches(PyExc_AttributeError)) + return -1; + PyErr_Clear(); + for(i=0;;i++) { + PyObject *obj = instance_item(inst, i); + int ret = 0; + + if(obj == NULL) { + if(!PyErr_ExceptionMatches(PyExc_IndexError)) + return -1; + PyErr_Clear(); + return 0; + } + if(PyObject_Cmp(obj, member, &cmp_res) == -1) + ret = -1; + if(cmp_res == 0) + ret = 1; + Py_DECREF(obj); + if(ret) + return ret; + } + } + arg = Py_BuildValue("(O)", member); + if(arg == NULL) { + Py_DECREF(func); + return -1; + } + res = PyEval_CallObject(func, arg); + Py_DECREF(func); + Py_DECREF(arg); + if(res == NULL) + return -1; + ret = PyObject_IsTrue(res); + Py_DECREF(res); + return ret; + } + static PySequenceMethods instance_as_sequence = { (inquiry)instance_length, /*sq_length*/ 0, /*sq_concat*/ *************** *** 1073,1078 **** --- 1126,1132 ---- (intintargfunc)instance_slice, /*sq_slice*/ (intobjargproc)instance_ass_item, /*sq_ass_item*/ (intintobjargproc)instance_ass_slice, /*sq_ass_slice*/ + (objobjproc)instance_contains, /* sq_contains */ }; static PyObject * *************** *** 1405,1410 **** --- 1459,1466 ---- 0, /*tp_str*/ (getattrofunc)instance_getattr, /*tp_getattro*/ (setattrofunc)instance_setattr, /*tp_setattro*/ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /*tp_flags */ }; --- -- Legalese: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@python.org Sat Feb 26 16:20:43 2000 From: guido@python.org (Guido van Rossum) Date: Sat, 26 Feb 2000 11:20:43 -0500 Subject: [Patches] Readline replacement under QNX in myreadline.c In-Reply-To: Your message of "Sat, 26 Feb 2000 12:39:16 +0200." <38B7AD54.57498D82@elisra.com> References: <38B7AD54.57498D82@elisra.com> Message-ID: <200002261620.LAA21156@eric.cnri.reston.va.us> > This patch for Python 1.52 , /Parser/myreadline.c on QNX using Watcom > C++. > Readline does not work properly . > Using QNX input_line function instead of Linux readline. > ------------------------------------------------------------ > 30,38d29 > < > < #ifdef __QNX__ > < p = input_line( fp, buf, len ); > < if( p ) { > < int n = strlen(p); > < p[n] = '\n'; > < p[n+1] = 0; > < } > < #else > 40d30 > < #endif > ------------------------------------------------------------- I seem to recall that this came up recently but I don't remember where. Can anybody jog my memory? What did we decide in the end? --Guido van Rossum (home page: http://www.python.org/~guido/) From Gerrit Sun Feb 27 11:14:31 2000 From: Gerrit (Gerrit Holl) Date: Sun, 27 Feb 2000 12:14:31 +0100 Subject: [Patches] SyntaxError in Tkinter Message-ID: <20000227121431.A19168@stopcontact.palga.uucp> Hello, I finally installed the right headers to run Tkinter to see Idle's state (I used to install by RPM but wanted to run 1.5.2+), and I ran on a SyntaxError (!) in Tkinter: >>> import Tkinter Traceback (innermost last): File "", line 1, in ? File "/usr/local/lib/python1.5/lib-tk/Tkinter.py", line 1813 if kwargs.has_key('command') ^ SyntaxError: invalid syntax you-haven't-ripped-off-the-semicolon,-have-you?-ly y'rs - gerrit From Gerrit Sun Feb 27 09:39:15 2000 From: Gerrit (Gerrit Holl) Date: Sun, 27 Feb 2000 10:39:15 +0100 Subject: [Patches] Acknowledgements: docstrings -> comments Message-ID: <20000227103915.A1526@stopcontact.palga.uucp> --UugvWAfsgieZRqgk Content-Type: text/plain; charset=us-ascii Hello, the attached patch moves all acknowledgements for authors from docstrings to normal comments. It also adds one line to the docstring of StringIO, namely that cStringIO is also available. I did not do it extremely careful, so it's possible that there's an acknowledgement left in a docstring (e.g., I only did module docstrings). Output of 'diff -cr /tmp/Lib/ .' is attached, where /tmp/Lib contains the old standard library and . the changed one. DISCLAIMER ~~~~~~~~~~ I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. regards, Gerrit. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --UugvWAfsgieZRqgk Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="acks.diff" diff -cr /tmp/Lib/StringIO.py ./StringIO.py *** /tmp/Lib/StringIO.py Wed Feb 2 16:10:14 2000 --- ./StringIO.py Sun Feb 27 10:13:36 2000 *************** *** 19,24 **** --- 19,26 ---- Notes: - Using a real file is often faster (but less convenient). + - There's also a much faster implementation in C, called cStringIO, but + it's not subclassable. - fileno() is left unimplemented so that code which uses it triggers an exception early. - Seeking far beyond EOF and then writing will insert real null diff -cr /tmp/Lib/cgi.py ./cgi.py *** /tmp/Lib/cgi.py Fri Feb 25 12:44:03 2000 --- ./cgi.py Sun Feb 27 10:15:20 2000 *************** *** 399,420 **** - Don't try to give a CGI script a set-uid mode. This doesn't work on most systems, and is a security liability as well. - - History - ------- - - Michael McLay started this module. Steve Majewski changed the - interface to SvFormContentDict and FormContentDict. The multipart - parsing was inspired by code submitted by Andreas Paepcke. Guido van - Rossum rewrote, reformatted and documented the module and is currently - responsible for its maintenance. - - XXX The module is getting pretty heavy with all those docstrings. Perhaps there should be a slimmed version that doesn't contain all those backwards compatible and debugging classes and functions? """ __version__ = "2.2" --- 399,419 ---- - Don't try to give a CGI script a set-uid mode. This doesn't work on most systems, and is a security liability as well. XXX The module is getting pretty heavy with all those docstrings. Perhaps there should be a slimmed version that doesn't contain all those backwards compatible and debugging classes and functions? """ + + # History + # ------- + # + # Michael McLay started this module. Steve Majewski changed the + # interface to SvFormContentDict and FormContentDict. The multipart + # parsing was inspired by code submitted by Andreas Paepcke. Guido van + # Rossum rewrote, reformatted and documented the module and is currently + # responsible for its maintenance. + # __version__ = "2.2" diff -cr /tmp/Lib/code.py ./code.py *** /tmp/Lib/code.py Thu Oct 22 23:56:44 1998 --- ./code.py Sun Feb 27 10:15:44 2000 *************** *** 1,7 **** """Utilities needed to emulate Python's interactive interpreter. - Inspired by similar code by Jeff Epler and Fredrik Lundh. """ import sys --- 1,8 ---- """Utilities needed to emulate Python's interactive interpreter. """ + + # Inspired by similar code by Jeff Epler and Fredrik Lundh. import sys diff -cr /tmp/Lib/ftplib.py ./ftplib.py *** /tmp/Lib/ftplib.py Fri Feb 4 16:39:29 2000 --- ./ftplib.py Sun Feb 27 10:17:39 2000 *************** *** 1,12 **** """An FTP client class and some helper functions. Based on RFC 959: File Transfer Protocol - (FTP), by J. Postel and J. Reynolds - - Changes and improvements suggested by Steve Majewski. - Modified by Jack to work on the mac. - Modified by Siebren to support docstrings and PASV. - Example: --- 1,6 ---- *************** *** 33,38 **** --- 27,38 ---- A nice test that reveals some of the network dialogue would be: python ftplib.py -d localhost -l -p -l """ + + # (FTP), by J. Postel and J. Reynolds + # + # Changes and improvements suggested by Steve Majewski. + # Modified by Jack to work on the mac. + # Modified by Siebren to support docstrings and PASV. import os diff -cr /tmp/Lib/getpass.py ./getpass.py *** /tmp/Lib/getpass.py Tue Oct 19 00:25:22 1999 --- ./getpass.py Sun Feb 27 10:17:56 2000 *************** *** 6,14 **** On Windows, the msvcrt module will be used. On the Mac EasyDialogs.AskPassword is used, if available. - Authors: Piers Lauder (original) - Guido van Rossum (Windows support and cleanup) """ import sys --- 6,15 ---- On Windows, the msvcrt module will be used. On the Mac EasyDialogs.AskPassword is used, if available. """ + + # Authors: Piers Lauder (original) + # Guido van Rossum (Windows support and cleanup) import sys diff -cr /tmp/Lib/imaplib.py ./imaplib.py *** /tmp/Lib/imaplib.py Thu Feb 24 03:24:50 2000 --- ./imaplib.py Sun Feb 27 10:18:53 2000 *************** *** 3,12 **** Based on RFC 2060. - Author: Piers Lauder December 1997. - - Authentication code contributed by Donn Cave June 1998. - Public class: IMAP4 Public variable: Debug Public functions: Internaldate2tuple --- 3,8 ---- *************** *** 14,19 **** --- 10,19 ---- ParseFlags Time2Internaldate """ + + # Author: Piers Lauder December 1997. + # + # Authentication code contributed by Donn Cave June 1998. __version__ = "2.32" diff -cr /tmp/Lib/poplib.py ./poplib.py *** /tmp/Lib/poplib.py Thu May 13 20:47:25 1999 --- ./poplib.py Sun Feb 27 10:21:02 2000 *************** *** 1,11 **** """A POP3 client class. Based on the J. Myers POP3 draft, Jan. 96 - - Author: David Ascher - [heavily stealing from nntplib.py] - Updated: Piers Lauder [Jul '97] """ # Example (see the test function at the end of this file) --- 1,11 ---- """A POP3 client class. Based on the J. Myers POP3 draft, Jan. 96 """ + + # Author: David Ascher + # [heavily stealing from nntplib.py] + # Updated: Piers Lauder [Jul '97] # Example (see the test function at the end of this file) diff -cr /tmp/Lib/smtplib.py ./smtplib.py *** /tmp/Lib/smtplib.py Fri Feb 4 16:39:30 2000 --- ./smtplib.py Sun Feb 27 10:12:15 2000 *************** *** 2,15 **** """SMTP/ESMTP client class. - Author: The Dragon De Monsyne - ESMTP support, test code and doc fixes added by - Eric S. Raymond - Better RFC 821 compliance (MAIL and RCPT, and CRLF in data) - by Carey Evans , for picky mail servers. - - This was modified from the Python 1.5 library HTTP lib. - This should follow RFC 821 (SMTP) and RFC 1869 (ESMTP). Notes: --- 2,7 ---- *************** *** 38,43 **** --- 30,43 ---- (250, "Somebody OverHere ") >>> s.quit() """ + + # Author: The Dragon De Monsyne + # ESMTP support, test code and doc fixes added by + # Eric S. Raymond + # Better RFC 821 compliance (MAIL and RCPT, and CRLF in data) + # by Carey Evans , for picky mail servers. + # + # This was modified from the Python 1.5 library HTTP lib. import socket import string diff -cr /tmp/Lib/telnetlib.py ./telnetlib.py *** /tmp/Lib/telnetlib.py Thu Dec 24 00:04:17 1998 --- ./telnetlib.py Sun Feb 27 10:23:53 2000 *************** *** 1,7 **** """TELNET client class. ! Based on RFC 854: TELNET Protocol Specification, by J. Postel and ! J. Reynolds Example: --- 1,6 ---- """TELNET client class. ! Based on RFC 854: TELNET Protocol Specification Example: *************** *** 35,40 **** --- 34,40 ---- """ + # by J. Postel and J. Reynolds # Imported modules import sys diff -cr /tmp/Lib/urlparse.py ./urlparse.py *** /tmp/Lib/urlparse.py Fri Feb 4 16:28:42 2000 --- ./urlparse.py Sun Feb 27 10:25:20 2000 *************** *** 1,8 **** """Parse (absolute and relative) URLs. ! See RFC 1808: "Relative Uniform Resource Locators", by R. Fielding, ! UC Irvine, June 1995. """ # Standard/builtin Python modules import string --- 1,9 ---- """Parse (absolute and relative) URLs. ! See RFC 1808: "Relative Uniform Resource Locators" """ + + # by R. Fielding, UC Irvine, June 1995. # Standard/builtin Python modules import string diff -cr /tmp/Lib/whrandom.py ./whrandom.py *** /tmp/Lib/whrandom.py Fri Feb 4 16:28:42 2000 --- ./whrandom.py Sun Feb 27 10:25:41 2000 *************** *** 25,34 **** random generators, and to choose from other ranges. - Translated by Guido van Rossum from C source provided by - Adrian Baddeley. - - Multi-threading note: the random number generator used here is not thread-safe; it is possible that nearly simultaneous calls in --- 25,30 ---- *************** *** 36,41 **** --- 32,41 ---- have to use a lock around all calls. (I didn't want to slow this down in the serial case by using a lock here.) """ + + # Translated by Guido van Rossum from C source provided by + # Adrian Baddeley. + class whrandom: def __init__(self, x = 0, y = 0, z = 0): --UugvWAfsgieZRqgk-- From m.favas@per.dem.csiro.au Sun Feb 27 13:00:13 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Sun, 27 Feb 2000 21:00:13 +0800 Subject: [Patches] Bug in CVS version of Tkinter.py - missing ":" on line 1813 Message-ID: <38B91FDD.7DF0976E@per.dem.csiro.au> The CVS version of Tkinter.py has a missing ":" at the end of an if... patch is: *** Tkinter.py.orig Sun Feb 27 20:49:37 2000 --- Tkinter.py Sun Feb 27 20:49:55 2000 *************** *** 1810,1816 **** self.menuname = menu._w # 'command' is the only supported keyword callback = kwargs.get('command') ! if kwargs.has_key('command') del kwargs['command'] if kwargs: raise TclError, 'unknown option -'+kwargs.keys()[0] --- 1810,1816 ---- self.menuname = menu._w # 'command' is the only supported keyword callback = kwargs.get('command') ! if kwargs.has_key('command'): del kwargs['command'] if kwargs: raise TclError, 'unknown option -'+kwargs.keys()[0] I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Email - m.favas@per.dem.csiro.au Mark C Favas Phone - +61 8 9333 6268, 0418 926 074 CSIRO Exploration & Mining Fax - +61 8 9383 9891 Private Bag Post Office Wembley GPS - 31.97 S, 115.81 E Western Australia 6014 From m.favas@per.dem.csiro.au Sun Feb 27 13:04:58 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Sun, 27 Feb 2000 21:04:58 +0800 Subject: [Patches] fmt.py - arg to blah.append() not a tuple Message-ID: <38B920FA.BF70C065@per.dem.csiro.au> I know this module is in Lib/lib-old and hence deprecated, but since it's still being distributed, it might as well be fixed . Patch is: *** fmt.py.orig Sun Feb 27 21:00:48 2000 --- fmt.py Sun Feb 27 21:01:46 2000 *************** *** 222,230 **** self.para.just = self.just self.nextfont = self.font space = int(space * self.space) ! self.para.words.append(self.nextfont, text, \ self.d.textwidth(text), space, space, \ ! self.ascent, self.descent) self.nextfont = None # def bgn_anchor(self, id): --- 222,230 ---- self.para.just = self.just self.nextfont = self.font space = int(space * self.space) ! self.para.words.append((self.nextfont, text, \ self.d.textwidth(text), space, space, \ ! self.ascent, self.descent)) self.nextfont = None # def bgn_anchor(self, id): I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Email - m.favas@per.dem.csiro.au Mark C Favas Phone - +61 8 9333 6268, 0418 926 074 CSIRO Exploration & Mining Fax - +61 8 9383 9891 Private Bag Post Office Wembley GPS - 31.97 S, 115.81 E Western Australia 6014 From guido@python.org Sun Feb 27 15:37:30 2000 From: guido@python.org (Guido van Rossum) Date: Sun, 27 Feb 2000 10:37:30 -0500 Subject: [Patches] Bug in CVS version of Tkinter.py - missing ":" on line 1813 In-Reply-To: Your message of "Sun, 27 Feb 2000 21:00:13 +0800." <38B91FDD.7DF0976E@per.dem.csiro.au> References: <38B91FDD.7DF0976E@per.dem.csiro.au> Message-ID: <200002271537.KAA21735@eric.cnri.reston.va.us> Thanks, Mark, for fixing this. And thanks to Gerrit for reporting it first. :-) Also thanks to Mark for the fmt.py patch. Wow, it must be the weekend of trivial patches, eh? :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From Gerrit Sun Feb 27 09:39:15 2000 From: Gerrit (Gerrit Holl) Date: Sun, 27 Feb 2000 10:39:15 +0100 Subject: [Patches] Acknowledgements: docstrings -> comments Message-ID: <20000227103915.A1526@stopcontact.palga.uucp> --sm4nu43k4a2Rpi4c Content-Type: text/plain; charset=us-ascii [according to the archives, this mail didn't reach the list, if it did - sorry] Hello, the attached patch moves all acknowledgements for authors from docstrings to normal comments. It also adds one line to the docstring of StringIO, namely that cStringIO is also available. I did not do it extremely careful, so it's possible that there's an acknowledgement left in a docstring (e.g., I only did module docstrings). Output of 'diff -cr /tmp/Lib/ .' is attached, where /tmp/Lib contains the old standard library and . the changed one. DISCLAIMER ~~~~~~~~~~ I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. regards, Gerrit. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --sm4nu43k4a2Rpi4c Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="acks.YMd3oX.diff" diff -cr /tmp/Lib/StringIO.py ./StringIO.py *** /tmp/Lib/StringIO.py Wed Feb 2 16:10:14 2000 --- ./StringIO.py Sun Feb 27 10:13:36 2000 *************** *** 19,24 **** --- 19,26 ---- Notes: - Using a real file is often faster (but less convenient). + - There's also a much faster implementation in C, called cStringIO, but + it's not subclassable. - fileno() is left unimplemented so that code which uses it triggers an exception early. - Seeking far beyond EOF and then writing will insert real null diff -cr /tmp/Lib/cgi.py ./cgi.py *** /tmp/Lib/cgi.py Fri Feb 25 12:44:03 2000 --- ./cgi.py Sun Feb 27 10:15:20 2000 *************** *** 399,420 **** - Don't try to give a CGI script a set-uid mode. This doesn't work on most systems, and is a security liability as well. - - History - ------- - - Michael McLay started this module. Steve Majewski changed the - interface to SvFormContentDict and FormContentDict. The multipart - parsing was inspired by code submitted by Andreas Paepcke. Guido van - Rossum rewrote, reformatted and documented the module and is currently - responsible for its maintenance. - - XXX The module is getting pretty heavy with all those docstrings. Perhaps there should be a slimmed version that doesn't contain all those backwards compatible and debugging classes and functions? """ __version__ = "2.2" --- 399,419 ---- - Don't try to give a CGI script a set-uid mode. This doesn't work on most systems, and is a security liability as well. XXX The module is getting pretty heavy with all those docstrings. Perhaps there should be a slimmed version that doesn't contain all those backwards compatible and debugging classes and functions? """ + + # History + # ------- + # + # Michael McLay started this module. Steve Majewski changed the + # interface to SvFormContentDict and FormContentDict. The multipart + # parsing was inspired by code submitted by Andreas Paepcke. Guido van + # Rossum rewrote, reformatted and documented the module and is currently + # responsible for its maintenance. + # __version__ = "2.2" diff -cr /tmp/Lib/code.py ./code.py *** /tmp/Lib/code.py Thu Oct 22 23:56:44 1998 --- ./code.py Sun Feb 27 10:15:44 2000 *************** *** 1,7 **** """Utilities needed to emulate Python's interactive interpreter. - Inspired by similar code by Jeff Epler and Fredrik Lundh. """ import sys --- 1,8 ---- """Utilities needed to emulate Python's interactive interpreter. """ + + # Inspired by similar code by Jeff Epler and Fredrik Lundh. import sys diff -cr /tmp/Lib/ftplib.py ./ftplib.py *** /tmp/Lib/ftplib.py Fri Feb 4 16:39:29 2000 --- ./ftplib.py Sun Feb 27 10:17:39 2000 *************** *** 1,12 **** """An FTP client class and some helper functions. Based on RFC 959: File Transfer Protocol - (FTP), by J. Postel and J. Reynolds - - Changes and improvements suggested by Steve Majewski. - Modified by Jack to work on the mac. - Modified by Siebren to support docstrings and PASV. - Example: --- 1,6 ---- *************** *** 33,38 **** --- 27,38 ---- A nice test that reveals some of the network dialogue would be: python ftplib.py -d localhost -l -p -l """ + + # (FTP), by J. Postel and J. Reynolds + # + # Changes and improvements suggested by Steve Majewski. + # Modified by Jack to work on the mac. + # Modified by Siebren to support docstrings and PASV. import os diff -cr /tmp/Lib/getpass.py ./getpass.py *** /tmp/Lib/getpass.py Tue Oct 19 00:25:22 1999 --- ./getpass.py Sun Feb 27 10:17:56 2000 *************** *** 6,14 **** On Windows, the msvcrt module will be used. On the Mac EasyDialogs.AskPassword is used, if available. - Authors: Piers Lauder (original) - Guido van Rossum (Windows support and cleanup) """ import sys --- 6,15 ---- On Windows, the msvcrt module will be used. On the Mac EasyDialogs.AskPassword is used, if available. """ + + # Authors: Piers Lauder (original) + # Guido van Rossum (Windows support and cleanup) import sys diff -cr /tmp/Lib/imaplib.py ./imaplib.py *** /tmp/Lib/imaplib.py Thu Feb 24 03:24:50 2000 --- ./imaplib.py Sun Feb 27 10:18:53 2000 *************** *** 3,12 **** Based on RFC 2060. - Author: Piers Lauder December 1997. - - Authentication code contributed by Donn Cave June 1998. - Public class: IMAP4 Public variable: Debug Public functions: Internaldate2tuple --- 3,8 ---- *************** *** 14,19 **** --- 10,19 ---- ParseFlags Time2Internaldate """ + + # Author: Piers Lauder December 1997. + # + # Authentication code contributed by Donn Cave June 1998. __version__ = "2.32" diff -cr /tmp/Lib/poplib.py ./poplib.py *** /tmp/Lib/poplib.py Thu May 13 20:47:25 1999 --- ./poplib.py Sun Feb 27 10:21:02 2000 *************** *** 1,11 **** """A POP3 client class. Based on the J. Myers POP3 draft, Jan. 96 - - Author: David Ascher - [heavily stealing from nntplib.py] - Updated: Piers Lauder [Jul '97] """ # Example (see the test function at the end of this file) --- 1,11 ---- """A POP3 client class. Based on the J. Myers POP3 draft, Jan. 96 """ + + # Author: David Ascher + # [heavily stealing from nntplib.py] + # Updated: Piers Lauder [Jul '97] # Example (see the test function at the end of this file) diff -cr /tmp/Lib/smtplib.py ./smtplib.py *** /tmp/Lib/smtplib.py Fri Feb 4 16:39:30 2000 --- ./smtplib.py Sun Feb 27 10:12:15 2000 *************** *** 2,15 **** """SMTP/ESMTP client class. - Author: The Dragon De Monsyne - ESMTP support, test code and doc fixes added by - Eric S. Raymond - Better RFC 821 compliance (MAIL and RCPT, and CRLF in data) - by Carey Evans , for picky mail servers. - - This was modified from the Python 1.5 library HTTP lib. - This should follow RFC 821 (SMTP) and RFC 1869 (ESMTP). Notes: --- 2,7 ---- *************** *** 38,43 **** --- 30,43 ---- (250, "Somebody OverHere ") >>> s.quit() """ + + # Author: The Dragon De Monsyne + # ESMTP support, test code and doc fixes added by + # Eric S. Raymond + # Better RFC 821 compliance (MAIL and RCPT, and CRLF in data) + # by Carey Evans , for picky mail servers. + # + # This was modified from the Python 1.5 library HTTP lib. import socket import string diff -cr /tmp/Lib/telnetlib.py ./telnetlib.py *** /tmp/Lib/telnetlib.py Thu Dec 24 00:04:17 1998 --- ./telnetlib.py Sun Feb 27 10:23:53 2000 *************** *** 1,7 **** """TELNET client class. ! Based on RFC 854: TELNET Protocol Specification, by J. Postel and ! J. Reynolds Example: --- 1,6 ---- """TELNET client class. ! Based on RFC 854: TELNET Protocol Specification Example: *************** *** 35,40 **** --- 34,40 ---- """ + # by J. Postel and J. Reynolds # Imported modules import sys diff -cr /tmp/Lib/urlparse.py ./urlparse.py *** /tmp/Lib/urlparse.py Fri Feb 4 16:28:42 2000 --- ./urlparse.py Sun Feb 27 10:25:20 2000 *************** *** 1,8 **** """Parse (absolute and relative) URLs. ! See RFC 1808: "Relative Uniform Resource Locators", by R. Fielding, ! UC Irvine, June 1995. """ # Standard/builtin Python modules import string --- 1,9 ---- """Parse (absolute and relative) URLs. ! See RFC 1808: "Relative Uniform Resource Locators" """ + + # by R. Fielding, UC Irvine, June 1995. # Standard/builtin Python modules import string diff -cr /tmp/Lib/whrandom.py ./whrandom.py *** /tmp/Lib/whrandom.py Fri Feb 4 16:28:42 2000 --- ./whrandom.py Sun Feb 27 10:25:41 2000 *************** *** 25,34 **** random generators, and to choose from other ranges. - Translated by Guido van Rossum from C source provided by - Adrian Baddeley. - - Multi-threading note: the random number generator used here is not thread-safe; it is possible that nearly simultaneous calls in --- 25,30 ---- *************** *** 36,41 **** --- 32,41 ---- have to use a lock around all calls. (I didn't want to slow this down in the serial case by using a lock here.) """ + + # Translated by Guido van Rossum from C source provided by + # Adrian Baddeley. + class whrandom: def __init__(self, x = 0, y = 0, z = 0): --sm4nu43k4a2Rpi4c-- From Gerrit Mon Feb 28 08:35:36 2000 From: Gerrit (Gerrit Holl) Date: Mon, 28 Feb 2000 09:35:36 +0100 Subject: [Patches] Re: Acknowledgements: docstrings -> comments In-Reply-To: <20000227103915.A1526@stopcontact.palga.uucp>; from gerrit on Sun, Feb 27, 2000 at 10:39:15AM +0100 References: <20000227103915.A1526@stopcontact.palga.uucp> Message-ID: <20000228093536.A1596@stopcontact.palga.uucp> > [according to the archives, this mail didn't reach the list, if it did - sorry] Oh, when I look again, I see it did - sorry! regards, Gerrit. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! From Gerrit Mon Feb 28 10:50:14 2000 From: Gerrit (Gerrit Holl) Date: Mon, 28 Feb 2000 11:50:14 +0100 Subject: [Patches] optimizing posixpath.walk Message-ID: <20000228115014.A9702@stopcontact.palga.uucp> --/04w6evG8XlLl3ft Content-Type: text/plain; charset=us-ascii Hello, the posixpath.walk function contains the following code: if isdir(name) and not islink(name): This is, of course, very inefficient: it does os.stat on the same file twice directly after each other. So I changed this code to: st = os.lstat(name) if stat.S_ISDIR(st[stat.ST_MODE]): Must be much faster... minor context diff attached. ------------------------------------------------------------------ | I confirm that, to the best of my knowledge and belief, this | | contribution is free of any claims of third parties under | | copyright, patent or other rights or interests ("claims"). To | | the extent that I have any such claims, I hereby grant to CNRI a | | nonexclusive, irrevocable, royalty-free, worldwide license to | | reproduce, distribute, perform and/or display publicly, prepare | | derivative versions, and otherwise use this contribution as part | | of the Python software and its related documentation, or any | | derivative versions thereof, at no cost to CNRI or its licensed | | users, and to authorize others to do so. | | | | I acknowledge that CNRI may, at its sole discretion, decide | | whether or not to incorporate this contribution in the Python | | software and its related documentation. I further grant CNRI | | permission to use my name and other identifying information | | provided to CNRI by me for use in connection with the Python | | software and its related documentation. | ------------------------------------------------------------------ I sent those two patches in two different mails because of the different nature of those patches: you might want to refuse the former and accept the latter or vica versa. Do you agree with this decision? Or should I just put them in one email message? regards, Gerrit. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --/04w6evG8XlLl3ft Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="posixpath.py.diff" *** /tmp/posixpath.py Mon Feb 28 11:21:52 2000 --- posixpath.py Mon Feb 28 11:32:55 2000 *************** *** 267,273 **** for name in names: if name not in exceptions: name = join(top, name) ! if isdir(name) and not islink(name): walk(name, func, arg) --- 267,274 ---- for name in names: if name not in exceptions: name = join(top, name) ! st = os.lstat(name) ! if stat.S_ISDIR(st[stat.ST_MODE]): walk(name, func, arg) --/04w6evG8XlLl3ft-- From Gerrit Mon Feb 28 10:05:14 2000 From: Gerrit (Gerrit Holl) Date: Mon, 28 Feb 2000 11:05:14 +0100 Subject: [Patches] append(a,b) -> append((a, b)) Message-ID: <20000228110514.A2021@stopcontact.palga.uucp> --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Hello, while grepping, I found three modules where the old way to use append was still used. This patch fixes it. The involved modules are toaiff.py, ConfigParser.py and pipes.py. DISCLAIMER I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. regards, Gerrit. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="ConfigParser.py.diff" *** /tmp/ConfigParser.py Mon Feb 28 10:59:04 2000 --- ConfigParser.py Mon Feb 28 10:59:25 2000 *************** *** 370,376 **** # list of all bogus lines if not e: e = ParsingError(fpname) ! e.append(lineno, `line`) # if any parsing errors occurred, raise an exception if e: raise e --- 370,376 ---- # list of all bogus lines if not e: e = ParsingError(fpname) ! e.append((lineno, `line`)) # if any parsing errors occurred, raise an exception if e: raise e --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="toaiff.py.diff" *** /tmp/toaiff.py Mon Feb 28 10:59:04 2000 --- toaiff.py Mon Feb 28 11:00:38 2000 *************** *** 16,22 **** table = {} t = pipes.Template() ! t.append('sox -t au - -t aiff -r 8000 -', '--') table['au'] = t # XXX The following is actually sub-optimal. --- 16,22 ---- table = {} t = pipes.Template() ! t.append(('sox -t au - -t aiff -r 8000 -', '--')) table['au'] = t # XXX The following is actually sub-optimal. *************** *** 26,56 **** # XXX sampled at 11k are unnecessarily expanded. # XXX Similar comments apply to some other file types. t = pipes.Template() ! t.append('sox -t hcom - -t aiff -r 22050 -', '--') table['hcom'] = t t = pipes.Template() ! t.append('sox -t voc - -t aiff -r 11025 -', '--') table['voc'] = t t = pipes.Template() ! t.append('sox -t wav - -t aiff -', '--') table['wav'] = t t = pipes.Template() ! t.append('sox -t 8svx - -t aiff -r 16000 -', '--') table['8svx'] = t t = pipes.Template() ! t.append('sox -t sndt - -t aiff -r 16000 -', '--') table['sndt'] = t t = pipes.Template() ! t.append('sox -t sndr - -t aiff -r 16000 -', '--') table['sndr'] = t uncompress = pipes.Template() ! uncompress.append('uncompress', '--') error = 'toaiff.error' # Exception --- 26,56 ---- # XXX sampled at 11k are unnecessarily expanded. # XXX Similar comments apply to some other file types. t = pipes.Template() ! t.append(('sox -t hcom - -t aiff -r 22050 -', '--')) table['hcom'] = t t = pipes.Template() ! t.append(('sox -t voc - -t aiff -r 11025 -', '--')) table['voc'] = t t = pipes.Template() ! t.append(('sox -t wav - -t aiff -', '--')) table['wav'] = t t = pipes.Template() ! t.append(('sox -t 8svx - -t aiff -r 16000 -', '--')) table['8svx'] = t t = pipes.Template() ! t.append(('sox -t sndt - -t aiff -r 16000 -', '--')) table['sndt'] = t t = pipes.Template() ! t.append(('sox -t sndr - -t aiff -r 16000 -', '--')) table['sndr'] = t uncompress = pipes.Template() ! uncompress.append(('uncompress', '--')) error = 'toaiff.error' # Exception --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="pipes.py.diff" *** /tmp/pipes.py Mon Feb 28 10:59:04 2000 --- pipes.py Mon Feb 28 11:00:03 2000 *************** *** 287,296 **** def test(): print 'Testing...' t = Template() ! t.append('togif $IN $OUT', 'ff') ! t.append('giftoppm', '--') ! t.append('ppmtogif >$OUT', '-f') ! t.append('fromgif $IN $OUT', 'ff') t.debug(1) FILE = '/usr/local/images/rgb/rogues/guido.rgb' t.copy(FILE, '@temp') --- 287,296 ---- def test(): print 'Testing...' t = Template() ! t.append(('togif $IN $OUT', 'ff')) ! t.append(('giftoppm', '--')) ! t.append(('ppmtogif >$OUT', '-f')) ! t.append(('fromgif $IN $OUT', 'ff')) t.debug(1) FILE = '/usr/local/images/rgb/rogues/guido.rgb' t.copy(FILE, '@temp') --tThc/1wpZn/ma/RB-- From guido@python.org Mon Feb 28 14:16:36 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 09:16:36 -0500 Subject: [Patches] append(a,b) -> append((a, b)) In-Reply-To: Your message of "Mon, 28 Feb 2000 11:05:14 +0100." <20000228110514.A2021@stopcontact.palga.uucp> References: <20000228110514.A2021@stopcontact.palga.uucp> Message-ID: <200002281416.JAA22632@eric.cnri.reston.va.us> > while grepping, I found three modules where the old way to use append > was still used. This patch fixes it. The involved modules are toaiff.py, > ConfigParser.py and pipes.py. Gerrit, did you test any of these patches? They are all wrong! I'll leave it to you to discover why. Hint: only list.append() was changed. --Guido van Rossum (home page: http://www.python.org/~guido/) From gerrit@nl.linux.org Mon Feb 28 14:27:47 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Mon, 28 Feb 2000 15:27:47 +0100 Subject: [Patches] append(a,b) -> append((a, b)) In-Reply-To: <200002281416.JAA22632@eric.cnri.reston.va.us>; from guido@python.org on Mon, Feb 28, 2000 at 09:16:36AM -0500 References: <20000228110514.A2021@stopcontact.palga.uucp> <200002281416.JAA22632@eric.cnri.reston.va.us> Message-ID: <20000228152747.A10344@humbolt.nl.linux.org> On Mon, Feb 28, 2000 at 09:16:36AM -0500, Guido van Rossum wrote: > > while grepping, I found three modules where the old way to use append > > was still used. This patch fixes it. The involved modules are toaiff.py, > > ConfigParser.py and pipes.py. > > Gerrit, did you test any of these patches? > > They are all wrong! > > I'll leave it to you to discover why. > > Hint: only list.append() was changed. Ooooops! Sorry! I was too enthuishast...... Sorry! regards, Gerrit. -- -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- From guido@python.org Mon Feb 28 14:28:46 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 09:28:46 -0500 Subject: [Patches] optimizing posixpath.walk In-Reply-To: Your message of "Mon, 28 Feb 2000 11:50:14 +0100." <20000228115014.A9702@stopcontact.palga.uucp> References: <20000228115014.A9702@stopcontact.palga.uucp> Message-ID: <200002281428.JAA22704@eric.cnri.reston.va.us> > the posixpath.walk function contains the following code: > > if isdir(name) and not islink(name): > > This is, of course, very inefficient: it does os.stat on the same file > twice directly after each other. So I changed this code to: > > st = os.lstat(name) > if stat.S_ISDIR(st[stat.ST_MODE]): > > Must be much faster... minor context diff attached. You're right -- although on my system the difference is in the noise, I agree that it should be done this way. You may consider patching the corresponding code in ntpath.py, macpath.py and dospath.py as well. > I sent those two patches in two different mails because of > the different nature of those patches: you might want to > refuse the former and accept the latter or vica versa. Which two? If you refer to your append() patches, I'm glad you put them in separate mails! --Guido van Rossum (home page: http://www.python.org/~guido/) From gerrit@nl.linux.org Mon Feb 28 14:38:52 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Mon, 28 Feb 2000 15:38:52 +0100 Subject: [Patches] optimizing posixpath.walk In-Reply-To: <200002281428.JAA22704@eric.cnri.reston.va.us>; from guido@python.org on Mon, Feb 28, 2000 at 09:28:46AM -0500 References: <20000228115014.A9702@stopcontact.palga.uucp> <200002281428.JAA22704@eric.cnri.reston.va.us> Message-ID: <20000228153852.A10714@humbolt.nl.linux.org> On Mon, Feb 28, 2000 at 09:28:46AM -0500, Guido van Rossum wrote: > You may consider patching the corresponding code in ntpath.py, > macpath.py and dospath.py as well. This line doesn't contain islink() there, because those systems don't have symbolic links. > > I sent those two patches in two different mails because of > > the different nature of those patches: you might want to > > refuse the former and accept the latter or vica versa. > > Which two? If you refer to your append() patches, I'm glad you put > them in separate mails! OK! regards, Gerrit. -- -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- From gerrit@nl.linux.org Mon Feb 28 15:03:21 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Mon, 28 Feb 2000 16:03:21 +0100 Subject: [Patches] Acknowledgements: docstrings -> comments In-Reply-To: <200002281433.JAA22734@eric.cnri.reston.va.us>; from guido@python.org on Mon, Feb 28, 2000 at 09:33:05AM -0500 References: <20000227103915.A1526@stopcontact.palga.uucp> <200002281433.JAA22734@eric.cnri.reston.va.us> Message-ID: <20000228160321.A11180@humbolt.nl.linux.org> --dDRMvlgZJXvWKvBx Content-Type: text/plain; charset=us-ascii On Mon, Feb 28, 2000 at 09:33:05AM -0500, Guido van Rossum wrote: > Gerrit, > > Can you send different patches for ftplib, telnetlib and urlparse? > You mistook the authors of the quoted RFCs for module authors! Oops! Here is the same patch again, but now without those changes to those three modules. > Also, as long as you're shortening the cgi.py docstring, you might as > well move the XXX paragraph about docstrings to a comment too. :-) Done. Is it OK this way (I hope so...)? I'm not sure about recursive diffs. regards, Gerrit. -- -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- --dDRMvlgZJXvWKvBx Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="acks.diff" diff -cr /tmp/Lib/StringIO.py ./StringIO.py *** /tmp/Lib/StringIO.py Wed Feb 2 16:10:14 2000 --- ./StringIO.py Mon Feb 28 15:51:09 2000 *************** *** 19,24 **** --- 19,26 ---- Notes: - Using a real file is often faster (but less convenient). + - There's also a much faster implementation in C, called cStringIO, but + it's not subclassable. - fileno() is left unimplemented so that code which uses it triggers an exception early. - Seeking far beyond EOF and then writing will insert real null diff -cr /tmp/Lib/cgi.py ./cgi.py *** /tmp/Lib/cgi.py Fri Feb 25 12:44:03 2000 --- ./cgi.py Mon Feb 28 15:56:19 2000 *************** *** 399,420 **** - Don't try to give a CGI script a set-uid mode. This doesn't work on most systems, and is a security liability as well. ! History ! ------- ! ! Michael McLay started this module. Steve Majewski changed the ! interface to SvFormContentDict and FormContentDict. The multipart ! parsing was inspired by code submitted by Andreas Paepcke. Guido van ! Rossum rewrote, reformatted and documented the module and is currently ! responsible for its maintenance. ! ! ! XXX The module is getting pretty heavy with all those docstrings. ! Perhaps there should be a slimmed version that doesn't contain all those ! backwards compatible and debugging classes and functions? ! """ __version__ = "2.2" --- 399,419 ---- - Don't try to give a CGI script a set-uid mode. This doesn't work on most systems, and is a security liability as well. + """ ! # XXX The module is getting pretty heavy with all those docstrings. ! # Perhaps there should be a slimmed version that doesn't contain all those ! # backwards compatible and debugging classes and functions? ! # History ! # ------- ! # ! # Michael McLay started this module. Steve Majewski changed the ! # interface to SvFormContentDict and FormContentDict. The multipart ! # parsing was inspired by code submitted by Andreas Paepcke. Guido van ! # Rossum rewrote, reformatted and documented the module and is currently ! # responsible for its maintenance. ! # __version__ = "2.2" diff -cr /tmp/Lib/code.py ./code.py *** /tmp/Lib/code.py Thu Oct 22 23:56:44 1998 --- ./code.py Mon Feb 28 15:51:09 2000 *************** *** 1,7 **** """Utilities needed to emulate Python's interactive interpreter. - Inspired by similar code by Jeff Epler and Fredrik Lundh. """ import sys --- 1,8 ---- """Utilities needed to emulate Python's interactive interpreter. """ + + # Inspired by similar code by Jeff Epler and Fredrik Lundh. import sys diff -cr /tmp/Lib/ftplib.py ./ftplib.py *** /tmp/Lib/ftplib.py Mon Feb 28 15:53:31 2000 --- ./ftplib.py Mon Feb 28 15:53:30 2000 *************** *** 3,13 **** Based on RFC 959: File Transfer Protocol (FTP), by J. Postel and J. Reynolds - Changes and improvements suggested by Steve Majewski. - Modified by Jack to work on the mac. - Modified by Siebren to support docstrings and PASV. - - Example: >>> from ftplib import FTP --- 3,8 ---- *************** *** 34,39 **** --- 29,39 ---- python ftplib.py -d localhost -l -p -l """ + # + # Changes and improvements suggested by Steve Majewski. + # Modified by Jack to work on the mac. + # Modified by Siebren to support docstrings and PASV. + # import os import sys diff -cr /tmp/Lib/getpass.py ./getpass.py *** /tmp/Lib/getpass.py Tue Oct 19 00:25:22 1999 --- ./getpass.py Mon Feb 28 15:51:09 2000 *************** *** 6,14 **** On Windows, the msvcrt module will be used. On the Mac EasyDialogs.AskPassword is used, if available. - Authors: Piers Lauder (original) - Guido van Rossum (Windows support and cleanup) """ import sys --- 6,15 ---- On Windows, the msvcrt module will be used. On the Mac EasyDialogs.AskPassword is used, if available. """ + + # Authors: Piers Lauder (original) + # Guido van Rossum (Windows support and cleanup) import sys diff -cr /tmp/Lib/imaplib.py ./imaplib.py *** /tmp/Lib/imaplib.py Thu Feb 24 03:24:50 2000 --- ./imaplib.py Mon Feb 28 15:51:09 2000 *************** *** 3,12 **** Based on RFC 2060. - Author: Piers Lauder December 1997. - - Authentication code contributed by Donn Cave June 1998. - Public class: IMAP4 Public variable: Debug Public functions: Internaldate2tuple --- 3,8 ---- *************** *** 14,19 **** --- 10,19 ---- ParseFlags Time2Internaldate """ + + # Author: Piers Lauder December 1997. + # + # Authentication code contributed by Donn Cave June 1998. __version__ = "2.32" diff -cr /tmp/Lib/poplib.py ./poplib.py *** /tmp/Lib/poplib.py Thu May 13 20:47:25 1999 --- ./poplib.py Mon Feb 28 15:51:09 2000 *************** *** 1,11 **** """A POP3 client class. Based on the J. Myers POP3 draft, Jan. 96 - - Author: David Ascher - [heavily stealing from nntplib.py] - Updated: Piers Lauder [Jul '97] """ # Example (see the test function at the end of this file) --- 1,11 ---- """A POP3 client class. Based on the J. Myers POP3 draft, Jan. 96 """ + + # Author: David Ascher + # [heavily stealing from nntplib.py] + # Updated: Piers Lauder [Jul '97] # Example (see the test function at the end of this file) diff -cr /tmp/Lib/smtplib.py ./smtplib.py *** /tmp/Lib/smtplib.py Fri Feb 4 16:39:30 2000 --- ./smtplib.py Mon Feb 28 15:51:09 2000 *************** *** 2,15 **** """SMTP/ESMTP client class. - Author: The Dragon De Monsyne - ESMTP support, test code and doc fixes added by - Eric S. Raymond - Better RFC 821 compliance (MAIL and RCPT, and CRLF in data) - by Carey Evans , for picky mail servers. - - This was modified from the Python 1.5 library HTTP lib. - This should follow RFC 821 (SMTP) and RFC 1869 (ESMTP). Notes: --- 2,7 ---- *************** *** 38,43 **** --- 30,43 ---- (250, "Somebody OverHere ") >>> s.quit() """ + + # Author: The Dragon De Monsyne + # ESMTP support, test code and doc fixes added by + # Eric S. Raymond + # Better RFC 821 compliance (MAIL and RCPT, and CRLF in data) + # by Carey Evans , for picky mail servers. + # + # This was modified from the Python 1.5 library HTTP lib. import socket import string diff -cr /tmp/Lib/telnetlib.py ./telnetlib.py *** /tmp/Lib/telnetlib.py Thu Dec 24 00:04:17 1998 --- ./telnetlib.py Mon Feb 28 15:54:04 2000 *************** *** 35,41 **** """ - # Imported modules import sys import socket --- 35,40 ---- diff -cr /tmp/Lib/whrandom.py ./whrandom.py *** /tmp/Lib/whrandom.py Fri Feb 4 16:28:42 2000 --- ./whrandom.py Mon Feb 28 15:51:09 2000 *************** *** 25,34 **** random generators, and to choose from other ranges. - Translated by Guido van Rossum from C source provided by - Adrian Baddeley. - - Multi-threading note: the random number generator used here is not thread-safe; it is possible that nearly simultaneous calls in --- 25,30 ---- *************** *** 36,41 **** --- 32,41 ---- have to use a lock around all calls. (I didn't want to slow this down in the serial case by using a lock here.) """ + + # Translated by Guido van Rossum from C source provided by + # Adrian Baddeley. + class whrandom: def __init__(self, x = 0, y = 0, z = 0): --dDRMvlgZJXvWKvBx-- From guido@python.org Mon Feb 28 15:07:04 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 10:07:04 -0500 Subject: [Patches] in-overloading: done right, with Guido's comments folded in (fwd) In-Reply-To: Your message of "Sat, 26 Feb 2000 15:00:47 +0200." References: Message-ID: <200002281507.KAA23049@eric.cnri.reston.va.us> Thanks, Moshe (and Greg for reviewing). All checked in! Request: please submit some testcases for the test suite. This should check that (1) a class defining __contains__ now supports 'in' correctly; (2) a class defining __getitem__ but not __contains__ still works correctly; (3) a class defining neither does not support 'in' and gets a decent error message. And of course, documentation is still missing. Two nits: (1) I couldn't find the checkin messages. Please refrain from using phrases like "same checkin message from last e-mail." (2) Somehow, the patch program got confused by the "---" that you appended to the end of the patch. It's better not to use rows of dashes or starts, since these mean something to the context-diff recognizer. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Feb 28 15:14:33 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 10:14:33 -0500 Subject: [Patches] Acknowledgements: docstrings -> comments In-Reply-To: Your message of "Mon, 28 Feb 2000 16:03:21 +0100." <20000228160321.A11180@humbolt.nl.linux.org> References: <20000227103915.A1526@stopcontact.palga.uucp> <200002281433.JAA22734@eric.cnri.reston.va.us> <20000228160321.A11180@humbolt.nl.linux.org> Message-ID: <200002281514.KAA23157@eric.cnri.reston.va.us> > Oops! Here is the same patch again, but now without those changes to > those three modules. Thanks. Done. > Is it OK this way (I hope so...)? I'm not sure about recursive diffs. Not sure what you mean. Certainly, as long as I haven't checked in a patch, sending me a patch on top of a previous patch is a recipe for disaster; I need patches relative to the CVS tree. --Guido van Rossum (home page: http://www.python.org/~guido/) From kirkland@blarg.net Mon Feb 28 17:44:54 2000 From: kirkland@blarg.net (Douglas Kirkland) Date: Mon, 28 Feb 2000 17:44:54 +0000 Subject: [Patches] find file permissions error Message-ID: <38BAB416.1C3E44AE@blarg.net> This is a multi-part message in MIME format. --------------CD7CF55A9676CA9115F2DA2D Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Find file has permissions error in Python 1.5.2. I add error correction so it does not crash because of permission denied. -- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Douglas Kirkland kirkland@blarg.net --------------CD7CF55A9676CA9115F2DA2D Content-Type: text/plain; charset=us-ascii; name="find.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="find.py" import fnmatch import os _debug = 0 _prune = ['(*)'] def find(pattern, dir = os.curdir): list = [] try: names = os.listdir(dir) names.sort() for name in names: fullname = os.path.join(dir, name) if fnmatch.fnmatch(name, pattern): list.append(fullname) if os.path.isdir(fullname) and not os.path.islink(fullname): if _debug: print "descend into", `fullname` list = list + find(pattern, fullname) return list except OSError: if _debug: print 'Permission denied' return list --------------CD7CF55A9676CA9115F2DA2D-- From jim@interet.com Mon Feb 28 19:14:58 2000 From: jim@interet.com (James C. Ahlstrom) Date: Mon, 28 Feb 2000 14:14:58 -0500 Subject: [Patches] Re: New module zipfile.py References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> <200002232245.RAA07934@eric.cnri.reston.va.us> Message-ID: <38BAC932.D697A533@interet.com> This is a multi-part message in MIME format. --------------18734A03406EE5D854B4AC13 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Guido van Rossum wrote: > [A lot of comments on zipfile.py....] I rewrote zipfile.py and make all the changes you suggested except as noted below. I think zipfile.py is now a lot better, but since there are so many changes, it needs more testing. Help with testing please... The new zipfile.py, new docs and a ./test/test_zipfile.py is attached. Please note that the interface changed from the last version. LEGAL ===== I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. DETAILS ======= > of not indenting doc strings and not leaving a blank line between the > "summary line" of the doc string (its first line) and the rest. None > of that is a showstopper, and a separate coding style conformance pass > can be made to fix these issues. I'm not sure I got this just right yet. "Not indenting"??? > Also, I noticed that this emits a warning to stdout if the name > already exists. The idea that the module prints to stdout for various > reasons is a little scary -- as you often repeat yourself, in GUI > apps, there may be no stdout, or writing to it may be frowned upon. > (Maybe _debug should just be set to 0 by default? That would shut up > everything except writepy() and printdir(), I believe.) I changed the default debug to zero, but.. In GUI apps it is necessary to capture stdout and present it in a popup window. Any reasonable Python GUI should do this as a standard feature. Otherwise tracebacks etc. have no place to go. So I think writing messages to stdout is OK. > Another comment: when I use the Unix zipinfo utility to list the > contents of an archive created with this module, the first character > of the file mode is listed as '?'. It should be '-', indicating a > plain file. No idea which flag you should set tohow get this effect, > but the zip documentation should tell you. I don't have zipinfo so I couldn't fix this. Where is it? JimA --------------18734A03406EE5D854B4AC13 Content-Type: text/plain; charset=us-ascii; name="zipfile.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="zipfile.py" "Read and write ZIP files" # Written by James C. Ahlstrom jim@interet.com # All rights transferred to CNRI pursuant to the Python contribution agreement import struct, os, time import binascii, py_compile try: import zlib # We may need its compression method except: zlib = None class _BadZipfile(Exception): pass error = _BadZipfile # The exception raised by this module # constants for Zip file compression methods ZIP_STORED = 0 ZIP_DEFLATED = 8 # Other ZIP compression methods not supported # Here are some struct module formats for reading headers structEndArchive = "<4s4H2lH" # 9 items, end of archive, 22 bytes stringEndArchive = "PK\005\006" # magic number for end of archive record structCentralDir = "<4s4B4H3l5H2l"# 19 items, central directory, 46 bytes stringCentralDir = "PK\001\002" # magic number for central directory structFileHeader = "<4s2B4H3l2H" # 12 items, file header record, 30 bytes stringFileHeader = "PK\003\004" # magic number for file header def is_zipfile(filename): """Quickly see if file is a ZIP file by checking the magic number. Will not accept a ZIP archive with an ending comment.""" try: fpin = open(filename, "rb") fpin.seek(-22, 2) # Seek to end-of-file record endrec = fpin.read() fpin.close() if endrec[0:4] == "PK\005\006" and endrec[-2:] == "\000\000": return 1 # file has correct magic number except: pass class ZipInfo: "Class with attributes describing each file in the ZIP archive" def __init__(self, filename="NoName", date_time=(1980,1,1,0,0,0)): self.filename = filename # Name of the file in the archive self.date_time = date_time # year, month, day, hour, min, sec # Standard values: self.compress_type = ZIP_STORED # Type of compression for the file self.comment = "" # Comment for each file self.extra = "" # ZIP extra data self.create_system = 0 # System which created ZIP archive self.create_version = 20 # Version which created ZIP archive self.extract_version = 20 # Version needed to extract archive self.reserved = 0 # Must be zero self.flag_bits = 0 # ZIP flag bits self.volume = 0 # Volume number of file header self.internal_attr = 0 # Internal attributes self.external_attr = 0 # External file attributes # Other attributes are set by class ZipFile: # header_offset Byte offset to the file header # file_offset Byte offset to the start of the file data # CRC CRC-32 of the uncompressed file # compress_size Size of the compressed file # file_size Size of the uncompressed file def FileHeader(self): 'Return the per-file header as a string' dt = self.date_time dosdate = (dt[0] - 1980) << 9 | dt[1] << 5 | dt[2] dostime = dt[3] << 11 | dt[4] << 5 | dt[5] / 2 if self.flag_bits & 0x08: # Set these to zero because we write them after the file data CRC = compress_size = file_size = 0 else: CRC = self.CRC compress_size = self.compress_size file_size = self.file_size header = struct.pack(structFileHeader, stringFileHeader, self.extract_version, self.reserved, self.flag_bits, self.compress_type, dostime, dosdate, CRC, compress_size, file_size, len(self.filename), len(self.extra)) return header + self.filename + self.extra class ZipFile: "Class with methods to open, read, write, close, list zip files" def __init__(self, filename, mode="r", compression=ZIP_STORED): 'Open the ZIP file with mode read "r", write "w" or append "a".' if compression == ZIP_STORED: pass elif compression == ZIP_DEFLATED: if not zlib: raise RuntimeError,\ "Compression requires the (missing) zlib module" else: raise RuntimeError, "That compression method is not supported" self.debug = 0 # Level of printing: 0 through 3 self.NameToInfo = {} # Find file info given name self.filelist = [] # List of ZipInfo instances for archive self.compression = compression # Method of compression self.filename = filename self.mode = key = mode[0] if key == 'r': self.fp = open(filename, "rb") self._GetContents() elif key == 'w': self.fp = open(filename, "wb") elif key == 'a': fp = self.fp = open(filename, "r+b") fp.seek(-22, 2) # Seek to end-of-file record endrec = fp.read() if endrec[0:4] == stringEndArchive and \ endrec[-2:] == "\000\000": self._GetContents() # file is a zip file # seek to start of directory and overwrite fp.seek(self.start_dir, 0) else: # file is not a zip file, just append fp.seek(0, 2) else: raise RuntimeError, 'Mode must be "r", "w" or "a"' def _GetContents(self): "Read in the table of contents for the zip file" fp = self.fp fp.seek(-22, 2) # Start of end-of-archive record filesize = fp.tell() + 22 # Get file size endrec = fp.read(22) # Archive must not end with a comment! if endrec[0:4] != stringEndArchive or endrec[-2:] != "\000\000": raise BadZipfile, "File is not a zip file, or ends with a comment" endrec = struct.unpack(structEndArchive, endrec) if self.debug > 1: print endrec size_cd = endrec[5] # bytes in central directory offset_cd = endrec[6] # offset of central directory x = filesize - 22 - size_cd # "concat" is zero, unless zip was concatenated to another file concat = x - offset_cd if self.debug > 2: print "given, inferred, offset", offset_cd, x, concat # self.start_dir: Position of start of central directory self.start_dir = offset_cd + concat fp.seek(self.start_dir, 0) total = 0 while total < size_cd: centdir = fp.read(46) total = total + 46 if centdir[0:4] != stringCentralDir: raise BadZipfile, "Bad magic number for central directory" centdir = struct.unpack(structCentralDir, centdir) if self.debug > 2: print centdir filename = fp.read(centdir[12]) # Create ZipInfo instance to store file information x = ZipInfo(filename) x.extra = fp.read(centdir[13]) x.comment = fp.read(centdir[14]) total = total + centdir[12] + centdir[13] + centdir[14] x.header_offset = centdir[18] + concat x.file_offset = x.header_offset + 30 + centdir[12] + centdir[13] (x.create_version, x.create_system, x.extract_version, x.reserved, x.flag_bits, x.compress_type, t, d, x.CRC, x.compress_size, x.file_size) = centdir[1:12] x.volume, x.internal_attr, x.external_attr = centdir[15:18] # Convert date/time code to (year, month, day, hour, min, sec) x.date_time = ( (d>>9)+1980, (d>>5)&0xF, d&0x1F, t>>11, (t>>5)&0x3F, t&0x1F * 2 ) self.filelist.append(x) self.NameToInfo[x.filename] = x if self.debug > 2: print "total", total for data in self.filelist: fp.seek(data.header_offset, 0) fheader = fp.read(30) if fheader[0:4] != stringFileHeader: raise BadZipfile, "Bad magic number for file header" fheader = struct.unpack(structFileHeader, fheader) fname = fp.read(fheader[10]) if fname != data.filename: raise RuntimeError, \ 'File name in Central Directory "%s" and File Header "%s" differ.' % ( data.filename, fname) def namelist(self): "Return a list of file names in the archive" l = [] for data in self.filelist: l.append(data.filename) return l def infolist(self): "Return a list of class ZipInfo instances for files in the archive" return self.filelist def printdir(self): "Print a table of contents for the zip file" print "%-46s %19s %12s" % ("File Name", "Modified ", "Size") for zinfo in self.filelist: date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time print "%-46s %s %12d" % (zinfo.filename, date, zinfo.file_size) def testzip(self): "Read all the files and check the CRC" for zinfo in self.filelist: try: self.read(zinfo.filename) # Check CRC-32 except: return zinfo.filename def getinfo(self, name): 'Return the instance of ZipInfo given "name"' return self.NameToInfo[name] def read(self, name): "Return file bytes (as a string) for name" if self.mode not in ("r", "a"): raise RuntimeError, 'read() requires mode "r" or "a"' if not self.fp: raise RuntimeError, \ "Attempt to read ZIP archive that was already closed" zinfo = self.getinfo(name) filepos = self.fp.tell() self.fp.seek(zinfo.file_offset, 0) bytes = self.fp.read(zinfo.compress_size) self.fp.seek(filepos, 0) if zinfo.compress_type == ZIP_STORED: pass elif zinfo.compress_type == ZIP_DEFLATED: if not zlib: raise RuntimeError, \ "De-compression requires the (missing) zlib module" # zlib compress/decompress code by Jeremy Hylton of CNRI dc = zlib.decompressobj(-15) bytes = dc.decompress(bytes) # need to feed in unused pad byte so that zlib won't choke ex = dc.decompress('Z') + dc.flush() if ex: bytes = bytes + ex else: raise BadZipfile, \ "Unsupported compression method %d for file %s" % \ (zinfo.compress_type, name) crc = binascii.crc32(bytes) if crc != zinfo.CRC: raise BadZipfile, "Bad CRC-32 for file %s" % name return bytes def _writecheck(self, zinfo): 'Check for errors before writing a file to the archive' if self.NameToInfo.has_key(zinfo.filename): if self.debug: # Warning for duplicate names print "Duplicate name:", zinfo.filename if self.mode not in ("w", "a"): raise RuntimeError, 'write() requires mode "w" or "a"' if not self.fp: raise RuntimeError, \ "Attempt to write ZIP archive that was already closed" if zinfo.compress_type == ZIP_DEFLATED and not zlib: raise RuntimeError, \ "Compression requires the (missing) zlib module" if zinfo.compress_type not in (ZIP_STORED, ZIP_DEFLATED): raise RuntimeError, \ "That compression method is not supported" def write(self, filename, arcname=None, compress_type=None): 'Put the bytes from filename into the archive under the name arcname.' st = os.stat(filename) mtime = time.localtime(st[8]) date_time = mtime[0:6] # Create ZipInfo instance to store file information if arcname is None: zinfo = ZipInfo(filename, date_time) else: zinfo = ZipInfo(arcname, date_time) zinfo.external_attr = st[0] << 16 # Unix attributes if compress_type is None: zinfo.compress_type = self.compression else: zinfo.compress_type = compress_type self._writecheck(zinfo) fp = open(filename, "rb") zinfo.flag_bits = 0x08 zinfo.header_offset = self.fp.tell() # Start of header bytes self.fp.write(zinfo.FileHeader()) zinfo.file_offset = self.fp.tell() # Start of file bytes CRC = 0 compress_size = 0 file_size = 0 if zinfo.compress_type == ZIP_DEFLATED: cmpr = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -15) else: cmpr = None while 1: buf = fp.read(1024 * 8) if not buf: break file_size = file_size + len(buf) CRC = binascii.crc32(buf, CRC) if cmpr: buf = cmpr.compress(buf) compress_size = compress_size + len(buf) self.fp.write(buf) fp.close() if cmpr: buf = cmpr.flush() compress_size = compress_size + len(buf) self.fp.write(buf) zinfo.compress_size = compress_size else: zinfo.compress_size = file_size zinfo.CRC = CRC zinfo.file_size = file_size # Write CRC and file sizes after the file data self.fp.write(struct.pack("= os.stat(file_py)[8]: fname = file_pyo # Use .pyo file elif not os.path.isfile(file_pyc) or \ os.stat(file_pyc)[8] < os.stat(file_py)[8]: if self.debug: print "Compiling", file_py py_compile.compile(file_py, file_pyc) fname = file_pyc else: fname = file_pyc archivename = os.path.split(fname)[1] if basename: archivename = "%s/%s" % (basename, archivename) return (fname, archivename) --------------18734A03406EE5D854B4AC13 Content-Type: text/plain; charset=us-ascii; name="test_zipfile.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="test_zipfile.py" import zipfile, os srcname = "junk9630.tmp" zipname = "junk9708.tmp" try: fp = open(srcname, "w") # Make a source file with some lines for i in range(0, 1000): fp.write("Test of zipfile line %d.\n" % i) fp.close() zip = zipfile.ZipFile(zipname, "w") # Create the ZIP archive zip.write(srcname, srcname) zip.write(srcname, "another.name") zip.close() zip = zipfile.ZipFile(zipname, "r") # Read the ZIP archive zip.read("another.name") zip.read(srcname) zip.close() finally: if os.path.isfile(srcname): # Remove temporary files os.unlink(srcname) if os.path.isfile(zipname): os.unlink(zipname) --------------18734A03406EE5D854B4AC13 Content-Type: text/plain; charset=us-ascii; name="zipdocs.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="zipdocs.txt" Documentation for module zipfile.py =================================== zipfile -- Read and write files in ZIP format The ZIP file format is a common archive and compression standard. This module provides tools to create, read, write, append, and list a ZIP file. The available attributes of this module are: error The error raised for bad ZIP files. class ZipFile The class for reading and writing ZIP files. class PyZipFile(ZipFile) The class for making ZIP archives of Python libraries and packages. class ZipInfo The class for storing information about the files in a ZIP archive. is_zipfile(path) Return 1/0 if "path" is/is not a valid ZIP file based on its magic number. This module does not currently handle ZIP files which have appended comments. ZIP_STORED The numeric constant (zero) for an uncompressed file. ZIP_DEFLATED The numeric constant (eight) for the usual ZIP file compression method. This requires the zlib module. No other compression methods are currently supported. The class ZipFile has this data attribute: ------------------------------------------ debug The level of printing, an integer 0 through 3, default 0. The class ZipFile has these methods: ------------------------------------ __init__(self, filename, mode="r", compression=ZIP_STORED) Open a ZIP file named "filename". Mode is "r" to read an existing file, "w" to truncate and write a new file, or "a" to append. Despite the lack of a "b", the file is opened in binary mode. For mode "a", if filename is a ZIP file, then additional files are added to it. If filename is not a ZIP file, then a new ZIP file is appended to the file. This is meant for adding a ZIP archive to another file such as python.exe. But "cat myzip.zip >> python.exe" also works, and at least WinZip can read such files. The "compression" is the ZIP compression method to use when writing the archive, and must be ZIP_STORED or ZIP_DEFLATED. namelist(self) Return a list of file names in the ZIP archive. infolist(self) Return a list of class ZipInfo instances, one for each file in the archive. Each instance contains information about the file. The list order is the same as the order of files in the archive. getinfo(self, name) Return the instance of ZipInfo given the file name. printdir(self) Print a table of contents for the archive to stdout. testzip(self) Read all the files and check their CRC's. Return the name of the first bad file, or else return None. read(self, name) Return the bytes of the file in the archive. The archive must be open for read or append. write(self, filename, arcname=None, compress_type=None) Write the file named "filename" to the archive, and give it the archive name "arcname", default "filename". Use "compress_type" to override the compression method on a per-file basis. The archive must be open with mode "w" or "a". writestr(self, zinfo, bytes): Write the string "bytes" and the other data to the archive. The "zinfo" is an instance of class ZipInfo, and you should set at least zinfo.filename and zinfo.date_time to valid values. The archive must be open with mode "w" or "a". close(self) Close the archive file. You must call close() before exiting your program or essential records will not be written. The class PyZipFile has this one method: ---------------------------------------- writepy(self, pathname, basename = "") Search for files *.py and add the corresponding file to the archive. The corresponding file is a *.pyo file if available, else a *.pyc file, compiling if necessary. If the pathname is a file, the file must end with ".py", and just the (corresponding *.py[oc]) file is added at the top level (no path information). If it is a directory, and the directory is not a package directory, then all the files *.py[oc] are added at the top level. If the directory is a package directory, then all *.py[oc] are added under the package name as a file path, and if any subdirectories are package directories, all of these are added recursively. The "basename" is intended for internal use only. The writepy() method makes archives with file names like this: string.pyc # Top level name test/__init__.pyc # Package directory test/testall.pyc # Package "test.testall" file test/bogus/__init__.pyc # Subpackage directory test/bogus/myfile.pyc # Subpackage "test.bogus.myfile" file The class ZipInfo has these data attributes: -------------------------------------------- filename Name of the file in the archive. extra The "extra" ZIP data string. comment File comment string. header_offset Byte offset to the file header. file_offset Byte offset to the start of the file data. create_version Zip version of the software which created the archive. create_system Type of system used to create the archive. extract_version Zip version needed to extract the files in the archive. reserved Must be zero. flag_bits Zip flag bits. compress_type Type of compression for the file. date_time The file time tuple (year, month, day, hour, min, sec). CRC CRC-32 of the uncompressed file. compress_size Size of the compressed file. file_size Size of the uncompressed file. volume Disk volume number. internal_attr Internal file attributes. external_attr External file attributes. The class ZipInfo has these methods: -------------------------------------- __init__(self, filename="NoName", date_time=(1980,1,1,0,0,0)) Create an instance with the specified name and date. FileHeader(self) Return the ZIP per-file header as a string. Intended for internal use. Bugs ==== Flag bit 0x08 is in use, and the DOS version PkZip 2.04g will not be able to read the archive unless compression methon ZIP_DEFLATED is used. In the interest of speed, this module does not attempt to read ZIP archives with an appended archive comment string. This module puts the Unix file attributes in the top two bytes of the external_attr, but sets the low-byte DOS attributes (if any) to zero. Bit 0x01 of internal_attr should be set for a text file, but it is always zero. --------------18734A03406EE5D854B4AC13-- From guido@python.org Mon Feb 28 19:37:43 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 14:37:43 -0500 Subject: [Patches] Re: New module zipfile.py In-Reply-To: Your message of "Mon, 28 Feb 2000 14:14:58 EST." <38BAC932.D697A533@interet.com> References: <38A4378A.CD4BE4B3@interet.com> <38B2E8D4.75E142@interet.com> <200002221958.OAA05758@eric.cnri.reston.va.us> <38B40B5B.D1619CCE@interet.com> <200002232245.RAA07934@eric.cnri.reston.va.us> <38BAC932.D697A533@interet.com> Message-ID: <200002281937.OAA28379@eric.cnri.reston.va.us> I like Jim's new offering. It works with my test program (after simple adaptation). But like Jim, I don't have the time to give it a more thorough test... Anybody else, please? It would be fun to use this to create a simple zipper/unzipper using Tkinter, and see how well it works on Windows. --Guido van Rossum (home page: http://www.python.org/~guido/) From piers@cs.su.oz.au Mon Feb 28 22:04:04 2000 From: piers@cs.su.oz.au (Piers Lauder) Date: Tue, 29 Feb 2000 09:04:04 +1100 Subject: [Patches] imaplib.py patch Message-ID: <951776548.596.775147904@cs.usyd.edu.au> --129.78.10.43.126.26379.951776548.602.5065 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi! Reason for patch: This patch is re: Lucas.Dejonge@awtpl.com.au: [Python-bugs-list] imaplib - not complying with RFC (PR#218) Lucas de Jonge reported that the code in imaplib that detects a read-write to read-only change doesn't comply with RFC 2060. I've modified the code to comply, and tested it, and here is the resulting "diff -c" patch against the version in the CVS tree as of 1/2 hour ago. Standard disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Piers Lauder. --129.78.10.43.126.26379.951776548.602.5065 Content-Type: application/octet-stream; name=imaplib.diff Content-Transfer-Encoding: 7bit *** dist/src/Lib/imaplib.py Tue Feb 29 09:06:12 2000 --- /usr/staff/piers/lib/html/python/imaplib.py Tue Feb 29 09:13:43 2000 *************** *** 15,21 **** # # Authentication code contributed by Donn Cave June 1998. ! __version__ = "2.32" import binascii, re, socket, string, time, random, sys --- 15,21 ---- # # Authentication code contributed by Donn Cave June 1998. ! __version__ = "2.33" import binascii, re, socket, string, time, random, sys *************** *** 128,133 **** --- 128,134 ---- self.tagged_commands = {} # Tagged commands awaiting response self.untagged_responses = {} # {typ: [data, ...], ...} self.continuation_response = '' # Last continuation response + self.is_readonly = None # READ-ONLY desired state self.tagnum = 0 # Open socket to server. *************** *** 470,475 **** --- 471,477 ---- """ # Mandated responses are ('FLAGS', 'EXISTS', 'RECENT', 'UIDVALIDITY') self.untagged_responses = {} # Flush old responses. + self.is_readonly = readonly if readonly: name = 'EXAMINE' else: *************** *** 479,485 **** self.state = 'AUTH' # Might have been 'SELECTED' return typ, dat self.state = 'SELECTED' ! if not self.untagged_responses.has_key('READ-WRITE') \ and not readonly: if __debug__: if self.debug >= 1: --- 481,487 ---- self.state = 'AUTH' # Might have been 'SELECTED' return typ, dat self.state = 'SELECTED' ! if self.untagged_responses.has_key('READ-ONLY') \ and not readonly: if __debug__: if self.debug >= 1: *************** *** 594,602 **** if self.untagged_responses.has_key(typ): del self.untagged_responses[typ] ! if self.untagged_responses.has_key('READ-WRITE') \ ! and self.untagged_responses.has_key('READ-ONLY'): ! del self.untagged_responses['READ-WRITE'] raise self.readonly('mailbox status changed to READ-ONLY') tag = self._new_tag() --- 596,603 ---- if self.untagged_responses.has_key(typ): del self.untagged_responses[typ] ! if self.untagged_responses.has_key('READ-ONLY') \ ! and not self.is_readonly: raise self.readonly('mailbox status changed to READ-ONLY') tag = self._new_tag() --129.78.10.43.126.26379.951776548.602.5065-- From guido@python.org Mon Feb 28 22:39:29 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 17:39:29 -0500 Subject: [Patches] imaplib.py patch In-Reply-To: Your message of "Tue, 29 Feb 2000 09:04:04 +1100." <951776548.596.775147904@cs.usyd.edu.au> References: <951776548.596.775147904@cs.usyd.edu.au> Message-ID: <200002282239.RAA03810@eric.cnri.reston.va.us> Thanks, Piers! --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@mojam.com (Skip Montanaro) Mon Feb 28 23:44:47 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 28 Feb 2000 17:44:47 -0600 Subject: [Patches] Patch to add :name to extension modules Message-ID: <200002282344.RAA09719@beluga2.mojam.com> I patched the extension modules in the core distribution to add :name info to as many format arts for PyArt_ParseTuple as easily possible. Some of the internal functions that are called from several places were left as-is, but I suspect I got well over 90% of the possible cases. Because the generated context diff was so large I decided not to attach it to this message. Instead, it's available at http://www.musi-cal.com/~skip/big.diff Once I see some activity on the checkins alias I'll delete the above URL. Release: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ From gstein@lyra.org Tue Feb 29 00:06:25 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 28 Feb 2000 16:06:25 -0800 (PST) Subject: [Patches] Patch to add :name to extension modules In-Reply-To: <200002282344.RAA09719@beluga2.mojam.com> Message-ID: On Mon, 28 Feb 2000, Skip Montanaro wrote: > I patched the extension modules in the core distribution to add :name info > to as many format arts for PyArt_ParseTuple as easily possible. Some of the > internal functions that are called from several places were left as-is, but > I suspect I got well over 90% of the possible cases. > > Because the generated context diff was so large I decided not to attach it > to this message. Instead, it's available at > > http://www.musi-cal.com/~skip/big.diff > > Once I see some activity on the checkins alias I'll delete the above URL. I've reviewed this and it looks good. I agree with the notion of "make simple changes and leave internal functions as-is". I did find one problem: * almodule.c, line 2077: the patch adds ":queryparams", which is incorrect. The function is called from al_getparams and al_setparams. The patch to that line should not be applied (and fall under Skip's "as-is" rationale) +1 Cheers, -g -- Greg Stein, http://www.lyra.org/ From skip@mojam.com (Skip Montanaro) Tue Feb 29 00:39:57 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 28 Feb 2000 18:39:57 -0600 (CST) Subject: [Patches] Patch to add :name to extension modules In-Reply-To: References: <200002282344.RAA09719@beluga2.mojam.com> Message-ID: <14523.5469.426562.42207@beluga2.mojam.com> Greg> I did find one problem: Greg> * almodule.c, line 2077: the patch adds ":queryparams", which is Greg> incorrect. The function is called from al_getparams and Greg> al_setparams. The patch to that line should not be applied (and Greg> fall under Skip's "as-is" rationale) Good catch. The almodule was the guinea pig upon which I developed the Emacs macro I used to do all the heavily lifting. Took me a couple tries to get it right enough. Skip From skip@mojam.com (Skip Montanaro) Tue Feb 29 00:45:58 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 28 Feb 2000 18:45:58 -0600 (CST) Subject: [Patches] moving acknowledgements from docstrings to comments In-Reply-To: <14518.44388.214657.775918@anthem.cnri.reston.va.us> References: <20000225164120.A19099@stopcontact.palga.uucp> <14518.44388.214657.775918@anthem.cnri.reston.va.us> Message-ID: <14523.5830.51598.669522@beluga2.mojam.com> Barry> Actually, this is an interesting point. How zealous should we be Barry> in using string meths? I wouldn't propose gobnourmous patches to Barry> convert the entire standard library, but when I'm hacking on a Barry> module anyway, can I do the conversion where it makes sense? Dittoing Guido's response here. When I hacked up a soundex.py today based upon some stuff I got over six months ago from Tim and Fred I went ahead and converted to string methods, but more for demonstration purposes than anything else. At this point, they are new enough that we should make sure there are some examples in the core libraries, but since there is no obvious functional improvement (slight performance boost in the occasional inner loop is about it) to using them, I see no reason to convert things wholesale. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From tim_one@email.msn.com Tue Feb 29 05:33:10 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 29 Feb 2000 00:33:10 -0500 Subject: [Patches] New checkappend.py tool Message-ID: <000301bf8276$77632000$732d153f@tim> New script (akin to tabnanny.py) for finding multi-argument .append() calls. Much easier than trying to wade thru a gazillion false positives via e.g. grep. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. #! /usr/bin/env python # Released to the public domain, by Tim Peters, 28 February 2000. """checkappend.py -- search for multi-argument .append() calls. Usage: specify one or more file or directory paths: checkappend [-v] file_or_dir [file_or_dir] ... Each file_or_dir is checked for multi-argument .append() calls. When a directory, all .py files in the directory, and recursively in its subdirectories, are checked. Use -v for status msgs. Use -vv for more status msgs. In the absence of -v, the only output is pairs of the form filename(linenumber): line containing the suspicious append Note that this finds multi-argument append calls regardless of whether they're attached to list objects. If a module defines a class with an append method that takes more than one argument, calls to that method will be listed. Note that this will not find multi-argument list.append calls made via a bound method object. For example, this is not caught: somelist = [] push = somelist.append push(1, 2, 3) """ __version__ = 1, 0, 0 import os import sys import string import getopt import tokenize verbose = 0 def errprint(*args): msg = string.join(args) sys.stderr.write(msg) sys.stderr.write("\n") def main(): args = sys.argv[1:] global verbose try: opts, args = getopt.getopt(sys.argv[1:], "v") except getopt.error, msg: errprint(msg + "\n\n" + __doc__) return for opt, optarg in opts: if opt == '-v': verbose = verbose + 1 if not args: errprint(__doc__) return for arg in args: check(arg) def check(file): if os.path.isdir(file) and not os.path.islink(file): if verbose: print "%s: listing directory" % `file` names = os.listdir(file) for name in names: fullname = os.path.join(file, name) if ((os.path.isdir(fullname) and not os.path.islink(fullname)) or os.path.normcase(name[-3:]) == ".py"): check(fullname) return try: f = open(file) except IOError, msg: errprint("%s: I/O Error: %s" % (`file`, str(msg))) return if verbose > 1: print "checking", `file`, "..." ok = AppendChecker(file, f).run() if verbose and ok: print "%s: Clean bill of health." % `file` [FIND_DOT, FIND_APPEND, FIND_LPAREN, FIND_COMMA, FIND_STMT] = range(5) class AppendChecker: def __init__(self, fname, file): self.fname = fname self.file = file self.state = FIND_DOT self.nerrors = 0 def run(self): try: tokenize.tokenize(self.file.readline, self.tokeneater) except tokenize.TokenError, msg: errprint("%s: Token Error: %s" % (`self.fname`, str(msg))) self.nerrors = self.nerrors + 1 return self.nerrors == 0 def tokeneater(self, type, token, start, end, line, NEWLINE=tokenize.NEWLINE, JUNK=(tokenize.COMMENT, tokenize.NL), OP=tokenize.OP, NAME=tokenize.NAME): state = self.state if type in JUNK: pass elif state is FIND_DOT: if type is OP and token == ".": state = FIND_APPEND elif state is FIND_APPEND: if type is NAME and token == "append": self.line = line self.lineno = start[0] state = FIND_LPAREN else: state = FIND_DOT elif state is FIND_LPAREN: if type is OP and token == "(": self.level = 1 state = FIND_COMMA else: state = FIND_DOT elif state is FIND_COMMA: if type is OP: if token in ("(", "{", "["): self.level = self.level + 1 elif token in (")", "}", "]"): self.level = self.level - 1 if self.level == 0: state = FIND_DOT elif token == "," and self.level == 1: self.nerrors = self.nerrors + 1 print "%s(%d):\n%s" % (self.fname, self.lineno, self.line) # don't gripe about this stmt again state = FIND_STMT elif state is FIND_STMT: if type is NEWLINE: state = FIND_DOT else: raise SystemError("unknown internal state '%s'" % `state`) self.state = state if __name__ == '__main__': main() From Moshe Zadka Tue Feb 29 05:57:20 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 29 Feb 2000 07:57:20 +0200 (IST) Subject: [Patches] find file permissions error In-Reply-To: <38BAB416.1C3E44AE@blarg.net> Message-ID: On Mon, 28 Feb 2000, Douglas Kirkland wrote: > Find file has permissions error in Python 1.5.2. I add error correction > so it does not crash because of permission denied. First thing first: the function does not crash (else it would be a bug in the interpreter), but raises OSError. It could be troublesome or some users, but it could be necessary for others. If anything, the fix should be to receive an optional function object to call on errors, and if it is not given, continue to raise OSError. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@python.org Tue Feb 29 13:09:23 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 29 Feb 2000 08:09:23 -0500 Subject: [Patches] New checkappend.py tool In-Reply-To: Your message of "Tue, 29 Feb 2000 00:33:10 EST." <000301bf8276$77632000$732d153f@tim> References: <000301bf8276$77632000$732d153f@tim> Message-ID: <200002291309.IAA04656@eric.cnri.reston.va.us> Thanks, Tim! I fixed a small error in the getopt error handling. Do you routinely run with string exceptions? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Feb 29 13:30:05 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 29 Feb 2000 08:30:05 -0500 Subject: [Patches] find file permissions error In-Reply-To: Your message of "Tue, 29 Feb 2000 07:57:20 +0200." References: Message-ID: <200002291330.IAA04715@eric.cnri.reston.va.us> > > Find file has permissions error in Python 1.5.2. I add error correction > > so it does not crash because of permission denied. > > First thing first: the function does not crash (else it would be a bug in > the interpreter), but raises OSError. It could be troublesome or some > users, but it could be necessary for others. If anything, the fix should > be to receive an optional function object to call on errors, and if it is > not given, continue to raise OSError. In Python 1.6, find.py will be declared obsolete, and live in Lib/lib-old (which means that it's not on the default module search path). Thus, changing the API seems bogus. The reason for making it obsolete is that it really was a mistake to add it in the first place; it looks like it was a utility used for the Mac development at the time. In 1.6, you can whip up the same thing using os.path.walk(): import os, fnmatch def find(pat, dir=os.curdir): list = [] def visit((list, pat), dir, names): for name in names: if fnmatch.fnmatch(name, pat): fullname = os.path.join(dir, name) list.append(fullname) os.path.walk(dir, visit, (list, pat)) return list By the way, has anybody else noticed some redundant logic in posixpath.walk()? It contains code to skip "." and "..", but os.listdir() stopped returning those many years ago... --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Feb 29 14:00:02 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 29 Feb 2000 09:00:02 -0500 Subject: [Patches] Patch to add :name to extension modules In-Reply-To: Your message of "Mon, 28 Feb 2000 17:44:47 CST." <200002282344.RAA09719@beluga2.mojam.com> References: <200002282344.RAA09719@beluga2.mojam.com> Message-ID: <200002291400.JAA05388@eric.cnri.reston.va.us> Thanks, Skip. And thanks again to Greg for checking. All done. --Guido van Rossum (home page: http://www.python.org/~guido/) From gansevle@cs.utwente.nl Tue Feb 29 15:43:40 2000 From: gansevle@cs.utwente.nl (Fred Gansevles) Date: Tue, 29 Feb 2000 16:43:40 +0100 Subject: [Patches] nismodule.c patches Message-ID: <200002291543.QAA17022@localhost.localdomain> This patch fixes 3 small problems. 1) If a map is used which is generated with 'makedbm -a', the trailing '\0' is now handled correctely. 2) The nis.maps() function skipped the first map in the output list. 3) The library '-lnsl' is added in Setup.in (needed on Linux glibc2 and Solaris systems. Maybe on other systems too?) ---------------------------------------------------------------- I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. FAQ about legal I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ---------------------------------------------------------------- Index: nismodule.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/nismodule.c,v retrieving revision 2.14 diff -c -r2.14 nismodule.c *** nismodule.c 1998/10/08 02:25:18 2.14 --- nismodule.c 2000/02/29 15:24:11 *************** *** 1,7 **** /*********************************************************** Written by: Fred Gansevles ! Vakgroep Spa, Faculteit der Informatica, Universiteit Twente, Enschede, --- 1,7 ---- /*********************************************************** Written by: Fred Gansevles ! B&O group, Faculteit der Informatica, Universiteit Twente, Enschede, *************** *** 36,67 **** static struct nis_map { char *alias; char *map; } aliases [] = { ! {"passwd", "passwd.byname"}, ! {"group", "group.byname"}, ! {"networks", "networks.byaddr"}, ! {"hosts", "hosts.byname"}, ! {"protocols", "protocols.bynumber"}, ! {"services", "services.byname"}, ! {"aliases", "mail.aliases"}, ! {"ethers", "ethers.byname"}, ! {0L, 0L} }; static char * ! nis_mapname (map) char *map; { int i; ! for (i=0; aliases[i].alias != 0L; i++) ! if (!strcmp (aliases[i].alias, map)) ! map = aliases[i].map; return map; } typedef int (*foreachfunc) Py_PROTO((int, char *, int, char *, int, char *)); static int nis_foreach (instatus, inkey, inkeylen, inval, invallen, indata) int instatus; --- 36,83 ---- static struct nis_map { char *alias; char *map; + int fix; } aliases [] = { ! {"passwd", "passwd.byname", 0}, ! {"group", "group.byname", 0}, ! {"networks", "networks.byaddr", 0}, ! {"hosts", "hosts.byname", 0}, ! {"protocols", "protocols.bynumber", 0}, ! {"services", "services.byname", 0}, ! {"aliases", "mail.aliases", 1}, /* created with 'makedbm -a' */ ! {"ethers", "ethers.byname", 0}, ! {0L, 0L, 0} }; static char * ! nis_mapname (map, pfix) char *map; + int *pfix; { int i; ! *pfix = 0; ! for (i=0; aliases[i].alias != 0L; i++) { ! if (!strcmp (aliases[i].alias, map)) { ! *pfix = aliases[i].fix; ! return aliases[i].map; ! } ! if (!strcmp (aliases[i].map, map)) { ! *pfix = aliases[i].fix; ! return aliases[i].map; ! } ! } ! return map; } typedef int (*foreachfunc) Py_PROTO((int, char *, int, char *, int, char *)); + struct ypcallback_data { + PyObject *dict; + int fix; + }; + static int nis_foreach (instatus, inkey, inkeylen, inval, invallen, indata) int instatus; *************** *** 69,80 **** int inkeylen; char *inval; int invallen; ! PyObject *indata; { if (instatus == YP_TRUE) { ! PyObject *key = PyString_FromStringAndSize(inkey, inkeylen); ! PyObject *val = PyString_FromStringAndSize(inval, invallen); int err; if (key == NULL || val == NULL) { /* XXX error -- don't know how to handle */ PyErr_Clear(); --- 85,103 ---- int inkeylen; char *inval; int invallen; ! struct ypcallback_data *indata; { if (instatus == YP_TRUE) { ! PyObject *key; ! PyObject *val; int err; + + if (indata->fix) { + inkeylen--; + invallen--; + } + key = PyString_FromStringAndSize(inkey, inkeylen); + val = PyString_FromStringAndSize(inval, invallen); if (key == NULL || val == NULL) { /* XXX error -- don't know how to handle */ PyErr_Clear(); *************** *** 82,88 **** Py_XDECREF(val); return 1; } ! err = PyDict_SetItem(indata, key, val); Py_DECREF(key); Py_DECREF(val); if (err != 0) { --- 105,111 ---- Py_XDECREF(val); return 1; } ! err = PyDict_SetItem(indata->dict, key, val); Py_DECREF(key); Py_DECREF(val); if (err != 0) { *************** *** 105,119 **** char *key, *map; int err; PyObject *res; if (!PyArg_Parse(args, "(t#s)", &key, &keylen, &map)) return NULL; if ((err = yp_get_default_domain(&domain)) != 0) return nis_error(err); Py_BEGIN_ALLOW_THREADS - map = nis_mapname (map); err = yp_match (domain, map, key, keylen, &match, &len); Py_END_ALLOW_THREADS if (err != 0) return nis_error(err); res = PyString_FromStringAndSize (match, len); --- 128,147 ---- char *key, *map; int err; PyObject *res; + int fix; if (!PyArg_Parse(args, "(t#s)", &key, &keylen, &map)) return NULL; if ((err = yp_get_default_domain(&domain)) != 0) return nis_error(err); + map = nis_mapname (map, &fix); + if (fix) + keylen++; Py_BEGIN_ALLOW_THREADS err = yp_match (domain, map, key, keylen, &match, &len); Py_END_ALLOW_THREADS + if (fix) + len--; if (err != 0) return nis_error(err); res = PyString_FromStringAndSize (match, len); *************** *** 129,155 **** char *domain; char *map; struct ypall_callback cb; ! PyObject *cat; int err; if (!PyArg_Parse(args, "s", &map)) return NULL; if ((err = yp_get_default_domain(&domain)) != 0) return nis_error(err); ! cat = PyDict_New (); ! if (cat == NULL) return NULL; cb.foreach = (foreachfunc)nis_foreach; ! cb.data = (char *)cat; Py_BEGIN_ALLOW_THREADS - map = nis_mapname (map); err = yp_all (domain, map, &cb); Py_END_ALLOW_THREADS if (err != 0) { ! Py_DECREF(cat); return nis_error(err); } ! return cat; } /* These should be u_long on Sun h/w but not on 64-bit h/w. --- 157,185 ---- char *domain; char *map; struct ypall_callback cb; ! struct ypcallback_data data; ! PyObject *dict; int err; if (!PyArg_Parse(args, "s", &map)) return NULL; if ((err = yp_get_default_domain(&domain)) != 0) return nis_error(err); ! dict = PyDict_New (); ! if (dict == NULL) return NULL; cb.foreach = (foreachfunc)nis_foreach; ! data.dict = dict; ! map = nis_mapname (map, &data.fix); ! cb.data = (char *)&data; Py_BEGIN_ALLOW_THREADS err = yp_all (domain, map, &cb); Py_END_ALLOW_THREADS if (err != 0) { ! Py_DECREF(dict); return nis_error(err); } ! return dict; } /* These should be u_long on Sun h/w but not on 64-bit h/w. *************** *** 345,351 **** return NULL; if ((list = PyList_New(0)) == NULL) return NULL; ! for (maps = maps->next; maps; maps = maps->next) { PyObject *str = PyString_FromString(maps->map); if (!str || PyList_Append(list, str) < 0) { --- 375,381 ---- return NULL; if ((list = PyList_New(0)) == NULL) return NULL; ! for (maps = maps; maps; maps = maps->next) { PyObject *str = PyString_FromString(maps->map); if (!str || PyList_Append(list, str) < 0) { Index: Setup.in =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/Setup.in,v retrieving revision 1.89 diff -c -r1.89 Setup.in *** Setup.in 1999/12/07 21:47:09 1.89 --- Setup.in 2000/02/29 15:24:11 *************** *** 164,170 **** # Some more UNIX dependent modules -- off by default, since these # are not supported by all UNIX systems: ! #nis nismodule.c # Sun yellow pages -- not everywhere #termios termios.c # Steen Lumholt's termios module #resource resource.c # Jeremy Hylton's rlimit interface --- 164,170 ---- # Some more UNIX dependent modules -- off by default, since these # are not supported by all UNIX systems: ! #nis nismodule.c -lnsl # Sun yellow pages -- not everywhere #termios termios.c # Steen Lumholt's termios module #resource resource.c # Jeremy Hylton's rlimit interface From guido@python.org Tue Feb 29 15:53:18 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 29 Feb 2000 10:53:18 -0500 Subject: [Patches] nismodule.c patches In-Reply-To: Your message of "Tue, 29 Feb 2000 16:43:40 +0100." <200002291543.QAA17022@localhost.localdomain> References: <200002291543.QAA17022@localhost.localdomain> Message-ID: <200002291553.KAA06617@eric.cnri.reston.va.us> Thanks, Fred! All checked in. One note: I can't use this here, because we're using NIS+. I don't know anything about NIS+ but apparently it's not compatible. Any chance of a NIS+ version? --Guido van Rossum (home page: http://www.python.org/~guido/) From gansevle@cs.utwente.nl Tue Feb 29 15:56:38 2000 From: gansevle@cs.utwente.nl (Fred Gansevles) Date: Tue, 29 Feb 2000 16:56:38 +0100 Subject: [Patches] nismodule.c patches In-Reply-To: Your message of Tue, 29 Feb 2000 10:53:18 EST Message-ID: <200002291556.QAA17343@localhost.localdomain> > Thanks, Fred! > > All checked in. > > One note: I can't use this here, because we're using NIS+. I don't > know anything about NIS+ but apparently it's not compatible. > > Any chance of a NIS+ version? I've never looked at NIS+ (we only run NIS) but I could give it a shot. Maybe in a few weeks time when I get around to it. > > --Guido van Rossum (home page: http://www.python.org/~guido/) ____________________________________________________________________________ Fred Gansevles Phone: +31 53 489 4613 >>> Your one-stop-shop for Linux/WinNT/NetWare <<< Org.: Twente University, Fac. of CS, Box 217, 7500 AE Enschede, Netherlands "Bill needs more time to learn Linux" - Steve B. From guido@python.org Tue Feb 29 16:43:38 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 29 Feb 2000 11:43:38 -0500 Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: Your message of "Wed, 09 Feb 2000 17:14:19 MST." <20000209171419.A10163@acs.ucalgary.ca> References: <20000205092822.A2204@acs.ucalgary.ca> <200002091459.PAA02594@python.inrialpes.fr> <20000209171419.A10163@acs.ucalgary.ca> Message-ID: <200002291643.LAA11666@eric.cnri.reston.va.us> I'm perusing the malloc and gc threads in more depth... Just responding to one misleading issue: [Vladimir] > > Every chunk of memory must be manupulated via the same malloc family. [Neil] > Yes, and this is where things get tricky. Extension modules can > use malloc to allocate objects and pass them to the Python core. > Python uses PyMem_FREE or similar and *boom*, memory corruption > (if they are different mallocs). Note that for object allocation, this is not true. Py_DECREF never deallocates the object directly -- it calls the function stored as tp_dealloc in the object's type object. Thus, the extension that defines the object type always gets control back to deallocate the object. Of course, for chunks of memory that aren't objects, all the above precautions hold. Note that e.g. Tcl is also careful with this. Its defines memory allocation and deallocation APIs: Tcl_Alloc, Tcl_Free, Tcl_Realloc. It has a few places where a routine returns a chunk of memory allocated by Tcl that should be freed by the caller, but the caller should always do this using Tcl_Free. (This is very important on Windows, where the Tcl/Tk DLL uses a different allocator than Python!) I'm not aware of Python APIs that have such a problem (since they almost all exclusively deal with objects) but if there are any, I presume that Vladimir's patch set will fix them... --Guido van Rossum (home page: http://www.python.org/~guido/) From nascheme@enme.ucalgary.ca Tue Feb 29 16:54:23 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Tue, 29 Feb 2000 09:54:23 -0700 Subject: [Patches] Re: Garbage collection patches for Python In-Reply-To: <200002291643.LAA11666@eric.cnri.reston.va.us>; from guido@python.org on Tue, Feb 29, 2000 at 11:43:38AM -0500 References: <20000205092822.A2204@acs.ucalgary.ca> <200002091459.PAA02594@python.inrialpes.fr> <20000209171419.A10163@acs.ucalgary.ca> <200002291643.LAA11666@eric.cnri.reston.va.us> Message-ID: <20000229095423.A24924@acs.ucalgary.ca> On Tue, Feb 29, 2000 at 11:43:38AM -0500, Guido van Rossum wrote: > [Vladimir] > > > Every chunk of memory must be manupulated via the same malloc family. > > [Neil] > > Yes, and this is where things get tricky. Extension modules can > > use malloc to allocate objects and pass them to the Python core. > > Python uses PyMem_FREE or similar and *boom*, memory corruption > > (if they are different mallocs). > > I'm not aware of Python APIs that have such a problem (since they > almost all exclusively deal with objects) but if there are any, I > presume that Vladimir's patch set will fix them... I was thinking of the readline module when I wrote this not too well thought out comment. It seems to be the only case. Vladimir has sorted this out with his patch. Neil From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 29 17:15:19 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 29 Feb 2000 12:15:19 -0500 (EST) Subject: [Patches] nismodule.c patches References: <200002291556.QAA17343@localhost.localdomain> Message-ID: <14523.65191.478645.169677@anthem.cnri.reston.va.us> >>>>> "FG" == Fred Gansevles writes: >> Thanks, Fred! All checked in. One note: I can't use this >> here, because we're using NIS+. I don't know anything about >> NIS+ but apparently it's not compatible. Any chance of a NIS+ >> version? FG> I've never looked at NIS+ (we only run NIS) but I could give FG> it a shot. Maybe in a few weeks time when I get around to it. I looked at it very briefly a few years ago. IIRC, NIS+ is more OO than NIS and what you really want is a more object-y API than a straight revelation of the C API. It would be cool to have, but I think it's a lot of work to do right. -Barry From nascheme@enme.ucalgary.ca Tue Feb 29 22:34:21 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Tue, 29 Feb 2000 15:34:21 -0700 Subject: [Patches] Reference cycle collection for Python Message-ID: <20000229153421.A16502@acs.ucalgary.ca> Here is my latest patch to add reference cycle collection to Python. This patch adds a new configure option called --with-cycle-gc and a new module called gc. Comments welcome. Neil (aka nas) diff -rcN Python-cvs/Include/Python.h Python-gc/Include/Python.h *** Python-cvs/Include/Python.h Wed Feb 9 04:38:15 2000 --- Python-gc/Include/Python.h Tue Feb 29 15:09:59 2000 *************** *** 72,77 **** --- 72,81 ---- #include "pydebug.h" + #ifdef WITH_CYCLE_GC + #include "cycle_gc.h" + #endif + #include "intobject.h" #include "longobject.h" #include "floatobject.h" diff -rcN Python-cvs/Include/cycle_gc.h Python-gc/Include/cycle_gc.h *** Python-cvs/Include/cycle_gc.h Wed Dec 31 17:00:00 1969 --- Python-gc/Include/cycle_gc.h Tue Feb 29 15:09:59 2000 *************** *** 0 **** --- 1,19 ---- + #ifndef Py_CYCLE_GC_H + #define Py_CYCLE_GC_H + + struct _gc_state; /* Forward */ + + #define PyGCObject_HEAD \ + PyObject_HEAD \ + struct _gc_linked *gc_next; \ + struct _gc_linked *gc_prev; + + typedef struct _gc_linked { + PyGCObject_HEAD + } PyGCObject; + + struct _gc_state *PyGC_Initialize(void); + int PyGC_Insert(PyGCObject *op); + void PyGC_Remove(PyGCObject *op); + + #endif diff -rcN Python-cvs/Include/object.h Python-gc/Include/object.h *** Python-cvs/Include/object.h Tue Feb 29 14:55:02 2000 --- Python-gc/Include/object.h Tue Feb 29 15:09:59 2000 *************** *** 212,217 **** --- 212,221 ---- typedef int (*cmpfunc) Py_PROTO((PyObject *, PyObject *)); typedef PyObject *(*reprfunc) Py_PROTO((PyObject *)); typedef long (*hashfunc) Py_PROTO((PyObject *)); + #ifdef WITH_CYCLE_GC + typedef int (*accountfunc) Py_PROTO((PyObject *, void *)); + typedef int (*recursefunc) Py_PROTO((PyObject *, accountfunc, void *)); + #endif typedef struct _typeobject { PyObject_VAR_HEAD *************** *** 249,256 **** char *tp_doc; /* Documentation string */ ! /* More spares */ long tp_xxx5; long tp_xxx6; long tp_xxx7; long tp_xxx8; --- 253,266 ---- char *tp_doc; /* Documentation string */ ! #ifdef WITH_CYCLE_GC ! /* call function for all accessible objects */ ! recursefunc tp_recurse; ! #else long tp_xxx5; + #endif + + /* More spares */ long tp_xxx6; long tp_xxx7; long tp_xxx8; diff -rcN Python-cvs/Include/pystate.h Python-gc/Include/pystate.h *** Python-cvs/Include/pystate.h Mon Dec 21 13:21:19 1998 --- Python-gc/Include/pystate.h Tue Feb 29 15:09:59 2000 *************** *** 54,59 **** --- 54,63 ---- int checkinterval; + #ifdef WITH_CYCLE_GC + struct _gc_state *gc_state; + #endif + } PyInterpreterState; diff -rcN Python-cvs/Modules/Setup.in Python-gc/Modules/Setup.in *** Python-cvs/Modules/Setup.in Tue Feb 29 14:55:08 2000 --- Python-gc/Modules/Setup.in Tue Feb 29 15:09:59 2000 *************** *** 94,99 **** --- 94,101 ---- posix posixmodule.c # posix (UNIX) system calls signal signalmodule.c # signal(2) + #gc gcmodule.c # cycle garbage collection + # The SGI specific GL module: GLHACK=-Dclear=__GLclear diff -rcN Python-cvs/Modules/gcmodule.c Python-gc/Modules/gcmodule.c *** Python-cvs/Modules/gcmodule.c Wed Dec 31 17:00:00 1969 --- Python-gc/Modules/gcmodule.c Tue Feb 29 15:26:55 2000 *************** *** 0 **** --- 1,820 ---- + /* + + Reference Cycle Garbage Collection + ================================== + + Neil Schemenauer + + Based on a patch by Toby Kelsey and ideas from Guido van Rossum and Tim + Peters. + + This garbage collector uses local information to determine inaccessible + cycles. + + How this method differs from mark-and-sweep + ------------------------------------------- + + Mark and sweep works by first keeping a list of all alive objects, then + starting from base (directly referenced) objects, recursively marking + all objects which are referenced by them. Any alive objects that are + not marked after this are inaccessible and are deleted. Consequently + 1) it requires a global list of objects + 2) it must find all base references + 3) it must access all reachable objects during the sweep phase. + There are problems with this method in locating all relevant objects + in the system, especially when libraries are used. + + This method does not require such global analysis. Instead it finds + sets of objects that are only refered to by objects within that set. + Since the base references are not included, objects in these sets are + not accessible from the base references and are therefore garbage. + + The first phase then consists of maintaining a set of objects that can + possibly be involved in reference cycles. + + The second phase, collection, which may be activated periodically, at + user choice or when the set of watched objects is large enough, is to + check the objects in the set and determine from local evidence whether + they are inaccessible. + + The first step is for each object in the set of watched objects, + determine the objects reachable from it. The second step is to + determine, for each object in this new set, whether any of its + references are due to objects not in the set. If there are no + external references, the set is isolated and can be deleted. + + This is not sufficient to find all inaccessible groups, however. + Consider the link structure: A<=>B -> E<=>F <- C<=>D + The set will contain A (or B) and C (or D). + But + get_accessible(A or B) gives [A,B,E,F], and F has an external link + while + get_accessible(C or D) gives [C,D,E,F], and E has an external link + so this group would always escape deletion. + + One option is to ignore this type of group and accept some leakage. + A better option is to remove from the group all items accessible + externally. In this example, remove [E,F] to leave [A,B] or [C,D] resp. + The remaining group objects are not externally accessible and can be + deleted. + + The second phase thus consists firstly of checking all objects in the + set of watched objects and finding all reachable objects; secondly + finding all external references and removing all objects reachable from + them; and finally deleting any remaining objects, which are + inaccessible. + + This implementation currently only considers cycles involving dictionary + objects. Also, only objects that define tp_recurse can be involved in + collected cycles. + + TODO: + make thread safe (okay already?) + + IDEAS: + use dummy element at the head of the lists + change debug flag to use bits + include module automaticly if WITH_CYCLE_GC defined + command line options + + */ + + + #include "Python.h" + + #ifndef WITH_CYCLE_GC + #error "You must define WITH_CYCLE_GC to include this module" + #endif + + #ifdef WITH_THREAD + #error "This module is not yet thread safe. Compile without threads." + #endif + + /* #define DEBUG */ + + /* ==== begin hash table implementation ================================== */ + + /* + * hash table implementation is stolen from dictobject.c. Look there for + * comments. + */ + + #define MINSIZE 4 /* XXX make this bigger? */ + + static long polys[] = { + 4 + 3, + 8 + 3, + 16 + 3, + 32 + 5, + 64 + 3, + 128 + 3, + 256 + 29, + 512 + 17, + 1024 + 9, + 2048 + 5, + 4096 + 83, + 8192 + 27, + 16384 + 43, + 32768 + 3, + 65536 + 45, + 131072 + 9, + 262144 + 39, + 524288 + 39, + 1048576 + 9, + 2097152 + 5, + 4194304 + 3, + 8388608 + 33, + 16777216 + 27, + 33554432 + 9, + 67108864 + 71, + 134217728 + 39, + 268435456 + 9, + 536870912 + 5, + 1073741824 + 83, + 0 + }; + + #define DUMMY -9999999 /* value for deleted entries */ + + /* empty: key is NULL + * deleted: key is not NULL, value is DUMMY + * used: key is not NULL, value is not DUMMY + */ + typedef struct { + PyObject *key; + int value; + } hashentry; + + typedef struct _hashtable { + int fill; + int used; + int size; + int poly; + hashentry *table; + } hashtable; + + + static hashtable * + hash_new() + { + register hashtable *mp; + mp = PyMem_NEW(hashtable, 1); + if (mp == NULL) + return NULL; + mp->size = 0; + mp->poly = 0; + mp->table = NULL; + mp->fill = 0; + mp->used = 0; + return mp; + } + + static hashentry * + hash_lookup(hashtable *mp, PyObject *key) + { + register int i; + register long hash; + register unsigned incr; + register hashentry *freeslot; + register unsigned int mask = mp->size-1; + hashentry *ep0 = mp->table; + register hashentry *ep; + + hash = (long)key; + i = (~hash) & mask; + ep = &ep0[i]; + if (ep->key == NULL || ep->key == key) + return ep; + if (ep->value == DUMMY) { + freeslot = ep; + } else { + freeslot = NULL; + } + incr = (hash ^ ((unsigned long)hash >> 3)) & mask; + if (!incr) + incr = mask; + for (;;) { + ep = &ep0[(i+incr)&mask]; + if (ep->key == NULL) { + if (freeslot != NULL) + return freeslot; + else + return ep; + } + if (ep->value == DUMMY && freeslot == NULL) { + freeslot = ep; + } else if (ep->key == key) { + return ep; + } + /* Cycle through GF(2^n)-{0} */ + incr = incr << 1; + if (incr > mask) + incr ^= mp->poly; /* This will implicitely clear + the highest bit */ + } + } + + static int + hash_insert(hashtable *mp, PyObject *key, int value) + { + register hashentry *ep; + ep = hash_lookup(mp, key); + if (ep->key == key && ep->value != DUMMY) { + ep->value = value; + return 1; + } + else { + if (ep->key == NULL) + mp->fill++; + ep->key = key; + ep->value = value; + mp->used++; + return 0; + } + } + + static int + hash_resize(hashtable *mp, int minused) + { + register int oldsize = mp->size; + register int newsize, newpoly; + register hashentry *oldtable = mp->table; + register hashentry *newtable; + register hashentry *ep; + register int i; + for (i = 0, newsize = MINSIZE; ; i++, newsize <<= 1) { + if (i > sizeof(polys)/sizeof(polys[0])) { + /* Ran out of polynomials */ + PyErr_NoMemory(); + return -1; + } + if (newsize > minused) { + newpoly = polys[i]; + break; + } + } + newtable = (hashentry *) malloc(sizeof(hashentry) * newsize); + if (newtable == NULL) { + PyErr_NoMemory(); + return -1; + } + memset(newtable, '\0', sizeof(hashentry) * newsize); + mp->size = newsize; + mp->poly = newpoly; + mp->table = newtable; + mp->fill = 0; + mp->used = 0; + + for (i = 0, ep = oldtable; i < oldsize; i++, ep++) { + if (ep->key != NULL && ep->value != DUMMY) + hash_insert(mp,ep->key,ep->value); + } + + PyMem_XDEL(oldtable); + return 0; + } + + static int + hash_setitem(hashtable *mp, PyObject *key, int value) + { + /* if fill >= 2/3 size, double in size */ + if (mp->fill*3 >= mp->size*2) { + if (hash_resize(mp, mp->used*2) != 0) { + if (mp->fill+1 > mp->size) + return -1; + } + } + return hash_insert(mp, key, value); + } + + static int + hash_delitem(hashtable *mp, PyObject *key) + { + register hashentry *ep; + + if (mp->table == NULL) + goto empty; + ep = hash_lookup(mp, key); + if (ep->key == NULL || ep->value == DUMMY) { + empty: + return 0; + } + ep->value = DUMMY; + mp->used--; + return 1; + } + + static void + hash_dealloc(hashtable *mp) + { + PyMem_XDEL(mp->table); + PyMem_DEL(mp); + } + + static PyObject * + hash_next(hashtable *mp, int *ppos, int *value) + { + int i; + i = *ppos; + if (i < 0) + return NULL; + while (i < mp->size && (mp->table[i].key == NULL || + mp->table[i].value == DUMMY)) + i++; + *ppos = i+1; + if (i >= mp->size) + return NULL; + + if (value) + *value = mp->table[i].value; + + return mp->table[i].key; + } + + /* ==== end hash table implementation ================================== */ + + + typedef struct _gc_state { + PyGCObject *pcr; /* young "interesting" objects */ + PyGCObject *pcr_old; /* old "interesting" objects */ + hashtable *dying; /* none NULL if collecting */ + int threshold; /* collection frequency */ + int threshold_old; /* old collection frequency */ + int allocated; /* objects allocated since collection */ + int aged; /* objects aged since old collection */ + int debug; /* set to > 0 to debug cycles */ + } gc_state; + + + #define GC_STATE (_PyThreadState_Current ? \ + _PyThreadState_Current->interp->gc_state : NULL) + + gc_state * + PyGC_Initialize(void) + { + gc_state *gc; + + gc = PyMem_NEW(gc_state, 1); + if (!gc) { + return NULL; + } + gc->pcr = NULL; + gc->pcr_old = NULL; + /* set threshold to zero to disable collection */ + gc->threshold = 25; + gc->threshold_old = 250; + gc->allocated = 0; + gc->aged = 0; + #ifdef DEBUG + gc->debug = 2; + #else + gc->debug = 0; + #endif + return gc; + } + + #ifdef DEBUG + static void + print_table(hashtable *t) + { + int i; + int value; + PyObject *op; + + i = 0; + while ((op = hash_next(t, &i, &value))) { + printf("0x%lx => %d\n", (long)op, value); + } + } + #endif + + static int + decref(PyObject *op, hashtable *t) + { + hashentry *ep; + + ep = hash_lookup(t, op); + if (ep->key != NULL && ep->value != DUMMY) { + ep->value--; + } + return 1; + } + + /* + * Recursively walk over objects accessible from op and add to t. + */ + static int + walk_add(PyObject *op, hashtable *t) + { + int rv; + recursefunc recurse; + + rv = hash_setitem(t, op, op->ob_refcnt); + if (rv == -1) { + PyErr_SetString(PyExc_MemoryError, + "Out of memory for garbage collection"); + return 0; /* error */ + } + if (rv) { + return 1; /* already in table */ + } else { + if ((recurse = op->ob_type->tp_recurse) != NULL) { + return recurse(op, (accountfunc)walk_add, (void *)t); + } else { + return 1; /* not walkable */ + } + } + } + + /* + * Recursively walk over objects accessible from op and remove from t. + */ + static int + walk_remove(PyObject *op, hashtable *t) + { + recursefunc recurse; + + if (!hash_delitem(t, op)) { + return 1; /* not in table */ + } + if ((recurse = op->ob_type->tp_recurse) != NULL) { + return recurse(op, (accountfunc)walk_remove, (void *)t); + } else { + return 1; /* not walkable */ + } + } + + /* + * Find all objects accessible from items in pcr. + */ + static int + add_reachable(hashtable *reachable, PyGCObject *op) + { + for (; op != NULL; op=op->gc_next) { + if (!walk_add((PyObject *)op, reachable)) { + return 0; + } + } + return 1; + } + + /* + * Subtract internal link-count off stored ref-count. + */ + static void + subtract_refcounts(hashtable *reachable) + { + PyObject *op; + recursefunc recurse; + int i = 0; + + while ((op = hash_next(reachable, &i, NULL))) { + if ((recurse = op->ob_type->tp_recurse) != NULL) { + (void) recurse(op, (accountfunc)decref, + (void *)reachable); + } + } + } + + /* + * Remove objects accessible externally (non-zero refcnt). + */ + static void + remove_external(hashtable *reachable) + { + int i; + PyObject *op; + int refcnt; + + i = 0; + while ((op = hash_next(reachable, &i, &refcnt))) { + if (refcnt > 0) { + /* has external references */ + walk_remove(op, reachable); + } + } + } + + /* be careful not to create new dictionaries */ + static void + debug_instance(PyInstanceObject *inst) + { + PyObject *classname = inst->in_class->cl_name; + char *cname; + if (classname != NULL && PyString_Check(classname)) + cname = PyString_AsString(classname); + else + cname = "?"; + PyErr_Clear(); + fprintf(stderr, "gc: <%.100s instance at %lx>\n", cname, (long)inst); + } + + static void + debug_object(PyObject *op) + { + PyObject *repr; + + repr = PyObject_Repr(op); + fprintf(stderr, "gc: %s\n", PyString_AsString(repr)); + Py_DECREF(repr); + } + + static void + debug_cycles(hashtable *reachable) + { + PyObject *op; + int i; + + i = 0; + while ((op = hash_next(reachable, &i, NULL))) { + if (PyInstance_Check(op)) { + debug_instance((PyInstanceObject *)op); + } else if (PyClass_Check(op) || PyModule_Check(op)) { + debug_object(op); + } + } + } + + /* + * Effectively deletes objects in cycles by breaking links between them. + * Breaking links may invalidate other objects in cycle, these are removed + * from reachable automatically via PyGC_Remove(). + */ + static void + delete_garbage(hashtable *reachable) + { + PyObject *op; + int i; + gc_state *gc = GC_STATE; + + gc->dying = reachable; + + if (gc->debug >= 2) { + debug_cycles(reachable); + } + i = 0; + while ((op = hash_next(reachable, &i, NULL))) { + if (PyDict_Check(op)) { + Py_INCREF(op); + /* XXX just delete things in reachable? */ + PyDict_Clear(op); + Py_DECREF(op); + } + } + + gc->dying = NULL; + } + + static long + collect(PyGCObject *pcr, PyGCObject *pcr_old) + { + long nd; + hashtable *reachable = hash_new(); + + if (!reachable) { + PyErr_SetString(PyExc_MemoryError, "Out of memory for GC"); + return -1; + } + + /* Collect reachable objects from pcr, noting refcounts */ + if (!add_reachable(reachable, pcr)) { + return -1; + } + if (!add_reachable(reachable, pcr_old)) { + return -1; + } + + /* Subtract off internal ref-counts */ + subtract_refcounts(reachable); + + /* Remove objects reachable from externally referenced items */ + remove_external(reachable); + + + /* Delete remaining (inaccessible) items */ + nd = reachable->used; + if (nd > 0) { + gc_state *gc = GC_STATE; + if (gc->debug) { + fprintf(stderr, "collection: %ld inaccessible\n", + nd); + } + delete_garbage(reachable); + } + + hash_dealloc(reachable); + + return nd; + } + + static long + gc_collect(gc_state *gc) + { + PyGCObject *op; + int n; + + if (gc->threshold <= 0) { + return 0; /* collection disabled */ + } + if (gc->aged > gc->threshold_old) { + /* collecting old set */ + n = collect(gc->pcr, gc->pcr_old); + gc->aged = 0; + } else { + n = collect(gc->pcr, NULL); + } + gc->allocated = 0; + if (n >= 0 && gc->pcr) { + /* move survivors into the old set*/ + op = gc->pcr; + while (op != NULL) { + /* XXX could be smarter here */ + PyGCObject *next = op->gc_next; + PyGCObject *tail = gc->pcr_old; + gc->pcr_old = op; + op->gc_prev = NULL; + op->gc_next = tail; + if (tail != NULL) { + tail->gc_prev = op; + } + gc->aged++; + op = next; + } + gc->pcr = NULL; + } + return n; + } + + int + PyGC_Insert(PyGCObject *op) + { + gc_state *gc = GC_STATE; + + if (gc->allocated++ > gc->threshold) { + if (gc_collect(gc) < 0) { + return 0; + } + } + op->gc_next = gc->pcr; + op->gc_prev = NULL; + if (op->gc_next != NULL) { + op->gc_next->gc_prev = op; + } + gc->pcr = op; + return 1; + } + + void + PyGC_Remove(PyGCObject *op) + { + gc_state *gc = GC_STATE; + + if (!gc) { + return; /* XXX interpreter shutting down? */ + } + if (gc->dying) { + /* Collection is happening. We don't want to touch this object + * if it is in the dying set as it has already been freed. */ + hash_delitem(gc->dying, (PyObject *)op); + } + + if (op->gc_prev == NULL) { + if (op == gc->pcr) { + gc->pcr = op->gc_next; + } else { + gc->pcr_old = op->gc_next; + } + } else { + op->gc_prev->gc_next = op->gc_next; + } + if (op->gc_next != NULL) { + op->gc_next->gc_prev = op->gc_prev; + } + gc->allocated--; + } + + + static char collect__doc__[] = + "collect() -> n\n" + "\n" + "Run a full collection right now. The number of objects collected is\n" + "returned." + ; + + static PyObject * + Py_collect(self, args) + PyObject *self; + PyObject *args; + { + long n; + gc_state *gc = GC_STATE; + + if(!PyArg_ParseTuple(args, "")) /* check no args */ + return NULL; + + gc->aged = gc->threshold_old+1; /* do full collection */ + + n = gc_collect(gc); + if (n < 0) /* exception already set */ + return NULL; + + return Py_BuildValue("l", n); + } + + static char set_debug__doc__[] = + "set_debug(n) -> None\n" + "\n" + "Set the current garbage collection debugging level. Debugging information\n" + "is printed on sys.stderr.\n" + "\n" + " n == 0 - Don't print any debugging information.\n" + " n >= 1 - Print number of inaccessible objects after each collection.\n" + " n >= 2 - Print information on objects being collected." + ; + + static PyObject * + Py_set_debug(self, args) + PyObject *self; + PyObject *args; + { + gc_state *gc = GC_STATE; + + if (!PyArg_ParseTuple(args, "l", &gc->debug)) + return NULL; + + Py_INCREF(Py_None); + return Py_None; + } + + static char set_thresh__doc__[] = + "set_threshold(threshold, [threshold_old]) -> None\n" + "\n" + "Sets the collection thresholds. Setting threshold to zero disables\n" + "collection." + "\n" + "threshold - number of new dictionaries before collecting\n" + "threshold_old - number of dictionaries aged before collecting old set.\n" + "\n" + ; + + static PyObject * + Py_set_thresh(self, args) + PyObject *self; + PyObject *args; + { + gc_state *gc = GC_STATE; + + if (!PyArg_ParseTuple(args, "l|l", &gc->threshold, &gc->threshold_old)) + return NULL; + + Py_INCREF(Py_None); + return Py_None; + } + + static char get_thresh__doc__[] = + "get_treshold() -> (threshold, threshold_old)" + "\n" + "Return the current collection thresholds" + ; + + static PyObject * + Py_get_thresh(self, args) + PyObject *self; + PyObject *args; + { + gc_state *gc = GC_STATE; + + if(!PyArg_ParseTuple(args, "")) /* no args */ + return NULL; + + return Py_BuildValue("(ii)", gc->threshold, gc->threshold_old); + } + + + static char gc__doc__ [] = + "This module provides access to the garbage collector for reference cycles.\n" + "\n" + "collect() -- Do a full collection right now.\n" + "set_debug(n) -- Set the debugging level.\n" + "set_threshold(threshold, [threshold_old]) -- Set the collection threshold.\n" + "get_threshold() -- Return the current the collection thresholds.\n" + ; + + static PyMethodDef GcMethods[] = { + {"set_debug", Py_set_debug, METH_VARARGS, set_debug__doc__}, + {"set_threshold", Py_set_thresh, METH_VARARGS, set_thresh__doc__}, + {"get_threshold", Py_get_thresh, METH_VARARGS, get_thresh__doc__}, + {"collect", Py_collect, METH_VARARGS, collect__doc__}, + {NULL, NULL} /* Sentinel */ + }; + + void + initgc(void) + { + (void) Py_InitModule4("gc", + GcMethods, + gc__doc__, + NULL, + PYTHON_API_VERSION); + } + diff -rcN Python-cvs/Objects/classobject.c Python-gc/Objects/classobject.c *** Python-cvs/Objects/classobject.c Tue Feb 29 14:55:15 2000 --- Python-gc/Objects/classobject.c Tue Feb 29 15:09:59 2000 *************** *** 386,391 **** --- 386,406 ---- return res; } + #ifdef WITH_CYCLE_GC + static int + class_recurse(PyClassObject *cl, accountfunc func, void *data) + { + if (!(*func)(cl->cl_bases, data)) { + return 0; + } + if (!(*func)(cl->cl_dict, data)) { + return 0; + } + return 1; + } + #endif /* WITH_CYCLE_GC */ + + PyTypeObject PyClass_Type = { PyObject_HEAD_INIT(&PyType_Type) 0, *************** *** 406,411 **** --- 421,432 ---- (reprfunc)class_str, /*tp_str*/ (getattrofunc)class_getattr, /*tp_getattro*/ (setattrofunc)class_setattr, /*tp_setattro*/ + 0, /* tp_as_buffer */ + 0, /* tp_flags */ + 0, /* tp_doc */ + #ifdef WITH_CYCLE_GC + (recursefunc)class_recurse, /* tp_recurse */ + #endif }; int *************** *** 837,842 **** --- 858,877 ---- return outcome; } + #ifdef WITH_CYCLE_GC + static int + instance_recurse(PyInstanceObject *in, accountfunc func, void *data) + { + if (!(*func)((PyObject *)in->in_class, data)) { + return 0; + } + if (!(*func)(in->in_dict, data)) { + return 0; + } + return 1; + } + #endif /* WITH_CYCLE_GC */ + static PyObject *getitemstr, *setitemstr, *delitemstr, *lenstr; static int *************** *** 1461,1466 **** --- 1496,1505 ---- (setattrofunc)instance_setattr, /*tp_setattro*/ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT, /*tp_flags */ + 0, /* tp_doc */ + #ifdef WITH_CYCLE_GC + (recursefunc)instance_recurse, /* tp_recurse */ + #endif }; diff -rcN Python-cvs/Objects/dictobject.c Python-gc/Objects/dictobject.c *** Python-cvs/Objects/dictobject.c Tue Feb 29 14:55:15 2000 --- Python-gc/Objects/dictobject.c Tue Feb 29 15:09:59 2000 *************** *** 104,110 **** --- 104,114 ---- when it is more than half filled. */ typedef struct { + #ifdef WITH_CYCLE_GC + PyGCObject_HEAD + #else PyObject_HEAD + #endif int ma_fill; int ma_used; int ma_size; *************** *** 129,134 **** --- 133,144 ---- mp->ma_table = NULL; mp->ma_fill = 0; mp->ma_used = 0; + #ifdef WITH_CYCLE_GC + if (!PyGC_Insert((PyGCObject *)mp)) { + Py_DECREF(mp); + return NULL; + } + #endif return (PyObject *)mp; } *************** *** 487,492 **** --- 497,505 ---- Py_DECREF(ep->me_value); } } + #ifdef WITH_CYCLE_GC + PyGC_Remove((PyGCObject *)mp); + #endif PyMem_XDEL(mp->ma_table); PyMem_DEL(mp); } *************** *** 1020,1025 **** --- 1033,1058 ---- return Py_None; } + #ifdef WITH_CYCLE_GC + static int + dict_recurse(PyObject *op, accountfunc func, void *data) + { + int i = 0; + PyObject *pk; + PyObject *pv; + + while (PyDict_Next(op, &i, &pk, &pv)) { + if (!(*func)(pk, data)) { + return 0; + } + if (!(*func)(pv, data)) { + return 0; + } + } + return 1; + } + #endif /* WITH_CYCLE_GC */ + static PyMethodDef mapp_methods[] = { {"has_key", (PyCFunction)dict_has_key, METH_VARARGS}, {"keys", (PyCFunction)dict_keys}, *************** *** 1055,1060 **** --- 1088,1104 ---- 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ &dict_as_mapping, /*tp_as_mapping*/ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + 0, /* tp_flags */ + 0, /* tp_doc */ + #ifdef WITH_CYCLE_GC + (recursefunc)dict_recurse, /* tp_recurse */ + #endif }; /* For backward compatibility with old dictionary interface */ diff -rcN Python-cvs/Objects/listobject.c Python-gc/Objects/listobject.c *** Python-cvs/Objects/listobject.c Tue Feb 29 14:55:15 2000 --- Python-gc/Objects/listobject.c Tue Feb 29 15:09:59 2000 *************** *** 1383,1388 **** --- 1383,1405 ---- return NULL; } + #ifdef WITH_CYCLE_GC + static int + list_recurse(PyListObject *lp, accountfunc func, void *data) + { + int i; + if (lp->ob_item != NULL) { + i = lp->ob_size; + while (--i >= 0) { + if (!(*func)(lp->ob_item[i], data)) { + return 0; + } + } + } + return 1; + } + #endif /* WITH_CYCLE_GC */ + static char append_doc[] = "L.append(object) -- append object to end"; static char extend_doc[] = *************** *** 1448,1453 **** --- 1465,1481 ---- 0, /*tp_as_number*/ &list_as_sequence, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + 0, /* tp_flags */ + 0, /* tp_doc */ + #ifdef WITH_CYCLE_GC + (recursefunc)list_recurse, /* tp_recurse */ + #endif }; diff -rcN Python-cvs/Objects/tupleobject.c Python-gc/Objects/tupleobject.c *** Python-cvs/Objects/tupleobject.c Thu Jan 20 15:32:54 2000 --- Python-gc/Objects/tupleobject.c Tue Feb 29 15:09:59 2000 *************** *** 390,395 **** --- 390,412 ---- return (PyObject *) np; } + #ifdef WITH_CYCLE_GC + static int + tuplerecurse(PyTupleObject *tp, accountfunc func, void *data) + { + int i; + if (tp->ob_item != NULL) { + i = tp->ob_size; + while (--i >= 0) { + if (!(*func)(tp->ob_item[i], data)) { + return 0; + } + } + } + return 1; + } + #endif /* WITH_CYCLE_GC */ + static PySequenceMethods tuple_as_sequence = { (inquiry)tuplelength, /*sq_length*/ (binaryfunc)tupleconcat, /*sq_concat*/ *************** *** 416,421 **** --- 433,448 ---- &tuple_as_sequence, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ (hashfunc)tuplehash, /*tp_hash*/ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + 0, /* tp_flags */ + 0, /* tp_doc */ + #ifdef WITH_CYCLE_GC + (recursefunc)tuplerecurse, /* tp_recurse */ + #endif }; /* The following function breaks the notion that tuples are immutable: diff -rcN Python-cvs/Python/pystate.c Python-gc/Python/pystate.c *** Python-cvs/Python/pystate.c Fri Jun 18 08:22:24 1999 --- Python-gc/Python/pystate.c Tue Feb 29 15:09:59 2000 *************** *** 63,68 **** --- 63,75 ---- PyInterpreterState *interp = PyMem_NEW(PyInterpreterState, 1); if (interp != NULL) { + #ifdef WITH_CYCLE_GC + interp->gc_state = PyGC_Initialize(); + if (!interp->gc_state) { + PyMem_DEL(interp); + return NULL; + } + #endif HEAD_INIT(); interp->modules = NULL; interp->sysdict = NULL; diff -rcN Python-cvs/config.h.in Python-gc/config.h.in *** Python-cvs/config.h.in Mon Dec 20 14:25:59 1999 --- Python-gc/config.h.in Tue Feb 29 15:09:59 2000 *************** *** 185,190 **** --- 185,193 ---- (shared library plus accessory files). */ #undef WITH_NEXT_FRAMEWORK + /* Define if you want cycle garbage collection */ + #undef WITH_CYCLE_GC + /* The number of bytes in an off_t. */ #undef SIZEOF_OFF_T diff -rcN Python-cvs/configure.in Python-gc/configure.in *** Python-cvs/configure.in Tue Feb 29 14:55:01 2000 --- Python-gc/configure.in Tue Feb 29 15:09:59 2000 *************** *** 1025,1030 **** --- 1025,1037 ---- AC_DEFINE(MALLOC_ZERO_RETURNS_NULL) fi + AC_MSG_CHECKING(for --with-cycle-gc) + AC_ARG_WITH(cycle-gc, [--with-cycle-gc enable reference cycle collection], [ + AC_MSG_RESULT($withval) + AC_DEFINE(WITH_CYCLE_GC) + ], + AC_MSG_RESULT(no)) + # generate output files AC_OUTPUT(Makefile \ Objects/Makefile \