From blais at furius.ca Thu Jun 1 04:24:08 2006 From: blais at furius.ca (Martin Blais) Date: Wed, 31 May 2006 22:24:08 -0400 Subject: [Python-checkins] [Python-Dev] r46300 - in python/trunk: Lib/socket.py Lib/test/test_socket.py Lib/test/test_struct.py Modules/_struct.c Modules/arraymodule.c Modules/socketmodule.c In-Reply-To: References: <20060526120329.9A9671E400C@bag.python.org>

<8393fff0605310335o12b2b26ew43ba63cfd69c26e9@mail.gmail.com> Message-ID: <8393fff0605311924n23bf5971ve39669cfffa9bdfc@mail.gmail.com> On 5/31/06, Guido van Rossum wrote: > On 5/31/06, Martin Blais wrote: > > So I assume you're suggesting the following renames: > > > > pack_to -> packinto > > recv_buf -> recvinto > > recvfrom_buf -> recvfrominto > > > > (I don't like that last one very much. > > I'll go ahead and make those renames once I return.) > > You could add an underscore before _into. Will do! cheers, From nnorwitz at gmail.com Thu Jun 1 07:24:47 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 31 May 2006 22:24:47 -0700 Subject: [Python-checkins] r46472 - python/trunk/PCbuild8 python/trunk/PCbuild8/Uninstal.wse python/trunk/PCbuild8/_bsddb.vcproj python/trunk/PCbuild8/_ctypes.vcproj python/trunk/PCbuild8/_ctypes_test.vcproj python/trunk/PCbuild8/_elementtree.vcproj pytho In-Reply-To: <1149108731.447e01fb8520c@www.domainfactory-webmail.de> References: <129CEF95A523704B9D46959C922A28000282A8ED@nemesis.central.ccp.cc> <1149108731.447e01fb8520c@www.domainfactory-webmail.de> Message-ID: On 5/31/06, martin at v.loewis.de wrote: > Zitat von "Kristj?n V. J?nsson" : > > > p.s. do you think I ought to subscribe to python-checkins? > > No. I personally didn't for quite a long time. I'll just remember > to CC you the next time. I wish you would subscribe. I wish everyone with checkin privs would subscribe, even if they don't review the checkins it would be nice if they at least read responses to their own checkins. There seem to be so many exceptions in dealing with the core, it's hard to keep them all straight. Things like: which module needs to be kept compatible with which version, which developers don't read python-checkins, which modules are externally maintained and should get patches instead of commits, who works on which modules, which buildbots are expected to fail and for what reason, which tests leak inconsistently, who has outstanding actions for 2.5, etc. Ok, most of those are just things I need to remember. I guess that's an indication I should be doing a lot less work. n PS. BTW, I don't do any work on the language as part of my day job. Except this week I did suggest that 2 internal patches be submitted. PPS. I blame my whining on lack of sleep. :-) From python-checkins at python.org Thu Jun 1 07:32:50 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 1 Jun 2006 07:32:50 +0200 (CEST) Subject: [Python-checkins] r46584 - python/trunk/Objects/stringobject.c Message-ID: <20060601053250.A3A4D1E400A@bag.python.org> Author: neal.norwitz Date: Thu Jun 1 07:32:49 2006 New Revision: 46584 Modified: python/trunk/Objects/stringobject.c Log: Remove ; at end of macro. There was a compiler recently that warned about extra semi-colons. It may have been the HP C compiler. This file will trigger a bunch of those warnings now. Modified: python/trunk/Objects/stringobject.c ============================================================================== --- python/trunk/Objects/stringobject.c (original) +++ python/trunk/Objects/stringobject.c Thu Jun 1 07:32:49 2006 @@ -1363,7 +1363,7 @@ count++; } /* Always force the list to the expected size. */ -#define FIX_PREALLOC_SIZE(list) ((PyListObject *)list)->ob_size = count; +#define FIX_PREALLOC_SIZE(list) ((PyListObject *)list)->ob_size = count #define SKIP_SPACE(s, i, len) { while (i Author: georg.brandl Date: Thu Jun 1 08:39:19 2006 New Revision: 46585 Modified: python/trunk/Objects/exceptions.c Log: Correctly unpickle 2.4 exceptions via __setstate__ (patch #1498571) Modified: python/trunk/Objects/exceptions.c ============================================================================== --- python/trunk/Objects/exceptions.c (original) +++ python/trunk/Objects/exceptions.c Thu Jun 1 08:39:19 2006 @@ -150,6 +150,29 @@ return PyTuple_Pack(2, self->ob_type, self->args); } +/* + * Needed for backward compatibility, since exceptions used to store + * all their attributes in the __dict__. Code is taken from cPickle's + * load_build function. + */ +static PyObject * +BaseException_setstate(PyObject *self, PyObject *state) +{ + PyObject *d_key, *d_value; + Py_ssize_t i = 0; + + if (state != Py_None) { + if (!PyDict_Check(state)) { + PyErr_SetString(PyExc_TypeError, "state is not a dictionary"); + return NULL; + } + while (PyDict_Next(state, &i, &d_key, &d_value)) { + if (PyObject_SetAttr(self, d_key, d_value) < 0) + return NULL; + } + } + Py_RETURN_NONE; +} #ifdef Py_USING_UNICODE /* while this method generates fairly uninspired output, it a least @@ -168,6 +191,7 @@ static PyMethodDef BaseException_methods[] = { {"__reduce__", (PyCFunction)BaseException_reduce, METH_NOARGS }, + {"__setstate__", (PyCFunction)BaseException_setstate, METH_O }, #ifdef Py_USING_UNICODE {"__unicode__", (PyCFunction)BaseException_unicode, METH_NOARGS }, #endif From python-checkins at python.org Thu Jun 1 10:27:33 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 1 Jun 2006 10:27:33 +0200 (CEST) Subject: [Python-checkins] r46586 - python/trunk/Objects/complexobject.c Message-ID: <20060601082733.392181E400A@bag.python.org> Author: georg.brandl Date: Thu Jun 1 10:27:32 2006 New Revision: 46586 Modified: python/trunk/Objects/complexobject.c Log: Correctly allocate complex types with tp_alloc. (bug #1498638) Modified: python/trunk/Objects/complexobject.c ============================================================================== --- python/trunk/Objects/complexobject.c (original) +++ python/trunk/Objects/complexobject.c Thu Jun 1 10:27:32 2006 @@ -188,7 +188,7 @@ { PyObject *op; - op = PyType_GenericAlloc(type, 0); + op = type->tp_alloc(type, 0); if (op != NULL) ((PyComplexObject *)op)->cval = cval; return op; @@ -1023,7 +1023,7 @@ 0, /* tp_descr_set */ 0, /* tp_dictoffset */ 0, /* tp_init */ - 0, /* tp_alloc */ + PyType_GenericAlloc, /* tp_alloc */ complex_new, /* tp_new */ PyObject_Del, /* tp_free */ }; From buildbot at python.org Thu Jun 1 11:07:35 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 01 Jun 2006 09:07:35 +0000 Subject: [Python-checkins] buildbot failure in S/390 Debian trunk Message-ID: <20060601090735.94F731E400C@bag.python.org> The Buildbot has detected a new failure of S/390 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/S%252F390%2520Debian%2520trunk/builds/0 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl,neal.norwitz BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Thu Jun 1 11:49:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 01 Jun 2006 09:49:37 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060601094937.C23071E400A@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/887 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Thu Jun 1 14:30:47 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 1 Jun 2006 14:30:47 +0200 (CEST) Subject: [Python-checkins] r46587 - python/trunk/Lib/SimpleXMLRPCServer.py Message-ID: <20060601123047.0D9FF1E4010@bag.python.org> Author: georg.brandl Date: Thu Jun 1 14:30:46 2006 New Revision: 46587 Modified: python/trunk/Lib/SimpleXMLRPCServer.py Log: Always close a BZ2Proxy fileobject. Remove silly struct usage. Modified: python/trunk/Lib/SimpleXMLRPCServer.py ============================================================================== --- python/trunk/Lib/SimpleXMLRPCServer.py (original) +++ python/trunk/Lib/SimpleXMLRPCServer.py Thu Jun 1 14:30:46 2006 @@ -247,10 +247,10 @@ of changing method dispatch behavior. """ - params, method = xmlrpclib.loads(data) - - # generate response try: + params, method = xmlrpclib.loads(data) + + # generate response if dispatch_method is not None: response = dispatch_method(method, params) else: From amk at amk.ca Thu Jun 1 14:51:12 2006 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 1 Jun 2006 08:51:12 -0400 Subject: [Python-checkins] r46587 - python/trunk/Lib/SimpleXMLRPCServer.py In-Reply-To: <20060601123047.0D9FF1E4010@bag.python.org> References: <20060601123047.0D9FF1E4010@bag.python.org> Message-ID: <20060601125112.GA9290@localhost.localdomain> On Thu, Jun 01, 2006 at 02:30:47PM +0200, georg.brandl wrote: > Author: georg.brandl > Date: Thu Jun 1 14:30:46 2006 > New Revision: 46587 > > Modified: > python/trunk/Lib/SimpleXMLRPCServer.py > Log: > Always close a BZ2Proxy fileobject. Remove silly struct usage. Looks like an accidental checkin. There are instructions on changing commit messages in the Subversion book at svnbook.red-bean.com. --amk From fredrik at pythonware.com Thu Jun 1 14:57:18 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 1 Jun 2006 14:57:18 +0200 Subject: [Python-checkins] r46587 -python/trunk/Lib/SimpleXMLRPCServer.py References: <20060601123047.0D9FF1E4010@bag.python.org> <20060601125112.GA9290@localhost.localdomain> Message-ID: A.M. Kuchling wrote: >> Modified: >> python/trunk/Lib/SimpleXMLRPCServer.py >> Log: >> Always close a BZ2Proxy fileobject. Remove silly struct usage. > > Looks like an accidental checkin. There are instructions on changing > commit messages in the Subversion book at svnbook.red-bean.com. and in the developer FAQ: http://www.python.org/dev/devfaq.html#how-can-i-edit-the-log-message-of-a-committed-revision From python-checkins at python.org Thu Jun 1 15:00:50 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 1 Jun 2006 15:00:50 +0200 (CEST) Subject: [Python-checkins] r46588 - python/trunk/Lib/test/test_exceptions.py Message-ID: <20060601130050.F24351E400A@bag.python.org> Author: georg.brandl Date: Thu Jun 1 15:00:49 2006 New Revision: 46588 Modified: python/trunk/Lib/test/test_exceptions.py Log: Some code style tweaks, and remove apply. Modified: python/trunk/Lib/test/test_exceptions.py ============================================================================== --- python/trunk/Lib/test/test_exceptions.py (original) +++ python/trunk/Lib/test/test_exceptions.py Thu Jun 1 15:00:49 2006 @@ -189,69 +189,68 @@ except Exception, e: sampleUnicodeDecodeError = e exceptionList = [ - ( BaseException, (), { 'message' : '', 'args' : () }), - ( BaseException, (1, ), { 'message' : 1, 'args' : ( 1, ) }), - ( BaseException, ('foo', ), { 'message' : 'foo', 'args' : ( 'foo', ) }), - ( BaseException, ('foo', 1), { 'message' : '', 'args' : ( 'foo', 1 ) }), - ( SystemExit, ('foo',), { 'message' : 'foo', 'args' : ( 'foo', ), - 'code' : 'foo' }), - ( IOError, ('foo',), { 'message' : 'foo', 'args' : ( 'foo', ), }), - ( IOError, ('foo', 'bar'), { 'message' : '', - 'args' : ('foo', 'bar'), }), - ( IOError, ('foo', 'bar', 'baz'), - { 'message' : '', 'args' : ('foo', 'bar'), }), - ( EnvironmentError, ('errnoStr', 'strErrorStr', 'filenameStr'), - { 'message' : '', 'args' : ('errnoStr', 'strErrorStr'), - 'strerror' : 'strErrorStr', - 'errno' : 'errnoStr', 'filename' : 'filenameStr' }), - ( EnvironmentError, (1, 'strErrorStr', 'filenameStr'), - { 'message' : '', 'args' : (1, 'strErrorStr'), - 'strerror' : 'strErrorStr', 'errno' : 1, - 'filename' : 'filenameStr' }), - ( SyntaxError, ('msgStr',), - { 'message' : 'msgStr', 'args' : ('msgStr', ), - 'print_file_and_line' : None, 'msg' : 'msgStr', - 'filename' : None, 'lineno' : None, 'offset' : None, - 'text' : None }), - ( SyntaxError, ('msgStr', ('filenameStr', 'linenoStr', 'offsetStr', - 'textStr')), - { 'message' : '', 'args' : ('msgStr', ('filenameStr', - 'linenoStr', 'offsetStr', 'textStr' )), - 'print_file_and_line' : None, 'msg' : 'msgStr', - 'filename' : 'filenameStr', 'lineno' : 'linenoStr', - 'offset' : 'offsetStr', 'text' : 'textStr' }), - ( SyntaxError, ('msgStr', 'filenameStr', 'linenoStr', 'offsetStr', - 'textStr', 'print_file_and_lineStr'), - { 'message' : '', 'args' : ('msgStr', 'filenameStr', - 'linenoStr', 'offsetStr', 'textStr', - 'print_file_and_lineStr'), - 'print_file_and_line' : None, 'msg' : 'msgStr', - 'filename' : None, 'lineno' : None, 'offset' : None, - 'text' : None }), - ( UnicodeError, (), - { 'message' : '', 'args' : (), }), - ( sampleUnicodeEncodeError, - { 'message' : '', 'args' : ('ascii', u'Hello \xe1', 6, 7, - 'ordinal not in range(128)'), - 'encoding' : 'ascii', 'object' : u'Hello \xe1', - 'start' : 6, 'reason' : 'ordinal not in range(128)' }), - ( sampleUnicodeDecodeError, - { 'message' : '', 'args' : ('ascii', '\xff', 0, 1, - 'ordinal not in range(128)'), - 'encoding' : 'ascii', 'object' : '\xff', - 'start' : 0, 'reason' : 'ordinal not in range(128)' }), - ( UnicodeTranslateError, (u"\u3042", 0, 1, "ouch"), - { 'message' : '', 'args' : (u'\u3042', 0, 1, 'ouch'), - 'object' : u'\u3042', 'reason' : 'ouch', - 'start' : 0, 'end' : 1 }), - ] + (BaseException, (), {'message' : '', 'args' : ()}), + (BaseException, (1, ), {'message' : 1, 'args' : (1,)}), + (BaseException, ('foo',), + {'message' : 'foo', 'args' : ('foo',)}), + (BaseException, ('foo', 1), + {'message' : '', 'args' : ('foo', 1)}), + (SystemExit, ('foo',), + {'message' : 'foo', 'args' : ('foo',), 'code' : 'foo'}), + (IOError, ('foo',), + {'message' : 'foo', 'args' : ('foo',)}), + (IOError, ('foo', 'bar'), + {'message' : '', 'args' : ('foo', 'bar')}), + (IOError, ('foo', 'bar', 'baz'), + {'message' : '', 'args' : ('foo', 'bar')}), + (EnvironmentError, ('errnoStr', 'strErrorStr', 'filenameStr'), + {'message' : '', 'args' : ('errnoStr', 'strErrorStr'), + 'strerror' : 'strErrorStr', 'errno' : 'errnoStr', + 'filename' : 'filenameStr'}), + (EnvironmentError, (1, 'strErrorStr', 'filenameStr'), + {'message' : '', 'args' : (1, 'strErrorStr'), 'errno' : 1, + 'strerror' : 'strErrorStr', 'filename' : 'filenameStr'}), + (SyntaxError, ('msgStr',), + {'message' : 'msgStr', 'args' : ('msgStr',), 'text' : None, + 'print_file_and_line' : None, 'msg' : 'msgStr', + 'filename' : None, 'lineno' : None, 'offset' : None}), + (SyntaxError, ('msgStr', ('filenameStr', 'linenoStr', 'offsetStr', + 'textStr')), + {'message' : '', 'offset' : 'offsetStr', 'text' : 'textStr', + 'args' : ('msgStr', ('filenameStr', 'linenoStr', + 'offsetStr', 'textStr')), + 'print_file_and_line' : None, 'msg' : 'msgStr', + 'filename' : 'filenameStr', 'lineno' : 'linenoStr'}), + (SyntaxError, ('msgStr', 'filenameStr', 'linenoStr', 'offsetStr', + 'textStr', 'print_file_and_lineStr'), + {'message' : '', 'text' : None, + 'args' : ('msgStr', 'filenameStr', 'linenoStr', 'offsetStr', + 'textStr', 'print_file_and_lineStr'), + 'print_file_and_line' : None, 'msg' : 'msgStr', + 'filename' : None, 'lineno' : None, 'offset' : None}), + (UnicodeError, (), {'message' : '', 'args' : (),}), + (sampleUnicodeEncodeError, + {'message' : '', 'args' : ('ascii', u'Hello \xe1', 6, 7, + 'ordinal not in range(128)'), + 'encoding' : 'ascii', 'object' : u'Hello \xe1', + 'start' : 6, 'reason' : 'ordinal not in range(128)'}), + (sampleUnicodeDecodeError, + {'message' : '', 'args' : ('ascii', '\xff', 0, 1, + 'ordinal not in range(128)'), + 'encoding' : 'ascii', 'object' : '\xff', + 'start' : 0, 'reason' : 'ordinal not in range(128)'}), + (UnicodeTranslateError, (u"\u3042", 0, 1, "ouch"), + {'message' : '', 'args' : (u'\u3042', 0, 1, 'ouch'), + 'object' : u'\u3042', 'reason' : 'ouch', + 'start' : 0, 'end' : 1}), + ] try: exceptionList.append( - ( WindowsError, (1, 'strErrorStr', 'filenameStr'), - { 'message' : '', 'args' : (1, 'strErrorStr'), - 'strerror' : 'strErrorStr', - 'errno' : 22, 'filename' : 'filenameStr', - 'winerror' : 1 })) + (WindowsError, (1, 'strErrorStr', 'filenameStr'), + {'message' : '', 'args' : (1, 'strErrorStr'), + 'strerror' : 'strErrorStr', 'winerror' : 1, + 'errno' : 22, 'filename' : 'filenameStr'}) + ) except NameError: pass import pickle, random @@ -259,9 +258,13 @@ for args in exceptionList: expected = args[-1] try: - if len(args) == 2: raise args[0] - else: raise apply(args[0], args[1]) + exc = args[0] + if len(args) == 2: raise exc + else: raise exc(*args[1]) except BaseException, e: + if (e is not exc and # needed for sampleUnicode errors + type(e) is not exc): + raise for checkArgName in expected.keys(): self.assertEquals(repr(getattr(e, checkArgName)), repr(expected[checkArgName]), @@ -280,6 +283,7 @@ """test that builtin exception don't take keyword args, but user-defined subclasses can if they want""" self.assertRaises(TypeError, BaseException, a=1) + class DerivedException(BaseException): def __init__(self, fancy_arg): BaseException.__init__(self) From python-checkins at python.org Thu Jun 1 15:03:25 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 1 Jun 2006 15:03:25 +0200 (CEST) Subject: [Python-checkins] r46587 - svn:log Message-ID: <20060601130325.4D5D51E400A@bag.python.org> Author: georg.brandl Revision: 46587 Property Name: svn:log New Property Value: Correctly dispatch Faults in loads (patch #1498627) From g.brandl at gmx.net Thu Jun 1 15:03:45 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 01 Jun 2006 15:03:45 +0200 Subject: [Python-checkins] r46587 -python/trunk/Lib/SimpleXMLRPCServer.py In-Reply-To: References: <20060601123047.0D9FF1E4010@bag.python.org> <20060601125112.GA9290@localhost.localdomain> Message-ID: Fredrik Lundh wrote: > A.M. Kuchling wrote: > >>> Modified: >>> python/trunk/Lib/SimpleXMLRPCServer.py >>> Log: >>> Always close a BZ2Proxy fileobject. Remove silly struct usage. >> >> Looks like an accidental checkin. There are instructions on changing >> commit messages in the Subversion book at svnbook.red-bean.com. > > and in the developer FAQ: > > http://www.python.org/dev/devfaq.html#how-can-i-edit-the-log-message-of-a-committed-revision Thanks! I wish SVN would overwrite its svn-commit.tmp files, or clean them up somehow magically... Georg From python-checkins at python.org Thu Jun 1 15:19:13 2006 From: python-checkins at python.org (armin.rigo) Date: Thu, 1 Jun 2006 15:19:13 +0200 (CEST) Subject: [Python-checkins] r46589 - in python/trunk: Lib/test/crashers/dictresize_attack.py Lib/test/output/test_operations Lib/test/test_operations.py Misc/NEWS Objects/dictobject.c Python/ceval.c Message-ID: <20060601131913.98AE31E400A@bag.python.org> Author: armin.rigo Date: Thu Jun 1 15:19:12 2006 New Revision: 46589 Removed: python/trunk/Lib/test/crashers/dictresize_attack.py Modified: python/trunk/Lib/test/output/test_operations python/trunk/Lib/test/test_operations.py python/trunk/Misc/NEWS python/trunk/Objects/dictobject.c python/trunk/Python/ceval.c Log: [ 1497053 ] Let dicts propagate the exceptions in user __eq__(). [ 1456209 ] dictresize() vulnerability ( <- backport candidate ). Deleted: /python/trunk/Lib/test/crashers/dictresize_attack.py ============================================================================== --- /python/trunk/Lib/test/crashers/dictresize_attack.py Thu Jun 1 15:19:12 2006 +++ (empty file) @@ -1,32 +0,0 @@ -# http://www.python.org/sf/1456209 - -# A dictresize() attack. If oldtable == mp->ma_smalltable then pure -# Python code can mangle with mp->ma_smalltable while it is being walked -# over. - -class X(object): - - def __hash__(self): - return 5 - - def __eq__(self, other): - if resizing: - d.clear() - return False - - -d = {} - -resizing = False - -d[X()] = 1 -d[X()] = 2 -d[X()] = 3 -d[X()] = 4 -d[X()] = 5 - -# now trigger a resize -resizing = True -d[9] = 6 - -# ^^^ I get Segmentation fault or Illegal instruction here. Modified: python/trunk/Lib/test/output/test_operations ============================================================================== --- python/trunk/Lib/test/output/test_operations (original) +++ python/trunk/Lib/test/output/test_operations Thu Jun 1 15:19:12 2006 @@ -1,6 +1,21 @@ test_operations 3. Operations XXX Mostly not yet implemented -3.1 Dictionary lookups succeed even if __cmp__() raises an exception +3.1 Dictionary lookups fail if __cmp__() raises an exception raising error -No exception passed through. +d[x2] = 2: caught the RuntimeError outside +raising error +z = d[x2]: caught the RuntimeError outside +raising error +x2 in d: caught the RuntimeError outside +raising error +d.has_key(x2): caught the RuntimeError outside +raising error +d.get(x2): caught the RuntimeError outside +raising error +d.setdefault(x2, 42): caught the RuntimeError outside +raising error +d.pop(x2): caught the RuntimeError outside +raising error +d.update({x2: 2}): caught the RuntimeError outside +resize bugs not triggered. Modified: python/trunk/Lib/test/test_operations.py ============================================================================== --- python/trunk/Lib/test/test_operations.py (original) +++ python/trunk/Lib/test/test_operations.py Thu Jun 1 15:19:12 2006 @@ -5,27 +5,16 @@ print 'XXX Mostly not yet implemented' -print '3.1 Dictionary lookups succeed even if __cmp__() raises an exception' - -# SourceForge bug #112558: -# http://sourceforge.net/bugs/?func=detailbug&bug_id=112558&group_id=5470 +print '3.1 Dictionary lookups fail if __cmp__() raises an exception' class BadDictKey: - already_printed_raising_error = 0 def __hash__(self): return hash(self.__class__) def __cmp__(self, other): if isinstance(other, self.__class__): - if not BadDictKey.already_printed_raising_error: - # How many times __cmp__ gets called depends on the hash - # code and the internals of the dict implementation; we - # know it will be called at least once, but that's it. - # already_printed_raising_error makes sure the expected- - # output file prints the msg at most once. - BadDictKey.already_printed_raising_error = 1 - print "raising error" + print "raising error" raise RuntimeError, "gotcha" return other @@ -33,8 +22,21 @@ x1 = BadDictKey() x2 = BadDictKey() d[x1] = 1 -d[x2] = 2 -print "No exception passed through." +for stmt in ['d[x2] = 2', + 'z = d[x2]', + 'x2 in d', + 'd.has_key(x2)', + 'd.get(x2)', + 'd.setdefault(x2, 42)', + 'd.pop(x2)', + 'd.update({x2: 2})']: + try: + exec stmt + except RuntimeError: + print "%s: caught the RuntimeError outside" % (stmt,) + else: + print "%s: No exception passed through!" # old CPython behavior + # Dict resizing bug, found by Jack Jansen in 2.2 CVS development. # This version got an assert failure in debug build, infinite loop in @@ -50,3 +52,27 @@ del d[i] for i in range(5, 9): # i==8 was the problem d[i] = i + + +# Another dict resizing bug (SF bug #1456209). +# This caused Segmentation faults or Illegal instructions. + +class X(object): + def __hash__(self): + return 5 + def __eq__(self, other): + if resizing: + d.clear() + return False +d = {} +resizing = False +d[X()] = 1 +d[X()] = 2 +d[X()] = 3 +d[X()] = 4 +d[X()] = 5 +# now trigger a resize +resizing = True +d[9] = 6 + +print 'resize bugs not triggered.' Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 1 15:19:12 2006 @@ -59,6 +59,11 @@ - Patch #1479181: split open() and file() from being aliases for each other. +- Patch #1497053: Exceptions occurring in __eq__() methods were always + silently ignored by dictionaries when comparing keys. They are now + passed through (except when using the C API function PyDict_GetItem(), + whose semantics did not change). + Extension Modules ----------------- Modified: python/trunk/Objects/dictobject.c ============================================================================== --- python/trunk/Objects/dictobject.c (original) +++ python/trunk/Objects/dictobject.c Thu Jun 1 15:19:12 2006 @@ -241,10 +241,7 @@ register Py_ssize_t mask = mp->ma_mask; dictentry *ep0 = mp->ma_table; register dictentry *ep; - register int restore_error; - register int checked_error; register int cmp; - PyObject *err_type, *err_value, *err_tb; PyObject *startkey; i = hash & mask; @@ -252,24 +249,17 @@ if (ep->me_key == NULL || ep->me_key == key) return ep; - restore_error = checked_error = 0; if (ep->me_key == dummy) freeslot = ep; else { if (ep->me_hash == hash) { - /* error can't have been checked yet */ - checked_error = 1; - if (PyErr_Occurred()) { - restore_error = 1; - PyErr_Fetch(&err_type, &err_value, &err_tb); - } startkey = ep->me_key; cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); if (cmp < 0) - PyErr_Clear(); + return NULL; if (ep0 == mp->ma_table && ep->me_key == startkey) { if (cmp > 0) - goto Done; + return ep; } else { /* The compare did major nasty stuff to the @@ -277,8 +267,7 @@ * XXX A clever adversary could prevent this * XXX from terminating. */ - ep = lookdict(mp, key, hash); - goto Done; + return lookdict(mp, key, hash); } } freeslot = NULL; @@ -289,29 +278,18 @@ for (perturb = hash; ; perturb >>= PERTURB_SHIFT) { i = (i << 2) + i + perturb + 1; ep = &ep0[i & mask]; - if (ep->me_key == NULL) { - if (freeslot != NULL) - ep = freeslot; - break; - } + if (ep->me_key == NULL) + return freeslot == NULL ? ep : freeslot; if (ep->me_key == key) - break; + return ep; if (ep->me_hash == hash && ep->me_key != dummy) { - if (!checked_error) { - checked_error = 1; - if (PyErr_Occurred()) { - restore_error = 1; - PyErr_Fetch(&err_type, &err_value, - &err_tb); - } - } startkey = ep->me_key; cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); if (cmp < 0) - PyErr_Clear(); + return NULL; if (ep0 == mp->ma_table && ep->me_key == startkey) { if (cmp > 0) - break; + return ep; } else { /* The compare did major nasty stuff to the @@ -319,18 +297,12 @@ * XXX A clever adversary could prevent this * XXX from terminating. */ - ep = lookdict(mp, key, hash); - break; + return lookdict(mp, key, hash); } } else if (ep->me_key == dummy && freeslot == NULL) freeslot = ep; } - -Done: - if (restore_error) - PyErr_Restore(err_type, err_value, err_tb); - return ep; } /* @@ -400,7 +372,7 @@ Used both by the internal resize routine and by the public insert routine. Eats a reference to key and one to value. */ -static void +static int insertdict(register dictobject *mp, PyObject *key, long hash, PyObject *value) { PyObject *old_value; @@ -409,6 +381,11 @@ assert(mp->ma_lookup != NULL); ep = mp->ma_lookup(mp, key, hash); + if (ep == NULL) { + Py_DECREF(key); + Py_DECREF(value); + return -1; + } if (ep->me_value != NULL) { old_value = ep->me_value; ep->me_value = value; @@ -427,6 +404,36 @@ ep->me_value = value; mp->ma_used++; } + return 0; +} + +/* +Internal routine used by dictresize() to insert an item which is +known to be absent from the dict. This routine also assumes that +the dict contains no deleted entries. Besides the performance benefit, +using insertdict() in dictresize() is dangerous (SF bug #1456209). +*/ +static void +insertdict_clean(register dictobject *mp, PyObject *key, long hash, + PyObject *value) +{ + register Py_ssize_t i; + register size_t perturb; + register unsigned int mask = mp->ma_mask; + dictentry *ep0 = mp->ma_table; + register dictentry *ep; + + i = hash & mask; + ep = &ep0[i]; + for (perturb = hash; ep->me_key != NULL; perturb >>= PERTURB_SHIFT) { + i = (i << 2) + i + perturb + 1; + ep = &ep0[i & mask]; + } + mp->ma_fill++; + ep->me_key = key; + ep->me_hash = (Py_ssize_t)hash; + ep->me_value = value; + mp->ma_used++; } /* @@ -501,7 +508,8 @@ for (ep = oldtable; i > 0; ep++) { if (ep->me_value != NULL) { /* active entry */ --i; - insertdict(mp, ep->me_key, ep->me_hash, ep->me_value); + insertdict_clean(mp, ep->me_key, (long)ep->me_hash, + ep->me_value); } else if (ep->me_key != NULL) { /* dummy entry */ --i; @@ -521,6 +529,8 @@ { long hash; dictobject *mp = (dictobject *)op; + dictentry *ep; + PyThreadState *tstate; if (!PyDict_Check(op)) { return NULL; } @@ -533,7 +543,29 @@ return NULL; } } - return (mp->ma_lookup)(mp, key, hash)->me_value; + + /* We can arrive here with a NULL tstate during initialization: + try running "python -Wi" for an example related to string + interning. Let's just hope that no exception occurs then... */ + tstate = PyThreadState_GET(); + if (tstate != NULL && tstate->curexc_type != NULL) { + /* preserve the existing exception */ + PyObject *err_type, *err_value, *err_tb; + PyErr_Fetch(&err_type, &err_value, &err_tb); + ep = (mp->ma_lookup)(mp, key, hash); + /* ignore errors */ + PyErr_Restore(err_type, err_value, err_tb); + if (ep == NULL) + return NULL; + } + else { + ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) { + PyErr_Clear(); + return NULL; + } + } + return ep->me_value; } /* CAUTION: PyDict_SetItem() must guarantee that it won't resize the @@ -568,7 +600,8 @@ n_used = mp->ma_used; Py_INCREF(value); Py_INCREF(key); - insertdict(mp, key, hash, value); + if (insertdict(mp, key, hash, value) != 0) + return -1; /* If we added a key, we can safely resize. Otherwise just return! * If fill >= 2/3 size, adjust size. Normally, this doubles or * quaduples the size, but it's also possible for the dict to shrink @@ -608,6 +641,8 @@ } mp = (dictobject *)op; ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) + return -1; if (ep->me_value == NULL) { PyErr_SetObject(PyExc_KeyError, key); return -1; @@ -893,6 +928,7 @@ { PyObject *v; long hash; + dictentry *ep; assert(mp->ma_table != NULL); if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { @@ -900,7 +936,10 @@ if (hash == -1) return NULL; } - v = (mp->ma_lookup)(mp, key, hash) -> me_value; + ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) + return NULL; + v = ep->me_value; if (v == NULL) { if (!PyDict_CheckExact(mp)) { /* Look up __missing__ method if we're a subclass. */ @@ -1258,9 +1297,10 @@ PyDict_GetItem(a, entry->me_key) == NULL)) { Py_INCREF(entry->me_key); Py_INCREF(entry->me_value); - insertdict(mp, entry->me_key, - (long)entry->me_hash, - entry->me_value); + if (insertdict(mp, entry->me_key, + (long)entry->me_hash, + entry->me_value) != 0) + return -1; } } } @@ -1568,13 +1608,17 @@ { long hash; register long ok; + dictentry *ep; if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { hash = PyObject_Hash(key); if (hash == -1) return NULL; } - ok = (mp->ma_lookup)(mp, key, hash)->me_value != NULL; + ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) + return NULL; + ok = ep->me_value != NULL; return PyBool_FromLong(ok); } @@ -1585,6 +1629,7 @@ PyObject *failobj = Py_None; PyObject *val = NULL; long hash; + dictentry *ep; if (!PyArg_UnpackTuple(args, "get", 1, 2, &key, &failobj)) return NULL; @@ -1595,8 +1640,10 @@ if (hash == -1) return NULL; } - val = (mp->ma_lookup)(mp, key, hash)->me_value; - + ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) + return NULL; + val = ep->me_value; if (val == NULL) val = failobj; Py_INCREF(val); @@ -1611,6 +1658,7 @@ PyObject *failobj = Py_None; PyObject *val = NULL; long hash; + dictentry *ep; if (!PyArg_UnpackTuple(args, "setdefault", 1, 2, &key, &failobj)) return NULL; @@ -1621,7 +1669,10 @@ if (hash == -1) return NULL; } - val = (mp->ma_lookup)(mp, key, hash)->me_value; + ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) + return NULL; + val = ep->me_value; if (val == NULL) { val = failobj; if (PyDict_SetItem((PyObject*)mp, key, failobj)) @@ -1665,6 +1716,8 @@ return NULL; } ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) + return NULL; if (ep->me_value == NULL) { if (deflt) { Py_INCREF(deflt); @@ -1884,6 +1937,7 @@ { long hash; dictobject *mp = (dictobject *)op; + dictentry *ep; if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { @@ -1891,7 +1945,10 @@ if (hash == -1) return -1; } - return (mp->ma_lookup)(mp, key, hash)->me_value != NULL; + ep = (mp->ma_lookup)(mp, key, hash); + if (ep == NULL) + return -1; + return ep->me_value != NULL; } /* Hack to implement "key in dict" */ Modified: python/trunk/Python/ceval.c ============================================================================== --- python/trunk/Python/ceval.c (original) +++ python/trunk/Python/ceval.c Thu Jun 1 15:19:12 2006 @@ -1855,15 +1855,26 @@ long hash = ((PyStringObject *)w)->ob_shash; if (hash != -1) { PyDictObject *d; + PyDictEntry *e; d = (PyDictObject *)(f->f_globals); - x = d->ma_lookup(d, w, hash)->me_value; + e = d->ma_lookup(d, w, hash); + if (e == NULL) { + x = NULL; + break; + } + x = e->me_value; if (x != NULL) { Py_INCREF(x); PUSH(x); continue; } d = (PyDictObject *)(f->f_builtins); - x = d->ma_lookup(d, w, hash)->me_value; + e = d->ma_lookup(d, w, hash); + if (e == NULL) { + x = NULL; + break; + } + x = e->me_value; if (x != NULL) { Py_INCREF(x); PUSH(x); From python-checkins at python.org Thu Jun 1 15:41:46 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 1 Jun 2006 15:41:46 +0200 (CEST) Subject: [Python-checkins] r46590 - python/trunk/Lib/SimpleXMLRPCServer.py Message-ID: <20060601134146.8EEB21E400E@bag.python.org> Author: tim.peters Date: Thu Jun 1 15:41:46 2006 New Revision: 46590 Modified: python/trunk/Lib/SimpleXMLRPCServer.py Log: Whitespace normalization. Modified: python/trunk/Lib/SimpleXMLRPCServer.py ============================================================================== --- python/trunk/Lib/SimpleXMLRPCServer.py (original) +++ python/trunk/Lib/SimpleXMLRPCServer.py Thu Jun 1 15:41:46 2006 @@ -445,7 +445,7 @@ if not self.is_rpc_path_valid(): self.report_404() return - + try: # Get arguments by reading body of request. # We read this in chunks to avoid straining @@ -486,15 +486,15 @@ def report_404 (self): # Report a 404 error - self.send_response(404) - response = 'No such page' - self.send_header("Content-type", "text/plain") - self.send_header("Content-length", str(len(response))) - self.end_headers() - self.wfile.write(response) - # shut down the connection - self.wfile.flush() - self.connection.shutdown(1) + self.send_response(404) + response = 'No such page' + self.send_header("Content-type", "text/plain") + self.send_header("Content-length", str(len(response))) + self.end_headers() + self.wfile.write(response) + # shut down the connection + self.wfile.flush() + self.connection.shutdown(1) def log_request(self, code='-', size='-'): """Selectively log an accepted request.""" From python-checkins at python.org Thu Jun 1 15:49:23 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 1 Jun 2006 15:49:23 +0200 (CEST) Subject: [Python-checkins] r46591 - python/trunk/Misc/NEWS Message-ID: <20060601134923.B816B1E400A@bag.python.org> Author: tim.peters Date: Thu Jun 1 15:49:23 2006 New Revision: 46591 Modified: python/trunk/Misc/NEWS Log: Record bugs 1275608 and 1456209 as being fixed. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 1 15:49:23 2006 @@ -59,10 +59,15 @@ - Patch #1479181: split open() and file() from being aliases for each other. -- Patch #1497053: Exceptions occurring in __eq__() methods were always - silently ignored by dictionaries when comparing keys. They are now - passed through (except when using the C API function PyDict_GetItem(), - whose semantics did not change). +- Patch #1497053 & bug #1275608: Exceptions occurring in ``__eq__()`` + methods were always silently ignored by dictionaries when comparing keys. + They are now passed through (except when using the C API function + ``PyDict_GetItem()``, whose semantics did not change). + +- Bug #1456209: In some obscure cases it was possible for a class with a + custom ``__eq__()`` method to confuse dict internals when class instances + were used as a dict's keys and the ``__eq__()`` method mutated the dict. + No, you don't have any code that did this ;-) Extension Modules ----------------- From python-checkins at python.org Thu Jun 1 15:56:28 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 1 Jun 2006 15:56:28 +0200 (CEST) Subject: [Python-checkins] r46592 - python/trunk/Lib/test/string_tests.py Message-ID: <20060601135628.4B9991E400C@bag.python.org> Author: tim.peters Date: Thu Jun 1 15:56:26 2006 New Revision: 46592 Modified: python/trunk/Lib/test/string_tests.py Log: Re-enable a new empty-string test added during the NFS sprint, but disabled then because str and unicode strings gave different results. The implementations were repaired later during the sprint, but the new test remained disabled. Modified: python/trunk/Lib/test/string_tests.py ============================================================================== --- python/trunk/Lib/test/string_tests.py (original) +++ python/trunk/Lib/test/string_tests.py Thu Jun 1 15:56:26 2006 @@ -494,12 +494,7 @@ # Operations on the empty string EQ("", "", "replace", "", "") - - #EQ("A", "", "replace", "", "A") - # That was the correct result; this is the result we actually get - # now (for str, but not for unicode): - #EQ("", "", "replace", "", "A") - + EQ("A", "", "replace", "", "A") EQ("", "", "replace", "A", "") EQ("", "", "replace", "A", "A") EQ("", "", "replace", "", "", 100) From buildbot at python.org Thu Jun 1 16:00:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 01 Jun 2006 14:00:55 +0000 Subject: [Python-checkins] buildbot warnings in S/390 Debian trunk Message-ID: <20060601140056.174C41E400D@bag.python.org> The Buildbot has detected a new failure of S/390 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/S%252F390%2520Debian%2520trunk/builds/1 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 1 16:12:47 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 01 Jun 2006 14:12:47 +0000 Subject: [Python-checkins] buildbot failure in x86 XP trunk Message-ID: <20060601141247.633781E400F@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/897 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From buildbot at python.org Thu Jun 1 16:34:38 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 01 Jun 2006 14:34:38 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060601143439.020D11E400A@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/549 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 1 17:26:04 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 01 Jun 2006 15:26:04 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060601152604.EFE161E400B@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/515 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 1 17:39:04 2006 From: python-checkins at python.org (phillip.eby) Date: Thu, 1 Jun 2006 17:39:04 +0200 (CEST) Subject: [Python-checkins] r46593 - sandbox/branches/setuptools-0.6/EasyInstall.txt Message-ID: <20060601153904.E801D1E400A@bag.python.org> Author: phillip.eby Date: Thu Jun 1 17:39:04 2006 New Revision: 46593 Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt Log: Add missing 0.6b2 release note Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt ============================================================================== --- sandbox/branches/setuptools-0.6/EasyInstall.txt (original) +++ sandbox/branches/setuptools-0.6/EasyInstall.txt Thu Jun 1 17:39:04 2006 @@ -1103,6 +1103,8 @@ * Construct ``.pth`` file paths in such a way that installing an egg whose name begins with ``import`` doesn't cause a syntax error. + * Fixed a bogus warning message that wasn't updated since the 0.5 versions. + 0.6b1 * Better ambiguity management: accept ``#egg`` name/version even if processing what appears to be a correctly-named distutils file, and ignore ``.egg`` From brett at python.org Thu Jun 1 17:50:29 2006 From: brett at python.org (Brett Cannon) Date: Thu, 1 Jun 2006 08:50:29 -0700 Subject: [Python-checkins] r46472 - python/trunk/PCbuild8 python/trunk/PCbuild8/Uninstal.wse python/trunk/PCbuild8/_bsddb.vcproj python/trunk/PCbuild8/_ctypes.vcproj python/trunk/PCbuild8/_ctypes_test.vcproj python/trunk/PCbuild8/_elementtree.vcproj pytho In-Reply-To: References: <129CEF95A523704B9D46959C922A28000282A8ED@nemesis.central.ccp.cc> <1149108731.447e01fb8520c@www.domainfactory-webmail.de> Message-ID: On 5/31/06, Neal Norwitz wrote: > > On 5/31/06, martin at v.loewis.de wrote: > > Zitat von "Kristj?n V. J?nsson" : > > > > > p.s. do you think I ought to subscribe to python-checkins? > > > > No. I personally didn't for quite a long time. I'll just remember > > to CC you the next time. > > I wish you would subscribe. I wish everyone with checkin privs would > subscribe, even if they don't review the checkins it would be nice if > they at least read responses to their own checkins. > > There seem to be so many exceptions in dealing with the core, it's > hard to keep them all straight. Things like: which module needs to > be kept compatible with which version, which developers don't read > python-checkins, which modules are externally maintained and should > get patches instead of commits, who works on which modules, which > buildbots are expected to fail and for what reason, which tests leak > inconsistently, who has outstanding actions for 2.5, etc. > > Ok, most of those are just things I need to remember. I guess that's > an indication I should be doing a lot less work. =) Well, the compatibility and what modules are maintained externally are both PEPs at this point (the externally maintained PEP is 360 and has not been pushed to the site yet). So at least for those you just need to remember to read something. =) As for expected Buildbot failures, it might be nice to have something somewhere written down about that since I know I don't remember those details either. -Brett n > > PS. BTW, I don't do any work on the language as part of my day job. > Except this week I did suggest that 2 internal patches be submitted. > > PPS. I blame my whining on lack of sleep. :-) > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060601/27db9adc/attachment.html From python-checkins at python.org Thu Jun 1 17:50:45 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 1 Jun 2006 17:50:45 +0200 (CEST) Subject: [Python-checkins] r46594 - python/trunk/Objects/dictobject.c Message-ID: <20060601155045.83BD71E400A@bag.python.org> Author: tim.peters Date: Thu Jun 1 17:50:44 2006 New Revision: 46594 Modified: python/trunk/Objects/dictobject.c Log: Armin committed his patch while I was reviewing it (I'm sure he didn't know this), so merged in some changes I made during review. Nothing material apart from changing a new `mask` local from int to Py_ssize_t. Mostly this is repairing comments that were made incorrect, and adding new comments. Also a few minor code rewrites for clarity or helpful succinctness. Modified: python/trunk/Objects/dictobject.c ============================================================================== --- python/trunk/Objects/dictobject.c (original) +++ python/trunk/Objects/dictobject.c Thu Jun 1 17:50:44 2006 @@ -227,24 +227,28 @@ contributions by Reimer Behrends, Jyrki Alakuijala, Vladimir Marangozov and Christian Tismer). -This function must never return NULL; failures are indicated by returning -a dictentry* for which the me_value field is NULL. Exceptions are never -reported by this function, and outstanding exceptions are maintained. +lookdict() is general-purpose, and may return NULL if (and only if) a +comparison raises an exception (this was new in Python 2.5). +lookdict_string() below is specialized to string keys, comparison of which can +never raise an exception; that function can never return NULL. For both, when +the key isn't found a dictentry* is returned for which the me_value field is +NULL; this is the slot in the dict at which the key would have been found, and +the caller can (if it wishes) add the pair to the returned +dictentry*. */ - static dictentry * lookdict(dictobject *mp, PyObject *key, register long hash) { - register Py_ssize_t i; + register size_t i; register size_t perturb; register dictentry *freeslot; - register Py_ssize_t mask = mp->ma_mask; + register size_t mask = (size_t)mp->ma_mask; dictentry *ep0 = mp->ma_table; register dictentry *ep; register int cmp; PyObject *startkey; - i = hash & mask; + i = (size_t)hash & mask; ep = &ep0[i]; if (ep->me_key == NULL || ep->me_key == key) return ep; @@ -307,21 +311,20 @@ /* * Hacked up version of lookdict which can assume keys are always strings; - * this assumption allows testing for errors during PyObject_Compare() to - * be dropped; string-string comparisons never raise exceptions. This also - * means we don't need to go through PyObject_Compare(); we can always use - * _PyString_Eq directly. + * this assumption allows testing for errors during PyObject_RichCompareBool() + * to be dropped; string-string comparisons never raise exceptions. This also + * means we don't need to go through PyObject_RichCompareBool(); we can always + * use _PyString_Eq() directly. * - * This is valuable because the general-case error handling in lookdict() is - * expensive, and dicts with pure-string keys are very common. + * This is valuable because dicts with only string keys are very common. */ static dictentry * lookdict_string(dictobject *mp, PyObject *key, register long hash) { - register Py_ssize_t i; + register size_t i; register size_t perturb; register dictentry *freeslot; - register Py_ssize_t mask = mp->ma_mask; + register size_t mask = (size_t)mp->ma_mask; dictentry *ep0 = mp->ma_table; register dictentry *ep; @@ -343,10 +346,8 @@ if (ep->me_key == dummy) freeslot = ep; else { - if (ep->me_hash == hash - && _PyString_Eq(ep->me_key, key)) { + if (ep->me_hash == hash && _PyString_Eq(ep->me_key, key)) return ep; - } freeslot = NULL; } @@ -371,6 +372,7 @@ Internal routine to insert a new item into the table. Used both by the internal resize routine and by the public insert routine. Eats a reference to key and one to value. +Returns -1 if an error occurred, or 0 on success. */ static int insertdict(register dictobject *mp, PyObject *key, long hash, PyObject *value) @@ -412,14 +414,16 @@ known to be absent from the dict. This routine also assumes that the dict contains no deleted entries. Besides the performance benefit, using insertdict() in dictresize() is dangerous (SF bug #1456209). +Note that no refcounts are changed by this routine; if needed, the caller +is responsible for incref'ing `key` and `value`. */ static void insertdict_clean(register dictobject *mp, PyObject *key, long hash, PyObject *value) { - register Py_ssize_t i; + register size_t i; register size_t perturb; - register unsigned int mask = mp->ma_mask; + register size_t mask = (size_t)mp->ma_mask; dictentry *ep0 = mp->ma_table; register dictentry *ep; @@ -429,6 +433,7 @@ i = (i << 2) + i + perturb + 1; ep = &ep0[i & mask]; } + assert(ep->me_value == NULL); mp->ma_fill++; ep->me_key = key; ep->me_hash = (Py_ssize_t)hash; @@ -524,6 +529,16 @@ return 0; } +/* Note that, for historical reasons, PyDict_GetItem() suppresses all errors + * that may occur (originally dicts supported only string keys, and exceptions + * weren't possible). So, while the original intent was that a NULL return + * meant the key wasn't present, it reality it can mean that, or that an error + * (suppressed) occurred while computing the key's hash, or that some error + * (suppressed) occurred when comparing keys in the dict's internal probe + * sequence. A nasty example of the latter is when a Python-coded comparison + * function hits a stack-depth error, which can cause this to return NULL + * even if the key is present. + */ PyObject * PyDict_GetItem(PyObject *op, PyObject *key) { @@ -531,9 +546,8 @@ dictobject *mp = (dictobject *)op; dictentry *ep; PyThreadState *tstate; - if (!PyDict_Check(op)) { + if (!PyDict_Check(op)) return NULL; - } if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { @@ -1607,8 +1621,8 @@ dict_has_key(register dictobject *mp, PyObject *key) { long hash; - register long ok; dictentry *ep; + if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { hash = PyObject_Hash(key); @@ -1618,8 +1632,7 @@ ep = (mp->ma_lookup)(mp, key, hash); if (ep == NULL) return NULL; - ok = ep->me_value != NULL; - return PyBool_FromLong(ok); + return PyBool_FromLong(ep->me_value != NULL); } static PyObject * @@ -1932,6 +1945,7 @@ {NULL, NULL} /* sentinel */ }; +/* Return 1 if `key` is in dict `op`, 0 if not, and -1 on error. */ int PyDict_Contains(PyObject *op, PyObject *key) { @@ -1946,9 +1960,7 @@ return -1; } ep = (mp->ma_lookup)(mp, key, hash); - if (ep == NULL) - return -1; - return ep->me_value != NULL; + return ep == NULL ? -1 : (ep->me_value != NULL); } /* Hack to implement "key in dict" */ From python-checkins at python.org Thu Jun 1 18:08:57 2006 From: python-checkins at python.org (david.goodger) Date: Thu, 1 Jun 2006 18:08:57 +0200 (CEST) Subject: [Python-checkins] r46595 - peps/trunk/pep-3100.txt Message-ID: <20060601160857.50FAF1E400A@bag.python.org> Author: david.goodger Date: Thu Jun 1 18:08:56 2006 New Revision: 46595 Modified: peps/trunk/pep-3100.txt Log: markup fix Modified: peps/trunk/pep-3100.txt ============================================================================== --- peps/trunk/pep-3100.txt (original) +++ peps/trunk/pep-3100.txt Thu Jun 1 18:08:56 2006 @@ -318,10 +318,10 @@ .. [27] python-3000 email ("sets in P3K?") http://mail.python.org/pipermail/python-3000/2006-April/001286.html - [28] python-3000 email ("sets in P3K?") +.. [28] python-3000 email ("sets in P3K?") http://mail.python.org/pipermail/python-3000/2006-May/001666.html - [29] python-3000 email ("bug in modulus?") +.. [29] python-3000 email ("bug in modulus?") http://mail.python.org/pipermail/python-3000/2006-May/001735.html .. [#pep238] PEP 238 (Changing the Division Operator) From buildbot at python.org Thu Jun 1 18:16:13 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 01 Jun 2006 16:16:13 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060601161614.138711E400A@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/261 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,georg.brandl,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 1 18:17:52 2006 From: python-checkins at python.org (david.goodger) Date: Thu, 1 Jun 2006 18:17:52 +0200 (CEST) Subject: [Python-checkins] r46596 - peps/trunk/pep-0360.txt Message-ID: <20060601161752.99D301E400B@bag.python.org> Author: david.goodger Date: Thu Jun 1 18:17:51 2006 New Revision: 46596 Modified: peps/trunk/pep-0360.txt Log: minor edits; use more appropriate markup for data records Modified: peps/trunk/pep-0360.txt ============================================================================== --- peps/trunk/pep-0360.txt (original) +++ peps/trunk/pep-0360.txt Thu Jun 1 18:17:51 2006 @@ -13,9 +13,9 @@ ======== There are many great pieces of Python software developed outside of -the Python standard library (aka, stdlib). Sometimes it makes sense -to incorporate these externally maintained packages into the stdlib in -order to fill a gap in the tools provided by Python. +the Python standard library (a.k.a., the "stdlib"). Sometimes it +makes sense to incorporate these externally maintained packages into +the stdlib in order to fill a gap in the tools provided by Python. But by having the packages maintained externally it means Python's developers do not have direct control over the packages' evolution and @@ -33,24 +33,25 @@ Externally Maintained Packages ============================== -The section title is the name of the package as known outside of the -Python standard library. The "standard library name" is what the +The section title is the name of the package as it is known outside of +the Python standard library. The "standard library name" is what the package is named within Python. The "contact person" is the Python -developer in charge of maintaining the package. The -"synchronisation history" lists what external version of the package -was included in each version of Python (if different from the previous -Python release). +developer in charge of maintaining the package. The "synchronisation +history" lists what external version of the package was included in +each version of Python (if different from the previous Python +release). ctypes ------ -- Web page + +:Web site: http://starship.python.net/crew/theller/ctypes/ -- Standard library name +:Standard library name: ctypes -- Contact person +:Contact person: Thomas Heller -- Synchronisation history +:Synchronisation history: * 0.9.9.6 (2.5) Bugs can be reported to either the Python tracker [#python-tracker]_ @@ -60,13 +61,14 @@ ElementTree ----------- -- Web page + +:Web site: http://effbot.org/zone/element-index.htm -- Standard library name +:Standard library name: xml.etree -- Contact person +:Contact person: Fredrik Lundh -- Synchronisation history +:Synchronisation history: * 1.2.6 [ElementTree] / 1.0.5 [cElementTree] (2.5) Patches should not be directly applied to Python HEAD, but instead @@ -77,27 +79,29 @@ Expat XML parser ---------------- -- Web page + +:Web site: http://www.libexpat.org/ -- Standard library name +:Standard library name: N/A (this refers to the parser itself, and not the Python bindings) -- Contact person +:Contact person: None -- Synchronisation history +:Synchronisation history: * 1.95.8 (2.4) * 1.95.7 (2.3) Optik ----- -- Web site + +:Web site: http://optik.sourceforge.net/ -- Standard library name +:Standard library name: optparse -- Contact person +:Contact person: Greg Ward -- Synchronisation history +:Synchronisation history: * 1.5.1 (2.5) * 1.5a1 (2.4) * 1.4 (2.3) @@ -105,13 +109,14 @@ pysqlite -------- -- Web site + +:Web site: http://www.sqlite.org/ -- Standard library name +:Standard library name: sqlite3 -- Contact person +:Contact person: Gerhard H?ring -- Synchronisation history +:Synchronisation history: * 2.2.2 (2.5) Bugs should be reported to the pysqlite bug @@ -137,7 +142,6 @@ This document has been placed in the public domain. - .. Local Variables: From python-checkins at python.org Thu Jun 1 20:50:00 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 1 Jun 2006 20:50:00 +0200 (CEST) Subject: [Python-checkins] r46597 - peps/trunk/pep-3099.txt Message-ID: <20060601185000.15E5D1E400B@bag.python.org> Author: georg.brandl Date: Thu Jun 1 20:49:59 2006 New Revision: 46597 Modified: peps/trunk/pep-3099.txt Log: Fold PEP 13 draft into PEP 3099. Modified: peps/trunk/pep-3099.txt ============================================================================== --- peps/trunk/pep-3099.txt (original) +++ peps/trunk/pep-3099.txt Thu Jun 1 20:49:59 2006 @@ -12,17 +12,47 @@ Abstract ======== -This PEP tries to list all BDFL pronouncements on Python 3000 that -refer to changes that will not happen and new features that will not -be introduced, sorted by topics, along with a short explanation or a -reference to the relevant thread on the python-3000 mailing list. +Some ideas are just bad. While some thoughts on Python evolution are +constructive, some go against the basic tenets of Python so +egregiously that it would be like asking someone to run in a circle: +it gets you nowhere, even for Python 3000, where extraordinary +proposals are allowed. This PEP tries to list all BDFL pronouncements +on Python 3000 that refer to changes that will not happen and new +features that will not be introduced, sorted by topics, along with +a short explanation or a reference to the relevant thread on the +python-3000 mailing list. + +If you think you should suggest any of the listed ideas it would be +better to just step away from the computer, go outside, and enjoy +yourself. Being active outdoors by napping in a nice patch of grass +is more productive than bringing up a beating-a-dead-horse idea and +having people tell you how dead the idea is. Consider yourself warned. Core language ============= +* ``self`` will not become implicit. + + Having ``self`` be explicit is a *good thing*. It makes the code + clear by removing ambiguity about how a variable resolves. It also + makes the difference between functions and methods small. + + Thread: "Draft proposal: Implicit self in Python 3.0" + http://mail.python.org/pipermail/python-dev/2006-January/059468.html + * ``lambda`` will not be renamed. + At one point lambda was slated for removal in Python 3000. + Unfortunately no one was able to come up with a better way of + providing anonymous functions. And so lambda is here to stay. + + But it is here to stay as-is. Adding support for statements is a + non-starter. It would require allowing multi-line lambda + expressions which would mean a multi-line expression could suddenly + exist. That would allow for multi-line arguments to function + calls, for instance. That is just plain ugly. + Thread: "genexp syntax / lambda", http://mail.python.org/pipermail/python-3000/2006-April/001042.html @@ -66,6 +96,20 @@ Thread: elimination of scope bleeding of iteration variables http://mail.python.org/pipermail/python-dev/2006-May/064761.html +* The parser won't be more complex than LL(1). + + Simple is better than complex. This idea extends to the parser. + Restricting Python's grammar to an LL(1) parser is a blessing, + not a curse. It puts us in handcuffs that prevent us from going + overboard and ending up with funky grammar rules like some other + dynamic languages that will go unnamed, like Perl. + +* No braces. + + This is so obvious that it doesn't need a reference to a mailing + list. Do ``from __future__ import braces`` to get a definitive + answer on this subject. + Builtins ======== From python-checkins at python.org Thu Jun 1 22:04:17 2006 From: python-checkins at python.org (phillip.eby) Date: Thu, 1 Jun 2006 22:04:17 +0200 (CEST) Subject: [Python-checkins] r46598 - sandbox/branches/setuptools-0.6/ez_setup.py Message-ID: <20060601200417.0A0A21E4020@bag.python.org> Author: phillip.eby Date: Thu Jun 1 22:04:16 2006 New Revision: 46598 Modified: sandbox/branches/setuptools-0.6/ez_setup.py Log: 0.6b2 MD5's Modified: sandbox/branches/setuptools-0.6/ez_setup.py ============================================================================== --- sandbox/branches/setuptools-0.6/ez_setup.py (original) +++ sandbox/branches/setuptools-0.6/ez_setup.py Thu Jun 1 22:04:16 2006 @@ -24,6 +24,8 @@ 'setuptools-0.6a11-py2.4.egg': 'a95d5bc7a070aa1028bc4dcb5270b133', 'setuptools-0.6b1-py2.3.egg': '8822caf901250d848b996b7f25c6e6ca', 'setuptools-0.6b1-py2.4.egg': 'b79a8a403e4502fbb85ee3f1941735cb', + 'setuptools-0.6b2-py2.3.egg': '5657759d8a6d8fc44070a9d07272d99b', + 'setuptools-0.6b2-py2.4.egg': '4996a8d169d2be661fa32a6e52e4f82a', } import sys, os From tim.peters at gmail.com Fri Jun 2 03:31:48 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 1 Jun 2006 21:31:48 -0400 Subject: [Python-checkins] r46505 - python/trunk/Tools/pybench/systimes.py In-Reply-To: <20060528174658.748AD1E4006@bag.python.org> References: <20060528174658.748AD1E4006@bag.python.org> Message-ID: <1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com> > Author: marc-andre.lemburg > Date: Sun May 28 19:46:58 2006 > New Revision: 46505 > > Added: > python/trunk/Tools/pybench/systimes.py (contents, props changed) ... > +def win32process_getprocesstimes_systimes(): > + d = win32process.GetProcessTimes(win32process.GetCurrentProcess()) > + # Note: I'm not sure whether KernelTime on Windows is the same as > + # system time on Unix - I've yet to see a non-zero value for > + # KernelTime on Windows. > + return (d['UserTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, > + d['KernelTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) FYI, I always see a non-zero, and increasing, value for KernelTime on my box (WinXP Pro SP2). Alas, these counters appear to have even worse resolution than time.time() on my box, incrementing a seemingly variable(!) number of times per second, but well under 100Hz: """ import win32process def scream(): from time import clock as now p = win32process.GetCurrentProcess() d = {} count = 0 start = now() deadline = start + 1.0 while now() < deadline: count += 1 x = win32process.GetProcessTimes(p) d[x['UserTime']] = 1 elapsed = now() - start print "saw", len(d), "distinct values in", \ count, "tries across", elapsed, "seconds" print "user", x['UserTime'], "kernel", x['KernelTime'] print scream() scream() scream() """ One run of that here: saw 74 distinct values in 134542 tries across 1.00000616787 seconds user 11562500 kernel 2187500 saw 81 distinct values in 133365 tries across 1.00000220027 seconds user 24062500 kernel 2968750 saw 68 distinct values in 132984 tries across 1.0000078902 seconds user 34531250 kernel 4687500 From nnorwitz at gmail.com Fri Jun 2 06:04:23 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 1 Jun 2006 21:04:23 -0700 Subject: [Python-checkins] r46472 - python/trunk/PCbuild8 python/trunk/PCbuild8/Uninstal.wse python/trunk/PCbuild8/_bsddb.vcproj python/trunk/PCbuild8/_ctypes.vcproj python/trunk/PCbuild8/_ctypes_test.vcproj python/trunk/PCbuild8/_elementtree.vcproj pytho In-Reply-To: References: <129CEF95A523704B9D46959C922A28000282A8ED@nemesis.central.ccp.cc> <1149108731.447e01fb8520c@www.domainfactory-webmail.de> Message-ID: On 6/1/06, Brett Cannon wrote: > > =) Well, the compatibility and what modules are maintained externally are > both PEPs at this point (the externally maintained PEP is 360 and has not > been pushed to the site yet). So at least for those you just need to > remember to read something. =) I know there's a smiley there, but that's still part of my point. For newcomers it's very difficult to start because the learning curve is so large and keeps increasing. There are all these indirections. The more we can simplify the process and make it easier to contribute the more likely it will be for people to contribute. It's the same thing with making the bug/patch process easier and updating the web/doc. The easier it is to contribute, the more likely we are to get contributions. > As for expected Buildbot failures, it might be nice to have something > somewhere written down about that since I know I don't remember those > details either. There is: http://wiki.python.org/moin/BuildBot Except other than Tim making a section for Windows at one point, I'm not sure anyone else has ever modified or viewed the page. Ok, I just looked, Anthony touched it a few times too. :-) n From python-checkins at python.org Fri Jun 2 06:45:57 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 2 Jun 2006 06:45:57 +0200 (CEST) Subject: [Python-checkins] r46599 - python/trunk/Lib/test/test_exceptions.py Message-ID: <20060602044557.A29391E400B@bag.python.org> Author: neal.norwitz Date: Fri Jun 2 06:45:53 2006 New Revision: 46599 Modified: python/trunk/Lib/test/test_exceptions.py Log: Convert docstrings to comments so regrtest -v prints method names Modified: python/trunk/Lib/test/test_exceptions.py ============================================================================== --- python/trunk/Lib/test/test_exceptions.py (original) +++ python/trunk/Lib/test/test_exceptions.py Fri Jun 2 06:45:53 2006 @@ -112,8 +112,8 @@ except Exception, e: pass def testSyntaxErrorMessage(self): - """make sure the right exception message is raised for each of - these code fragments""" + # make sure the right exception message is raised for each of + # these code fragments def ckmsg(src, msg): try: @@ -143,8 +143,8 @@ ckmsg("continue\n", "'continue' not properly in loop") def testSettingException(self): - """test that setting an exception at the C level works even if the - exception object can't be constructed.""" + # test that setting an exception at the C level works even if the + # exception object can't be constructed. class BadException: def __init__(self_): @@ -181,7 +181,7 @@ test_capi2() def testAttributes(self): - """test that exception attributes are happy.""" + # test that exception attributes are happy try: str(u'Hello \u00E1') except Exception, e: sampleUnicodeEncodeError = e @@ -280,8 +280,8 @@ (repr(e), checkArgName)) def testKeywordArgs(self): - """test that builtin exception don't take keyword args, - but user-defined subclasses can if they want""" + # test that builtin exception don't take keyword args, + # but user-defined subclasses can if they want self.assertRaises(TypeError, BaseException, a=1) class DerivedException(BaseException): From python-checkins at python.org Fri Jun 2 06:50:55 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 2 Jun 2006 06:50:55 +0200 (CEST) Subject: [Python-checkins] r46600 - in python/trunk: Lib/test/test_exceptions.py Objects/exceptions.c Message-ID: <20060602045055.6B7071E400B@bag.python.org> Author: neal.norwitz Date: Fri Jun 2 06:50:49 2006 New Revision: 46600 Modified: python/trunk/Lib/test/test_exceptions.py python/trunk/Objects/exceptions.c Log: Fix memory leak found by valgrind. Modified: python/trunk/Lib/test/test_exceptions.py ============================================================================== --- python/trunk/Lib/test/test_exceptions.py (original) +++ python/trunk/Lib/test/test_exceptions.py Fri Jun 2 06:50:49 2006 @@ -265,7 +265,9 @@ if (e is not exc and # needed for sampleUnicode errors type(e) is not exc): raise - for checkArgName in expected.keys(): + # Verify no ref leaks in Exc_str() + s = str(e) + for checkArgName in expected: self.assertEquals(repr(getattr(e, checkArgName)), repr(expected[checkArgName]), 'exception "%s", attribute "%s"' % @@ -273,7 +275,7 @@ # test for pickling support new = pickle.loads(pickle.dumps(e, random.randint(0, 2))) - for checkArgName in expected.keys(): + for checkArgName in expected: self.assertEquals(repr(getattr(e, checkArgName)), repr(expected[checkArgName]), 'pickled exception "%s", attribute "%s' % Modified: python/trunk/Objects/exceptions.c ============================================================================== --- python/trunk/Objects/exceptions.c (original) +++ python/trunk/Objects/exceptions.c Fri Jun 2 06:50:49 2006 @@ -619,7 +619,6 @@ PyTuple_SET_ITEM(tuple, 1, Py_None); } - Py_INCREF(repr); PyTuple_SET_ITEM(tuple, 2, repr); rtnval = PyString_Format(fmt, tuple); From python-checkins at python.org Fri Jun 2 06:54:59 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 2 Jun 2006 06:54:59 +0200 (CEST) Subject: [Python-checkins] r46601 - python/trunk/Modules/_sqlite/module.c Message-ID: <20060602045459.3CE081E400B@bag.python.org> Author: neal.norwitz Date: Fri Jun 2 06:54:52 2006 New Revision: 46601 Modified: python/trunk/Modules/_sqlite/module.c Log: More memory leaks from valgrind Modified: python/trunk/Modules/_sqlite/module.c ============================================================================== --- python/trunk/Modules/_sqlite/module.c (original) +++ python/trunk/Modules/_sqlite/module.c Fri Jun 2 06:54:52 2006 @@ -280,21 +280,25 @@ goto error; } PyDict_SetItemString(dict, "PARSE_DECLTYPES", tmp_obj); + Py_DECREF(tmp_obj); if (!(tmp_obj = PyInt_FromLong(PARSE_COLNAMES))) { goto error; } PyDict_SetItemString(dict, "PARSE_COLNAMES", tmp_obj); + Py_DECREF(tmp_obj); if (!(tmp_obj = PyString_FromString(PYSQLITE_VERSION))) { goto error; } PyDict_SetItemString(dict, "version", tmp_obj); + Py_DECREF(tmp_obj); if (!(tmp_obj = PyString_FromString(sqlite3_libversion()))) { goto error; } PyDict_SetItemString(dict, "sqlite_version", tmp_obj); + Py_DECREF(tmp_obj); /* initialize microprotocols layer */ microprotocols_init(dict); From buildbot at python.org Fri Jun 2 07:44:46 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 02 Jun 2006 05:44:46 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060602054446.9FA151E400B@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/552 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 2 08:23:01 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 2 Jun 2006 08:23:01 +0200 (CEST) Subject: [Python-checkins] r46602 - python/trunk/Parser/tokenizer.c Message-ID: <20060602062301.0FF9A1E4004@bag.python.org> Author: neal.norwitz Date: Fri Jun 2 08:23:00 2006 New Revision: 46602 Modified: python/trunk/Parser/tokenizer.c Log: Patch #1357836: Prevent an invalid memory read from test_coding in case the done flag is set. In that case, the loop isn't entered. I wonder if rather than setting the done flag in the cases before the loop, if they should just exit early. This code looks like it should be refactored. Backport candidate (also the early break above if decoding_fgets fails) Modified: python/trunk/Parser/tokenizer.c ============================================================================== --- python/trunk/Parser/tokenizer.c (original) +++ python/trunk/Parser/tokenizer.c Fri Jun 2 08:23:00 2006 @@ -893,15 +893,17 @@ tok->inp = strchr(tok->inp, '\0'); done = tok->inp[-1] == '\n'; } - tok->cur = tok->buf + cur; - tok->line_start = tok->cur; - /* replace "\r\n" with "\n" */ - /* For Mac we leave the \r, giving a syntax error */ - pt = tok->inp - 2; - if (pt >= tok->buf && *pt == '\r') { - *pt++ = '\n'; - *pt = '\0'; - tok->inp = pt; + if (tok->buf != NULL) { + tok->cur = tok->buf + cur; + tok->line_start = tok->cur; + /* replace "\r\n" with "\n" */ + /* For Mac leave the \r, giving syntax error */ + pt = tok->inp - 2; + if (pt >= tok->buf && *pt == '\r') { + *pt++ = '\n'; + *pt = '\0'; + tok->inp = pt; + } } } if (tok->done != E_OK) { From buildbot at python.org Fri Jun 2 08:28:03 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 02 Jun 2006 06:28:03 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060602062804.0542D1E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/263 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From mal at egenix.com Fri Jun 2 11:01:26 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 02 Jun 2006 11:01:26 +0200 Subject: [Python-checkins] r46505 - python/trunk/Tools/pybench/systimes.py In-Reply-To: <1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com> References: <20060528174658.748AD1E4006@bag.python.org> <1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com> Message-ID: <447FFE66.4020202@egenix.com> Tim Peters wrote: >> Author: marc-andre.lemburg >> Date: Sun May 28 19:46:58 2006 >> New Revision: 46505 >> >> Added: >> python/trunk/Tools/pybench/systimes.py (contents, props changed) > > ... > >> +def win32process_getprocesstimes_systimes(): >> + d = win32process.GetProcessTimes(win32process.GetCurrentProcess()) >> + # Note: I'm not sure whether KernelTime on Windows is the same as >> + # system time on Unix - I've yet to see a non-zero value for >> + # KernelTime on Windows. >> + return (d['UserTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, >> + d['KernelTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) > > FYI, I always see a non-zero, and increasing, value for KernelTime on > my box (WinXP Pro SP2). Using your code I now see non-zero values as well - perhaps KernelTime refers to calls being made to the win32 APIs ?! I tested using standard Python code, such as long for loops which don't call out to win32 APIs. > Alas, these counters appear to have even worse resolution than > time.time() on my box, incrementing a seemingly variable(!) number of > times per second, but well under 100Hz: > > """ > import win32process > > def scream(): > from time import clock as now > p = win32process.GetCurrentProcess() > d = {} > count = 0 > start = now() > deadline = start + 1.0 > while now() < deadline: > count += 1 > x = win32process.GetProcessTimes(p) > d[x['UserTime']] = 1 > elapsed = now() - start > print "saw", len(d), "distinct values in", \ > count, "tries across", elapsed, "seconds" > print "user", x['UserTime'], "kernel", x['KernelTime'] > print > > scream() > scream() > scream() > """ > > One run of that here: > > saw 74 distinct values in 134542 tries across 1.00000616787 seconds > user 11562500 kernel 2187500 > > saw 81 distinct values in 133365 tries across 1.00000220027 seconds > user 24062500 kernel 2968750 > > saw 68 distinct values in 132984 tries across 1.0000078902 seconds > user 34531250 kernel 4687500 Question is whether this is really bad resolution or just the effect of the OS not fetching the timer value in real-time, but using some variable that is only updated at 50-100 Hz. It is interesting to see how WinXP assigns slots to the processes (the elapsed ticks values are number of ticks seen in 1 second). saw 44 distinct values in 145439 tries across 1.00002849524 seconds user 7500000 kernel 4062500 elapsed kernel ticks: 3281250 elapsed user ticks: 6718750 saw 48 distinct values in 146123 tries across 1.00000726349 seconds user 15156250 kernel 6562500 elapsed kernel ticks: 2500000 elapsed user ticks: 7343750 saw 45 distinct values in 146262 tries across 1.00000279365 seconds user 22187500 kernel 9531250 elapsed kernel ticks: 2968750 elapsed user ticks: 6875000 Here's the modified code: """ import win32process def scream(): from time import clock as now p = win32process.GetCurrentProcess() d = {} count = 0 start = now() deadline = start + 1.0 while now() < deadline: count += 1 x = win32process.GetProcessTimes(p) d[x['UserTime']] = 1 elapsed = now() - start print "saw", len(d), "distinct values in", \ count, "tries across", elapsed, "seconds" print "user", x['UserTime'], "kernel", x['KernelTime'] print scream() scream() scream() """ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 02 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2006-07-03: EuroPython 2006, CERN, Switzerland 30 days left ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fredrik at pythonware.com Fri Jun 2 14:12:32 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 2 Jun 2006 14:12:32 +0200 Subject: [Python-checkins] r46505 -python/trunk/Tools/pybench/systimes.py References: <20060528174658.748AD1E4006@bag.python.org><1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com> <447FFE66.4020202@egenix.com> Message-ID: M.-A. Lemburg wrote: >> FYI, I always see a non-zero, and increasing, value for KernelTime on >> my box (WinXP Pro SP2). > > Using your code I now see non-zero values as well - perhaps > KernelTime refers to calls being made to the win32 APIs ?! huh? what else would it refer to ? (what do you think "system time" refers to on a Unix machine?) >> One run of that here: >> >> saw 74 distinct values in 134542 tries across 1.00000616787 seconds >> user 11562500 kernel 2187500 >> >> saw 81 distinct values in 133365 tries across 1.00000220027 seconds >> user 24062500 kernel 2968750 >> >> saw 68 distinct values in 132984 tries across 1.0000078902 seconds >> user 34531250 kernel 4687500 > > Question is whether this is really bad resolution or just the > effect of the OS not fetching the timer value in real-time, > but using some variable that is only updated at 50-100 Hz. but, but, but... didn't I just point out that process time is based on *sampling* of the program counter by the system's scheduler, not on measurements using some high-performance timer. the standard tick interval on a Windows machine is 10-15 ms, depending on platform. (to check the tick on your machine, you can write a simple "time.time() - time.time() until you get back a non-zero value" will reveal, or use http://www.sysinternals.com/Utilities/ClockRes.html ). there is no process timer. this is getting too weird for me. I need some coffee and a break. From fredrik at pythonware.com Fri Jun 2 14:27:41 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 2 Jun 2006 14:27:41 +0200 Subject: [Python-checkins] r46505 -python/trunk/Tools/pybench/systimes.py References: <20060528174658.748AD1E4006@bag.python.org><1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com><447FFE66.4020202@egenix.com> Message-ID: > ms, depending on platform. (to check the tick on your machine, you can write a > simple "time.time() - time.time() until you get back a non-zero value" [-loop], or > use /.../ From python-checkins at python.org Fri Jun 2 15:03:46 2006 From: python-checkins at python.org (martin.blais) Date: Fri, 2 Jun 2006 15:03:46 +0200 (CEST) Subject: [Python-checkins] r46603 - python/trunk/Lib/test/test_struct.py Message-ID: <20060602130346.9071C1E400B@bag.python.org> Author: martin.blais Date: Fri Jun 2 15:03:43 2006 New Revision: 46603 Modified: python/trunk/Lib/test/test_struct.py Log: Fixed struct test to not use unittest. Modified: python/trunk/Lib/test/test_struct.py ============================================================================== --- python/trunk/Lib/test/test_struct.py (original) +++ python/trunk/Lib/test/test_struct.py Fri Jun 2 15:03:43 2006 @@ -2,7 +2,6 @@ import test.test_support import struct import array -import unittest import warnings import sys @@ -494,80 +493,87 @@ if PY_STRUCT_RANGE_CHECKING: test_1229380() -class PackBufferTestCase(unittest.TestCase): - """ - Test the packing methods that work on buffers. - """ - - def test_unpack_from( self ): - test_string = 'abcd01234' - fmt = '4s' - s = struct.Struct(fmt) - for cls in (str, buffer): - data = cls(test_string) - self.assertEquals(s.unpack_from(data), ('abcd',)) - self.assertEquals(s.unpack_from(data, 2), ('cd01',)) - self.assertEquals(s.unpack_from(data, 4), ('0123',)) - for i in xrange(6): - self.assertEquals(s.unpack_from(data, i), (data[i:i+4],)) - for i in xrange(6, len(test_string) + 1): - simple_err(s.unpack_from, data, i) - for cls in (str, buffer): - data = cls(test_string) - self.assertEquals(struct.unpack_from(fmt, data), ('abcd',)) - self.assertEquals(struct.unpack_from(fmt, data, 2), ('cd01',)) - self.assertEquals(struct.unpack_from(fmt, data, 4), ('0123',)) - for i in xrange(6): - self.assertEquals(struct.unpack_from(fmt, data, i), - (data[i:i+4],)) - for i in xrange(6, len(test_string) + 1): - simple_err(struct.unpack_from, fmt, data, i) - - def test_pack_to( self ): - test_string = 'Reykjavik rocks, eow!' - writable_buf = array.array('c', ' '*100) - fmt = '21s' - s = struct.Struct(fmt) - - # Test without offset - s.pack_to(writable_buf, 0, test_string) - from_buf = writable_buf.tostring()[:len(test_string)] - self.assertEquals(from_buf, test_string) - - # Test with offset. - s.pack_to(writable_buf, 10, test_string) - from_buf = writable_buf.tostring()[:len(test_string)+10] - self.assertEquals(from_buf, (test_string[:10] + test_string)) - - # Go beyond boundaries. - small_buf = array.array('c', ' '*10) - self.assertRaises(struct.error, s.pack_to, small_buf, 0, test_string) - self.assertRaises(struct.error, s.pack_to, small_buf, 2, test_string) - - def test_pack_to_fn( self ): - test_string = 'Reykjavik rocks, eow!' - writable_buf = array.array('c', ' '*100) - fmt = '21s' - pack_to = lambda *args: struct.pack_to(fmt, *args) - - # Test without offset - pack_to(writable_buf, 0, test_string) - from_buf = writable_buf.tostring()[:len(test_string)] - self.assertEquals(from_buf, test_string) - - # Test with offset. - pack_to(writable_buf, 10, test_string) - from_buf = writable_buf.tostring()[:len(test_string)+10] - self.assertEquals(from_buf, (test_string[:10] + test_string)) - - # Go beyond boundaries. - small_buf = array.array('c', ' '*10) - self.assertRaises(struct.error, pack_to, small_buf, 0, test_string) - self.assertRaises(struct.error, pack_to, small_buf, 2, test_string) +########################################################################### +# Packing and unpacking to/from buffers. + +# Copied and modified from unittest. +def assertRaises(excClass, callableObj, *args, **kwargs): + try: + callableObj(*args, **kwargs) + except excClass: + return + else: + raise RuntimeError("%s not raised." % excClass) -def test_main(): - test.test_support.run_unittest(PackBufferTestCase) +def test_unpack_from(): + test_string = 'abcd01234' + fmt = '4s' + s = struct.Struct(fmt) + for cls in (str, buffer): + data = cls(test_string) + assert s.unpack_from(data) == ('abcd',) + assert s.unpack_from(data, 2) == ('cd01',) + assert s.unpack_from(data, 4) == ('0123',) + for i in xrange(6): + assert s.unpack_from(data, i) == (data[i:i+4],) + for i in xrange(6, len(test_string) + 1): + simple_err(s.unpack_from, data, i) + for cls in (str, buffer): + data = cls(test_string) + assert struct.unpack_from(fmt, data) == ('abcd',) + assert struct.unpack_from(fmt, data, 2) == ('cd01',) + assert struct.unpack_from(fmt, data, 4) == ('0123',) + for i in xrange(6): + assert (struct.unpack_from(fmt, data, i) == (data[i:i+4],)) + for i in xrange(6, len(test_string) + 1): + simple_err(struct.unpack_from, fmt, data, i) + +def test_pack_to(): + test_string = 'Reykjavik rocks, eow!' + writable_buf = array.array('c', ' '*100) + fmt = '21s' + s = struct.Struct(fmt) + + # Test without offset + s.pack_to(writable_buf, 0, test_string) + from_buf = writable_buf.tostring()[:len(test_string)] + assert from_buf == test_string + + # Test with offset. + s.pack_to(writable_buf, 10, test_string) + from_buf = writable_buf.tostring()[:len(test_string)+10] + assert from_buf == (test_string[:10] + test_string) + + # Go beyond boundaries. + small_buf = array.array('c', ' '*10) + assertRaises(struct.error, s.pack_to, small_buf, 0, test_string) + assertRaises(struct.error, s.pack_to, small_buf, 2, test_string) + +def test_pack_to_fn(): + test_string = 'Reykjavik rocks, eow!' + writable_buf = array.array('c', ' '*100) + fmt = '21s' + pack_to = lambda *args: struct.pack_to(fmt, *args) + + # Test without offset + pack_to(writable_buf, 0, test_string) + from_buf = writable_buf.tostring()[:len(test_string)] + assert from_buf == test_string + + # Test with offset. + pack_to(writable_buf, 10, test_string) + from_buf = writable_buf.tostring()[:len(test_string)+10] + assert from_buf == (test_string[:10] + test_string) + + # Go beyond boundaries. + small_buf = array.array('c', ' '*10) + assertRaises(struct.error, pack_to, small_buf, 0, test_string) + assertRaises(struct.error, pack_to, small_buf, 2, test_string) + + +# Test methods to pack and unpack from buffers rather than strings. +test_unpack_from() +test_pack_to() +test_pack_to_fn() -if __name__ == "__main__": - test_main() From neal at metaslash.com Fri Jun 2 11:16:39 2006 From: neal at metaslash.com (Neal Norwitz) Date: Fri, 2 Jun 2006 05:16:39 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060602091639.GA3892@python.psfb.org> test_exceptions leaked [0, 0, 2] references test_threading_local leaked [0, -91, 0] references From mal at egenix.com Fri Jun 2 16:14:03 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 02 Jun 2006 16:14:03 +0200 Subject: [Python-checkins] r46505 -python/trunk/Tools/pybench/systimes.py In-Reply-To: References: <20060528174658.748AD1E4006@bag.python.org><1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com> <447FFE66.4020202@egenix.com> Message-ID: <448047AB.50507@egenix.com> Fredrik Lundh wrote: > M.-A. Lemburg wrote: > >>> FYI, I always see a non-zero, and increasing, value for KernelTime on >>> my box (WinXP Pro SP2). >> Using your code I now see non-zero values as well - perhaps >> KernelTime refers to calls being made to the win32 APIs ?! > > huh? what else would it refer to ? Not sure, that's why I'm asking. The MS docs say: "A pointer to a FILETIME structure that receives the amount of time that the process has executed in kernel mode.". http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/getprocesstimes.asp > (what do you think "system time" refers > to on a Unix machine?) man 3 times: "The tms_stime field contains the CPU time spent in the system while executing tasks on behalf of the calling process." tms_stime refers to "system time". Both definitions are, well, fuzzy :-) E.g. it is not clear whether time.sleep(10) or time spent waiting for I/O should be considered system time, even though the system is spending time on behalf of the process, albeit usually giving it to other processes. I just wonder whether they both refer to the same fuzzy definition. >>> One run of that here: >>> >>> saw 74 distinct values in 134542 tries across 1.00000616787 seconds >>> user 11562500 kernel 2187500 >>> >>> saw 81 distinct values in 133365 tries across 1.00000220027 seconds >>> user 24062500 kernel 2968750 >>> >>> saw 68 distinct values in 132984 tries across 1.0000078902 seconds >>> user 34531250 kernel 4687500 >> Question is whether this is really bad resolution or just the >> effect of the OS not fetching the timer value in real-time, >> but using some variable that is only updated at 50-100 Hz. > > but, but, but... didn't I just point out that process time is based on *sampling* of > the program counter by the system's scheduler, not on measurements using some > high-performance timer. the standard tick interval on a Windows machine is 10-15 > ms, depending on platform. (to check the tick on your machine, you can write a > simple "time.time() - time.time() until you get back a non-zero value" will reveal, > or use http://www.sysinternals.com/Utilities/ClockRes.html ). there is no process > timer. Maybe there's just a misunderstanding of terms here: what I refer to as timer is really just some counter that gets incremented at fixed intervals. This could be an interrupt based counter such as the jiffies counter on Linux or a CPU based one such as the one used by the clock_gettime() real-time POSIX APIs. The question is whether the process time is accounted for in an accurate way or not, ie. which timer the system scheduler uses to update the process time counter and what it's minimum time slice is. If the scheduler just allows an accuracy of 10ms, then process time will only ever be as accurate as that. If you have code to run that uses less CPU time, a wall-clock timer (in hardware) will likely give you more accurate results, than counting clock ticks. OTOH, if you repeat the code execution for several seconds, then a process time based approach will give you more accurate results (unless you happen to run on a real-time OS with fixed and guaranteed scheduling time slices). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 02 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2006-07-03: EuroPython 2006, CERN, Switzerland 30 days left ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From buildbot at python.org Fri Jun 2 16:58:34 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 02 Jun 2006 14:58:34 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060602145834.8C0211E4018@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/265 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.blais Build Had Warnings: warnings test sincerely, -The Buildbot From jimjjewett at gmail.com Fri Jun 2 17:19:33 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 2 Jun 2006 11:19:33 -0400 Subject: [Python-checkins] r46599 - python/trunk/Lib/test/test_exceptions.py In-Reply-To: <20060602044557.A29391E400B@bag.python.org> References: <20060602044557.A29391E400B@bag.python.org> Message-ID: On 6/2/06, neal.norwitz wrote: > Author: neal.norwitz > Date: Fri Jun 2 06:45:53 2006 > New Revision: 46599 > > Modified: > python/trunk/Lib/test/test_exceptions.py > Log: > Convert docstrings to comments so regrtest -v prints method names Wouldn't it make more sense to modify regrtest -v to print method names (possibly in addition to the docstring)? -jJ From tim.peters at gmail.com Fri Jun 2 17:44:39 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 2 Jun 2006 11:44:39 -0400 Subject: [Python-checkins] r46599 - python/trunk/Lib/test/test_exceptions.py In-Reply-To: References: <20060602044557.A29391E400B@bag.python.org> Message-ID: <1f7befae0606020844o50838d1dmb4a54f98af4b5373@mail.gmail.com> >> Author: neal.norwitz >> Date: Fri Jun 2 06:45:53 2006 >> New Revision: 46599 >> >> Modified: >> python/trunk/Lib/test/test_exceptions.py >> Log: >> Convert docstrings to comments so regrtest -v prints method names [Jim Jewett] > Wouldn't it make more sense to modify regrtest -v to print method > names (possibly in addition to the docstring)? It has nothing to do with regrtest -- it's an (undocumented?) (mis)feature of unittest's TextTestRunner that it displays the docstring instead of a test method's name and test class whenever a test method has a non-empty docstring, and unittest (not regrtest) is what produces the output here. For example, try this minimal unittest: """ import unittest class Whatever(unittest.TestCase): def testequal(self): self.assertEqual(2, 2) if __name__ == '__main__': unittest.main() """ Then running "python THATFILE.py -v" displays """ testequal (__main__.Whatever) ... ok ---------------------------------------------------------------------- Ran 1 test in 0.000s OK """ Add a non-empty docstring to the method: """ def testequal(self): """I think I'm being helpful.""" self.assertEqual(2, 2) """ and it displays """ I think I'm being helpful. ... ok ---------------------------------------------------------------------- Ran 1 test in 0.000s OK """ instead. For that reason, most people avoid docstrings in unittest test methods like plague. regrtest has no control of its own over unittest's output. From tim.peters at gmail.com Fri Jun 2 18:23:33 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 2 Jun 2006 12:23:33 -0400 Subject: [Python-checkins] r46505 -python/trunk/Tools/pybench/systimes.py In-Reply-To: <448047AB.50507@egenix.com> References: <20060528174658.748AD1E4006@bag.python.org> <1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com> <447FFE66.4020202@egenix.com> <448047AB.50507@egenix.com> Message-ID: <1f7befae0606020923k2bab81fewf30417ff43296339@mail.gmail.com> [Marc-Andre] >>> Using your code I now see non-zero values as well - perhaps >>> KernelTime refers to calls being made to the win32 APIs ?! [Fredrik] >> huh? what else would it refer to ? [Marc-Andre] > Not sure, that's why I'm asking. The MS docs say: "A pointer to a > FILETIME structure that receives the amount of time that the process has > executed in kernel mode.". My understanding is that the docs are correct there, and "kernel mode" could just as well (perhaps better) be called "privileged mode". The OS kernel runs at a higher privilege level, which is partly enforced by modern CPU hardware. For example, code triggered by HW interrupts runs in privileged mode, and can execute instructions that the HW refuses to execute in "user mode", and access the entirety of physical RAM directly. Device drivers usually run in privileged/kernel mode too. This isn't the same as "Win32 API call", neither is it the same as anything else that can make instant sense to a non-OS hacker. More here: http://en.wikipedia.org/wiki/CPU_modes http://en.wikipedia.org/wiki/Architecture_of_the_Windows_NT_operating_system_line In any case, distinguishing between user time and system time is pretty much pointless in most benchmarks. I want to know how long the code takes, and couldn't care less how much of it runs with some CPU privilege bit set (unless, as mentioned before, I'm running on a timesharing mainframe and get charged different rates for user time than for system time). BTW, it may be the case that time waiting for a page fault to get resolved from disk doesn't get charged against "user time" or "kernel time" (depends on OS details). But if a code change is made that significantly increases or decreases page faults, that's certainly something a benchmarker wants to know about. A high-resolution wall-clock timer is typically the only way to approach measuring that kind of thing directly. Example: I ran this on WinXP just now: """ i = 0 d = {} while True: i += 1 d[i] = 1 if i % 1000000 == 0: print i """ It quickly brought my box to its knees, but _total_ system-wide "user time" and "kernel time" essentially stopped increasing. The paging time wasn't getting charged to anything. Likewise after hitting Ctrl-C, the time required to swap all of the dict back in from disk (in order to decref millions of int objects) didn't appear to be charged to anything either. The CPU was simply idle most of that time, waiting for disk traffic. From guido at python.org Fri Jun 2 18:23:55 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 2 Jun 2006 09:23:55 -0700 Subject: [Python-checkins] r46599 - python/trunk/Lib/test/test_exceptions.py In-Reply-To: <1f7befae0606020844o50838d1dmb4a54f98af4b5373@mail.gmail.com> References: <20060602044557.A29391E400B@bag.python.org> <1f7befae0606020844o50838d1dmb4a54f98af4b5373@mail.gmail.com> Message-ID: On 6/2/06, Tim Peters wrote: > instead. For that reason, most people avoid docstrings in unittest > test methods like plague. regrtest has no control of its own over > unittest's output. Perhaps it's time to change this unittest behavior, or at least provide a flag to disable it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Fri Jun 2 19:08:49 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 02 Jun 2006 19:08:49 +0200 Subject: [Python-checkins] r46505 -python/trunk/Tools/pybench/systimes.py In-Reply-To: <1f7befae0606020923k2bab81fewf30417ff43296339@mail.gmail.com> References: <20060528174658.748AD1E4006@bag.python.org> <1f7befae0606011831o330b610cybdc780d97c453ad@mail.gmail.com> <447FFE66.4020202@egenix.com> <448047AB.50507@egenix.com> <1f7befae0606020923k2bab81fewf30417ff43296339@mail.gmail.com> Message-ID: Tim Peters wrote: > My understanding is that the docs are correct there, and "kernel mode" > could just as well (perhaps better) be called "privileged mode". The > OS kernel runs at a higher privilege level, which is partly enforced > by modern CPU hardware. For example, code triggered by HW interrupts > runs in privileged mode, and can execute instructions that the HW > refuses to execute in "user mode", and access the entirety of physical > RAM directly. Device drivers usually run in privileged/kernel mode > too. This isn't the same as "Win32 API call", neither is it the same > as anything else that can make instant sense to a non-OS hacker. well, yes and no -- the time needed to handle HW interrupts, kernel processes, etc will of course be assigned to those processes, not your user process. when the scheduler interrupts your process to "handle a tick", it knows very well what process it was interrupting, and whether that process was running in privileged mode or not. (and for an ordinary user process running in the win32 subsystem, the best way to end up in kernel space is to call a win32 api function that (directly or indirectly) requires kernel-level support). > In any case, distinguishing between user time and system time is > pretty much pointless in most benchmarks. I want to know how long the > code takes, and couldn't care less how much of it runs with some CPU > privilege bit set (unless, as mentioned before, I'm running on a > timesharing mainframe and get charged different rates for user time > than for system time). if you're running on a mainframe, chances are that you have excellent hardware support for this ;-) > BTW, it may be the case that time waiting for a page fault to get > resolved from disk doesn't get charged against "user time" or "kernel > time" (depends on OS details). oh, I find it hard to believe that any modern operating system would consider waiting for I/O to finish to be a user process issue... > But if a code change is made that > significantly increases or decreases page faults, that's certainly > something a benchmarker wants to know about. A high-resolution > wall-clock timer is typically the only way to approach measuring that > kind of thing directly. absolutely. but I don't think we have that kind of tests in PyBench; they are all interpreter microbenchmarks, and virtually the only thing that can cause them to end up in kernel space on their own is memory allocations... From jimjjewett at gmail.com Fri Jun 2 19:38:55 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 2 Jun 2006 13:38:55 -0400 Subject: [Python-checkins] r46599 - python/trunk/Lib/test/test_exceptions.py In-Reply-To: References: <20060602044557.A29391E400B@bag.python.org> <1f7befae0606020844o50838d1dmb4a54f98af4b5373@mail.gmail.com> Message-ID: On 6/2/06, Guido van Rossum wrote: > On 6/2/06, Tim Peters wrote: > > instead. For that reason, most people avoid docstrings in unittest > > test methods like plague. regrtest has no control of its own over > > unittest's output. > Perhaps it's time to change this unittest behavior, or at least > provide a flag to disable it? Sounds good to me. Should it ignore the docstring entirely, or just append (the first line of) it to the method name? In other words, given def testStupidData(...): """Ensure that certain nonsense input combinations don't cause a crash. Resolves bug 843, crash on negative height""" Should the output be testStupidData or testStupidData - Ensure that certain nonsense input combinations don't cause a crash. Or should I let this be controlled by verbosity level? (verbosity 2 adding the string) -jJ From python-checkins at python.org Fri Jun 2 23:52:33 2006 From: python-checkins at python.org (steven.bethard) Date: Fri, 2 Jun 2006 23:52:33 +0200 (CEST) Subject: [Python-checkins] r46604 - peps/trunk/pep-3099.txt Message-ID: <20060602215233.9F2331E400D@bag.python.org> Author: steven.bethard Date: Fri Jun 2 23:52:33 2006 New Revision: 46604 Modified: peps/trunk/pep-3099.txt Log: Add thread for "Iterating over a dictionary..." that discusses dict() and dict.update() Modified: peps/trunk/pep-3099.txt ============================================================================== --- peps/trunk/pep-3099.txt (original) +++ peps/trunk/pep-3099.txt Fri Jun 2 23:52:33 2006 @@ -128,6 +128,9 @@ Thread: "Iterating over a dict", http://mail.python.org/pipermail/python-3000/2006-April/000283.html + + Thread: have iter(mapping) generate (key, value) pairs + http://mail.python.org/pipermail/python-3000/2006-June/002368.html * There will be no ``frozenlist`` type. From python-checkins at python.org Sat Jun 3 01:22:53 2006 From: python-checkins at python.org (tim.peters) Date: Sat, 3 Jun 2006 01:22:53 +0200 (CEST) Subject: [Python-checkins] r46605 - in python/trunk: Doc/lib/libpprint.tex Lib/pprint.py Lib/test/test_pprint.py Misc/NEWS Message-ID: <20060602232253.A90671E400D@bag.python.org> Author: tim.peters Date: Sat Jun 3 01:22:51 2006 New Revision: 46605 Modified: python/trunk/Doc/lib/libpprint.tex python/trunk/Lib/pprint.py python/trunk/Lib/test/test_pprint.py python/trunk/Misc/NEWS Log: pprint functions used to sort a dict (by key) if and only if the output required more than one line. "Small" dicts got displayed in seemingly random order (the hash-induced order produced by dict.__repr__). None of this was documented. Now pprint functions always sort dicts by key, and the docs promise it. This was proposed and agreed to during the PyCon 2006 core sprint -- I just didn't have time for it before now. Modified: python/trunk/Doc/lib/libpprint.tex ============================================================================== --- python/trunk/Doc/lib/libpprint.tex (original) +++ python/trunk/Doc/lib/libpprint.tex Sat Jun 3 01:22:51 2006 @@ -20,6 +20,10 @@ allowed width. Construct \class{PrettyPrinter} objects explicitly if you need to adjust the width constraint. +\versionchanged[Dictionaries are sorted by key before the display is +computed; before 2.5, a dictionary was sorted only if its display +required more than one line, although that wasn't documented]{2.5} + The \module{pprint} module defines one class: Modified: python/trunk/Lib/pprint.py ============================================================================== --- python/trunk/Lib/pprint.py (original) +++ python/trunk/Lib/pprint.py Sat Jun 3 01:22:51 2006 @@ -246,7 +246,7 @@ append = components.append level += 1 saferepr = _safe_repr - for k, v in object.iteritems(): + for k, v in sorted(object.items()): krepr, kreadable, krecur = saferepr(k, context, maxlevels, level) vrepr, vreadable, vrecur = saferepr(v, context, maxlevels, level) append("%s: %s" % (krepr, vrepr)) Modified: python/trunk/Lib/test/test_pprint.py ============================================================================== --- python/trunk/Lib/test/test_pprint.py (original) +++ python/trunk/Lib/test/test_pprint.py Sat Jun 3 01:22:51 2006 @@ -11,16 +11,21 @@ # list, tuple and dict subclasses that do or don't overwrite __repr__ class list2(list): pass + class list3(list): def __repr__(self): return list.__repr__(self) + class tuple2(tuple): pass + class tuple3(tuple): def __repr__(self): return tuple.__repr__(self) + class dict2(dict): pass + class dict3(dict): def __repr__(self): return dict.__repr__(self) @@ -101,7 +106,13 @@ def test_same_as_repr(self): # Simple objects, small containers and classes that overwrite __repr__ - # For those the result should be the same as repr() + # For those the result should be the same as repr(). + # Ahem. The docs don't say anything about that -- this appears to + # be testing an implementation quirk. Starting in Python 2.5, it's + # not true for dicts: pprint always sorts dicts by key now; before, + # it sorted a dict display if and only if the display required + # multiple lines. For that reason, dicts with more than one element + # aren't tested here. verify = self.assert_ for simple in (0, 0L, 0+0j, 0.0, "", uni(""), (), tuple2(), tuple3(), @@ -112,9 +123,7 @@ (1,2), [3,4], {5: 6, 7: 8}, tuple2((1,2)), tuple3((1,2)), tuple3(range(100)), [3,4], list2([3,4]), list3([3,4]), list3(range(100)), - {5: 6, 7: 8}, dict2({5: 6, 7: 8}), dict3({5: 6, 7: 8}), - dict3([(x,x) for x in range(100)]), - {"xy\tab\n": (3,), 5: [[]], (): {}}, + {5: 6, 7: 8}, dict2({5: 6}), dict3({5: 6}), range(10, -11, -1) ): native = repr(simple) @@ -160,6 +169,24 @@ for type in [list, list2]: self.assertEqual(pprint.pformat(type(o), indent=4), exp) + def test_sorted_dict(self): + # Starting in Python 2.5, pprint sorts dict displays by key regardless + # of how small the dictionary may be. + # Before the change, on 32-bit Windows pformat() gave order + # 'a', 'c', 'b' here, so this test failed. + d = {'a': 1, 'b': 1, 'c': 1} + self.assertEqual(pprint.pformat(d), "{'a': 1, 'b': 1, 'c': 1}") + self.assertEqual(pprint.pformat([d, d]), + "[{'a': 1, 'b': 1, 'c': 1}, {'a': 1, 'b': 1, 'c': 1}]") + + # The next one is kind of goofy. The sorted order depends on the + # alphabetic order of type names: "int" < "str" < "tuple". Before + # Python 2.5, this was in the test_same_as_repr() test. It's worth + # keeping around for now because it's one of few tests of pprint + # against a crazy mix of types. + self.assertEqual(pprint.pformat({"xy\tab\n": (3,), 5: [[]], (): {}}), + r"{5: [[]], 'xy\tab\n': (3,), (): {}}") + def test_subclassing(self): o = {'names with spaces': 'should be presented using repr()', 'others.should.not.be': 'like.this'} Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 3 01:22:51 2006 @@ -101,6 +101,12 @@ Library ------- +- The functions in the ``pprint`` module now sort dictionaries by key + before computing the display. Before 2.5, ``pprint`` sorted a dictionary + if and only if its display required more than one line, although that + wasn't documented. The new behavior increases predictability; e.g., + using ``pprint.pprint(a_dict)`` in a doctest is now reliable. + - Patch #1497027: try HTTP digest auth before basic auth in urllib2 (thanks for J. J. Lee). From buildbot at python.org Sat Jun 3 01:36:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 02 Jun 2006 23:36:14 +0000 Subject: [Python-checkins] buildbot failure in x86 cygwin trunk Message-ID: <20060602233614.7E5DC1E400D@bag.python.org> The Buildbot has detected a new failure of x86 cygwin trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520cygwin%2520trunk/builds/659 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Sat Jun 3 03:45:30 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 03:45:30 +0200 (CEST) Subject: [Python-checkins] r46606 - sandbox/trunk/Doc/functional.rst Message-ID: <20060603014530.66C941E4014@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 03:45:26 2006 New Revision: 46606 Modified: sandbox/trunk/Doc/functional.rst Log: Add some incomplete text and some XXX markers Modified: sandbox/trunk/Doc/functional.rst ============================================================================== --- sandbox/trunk/Doc/functional.rst (original) +++ sandbox/trunk/Doc/functional.rst Sat Jun 3 03:45:26 2006 @@ -597,6 +597,49 @@ many different points (the ``yield`` statements). +Built-in functions +---------------------------------------------- + +Let's look in more detail at those built-in functions that are +relevant to iterators. + +any(), all() + +enumerate() + +sorted() + +sum() + +Two Python's built-in functions, ``map()`` and ``filter()``, are +somewhat obsolete; they duplicate the features of list comprehensions +and return actual lists instead of iterators. + +``map(f, iterA, iterB, ...)`` returns a list containing ``f(iterA[0], +iterB[0]), f(iterA[1], iterB[1]), f(iterA[2], iterB[2]), ...``. + +:: + + def upper(s): + return s.upper() + map(upper, ['sentence', 'fragment']) => + ['SENTENCE', 'FRAGMENT'] + + [upper(s) for s in ['sentence', 'fragment']] => + ['SENTENCE', 'FRAGMENT'] + +As shown above, you can achieve the same effect with a list +comprehension. The ``itertools.imap()`` function does the same thing +but can handle infinite iterators; it'll be discussed in the section on +the ``itertools`` module. + +``filter(predicate, iter)`` is similarly duplicated by list comprehensions. +**, a function that returns the truth value of +some condition. + +reduce() + + The itertools module ----------------------- @@ -726,8 +769,7 @@ /usr/bin/java, /bin/python, /usr/bin/perl, /usr/bin/ruby Another group of functions chooses a subset of an iterator's elements -based on a **predicate**, a function that returns the truth value of -some condition. +based on a predicate. ``itertools.ifilter(predicate, iter)`` returns all the elements for which the predicate returns true:: @@ -812,12 +854,6 @@ of iterator-1 before requesting iterator-2 and its corresponding key. -Built-in functions ----------------------------------------------- - -map(), filter(), reduce() -os.walk() - Small functions and the lambda statement ---------------------------------------------- @@ -826,6 +862,18 @@ The functools module ---------------------------------------------- +XXX + +Topics to place +----------------------------- + +XXX + +os.walk() + +Need a large example. + +======= Acknowledgements ------------------------------ From buildbot at python.org Sat Jun 3 05:58:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 03:58:36 +0000 Subject: [Python-checkins] buildbot failure in x86 OpenBSD trunk Message-ID: <20060603035836.EA66C1E400D@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%2520trunk/builds/814 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by '': Build Source Stamp: [branch trunk] HEAD Blamelist: BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Sat Jun 3 06:49:01 2006 From: python-checkins at python.org (neal.norwitz) Date: Sat, 3 Jun 2006 06:49:01 +0200 (CEST) Subject: [Python-checkins] r46607 - in python/trunk/Mac: IDE scripts MPW Tools Unsupported Wastemods Message-ID: <20060603044901.695801E400D@bag.python.org> Author: neal.norwitz Date: Sat Jun 3 06:49:00 2006 New Revision: 46607 Removed: python/trunk/Mac/IDE scripts/ python/trunk/Mac/MPW/ python/trunk/Mac/Tools/ python/trunk/Mac/Unsupported/ python/trunk/Mac/Wastemods/ Log: Remove Mac OS 9 support (http://mail.python.org/pipermail/python-dev/2006-June/065538.html) From python-checkins at python.org Sat Jun 3 09:37:17 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 3 Jun 2006 09:37:17 +0200 (CEST) Subject: [Python-checkins] r46608 - in python/trunk: Misc/NEWS configure configure.in Message-ID: <20060603073717.917E91E400D@bag.python.org> Author: martin.v.loewis Date: Sat Jun 3 09:37:13 2006 New Revision: 46608 Modified: python/trunk/Misc/NEWS python/trunk/configure python/trunk/configure.in Log: Port to OpenBSD 3.9. Patch from Aldo Cortesi. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 3 09:37:13 2006 @@ -149,6 +149,8 @@ Build ----- +- OpenBSD 3.9 is supported now. + - Patch #1492356: Port to Windows CE. - Bug/Patch #1481770: Use .so extension for shared libraries on HP-UX for ia64. Modified: python/trunk/configure ============================================================================== --- python/trunk/configure (original) +++ python/trunk/configure Sat Jun 3 09:37:13 2006 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 46046 . +# From configure.in Revision: 46295 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59 for python 2.5. # @@ -1529,7 +1529,7 @@ # On OpenBSD, select(2) is not available if _XOPEN_SOURCE is defined, # even though select is a POSIX function. Reported by J. Ribbens. # Reconfirmed for OpenBSD 3.3 by Zachary Hamm, for 3.4 by Jason Ish. - OpenBSD/2.* | OpenBSD/3.[012345678]) + OpenBSD/2.* | OpenBSD/3.[0123456789]) define_xopen_source=no;; # On Solaris 2.6, sys/wait.h is inconsistent in the usage # of union __?sigval. Reported by Stuart Bishop. @@ -10990,13 +10990,27 @@ ;; Linux*|GNU*) LDSHARED='$(CC) -shared';; BSD/OS*/4*) LDSHARED="gcc -shared";; - OpenBSD*|FreeBSD*) + FreeBSD*) if [ "`$CC -dM -E - @) + OpenBSD/2.* | OpenBSD/3.@<:@0123456789@:>@) define_xopen_source=no;; # On Solaris 2.6, sys/wait.h is inconsistent in the usage # of union __?sigval. Reported by Stuart Bishop. @@ -1474,13 +1474,27 @@ ;; Linux*|GNU*) LDSHARED='$(CC) -shared';; BSD/OS*/4*) LDSHARED="gcc -shared";; - OpenBSD*|FreeBSD*) + FreeBSD*) if [[ "`$CC -dM -E - Author: martin.v.loewis Date: Sat Jun 3 09:40:03 2006 New Revision: 46609 Modified: python/branches/release24-maint/Misc/NEWS python/branches/release24-maint/configure python/branches/release24-maint/configure.in Log: Backport of 46608: Port to OpenBSD 3.9. Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Sat Jun 3 09:40:03 2006 @@ -60,6 +60,8 @@ Build ----- +- OpenBSD 3.9 is now supported. + - Test for sys/statvfs.h before including it, as statvfs is present on some OSX installation, but its header file is not. Modified: python/branches/release24-maint/configure ============================================================================== --- python/branches/release24-maint/configure (original) +++ python/branches/release24-maint/configure Sat Jun 3 09:40:03 2006 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 43618 . +# From configure.in Revision: 46011 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59 for python 2.4. # @@ -1480,7 +1480,7 @@ # On OpenBSD, select(2) is not available if _XOPEN_SOURCE is defined, # even though select is a POSIX function. Reported by J. Ribbens. # Reconfirmed for OpenBSD 3.3 by Zachary Hamm, for 3.4 by Jason Ish. - OpenBSD/2.* | OpenBSD/3.[012345678]) + OpenBSD/2.* | OpenBSD/3.[0123456789]) define_xopen_source=no;; # On Solaris 2.6, sys/wait.h is inconsistent in the usage # of union __?sigval. Reported by Stuart Bishop. @@ -10386,13 +10386,27 @@ ;; Linux*|GNU*) LDSHARED='$(CC) -shared';; BSD/OS*/4*) LDSHARED="gcc -shared";; - OpenBSD*|FreeBSD*) + FreeBSD*) if [ "`$CC -dM -E - @) + OpenBSD/2.* | OpenBSD/3.@<:@0123456789@:>@) define_xopen_source=no;; # On Solaris 2.6, sys/wait.h is inconsistent in the usage # of union __?sigval. Reported by Stuart Bishop. @@ -1385,13 +1385,27 @@ ;; Linux*|GNU*) LDSHARED='$(CC) -shared';; BSD/OS*/4*) LDSHARED="gcc -shared";; - OpenBSD*|FreeBSD*) + FreeBSD*) if [[ "`$CC -dM -E - Author: martin.v.loewis Date: Sat Jun 3 09:42:26 2006 New Revision: 46610 Modified: python/trunk/PC/py.ico python/trunk/PC/pyc.ico python/trunk/PC/pycon.ico Log: Updated version (win32-icons2.zip) from #1490384. Modified: python/trunk/PC/py.ico ============================================================================== Binary files. No diff available. Modified: python/trunk/PC/pyc.ico ============================================================================== Binary files. No diff available. Modified: python/trunk/PC/pycon.ico ============================================================================== Binary files. No diff available. From buildbot at python.org Sat Jun 3 10:03:38 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 08:03:38 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD trunk Message-ID: <20060603080338.AD9251E4003@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%2520trunk/builds/816 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 10:53:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 08:53:14 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper 2.4 Message-ID: <20060603085314.D41071E4003@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%25202.4/builds/72 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 11:13:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 09:13:17 +0000 Subject: [Python-checkins] buildbot warnings in S-390 Debian 2.4 Message-ID: <20060603091317.81FB21E4003@bag.python.org> The Buildbot has detected a new failure of S-390 Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/S-390%2520Debian%25202.4/builds/0 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 11:52:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 09:52:14 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060603095214.D9BAF1E4003@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/35 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 11:59:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 09:59:28 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian 2.4 Message-ID: <20060603095928.272ED1E4003@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%25202.4/builds/11 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: martin.v.loewis BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From nnorwitz at gmail.com Sat Jun 3 18:50:41 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 3 Jun 2006 09:50:41 -0700 Subject: [Python-checkins] r46603 - python/trunk/Lib/test/test_struct.py In-Reply-To: <20060602130346.9071C1E400B@bag.python.org> References: <20060602130346.9071C1E400B@bag.python.org> Message-ID: On 6/2/06, martin.blais wrote: > Author: martin.blais > Date: Fri Jun 2 15:03:43 2006 > New Revision: 46603 > > Modified: > python/trunk/Lib/test/test_struct.py > Log: > Fixed struct test to not use unittest. Shoot, I had hoped you would go the other way and convert all the old test cases to use unittest. :-) n From python-checkins at python.org Sat Jun 3 19:20:13 2006 From: python-checkins at python.org (george.yoshida) Date: Sat, 3 Jun 2006 19:20:13 +0200 (CEST) Subject: [Python-checkins] r46611 - peps/trunk/pep-0339.txt peps/trunk/pep-0347.txt peps/trunk/pep-0355.txt Message-ID: <20060603172013.1441D1E401B@bag.python.org> Author: george.yoshida Date: Sat Jun 3 19:20:12 2006 New Revision: 46611 Modified: peps/trunk/pep-0339.txt peps/trunk/pep-0347.txt peps/trunk/pep-0355.txt Log: Repair typos Modified: peps/trunk/pep-0339.txt ============================================================================== --- peps/trunk/pep-0339.txt (original) +++ peps/trunk/pep-0339.txt Sat Jun 3 19:20:12 2006 @@ -29,7 +29,7 @@ Starting with Python 2.5, the above steps are now used. This change was done to simplify compilation by breaking it into three steps. -The purpose of this document is to outline how the lattter three steps +The purpose of this document is to outline how the latter three steps of the process works. This document does not touch on how parsing works beyond what is needed @@ -515,7 +515,7 @@ changed. * pycodegen.py - One of the files that muc be modified if Include/opcode.h is + One of the files that must be modified if Include/opcode.h is changed. Modified: peps/trunk/pep-0347.txt ============================================================================== --- peps/trunk/pep-0347.txt (original) +++ peps/trunk/pep-0347.txt Sat Jun 3 19:20:12 2006 @@ -56,8 +56,8 @@ operators could not always respond in a timely manner. In particular, for CVS, they had to reduce the load on the primary CVS server by introducing a second, read-only CVS server for anonymous access. This -server is regularly synchronized, but lags behind the the read-write -CVS repository between synchronizations. As a result, users without +server is regularly synchronized, but lags behind the read-write CVS +repository between synchronizations. As a result, users without commit access can see recent changes to the repository only after a delay. Modified: peps/trunk/pep-0355.txt ============================================================================== --- peps/trunk/pep-0355.txt (original) +++ peps/trunk/pep-0355.txt Sat Jun 3 19:20:12 2006 @@ -33,7 +33,7 @@ were made to get the path module included in the Python standard library; [4], [5], [6], [7]. - This PEP summarizes the the ideas and suggestions people have + This PEP summarizes the ideas and suggestions people have expressed about the path module and proposes that a modified version should be included in the standard library. @@ -107,7 +107,7 @@ the manipulation of file contents, for which file objects are better suited. - - Platform incompatibilites are dealt with by not instantiating + - Platform incompatibilities are dealt with by not instantiating system specific methods. @@ -389,7 +389,7 @@ * The __div__() method was removed. Overloading the / (division) operator may be "too much magic" and make path concatenation appear to be division. The method can always be re-added later - if the BFDL so desires. In its place, __new__() got an *args + if the BDFL so desires. In its place, __new__() got an *args argument that accepts both Path and string objects. The *args are concatenated with os.path.join() which is used to construct the Path object. These changes obsoleted the problematic @@ -429,7 +429,7 @@ should it live? In its own module or in os? * Due to Path subclassing either str or unicode, the following - non-magic, public methods are availible on Path objects: + non-magic, public methods are available on Path objects: capitalize(), center(), count(), decode(), encode(), endswith(), expandtabs(), find(), index(), isalnum(), @@ -525,7 +525,7 @@ References and Footnotes - [1] Method is not guaranteed to be availible on all platforms. + [1] Method is not guaranteed to be available on all platforms. [2] "(idea) subclassable string: path object?", van Rossum, 2001 http://mail.python.org/pipermail/python-dev/2001-August/016663.html From python-checkins at python.org Sat Jun 3 20:09:44 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 20:09:44 +0200 (CEST) Subject: [Python-checkins] r46612 - python/trunk/Doc/lib/libsgmllib.tex Message-ID: <20060603180944.4A6011E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 20:09:41 2006 New Revision: 46612 Modified: python/trunk/Doc/lib/libsgmllib.tex Log: [Bug #1472084] Fix description of do_tag Modified: python/trunk/Doc/lib/libsgmllib.tex ============================================================================== --- python/trunk/Doc/lib/libsgmllib.tex (original) +++ python/trunk/Doc/lib/libsgmllib.tex Sat Jun 3 20:09:41 2006 @@ -218,8 +218,9 @@ \end{methoddescni} \begin{methoddescni}{do_\var{tag}}{attributes} -This method is called to process an opening tag \var{tag} that does -not come with a matching closing tag. The \var{attributes} argument +This method is called to process an opening tag \var{tag} +for which no \method{start_\var{tag}} method is defined. +The \var{attributes} argument has the same meaning as described for \method{handle_starttag()} above. \end{methoddescni} From python-checkins at python.org Sat Jun 3 20:10:03 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 20:10:03 +0200 (CEST) Subject: [Python-checkins] r46613 - python/branches/release24-maint/Doc/lib/libsgmllib.tex Message-ID: <20060603181003.984D41E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 20:10:03 2006 New Revision: 46613 Modified: python/branches/release24-maint/Doc/lib/libsgmllib.tex Log: [Bug #1472084] Fix description of do_tag Modified: python/branches/release24-maint/Doc/lib/libsgmllib.tex ============================================================================== --- python/branches/release24-maint/Doc/lib/libsgmllib.tex (original) +++ python/branches/release24-maint/Doc/lib/libsgmllib.tex Sat Jun 3 20:10:03 2006 @@ -208,8 +208,9 @@ \end{methoddescni} \begin{methoddescni}{do_\var{tag}}{attributes} -This method is called to process an opening tag \var{tag} that does -not come with a matching closing tag. The \var{attributes} argument +This method is called to process an opening tag \var{tag} +for which no \method{start_\var{tag}} method is defined. +The \var{attributes} argument has the same meaning as described for \method{handle_starttag()} above. \end{methoddescni} From python-checkins at python.org Sat Jun 3 20:33:36 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 20:33:36 +0200 (CEST) Subject: [Python-checkins] r46614 - python/trunk/Doc/lib/libsocket.tex Message-ID: <20060603183336.34DBC1E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 20:33:35 2006 New Revision: 46614 Modified: python/trunk/Doc/lib/libsocket.tex Log: [Bug #1475554] Strengthen text to say 'must' instead of 'should' Modified: python/trunk/Doc/lib/libsocket.tex ============================================================================== --- python/trunk/Doc/lib/libsocket.tex (original) +++ python/trunk/Doc/lib/libsocket.tex Sat Jun 3 20:33:35 2006 @@ -548,7 +548,7 @@ The file object references a \cfunction{dup()}ped version of the socket file descriptor, so the file object and socket object may be closed or garbage-collected independently. -The socket should be in blocking mode. +The socket must be in blocking mode. \index{I/O control!buffering}The optional \var{mode} and \var{bufsize} arguments are interpreted the same way as by the built-in \function{file()} function; see ``Built-in Functions'' @@ -647,7 +647,7 @@ blocking and timeout modes are shared between file descriptors and socket objects that refer to the same network endpoint. A consequence of this is that file objects returned by the \method{makefile()} -method should only be used when the socket is in blocking mode; in +method must only be used when the socket is in blocking mode; in timeout or non-blocking mode file operations that cannot be completed immediately will fail. From python-checkins at python.org Sat Jun 3 20:34:03 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 20:34:03 +0200 (CEST) Subject: [Python-checkins] r46615 - python/branches/release24-maint/Doc/lib/libsocket.tex Message-ID: <20060603183403.9D5D31E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 20:34:03 2006 New Revision: 46615 Modified: python/branches/release24-maint/Doc/lib/libsocket.tex Log: [Bug #1475554] Strengthen text to say 'must' instead of 'should' Modified: python/branches/release24-maint/Doc/lib/libsocket.tex ============================================================================== --- python/branches/release24-maint/Doc/lib/libsocket.tex (original) +++ python/branches/release24-maint/Doc/lib/libsocket.tex Sat Jun 3 20:34:03 2006 @@ -545,7 +545,7 @@ The file object references a \cfunction{dup()}ped version of the socket file descriptor, so the file object and socket object may be closed or garbage-collected independently. -The socket should be in blocking mode. +The socket must be in blocking mode. \index{I/O control!buffering}The optional \var{mode} and \var{bufsize} arguments are interpreted the same way as by the built-in \function{file()} function; see ``Built-in Functions'' @@ -644,7 +644,7 @@ blocking and timeout modes are shared between file descriptors and socket objects that refer to the same network endpoint. A consequence of this is that file objects returned by the \method{makefile()} -method should only be used when the socket is in blocking mode; in +method must only be used when the socket is in blocking mode; in timeout or non-blocking mode file operations that cannot be completed immediately will fail. From python-checkins at python.org Sat Jun 3 20:41:30 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 20:41:30 +0200 (CEST) Subject: [Python-checkins] r46616 - python/trunk/Doc/lib/liburllib2.tex Message-ID: <20060603184130.AC7B51E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 20:41:28 2006 New Revision: 46616 Modified: python/trunk/Doc/lib/liburllib2.tex Log: [Bug #1441864] Clarify description of 'data' argument Modified: python/trunk/Doc/lib/liburllib2.tex ============================================================================== --- python/trunk/Doc/lib/liburllib2.tex (original) +++ python/trunk/Doc/lib/liburllib2.tex Sat Jun 3 20:41:28 2006 @@ -18,11 +18,13 @@ Open the URL \var{url}, which can be either a string or a \class{Request} object. -\var{data} should be a string, which specifies additional data to -send to the server. In HTTP requests, which are the only ones that -support \var{data}, it should be a buffer in the format of -\mimetype{application/x-www-form-urlencoded}, for example one returned -from \function{urllib.urlencode()}. +\var{data} should be a string, which specifies additional data to send +to the server. Currently HTTP requests are the only ones that use +\var{data}. For HTTP, the request will be a POST instead of a GET +when the \var{data} parameter is provided. \var{data} should be a +buffer in the standard \mimetype{application/x-www-form-urlencoded} format. +The \function{urllib.urlencode()} function takes a mapping or +sequence of 2-tuples and returns a string in this format. This function returns a file-like object with two additional methods: From python-checkins at python.org Sat Jun 3 20:43:25 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 20:43:25 +0200 (CEST) Subject: [Python-checkins] r46617 - python/trunk/Doc/lib/liburllib2.tex Message-ID: <20060603184325.346C91E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 20:43:24 2006 New Revision: 46617 Modified: python/trunk/Doc/lib/liburllib2.tex Log: Minor rewording Modified: python/trunk/Doc/lib/liburllib2.tex ============================================================================== --- python/trunk/Doc/lib/liburllib2.tex (original) +++ python/trunk/Doc/lib/liburllib2.tex Sat Jun 3 20:43:24 2006 @@ -18,13 +18,13 @@ Open the URL \var{url}, which can be either a string or a \class{Request} object. -\var{data} should be a string, which specifies additional data to send -to the server. Currently HTTP requests are the only ones that use -\var{data}. For HTTP, the request will be a POST instead of a GET -when the \var{data} parameter is provided. \var{data} should be a -buffer in the standard \mimetype{application/x-www-form-urlencoded} format. -The \function{urllib.urlencode()} function takes a mapping or -sequence of 2-tuples and returns a string in this format. +\var{data} may be a string specifying additional data to send to the +server. Currently HTTP requests are the only ones that use \var{data}; +the HTTP request will be a POST instead of a GET when the \var{data} +parameter is provided. \var{data} should be a buffer in the standard +\mimetype{application/x-www-form-urlencoded} format. The +\function{urllib.urlencode()} function takes a mapping or sequence of +2-tuples and returns a string in this format. This function returns a file-like object with two additional methods: From python-checkins at python.org Sat Jun 3 20:46:03 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 20:46:03 +0200 (CEST) Subject: [Python-checkins] r46618 - python/branches/release24-maint/Doc/lib/liburllib2.tex Message-ID: <20060603184603.B57C71E4014@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 20:46:03 2006 New Revision: 46618 Modified: python/branches/release24-maint/Doc/lib/liburllib2.tex Log: [Bug #1441864] Clarify description of 'data' argument Modified: python/branches/release24-maint/Doc/lib/liburllib2.tex ============================================================================== --- python/branches/release24-maint/Doc/lib/liburllib2.tex (original) +++ python/branches/release24-maint/Doc/lib/liburllib2.tex Sat Jun 3 20:46:03 2006 @@ -18,11 +18,13 @@ Open the URL \var{url}, which can be either a string or a \class{Request} object. -\var{data} should be a string, which specifies additional data to -send to the server. In HTTP requests, which are the only ones that -support \var{data}, it should be a buffer in the format of -\mimetype{application/x-www-form-urlencoded}, for example one returned -from \function{urllib.urlencode()}. +\var{data} may be a string specifying additional data to send to the +server. Currently HTTP requests are the only ones that use \var{data}; +the HTTP request will be a POST instead of a GET when the \var{data} +parameter is provided. \var{data} should be a buffer in the standard +\mimetype{application/x-www-form-urlencoded} format. The +\function{urllib.urlencode()} function takes a mapping or sequence of +2-tuples and returns a string in this format. This function returns a file-like object with two additional methods: From neal at metaslash.com Sat Jun 3 15:52:30 2006 From: neal at metaslash.com (Neal Norwitz) Date: Sat, 3 Jun 2006 09:52:30 -0400 Subject: [Python-checkins] Python Regression Test Failures doc (1) Message-ID: <20060603135230.GA25565@python.psfb.org> TEXINPUTS=/home/neal/python/r24/Doc/commontex: python /home/neal/python/r24/Doc/tools/mkhowto --html --about html/stdabout.dat --iconserver ../icons --favicon ../icons/pyfav.png --address "See About this document... for information on suggesting changes." --up-link ../index.html --up-title "Python Documentation Index" --global-module-index "../modindex.html" --dvips-safe --dir html/lib lib/lib.tex *** Session transcript and error messages are in /home/neal/python/r24/Doc/html/lib/lib.how. +++ TEXINPUTS=/home/neal/python/r24/Doc/lib:/home/neal/python/r24/Doc/commontex:/home/neal/python/r24/Doc/paper-letter:/home/neal/python/r24/Doc/texinputs: +++ latex lib +++ latex2html -init_file lib.l2h -dir /home/neal/python/r24/Doc/html/lib /home/neal/python/r24/Doc/lib/lib.tex Traceback (most recent call last): File "/home/neal/python/r24/Doc/tools/mkhowto", line 659, in ? main() File "/home/neal/python/r24/Doc/tools/mkhowto", line 615, in main Job(options, path).build() File "/home/neal/python/r24/Doc/tools/mkhowto", line 284, in build self.build_html(self.builddir) File "/home/neal/python/r24/Doc/tools/mkhowto", line 408, in build_html self.run(" ".join(args)) # XXX need quoting! File "/home/neal/python/r24/Doc/tools/mkhowto", line 517, in run self.warning( File "/home/neal/python/r24/Doc/tools/mkhowto", line 543, in warning self.log(msg) File "/home/neal/python/r24/Doc/tools/mkhowto", line 546, in log fp = open(self.log_filename, "a") IOError: [Errno 2] No such file or directory: '/home/neal/python/r24/Doc/html/lib/lib.how' make: *** [html/lib/lib.html] Error 1 From buildbot at python.org Sat Jun 3 20:54:51 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 18:54:51 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060603185451.5A09B1E4003@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/109 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 20:56:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 18:56:37 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060603185637.8FEEA1E4003@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/559 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 21:00:21 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 19:00:21 +0000 Subject: [Python-checkins] buildbot failure in x86 cygwin trunk Message-ID: <20060603190021.AA6021E4010@bag.python.org> The Buildbot has detected a new failure of x86 cygwin trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520cygwin%2520trunk/builds/664 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling BUILD FAILED: failed svn sincerely, -The Buildbot From python-checkins at python.org Sat Jun 3 21:02:42 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 21:02:42 +0200 (CEST) Subject: [Python-checkins] r46619 - python/trunk/Modules/_tkinter.c Message-ID: <20060603190242.27D051E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 21:02:35 2006 New Revision: 46619 Modified: python/trunk/Modules/_tkinter.c Log: [Bug #1497414] _self is a reserved word in the WATCOM 10.6 C compiler. Fix by renaming the variable. In a different module, Neal fixed it by renaming _self to self. There's already a variable named 'self' here, so I used selfptr. (I'm committing this on a Mac without Tk, but it's a simple search-and-replace. , so I'll watch the buildbots and see what happens.) Modified: python/trunk/Modules/_tkinter.c ============================================================================== --- python/trunk/Modules/_tkinter.c (original) +++ python/trunk/Modules/_tkinter.c Sat Jun 3 21:02:35 2006 @@ -1274,13 +1274,13 @@ and perform processing there. */ static PyObject * -Tkapp_Call(PyObject *_self, PyObject *args) +Tkapp_Call(PyObject *selfptr, PyObject *args) { Tcl_Obj *objStore[ARGSZ]; Tcl_Obj **objv = NULL; int objc, i; PyObject *res = NULL; - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; /* Could add TCL_EVAL_GLOBAL if wrapped by GlobalCall... */ int flags = TCL_EVAL_DIRECT; @@ -1326,7 +1326,7 @@ ENTER_OVERLAP if (i == TCL_ERROR) - Tkinter_Error(_self); + Tkinter_Error(selfptr); else res = Tkapp_CallResult(self); @@ -1542,12 +1542,12 @@ } static PyObject* -var_invoke(EventFunc func, PyObject *_self, PyObject *args, int flags) +var_invoke(EventFunc func, PyObject *selfptr, PyObject *args, int flags) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; #ifdef WITH_THREAD if (self->threaded && self->thread_id != Tcl_GetCurrentThread()) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; VarEvent *ev; PyObject *res, *exc_type, *exc_val; @@ -1559,7 +1559,7 @@ ev = (VarEvent*)ckalloc(sizeof(VarEvent)); - ev->self = _self; + ev->self = selfptr; ev->args = args; ev->flags = flags; ev->func = func; @@ -1579,7 +1579,7 @@ } #endif /* Tcl is not threaded, or this is the interpreter thread. */ - return func(_self, args, flags); + return func(selfptr, args, flags); } static PyObject * @@ -2079,9 +2079,9 @@ } static PyObject * -Tkapp_CreateCommand(PyObject *_self, PyObject *args) +Tkapp_CreateCommand(PyObject *selfptr, PyObject *args) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; PythonCmd_ClientData *data; char *cmdName; PyObject *func; @@ -2105,7 +2105,7 @@ return PyErr_NoMemory(); Py_XINCREF(self); Py_XINCREF(func); - data->self = _self; + data->self = selfptr; data->func = func; if (self->threaded && self->thread_id != Tcl_GetCurrentThread()) { @@ -2139,9 +2139,9 @@ static PyObject * -Tkapp_DeleteCommand(PyObject *_self, PyObject *args) +Tkapp_DeleteCommand(PyObject *selfptr, PyObject *args) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; char *cmdName; int err; @@ -2502,10 +2502,10 @@ /** Event Loop **/ static PyObject * -Tkapp_MainLoop(PyObject *_self, PyObject *args) +Tkapp_MainLoop(PyObject *selfptr, PyObject *args) { int threshold = 0; - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; #ifdef WITH_THREAD PyThreadState *tstate = PyThreadState_Get(); #endif From buildbot at python.org Sat Jun 3 21:27:45 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 19:27:45 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian 2.4 Message-ID: <20060603192745.418C81E4003@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%25202.4/builds/12 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 21:32:24 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 19:32:24 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060603193224.3EAA11E4003@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/524 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From nnorwitz at gmail.com Sat Jun 3 21:59:55 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 3 Jun 2006 12:59:55 -0700 Subject: [Python-checkins] r46144 - in python/trunk: Lib/struct.py Modules/_struct.c Modules/structmodule.c In-Reply-To: <20060523191241.B5FEF1E4003@bag.python.org> References: <20060523191241.B5FEF1E4003@bag.python.org> Message-ID: Note: I'm reviewing the version current as of 3 June 2006, Revision: 46613. > Added: python/trunk/Modules/_struct.c > +/* compatibility macros */ > +#if (PY_VERSION_HEX < 0x02050000) > +typedef int Py_ssize_t; > +#endif Why is this needed? Is this maintained for older versions of Python? (There are a few others elsewhere in this module.) If it is needed, is int correct or should it be long? > +#ifdef __powerc > +#pragma options align=reset > +#endif What is this? Is it required? If so it should be documented. In get_wrapped_long, there's this code: x = (long)PyLong_AsUnsignedLong(wrapped); Why not use PyLongAsLong() so you don't need to cast? It really required as is? If so, a comment explaining why would be nice. In get_wrapped_ulong(), why is x a long rather than an unsigned long with all the casts? > +/* Floating point helpers */ > + > +static PyObject * > +unpack_float(const char *p, /* start of 4-byte string */ > + int le) /* true for little-endian, false for big-endian */ If you renamed le to little_endian, you could get rid of the comments and would be more readable IMO. (Same with unpack_double.) > +static int > +bp_uint(char *p, PyObject *v, const formatdef *f) > +{ unsigned long x; Py_ssize_t i; if (get_wrapped_ulong(v, &x) < 0) return -1; i = f->size; if (i != SIZEOF_LONG) { unsigned long maxint = 1; maxint <<= (unsigned long)(i * 8); if (x >= maxint) RANGE_ERROR(x, f, 1, maxint - 1); } when you say i != SIZEOF_LONG, you really mean that i < SIZEOF_LONG, right? Otherwise the maxint calc wouldn't work. Same with lp_uint. > +static PyObject * > +lu_int(const char *p, const formatdef *f) > +{ long x; Py_ssize_t i; if (get_wrapped_long(v, &x) < 0) return -1; i = f->size; if (i != SIZEOF_LONG) { if ((i == 2) && (x < -32768 || x > 32767)) RANGE_ERROR(x, f, 0, 0xffffL); #if (SIZEOF_LONG != 4) else if ((i == 4) && (x < -2147483648L || x > 2147483647L)) RANGE_ERROR(x, f, 0, 0xffffffffL); #endif #ifdef PY_STRUCT_OVERFLOW_MASKING else if ((i == 1) && (x < -128 || x > 127)) RANGE_ERROR(x, f, 0, 0xffL); #endif } *** Why not use a switch(i)? (Now pack_internal) > +static PyObject * > +s_pack(PyObject *self, PyObject *args) > +{ ... > + } else if (e->format == 'p') { ... > + n = PyString_GET_SIZE(v); > + if (n > (code->repeat - 1)) > + n = code->repeat - 1; > + if (n > 0) > + memcpy(res + 1, PyString_AS_STRING(v), n); > + if (n > 255) > + n = 255; > + *res = Py_SAFE_DOWNCAST(n, Py_ssize_t, unsigned char); Wait, don't you want to truncate n (length of string) *before* doing the memcpy? If this code is wrong, there should be a test for it. > +PyMODINIT_FUNC > +init_struct(void) > +{ ... PyModule_AddIntConstant(m, "_PY_STRUCT_RANGE_CHECKING", 1); #ifdef PY_STRUCT_OVERFLOW_MASKING PyModule_AddIntConstant(m, "_PY_STRUCT_OVERFLOW_MASKING", 1); #endif Shouldn't those be bools? n From buildbot at python.org Sat Jun 3 22:01:38 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 20:01:38 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060603200138.A81331E4003@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/270 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 3 22:31:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 20:31:28 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) 2.4 Message-ID: <20060603203128.967571E400D@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%25202.4/builds/77 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 3 22:34:34 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 3 Jun 2006 22:34:34 +0200 (CEST) Subject: [Python-checkins] r46620 - python/branches/release24-maint/Modules/_tkinter.c Message-ID: <20060603203434.2A8031E4003@bag.python.org> Author: andrew.kuchling Date: Sat Jun 3 22:34:33 2006 New Revision: 46620 Modified: python/branches/release24-maint/Modules/_tkinter.c Log: [Bug #1497414] _self is a reserved word in the WATCOM 10.6 C compiler. Fix by renaming the variable. In a different module, Neal fixed it by renaming _self to self. There's already a variable named 'self' here, so I used selfptr. Modified: python/branches/release24-maint/Modules/_tkinter.c ============================================================================== --- python/branches/release24-maint/Modules/_tkinter.c (original) +++ python/branches/release24-maint/Modules/_tkinter.c Sat Jun 3 22:34:33 2006 @@ -1264,13 +1264,13 @@ and perform processing there. */ static PyObject * -Tkapp_Call(PyObject *_self, PyObject *args) +Tkapp_Call(PyObject *selfptr, PyObject *args) { Tcl_Obj *objStore[ARGSZ]; Tcl_Obj **objv = NULL; int objc, i; PyObject *res = NULL; - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; /* Could add TCL_EVAL_GLOBAL if wrapped by GlobalCall... */ int flags = TCL_EVAL_DIRECT; @@ -1316,7 +1316,7 @@ ENTER_OVERLAP if (i == TCL_ERROR) - Tkinter_Error(_self); + Tkinter_Error(selfptr); else res = Tkapp_CallResult(self); @@ -1532,12 +1532,12 @@ } static PyObject* -var_invoke(EventFunc func, PyObject *_self, PyObject *args, int flags) +var_invoke(EventFunc func, PyObject *selfptr, PyObject *args, int flags) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; #ifdef WITH_THREAD if (self->threaded && self->thread_id != Tcl_GetCurrentThread()) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; VarEvent *ev; PyObject *res, *exc_type, *exc_val; @@ -1549,7 +1549,7 @@ ev = (VarEvent*)ckalloc(sizeof(VarEvent)); - ev->self = _self; + ev->self = selfptr; ev->args = args; ev->flags = flags; ev->func = func; @@ -1569,7 +1569,7 @@ } #endif /* Tcl is not threaded, or this is the interpreter thread. */ - return func(_self, args, flags); + return func(selfptr, args, flags); } static PyObject * @@ -2069,9 +2069,9 @@ } static PyObject * -Tkapp_CreateCommand(PyObject *_self, PyObject *args) +Tkapp_CreateCommand(PyObject *selfptr, PyObject *args) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; PythonCmd_ClientData *data; char *cmdName; PyObject *func; @@ -2095,7 +2095,7 @@ return PyErr_NoMemory(); Py_XINCREF(self); Py_XINCREF(func); - data->self = _self; + data->self = selfptr; data->func = func; if (self->threaded && self->thread_id != Tcl_GetCurrentThread()) { @@ -2129,9 +2129,9 @@ static PyObject * -Tkapp_DeleteCommand(PyObject *_self, PyObject *args) +Tkapp_DeleteCommand(PyObject *selfptr, PyObject *args) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; char *cmdName; int err; @@ -2492,10 +2492,10 @@ /** Event Loop **/ static PyObject * -Tkapp_MainLoop(PyObject *_self, PyObject *args) +Tkapp_MainLoop(PyObject *selfptr, PyObject *args) { int threshold = 0; - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; #ifdef WITH_THREAD PyThreadState *tstate = PyThreadState_Get(); #endif From amk at amk.ca Sat Jun 3 22:38:44 2006 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 3 Jun 2006 16:38:44 -0400 Subject: [Python-checkins] r46620 - python/branches/release24-maint/Modules/_tkinter.c In-Reply-To: <20060603203434.2A8031E4003@bag.python.org> References: <20060603203434.2A8031E4003@bag.python.org> Message-ID: <20060603203843.GA13691@Andrew-iBook2.local> On Sat, Jun 03, 2006 at 10:34:34PM +0200, andrew.kuchling wrote: > [Bug #1497414] _self is a reserved word in the WATCOM 10.6 C compiler. > Fix by renaming the variable. Fredrik -- grepping for _self turns it up as a variable in _elementtree.c. You should probably change that usage, too. --amk From martin at v.loewis.de Sat Jun 3 23:08:25 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 03 Jun 2006 23:08:25 +0200 Subject: [Python-checkins] r46144 - in python/trunk: Lib/struct.py Modules/_struct.c Modules/structmodule.c In-Reply-To: References: <20060523191241.B5FEF1E4003@bag.python.org> Message-ID: <4481FA49.4040504@v.loewis.de> Neal Norwitz wrote: >> +/* compatibility macros */ >> +#if (PY_VERSION_HEX < 0x02050000) >> +typedef int Py_ssize_t; >> +#endif > > Why is this needed? Is this maintained for older versions of Python? > (There are a few others elsewhere in this module.) If it is needed, > is int correct or should it be long? It's needed if the module is mean to work with earlier Python versions. Defining Py_ssize_t as int is correct: collections where previously counted in ints. >> +#ifdef __powerc >> +#pragma options align=reset >> +#endif > > What is this? Is it required? If so it should be documented. Not sure what it is; it looks wrong. If there are alignment errors in the module, they should be corrected instead of working around. Regards, Martin From python-checkins at python.org Sat Jun 3 23:56:06 2006 From: python-checkins at python.org (fredrik.lundh) Date: Sat, 3 Jun 2006 23:56:06 +0200 (CEST) Subject: [Python-checkins] r46621 - python/trunk/Modules/_elementtree.c Message-ID: <20060603215606.17E0C1E4003@bag.python.org> Author: fredrik.lundh Date: Sat Jun 3 23:56:05 2006 New Revision: 46621 Modified: python/trunk/Modules/_elementtree.c Log: "_self" is a said to be a reserved word in Watcom C 10.6. I'm not sure that's really standard compliant behaviour, but I guess we have to fix that anyway... Modified: python/trunk/Modules/_elementtree.c ============================================================================== --- python/trunk/Modules/_elementtree.c (original) +++ python/trunk/Modules/_elementtree.c Sat Jun 3 23:56:05 2006 @@ -1476,7 +1476,7 @@ } static PyObject* -treebuilder(PyObject* _self, PyObject* args) +treebuilder(PyObject* self_, PyObject* args) { if (!PyArg_ParseTuple(args, ":TreeBuilder")) return NULL; @@ -2201,7 +2201,7 @@ /* constructor and destructor */ static PyObject* -xmlparser(PyObject* _self, PyObject* args, PyObject* kw) +xmlparser(PyObject* self_, PyObject* args, PyObject* kw) { XMLParserObject* self; /* FIXME: does this need to be static? */ From python-checkins at python.org Sun Jun 4 00:44:43 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 00:44:43 +0200 (CEST) Subject: [Python-checkins] r46622 - python/trunk/Demo/curses/README Message-ID: <20060603224443.3724D1E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 00:44:42 2006 New Revision: 46622 Modified: python/trunk/Demo/curses/README Log: Update readme Modified: python/trunk/Demo/curses/README ============================================================================== --- python/trunk/Demo/curses/README (original) +++ python/trunk/Demo/curses/README Sun Jun 4 00:44:42 2006 @@ -11,14 +11,11 @@ course. ncurses.py -- currently only a panels demo - XXX this won't work until panel support is checked in rain.py -- raindrops keep falling on my desktop tclock.py -- ASCII clock, by Howard Jones xmas.py -- I'm dreaming of an ASCII christmas -Please send bugfixes and new contributions to me or, even better, -submit them to the Python Bug Tracker on SourceForge -(). +Please submit bugfixes and new contributions to the Python bug tracker. Other demos From buildbot at python.org Sun Jun 4 00:46:16 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 22:46:16 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060603224616.127671E4003@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/561 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 00:59:23 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 00:59:23 +0200 (CEST) Subject: [Python-checkins] r46623 - python/trunk/Demo/curses/xmas.py Message-ID: <20060603225923.D1C9A1E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 00:59:23 2006 New Revision: 46623 Modified: python/trunk/Demo/curses/xmas.py Log: Drop 0 parameter Modified: python/trunk/Demo/curses/xmas.py ============================================================================== --- python/trunk/Demo/curses/xmas.py (original) +++ python/trunk/Demo/curses/xmas.py Sun Jun 4 00:59:23 2006 @@ -4,7 +4,7 @@ # $Id$ # # I'm dreaming of an ascii character-based monochrome Christmas, -# Just like the one's I used to know! +# Just like the ones I used to know! # Via a full duplex communications channel, # At 9600 bits per second, # Even though it's kinda slow. @@ -272,7 +272,7 @@ def blinkit(): treescrn8.touchwin() - for cycle in range(0, 5): + for cycle in range(5): if cycle == 0: treescrn3.overlay(treescrn8) treescrn8.refresh() @@ -380,7 +380,7 @@ middeer0.refresh() w_del_msg.refresh() - for looper in range(0, 2): + for looper in range(2): deer_step(middeer3, y_pos, x_pos) deer_step(middeer2, y_pos, x_pos) deer_step(middeer1, y_pos, x_pos) From python-checkins at python.org Sun Jun 4 01:00:00 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:00:00 +0200 (CEST) Subject: [Python-checkins] r46624 - python/trunk/Demo/curses/life.py Message-ID: <20060603230000.3DC741E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 00:59:59 2006 New Revision: 46624 Modified: python/trunk/Demo/curses/life.py Log: Some code tidying; use curses.wrapper Modified: python/trunk/Demo/curses/life.py ============================================================================== --- python/trunk/Demo/curses/life.py (original) +++ python/trunk/Demo/curses/life.py Sun Jun 4 00:59:59 2006 @@ -44,14 +44,15 @@ scr -- curses screen object to use for display char -- character used to render live cells (default: '*') """ - self.state={} ; self.scr=scr + self.state = {} + self.scr = scr Y, X = self.scr.getmaxyx() self.X, self.Y = X-2, Y-2-1 self.char = char self.scr.clear() # Draw a border around the board - border_line='+'+(self.X*'-')+'+' + border_line = '+'+(self.X*'-')+'+' self.scr.addstr(0, 0, border_line) self.scr.addstr(self.Y+1,0, border_line) for y in range(0, self.Y): @@ -73,16 +74,16 @@ del self.state[x,y] self.scr.addch(y+1, x+1, ' ') else: - self.state[x,y]=1 + self.state[x,y] = 1 self.scr.addch(y+1, x+1, self.char) self.scr.refresh() def erase(self): """Clear the entire board and update the board display""" - self.state={} - self.display(update_board=0) + self.state = {} + self.display(update_board=False) - def display(self, update_board=1): + def display(self, update_board=True): """Display the whole board, optionally computing one generation""" M,N = self.X, self.Y if not update_board: @@ -95,42 +96,46 @@ self.scr.refresh() return - d={} ; self.boring=1 + d = {} + self.boring = 1 for i in range(0, M): - L=range( max(0, i-1), min(M, i+2) ) + L = range( max(0, i-1), min(M, i+2) ) for j in range(0, N): - s=0 - live=self.state.has_key( (i,j) ) + s = 0 + live = self.state.has_key( (i,j) ) for k in range( max(0, j-1), min(N, j+2) ): for l in L: if self.state.has_key( (l,k) ): - s=s+1 - s=s-live - if s==3: + s += 1 + s -= live + if s == 3: # Birth - d[i,j]=1 + d[i,j] = 1 self.scr.addch(j+1, i+1, self.char) - if not live: self.boring=0 - elif s==2 and live: d[i,j]=1 # Survival + if not live: self.boring = 0 + elif s == 2 and live: d[i,j] = 1 # Survival elif live: # Death self.scr.addch(j+1, i+1, ' ') - self.boring=0 - self.state=d + self.boring = 0 + self.state = d self.scr.refresh() def makeRandom(self): "Fill the board with a random pattern" - self.state={} + self.state = {} for i in range(0, self.X): for j in range(0, self.Y): - if random.random() > 0.5: self.set(j,i) + if random.random() > 0.5: + self.set(j,i) def erase_menu(stdscr, menu_y): "Clear the space where the menu resides" - stdscr.move(menu_y, 0) ; stdscr.clrtoeol() - stdscr.move(menu_y+1, 0) ; stdscr.clrtoeol() + stdscr.move(menu_y, 0) + stdscr.clrtoeol() + stdscr.move(menu_y+1, 0) + stdscr.clrtoeol() def display_menu(stdscr, menu_y): "Display the menu of possible keystroke commands" @@ -140,18 +145,17 @@ stdscr.addstr(menu_y+1, 4, 'E)rase the board, R)andom fill, S)tep once or C)ontinuously, Q)uit') -def main(stdscr): - +def keyloop(stdscr): # Clear the screen and display the menu of keys stdscr.clear() stdscr_y, stdscr_x = stdscr.getmaxyx() - menu_y=(stdscr_y-3)-1 + menu_y = (stdscr_y-3)-1 display_menu(stdscr, menu_y) # Allocate a subwindow for the Life board and create the board object - subwin=stdscr.subwin(stdscr_y-3, stdscr_x, 0, 0) - board=LifeBoard(subwin, char=ord('*')) - board.display(update_board=0) + subwin = stdscr.subwin(stdscr_y-3, stdscr_x, 0, 0) + board = LifeBoard(subwin, char=ord('*')) + board.display(update_board=False) # xpos, ypos are the cursor's position xpos, ypos = board.X/2, board.Y/2 @@ -159,9 +163,9 @@ # Main loop: while (1): stdscr.move(1+ypos, 1+xpos) # Move the cursor - c=stdscr.getch() # Get a keystroke + c = stdscr.getch() # Get a keystroke if 00: ypos=ypos-1 - elif c==curses.KEY_DOWN and ypos0: xpos=xpos-1 - elif c==curses.KEY_RIGHT and xpos0: ypos -= 1 + elif c == curses.KEY_DOWN and ypos0: xpos -= 1 + elif c == curses.KEY_RIGHT and xpos Author: andrew.kuchling Date: Sun Jun 4 01:02:15 2006 New Revision: 46625 Modified: python/trunk/Demo/curses/rain.py Log: Use True; value returned from main is unused Modified: python/trunk/Demo/curses/rain.py ============================================================================== --- python/trunk/Demo/curses/rain.py (original) +++ python/trunk/Demo/curses/rain.py Sun Jun 4 01:02:15 2006 @@ -48,7 +48,7 @@ ypos[j] = randrange(0, r) + 2 j = 0 - while 1: + while True: x = randrange(0, c) + 2 y = randrange(0, r) + 2 @@ -83,7 +83,7 @@ ch = stdscr.getch() if ch == ord('q') or ch == ord('Q'): - return 0 + return elif ch == ord('s'): stdscr.nodelay(0) elif ch == ord(' '): From python-checkins at python.org Sun Jun 4 01:07:22 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:07:22 +0200 (CEST) Subject: [Python-checkins] r46626 - python/trunk/Demo/curses/tclock.py Message-ID: <20060603230722.135E11E400D@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:07:21 2006 New Revision: 46626 Modified: python/trunk/Demo/curses/tclock.py Log: Use true division, and the True value Modified: python/trunk/Demo/curses/tclock.py ============================================================================== --- python/trunk/Demo/curses/tclock.py (original) +++ python/trunk/Demo/curses/tclock.py Sun Jun 4 01:07:21 2006 @@ -14,7 +14,8 @@ return 1 def A2XY(angle, radius): - return int(round(ASPECT * radius * sin(angle))), int(round(radius * cos(angle))) + return (int(round(ASPECT * radius * sin(angle))), + int(round(radius * cos(angle)))) def plot(x, y, col): stdscr.addch(y, x, col) @@ -37,9 +38,9 @@ y = from_y if ax > ay: - d = ay - ax / 2 + d = ay - ax // 2 - while 1: + while True: plot(x, y, ch) if x == x2: return @@ -50,9 +51,9 @@ x += sx d += ay else: - d = ax - ay / 2 + d = ax - ay // 2 - while 1: + while True: plot(x, y, ch) if y == y2: return @@ -78,12 +79,12 @@ curses.init_pair(2, curses.COLOR_MAGENTA, my_bg) curses.init_pair(3, curses.COLOR_GREEN, my_bg) - cx = (curses.COLS - 1) / 2 - cy = curses.LINES / 2 - ch = min( cy-1, int(cx / ASPECT) - 1) - mradius = (3 * ch) / 4 - hradius = ch / 2 - sradius = 5 * ch / 6 + cx = (curses.COLS - 1) // 2 + cy = curses.LINES // 2 + ch = min( cy-1, int(cx // ASPECT) - 1) + mradius = (3 * ch) // 4 + hradius = ch // 2 + sradius = 5 * ch // 6 for i in range(0, 12): sangle = (i + 1) * 2.0 * pi / 12.0 @@ -96,7 +97,7 @@ sradius = max(sradius-4, 8) - while 1: + while True: curses.napms(1000) tim = time.time() From python-checkins at python.org Sun Jun 4 01:09:58 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:09:58 +0200 (CEST) Subject: [Python-checkins] r46627 - python/trunk/Demo/curses/repeat.py Message-ID: <20060603230958.C2F431E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:09:58 2006 New Revision: 46627 Modified: python/trunk/Demo/curses/repeat.py Log: Docstring fix; use True Modified: python/trunk/Demo/curses/repeat.py ============================================================================== --- python/trunk/Demo/curses/repeat.py (original) +++ python/trunk/Demo/curses/repeat.py Sun Jun 4 01:09:58 2006 @@ -2,7 +2,7 @@ """repeat -This simple program repeatedly (with 1-second intervals) executes the +This simple program repeatedly (at 1-second intervals) executes the shell command given on the command line and displays the output (or as much of it as fits on the screen). It uses curses to paint each new output on top of the old output, so that if nothing changes, the @@ -38,7 +38,7 @@ sys.exit(sts) w = curses.initscr() try: - while 1: + while True: w.erase() try: w.addstr(text) From python-checkins at python.org Sun Jun 4 01:15:56 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:15:56 +0200 (CEST) Subject: [Python-checkins] r46628 - python/trunk/Demo/zlib/zlibdemo.py Message-ID: <20060603231556.C0D571E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:15:56 2006 New Revision: 46628 Modified: python/trunk/Demo/zlib/zlibdemo.py Log: Put code in a main() function; loosen up the spacing to match current code style Modified: python/trunk/Demo/zlib/zlibdemo.py ============================================================================== --- python/trunk/Demo/zlib/zlibdemo.py (original) +++ python/trunk/Demo/zlib/zlibdemo.py Sun Jun 4 01:15:56 2006 @@ -1,35 +1,48 @@ #!/usr/bin/env python +# Takes an optional filename, defaulting to this file itself. +# Reads the file and compresses the content using level 1 and level 9 +# compression, printing a summary of the results. + import zlib, sys -if len(sys.argv)>1: filename=sys.argv[1] -else: filename='zlibdemo.py' -print 'Reading', filename -f=open(filename, 'r') # Get the data to compress -s=f.read() -f.close() - -# First, we'll compress the string in one step -comptext=zlib.compress(s, 1) -decomp=zlib.decompress(comptext) - -print '1-step compression: (level 1)' -print ' Original:', len(s), 'Compressed:', len(comptext), -print 'Uncompressed:', len(decomp) - -# Now, let's compress the string in stages; set chunk to work in smaller steps - -chunk=256 -compressor=zlib.compressobj(9) -decompressor=zlib.decompressobj() -comptext=decomp='' -for i in range(0, len(s), chunk): - comptext=comptext+compressor.compress(s[i:i+chunk]) -comptext=comptext+compressor.flush() # Don't forget to call flush()!! - -for i in range(0, len(comptext), chunk): - decomp=decomp+decompressor.decompress(comptext[i:i+chunk]) -decomp=decomp+decompressor.flush() - -print 'Progressive compression (level 9):' -print ' Original:', len(s), 'Compressed:', len(comptext), -print 'Uncompressed:', len(decomp) + +def main(): + if len(sys.argv) > 1: + filename = sys.argv[1] + else: + filename = sys.argv[0] + print 'Reading', filename + + f = open(filename, 'rb') # Get the data to compress + s = f.read() + f.close() + + # First, we'll compress the string in one step + comptext = zlib.compress(s, 1) + decomp = zlib.decompress(comptext) + + print '1-step compression: (level 1)' + print ' Original:', len(s), 'Compressed:', len(comptext), + print 'Uncompressed:', len(decomp) + + # Now, let's compress the string in stages; set chunk to work in smaller steps + + chunk = 256 + compressor = zlib.compressobj(9) + decompressor = zlib.decompressobj() + comptext = decomp = '' + for i in range(0, len(s), chunk): + comptext = comptext+compressor.compress(s[i:i+chunk]) + # Don't forget to call flush()!! + comptext = comptext + compressor.flush() + + for i in range(0, len(comptext), chunk): + decomp = decomp + decompressor.decompress(comptext[i:i+chunk]) + decomp=decomp+decompressor.flush() + + print 'Progressive compression (level 9):' + print ' Original:', len(s), 'Compressed:', len(comptext), + print 'Uncompressed:', len(decomp) + +if __name__ == '__main__': + main() From python-checkins at python.org Sun Jun 4 01:39:08 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:39:08 +0200 (CEST) Subject: [Python-checkins] r46629 - python/trunk/Demo/zlib/minigzip.py Message-ID: <20060603233908.222171E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:39:07 2006 New Revision: 46629 Modified: python/trunk/Demo/zlib/minigzip.py Log: Use functions; modernize code Modified: python/trunk/Demo/zlib/minigzip.py ============================================================================== --- python/trunk/Demo/zlib/minigzip.py (original) +++ python/trunk/Demo/zlib/minigzip.py Sun Jun 4 01:39:07 2006 @@ -1,106 +1,133 @@ #!/usr/bin/env python # Demo program for zlib; it compresses or decompresses files, but *doesn't* # delete the original. This doesn't support all of gzip's options. +# +# The 'gzip' module in the standard library provides a more complete +# implementation of gzip-format files. + +import zlib, sys, os FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16 def write32(output, value): - output.write(chr(value & 255)) ; value=value / 256 - output.write(chr(value & 255)) ; value=value / 256 - output.write(chr(value & 255)) ; value=value / 256 + output.write(chr(value & 255)) ; value=value // 256 + output.write(chr(value & 255)) ; value=value // 256 + output.write(chr(value & 255)) ; value=value // 256 output.write(chr(value & 255)) def read32(input): - v=ord(input.read(1)) - v=v+ (ord(input.read(1))<<8 ) - v=v+ (ord(input.read(1))<<16) - v=v+ (ord(input.read(1))<<24) + v = ord(input.read(1)) + v += (ord(input.read(1)) << 8 ) + v += (ord(input.read(1)) << 16) + v += (ord(input.read(1)) << 24) return v -import zlib, sys -if len(sys.argv)!=2: - print 'Usage: minigzip.py ' - print ' The file will be compressed or decompressed.' - sys.exit(0) - -filename=sys.argv[1] -compressing=1 ; outputname=filename+'.gz' -if filename[-3:]=='.gz': - compressing=0 ; outputname=filename[:-3] -input=open(filename) ; output=open(outputname, 'w') - -if compressing: +def compress (filename, input, output): output.write('\037\213\010') # Write the header, ... output.write(chr(FNAME)) # ... flag byte ... - import os # ... modification time ... - statval=os.stat(filename) - mtime=statval[8] + statval = os.stat(filename) # ... modification time ... + mtime = statval[8] write32(output, mtime) output.write('\002') # ... slowest compression alg. ... output.write('\377') # ... OS (=unknown) ... output.write(filename+'\000') # ... original filename ... - crcval=zlib.crc32("") - compobj=zlib.compressobj(9, zlib.DEFLATED, -zlib.MAX_WBITS, + crcval = zlib.crc32("") + compobj = zlib.compressobj(9, zlib.DEFLATED, -zlib.MAX_WBITS, zlib.DEF_MEM_LEVEL, 0) - while (1): - data=input.read(1024) - if data=="": break - crcval=zlib.crc32(data, crcval) + while True: + data = input.read(1024) + if data == "": + break + crcval = zlib.crc32(data, crcval) output.write(compobj.compress(data)) output.write(compobj.flush()) write32(output, crcval) # ... the CRC ... write32(output, statval[6]) # and the file size. -else: - magic=input.read(2) - if magic!='\037\213': - print 'Not a gzipped file' ; sys.exit(0) - if ord(input.read(1))!=8: - print 'Unknown compression method' ; sys.exit(0) - flag=ord(input.read(1)) +def decompress (input, output): + magic = input.read(2) + if magic != '\037\213': + print 'Not a gzipped file' + sys.exit(0) + if ord(input.read(1)) != 8: + print 'Unknown compression method' + sys.exit(0) + flag = ord(input.read(1)) input.read(4+1+1) # Discard modification time, # extra flags, and OS byte. if flag & FEXTRA: # Read & discard the extra field, if present - xlen=ord(input.read(1)) - xlen=xlen+256*ord(input.read(1)) + xlen = ord(input.read(1)) + xlen += 256*ord(input.read(1)) input.read(xlen) if flag & FNAME: # Read and discard a null-terminated string containing the filename - while (1): - s=input.read(1) - if s=='\000': break + while True: + s = input.read(1) + if s == '\0': break if flag & FCOMMENT: # Read and discard a null-terminated string containing a comment - while (1): + while True: s=input.read(1) - if s=='\000': break + if s=='\0': break if flag & FHCRC: input.read(2) # Read & discard the 16-bit header CRC - decompobj=zlib.decompressobj(-zlib.MAX_WBITS) - crcval=zlib.crc32("") - length=0 - while (1): + + decompobj = zlib.decompressobj(-zlib.MAX_WBITS) + crcval = zlib.crc32("") + length = 0 + while True: data=input.read(1024) - if data=="": break - decompdata=decompobj.decompress(data) - print len(decompdata) - output.write(decompdata) ; length=length+len(decompdata) - crcval=zlib.crc32(decompdata, crcval) - decompdata=decompobj.flush() - output.write(decompdata) ; length=length+len(decompdata) - crcval=zlib.crc32(decompdata, crcval) + if data == "": + break + decompdata = decompobj.decompress(data) + output.write(decompdata) + length += len(decompdata) + crcval = zlib.crc32(decompdata, crcval) + + decompdata = decompobj.flush() + output.write(decompdata) + length += len(decompdata) + crcval = zlib.crc32(decompdata, crcval) # We've read to the end of the file, so we have to rewind in order # to reread the 8 bytes containing the CRC and the file size. The # decompressor is smart and knows when to stop, so feeding it # extra data is harmless. input.seek(-8, 2) - crc32=read32(input) - isize=read32(input) - if crc32!=crcval: print 'CRC check failed.' - if isize!=length: print 'Incorrect length of data produced' + crc32 = read32(input) + isize = read32(input) + if crc32 != crcval: + print 'CRC check failed.' + if isize != length: + print 'Incorrect length of data produced' + +def main(): + if len(sys.argv)!=2: + print 'Usage: minigzip.py ' + print ' The file will be compressed or decompressed.' + sys.exit(0) + + filename = sys.argv[1] + if filename.endswith('.gz'): + compressing = False + outputname = filename[:-3] + else: + compressing = True + outputname = filename + '.gz' + + input = open(filename, 'rb') + output = open(outputname, 'wb') + + if compressing: + compress(filename, input, output) + else: + decompress(input, output) + + input.close() + output.close() -input.close() ; output.close() +if __name__ == '__main__': + main() From python-checkins at python.org Sun Jun 4 01:43:22 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:43:22 +0200 (CEST) Subject: [Python-checkins] r46630 - python/trunk/Demo/xmlrpc/xmlrpc_handler.py Message-ID: <20060603234322.DB0BE1E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:43:22 2006 New Revision: 46630 Removed: python/trunk/Demo/xmlrpc/xmlrpc_handler.py Log: This demo requires Medusa (not just asyncore); remove it Deleted: /python/trunk/Demo/xmlrpc/xmlrpc_handler.py ============================================================================== --- /python/trunk/Demo/xmlrpc/xmlrpc_handler.py Sun Jun 4 01:43:22 2006 +++ (empty file) @@ -1,104 +0,0 @@ -# -# XML-RPC SERVER -# $Id$ -# -# an asynchronous XML-RPC server for Medusa -# -# written by Sam Rushing -# -# Based on "xmlrpcserver.py" by Fredrik Lundh (fredrik at pythonware.com) -# - -import http_server -import xmlrpclib - -import sys - -class xmlrpc_handler: - - def match (self, request): - # Note: /RPC2 is not required by the spec, so you may override this method. - if request.uri[:5] == '/RPC2': - return 1 - else: - return 0 - - def handle_request (self, request): - [path, params, query, fragment] = request.split_uri() - - if request.command.lower() in ('post', 'put'): - request.collector = collector (self, request) - else: - request.error (400) - - def continue_request (self, data, request): - params, method = xmlrpclib.loads (data) - try: - # generate response - try: - response = self.call (method, params) - response = (response,) - except: - # report exception back to server - response = xmlrpclib.dumps ( - xmlrpclib.Fault (1, "%s:%s" % sys.exc_info()[:2]) - ) - else: - response = xmlrpclib.dumps (response, methodresponse=1) - except: - # internal error, report as HTTP server error - request.error (500) - else: - # got a valid XML RPC response - request['Content-Type'] = 'text/xml' - request.push (response) - request.done() - - def call (self, method, params): - # override this method to implement RPC methods - raise "NotYetImplemented" - -class collector: - - "gathers input for POST and PUT requests" - - def __init__ (self, handler, request): - - self.handler = handler - self.request = request - self.data = '' - - # make sure there's a content-length header - cl = request.get_header ('content-length') - - if not cl: - request.error (411) - else: - cl = int (cl) - # using a 'numeric' terminator - self.request.channel.set_terminator (cl) - - def collect_incoming_data (self, data): - self.data = self.data + data - - def found_terminator (self): - # set the terminator back to the default - self.request.channel.set_terminator ('\r\n\r\n') - self.handler.continue_request (self.data, self.request) - -if __name__ == '__main__': - - class rpc_demo (xmlrpc_handler): - - def call (self, method, params): - print 'method="%s" params=%s' % (method, params) - return "Sure, that works" - - import asyncore - import http_server - - hs = http_server.http_server ('', 8000) - rpc = rpc_demo() - hs.install_handler (rpc) - - asyncore.loop() From python-checkins at python.org Sun Jun 4 01:46:37 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:46:37 +0200 (CEST) Subject: [Python-checkins] r46631 - python/trunk/Demo/xmlrpc Message-ID: <20060603234637.08EFC1E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:46:36 2006 New Revision: 46631 Removed: python/trunk/Demo/xmlrpc/ Log: Remove xmlrpc demo -- it duplicates the SimpleXMLRPCServer module. From python-checkins at python.org Sun Jun 4 01:47:25 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:47:25 +0200 (CEST) Subject: [Python-checkins] r46632 - python/trunk/Demo/README Message-ID: <20060603234725.473021E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:47:22 2006 New Revision: 46632 Modified: python/trunk/Demo/README Log: Remove xmlrpc/ directory Modified: python/trunk/Demo/README ============================================================================== --- python/trunk/Demo/README (original) +++ python/trunk/Demo/README Sun Jun 4 01:47:22 2006 @@ -57,8 +57,5 @@ xml Some XML demos. -xmlrpc XML-RPC server framework (but see the standard library - module SimpleXMLRPCServer.py for a replacement). - zlib Some demos for the zlib module (see also the standard library module gzip.py). From thomas at python.org Sun Jun 4 01:51:13 2006 From: thomas at python.org (Thomas Wouters) Date: Sun, 4 Jun 2006 01:51:13 +0200 Subject: [Python-checkins] r46603 - python/trunk/Lib/test/test_struct.py In-Reply-To: References: <20060602130346.9071C1E400B@bag.python.org> Message-ID: <9e804ac0606031651q56199989ra8662d9e57ba8660@mail.gmail.com> On 6/3/06, Neal Norwitz wrote: > > On 6/2/06, martin.blais wrote: > > Author: martin.blais > > Date: Fri Jun 2 15:03:43 2006 > > New Revision: 46603 > > > > Modified: > > python/trunk/Lib/test/test_struct.py > > Log: > > Fixed struct test to not use unittest. > > Shoot, I had hoped you would go the other way and convert all the old > test cases to use unittest. :-) Heh, my reaction was both "huh?" and "yay" at the same time. I thought unittest was still preferred for the stdlib testsuite? I don't particularly see the benefits of unittest myself, although I admit it's a _tad_ cleaner than most of the non-unittest scripts the stdlib had before. However, having written a few tests with py.test, in particular test-generators, well, unittest's just cumbersome ;-P I'm not arguing for (or against) py.test, just wondering what the support and opposition for changing the policy would be like ;-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060604/bd1484dc/attachment.htm From python-checkins at python.org Sun Jun 4 01:51:22 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:51:22 +0200 (CEST) Subject: [Python-checkins] r46633 - python/trunk/Demo/sockets/README Message-ID: <20060603235122.465D51E400E@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:51:21 2006 New Revision: 46633 Modified: python/trunk/Demo/sockets/README Log: Remove dangling reference Modified: python/trunk/Demo/sockets/README ============================================================================== --- python/trunk/Demo/sockets/README (original) +++ python/trunk/Demo/sockets/README Sun Jun 4 01:51:21 2006 @@ -19,4 +19,3 @@ /usr/people/4Dgifts/examples/network/mcast.c (Note that IN.py is in ../../lib/sgi.) -See also ../../lib/nntp.py for another example of socket code. From buildbot at python.org Sun Jun 4 01:53:32 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 03 Jun 2006 23:53:32 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060603235332.6FB901E4003@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/37 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 01:59:36 2006 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 4 Jun 2006 01:59:36 +0200 (CEST) Subject: [Python-checkins] r46634 - python/trunk/Demo/sockets/unixclient.py python/trunk/Demo/sockets/unixserver.py Message-ID: <20060603235936.6BC361E4003@bag.python.org> Author: andrew.kuchling Date: Sun Jun 4 01:59:36 2006 New Revision: 46634 Modified: python/trunk/Demo/sockets/unixclient.py python/trunk/Demo/sockets/unixserver.py Log: Add more whitespace; use a better socket name Modified: python/trunk/Demo/sockets/unixclient.py ============================================================================== --- python/trunk/Demo/sockets/unixclient.py (original) +++ python/trunk/Demo/sockets/unixclient.py Sun Jun 4 01:59:36 2006 @@ -1,7 +1,9 @@ # Echo client demo using Unix sockets # Piet van Oostrum + from socket import * -FILE = 'blabla' + +FILE = 'unix-socket' s = socket(AF_UNIX, SOCK_STREAM) s.connect(FILE) s.send('Hello, world') Modified: python/trunk/Demo/sockets/unixserver.py ============================================================================== --- python/trunk/Demo/sockets/unixserver.py (original) +++ python/trunk/Demo/sockets/unixserver.py Sun Jun 4 01:59:36 2006 @@ -1,17 +1,24 @@ # Echo server demo using Unix sockets (handles one connection only) # Piet van Oostrum + import os from socket import * -FILE = 'blabla' + +FILE = 'unix-socket' s = socket(AF_UNIX, SOCK_STREAM) s.bind(FILE) + print 'Sock name is: ['+s.getsockname()+']' + +# Wait for a connection s.listen(1) conn, addr = s.accept() -print 'Connected by', addr -while 1: + +while True: data = conn.recv(1024) - if not data: break + if not data: + break conn.send(data) + conn.close() os.unlink(FILE) From python-checkins at python.org Sun Jun 4 03:22:53 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 4 Jun 2006 03:22:53 +0200 (CEST) Subject: [Python-checkins] r46635 - python/trunk/Lib/test/test_struct.py Message-ID: <20060604012253.C42041E4003@bag.python.org> Author: tim.peters Date: Sun Jun 4 03:22:53 2006 New Revision: 46635 Modified: python/trunk/Lib/test/test_struct.py Log: Whitespace normalization. Modified: python/trunk/Lib/test/test_struct.py ============================================================================== --- python/trunk/Lib/test/test_struct.py (original) +++ python/trunk/Lib/test/test_struct.py Sun Jun 4 03:22:53 2006 @@ -571,9 +571,8 @@ assertRaises(struct.error, pack_to, small_buf, 0, test_string) assertRaises(struct.error, pack_to, small_buf, 2, test_string) - + # Test methods to pack and unpack from buffers rather than strings. test_unpack_from() test_pack_to() test_pack_to_fn() - From python-checkins at python.org Sun Jun 4 03:33:33 2006 From: python-checkins at python.org (matt.fleming) Date: Sun, 4 Jun 2006 03:33:33 +0200 (CEST) Subject: [Python-checkins] r46636 - sandbox/trunk/pdb/mpdb.py Message-ID: <20060604013333.478721E4003@bag.python.org> Author: matt.fleming Date: Sun Jun 4 03:33:31 2006 New Revision: 46636 Modified: sandbox/trunk/pdb/mpdb.py Log: Trying to follow gdb's command set as closely as possible. Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Sun Jun 4 03:33:31 2006 @@ -3,224 +3,273 @@ # This is a Google Summer of Code project # Student: Matthew J. Fleming # Mentor: Robert L. Bernstein +""" +This module provides improvements over the Python Debugger (Pdb). +This module allows, + +- debugging of applications running in a separate process to the debugger +- debugging of applications on a remote machine +- debugging of threaded applications. + +""" -import cmd import os from optparse import OptionParser import pdb import socket import sys import traceback -import thread # Need custom safe Repr just like pdb _saferepr = pdb._repr.repr line_prefix = '\n-> ' class MPdb(pdb.Pdb): - def __init__(self): - pdb.Pdb.__init__(self) - self.fd = sys.stdout # The file descriptor we're writing output to + """ This class extends the command set and functionality of the + Python debugger and provides support for, + + - debugging separate processes + - debugging applications on remote machines + - debugging threaded applications + """ + def __init__(self, completekey='tab', stdin=None, stdout=None): + """ Instantiate a debugger. + + The optional argument 'completekey' is the readline name of a + completion key; it defaults to the Tab key. If completekey is + not None and the readline module is available, command completion + is done automatically. The optional arguments stdin and stdout + specify alternate input and output file objects; if not specified, + sys.stdin and sys.stdout are used. + """ + pdb.Pdb.__init__(self, completekey, stdin, stdout) self.prompt = '(MPdb)' - def read(self): - """ Read a line from our input stream/socket. """ - try: - line = raw_input(self.prompt) - except EOFError: - line = 'EOF' - return line - -class DebuggerConsoleConnected(Exception): pass - -# Below are the classes that allow MPdb instances to be debugged locally and -# remotely by a debugger running in a different process -class MPdbServer(MPdb): - def __init__(self): - MPdb.__init__(self) - self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.debugger_list = [] - self.attached = False + def _rebind_input(self, new_input): + """ This method rebinds the debugger's input to the object specified + by 'new_input'. + """ + self.raw_input = 1 + print >> sys.stderr, "From %s to %s" % (self.stdin, new_input) + self.stdin.flush() + self.stdin = new_input + self.stdin.flush() + + def _rebind_output(self, new_output): + """ This method rebinds the debugger's output to the object specified + by 'new_output'. + """ + self.stdout.flush() + self.stdout = new_output + self.stdout.flush() + + # Debugger commands + def do_attach(self, addr): + """ Attach to a process or file outside of Pdb. +This command attaches to another target, of the same type as your last +"target" command. The command may take as argument a process id or a +device file. For a process id, you must have permission to send the +process a signal, and it must have the same effective uid as the debugger. +When using "attach" with a process id, the debugger finds the +program running in the process, looking first in the current working +directory, or (if not found there) using the source file search path +(see the "directory" command). You can also use the "file" command +to specify the program, and to load its symbol table. +""" + + def do_target(self, args): + """ Connect to a target machine or process. +The first argument is the type or protocol of the target machine +(which can be the name of a class that is avaible either in the current +working directory or in Python's PYTHONPATH environtment variable). +Remaining arguments are interpreted by the target protocol. For more +information on the arguments for a particular protocol, type +`help target ' followed by the protocol name. + +List of target subcommands: + +target serial -- Use a remote computer via a serial line +target tcp -- Use a remote computer via a TCP connection +target udp -- Use a remote computer via a UDP connection +target xml -- Use a remote computer via the xmlrpc lib +""" + cls, addr = args.split(' ') + self.connection = eval(cls+'(addr)') + self.connection.setup() + # XXX currently this method doesn't do anything + + def do_serve(self, args): + """ Allow a debugger to connect to this session. +The first argument is the type or protocol that is used for this connection +(which can be the name of a class that is avaible either in the current +working directory or in Python's PYTHONPATH environtment variable). +Remaining arguments are interpreted by the protocol. For more +information on the arguments for a particular protocol, type +`help target ' followed by the protocol name. +""" + cls, addr = args.split(' ') + self.connection = eval(cls+'(addr)') + self.connection.setup() + self._rebind_output(self.connection.output) + +NotImplementedMessage = "This method must be overriden in a subclass" + +class MTargetInterface(object): + """ This is an abstract class that specifies the interface a debugging + target class must implement. + """ + def accept(self, console, addr): + """ This method is called when a connection from a debugger + is accepted by this target. + """ + raise NotImplementedError, NotImplementedMessage + + def disconnect(self): + """ This method is called to disconnect any debuggers that + are connected and to stop accepting any more connections from + debuggers. + """ + raise NotImplementedError, NotImplementedMessage + + def listen(self): + """ This method is called when a target is initially + configured to listen for connections from debuggers. + """ + raise NotImplementedError, NotImplementedMessage + +class MTargetConnectionTCP(MTargetInterface): + """ This is an implementation of a target class that uses the TCP + protocol as its means of communication. Debuggers wishing to connect + to this target must use this syntax for the target command, - def listen(self, port): - self.port = port - self._sock.bind((self.host, self.port)) - self._sock.listen(1) # Only one debugger is allowed control - self.thread_handle = thread.start_new_thread(self.accept, ()) + `target tcp hostname:port - def accept(self): - c, a = self._sock.accept() - print "\n\nDebugger attached..\n" - self.debugger_list.append((c,a)) - self.fd = self.debugger_list[-1][0].makefile("w") - self.attached = True - # We've got a debugger console attached so we hijack stdout - self.old_stdout = os.dup(1) - os.close(1) - sys.stdout = self.fd - # XXX Need a way to enter the current 'waiting' on stdin for input - # and start reading from the debugger console + """ + def __init__(self, addr): + """ 'addr' specifies the hostname and port combination of + the target. + """ + MTargetInterface.__init__(self) + h,p = addr.split(':') + self.host = h + self.port = int(p) - def cmdloop(self): - self.preloop() - stop = None - while not stop: - if self.cmdqueue: - line = self.cmdqueue.pop(0) - else: - line = self.read() - if not len(line): - line = 'EOF' - line = self.precmd(line) - stop = self.onecmd(line) - stop = self.postcmd(stop, line) - # Force stdout flush its contents - sys.stdout.flush() - self.postloop() - - def read(self): - # We need to check whether we've actually been attached to yet - if self.attached: - line = self.debugger_list[-1][0].recv(1024) - return line - else: - line = raw_input(self.prompt) - return line - -class LMPdbServer(MPdbServer): - """ A local debugee which can only communicate with a debugger running - on the localhost. - """ - def __init__(self): - MPdbServer.__init__(self) - self.host = 'localhost' - -class MPdbConsole(cmd.Cmd): - def __init__(self): - cmd.Cmd.__init__(self, stdin=None, stdout=None) + + def setup(self): self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.attached = False - self.prompt = '(MPdb)' - self.lastcmd = "" + self._sock.bind((self.host, self.port)) + self.listen() - def connect(self, host, port): - self._sock.connect((host, port)) - self.th_r = thread.start_new_thread(self.recv, ()) - - def do_attach(self, args): - if self.attached: - print "Already attached to a debugger.." - return None - h = args.split(" ") - self.connect(h[0], int(h[1])) - print "Attaching..\n" - self.attached = True - - def help_attach(self): - print """attach [host] [port] - Attach to a debugging session at [host] on [port].""" - - def cmdloop(self): - """ Repeatedly issue a prompt and accept input from a debugger - console. All commands (apart from the 'attach' and 'help' commands) - are sent to an attached debugee. - """ - print self.prompt, - stop = None - while not stop: - line = self.input() - if not len(line): - line = 'EOF' - if self.attached: - self.send(line) - else: - if 'attach' in line: - line = line.split(' ') - line = line[1:] - line = line[0] + " " + line[1] - self.do_attach(line) - elif 'help' in line: - self.help_attach() - else: - print "Not currently attached to a debugger..\n" - continue - self.postloop() - - def send(self, line): - """ Send the line to the debugee server for parsing and execution. """ - self._sock.send(line) + def listen(self): + self._sock.listen(5) + debugger, addr = self._sock.accept() + self.accept(debugger, addr) + + def accept(self, debugger, addr): + self.output = debugger.makefile('w') + + def disconnect(self): + self.output.flush() + self.output.close() + self._sock.close() + +class MTargetConnectionSerial(MTargetInterface): + """ This is a target connection class that allows a debugger + to connect via a serial line. A serial device must be specified + (e.g. /dev/ttyS0, COM1, etc.). + """ + def __init__(self, device): + self._dev + MTargetInterface.__init__(self) + + def setup(self): + self.output = open(self._dev, 'w') + self.input = open(self._dev, 'r') + + def listen(self): + pass + + def accept(self, debugger, addr): + pass + + def disconnect(self): + self.output.close() + self.input.close() - def input(self): - """ This method will be changed later. """ - line = raw_input() - # Hitting enter repeatedly should execute the last command - if line is not '': - self.lastcmd = line - elif line is '': - if self.lastcmd is '': - print "No previous command.. \n" - line = 'EOF' - else: - line = self.lastcmd - return line - - def recv(self): - """ Receive all data being sent from the debugee. """ - while 1: - data = self._sock.recv(1024) - if data: - print data, - print self.prompt, +class MDebuggerConnectionTCP(object): + """ A class that allows a connection to be made from a debugger + to a target via TCP. Specify the address of the target + (e.g. host:2020). + """ + def __init__(self, addr): + """ Specify the address to connection to. """ + h, p = addr.split(':') + self.host = h + self.port = int(p) + self.setup() - def close(self): - self._sock.close() + def setup(self): + """ Connect to the target. """ + self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self._sock.connect((self.host, self.port)) + +class MDebuggerConnectionSerial(object): + """ A class that allows a connection to be made from a debugger + to a target via a serial line. Specify the serial device it is + connected to (e.g. /dev/ttya). + """ + def __init__(self, device): + """ Specify the serial device. """ + self._dev = device + self.input = None + self.output = None + self.setup() + + def setup(self): + """ Create our fileobject by opening the serial device for + this connection. + """ + self.output = open(self._dev, "w") + +# These are the default classes that are instantiated when connecting +# to a target, +# +# `target tcp localhost:8080 ' +# +# These can be changed at runtime. + +serial = MDebuggerConnectionSerial +tcp = MDebuggerConnectionTCP def main(options): opts = options[0] args = options[1] - if not args: - print 'usage: mpdb.py scriptfile [arg] ... ' - sys.exit(1) - mainpyfile = args[0] - if not os.path.exists(mainpyfile): - print 'Error:', mainpyfile, 'does not exist' - sys.exit(1) - if opts.debugger: - # We may be local or remote but we're still a debugger console - # This console acts differently to everything else, for now, it's just - # a glorified telnet client that works _properly_ with our server - mpdb = MPdbConsole() - # All we can actually do is issue commands and receieve the result - mpdb.cmdloop() - else: - if opts.local_debugee: - mpdb = LMPdbServer() - mpdb.listen(7000) - elif opts.remote_debugee: - pass - else: - # Regular interactive debugger session - mpdb = MPdb() - while 1: - try: - mpdb._runscript(mainpyfile) - if mpdb._user_requested_quit: - break - print "The program finished and will be restarted" - except SystemExit: - # In most cases SystemExit does not warrant a post-mortem session. - print "The program exited via sys.exit(). " + \ - "Exit status:",sys.exc_info()[1] - except: - print traceback.format_exc() - print "Uncaught exception. Entering post mortem debugging" - print "Running 'cont' or 'step' will restart the program" - t = sys.exc_info()[2] - while t.tb_next is not None: - t = t.tb_next - mpdb.interaction(t.tb_frame,t) - print "Post mortem debugger finished. The " + \ - mainpyfile + " will be restarted" + if args: + mainpyfile = args[0] + if not os.path.exists(mainpyfile): + print 'Error:', mainpyfile, 'does not exist' + sys.exit(1) + mpdb = MPdb() + while 1: + try: + mpdb._runscript(mainpyfile) + if mpdb._user_requested_quit: + break + print "The program finished and will be restarted" + except SystemExit: + # In most cases SystemExit does not warrant a post-mortem session. + print "The program exited via sys.exit(). " + \ + "Exit status:",sys.exc_info()[1] + except: + print traceback.format_exc() + print "Uncaught exception. Entering post mortem debugging" + print "Running 'cont' or 'step' will restart the program" + t = sys.exc_info()[2] + while t.tb_next is not None: + t = t.tb_next + mpdb.interaction(t.tb_frame,t) + print "Post mortem debugger finished. The " + \ + mainpyfile + " will be restarted" # A utility function to parse the options from the command line def parse_opts(): From tim.peters at gmail.com Sun Jun 4 03:39:59 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 3 Jun 2006 21:39:59 -0400 Subject: [Python-checkins] r46603 - python/trunk/Lib/test/test_struct.py In-Reply-To: <9e804ac0606031651q56199989ra8662d9e57ba8660@mail.gmail.com> References: <20060602130346.9071C1E400B@bag.python.org> <9e804ac0606031651q56199989ra8662d9e57ba8660@mail.gmail.com> Message-ID: <1f7befae0606031839t32ca5ddch361c2074fc81f20f@mail.gmail.com> >>> Author: martin.blais >>> Date: Fri Jun 2 15:03:43 2006 >>> New Revision: 46603 >>> >>> Modified: >>> python/trunk/Lib/test/test_struct.py >>> Log: >>> Fixed struct test to not use unittest. [Neal Norwitz] >> Shoot, I had hoped you would go the other way and convert all the old >> test cases to use unittest. :-) [Thomas Wouters] > Heh, my reaction was both "huh?" and "yay" at the same time. I thought > unittest was still preferred for the stdlib testsuite? That or doctest. What's definitely unwanted now is the original style of test that compares the whole test's output to an expected-output file in Lib/test/output/. Those can be horrid to figure out when they go wrong, and don't check at all when running regrtest with -v. unittest, and especially doctest, encourage breaking tests into small units. An example of neither is test_descr.py, which can be a real bitch to untangle when it fails. > I don't particularly see the benefits of unittest myself, although I admit it's a _tad_ > cleaner than most of the non-unittest scripts the stdlib had before. However, having > written a few tests with py.test, in particular test-generators, well, unittest's just > cumbersome ;-P I haven't used py.test, but I'm sure it's not used at all in Python's test suite. Don't know what "test-generators" refers to -- Python's Lib/test/test_generators.py is a doctest. > I'm not arguing for (or against) py.test, just wondering what the support > and opposition for changing the policy would be like ;-) py.test doesn't ship with Python, so we certainly can't use py.test in Python's standard test suite unless/until py.test does ship with Python. From buildbot at python.org Sun Jun 4 04:33:00 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 02:33:00 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060604023300.9E2AB1E4003@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/880 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 05:10:39 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 03:10:39 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060604031039.F253E1E4003@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/273 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 05:26:05 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 4 Jun 2006 05:26:05 +0200 (CEST) Subject: [Python-checkins] r46637 - in python/trunk: Misc/NEWS Misc/SpecialBuilds.txt Objects/obmalloc.c Message-ID: <20060604032605.0A2FA1E4003@bag.python.org> Author: tim.peters Date: Sun Jun 4 05:26:02 2006 New Revision: 46637 Modified: python/trunk/Misc/NEWS python/trunk/Misc/SpecialBuilds.txt python/trunk/Objects/obmalloc.c Log: In a PYMALLOC_DEBUG build obmalloc adds extra debugging info to each allocated block. This was using 4 bytes for each such piece of info regardless of platform. This didn't really matter before (proof: no bug reports, and the debug-build obmalloc would have assert-failed if it was ever asked for a chunk of memory >= 2**32 bytes), since container indices were plain ints. But after the Py_ssize_t changes, it's at least theoretically possible to allocate a list or string whose guts exceed 2**32 bytes, and the PYMALLOC_DEBUG routines would fail then (having only 4 bytes to record the originally requested size). Now we use sizeof(size_t) bytes for each of a PYMALLOC_DEBUG build's extra debugging fields. This won't make any difference on 32-bit boxes, but will add 16 bytes to each allocation in a debug build on a 64-bit box. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sun Jun 4 05:26:02 2006 @@ -464,7 +464,13 @@ Note: Codec packages should implement and register their own codec search function. PEP 100 has the details. -- PEP 353: Using ssize_t as the index type. +- PEP 353: Using ``Py_ssize_t`` as the index type. + +- ``PYMALLOC_DEBUG`` builds now add ``4*sizeof(size_t)`` bytes of debugging + info to each allocated block, since the ``Py_ssize_t`` changes (PEP 353) + now allow Python to make use of memory blocks exceeding 2**32 bytes for + some purposes on 64-bit boxes. A ``PYMALLOC_DEBUG`` build was limited + to 4-byte allocations before. - Patch #1400181, fix unicode string formatting to not use the locale. This is how string objects work. u'%f' could use , instead of . Modified: python/trunk/Misc/SpecialBuilds.txt ============================================================================== --- python/trunk/Misc/SpecialBuilds.txt (original) +++ python/trunk/Misc/SpecialBuilds.txt Sun Jun 4 05:26:02 2006 @@ -96,16 +96,16 @@ Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit ASCII strings. -8 bytes are added at each end of each block of N bytes requested. The -memory layout is like so, where p represents the address returned by a -malloc-like or realloc-like function (p[i:j] means the slice of bytes -from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment -of negative indices differs from a Python slice): - -p[-8:-4] - Number of bytes originally asked for. 4-byte unsigned integer, - big-endian (easier to read in a memory dump). -p[-4:0] +Let S = sizeof(size_t). 2*S bytes are added at each end of each block of N +bytes requested. The memory layout is like so, where p represents the +address returned by a malloc-like or realloc-like function (p[i:j] means +the slice of bytes from *(p+i) inclusive up to *(p+j) exclusive; note that +the treatment of negative indices differs from a Python slice): + +p[-2*S:-S] + Number of bytes originally asked for. This is a size_t, big-endian + (easier to read in a memory dump). +p[-S:0] Copies of FORBIDDENBYTE. Used to catch under- writes and reads. p[0:N] The requested memory, filled with copies of CLEANBYTE, used to catch @@ -116,12 +116,12 @@ DEADBYTE, to catch reference to freed memory. When a realloc- like function is called requesting a smaller memory block, the excess old bytes are also filled with DEADBYTE. -p[N:N+4] +p[N:N+S] Copies of FORBIDDENBYTE. Used to catch over- writes and reads. -p[N+4:N+8] +p[N+S:N+2*S] A serial number, incremented by 1 on each call to a malloc-like or realloc-like function. - 4-byte unsigned integer, big-endian. + Big-endian size_t. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. The static function @@ -145,6 +145,10 @@ If this envar exists, a report of pymalloc summary statistics is printed to stderr whenever a new arena is allocated, and also by Py_Finalize(). + +Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t). +Before it was 16 on all boxes, reflecting that Python couldn't make use of +allocations >= 2**32 bytes even on 64-bit boxes before 2.5. --------------------------------------------------------------------------- Py_DEBUG introduced in 1.5 named DEBUG before 1.5 @@ -251,7 +255,7 @@ find the manual for your specific processor. For the 750CX, 750CXe and 750FX (all sold as the G3) we find: - The time base counter is clocked at a frequency that is + The time base counter is clocked at a frequency that is one-fourth that of the bus clock. This build is enabled by the --with-tsc flag to configure. Modified: python/trunk/Objects/obmalloc.c ============================================================================== --- python/trunk/Objects/obmalloc.c (original) +++ python/trunk/Objects/obmalloc.c Sun Jun 4 05:26:02 2006 @@ -491,13 +491,13 @@ #define INITIAL_ARENA_OBJECTS 16 /* Number of arenas allocated that haven't been free()'d. */ -static ulong narenas_currently_allocated = 0; +static size_t narenas_currently_allocated = 0; #ifdef PYMALLOC_DEBUG /* Total number of times malloc() called to allocate an arena. */ -static ulong ntimes_arena_allocated = 0; +static size_t ntimes_arena_allocated = 0; /* High water mark (max value ever seen) for narenas_currently_allocated. */ -static ulong narenas_highwater = 0; +static size_t narenas_highwater = 0; #endif /* Allocate a new arena. If we run out of memory, return NULL. Else @@ -1220,39 +1220,45 @@ #define DEADBYTE 0xDB /* dead (newly freed) memory */ #define FORBIDDENBYTE 0xFB /* untouchable bytes at each end of a block */ -static ulong serialno = 0; /* incremented on each debug {m,re}alloc */ +static size_t serialno = 0; /* incremented on each debug {m,re}alloc */ /* serialno is always incremented via calling this routine. The point is - to supply a single place to set a breakpoint. -*/ + * to supply a single place to set a breakpoint. + */ static void bumpserialno(void) { ++serialno; } +#define SST SIZEOF_SIZE_T -/* Read 4 bytes at p as a big-endian ulong. */ -static ulong -read4(const void *p) +/* Read sizeof(size_t) bytes at p as a big-endian size_t. */ +static size_t +read_size_t(const void *p) { const uchar *q = (const uchar *)p; - return ((ulong)q[0] << 24) | - ((ulong)q[1] << 16) | - ((ulong)q[2] << 8) | - (ulong)q[3]; + size_t result = *q++; + int i; + + for (i = SST; --i > 0; ++q) + result = (result << 8) | *q; + return result; } -/* Write the 4 least-significant bytes of n as a big-endian unsigned int, - MSB at address p, LSB at p+3. */ +/* Write n as a big-endian size_t, MSB at address p, LSB at + * p + sizeof(size_t) - 1. + */ static void -write4(void *p, ulong n) +write_size_t(void *p, size_t n) { - uchar *q = (uchar *)p; - q[0] = (uchar)((n >> 24) & 0xff); - q[1] = (uchar)((n >> 16) & 0xff); - q[2] = (uchar)((n >> 8) & 0xff); - q[3] = (uchar)( n & 0xff); + uchar *q = (uchar *)p + SST - 1; + int i; + + for (i = SST; --i >= 0; --q) { + *q = (uchar)(n & 0xff); + n >>= 8; + } } #ifdef Py_DEBUG @@ -1280,25 +1286,25 @@ #endif /* Py_DEBUG */ -/* The debug malloc asks for 16 extra bytes and fills them with useful stuff, - here calling the underlying malloc's result p: +/* Let S = sizeof(size_t). The debug malloc asks for 4*S extra bytes and + fills them with useful stuff, here calling the underlying malloc's result p: -p[0:4] - Number of bytes originally asked for. 4-byte unsigned integer, - big-endian (easier to read in a memory dump). -p[4:8] +p[0: S] + Number of bytes originally asked for. This is a size_t, big-endian (easier + to read in a memory dump). +p[S: 2*S] Copies of FORBIDDENBYTE. Used to catch under- writes and reads. -p[8:8+n] +p[2*S: 2*S+n] The requested memory, filled with copies of CLEANBYTE. Used to catch reference to uninitialized memory. - &p[8] is returned. Note that this is 8-byte aligned if pymalloc + &p[2*S] is returned. Note that this is 8-byte aligned if pymalloc handled the request itself. -p[8+n:8+n+4] +p[2*S+n: 2*S+n+S] Copies of FORBIDDENBYTE. Used to catch over- writes and reads. -p[8+n+4:8+n+8] +p[2*S+n+S: 2*S+n+2*S] A serial number, incremented by 1 on each call to _PyObject_DebugMalloc and _PyObject_DebugRealloc. - 4-byte unsigned integer, big-endian. + This is a big-endian size_t. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. @@ -1308,41 +1314,33 @@ _PyObject_DebugMalloc(size_t nbytes) { uchar *p; /* base address of malloc'ed block */ - uchar *tail; /* p + 8 + nbytes == pointer to tail pad bytes */ - size_t total; /* nbytes + 16 */ + uchar *tail; /* p + 2*SST + nbytes == pointer to tail pad bytes */ + size_t total; /* nbytes + 4*SST */ bumpserialno(); - total = nbytes + 16; -#if SIZEOF_SIZE_T < 8 - /* XXX do this check only on 32-bit machines */ - if (total < nbytes || (total >> 31) > 1) { - /* overflow, or we can't represent it in 4 bytes */ - /* Obscure: can't do (total >> 32) != 0 instead, because - C doesn't define what happens for a right-shift of 32 - when size_t is a 32-bit type. At least C guarantees - size_t is an unsigned type. */ + total = nbytes + 4*SST; + if (total < nbytes) + /* overflow: can't represent total as a size_t */ return NULL; - } -#endif p = (uchar *)PyObject_Malloc(total); if (p == NULL) return NULL; - write4(p, (ulong)nbytes); - p[4] = p[5] = p[6] = p[7] = FORBIDDENBYTE; + write_size_t(p, nbytes); + memset(p + SST, FORBIDDENBYTE, SST); if (nbytes > 0) - memset(p+8, CLEANBYTE, nbytes); + memset(p + 2*SST, CLEANBYTE, nbytes); - tail = p + 8 + nbytes; - tail[0] = tail[1] = tail[2] = tail[3] = FORBIDDENBYTE; - write4(tail + 4, serialno); + tail = p + 2*SST + nbytes; + memset(tail, FORBIDDENBYTE, SST); + write_size_t(tail + SST, serialno); return p+8; } -/* The debug free first checks the 8 bytes on each end for sanity (in +/* The debug free first checks the 2*SST bytes on each end for sanity (in particular, that the FORBIDDENBYTEs are still intact). Then fills the original bytes with DEADBYTE. Then calls the underlying free. @@ -1350,16 +1348,16 @@ void _PyObject_DebugFree(void *p) { - uchar *q = (uchar *)p; + uchar *q = (uchar *)p - 2*SST; /* address returned from malloc */ size_t nbytes; if (p == NULL) return; _PyObject_DebugCheckAddress(p); - nbytes = read4(q-8); + nbytes = read_size_t(q); if (nbytes > 0) memset(q, DEADBYTE, nbytes); - PyObject_Free(q-8); + PyObject_Free(q); } void * @@ -1367,20 +1365,20 @@ { uchar *q = (uchar *)p; uchar *tail; - size_t total; /* nbytes + 16 */ + size_t total; /* nbytes + 4*SST */ size_t original_nbytes; + int i; if (p == NULL) return _PyObject_DebugMalloc(nbytes); _PyObject_DebugCheckAddress(p); bumpserialno(); - original_nbytes = read4(q-8); - total = nbytes + 16; - if (total < nbytes || (total >> 31) > 1) { - /* overflow, or we can't represent it in 4 bytes */ + original_nbytes = read_size_t(q - 2*SST); + total = nbytes + 4*SST; + if (total < nbytes) + /* overflow: can't represent total as a size_t */ return NULL; - } if (nbytes < original_nbytes) { /* shrinking: mark old extra memory dead */ @@ -1388,19 +1386,17 @@ } /* Resize and add decorations. */ - q = (uchar *)PyObject_Realloc(q-8, total); + q = (uchar *)PyObject_Realloc(q - 2*SST, total); if (q == NULL) return NULL; - write4(q, (ulong)nbytes); - assert(q[4] == FORBIDDENBYTE && - q[5] == FORBIDDENBYTE && - q[6] == FORBIDDENBYTE && - q[7] == FORBIDDENBYTE); - q += 8; + write_size_t(q, nbytes); + for (i = 0; i < SST; ++i) + assert(q[SST + i] == FORBIDDENBYTE); + q += 2*SST; tail = q + nbytes; - tail[0] = tail[1] = tail[2] = tail[3] = FORBIDDENBYTE; - write4(tail + 4, serialno); + memset(tail, FORBIDDENBYTE, SST); + write_size_t(tail + SST, serialno); if (nbytes > original_nbytes) { /* growing: mark new extra memory clean */ @@ -1420,7 +1416,7 @@ { const uchar *q = (const uchar *)p; char *msg; - ulong nbytes; + size_t nbytes; const uchar *tail; int i; @@ -1433,16 +1429,16 @@ * corruption, the number-of-bytes field may be nuts, and checking * the tail could lead to a segfault then. */ - for (i = 4; i >= 1; --i) { + for (i = SST; i >= 1; --i) { if (*(q-i) != FORBIDDENBYTE) { msg = "bad leading pad byte"; goto error; } } - nbytes = read4(q-8); + nbytes = read_size_t(q - 2*SST); tail = q + nbytes; - for (i = 0; i < 4; ++i) { + for (i = 0; i < SST; ++i) { if (tail[i] != FORBIDDENBYTE) { msg = "bad trailing pad byte"; goto error; @@ -1462,28 +1458,33 @@ { const uchar *q = (const uchar *)p; const uchar *tail; - ulong nbytes, serial; + size_t nbytes, serial; int i; + int ok; fprintf(stderr, "Debug memory block at address p=%p:\n", p); if (p == NULL) return; - nbytes = read4(q-8); - fprintf(stderr, " %lu bytes originally requested\n", nbytes); + nbytes = read_size_t(q - 2*SST); + fprintf(stderr, " %" PY_FORMAT_SIZE_T "u bytes originally " + "requested\n", nbytes); /* In case this is nuts, check the leading pad bytes first. */ - fputs(" The 4 pad bytes at p-4 are ", stderr); - if (*(q-4) == FORBIDDENBYTE && - *(q-3) == FORBIDDENBYTE && - *(q-2) == FORBIDDENBYTE && - *(q-1) == FORBIDDENBYTE) { - fputs("FORBIDDENBYTE, as expected.\n", stderr); + fprintf(stderr, " The %d pad bytes at p-%d are ", SST, SST); + ok = 1; + for (i = 1; i <= SST; ++i) { + if (*(q-i) != FORBIDDENBYTE) { + ok = 0; + break; + } } + if (ok) + fputs("FORBIDDENBYTE, as expected.\n", stderr); else { fprintf(stderr, "not all FORBIDDENBYTE (0x%02x):\n", FORBIDDENBYTE); - for (i = 4; i >= 1; --i) { + for (i = SST; i >= 1; --i) { const uchar byte = *(q-i); fprintf(stderr, " at p-%d: 0x%02x", i, byte); if (byte != FORBIDDENBYTE) @@ -1498,17 +1499,20 @@ } tail = q + nbytes; - fprintf(stderr, " The 4 pad bytes at tail=%p are ", tail); - if (tail[0] == FORBIDDENBYTE && - tail[1] == FORBIDDENBYTE && - tail[2] == FORBIDDENBYTE && - tail[3] == FORBIDDENBYTE) { - fputs("FORBIDDENBYTE, as expected.\n", stderr); + fprintf(stderr, " The %d pad bytes at tail=%p are ", SST, tail); + ok = 1; + for (i = 0; i < SST; ++i) { + if (tail[i] != FORBIDDENBYTE) { + ok = 0; + break; + } } + if (ok) + fputs("FORBIDDENBYTE, as expected.\n", stderr); else { fprintf(stderr, "not all FORBIDDENBYTE (0x%02x):\n", FORBIDDENBYTE); - for (i = 0; i < 4; ++i) { + for (i = 0; i < SST; ++i) { const uchar byte = tail[i]; fprintf(stderr, " at tail+%d: 0x%02x", i, byte); @@ -1518,12 +1522,12 @@ } } - serial = read4(tail+4); - fprintf(stderr, " The block was made by call #%lu to " - "debug malloc/realloc.\n", serial); + serial = read_size_t(tail + SST); + fprintf(stderr, " The block was made by call #%" PY_FORMAT_SIZE_T + "u to debug malloc/realloc.\n", serial); if (nbytes > 0) { - int i = 0; + i = 0; fputs(" Data at p:", stderr); /* print up to 8 bytes at the start */ while (q < tail && i < 8) { @@ -1546,12 +1550,12 @@ } } -static ulong -printone(const char* msg, ulong value) +static size_t +printone(const char* msg, size_t value) { int i, k; char buf[100]; - ulong origvalue = value; + size_t origvalue = value; fputs(msg, stderr); for (i = (int)strlen(msg); i < 35; ++i) @@ -1564,8 +1568,8 @@ buf[i--] = '\n'; k = 3; do { - ulong nextvalue = value / 10UL; - uint digit = value - nextvalue * 10UL; + size_t nextvalue = value / 10; + uint digit = (uint)(value - nextvalue * 10); value = nextvalue; buf[i--] = (char)(digit + '0'); --k; @@ -1592,28 +1596,28 @@ uint i; const uint numclasses = SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT; /* # of pools, allocated blocks, and free blocks per class index */ - ulong numpools[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; - ulong numblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; - ulong numfreeblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; + size_t numpools[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; + size_t numblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; + size_t numfreeblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; /* total # of allocated bytes in used and full pools */ - ulong allocated_bytes = 0; + size_t allocated_bytes = 0; /* total # of available bytes in used pools */ - ulong available_bytes = 0; + size_t available_bytes = 0; /* # of free pools + pools not yet carved out of current arena */ uint numfreepools = 0; /* # of bytes for arena alignment padding */ - ulong arena_alignment = 0; + size_t arena_alignment = 0; /* # of bytes in used and full pools used for pool_headers */ - ulong pool_header_bytes = 0; + size_t pool_header_bytes = 0; /* # of bytes in used and full pools wasted due to quantization, * i.e. the necessarily leftover space at the ends of used and * full pools. */ - ulong quantization = 0; + size_t quantization = 0; /* # of arenas actually allocated. */ - ulong narenas = 0; + size_t narenas = 0; /* running total -- should equal narenas * ARENA_SIZE */ - ulong total; + size_t total; char buf[128]; fprintf(stderr, "Small block threshold = %d, in %u size classes.\n", @@ -1678,15 +1682,18 @@ stderr); for (i = 0; i < numclasses; ++i) { - ulong p = numpools[i]; - ulong b = numblocks[i]; - ulong f = numfreeblocks[i]; + size_t p = numpools[i]; + size_t b = numblocks[i]; + size_t f = numfreeblocks[i]; uint size = INDEX2SIZE(i); if (p == 0) { assert(b == 0 && f == 0); continue; } - fprintf(stderr, "%5u %6u %11lu %15lu %13lu\n", + fprintf(stderr, "%5u %6u " + "%11" PY_FORMAT_SIZE_T "u " + "%15" PY_FORMAT_SIZE_T "u " + "%13" PY_FORMAT_SIZE_T "u\n", i, size, p, b, f); allocated_bytes += b * size; available_bytes += f * size; @@ -1702,7 +1709,8 @@ (void)printone("# arenas allocated current", narenas); PyOS_snprintf(buf, sizeof(buf), - "%lu arenas * %d bytes/arena", narenas, ARENA_SIZE); + "%" PY_FORMAT_SIZE_T "u arenas * %d bytes/arena", + narenas, ARENA_SIZE); (void)printone(buf, narenas * ARENA_SIZE); fputc('\n', stderr); @@ -1712,7 +1720,7 @@ PyOS_snprintf(buf, sizeof(buf), "%u unused pools * %d bytes", numfreepools, POOL_SIZE); - total += printone(buf, (ulong)numfreepools * POOL_SIZE); + total += printone(buf, (size_t)numfreepools * POOL_SIZE); total += printone("# bytes lost to pool headers", pool_header_bytes); total += printone("# bytes lost to quantization", quantization); From buildbot at python.org Sun Jun 4 05:31:59 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 03:31:59 +0000 Subject: [Python-checkins] buildbot failure in amd64 gentoo trunk Message-ID: <20060604033159.8E41D1E400D@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/912 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters BUILD FAILED: failed compile sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 05:32:09 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 03:32:09 +0000 Subject: [Python-checkins] buildbot failure in ia64 Debian unstable trunk Message-ID: <20060604033209.BE50C1E400D@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/599 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters BUILD FAILED: failed compile sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 05:33:01 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 03:33:01 +0000 Subject: [Python-checkins] buildbot failure in alpha Tru64 5.1 trunk Message-ID: <20060604033301.AE47D1E400D@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/634 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters BUILD FAILED: failed compile sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 05:34:08 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 03:34:08 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20060604033408.438DE1E4003@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/117 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 05:38:10 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 4 Jun 2006 05:38:10 +0200 (CEST) Subject: [Python-checkins] r46638 - python/trunk/Objects/obmalloc.c Message-ID: <20060604033810.780601E4003@bag.python.org> Author: tim.peters Date: Sun Jun 4 05:38:04 2006 New Revision: 46638 Modified: python/trunk/Objects/obmalloc.c Log: _PyObject_DebugMalloc(): The return value should add 2*sizeof(size_t) now, not 8. This probably accounts for current disasters on the 64-bit buildbot slaves. Modified: python/trunk/Objects/obmalloc.c ============================================================================== --- python/trunk/Objects/obmalloc.c (original) +++ python/trunk/Objects/obmalloc.c Sun Jun 4 05:38:04 2006 @@ -1337,7 +1337,7 @@ memset(tail, FORBIDDENBYTE, SST); write_size_t(tail + SST, serialno); - return p+8; + return p + 2*SST; } /* The debug free first checks the 2*SST bytes on each end for sanity (in From nnorwitz at gmail.com Sun Jun 4 06:09:40 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 3 Jun 2006 21:09:40 -0700 Subject: [Python-checkins] r46603 - python/trunk/Lib/test/test_struct.py In-Reply-To: <9e804ac0606031651q56199989ra8662d9e57ba8660@mail.gmail.com> References: <20060602130346.9071C1E400B@bag.python.org> <9e804ac0606031651q56199989ra8662d9e57ba8660@mail.gmail.com> Message-ID: On 6/3/06, Thomas Wouters wrote: > > Heh, my reaction was both "huh?" and "yay" at the same time. I thought > unittest was still preferred for the stdlib testsuite? I don't particularly > see the benefits of unittest myself, although I admit it's a _tad_ cleaner > than most of the non-unittest scripts the stdlib had before. However, having > written a few tests with py.test, in particular test-generators, well, > unittest's just cumbersome ;-P unittest inherited too much from JUnit IMO (does that make it Java-ick?). I don't love it, but it's much better than the old style as Tim described. > I'm not arguing for (or against) py.test, just wondering what the support > and opposition for changing the policy would be like ;-) I vaguely remember hearing positive things about py.test. I haven't looked into it myself. I agree with Tim that until py.test was in the stdlib, it's not practical to use. I'm -1e6 on it going into 2.5. :-) No opinion (yet) on 2.6. It would suck to have 2 testing frameworks in the stdlib though. unittest would really need to be deprecated. I'm not sure the pain is worth it. If there was a way to enhance unittest to be more pythonic (perhaps like py.test?), that would be just fine with me. n From buildbot at python.org Sun Jun 4 06:37:49 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 04:37:49 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060604043749.A32E51E4003@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1008 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 06:42:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 04:42:37 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060604044237.3D8651E4003@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/118 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 07:46:53 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 05:46:53 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060604054653.DDC211E4003@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/566 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 08:19:31 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 4 Jun 2006 08:19:31 +0200 (CEST) Subject: [Python-checkins] r46639 - python/trunk/Objects/exceptions.c Message-ID: <20060604061931.B40161E4003@bag.python.org> Author: neal.norwitz Date: Sun Jun 4 08:19:31 2006 New Revision: 46639 Modified: python/trunk/Objects/exceptions.c Log: SF #1499797, Fix for memory leak in WindowsError_str Modified: python/trunk/Objects/exceptions.c ============================================================================== --- python/trunk/Objects/exceptions.c (original) +++ python/trunk/Objects/exceptions.c Sun Jun 4 08:19:31 2006 @@ -851,7 +851,6 @@ PyTuple_SET_ITEM(tuple, 1, Py_None); } - Py_INCREF(repr); PyTuple_SET_ITEM(tuple, 2, repr); rtnval = PyString_Format(fmt, tuple); From buildbot at python.org Sun Jun 4 08:46:07 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 06:46:07 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060604064607.20D761E401C@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/275 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 09:23:27 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 07:23:27 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060604072327.CE5401E4003@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/531 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From mwh at python.net Sun Jun 4 10:28:23 2006 From: mwh at python.net (Michael Hudson) Date: Sun, 04 Jun 2006 09:28:23 +0100 Subject: [Python-checkins] r46603 - python/trunk/Lib/test/test_struct.py In-Reply-To: <1f7befae0606031839t32ca5ddch361c2074fc81f20f@mail.gmail.com> (Tim Peters's message of "Sat, 3 Jun 2006 21:39:59 -0400") References: <20060602130346.9071C1E400B@bag.python.org> <9e804ac0606031651q56199989ra8662d9e57ba8660@mail.gmail.com> <1f7befae0606031839t32ca5ddch361c2074fc81f20f@mail.gmail.com> Message-ID: <2m8xoduxy0.fsf@starship.python.net> "Tim Peters" writes: >>>> Author: martin.blais >>>> Date: Fri Jun 2 15:03:43 2006 >>>> New Revision: 46603 >>>> >>>> Modified: >>>> python/trunk/Lib/test/test_struct.py >>>> Log: >>>> Fixed struct test to not use unittest. > > [Neal Norwitz] >>> Shoot, I had hoped you would go the other way and convert all the old >>> test cases to use unittest. :-) > > [Thomas Wouters] >> Heh, my reaction was both "huh?" and "yay" at the same time. I thought >> unittest was still preferred for the stdlib testsuite? > > That or doctest. What's definitely unwanted now is the original style > of test that compares the whole test's output to an expected-output > file in Lib/test/output/. Those can be horrid to figure out when they > go wrong, and don't check at all when running regrtest with -v. > > unittest, and especially doctest, encourage breaking tests into small > units. An example of neither is test_descr.py, which can be a real > bitch to untangle when it fails. Also, there is an advantage to have more structure to the tests; if all of python's tests used unittest, my regrtest -R gimmickery would be able to identify tests, rather than test files, that leaked and I'm pretty sure that this would have saved me a few hours in the last couple of years. Also, you can more easily identify particular tests that fail intermittently. Etc. Cheers, mwh -- dash: where do you go to school? mitiege: PSU tpck: didn't faassen go there too? -- from Twisted.Quotes From neal at metaslash.com Sun Jun 4 11:07:04 2006 From: neal at metaslash.com (Neal Norwitz) Date: Sun, 4 Jun 2006 05:07:04 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060604090704.GA17397@python.psfb.org> test_exceptions leaked [0, 2, 0] references test_filecmp leaked [13, 0, 0] references From richard at commonground.com.au Sun Jun 4 12:14:29 2006 From: richard at commonground.com.au (Richard Jones) Date: Sun, 4 Jun 2006 20:14:29 +1000 Subject: [Python-checkins] Python Regression Test Failures refleak (1) In-Reply-To: <20060604090704.GA17397@python.psfb.org> References: <20060604090704.GA17397@python.psfb.org> Message-ID: <43E59BC0-9CD0-4524-9DB3-6CFD5E155721@commonground.com.au> On 04/06/2006, at 7:07 PM, Neal Norwitz wrote: > test_exceptions leaked [0, 2, 0] references > test_filecmp leaked [13, 0, 0] references What's producing this output? On what OS? I'd like to try to find that leak in exceptions if possible. Richard From python-checkins at python.org Sun Jun 4 14:32:33 2006 From: python-checkins at python.org (andrew.macintyre) Date: Sun, 4 Jun 2006 14:32:33 +0200 (CEST) Subject: [Python-checkins] r46640 - in python/trunk: Doc/lib/libthread.tex Doc/lib/libthreading.tex Include/pythread.h Lib/dummy_thread.py Lib/test/output/test_thread Lib/test/test_thread.py Lib/test/test_threading.py Lib/threading.py Misc/NEWS Modules/threadmodule.c Python/thread.c Python/thread_nt.h Python/thread_os2.h Python/thread_pthread.h Message-ID: <20060604123233.D222B1E4003@bag.python.org> Author: andrew.macintyre Date: Sun Jun 4 14:31:09 2006 New Revision: 46640 Modified: python/trunk/Doc/lib/libthread.tex python/trunk/Doc/lib/libthreading.tex python/trunk/Include/pythread.h python/trunk/Lib/dummy_thread.py python/trunk/Lib/test/output/test_thread python/trunk/Lib/test/test_thread.py python/trunk/Lib/test/test_threading.py python/trunk/Lib/threading.py python/trunk/Misc/NEWS python/trunk/Modules/threadmodule.c python/trunk/Python/thread.c python/trunk/Python/thread_nt.h python/trunk/Python/thread_os2.h python/trunk/Python/thread_pthread.h Log: Patch #1454481: Make thread stack size runtime tunable. Modified: python/trunk/Doc/lib/libthread.tex ============================================================================== --- python/trunk/Doc/lib/libthread.tex (original) +++ python/trunk/Doc/lib/libthread.tex Sun Jun 4 14:31:09 2006 @@ -74,6 +74,25 @@ another thread is created. \end{funcdesc} +\begin{funcdesc}{stack_size}{\optional{size}} +Return the thread stack size used when creating new threads. The +optional \var{size} argument specifies the stack size to be used for +subsequently created threads, and must be 0 (use platform or +configured default) or a positive integer value of at least 32,768 (32kB). +If changing the thread stack size is unsupported, or the specified size +is invalid, a RuntimeWarning is issued and the stack size is unmodified. +32kB is currently the minimum supported stack size value, to guarantee +sufficient stack space for the interpreter itself. +Note that some platforms may have particular restrictions on values for +the stack size, such as requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for more +information (4kB pages are common; using multiples of 4096 for the +stack size is the suggested approach in the absence of more specific +information). +Availability: Windows, systems with \POSIX{} threads. +\versionadded{2.5} +\end{funcdesc} + Lock objects have the following methods: Modified: python/trunk/Doc/lib/libthreading.tex ============================================================================== --- python/trunk/Doc/lib/libthreading.tex (original) +++ python/trunk/Doc/lib/libthreading.tex Sun Jun 4 14:31:09 2006 @@ -125,6 +125,25 @@ \versionadded{2.3} \end{funcdesc} +\begin{funcdesc}{stack_size}{\optional{size}} +Return the thread stack size used when creating new threads. The +optional \var{size} argument specifies the stack size to be used for +subsequently created threads, and must be 0 (use platform or +configured default) or a positive integer value of at least 32,768 (32kB). +If changing the thread stack size is unsupported, or the specified size +is invalid, a RuntimeWarning is issued and the stack size is unmodified. +32kB is currently the minimum supported stack size value, to guarantee +sufficient stack space for the interpreter itself. +Note that some platforms may have particular restrictions on values for +the stack size, such as requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for more +information (4kB pages are common; using multiples of 4096 for the +stack size is the suggested approach in the absence of more specific +information). +Availability: Windows, systems with \POSIX{} threads. +\versionadded{2.5} +\end{funcdesc} + Detailed interfaces for the objects are documented below. The design of this module is loosely based on Java's threading model. Modified: python/trunk/Include/pythread.h ============================================================================== --- python/trunk/Include/pythread.h (original) +++ python/trunk/Include/pythread.h Sun Jun 4 14:31:09 2006 @@ -25,6 +25,9 @@ #define NOWAIT_LOCK 0 PyAPI_FUNC(void) PyThread_release_lock(PyThread_type_lock); +PyAPI_FUNC(size_t) PyThread_get_stacksize(void); +PyAPI_FUNC(int) PyThread_set_stacksize(size_t); + #ifndef NO_EXIT_PROG PyAPI_FUNC(void) PyThread_exit_prog(int); PyAPI_FUNC(void) PyThread__PyThread_exit_prog(int); Modified: python/trunk/Lib/dummy_thread.py ============================================================================== --- python/trunk/Lib/dummy_thread.py (original) +++ python/trunk/Lib/dummy_thread.py Sun Jun 4 14:31:09 2006 @@ -20,6 +20,7 @@ 'interrupt_main', 'LockType'] import traceback as _traceback +import warnings class error(Exception): """Dummy implementation of thread.error.""" @@ -75,6 +76,13 @@ """Dummy implementation of thread.allocate_lock().""" return LockType() +def stack_size(size=None): + """Dummy implementation of thread.stack_size().""" + if size is not None: + msg = "setting thread stack size not supported on this platform" + warnings.warn(msg, RuntimeWarning) + return 0 + class LockType(object): """Class implementing dummy implementation of thread.LockType. Modified: python/trunk/Lib/test/output/test_thread ============================================================================== --- python/trunk/Lib/test/output/test_thread (original) +++ python/trunk/Lib/test/output/test_thread Sun Jun 4 14:31:09 2006 @@ -4,3 +4,11 @@ *** Barrier Test *** all tasks done + +*** Changing thread stack size *** +trying stack_size = 32768 +waiting for all tasks to complete +all tasks done +trying stack_size = 4194304 +waiting for all tasks to complete +all tasks done Modified: python/trunk/Lib/test/test_thread.py ============================================================================== --- python/trunk/Lib/test/test_thread.py (original) +++ python/trunk/Lib/test/test_thread.py Sun Jun 4 14:31:09 2006 @@ -115,3 +115,38 @@ thread.start_new_thread(task2, (i,)) done.acquire() print 'all tasks done' + +# not all platforms support changing thread stack size +print '\n*** Changing thread stack size ***' +if thread.stack_size() != 0: + raise ValueError, "initial stack_size not 0" + +thread.stack_size(0) +if thread.stack_size() != 0: + raise ValueError, "stack_size not reset to default" + +from os import name as os_name +if os_name in ("nt", "os2", "posix"): + + for tss, ok in ((4096, 0), (32768, 1), (0x400000, 1), (0, 1)): + if ok: + failed = lambda s, e: s != e + fail_msg = "stack_size(%d) failed - should succeed" + else: + failed = lambda s, e: s == e + fail_msg = "stack_size(%d) succeeded - should fail" + thread.stack_size(tss) + if failed(thread.stack_size(), tss): + raise ValueError, fail_msg % tss + + for tss in (32768, 0x400000): + print 'trying stack_size = %d' % tss + next_ident = 0 + for i in range(numtasks): + newtask() + + print 'waiting for all tasks to complete' + done.acquire() + print 'all tasks done' + + thread.stack_size(0) Modified: python/trunk/Lib/test/test_threading.py ============================================================================== --- python/trunk/Lib/test/test_threading.py (original) +++ python/trunk/Lib/test/test_threading.py Sun Jun 4 14:31:09 2006 @@ -85,6 +85,22 @@ print 'all tasks done' self.assertEqual(numrunning.get(), 0) + # run with a minimum thread stack size (32kB) + def test_various_ops_small_stack(self): + if verbose: + print 'with 32kB thread stack size...' + threading.stack_size(0x8000) + self.test_various_ops() + threading.stack_size(0) + + # run with a large thread stack size (16MB) + def test_various_ops_large_stack(self): + if verbose: + print 'with 16MB thread stack size...' + threading.stack_size(0x1000000) + self.test_various_ops() + threading.stack_size(0) + def test_foreign_thread(self): # Check that a "foreign" thread can use the threading module. def f(mutex): Modified: python/trunk/Lib/threading.py ============================================================================== --- python/trunk/Lib/threading.py (original) +++ python/trunk/Lib/threading.py Sun Jun 4 14:31:09 2006 @@ -15,7 +15,7 @@ # Rename some stuff so "from threading import *" is safe __all__ = ['activeCount', 'Condition', 'currentThread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', - 'Timer', 'setprofile', 'settrace', 'local'] + 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'] _start_new_thread = thread.start_new_thread _allocate_lock = thread.allocate_lock @@ -713,6 +713,8 @@ _active_limbo_lock.release() return active +from thread import stack_size + # Create the main thread object _MainThread() Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sun Jun 4 14:31:09 2006 @@ -84,6 +84,9 @@ - Patch #1435422: zlib's compress and decompress objects now have a copy() method. +- Patch #1454481: thread stack size is now tunable at runtime for thread + enabled builds on Windows and systems with Posix threads support. + - On Win32, os.listdir now supports arbitrarily-long Unicode path names (up to the system limit of 32K characters). Modified: python/trunk/Modules/threadmodule.c ============================================================================== --- python/trunk/Modules/threadmodule.c (original) +++ python/trunk/Modules/threadmodule.c Sun Jun 4 14:31:09 2006 @@ -586,6 +586,51 @@ be relied upon, and the number should be seen purely as a magic cookie.\n\ A thread's identity may be reused for another thread after it exits."); +static PyObject * +thread_stack_size(PyObject *self, PyObject *args) +{ + size_t old_size, new_size; + PyObject *set_size = NULL; + + if (!PyArg_UnpackTuple(args, "stack_size", 0, 1, &set_size)) + return NULL; + + old_size = PyThread_get_stacksize(); + + if (set_size != NULL) { + if (PyInt_Check(set_size)) + new_size = (size_t) PyInt_AsLong(set_size); + else { + PyErr_SetString(PyExc_TypeError, + "size must be an integer"); + return NULL; + } + if (PyThread_set_stacksize(new_size)) + return NULL; + } + + return PyInt_FromLong((long) old_size); +} + +PyDoc_STRVAR(stack_size_doc, +"stack_size([size]) -> size\n\ +\n\ +Return the thread stack size used when creating new threads. The\n\ +optional size argument specifies the stack size (in bytes) to be used\n\ +for subsequently created threads, and must be 0 (use platform or\n\ +configured default) or a positive integer value of at least 32,768 (32kB).\n\ +If changing the thread stack size is unsupported, or the specified size\n\ +is invalid, a RuntimeWarning is issued and the stack size is unmodified.\n\ +32kB is currently the minimum supported stack size value, to guarantee\n\ +sufficient stack space for the interpreter itself.\n\ +\n\ +Note that some platforms may have particular restrictions on values for\n\ +the stack size, such as requiring allocation in multiples of the system\n\ +memory page size - platform documentation should be referred to for more\n\ +information (4kB pages are common; using multiples of 4096 for the\n\ +stack size is the suggested approach in the absence of more specific\n\ +information)."); + static PyMethodDef thread_methods[] = { {"start_new_thread", (PyCFunction)thread_PyThread_start_new_thread, METH_VARARGS, @@ -605,6 +650,9 @@ METH_NOARGS, interrupt_doc}, {"get_ident", (PyCFunction)thread_get_ident, METH_NOARGS, get_ident_doc}, + {"stack_size", (PyCFunction)thread_stack_size, + METH_VARARGS, + stack_size_doc}, #ifndef NO_EXIT_PROG {"exit_prog", (PyCFunction)thread_PyThread_exit_prog, METH_VARARGS}, Modified: python/trunk/Python/thread.c ============================================================================== --- python/trunk/Python/thread.c (original) +++ python/trunk/Python/thread.c Sun Jun 4 14:31:09 2006 @@ -94,6 +94,31 @@ PyThread__init_thread(); } +/* Support for runtime thread stack size tuning. + A value of 0 means using the platform's default stack size + or the size specified by the THREAD_STACK_SIZE macro. */ +static size_t _pythread_stacksize = 0; + +size_t +PyThread_get_stacksize(void) +{ + return _pythread_stacksize; +} + +static int +_pythread_unsupported_set_stacksize(size_t size) +{ + return PyErr_Warn(PyExc_RuntimeWarning, + "setting thread stack size not supported on " + "this platform"); +} + +/* Only platforms with THREAD_SET_STACKSIZE() defined in + pthread_.h, overriding this default definition, + will support changing the stack size. + Return 1 if an exception is pending, 0 otherwise. */ +#define THREAD_SET_STACKSIZE(x) _pythread_unsupported_set_stacksize(x) + #ifdef SGI_THREADS #include "thread_sgi.h" #endif @@ -149,6 +174,14 @@ #endif */ +/* use appropriate thread stack size setting routine. + Return 1 if an exception is pending, 0 otherwise. */ +int +PyThread_set_stacksize(size_t size) +{ + return THREAD_SET_STACKSIZE(size); +} + #ifndef Py_HAVE_NATIVE_TLS /* If the platform has not supplied a platform specific TLS implementation, provide our own. Modified: python/trunk/Python/thread_nt.h ============================================================================== --- python/trunk/Python/thread_nt.h (original) +++ python/trunk/Python/thread_nt.h Sun Jun 4 14:31:09 2006 @@ -185,7 +185,7 @@ if (obj.done == NULL) return -1; - rv = _beginthread(bootstrap, 0, &obj); /* use default stack size */ + rv = _beginthread(bootstrap, _pythread_stacksize, &obj); if (rv == (Py_uintptr_t)-1) { /* I've seen errno == EAGAIN here, which means "there are * too many threads". @@ -313,3 +313,37 @@ if (!(aLock && LeaveNonRecursiveMutex((PNRMUTEX) aLock))) dprintf(("%ld: Could not PyThread_release_lock(%p) error: %l\n", PyThread_get_thread_ident(), aLock, GetLastError())); } + +/* minimum/maximum thread stack sizes supported */ +#define THREAD_MIN_STACKSIZE 0x8000 /* 32kB */ +#define THREAD_MAX_STACKSIZE 0x10000000 /* 256MB */ + +/* set the thread stack size. + * Return 1 if an exception is pending, 0 otherwise. + */ +static int +_pythread_nt_set_stacksize(size_t size) +{ + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + + /* valid range? */ + if (size >= THREAD_MIN_STACKSIZE && size < THREAD_MAX_STACKSIZE) { + _pythread_stacksize = size; + return 0; + } + else { + char warning[128]; + snprintf(warning, + 128, + "thread stack size of %#x bytes not supported on Win32", + size); + return PyErr_Warn(PyExc_RuntimeWarning, warning); + } +} + +#undef THREAD_SET_STACKSIZE +#define THREAD_SET_STACKSIZE(x) _pythread_nt_set_stacksize(x) Modified: python/trunk/Python/thread_os2.h ============================================================================== --- python/trunk/Python/thread_os2.h (original) +++ python/trunk/Python/thread_os2.h Sun Jun 4 14:31:09 2006 @@ -14,10 +14,13 @@ long PyThread_get_thread_ident(void); #endif +/* default thread stack size of 64kB */ #if !defined(THREAD_STACK_SIZE) #define THREAD_STACK_SIZE 0x10000 #endif +#define OS2_STACKSIZE(x) (x ? x : THREAD_STACK_SIZE) + /* * Initialization of the C package, should not be needed. */ @@ -35,7 +38,10 @@ int aThread; int success = 0; - aThread = _beginthread(func, NULL, THREAD_STACK_SIZE, arg); + aThread = _beginthread(func, + NULL, + OS2_STACKSIZE(_pythread_stacksize), + arg); if (aThread == -1) { success = -1; @@ -274,3 +280,37 @@ DosExitCritSec(); #endif } + +/* minimum/maximum thread stack sizes supported */ +#define THREAD_MIN_STACKSIZE 0x8000 /* 32kB */ +#define THREAD_MAX_STACKSIZE 0x2000000 /* 32MB */ + +/* set the thread stack size. + * Return 1 if an exception is pending, 0 otherwise. + */ +static int +_pythread_os2_set_stacksize(size_t size) +{ + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + + /* valid range? */ + if (size >= THREAD_MIN_STACKSIZE && size < THREAD_MAX_STACKSIZE) { + _pythread_stacksize = size; + return 0; + } + else { + char warning[128]; + snprintf(warning, + 128, + "thread stack size of %#x bytes not supported on OS/2", + size); + return PyErr_Warn(PyExc_RuntimeWarning, warning); + } +} + +#undef THREAD_SET_STACKSIZE +#define THREAD_SET_STACKSIZE(x) _pythread_os2_set_stacksize(x) Modified: python/trunk/Python/thread_pthread.h ============================================================================== --- python/trunk/Python/thread_pthread.h (original) +++ python/trunk/Python/thread_pthread.h Sun Jun 4 14:31:09 2006 @@ -12,6 +12,24 @@ #endif #include +/* The POSIX spec requires that use of pthread_attr_setstacksize + be conditional on _POSIX_THREAD_ATTR_STACKSIZE being defined. */ +#ifdef _POSIX_THREAD_ATTR_STACKSIZE +#ifndef THREAD_STACK_SIZE +#define THREAD_STACK_SIZE 0 /* use default stack size */ +#endif +/* for safety, ensure a viable minimum stacksize */ +#define THREAD_STACK_MIN 0x8000 /* 32kB */ +#if THREAD_STACK_MIN < PTHREAD_STACK_MIN +#undef THREAD_STACK_MIN +#define THREAD_STACK_MIN PTHREAD_STACK_MIN +#endif +#else /* !_POSIX_THREAD_ATTR_STACKSIZE */ +#ifdef THREAD_STACK_SIZE +#error "THREAD_STACK_SIZE defined but _POSIX_THREAD_ATTR_STACKSIZE undefined" +#endif +#endif + /* The POSIX spec says that implementations supporting the sem_* family of functions must indicate this by defining _POSIX_SEMAPHORES. */ @@ -138,6 +156,10 @@ #if defined(THREAD_STACK_SIZE) || defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) pthread_attr_t attrs; #endif +#if defined(THREAD_STACK_SIZE) + size_t tss; +#endif + dprintf(("PyThread_start_new_thread called\n")); if (!initialized) PyThread_init_thread(); @@ -145,8 +167,15 @@ #if defined(THREAD_STACK_SIZE) || defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) pthread_attr_init(&attrs); #endif -#ifdef THREAD_STACK_SIZE - pthread_attr_setstacksize(&attrs, THREAD_STACK_SIZE); +#if defined(THREAD_STACK_SIZE) + tss = (_pythread_stacksize != 0) ? _pythread_stacksize + : THREAD_STACK_SIZE; + if (tss != 0) { + if (pthread_attr_setstacksize(&attrs, tss) != 0) { + pthread_attr_destroy(&attrs); + return -1; + } + } #endif #if defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) pthread_attr_setscope(&attrs, PTHREAD_SCOPE_SYSTEM); @@ -460,3 +489,33 @@ } #endif /* USE_SEMAPHORES */ + +/* set the thread stack size. + * Return 1 if an exception is pending, 0 otherwise. + */ +static int +_pythread_pthread_set_stacksize(size_t size) +{ + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + + /* valid range? */ + if (size >= THREAD_STACK_MIN) { + _pythread_stacksize = size; + return 0; + } + else { + char warning[128]; + snprintf(warning, + 128, + "thread stack size of %#x bytes not supported", + size); + return PyErr_Warn(PyExc_RuntimeWarning, warning); + } +} + +#undef THREAD_SET_STACKSIZE +#define THREAD_SET_STACKSIZE(x) _pythread_pthread_set_stacksize(x) From buildbot at python.org Sun Jun 4 14:40:11 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 12:40:11 +0000 Subject: [Python-checkins] buildbot failure in sparc solaris10 gcc trunk Message-ID: <20060604124012.52AF31E4003@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%2520trunk/builds/889 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre BUILD FAILED: failed compile sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 14:54:43 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 12:54:43 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060604125450.009F61E4003@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1011 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 14:55:27 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 12:55:27 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060604125527.E77FC1E4003@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/915 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 15:00:00 2006 From: python-checkins at python.org (andrew.macintyre) Date: Sun, 4 Jun 2006 15:00:00 +0200 (CEST) Subject: [Python-checkins] r46641 - python/trunk/Python/thread.c python/trunk/Python/thread_nt.h python/trunk/Python/thread_os2.h Message-ID: <20060604130000.2181A1E4003@bag.python.org> Author: andrew.macintyre Date: Sun Jun 4 14:59:59 2006 New Revision: 46641 Modified: python/trunk/Python/thread.c python/trunk/Python/thread_nt.h python/trunk/Python/thread_os2.h Log: clean up function declarations to conform to PEP-8 style. Modified: python/trunk/Python/thread.c ============================================================================== --- python/trunk/Python/thread.c (original) +++ python/trunk/Python/thread.c Sun Jun 4 14:59:59 2006 @@ -75,7 +75,8 @@ static void PyThread__init_thread(void); /* Forward */ -void PyThread_init_thread(void) +void +PyThread_init_thread(void) { #ifdef Py_DEBUG char *p = getenv("THREADDEBUG"); Modified: python/trunk/Python/thread_nt.h ============================================================================== --- python/trunk/Python/thread_nt.h (original) +++ python/trunk/Python/thread_nt.h Sun Jun 4 14:59:59 2006 @@ -16,7 +16,8 @@ typedef PVOID WINAPI interlocked_cmp_xchg_t(PVOID *dest, PVOID exc, PVOID comperand) ; /* Sorry mate, but we haven't got InterlockedCompareExchange in Win95! */ -static PVOID WINAPI interlocked_cmp_xchg(PVOID *dest, PVOID exc, PVOID comperand) +static PVOID WINAPI +interlocked_cmp_xchg(PVOID *dest, PVOID exc, PVOID comperand) { static LONG spinlock = 0 ; PVOID result ; @@ -54,8 +55,10 @@ return result ; } ; -static interlocked_cmp_xchg_t *ixchg ; -BOOL InitializeNonRecursiveMutex(PNRMUTEX mutex) +static interlocked_cmp_xchg_t *ixchg; + +BOOL +InitializeNonRecursiveMutex(PNRMUTEX mutex) { if (!ixchg) { @@ -76,14 +79,16 @@ #endif #define InterlockedCompareExchange(dest,exchange,comperand) (ixchg((dest), (exchange), (comperand))) -VOID DeleteNonRecursiveMutex(PNRMUTEX mutex) +VOID +DeleteNonRecursiveMutex(PNRMUTEX mutex) { /* No in-use check */ CloseHandle(mutex->hevent) ; mutex->hevent = NULL ; /* Just in case */ } -DWORD EnterNonRecursiveMutex(PNRMUTEX mutex, BOOL wait) +DWORD +EnterNonRecursiveMutex(PNRMUTEX mutex, BOOL wait) { /* Assume that the thread waits successfully */ DWORD ret ; @@ -104,7 +109,8 @@ return ret ; } -BOOL LeaveNonRecursiveMutex(PNRMUTEX mutex) +BOOL +LeaveNonRecursiveMutex(PNRMUTEX mutex) { /* We don't own the mutex */ mutex->thread_id = 0 ; @@ -113,7 +119,8 @@ SetEvent(mutex->hevent) ; /* Other threads are waiting, wake one on them up */ } -PNRMUTEX AllocNonRecursiveMutex(void) +PNRMUTEX +AllocNonRecursiveMutex(void) { PNRMUTEX mutex = (PNRMUTEX)malloc(sizeof(NRMUTEX)) ; if (mutex && !InitializeNonRecursiveMutex(mutex)) @@ -124,7 +131,8 @@ return mutex ; } -void FreeNonRecursiveMutex(PNRMUTEX mutex) +void +FreeNonRecursiveMutex(PNRMUTEX mutex) { if (mutex) { @@ -138,7 +146,8 @@ /* * Initialization of the C package, should not be needed. */ -static void PyThread__init_thread(void) +static void +PyThread__init_thread(void) { } @@ -209,7 +218,8 @@ * Return the thread Id instead of an handle. The Id is said to uniquely identify the * thread in the system */ -long PyThread_get_thread_ident(void) +long +PyThread_get_thread_ident(void) { if (!initialized) PyThread_init_thread(); @@ -217,7 +227,8 @@ return GetCurrentThreadId(); } -static void do_PyThread_exit_thread(int no_cleanup) +static void +do_PyThread_exit_thread(int no_cleanup) { dprintf(("%ld: PyThread_exit_thread called\n", PyThread_get_thread_ident())); if (!initialized) @@ -228,18 +239,21 @@ _endthread(); } -void PyThread_exit_thread(void) +void +PyThread_exit_thread(void) { do_PyThread_exit_thread(0); } -void PyThread__exit_thread(void) +void +PyThread__exit_thread(void) { do_PyThread_exit_thread(1); } #ifndef NO_EXIT_PROG -static void do_PyThread_exit_prog(int status, int no_cleanup) +static void +do_PyThread_exit_prog(int status, int no_cleanup) { dprintf(("PyThread_exit_prog(%d) called\n", status)); if (!initialized) @@ -249,12 +263,14 @@ exit(status); } -void PyThread_exit_prog(int status) +void +PyThread_exit_prog(int status) { do_PyThread_exit_prog(status, 0); } -void PyThread__exit_prog(int status) +void +PyThread__exit_prog(int status) { do_PyThread_exit_prog(status, 1); } @@ -265,7 +281,8 @@ * I [Dag] tried to implement it with mutex but I could find a way to * tell whether a thread already own the lock or not. */ -PyThread_type_lock PyThread_allocate_lock(void) +PyThread_type_lock +PyThread_allocate_lock(void) { PNRMUTEX aLock; @@ -280,7 +297,8 @@ return (PyThread_type_lock) aLock; } -void PyThread_free_lock(PyThread_type_lock aLock) +void +PyThread_free_lock(PyThread_type_lock aLock) { dprintf(("%ld: PyThread_free_lock(%p) called\n", PyThread_get_thread_ident(),aLock)); @@ -293,7 +311,8 @@ * and 0 if the lock was not acquired. This means a 0 is returned * if the lock has already been acquired by this thread! */ -int PyThread_acquire_lock(PyThread_type_lock aLock, int waitflag) +int +PyThread_acquire_lock(PyThread_type_lock aLock, int waitflag) { int success ; @@ -306,7 +325,8 @@ return success; } -void PyThread_release_lock(PyThread_type_lock aLock) +void +PyThread_release_lock(PyThread_type_lock aLock) { dprintf(("%ld: PyThread_release_lock(%p) called\n", PyThread_get_thread_ident(),aLock)); Modified: python/trunk/Python/thread_os2.h ============================================================================== --- python/trunk/Python/thread_os2.h (original) +++ python/trunk/Python/thread_os2.h Sun Jun 4 14:59:59 2006 @@ -244,7 +244,8 @@ return 1; } -void PyThread_release_lock(PyThread_type_lock aLock) +void +PyThread_release_lock(PyThread_type_lock aLock) { #if !defined(PYCC_GCC) type_os2_lock lock = (type_os2_lock)aLock; From buildbot at python.org Sun Jun 4 15:21:34 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 13:21:34 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060604132150.EA1CF1E4003@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/602 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 15:25:50 2006 From: python-checkins at python.org (andrew.macintyre) Date: Sun, 4 Jun 2006 15:25:50 +0200 (CEST) Subject: [Python-checkins] r46641 - svn:log Message-ID: <20060604132550.CD69E1E4003@bag.python.org> Author: andrew.macintyre Revision: 46641 Property Name: svn:log New Property Value: clean up function declarations to conform to PEP-7 style. From blais at furius.ca Sun Jun 4 15:44:49 2006 From: blais at furius.ca (Martin Blais) Date: Sun, 4 Jun 2006 09:44:49 -0400 Subject: [Python-checkins] r46603 - python/trunk/Lib/test/test_struct.py In-Reply-To: <1f7befae0606031839t32ca5ddch361c2074fc81f20f@mail.gmail.com> References: <20060602130346.9071C1E400B@bag.python.org> <9e804ac0606031651q56199989ra8662d9e57ba8660@mail.gmail.com> <1f7befae0606031839t32ca5ddch361c2074fc81f20f@mail.gmail.com> Message-ID: <8393fff0606040644s185a6876i99abafc12e53264a@mail.gmail.com> On 6/3/06, Tim Peters wrote: > >>> Author: martin.blais > >>> Date: Fri Jun 2 15:03:43 2006 > >>> New Revision: 46603 > >>> > >>> Modified: > >>> python/trunk/Lib/test/test_struct.py > >>> Log: > >>> Fixed struct test to not use unittest. > > [Neal Norwitz] > >> Shoot, I had hoped you would go the other way and convert all the old > >> test cases to use unittest. :-) > > [Thomas Wouters] > > Heh, my reaction was both "huh?" and "yay" at the same time. I thought > > unittest was still preferred for the stdlib testsuite? > > That or doctest. What's definitely unwanted now is the original style > of test that compares the whole test's output to an expected-output > file in Lib/test/output/. Those can be horrid to figure out when they > go wrong, and don't check at all when running regrtest with -v. I looked at the existing tests, I did not observe consistency, so I took the path of minimal effort. If you really care, then I suggest you should all agree on something, and then someone should show courage and convert all the existing tests to that thing (ideally, someone who cares). I would be happy to comply with whatever thing you choose to agree on. cheers, From buildbot at python.org Sun Jun 4 15:47:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 13:47:17 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060604134721.51E8D1E4003@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/568 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 15:50:02 2006 From: python-checkins at python.org (martin.blais) Date: Sun, 4 Jun 2006 15:50:02 +0200 (CEST) Subject: [Python-checkins] r46642 - in python/trunk: Lib/socket.py Lib/struct.py Lib/test/test_socket.py Lib/test/test_struct.py Modules/_struct.c Modules/socketmodule.c Message-ID: <20060604135002.0DC1B1E4003@bag.python.org> Author: martin.blais Date: Sun Jun 4 15:49:49 2006 New Revision: 46642 Modified: python/trunk/Lib/socket.py python/trunk/Lib/struct.py python/trunk/Lib/test/test_socket.py python/trunk/Lib/test/test_struct.py python/trunk/Modules/_struct.c python/trunk/Modules/socketmodule.c Log: Fixes in struct and socket from merge reviews. - Following Guido's comments, renamed * pack_to -> pack_into * recv_buf -> recv_into * recvfrom_buf -> recvfrom_into - Made fixes to _struct.c according to Neal Norwitz comments on the checkins list. - Converted some ints into the appropriate -- I hope -- ssize_t and size_t. Modified: python/trunk/Lib/socket.py ============================================================================== --- python/trunk/Lib/socket.py (original) +++ python/trunk/Lib/socket.py Sun Jun 4 15:49:49 2006 @@ -141,7 +141,7 @@ __doc__ = _realsocket.__doc__ __slots__ = ["_sock", - "recv", "recv_buf", "recvfrom_buf", + "recv", "recv_into", "recvfrom_into", "send", "sendto", "recvfrom", "__weakref__"] @@ -151,10 +151,10 @@ self._sock = _sock self.send = self._sock.send self.recv = self._sock.recv - self.recv_buf = self._sock.recv_buf + self.recv_into = self._sock.recv_into self.sendto = self._sock.sendto self.recvfrom = self._sock.recvfrom - self.recvfrom_buf = self._sock.recvfrom_buf + self.recvfrom_into = self._sock.recvfrom_into def close(self): self._sock = _closedsocket() Modified: python/trunk/Lib/struct.py ============================================================================== --- python/trunk/Lib/struct.py (original) +++ python/trunk/Lib/struct.py Sun Jun 4 15:49:49 2006 @@ -62,7 +62,7 @@ o = _compile(fmt) return o.pack(*args) -def pack_to(fmt, buf, offset, *args): +def pack_into(fmt, buf, offset, *args): """ Pack the values v2, v2, ... according to fmt, write the packed bytes into the writable buffer buf starting at offset. @@ -72,7 +72,7 @@ o = _cache[fmt] except KeyError: o = _compile(fmt) - return o.pack_to(buf, offset, *args) + return o.pack_into(buf, offset, *args) def unpack(fmt, s): """ Modified: python/trunk/Lib/test/test_socket.py ============================================================================== --- python/trunk/Lib/test/test_socket.py (original) +++ python/trunk/Lib/test/test_socket.py Sun Jun 4 15:49:49 2006 @@ -860,25 +860,25 @@ def __init__(self, methodName='runTest'): SocketConnectedTest.__init__(self, methodName=methodName) - def testRecvBuf(self): + def testRecvInto(self): buf = array.array('c', ' '*1024) - nbytes = self.cli_conn.recv_buf(buf) + nbytes = self.cli_conn.recv_into(buf) self.assertEqual(nbytes, len(MSG)) msg = buf.tostring()[:len(MSG)] self.assertEqual(msg, MSG) - def _testRecvBuf(self): + def _testRecvInto(self): buf = buffer(MSG) self.serv_conn.send(buf) - def testRecvFromBuf(self): + def testRecvFromInto(self): buf = array.array('c', ' '*1024) - nbytes, addr = self.cli_conn.recvfrom_buf(buf) + nbytes, addr = self.cli_conn.recvfrom_into(buf) self.assertEqual(nbytes, len(MSG)) msg = buf.tostring()[:len(MSG)] self.assertEqual(msg, MSG) - def _testRecvFromBuf(self): + def _testRecvFromInto(self): buf = buffer(MSG) self.serv_conn.send(buf) Modified: python/trunk/Lib/test/test_struct.py ============================================================================== --- python/trunk/Lib/test/test_struct.py (original) +++ python/trunk/Lib/test/test_struct.py Sun Jun 4 15:49:49 2006 @@ -529,50 +529,50 @@ for i in xrange(6, len(test_string) + 1): simple_err(struct.unpack_from, fmt, data, i) -def test_pack_to(): +def test_pack_into(): test_string = 'Reykjavik rocks, eow!' writable_buf = array.array('c', ' '*100) fmt = '21s' s = struct.Struct(fmt) # Test without offset - s.pack_to(writable_buf, 0, test_string) + s.pack_into(writable_buf, 0, test_string) from_buf = writable_buf.tostring()[:len(test_string)] assert from_buf == test_string # Test with offset. - s.pack_to(writable_buf, 10, test_string) + s.pack_into(writable_buf, 10, test_string) from_buf = writable_buf.tostring()[:len(test_string)+10] assert from_buf == (test_string[:10] + test_string) # Go beyond boundaries. small_buf = array.array('c', ' '*10) - assertRaises(struct.error, s.pack_to, small_buf, 0, test_string) - assertRaises(struct.error, s.pack_to, small_buf, 2, test_string) + assertRaises(struct.error, s.pack_into, small_buf, 0, test_string) + assertRaises(struct.error, s.pack_into, small_buf, 2, test_string) -def test_pack_to_fn(): +def test_pack_into_fn(): test_string = 'Reykjavik rocks, eow!' writable_buf = array.array('c', ' '*100) fmt = '21s' - pack_to = lambda *args: struct.pack_to(fmt, *args) + pack_into = lambda *args: struct.pack_into(fmt, *args) # Test without offset - pack_to(writable_buf, 0, test_string) + pack_into(writable_buf, 0, test_string) from_buf = writable_buf.tostring()[:len(test_string)] assert from_buf == test_string # Test with offset. - pack_to(writable_buf, 10, test_string) + pack_into(writable_buf, 10, test_string) from_buf = writable_buf.tostring()[:len(test_string)+10] assert from_buf == (test_string[:10] + test_string) # Go beyond boundaries. small_buf = array.array('c', ' '*10) - assertRaises(struct.error, pack_to, small_buf, 0, test_string) - assertRaises(struct.error, pack_to, small_buf, 2, test_string) + assertRaises(struct.error, pack_into, small_buf, 0, test_string) + assertRaises(struct.error, pack_into, small_buf, 2, test_string) # Test methods to pack and unpack from buffers rather than strings. test_unpack_from() -test_pack_to() -test_pack_to_fn() +test_pack_into() +test_pack_into_fn() Modified: python/trunk/Modules/_struct.c ============================================================================== --- python/trunk/Modules/_struct.c (original) +++ python/trunk/Modules/_struct.c Sun Jun 4 15:49:49 2006 @@ -1572,8 +1572,7 @@ soself = (PyStructObject *)self; assert(PyStruct_Check(self)); assert(soself->s_codes != NULL); - if (args == NULL || !PyTuple_Check(args) || - PyTuple_GET_SIZE(args) != soself->s_len) + if (PyTuple_GET_SIZE(args) != soself->s_len) { PyErr_Format(StructError, "pack requires exactly %zd arguments", soself->s_len); @@ -1594,16 +1593,16 @@ return result; } -PyDoc_STRVAR(s_pack_to__doc__, -"S.pack_to(buffer, offset, v1, v2, ...)\n\ +PyDoc_STRVAR(s_pack_into__doc__, +"S.pack_into(buffer, offset, v1, v2, ...)\n\ \n\ -Pack the values v2, v2, ... according to this Struct's format, write \n\ +Pack the values v1, v2, ... according to this Struct's format, write \n\ the packed bytes into the writable buffer buf starting at offset. Note\n\ that the offset is not an optional argument. See struct.__doc__ for \n\ more on format strings."); static PyObject * -s_pack_to(PyObject *self, PyObject *args) +s_pack_into(PyObject *self, PyObject *args) { PyStructObject *soself; char *buffer; @@ -1613,11 +1612,10 @@ soself = (PyStructObject *)self; assert(PyStruct_Check(self)); assert(soself->s_codes != NULL); - if (args == NULL || !PyTuple_Check(args) || - PyTuple_GET_SIZE(args) != (soself->s_len + 2)) + if (PyTuple_GET_SIZE(args) != (soself->s_len + 2)) { PyErr_Format(StructError, - "pack_to requires exactly %zd arguments", + "pack_into requires exactly %zd arguments", (soself->s_len + 2)); return NULL; } @@ -1630,7 +1628,7 @@ assert( buffer_len >= 0 ); /* Extract the offset from the first argument */ - offset = PyInt_AsLong(PyTuple_GET_ITEM(args, 1)); + offset = PyInt_AsSsize_t(PyTuple_GET_ITEM(args, 1)); /* Support negative offsets. */ if (offset < 0) @@ -1639,7 +1637,7 @@ /* Check boundaries */ if (offset < 0 || (buffer_len - offset) < soself->s_size) { PyErr_Format(StructError, - "pack_to requires a buffer of at least %zd bytes", + "pack_into requires a buffer of at least %zd bytes", soself->s_size); return NULL; } @@ -1668,10 +1666,10 @@ /* List of functions */ static struct PyMethodDef s_methods[] = { - {"pack", (PyCFunction)s_pack, METH_VARARGS, s_pack__doc__}, - {"pack_to", (PyCFunction)s_pack_to, METH_VARARGS, s_pack_to__doc__}, - {"unpack", (PyCFunction)s_unpack, METH_O, s_unpack__doc__}, - {"unpack_from", (PyCFunction)s_unpack_from, METH_KEYWORDS, s_unpack_from__doc__}, + {"pack", s_pack, METH_VARARGS, s_pack__doc__}, + {"pack_into", s_pack_into, METH_VARARGS, s_pack_into__doc__}, + {"unpack", s_unpack, METH_O, s_unpack__doc__}, + {"unpack_from", s_unpack_from, METH_KEYWORDS, s_unpack_from__doc__}, {NULL, NULL} /* sentinel */ }; Modified: python/trunk/Modules/socketmodule.c ============================================================================== --- python/trunk/Modules/socketmodule.c (original) +++ python/trunk/Modules/socketmodule.c Sun Jun 4 15:49:49 2006 @@ -104,9 +104,9 @@ listen(n) -- start listening for incoming connections\n\ makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ recv(buflen[, flags]) -- receive data\n\ -recv_buf(buffer[, nbytes[, flags]]) -- receive data (into a buffer)\n\ +recv_into(buffer[, nbytes[, flags]]) -- receive data (into a buffer)\n\ recvfrom(buflen[, flags]) -- receive data and sender\'s address\n\ -recvfrom_buf(buffer[, nbytes, [, flags])\n\ +recvfrom_into(buffer[, nbytes, [, flags])\n\ -- receive data and sender\'s address (into a buffer)\n\ sendall(data[, flags]) -- send all data\n\ send(data[, flags]) -- send data, may not send all of it\n\ @@ -2139,17 +2139,18 @@ #endif /* NO_DUP */ /* - * This is the guts of the recv() and recv_buf() methods, which reads into a + * This is the guts of the recv() and recv_into() methods, which reads into a * char buffer. If you have any inc/def ref to do to the objects that contain * the buffer, do it in the caller. This function returns the number of bytes * succesfully read. If there was an error, it returns -1. Note that it is * also possible that we return a number of bytes smaller than the request * bytes. */ -static int +static ssize_t sock_recv_guts(PySocketSockObject *s, char* cbuf, int len, int flags) { - int timeout, outlen = 0; + ssize_t outlen = 0; + int timeout; #ifdef __VMS int remaining, nread; char *read_buf; @@ -2225,7 +2226,8 @@ static PyObject * sock_recv(PySocketSockObject *s, PyObject *args) { - int recvlen, flags = 0, outlen; + int recvlen, flags = 0; + ssize_t outlen; PyObject *buf; if (!PyArg_ParseTuple(args, "i|i:recv", &recvlen, &flags)) @@ -2243,7 +2245,7 @@ return NULL; /* Call the guts */ - outlen = sock_recv_guts(s, PyString_AsString(buf), recvlen, flags); + outlen = sock_recv_guts(s, PyString_AS_STRING(buf), recvlen, flags); if (outlen < 0) { /* An error occured, release the string and return an error. */ @@ -2270,19 +2272,20 @@ the remote end is closed and all data is read, return the empty string."); -/* s.recv_buf(buffer, [nbytes [,flags]]) method */ +/* s.recv_into(buffer, [nbytes [,flags]]) method */ static PyObject* -sock_recv_buf(PySocketSockObject *s, PyObject *args, PyObject *kwds) +sock_recv_into(PySocketSockObject *s, PyObject *args, PyObject *kwds) { static char *kwlist[] = {"buffer", "nbytes", "flags", 0}; - int recvlen = 0, flags = 0, readlen; + int recvlen = 0, flags = 0; + ssize_t readlen; char *buf; int buflen; /* Get the buffer's memory */ - if (!PyArg_ParseTupleAndKeywords(args, kwds, "s#|ii:recv", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kwds, "w#|ii:recv", kwlist, &buf, &buflen, &recvlen, &flags)) return NULL; assert(buf != 0 && buflen > 0); @@ -2313,11 +2316,11 @@ /* Return the number of bytes read. Note that we do not do anything special here in the case that readlen < recvlen. */ - return PyInt_FromLong(readlen); + return PyInt_FromSsize_t(readlen); } -PyDoc_STRVAR(recv_buf_doc, -"recv_buf(buffer, [nbytes[, flags]]) -> nbytes_read\n\ +PyDoc_STRVAR(recv_into_doc, +"recv_into(buffer, [nbytes[, flags]]) -> nbytes_read\n\ \n\ A version of recv() that stores its data into a buffer rather than creating \n\ a new string. Receive up to buffersize bytes from the socket. If buffersize \n\ @@ -2327,7 +2330,7 @@ /* - * This is the guts of the recv() and recv_buf() methods, which reads into a + * This is the guts of the recv() and recv_into() methods, which reads into a * char buffer. If you have any inc/def ref to do to the objects that contain * the buffer, do it in the caller. This function returns the number of bytes * succesfully read. If there was an error, it returns -1. Note that it is @@ -2337,12 +2340,13 @@ * 'addr' is a return value for the address object. Note that you must decref * it yourself. */ -static int +static ssize_t sock_recvfrom_guts(PySocketSockObject *s, char* cbuf, int len, int flags, PyObject** addr) { sock_addr_t addrbuf; - int n = 0, timeout; + int timeout; + ssize_t n = 0; socklen_t addrlen; *addr = NULL; @@ -2398,7 +2402,8 @@ PyObject *buf = NULL; PyObject *addr = NULL; PyObject *ret = NULL; - int recvlen, outlen, flags = 0; + int recvlen, flags = 0; + ssize_t outlen; if (!PyArg_ParseTuple(args, "i|i:recvfrom", &recvlen, &flags)) return NULL; @@ -2435,21 +2440,22 @@ Like recv(buffersize, flags) but also return the sender's address info."); -/* s.recvfrom_buf(buffer[, nbytes [,flags]]) method */ +/* s.recvfrom_into(buffer[, nbytes [,flags]]) method */ static PyObject * -sock_recvfrom_buf(PySocketSockObject *s, PyObject *args, PyObject* kwds) +sock_recvfrom_into(PySocketSockObject *s, PyObject *args, PyObject* kwds) { static char *kwlist[] = {"buffer", "nbytes", "flags", 0}; - int recvlen = 0, flags = 0, readlen; + int recvlen = 0, flags = 0; + ssize_t readlen; char *buf; int buflen; PyObject *addr = NULL; PyObject *ret = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwds, "s#|ii:recvfrom", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kwds, "w#|ii:recvfrom", kwlist, &buf, &buflen, &recvlen, &flags)) return NULL; assert(buf != 0 && buflen > 0); @@ -2467,22 +2473,19 @@ readlen = sock_recvfrom_guts(s, buf, recvlen, flags, &addr); if (readlen < 0) { /* Return an error */ - goto finally; + Py_XDECREF(addr); + return NULL; } /* Return the number of bytes read and the address. Note that we do not do anything special here in the case that readlen < recvlen. */ - ret = Py_BuildValue("lO", readlen, addr); - -finally: - Py_XDECREF(addr); - return ret; + return Py_BuildValue("lO", readlen, addr); } -PyDoc_STRVAR(recvfrom_buf_doc, -"recvfrom_buf(buffer[, nbytes[, flags]]) -> (nbytes, address info)\n\ +PyDoc_STRVAR(recvfrom_into_doc, +"recvfrom_into(buffer[, nbytes[, flags]]) -> (nbytes, address info)\n\ \n\ -Like recv_buf(buffer[, nbytes[, flags]]) but also return the sender's address info."); +Like recv_into(buffer[, nbytes[, flags]]) but also return the sender's address info."); /* s.send(data [,flags]) method */ @@ -2711,12 +2714,12 @@ #endif {"recv", (PyCFunction)sock_recv, METH_VARARGS, recv_doc}, - {"recv_buf", (PyCFunction)sock_recv_buf, METH_VARARGS | METH_KEYWORDS, - recv_buf_doc}, + {"recv_into", (PyCFunction)sock_recv_into, METH_VARARGS | METH_KEYWORDS, + recv_into_doc}, {"recvfrom", (PyCFunction)sock_recvfrom, METH_VARARGS, recvfrom_doc}, - {"recvfrom_buf", (PyCFunction)sock_recvfrom_buf, METH_VARARGS | METH_KEYWORDS, - recvfrom_buf_doc}, + {"recvfrom_into", (PyCFunction)sock_recvfrom_into, METH_VARARGS | METH_KEYWORDS, + recvfrom_into_doc}, {"send", (PyCFunction)sock_send, METH_VARARGS, send_doc}, {"sendall", (PyCFunction)sock_sendall, METH_VARARGS, From python-checkins at python.org Sun Jun 4 16:05:45 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 4 Jun 2006 16:05:45 +0200 (CEST) Subject: [Python-checkins] r46643 - python/trunk/Mac/OSX/Makefile.in Message-ID: <20060604140545.95C931E4003@bag.python.org> Author: ronald.oussoren Date: Sun Jun 4 16:05:28 2006 New Revision: 46643 Modified: python/trunk/Mac/OSX/Makefile.in Log: "Import" LDFLAGS in Mac/OSX/Makefile.in to ensure pythonw gets build with the right compiler flags. Modified: python/trunk/Mac/OSX/Makefile.in ============================================================================== --- python/trunk/Mac/OSX/Makefile.in (original) +++ python/trunk/Mac/OSX/Makefile.in Sun Jun 4 16:05:28 2006 @@ -9,6 +9,7 @@ LIBDEST=$(prefix)/lib/python$(VERSION) BUILDPYTHON=$(builddir)/python.exe DESTDIR= +LDFLAGS= @LDFLAGS@ # These are normally glimpsed from the previous set bindir=@exec_prefix@/bin From python-checkins at python.org Sun Jun 4 16:25:34 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 4 Jun 2006 16:25:34 +0200 (CEST) Subject: [Python-checkins] r46644 - in python/trunk: Doc/mac/undoc.tex Lib/plat-mac/Carbon/WASTEconst.py Lib/plat-mac/WASTEconst.py Mac/Demo/index.html Mac/Demo/textedit.html Mac/Demo/waste Mac/Demo/waste.html Mac/Modules/waste setup.py Message-ID: <20060604142534.1973C1E4003@bag.python.org> Author: ronald.oussoren Date: Sun Jun 4 16:24:59 2006 New Revision: 46644 Removed: python/trunk/Lib/plat-mac/Carbon/WASTEconst.py python/trunk/Lib/plat-mac/WASTEconst.py python/trunk/Mac/Demo/waste/ python/trunk/Mac/Demo/waste.html python/trunk/Mac/Modules/waste/ Modified: python/trunk/Doc/mac/undoc.tex python/trunk/Mac/Demo/index.html python/trunk/Mac/Demo/textedit.html python/trunk/setup.py Log: Drop Mac wrappers for the WASTE library. Modified: python/trunk/Doc/mac/undoc.tex ============================================================================== --- python/trunk/Doc/mac/undoc.tex (original) +++ python/trunk/Doc/mac/undoc.tex Sun Jun 4 16:24:59 2006 @@ -95,14 +95,3 @@ The \module{W} widgets are used extensively in the \program{IDE}. -\section{\module{waste} --- non-Apple \program{TextEdit} replacement} -\declaremodule{standard}{waste} - \platform{Mac} -\modulesynopsis{Interface to the ``WorldScript-Aware Styled Text Engine.''} - -\begin{seealso} - \seetitle[http://www.merzwaren.com/waste/]{About WASTE}{Information - about the WASTE widget and library, including - documentation and downloads.} -\end{seealso} - Deleted: /python/trunk/Lib/plat-mac/Carbon/WASTEconst.py ============================================================================== --- /python/trunk/Lib/plat-mac/Carbon/WASTEconst.py Sun Jun 4 16:24:59 2006 +++ (empty file) @@ -1,207 +0,0 @@ -# Generated from 'WASTE.h' - -kPascalStackBased = None # workaround for header parsing -def FOUR_CHAR_CODE(x): return x -weCantUndoErr = -10015 -weEmptySelectionErr = -10013 -weUnknownObjectTypeErr = -9478 -weObjectNotFoundErr = -9477 -weReadOnlyErr = -9476 -weTextNotFoundErr = -9474 -weInvalidTextEncodingErr = -9473 -weDuplicateAttributeErr = -9472 -weInvalidAttributeSizeErr = -9471 -weReadOnlyAttributeErr = -9470 -weOddByteCountErr = -9469 -weHandlerNotFoundErr = -1717 -weNotHandledErr = -1708 -weNewerVersionErr = -1706 -weCorruptDataErr = -1702 -weProtocolErr = -603 -weUndefinedSelectorErr = -50 -weFlushLeft = -2 -weFlushRight = -1 -weFlushDefault = 0 -weCenter = 1 -weJustify = 2 -weDirDefault = 1 -weDirRightToLeft = -1 -weDirLeftToRight = 0 -weDoFont = 0x0001 -weDoFace = 0x0002 -weDoSize = 0x0004 -weDoColor = 0x0008 -weDoAll = weDoFont | weDoFace | weDoSize | weDoColor -weDoAddSize = 0x0010 -weDoToggleFace = 0x0020 -weDoReplaceFace = 0x0040 -weDoPreserveScript = 0x0080 -weDoExtractSubscript = 0x0100 -weDoFaceMask = 0x0200 -weDoDirection = 0x00000001 -weDoAlignment = 0x00000002 -weDoLeftIndent = 0x00000004 -weDoRightIndent = 0x00000008 -weDoFirstLineIndent = 0x00000010 -weDoLineSpacing = 0x00000020 -weDoSpaceBefore = 0x00000040 -weDoSpaceAfter = 0x00000080 -weDoBottomBorderStyle = 0x00000400 -kLeadingEdge = -1 -kTrailingEdge = 0 -kObjectEdge = 2 -weFAutoScroll = 0 -weFOutlineHilite = 2 -weFReadOnly = 5 -weFUndo = 6 -weFIntCutAndPaste = 7 -weFDragAndDrop = 8 -weFInhibitRecal = 9 -weFUseTempMem = 10 -weFDrawOffscreen = 11 -weFInhibitRedraw = 12 -weFMonoStyled = 13 -weFMultipleUndo = 14 -weFNoKeyboardSync = 29 -weFInhibitICSupport = 30 -weFInhibitColor = 31 -# weDoAutoScroll = 1UL << weFAutoScroll -# weDoOutlineHilite = 1UL << weFOutlineHilite -# weDoReadOnly = 1UL << weFReadOnly -# weDoUndo = 1UL << weFUndo -# weDoIntCutAndPaste = 1UL << weFIntCutAndPaste -# weDoDragAndDrop = 1UL << weFDragAndDrop -# weDoInhibitRecal = 1UL << weFInhibitRecal -# weDoUseTempMem = 1UL << weFUseTempMem -# weDoDrawOffscreen = 1UL << weFDrawOffscreen -# weDoInhibitRedraw = 1UL << weFInhibitRedraw -# weDoMonoStyled = 1UL << weFMonoStyled -# weDoMultipleUndo = 1UL << weFMultipleUndo -# weDoNoKeyboardSync = 1UL << weFNoKeyboardSync -# weDoInhibitICSupport = 1UL << weFInhibitICSupport -# weDoInhibitColor = 1UL << weFInhibitColor -weBitToggle = -2 -weBitTest = -1 -weBitClear = 0 -weBitSet = 1 -weLowerCase = 0 -weUpperCase = 1 -weFindWholeWords = 0x00000001 -weFindCaseInsensitive = 0x00000002 -weFindDiacriticalInsensitive = 0x00000004 -wePutIntCutAndPaste = 0x00000001 -wePutAddToTypingSequence = 0x00000002 -wePutDetectUnicodeBOM = 0x00000200 -weStreamDestinationKindMask = 0x000000FF -weStreamIncludeObjects = 0x00000100 -weGetAddUnicodeBOM = 0x00000200 -weGetLittleEndian = 0x00000400 -weTagFontFamily = FOUR_CHAR_CODE('font') -weTagFontSize = FOUR_CHAR_CODE('ptsz') -weTagPlain = FOUR_CHAR_CODE('plan') -weTagBold = FOUR_CHAR_CODE('bold') -weTagItalic = FOUR_CHAR_CODE('ital') -weTagUnderline = FOUR_CHAR_CODE('undl') -weTagOutline = FOUR_CHAR_CODE('outl') -weTagShadow = FOUR_CHAR_CODE('shad') -weTagCondensed = FOUR_CHAR_CODE('cond') -weTagExtended = FOUR_CHAR_CODE('pexp') -weTagStrikethrough = FOUR_CHAR_CODE('strk') -weTagTextColor = FOUR_CHAR_CODE('colr') -weTagBackgroundColor = FOUR_CHAR_CODE('pbcl') -weTagTransferMode = FOUR_CHAR_CODE('pptm') -weTagVerticalShift = FOUR_CHAR_CODE('xshf') -weTagAlignment = FOUR_CHAR_CODE('pjst') -weTagDirection = FOUR_CHAR_CODE('LDIR') -weTagLineSpacing = FOUR_CHAR_CODE('ledg') -weTagLeftIndent = FOUR_CHAR_CODE('lein') -weTagRightIndent = FOUR_CHAR_CODE('riin') -weTagFirstLineIndent = FOUR_CHAR_CODE('fidt') -weTagSpaceBefore = FOUR_CHAR_CODE('spbe') -weTagSpaceAfter = FOUR_CHAR_CODE('spaf') -weTagBottomBorderStyle = FOUR_CHAR_CODE('BBRD') -weTagForceFontFamily = FOUR_CHAR_CODE('ffnt') -weTagAddFontSize = FOUR_CHAR_CODE('+siz') -weTagAddVerticalShift = FOUR_CHAR_CODE('+shf') -weTagTextEncoding = FOUR_CHAR_CODE('ptxe') -weTagQDStyles = FOUR_CHAR_CODE('qdst') -weTagTETextStyle = FOUR_CHAR_CODE('tets') -weTagAlignmentDefault = FOUR_CHAR_CODE('deft') -weTagAlignmentLeft = FOUR_CHAR_CODE('left') -weTagAlignmentCenter = FOUR_CHAR_CODE('cent') -weTagAlignmentRight = FOUR_CHAR_CODE('rght') -weTagAlignmentFull = FOUR_CHAR_CODE('full') -weTagDirectionDefault = FOUR_CHAR_CODE('deft') -weTagDirectionLeftToRight = FOUR_CHAR_CODE('L->R') -weTagDirectionRightToLeft = FOUR_CHAR_CODE('R->L') -weTagBorderStyleNone = FOUR_CHAR_CODE('NONE') -weTagBorderStyleThin = FOUR_CHAR_CODE('SLDL') -weTagBorderStyleDotted = FOUR_CHAR_CODE('DTDL') -weTagBorderStyleThick = FOUR_CHAR_CODE('THKL') -weLineSpacingSingle = 0x00000000 -weLineSpacingOneAndHalf = 0x00008000 -weLineSpacingDouble = 0x00010000 -weCharByteHook = FOUR_CHAR_CODE('cbyt') -weCharToPixelHook = FOUR_CHAR_CODE('c2p ') -weCharTypeHook = FOUR_CHAR_CODE('ctyp') -weClickLoop = FOUR_CHAR_CODE('clik') -weCurrentDrag = FOUR_CHAR_CODE('drag') -weDrawTextHook = FOUR_CHAR_CODE('draw') -weDrawTSMHiliteHook = FOUR_CHAR_CODE('dtsm') -weEraseHook = FOUR_CHAR_CODE('eras') -weFontFamilyToNameHook = FOUR_CHAR_CODE('ff2n') -weFontNameToFamilyHook = FOUR_CHAR_CODE('fn2f') -weFluxProc = FOUR_CHAR_CODE('flux') -weHiliteDropAreaHook = FOUR_CHAR_CODE('hidr') -weLineBreakHook = FOUR_CHAR_CODE('lbrk') -wePixelToCharHook = FOUR_CHAR_CODE('p2c ') -wePort = FOUR_CHAR_CODE('port') -wePreTrackDragHook = FOUR_CHAR_CODE('ptrk') -weRefCon = FOUR_CHAR_CODE('refc') -weScrollProc = FOUR_CHAR_CODE('scrl') -weText = FOUR_CHAR_CODE('text') -weTranslateDragHook = FOUR_CHAR_CODE('xdrg') -weTranslucencyThreshold = FOUR_CHAR_CODE('tluc') -weTSMDocumentID = FOUR_CHAR_CODE('tsmd') -weTSMPreUpdate = FOUR_CHAR_CODE('pre ') -weTSMPostUpdate = FOUR_CHAR_CODE('post') -weURLHint = FOUR_CHAR_CODE('urlh') -weWordBreakHook = FOUR_CHAR_CODE('wbrk') -weNewHandler = FOUR_CHAR_CODE('new ') -weDisposeHandler = FOUR_CHAR_CODE('free') -weDrawHandler = FOUR_CHAR_CODE('draw') -weClickHandler = FOUR_CHAR_CODE('clik') -weStreamHandler = FOUR_CHAR_CODE('strm') -weHoverHandler = FOUR_CHAR_CODE('hovr') -kTypeText = FOUR_CHAR_CODE('TEXT') -kTypeStyles = FOUR_CHAR_CODE('styl') -kTypeSoup = FOUR_CHAR_CODE('SOUP') -kTypeFontTable = FOUR_CHAR_CODE('FISH') -kTypeParaFormat = FOUR_CHAR_CODE('WEpf') -kTypeRulerScrap = FOUR_CHAR_CODE('WEru') -kTypeCharFormat = FOUR_CHAR_CODE('WEcf') -kTypeStyleScrap = FOUR_CHAR_CODE('WEst') -kTypeUnicodeText = FOUR_CHAR_CODE('utxt') -kTypeUTF8Text = FOUR_CHAR_CODE('UTF8') -kTypeStyledText = FOUR_CHAR_CODE('STXT') -weAKNone = 0 -weAKUnspecified = 1 -weAKTyping = 2 -weAKCut = 3 -weAKPaste = 4 -weAKClear = 5 -weAKDrag = 6 -weAKSetStyle = 7 -weAKSetRuler = 8 -weAKBackspace = 9 -weAKFwdDelete = 10 -weAKCaseChange = 11 -weAKObjectChange = 12 -weToScrap = 0 -weToDrag = 1 -weToSoup = 2 -weMouseEnter = 0 -weMouseWithin = 1 -weMouseLeave = 2 -kCurrentSelection = -1 -kNullStyle = -2 Deleted: /python/trunk/Lib/plat-mac/WASTEconst.py ============================================================================== --- /python/trunk/Lib/plat-mac/WASTEconst.py Sun Jun 4 16:24:59 2006 +++ (empty file) @@ -1,207 +0,0 @@ -# Generated from 'WASTE.h' - -kPascalStackBased = None # workaround for header parsing -def FOUR_CHAR_CODE(x): return x -weCantUndoErr = -10015 -weEmptySelectionErr = -10013 -weUnknownObjectTypeErr = -9478 -weObjectNotFoundErr = -9477 -weReadOnlyErr = -9476 -weTextNotFoundErr = -9474 -weInvalidTextEncodingErr = -9473 -weDuplicateAttributeErr = -9472 -weInvalidAttributeSizeErr = -9471 -weReadOnlyAttributeErr = -9470 -weOddByteCountErr = -9469 -weHandlerNotFoundErr = -1717 -weNotHandledErr = -1708 -weNewerVersionErr = -1706 -weCorruptDataErr = -1702 -weProtocolErr = -603 -weUndefinedSelectorErr = -50 -weFlushLeft = -2 -weFlushRight = -1 -weFlushDefault = 0 -weCenter = 1 -weJustify = 2 -weDirDefault = 1 -weDirRightToLeft = -1 -weDirLeftToRight = 0 -weDoFont = 0x0001 -weDoFace = 0x0002 -weDoSize = 0x0004 -weDoColor = 0x0008 -weDoAll = weDoFont | weDoFace | weDoSize | weDoColor -weDoAddSize = 0x0010 -weDoToggleFace = 0x0020 -weDoReplaceFace = 0x0040 -weDoPreserveScript = 0x0080 -weDoExtractSubscript = 0x0100 -weDoFaceMask = 0x0200 -weDoDirection = 0x00000001 -weDoAlignment = 0x00000002 -weDoLeftIndent = 0x00000004 -weDoRightIndent = 0x00000008 -weDoFirstLineIndent = 0x00000010 -weDoLineSpacing = 0x00000020 -weDoSpaceBefore = 0x00000040 -weDoSpaceAfter = 0x00000080 -weDoBottomBorderStyle = 0x00000400 -kLeadingEdge = -1 -kTrailingEdge = 0 -kObjectEdge = 2 -weFAutoScroll = 0 -weFOutlineHilite = 2 -weFReadOnly = 5 -weFUndo = 6 -weFIntCutAndPaste = 7 -weFDragAndDrop = 8 -weFInhibitRecal = 9 -weFUseTempMem = 10 -weFDrawOffscreen = 11 -weFInhibitRedraw = 12 -weFMonoStyled = 13 -weFMultipleUndo = 14 -weFNoKeyboardSync = 29 -weFInhibitICSupport = 30 -weFInhibitColor = 31 -weDoAutoScroll = 1 << weFAutoScroll -weDoOutlineHilite = 1 << weFOutlineHilite -weDoReadOnly = 1 << weFReadOnly -weDoUndo = 1 << weFUndo -weDoIntCutAndPaste = 1 << weFIntCutAndPaste -weDoDragAndDrop = 1 << weFDragAndDrop -weDoInhibitRecal = 1 << weFInhibitRecal -weDoUseTempMem = 1 << weFUseTempMem -weDoDrawOffscreen = 1 << weFDrawOffscreen -weDoInhibitRedraw = 1 << weFInhibitRedraw -weDoMonoStyled = 1 << weFMonoStyled -weDoMultipleUndo = 1 << weFMultipleUndo -weDoNoKeyboardSync = 1 << weFNoKeyboardSync -weDoInhibitICSupport = 1 << weFInhibitICSupport -# weDoInhibitColor = 1 << weFInhibitColor -weBitToggle = -2 -weBitTest = -1 -weBitClear = 0 -weBitSet = 1 -weLowerCase = 0 -weUpperCase = 1 -weFindWholeWords = 0x00000001 -weFindCaseInsensitive = 0x00000002 -weFindDiacriticalInsensitive = 0x00000004 -wePutIntCutAndPaste = 0x00000001 -wePutAddToTypingSequence = 0x00000002 -wePutDetectUnicodeBOM = 0x00000200 -weStreamDestinationKindMask = 0x000000FF -weStreamIncludeObjects = 0x00000100 -weGetAddUnicodeBOM = 0x00000200 -weGetLittleEndian = 0x00000400 -weTagFontFamily = FOUR_CHAR_CODE('font') -weTagFontSize = FOUR_CHAR_CODE('ptsz') -weTagPlain = FOUR_CHAR_CODE('plan') -weTagBold = FOUR_CHAR_CODE('bold') -weTagItalic = FOUR_CHAR_CODE('ital') -weTagUnderline = FOUR_CHAR_CODE('undl') -weTagOutline = FOUR_CHAR_CODE('outl') -weTagShadow = FOUR_CHAR_CODE('shad') -weTagCondensed = FOUR_CHAR_CODE('cond') -weTagExtended = FOUR_CHAR_CODE('pexp') -weTagStrikethrough = FOUR_CHAR_CODE('strk') -weTagTextColor = FOUR_CHAR_CODE('colr') -weTagBackgroundColor = FOUR_CHAR_CODE('pbcl') -weTagTransferMode = FOUR_CHAR_CODE('pptm') -weTagVerticalShift = FOUR_CHAR_CODE('xshf') -weTagAlignment = FOUR_CHAR_CODE('pjst') -weTagDirection = FOUR_CHAR_CODE('LDIR') -weTagLineSpacing = FOUR_CHAR_CODE('ledg') -weTagLeftIndent = FOUR_CHAR_CODE('lein') -weTagRightIndent = FOUR_CHAR_CODE('riin') -weTagFirstLineIndent = FOUR_CHAR_CODE('fidt') -weTagSpaceBefore = FOUR_CHAR_CODE('spbe') -weTagSpaceAfter = FOUR_CHAR_CODE('spaf') -weTagBottomBorderStyle = FOUR_CHAR_CODE('BBRD') -weTagForceFontFamily = FOUR_CHAR_CODE('ffnt') -weTagAddFontSize = FOUR_CHAR_CODE('+siz') -weTagAddVerticalShift = FOUR_CHAR_CODE('+shf') -weTagTextEncoding = FOUR_CHAR_CODE('ptxe') -weTagQDStyles = FOUR_CHAR_CODE('qdst') -weTagTETextStyle = FOUR_CHAR_CODE('tets') -weTagAlignmentDefault = FOUR_CHAR_CODE('deft') -weTagAlignmentLeft = FOUR_CHAR_CODE('left') -weTagAlignmentCenter = FOUR_CHAR_CODE('cent') -weTagAlignmentRight = FOUR_CHAR_CODE('rght') -weTagAlignmentFull = FOUR_CHAR_CODE('full') -weTagDirectionDefault = FOUR_CHAR_CODE('deft') -weTagDirectionLeftToRight = FOUR_CHAR_CODE('L->R') -weTagDirectionRightToLeft = FOUR_CHAR_CODE('R->L') -weTagBorderStyleNone = FOUR_CHAR_CODE('NONE') -weTagBorderStyleThin = FOUR_CHAR_CODE('SLDL') -weTagBorderStyleDotted = FOUR_CHAR_CODE('DTDL') -weTagBorderStyleThick = FOUR_CHAR_CODE('THKL') -weLineSpacingSingle = 0x00000000 -weLineSpacingOneAndHalf = 0x00008000 -weLineSpacingDouble = 0x00010000 -weCharByteHook = FOUR_CHAR_CODE('cbyt') -weCharToPixelHook = FOUR_CHAR_CODE('c2p ') -weCharTypeHook = FOUR_CHAR_CODE('ctyp') -weClickLoop = FOUR_CHAR_CODE('clik') -weCurrentDrag = FOUR_CHAR_CODE('drag') -weDrawTextHook = FOUR_CHAR_CODE('draw') -weDrawTSMHiliteHook = FOUR_CHAR_CODE('dtsm') -weEraseHook = FOUR_CHAR_CODE('eras') -weFontFamilyToNameHook = FOUR_CHAR_CODE('ff2n') -weFontNameToFamilyHook = FOUR_CHAR_CODE('fn2f') -weFluxProc = FOUR_CHAR_CODE('flux') -weHiliteDropAreaHook = FOUR_CHAR_CODE('hidr') -weLineBreakHook = FOUR_CHAR_CODE('lbrk') -wePixelToCharHook = FOUR_CHAR_CODE('p2c ') -wePort = FOUR_CHAR_CODE('port') -wePreTrackDragHook = FOUR_CHAR_CODE('ptrk') -weRefCon = FOUR_CHAR_CODE('refc') -weScrollProc = FOUR_CHAR_CODE('scrl') -weText = FOUR_CHAR_CODE('text') -weTranslateDragHook = FOUR_CHAR_CODE('xdrg') -weTranslucencyThreshold = FOUR_CHAR_CODE('tluc') -weTSMDocumentID = FOUR_CHAR_CODE('tsmd') -weTSMPreUpdate = FOUR_CHAR_CODE('pre ') -weTSMPostUpdate = FOUR_CHAR_CODE('post') -weURLHint = FOUR_CHAR_CODE('urlh') -weWordBreakHook = FOUR_CHAR_CODE('wbrk') -weNewHandler = FOUR_CHAR_CODE('new ') -weDisposeHandler = FOUR_CHAR_CODE('free') -weDrawHandler = FOUR_CHAR_CODE('draw') -weClickHandler = FOUR_CHAR_CODE('clik') -weStreamHandler = FOUR_CHAR_CODE('strm') -weHoverHandler = FOUR_CHAR_CODE('hovr') -kTypeText = FOUR_CHAR_CODE('TEXT') -kTypeStyles = FOUR_CHAR_CODE('styl') -kTypeSoup = FOUR_CHAR_CODE('SOUP') -kTypeFontTable = FOUR_CHAR_CODE('FISH') -kTypeParaFormat = FOUR_CHAR_CODE('WEpf') -kTypeRulerScrap = FOUR_CHAR_CODE('WEru') -kTypeCharFormat = FOUR_CHAR_CODE('WEcf') -kTypeStyleScrap = FOUR_CHAR_CODE('WEst') -kTypeUnicodeText = FOUR_CHAR_CODE('utxt') -kTypeUTF8Text = FOUR_CHAR_CODE('UTF8') -kTypeStyledText = FOUR_CHAR_CODE('STXT') -weAKNone = 0 -weAKUnspecified = 1 -weAKTyping = 2 -weAKCut = 3 -weAKPaste = 4 -weAKClear = 5 -weAKDrag = 6 -weAKSetStyle = 7 -weAKSetRuler = 8 -weAKBackspace = 9 -weAKFwdDelete = 10 -weAKCaseChange = 11 -weAKObjectChange = 12 -weToScrap = 0 -weToDrag = 1 -weToSoup = 2 -weMouseEnter = 0 -weMouseWithin = 1 -weMouseLeave = 2 -kCurrentSelection = -1 -kNullStyle = -2 Modified: python/trunk/Mac/Demo/index.html ============================================================================== --- python/trunk/Mac/Demo/index.html (original) +++ python/trunk/Mac/Demo/index.html Sun Jun 4 16:24:59 2006 @@ -74,10 +74,6 @@ TextEdit toolbox to build a text editor.

-Using WASTE expands on this editor by using -WASTE, an extended TextEdit replacement. - -

+as if you can resize the window but it does not work. Still, it serves as an example. Ped creates two classes, TEWindow and Ped. Let us start with the latter one, which is a subclass of FrameWork.Application and our main application. The init function Deleted: /python/trunk/Mac/Demo/waste.html ============================================================================== --- /python/trunk/Mac/Demo/waste.html Sun Jun 4 16:24:59 2006 +++ (empty file) @@ -1,72 +0,0 @@ -Using WASTE - -

Using WASTE

- -WASTE is an almost-compatible TextEdit replacement which overcomes -some of the limitations of it (like the 32K limit) and provides some extensions -(drag and drop, images, undo support). Moreover, it has a much cleaner interface -and is therefore easier integrated in Python.

- -WASTE is written by Marco Piovanelli, <piovanel at kagi.com>, -and copyrighted by him. You can always obtain the latest version (for use in C -or Pascal programs) and the documentation from -<http://www.boingo.com/waste/>. - -We explain the useage of waste here by showing how to modify the TextEdit based -ped.py of the -previous example into the waste-based wed.py, -so you should have both sources handy.

- -Functionally, wed.py provides three new things: resizable windows, a horizontal -scroll bar and undo.

- -Let us look at the code, first at the application class Wed. The only real change is that -we now handle undo. Aside from enabling it in the creation routine and the addition of -a callback routine there is a bit of new code in updatemenubar: Waste not only handles -the full details of implementing undo, it will also tell us what the next undo operation will undo -(or redo). We use this to our advantage by changing the undo menu label to tell the user.

- -The WasteWindow has seen a bit more change. Initialization of the waste data structure is -a bit different, in that we can specify some options at creation time. Also, waste has no SetText -method but a UseText which expects a handle as parameter. We have to be very careful -that we keep this handle around, because Python will happily free the handle if we have no more references -to it (and I doubt that Waste would like this:-). A final difference in open -is that we use a large number for the destination rectangle width, because we will use a horizontal scroll -bar.

- -The idle method is a bit more involved, since we also call WEAdjustCursor to -provide the correct cursor based on mouse-position. Users like this.

- -Getscrollbarvalues is simpler than its' TextEdit counterpart because Waste correctly -updates the destination rectangle when the document changes. Also note that waste uses accessor functions -to get at internal values, as opposed to direct struct access for TextEdit.

- -Scrollbar_callback on the other hand is more elaborate (but also provides more functionality). -It also handles horizontal scrolls (scrolling one-tenth and half a screenful with the buttons). This -function is also "multi-font-ready" in that scrolling one line will do the expected thing in case of multiple -fonts. We will implement a multi-font editor later. A minor annoyance of Waste is that is does not provide -a pinned scroll, so at the end of our callback routine we have to check that we have not scrolled past the -beginning or end of the document, and adjust when needed.

- -do_update is also changed, because Waste is completely region-based (as opposed to rect-based). -Hence, we erase regions here and we can also return immedeately if there is nothing to update.

- -Do_postresize is new: because Waste uses accessor functions we can now modify the viewRect from -Python, which is impossible in the Python TextEdit interface, and hence we can implement resize. The -do_key and do_contentclick methods have also seen minor changes, because the -corresponding waste routines need a bit more information than their TextEdit counterparts. The Cut/copy/paste -code is simplified, because Waste uses the normal desktop scrap.

- -Implementing undo is a wonder of simplicity: Waste handles all the details for us. Also, the new -can_paste method (which controls greying out of the paste menu entry) is an improvement -over what ped did: in ped it was possible that paste was enabled but that the data on the -scrap was incompatible with TextEdit. No more such problems here.

- -That is all for now. There is an undocumented extended version of wed, swed.py, -which supports multiple fonts, sizes and faces, and uses Waste's tab-calculation to do tab characters "right". -There is also an even more elaborate example, htmled.py which extends swed with -the ability to import html files, showing the use of color and how to use embedded object (rulers, in this case). -These two programs have not been documented yet, though, so you will have to look at them without guidance.

-Back to the index to pick another example. Modified: python/trunk/setup.py ============================================================================== --- python/trunk/setup.py (original) +++ python/trunk/setup.py Sun Jun 4 16:24:59 2006 @@ -1095,29 +1095,6 @@ extra_link_args=['-framework', 'QuickTime', '-framework', 'Carbon']) ) - # As there is no standardized place (yet) to put - # user-installed Mac libraries on OSX, we search for "waste" - # in parent directories of the Python source tree. You - # should put a symlink to your Waste installation in the - # same folder as your python source tree. Or modify the - # next few lines:-) - waste_incs = find_file("WASTE.h", [], - ['../'*n + 'waste/C_C++ Headers' for n in (0,1,2,3,4)]) - waste_libs = find_library_file(self.compiler, "WASTE", [], - ["../"*n + "waste/Static Libraries" for n in (0,1,2,3,4)]) - if waste_incs != None and waste_libs != None: - exts.append( Extension('waste', - ['waste/wastemodule.c'] + [ - os.path.join(srcdir, d) for d in - 'Mac/Wastemods/WEObjectHandlers.c', - 'Mac/Wastemods/WETabHooks.c', - 'Mac/Wastemods/WETabs.c' - ], - include_dirs = waste_incs + [os.path.join(srcdir, 'Mac/Wastemods')], - library_dirs = waste_libs, - libraries = ['WASTE'], - extra_link_args = ['-framework', 'Carbon'], - ) ) self.extensions.extend(exts) From buildbot at python.org Sun Jun 4 16:56:49 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 14:56:49 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP trunk Message-ID: <20060604145651.8B2F91E4003@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/923 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 16:58:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 14:58:14 +0000 Subject: [Python-checkins] buildbot failure in x86 XP trunk Message-ID: <20060604145814.B20E51E4003@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/924 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 17:49:41 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 4 Jun 2006 17:49:41 +0200 (CEST) Subject: [Python-checkins] r46645 - python/trunk/Modules/_struct.c Message-ID: <20060604154941.590861E4003@bag.python.org> Author: tim.peters Date: Sun Jun 4 17:49:07 2006 New Revision: 46645 Modified: python/trunk/Modules/_struct.c Log: s_methods[]: Stop compiler warnings by casting s_unpack_from to PyCFunction. Modified: python/trunk/Modules/_struct.c ============================================================================== --- python/trunk/Modules/_struct.c (original) +++ python/trunk/Modules/_struct.c Sun Jun 4 17:49:07 2006 @@ -1669,7 +1669,8 @@ {"pack", s_pack, METH_VARARGS, s_pack__doc__}, {"pack_into", s_pack_into, METH_VARARGS, s_pack_into__doc__}, {"unpack", s_unpack, METH_O, s_unpack__doc__}, - {"unpack_from", s_unpack_from, METH_KEYWORDS, s_unpack_from__doc__}, + {"unpack_from", (PyCFunction)s_unpack_from, METH_KEYWORDS, + s_unpack_from__doc__}, {NULL, NULL} /* sentinel */ }; From buildbot at python.org Sun Jun 4 18:52:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 16:52:37 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060604165251.9CD561E4012@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/535 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From tim.peters at gmail.com Sun Jun 4 18:47:50 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 4 Jun 2006 12:47:50 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) In-Reply-To: <43E59BC0-9CD0-4524-9DB3-6CFD5E155721@commonground.com.au> References: <20060604090704.GA17397@python.psfb.org> <43E59BC0-9CD0-4524-9DB3-6CFD5E155721@commonground.com.au> Message-ID: <1f7befae0606040947i7b3fdbc3p23ba363a4103cd24@mail.gmail.com> [Neal Norwitz] >> test_exceptions leaked [0, 2, 0] references >> test_filecmp leaked [13, 0, 0] references [Richard Jones] > What's producing this output? While I've never run it, I believe it's produced by Misc/build.sh. > On what OS? Windows 3.1 with Neal's cool hack to implement the Bourne shell as a Windows device driver ;-) > I'd like to try to find that leak in exceptions if possible. There probably isn't a leak. You can run the relevant bits "by hand" in isolation (here on Windows, but it's rare for a leak to be platform specific -- note that you need a debug build of Python to use -R): $ python_d ../Lib/test/regrtest.py -R 2:40: test_filecmp test_exceptions test_filecmp beginning 42 repetitions 123456789012345678901234567890123456789012 .......................................... test_exceptions beginning 42 repetitions 123456789012345678901234567890123456789012 .......................................... All 2 tests OK. [25878 refs] No leaks. It's one of the charms of -R that its leak output (when it produces some) can be very hard to track down. Here I ran exactly the same thing a second time, but the output differed: $ python_d ../Lib/test/regrtest.py -R 2:40: test_filecmp test_exceptions test_filecmp beginning 42 repetitions 123456789012345678901234567890123456789012 .......................................... test_filecmp leaked [0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] references test_exceptions beginning 42 repetitions 123456789012345678901234567890123456789012 .......................................... All 2 tests OK. [25883 refs] After about 10 more tries at that, yet a third outcome showed up: $ python_d ../Lib/test/regrtest.py -R 2:40: test_filecmp test_exceptions test_filecmp beginning 42 repetitions 123456789012345678901234567890123456789012 .......................................... test_exceptions beginning 42 repetitions 123456789012345678901234567890123456789012 .......................................... test_exceptions leaked [0, 203, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] references All 2 tests OK. [25883 refs] Does that make any sense? Not to me -- I don't know of a clear reason other than wild loads/stores for why such runs should ever differ. From python-checkins at python.org Sun Jun 4 19:04:26 2006 From: python-checkins at python.org (george.yoshida) Date: Sun, 4 Jun 2006 19:04:26 +0200 (CEST) Subject: [Python-checkins] r46646 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060604170426.C70921E400D@bag.python.org> Author: george.yoshida Date: Sun Jun 4 19:04:12 2006 New Revision: 46646 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Remove a redundant word Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Sun Jun 4 19:04:12 2006 @@ -1358,7 +1358,6 @@ now support a \code{key} keyword parameter similar to the one provided by the \function{min()}/\function{max()} functions and the \method{sort()} methods. For example: -Example: \begin{verbatim} >>> import heapq From buildbot at python.org Sun Jun 4 19:08:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 17:08:37 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060604170912.597041E4003@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/278 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre,martin.blais,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 4 19:18:44 2006 From: python-checkins at python.org (george.yoshida) Date: Sun, 4 Jun 2006 19:18:44 +0200 (CEST) Subject: [Python-checkins] r46647 - python/trunk/Doc/lib/libthread.tex python/trunk/Doc/lib/libthreading.tex Message-ID: <20060604171844.6D4041E4003@bag.python.org> Author: george.yoshida Date: Sun Jun 4 19:17:25 2006 New Revision: 46647 Modified: python/trunk/Doc/lib/libthread.tex python/trunk/Doc/lib/libthreading.tex Log: Markup fix Modified: python/trunk/Doc/lib/libthread.tex ============================================================================== --- python/trunk/Doc/lib/libthread.tex (original) +++ python/trunk/Doc/lib/libthread.tex Sun Jun 4 19:17:25 2006 @@ -80,9 +80,9 @@ subsequently created threads, and must be 0 (use platform or configured default) or a positive integer value of at least 32,768 (32kB). If changing the thread stack size is unsupported, or the specified size -is invalid, a RuntimeWarning is issued and the stack size is unmodified. -32kB is currently the minimum supported stack size value, to guarantee -sufficient stack space for the interpreter itself. +is invalid, a \exception{RuntimeWarning} is issued and the stack size is +unmodified. 32kB is currently the minimum supported stack size value, +to guarantee sufficient stack space for the interpreter itself. Note that some platforms may have particular restrictions on values for the stack size, such as requiring allocation in multiples of the system memory page size - platform documentation should be referred to for more Modified: python/trunk/Doc/lib/libthreading.tex ============================================================================== --- python/trunk/Doc/lib/libthreading.tex (original) +++ python/trunk/Doc/lib/libthreading.tex Sun Jun 4 19:17:25 2006 @@ -131,9 +131,9 @@ subsequently created threads, and must be 0 (use platform or configured default) or a positive integer value of at least 32,768 (32kB). If changing the thread stack size is unsupported, or the specified size -is invalid, a RuntimeWarning is issued and the stack size is unmodified. -32kB is currently the minimum supported stack size value, to guarantee -sufficient stack space for the interpreter itself. +is invalid, a \exception{RuntimeWarning} is issued and the stack size is +unmodified. 32kB is currently the minimum supported stack size value, +to guarantee sufficient stack space for the interpreter itself. Note that some platforms may have particular restrictions on values for the stack size, such as requiring allocation in multiples of the system memory page size - platform documentation should be referred to for more From nnorwitz at gmail.com Sun Jun 4 19:19:00 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 4 Jun 2006 10:19:00 -0700 Subject: [Python-checkins] r46640 - in python/trunk: Doc/lib/libthread.tex Doc/lib/libthreading.tex Include/pythread.h Lib/dummy_thread.py Lib/test/output/test_thread Lib/test/test_thread.py Lib/test/test_threading.py Lib/threading.py Misc/NEWS Modules/th Message-ID: On 6/4/06, andrew.macintyre wrote: > Author: andrew.macintyre > Date: Sun Jun 4 14:31:09 2006 > New Revision: 46640 > > Modified: python/trunk/Modules/threadmodule.c > ============================================================================== > --- python/trunk/Modules/threadmodule.c (original) > +++ python/trunk/Modules/threadmodule.c Sun Jun 4 14:31:09 2006 > @@ -586,6 +586,51 @@ > be relied upon, and the number should be seen purely as a magic cookie.\n\ > A thread's identity may be reused for another thread after it exits."); > > +static PyObject * > +thread_stack_size(PyObject *self, PyObject *args) > +{ > + size_t old_size, new_size; > + PyObject *set_size = NULL; > + > + if (!PyArg_UnpackTuple(args, "stack_size", 0, 1, &set_size)) > + return NULL; > + > + old_size = PyThread_get_stacksize(); > + > + if (set_size != NULL) { > + if (PyInt_Check(set_size)) > + new_size = (size_t) PyInt_AsLong(set_size); > + else { > + PyErr_SetString(PyExc_TypeError, > + "size must be an integer"); > + return NULL; > + } > + if (PyThread_set_stacksize(new_size)) > + return NULL; > + } > + > + return PyInt_FromLong((long) old_size); > +} You should accept PyLongs too and deal with Py_ssize_t features also. I don't think you should need any casts. > Modified: python/trunk/Python/thread.c > ============================================================================== > --- python/trunk/Python/thread.c (original) > +++ python/trunk/Python/thread.c Sun Jun 4 14:31:09 2006 > @@ -94,6 +94,31 @@ > PyThread__init_thread(); > } > > +/* Support for runtime thread stack size tuning. > + A value of 0 means using the platform's default stack size > + or the size specified by the THREAD_STACK_SIZE macro. */ > +static size_t _pythread_stacksize = 0; > + > +size_t > +PyThread_get_stacksize(void) > +{ > + return _pythread_stacksize; > +} > + > +static int > +_pythread_unsupported_set_stacksize(size_t size) > +{ > + return PyErr_Warn(PyExc_RuntimeWarning, > + "setting thread stack size not supported on " > + "this platform"); > +} > + > +/* Only platforms with THREAD_SET_STACKSIZE() defined in > + pthread_.h, overriding this default definition, > + will support changing the stack size. > + Return 1 if an exception is pending, 0 otherwise. */ > +#define THREAD_SET_STACKSIZE(x) _pythread_unsupported_set_stacksize(x) > + > #ifdef SGI_THREADS > #include "thread_sgi.h" > #endif > @@ -149,6 +174,14 @@ > #endif > */ > > +/* use appropriate thread stack size setting routine. > + Return 1 if an exception is pending, 0 otherwise. */ This looks like it returns the result of PyErr_Warn which returns 0 on success and -1 if an error occurs. This comment doesn't seem to agree with the code. I think there are multiple occurrences of this comment. > +int > +PyThread_set_stacksize(size_t size) > +{ > + return THREAD_SET_STACKSIZE(size); > +} > Modified: python/trunk/Python/thread_pthread.h > ============================================================================== > --- python/trunk/Python/thread_pthread.h (original) > +++ python/trunk/Python/thread_pthread.h Sun Jun 4 14:31:09 2006 ... > +static int > +_pythread_pthread_set_stacksize(size_t size) > +{ > + /* set to default */ > + if (size == 0) { > + _pythread_stacksize = 0; > + return 0; > + } > + > + /* valid range? */ > + if (size >= THREAD_STACK_MIN) { > + _pythread_stacksize = size; > + return 0; > + } > + else { > + char warning[128]; > + snprintf(warning, > + 128, > + "thread stack size of %#x bytes not supported", > + size); Why not use PyOS_snprintf for more portability, just in case? Ditto for other header changes (even though those are more windows/platform specific). > + return PyErr_Warn(PyExc_RuntimeWarning, warning); > + } > +} n From buildbot at python.org Sun Jun 4 19:29:22 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 17:29:22 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20060604172950.90B891E4003@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/123 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: george.yoshida,tim.peters BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From nnorwitz at gmail.com Sun Jun 4 19:37:23 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 4 Jun 2006 10:37:23 -0700 Subject: [Python-checkins] Python Regression Test Failures refleak (1) In-Reply-To: <1f7befae0606040947i7b3fdbc3p23ba363a4103cd24@mail.gmail.com> References: <20060604090704.GA17397@python.psfb.org> <43E59BC0-9CD0-4524-9DB3-6CFD5E155721@commonground.com.au> <1f7befae0606040947i7b3fdbc3p23ba363a4103cd24@mail.gmail.com> Message-ID: On 6/4/06, Tim Peters wrote: > [Neal Norwitz] > >> test_exceptions leaked [0, 2, 0] references > >> test_filecmp leaked [13, 0, 0] references > > [Richard Jones] > > What's producing this output? > > While I've never run it, I believe it's produced by Misc/build.sh. Correct. It's run every 12 hours. Sometimes it hangs due to test_logging or one of the various socket tests. > > On what OS? > > Windows 3.1 with Neal's cool hack to implement the Bourne shell as a > Windows device driver ;-) Oooooh, so close. :-) It's the x86 buildbot (gentoo). > > I'd like to try to find that leak in exceptions if possible. > > There probably isn't a leak. I'm 99.44% sure there's no leak. > Does that make any sense? Not to me -- I don't know of a clear reason > other than wild loads/stores for why such runs should ever differ. The problem generally has to do with modules cacheing things. The way to cleanup the cache and eliminate these false positives is in regrtest.py, see cleanup. To give you an idea of how much needs to be cleaned up currently, I pasting the code: def cleanup(): _path_created.clear() warnings.filters[:] = fs gc.collect() re.purge() _strptime._regex_cache.clear() urlparse.clear_cache() urllib.urlcleanup() urllib2.install_opener(None) copy_reg.dispatch_table.clear() copy_reg.dispatch_table.update(ps) sys.path_importer_cache.clear() sys.path_importer_cache.update(pic) dircache.reset() linecache.clearcache() mimetypes._default_mime_types() struct._cache.clear() doctest.master = None It's possible that gc.collect() should also be called at the end of it. On the box this is run on, I added 2 gc.collect() calls at the end of cleanup, just to be safe. So to fix this problem, you'd have to find where those refs are being stuck and clear them out somehow. n From buildbot at python.org Sun Jun 4 19:40:49 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 17:40:49 +0000 Subject: [Python-checkins] buildbot warnings in MIPS Debian trunk Message-ID: <20060604174116.57B0E1E4003@bag.python.org> The Buildbot has detected a new failure of MIPS Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/MIPS%2520Debian%2520trunk/builds/187 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre,martin.blais Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sun Jun 4 20:05:46 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 04 Jun 2006 18:05:46 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060604180556.216E41E4026@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/642 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: george.yoshida Build Had Warnings: warnings test sincerely, -The Buildbot From martin at v.loewis.de Sun Jun 4 21:10:20 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 04 Jun 2006 21:10:20 +0200 Subject: [Python-checkins] Python Regression Test Failures refleak (1) In-Reply-To: References: <20060604090704.GA17397@python.psfb.org> <43E59BC0-9CD0-4524-9DB3-6CFD5E155721@commonground.com.au> <1f7befae0606040947i7b3fdbc3p23ba363a4103cd24@mail.gmail.com> Message-ID: <4483301C.8000601@v.loewis.de> Neal Norwitz wrote: >> Does that make any sense? Not to me -- I don't know of a clear reason >> other than wild loads/stores for why such runs should ever differ. > > The problem generally has to do with modules cacheing things. Then the numbers should repeat on each run. So wild loads/stores are a more compelling explanation. Of course, it *should* even be repeatable with wild loads/stores, unless the OS shuffles the address space on each run (which at least Linux, unfortunately, does). Regards, Martin From python-checkins at python.org Sun Jun 4 21:36:57 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sun, 4 Jun 2006 21:36:57 +0200 (CEST) Subject: [Python-checkins] r46648 - in python/trunk: Include/unicodeobject.h Lib/encodings/cp037.py Lib/encodings/cp1006.py Lib/encodings/cp1026.py Lib/encodings/cp1140.py Lib/encodings/cp1250.py Lib/encodings/cp1251.py Lib/encodings/cp1252.py Lib/encodings/cp1253.py Lib/encodings/cp1254.py Lib/encodings/cp1255.py Lib/encodings/cp1256.py Lib/encodings/cp1257.py Lib/encodings/cp1258.py Lib/encodings/cp424.py Lib/encodings/cp500.py Lib/encodings/cp856.py Lib/encodings/cp874.py Lib/encodings/cp875.py Lib/encodings/iso8859_1.py Lib/encodings/iso8859_10.py Lib/encodings/iso8859_11.py Lib/encodings/iso8859_13.py Lib/encodings/iso8859_14.py Lib/encodings/iso8859_15.py Lib/encodings/iso8859_16.py Lib/encodings/iso8859_2.py Lib/encodings/iso8859_3.py Lib/encodings/iso8859_4.py Lib/encodings/iso8859_5.py Lib/encodings/iso8859_6.py Lib/encodings/iso8859_7.py Lib/encodings/iso8859_8.py Lib/encodings/iso8859_9.py Lib/encodings/koi8_r.py Lib/encodings/koi8_u.py Lib/encodings/mac_centeuro.py Lib/encodings/mac_croatian.py Lib/encodings/mac_cyrillic.py Lib/encodings/mac_farsi.py Lib/encodings/mac_greek.py Lib/encodings/mac_iceland.py Lib/encodings/mac_roman.py Lib/encodings/mac_romanian.py Lib/encodings/mac_turkish.py Lib/encodings/tis_620.py Misc/NEWS Modules/_codecsmodule.c Objects/unicodeobject.c Tools/unicode/Makefile Tools/unicode/gencodec.py Message-ID: <20060604193657.122BE1E4003@bag.python.org> Author: martin.v.loewis Date: Sun Jun 4 21:36:28 2006 New Revision: 46648 Modified: python/trunk/Include/unicodeobject.h python/trunk/Lib/encodings/cp037.py python/trunk/Lib/encodings/cp1006.py python/trunk/Lib/encodings/cp1026.py python/trunk/Lib/encodings/cp1140.py python/trunk/Lib/encodings/cp1250.py python/trunk/Lib/encodings/cp1251.py python/trunk/Lib/encodings/cp1252.py python/trunk/Lib/encodings/cp1253.py python/trunk/Lib/encodings/cp1254.py python/trunk/Lib/encodings/cp1255.py python/trunk/Lib/encodings/cp1256.py python/trunk/Lib/encodings/cp1257.py python/trunk/Lib/encodings/cp1258.py python/trunk/Lib/encodings/cp424.py python/trunk/Lib/encodings/cp500.py python/trunk/Lib/encodings/cp856.py python/trunk/Lib/encodings/cp874.py python/trunk/Lib/encodings/cp875.py python/trunk/Lib/encodings/iso8859_1.py python/trunk/Lib/encodings/iso8859_10.py python/trunk/Lib/encodings/iso8859_11.py python/trunk/Lib/encodings/iso8859_13.py python/trunk/Lib/encodings/iso8859_14.py python/trunk/Lib/encodings/iso8859_15.py python/trunk/Lib/encodings/iso8859_16.py python/trunk/Lib/encodings/iso8859_2.py python/trunk/Lib/encodings/iso8859_3.py python/trunk/Lib/encodings/iso8859_4.py python/trunk/Lib/encodings/iso8859_5.py python/trunk/Lib/encodings/iso8859_6.py python/trunk/Lib/encodings/iso8859_7.py python/trunk/Lib/encodings/iso8859_8.py python/trunk/Lib/encodings/iso8859_9.py python/trunk/Lib/encodings/koi8_r.py python/trunk/Lib/encodings/koi8_u.py python/trunk/Lib/encodings/mac_centeuro.py python/trunk/Lib/encodings/mac_croatian.py python/trunk/Lib/encodings/mac_cyrillic.py python/trunk/Lib/encodings/mac_farsi.py python/trunk/Lib/encodings/mac_greek.py python/trunk/Lib/encodings/mac_iceland.py python/trunk/Lib/encodings/mac_roman.py python/trunk/Lib/encodings/mac_romanian.py python/trunk/Lib/encodings/mac_turkish.py python/trunk/Lib/encodings/tis_620.py python/trunk/Misc/NEWS python/trunk/Modules/_codecsmodule.c python/trunk/Objects/unicodeobject.c python/trunk/Tools/unicode/Makefile python/trunk/Tools/unicode/gencodec.py Log: Patch #1359618: Speed-up charmap encoder. Modified: python/trunk/Include/unicodeobject.h ============================================================================== --- python/trunk/Include/unicodeobject.h (original) +++ python/trunk/Include/unicodeobject.h Sun Jun 4 21:36:28 2006 @@ -650,6 +650,11 @@ const char *errors /* error handling */ ); +PyAPI_FUNC(PyObject*) PyUnicode_BuildEncodingMap( + PyObject* string /* 256 character map */ + ); + + /* --- UTF-7 Codecs ------------------------------------------------------- */ PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF7( Modified: python/trunk/Lib/encodings/cp037.py ============================================================================== --- python/trunk/Lib/encodings/cp037.py (original) +++ python/trunk/Lib/encodings/cp037.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\x9f' # 0xFF -> CONTROL ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x37, # END OF TRANSMISSION - 0x0005: 0x2D, # ENQUIRY - 0x0006: 0x2E, # ACKNOWLEDGE - 0x0007: 0x2F, # BELL - 0x0008: 0x16, # BACKSPACE - 0x0009: 0x05, # HORIZONTAL TABULATION - 0x000A: 0x25, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x3C, # DEVICE CONTROL FOUR - 0x0015: 0x3D, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x32, # SYNCHRONOUS IDLE - 0x0017: 0x26, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x3F, # SUBSTITUTE - 0x001B: 0x27, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x40, # SPACE - 0x0021: 0x5A, # EXCLAMATION MARK - 0x0022: 0x7F, # QUOTATION MARK - 0x0023: 0x7B, # NUMBER SIGN - 0x0024: 0x5B, # DOLLAR SIGN - 0x0025: 0x6C, # PERCENT SIGN - 0x0026: 0x50, # AMPERSAND - 0x0027: 0x7D, # APOSTROPHE - 0x0028: 0x4D, # LEFT PARENTHESIS - 0x0029: 0x5D, # RIGHT PARENTHESIS - 0x002A: 0x5C, # ASTERISK - 0x002B: 0x4E, # PLUS SIGN - 0x002C: 0x6B, # COMMA - 0x002D: 0x60, # HYPHEN-MINUS - 0x002E: 0x4B, # FULL STOP - 0x002F: 0x61, # SOLIDUS - 0x0030: 0xF0, # DIGIT ZERO - 0x0031: 0xF1, # DIGIT ONE - 0x0032: 0xF2, # DIGIT TWO - 0x0033: 0xF3, # DIGIT THREE - 0x0034: 0xF4, # DIGIT FOUR - 0x0035: 0xF5, # DIGIT FIVE - 0x0036: 0xF6, # DIGIT SIX - 0x0037: 0xF7, # DIGIT SEVEN - 0x0038: 0xF8, # DIGIT EIGHT - 0x0039: 0xF9, # DIGIT NINE - 0x003A: 0x7A, # COLON - 0x003B: 0x5E, # SEMICOLON - 0x003C: 0x4C, # LESS-THAN SIGN - 0x003D: 0x7E, # EQUALS SIGN - 0x003E: 0x6E, # GREATER-THAN SIGN - 0x003F: 0x6F, # QUESTION MARK - 0x0040: 0x7C, # COMMERCIAL AT - 0x0041: 0xC1, # LATIN CAPITAL LETTER A - 0x0042: 0xC2, # LATIN CAPITAL LETTER B - 0x0043: 0xC3, # LATIN CAPITAL LETTER C - 0x0044: 0xC4, # LATIN CAPITAL LETTER D - 0x0045: 0xC5, # LATIN CAPITAL LETTER E - 0x0046: 0xC6, # LATIN CAPITAL LETTER F - 0x0047: 0xC7, # LATIN CAPITAL LETTER G - 0x0048: 0xC8, # LATIN CAPITAL LETTER H - 0x0049: 0xC9, # LATIN CAPITAL LETTER I - 0x004A: 0xD1, # LATIN CAPITAL LETTER J - 0x004B: 0xD2, # LATIN CAPITAL LETTER K - 0x004C: 0xD3, # LATIN CAPITAL LETTER L - 0x004D: 0xD4, # LATIN CAPITAL LETTER M - 0x004E: 0xD5, # LATIN CAPITAL LETTER N - 0x004F: 0xD6, # LATIN CAPITAL LETTER O - 0x0050: 0xD7, # LATIN CAPITAL LETTER P - 0x0051: 0xD8, # LATIN CAPITAL LETTER Q - 0x0052: 0xD9, # LATIN CAPITAL LETTER R - 0x0053: 0xE2, # LATIN CAPITAL LETTER S - 0x0054: 0xE3, # LATIN CAPITAL LETTER T - 0x0055: 0xE4, # LATIN CAPITAL LETTER U - 0x0056: 0xE5, # LATIN CAPITAL LETTER V - 0x0057: 0xE6, # LATIN CAPITAL LETTER W - 0x0058: 0xE7, # LATIN CAPITAL LETTER X - 0x0059: 0xE8, # LATIN CAPITAL LETTER Y - 0x005A: 0xE9, # LATIN CAPITAL LETTER Z - 0x005B: 0xBA, # LEFT SQUARE BRACKET - 0x005C: 0xE0, # REVERSE SOLIDUS - 0x005D: 0xBB, # RIGHT SQUARE BRACKET - 0x005E: 0xB0, # CIRCUMFLEX ACCENT - 0x005F: 0x6D, # LOW LINE - 0x0060: 0x79, # GRAVE ACCENT - 0x0061: 0x81, # LATIN SMALL LETTER A - 0x0062: 0x82, # LATIN SMALL LETTER B - 0x0063: 0x83, # LATIN SMALL LETTER C - 0x0064: 0x84, # LATIN SMALL LETTER D - 0x0065: 0x85, # LATIN SMALL LETTER E - 0x0066: 0x86, # LATIN SMALL LETTER F - 0x0067: 0x87, # LATIN SMALL LETTER G - 0x0068: 0x88, # LATIN SMALL LETTER H - 0x0069: 0x89, # LATIN SMALL LETTER I - 0x006A: 0x91, # LATIN SMALL LETTER J - 0x006B: 0x92, # LATIN SMALL LETTER K - 0x006C: 0x93, # LATIN SMALL LETTER L - 0x006D: 0x94, # LATIN SMALL LETTER M - 0x006E: 0x95, # LATIN SMALL LETTER N - 0x006F: 0x96, # LATIN SMALL LETTER O - 0x0070: 0x97, # LATIN SMALL LETTER P - 0x0071: 0x98, # LATIN SMALL LETTER Q - 0x0072: 0x99, # LATIN SMALL LETTER R - 0x0073: 0xA2, # LATIN SMALL LETTER S - 0x0074: 0xA3, # LATIN SMALL LETTER T - 0x0075: 0xA4, # LATIN SMALL LETTER U - 0x0076: 0xA5, # LATIN SMALL LETTER V - 0x0077: 0xA6, # LATIN SMALL LETTER W - 0x0078: 0xA7, # LATIN SMALL LETTER X - 0x0079: 0xA8, # LATIN SMALL LETTER Y - 0x007A: 0xA9, # LATIN SMALL LETTER Z - 0x007B: 0xC0, # LEFT CURLY BRACKET - 0x007C: 0x4F, # VERTICAL LINE - 0x007D: 0xD0, # RIGHT CURLY BRACKET - 0x007E: 0xA1, # TILDE - 0x007F: 0x07, # DELETE - 0x0080: 0x20, # CONTROL - 0x0081: 0x21, # CONTROL - 0x0082: 0x22, # CONTROL - 0x0083: 0x23, # CONTROL - 0x0084: 0x24, # CONTROL - 0x0085: 0x15, # CONTROL - 0x0086: 0x06, # CONTROL - 0x0087: 0x17, # CONTROL - 0x0088: 0x28, # CONTROL - 0x0089: 0x29, # CONTROL - 0x008A: 0x2A, # CONTROL - 0x008B: 0x2B, # CONTROL - 0x008C: 0x2C, # CONTROL - 0x008D: 0x09, # CONTROL - 0x008E: 0x0A, # CONTROL - 0x008F: 0x1B, # CONTROL - 0x0090: 0x30, # CONTROL - 0x0091: 0x31, # CONTROL - 0x0092: 0x1A, # CONTROL - 0x0093: 0x33, # CONTROL - 0x0094: 0x34, # CONTROL - 0x0095: 0x35, # CONTROL - 0x0096: 0x36, # CONTROL - 0x0097: 0x08, # CONTROL - 0x0098: 0x38, # CONTROL - 0x0099: 0x39, # CONTROL - 0x009A: 0x3A, # CONTROL - 0x009B: 0x3B, # CONTROL - 0x009C: 0x04, # CONTROL - 0x009D: 0x14, # CONTROL - 0x009E: 0x3E, # CONTROL - 0x009F: 0xFF, # CONTROL - 0x00A0: 0x41, # NO-BREAK SPACE - 0x00A1: 0xAA, # INVERTED EXCLAMATION MARK - 0x00A2: 0x4A, # CENT SIGN - 0x00A3: 0xB1, # POUND SIGN - 0x00A4: 0x9F, # CURRENCY SIGN - 0x00A5: 0xB2, # YEN SIGN - 0x00A6: 0x6A, # BROKEN BAR - 0x00A7: 0xB5, # SECTION SIGN - 0x00A8: 0xBD, # DIAERESIS - 0x00A9: 0xB4, # COPYRIGHT SIGN - 0x00AA: 0x9A, # FEMININE ORDINAL INDICATOR - 0x00AB: 0x8A, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0x5F, # NOT SIGN - 0x00AD: 0xCA, # SOFT HYPHEN - 0x00AE: 0xAF, # REGISTERED SIGN - 0x00AF: 0xBC, # MACRON - 0x00B0: 0x90, # DEGREE SIGN - 0x00B1: 0x8F, # PLUS-MINUS SIGN - 0x00B2: 0xEA, # SUPERSCRIPT TWO - 0x00B3: 0xFA, # SUPERSCRIPT THREE - 0x00B4: 0xBE, # ACUTE ACCENT - 0x00B5: 0xA0, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB3, # MIDDLE DOT - 0x00B8: 0x9D, # CEDILLA - 0x00B9: 0xDA, # SUPERSCRIPT ONE - 0x00BA: 0x9B, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0x8B, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xB7, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xB8, # VULGAR FRACTION ONE HALF - 0x00BE: 0xB9, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xAB, # INVERTED QUESTION MARK - 0x00C0: 0x64, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0x65, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0x62, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0x66, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0x63, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0x67, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0x9E, # LATIN CAPITAL LIGATURE AE - 0x00C7: 0x68, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0x74, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0x71, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0x72, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0x73, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0x78, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0x75, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0x76, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0x77, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D0: 0xAC, # LATIN CAPITAL LETTER ETH (ICELANDIC) - 0x00D1: 0x69, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xED, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xEE, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xEB, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xEF, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xEC, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xBF, # MULTIPLICATION SIGN - 0x00D8: 0x80, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xFD, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xFE, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xFB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xFC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xAD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DE: 0xAE, # LATIN CAPITAL LETTER THORN (ICELANDIC) - 0x00DF: 0x59, # LATIN SMALL LETTER SHARP S (GERMAN) - 0x00E0: 0x44, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0x45, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0x42, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0x46, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0x43, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0x47, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0x9C, # LATIN SMALL LIGATURE AE - 0x00E7: 0x48, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0x54, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0x51, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0x52, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0x53, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0x58, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0x55, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0x56, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0x57, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F0: 0x8C, # LATIN SMALL LETTER ETH (ICELANDIC) - 0x00F1: 0x49, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xCD, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xCE, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xCB, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xCF, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xCC, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xE1, # DIVISION SIGN - 0x00F8: 0x70, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xDD, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xDE, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xDB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xDC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0x8D, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FE: 0x8E, # LATIN SMALL LETTER THORN (ICELANDIC) - 0x00FF: 0xDF, # LATIN SMALL LETTER Y WITH DIAERESIS -} Modified: python/trunk/Lib/encodings/cp1006.py ============================================================================== --- python/trunk/Lib/encodings/cp1006.py (original) +++ python/trunk/Lib/encodings/cp1006.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,262 +303,6 @@ u'\ufe7d' # 0xFF -> ARABIC SHADDA MEDIAL FORM ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00AD: 0xAD, # SOFT HYPHEN - 0x060C: 0xAB, # ARABIC COMMA - 0x061B: 0xAC, # ARABIC SEMICOLON - 0x061F: 0xAE, # ARABIC QUESTION MARK - 0x06F0: 0xA1, # EXTENDED ARABIC-INDIC DIGIT ZERO - 0x06F1: 0xA2, # EXTENDED ARABIC-INDIC DIGIT ONE - 0x06F2: 0xA3, # EXTENDED ARABIC-INDIC DIGIT TWO - 0x06F3: 0xA4, # EXTENDED ARABIC-INDIC DIGIT THREE - 0x06F4: 0xA5, # EXTENDED ARABIC-INDIC DIGIT FOUR - 0x06F5: 0xA6, # EXTENDED ARABIC-INDIC DIGIT FIVE - 0x06F6: 0xA7, # EXTENDED ARABIC-INDIC DIGIT SIX - 0x06F7: 0xA8, # EXTENDED ARABIC-INDIC DIGIT SEVEN - 0x06F8: 0xA9, # EXTENDED ARABIC-INDIC DIGIT EIGHT - 0x06F9: 0xAA, # EXTENDED ARABIC-INDIC DIGIT NINE - 0xFB56: 0xB5, # ARABIC LETTER PEH ISOLATED FORM - 0xFB58: 0xB6, # ARABIC LETTER PEH INITIAL FORM - 0xFB66: 0xBA, # ARABIC LETTER TTEH ISOLATED FORM - 0xFB68: 0xBB, # ARABIC LETTER TTEH INITIAL FORM - 0xFB7A: 0xC0, # ARABIC LETTER TCHEH ISOLATED FORM - 0xFB7C: 0xC1, # ARABIC LETTER TCHEH INITIAL FORM - 0xFB84: 0xC7, # ARABIC LETTER DAHAL ISOLATED FORMN - 0xFB8A: 0xCC, # ARABIC LETTER JEH ISOLATED FORM - 0xFB8C: 0xCA, # ARABIC LETTER RREH ISOLATED FORM - 0xFB92: 0xE5, # ARABIC LETTER GAF ISOLATED FORM - 0xFB94: 0xE6, # ARABIC LETTER GAF INITIAL FORM - 0xFB9E: 0xEC, # ARABIC LETTER NOON GHUNNA ISOLATED FORM - 0xFBA6: 0xF1, # ARABIC LETTER HEH GOAL ISOLATED FORM - 0xFBA8: 0xF2, # ARABIC LETTER HEH GOAL INITIAL FORM - 0xFBA9: 0xF3, # ARABIC LETTER HEH GOAL MEDIAL FORM - 0xFBAA: 0xF4, # ARABIC LETTER HEH DOACHASHMEE ISOLATED FORM - 0xFBAE: 0xFD, # ARABIC LETTER YEH BARREE ISOLATED FORM - 0xFBB0: 0xFC, # ARABIC LETTER YEH BARREE WITH HAMZA ABOVE ISOLATED FORM - 0xFE7C: 0xFE, # ARABIC SHADDA ISOLATED FORM - 0xFE7D: 0xFF, # ARABIC SHADDA MEDIAL FORM - 0xFE80: 0xF5, # ARABIC LETTER HAMZA ISOLATED FORM - 0xFE81: 0xAF, # ARABIC LETTER ALEF WITH MADDA ABOVE ISOLATED FORM - 0xFE85: 0xEF, # ARABIC LETTER WAW WITH HAMZA ABOVE ISOLATED FORM - 0xFE89: 0xF6, # ARABIC LETTER YEH WITH HAMZA ABOVE ISOLATED FORM - 0xFE8A: 0xF7, # ARABIC LETTER YEH WITH HAMZA ABOVE FINAL FORM - 0xFE8B: 0xF8, # ARABIC LETTER YEH WITH HAMZA ABOVE INITIAL FORM - 0xFE8D: 0xB0, # ARABIC LETTER ALEF ISOLATED FORM - 0xFE8E: None, # ARABIC LETTER ALEF FINAL FORM - 0xFE8F: 0xB3, # ARABIC LETTER BEH ISOLATED FORM - 0xFE91: 0xB4, # ARABIC LETTER BEH INITIAL FORM - 0xFE93: 0xB7, # ARABIC LETTER TEH MARBUTA ISOLATED FORM - 0xFE95: 0xB8, # ARABIC LETTER TEH ISOLATED FORM - 0xFE97: 0xB9, # ARABIC LETTER TEH INITIAL FORM - 0xFE99: 0xBC, # ARABIC LETTER THEH ISOLATED FORM - 0xFE9B: 0xBD, # ARABIC LETTER THEH INITIAL FORM - 0xFE9D: 0xBE, # ARABIC LETTER JEEM ISOLATED FORM - 0xFE9F: 0xBF, # ARABIC LETTER JEEM INITIAL FORM - 0xFEA1: 0xC2, # ARABIC LETTER HAH ISOLATED FORM - 0xFEA3: 0xC3, # ARABIC LETTER HAH INITIAL FORM - 0xFEA5: 0xC4, # ARABIC LETTER KHAH ISOLATED FORM - 0xFEA7: 0xC5, # ARABIC LETTER KHAH INITIAL FORM - 0xFEA9: 0xC6, # ARABIC LETTER DAL ISOLATED FORM - 0xFEAB: 0xC8, # ARABIC LETTER THAL ISOLATED FORM - 0xFEAD: 0xC9, # ARABIC LETTER REH ISOLATED FORM - 0xFEAF: 0xCB, # ARABIC LETTER ZAIN ISOLATED FORM - 0xFEB1: 0xCD, # ARABIC LETTER SEEN ISOLATED FORM - 0xFEB3: 0xCE, # ARABIC LETTER SEEN INITIAL FORM - 0xFEB5: 0xCF, # ARABIC LETTER SHEEN ISOLATED FORM - 0xFEB7: 0xD0, # ARABIC LETTER SHEEN INITIAL FORM - 0xFEB9: 0xD1, # ARABIC LETTER SAD ISOLATED FORM - 0xFEBB: 0xD2, # ARABIC LETTER SAD INITIAL FORM - 0xFEBD: 0xD3, # ARABIC LETTER DAD ISOLATED FORM - 0xFEBF: 0xD4, # ARABIC LETTER DAD INITIAL FORM - 0xFEC1: 0xD5, # ARABIC LETTER TAH ISOLATED FORM - 0xFEC5: 0xD6, # ARABIC LETTER ZAH ISOLATED FORM - 0xFEC9: 0xD7, # ARABIC LETTER AIN ISOLATED FORM - 0xFECA: 0xD8, # ARABIC LETTER AIN FINAL FORM - 0xFECB: 0xD9, # ARABIC LETTER AIN INITIAL FORM - 0xFECC: 0xDA, # ARABIC LETTER AIN MEDIAL FORM - 0xFECD: 0xDB, # ARABIC LETTER GHAIN ISOLATED FORM - 0xFECE: 0xDC, # ARABIC LETTER GHAIN FINAL FORM - 0xFECF: 0xDD, # ARABIC LETTER GHAIN INITIAL FORM - 0xFED0: 0xDE, # ARABIC LETTER GHAIN MEDIAL FORM - 0xFED1: 0xDF, # ARABIC LETTER FEH ISOLATED FORM - 0xFED3: 0xE0, # ARABIC LETTER FEH INITIAL FORM - 0xFED5: 0xE1, # ARABIC LETTER QAF ISOLATED FORM - 0xFED7: 0xE2, # ARABIC LETTER QAF INITIAL FORM - 0xFED9: 0xE3, # ARABIC LETTER KAF ISOLATED FORM - 0xFEDB: 0xE4, # ARABIC LETTER KAF INITIAL FORM - 0xFEDD: 0xE7, # ARABIC LETTER LAM ISOLATED FORM - 0xFEDF: 0xE8, # ARABIC LETTER LAM INITIAL FORM - 0xFEE0: 0xE9, # ARABIC LETTER LAM MEDIAL FORM - 0xFEE1: 0xEA, # ARABIC LETTER MEEM ISOLATED FORM - 0xFEE3: 0xEB, # ARABIC LETTER MEEM INITIAL FORM - 0xFEE5: 0xED, # ARABIC LETTER NOON ISOLATED FORM - 0xFEE7: 0xEE, # ARABIC LETTER NOON INITIAL FORM - 0xFEED: 0xF0, # ARABIC LETTER WAW ISOLATED FORM - 0xFEF1: 0xF9, # ARABIC LETTER YEH ISOLATED FORM - 0xFEF2: 0xFA, # ARABIC LETTER YEH FINAL FORM - 0xFEF3: 0xFB, # ARABIC LETTER YEH INITIAL FORM -} Modified: python/trunk/Lib/encodings/cp1026.py ============================================================================== --- python/trunk/Lib/encodings/cp1026.py (original) +++ python/trunk/Lib/encodings/cp1026.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\x9f' # 0xFF -> CONTROL ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x37, # END OF TRANSMISSION - 0x0005: 0x2D, # ENQUIRY - 0x0006: 0x2E, # ACKNOWLEDGE - 0x0007: 0x2F, # BELL - 0x0008: 0x16, # BACKSPACE - 0x0009: 0x05, # HORIZONTAL TABULATION - 0x000A: 0x25, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x3C, # DEVICE CONTROL FOUR - 0x0015: 0x3D, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x32, # SYNCHRONOUS IDLE - 0x0017: 0x26, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x3F, # SUBSTITUTE - 0x001B: 0x27, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x40, # SPACE - 0x0021: 0x4F, # EXCLAMATION MARK - 0x0022: 0xFC, # QUOTATION MARK - 0x0023: 0xEC, # NUMBER SIGN - 0x0024: 0xAD, # DOLLAR SIGN - 0x0025: 0x6C, # PERCENT SIGN - 0x0026: 0x50, # AMPERSAND - 0x0027: 0x7D, # APOSTROPHE - 0x0028: 0x4D, # LEFT PARENTHESIS - 0x0029: 0x5D, # RIGHT PARENTHESIS - 0x002A: 0x5C, # ASTERISK - 0x002B: 0x4E, # PLUS SIGN - 0x002C: 0x6B, # COMMA - 0x002D: 0x60, # HYPHEN-MINUS - 0x002E: 0x4B, # FULL STOP - 0x002F: 0x61, # SOLIDUS - 0x0030: 0xF0, # DIGIT ZERO - 0x0031: 0xF1, # DIGIT ONE - 0x0032: 0xF2, # DIGIT TWO - 0x0033: 0xF3, # DIGIT THREE - 0x0034: 0xF4, # DIGIT FOUR - 0x0035: 0xF5, # DIGIT FIVE - 0x0036: 0xF6, # DIGIT SIX - 0x0037: 0xF7, # DIGIT SEVEN - 0x0038: 0xF8, # DIGIT EIGHT - 0x0039: 0xF9, # DIGIT NINE - 0x003A: 0x7A, # COLON - 0x003B: 0x5E, # SEMICOLON - 0x003C: 0x4C, # LESS-THAN SIGN - 0x003D: 0x7E, # EQUALS SIGN - 0x003E: 0x6E, # GREATER-THAN SIGN - 0x003F: 0x6F, # QUESTION MARK - 0x0040: 0xAE, # COMMERCIAL AT - 0x0041: 0xC1, # LATIN CAPITAL LETTER A - 0x0042: 0xC2, # LATIN CAPITAL LETTER B - 0x0043: 0xC3, # LATIN CAPITAL LETTER C - 0x0044: 0xC4, # LATIN CAPITAL LETTER D - 0x0045: 0xC5, # LATIN CAPITAL LETTER E - 0x0046: 0xC6, # LATIN CAPITAL LETTER F - 0x0047: 0xC7, # LATIN CAPITAL LETTER G - 0x0048: 0xC8, # LATIN CAPITAL LETTER H - 0x0049: 0xC9, # LATIN CAPITAL LETTER I - 0x004A: 0xD1, # LATIN CAPITAL LETTER J - 0x004B: 0xD2, # LATIN CAPITAL LETTER K - 0x004C: 0xD3, # LATIN CAPITAL LETTER L - 0x004D: 0xD4, # LATIN CAPITAL LETTER M - 0x004E: 0xD5, # LATIN CAPITAL LETTER N - 0x004F: 0xD6, # LATIN CAPITAL LETTER O - 0x0050: 0xD7, # LATIN CAPITAL LETTER P - 0x0051: 0xD8, # LATIN CAPITAL LETTER Q - 0x0052: 0xD9, # LATIN CAPITAL LETTER R - 0x0053: 0xE2, # LATIN CAPITAL LETTER S - 0x0054: 0xE3, # LATIN CAPITAL LETTER T - 0x0055: 0xE4, # LATIN CAPITAL LETTER U - 0x0056: 0xE5, # LATIN CAPITAL LETTER V - 0x0057: 0xE6, # LATIN CAPITAL LETTER W - 0x0058: 0xE7, # LATIN CAPITAL LETTER X - 0x0059: 0xE8, # LATIN CAPITAL LETTER Y - 0x005A: 0xE9, # LATIN CAPITAL LETTER Z - 0x005B: 0x68, # LEFT SQUARE BRACKET - 0x005C: 0xDC, # REVERSE SOLIDUS - 0x005D: 0xAC, # RIGHT SQUARE BRACKET - 0x005E: 0x5F, # CIRCUMFLEX ACCENT - 0x005F: 0x6D, # LOW LINE - 0x0060: 0x8D, # GRAVE ACCENT - 0x0061: 0x81, # LATIN SMALL LETTER A - 0x0062: 0x82, # LATIN SMALL LETTER B - 0x0063: 0x83, # LATIN SMALL LETTER C - 0x0064: 0x84, # LATIN SMALL LETTER D - 0x0065: 0x85, # LATIN SMALL LETTER E - 0x0066: 0x86, # LATIN SMALL LETTER F - 0x0067: 0x87, # LATIN SMALL LETTER G - 0x0068: 0x88, # LATIN SMALL LETTER H - 0x0069: 0x89, # LATIN SMALL LETTER I - 0x006A: 0x91, # LATIN SMALL LETTER J - 0x006B: 0x92, # LATIN SMALL LETTER K - 0x006C: 0x93, # LATIN SMALL LETTER L - 0x006D: 0x94, # LATIN SMALL LETTER M - 0x006E: 0x95, # LATIN SMALL LETTER N - 0x006F: 0x96, # LATIN SMALL LETTER O - 0x0070: 0x97, # LATIN SMALL LETTER P - 0x0071: 0x98, # LATIN SMALL LETTER Q - 0x0072: 0x99, # LATIN SMALL LETTER R - 0x0073: 0xA2, # LATIN SMALL LETTER S - 0x0074: 0xA3, # LATIN SMALL LETTER T - 0x0075: 0xA4, # LATIN SMALL LETTER U - 0x0076: 0xA5, # LATIN SMALL LETTER V - 0x0077: 0xA6, # LATIN SMALL LETTER W - 0x0078: 0xA7, # LATIN SMALL LETTER X - 0x0079: 0xA8, # LATIN SMALL LETTER Y - 0x007A: 0xA9, # LATIN SMALL LETTER Z - 0x007B: 0x48, # LEFT CURLY BRACKET - 0x007C: 0xBB, # VERTICAL LINE - 0x007D: 0x8C, # RIGHT CURLY BRACKET - 0x007E: 0xCC, # TILDE - 0x007F: 0x07, # DELETE - 0x0080: 0x20, # CONTROL - 0x0081: 0x21, # CONTROL - 0x0082: 0x22, # CONTROL - 0x0083: 0x23, # CONTROL - 0x0084: 0x24, # CONTROL - 0x0085: 0x15, # CONTROL - 0x0086: 0x06, # CONTROL - 0x0087: 0x17, # CONTROL - 0x0088: 0x28, # CONTROL - 0x0089: 0x29, # CONTROL - 0x008A: 0x2A, # CONTROL - 0x008B: 0x2B, # CONTROL - 0x008C: 0x2C, # CONTROL - 0x008D: 0x09, # CONTROL - 0x008E: 0x0A, # CONTROL - 0x008F: 0x1B, # CONTROL - 0x0090: 0x30, # CONTROL - 0x0091: 0x31, # CONTROL - 0x0092: 0x1A, # CONTROL - 0x0093: 0x33, # CONTROL - 0x0094: 0x34, # CONTROL - 0x0095: 0x35, # CONTROL - 0x0096: 0x36, # CONTROL - 0x0097: 0x08, # CONTROL - 0x0098: 0x38, # CONTROL - 0x0099: 0x39, # CONTROL - 0x009A: 0x3A, # CONTROL - 0x009B: 0x3B, # CONTROL - 0x009C: 0x04, # CONTROL - 0x009D: 0x14, # CONTROL - 0x009E: 0x3E, # CONTROL - 0x009F: 0xFF, # CONTROL - 0x00A0: 0x41, # NO-BREAK SPACE - 0x00A1: 0xAA, # INVERTED EXCLAMATION MARK - 0x00A2: 0xB0, # CENT SIGN - 0x00A3: 0xB1, # POUND SIGN - 0x00A4: 0x9F, # CURRENCY SIGN - 0x00A5: 0xB2, # YEN SIGN - 0x00A6: 0x8E, # BROKEN BAR - 0x00A7: 0xB5, # SECTION SIGN - 0x00A8: 0xBD, # DIAERESIS - 0x00A9: 0xB4, # COPYRIGHT SIGN - 0x00AA: 0x9A, # FEMININE ORDINAL INDICATOR - 0x00AB: 0x8A, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xBA, # NOT SIGN - 0x00AD: 0xCA, # SOFT HYPHEN - 0x00AE: 0xAF, # REGISTERED SIGN - 0x00AF: 0xBC, # MACRON - 0x00B0: 0x90, # DEGREE SIGN - 0x00B1: 0x8F, # PLUS-MINUS SIGN - 0x00B2: 0xEA, # SUPERSCRIPT TWO - 0x00B3: 0xFA, # SUPERSCRIPT THREE - 0x00B4: 0xBE, # ACUTE ACCENT - 0x00B5: 0xA0, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB3, # MIDDLE DOT - 0x00B8: 0x9D, # CEDILLA - 0x00B9: 0xDA, # SUPERSCRIPT ONE - 0x00BA: 0x9B, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0x8B, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xB7, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xB8, # VULGAR FRACTION ONE HALF - 0x00BE: 0xB9, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xAB, # INVERTED QUESTION MARK - 0x00C0: 0x64, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0x65, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0x62, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0x66, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0x63, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0x67, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0x9E, # LATIN CAPITAL LIGATURE AE - 0x00C7: 0x4A, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0x74, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0x71, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0x72, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0x73, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0x78, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0x75, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0x76, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0x77, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D1: 0x69, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xED, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xEE, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xEB, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xEF, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0x7B, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xBF, # MULTIPLICATION SIGN - 0x00D8: 0x80, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xFD, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xFE, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xFB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0x7F, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DF: 0x59, # LATIN SMALL LETTER SHARP S (GERMAN) - 0x00E0: 0x44, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0x45, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0x42, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0x46, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0x43, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0x47, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0x9C, # LATIN SMALL LIGATURE AE - 0x00E7: 0xC0, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0x54, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0x51, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0x52, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0x53, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0x58, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0x55, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0x56, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0x57, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F1: 0x49, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xCD, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xCE, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xCB, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xCF, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xA1, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xE1, # DIVISION SIGN - 0x00F8: 0x70, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xDD, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xDE, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xDB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xE0, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FF: 0xDF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x011E: 0x5A, # LATIN CAPITAL LETTER G WITH BREVE - 0x011F: 0xD0, # LATIN SMALL LETTER G WITH BREVE - 0x0130: 0x5B, # LATIN CAPITAL LETTER I WITH DOT ABOVE - 0x0131: 0x79, # LATIN SMALL LETTER DOTLESS I - 0x015E: 0x7C, # LATIN CAPITAL LETTER S WITH CEDILLA - 0x015F: 0x6A, # LATIN SMALL LETTER S WITH CEDILLA -} Modified: python/trunk/Lib/encodings/cp1140.py ============================================================================== --- python/trunk/Lib/encodings/cp1140.py (original) +++ python/trunk/Lib/encodings/cp1140.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\x9f' # 0xFF -> CONTROL ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x37, # END OF TRANSMISSION - 0x0005: 0x2D, # ENQUIRY - 0x0006: 0x2E, # ACKNOWLEDGE - 0x0007: 0x2F, # BELL - 0x0008: 0x16, # BACKSPACE - 0x0009: 0x05, # HORIZONTAL TABULATION - 0x000A: 0x25, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x3C, # DEVICE CONTROL FOUR - 0x0015: 0x3D, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x32, # SYNCHRONOUS IDLE - 0x0017: 0x26, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x3F, # SUBSTITUTE - 0x001B: 0x27, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x40, # SPACE - 0x0021: 0x5A, # EXCLAMATION MARK - 0x0022: 0x7F, # QUOTATION MARK - 0x0023: 0x7B, # NUMBER SIGN - 0x0024: 0x5B, # DOLLAR SIGN - 0x0025: 0x6C, # PERCENT SIGN - 0x0026: 0x50, # AMPERSAND - 0x0027: 0x7D, # APOSTROPHE - 0x0028: 0x4D, # LEFT PARENTHESIS - 0x0029: 0x5D, # RIGHT PARENTHESIS - 0x002A: 0x5C, # ASTERISK - 0x002B: 0x4E, # PLUS SIGN - 0x002C: 0x6B, # COMMA - 0x002D: 0x60, # HYPHEN-MINUS - 0x002E: 0x4B, # FULL STOP - 0x002F: 0x61, # SOLIDUS - 0x0030: 0xF0, # DIGIT ZERO - 0x0031: 0xF1, # DIGIT ONE - 0x0032: 0xF2, # DIGIT TWO - 0x0033: 0xF3, # DIGIT THREE - 0x0034: 0xF4, # DIGIT FOUR - 0x0035: 0xF5, # DIGIT FIVE - 0x0036: 0xF6, # DIGIT SIX - 0x0037: 0xF7, # DIGIT SEVEN - 0x0038: 0xF8, # DIGIT EIGHT - 0x0039: 0xF9, # DIGIT NINE - 0x003A: 0x7A, # COLON - 0x003B: 0x5E, # SEMICOLON - 0x003C: 0x4C, # LESS-THAN SIGN - 0x003D: 0x7E, # EQUALS SIGN - 0x003E: 0x6E, # GREATER-THAN SIGN - 0x003F: 0x6F, # QUESTION MARK - 0x0040: 0x7C, # COMMERCIAL AT - 0x0041: 0xC1, # LATIN CAPITAL LETTER A - 0x0042: 0xC2, # LATIN CAPITAL LETTER B - 0x0043: 0xC3, # LATIN CAPITAL LETTER C - 0x0044: 0xC4, # LATIN CAPITAL LETTER D - 0x0045: 0xC5, # LATIN CAPITAL LETTER E - 0x0046: 0xC6, # LATIN CAPITAL LETTER F - 0x0047: 0xC7, # LATIN CAPITAL LETTER G - 0x0048: 0xC8, # LATIN CAPITAL LETTER H - 0x0049: 0xC9, # LATIN CAPITAL LETTER I - 0x004A: 0xD1, # LATIN CAPITAL LETTER J - 0x004B: 0xD2, # LATIN CAPITAL LETTER K - 0x004C: 0xD3, # LATIN CAPITAL LETTER L - 0x004D: 0xD4, # LATIN CAPITAL LETTER M - 0x004E: 0xD5, # LATIN CAPITAL LETTER N - 0x004F: 0xD6, # LATIN CAPITAL LETTER O - 0x0050: 0xD7, # LATIN CAPITAL LETTER P - 0x0051: 0xD8, # LATIN CAPITAL LETTER Q - 0x0052: 0xD9, # LATIN CAPITAL LETTER R - 0x0053: 0xE2, # LATIN CAPITAL LETTER S - 0x0054: 0xE3, # LATIN CAPITAL LETTER T - 0x0055: 0xE4, # LATIN CAPITAL LETTER U - 0x0056: 0xE5, # LATIN CAPITAL LETTER V - 0x0057: 0xE6, # LATIN CAPITAL LETTER W - 0x0058: 0xE7, # LATIN CAPITAL LETTER X - 0x0059: 0xE8, # LATIN CAPITAL LETTER Y - 0x005A: 0xE9, # LATIN CAPITAL LETTER Z - 0x005B: 0xBA, # LEFT SQUARE BRACKET - 0x005C: 0xE0, # REVERSE SOLIDUS - 0x005D: 0xBB, # RIGHT SQUARE BRACKET - 0x005E: 0xB0, # CIRCUMFLEX ACCENT - 0x005F: 0x6D, # LOW LINE - 0x0060: 0x79, # GRAVE ACCENT - 0x0061: 0x81, # LATIN SMALL LETTER A - 0x0062: 0x82, # LATIN SMALL LETTER B - 0x0063: 0x83, # LATIN SMALL LETTER C - 0x0064: 0x84, # LATIN SMALL LETTER D - 0x0065: 0x85, # LATIN SMALL LETTER E - 0x0066: 0x86, # LATIN SMALL LETTER F - 0x0067: 0x87, # LATIN SMALL LETTER G - 0x0068: 0x88, # LATIN SMALL LETTER H - 0x0069: 0x89, # LATIN SMALL LETTER I - 0x006A: 0x91, # LATIN SMALL LETTER J - 0x006B: 0x92, # LATIN SMALL LETTER K - 0x006C: 0x93, # LATIN SMALL LETTER L - 0x006D: 0x94, # LATIN SMALL LETTER M - 0x006E: 0x95, # LATIN SMALL LETTER N - 0x006F: 0x96, # LATIN SMALL LETTER O - 0x0070: 0x97, # LATIN SMALL LETTER P - 0x0071: 0x98, # LATIN SMALL LETTER Q - 0x0072: 0x99, # LATIN SMALL LETTER R - 0x0073: 0xA2, # LATIN SMALL LETTER S - 0x0074: 0xA3, # LATIN SMALL LETTER T - 0x0075: 0xA4, # LATIN SMALL LETTER U - 0x0076: 0xA5, # LATIN SMALL LETTER V - 0x0077: 0xA6, # LATIN SMALL LETTER W - 0x0078: 0xA7, # LATIN SMALL LETTER X - 0x0079: 0xA8, # LATIN SMALL LETTER Y - 0x007A: 0xA9, # LATIN SMALL LETTER Z - 0x007B: 0xC0, # LEFT CURLY BRACKET - 0x007C: 0x4F, # VERTICAL LINE - 0x007D: 0xD0, # RIGHT CURLY BRACKET - 0x007E: 0xA1, # TILDE - 0x007F: 0x07, # DELETE - 0x0080: 0x20, # CONTROL - 0x0081: 0x21, # CONTROL - 0x0082: 0x22, # CONTROL - 0x0083: 0x23, # CONTROL - 0x0084: 0x24, # CONTROL - 0x0085: 0x15, # CONTROL - 0x0086: 0x06, # CONTROL - 0x0087: 0x17, # CONTROL - 0x0088: 0x28, # CONTROL - 0x0089: 0x29, # CONTROL - 0x008A: 0x2A, # CONTROL - 0x008B: 0x2B, # CONTROL - 0x008C: 0x2C, # CONTROL - 0x008D: 0x09, # CONTROL - 0x008E: 0x0A, # CONTROL - 0x008F: 0x1B, # CONTROL - 0x0090: 0x30, # CONTROL - 0x0091: 0x31, # CONTROL - 0x0092: 0x1A, # CONTROL - 0x0093: 0x33, # CONTROL - 0x0094: 0x34, # CONTROL - 0x0095: 0x35, # CONTROL - 0x0096: 0x36, # CONTROL - 0x0097: 0x08, # CONTROL - 0x0098: 0x38, # CONTROL - 0x0099: 0x39, # CONTROL - 0x009A: 0x3A, # CONTROL - 0x009B: 0x3B, # CONTROL - 0x009C: 0x04, # CONTROL - 0x009D: 0x14, # CONTROL - 0x009E: 0x3E, # CONTROL - 0x009F: 0xFF, # CONTROL - 0x00A0: 0x41, # NO-BREAK SPACE - 0x00A1: 0xAA, # INVERTED EXCLAMATION MARK - 0x00A2: 0x4A, # CENT SIGN - 0x00A3: 0xB1, # POUND SIGN - 0x00A5: 0xB2, # YEN SIGN - 0x00A6: 0x6A, # BROKEN BAR - 0x00A7: 0xB5, # SECTION SIGN - 0x00A8: 0xBD, # DIAERESIS - 0x00A9: 0xB4, # COPYRIGHT SIGN - 0x00AA: 0x9A, # FEMININE ORDINAL INDICATOR - 0x00AB: 0x8A, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0x5F, # NOT SIGN - 0x00AD: 0xCA, # SOFT HYPHEN - 0x00AE: 0xAF, # REGISTERED SIGN - 0x00AF: 0xBC, # MACRON - 0x00B0: 0x90, # DEGREE SIGN - 0x00B1: 0x8F, # PLUS-MINUS SIGN - 0x00B2: 0xEA, # SUPERSCRIPT TWO - 0x00B3: 0xFA, # SUPERSCRIPT THREE - 0x00B4: 0xBE, # ACUTE ACCENT - 0x00B5: 0xA0, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB3, # MIDDLE DOT - 0x00B8: 0x9D, # CEDILLA - 0x00B9: 0xDA, # SUPERSCRIPT ONE - 0x00BA: 0x9B, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0x8B, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xB7, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xB8, # VULGAR FRACTION ONE HALF - 0x00BE: 0xB9, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xAB, # INVERTED QUESTION MARK - 0x00C0: 0x64, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0x65, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0x62, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0x66, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0x63, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0x67, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0x9E, # LATIN CAPITAL LIGATURE AE - 0x00C7: 0x68, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0x74, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0x71, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0x72, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0x73, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0x78, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0x75, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0x76, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0x77, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D0: 0xAC, # LATIN CAPITAL LETTER ETH (ICELANDIC) - 0x00D1: 0x69, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xED, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xEE, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xEB, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xEF, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xEC, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xBF, # MULTIPLICATION SIGN - 0x00D8: 0x80, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xFD, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xFE, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xFB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xFC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xAD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DE: 0xAE, # LATIN CAPITAL LETTER THORN (ICELANDIC) - 0x00DF: 0x59, # LATIN SMALL LETTER SHARP S (GERMAN) - 0x00E0: 0x44, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0x45, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0x42, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0x46, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0x43, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0x47, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0x9C, # LATIN SMALL LIGATURE AE - 0x00E7: 0x48, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0x54, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0x51, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0x52, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0x53, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0x58, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0x55, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0x56, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0x57, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F0: 0x8C, # LATIN SMALL LETTER ETH (ICELANDIC) - 0x00F1: 0x49, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xCD, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xCE, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xCB, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xCF, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xCC, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xE1, # DIVISION SIGN - 0x00F8: 0x70, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xDD, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xDE, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xDB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xDC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0x8D, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FE: 0x8E, # LATIN SMALL LETTER THORN (ICELANDIC) - 0x00FF: 0xDF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x20AC: 0x9F, # EURO SIGN -} Modified: python/trunk/Lib/encodings/cp1250.py ============================================================================== --- python/trunk/Lib/encodings/cp1250.py (original) +++ python/trunk/Lib/encodings/cp1250.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,258 +303,6 @@ u'\u02d9' # 0xFF -> DOT ABOVE ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0xB8, # CEDILLA - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xDD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0xFD, # LATIN SMALL LETTER Y WITH ACUTE - 0x0102: 0xC3, # LATIN CAPITAL LETTER A WITH BREVE - 0x0103: 0xE3, # LATIN SMALL LETTER A WITH BREVE - 0x0104: 0xA5, # LATIN CAPITAL LETTER A WITH OGONEK - 0x0105: 0xB9, # LATIN SMALL LETTER A WITH OGONEK - 0x0106: 0xC6, # LATIN CAPITAL LETTER C WITH ACUTE - 0x0107: 0xE6, # LATIN SMALL LETTER C WITH ACUTE - 0x010C: 0xC8, # LATIN CAPITAL LETTER C WITH CARON - 0x010D: 0xE8, # LATIN SMALL LETTER C WITH CARON - 0x010E: 0xCF, # LATIN CAPITAL LETTER D WITH CARON - 0x010F: 0xEF, # LATIN SMALL LETTER D WITH CARON - 0x0110: 0xD0, # LATIN CAPITAL LETTER D WITH STROKE - 0x0111: 0xF0, # LATIN SMALL LETTER D WITH STROKE - 0x0118: 0xCA, # LATIN CAPITAL LETTER E WITH OGONEK - 0x0119: 0xEA, # LATIN SMALL LETTER E WITH OGONEK - 0x011A: 0xCC, # LATIN CAPITAL LETTER E WITH CARON - 0x011B: 0xEC, # LATIN SMALL LETTER E WITH CARON - 0x0139: 0xC5, # LATIN CAPITAL LETTER L WITH ACUTE - 0x013A: 0xE5, # LATIN SMALL LETTER L WITH ACUTE - 0x013D: 0xBC, # LATIN CAPITAL LETTER L WITH CARON - 0x013E: 0xBE, # LATIN SMALL LETTER L WITH CARON - 0x0141: 0xA3, # LATIN CAPITAL LETTER L WITH STROKE - 0x0142: 0xB3, # LATIN SMALL LETTER L WITH STROKE - 0x0143: 0xD1, # LATIN CAPITAL LETTER N WITH ACUTE - 0x0144: 0xF1, # LATIN SMALL LETTER N WITH ACUTE - 0x0147: 0xD2, # LATIN CAPITAL LETTER N WITH CARON - 0x0148: 0xF2, # LATIN SMALL LETTER N WITH CARON - 0x0150: 0xD5, # LATIN CAPITAL LETTER O WITH DOUBLE ACUTE - 0x0151: 0xF5, # LATIN SMALL LETTER O WITH DOUBLE ACUTE - 0x0154: 0xC0, # LATIN CAPITAL LETTER R WITH ACUTE - 0x0155: 0xE0, # LATIN SMALL LETTER R WITH ACUTE - 0x0158: 0xD8, # LATIN CAPITAL LETTER R WITH CARON - 0x0159: 0xF8, # LATIN SMALL LETTER R WITH CARON - 0x015A: 0x8C, # LATIN CAPITAL LETTER S WITH ACUTE - 0x015B: 0x9C, # LATIN SMALL LETTER S WITH ACUTE - 0x015E: 0xAA, # LATIN CAPITAL LETTER S WITH CEDILLA - 0x015F: 0xBA, # LATIN SMALL LETTER S WITH CEDILLA - 0x0160: 0x8A, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0x9A, # LATIN SMALL LETTER S WITH CARON - 0x0162: 0xDE, # LATIN CAPITAL LETTER T WITH CEDILLA - 0x0163: 0xFE, # LATIN SMALL LETTER T WITH CEDILLA - 0x0164: 0x8D, # LATIN CAPITAL LETTER T WITH CARON - 0x0165: 0x9D, # LATIN SMALL LETTER T WITH CARON - 0x016E: 0xD9, # LATIN CAPITAL LETTER U WITH RING ABOVE - 0x016F: 0xF9, # LATIN SMALL LETTER U WITH RING ABOVE - 0x0170: 0xDB, # LATIN CAPITAL LETTER U WITH DOUBLE ACUTE - 0x0171: 0xFB, # LATIN SMALL LETTER U WITH DOUBLE ACUTE - 0x0179: 0x8F, # LATIN CAPITAL LETTER Z WITH ACUTE - 0x017A: 0x9F, # LATIN SMALL LETTER Z WITH ACUTE - 0x017B: 0xAF, # LATIN CAPITAL LETTER Z WITH DOT ABOVE - 0x017C: 0xBF, # LATIN SMALL LETTER Z WITH DOT ABOVE - 0x017D: 0x8E, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0x9E, # LATIN SMALL LETTER Z WITH CARON - 0x02C7: 0xA1, # CARON - 0x02D8: 0xA2, # BREVE - 0x02D9: 0xFF, # DOT ABOVE - 0x02DB: 0xB2, # OGONEK - 0x02DD: 0xBD, # DOUBLE ACUTE ACCENT - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1251.py ============================================================================== --- python/trunk/Lib/encodings/cp1251.py (original) +++ python/trunk/Lib/encodings/cp1251.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,262 +303,6 @@ u'\u044f' # 0xFF -> CYRILLIC SMALL LETTER YA ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x0401: 0xA8, # CYRILLIC CAPITAL LETTER IO - 0x0402: 0x80, # CYRILLIC CAPITAL LETTER DJE - 0x0403: 0x81, # CYRILLIC CAPITAL LETTER GJE - 0x0404: 0xAA, # CYRILLIC CAPITAL LETTER UKRAINIAN IE - 0x0405: 0xBD, # CYRILLIC CAPITAL LETTER DZE - 0x0406: 0xB2, # CYRILLIC CAPITAL LETTER BYELORUSSIAN-UKRAINIAN I - 0x0407: 0xAF, # CYRILLIC CAPITAL LETTER YI - 0x0408: 0xA3, # CYRILLIC CAPITAL LETTER JE - 0x0409: 0x8A, # CYRILLIC CAPITAL LETTER LJE - 0x040A: 0x8C, # CYRILLIC CAPITAL LETTER NJE - 0x040B: 0x8E, # CYRILLIC CAPITAL LETTER TSHE - 0x040C: 0x8D, # CYRILLIC CAPITAL LETTER KJE - 0x040E: 0xA1, # CYRILLIC CAPITAL LETTER SHORT U - 0x040F: 0x8F, # CYRILLIC CAPITAL LETTER DZHE - 0x0410: 0xC0, # CYRILLIC CAPITAL LETTER A - 0x0411: 0xC1, # CYRILLIC CAPITAL LETTER BE - 0x0412: 0xC2, # CYRILLIC CAPITAL LETTER VE - 0x0413: 0xC3, # CYRILLIC CAPITAL LETTER GHE - 0x0414: 0xC4, # CYRILLIC CAPITAL LETTER DE - 0x0415: 0xC5, # CYRILLIC CAPITAL LETTER IE - 0x0416: 0xC6, # CYRILLIC CAPITAL LETTER ZHE - 0x0417: 0xC7, # CYRILLIC CAPITAL LETTER ZE - 0x0418: 0xC8, # CYRILLIC CAPITAL LETTER I - 0x0419: 0xC9, # CYRILLIC CAPITAL LETTER SHORT I - 0x041A: 0xCA, # CYRILLIC CAPITAL LETTER KA - 0x041B: 0xCB, # CYRILLIC CAPITAL LETTER EL - 0x041C: 0xCC, # CYRILLIC CAPITAL LETTER EM - 0x041D: 0xCD, # CYRILLIC CAPITAL LETTER EN - 0x041E: 0xCE, # CYRILLIC CAPITAL LETTER O - 0x041F: 0xCF, # CYRILLIC CAPITAL LETTER PE - 0x0420: 0xD0, # CYRILLIC CAPITAL LETTER ER - 0x0421: 0xD1, # CYRILLIC CAPITAL LETTER ES - 0x0422: 0xD2, # CYRILLIC CAPITAL LETTER TE - 0x0423: 0xD3, # CYRILLIC CAPITAL LETTER U - 0x0424: 0xD4, # CYRILLIC CAPITAL LETTER EF - 0x0425: 0xD5, # CYRILLIC CAPITAL LETTER HA - 0x0426: 0xD6, # CYRILLIC CAPITAL LETTER TSE - 0x0427: 0xD7, # CYRILLIC CAPITAL LETTER CHE - 0x0428: 0xD8, # CYRILLIC CAPITAL LETTER SHA - 0x0429: 0xD9, # CYRILLIC CAPITAL LETTER SHCHA - 0x042A: 0xDA, # CYRILLIC CAPITAL LETTER HARD SIGN - 0x042B: 0xDB, # CYRILLIC CAPITAL LETTER YERU - 0x042C: 0xDC, # CYRILLIC CAPITAL LETTER SOFT SIGN - 0x042D: 0xDD, # CYRILLIC CAPITAL LETTER E - 0x042E: 0xDE, # CYRILLIC CAPITAL LETTER YU - 0x042F: 0xDF, # CYRILLIC CAPITAL LETTER YA - 0x0430: 0xE0, # CYRILLIC SMALL LETTER A - 0x0431: 0xE1, # CYRILLIC SMALL LETTER BE - 0x0432: 0xE2, # CYRILLIC SMALL LETTER VE - 0x0433: 0xE3, # CYRILLIC SMALL LETTER GHE - 0x0434: 0xE4, # CYRILLIC SMALL LETTER DE - 0x0435: 0xE5, # CYRILLIC SMALL LETTER IE - 0x0436: 0xE6, # CYRILLIC SMALL LETTER ZHE - 0x0437: 0xE7, # CYRILLIC SMALL LETTER ZE - 0x0438: 0xE8, # CYRILLIC SMALL LETTER I - 0x0439: 0xE9, # CYRILLIC SMALL LETTER SHORT I - 0x043A: 0xEA, # CYRILLIC SMALL LETTER KA - 0x043B: 0xEB, # CYRILLIC SMALL LETTER EL - 0x043C: 0xEC, # CYRILLIC SMALL LETTER EM - 0x043D: 0xED, # CYRILLIC SMALL LETTER EN - 0x043E: 0xEE, # CYRILLIC SMALL LETTER O - 0x043F: 0xEF, # CYRILLIC SMALL LETTER PE - 0x0440: 0xF0, # CYRILLIC SMALL LETTER ER - 0x0441: 0xF1, # CYRILLIC SMALL LETTER ES - 0x0442: 0xF2, # CYRILLIC SMALL LETTER TE - 0x0443: 0xF3, # CYRILLIC SMALL LETTER U - 0x0444: 0xF4, # CYRILLIC SMALL LETTER EF - 0x0445: 0xF5, # CYRILLIC SMALL LETTER HA - 0x0446: 0xF6, # CYRILLIC SMALL LETTER TSE - 0x0447: 0xF7, # CYRILLIC SMALL LETTER CHE - 0x0448: 0xF8, # CYRILLIC SMALL LETTER SHA - 0x0449: 0xF9, # CYRILLIC SMALL LETTER SHCHA - 0x044A: 0xFA, # CYRILLIC SMALL LETTER HARD SIGN - 0x044B: 0xFB, # CYRILLIC SMALL LETTER YERU - 0x044C: 0xFC, # CYRILLIC SMALL LETTER SOFT SIGN - 0x044D: 0xFD, # CYRILLIC SMALL LETTER E - 0x044E: 0xFE, # CYRILLIC SMALL LETTER YU - 0x044F: 0xFF, # CYRILLIC SMALL LETTER YA - 0x0451: 0xB8, # CYRILLIC SMALL LETTER IO - 0x0452: 0x90, # CYRILLIC SMALL LETTER DJE - 0x0453: 0x83, # CYRILLIC SMALL LETTER GJE - 0x0454: 0xBA, # CYRILLIC SMALL LETTER UKRAINIAN IE - 0x0455: 0xBE, # CYRILLIC SMALL LETTER DZE - 0x0456: 0xB3, # CYRILLIC SMALL LETTER BYELORUSSIAN-UKRAINIAN I - 0x0457: 0xBF, # CYRILLIC SMALL LETTER YI - 0x0458: 0xBC, # CYRILLIC SMALL LETTER JE - 0x0459: 0x9A, # CYRILLIC SMALL LETTER LJE - 0x045A: 0x9C, # CYRILLIC SMALL LETTER NJE - 0x045B: 0x9E, # CYRILLIC SMALL LETTER TSHE - 0x045C: 0x9D, # CYRILLIC SMALL LETTER KJE - 0x045E: 0xA2, # CYRILLIC SMALL LETTER SHORT U - 0x045F: 0x9F, # CYRILLIC SMALL LETTER DZHE - 0x0490: 0xA5, # CYRILLIC CAPITAL LETTER GHE WITH UPTURN - 0x0491: 0xB4, # CYRILLIC SMALL LETTER GHE WITH UPTURN - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AC: 0x88, # EURO SIGN - 0x2116: 0xB9, # NUMERO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1252.py ============================================================================== --- python/trunk/Lib/encodings/cp1252.py (original) +++ python/trunk/Lib/encodings/cp1252.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,258 +303,6 @@ u'\xff' # 0xFF -> LATIN SMALL LETTER Y WITH DIAERESIS ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A1: 0xA1, # INVERTED EXCLAMATION MARK - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AA: 0xAA, # FEMININE ORDINAL INDICATOR - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0xAF, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0xB8, # CEDILLA - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BA: 0xBA, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xBF, # INVERTED QUESTION MARK - 0x00C0: 0xC0, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0xC3, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0xC8, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0xCA, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0xCC, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D0: 0xD0, # LATIN CAPITAL LETTER ETH - 0x00D1: 0xD1, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xD2, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00D8: 0xD8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xD9, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xDD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DE: 0xDE, # LATIN CAPITAL LETTER THORN - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0xE3, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0xEC, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F0: 0xF0, # LATIN SMALL LETTER ETH - 0x00F1: 0xF1, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xF2, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F8: 0xF8, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0xFD, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FE: 0xFE, # LATIN SMALL LETTER THORN - 0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x0152: 0x8C, # LATIN CAPITAL LIGATURE OE - 0x0153: 0x9C, # LATIN SMALL LIGATURE OE - 0x0160: 0x8A, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0x9A, # LATIN SMALL LETTER S WITH CARON - 0x0178: 0x9F, # LATIN CAPITAL LETTER Y WITH DIAERESIS - 0x017D: 0x8E, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0x9E, # LATIN SMALL LETTER Z WITH CARON - 0x0192: 0x83, # LATIN SMALL LETTER F WITH HOOK - 0x02C6: 0x88, # MODIFIER LETTER CIRCUMFLEX ACCENT - 0x02DC: 0x98, # SMALL TILDE - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1253.py ============================================================================== --- python/trunk/Lib/encodings/cp1253.py (original) +++ python/trunk/Lib/encodings/cp1253.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,246 +303,6 @@ u'\ufffe' # 0xFF -> UNDEFINED ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x0192: 0x83, # LATIN SMALL LETTER F WITH HOOK - 0x0384: 0xB4, # GREEK TONOS - 0x0385: 0xA1, # GREEK DIALYTIKA TONOS - 0x0386: 0xA2, # GREEK CAPITAL LETTER ALPHA WITH TONOS - 0x0388: 0xB8, # GREEK CAPITAL LETTER EPSILON WITH TONOS - 0x0389: 0xB9, # GREEK CAPITAL LETTER ETA WITH TONOS - 0x038A: 0xBA, # GREEK CAPITAL LETTER IOTA WITH TONOS - 0x038C: 0xBC, # GREEK CAPITAL LETTER OMICRON WITH TONOS - 0x038E: 0xBE, # GREEK CAPITAL LETTER UPSILON WITH TONOS - 0x038F: 0xBF, # GREEK CAPITAL LETTER OMEGA WITH TONOS - 0x0390: 0xC0, # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND TONOS - 0x0391: 0xC1, # GREEK CAPITAL LETTER ALPHA - 0x0392: 0xC2, # GREEK CAPITAL LETTER BETA - 0x0393: 0xC3, # GREEK CAPITAL LETTER GAMMA - 0x0394: 0xC4, # GREEK CAPITAL LETTER DELTA - 0x0395: 0xC5, # GREEK CAPITAL LETTER EPSILON - 0x0396: 0xC6, # GREEK CAPITAL LETTER ZETA - 0x0397: 0xC7, # GREEK CAPITAL LETTER ETA - 0x0398: 0xC8, # GREEK CAPITAL LETTER THETA - 0x0399: 0xC9, # GREEK CAPITAL LETTER IOTA - 0x039A: 0xCA, # GREEK CAPITAL LETTER KAPPA - 0x039B: 0xCB, # GREEK CAPITAL LETTER LAMDA - 0x039C: 0xCC, # GREEK CAPITAL LETTER MU - 0x039D: 0xCD, # GREEK CAPITAL LETTER NU - 0x039E: 0xCE, # GREEK CAPITAL LETTER XI - 0x039F: 0xCF, # GREEK CAPITAL LETTER OMICRON - 0x03A0: 0xD0, # GREEK CAPITAL LETTER PI - 0x03A1: 0xD1, # GREEK CAPITAL LETTER RHO - 0x03A3: 0xD3, # GREEK CAPITAL LETTER SIGMA - 0x03A4: 0xD4, # GREEK CAPITAL LETTER TAU - 0x03A5: 0xD5, # GREEK CAPITAL LETTER UPSILON - 0x03A6: 0xD6, # GREEK CAPITAL LETTER PHI - 0x03A7: 0xD7, # GREEK CAPITAL LETTER CHI - 0x03A8: 0xD8, # GREEK CAPITAL LETTER PSI - 0x03A9: 0xD9, # GREEK CAPITAL LETTER OMEGA - 0x03AA: 0xDA, # GREEK CAPITAL LETTER IOTA WITH DIALYTIKA - 0x03AB: 0xDB, # GREEK CAPITAL LETTER UPSILON WITH DIALYTIKA - 0x03AC: 0xDC, # GREEK SMALL LETTER ALPHA WITH TONOS - 0x03AD: 0xDD, # GREEK SMALL LETTER EPSILON WITH TONOS - 0x03AE: 0xDE, # GREEK SMALL LETTER ETA WITH TONOS - 0x03AF: 0xDF, # GREEK SMALL LETTER IOTA WITH TONOS - 0x03B0: 0xE0, # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND TONOS - 0x03B1: 0xE1, # GREEK SMALL LETTER ALPHA - 0x03B2: 0xE2, # GREEK SMALL LETTER BETA - 0x03B3: 0xE3, # GREEK SMALL LETTER GAMMA - 0x03B4: 0xE4, # GREEK SMALL LETTER DELTA - 0x03B5: 0xE5, # GREEK SMALL LETTER EPSILON - 0x03B6: 0xE6, # GREEK SMALL LETTER ZETA - 0x03B7: 0xE7, # GREEK SMALL LETTER ETA - 0x03B8: 0xE8, # GREEK SMALL LETTER THETA - 0x03B9: 0xE9, # GREEK SMALL LETTER IOTA - 0x03BA: 0xEA, # GREEK SMALL LETTER KAPPA - 0x03BB: 0xEB, # GREEK SMALL LETTER LAMDA - 0x03BC: 0xEC, # GREEK SMALL LETTER MU - 0x03BD: 0xED, # GREEK SMALL LETTER NU - 0x03BE: 0xEE, # GREEK SMALL LETTER XI - 0x03BF: 0xEF, # GREEK SMALL LETTER OMICRON - 0x03C0: 0xF0, # GREEK SMALL LETTER PI - 0x03C1: 0xF1, # GREEK SMALL LETTER RHO - 0x03C2: 0xF2, # GREEK SMALL LETTER FINAL SIGMA - 0x03C3: 0xF3, # GREEK SMALL LETTER SIGMA - 0x03C4: 0xF4, # GREEK SMALL LETTER TAU - 0x03C5: 0xF5, # GREEK SMALL LETTER UPSILON - 0x03C6: 0xF6, # GREEK SMALL LETTER PHI - 0x03C7: 0xF7, # GREEK SMALL LETTER CHI - 0x03C8: 0xF8, # GREEK SMALL LETTER PSI - 0x03C9: 0xF9, # GREEK SMALL LETTER OMEGA - 0x03CA: 0xFA, # GREEK SMALL LETTER IOTA WITH DIALYTIKA - 0x03CB: 0xFB, # GREEK SMALL LETTER UPSILON WITH DIALYTIKA - 0x03CC: 0xFC, # GREEK SMALL LETTER OMICRON WITH TONOS - 0x03CD: 0xFD, # GREEK SMALL LETTER UPSILON WITH TONOS - 0x03CE: 0xFE, # GREEK SMALL LETTER OMEGA WITH TONOS - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2015: 0xAF, # HORIZONTAL BAR - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1254.py ============================================================================== --- python/trunk/Lib/encodings/cp1254.py (original) +++ python/trunk/Lib/encodings/cp1254.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,256 +303,6 @@ u'\xff' # 0xFF -> LATIN SMALL LETTER Y WITH DIAERESIS ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A1: 0xA1, # INVERTED EXCLAMATION MARK - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AA: 0xAA, # FEMININE ORDINAL INDICATOR - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0xAF, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0xB8, # CEDILLA - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BA: 0xBA, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xBF, # INVERTED QUESTION MARK - 0x00C0: 0xC0, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0xC3, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0xC8, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0xCA, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0xCC, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D1: 0xD1, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xD2, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00D8: 0xD8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xD9, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0xE3, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0xEC, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F1: 0xF1, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xF2, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F8: 0xF8, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x011E: 0xD0, # LATIN CAPITAL LETTER G WITH BREVE - 0x011F: 0xF0, # LATIN SMALL LETTER G WITH BREVE - 0x0130: 0xDD, # LATIN CAPITAL LETTER I WITH DOT ABOVE - 0x0131: 0xFD, # LATIN SMALL LETTER DOTLESS I - 0x0152: 0x8C, # LATIN CAPITAL LIGATURE OE - 0x0153: 0x9C, # LATIN SMALL LIGATURE OE - 0x015E: 0xDE, # LATIN CAPITAL LETTER S WITH CEDILLA - 0x015F: 0xFE, # LATIN SMALL LETTER S WITH CEDILLA - 0x0160: 0x8A, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0x9A, # LATIN SMALL LETTER S WITH CARON - 0x0178: 0x9F, # LATIN CAPITAL LETTER Y WITH DIAERESIS - 0x0192: 0x83, # LATIN SMALL LETTER F WITH HOOK - 0x02C6: 0x88, # MODIFIER LETTER CIRCUMFLEX ACCENT - 0x02DC: 0x98, # SMALL TILDE - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1255.py ============================================================================== --- python/trunk/Lib/encodings/cp1255.py (original) +++ python/trunk/Lib/encodings/cp1255.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,240 +303,6 @@ u'\ufffe' # 0xFF -> UNDEFINED ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A1: 0xA1, # INVERTED EXCLAMATION MARK - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0xAF, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0xB8, # CEDILLA - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xBF, # INVERTED QUESTION MARK - 0x00D7: 0xAA, # MULTIPLICATION SIGN - 0x00F7: 0xBA, # DIVISION SIGN - 0x0192: 0x83, # LATIN SMALL LETTER F WITH HOOK - 0x02C6: 0x88, # MODIFIER LETTER CIRCUMFLEX ACCENT - 0x02DC: 0x98, # SMALL TILDE - 0x05B0: 0xC0, # HEBREW POINT SHEVA - 0x05B1: 0xC1, # HEBREW POINT HATAF SEGOL - 0x05B2: 0xC2, # HEBREW POINT HATAF PATAH - 0x05B3: 0xC3, # HEBREW POINT HATAF QAMATS - 0x05B4: 0xC4, # HEBREW POINT HIRIQ - 0x05B5: 0xC5, # HEBREW POINT TSERE - 0x05B6: 0xC6, # HEBREW POINT SEGOL - 0x05B7: 0xC7, # HEBREW POINT PATAH - 0x05B8: 0xC8, # HEBREW POINT QAMATS - 0x05B9: 0xC9, # HEBREW POINT HOLAM - 0x05BB: 0xCB, # HEBREW POINT QUBUTS - 0x05BC: 0xCC, # HEBREW POINT DAGESH OR MAPIQ - 0x05BD: 0xCD, # HEBREW POINT METEG - 0x05BE: 0xCE, # HEBREW PUNCTUATION MAQAF - 0x05BF: 0xCF, # HEBREW POINT RAFE - 0x05C0: 0xD0, # HEBREW PUNCTUATION PASEQ - 0x05C1: 0xD1, # HEBREW POINT SHIN DOT - 0x05C2: 0xD2, # HEBREW POINT SIN DOT - 0x05C3: 0xD3, # HEBREW PUNCTUATION SOF PASUQ - 0x05D0: 0xE0, # HEBREW LETTER ALEF - 0x05D1: 0xE1, # HEBREW LETTER BET - 0x05D2: 0xE2, # HEBREW LETTER GIMEL - 0x05D3: 0xE3, # HEBREW LETTER DALET - 0x05D4: 0xE4, # HEBREW LETTER HE - 0x05D5: 0xE5, # HEBREW LETTER VAV - 0x05D6: 0xE6, # HEBREW LETTER ZAYIN - 0x05D7: 0xE7, # HEBREW LETTER HET - 0x05D8: 0xE8, # HEBREW LETTER TET - 0x05D9: 0xE9, # HEBREW LETTER YOD - 0x05DA: 0xEA, # HEBREW LETTER FINAL KAF - 0x05DB: 0xEB, # HEBREW LETTER KAF - 0x05DC: 0xEC, # HEBREW LETTER LAMED - 0x05DD: 0xED, # HEBREW LETTER FINAL MEM - 0x05DE: 0xEE, # HEBREW LETTER MEM - 0x05DF: 0xEF, # HEBREW LETTER FINAL NUN - 0x05E0: 0xF0, # HEBREW LETTER NUN - 0x05E1: 0xF1, # HEBREW LETTER SAMEKH - 0x05E2: 0xF2, # HEBREW LETTER AYIN - 0x05E3: 0xF3, # HEBREW LETTER FINAL PE - 0x05E4: 0xF4, # HEBREW LETTER PE - 0x05E5: 0xF5, # HEBREW LETTER FINAL TSADI - 0x05E6: 0xF6, # HEBREW LETTER TSADI - 0x05E7: 0xF7, # HEBREW LETTER QOF - 0x05E8: 0xF8, # HEBREW LETTER RESH - 0x05E9: 0xF9, # HEBREW LETTER SHIN - 0x05EA: 0xFA, # HEBREW LETTER TAV - 0x05F0: 0xD4, # HEBREW LIGATURE YIDDISH DOUBLE VAV - 0x05F1: 0xD5, # HEBREW LIGATURE YIDDISH VAV YOD - 0x05F2: 0xD6, # HEBREW LIGATURE YIDDISH DOUBLE YOD - 0x05F3: 0xD7, # HEBREW PUNCTUATION GERESH - 0x05F4: 0xD8, # HEBREW PUNCTUATION GERSHAYIM - 0x200E: 0xFD, # LEFT-TO-RIGHT MARK - 0x200F: 0xFE, # RIGHT-TO-LEFT MARK - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AA: 0xA4, # NEW SHEQEL SIGN - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1256.py ============================================================================== --- python/trunk/Lib/encodings/cp1256.py (original) +++ python/trunk/Lib/encodings/cp1256.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\u06d2' # 0xFF -> ARABIC LETTER YEH BARREE ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0xAF, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0xB8, # CEDILLA - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x0152: 0x8C, # LATIN CAPITAL LIGATURE OE - 0x0153: 0x9C, # LATIN SMALL LIGATURE OE - 0x0192: 0x83, # LATIN SMALL LETTER F WITH HOOK - 0x02C6: 0x88, # MODIFIER LETTER CIRCUMFLEX ACCENT - 0x060C: 0xA1, # ARABIC COMMA - 0x061B: 0xBA, # ARABIC SEMICOLON - 0x061F: 0xBF, # ARABIC QUESTION MARK - 0x0621: 0xC1, # ARABIC LETTER HAMZA - 0x0622: 0xC2, # ARABIC LETTER ALEF WITH MADDA ABOVE - 0x0623: 0xC3, # ARABIC LETTER ALEF WITH HAMZA ABOVE - 0x0624: 0xC4, # ARABIC LETTER WAW WITH HAMZA ABOVE - 0x0625: 0xC5, # ARABIC LETTER ALEF WITH HAMZA BELOW - 0x0626: 0xC6, # ARABIC LETTER YEH WITH HAMZA ABOVE - 0x0627: 0xC7, # ARABIC LETTER ALEF - 0x0628: 0xC8, # ARABIC LETTER BEH - 0x0629: 0xC9, # ARABIC LETTER TEH MARBUTA - 0x062A: 0xCA, # ARABIC LETTER TEH - 0x062B: 0xCB, # ARABIC LETTER THEH - 0x062C: 0xCC, # ARABIC LETTER JEEM - 0x062D: 0xCD, # ARABIC LETTER HAH - 0x062E: 0xCE, # ARABIC LETTER KHAH - 0x062F: 0xCF, # ARABIC LETTER DAL - 0x0630: 0xD0, # ARABIC LETTER THAL - 0x0631: 0xD1, # ARABIC LETTER REH - 0x0632: 0xD2, # ARABIC LETTER ZAIN - 0x0633: 0xD3, # ARABIC LETTER SEEN - 0x0634: 0xD4, # ARABIC LETTER SHEEN - 0x0635: 0xD5, # ARABIC LETTER SAD - 0x0636: 0xD6, # ARABIC LETTER DAD - 0x0637: 0xD8, # ARABIC LETTER TAH - 0x0638: 0xD9, # ARABIC LETTER ZAH - 0x0639: 0xDA, # ARABIC LETTER AIN - 0x063A: 0xDB, # ARABIC LETTER GHAIN - 0x0640: 0xDC, # ARABIC TATWEEL - 0x0641: 0xDD, # ARABIC LETTER FEH - 0x0642: 0xDE, # ARABIC LETTER QAF - 0x0643: 0xDF, # ARABIC LETTER KAF - 0x0644: 0xE1, # ARABIC LETTER LAM - 0x0645: 0xE3, # ARABIC LETTER MEEM - 0x0646: 0xE4, # ARABIC LETTER NOON - 0x0647: 0xE5, # ARABIC LETTER HEH - 0x0648: 0xE6, # ARABIC LETTER WAW - 0x0649: 0xEC, # ARABIC LETTER ALEF MAKSURA - 0x064A: 0xED, # ARABIC LETTER YEH - 0x064B: 0xF0, # ARABIC FATHATAN - 0x064C: 0xF1, # ARABIC DAMMATAN - 0x064D: 0xF2, # ARABIC KASRATAN - 0x064E: 0xF3, # ARABIC FATHA - 0x064F: 0xF5, # ARABIC DAMMA - 0x0650: 0xF6, # ARABIC KASRA - 0x0651: 0xF8, # ARABIC SHADDA - 0x0652: 0xFA, # ARABIC SUKUN - 0x0679: 0x8A, # ARABIC LETTER TTEH - 0x067E: 0x81, # ARABIC LETTER PEH - 0x0686: 0x8D, # ARABIC LETTER TCHEH - 0x0688: 0x8F, # ARABIC LETTER DDAL - 0x0691: 0x9A, # ARABIC LETTER RREH - 0x0698: 0x8E, # ARABIC LETTER JEH - 0x06A9: 0x98, # ARABIC LETTER KEHEH - 0x06AF: 0x90, # ARABIC LETTER GAF - 0x06BA: 0x9F, # ARABIC LETTER NOON GHUNNA - 0x06BE: 0xAA, # ARABIC LETTER HEH DOACHASHMEE - 0x06C1: 0xC0, # ARABIC LETTER HEH GOAL - 0x06D2: 0xFF, # ARABIC LETTER YEH BARREE - 0x200C: 0x9D, # ZERO WIDTH NON-JOINER - 0x200D: 0x9E, # ZERO WIDTH JOINER - 0x200E: 0xFD, # LEFT-TO-RIGHT MARK - 0x200F: 0xFE, # RIGHT-TO-LEFT MARK - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1257.py ============================================================================== --- python/trunk/Lib/encodings/cp1257.py (original) +++ python/trunk/Lib/encodings/cp1257.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,251 +303,6 @@ u'\u02d9' # 0xFF -> DOT ABOVE ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0x8D, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0x9D, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0x8F, # CEDILLA - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xAF, # LATIN CAPITAL LETTER AE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00D8: 0xA8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xBF, # LATIN SMALL LETTER AE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F8: 0xB8, # LATIN SMALL LETTER O WITH STROKE - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x0100: 0xC2, # LATIN CAPITAL LETTER A WITH MACRON - 0x0101: 0xE2, # LATIN SMALL LETTER A WITH MACRON - 0x0104: 0xC0, # LATIN CAPITAL LETTER A WITH OGONEK - 0x0105: 0xE0, # LATIN SMALL LETTER A WITH OGONEK - 0x0106: 0xC3, # LATIN CAPITAL LETTER C WITH ACUTE - 0x0107: 0xE3, # LATIN SMALL LETTER C WITH ACUTE - 0x010C: 0xC8, # LATIN CAPITAL LETTER C WITH CARON - 0x010D: 0xE8, # LATIN SMALL LETTER C WITH CARON - 0x0112: 0xC7, # LATIN CAPITAL LETTER E WITH MACRON - 0x0113: 0xE7, # LATIN SMALL LETTER E WITH MACRON - 0x0116: 0xCB, # LATIN CAPITAL LETTER E WITH DOT ABOVE - 0x0117: 0xEB, # LATIN SMALL LETTER E WITH DOT ABOVE - 0x0118: 0xC6, # LATIN CAPITAL LETTER E WITH OGONEK - 0x0119: 0xE6, # LATIN SMALL LETTER E WITH OGONEK - 0x0122: 0xCC, # LATIN CAPITAL LETTER G WITH CEDILLA - 0x0123: 0xEC, # LATIN SMALL LETTER G WITH CEDILLA - 0x012A: 0xCE, # LATIN CAPITAL LETTER I WITH MACRON - 0x012B: 0xEE, # LATIN SMALL LETTER I WITH MACRON - 0x012E: 0xC1, # LATIN CAPITAL LETTER I WITH OGONEK - 0x012F: 0xE1, # LATIN SMALL LETTER I WITH OGONEK - 0x0136: 0xCD, # LATIN CAPITAL LETTER K WITH CEDILLA - 0x0137: 0xED, # LATIN SMALL LETTER K WITH CEDILLA - 0x013B: 0xCF, # LATIN CAPITAL LETTER L WITH CEDILLA - 0x013C: 0xEF, # LATIN SMALL LETTER L WITH CEDILLA - 0x0141: 0xD9, # LATIN CAPITAL LETTER L WITH STROKE - 0x0142: 0xF9, # LATIN SMALL LETTER L WITH STROKE - 0x0143: 0xD1, # LATIN CAPITAL LETTER N WITH ACUTE - 0x0144: 0xF1, # LATIN SMALL LETTER N WITH ACUTE - 0x0145: 0xD2, # LATIN CAPITAL LETTER N WITH CEDILLA - 0x0146: 0xF2, # LATIN SMALL LETTER N WITH CEDILLA - 0x014C: 0xD4, # LATIN CAPITAL LETTER O WITH MACRON - 0x014D: 0xF4, # LATIN SMALL LETTER O WITH MACRON - 0x0156: 0xAA, # LATIN CAPITAL LETTER R WITH CEDILLA - 0x0157: 0xBA, # LATIN SMALL LETTER R WITH CEDILLA - 0x015A: 0xDA, # LATIN CAPITAL LETTER S WITH ACUTE - 0x015B: 0xFA, # LATIN SMALL LETTER S WITH ACUTE - 0x0160: 0xD0, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0xF0, # LATIN SMALL LETTER S WITH CARON - 0x016A: 0xDB, # LATIN CAPITAL LETTER U WITH MACRON - 0x016B: 0xFB, # LATIN SMALL LETTER U WITH MACRON - 0x0172: 0xD8, # LATIN CAPITAL LETTER U WITH OGONEK - 0x0173: 0xF8, # LATIN SMALL LETTER U WITH OGONEK - 0x0179: 0xCA, # LATIN CAPITAL LETTER Z WITH ACUTE - 0x017A: 0xEA, # LATIN SMALL LETTER Z WITH ACUTE - 0x017B: 0xDD, # LATIN CAPITAL LETTER Z WITH DOT ABOVE - 0x017C: 0xFD, # LATIN SMALL LETTER Z WITH DOT ABOVE - 0x017D: 0xDE, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0xFE, # LATIN SMALL LETTER Z WITH CARON - 0x02C7: 0x8E, # CARON - 0x02D9: 0xFF, # DOT ABOVE - 0x02DB: 0x9E, # OGONEK - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp1258.py ============================================================================== --- python/trunk/Lib/encodings/cp1258.py (original) +++ python/trunk/Lib/encodings/cp1258.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,254 +303,6 @@ u'\xff' # 0xFF -> LATIN SMALL LETTER Y WITH DIAERESIS ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A1: 0xA1, # INVERTED EXCLAMATION MARK - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AA: 0xAA, # FEMININE ORDINAL INDICATOR - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0xAF, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0xB8, # CEDILLA - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BA: 0xBA, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xBF, # INVERTED QUESTION MARK - 0x00C0: 0xC0, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0xC8, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0xCA, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D1: 0xD1, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00D8: 0xD8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xD9, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F1: 0xF1, # LATIN SMALL LETTER N WITH TILDE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F8: 0xF8, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x0102: 0xC3, # LATIN CAPITAL LETTER A WITH BREVE - 0x0103: 0xE3, # LATIN SMALL LETTER A WITH BREVE - 0x0110: 0xD0, # LATIN CAPITAL LETTER D WITH STROKE - 0x0111: 0xF0, # LATIN SMALL LETTER D WITH STROKE - 0x0152: 0x8C, # LATIN CAPITAL LIGATURE OE - 0x0153: 0x9C, # LATIN SMALL LIGATURE OE - 0x0178: 0x9F, # LATIN CAPITAL LETTER Y WITH DIAERESIS - 0x0192: 0x83, # LATIN SMALL LETTER F WITH HOOK - 0x01A0: 0xD5, # LATIN CAPITAL LETTER O WITH HORN - 0x01A1: 0xF5, # LATIN SMALL LETTER O WITH HORN - 0x01AF: 0xDD, # LATIN CAPITAL LETTER U WITH HORN - 0x01B0: 0xFD, # LATIN SMALL LETTER U WITH HORN - 0x02C6: 0x88, # MODIFIER LETTER CIRCUMFLEX ACCENT - 0x02DC: 0x98, # SMALL TILDE - 0x0300: 0xCC, # COMBINING GRAVE ACCENT - 0x0301: 0xEC, # COMBINING ACUTE ACCENT - 0x0303: 0xDE, # COMBINING TILDE - 0x0309: 0xD2, # COMBINING HOOK ABOVE - 0x0323: 0xF2, # COMBINING DOT BELOW - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201A: 0x82, # SINGLE LOW-9 QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0x84, # DOUBLE LOW-9 QUOTATION MARK - 0x2020: 0x86, # DAGGER - 0x2021: 0x87, # DOUBLE DAGGER - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x2030: 0x89, # PER MILLE SIGN - 0x2039: 0x8B, # SINGLE LEFT-POINTING ANGLE QUOTATION MARK - 0x203A: 0x9B, # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK - 0x20AB: 0xFE, # DONG SIGN - 0x20AC: 0x80, # EURO SIGN - 0x2122: 0x99, # TRADE MARK SIGN -} Modified: python/trunk/Lib/encodings/cp424.py ============================================================================== --- python/trunk/Lib/encodings/cp424.py (original) +++ python/trunk/Lib/encodings/cp424.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,225 +303,6 @@ u'\x9f' # 0xFF -> EIGHT ONES ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x37, # END OF TRANSMISSION - 0x0005: 0x2D, # ENQUIRY - 0x0006: 0x2E, # ACKNOWLEDGE - 0x0007: 0x2F, # BELL - 0x0008: 0x16, # BACKSPACE - 0x0009: 0x05, # HORIZONTAL TABULATION - 0x000A: 0x25, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x3C, # DEVICE CONTROL FOUR - 0x0015: 0x3D, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x32, # SYNCHRONOUS IDLE - 0x0017: 0x26, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x3F, # SUBSTITUTE - 0x001B: 0x27, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x40, # SPACE - 0x0021: 0x5A, # EXCLAMATION MARK - 0x0022: 0x7F, # QUOTATION MARK - 0x0023: 0x7B, # NUMBER SIGN - 0x0024: 0x5B, # DOLLAR SIGN - 0x0025: 0x6C, # PERCENT SIGN - 0x0026: 0x50, # AMPERSAND - 0x0027: 0x7D, # APOSTROPHE - 0x0028: 0x4D, # LEFT PARENTHESIS - 0x0029: 0x5D, # RIGHT PARENTHESIS - 0x002A: 0x5C, # ASTERISK - 0x002B: 0x4E, # PLUS SIGN - 0x002C: 0x6B, # COMMA - 0x002D: 0x60, # HYPHEN-MINUS - 0x002E: 0x4B, # FULL STOP - 0x002F: 0x61, # SOLIDUS - 0x0030: 0xF0, # DIGIT ZERO - 0x0031: 0xF1, # DIGIT ONE - 0x0032: 0xF2, # DIGIT TWO - 0x0033: 0xF3, # DIGIT THREE - 0x0034: 0xF4, # DIGIT FOUR - 0x0035: 0xF5, # DIGIT FIVE - 0x0036: 0xF6, # DIGIT SIX - 0x0037: 0xF7, # DIGIT SEVEN - 0x0038: 0xF8, # DIGIT EIGHT - 0x0039: 0xF9, # DIGIT NINE - 0x003A: 0x7A, # COLON - 0x003B: 0x5E, # SEMICOLON - 0x003C: 0x4C, # LESS-THAN SIGN - 0x003D: 0x7E, # EQUALS SIGN - 0x003E: 0x6E, # GREATER-THAN SIGN - 0x003F: 0x6F, # QUESTION MARK - 0x0040: 0x7C, # COMMERCIAL AT - 0x0041: 0xC1, # LATIN CAPITAL LETTER A - 0x0042: 0xC2, # LATIN CAPITAL LETTER B - 0x0043: 0xC3, # LATIN CAPITAL LETTER C - 0x0044: 0xC4, # LATIN CAPITAL LETTER D - 0x0045: 0xC5, # LATIN CAPITAL LETTER E - 0x0046: 0xC6, # LATIN CAPITAL LETTER F - 0x0047: 0xC7, # LATIN CAPITAL LETTER G - 0x0048: 0xC8, # LATIN CAPITAL LETTER H - 0x0049: 0xC9, # LATIN CAPITAL LETTER I - 0x004A: 0xD1, # LATIN CAPITAL LETTER J - 0x004B: 0xD2, # LATIN CAPITAL LETTER K - 0x004C: 0xD3, # LATIN CAPITAL LETTER L - 0x004D: 0xD4, # LATIN CAPITAL LETTER M - 0x004E: 0xD5, # LATIN CAPITAL LETTER N - 0x004F: 0xD6, # LATIN CAPITAL LETTER O - 0x0050: 0xD7, # LATIN CAPITAL LETTER P - 0x0051: 0xD8, # LATIN CAPITAL LETTER Q - 0x0052: 0xD9, # LATIN CAPITAL LETTER R - 0x0053: 0xE2, # LATIN CAPITAL LETTER S - 0x0054: 0xE3, # LATIN CAPITAL LETTER T - 0x0055: 0xE4, # LATIN CAPITAL LETTER U - 0x0056: 0xE5, # LATIN CAPITAL LETTER V - 0x0057: 0xE6, # LATIN CAPITAL LETTER W - 0x0058: 0xE7, # LATIN CAPITAL LETTER X - 0x0059: 0xE8, # LATIN CAPITAL LETTER Y - 0x005A: 0xE9, # LATIN CAPITAL LETTER Z - 0x005B: 0xBA, # LEFT SQUARE BRACKET - 0x005C: 0xE0, # REVERSE SOLIDUS - 0x005D: 0xBB, # RIGHT SQUARE BRACKET - 0x005E: 0xB0, # CIRCUMFLEX ACCENT - 0x005F: 0x6D, # LOW LINE - 0x0060: 0x79, # GRAVE ACCENT - 0x0061: 0x81, # LATIN SMALL LETTER A - 0x0062: 0x82, # LATIN SMALL LETTER B - 0x0063: 0x83, # LATIN SMALL LETTER C - 0x0064: 0x84, # LATIN SMALL LETTER D - 0x0065: 0x85, # LATIN SMALL LETTER E - 0x0066: 0x86, # LATIN SMALL LETTER F - 0x0067: 0x87, # LATIN SMALL LETTER G - 0x0068: 0x88, # LATIN SMALL LETTER H - 0x0069: 0x89, # LATIN SMALL LETTER I - 0x006A: 0x91, # LATIN SMALL LETTER J - 0x006B: 0x92, # LATIN SMALL LETTER K - 0x006C: 0x93, # LATIN SMALL LETTER L - 0x006D: 0x94, # LATIN SMALL LETTER M - 0x006E: 0x95, # LATIN SMALL LETTER N - 0x006F: 0x96, # LATIN SMALL LETTER O - 0x0070: 0x97, # LATIN SMALL LETTER P - 0x0071: 0x98, # LATIN SMALL LETTER Q - 0x0072: 0x99, # LATIN SMALL LETTER R - 0x0073: 0xA2, # LATIN SMALL LETTER S - 0x0074: 0xA3, # LATIN SMALL LETTER T - 0x0075: 0xA4, # LATIN SMALL LETTER U - 0x0076: 0xA5, # LATIN SMALL LETTER V - 0x0077: 0xA6, # LATIN SMALL LETTER W - 0x0078: 0xA7, # LATIN SMALL LETTER X - 0x0079: 0xA8, # LATIN SMALL LETTER Y - 0x007A: 0xA9, # LATIN SMALL LETTER Z - 0x007B: 0xC0, # LEFT CURLY BRACKET - 0x007C: 0x4F, # VERTICAL LINE - 0x007D: 0xD0, # RIGHT CURLY BRACKET - 0x007E: 0xA1, # TILDE - 0x007F: 0x07, # DELETE - 0x0080: 0x20, # DIGIT SELECT - 0x0081: 0x21, # START OF SIGNIFICANCE - 0x0082: 0x22, # FIELD SEPARATOR - 0x0083: 0x23, # WORD UNDERSCORE - 0x0084: 0x24, # BYPASS OR INHIBIT PRESENTATION - 0x0085: 0x15, # NEW LINE - 0x0086: 0x06, # REQUIRED NEW LINE - 0x0087: 0x17, # PROGRAM OPERATOR COMMUNICATION - 0x0088: 0x28, # SET ATTRIBUTE - 0x0089: 0x29, # START FIELD EXTENDED - 0x008A: 0x2A, # SET MODE OR SWITCH - 0x008B: 0x2B, # CONTROL SEQUENCE PREFIX - 0x008C: 0x2C, # MODIFY FIELD ATTRIBUTE - 0x008D: 0x09, # SUPERSCRIPT - 0x008E: 0x0A, # REPEAT - 0x008F: 0x1B, # CUSTOMER USE ONE - 0x0090: 0x30, # - 0x0091: 0x31, # - 0x0092: 0x1A, # UNIT BACK SPACE - 0x0093: 0x33, # INDEX RETURN - 0x0094: 0x34, # PRESENTATION POSITION - 0x0095: 0x35, # TRANSPARENT - 0x0096: 0x36, # NUMERIC BACKSPACE - 0x0097: 0x08, # GRAPHIC ESCAPE - 0x0098: 0x38, # SUBSCRIPT - 0x0099: 0x39, # INDENT TABULATION - 0x009A: 0x3A, # REVERSE FORM FEED - 0x009B: 0x3B, # CUSTOMER USE THREE - 0x009C: 0x04, # SELECT - 0x009D: 0x14, # RESTORE/ENABLE PRESENTATION - 0x009E: 0x3E, # - 0x009F: 0xFF, # EIGHT ONES - 0x00A0: 0x74, # NO-BREAK SPACE - 0x00A2: 0x4A, # CENT SIGN - 0x00A3: 0xB1, # POUND SIGN - 0x00A4: 0x9F, # CURRENCY SIGN - 0x00A5: 0xB2, # YEN SIGN - 0x00A6: 0x6A, # BROKEN BAR - 0x00A7: 0xB5, # SECTION SIGN - 0x00A8: 0xBD, # DIAERESIS - 0x00A9: 0xB4, # COPYRIGHT SIGN - 0x00AB: 0x8A, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0x5F, # NOT SIGN - 0x00AD: 0xCA, # SOFT HYPHEN - 0x00AE: 0xAF, # REGISTERED SIGN - 0x00AF: 0xBC, # MACRON - 0x00B0: 0x90, # DEGREE SIGN - 0x00B1: 0x8F, # PLUS-MINUS SIGN - 0x00B2: 0xEA, # SUPERSCRIPT TWO - 0x00B3: 0xFA, # SUPERSCRIPT THREE - 0x00B4: 0xBE, # ACUTE ACCENT - 0x00B5: 0xA0, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB3, # MIDDLE DOT - 0x00B8: 0x9D, # CEDILLA - 0x00B9: 0xDA, # SUPERSCRIPT ONE - 0x00BB: 0x8B, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xB7, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xB8, # VULGAR FRACTION ONE HALF - 0x00BE: 0xB9, # VULGAR FRACTION THREE QUARTERS - 0x00D7: 0xBF, # MULTIPLICATION SIGN - 0x00F7: 0xE1, # DIVISION SIGN - 0x05D0: 0x41, # HEBREW LETTER ALEF - 0x05D1: 0x42, # HEBREW LETTER BET - 0x05D2: 0x43, # HEBREW LETTER GIMEL - 0x05D3: 0x44, # HEBREW LETTER DALET - 0x05D4: 0x45, # HEBREW LETTER HE - 0x05D5: 0x46, # HEBREW LETTER VAV - 0x05D6: 0x47, # HEBREW LETTER ZAYIN - 0x05D7: 0x48, # HEBREW LETTER HET - 0x05D8: 0x49, # HEBREW LETTER TET - 0x05D9: 0x51, # HEBREW LETTER YOD - 0x05DA: 0x52, # HEBREW LETTER FINAL KAF - 0x05DB: 0x53, # HEBREW LETTER KAF - 0x05DC: 0x54, # HEBREW LETTER LAMED - 0x05DD: 0x55, # HEBREW LETTER FINAL MEM - 0x05DE: 0x56, # HEBREW LETTER MEM - 0x05DF: 0x57, # HEBREW LETTER FINAL NUN - 0x05E0: 0x58, # HEBREW LETTER NUN - 0x05E1: 0x59, # HEBREW LETTER SAMEKH - 0x05E2: 0x62, # HEBREW LETTER AYIN - 0x05E3: 0x63, # HEBREW LETTER FINAL PE - 0x05E4: 0x64, # HEBREW LETTER PE - 0x05E5: 0x65, # HEBREW LETTER FINAL TSADI - 0x05E6: 0x66, # HEBREW LETTER TSADI - 0x05E7: 0x67, # HEBREW LETTER QOF - 0x05E8: 0x68, # HEBREW LETTER RESH - 0x05E9: 0x69, # HEBREW LETTER SHIN - 0x05EA: 0x71, # HEBREW LETTER TAV - 0x2017: 0x78, # DOUBLE LOW LINE -} Modified: python/trunk/Lib/encodings/cp500.py ============================================================================== --- python/trunk/Lib/encodings/cp500.py (original) +++ python/trunk/Lib/encodings/cp500.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\x9f' # 0xFF -> CONTROL ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x37, # END OF TRANSMISSION - 0x0005: 0x2D, # ENQUIRY - 0x0006: 0x2E, # ACKNOWLEDGE - 0x0007: 0x2F, # BELL - 0x0008: 0x16, # BACKSPACE - 0x0009: 0x05, # HORIZONTAL TABULATION - 0x000A: 0x25, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x3C, # DEVICE CONTROL FOUR - 0x0015: 0x3D, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x32, # SYNCHRONOUS IDLE - 0x0017: 0x26, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x3F, # SUBSTITUTE - 0x001B: 0x27, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x40, # SPACE - 0x0021: 0x4F, # EXCLAMATION MARK - 0x0022: 0x7F, # QUOTATION MARK - 0x0023: 0x7B, # NUMBER SIGN - 0x0024: 0x5B, # DOLLAR SIGN - 0x0025: 0x6C, # PERCENT SIGN - 0x0026: 0x50, # AMPERSAND - 0x0027: 0x7D, # APOSTROPHE - 0x0028: 0x4D, # LEFT PARENTHESIS - 0x0029: 0x5D, # RIGHT PARENTHESIS - 0x002A: 0x5C, # ASTERISK - 0x002B: 0x4E, # PLUS SIGN - 0x002C: 0x6B, # COMMA - 0x002D: 0x60, # HYPHEN-MINUS - 0x002E: 0x4B, # FULL STOP - 0x002F: 0x61, # SOLIDUS - 0x0030: 0xF0, # DIGIT ZERO - 0x0031: 0xF1, # DIGIT ONE - 0x0032: 0xF2, # DIGIT TWO - 0x0033: 0xF3, # DIGIT THREE - 0x0034: 0xF4, # DIGIT FOUR - 0x0035: 0xF5, # DIGIT FIVE - 0x0036: 0xF6, # DIGIT SIX - 0x0037: 0xF7, # DIGIT SEVEN - 0x0038: 0xF8, # DIGIT EIGHT - 0x0039: 0xF9, # DIGIT NINE - 0x003A: 0x7A, # COLON - 0x003B: 0x5E, # SEMICOLON - 0x003C: 0x4C, # LESS-THAN SIGN - 0x003D: 0x7E, # EQUALS SIGN - 0x003E: 0x6E, # GREATER-THAN SIGN - 0x003F: 0x6F, # QUESTION MARK - 0x0040: 0x7C, # COMMERCIAL AT - 0x0041: 0xC1, # LATIN CAPITAL LETTER A - 0x0042: 0xC2, # LATIN CAPITAL LETTER B - 0x0043: 0xC3, # LATIN CAPITAL LETTER C - 0x0044: 0xC4, # LATIN CAPITAL LETTER D - 0x0045: 0xC5, # LATIN CAPITAL LETTER E - 0x0046: 0xC6, # LATIN CAPITAL LETTER F - 0x0047: 0xC7, # LATIN CAPITAL LETTER G - 0x0048: 0xC8, # LATIN CAPITAL LETTER H - 0x0049: 0xC9, # LATIN CAPITAL LETTER I - 0x004A: 0xD1, # LATIN CAPITAL LETTER J - 0x004B: 0xD2, # LATIN CAPITAL LETTER K - 0x004C: 0xD3, # LATIN CAPITAL LETTER L - 0x004D: 0xD4, # LATIN CAPITAL LETTER M - 0x004E: 0xD5, # LATIN CAPITAL LETTER N - 0x004F: 0xD6, # LATIN CAPITAL LETTER O - 0x0050: 0xD7, # LATIN CAPITAL LETTER P - 0x0051: 0xD8, # LATIN CAPITAL LETTER Q - 0x0052: 0xD9, # LATIN CAPITAL LETTER R - 0x0053: 0xE2, # LATIN CAPITAL LETTER S - 0x0054: 0xE3, # LATIN CAPITAL LETTER T - 0x0055: 0xE4, # LATIN CAPITAL LETTER U - 0x0056: 0xE5, # LATIN CAPITAL LETTER V - 0x0057: 0xE6, # LATIN CAPITAL LETTER W - 0x0058: 0xE7, # LATIN CAPITAL LETTER X - 0x0059: 0xE8, # LATIN CAPITAL LETTER Y - 0x005A: 0xE9, # LATIN CAPITAL LETTER Z - 0x005B: 0x4A, # LEFT SQUARE BRACKET - 0x005C: 0xE0, # REVERSE SOLIDUS - 0x005D: 0x5A, # RIGHT SQUARE BRACKET - 0x005E: 0x5F, # CIRCUMFLEX ACCENT - 0x005F: 0x6D, # LOW LINE - 0x0060: 0x79, # GRAVE ACCENT - 0x0061: 0x81, # LATIN SMALL LETTER A - 0x0062: 0x82, # LATIN SMALL LETTER B - 0x0063: 0x83, # LATIN SMALL LETTER C - 0x0064: 0x84, # LATIN SMALL LETTER D - 0x0065: 0x85, # LATIN SMALL LETTER E - 0x0066: 0x86, # LATIN SMALL LETTER F - 0x0067: 0x87, # LATIN SMALL LETTER G - 0x0068: 0x88, # LATIN SMALL LETTER H - 0x0069: 0x89, # LATIN SMALL LETTER I - 0x006A: 0x91, # LATIN SMALL LETTER J - 0x006B: 0x92, # LATIN SMALL LETTER K - 0x006C: 0x93, # LATIN SMALL LETTER L - 0x006D: 0x94, # LATIN SMALL LETTER M - 0x006E: 0x95, # LATIN SMALL LETTER N - 0x006F: 0x96, # LATIN SMALL LETTER O - 0x0070: 0x97, # LATIN SMALL LETTER P - 0x0071: 0x98, # LATIN SMALL LETTER Q - 0x0072: 0x99, # LATIN SMALL LETTER R - 0x0073: 0xA2, # LATIN SMALL LETTER S - 0x0074: 0xA3, # LATIN SMALL LETTER T - 0x0075: 0xA4, # LATIN SMALL LETTER U - 0x0076: 0xA5, # LATIN SMALL LETTER V - 0x0077: 0xA6, # LATIN SMALL LETTER W - 0x0078: 0xA7, # LATIN SMALL LETTER X - 0x0079: 0xA8, # LATIN SMALL LETTER Y - 0x007A: 0xA9, # LATIN SMALL LETTER Z - 0x007B: 0xC0, # LEFT CURLY BRACKET - 0x007C: 0xBB, # VERTICAL LINE - 0x007D: 0xD0, # RIGHT CURLY BRACKET - 0x007E: 0xA1, # TILDE - 0x007F: 0x07, # DELETE - 0x0080: 0x20, # CONTROL - 0x0081: 0x21, # CONTROL - 0x0082: 0x22, # CONTROL - 0x0083: 0x23, # CONTROL - 0x0084: 0x24, # CONTROL - 0x0085: 0x15, # CONTROL - 0x0086: 0x06, # CONTROL - 0x0087: 0x17, # CONTROL - 0x0088: 0x28, # CONTROL - 0x0089: 0x29, # CONTROL - 0x008A: 0x2A, # CONTROL - 0x008B: 0x2B, # CONTROL - 0x008C: 0x2C, # CONTROL - 0x008D: 0x09, # CONTROL - 0x008E: 0x0A, # CONTROL - 0x008F: 0x1B, # CONTROL - 0x0090: 0x30, # CONTROL - 0x0091: 0x31, # CONTROL - 0x0092: 0x1A, # CONTROL - 0x0093: 0x33, # CONTROL - 0x0094: 0x34, # CONTROL - 0x0095: 0x35, # CONTROL - 0x0096: 0x36, # CONTROL - 0x0097: 0x08, # CONTROL - 0x0098: 0x38, # CONTROL - 0x0099: 0x39, # CONTROL - 0x009A: 0x3A, # CONTROL - 0x009B: 0x3B, # CONTROL - 0x009C: 0x04, # CONTROL - 0x009D: 0x14, # CONTROL - 0x009E: 0x3E, # CONTROL - 0x009F: 0xFF, # CONTROL - 0x00A0: 0x41, # NO-BREAK SPACE - 0x00A1: 0xAA, # INVERTED EXCLAMATION MARK - 0x00A2: 0xB0, # CENT SIGN - 0x00A3: 0xB1, # POUND SIGN - 0x00A4: 0x9F, # CURRENCY SIGN - 0x00A5: 0xB2, # YEN SIGN - 0x00A6: 0x6A, # BROKEN BAR - 0x00A7: 0xB5, # SECTION SIGN - 0x00A8: 0xBD, # DIAERESIS - 0x00A9: 0xB4, # COPYRIGHT SIGN - 0x00AA: 0x9A, # FEMININE ORDINAL INDICATOR - 0x00AB: 0x8A, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xBA, # NOT SIGN - 0x00AD: 0xCA, # SOFT HYPHEN - 0x00AE: 0xAF, # REGISTERED SIGN - 0x00AF: 0xBC, # MACRON - 0x00B0: 0x90, # DEGREE SIGN - 0x00B1: 0x8F, # PLUS-MINUS SIGN - 0x00B2: 0xEA, # SUPERSCRIPT TWO - 0x00B3: 0xFA, # SUPERSCRIPT THREE - 0x00B4: 0xBE, # ACUTE ACCENT - 0x00B5: 0xA0, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB3, # MIDDLE DOT - 0x00B8: 0x9D, # CEDILLA - 0x00B9: 0xDA, # SUPERSCRIPT ONE - 0x00BA: 0x9B, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0x8B, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xB7, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xB8, # VULGAR FRACTION ONE HALF - 0x00BE: 0xB9, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xAB, # INVERTED QUESTION MARK - 0x00C0: 0x64, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0x65, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0x62, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0x66, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0x63, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0x67, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0x9E, # LATIN CAPITAL LIGATURE AE - 0x00C7: 0x68, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0x74, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0x71, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0x72, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0x73, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0x78, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0x75, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0x76, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0x77, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D0: 0xAC, # LATIN CAPITAL LETTER ETH (ICELANDIC) - 0x00D1: 0x69, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xED, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xEE, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xEB, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xEF, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xEC, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xBF, # MULTIPLICATION SIGN - 0x00D8: 0x80, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xFD, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xFE, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xFB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xFC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xAD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DE: 0xAE, # LATIN CAPITAL LETTER THORN (ICELANDIC) - 0x00DF: 0x59, # LATIN SMALL LETTER SHARP S (GERMAN) - 0x00E0: 0x44, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0x45, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0x42, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0x46, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0x43, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0x47, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0x9C, # LATIN SMALL LIGATURE AE - 0x00E7: 0x48, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0x54, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0x51, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0x52, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0x53, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0x58, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0x55, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0x56, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0x57, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F0: 0x8C, # LATIN SMALL LETTER ETH (ICELANDIC) - 0x00F1: 0x49, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xCD, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xCE, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xCB, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xCF, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xCC, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xE1, # DIVISION SIGN - 0x00F8: 0x70, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xDD, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xDE, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xDB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xDC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0x8D, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FE: 0x8E, # LATIN SMALL LETTER THORN (ICELANDIC) - 0x00FF: 0xDF, # LATIN SMALL LETTER Y WITH DIAERESIS -} Modified: python/trunk/Lib/encodings/cp856.py ============================================================================== --- python/trunk/Lib/encodings/cp856.py (original) +++ python/trunk/Lib/encodings/cp856.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,222 +303,6 @@ u'\xa0' # 0xFF -> NO-BREAK SPACE ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xFF, # NO-BREAK SPACE - 0x00A2: 0xBD, # CENT SIGN - 0x00A3: 0x9C, # POUND SIGN - 0x00A4: 0xCF, # CURRENCY SIGN - 0x00A5: 0xBE, # YEN SIGN - 0x00A6: 0xDD, # BROKEN BAR - 0x00A7: 0xF5, # SECTION SIGN - 0x00A8: 0xF9, # DIAERESIS - 0x00A9: 0xB8, # COPYRIGHT SIGN - 0x00AB: 0xAE, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAA, # NOT SIGN - 0x00AD: 0xF0, # SOFT HYPHEN - 0x00AE: 0xA9, # REGISTERED SIGN - 0x00AF: 0xEE, # MACRON - 0x00B0: 0xF8, # DEGREE SIGN - 0x00B1: 0xF1, # PLUS-MINUS SIGN - 0x00B2: 0xFD, # SUPERSCRIPT TWO - 0x00B3: 0xFC, # SUPERSCRIPT THREE - 0x00B4: 0xEF, # ACUTE ACCENT - 0x00B5: 0xE6, # MICRO SIGN - 0x00B6: 0xF4, # PILCROW SIGN - 0x00B7: 0xFA, # MIDDLE DOT - 0x00B8: 0xF7, # CEDILLA - 0x00B9: 0xFB, # SUPERSCRIPT ONE - 0x00BB: 0xAF, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xAC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xAB, # VULGAR FRACTION ONE HALF - 0x00BE: 0xF3, # VULGAR FRACTION THREE QUARTERS - 0x00D7: 0x9E, # MULTIPLICATION SIGN - 0x00F7: 0xF6, # DIVISION SIGN - 0x05D0: 0x80, # HEBREW LETTER ALEF - 0x05D1: 0x81, # HEBREW LETTER BET - 0x05D2: 0x82, # HEBREW LETTER GIMEL - 0x05D3: 0x83, # HEBREW LETTER DALET - 0x05D4: 0x84, # HEBREW LETTER HE - 0x05D5: 0x85, # HEBREW LETTER VAV - 0x05D6: 0x86, # HEBREW LETTER ZAYIN - 0x05D7: 0x87, # HEBREW LETTER HET - 0x05D8: 0x88, # HEBREW LETTER TET - 0x05D9: 0x89, # HEBREW LETTER YOD - 0x05DA: 0x8A, # HEBREW LETTER FINAL KAF - 0x05DB: 0x8B, # HEBREW LETTER KAF - 0x05DC: 0x8C, # HEBREW LETTER LAMED - 0x05DD: 0x8D, # HEBREW LETTER FINAL MEM - 0x05DE: 0x8E, # HEBREW LETTER MEM - 0x05DF: 0x8F, # HEBREW LETTER FINAL NUN - 0x05E0: 0x90, # HEBREW LETTER NUN - 0x05E1: 0x91, # HEBREW LETTER SAMEKH - 0x05E2: 0x92, # HEBREW LETTER AYIN - 0x05E3: 0x93, # HEBREW LETTER FINAL PE - 0x05E4: 0x94, # HEBREW LETTER PE - 0x05E5: 0x95, # HEBREW LETTER FINAL TSADI - 0x05E6: 0x96, # HEBREW LETTER TSADI - 0x05E7: 0x97, # HEBREW LETTER QOF - 0x05E8: 0x98, # HEBREW LETTER RESH - 0x05E9: 0x99, # HEBREW LETTER SHIN - 0x05EA: 0x9A, # HEBREW LETTER TAV - 0x2017: 0xF2, # DOUBLE LOW LINE - 0x2500: 0xC4, # BOX DRAWINGS LIGHT HORIZONTAL - 0x2502: 0xB3, # BOX DRAWINGS LIGHT VERTICAL - 0x250C: 0xDA, # BOX DRAWINGS LIGHT DOWN AND RIGHT - 0x2510: 0xBF, # BOX DRAWINGS LIGHT DOWN AND LEFT - 0x2514: 0xC0, # BOX DRAWINGS LIGHT UP AND RIGHT - 0x2518: 0xD9, # BOX DRAWINGS LIGHT UP AND LEFT - 0x251C: 0xC3, # BOX DRAWINGS LIGHT VERTICAL AND RIGHT - 0x2524: 0xB4, # BOX DRAWINGS LIGHT VERTICAL AND LEFT - 0x252C: 0xC2, # BOX DRAWINGS LIGHT DOWN AND HORIZONTAL - 0x2534: 0xC1, # BOX DRAWINGS LIGHT UP AND HORIZONTAL - 0x253C: 0xC5, # BOX DRAWINGS LIGHT VERTICAL AND HORIZONTAL - 0x2550: 0xCD, # BOX DRAWINGS DOUBLE HORIZONTAL - 0x2551: 0xBA, # BOX DRAWINGS DOUBLE VERTICAL - 0x2554: 0xC9, # BOX DRAWINGS DOUBLE DOWN AND RIGHT - 0x2557: 0xBB, # BOX DRAWINGS DOUBLE DOWN AND LEFT - 0x255A: 0xC8, # BOX DRAWINGS DOUBLE UP AND RIGHT - 0x255D: 0xBC, # BOX DRAWINGS DOUBLE UP AND LEFT - 0x2560: 0xCC, # BOX DRAWINGS DOUBLE VERTICAL AND RIGHT - 0x2563: 0xB9, # BOX DRAWINGS DOUBLE VERTICAL AND LEFT - 0x2566: 0xCB, # BOX DRAWINGS DOUBLE DOWN AND HORIZONTAL - 0x2569: 0xCA, # BOX DRAWINGS DOUBLE UP AND HORIZONTAL - 0x256C: 0xCE, # BOX DRAWINGS DOUBLE VERTICAL AND HORIZONTAL - 0x2580: 0xDF, # UPPER HALF BLOCK - 0x2584: 0xDC, # LOWER HALF BLOCK - 0x2588: 0xDB, # FULL BLOCK - 0x2591: 0xB0, # LIGHT SHADE - 0x2592: 0xB1, # MEDIUM SHADE - 0x2593: 0xB2, # DARK SHADE - 0x25A0: 0xFE, # BLACK SQUARE -} Modified: python/trunk/Lib/encodings/cp874.py ============================================================================== --- python/trunk/Lib/encodings/cp874.py (original) +++ python/trunk/Lib/encodings/cp874.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,232 +303,6 @@ u'\ufffe' # 0xFF -> UNDEFINED ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x0E01: 0xA1, # THAI CHARACTER KO KAI - 0x0E02: 0xA2, # THAI CHARACTER KHO KHAI - 0x0E03: 0xA3, # THAI CHARACTER KHO KHUAT - 0x0E04: 0xA4, # THAI CHARACTER KHO KHWAI - 0x0E05: 0xA5, # THAI CHARACTER KHO KHON - 0x0E06: 0xA6, # THAI CHARACTER KHO RAKHANG - 0x0E07: 0xA7, # THAI CHARACTER NGO NGU - 0x0E08: 0xA8, # THAI CHARACTER CHO CHAN - 0x0E09: 0xA9, # THAI CHARACTER CHO CHING - 0x0E0A: 0xAA, # THAI CHARACTER CHO CHANG - 0x0E0B: 0xAB, # THAI CHARACTER SO SO - 0x0E0C: 0xAC, # THAI CHARACTER CHO CHOE - 0x0E0D: 0xAD, # THAI CHARACTER YO YING - 0x0E0E: 0xAE, # THAI CHARACTER DO CHADA - 0x0E0F: 0xAF, # THAI CHARACTER TO PATAK - 0x0E10: 0xB0, # THAI CHARACTER THO THAN - 0x0E11: 0xB1, # THAI CHARACTER THO NANGMONTHO - 0x0E12: 0xB2, # THAI CHARACTER THO PHUTHAO - 0x0E13: 0xB3, # THAI CHARACTER NO NEN - 0x0E14: 0xB4, # THAI CHARACTER DO DEK - 0x0E15: 0xB5, # THAI CHARACTER TO TAO - 0x0E16: 0xB6, # THAI CHARACTER THO THUNG - 0x0E17: 0xB7, # THAI CHARACTER THO THAHAN - 0x0E18: 0xB8, # THAI CHARACTER THO THONG - 0x0E19: 0xB9, # THAI CHARACTER NO NU - 0x0E1A: 0xBA, # THAI CHARACTER BO BAIMAI - 0x0E1B: 0xBB, # THAI CHARACTER PO PLA - 0x0E1C: 0xBC, # THAI CHARACTER PHO PHUNG - 0x0E1D: 0xBD, # THAI CHARACTER FO FA - 0x0E1E: 0xBE, # THAI CHARACTER PHO PHAN - 0x0E1F: 0xBF, # THAI CHARACTER FO FAN - 0x0E20: 0xC0, # THAI CHARACTER PHO SAMPHAO - 0x0E21: 0xC1, # THAI CHARACTER MO MA - 0x0E22: 0xC2, # THAI CHARACTER YO YAK - 0x0E23: 0xC3, # THAI CHARACTER RO RUA - 0x0E24: 0xC4, # THAI CHARACTER RU - 0x0E25: 0xC5, # THAI CHARACTER LO LING - 0x0E26: 0xC6, # THAI CHARACTER LU - 0x0E27: 0xC7, # THAI CHARACTER WO WAEN - 0x0E28: 0xC8, # THAI CHARACTER SO SALA - 0x0E29: 0xC9, # THAI CHARACTER SO RUSI - 0x0E2A: 0xCA, # THAI CHARACTER SO SUA - 0x0E2B: 0xCB, # THAI CHARACTER HO HIP - 0x0E2C: 0xCC, # THAI CHARACTER LO CHULA - 0x0E2D: 0xCD, # THAI CHARACTER O ANG - 0x0E2E: 0xCE, # THAI CHARACTER HO NOKHUK - 0x0E2F: 0xCF, # THAI CHARACTER PAIYANNOI - 0x0E30: 0xD0, # THAI CHARACTER SARA A - 0x0E31: 0xD1, # THAI CHARACTER MAI HAN-AKAT - 0x0E32: 0xD2, # THAI CHARACTER SARA AA - 0x0E33: 0xD3, # THAI CHARACTER SARA AM - 0x0E34: 0xD4, # THAI CHARACTER SARA I - 0x0E35: 0xD5, # THAI CHARACTER SARA II - 0x0E36: 0xD6, # THAI CHARACTER SARA UE - 0x0E37: 0xD7, # THAI CHARACTER SARA UEE - 0x0E38: 0xD8, # THAI CHARACTER SARA U - 0x0E39: 0xD9, # THAI CHARACTER SARA UU - 0x0E3A: 0xDA, # THAI CHARACTER PHINTHU - 0x0E3F: 0xDF, # THAI CURRENCY SYMBOL BAHT - 0x0E40: 0xE0, # THAI CHARACTER SARA E - 0x0E41: 0xE1, # THAI CHARACTER SARA AE - 0x0E42: 0xE2, # THAI CHARACTER SARA O - 0x0E43: 0xE3, # THAI CHARACTER SARA AI MAIMUAN - 0x0E44: 0xE4, # THAI CHARACTER SARA AI MAIMALAI - 0x0E45: 0xE5, # THAI CHARACTER LAKKHANGYAO - 0x0E46: 0xE6, # THAI CHARACTER MAIYAMOK - 0x0E47: 0xE7, # THAI CHARACTER MAITAIKHU - 0x0E48: 0xE8, # THAI CHARACTER MAI EK - 0x0E49: 0xE9, # THAI CHARACTER MAI THO - 0x0E4A: 0xEA, # THAI CHARACTER MAI TRI - 0x0E4B: 0xEB, # THAI CHARACTER MAI CHATTAWA - 0x0E4C: 0xEC, # THAI CHARACTER THANTHAKHAT - 0x0E4D: 0xED, # THAI CHARACTER NIKHAHIT - 0x0E4E: 0xEE, # THAI CHARACTER YAMAKKAN - 0x0E4F: 0xEF, # THAI CHARACTER FONGMAN - 0x0E50: 0xF0, # THAI DIGIT ZERO - 0x0E51: 0xF1, # THAI DIGIT ONE - 0x0E52: 0xF2, # THAI DIGIT TWO - 0x0E53: 0xF3, # THAI DIGIT THREE - 0x0E54: 0xF4, # THAI DIGIT FOUR - 0x0E55: 0xF5, # THAI DIGIT FIVE - 0x0E56: 0xF6, # THAI DIGIT SIX - 0x0E57: 0xF7, # THAI DIGIT SEVEN - 0x0E58: 0xF8, # THAI DIGIT EIGHT - 0x0E59: 0xF9, # THAI DIGIT NINE - 0x0E5A: 0xFA, # THAI CHARACTER ANGKHANKHU - 0x0E5B: 0xFB, # THAI CHARACTER KHOMUT - 0x2013: 0x96, # EN DASH - 0x2014: 0x97, # EM DASH - 0x2018: 0x91, # LEFT SINGLE QUOTATION MARK - 0x2019: 0x92, # RIGHT SINGLE QUOTATION MARK - 0x201C: 0x93, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0x94, # RIGHT DOUBLE QUOTATION MARK - 0x2022: 0x95, # BULLET - 0x2026: 0x85, # HORIZONTAL ELLIPSIS - 0x20AC: 0x80, # EURO SIGN -} Modified: python/trunk/Lib/encodings/cp875.py ============================================================================== --- python/trunk/Lib/encodings/cp875.py (original) +++ python/trunk/Lib/encodings/cp875.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,257 +303,6 @@ u'\x9f' # 0xFF -> CONTROL ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x37, # END OF TRANSMISSION - 0x0005: 0x2D, # ENQUIRY - 0x0006: 0x2E, # ACKNOWLEDGE - 0x0007: 0x2F, # BELL - 0x0008: 0x16, # BACKSPACE - 0x0009: 0x05, # HORIZONTAL TABULATION - 0x000A: 0x25, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x3C, # DEVICE CONTROL FOUR - 0x0015: 0x3D, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x32, # SYNCHRONOUS IDLE - 0x0017: 0x26, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: None, # SUBSTITUTE - 0x001B: 0x27, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x40, # SPACE - 0x0021: 0x4F, # EXCLAMATION MARK - 0x0022: 0x7F, # QUOTATION MARK - 0x0023: 0x7B, # NUMBER SIGN - 0x0024: 0x5B, # DOLLAR SIGN - 0x0025: 0x6C, # PERCENT SIGN - 0x0026: 0x50, # AMPERSAND - 0x0027: 0x7D, # APOSTROPHE - 0x0028: 0x4D, # LEFT PARENTHESIS - 0x0029: 0x5D, # RIGHT PARENTHESIS - 0x002A: 0x5C, # ASTERISK - 0x002B: 0x4E, # PLUS SIGN - 0x002C: 0x6B, # COMMA - 0x002D: 0x60, # HYPHEN-MINUS - 0x002E: 0x4B, # FULL STOP - 0x002F: 0x61, # SOLIDUS - 0x0030: 0xF0, # DIGIT ZERO - 0x0031: 0xF1, # DIGIT ONE - 0x0032: 0xF2, # DIGIT TWO - 0x0033: 0xF3, # DIGIT THREE - 0x0034: 0xF4, # DIGIT FOUR - 0x0035: 0xF5, # DIGIT FIVE - 0x0036: 0xF6, # DIGIT SIX - 0x0037: 0xF7, # DIGIT SEVEN - 0x0038: 0xF8, # DIGIT EIGHT - 0x0039: 0xF9, # DIGIT NINE - 0x003A: 0x7A, # COLON - 0x003B: 0x5E, # SEMICOLON - 0x003C: 0x4C, # LESS-THAN SIGN - 0x003D: 0x7E, # EQUALS SIGN - 0x003E: 0x6E, # GREATER-THAN SIGN - 0x003F: 0x6F, # QUESTION MARK - 0x0040: 0x7C, # COMMERCIAL AT - 0x0041: 0xC1, # LATIN CAPITAL LETTER A - 0x0042: 0xC2, # LATIN CAPITAL LETTER B - 0x0043: 0xC3, # LATIN CAPITAL LETTER C - 0x0044: 0xC4, # LATIN CAPITAL LETTER D - 0x0045: 0xC5, # LATIN CAPITAL LETTER E - 0x0046: 0xC6, # LATIN CAPITAL LETTER F - 0x0047: 0xC7, # LATIN CAPITAL LETTER G - 0x0048: 0xC8, # LATIN CAPITAL LETTER H - 0x0049: 0xC9, # LATIN CAPITAL LETTER I - 0x004A: 0xD1, # LATIN CAPITAL LETTER J - 0x004B: 0xD2, # LATIN CAPITAL LETTER K - 0x004C: 0xD3, # LATIN CAPITAL LETTER L - 0x004D: 0xD4, # LATIN CAPITAL LETTER M - 0x004E: 0xD5, # LATIN CAPITAL LETTER N - 0x004F: 0xD6, # LATIN CAPITAL LETTER O - 0x0050: 0xD7, # LATIN CAPITAL LETTER P - 0x0051: 0xD8, # LATIN CAPITAL LETTER Q - 0x0052: 0xD9, # LATIN CAPITAL LETTER R - 0x0053: 0xE2, # LATIN CAPITAL LETTER S - 0x0054: 0xE3, # LATIN CAPITAL LETTER T - 0x0055: 0xE4, # LATIN CAPITAL LETTER U - 0x0056: 0xE5, # LATIN CAPITAL LETTER V - 0x0057: 0xE6, # LATIN CAPITAL LETTER W - 0x0058: 0xE7, # LATIN CAPITAL LETTER X - 0x0059: 0xE8, # LATIN CAPITAL LETTER Y - 0x005A: 0xE9, # LATIN CAPITAL LETTER Z - 0x005B: 0x4A, # LEFT SQUARE BRACKET - 0x005C: 0xE0, # REVERSE SOLIDUS - 0x005D: 0x5A, # RIGHT SQUARE BRACKET - 0x005E: 0x5F, # CIRCUMFLEX ACCENT - 0x005F: 0x6D, # LOW LINE - 0x0060: 0x79, # GRAVE ACCENT - 0x0061: 0x81, # LATIN SMALL LETTER A - 0x0062: 0x82, # LATIN SMALL LETTER B - 0x0063: 0x83, # LATIN SMALL LETTER C - 0x0064: 0x84, # LATIN SMALL LETTER D - 0x0065: 0x85, # LATIN SMALL LETTER E - 0x0066: 0x86, # LATIN SMALL LETTER F - 0x0067: 0x87, # LATIN SMALL LETTER G - 0x0068: 0x88, # LATIN SMALL LETTER H - 0x0069: 0x89, # LATIN SMALL LETTER I - 0x006A: 0x91, # LATIN SMALL LETTER J - 0x006B: 0x92, # LATIN SMALL LETTER K - 0x006C: 0x93, # LATIN SMALL LETTER L - 0x006D: 0x94, # LATIN SMALL LETTER M - 0x006E: 0x95, # LATIN SMALL LETTER N - 0x006F: 0x96, # LATIN SMALL LETTER O - 0x0070: 0x97, # LATIN SMALL LETTER P - 0x0071: 0x98, # LATIN SMALL LETTER Q - 0x0072: 0x99, # LATIN SMALL LETTER R - 0x0073: 0xA2, # LATIN SMALL LETTER S - 0x0074: 0xA3, # LATIN SMALL LETTER T - 0x0075: 0xA4, # LATIN SMALL LETTER U - 0x0076: 0xA5, # LATIN SMALL LETTER V - 0x0077: 0xA6, # LATIN SMALL LETTER W - 0x0078: 0xA7, # LATIN SMALL LETTER X - 0x0079: 0xA8, # LATIN SMALL LETTER Y - 0x007A: 0xA9, # LATIN SMALL LETTER Z - 0x007B: 0xC0, # LEFT CURLY BRACKET - 0x007C: 0x6A, # VERTICAL LINE - 0x007D: 0xD0, # RIGHT CURLY BRACKET - 0x007E: 0xA1, # TILDE - 0x007F: 0x07, # DELETE - 0x0080: 0x20, # CONTROL - 0x0081: 0x21, # CONTROL - 0x0082: 0x22, # CONTROL - 0x0083: 0x23, # CONTROL - 0x0084: 0x24, # CONTROL - 0x0085: 0x15, # CONTROL - 0x0086: 0x06, # CONTROL - 0x0087: 0x17, # CONTROL - 0x0088: 0x28, # CONTROL - 0x0089: 0x29, # CONTROL - 0x008A: 0x2A, # CONTROL - 0x008B: 0x2B, # CONTROL - 0x008C: 0x2C, # CONTROL - 0x008D: 0x09, # CONTROL - 0x008E: 0x0A, # CONTROL - 0x008F: 0x1B, # CONTROL - 0x0090: 0x30, # CONTROL - 0x0091: 0x31, # CONTROL - 0x0092: 0x1A, # CONTROL - 0x0093: 0x33, # CONTROL - 0x0094: 0x34, # CONTROL - 0x0095: 0x35, # CONTROL - 0x0096: 0x36, # CONTROL - 0x0097: 0x08, # CONTROL - 0x0098: 0x38, # CONTROL - 0x0099: 0x39, # CONTROL - 0x009A: 0x3A, # CONTROL - 0x009B: 0x3B, # CONTROL - 0x009C: 0x04, # CONTROL - 0x009D: 0x14, # CONTROL - 0x009E: 0x3E, # CONTROL - 0x009F: 0xFF, # CONTROL - 0x00A0: 0x74, # NO-BREAK SPACE - 0x00A3: 0xB0, # POUND SIGN - 0x00A6: 0xDF, # BROKEN BAR - 0x00A7: 0xEB, # SECTION SIGN - 0x00A8: 0x70, # DIAERESIS - 0x00A9: 0xFB, # COPYRIGHT SIGN - 0x00AB: 0xEE, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xEF, # NOT SIGN - 0x00AD: 0xCA, # SOFT HYPHEN - 0x00B0: 0x90, # DEGREE SIGN - 0x00B1: 0xDA, # PLUS-MINUS SIGN - 0x00B2: 0xEA, # SUPERSCRIPT TWO - 0x00B3: 0xFA, # SUPERSCRIPT THREE - 0x00B4: 0xA0, # ACUTE ACCENT - 0x00BB: 0xFE, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BD: 0xDB, # VULGAR FRACTION ONE HALF - 0x0385: 0x80, # GREEK DIALYTIKA TONOS - 0x0386: 0x71, # GREEK CAPITAL LETTER ALPHA WITH TONOS - 0x0387: 0xDD, # GREEK ANO TELEIA - 0x0388: 0x72, # GREEK CAPITAL LETTER EPSILON WITH TONOS - 0x0389: 0x73, # GREEK CAPITAL LETTER ETA WITH TONOS - 0x038A: 0x75, # GREEK CAPITAL LETTER IOTA WITH TONOS - 0x038C: 0x76, # GREEK CAPITAL LETTER OMICRON WITH TONOS - 0x038E: 0x77, # GREEK CAPITAL LETTER UPSILON WITH TONOS - 0x038F: 0x78, # GREEK CAPITAL LETTER OMEGA WITH TONOS - 0x0390: 0xCC, # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND TONOS - 0x0391: 0x41, # GREEK CAPITAL LETTER ALPHA - 0x0392: 0x42, # GREEK CAPITAL LETTER BETA - 0x0393: 0x43, # GREEK CAPITAL LETTER GAMMA - 0x0394: 0x44, # GREEK CAPITAL LETTER DELTA - 0x0395: 0x45, # GREEK CAPITAL LETTER EPSILON - 0x0396: 0x46, # GREEK CAPITAL LETTER ZETA - 0x0397: 0x47, # GREEK CAPITAL LETTER ETA - 0x0398: 0x48, # GREEK CAPITAL LETTER THETA - 0x0399: 0x49, # GREEK CAPITAL LETTER IOTA - 0x039A: 0x51, # GREEK CAPITAL LETTER KAPPA - 0x039B: 0x52, # GREEK CAPITAL LETTER LAMDA - 0x039C: 0x53, # GREEK CAPITAL LETTER MU - 0x039D: 0x54, # GREEK CAPITAL LETTER NU - 0x039E: 0x55, # GREEK CAPITAL LETTER XI - 0x039F: 0x56, # GREEK CAPITAL LETTER OMICRON - 0x03A0: 0x57, # GREEK CAPITAL LETTER PI - 0x03A1: 0x58, # GREEK CAPITAL LETTER RHO - 0x03A3: 0x59, # GREEK CAPITAL LETTER SIGMA - 0x03A4: 0x62, # GREEK CAPITAL LETTER TAU - 0x03A5: 0x63, # GREEK CAPITAL LETTER UPSILON - 0x03A6: 0x64, # GREEK CAPITAL LETTER PHI - 0x03A7: 0x65, # GREEK CAPITAL LETTER CHI - 0x03A8: 0x66, # GREEK CAPITAL LETTER PSI - 0x03A9: 0x67, # GREEK CAPITAL LETTER OMEGA - 0x03AA: 0x68, # GREEK CAPITAL LETTER IOTA WITH DIALYTIKA - 0x03AB: 0x69, # GREEK CAPITAL LETTER UPSILON WITH DIALYTIKA - 0x03AC: 0xB1, # GREEK SMALL LETTER ALPHA WITH TONOS - 0x03AD: 0xB2, # GREEK SMALL LETTER EPSILON WITH TONOS - 0x03AE: 0xB3, # GREEK SMALL LETTER ETA WITH TONOS - 0x03AF: 0xB5, # GREEK SMALL LETTER IOTA WITH TONOS - 0x03B0: 0xCD, # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND TONOS - 0x03B1: 0x8A, # GREEK SMALL LETTER ALPHA - 0x03B2: 0x8B, # GREEK SMALL LETTER BETA - 0x03B3: 0x8C, # GREEK SMALL LETTER GAMMA - 0x03B4: 0x8D, # GREEK SMALL LETTER DELTA - 0x03B5: 0x8E, # GREEK SMALL LETTER EPSILON - 0x03B6: 0x8F, # GREEK SMALL LETTER ZETA - 0x03B7: 0x9A, # GREEK SMALL LETTER ETA - 0x03B8: 0x9B, # GREEK SMALL LETTER THETA - 0x03B9: 0x9C, # GREEK SMALL LETTER IOTA - 0x03BA: 0x9D, # GREEK SMALL LETTER KAPPA - 0x03BB: 0x9E, # GREEK SMALL LETTER LAMDA - 0x03BC: 0x9F, # GREEK SMALL LETTER MU - 0x03BD: 0xAA, # GREEK SMALL LETTER NU - 0x03BE: 0xAB, # GREEK SMALL LETTER XI - 0x03BF: 0xAC, # GREEK SMALL LETTER OMICRON - 0x03C0: 0xAD, # GREEK SMALL LETTER PI - 0x03C1: 0xAE, # GREEK SMALL LETTER RHO - 0x03C2: 0xBA, # GREEK SMALL LETTER FINAL SIGMA - 0x03C3: 0xAF, # GREEK SMALL LETTER SIGMA - 0x03C4: 0xBB, # GREEK SMALL LETTER TAU - 0x03C5: 0xBC, # GREEK SMALL LETTER UPSILON - 0x03C6: 0xBD, # GREEK SMALL LETTER PHI - 0x03C7: 0xBE, # GREEK SMALL LETTER CHI - 0x03C8: 0xBF, # GREEK SMALL LETTER PSI - 0x03C9: 0xCB, # GREEK SMALL LETTER OMEGA - 0x03CA: 0xB4, # GREEK SMALL LETTER IOTA WITH DIALYTIKA - 0x03CB: 0xB8, # GREEK SMALL LETTER UPSILON WITH DIALYTIKA - 0x03CC: 0xB6, # GREEK SMALL LETTER OMICRON WITH TONOS - 0x03CD: 0xB7, # GREEK SMALL LETTER UPSILON WITH TONOS - 0x03CE: 0xB9, # GREEK SMALL LETTER OMEGA WITH TONOS - 0x2015: 0xCF, # HORIZONTAL BAR - 0x2018: 0xCE, # LEFT SINGLE QUOTATION MARK - 0x2019: 0xDE, # RIGHT SINGLE QUOTATION MARK -} Modified: python/trunk/Lib/encodings/iso8859_1.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_1.py (original) +++ python/trunk/Lib/encodings/iso8859_1.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\xff' # 0xFF -> LATIN SMALL LETTER Y WITH DIAERESIS ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A1: 0xA1, # INVERTED EXCLAMATION MARK - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AA: 0xAA, # FEMININE ORDINAL INDICATOR - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0xAF, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B8: 0xB8, # CEDILLA - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BA: 0xBA, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00BF: 0xBF, # INVERTED QUESTION MARK - 0x00C0: 0xC0, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0xC3, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0xC8, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0xCA, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0xCC, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D0: 0xD0, # LATIN CAPITAL LETTER ETH (Icelandic) - 0x00D1: 0xD1, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xD2, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00D8: 0xD8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xD9, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xDD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DE: 0xDE, # LATIN CAPITAL LETTER THORN (Icelandic) - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S (German) - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0xE3, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0xEC, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F0: 0xF0, # LATIN SMALL LETTER ETH (Icelandic) - 0x00F1: 0xF1, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xF2, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F8: 0xF8, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0xFD, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FE: 0xFE, # LATIN SMALL LETTER THORN (Icelandic) - 0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS -} Modified: python/trunk/Lib/encodings/iso8859_10.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_10.py (original) +++ python/trunk/Lib/encodings/iso8859_10.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\u0138' # 0xFF -> LATIN SMALL LETTER KRA ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A7: 0xA7, # SECTION SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0xC3, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D0: 0xD0, # LATIN CAPITAL LETTER ETH (Icelandic) - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D8: 0xD8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xDD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DE: 0xDE, # LATIN CAPITAL LETTER THORN (Icelandic) - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S (German) - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0xE3, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F0: 0xF0, # LATIN SMALL LETTER ETH (Icelandic) - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F8: 0xF8, # LATIN SMALL LETTER O WITH STROKE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0xFD, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FE: 0xFE, # LATIN SMALL LETTER THORN (Icelandic) - 0x0100: 0xC0, # LATIN CAPITAL LETTER A WITH MACRON - 0x0101: 0xE0, # LATIN SMALL LETTER A WITH MACRON - 0x0104: 0xA1, # LATIN CAPITAL LETTER A WITH OGONEK - 0x0105: 0xB1, # LATIN SMALL LETTER A WITH OGONEK - 0x010C: 0xC8, # LATIN CAPITAL LETTER C WITH CARON - 0x010D: 0xE8, # LATIN SMALL LETTER C WITH CARON - 0x0110: 0xA9, # LATIN CAPITAL LETTER D WITH STROKE - 0x0111: 0xB9, # LATIN SMALL LETTER D WITH STROKE - 0x0112: 0xA2, # LATIN CAPITAL LETTER E WITH MACRON - 0x0113: 0xB2, # LATIN SMALL LETTER E WITH MACRON - 0x0116: 0xCC, # LATIN CAPITAL LETTER E WITH DOT ABOVE - 0x0117: 0xEC, # LATIN SMALL LETTER E WITH DOT ABOVE - 0x0118: 0xCA, # LATIN CAPITAL LETTER E WITH OGONEK - 0x0119: 0xEA, # LATIN SMALL LETTER E WITH OGONEK - 0x0122: 0xA3, # LATIN CAPITAL LETTER G WITH CEDILLA - 0x0123: 0xB3, # LATIN SMALL LETTER G WITH CEDILLA - 0x0128: 0xA5, # LATIN CAPITAL LETTER I WITH TILDE - 0x0129: 0xB5, # LATIN SMALL LETTER I WITH TILDE - 0x012A: 0xA4, # LATIN CAPITAL LETTER I WITH MACRON - 0x012B: 0xB4, # LATIN SMALL LETTER I WITH MACRON - 0x012E: 0xC7, # LATIN CAPITAL LETTER I WITH OGONEK - 0x012F: 0xE7, # LATIN SMALL LETTER I WITH OGONEK - 0x0136: 0xA6, # LATIN CAPITAL LETTER K WITH CEDILLA - 0x0137: 0xB6, # LATIN SMALL LETTER K WITH CEDILLA - 0x0138: 0xFF, # LATIN SMALL LETTER KRA - 0x013B: 0xA8, # LATIN CAPITAL LETTER L WITH CEDILLA - 0x013C: 0xB8, # LATIN SMALL LETTER L WITH CEDILLA - 0x0145: 0xD1, # LATIN CAPITAL LETTER N WITH CEDILLA - 0x0146: 0xF1, # LATIN SMALL LETTER N WITH CEDILLA - 0x014A: 0xAF, # LATIN CAPITAL LETTER ENG - 0x014B: 0xBF, # LATIN SMALL LETTER ENG - 0x014C: 0xD2, # LATIN CAPITAL LETTER O WITH MACRON - 0x014D: 0xF2, # LATIN SMALL LETTER O WITH MACRON - 0x0160: 0xAA, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0xBA, # LATIN SMALL LETTER S WITH CARON - 0x0166: 0xAB, # LATIN CAPITAL LETTER T WITH STROKE - 0x0167: 0xBB, # LATIN SMALL LETTER T WITH STROKE - 0x0168: 0xD7, # LATIN CAPITAL LETTER U WITH TILDE - 0x0169: 0xF7, # LATIN SMALL LETTER U WITH TILDE - 0x016A: 0xAE, # LATIN CAPITAL LETTER U WITH MACRON - 0x016B: 0xBE, # LATIN SMALL LETTER U WITH MACRON - 0x0172: 0xD9, # LATIN CAPITAL LETTER U WITH OGONEK - 0x0173: 0xF9, # LATIN SMALL LETTER U WITH OGONEK - 0x017D: 0xAC, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0xBC, # LATIN SMALL LETTER Z WITH CARON - 0x2015: 0xBD, # HORIZONTAL BAR -} Modified: python/trunk/Lib/encodings/iso8859_11.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_11.py (original) +++ python/trunk/Lib/encodings/iso8859_11.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,255 +303,6 @@ u'\ufffe' ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x0E01: 0xA1, # THAI CHARACTER KO KAI - 0x0E02: 0xA2, # THAI CHARACTER KHO KHAI - 0x0E03: 0xA3, # THAI CHARACTER KHO KHUAT - 0x0E04: 0xA4, # THAI CHARACTER KHO KHWAI - 0x0E05: 0xA5, # THAI CHARACTER KHO KHON - 0x0E06: 0xA6, # THAI CHARACTER KHO RAKHANG - 0x0E07: 0xA7, # THAI CHARACTER NGO NGU - 0x0E08: 0xA8, # THAI CHARACTER CHO CHAN - 0x0E09: 0xA9, # THAI CHARACTER CHO CHING - 0x0E0A: 0xAA, # THAI CHARACTER CHO CHANG - 0x0E0B: 0xAB, # THAI CHARACTER SO SO - 0x0E0C: 0xAC, # THAI CHARACTER CHO CHOE - 0x0E0D: 0xAD, # THAI CHARACTER YO YING - 0x0E0E: 0xAE, # THAI CHARACTER DO CHADA - 0x0E0F: 0xAF, # THAI CHARACTER TO PATAK - 0x0E10: 0xB0, # THAI CHARACTER THO THAN - 0x0E11: 0xB1, # THAI CHARACTER THO NANGMONTHO - 0x0E12: 0xB2, # THAI CHARACTER THO PHUTHAO - 0x0E13: 0xB3, # THAI CHARACTER NO NEN - 0x0E14: 0xB4, # THAI CHARACTER DO DEK - 0x0E15: 0xB5, # THAI CHARACTER TO TAO - 0x0E16: 0xB6, # THAI CHARACTER THO THUNG - 0x0E17: 0xB7, # THAI CHARACTER THO THAHAN - 0x0E18: 0xB8, # THAI CHARACTER THO THONG - 0x0E19: 0xB9, # THAI CHARACTER NO NU - 0x0E1A: 0xBA, # THAI CHARACTER BO BAIMAI - 0x0E1B: 0xBB, # THAI CHARACTER PO PLA - 0x0E1C: 0xBC, # THAI CHARACTER PHO PHUNG - 0x0E1D: 0xBD, # THAI CHARACTER FO FA - 0x0E1E: 0xBE, # THAI CHARACTER PHO PHAN - 0x0E1F: 0xBF, # THAI CHARACTER FO FAN - 0x0E20: 0xC0, # THAI CHARACTER PHO SAMPHAO - 0x0E21: 0xC1, # THAI CHARACTER MO MA - 0x0E22: 0xC2, # THAI CHARACTER YO YAK - 0x0E23: 0xC3, # THAI CHARACTER RO RUA - 0x0E24: 0xC4, # THAI CHARACTER RU - 0x0E25: 0xC5, # THAI CHARACTER LO LING - 0x0E26: 0xC6, # THAI CHARACTER LU - 0x0E27: 0xC7, # THAI CHARACTER WO WAEN - 0x0E28: 0xC8, # THAI CHARACTER SO SALA - 0x0E29: 0xC9, # THAI CHARACTER SO RUSI - 0x0E2A: 0xCA, # THAI CHARACTER SO SUA - 0x0E2B: 0xCB, # THAI CHARACTER HO HIP - 0x0E2C: 0xCC, # THAI CHARACTER LO CHULA - 0x0E2D: 0xCD, # THAI CHARACTER O ANG - 0x0E2E: 0xCE, # THAI CHARACTER HO NOKHUK - 0x0E2F: 0xCF, # THAI CHARACTER PAIYANNOI - 0x0E30: 0xD0, # THAI CHARACTER SARA A - 0x0E31: 0xD1, # THAI CHARACTER MAI HAN-AKAT - 0x0E32: 0xD2, # THAI CHARACTER SARA AA - 0x0E33: 0xD3, # THAI CHARACTER SARA AM - 0x0E34: 0xD4, # THAI CHARACTER SARA I - 0x0E35: 0xD5, # THAI CHARACTER SARA II - 0x0E36: 0xD6, # THAI CHARACTER SARA UE - 0x0E37: 0xD7, # THAI CHARACTER SARA UEE - 0x0E38: 0xD8, # THAI CHARACTER SARA U - 0x0E39: 0xD9, # THAI CHARACTER SARA UU - 0x0E3A: 0xDA, # THAI CHARACTER PHINTHU - 0x0E3F: 0xDF, # THAI CURRENCY SYMBOL BAHT - 0x0E40: 0xE0, # THAI CHARACTER SARA E - 0x0E41: 0xE1, # THAI CHARACTER SARA AE - 0x0E42: 0xE2, # THAI CHARACTER SARA O - 0x0E43: 0xE3, # THAI CHARACTER SARA AI MAIMUAN - 0x0E44: 0xE4, # THAI CHARACTER SARA AI MAIMALAI - 0x0E45: 0xE5, # THAI CHARACTER LAKKHANGYAO - 0x0E46: 0xE6, # THAI CHARACTER MAIYAMOK - 0x0E47: 0xE7, # THAI CHARACTER MAITAIKHU - 0x0E48: 0xE8, # THAI CHARACTER MAI EK - 0x0E49: 0xE9, # THAI CHARACTER MAI THO - 0x0E4A: 0xEA, # THAI CHARACTER MAI TRI - 0x0E4B: 0xEB, # THAI CHARACTER MAI CHATTAWA - 0x0E4C: 0xEC, # THAI CHARACTER THANTHAKHAT - 0x0E4D: 0xED, # THAI CHARACTER NIKHAHIT - 0x0E4E: 0xEE, # THAI CHARACTER YAMAKKAN - 0x0E4F: 0xEF, # THAI CHARACTER FONGMAN - 0x0E50: 0xF0, # THAI DIGIT ZERO - 0x0E51: 0xF1, # THAI DIGIT ONE - 0x0E52: 0xF2, # THAI DIGIT TWO - 0x0E53: 0xF3, # THAI DIGIT THREE - 0x0E54: 0xF4, # THAI DIGIT FOUR - 0x0E55: 0xF5, # THAI DIGIT FIVE - 0x0E56: 0xF6, # THAI DIGIT SIX - 0x0E57: 0xF7, # THAI DIGIT SEVEN - 0x0E58: 0xF8, # THAI DIGIT EIGHT - 0x0E59: 0xF9, # THAI DIGIT NINE - 0x0E5A: 0xFA, # THAI CHARACTER ANGKHANKHU - 0x0E5B: 0xFB, # THAI CHARACTER KHOMUT -} Modified: python/trunk/Lib/encodings/iso8859_13.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_13.py (original) +++ python/trunk/Lib/encodings/iso8859_13.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\u2019' # 0xFF -> RIGHT SINGLE QUOTATION MARK ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A6: 0xA6, # BROKEN BAR - 0x00A7: 0xA7, # SECTION SIGN - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BC: 0xBC, # VULGAR FRACTION ONE QUARTER - 0x00BD: 0xBD, # VULGAR FRACTION ONE HALF - 0x00BE: 0xBE, # VULGAR FRACTION THREE QUARTERS - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xAF, # LATIN CAPITAL LETTER AE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00D8: 0xA8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S (German) - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xBF, # LATIN SMALL LETTER AE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F8: 0xB8, # LATIN SMALL LETTER O WITH STROKE - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x0100: 0xC2, # LATIN CAPITAL LETTER A WITH MACRON - 0x0101: 0xE2, # LATIN SMALL LETTER A WITH MACRON - 0x0104: 0xC0, # LATIN CAPITAL LETTER A WITH OGONEK - 0x0105: 0xE0, # LATIN SMALL LETTER A WITH OGONEK - 0x0106: 0xC3, # LATIN CAPITAL LETTER C WITH ACUTE - 0x0107: 0xE3, # LATIN SMALL LETTER C WITH ACUTE - 0x010C: 0xC8, # LATIN CAPITAL LETTER C WITH CARON - 0x010D: 0xE8, # LATIN SMALL LETTER C WITH CARON - 0x0112: 0xC7, # LATIN CAPITAL LETTER E WITH MACRON - 0x0113: 0xE7, # LATIN SMALL LETTER E WITH MACRON - 0x0116: 0xCB, # LATIN CAPITAL LETTER E WITH DOT ABOVE - 0x0117: 0xEB, # LATIN SMALL LETTER E WITH DOT ABOVE - 0x0118: 0xC6, # LATIN CAPITAL LETTER E WITH OGONEK - 0x0119: 0xE6, # LATIN SMALL LETTER E WITH OGONEK - 0x0122: 0xCC, # LATIN CAPITAL LETTER G WITH CEDILLA - 0x0123: 0xEC, # LATIN SMALL LETTER G WITH CEDILLA - 0x012A: 0xCE, # LATIN CAPITAL LETTER I WITH MACRON - 0x012B: 0xEE, # LATIN SMALL LETTER I WITH MACRON - 0x012E: 0xC1, # LATIN CAPITAL LETTER I WITH OGONEK - 0x012F: 0xE1, # LATIN SMALL LETTER I WITH OGONEK - 0x0136: 0xCD, # LATIN CAPITAL LETTER K WITH CEDILLA - 0x0137: 0xED, # LATIN SMALL LETTER K WITH CEDILLA - 0x013B: 0xCF, # LATIN CAPITAL LETTER L WITH CEDILLA - 0x013C: 0xEF, # LATIN SMALL LETTER L WITH CEDILLA - 0x0141: 0xD9, # LATIN CAPITAL LETTER L WITH STROKE - 0x0142: 0xF9, # LATIN SMALL LETTER L WITH STROKE - 0x0143: 0xD1, # LATIN CAPITAL LETTER N WITH ACUTE - 0x0144: 0xF1, # LATIN SMALL LETTER N WITH ACUTE - 0x0145: 0xD2, # LATIN CAPITAL LETTER N WITH CEDILLA - 0x0146: 0xF2, # LATIN SMALL LETTER N WITH CEDILLA - 0x014C: 0xD4, # LATIN CAPITAL LETTER O WITH MACRON - 0x014D: 0xF4, # LATIN SMALL LETTER O WITH MACRON - 0x0156: 0xAA, # LATIN CAPITAL LETTER R WITH CEDILLA - 0x0157: 0xBA, # LATIN SMALL LETTER R WITH CEDILLA - 0x015A: 0xDA, # LATIN CAPITAL LETTER S WITH ACUTE - 0x015B: 0xFA, # LATIN SMALL LETTER S WITH ACUTE - 0x0160: 0xD0, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0xF0, # LATIN SMALL LETTER S WITH CARON - 0x016A: 0xDB, # LATIN CAPITAL LETTER U WITH MACRON - 0x016B: 0xFB, # LATIN SMALL LETTER U WITH MACRON - 0x0172: 0xD8, # LATIN CAPITAL LETTER U WITH OGONEK - 0x0173: 0xF8, # LATIN SMALL LETTER U WITH OGONEK - 0x0179: 0xCA, # LATIN CAPITAL LETTER Z WITH ACUTE - 0x017A: 0xEA, # LATIN SMALL LETTER Z WITH ACUTE - 0x017B: 0xDD, # LATIN CAPITAL LETTER Z WITH DOT ABOVE - 0x017C: 0xFD, # LATIN SMALL LETTER Z WITH DOT ABOVE - 0x017D: 0xDE, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0xFE, # LATIN SMALL LETTER Z WITH CARON - 0x2019: 0xFF, # RIGHT SINGLE QUOTATION MARK - 0x201C: 0xB4, # LEFT DOUBLE QUOTATION MARK - 0x201D: 0xA1, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0xA5, # DOUBLE LOW-9 QUOTATION MARK -} Modified: python/trunk/Lib/encodings/iso8859_14.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_14.py (original) +++ python/trunk/Lib/encodings/iso8859_14.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\xff' # 0xFF -> LATIN SMALL LETTER Y WITH DIAERESIS ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A3: 0xA3, # POUND SIGN - 0x00A7: 0xA7, # SECTION SIGN - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00C0: 0xC0, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0xC3, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0xC8, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0xCA, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0xCC, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D1: 0xD1, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xD2, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D8: 0xD8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xD9, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xDD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0xE3, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0xEC, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F1: 0xF1, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xF2, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F8: 0xF8, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0xFD, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x010A: 0xA4, # LATIN CAPITAL LETTER C WITH DOT ABOVE - 0x010B: 0xA5, # LATIN SMALL LETTER C WITH DOT ABOVE - 0x0120: 0xB2, # LATIN CAPITAL LETTER G WITH DOT ABOVE - 0x0121: 0xB3, # LATIN SMALL LETTER G WITH DOT ABOVE - 0x0174: 0xD0, # LATIN CAPITAL LETTER W WITH CIRCUMFLEX - 0x0175: 0xF0, # LATIN SMALL LETTER W WITH CIRCUMFLEX - 0x0176: 0xDE, # LATIN CAPITAL LETTER Y WITH CIRCUMFLEX - 0x0177: 0xFE, # LATIN SMALL LETTER Y WITH CIRCUMFLEX - 0x0178: 0xAF, # LATIN CAPITAL LETTER Y WITH DIAERESIS - 0x1E02: 0xA1, # LATIN CAPITAL LETTER B WITH DOT ABOVE - 0x1E03: 0xA2, # LATIN SMALL LETTER B WITH DOT ABOVE - 0x1E0A: 0xA6, # LATIN CAPITAL LETTER D WITH DOT ABOVE - 0x1E0B: 0xAB, # LATIN SMALL LETTER D WITH DOT ABOVE - 0x1E1E: 0xB0, # LATIN CAPITAL LETTER F WITH DOT ABOVE - 0x1E1F: 0xB1, # LATIN SMALL LETTER F WITH DOT ABOVE - 0x1E40: 0xB4, # LATIN CAPITAL LETTER M WITH DOT ABOVE - 0x1E41: 0xB5, # LATIN SMALL LETTER M WITH DOT ABOVE - 0x1E56: 0xB7, # LATIN CAPITAL LETTER P WITH DOT ABOVE - 0x1E57: 0xB9, # LATIN SMALL LETTER P WITH DOT ABOVE - 0x1E60: 0xBB, # LATIN CAPITAL LETTER S WITH DOT ABOVE - 0x1E61: 0xBF, # LATIN SMALL LETTER S WITH DOT ABOVE - 0x1E6A: 0xD7, # LATIN CAPITAL LETTER T WITH DOT ABOVE - 0x1E6B: 0xF7, # LATIN SMALL LETTER T WITH DOT ABOVE - 0x1E80: 0xA8, # LATIN CAPITAL LETTER W WITH GRAVE - 0x1E81: 0xB8, # LATIN SMALL LETTER W WITH GRAVE - 0x1E82: 0xAA, # LATIN CAPITAL LETTER W WITH ACUTE - 0x1E83: 0xBA, # LATIN SMALL LETTER W WITH ACUTE - 0x1E84: 0xBD, # LATIN CAPITAL LETTER W WITH DIAERESIS - 0x1E85: 0xBE, # LATIN SMALL LETTER W WITH DIAERESIS - 0x1EF2: 0xAC, # LATIN CAPITAL LETTER Y WITH GRAVE - 0x1EF3: 0xBC, # LATIN SMALL LETTER Y WITH GRAVE -} Modified: python/trunk/Lib/encodings/iso8859_15.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_15.py (original) +++ python/trunk/Lib/encodings/iso8859_15.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\xff' # 0xFF -> LATIN SMALL LETTER Y WITH DIAERESIS ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A1: 0xA1, # INVERTED EXCLAMATION MARK - 0x00A2: 0xA2, # CENT SIGN - 0x00A3: 0xA3, # POUND SIGN - 0x00A5: 0xA5, # YEN SIGN - 0x00A7: 0xA7, # SECTION SIGN - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AA: 0xAA, # FEMININE ORDINAL INDICATOR - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AC: 0xAC, # NOT SIGN - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00AE: 0xAE, # REGISTERED SIGN - 0x00AF: 0xAF, # MACRON - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B2: 0xB2, # SUPERSCRIPT TWO - 0x00B3: 0xB3, # SUPERSCRIPT THREE - 0x00B5: 0xB5, # MICRO SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00B9: 0xB9, # SUPERSCRIPT ONE - 0x00BA: 0xBA, # MASCULINE ORDINAL INDICATOR - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00BF: 0xBF, # INVERTED QUESTION MARK - 0x00C0: 0xC0, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C3: 0xC3, # LATIN CAPITAL LETTER A WITH TILDE - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C5: 0xC5, # LATIN CAPITAL LETTER A WITH RING ABOVE - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0xC8, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0xCA, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0xCC, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D0: 0xD0, # LATIN CAPITAL LETTER ETH - 0x00D1: 0xD1, # LATIN CAPITAL LETTER N WITH TILDE - 0x00D2: 0xD2, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D5: 0xD5, # LATIN CAPITAL LETTER O WITH TILDE - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00D8: 0xD8, # LATIN CAPITAL LETTER O WITH STROKE - 0x00D9: 0xD9, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xDD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DE: 0xDE, # LATIN CAPITAL LETTER THORN - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E3: 0xE3, # LATIN SMALL LETTER A WITH TILDE - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E5: 0xE5, # LATIN SMALL LETTER A WITH RING ABOVE - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0xEC, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F0: 0xF0, # LATIN SMALL LETTER ETH - 0x00F1: 0xF1, # LATIN SMALL LETTER N WITH TILDE - 0x00F2: 0xF2, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F5: 0xF5, # LATIN SMALL LETTER O WITH TILDE - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00F8: 0xF8, # LATIN SMALL LETTER O WITH STROKE - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0xFD, # LATIN SMALL LETTER Y WITH ACUTE - 0x00FE: 0xFE, # LATIN SMALL LETTER THORN - 0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x0152: 0xBC, # LATIN CAPITAL LIGATURE OE - 0x0153: 0xBD, # LATIN SMALL LIGATURE OE - 0x0160: 0xA6, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0xA8, # LATIN SMALL LETTER S WITH CARON - 0x0178: 0xBE, # LATIN CAPITAL LETTER Y WITH DIAERESIS - 0x017D: 0xB4, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0xB8, # LATIN SMALL LETTER Z WITH CARON - 0x20AC: 0xA4, # EURO SIGN -} Modified: python/trunk/Lib/encodings/iso8859_16.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_16.py (original) +++ python/trunk/Lib/encodings/iso8859_16.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\xff' # 0xFF -> LATIN SMALL LETTER Y WITH DIAERESIS ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A7: 0xA7, # SECTION SIGN - 0x00A9: 0xA9, # COPYRIGHT SIGN - 0x00AB: 0xAB, # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B1: 0xB1, # PLUS-MINUS SIGN - 0x00B6: 0xB6, # PILCROW SIGN - 0x00B7: 0xB7, # MIDDLE DOT - 0x00BB: 0xBB, # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK - 0x00C0: 0xC0, # LATIN CAPITAL LETTER A WITH GRAVE - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C6: 0xC6, # LATIN CAPITAL LETTER AE - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C8: 0xC8, # LATIN CAPITAL LETTER E WITH GRAVE - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CA: 0xCA, # LATIN CAPITAL LETTER E WITH CIRCUMFLEX - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CC: 0xCC, # LATIN CAPITAL LETTER I WITH GRAVE - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00CF: 0xCF, # LATIN CAPITAL LETTER I WITH DIAERESIS - 0x00D2: 0xD2, # LATIN CAPITAL LETTER O WITH GRAVE - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D9: 0xD9, # LATIN CAPITAL LETTER U WITH GRAVE - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DB: 0xDB, # LATIN CAPITAL LETTER U WITH CIRCUMFLEX - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E0: 0xE0, # LATIN SMALL LETTER A WITH GRAVE - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E6: 0xE6, # LATIN SMALL LETTER AE - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E8: 0xE8, # LATIN SMALL LETTER E WITH GRAVE - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EA: 0xEA, # LATIN SMALL LETTER E WITH CIRCUMFLEX - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00EC: 0xEC, # LATIN SMALL LETTER I WITH GRAVE - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00EF: 0xEF, # LATIN SMALL LETTER I WITH DIAERESIS - 0x00F2: 0xF2, # LATIN SMALL LETTER O WITH GRAVE - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F9: 0xF9, # LATIN SMALL LETTER U WITH GRAVE - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FB: 0xFB, # LATIN SMALL LETTER U WITH CIRCUMFLEX - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS - 0x0102: 0xC3, # LATIN CAPITAL LETTER A WITH BREVE - 0x0103: 0xE3, # LATIN SMALL LETTER A WITH BREVE - 0x0104: 0xA1, # LATIN CAPITAL LETTER A WITH OGONEK - 0x0105: 0xA2, # LATIN SMALL LETTER A WITH OGONEK - 0x0106: 0xC5, # LATIN CAPITAL LETTER C WITH ACUTE - 0x0107: 0xE5, # LATIN SMALL LETTER C WITH ACUTE - 0x010C: 0xB2, # LATIN CAPITAL LETTER C WITH CARON - 0x010D: 0xB9, # LATIN SMALL LETTER C WITH CARON - 0x0110: 0xD0, # LATIN CAPITAL LETTER D WITH STROKE - 0x0111: 0xF0, # LATIN SMALL LETTER D WITH STROKE - 0x0118: 0xDD, # LATIN CAPITAL LETTER E WITH OGONEK - 0x0119: 0xFD, # LATIN SMALL LETTER E WITH OGONEK - 0x0141: 0xA3, # LATIN CAPITAL LETTER L WITH STROKE - 0x0142: 0xB3, # LATIN SMALL LETTER L WITH STROKE - 0x0143: 0xD1, # LATIN CAPITAL LETTER N WITH ACUTE - 0x0144: 0xF1, # LATIN SMALL LETTER N WITH ACUTE - 0x0150: 0xD5, # LATIN CAPITAL LETTER O WITH DOUBLE ACUTE - 0x0151: 0xF5, # LATIN SMALL LETTER O WITH DOUBLE ACUTE - 0x0152: 0xBC, # LATIN CAPITAL LIGATURE OE - 0x0153: 0xBD, # LATIN SMALL LIGATURE OE - 0x015A: 0xD7, # LATIN CAPITAL LETTER S WITH ACUTE - 0x015B: 0xF7, # LATIN SMALL LETTER S WITH ACUTE - 0x0160: 0xA6, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0xA8, # LATIN SMALL LETTER S WITH CARON - 0x0170: 0xD8, # LATIN CAPITAL LETTER U WITH DOUBLE ACUTE - 0x0171: 0xF8, # LATIN SMALL LETTER U WITH DOUBLE ACUTE - 0x0178: 0xBE, # LATIN CAPITAL LETTER Y WITH DIAERESIS - 0x0179: 0xAC, # LATIN CAPITAL LETTER Z WITH ACUTE - 0x017A: 0xAE, # LATIN SMALL LETTER Z WITH ACUTE - 0x017B: 0xAF, # LATIN CAPITAL LETTER Z WITH DOT ABOVE - 0x017C: 0xBF, # LATIN SMALL LETTER Z WITH DOT ABOVE - 0x017D: 0xB4, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0xB8, # LATIN SMALL LETTER Z WITH CARON - 0x0218: 0xAA, # LATIN CAPITAL LETTER S WITH COMMA BELOW - 0x0219: 0xBA, # LATIN SMALL LETTER S WITH COMMA BELOW - 0x021A: 0xDE, # LATIN CAPITAL LETTER T WITH COMMA BELOW - 0x021B: 0xFE, # LATIN SMALL LETTER T WITH COMMA BELOW - 0x201D: 0xB5, # RIGHT DOUBLE QUOTATION MARK - 0x201E: 0xA5, # DOUBLE LOW-9 QUOTATION MARK - 0x20AC: 0xA4, # EURO SIGN -} Modified: python/trunk/Lib/encodings/iso8859_2.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_2.py (original) +++ python/trunk/Lib/encodings/iso8859_2.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,263 +303,6 @@ u'\u02d9' # 0xFF -> DOT ABOVE ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, # - 0x0095: 0x95, # - 0x0096: 0x96, # - 0x0097: 0x97, # - 0x0098: 0x98, # - 0x0099: 0x99, # - 0x009A: 0x9A, # - 0x009B: 0x9B, # - 0x009C: 0x9C, # - 0x009D: 0x9D, # - 0x009E: 0x9E, # - 0x009F: 0x9F, # - 0x00A0: 0xA0, # NO-BREAK SPACE - 0x00A4: 0xA4, # CURRENCY SIGN - 0x00A7: 0xA7, # SECTION SIGN - 0x00A8: 0xA8, # DIAERESIS - 0x00AD: 0xAD, # SOFT HYPHEN - 0x00B0: 0xB0, # DEGREE SIGN - 0x00B4: 0xB4, # ACUTE ACCENT - 0x00B8: 0xB8, # CEDILLA - 0x00C1: 0xC1, # LATIN CAPITAL LETTER A WITH ACUTE - 0x00C2: 0xC2, # LATIN CAPITAL LETTER A WITH CIRCUMFLEX - 0x00C4: 0xC4, # LATIN CAPITAL LETTER A WITH DIAERESIS - 0x00C7: 0xC7, # LATIN CAPITAL LETTER C WITH CEDILLA - 0x00C9: 0xC9, # LATIN CAPITAL LETTER E WITH ACUTE - 0x00CB: 0xCB, # LATIN CAPITAL LETTER E WITH DIAERESIS - 0x00CD: 0xCD, # LATIN CAPITAL LETTER I WITH ACUTE - 0x00CE: 0xCE, # LATIN CAPITAL LETTER I WITH CIRCUMFLEX - 0x00D3: 0xD3, # LATIN CAPITAL LETTER O WITH ACUTE - 0x00D4: 0xD4, # LATIN CAPITAL LETTER O WITH CIRCUMFLEX - 0x00D6: 0xD6, # LATIN CAPITAL LETTER O WITH DIAERESIS - 0x00D7: 0xD7, # MULTIPLICATION SIGN - 0x00DA: 0xDA, # LATIN CAPITAL LETTER U WITH ACUTE - 0x00DC: 0xDC, # LATIN CAPITAL LETTER U WITH DIAERESIS - 0x00DD: 0xDD, # LATIN CAPITAL LETTER Y WITH ACUTE - 0x00DF: 0xDF, # LATIN SMALL LETTER SHARP S - 0x00E1: 0xE1, # LATIN SMALL LETTER A WITH ACUTE - 0x00E2: 0xE2, # LATIN SMALL LETTER A WITH CIRCUMFLEX - 0x00E4: 0xE4, # LATIN SMALL LETTER A WITH DIAERESIS - 0x00E7: 0xE7, # LATIN SMALL LETTER C WITH CEDILLA - 0x00E9: 0xE9, # LATIN SMALL LETTER E WITH ACUTE - 0x00EB: 0xEB, # LATIN SMALL LETTER E WITH DIAERESIS - 0x00ED: 0xED, # LATIN SMALL LETTER I WITH ACUTE - 0x00EE: 0xEE, # LATIN SMALL LETTER I WITH CIRCUMFLEX - 0x00F3: 0xF3, # LATIN SMALL LETTER O WITH ACUTE - 0x00F4: 0xF4, # LATIN SMALL LETTER O WITH CIRCUMFLEX - 0x00F6: 0xF6, # LATIN SMALL LETTER O WITH DIAERESIS - 0x00F7: 0xF7, # DIVISION SIGN - 0x00FA: 0xFA, # LATIN SMALL LETTER U WITH ACUTE - 0x00FC: 0xFC, # LATIN SMALL LETTER U WITH DIAERESIS - 0x00FD: 0xFD, # LATIN SMALL LETTER Y WITH ACUTE - 0x0102: 0xC3, # LATIN CAPITAL LETTER A WITH BREVE - 0x0103: 0xE3, # LATIN SMALL LETTER A WITH BREVE - 0x0104: 0xA1, # LATIN CAPITAL LETTER A WITH OGONEK - 0x0105: 0xB1, # LATIN SMALL LETTER A WITH OGONEK - 0x0106: 0xC6, # LATIN CAPITAL LETTER C WITH ACUTE - 0x0107: 0xE6, # LATIN SMALL LETTER C WITH ACUTE - 0x010C: 0xC8, # LATIN CAPITAL LETTER C WITH CARON - 0x010D: 0xE8, # LATIN SMALL LETTER C WITH CARON - 0x010E: 0xCF, # LATIN CAPITAL LETTER D WITH CARON - 0x010F: 0xEF, # LATIN SMALL LETTER D WITH CARON - 0x0110: 0xD0, # LATIN CAPITAL LETTER D WITH STROKE - 0x0111: 0xF0, # LATIN SMALL LETTER D WITH STROKE - 0x0118: 0xCA, # LATIN CAPITAL LETTER E WITH OGONEK - 0x0119: 0xEA, # LATIN SMALL LETTER E WITH OGONEK - 0x011A: 0xCC, # LATIN CAPITAL LETTER E WITH CARON - 0x011B: 0xEC, # LATIN SMALL LETTER E WITH CARON - 0x0139: 0xC5, # LATIN CAPITAL LETTER L WITH ACUTE - 0x013A: 0xE5, # LATIN SMALL LETTER L WITH ACUTE - 0x013D: 0xA5, # LATIN CAPITAL LETTER L WITH CARON - 0x013E: 0xB5, # LATIN SMALL LETTER L WITH CARON - 0x0141: 0xA3, # LATIN CAPITAL LETTER L WITH STROKE - 0x0142: 0xB3, # LATIN SMALL LETTER L WITH STROKE - 0x0143: 0xD1, # LATIN CAPITAL LETTER N WITH ACUTE - 0x0144: 0xF1, # LATIN SMALL LETTER N WITH ACUTE - 0x0147: 0xD2, # LATIN CAPITAL LETTER N WITH CARON - 0x0148: 0xF2, # LATIN SMALL LETTER N WITH CARON - 0x0150: 0xD5, # LATIN CAPITAL LETTER O WITH DOUBLE ACUTE - 0x0151: 0xF5, # LATIN SMALL LETTER O WITH DOUBLE ACUTE - 0x0154: 0xC0, # LATIN CAPITAL LETTER R WITH ACUTE - 0x0155: 0xE0, # LATIN SMALL LETTER R WITH ACUTE - 0x0158: 0xD8, # LATIN CAPITAL LETTER R WITH CARON - 0x0159: 0xF8, # LATIN SMALL LETTER R WITH CARON - 0x015A: 0xA6, # LATIN CAPITAL LETTER S WITH ACUTE - 0x015B: 0xB6, # LATIN SMALL LETTER S WITH ACUTE - 0x015E: 0xAA, # LATIN CAPITAL LETTER S WITH CEDILLA - 0x015F: 0xBA, # LATIN SMALL LETTER S WITH CEDILLA - 0x0160: 0xA9, # LATIN CAPITAL LETTER S WITH CARON - 0x0161: 0xB9, # LATIN SMALL LETTER S WITH CARON - 0x0162: 0xDE, # LATIN CAPITAL LETTER T WITH CEDILLA - 0x0163: 0xFE, # LATIN SMALL LETTER T WITH CEDILLA - 0x0164: 0xAB, # LATIN CAPITAL LETTER T WITH CARON - 0x0165: 0xBB, # LATIN SMALL LETTER T WITH CARON - 0x016E: 0xD9, # LATIN CAPITAL LETTER U WITH RING ABOVE - 0x016F: 0xF9, # LATIN SMALL LETTER U WITH RING ABOVE - 0x0170: 0xDB, # LATIN CAPITAL LETTER U WITH DOUBLE ACUTE - 0x0171: 0xFB, # LATIN SMALL LETTER U WITH DOUBLE ACUTE - 0x0179: 0xAC, # LATIN CAPITAL LETTER Z WITH ACUTE - 0x017A: 0xBC, # LATIN SMALL LETTER Z WITH ACUTE - 0x017B: 0xAF, # LATIN CAPITAL LETTER Z WITH DOT ABOVE - 0x017C: 0xBF, # LATIN SMALL LETTER Z WITH DOT ABOVE - 0x017D: 0xAE, # LATIN CAPITAL LETTER Z WITH CARON - 0x017E: 0xBE, # LATIN SMALL LETTER Z WITH CARON - 0x02C7: 0xB7, # CARON - 0x02D8: 0xA2, # BREVE - 0x02D9: 0xFF, # DOT ABOVE - 0x02DB: 0xB2, # OGONEK - 0x02DD: 0xBD, # DOUBLE ACUTE ACCENT -} Modified: python/trunk/Lib/encodings/iso8859_3.py ============================================================================== --- python/trunk/Lib/encodings/iso8859_3.py (original) +++ python/trunk/Lib/encodings/iso8859_3.py Sun Jun 4 21:36:28 2006 @@ -9,14 +9,14 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) + return codecs.charmap_encode(input,errors,encoding_table) def decode(self,input,errors='strict'): return codecs.charmap_decode(input,errors,decoding_table) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_table)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): @@ -303,256 +303,6 @@ u'\u02d9' # 0xFF -> DOT ABOVE ) -### Encoding Map +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) -encoding_map = { - 0x0000: 0x00, # NULL - 0x0001: 0x01, # START OF HEADING - 0x0002: 0x02, # START OF TEXT - 0x0003: 0x03, # END OF TEXT - 0x0004: 0x04, # END OF TRANSMISSION - 0x0005: 0x05, # ENQUIRY - 0x0006: 0x06, # ACKNOWLEDGE - 0x0007: 0x07, # BELL - 0x0008: 0x08, # BACKSPACE - 0x0009: 0x09, # HORIZONTAL TABULATION - 0x000A: 0x0A, # LINE FEED - 0x000B: 0x0B, # VERTICAL TABULATION - 0x000C: 0x0C, # FORM FEED - 0x000D: 0x0D, # CARRIAGE RETURN - 0x000E: 0x0E, # SHIFT OUT - 0x000F: 0x0F, # SHIFT IN - 0x0010: 0x10, # DATA LINK ESCAPE - 0x0011: 0x11, # DEVICE CONTROL ONE - 0x0012: 0x12, # DEVICE CONTROL TWO - 0x0013: 0x13, # DEVICE CONTROL THREE - 0x0014: 0x14, # DEVICE CONTROL FOUR - 0x0015: 0x15, # NEGATIVE ACKNOWLEDGE - 0x0016: 0x16, # SYNCHRONOUS IDLE - 0x0017: 0x17, # END OF TRANSMISSION BLOCK - 0x0018: 0x18, # CANCEL - 0x0019: 0x19, # END OF MEDIUM - 0x001A: 0x1A, # SUBSTITUTE - 0x001B: 0x1B, # ESCAPE - 0x001C: 0x1C, # FILE SEPARATOR - 0x001D: 0x1D, # GROUP SEPARATOR - 0x001E: 0x1E, # RECORD SEPARATOR - 0x001F: 0x1F, # UNIT SEPARATOR - 0x0020: 0x20, # SPACE - 0x0021: 0x21, # EXCLAMATION MARK - 0x0022: 0x22, # QUOTATION MARK - 0x0023: 0x23, # NUMBER SIGN - 0x0024: 0x24, # DOLLAR SIGN - 0x0025: 0x25, # PERCENT SIGN - 0x0026: 0x26, # AMPERSAND - 0x0027: 0x27, # APOSTROPHE - 0x0028: 0x28, # LEFT PARENTHESIS - 0x0029: 0x29, # RIGHT PARENTHESIS - 0x002A: 0x2A, # ASTERISK - 0x002B: 0x2B, # PLUS SIGN - 0x002C: 0x2C, # COMMA - 0x002D: 0x2D, # HYPHEN-MINUS - 0x002E: 0x2E, # FULL STOP - 0x002F: 0x2F, # SOLIDUS - 0x0030: 0x30, # DIGIT ZERO - 0x0031: 0x31, # DIGIT ONE - 0x0032: 0x32, # DIGIT TWO - 0x0033: 0x33, # DIGIT THREE - 0x0034: 0x34, # DIGIT FOUR - 0x0035: 0x35, # DIGIT FIVE - 0x0036: 0x36, # DIGIT SIX - 0x0037: 0x37, # DIGIT SEVEN - 0x0038: 0x38, # DIGIT EIGHT - 0x0039: 0x39, # DIGIT NINE - 0x003A: 0x3A, # COLON - 0x003B: 0x3B, # SEMICOLON - 0x003C: 0x3C, # LESS-THAN SIGN - 0x003D: 0x3D, # EQUALS SIGN - 0x003E: 0x3E, # GREATER-THAN SIGN - 0x003F: 0x3F, # QUESTION MARK - 0x0040: 0x40, # COMMERCIAL AT - 0x0041: 0x41, # LATIN CAPITAL LETTER A - 0x0042: 0x42, # LATIN CAPITAL LETTER B - 0x0043: 0x43, # LATIN CAPITAL LETTER C - 0x0044: 0x44, # LATIN CAPITAL LETTER D - 0x0045: 0x45, # LATIN CAPITAL LETTER E - 0x0046: 0x46, # LATIN CAPITAL LETTER F - 0x0047: 0x47, # LATIN CAPITAL LETTER G - 0x0048: 0x48, # LATIN CAPITAL LETTER H - 0x0049: 0x49, # LATIN CAPITAL LETTER I - 0x004A: 0x4A, # LATIN CAPITAL LETTER J - 0x004B: 0x4B, # LATIN CAPITAL LETTER K - 0x004C: 0x4C, # LATIN CAPITAL LETTER L - 0x004D: 0x4D, # LATIN CAPITAL LETTER M - 0x004E: 0x4E, # LATIN CAPITAL LETTER N - 0x004F: 0x4F, # LATIN CAPITAL LETTER O - 0x0050: 0x50, # LATIN CAPITAL LETTER P - 0x0051: 0x51, # LATIN CAPITAL LETTER Q - 0x0052: 0x52, # LATIN CAPITAL LETTER R - 0x0053: 0x53, # LATIN CAPITAL LETTER S - 0x0054: 0x54, # LATIN CAPITAL LETTER T - 0x0055: 0x55, # LATIN CAPITAL LETTER U - 0x0056: 0x56, # LATIN CAPITAL LETTER V - 0x0057: 0x57, # LATIN CAPITAL LETTER W - 0x0058: 0x58, # LATIN CAPITAL LETTER X - 0x0059: 0x59, # LATIN CAPITAL LETTER Y - 0x005A: 0x5A, # LATIN CAPITAL LETTER Z - 0x005B: 0x5B, # LEFT SQUARE BRACKET - 0x005C: 0x5C, # REVERSE SOLIDUS - 0x005D: 0x5D, # RIGHT SQUARE BRACKET - 0x005E: 0x5E, # CIRCUMFLEX ACCENT - 0x005F: 0x5F, # LOW LINE - 0x0060: 0x60, # GRAVE ACCENT - 0x0061: 0x61, # LATIN SMALL LETTER A - 0x0062: 0x62, # LATIN SMALL LETTER B - 0x0063: 0x63, # LATIN SMALL LETTER C - 0x0064: 0x64, # LATIN SMALL LETTER D - 0x0065: 0x65, # LATIN SMALL LETTER E - 0x0066: 0x66, # LATIN SMALL LETTER F - 0x0067: 0x67, # LATIN SMALL LETTER G - 0x0068: 0x68, # LATIN SMALL LETTER H - 0x0069: 0x69, # LATIN SMALL LETTER I - 0x006A: 0x6A, # LATIN SMALL LETTER J - 0x006B: 0x6B, # LATIN SMALL LETTER K - 0x006C: 0x6C, # LATIN SMALL LETTER L - 0x006D: 0x6D, # LATIN SMALL LETTER M - 0x006E: 0x6E, # LATIN SMALL LETTER N - 0x006F: 0x6F, # LATIN SMALL LETTER O - 0x0070: 0x70, # LATIN SMALL LETTER P - 0x0071: 0x71, # LATIN SMALL LETTER Q - 0x0072: 0x72, # LATIN SMALL LETTER R - 0x0073: 0x73, # LATIN SMALL LETTER S - 0x0074: 0x74, # LATIN SMALL LETTER T - 0x0075: 0x75, # LATIN SMALL LETTER U - 0x0076: 0x76, # LATIN SMALL LETTER V - 0x0077: 0x77, # LATIN SMALL LETTER W - 0x0078: 0x78, # LATIN SMALL LETTER X - 0x0079: 0x79, # LATIN SMALL LETTER Y - 0x007A: 0x7A, # LATIN SMALL LETTER Z - 0x007B: 0x7B, # LEFT CURLY BRACKET - 0x007C: 0x7C, # VERTICAL LINE - 0x007D: 0x7D, # RIGHT CURLY BRACKET - 0x007E: 0x7E, # TILDE - 0x007F: 0x7F, # DELETE - 0x0080: 0x80, # - 0x0081: 0x81, # - 0x0082: 0x82, # - 0x0083: 0x83, # - 0x0084: 0x84, # - 0x0085: 0x85, # - 0x0086: 0x86, # - 0x0087: 0x87, # - 0x0088: 0x88, # - 0x0089: 0x89, # - 0x008A: 0x8A, # - 0x008B: 0x8B, # - 0x008C: 0x8C, # - 0x008D: 0x8D, # - 0x008E: 0x8E, # - 0x008F: 0x8F, # - 0x0090: 0x90, # - 0x0091: 0x91, # - 0x0092: 0x92, # - 0x0093: 0x93, # - 0x0094: 0x94, #

-Using WASTE expands on this editor by using -WASTE, an extended TextEdit replacement. - -

Creating a C extension module on the Macintosh is meant for the hardcore programmer, and shows how to create an extension module in C. It also handles using Modulator to create the Modified: python/branches/p3yk/Mac/Demo/textedit.html ============================================================================== --- python/branches/p3yk/Mac/Demo/textedit.html (original) +++ python/branches/p3yk/Mac/Demo/textedit.html Thu Jun 8 17:35:45 2006 @@ -80,8 +80,7 @@ Let us have a look at ped.py (in the Demo:textedit folder), the Pathetic EDitor. It has multiple windows, cut/copy/paste and keyboard input, but that is about all. It looks -as if you can resize the window but it does not work. Still, it serves as an example. We will improve -on ped later, in a waste-based example.
+as if you can resize the window but it does not work. Still, it serves as an example. Ped creates two classes, TEWindow and Ped. Let us start with the latter one, which is a subclass of FrameWork.Application and our main application. The init function Deleted: /python/branches/p3yk/Mac/Demo/waste.html ============================================================================== --- /python/branches/p3yk/Mac/Demo/waste.html Thu Jun 8 17:35:45 2006 +++ (empty file) @@ -1,72 +0,0 @@ -Using WASTE - -
Using WASTE
-
- -WASTE is an almost-compatible TextEdit replacement which overcomes -some of the limitations of it (like the 32K limit) and provides some extensions -(drag and drop, images, undo support). Moreover, it has a much cleaner interface -and is therefore easier integrated in Python.
- -WASTE is written by Marco Piovanelli, <piovanel at kagi.com>, -and copyrighted by him. You can always obtain the latest version (for use in C -or Pascal programs) and the documentation from -<http://www.boingo.com/waste/>. - -We explain the useage of waste here by showing how to modify the TextEdit based -ped.py of the -previous example into the waste-based wed.py, -so you should have both sources handy.
- -Functionally, wed.py provides three new things: resizable windows, a horizontal -scroll bar and undo.
- -Let us look at the code, first at the application class Wed. The only real change is that -we now handle undo. Aside from enabling it in the creation routine and the addition of -a callback routine there is a bit of new code in updatemenubar: Waste not only handles -the full details of implementing undo, it will also tell us what the next undo operation will undo -(or redo). We use this to our advantage by changing the undo menu label to tell the user.
- -The WasteWindow has seen a bit more change. Initialization of the waste data structure is -a bit different, in that we can specify some options at creation time. Also, waste has no SetText -method but a UseText which expects a handle as parameter. We have to be very careful -that we keep this handle around, because Python will happily free the handle if we have no more references -to it (and I doubt that Waste would like this:-). A final difference in open -is that we use a large number for the destination rectangle width, because we will use a horizontal scroll -bar.
- -The idle method is a bit more involved, since we also call WEAdjustCursor to -provide the correct cursor based on mouse-position. Users like this.
- -Getscrollbarvalues is simpler than its' TextEdit counterpart because Waste correctly -updates the destination rectangle when the document changes. Also note that waste uses accessor functions -to get at internal values, as opposed to direct struct access for TextEdit.
- -Scrollbar_callback on the other hand is more elaborate (but also provides more functionality). -It also handles horizontal scrolls (scrolling one-tenth and half a screenful with the buttons). This -function is also "multi-font-ready" in that scrolling one line will do the expected thing in case of multiple -fonts. We will implement a multi-font editor later. A minor annoyance of Waste is that is does not provide -a pinned scroll, so at the end of our callback routine we have to check that we have not scrolled past the -beginning or end of the document, and adjust when needed.
- -do_update is also changed, because Waste is completely region-based (as opposed to rect-based). -Hence, we erase regions here and we can also return immedeately if there is nothing to update.
- -Do_postresize is new: because Waste uses accessor functions we can now modify the viewRect from -Python, which is impossible in the Python TextEdit interface, and hence we can implement resize. The -do_key and do_contentclick methods have also seen minor changes, because the -corresponding waste routines need a bit more information than their TextEdit counterparts. The Cut/copy/paste -code is simplified, because Waste uses the normal desktop scrap.
- -Implementing undo is a wonder of simplicity: Waste handles all the details for us. Also, the new -can_paste method (which controls greying out of the paste menu entry) is an improvement -over what ped did: in ped it was possible that paste was enabled but that the data on the -scrap was incompatible with TextEdit. No more such problems here.
- -That is all for now. There is an undocumented extended version of wed, swed.py, -which supports multiple fonts, sizes and faces, and uses Waste's tab-calculation to do tab characters "right". -There is also an even more elaborate example, htmled.py which extends swed with -the ability to import html files, showing the use of color and how to use embedded object (rulers, in this case). -These two programs have not been documented yet, though, so you will have to look at them without guidance.
-
-Back to the index to pick another example. Modified: python/branches/p3yk/Mac/scripts/BuildApplet.py ============================================================================== --- python/branches/p3yk/Mac/scripts/BuildApplet.py (original) +++ python/branches/p3yk/Mac/scripts/BuildApplet.py Thu Jun 8 17:35:45 2006 @@ -16,6 +16,18 @@ import buildtools import getopt +if not sys.executable.startswith(sys.exec_prefix): + # Oh, the joys of using a python script to bootstrap applicatin bundles + # sys.executable points inside the current application bundle. Because this + # path contains blanks (two of them actually) this path isn't usable on + # #! lines. Reset sys.executable to point to the embedded python interpreter + sys.executable = os.path.join(sys.prefix, + 'Resources/Python.app/Contents/MacOS/Python') + + # Just in case we're not in a framework: + if not os.path.exists(sys.executable): + sys.executable = os.path.join(sys.exec_prefix, 'bin/python') + def main(): try: buildapplet() Modified: python/branches/p3yk/Makefile.pre.in ============================================================================== --- python/branches/p3yk/Makefile.pre.in (original) +++ python/branches/p3yk/Makefile.pre.in Thu Jun 8 17:35:45 2006 @@ -370,12 +370,12 @@ fi libpython$(VERSION).sl: $(LIBRARY_OBJS) - $(LDSHARED) -o $@ $(LIBRARY_OBJS) $(LIBC) $(LIBM) + $(LDSHARED) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM) # This rule is here for OPENSTEP/Rhapsody/MacOSX. It builds a temporary # minimal framework (not including the Lib directory and such) in the current # directory. -RESSRCDIR=$(srcdir)/Mac/OSXResources/framework +RESSRCDIR=$(srcdir)/Mac/Resources/framework $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK): \ $(LIBRARY) \ $(RESSRCDIR)/Info.plist \ @@ -619,8 +619,8 @@ install: @FRAMEWORKINSTALLFIRST@ altinstall bininstall maninstall @FRAMEWORKINSTALLLAST@ # Install almost everything without disturbing previous versions -altinstall: altbininstall libinstall inclinstall libainstall \ - sharedinstall oldsharedinstall +altinstall: @FRAMEWORKALTINSTALLFIRST@ altbininstall libinstall inclinstall libainstall \ + sharedinstall oldsharedinstall @FRAMEWORKALTINSTALLLAST@ # Install shared libraries enabled by Setup DESTDIRS= $(exec_prefix) $(LIBDIR) $(BINLIBDEST) $(DESTSHARED) @@ -899,7 +899,7 @@ # Here are a couple of targets for MacOSX again, to install a full # framework-based Python. frameworkinstall installs everything, the # subtargets install specific parts. Much of the actual work is offloaded to -# the Makefile in Mac/OSX +# the Makefile in Mac # # # This target is here for backward compatiblity, previous versions of Python @@ -940,22 +940,23 @@ # This installs Mac/Lib into the framework frameworkinstallmaclib: - cd Mac/OSX && $(MAKE) installmacsubtree DESTDIR="$(DESTDIR)" + cd Mac && $(MAKE) installmacsubtree DESTDIR="$(DESTDIR)" # This installs the IDE, the Launcher and other apps into /Applications frameworkinstallapps: - cd Mac/OSX && $(MAKE) installapps DESTDIR="$(DESTDIR)" + cd Mac && $(MAKE) installapps DESTDIR="$(DESTDIR)" # This install the unix python and pythonw tools in /usr/local/bin frameworkinstallunixtools: - cd Mac/OSX && $(MAKE) installunixtools DESTDIR="$(DESTDIR)" + cd Mac && $(MAKE) installunixtools DESTDIR="$(DESTDIR)" + +frameworkaltinstallunixtools: + cd Mac && $(MAKE) altinstallunixtools DESTDIR="$(DESTDIR)" # This installs the Demos and Tools into the applications directory. # It is not part of a normal frameworkinstall frameworkinstallextras: - $(MAKE) -f Mac/OSX/Makefile installextras \ - $(RUNSHARED) BUILDPYTHON=./$(BUILDPYTHON) DIRMODE=$(DIRMODE) FILEMODE=$(FILEMODE) \ - srcdir=$(srcdir) builddir=. DESTDIR=$(DESTDIR) + cd Mac && Make installextras DESTDIR="$(DESTDIR)" # This installs a few of the useful scripts in Tools/scripts scriptsinstall: @@ -1083,6 +1084,7 @@ .PHONY: maninstall libinstall inclinstall libainstall sharedinstall .PHONY: frameworkinstall frameworkinstallframework frameworkinstallstructure .PHONY: frameworkinstallmaclib frameworkinstallapps frameworkinstallunixtools -.PHONY: recheck autoconf clean clobber distclean smelly funny +.PHONY: frameworkaltinstallunixtools recheck autoconf clean clobber distclean +.PHONY: smelly funny # IF YOU PUT ANYTHING HERE IT WILL GO AWAY Modified: python/branches/p3yk/Misc/SpecialBuilds.txt ============================================================================== --- python/branches/p3yk/Misc/SpecialBuilds.txt (original) +++ python/branches/p3yk/Misc/SpecialBuilds.txt Thu Jun 8 17:35:45 2006 @@ -96,16 +96,16 @@ Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit ASCII strings. -8 bytes are added at each end of each block of N bytes requested. The -memory layout is like so, where p represents the address returned by a -malloc-like or realloc-like function (p[i:j] means the slice of bytes -from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment -of negative indices differs from a Python slice): - -p[-8:-4] - Number of bytes originally asked for. 4-byte unsigned integer, - big-endian (easier to read in a memory dump). -p[-4:0] +Let S = sizeof(size_t). 2*S bytes are added at each end of each block of N +bytes requested. The memory layout is like so, where p represents the +address returned by a malloc-like or realloc-like function (p[i:j] means +the slice of bytes from *(p+i) inclusive up to *(p+j) exclusive; note that +the treatment of negative indices differs from a Python slice): + +p[-2*S:-S] + Number of bytes originally asked for. This is a size_t, big-endian + (easier to read in a memory dump). +p[-S:0] Copies of FORBIDDENBYTE. Used to catch under- writes and reads. p[0:N] The requested memory, filled with copies of CLEANBYTE, used to catch @@ -116,12 +116,12 @@ DEADBYTE, to catch reference to freed memory. When a realloc- like function is called requesting a smaller memory block, the excess old bytes are also filled with DEADBYTE. -p[N:N+4] +p[N:N+S] Copies of FORBIDDENBYTE. Used to catch over- writes and reads. -p[N+4:N+8] +p[N+S:N+2*S] A serial number, incremented by 1 on each call to a malloc-like or realloc-like function. - 4-byte unsigned integer, big-endian. + Big-endian size_t. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. The static function @@ -145,6 +145,10 @@ If this envar exists, a report of pymalloc summary statistics is printed to stderr whenever a new arena is allocated, and also by Py_Finalize(). + +Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t). +Before it was 16 on all boxes, reflecting that Python couldn't make use of +allocations >= 2**32 bytes even on 64-bit boxes before 2.5. --------------------------------------------------------------------------- Py_DEBUG introduced in 1.5 named DEBUG before 1.5 @@ -251,7 +255,7 @@ find the manual for your specific processor. For the 750CX, 750CXe and 750FX (all sold as the G3) we find: - The time base counter is clocked at a frequency that is + The time base counter is clocked at a frequency that is one-fourth that of the bus clock. This build is enabled by the --with-tsc flag to configure. Modified: python/branches/p3yk/Modules/_bsddb.c ============================================================================== --- python/branches/p3yk/Modules/_bsddb.c (original) +++ python/branches/p3yk/Modules/_bsddb.c Thu Jun 8 17:35:45 2006 @@ -61,13 +61,14 @@ * * http://www.python.org/peps/pep-0291.html * - * This module contains 5 types: + * This module contains 6 types: * * DB (Database) * DBCursor (Database Cursor) * DBEnv (database environment) * DBTxn (An explicit database transaction) * DBLock (A lock handle) + * DBSequence (Sequence) * */ @@ -97,7 +98,7 @@ #error "eek! DBVER can't handle minor versions > 9" #endif -#define PY_BSDDB_VERSION "4.4.2" +#define PY_BSDDB_VERSION "4.4.4" static char *rcs_id = "$Id$"; @@ -285,7 +286,17 @@ #endif } DBLockObject; - +#if (DBVER >= 43) +typedef struct { + PyObject_HEAD + DB_SEQUENCE* sequence; + DBObject* mydb; +#ifdef HAVE_WEAKREF + PyObject *in_weakreflist; /* List of weak references */ +#endif +} DBSequenceObject; +static PyTypeObject DBSequence_Type; +#endif static PyTypeObject DB_Type, DBCursor_Type, DBEnv_Type, DBTxn_Type, DBLock_Type; @@ -294,6 +305,9 @@ #define DBEnvObject_Check(v) ((v)->ob_type == &DBEnv_Type) #define DBTxnObject_Check(v) ((v)->ob_type == &DBTxn_Type) #define DBLockObject_Check(v) ((v)->ob_type == &DBLock_Type) +#if (DBVER >= 43) +#define DBSequenceObject_Check(v) ((v)->ob_type == &DBSequence_Type) +#endif /* --------------------------------------------------------------------- */ @@ -324,6 +338,10 @@ #define CHECK_CURSOR_NOT_CLOSED(curs) \ _CHECK_OBJECT_NOT_CLOSED(curs->dbc, DBCursorClosedError, DBCursor) +#if (DBVER >= 43) +#define CHECK_SEQUENCE_NOT_CLOSED(curs) \ + _CHECK_OBJECT_NOT_CLOSED(curs->sequence, DBError, DBSequence) +#endif #define CHECK_DBFLAG(mydb, flag) (((mydb)->flags & (flag)) || \ (((mydb)->myenvobj != NULL) && ((mydb)->myenvobj->flags & (flag)))) @@ -724,7 +742,17 @@ Py_XDECREF(v); } +#if (DBVER >= 43) +/* add an db_seq_t to a dictionary using the given name as a key */ +static void _addDb_seq_tToDict(PyObject* dict, char *name, db_seq_t value) +{ + PyObject* v = PyLong_FromLongLong(value); + if (!v || PyDict_SetItemString(dict, name, v)) + PyErr_Clear(); + Py_XDECREF(v); +} +#endif @@ -777,7 +805,7 @@ MYDB_END_ALLOW_THREADS; /* TODO add a weakref(self) to the self->myenvobj->open_child_weakrefs * list so that a DBEnv can refuse to close without aborting any open - * open DBTxns and closing any open DBs first. */ + * DBTxns and closing any open DBs first. */ if (makeDBError(err)) { if (self->myenvobj) { Py_DECREF(self->myenvobj); @@ -1029,6 +1057,48 @@ } +#if (DBVER >= 43) +static DBSequenceObject* +newDBSequenceObject(DBObject* mydb, int flags) +{ + int err; + DBSequenceObject* self = PyObject_New(DBSequenceObject, &DBSequence_Type); + if (self == NULL) + return NULL; + Py_INCREF(mydb); + self->mydb = mydb; +#ifdef HAVE_WEAKREF + self->in_weakreflist = NULL; +#endif + + + MYDB_BEGIN_ALLOW_THREADS; + err = db_sequence_create(&self->sequence, self->mydb->db, flags); + MYDB_END_ALLOW_THREADS; + if (makeDBError(err)) { + Py_DECREF(self->mydb); + PyObject_Del(self); + self = NULL; + } + + return self; +} + + +static void +DBSequence_dealloc(DBSequenceObject* self) +{ +#ifdef HAVE_WEAKREF + if (self->in_weakreflist != NULL) { + PyObject_ClearWeakRefs((PyObject *) self); + } +#endif + + Py_DECREF(self->mydb); + PyObject_Del(self); +} +#endif + /* --------------------------------------------------------------------- */ /* DB methods */ @@ -2614,7 +2684,7 @@ Py_ssize_t DB_length(DBObject* self) { int err; - long size = 0; + Py_ssize_t size = 0; int flags = 0; void* sp; @@ -2627,10 +2697,11 @@ if (self->haveStat) { /* Has the stat function been called recently? If so, we can use the cached value. */ - flags = DB_CACHED_COUNTS; + flags = DB_FAST_STAT; } MYDB_BEGIN_ALLOW_THREADS; +redo_stat_for_length: #if (DBVER >= 43) err = self->db->stat(self->db, /*txnid*/ NULL, &sp, flags); #elif (DBVER >= 33) @@ -2638,6 +2709,20 @@ #else err = self->db->stat(self->db, &sp, NULL, flags); #endif + + /* All the stat structures have matching fields upto the ndata field, + so we can use any of them for the type cast */ + size = ((DB_BTREE_STAT*)sp)->bt_ndata; + + /* A size of 0 could mean that BerkeleyDB no longer had the stat values cached. + * redo a full stat to make sure. + * Fixes SF python bug 1493322, pybsddb bug 1184012 + */ + if (size == 0 && (flags & DB_FAST_STAT)) { + flags = 0; + goto redo_stat_for_length; + } + MYDB_END_ALLOW_THREADS; if (err) @@ -2645,9 +2730,6 @@ self->haveStat = 1; - /* All the stat structures have matching fields upto the ndata field, - so we can use any of them for the type cast */ - size = ((DB_BTREE_STAT*)sp)->bt_ndata; free(sp); return size; } @@ -3124,8 +3206,8 @@ int dlen = -1; int doff = -1; DBT key, pkey, data; - static char* kwnames[] = { "key","data", "flags", "dlen", "doff", - NULL }; + static char* kwnames_keyOnly[] = { "key", "flags", "dlen", "doff", NULL }; + static char* kwnames[] = { "key", "data", "flags", "dlen", "doff", NULL }; CLEAR_DBT(key); CLEAR_DBT(data); @@ -3134,7 +3216,7 @@ { PyErr_Clear(); if (!PyArg_ParseTupleAndKeywords(args, kwargs, "Oi|ii:pget", - &kwnames[1], + kwnames_keyOnly, &keyobj, &flags, &dlen, &doff)) { PyErr_Clear(); @@ -4294,6 +4376,93 @@ RETURN_NONE(); } +#if (DBVER >= 44) +static PyObject* +DBEnv_lsn_reset(DBEnvObject* self, PyObject* args, PyObject* kwargs) +{ + int err; + char *file; + u_int32_t flags = 0; + static char* kwnames[] = { "file", "flags", NULL}; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "z|i:lsn_reset", kwnames, + &file, &flags)) + return NULL; + CHECK_ENV_NOT_CLOSED(self); + + MYDB_BEGIN_ALLOW_THREADS; + err = self->db_env->lsn_reset(self->db_env, file, flags); + MYDB_END_ALLOW_THREADS; + RETURN_IF_ERR(); + RETURN_NONE(); +} +#endif /* DBVER >= 4.4 */ + +#if (DBVER >= 40) +static PyObject* +DBEnv_log_stat(DBEnvObject* self, PyObject* args) +{ + int err; + DB_LOG_STAT* statp = NULL; + PyObject* d = NULL; + u_int32_t flags = 0; + + if (!PyArg_ParseTuple(args, "|i:log_stat", &flags)) + return NULL; + CHECK_ENV_NOT_CLOSED(self); + + MYDB_BEGIN_ALLOW_THREADS; + err = self->db_env->log_stat(self->db_env, &statp, flags); + MYDB_END_ALLOW_THREADS; + RETURN_IF_ERR(); + + /* Turn the stat structure into a dictionary */ + d = PyDict_New(); + if (d == NULL) { + if (statp) + free(statp); + return NULL; + } + +#define MAKE_ENTRY(name) _addIntToDict(d, #name, statp->st_##name) + + MAKE_ENTRY(magic); + MAKE_ENTRY(version); + MAKE_ENTRY(mode); + MAKE_ENTRY(lg_bsize); +#if (DBVER >= 44) + MAKE_ENTRY(lg_size); + MAKE_ENTRY(record); +#endif +#if (DBVER <= 40) + MAKE_ENTRY(lg_max); +#endif + MAKE_ENTRY(w_mbytes); + MAKE_ENTRY(w_bytes); + MAKE_ENTRY(wc_mbytes); + MAKE_ENTRY(wc_bytes); + MAKE_ENTRY(wcount); + MAKE_ENTRY(wcount_fill); +#if (DBVER >= 44) + MAKE_ENTRY(rcount); +#endif + MAKE_ENTRY(scount); + MAKE_ENTRY(cur_file); + MAKE_ENTRY(cur_offset); + MAKE_ENTRY(disk_file); + MAKE_ENTRY(disk_offset); + MAKE_ENTRY(maxcommitperflush); + MAKE_ENTRY(mincommitperflush); + MAKE_ENTRY(regsize); + MAKE_ENTRY(region_wait); + MAKE_ENTRY(region_nowait); + +#undef MAKE_ENTRY + free(statp); + return d; +} /* DBEnv_log_stat */ +#endif /* DBVER >= 4.0 for log_stat method */ + static PyObject* DBEnv_lock_stat(DBEnvObject* self, PyObject* args) @@ -4371,7 +4540,7 @@ { int flags=0; int err; - char **log_list_start, **log_list; + char **log_list = NULL; PyObject* list; PyObject* item = NULL; @@ -4391,10 +4560,14 @@ RETURN_IF_ERR(); list = PyList_New(0); - if (list == NULL) + if (list == NULL) { + if (log_list) + free(log_list); return NULL; + } if (log_list) { + char **log_list_start; for (log_list_start = log_list; *log_list != NULL; ++log_list) { item = PyString_FromString (*log_list); if (item == NULL) { @@ -4624,6 +4797,294 @@ return PyInt_FromLong(id); } +#if (DBVER >= 43) +/* --------------------------------------------------------------------- */ +/* DBSequence methods */ + + +static PyObject* +DBSequence_close(DBSequenceObject* self, PyObject* args) +{ + int err, flags=0; + if (!PyArg_ParseTuple(args,"|i:close", &flags)) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->close(self->sequence, flags); + self->sequence = NULL; + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + + RETURN_NONE(); +} + +static PyObject* +DBSequence_get(DBSequenceObject* self, PyObject* args, PyObject* kwargs) +{ + int err, flags = 0; + int delta = 1; + db_seq_t value; + PyObject *txnobj = NULL; + DB_TXN *txn = NULL; + static char* kwnames[] = {"delta", "txn", "flags", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|iOi:get", kwnames, &delta, &txnobj, &flags)) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + if (!checkTxnObj(txnobj, &txn)) + return NULL; + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->get(self->sequence, txn, delta, &value, flags); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + return PyLong_FromLongLong(value); + +} + +static PyObject* +DBSequence_get_dbp(DBSequenceObject* self, PyObject* args) +{ + if (!PyArg_ParseTuple(args,":get_dbp")) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + Py_INCREF(self->mydb); + return (PyObject* )self->mydb; +} + +static PyObject* +DBSequence_get_key(DBSequenceObject* self, PyObject* args) +{ + int err; + DBT key; + CHECK_SEQUENCE_NOT_CLOSED(self) + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->get_key(self->sequence, &key); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + + return PyString_FromStringAndSize(key.data, key.size); +} + +static PyObject* +DBSequence_init_value(DBSequenceObject* self, PyObject* args) +{ + int err; + db_seq_t value; + if (!PyArg_ParseTuple(args,"L:init_value", &value)) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->initial_value(self->sequence, value); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + + RETURN_NONE(); +} + +static PyObject* +DBSequence_open(DBSequenceObject* self, PyObject* args, PyObject* kwargs) +{ + int err, flags = 0; + PyObject* keyobj; + PyObject *txnobj = NULL; + DB_TXN *txn = NULL; + DBT key; + + static char* kwnames[] = {"key", "txn", "flags", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|Oi:open", kwnames, &keyobj, &txnobj, &flags)) + return NULL; + + if (!checkTxnObj(txnobj, &txn)) + return NULL; + + if (!make_key_dbt(self->mydb, keyobj, &key, NULL)) + return NULL; + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->open(self->sequence, txn, &key, flags); + MYDB_END_ALLOW_THREADS + + CLEAR_DBT(key); + RETURN_IF_ERR(); + + RETURN_NONE(); +} + +static PyObject* +DBSequence_remove(DBSequenceObject* self, PyObject* args, PyObject* kwargs) +{ + int err, flags = 0; + PyObject *txnobj = NULL; + DB_TXN *txn = NULL; + + static char* kwnames[] = {"txn", "flags", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|Oi:remove", kwnames, &txnobj, &flags)) + return NULL; + + if (!checkTxnObj(txnobj, &txn)) + return NULL; + + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->remove(self->sequence, txn, flags); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + RETURN_NONE(); +} + +static PyObject* +DBSequence_set_cachesize(DBSequenceObject* self, PyObject* args) +{ + int err, size; + if (!PyArg_ParseTuple(args,"i:set_cachesize", &size)) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->set_cachesize(self->sequence, size); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + RETURN_NONE(); +} + +static PyObject* +DBSequence_get_cachesize(DBSequenceObject* self, PyObject* args) +{ + int err, size; + if (!PyArg_ParseTuple(args,":get_cachesize")) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->get_cachesize(self->sequence, &size); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + return PyInt_FromLong(size); +} + +static PyObject* +DBSequence_set_flags(DBSequenceObject* self, PyObject* args) +{ + int err, flags = 0; + if (!PyArg_ParseTuple(args,"i:set_flags", &flags)) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->set_flags(self->sequence, flags); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + RETURN_NONE(); + +} + +static PyObject* +DBSequence_get_flags(DBSequenceObject* self, PyObject* args) +{ + unsigned int flags; + int err; + if (!PyArg_ParseTuple(args,":get_flags")) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->get_flags(self->sequence, &flags); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + return PyInt_FromLong((int)flags); +} + +static PyObject* +DBSequence_set_range(DBSequenceObject* self, PyObject* args) +{ + int err; + db_seq_t min, max; + if (!PyArg_ParseTuple(args,"(LL):set_range", &min, &max)) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->set_range(self->sequence, min, max); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + RETURN_NONE(); +} + +static PyObject* +DBSequence_get_range(DBSequenceObject* self, PyObject* args) +{ + int err; + db_seq_t min, max; + if (!PyArg_ParseTuple(args,":get_range")) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self) + + MYDB_BEGIN_ALLOW_THREADS + err = self->sequence->get_range(self->sequence, &min, &max); + MYDB_END_ALLOW_THREADS + + RETURN_IF_ERR(); + return Py_BuildValue("(LL)", min, max); +} + +static PyObject* +DBSequence_stat(DBSequenceObject* self, PyObject* args, PyObject* kwargs) +{ + int err, flags = 0; + DB_SEQUENCE_STAT* sp = NULL; + PyObject* dict_stat; + static char* kwnames[] = {"flags", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|i:stat", kwnames, &flags)) + return NULL; + CHECK_SEQUENCE_NOT_CLOSED(self); + + MYDB_BEGIN_ALLOW_THREADS; + err = self->sequence->stat(self->sequence, &sp, flags); + MYDB_END_ALLOW_THREADS; + RETURN_IF_ERR(); + + if ((dict_stat = PyDict_New()) == NULL) { + free(sp); + return NULL; + } + + +#define MAKE_INT_ENTRY(name) _addIntToDict(dict_stat, #name, sp->st_##name) +#define MAKE_LONG_LONG_ENTRY(name) _addDb_seq_tToDict(dict_stat, #name, sp->st_##name) + + MAKE_INT_ENTRY(wait); + MAKE_INT_ENTRY(nowait); + MAKE_LONG_LONG_ENTRY(current); + MAKE_LONG_LONG_ENTRY(value); + MAKE_LONG_LONG_ENTRY(last_value); + MAKE_LONG_LONG_ENTRY(min); + MAKE_LONG_LONG_ENTRY(max); + MAKE_INT_ENTRY(cache_size); + MAKE_INT_ENTRY(flags); + +#undef MAKE_INT_ENTRY +#undef MAKE_LONG_LONG_ENTRY + + free(sp); + return dict_stat; +} +#endif + + /* --------------------------------------------------------------------- */ /* Method definition tables and type objects */ @@ -4777,6 +5238,12 @@ {"lock_put", (PyCFunction)DBEnv_lock_put, METH_VARARGS}, {"lock_stat", (PyCFunction)DBEnv_lock_stat, METH_VARARGS}, {"log_archive", (PyCFunction)DBEnv_log_archive, METH_VARARGS}, +#if (DBVER >= 40) + {"log_stat", (PyCFunction)DBEnv_log_stat, METH_VARARGS}, +#endif +#if (DBVER >= 44) + {"lsn_reset", (PyCFunction)DBEnv_lsn_reset, METH_VARARGS|METH_KEYWORDS}, +#endif {"set_get_returns_none",(PyCFunction)DBEnv_set_get_returns_none, METH_VARARGS}, {NULL, NULL} /* sentinel */ }; @@ -4791,6 +5258,27 @@ }; +#if (DBVER >= 43) +static PyMethodDef DBSequence_methods[] = { + {"close", (PyCFunction)DBSequence_close, METH_VARARGS}, + {"get", (PyCFunction)DBSequence_get, METH_VARARGS|METH_KEYWORDS}, + {"get_dbp", (PyCFunction)DBSequence_get_dbp, METH_VARARGS}, + {"get_key", (PyCFunction)DBSequence_get_key, METH_VARARGS}, + {"init_value", (PyCFunction)DBSequence_init_value, METH_VARARGS}, + {"open", (PyCFunction)DBSequence_open, METH_VARARGS|METH_KEYWORDS}, + {"remove", (PyCFunction)DBSequence_remove, METH_VARARGS|METH_KEYWORDS}, + {"set_cachesize", (PyCFunction)DBSequence_set_cachesize, METH_VARARGS}, + {"get_cachesize", (PyCFunction)DBSequence_get_cachesize, METH_VARARGS}, + {"set_flags", (PyCFunction)DBSequence_set_flags, METH_VARARGS}, + {"get_flags", (PyCFunction)DBSequence_get_flags, METH_VARARGS}, + {"set_range", (PyCFunction)DBSequence_set_range, METH_VARARGS}, + {"get_range", (PyCFunction)DBSequence_get_range, METH_VARARGS}, + {"stat", (PyCFunction)DBSequence_stat, METH_VARARGS|METH_KEYWORDS}, + {NULL, NULL} /* sentinel */ +}; +#endif + + static PyObject* DB_getattr(DBObject* self, char *name) { @@ -4831,6 +5319,14 @@ return NULL; } +#if (DBVER >= 43) +static PyObject* +DBSequence_getattr(DBSequenceObject* self, char *name) +{ + return Py_FindMethod(DBSequence_methods, (PyObject* )self, name); +} +#endif + static PyTypeObject DB_Type = { PyObject_HEAD_INIT(NULL) 0, /*ob_size*/ @@ -4994,6 +5490,39 @@ #endif }; +#if (DBVER >= 43) +static PyTypeObject DBSequence_Type = { + PyObject_HEAD_INIT(NULL) + 0, /*ob_size*/ + "DBSequence", /*tp_name*/ + sizeof(DBSequenceObject), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + /* methods */ + (destructor)DBSequence_dealloc, /*tp_dealloc*/ + 0, /*tp_print*/ + (getattrfunc)DBSequence_getattr,/*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_compare*/ + 0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash*/ +#ifdef HAVE_WEAKREF + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_WEAKREFS, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + offsetof(DBSequenceObject, in_weakreflist), /* tp_weaklistoffset */ +#endif +}; +#endif /* --------------------------------------------------------------------- */ /* Module-level functions */ @@ -5027,6 +5556,25 @@ return (PyObject* )newDBEnvObject(flags); } +#if (DBVER >= 43) +static PyObject* +DBSequence_construct(PyObject* self, PyObject* args, PyObject* kwargs) +{ + PyObject* dbobj = NULL; + int flags = 0; + static char* kwnames[] = { "db", "flags", NULL}; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|i:DBSequence", kwnames, &dbobj, &flags)) + return NULL; + if (dbobj == Py_None) + dbobj = NULL; + else if (dbobj && !DBObject_Check(dbobj)) { + makeTypeError("DB", dbobj); + return NULL; + } + return (PyObject* )newDBSequenceObject((DBObject*)dbobj, flags); +} +#endif static char bsddb_version_doc[] = "Returns a tuple of major, minor, and patch release numbers of the\n\ @@ -5047,9 +5595,12 @@ /* List of functions defined in the module */ static PyMethodDef bsddb_methods[] = { - {"DB", (PyCFunction)DB_construct, METH_VARARGS | METH_KEYWORDS }, - {"DBEnv", (PyCFunction)DBEnv_construct, METH_VARARGS}, - {"version", (PyCFunction)bsddb_version, METH_VARARGS, bsddb_version_doc}, + {"DB", (PyCFunction)DB_construct, METH_VARARGS | METH_KEYWORDS }, + {"DBEnv", (PyCFunction)DBEnv_construct, METH_VARARGS}, +#if (DBVER >= 43) + {"DBSequence", (PyCFunction)DBSequence_construct, METH_VARARGS | METH_KEYWORDS }, +#endif + {"version", (PyCFunction)bsddb_version, METH_VARARGS, bsddb_version_doc}, {NULL, NULL} /* sentinel */ }; @@ -5081,6 +5632,9 @@ DBEnv_Type.ob_type = &PyType_Type; DBTxn_Type.ob_type = &PyType_Type; DBLock_Type.ob_type = &PyType_Type; +#if (DBVER >= 43) + DBSequence_Type.ob_type = &PyType_Type; +#endif #if defined(WITH_THREAD) && !defined(MYDB_USE_GILSTATE) @@ -5247,6 +5801,9 @@ ADD_INT(d, DB_ARCH_ABS); ADD_INT(d, DB_ARCH_DATA); ADD_INT(d, DB_ARCH_LOG); +#if (DBVER >= 42) + ADD_INT(d, DB_ARCH_REMOVE); +#endif ADD_INT(d, DB_BTREE); ADD_INT(d, DB_HASH); @@ -5368,6 +5925,9 @@ #if (DBVER >= 43) ADD_INT(d, DB_LOG_INMEMORY); ADD_INT(d, DB_BUFFER_SMALL); + ADD_INT(d, DB_SEQ_DEC); + ADD_INT(d, DB_SEQ_INC); + ADD_INT(d, DB_SEQ_WRAP); #endif #if (DBVER >= 41) Modified: python/branches/p3yk/Modules/_codecsmodule.c ============================================================================== --- python/branches/p3yk/Modules/_codecsmodule.c (original) +++ python/branches/p3yk/Modules/_codecsmodule.c Thu Jun 8 17:35:45 2006 @@ -792,6 +792,15 @@ return v; } +static PyObject* +charmap_build(PyObject *self, PyObject *args) +{ + PyObject *map; + if (!PyArg_ParseTuple(args, "U:charmap_build", &map)) + return NULL; + return PyUnicode_BuildEncodingMap(map); +} + #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T) static PyObject * @@ -897,6 +906,7 @@ {"ascii_decode", ascii_decode, METH_VARARGS}, {"charmap_encode", charmap_encode, METH_VARARGS}, {"charmap_decode", charmap_decode, METH_VARARGS}, + {"charmap_build", charmap_build, METH_VARARGS}, {"readbuffer_encode", readbuffer_encode, METH_VARARGS}, {"charbuffer_encode", charbuffer_encode, METH_VARARGS}, #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T) Modified: python/branches/p3yk/Modules/_ctypes/callbacks.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/callbacks.c (original) +++ python/branches/p3yk/Modules/_ctypes/callbacks.c Thu Jun 8 17:35:45 2006 @@ -199,45 +199,16 @@ result = PyObject_CallObject(callable, arglist); CHECK("'calling callback function'", result); - if ((restype != &ffi_type_void) - && result && result != Py_None) { /* XXX What is returned for Py_None ? */ - /* another big endian hack */ - union { - char c; - short s; - int i; - long l; - } r; + if ((restype != &ffi_type_void) && result && result != Py_None) { PyObject *keep; assert(setfunc); - switch (restype->size) { - case 1: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.c; - break; - case SIZEOF_SHORT: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.s; - break; - case SIZEOF_INT: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.i; - break; -#if (SIZEOF_LONG != SIZEOF_INT) - case SIZEOF_LONG: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.l; - break; +#ifdef WORDS_BIGENDIAN + /* See the corresponding code in callproc.c, around line 961 */ + if (restype->type != FFI_TYPE_FLOAT && restype->size < sizeof(ffi_arg)) + mem = (char *)mem + sizeof(ffi_arg) - restype->size; #endif - default: - keep = setfunc(mem, result, 0); - CHECK("'converting callback result'", keep); - break; - } + keep = setfunc(mem, result, 0); + CHECK("'converting callback result'", keep); /* keep is an object we have to keep alive so that the result stays valid. If there is no such object, the setfunc will have returned Py_None. Modified: python/branches/p3yk/Modules/_ctypes/callproc.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/callproc.c (original) +++ python/branches/p3yk/Modules/_ctypes/callproc.c Thu Jun 8 17:35:45 2006 @@ -964,7 +964,14 @@ address cannot simply be used as result pointer, instead we must adjust the pointer value: */ - if (rtype->size < sizeof(ffi_arg)) + /* + XXX I should find out and clarify why this is needed at all, + especially why adjusting for ffi_type_float must be avoided on + 64-bit platforms. + */ + if (rtype->type != FFI_TYPE_FLOAT + && rtype->type != FFI_TYPE_STRUCT + && rtype->size < sizeof(ffi_arg)) resbuf = (char *)resbuf + sizeof(ffi_arg) - rtype->size; #endif Modified: python/branches/p3yk/Modules/_ctypes/cfield.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/cfield.c (original) +++ python/branches/p3yk/Modules/_ctypes/cfield.c Thu Jun 8 17:35:45 2006 @@ -1,5 +1,4 @@ #include "Python.h" -#include "structmember.h" #include #ifdef MS_WIN32 @@ -208,14 +207,30 @@ self->index, self->size, src->b_ptr + self->offset); } -static PyMemberDef CField_members[] = { - { "offset", T_UINT, - offsetof(CFieldObject, offset), READONLY, - "offset in bytes of this field"}, - { "size", T_UINT, - offsetof(CFieldObject, size), READONLY, - "size in bytes of this field"}, - { NULL }, +static PyObject * +CField_get_offset(PyObject *self, void *data) +{ +#if (PY_VERSION_HEX < 0x02050000) + return PyInt_FromLong(((CFieldObject *)self)->offset); +#else + return PyInt_FromSsize_t(((CFieldObject *)self)->offset); +#endif +} + +static PyObject * +CField_get_size(PyObject *self, void *data) +{ +#if (PY_VERSION_HEX < 0x02050000) + return PyInt_FromLong(((CFieldObject *)self)->size); +#else + return PyInt_FromSsize_t(((CFieldObject *)self)->size); +#endif +} + +static PyGetSetDef CField_getset[] = { + { "offset", CField_get_offset, NULL, "offset in bytes of this field" }, + { "size", CField_get_size, NULL, "size in bytes of this field" }, + { NULL, NULL, NULL, NULL }, }; static int @@ -298,8 +313,8 @@ 0, /* tp_iter */ 0, /* tp_iternext */ 0, /* tp_methods */ - CField_members, /* tp_members */ - 0, /* tp_getset */ + 0, /* tp_members */ + CField_getset, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ (descrgetfunc)CField_get, /* tp_descr_get */ Modified: python/branches/p3yk/Modules/_elementtree.c ============================================================================== --- python/branches/p3yk/Modules/_elementtree.c (original) +++ python/branches/p3yk/Modules/_elementtree.c Thu Jun 8 17:35:45 2006 @@ -1476,7 +1476,7 @@ } static PyObject* -treebuilder(PyObject* _self, PyObject* args) +treebuilder(PyObject* self_, PyObject* args) { if (!PyArg_ParseTuple(args, ":TreeBuilder")) return NULL; @@ -2201,7 +2201,7 @@ /* constructor and destructor */ static PyObject* -xmlparser(PyObject* _self, PyObject* args, PyObject* kw) +xmlparser(PyObject* self_, PyObject* args, PyObject* kw) { XMLParserObject* self; /* FIXME: does this need to be static? */ Modified: python/branches/p3yk/Modules/_struct.c ============================================================================== --- python/branches/p3yk/Modules/_struct.c (original) +++ python/branches/p3yk/Modules/_struct.c Thu Jun 8 17:35:45 2006 @@ -1572,8 +1572,7 @@ soself = (PyStructObject *)self; assert(PyStruct_Check(self)); assert(soself->s_codes != NULL); - if (args == NULL || !PyTuple_Check(args) || - PyTuple_GET_SIZE(args) != soself->s_len) + if (PyTuple_GET_SIZE(args) != soself->s_len) { PyErr_Format(StructError, "pack requires exactly %zd arguments", soself->s_len); @@ -1594,16 +1593,16 @@ return result; } -PyDoc_STRVAR(s_pack_to__doc__, -"S.pack_to(buffer, offset, v1, v2, ...)\n\ +PyDoc_STRVAR(s_pack_into__doc__, +"S.pack_into(buffer, offset, v1, v2, ...)\n\ \n\ -Pack the values v2, v2, ... according to this Struct's format, write \n\ +Pack the values v1, v2, ... according to this Struct's format, write \n\ the packed bytes into the writable buffer buf starting at offset. Note\n\ that the offset is not an optional argument. See struct.__doc__ for \n\ more on format strings."); static PyObject * -s_pack_to(PyObject *self, PyObject *args) +s_pack_into(PyObject *self, PyObject *args) { PyStructObject *soself; char *buffer; @@ -1613,11 +1612,10 @@ soself = (PyStructObject *)self; assert(PyStruct_Check(self)); assert(soself->s_codes != NULL); - if (args == NULL || !PyTuple_Check(args) || - PyTuple_GET_SIZE(args) != (soself->s_len + 2)) + if (PyTuple_GET_SIZE(args) != (soself->s_len + 2)) { PyErr_Format(StructError, - "pack_to requires exactly %zd arguments", + "pack_into requires exactly %zd arguments", (soself->s_len + 2)); return NULL; } @@ -1630,7 +1628,7 @@ assert( buffer_len >= 0 ); /* Extract the offset from the first argument */ - offset = PyInt_AsLong(PyTuple_GET_ITEM(args, 1)); + offset = PyInt_AsSsize_t(PyTuple_GET_ITEM(args, 1)); /* Support negative offsets. */ if (offset < 0) @@ -1639,7 +1637,7 @@ /* Check boundaries */ if (offset < 0 || (buffer_len - offset) < soself->s_size) { PyErr_Format(StructError, - "pack_to requires a buffer of at least %zd bytes", + "pack_into requires a buffer of at least %zd bytes", soself->s_size); return NULL; } @@ -1668,10 +1666,11 @@ /* List of functions */ static struct PyMethodDef s_methods[] = { - {"pack", (PyCFunction)s_pack, METH_VARARGS, s_pack__doc__}, - {"pack_to", (PyCFunction)s_pack_to, METH_VARARGS, s_pack_to__doc__}, - {"unpack", (PyCFunction)s_unpack, METH_O, s_unpack__doc__}, - {"unpack_from", (PyCFunction)s_unpack_from, METH_KEYWORDS, s_unpack_from__doc__}, + {"pack", s_pack, METH_VARARGS, s_pack__doc__}, + {"pack_into", s_pack_into, METH_VARARGS, s_pack_into__doc__}, + {"unpack", s_unpack, METH_O, s_unpack__doc__}, + {"unpack_from", (PyCFunction)s_unpack_from, METH_KEYWORDS, + s_unpack_from__doc__}, {NULL, NULL} /* sentinel */ }; Modified: python/branches/p3yk/Modules/_tkinter.c ============================================================================== --- python/branches/p3yk/Modules/_tkinter.c (original) +++ python/branches/p3yk/Modules/_tkinter.c Thu Jun 8 17:35:45 2006 @@ -1274,13 +1274,13 @@ and perform processing there. */ static PyObject * -Tkapp_Call(PyObject *_self, PyObject *args) +Tkapp_Call(PyObject *selfptr, PyObject *args) { Tcl_Obj *objStore[ARGSZ]; Tcl_Obj **objv = NULL; int objc, i; PyObject *res = NULL; - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; /* Could add TCL_EVAL_GLOBAL if wrapped by GlobalCall... */ int flags = TCL_EVAL_DIRECT; @@ -1326,7 +1326,7 @@ ENTER_OVERLAP if (i == TCL_ERROR) - Tkinter_Error(_self); + Tkinter_Error(selfptr); else res = Tkapp_CallResult(self); @@ -1542,12 +1542,12 @@ } static PyObject* -var_invoke(EventFunc func, PyObject *_self, PyObject *args, int flags) +var_invoke(EventFunc func, PyObject *selfptr, PyObject *args, int flags) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; #ifdef WITH_THREAD if (self->threaded && self->thread_id != Tcl_GetCurrentThread()) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; VarEvent *ev; PyObject *res, *exc_type, *exc_val; @@ -1559,7 +1559,7 @@ ev = (VarEvent*)ckalloc(sizeof(VarEvent)); - ev->self = _self; + ev->self = selfptr; ev->args = args; ev->flags = flags; ev->func = func; @@ -1579,7 +1579,7 @@ } #endif /* Tcl is not threaded, or this is the interpreter thread. */ - return func(_self, args, flags); + return func(selfptr, args, flags); } static PyObject * @@ -2079,9 +2079,9 @@ } static PyObject * -Tkapp_CreateCommand(PyObject *_self, PyObject *args) +Tkapp_CreateCommand(PyObject *selfptr, PyObject *args) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; PythonCmd_ClientData *data; char *cmdName; PyObject *func; @@ -2105,7 +2105,7 @@ return PyErr_NoMemory(); Py_XINCREF(self); Py_XINCREF(func); - data->self = _self; + data->self = selfptr; data->func = func; if (self->threaded && self->thread_id != Tcl_GetCurrentThread()) { @@ -2139,9 +2139,9 @@ static PyObject * -Tkapp_DeleteCommand(PyObject *_self, PyObject *args) +Tkapp_DeleteCommand(PyObject *selfptr, PyObject *args) { - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; char *cmdName; int err; @@ -2502,10 +2502,10 @@ /** Event Loop **/ static PyObject * -Tkapp_MainLoop(PyObject *_self, PyObject *args) +Tkapp_MainLoop(PyObject *selfptr, PyObject *args) { int threshold = 0; - TkappObject *self = (TkappObject*)_self; + TkappObject *self = (TkappObject*)selfptr; #ifdef WITH_THREAD PyThreadState *tstate = PyThreadState_Get(); #endif Modified: python/branches/p3yk/Modules/cjkcodecs/_codecs_jp.c ============================================================================== --- python/branches/p3yk/Modules/cjkcodecs/_codecs_jp.c (original) +++ python/branches/p3yk/Modules/cjkcodecs/_codecs_jp.c Thu Jun 8 17:35:45 2006 @@ -639,10 +639,11 @@ REQUIRE_OUTBUF(1) JISX0201_DECODE(c, **outbuf) else if ((c >= 0x81 && c <= 0x9f) || (c >= 0xe0 && c <= 0xfc)){ - unsigned char c1, c2 = IN2; + unsigned char c1, c2; ucs4_t code; REQUIRE_INBUF(2) + c2 = IN2; if (c2 < 0x40 || (c2 > 0x7e && c2 < 0x80) || c2 > 0xfc) return 2; Modified: python/branches/p3yk/Modules/socketmodule.c ============================================================================== --- python/branches/p3yk/Modules/socketmodule.c (original) +++ python/branches/p3yk/Modules/socketmodule.c Thu Jun 8 17:35:45 2006 @@ -104,9 +104,9 @@ listen(n) -- start listening for incoming connections\n\ makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ recv(buflen[, flags]) -- receive data\n\ -recv_buf(buffer[, nbytes[, flags]]) -- receive data (into a buffer)\n\ +recv_into(buffer[, nbytes[, flags]]) -- receive data (into a buffer)\n\ recvfrom(buflen[, flags]) -- receive data and sender\'s address\n\ -recvfrom_buf(buffer[, nbytes, [, flags])\n\ +recvfrom_into(buffer[, nbytes, [, flags])\n\ -- receive data and sender\'s address (into a buffer)\n\ sendall(data[, flags]) -- send all data\n\ send(data[, flags]) -- send data, may not send all of it\n\ @@ -2139,17 +2139,18 @@ #endif /* NO_DUP */ /* - * This is the guts of the recv() and recv_buf() methods, which reads into a + * This is the guts of the recv() and recv_into() methods, which reads into a * char buffer. If you have any inc/def ref to do to the objects that contain * the buffer, do it in the caller. This function returns the number of bytes * succesfully read. If there was an error, it returns -1. Note that it is * also possible that we return a number of bytes smaller than the request * bytes. */ -static int +static ssize_t sock_recv_guts(PySocketSockObject *s, char* cbuf, int len, int flags) { - int timeout, outlen = 0; + ssize_t outlen = 0; + int timeout; #ifdef __VMS int remaining, nread; char *read_buf; @@ -2225,7 +2226,8 @@ static PyObject * sock_recv(PySocketSockObject *s, PyObject *args) { - int recvlen, flags = 0, outlen; + int recvlen, flags = 0; + ssize_t outlen; PyObject *buf; if (!PyArg_ParseTuple(args, "i|i:recv", &recvlen, &flags)) @@ -2243,7 +2245,7 @@ return NULL; /* Call the guts */ - outlen = sock_recv_guts(s, PyString_AsString(buf), recvlen, flags); + outlen = sock_recv_guts(s, PyString_AS_STRING(buf), recvlen, flags); if (outlen < 0) { /* An error occured, release the string and return an error. */ @@ -2270,19 +2272,20 @@ the remote end is closed and all data is read, return the empty string."); -/* s.recv_buf(buffer, [nbytes [,flags]]) method */ +/* s.recv_into(buffer, [nbytes [,flags]]) method */ static PyObject* -sock_recv_buf(PySocketSockObject *s, PyObject *args, PyObject *kwds) +sock_recv_into(PySocketSockObject *s, PyObject *args, PyObject *kwds) { static char *kwlist[] = {"buffer", "nbytes", "flags", 0}; - int recvlen = 0, flags = 0, readlen; + int recvlen = 0, flags = 0; + ssize_t readlen; char *buf; int buflen; /* Get the buffer's memory */ - if (!PyArg_ParseTupleAndKeywords(args, kwds, "s#|ii:recv", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kwds, "w#|ii:recv", kwlist, &buf, &buflen, &recvlen, &flags)) return NULL; assert(buf != 0 && buflen > 0); @@ -2313,11 +2316,11 @@ /* Return the number of bytes read. Note that we do not do anything special here in the case that readlen < recvlen. */ - return PyInt_FromLong(readlen); + return PyInt_FromSsize_t(readlen); } -PyDoc_STRVAR(recv_buf_doc, -"recv_buf(buffer, [nbytes[, flags]]) -> nbytes_read\n\ +PyDoc_STRVAR(recv_into_doc, +"recv_into(buffer, [nbytes[, flags]]) -> nbytes_read\n\ \n\ A version of recv() that stores its data into a buffer rather than creating \n\ a new string. Receive up to buffersize bytes from the socket. If buffersize \n\ @@ -2327,7 +2330,7 @@ /* - * This is the guts of the recv() and recv_buf() methods, which reads into a + * This is the guts of the recv() and recv_into() methods, which reads into a * char buffer. If you have any inc/def ref to do to the objects that contain * the buffer, do it in the caller. This function returns the number of bytes * succesfully read. If there was an error, it returns -1. Note that it is @@ -2337,12 +2340,13 @@ * 'addr' is a return value for the address object. Note that you must decref * it yourself. */ -static int +static ssize_t sock_recvfrom_guts(PySocketSockObject *s, char* cbuf, int len, int flags, PyObject** addr) { sock_addr_t addrbuf; - int n = 0, timeout; + int timeout; + ssize_t n = 0; socklen_t addrlen; *addr = NULL; @@ -2398,7 +2402,8 @@ PyObject *buf = NULL; PyObject *addr = NULL; PyObject *ret = NULL; - int recvlen, outlen, flags = 0; + int recvlen, flags = 0; + ssize_t outlen; if (!PyArg_ParseTuple(args, "i|i:recvfrom", &recvlen, &flags)) return NULL; @@ -2435,21 +2440,21 @@ Like recv(buffersize, flags) but also return the sender's address info."); -/* s.recvfrom_buf(buffer[, nbytes [,flags]]) method */ +/* s.recvfrom_into(buffer[, nbytes [,flags]]) method */ static PyObject * -sock_recvfrom_buf(PySocketSockObject *s, PyObject *args, PyObject* kwds) +sock_recvfrom_into(PySocketSockObject *s, PyObject *args, PyObject* kwds) { static char *kwlist[] = {"buffer", "nbytes", "flags", 0}; - int recvlen = 0, flags = 0, readlen; + int recvlen = 0, flags = 0; + ssize_t readlen; char *buf; int buflen; PyObject *addr = NULL; - PyObject *ret = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwds, "s#|ii:recvfrom", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kwds, "w#|ii:recvfrom", kwlist, &buf, &buflen, &recvlen, &flags)) return NULL; assert(buf != 0 && buflen > 0); @@ -2467,22 +2472,19 @@ readlen = sock_recvfrom_guts(s, buf, recvlen, flags, &addr); if (readlen < 0) { /* Return an error */ - goto finally; + Py_XDECREF(addr); + return NULL; } /* Return the number of bytes read and the address. Note that we do not do anything special here in the case that readlen < recvlen. */ - ret = Py_BuildValue("lO", readlen, addr); - -finally: - Py_XDECREF(addr); - return ret; + return Py_BuildValue("lN", readlen, addr); } -PyDoc_STRVAR(recvfrom_buf_doc, -"recvfrom_buf(buffer[, nbytes[, flags]]) -> (nbytes, address info)\n\ +PyDoc_STRVAR(recvfrom_into_doc, +"recvfrom_into(buffer[, nbytes[, flags]]) -> (nbytes, address info)\n\ \n\ -Like recv_buf(buffer[, nbytes[, flags]]) but also return the sender's address info."); +Like recv_into(buffer[, nbytes[, flags]]) but also return the sender's address info."); /* s.send(data [,flags]) method */ @@ -2711,12 +2713,12 @@ #endif {"recv", (PyCFunction)sock_recv, METH_VARARGS, recv_doc}, - {"recv_buf", (PyCFunction)sock_recv_buf, METH_VARARGS | METH_KEYWORDS, - recv_buf_doc}, + {"recv_into", (PyCFunction)sock_recv_into, METH_VARARGS | METH_KEYWORDS, + recv_into_doc}, {"recvfrom", (PyCFunction)sock_recvfrom, METH_VARARGS, recvfrom_doc}, - {"recvfrom_buf", (PyCFunction)sock_recvfrom_buf, METH_VARARGS | METH_KEYWORDS, - recvfrom_buf_doc}, + {"recvfrom_into", (PyCFunction)sock_recvfrom_into, METH_VARARGS | METH_KEYWORDS, + recvfrom_into_doc}, {"send", (PyCFunction)sock_send, METH_VARARGS, send_doc}, {"sendall", (PyCFunction)sock_sendall, METH_VARARGS, Modified: python/branches/p3yk/Objects/abstract.c ============================================================================== --- python/branches/p3yk/Objects/abstract.c (original) +++ python/branches/p3yk/Objects/abstract.c Thu Jun 8 17:35:45 2006 @@ -216,7 +216,8 @@ return ret; } -int PyObject_AsCharBuffer(PyObject *obj, +int +PyObject_AsCharBuffer(PyObject *obj, const char **buffer, Py_ssize_t *buffer_len) { Modified: python/branches/p3yk/Objects/classobject.c ============================================================================== --- python/branches/p3yk/Objects/classobject.c (original) +++ python/branches/p3yk/Objects/classobject.c Thu Jun 8 17:35:45 2006 @@ -2217,9 +2217,17 @@ static int instancemethod_compare(PyMethodObject *a, PyMethodObject *b) { - if (a->im_self != b->im_self) + int cmp; + cmp = PyObject_Compare(a->im_func, b->im_func); + if (cmp) + return cmp; + + if (a->im_self == b->im_self) + return 0; + if (a->im_self == NULL || b->im_self == NULL) return (a->im_self < b->im_self) ? -1 : 1; - return PyObject_Compare(a->im_func, b->im_func); + else + return PyObject_Compare(a->im_self, b->im_self); } static PyObject * @@ -2295,7 +2303,10 @@ y = PyObject_Hash(a->im_func); if (y == -1) return -1; - return x ^ y; + x = x ^ y; + if (x == -1) + x = -2; + return x; } static int Modified: python/branches/p3yk/Objects/descrobject.c ============================================================================== --- python/branches/p3yk/Objects/descrobject.c (original) +++ python/branches/p3yk/Objects/descrobject.c Thu Jun 8 17:35:45 2006 @@ -901,16 +901,28 @@ static int wrapper_compare(wrapperobject *a, wrapperobject *b) { - if (a->descr == b->descr) { - if (a->self == b->self) - return 0; - else - return (a->self < b->self) ? -1 : 1; - } + if (a->descr == b->descr) + return PyObject_Compare(a->self, b->self); else return (a->descr < b->descr) ? -1 : 1; } +static long +wrapper_hash(wrapperobject *wp) +{ + int x, y; + x = _Py_HashPointer(wp->descr); + if (x == -1) + return -1; + y = PyObject_Hash(wp->self); + if (y == -1) + return -1; + x = x ^ y; + if (x == -1) + x = -2; + return x; +} + static PyObject * wrapper_repr(wrapperobject *wp) { @@ -1008,7 +1020,7 @@ 0, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ - 0, /* tp_hash */ + (hashfunc)wrapper_hash, /* tp_hash */ (ternaryfunc)wrapper_call, /* tp_call */ 0, /* tp_str */ PyObject_GenericGetAttr, /* tp_getattro */ Modified: python/branches/p3yk/Objects/exceptions.c ============================================================================== --- python/branches/p3yk/Objects/exceptions.c (original) +++ python/branches/p3yk/Objects/exceptions.c Thu Jun 8 17:35:45 2006 @@ -851,7 +851,6 @@ PyTuple_SET_ITEM(tuple, 1, Py_None); } - Py_INCREF(repr); PyTuple_SET_ITEM(tuple, 2, repr); rtnval = PyString_Format(fmt, tuple); Modified: python/branches/p3yk/Objects/fileobject.c ============================================================================== --- python/branches/p3yk/Objects/fileobject.c (original) +++ python/branches/p3yk/Objects/fileobject.c Thu Jun 8 17:35:45 2006 @@ -1640,6 +1640,20 @@ return (PyObject *)f; } +static PyObject * +file_exit(PyFileObject *f, PyObject *args) +{ + PyObject *ret = file_close(f); + if (!ret) + /* If error occurred, pass through */ + return NULL; + Py_DECREF(ret); + /* We cannot return the result of close since a true + * value will be interpreted as "yes, swallow the + * exception if one was raised inside the with block". */ + Py_RETURN_NONE; +} + PyDoc_STRVAR(readline_doc, "readline([size]) -> next line from the file, as a string.\n" "\n" @@ -1721,6 +1735,9 @@ PyDoc_STRVAR(enter_doc, "__enter__() -> self."); +PyDoc_STRVAR(exit_doc, + "__exit__(*excinfo) -> None. Closes the file."); + static PyMethodDef file_methods[] = { {"readline", (PyCFunction)file_readline, METH_VARARGS, readline_doc}, {"read", (PyCFunction)file_read, METH_VARARGS, read_doc}, @@ -1738,7 +1755,7 @@ {"close", (PyCFunction)file_close, METH_NOARGS, close_doc}, {"isatty", (PyCFunction)file_isatty, METH_NOARGS, isatty_doc}, {"__enter__", (PyCFunction)file_self, METH_NOARGS, enter_doc}, - {"__exit__", (PyCFunction)file_close, METH_VARARGS, close_doc}, + {"__exit__", (PyCFunction)file_exit, METH_VARARGS, exit_doc}, {NULL, NULL} /* sentinel */ }; Modified: python/branches/p3yk/Objects/obmalloc.c ============================================================================== --- python/branches/p3yk/Objects/obmalloc.c (original) +++ python/branches/p3yk/Objects/obmalloc.c Thu Jun 8 17:35:45 2006 @@ -491,13 +491,13 @@ #define INITIAL_ARENA_OBJECTS 16 /* Number of arenas allocated that haven't been free()'d. */ -static ulong narenas_currently_allocated = 0; +static size_t narenas_currently_allocated = 0; #ifdef PYMALLOC_DEBUG /* Total number of times malloc() called to allocate an arena. */ -static ulong ntimes_arena_allocated = 0; +static size_t ntimes_arena_allocated = 0; /* High water mark (max value ever seen) for narenas_currently_allocated. */ -static ulong narenas_highwater = 0; +static size_t narenas_highwater = 0; #endif /* Allocate a new arena. If we run out of memory, return NULL. Else @@ -1220,39 +1220,45 @@ #define DEADBYTE 0xDB /* dead (newly freed) memory */ #define FORBIDDENBYTE 0xFB /* untouchable bytes at each end of a block */ -static ulong serialno = 0; /* incremented on each debug {m,re}alloc */ +static size_t serialno = 0; /* incremented on each debug {m,re}alloc */ /* serialno is always incremented via calling this routine. The point is - to supply a single place to set a breakpoint. -*/ + * to supply a single place to set a breakpoint. + */ static void bumpserialno(void) { ++serialno; } +#define SST SIZEOF_SIZE_T -/* Read 4 bytes at p as a big-endian ulong. */ -static ulong -read4(const void *p) +/* Read sizeof(size_t) bytes at p as a big-endian size_t. */ +static size_t +read_size_t(const void *p) { const uchar *q = (const uchar *)p; - return ((ulong)q[0] << 24) | - ((ulong)q[1] << 16) | - ((ulong)q[2] << 8) | - (ulong)q[3]; + size_t result = *q++; + int i; + + for (i = SST; --i > 0; ++q) + result = (result << 8) | *q; + return result; } -/* Write the 4 least-significant bytes of n as a big-endian unsigned int, - MSB at address p, LSB at p+3. */ +/* Write n as a big-endian size_t, MSB at address p, LSB at + * p + sizeof(size_t) - 1. + */ static void -write4(void *p, ulong n) +write_size_t(void *p, size_t n) { - uchar *q = (uchar *)p; - q[0] = (uchar)((n >> 24) & 0xff); - q[1] = (uchar)((n >> 16) & 0xff); - q[2] = (uchar)((n >> 8) & 0xff); - q[3] = (uchar)( n & 0xff); + uchar *q = (uchar *)p + SST - 1; + int i; + + for (i = SST; --i >= 0; --q) { + *q = (uchar)(n & 0xff); + n >>= 8; + } } #ifdef Py_DEBUG @@ -1280,25 +1286,25 @@ #endif /* Py_DEBUG */ -/* The debug malloc asks for 16 extra bytes and fills them with useful stuff, - here calling the underlying malloc's result p: +/* Let S = sizeof(size_t). The debug malloc asks for 4*S extra bytes and + fills them with useful stuff, here calling the underlying malloc's result p: -p[0:4] - Number of bytes originally asked for. 4-byte unsigned integer, - big-endian (easier to read in a memory dump). -p[4:8] +p[0: S] + Number of bytes originally asked for. This is a size_t, big-endian (easier + to read in a memory dump). +p[S: 2*S] Copies of FORBIDDENBYTE. Used to catch under- writes and reads. -p[8:8+n] +p[2*S: 2*S+n] The requested memory, filled with copies of CLEANBYTE. Used to catch reference to uninitialized memory. - &p[8] is returned. Note that this is 8-byte aligned if pymalloc + &p[2*S] is returned. Note that this is 8-byte aligned if pymalloc handled the request itself. -p[8+n:8+n+4] +p[2*S+n: 2*S+n+S] Copies of FORBIDDENBYTE. Used to catch over- writes and reads. -p[8+n+4:8+n+8] +p[2*S+n+S: 2*S+n+2*S] A serial number, incremented by 1 on each call to _PyObject_DebugMalloc and _PyObject_DebugRealloc. - 4-byte unsigned integer, big-endian. + This is a big-endian size_t. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. @@ -1308,41 +1314,33 @@ _PyObject_DebugMalloc(size_t nbytes) { uchar *p; /* base address of malloc'ed block */ - uchar *tail; /* p + 8 + nbytes == pointer to tail pad bytes */ - size_t total; /* nbytes + 16 */ + uchar *tail; /* p + 2*SST + nbytes == pointer to tail pad bytes */ + size_t total; /* nbytes + 4*SST */ bumpserialno(); - total = nbytes + 16; -#if SIZEOF_SIZE_T < 8 - /* XXX do this check only on 32-bit machines */ - if (total < nbytes || (total >> 31) > 1) { - /* overflow, or we can't represent it in 4 bytes */ - /* Obscure: can't do (total >> 32) != 0 instead, because - C doesn't define what happens for a right-shift of 32 - when size_t is a 32-bit type. At least C guarantees - size_t is an unsigned type. */ + total = nbytes + 4*SST; + if (total < nbytes) + /* overflow: can't represent total as a size_t */ return NULL; - } -#endif p = (uchar *)PyObject_Malloc(total); if (p == NULL) return NULL; - write4(p, (ulong)nbytes); - p[4] = p[5] = p[6] = p[7] = FORBIDDENBYTE; + write_size_t(p, nbytes); + memset(p + SST, FORBIDDENBYTE, SST); if (nbytes > 0) - memset(p+8, CLEANBYTE, nbytes); + memset(p + 2*SST, CLEANBYTE, nbytes); - tail = p + 8 + nbytes; - tail[0] = tail[1] = tail[2] = tail[3] = FORBIDDENBYTE; - write4(tail + 4, serialno); + tail = p + 2*SST + nbytes; + memset(tail, FORBIDDENBYTE, SST); + write_size_t(tail + SST, serialno); - return p+8; + return p + 2*SST; } -/* The debug free first checks the 8 bytes on each end for sanity (in +/* The debug free first checks the 2*SST bytes on each end for sanity (in particular, that the FORBIDDENBYTEs are still intact). Then fills the original bytes with DEADBYTE. Then calls the underlying free. @@ -1350,16 +1348,16 @@ void _PyObject_DebugFree(void *p) { - uchar *q = (uchar *)p; + uchar *q = (uchar *)p - 2*SST; /* address returned from malloc */ size_t nbytes; if (p == NULL) return; _PyObject_DebugCheckAddress(p); - nbytes = read4(q-8); + nbytes = read_size_t(q); if (nbytes > 0) memset(q, DEADBYTE, nbytes); - PyObject_Free(q-8); + PyObject_Free(q); } void * @@ -1367,20 +1365,20 @@ { uchar *q = (uchar *)p; uchar *tail; - size_t total; /* nbytes + 16 */ + size_t total; /* nbytes + 4*SST */ size_t original_nbytes; + int i; if (p == NULL) return _PyObject_DebugMalloc(nbytes); _PyObject_DebugCheckAddress(p); bumpserialno(); - original_nbytes = read4(q-8); - total = nbytes + 16; - if (total < nbytes || (total >> 31) > 1) { - /* overflow, or we can't represent it in 4 bytes */ + original_nbytes = read_size_t(q - 2*SST); + total = nbytes + 4*SST; + if (total < nbytes) + /* overflow: can't represent total as a size_t */ return NULL; - } if (nbytes < original_nbytes) { /* shrinking: mark old extra memory dead */ @@ -1388,19 +1386,17 @@ } /* Resize and add decorations. */ - q = (uchar *)PyObject_Realloc(q-8, total); + q = (uchar *)PyObject_Realloc(q - 2*SST, total); if (q == NULL) return NULL; - write4(q, (ulong)nbytes); - assert(q[4] == FORBIDDENBYTE && - q[5] == FORBIDDENBYTE && - q[6] == FORBIDDENBYTE && - q[7] == FORBIDDENBYTE); - q += 8; + write_size_t(q, nbytes); + for (i = 0; i < SST; ++i) + assert(q[SST + i] == FORBIDDENBYTE); + q += 2*SST; tail = q + nbytes; - tail[0] = tail[1] = tail[2] = tail[3] = FORBIDDENBYTE; - write4(tail + 4, serialno); + memset(tail, FORBIDDENBYTE, SST); + write_size_t(tail + SST, serialno); if (nbytes > original_nbytes) { /* growing: mark new extra memory clean */ @@ -1420,7 +1416,7 @@ { const uchar *q = (const uchar *)p; char *msg; - ulong nbytes; + size_t nbytes; const uchar *tail; int i; @@ -1433,16 +1429,16 @@ * corruption, the number-of-bytes field may be nuts, and checking * the tail could lead to a segfault then. */ - for (i = 4; i >= 1; --i) { + for (i = SST; i >= 1; --i) { if (*(q-i) != FORBIDDENBYTE) { msg = "bad leading pad byte"; goto error; } } - nbytes = read4(q-8); + nbytes = read_size_t(q - 2*SST); tail = q + nbytes; - for (i = 0; i < 4; ++i) { + for (i = 0; i < SST; ++i) { if (tail[i] != FORBIDDENBYTE) { msg = "bad trailing pad byte"; goto error; @@ -1462,28 +1458,33 @@ { const uchar *q = (const uchar *)p; const uchar *tail; - ulong nbytes, serial; + size_t nbytes, serial; int i; + int ok; fprintf(stderr, "Debug memory block at address p=%p:\n", p); if (p == NULL) return; - nbytes = read4(q-8); - fprintf(stderr, " %lu bytes originally requested\n", nbytes); + nbytes = read_size_t(q - 2*SST); + fprintf(stderr, " %" PY_FORMAT_SIZE_T "u bytes originally " + "requested\n", nbytes); /* In case this is nuts, check the leading pad bytes first. */ - fputs(" The 4 pad bytes at p-4 are ", stderr); - if (*(q-4) == FORBIDDENBYTE && - *(q-3) == FORBIDDENBYTE && - *(q-2) == FORBIDDENBYTE && - *(q-1) == FORBIDDENBYTE) { - fputs("FORBIDDENBYTE, as expected.\n", stderr); + fprintf(stderr, " The %d pad bytes at p-%d are ", SST, SST); + ok = 1; + for (i = 1; i <= SST; ++i) { + if (*(q-i) != FORBIDDENBYTE) { + ok = 0; + break; + } } + if (ok) + fputs("FORBIDDENBYTE, as expected.\n", stderr); else { fprintf(stderr, "not all FORBIDDENBYTE (0x%02x):\n", FORBIDDENBYTE); - for (i = 4; i >= 1; --i) { + for (i = SST; i >= 1; --i) { const uchar byte = *(q-i); fprintf(stderr, " at p-%d: 0x%02x", i, byte); if (byte != FORBIDDENBYTE) @@ -1498,17 +1499,20 @@ } tail = q + nbytes; - fprintf(stderr, " The 4 pad bytes at tail=%p are ", tail); - if (tail[0] == FORBIDDENBYTE && - tail[1] == FORBIDDENBYTE && - tail[2] == FORBIDDENBYTE && - tail[3] == FORBIDDENBYTE) { - fputs("FORBIDDENBYTE, as expected.\n", stderr); + fprintf(stderr, " The %d pad bytes at tail=%p are ", SST, tail); + ok = 1; + for (i = 0; i < SST; ++i) { + if (tail[i] != FORBIDDENBYTE) { + ok = 0; + break; + } } + if (ok) + fputs("FORBIDDENBYTE, as expected.\n", stderr); else { fprintf(stderr, "not all FORBIDDENBYTE (0x%02x):\n", FORBIDDENBYTE); - for (i = 0; i < 4; ++i) { + for (i = 0; i < SST; ++i) { const uchar byte = tail[i]; fprintf(stderr, " at tail+%d: 0x%02x", i, byte); @@ -1518,12 +1522,12 @@ } } - serial = read4(tail+4); - fprintf(stderr, " The block was made by call #%lu to " - "debug malloc/realloc.\n", serial); + serial = read_size_t(tail + SST); + fprintf(stderr, " The block was made by call #%" PY_FORMAT_SIZE_T + "u to debug malloc/realloc.\n", serial); if (nbytes > 0) { - int i = 0; + i = 0; fputs(" Data at p:", stderr); /* print up to 8 bytes at the start */ while (q < tail && i < 8) { @@ -1546,12 +1550,12 @@ } } -static ulong -printone(const char* msg, ulong value) +static size_t +printone(const char* msg, size_t value) { int i, k; char buf[100]; - ulong origvalue = value; + size_t origvalue = value; fputs(msg, stderr); for (i = (int)strlen(msg); i < 35; ++i) @@ -1564,8 +1568,8 @@ buf[i--] = '\n'; k = 3; do { - ulong nextvalue = value / 10UL; - uint digit = value - nextvalue * 10UL; + size_t nextvalue = value / 10; + uint digit = (uint)(value - nextvalue * 10); value = nextvalue; buf[i--] = (char)(digit + '0'); --k; @@ -1592,28 +1596,28 @@ uint i; const uint numclasses = SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT; /* # of pools, allocated blocks, and free blocks per class index */ - ulong numpools[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; - ulong numblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; - ulong numfreeblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; + size_t numpools[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; + size_t numblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; + size_t numfreeblocks[SMALL_REQUEST_THRESHOLD >> ALIGNMENT_SHIFT]; /* total # of allocated bytes in used and full pools */ - ulong allocated_bytes = 0; + size_t allocated_bytes = 0; /* total # of available bytes in used pools */ - ulong available_bytes = 0; + size_t available_bytes = 0; /* # of free pools + pools not yet carved out of current arena */ uint numfreepools = 0; /* # of bytes for arena alignment padding */ - ulong arena_alignment = 0; + size_t arena_alignment = 0; /* # of bytes in used and full pools used for pool_headers */ - ulong pool_header_bytes = 0; + size_t pool_header_bytes = 0; /* # of bytes in used and full pools wasted due to quantization, * i.e. the necessarily leftover space at the ends of used and * full pools. */ - ulong quantization = 0; + size_t quantization = 0; /* # of arenas actually allocated. */ - ulong narenas = 0; + size_t narenas = 0; /* running total -- should equal narenas * ARENA_SIZE */ - ulong total; + size_t total; char buf[128]; fprintf(stderr, "Small block threshold = %d, in %u size classes.\n", @@ -1678,15 +1682,18 @@ stderr); for (i = 0; i < numclasses; ++i) { - ulong p = numpools[i]; - ulong b = numblocks[i]; - ulong f = numfreeblocks[i]; + size_t p = numpools[i]; + size_t b = numblocks[i]; + size_t f = numfreeblocks[i]; uint size = INDEX2SIZE(i); if (p == 0) { assert(b == 0 && f == 0); continue; } - fprintf(stderr, "%5u %6u %11lu %15lu %13lu\n", + fprintf(stderr, "%5u %6u " + "%11" PY_FORMAT_SIZE_T "u " + "%15" PY_FORMAT_SIZE_T "u " + "%13" PY_FORMAT_SIZE_T "u\n", i, size, p, b, f); allocated_bytes += b * size; available_bytes += f * size; @@ -1702,7 +1709,8 @@ (void)printone("# arenas allocated current", narenas); PyOS_snprintf(buf, sizeof(buf), - "%lu arenas * %d bytes/arena", narenas, ARENA_SIZE); + "%" PY_FORMAT_SIZE_T "u arenas * %d bytes/arena", + narenas, ARENA_SIZE); (void)printone(buf, narenas * ARENA_SIZE); fputc('\n', stderr); @@ -1712,7 +1720,7 @@ PyOS_snprintf(buf, sizeof(buf), "%u unused pools * %d bytes", numfreepools, POOL_SIZE); - total += printone(buf, (ulong)numfreepools * POOL_SIZE); + total += printone(buf, (size_t)numfreepools * POOL_SIZE); total += printone("# bytes lost to pool headers", pool_header_bytes); total += printone("# bytes lost to quantization", quantization); Modified: python/branches/p3yk/Objects/unicodeobject.c ============================================================================== --- python/branches/p3yk/Objects/unicodeobject.c (original) +++ python/branches/p3yk/Objects/unicodeobject.c Thu Jun 8 17:35:45 2006 @@ -3057,6 +3057,221 @@ return NULL; } +/* Charmap encoding: the lookup table */ + +struct encoding_map{ + PyObject_HEAD + unsigned char level1[32]; + int count2, count3; + unsigned char level23[1]; +}; + +static PyObject* +encoding_map_size(PyObject *obj, PyObject* args) +{ + struct encoding_map *map = (struct encoding_map*)obj; + return PyInt_FromLong(sizeof(*map) - 1 + 16*map->count2 + + 128*map->count3); +} + +static PyMethodDef encoding_map_methods[] = { + {"size", encoding_map_size, METH_NOARGS, + PyDoc_STR("Return the size (in bytes) of this object") }, + { 0 } +}; + +static void +encoding_map_dealloc(PyObject* o) +{ + PyObject_FREE(o); +} + +static PyTypeObject EncodingMapType = { + PyObject_HEAD_INIT(NULL) + 0, /*ob_size*/ + "EncodingMap", /*tp_name*/ + sizeof(struct encoding_map), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + /* methods */ + encoding_map_dealloc, /*tp_dealloc*/ + 0, /*tp_print*/ + 0, /*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_compare*/ + 0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash*/ + 0, /*tp_call*/ + 0, /*tp_str*/ + 0, /*tp_getattro*/ + 0, /*tp_setattro*/ + 0, /*tp_as_buffer*/ + Py_TPFLAGS_DEFAULT, /*tp_flags*/ + 0, /*tp_doc*/ + 0, /*tp_traverse*/ + 0, /*tp_clear*/ + 0, /*tp_richcompare*/ + 0, /*tp_weaklistoffset*/ + 0, /*tp_iter*/ + 0, /*tp_iternext*/ + encoding_map_methods, /*tp_methods*/ + 0, /*tp_members*/ + 0, /*tp_getset*/ + 0, /*tp_base*/ + 0, /*tp_dict*/ + 0, /*tp_descr_get*/ + 0, /*tp_descr_set*/ + 0, /*tp_dictoffset*/ + 0, /*tp_init*/ + 0, /*tp_alloc*/ + 0, /*tp_new*/ + 0, /*tp_free*/ + 0, /*tp_is_gc*/ +}; + +PyObject* +PyUnicode_BuildEncodingMap(PyObject* string) +{ + Py_UNICODE *decode; + PyObject *result; + struct encoding_map *mresult; + int i; + int need_dict = 0; + unsigned char level1[32]; + unsigned char level2[512]; + unsigned char *mlevel1, *mlevel2, *mlevel3; + int count2 = 0, count3 = 0; + + if (!PyUnicode_Check(string) || PyUnicode_GetSize(string) != 256) { + PyErr_BadArgument(); + return NULL; + } + decode = PyUnicode_AS_UNICODE(string); + memset(level1, 0xFF, sizeof level1); + memset(level2, 0xFF, sizeof level2); + + /* If there isn't a one-to-one mapping of NULL to \0, + or if there are non-BMP characters, we need to use + a mapping dictionary. */ + if (decode[0] != 0) + need_dict = 1; + for (i = 1; i < 256; i++) { + int l1, l2; + if (decode[i] == 0 + #ifdef Py_UNICODE_WIDE + || decode[i] > 0xFFFF + #endif + ) { + need_dict = 1; + break; + } + if (decode[i] == 0xFFFE) + /* unmapped character */ + continue; + l1 = decode[i] >> 11; + l2 = decode[i] >> 7; + if (level1[l1] == 0xFF) + level1[l1] = count2++; + if (level2[l2] == 0xFF) + level2[l2] = count3++; + } + + if (count2 >= 0xFF || count3 >= 0xFF) + need_dict = 1; + + if (need_dict) { + PyObject *result = PyDict_New(); + PyObject *key, *value; + if (!result) + return NULL; + for (i = 0; i < 256; i++) { + key = value = NULL; + key = PyInt_FromLong(decode[i]); + value = PyInt_FromLong(i); + if (!key || !value) + goto failed1; + if (PyDict_SetItem(result, key, value) == -1) + goto failed1; + Py_DECREF(key); + Py_DECREF(value); + } + return result; + failed1: + Py_XDECREF(key); + Py_XDECREF(value); + Py_DECREF(result); + return NULL; + } + + /* Create a three-level trie */ + result = PyObject_MALLOC(sizeof(struct encoding_map) + + 16*count2 + 128*count3 - 1); + if (!result) + return PyErr_NoMemory(); + PyObject_Init(result, &EncodingMapType); + mresult = (struct encoding_map*)result; + mresult->count2 = count2; + mresult->count3 = count3; + mlevel1 = mresult->level1; + mlevel2 = mresult->level23; + mlevel3 = mresult->level23 + 16*count2; + memcpy(mlevel1, level1, 32); + memset(mlevel2, 0xFF, 16*count2); + memset(mlevel3, 0, 128*count3); + count3 = 0; + for (i = 1; i < 256; i++) { + int o1, o2, o3, i2, i3; + if (decode[i] == 0xFFFE) + /* unmapped character */ + continue; + o1 = decode[i]>>11; + o2 = (decode[i]>>7) & 0xF; + i2 = 16*mlevel1[o1] + o2; + if (mlevel2[i2] == 0xFF) + mlevel2[i2] = count3++; + o3 = decode[i] & 0x7F; + i3 = 128*mlevel2[i2] + o3; + mlevel3[i3] = i; + } + return result; +} + +static int +encoding_map_lookup(Py_UNICODE c, PyObject *mapping) +{ + struct encoding_map *map = (struct encoding_map*)mapping; + int l1 = c>>11; + int l2 = (c>>7) & 0xF; + int l3 = c & 0x7F; + int i; + +#ifdef Py_UNICODE_WIDE + if (c > 0xFFFF) { + return -1; + } +#endif + if (c == 0) + return 0; + /* level 1*/ + i = map->level1[l1]; + if (i == 0xFF) { + return -1; + } + /* level 2*/ + i = map->level23[16*i+l2]; + if (i == 0xFF) { + return -1; + } + /* level 3 */ + i = map->level23[16*map->count2 + 128*i + l3]; + if (i == 0) { + return -1; + } + return i; +} + /* Lookup the character ch in the mapping. If the character can't be found, Py_None is returned (or NULL, if another error occurred). */ @@ -3102,6 +3317,22 @@ } } +static int +charmapencode_resize(PyObject **outobj, Py_ssize_t *outpos, Py_ssize_t requiredsize) +{ + Py_ssize_t outsize = PyString_GET_SIZE(*outobj); + /* exponentially overallocate to minimize reallocations */ + if (requiredsize < 2*outsize) + requiredsize = 2*outsize; + if (_PyString_Resize(outobj, requiredsize)) { + return 0; + } + return 1; +} + +typedef enum charmapencode_result { + enc_SUCCESS, enc_FAILED, enc_EXCEPTION +}charmapencode_result; /* lookup the character, put the result in the output string and adjust various state variables. Reallocate the output string if not enough space is available. Return a new reference to the object that @@ -3109,51 +3340,59 @@ (in which case no character was written) or NULL, if a reallocation error occurred. The caller must decref the result */ static -PyObject *charmapencode_output(Py_UNICODE c, PyObject *mapping, +charmapencode_result charmapencode_output(Py_UNICODE c, PyObject *mapping, PyObject **outobj, Py_ssize_t *outpos) { - PyObject *rep = charmapencode_lookup(c, mapping); + PyObject *rep; + char *outstart; + Py_ssize_t outsize = PyString_GET_SIZE(*outobj); + + if (mapping->ob_type == &EncodingMapType) { + int res = encoding_map_lookup(c, mapping); + Py_ssize_t requiredsize = *outpos+1; + if (res == -1) + return enc_FAILED; + if (outsizeob_type == &EncodingMapType) { + int res = encoding_map_lookup(p[collendpos], mapping); + if (res != -1) + break; + ++collendpos; + continue; + } + + rep = charmapencode_lookup(p[collendpos], mapping); + if (rep==NULL) return -1; - else if (x!=Py_None) { - Py_DECREF(x); + else if (rep!=Py_None) { + Py_DECREF(rep); break; } - Py_DECREF(x); + Py_DECREF(rep); ++collendpos; } /* cache callback name lookup @@ -3210,15 +3458,13 @@ case 2: /* replace */ for (collpos = collstartpos; collpos0; ++uni2) { x = charmapencode_output(*uni2, mapping, res, respos); - if (x==NULL) { - Py_DECREF(repunicode); + if (x==enc_EXCEPTION) { return -1; } - else if (x==Py_None) { + else if (x==enc_FAILED) { Py_DECREF(repunicode); - Py_DECREF(x); raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason); return -1; } - Py_DECREF(x); } *inpos = newpos; Py_DECREF(repunicode); @@ -3304,22 +3545,20 @@ while (inpos adjust input position */ ++inpos; - Py_DECREF(x); } /* Resize if we allocated to much */ Modified: python/branches/p3yk/PC/_subprocess.c ============================================================================== --- python/branches/p3yk/PC/_subprocess.c (original) +++ python/branches/p3yk/PC/_subprocess.c Thu Jun 8 17:35:45 2006 @@ -250,19 +250,23 @@ getint(PyObject* obj, char* name) { PyObject* value; + int ret; value = PyObject_GetAttrString(obj, name); if (! value) { PyErr_Clear(); /* FIXME: propagate error? */ return 0; } - return (int) PyInt_AsLong(value); + ret = (int) PyInt_AsLong(value); + Py_DECREF(value); + return ret; } static HANDLE gethandle(PyObject* obj, char* name) { sp_handle_object* value; + HANDLE ret; value = (sp_handle_object*) PyObject_GetAttrString(obj, name); if (! value) { @@ -270,8 +274,11 @@ return NULL; } if (value->ob_type != &sp_handle_type) - return NULL; - return value->handle; + ret = NULL; + else + ret = value->handle; + Py_DECREF(value); + return ret; } static PyObject* Modified: python/branches/p3yk/PC/py.ico ============================================================================== Binary files. No diff available. Modified: python/branches/p3yk/PC/pyc.ico ============================================================================== Binary files. No diff available. Modified: python/branches/p3yk/PC/pycon.ico ============================================================================== Binary files. No diff available. Modified: python/branches/p3yk/Parser/node.c ============================================================================== --- python/branches/p3yk/Parser/node.c (original) +++ python/branches/p3yk/Parser/node.c Thu Jun 8 17:35:45 2006 @@ -65,7 +65,7 @@ * reported that, with this scheme, 89% of PyObject_REALLOC calls in * PyNode_AddChild passed 1 for the size, and 9% passed 4. So this usually * wastes very little memory, but is very effective at sidestepping - * platform-realloc disasters on vulnernable platforms. + * platform-realloc disasters on vulnerable platforms. * * Note that this would be straightforward if a node stored its current * capacity. The code is tricky to avoid that. Modified: python/branches/p3yk/Python/compile.c ============================================================================== --- python/branches/p3yk/Python/compile.c (original) +++ python/branches/p3yk/Python/compile.c Thu Jun 8 17:35:45 2006 @@ -2141,7 +2141,7 @@ compiler_if(struct compiler *c, stmt_ty s) { basicblock *end, *next; - + int constant; assert(s->kind == If_kind); end = compiler_new_block(c); if (end == NULL) @@ -2149,15 +2149,27 @@ next = compiler_new_block(c); if (next == NULL) return 0; - VISIT(c, expr, s->v.If.test); - ADDOP_JREL(c, JUMP_IF_FALSE, next); - ADDOP(c, POP_TOP); - VISIT_SEQ(c, stmt, s->v.If.body); - ADDOP_JREL(c, JUMP_FORWARD, end); - compiler_use_next_block(c, next); - ADDOP(c, POP_TOP); - if (s->v.If.orelse) - VISIT_SEQ(c, stmt, s->v.If.orelse); + + constant = expr_constant(s->v.If.test); + /* constant = 0: "if 0" + * constant = 1: "if 1", "if 2", ... + * constant = -1: rest */ + if (constant == 0) { + if (s->v.If.orelse) + VISIT_SEQ(c, stmt, s->v.If.orelse); + } else if (constant == 1) { + VISIT_SEQ(c, stmt, s->v.If.body); + } else { + VISIT(c, expr, s->v.If.test); + ADDOP_JREL(c, JUMP_IF_FALSE, next); + ADDOP(c, POP_TOP); + VISIT_SEQ(c, stmt, s->v.If.body); + ADDOP_JREL(c, JUMP_FORWARD, end); + compiler_use_next_block(c, next); + ADDOP(c, POP_TOP); + if (s->v.If.orelse) + VISIT_SEQ(c, stmt, s->v.If.orelse); + } compiler_use_next_block(c, end); return 1; } @@ -2623,10 +2635,6 @@ if (c->u->u_ste->ste_type != FunctionBlock) return compiler_error(c, "'return' outside function"); if (s->v.Return.value) { - if (c->u->u_ste->ste_generator) { - return compiler_error(c, - "'return' with argument inside generator"); - } VISIT(c, expr, s->v.Return.value); } else @@ -3334,6 +3342,13 @@ return PyObject_IsTrue(e->v.Num.n); case Str_kind: return PyObject_IsTrue(e->v.Str.s); + case Name_kind: + /* __debug__ is not assignable, so we can optimize + * it away in if and while statements */ + if (strcmp(PyString_AS_STRING(e->v.Name.id), + "__debug__") == 0) + return ! Py_OptimizeFlag; + /* fall through */ default: return -1; } Modified: python/branches/p3yk/Python/symtable.c ============================================================================== --- python/branches/p3yk/Python/symtable.c (original) +++ python/branches/p3yk/Python/symtable.c Thu Jun 8 17:35:45 2006 @@ -13,6 +13,8 @@ #define IMPORT_STAR_WARNING "import * only allowed at module level" +#define RETURN_VAL_IN_GENERATOR \ + "'return' with argument inside generator" /* XXX(nnorwitz): change name since static? */ static PySTEntryObject * @@ -66,6 +68,7 @@ ste->ste_nested = 1; ste->ste_child_free = 0; ste->ste_generator = 0; + ste->ste_returns_value = 0; if (PyDict_SetItem(st->st_symbols, ste->ste_id, (PyObject *)ste) < 0) goto fail; @@ -944,8 +947,17 @@ break; } case Return_kind: - if (s->v.Return.value) + if (s->v.Return.value) { VISIT(st, expr, s->v.Return.value); + st->st_cur->ste_returns_value = 1; + if (st->st_cur->ste_generator) { + PyErr_SetString(PyExc_SyntaxError, + RETURN_VAL_IN_GENERATOR); + PyErr_SyntaxLocation(st->st_filename, + s->lineno); + return 0; + } + } break; case Delete_kind: VISIT_SEQ(st, expr, s->v.Delete.targets); @@ -1136,6 +1148,13 @@ if (e->v.Yield.value) VISIT(st, expr, e->v.Yield.value); st->st_cur->ste_generator = 1; + if (st->st_cur->ste_returns_value) { + PyErr_SetString(PyExc_SyntaxError, + RETURN_VAL_IN_GENERATOR); + PyErr_SyntaxLocation(st->st_filename, + e->lineno); + return 0; + } break; case Compare_kind: VISIT(st, expr, e->v.Compare.left); Modified: python/branches/p3yk/Python/sysmodule.c ============================================================================== --- python/branches/p3yk/Python/sysmodule.c (original) +++ python/branches/p3yk/Python/sysmodule.c Thu Jun 8 17:35:45 2006 @@ -1027,7 +1027,7 @@ PyObject *sysin, *sysout, *syserr; char *s; #ifdef MS_WINDOWS - char buf[10]; + char buf[128]; #endif m = Py_InitModule3("sys", sys_methods, sys_doc); Modified: python/branches/p3yk/Python/thread.c ============================================================================== --- python/branches/p3yk/Python/thread.c (original) +++ python/branches/p3yk/Python/thread.c Thu Jun 8 17:35:45 2006 @@ -75,7 +75,8 @@ static void PyThread__init_thread(void); /* Forward */ -void PyThread_init_thread(void) +void +PyThread_init_thread(void) { #ifdef Py_DEBUG char *p = getenv("THREADDEBUG"); Modified: python/branches/p3yk/Python/thread_nt.h ============================================================================== --- python/branches/p3yk/Python/thread_nt.h (original) +++ python/branches/p3yk/Python/thread_nt.h Thu Jun 8 17:35:45 2006 @@ -16,7 +16,8 @@ typedef PVOID WINAPI interlocked_cmp_xchg_t(PVOID *dest, PVOID exc, PVOID comperand) ; /* Sorry mate, but we haven't got InterlockedCompareExchange in Win95! */ -static PVOID WINAPI interlocked_cmp_xchg(PVOID *dest, PVOID exc, PVOID comperand) +static PVOID WINAPI +interlocked_cmp_xchg(PVOID *dest, PVOID exc, PVOID comperand) { static LONG spinlock = 0 ; PVOID result ; @@ -54,8 +55,10 @@ return result ; } ; -static interlocked_cmp_xchg_t *ixchg ; -BOOL InitializeNonRecursiveMutex(PNRMUTEX mutex) +static interlocked_cmp_xchg_t *ixchg; + +BOOL +InitializeNonRecursiveMutex(PNRMUTEX mutex) { if (!ixchg) { @@ -76,14 +79,16 @@ #endif #define InterlockedCompareExchange(dest,exchange,comperand) (ixchg((dest), (exchange), (comperand))) -VOID DeleteNonRecursiveMutex(PNRMUTEX mutex) +VOID +DeleteNonRecursiveMutex(PNRMUTEX mutex) { /* No in-use check */ CloseHandle(mutex->hevent) ; mutex->hevent = NULL ; /* Just in case */ } -DWORD EnterNonRecursiveMutex(PNRMUTEX mutex, BOOL wait) +DWORD +EnterNonRecursiveMutex(PNRMUTEX mutex, BOOL wait) { /* Assume that the thread waits successfully */ DWORD ret ; @@ -104,7 +109,8 @@ return ret ; } -BOOL LeaveNonRecursiveMutex(PNRMUTEX mutex) +BOOL +LeaveNonRecursiveMutex(PNRMUTEX mutex) { /* We don't own the mutex */ mutex->thread_id = 0 ; @@ -113,7 +119,8 @@ SetEvent(mutex->hevent) ; /* Other threads are waiting, wake one on them up */ } -PNRMUTEX AllocNonRecursiveMutex(void) +PNRMUTEX +AllocNonRecursiveMutex(void) { PNRMUTEX mutex = (PNRMUTEX)malloc(sizeof(NRMUTEX)) ; if (mutex && !InitializeNonRecursiveMutex(mutex)) @@ -124,7 +131,8 @@ return mutex ; } -void FreeNonRecursiveMutex(PNRMUTEX mutex) +void +FreeNonRecursiveMutex(PNRMUTEX mutex) { if (mutex) { @@ -138,7 +146,8 @@ /* * Initialization of the C package, should not be needed. */ -static void PyThread__init_thread(void) +static void +PyThread__init_thread(void) { } @@ -209,7 +218,8 @@ * Return the thread Id instead of an handle. The Id is said to uniquely identify the * thread in the system */ -long PyThread_get_thread_ident(void) +long +PyThread_get_thread_ident(void) { if (!initialized) PyThread_init_thread(); @@ -217,7 +227,8 @@ return GetCurrentThreadId(); } -static void do_PyThread_exit_thread(int no_cleanup) +static void +do_PyThread_exit_thread(int no_cleanup) { dprintf(("%ld: PyThread_exit_thread called\n", PyThread_get_thread_ident())); if (!initialized) @@ -228,18 +239,21 @@ _endthread(); } -void PyThread_exit_thread(void) +void +PyThread_exit_thread(void) { do_PyThread_exit_thread(0); } -void PyThread__exit_thread(void) +void +PyThread__exit_thread(void) { do_PyThread_exit_thread(1); } #ifndef NO_EXIT_PROG -static void do_PyThread_exit_prog(int status, int no_cleanup) +static void +do_PyThread_exit_prog(int status, int no_cleanup) { dprintf(("PyThread_exit_prog(%d) called\n", status)); if (!initialized) @@ -249,12 +263,14 @@ exit(status); } -void PyThread_exit_prog(int status) +void +PyThread_exit_prog(int status) { do_PyThread_exit_prog(status, 0); } -void PyThread__exit_prog(int status) +void +PyThread__exit_prog(int status) { do_PyThread_exit_prog(status, 1); } @@ -265,7 +281,8 @@ * I [Dag] tried to implement it with mutex but I could find a way to * tell whether a thread already own the lock or not. */ -PyThread_type_lock PyThread_allocate_lock(void) +PyThread_type_lock +PyThread_allocate_lock(void) { PNRMUTEX aLock; @@ -280,7 +297,8 @@ return (PyThread_type_lock) aLock; } -void PyThread_free_lock(PyThread_type_lock aLock) +void +PyThread_free_lock(PyThread_type_lock aLock) { dprintf(("%ld: PyThread_free_lock(%p) called\n", PyThread_get_thread_ident(),aLock)); @@ -293,7 +311,8 @@ * and 0 if the lock was not acquired. This means a 0 is returned * if the lock has already been acquired by this thread! */ -int PyThread_acquire_lock(PyThread_type_lock aLock, int waitflag) +int +PyThread_acquire_lock(PyThread_type_lock aLock, int waitflag) { int success ; @@ -306,7 +325,8 @@ return success; } -void PyThread_release_lock(PyThread_type_lock aLock) +void +PyThread_release_lock(PyThread_type_lock aLock) { dprintf(("%ld: PyThread_release_lock(%p) called\n", PyThread_get_thread_ident(),aLock)); Modified: python/branches/p3yk/Python/thread_os2.h ============================================================================== --- python/branches/p3yk/Python/thread_os2.h (original) +++ python/branches/p3yk/Python/thread_os2.h Thu Jun 8 17:35:45 2006 @@ -238,7 +238,8 @@ return 1; } -void PyThread_release_lock(PyThread_type_lock aLock) +void +PyThread_release_lock(PyThread_type_lock aLock) { #if !defined(PYCC_GCC) type_os2_lock lock = (type_os2_lock)aLock; Modified: python/branches/p3yk/Tools/unicode/Makefile ============================================================================== --- python/branches/p3yk/Tools/unicode/Makefile (original) +++ python/branches/p3yk/Tools/unicode/Makefile Thu Jun 8 17:35:45 2006 @@ -78,7 +78,7 @@ ### Cleanup clean: - $(RM) build/* + $(RM) -f build/* distclean: clean $(RM) -rf MAPPINGS/ Modified: python/branches/p3yk/Tools/unicode/gencodec.py ============================================================================== --- python/branches/p3yk/Tools/unicode/gencodec.py (original) +++ python/branches/p3yk/Tools/unicode/gencodec.py Thu Jun 8 17:35:45 2006 @@ -270,6 +270,11 @@ comments=comments, precisions=(4, 2)) + if decoding_table_code: + suffix = 'table' + else: + suffix = 'map' + l = [ '''\ """ Python Character Mapping Codec %s generated from '%s' with gencodec.py. @@ -283,30 +288,20 @@ class Codec(codecs.Codec): def encode(self,input,errors='strict'): - return codecs.charmap_encode(input,errors,encoding_map) - - def decode(self,input,errors='strict'):''' % (encodingname, name) - ] - if decoding_table_code: - l.append('''\ - return codecs.charmap_decode(input,errors,decoding_table)''') - else: - l.append('''\ - return codecs.charmap_decode(input,errors,decoding_map)''') + return codecs.charmap_encode(input,errors,encoding_%s) - l.append(''' + def decode(self,input,errors='strict'): + return codecs.charmap_decode(input,errors,decoding_%s) +''' % (encodingname, name, suffix, suffix)] + l.append('''\ class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.charmap_encode(input,self.errors,encoding_map)[0] + return codecs.charmap_encode(input,self.errors,encoding_%s)[0] class IncrementalDecoder(codecs.IncrementalDecoder): - def decode(self, input, final=False):''') - if decoding_table_code: - l.append('''\ - return codecs.charmap_decode(input,self.errors,decoding_table)[0]''') - else: - l.append('''\ - return codecs.charmap_decode(input,self.errors,decoding_map)[0]''') + def decode(self, input, final=False): + return codecs.charmap_decode(input,self.errors,decoding_%s)[0]''' % + (suffix, suffix)) l.append(''' class StreamWriter(Codec,codecs.StreamWriter): @@ -319,13 +314,13 @@ def getregentry(): return codecs.CodecInfo( - Codec().encode, - Codec().decode, name=%r, - streamwriter=StreamWriter, - streamreader=StreamReader, + encode=Codec().encode, + decode=Codec().decode, incrementalencoder=IncrementalEncoder, incrementaldecoder=IncrementalDecoder, + streamreader=StreamReader, + streamwriter=StreamWriter, ) ''' % encodingname.replace('_', '-')) @@ -342,10 +337,16 @@ l.extend(decoding_table_code) # Add encoding map - l.append(''' + if decoding_table_code: + l.append(''' +### Encoding table +encoding_table=codecs.charmap_build(decoding_table) +''') + else: + l.append(''' ### Encoding Map ''') - l.extend(encoding_map_code) + l.extend(encoding_map_code) # Final new-line l.append('') Modified: python/branches/p3yk/Tools/webchecker/webchecker.py ============================================================================== --- python/branches/p3yk/Tools/webchecker/webchecker.py (original) +++ python/branches/p3yk/Tools/webchecker/webchecker.py Thu Jun 8 17:35:45 2006 @@ -784,7 +784,7 @@ self.url = url sgmllib.SGMLParser.__init__(self) - def check_name_id( self, attributes ): + def check_name_id(self, attributes): """ Check the name or id attributes on an element. """ # We must rescue the NAME or id (name is deprecated in XHTML) @@ -799,7 +799,7 @@ else: self.names.append(value) break - def unknown_starttag( self, tag, attributes ): + def unknown_starttag(self, tag, attributes): """ In XHTML, you can have id attributes on any element. """ self.check_name_id(attributes) Modified: python/branches/p3yk/configure ============================================================================== --- python/branches/p3yk/configure (original) +++ python/branches/p3yk/configure Thu Jun 8 17:35:45 2006 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 46491 . +# From configure.in Revision: 46753 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59e for python 3.0. # @@ -648,6 +648,9 @@ PYTHONFRAMEWORKINSTALLDIR FRAMEWORKINSTALLFIRST FRAMEWORKINSTALLLAST +FRAMEWORKALTINSTALLFIRST +FRAMEWORKALTINSTALLLAST +FRAMEWORKUNIXTOOLSPREFIX MACHDEP SGI_ABI EXTRAPLATDIR @@ -1874,6 +1877,13 @@ PYTHONFRAMEWORKINSTALLDIR= FRAMEWORKINSTALLFIRST= FRAMEWORKINSTALLLAST= + FRAMEWORKALTINSTALLFIRST= + FRAMEWORKALTINSTALLLAST= + if test "x${prefix}" = "xNONE"; then + FRAMEWORKUNIXTOOLSPREFIX="${ac_default_prefix}" + else + FRAMEWORKUNIXTOOLSPREFIX="${prefix}" + fi enable_framework= ;; *) @@ -1883,15 +1893,22 @@ PYTHONFRAMEWORKINSTALLDIR=$PYTHONFRAMEWORKPREFIX/$PYTHONFRAMEWORKDIR FRAMEWORKINSTALLFIRST="frameworkinstallstructure" FRAMEWORKINSTALLLAST="frameworkinstallmaclib frameworkinstallapps frameworkinstallunixtools" + FRAMEWORKALTINSTALLFIRST="${FRAMEWORKINSTALLFIRST} bininstall maninstall" + FRAMEWORKALTINSTALLLAST="frameworkinstallmaclib frameworkinstallapps frameworkaltinstallunixtools" + if test "x${prefix}" = "xNONE" ; then + FRAMEWORKUNIXTOOLSPREFIX="${ac_default_prefix}" + else + FRAMEWORKUNIXTOOLSPREFIX="${prefix}" + fi prefix=$PYTHONFRAMEWORKINSTALLDIR/Versions/$VERSION # Add makefiles for Mac specific code to the list of output # files: - ac_config_files="$ac_config_files Mac/OSX/Makefile" + ac_config_files="$ac_config_files Mac/Makefile" - ac_config_files="$ac_config_files Mac/OSX/PythonLauncher/Makefile" + ac_config_files="$ac_config_files Mac/PythonLauncher/Makefile" - ac_config_files="$ac_config_files Mac/OSX/IDLE/Makefile" + ac_config_files="$ac_config_files Mac/IDLE/Makefile" esac @@ -1903,6 +1920,13 @@ PYTHONFRAMEWORKINSTALLDIR= FRAMEWORKINSTALLFIRST= FRAMEWORKINSTALLLAST= + FRAMEWORKALTINSTALLFIRST= + FRAMEWORKALTINSTALLLAST= + if test "x${prefix}" = "xNONE" ; then + FRAMEWORKUNIXTOOLSPREFIX="${ac_default_prefix}" + else + FRAMEWORKUNIXTOOLSPREFIX="${prefix}" + fi enable_framework= fi @@ -1914,6 +1938,9 @@ + + + ##AC_ARG_WITH(dyld, ## AC_HELP_STRING(--with-dyld, ## Use (OpenStep|Rhapsody) dynamic linker)) @@ -26168,9 +26195,9 @@ do case $ac_config_target in "pyconfig.h") CONFIG_HEADERS="$CONFIG_HEADERS pyconfig.h" ;; - "Mac/OSX/Makefile") CONFIG_FILES="$CONFIG_FILES Mac/OSX/Makefile" ;; - "Mac/OSX/PythonLauncher/Makefile") CONFIG_FILES="$CONFIG_FILES Mac/OSX/PythonLauncher/Makefile" ;; - "Mac/OSX/IDLE/Makefile") CONFIG_FILES="$CONFIG_FILES Mac/OSX/IDLE/Makefile" ;; + "Mac/Makefile") CONFIG_FILES="$CONFIG_FILES Mac/Makefile" ;; + "Mac/PythonLauncher/Makefile") CONFIG_FILES="$CONFIG_FILES Mac/PythonLauncher/Makefile" ;; + "Mac/IDLE/Makefile") CONFIG_FILES="$CONFIG_FILES Mac/IDLE/Makefile" ;; "Makefile.pre") CONFIG_FILES="$CONFIG_FILES Makefile.pre" ;; "Modules/Setup.config") CONFIG_FILES="$CONFIG_FILES Modules/Setup.config" ;; @@ -26281,6 +26308,9 @@ PYTHONFRAMEWORKINSTALLDIR!$PYTHONFRAMEWORKINSTALLDIR$ac_delim FRAMEWORKINSTALLFIRST!$FRAMEWORKINSTALLFIRST$ac_delim FRAMEWORKINSTALLLAST!$FRAMEWORKINSTALLLAST$ac_delim +FRAMEWORKALTINSTALLFIRST!$FRAMEWORKALTINSTALLFIRST$ac_delim +FRAMEWORKALTINSTALLLAST!$FRAMEWORKALTINSTALLLAST$ac_delim +FRAMEWORKUNIXTOOLSPREFIX!$FRAMEWORKUNIXTOOLSPREFIX$ac_delim MACHDEP!$MACHDEP$ac_delim SGI_ABI!$SGI_ABI$ac_delim EXTRAPLATDIR!$EXTRAPLATDIR$ac_delim @@ -26328,9 +26358,6 @@ SHLIBS!$SHLIBS$ac_delim USE_SIGNAL_MODULE!$USE_SIGNAL_MODULE$ac_delim SIGNAL_OBJS!$SIGNAL_OBJS$ac_delim -USE_THREAD_MODULE!$USE_THREAD_MODULE$ac_delim -LDLAST!$LDLAST$ac_delim -THREADOBJ!$THREADOBJ$ac_delim _ACEOF if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 97; then @@ -26372,6 +26399,9 @@ ac_delim='%!_!# ' for ac_last_try in false false false false false :; do cat >conf$$subs.sed <<_ACEOF +USE_THREAD_MODULE!$USE_THREAD_MODULE$ac_delim +LDLAST!$LDLAST$ac_delim +THREADOBJ!$THREADOBJ$ac_delim DLINCLDIR!$DLINCLDIR$ac_delim DYNLOADFILE!$DYNLOADFILE$ac_delim MACHDEP_OBJS!$MACHDEP_OBJS$ac_delim @@ -26390,7 +26420,7 @@ LTLIBOBJS!$LTLIBOBJS$ac_delim _ACEOF - if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 16; then + if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 19; then break elif $ac_last_try; then { { echo "$as_me:$LINENO: error: could not make $CONFIG_STATUS" >&5 Modified: python/branches/p3yk/configure.in ============================================================================== --- python/branches/p3yk/configure.in (original) +++ python/branches/p3yk/configure.in Thu Jun 8 17:35:45 2006 @@ -99,6 +99,13 @@ PYTHONFRAMEWORKINSTALLDIR= FRAMEWORKINSTALLFIRST= FRAMEWORKINSTALLLAST= + FRAMEWORKALTINSTALLFIRST= + FRAMEWORKALTINSTALLLAST= + if test "x${prefix}" = "xNONE"; then + FRAMEWORKUNIXTOOLSPREFIX="${ac_default_prefix}" + else + FRAMEWORKUNIXTOOLSPREFIX="${prefix}" + fi enable_framework= ;; *) @@ -108,13 +115,20 @@ PYTHONFRAMEWORKINSTALLDIR=$PYTHONFRAMEWORKPREFIX/$PYTHONFRAMEWORKDIR FRAMEWORKINSTALLFIRST="frameworkinstallstructure" FRAMEWORKINSTALLLAST="frameworkinstallmaclib frameworkinstallapps frameworkinstallunixtools" + FRAMEWORKALTINSTALLFIRST="${FRAMEWORKINSTALLFIRST} bininstall maninstall" + FRAMEWORKALTINSTALLLAST="frameworkinstallmaclib frameworkinstallapps frameworkaltinstallunixtools" + if test "x${prefix}" = "xNONE" ; then + FRAMEWORKUNIXTOOLSPREFIX="${ac_default_prefix}" + else + FRAMEWORKUNIXTOOLSPREFIX="${prefix}" + fi prefix=$PYTHONFRAMEWORKINSTALLDIR/Versions/$VERSION # Add makefiles for Mac specific code to the list of output # files: - AC_CONFIG_FILES(Mac/OSX/Makefile) - AC_CONFIG_FILES(Mac/OSX/PythonLauncher/Makefile) - AC_CONFIG_FILES(Mac/OSX/IDLE/Makefile) + AC_CONFIG_FILES(Mac/Makefile) + AC_CONFIG_FILES(Mac/PythonLauncher/Makefile) + AC_CONFIG_FILES(Mac/IDLE/Makefile) esac ],[ PYTHONFRAMEWORK= @@ -123,6 +137,13 @@ PYTHONFRAMEWORKINSTALLDIR= FRAMEWORKINSTALLFIRST= FRAMEWORKINSTALLLAST= + FRAMEWORKALTINSTALLFIRST= + FRAMEWORKALTINSTALLLAST= + if test "x${prefix}" = "xNONE" ; then + FRAMEWORKUNIXTOOLSPREFIX="${ac_default_prefix}" + else + FRAMEWORKUNIXTOOLSPREFIX="${prefix}" + fi enable_framework= ]) AC_SUBST(PYTHONFRAMEWORK) @@ -131,6 +152,9 @@ AC_SUBST(PYTHONFRAMEWORKINSTALLDIR) AC_SUBST(FRAMEWORKINSTALLFIRST) AC_SUBST(FRAMEWORKINSTALLLAST) +AC_SUBST(FRAMEWORKALTINSTALLFIRST) +AC_SUBST(FRAMEWORKALTINSTALLLAST) +AC_SUBST(FRAMEWORKUNIXTOOLSPREFIX) ##AC_ARG_WITH(dyld, ## AC_HELP_STRING(--with-dyld, Modified: python/branches/p3yk/setup.py ============================================================================== --- python/branches/p3yk/setup.py (original) +++ python/branches/p3yk/setup.py Thu Jun 8 17:35:45 2006 @@ -582,7 +582,9 @@ # The _md5 module implements the RSA Data Security, Inc. MD5 # Message-Digest Algorithm, described in RFC 1321. The # necessary files md5.c and md5.h are included here. - exts.append( Extension('_md5', ['md5module.c', 'md5.c']) ) + exts.append( Extension('_md5', + sources = ['md5module.c', 'md5.c'], + depends = ['md5.h']) ) if (openssl_ver < 0x00908000): # OpenSSL doesn't do these until 0.9.8 so we'll bring our own hash @@ -1095,29 +1097,6 @@ extra_link_args=['-framework', 'QuickTime', '-framework', 'Carbon']) ) - # As there is no standardized place (yet) to put - # user-installed Mac libraries on OSX, we search for "waste" - # in parent directories of the Python source tree. You - # should put a symlink to your Waste installation in the - # same folder as your python source tree. Or modify the - # next few lines:-) - waste_incs = find_file("WASTE.h", [], - ['../'*n + 'waste/C_C++ Headers' for n in (0,1,2,3,4)]) - waste_libs = find_library_file(self.compiler, "WASTE", [], - ["../"*n + "waste/Static Libraries" for n in (0,1,2,3,4)]) - if waste_incs != None and waste_libs != None: - exts.append( Extension('waste', - ['waste/wastemodule.c'] + [ - os.path.join(srcdir, d) for d in - 'Mac/Wastemods/WEObjectHandlers.c', - 'Mac/Wastemods/WETabHooks.c', - 'Mac/Wastemods/WETabs.c' - ], - include_dirs = waste_incs + [os.path.join(srcdir, 'Mac/Wastemods')], - library_dirs = waste_libs, - libraries = ['WASTE'], - extra_link_args = ['-framework', 'Carbon'], - ) ) self.extensions.extend(exts) From buildbot at python.org Thu Jun 8 17:37:38 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 15:37:38 +0000 Subject: [Python-checkins] buildbot warnings in x86 W2k trunk Message-ID: <20060608153738.D8FEB1E400E@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/990 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 8 18:07:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 16:07:30 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060608160730.844341E4004@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/601 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 8 18:09:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 16:09:55 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060608160955.CCB751E4004@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/933 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 8 18:23:05 2006 From: python-checkins at python.org (brett.cannon) Date: Thu, 8 Jun 2006 18:23:05 +0200 (CEST) Subject: [Python-checkins] r46755 - in python/trunk: Misc/NEWS Modules/binascii.c Message-ID: <20060608162305.81EB41E4004@bag.python.org> Author: brett.cannon Date: Thu Jun 8 18:23:04 2006 New Revision: 46755 Modified: python/trunk/Misc/NEWS python/trunk/Modules/binascii.c Log: Make binascii.hexlify() use s# for its arguments instead of t# to actually match its documentation stating it accepts any read-only buffer. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 8 18:23:04 2006 @@ -75,6 +75,9 @@ Extension Modules ----------------- +- Change binascii.hexlify to accept a read-only buffer instead of only a char + buffer and actually follow its documentation. + - Fixed a potentially invalid memory access of CJKCodecs' shift-jis decoder. - Patch #1478788 (modified version): The functional extension module has Modified: python/trunk/Modules/binascii.c ============================================================================== --- python/trunk/Modules/binascii.c (original) +++ python/trunk/Modules/binascii.c Thu Jun 8 18:23:04 2006 @@ -909,7 +909,7 @@ char* retbuf; Py_ssize_t i, j; - if (!PyArg_ParseTuple(args, "t#:b2a_hex", &argbuf, &arglen)) + if (!PyArg_ParseTuple(args, "s#:b2a_hex", &argbuf, &arglen)) return NULL; retval = PyString_FromStringAndSize(NULL, arglen*2); From python-checkins at python.org Thu Jun 8 18:24:57 2006 From: python-checkins at python.org (brett.cannon) Date: Thu, 8 Jun 2006 18:24:57 +0200 (CEST) Subject: [Python-checkins] r46756 - in python/branches/release24-maint: Misc/NEWS Modules/binascii.c Message-ID: <20060608162457.6B9271E4004@bag.python.org> Author: brett.cannon Date: Thu Jun 8 18:24:56 2006 New Revision: 46756 Modified: python/branches/release24-maint/Misc/NEWS python/branches/release24-maint/Modules/binascii.c Log: Backport change to binascii.hexlify() to use s# for its arguments instead of t# in order to match its documentation. Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Thu Jun 8 18:24:56 2006 @@ -19,6 +19,9 @@ Extension Modules ----------------- +- Change binascii.hexlify() to accept any read-only buffer and not just a char + buffer. + - Fixed a potentially invalid memory access of CJKCodecs' shift-jis decoder. - Calling Tk_Init twice is refused if the first call failed as that Modified: python/branches/release24-maint/Modules/binascii.c ============================================================================== --- python/branches/release24-maint/Modules/binascii.c (original) +++ python/branches/release24-maint/Modules/binascii.c Thu Jun 8 18:24:56 2006 @@ -908,7 +908,7 @@ char* retbuf; int i, j; - if (!PyArg_ParseTuple(args, "t#:b2a_hex", &argbuf, &arglen)) + if (!PyArg_ParseTuple(args, "s#:b2a_hex", &argbuf, &arglen)) return NULL; retval = PyString_FromStringAndSize(NULL, arglen*2); From buildbot at python.org Thu Jun 8 18:59:26 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 16:59:26 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060608165926.21C5F1E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/307 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 8 19:00:49 2006 From: python-checkins at python.org (brett.cannon) Date: Thu, 8 Jun 2006 19:00:49 +0200 (CEST) Subject: [Python-checkins] r46757 - in python/trunk: Lib/test/test_types.py Misc/NEWS Modules/arraymodule.c Objects/bufferobject.c Message-ID: <20060608170049.6A6411E4011@bag.python.org> Author: brett.cannon Date: Thu Jun 8 19:00:45 2006 New Revision: 46757 Modified: python/trunk/Lib/test/test_types.py python/trunk/Misc/NEWS python/trunk/Modules/arraymodule.c python/trunk/Objects/bufferobject.c Log: Buffer objects would return the read or write buffer for a wrapped object when the char buffer was requested. Now it actually returns the char buffer if available or raises a TypeError if it isn't (as is raised for the other buffer types if they are not present but requested). Not a backport candidate since it does change semantics of the buffer object (although it could be argued this is enough of a bug to bother backporting). Modified: python/trunk/Lib/test/test_types.py ============================================================================== --- python/trunk/Lib/test/test_types.py (original) +++ python/trunk/Lib/test/test_types.py Thu Jun 8 19:00:45 2006 @@ -276,3 +276,10 @@ try: a[0:1] = 'g' except TypeError: pass else: raise TestFailed, "buffer slice assignment should raise TypeError" + +# array.array() returns an object that does not implement a char buffer, +# something which int() uses for conversion. +import array +try: int(buffer(array.array('c'))) +except TypeError :pass +else: raise TestFailed, "char buffer (at C level) not working" Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 8 19:00:45 2006 @@ -12,6 +12,12 @@ Core and builtins ----------------- +- Buffer objects, at the C level, never used the char buffer + implementation even when the char buffer for the wrapped object was + explicitly requested (originally returned the read or write buffer). + Now a TypeError is raised if the char buffer is not present but is + requested. + - Patch #1346214: Statements like "if 0: suite" are now again optimized away like they were in Python 2.4. Modified: python/trunk/Modules/arraymodule.c ============================================================================== --- python/trunk/Modules/arraymodule.c (original) +++ python/trunk/Modules/arraymodule.c Thu Jun 8 19:00:45 2006 @@ -1787,6 +1787,7 @@ (readbufferproc)array_buffer_getreadbuf, (writebufferproc)array_buffer_getwritebuf, (segcountproc)array_buffer_getsegcount, + NULL, }; static PyObject * Modified: python/trunk/Objects/bufferobject.c ============================================================================== --- python/trunk/Objects/bufferobject.c (original) +++ python/trunk/Objects/bufferobject.c Thu Jun 8 19:00:45 2006 @@ -15,8 +15,16 @@ } PyBufferObject; +enum buffer_t { + READBUFFER, + WRITEBUFFER, + CHARBUFFER, + ANY_BUFFER, +}; + static int -get_buf(PyBufferObject *self, void **ptr, Py_ssize_t *size) +get_buf(PyBufferObject *self, void **ptr, Py_ssize_t *size, + enum buffer_t buffer_type) { if (self->b_base == NULL) { assert (ptr != NULL); @@ -32,10 +40,42 @@ "single-segment buffer object expected"); return 0; } - if (self->b_readonly) - proc = bp->bf_getreadbuffer; - else - proc = (readbufferproc)bp->bf_getwritebuffer; + if ((buffer_type == READBUFFER) || + ((buffer_type == ANY_BUFFER) && self->b_readonly)) + proc = bp->bf_getreadbuffer; + else if ((buffer_type == WRITEBUFFER) || + (buffer_type == ANY_BUFFER)) + proc = (readbufferproc)bp->bf_getwritebuffer; + else if (buffer_type == CHARBUFFER) { + if (!PyType_HasFeature(self->ob_type, + Py_TPFLAGS_HAVE_GETCHARBUFFER)) { + PyErr_SetString(PyExc_TypeError, + "Py_TPFLAGS_HAVE_GETCHARBUFFER needed"); + return 0; + } + proc = (readbufferproc)bp->bf_getcharbuffer; + } + if (!proc) { + char *buffer_type_name; + switch (buffer_type) { + case READBUFFER: + buffer_type_name = "read"; + break; + case WRITEBUFFER: + buffer_type_name = "write"; + break; + case CHARBUFFER: + buffer_type_name = "char"; + break; + default: + buffer_type_name = "no"; + break; + } + PyErr_Format(PyExc_TypeError, + "%s buffer type not available", + buffer_type_name); + return 0; + } if ((count = (*proc)(self->b_base, 0, ptr)) < 0) return 0; /* apply constraints to the start/end */ @@ -224,9 +264,9 @@ Py_ssize_t len_self, len_other, min_len; int cmp; - if (!get_buf(self, &p1, &len_self)) + if (!get_buf(self, &p1, &len_self, ANY_BUFFER)) return -1; - if (!get_buf(other, &p2, &len_other)) + if (!get_buf(other, &p2, &len_other, ANY_BUFFER)) return -1; min_len = (len_self < len_other) ? len_self : len_other; if (min_len > 0) { @@ -284,7 +324,7 @@ return -1; } - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return -1; p = (unsigned char *) ptr; len = size; @@ -303,7 +343,7 @@ { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; return PyString_FromStringAndSize((const char *)ptr, size); } @@ -315,7 +355,7 @@ { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return -1; return size; } @@ -344,7 +384,7 @@ return NULL; } - if (!get_buf(self, &ptr1, &size)) + if (!get_buf(self, &ptr1, &size, ANY_BUFFER)) return NULL; /* optimize special case */ @@ -380,7 +420,7 @@ if ( count < 0 ) count = 0; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; ob = PyString_FromStringAndSize(NULL, size * count); if ( ob == NULL ) @@ -404,7 +444,7 @@ { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; if ( idx < 0 || idx >= size ) { PyErr_SetString(PyExc_IndexError, "buffer index out of range"); @@ -418,7 +458,7 @@ { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; if ( left < 0 ) left = 0; @@ -446,7 +486,7 @@ return -1; } - if (!get_buf(self, &ptr1, &size)) + if (!get_buf(self, &ptr1, &size, ANY_BUFFER)) return -1; if (idx < 0 || idx >= size) { @@ -513,7 +553,7 @@ "single-segment buffer object expected"); return -1; } - if (!get_buf(self, &ptr1, &size)) + if (!get_buf(self, &ptr1, &size, ANY_BUFFER)) return -1; if ( (count = (*pb->bf_getreadbuffer)(other, 0, &ptr2)) < 0 ) return -1; @@ -552,7 +592,7 @@ "accessing non-existent buffer segment"); return -1; } - if (!get_buf(self, pp, &size)) + if (!get_buf(self, pp, &size, READBUFFER)) return -1; return size; } @@ -560,12 +600,22 @@ static Py_ssize_t buffer_getwritebuf(PyBufferObject *self, Py_ssize_t idx, void **pp) { + Py_ssize_t size; + if ( self->b_readonly ) { PyErr_SetString(PyExc_TypeError, "buffer is read-only"); return -1; } - return buffer_getreadbuf(self, idx, pp); + + if ( idx != 0 ) { + PyErr_SetString(PyExc_SystemError, + "accessing non-existent buffer segment"); + return -1; + } + if (!get_buf(self, pp, &size, WRITEBUFFER)) + return -1; + return size; } static Py_ssize_t @@ -573,7 +623,7 @@ { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return -1; if (lenp) *lenp = size; @@ -590,13 +640,12 @@ "accessing non-existent buffer segment"); return -1; } - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, CHARBUFFER)) return -1; *pp = (const char *)ptr; return size; } - static PySequenceMethods buffer_as_sequence = { (lenfunc)buffer_length, /*sq_length*/ (binaryfunc)buffer_concat, /*sq_concat*/ @@ -635,7 +684,7 @@ PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ &buffer_as_buffer, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT, /* tp_flags */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GETCHARBUFFER, /* tp_flags */ buffer_doc, /* tp_doc */ 0, /* tp_traverse */ 0, /* tp_clear */ From buildbot at python.org Thu Jun 8 19:08:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 17:08:17 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper 2.4 Message-ID: <20060608170817.DC5171E4004@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%25202.4/builds/85 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 8 19:32:09 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 17:32:09 +0000 Subject: [Python-checkins] buildbot warnings in x86 W2k trunk Message-ID: <20060608173209.28BE21E400F@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/992 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 8 22:18:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 20:18:20 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060608201820.705491E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/47 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 8 22:19:15 2006 From: python-checkins at python.org (ronald.oussoren) Date: Thu, 8 Jun 2006 22:19:15 +0200 (CEST) Subject: [Python-checkins] r46758 - python/branches/release24-maint/Lib/plat-mac/argvemulator.py Message-ID: <20060608201915.DE4621E4004@bag.python.org> Author: ronald.oussoren Date: Thu Jun 8 22:19:15 2006 New Revision: 46758 Modified: python/branches/release24-maint/Lib/plat-mac/argvemulator.py Log: Backport of endianness fix for argvemulator Modified: python/branches/release24-maint/Lib/plat-mac/argvemulator.py ============================================================================== --- python/branches/release24-maint/Lib/plat-mac/argvemulator.py (original) +++ python/branches/release24-maint/Lib/plat-mac/argvemulator.py Thu Jun 8 22:19:15 2006 @@ -7,6 +7,7 @@ from Carbon import AE from Carbon.AppleEvents import * from Carbon import Evt +from Carbon import File from Carbon.Events import * import aetools @@ -16,36 +17,36 @@ def __init__(self): self.quitting = 0 - self.ae_handlers = {} # Remove the funny -psn_xxx_xxx argument if len(sys.argv) > 1 and sys.argv[1][:4] == '-psn': del sys.argv[1] - self.installaehandler('aevt', 'oapp', self.open_app) - self.installaehandler('aevt', 'odoc', self.open_file) - def installaehandler(self, classe, type, callback): - AE.AEInstallEventHandler(classe, type, self.callback_wrapper) - self.ae_handlers[(classe, type)] = callback + AE.AEInstallEventHandler(kCoreEventClass, kAEOpenApplication, self.__runapp) + AE.AEInstallEventHandler(kCoreEventClass, kAEOpenDocuments, self.__openfiles) def close(self): - for classe, type in self.ae_handlers.keys(): - AE.AERemoveEventHandler(classe, type) + AE.AERemoveEventHandler(kCoreEventClass, kAEOpenApplication) + AE.AERemoveEventHandler(kCoreEventClass, kAEOpenDocuments) def mainloop(self, mask = highLevelEventMask, timeout = 1*60): + # Note: this is not the right way to run an event loop in OSX or even + # "recent" versions of MacOS9. This is however code that has proven + # itself. stoptime = Evt.TickCount() + timeout while not self.quitting and Evt.TickCount() < stoptime: - self.dooneevent(mask, timeout) - self.close() + self._dooneevent(mask, timeout) - def _quit(self): - self.quitting = 1 + if not self.quitting: + print "argvemulator: timeout waiting for arguments" - def dooneevent(self, mask = highLevelEventMask, timeout = 1*60): + self.close() + + def _dooneevent(self, mask = highLevelEventMask, timeout = 1*60): got, event = Evt.WaitNextEvent(mask, timeout) if got: - self.lowlevelhandler(event) + self._lowlevelhandler(event) - def lowlevelhandler(self, event): + def _lowlevelhandler(self, event): what, message, when, where, modifiers = event h, v = where if what == kHighLevelEvent: @@ -60,53 +61,28 @@ else: print "Unhandled event:", event - def callback_wrapper(self, _request, _reply): - _parameters, _attributes = aetools.unpackevent(_request) - _class = _attributes['evcl'].type - _type = _attributes['evid'].type - - if self.ae_handlers.has_key((_class, _type)): - _function = self.ae_handlers[(_class, _type)] - elif self.ae_handlers.has_key((_class, '****')): - _function = self.ae_handlers[(_class, '****')] - elif self.ae_handlers.has_key(('****', '****')): - _function = self.ae_handlers[('****', '****')] - else: - raise 'Cannot happen: AE callback without handler', (_class, _type) - - # XXXX Do key-to-name mapping here - _parameters['_attributes'] = _attributes - _parameters['_class'] = _class - _parameters['_type'] = _type - if _parameters.has_key('----'): - _object = _parameters['----'] - del _parameters['----'] - # The try/except that used to be here can mask programmer errors. - # Let the program crash, the programmer can always add a **args - # to the formal parameter list. - rv = _function(_object, **_parameters) - else: - #Same try/except comment as above - rv = _function(**_parameters) - - if rv == None: - aetools.packevent(_reply, {}) - else: - aetools.packevent(_reply, {'----':rv}) + def _quit(self): + self.quitting = 1 - def open_app(self, **args): + def __runapp(self, requestevent, replyevent): self._quit() - def open_file(self, _object=None, **args): - for alias in _object: - fsr = alias.FSResolveAlias(None)[0] - pathname = fsr.as_pathname() - sys.argv.append(pathname) - self._quit() + def __openfiles(self, requestevent, replyevent): + try: + listdesc = requestevent.AEGetParamDesc(keyDirectObject, typeAEList) + for i in range(listdesc.AECountItems()): + aliasdesc = listdesc.AEGetNthDesc(i+1, typeAlias)[1] + alias = File.Alias(rawdata=aliasdesc.data) + fsref = alias.FSResolveAlias(None)[0] + pathname = fsref.as_pathname() + sys.argv.append(pathname) + except Exception, e: + print "argvemulator.py warning: can't unpack an open document event" + import traceback + traceback.print_exc() - def other(self, _object=None, _class=None, _type=None, **args): - print 'Ignore AppleEvent', (_class, _type), 'for', _object, 'Other args:', args + self._quit() if __name__ == '__main__': ArgvCollector().mainloop() From buildbot at python.org Thu Jun 8 22:20:43 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 08 Jun 2006 20:20:43 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060608202043.D5CEC1E4004@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/603 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 00:14:14 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 00:14:14 +0200 (CEST) Subject: [Python-checkins] r46759 - sandbox/trunk/pycon Message-ID: <20060608221414.356B21E4008@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 00:14:13 2006 New Revision: 46759 Removed: sandbox/trunk/pycon/ Log: Remove pycon/ directory From brett at python.org Fri Jun 9 01:47:15 2006 From: brett at python.org (Brett Cannon) Date: Thu, 8 Jun 2006 16:47:15 -0700 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <20060608202043.D5CEC1E4004@bag.python.org> References: <20060608202043.D5CEC1E4004@bag.python.org> Message-ID: On 6/8/06, buildbot at python.org wrote: > > The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. > Full details are available at: > > http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/603 > > Buildbot URL: http://www.python.org/dev/buildbot/all/ > > Build Reason: > Build Source Stamp: [branch trunk] HEAD > Blamelist: brett.cannon > > Build Had Warnings: warnings test OK, I have no clue how my change could trigger this failure. Anyone have a clue? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060608/155634e5/attachment.htm From python-checkins at python.org Fri Jun 9 03:10:18 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 03:10:18 +0200 (CEST) Subject: [Python-checkins] r46760 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060609011018.8A7EE1E400D@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 03:10:17 2006 New Revision: 46760 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Update functools section Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Fri Jun 9 03:10:17 2006 @@ -126,19 +126,16 @@ \section{PEP 309: Partial Function Application\label{pep-309}} The \module{functools} module is intended to contain tools for -functional-style programming. Currently it only contains a -\class{partial()} function, but new functions will probably be added -in future versions of Python. +functional-style programming. -For programs written in a functional style, it can be useful to +One useful tool in this module is the \function{partial()} function. +For programs written in a functional style, you'll sometimes want to construct variants of existing functions that have some of the parameters filled in. Consider a Python function \code{f(a, b, c)}; you could create a new function \code{g(b, c)} that was equivalent to -\code{f(1, b, c)}. This is called ``partial function application'', -and is provided by the \class{partial} class in the new -\module{functools} module. +\code{f(1, b, c)}. This is called ``partial function application''. -The constructor for \class{partial} takes the arguments +\function{partial} takes the arguments \code{(\var{function}, \var{arg1}, \var{arg2}, ... \var{kwarg1}=\var{value1}, \var{kwarg2}=\var{value2})}. The resulting object is callable, so you can just call it to invoke \var{function} @@ -175,11 +172,40 @@ \end{verbatim} +Another function in the \module{functools} module is the +\function{update_wrapper(\var{wrapper, \var{wrapped})} function that +helps you write well-behaved decorators. \function{update_wrapper()} +copies the name, module, and docstring attribute to a wrapper function +so that tracebacks inside the wrapped function are easier to +understand. For example, you might write: + +\begin{verbatim} +def my_decorator(f): + def wrapper(*args, **kwds): + print 'Calling decorated function' + return f(*args, **kwds) + functools.update_wrapper(wrapper, f) + return wrapper +\end{verbatim} + +\function{wraps()} is a decorator that can be used inside your own +decorators to copy the wrapped function's information. An alternate +version of the previous example would be: + +\begin{verbatim} +def my_decorator(f): + @functools.wraps(f) + def wrapper(*args, **kwds): + print 'Calling decorated function' + return f(*args, **kwds) + return wrapper +\end{verbatim} + \begin{seealso} \seepep{309}{Partial Function Application}{PEP proposed and written by -Peter Harris; implemented by Hye-Shik Chang, with adaptations by -Raymond Hettinger.} +Peter Harris; implemented by Hye-Shik Chang and Nick Coghlan, with +adaptations by Raymond Hettinger.} \end{seealso} From python-checkins at python.org Fri Jun 9 03:12:19 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 03:12:19 +0200 (CEST) Subject: [Python-checkins] r46761 - sandbox/trunk/Doc/functional.rst Message-ID: <20060609011219.0510A1E4008@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 03:12:18 2006 New Revision: 46761 Modified: sandbox/trunk/Doc/functional.rst Log: Result of tonight's thesis night: * finish up built-in section * create functools section from the what's new * add lambda section Modified: sandbox/trunk/Doc/functional.rst ============================================================================== --- sandbox/trunk/Doc/functional.rst (original) +++ sandbox/trunk/Doc/functional.rst Fri Jun 9 03:12:18 2006 @@ -4,10 +4,10 @@ **Incomplete Draft** In this document, we'll take a tour of Python's features that are -well-suited to implementing programs in a functional style. -After an introduction to the concepts of functional programming, -we'll look at language features such as iterators and generators -and relevant library modules such as the ``itertools`` module. +well-suited to implementing programs in a functional style. After an +introduction to the concepts of functional programming, we'll look at +language features such as iterators and generators and relevant +library modules such as the ``itertools`` and ``functools`` modules. Introduction @@ -603,14 +603,6 @@ Let's look in more detail at those built-in functions that are relevant to iterators. -any(), all() - -enumerate() - -sorted() - -sum() - Two Python's built-in functions, ``map()`` and ``filter()``, are somewhat obsolete; they duplicate the features of list comprehensions and return actual lists instead of iterators. @@ -633,11 +625,192 @@ but can handle infinite iterators; it'll be discussed in the section on the ``itertools`` module. -``filter(predicate, iter)`` is similarly duplicated by list comprehensions. -**, a function that returns the truth value of -some condition. +``filter(predicate, iter)`` returns a list +that contains all the sequence elements that meet a certain condition, +and is similarly duplicated by list comprehensions. +A **predicate** is a function that returns the truth value of +some condition; for use with ``filter()``, the predicate must take a +single value. + +:: + + def is_even(x): + return (x % 2) == 0 + + filter(is_even, range(10)) => + [0, 2, 4, 6, 8] + +This can also be written as a list comprehension:: + + >>> [x for x in range(10) if is_even(x)] + [0, 2, 4, 6, 8] + +``filter()`` also has a counterpart in the ``itertools`` module, +``itertools.ifilter()``, that returns an iterator and +can therefore handle infinite sequences just as ``itertools.imap()`` can. + +``reduce(func, iter, [initial_value])`` doesn't have a counterpart in +the ``itertools`` module because it cumulatively performs an operation +on all the iterable's elements and therefore can't be applied to +infinite ones. ``func`` must be a function that takes two elements +and returns a single value. ``reduce()`` takes the first two elements +A and B returned by the iterator and calculates ``func(A, B)``. It +then requests the third element, C, calculates ``func(func(A, B), +C)``, combines this result with the fourth element returned, and +continues until the iterable is exhausted. If the iterable returns no +values at all, a ``TypeError`` exception is raised. If the initial +value is supplied, it's used as a starting point and +``func(initial_value, A)`` is the first calculation. + +:: -reduce() + import operator + reduce(operator.concat, ['A', 'BB', 'C']) => + 'ABBC' + reduce(operator.concat, []) => + TypeError: reduce() of empty sequence with no initial value + reduce(operator.mul, [1,2,3], 1) => + 6 + reduce(operator.mul, [], 1) => + 1 + +If you use ``operator.add`` with ``reduce()``, you'll add up all the +elements of the iterable. This case is so common that there's a special +built-in for it:: + + reduce(operator.add, [1,2,3,4], 0) => + 10 + sum([1,2,3,4]) => + 10 + sum([]) => + 0 + +``enumerate(iter)`` counts off the elements in the iterable, return +pairs of the count and each element. + +:: + + enumerate(['subject', 'verb', 'object']) => + [(0, 'subject'), (1, 'verb'), (2, 'object')] + +``enumerate()`` is often used when looping through a list +and recording the indexes at which certain conditions are met:: + + f = open('data.txt', 'r') + for i, line in enumerate(f): + if line.strip() == '': + print 'Blank line at', line + +``sorted(iterable, [cmp=None], [key=None], [reverse=False)`` +collects all the elements of the iterable into a list, sorts +the list, and returns the sorted result. The ``cmp``, ``key``, +and ``reverse`` arguments are passed through to the +constructed list's ``.sort()`` method. + +:: + + import random + # Generate 8 random numbers between [0, 10000) + rand_list = random.sample(range(10000), 8) + rand_list => + [769, 7953, 9828, 6431, 8442, 9878, 6213, 2207] + sorted(rand_list) => + [769, 2207, 6213, 6431, 7953, 8442, 9828, 9878] + sorted(rand_list, reverse=True) => + [9878, 9828, 8442, 7953, 6431, 6213, 2207, 769] + +(For a more detailed discussion of sorting, see the Sorting mini-HOWTO +in the Python wiki at \url{http://wiki.python.org/moin/SortingHowto}.) + +The ``any(iter)`` and ``all(iter)`` built-ins look at +the truth values of an iterable's contents. ``any()`` returns +True if any element in the iterable is a true value, and ``all()`` +retturn True if all of the elements are true values:: + + any([0,1,0]) => + True + any([0,0,0]) => + False + any([1,1,1]) => + True + all([0,1,0]) => + False + all([0,0,0]) => + False + all([1,1,1]) => + True + + +Small functions and the lambda statement +---------------------------------------------- + +When writing functional-style programs, you'll often need little +functions that act as predicates or that combine list elements in some +particular way. + +If there's a Python built-in that's suitable, or a module has a +function that does what you need, you don't need to define a new +function at all:: + + stripped_lines = [line.strip for line in lines] + existing_files = filter(os.path.exists, file_list) + XXX pi-notation example + +If the function you need doesn't exist, you need to write it. One way +to write small functions is to use the ``lambda`` statement. ``lambda`` +takes a number of parameters and an expression combining these parameters, +and creates a small function that returns the value of the expression: + + lowercase = lambda x: x.lower() + + print_assign = lambda name, value: name + '=' + str(value) + + adder = lambda x, y: x+y + +An alternative is to just use the ``def`` statement and define a +function in the usual way:: + + def lowercase(x): + return x.lower() + + def print_assign(name, value): + return name + '=' + str(value) + + def adder(x,y): + return x + y + +Which alternative is preferable? That's a style question. + +``lambda`` is quite limited in the functions it can define. The +result has to be computable as a single expression, which means you +can't have multiway ``if... elif... else`` comparisons or +``try... except`` statements. If you try to do too much in a +``lambda`` statement, you'll end up with an overly complicated +expression that's hard to read. Quick, what's the following code doing? + +:: + + freq = reduce(lambda a, b: (0, a[1] + b[1]), items)[1] + +Fredrik Lundh once suggested the following set of rules for refactoring +uses of ``lambda``: + +1) Write a lambda function. +2) Write a comment explaining what the heck that lambda does. +3) Study the comment for a while, and think of a name that captures + the essence of the comment. +4) Convert the lambda to a def statement, using that name. +5) Remove the comment. + +Personally I try to avoid lambdas, favoring short nested +``def`` statements like this:: + + def combine_freq (a, b): + return 0, a[1] + b[1] + + print reduce(combine_freq, items)[1] + +You might disagree that this style is better. The itertools module @@ -764,7 +937,7 @@ itertools.starmap(os.path.join, [('/usr', 'bin', 'java'), ('/bin', 'python'), - ('/usr', 'bin', 'perl'),('/usr', 'bin', 'ruby')]) + ('/usr', 'bin', 'perl'),('/usr', 'bin', 'ruby')]) => /usr/bin/java, /bin/python, /usr/bin/perl, /usr/bin/ruby @@ -827,13 +1000,13 @@ :: city_list = [('Decatur', 'AL'), ('Huntsville', 'AL'), ('Selma', 'AL'), - ('Anchorage', 'AK'), ('Nome', 'AK'), - ('Flagstaff', 'AZ'), ('Phoenix', 'AZ'), ('Tucson', 'AZ'), + ('Anchorage', 'AK'), ('Nome', 'AK'), + ('Flagstaff', 'AZ'), ('Phoenix', 'AZ'), ('Tucson', 'AZ'), ... - ] + ] def get_state ((city, state)): - return state + return state itertools.groupby(city_list, get_state) => ('AL', iterator-1), @@ -854,24 +1027,44 @@ of iterator-1 before requesting iterator-2 and its corresponding key. -Small functions and the lambda statement +The functools module ---------------------------------------------- -XXX +The ``functools`` module in Python 2.5 contains some higher-order +functions. A **higher-order function** takes functions as input and +returns new functions. The most useful tool in this module is the +``partial()`` function. + +For programs written in a functional style, you'll sometimes want to +construct variants of existing functions that have some of the +parameters filled in. Consider a Python function ``f(a, b, c)``; you +could create a new function ``g(b, c)`` that was equivalent to +``f(1, b, c)``. This is called "partial function application". + +The constructor for ``partial`` takes the arguments ``(function, arg1, +arg2, ... kwarg1=value1, kwarg2=value2)``. The resulting object is +callable, so you can just call it to invoke ``function`` with the +filled-in arguments. + +Here's a small but realistic example:: + + import functools + + def log (message, subsystem): + "Write the contents of 'message' to the specified subsystem." + print '%s: %s' % (subsystem, message) + ... -The functools module ----------------------------------------------- + server_log = functools.partial(log, subsystem='server') + server_log('Unable to open socket') -XXX Topics to place ----------------------------- -XXX - -os.walk() +XXX os.walk() -Need a large example. +XXX Need a large example. ======= @@ -880,7 +1073,7 @@ The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this -article: Jim Jewett. +article: Raymond Hettinger, Jim Jewett. .. comment From tim.peters at gmail.com Fri Jun 9 03:20:45 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 8 Jun 2006 21:20:45 -0400 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: References: <20060608202043.D5CEC1E4004@bag.python.org> Message-ID: <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> [buildbot at python.org] >> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. >> Full details are available at: >> > http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/603 >> >> Buildbot URL: http://www.python.org/dev/buildbot/all/ >> >> Build Reason: >> Build Source Stamp: [branch trunk] HEAD >> Blamelist: brett.cannon >> >> Build Had Warnings: warnings test [brett.cannon, trying to weasle out ;-)] > OK, I have no clue how my change could trigger this failure. Anyone have a > clue? Note something from that log: test_optparse originally failed in test_filetype_noexist: test test_optparse failed -- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-ubuntu-hppa/build/Lib/test/test_optparse.py", line 1042, in test_filetype_noexist test_support.TESTFN) File "/home/pybot/buildarea/trunk.klose-ubuntu-hppa/build/Lib/test/test_optparse.py", line 158, in assertParseFail self.assertFalse("expected parse failure") AssertionError But when test_optparse was re-run with -v, type_noexist passed, and test_optparse failed in test_version instead. OTOH, test_optparse's test_filetype_noexist _also_ failed on the "x86 W2k trunk" buildbot, build 992, with your rev 46757 alone in the blamelist, and that makes it harder to swallow that the HPPA buildbot is just flaky here. On the W2k run, test_optparse also passed (all of it) when it got rerun with -v. If there's "a real reason" for this, it may well require running some subset of tests in a specific order. Alas, since your patch changed C code, it's potentially to blame for any kind of failure whatsoever. I forced another test run on the HPPA box just to see what happens. I'm also running the tests on my box in exactly the same order the W2k box ran them during its test_optparse failure. Oops! That finished already, and test_optparse _did_ fail here. I'll attach the list of tests it ran. You use this by passing it to regrtest.py's "-f" argument. Next step is binary-search'ish to get a minimal set of failing tests. Happily, I forgot to use -uall, so the 4 skipped tests here can be removed at once: $ python_d -E -tt ../lib/test/regrtest.py -f f.txt test_distutils ... test_optparse test test_optparse failed -- Traceback (most recent call last): File "C:\Code\python\lib\test\test_optparse.py", line 1042, in test_filetype_noexist test_support.TESTFN) File "C:\Code\python\lib\test\test_optparse.py", line 158, in assertParseFail self.assertFalse("expected parse failure") AssertionError 135 tests OK. 1 test failed: test_optparse 4 tests skipped: test_largefile test_socket_ssl test_timeout test_urllibnet Those skips are all expected on win32. -------------- next part -------------- test_distutils test_capi test_fpformat test_compare test_sax test_bz2 test_with test_functools test_sets test_stringprep test_grammar test_wave test_fnmatch test_minidom test_iter test_urllibnet test_compile test_threaded_import test_peepholer test_asynchat test_set test_StringIO test_urlparse test_doctest2 test_slice test_module test_decimal test_htmllib test_xml_etree test_imp test_call test_csv test_unittest test_anydbm test_audioop test_email_codecs test_datetime test_ntpath test_types test_filecmp test_exception_variations test_coding test_new test_urllib2 test_syntax test_inspect test_xdrlib test_codecencodings_tw test_traceback test_defaultdict test_hotshot test_textwrap test_MimeWriter test_socket_ssl test_imaplib test_tuple test_codeccallbacks test_time test_unpack test_string test_array test_locale test_long_future test_getargs2 test_augassign test_fileinput test_softspace test_scope test_codecmaps_cn test_charmapcodec test_pyexpat test_genexps test_runpy test_userstring test_bsddb test_binhex test_urllib test_startfile test_repr test_quopri test_timeout test_weakref test_codecs test_structseq test_warnings test_frozen test_email_renamed test_shlex test_codecmaps_tw test_coercion test_largefile test_codecencodings_kr test_pprint test_dummy_threading test_global test_platform test_mimetypes test_cProfile test_complex test_bastion test_getopt test_tcl test_strftime test_copy test_unicode test_symtable test_bufio test_pep263 test_zlib test_zipfile test_zipimport test_code test_pkgimport test_winsound test_macpath test_profilehooks test_pickletools test_mailbox test_bool test_funcattrs test_calendar test_multibytecodec test_select test_trace test_math test_str test_pyclbr test_pickle test_site test_pkg test_bisect test_userdict test_univnewlines test_compiler test_rfc822 test_unicode_file test_file test_threadedtempfile test_class test_optparse From tim.peters at gmail.com Fri Jun 9 04:05:49 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 8 Jun 2006 22:05:49 -0400 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> Message-ID: <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> FYI, here's the minimal set of failing tests: $ python_d ../Lib/test/regrtest.py test_file test_optparse test_file test_optparse test test_optparse failed -- Traceback (most recent call last): File "C:\Code\python\lib\test\test_optparse.py", line 1042, in test_filetype_noexist test_support.TESTFN) File "C:\Code\python\lib\test\test_optparse.py", line 158, in assertParseFail self.assertFalse("expected parse failure") AssertionError 1 test OK. 1 test failed: test_optparse [23476 refs] That also, using -w and -f, reproduces the bizarre HPPA behavior when test_optparse is rerun in verbose mode (test_filetype_noexist passes then, but test_version fails). I have no idea why any of this is true, but there's good and bad news: reverting rev 46757 does _not_ make the problem go away. So you're off the hook, but we don't know who to crucify in your place ;-) As to why the failure only showed up recently, I'm not sure, but test_file must run before test_optparse, and it looks like the problem goes away if "too many"(!) other tests intervene. The Win2K buildbot is unique in that test_file has been followed very soon by test_optparse two builds in a row. From python-checkins at python.org Fri Jun 9 04:11:02 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 9 Jun 2006 04:11:02 +0200 (CEST) Subject: [Python-checkins] r46762 - python/trunk/Lib/test/test_file.py Message-ID: <20060609021102.A64421E4008@bag.python.org> Author: tim.peters Date: Fri Jun 9 04:11:02 2006 New Revision: 46762 Modified: python/trunk/Lib/test/test_file.py Log: Whitespace normalization. Since test_file is implicated in mysterious test failures when followed by test_optparse, if I had any brains I'd look at the checkin that last changed test_file ;-) Modified: python/trunk/Lib/test/test_file.py ============================================================================== --- python/trunk/Lib/test/test_file.py (original) +++ python/trunk/Lib/test/test_file.py Fri Jun 9 04:11:02 2006 @@ -86,7 +86,7 @@ self.assertEquals(f.name, TESTFN) self.assert_(not f.isatty()) self.assert_(not f.closed) - + self.assertRaises(TypeError, f.readinto, "") f.close() self.assert_(f.closed) From brett at python.org Fri Jun 9 04:30:55 2006 From: brett at python.org (Brett Cannon) Date: Thu, 8 Jun 2006 19:30:55 -0700 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> Message-ID: On 6/8/06, Tim Peters wrote: > > FYI, here's the minimal set of failing tests: > > $ python_d ../Lib/test/regrtest.py test_file test_optparse > test_file > test_optparse > test test_optparse failed -- Traceback (most recent call last): > File "C:\Code\python\lib\test\test_optparse.py", line 1042, in > test_filetype_noexist > test_support.TESTFN) > File "C:\Code\python\lib\test\test_optparse.py", line 158, in > assertParseFail > self.assertFalse("expected parse failure") > AssertionError Different type of failure as well; if you look at the original failure it has to do with the help output having an extra newline. 1 test OK. > 1 test failed: > test_optparse > [23476 refs] > > That also, using -w and -f, reproduces the bizarre HPPA behavior when > test_optparse is rerun in verbose mode (test_filetype_noexist passes > then, but test_version fails). > > I have no idea why any of this is true, but there's good and bad news: > reverting rev 46757 does _not_ make the problem go away. Actually, that run had two checkins; there was also 46755. But when I ``svn update -r46754`` it still fails on my OS X laptop. So still ain't my fault. =) So you're > off the hook, but we don't know who to crucify in your place ;-) Oh, we live in America; we have a list to pull from. =) As to why the failure only showed up recently, I'm not sure, but > test_file must run before test_optparse, and it looks like the problem > goes away if "too many"(!) other tests intervene. The Win2K buildbot > is unique in that test_file has been followed very soon by > test_optparse two builds in a row. > We don't have any mechanism in place to record when we find tests failing in a row to always run them in that order until we fix it, do we? Nor do we have a script to just continually check out older revisions in svn, compile, and test until the tests do pass, huh? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060608/57f96165/attachment.html From buildbot at python.org Fri Jun 9 04:32:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 02:32:02 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060609023203.0ED641E4008@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/605 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 04:48:23 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 02:48:23 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP trunk Message-ID: <20060609024823.4571D1E4008@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/975 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From tim.peters at gmail.com Fri Jun 9 04:56:27 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 8 Jun 2006 22:56:27 -0400 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> Message-ID: <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> [Tim] >> FYI, here's the minimal set of failing tests: >> >> $ python_d ../Lib/test/regrtest.py test_file test_optparse >> test_file >> test_optparse >> test test_optparse failed -- Traceback (most recent call last): >> File "C:\Code\python\lib\test\test_optparse.py", line 1042, in test_filetype_noexist >> test_support.TESTFN) >> File "C:\Code\python\lib\test\test_optparse.py", line 158, in assertParseFail >> self.assertFalse("expected parse failure") >> AssertionError [Brett] > Different type of failure as well; Not so. > if you look at the original failure it has to do with the help output > having an extra newline. While if you look at the original failure ;-), you'll see that _both_ failure modes occur. The one I showed above occurs when test_optparse runs the first time; the one you're thinking of occurs when regrest *re*runs test_optparse in verbose mode. The original HPPA log contained both failures. >> ... >> I have no idea why any of this is true, but there's good and bad news: >> reverting rev 46757 does _not_ make the problem go away. > Actually, that run had two checkins; there was also 46755. Build 992 on the W2k trunk buildbot had only 46757 in its blamelist, and was the first failing test run there. > But when I ``svn update -r46754`` it still fails on my OS X laptop. What revision was your laptop at before the update? It could help a lot to know the earliest revision at which this fails. > So still ain't my fault. =) No, you're so argumentative today I'm starting to suspect it is ;-) ... >> As to why the failure only showed up recently, I'm not sure, but >> test_file must run before test_optparse, and it looks like the problem >> goes away if "too many"(!) other tests intervene. The Win2K buildbot >> is unique in that test_file has been followed very soon by >> test_optparse two builds in a row. > We don't have any mechanism in place to record when we find tests failing in > a row to always run them in that order until we fix it, do we? That's right -- none. If would be easy to check in a little temporary tweak -- think I'll do that. > Nor do we have a script to just continually check out older revisions > in svn, compile, and test until the tests do pass, huh? We don't, and I don't either. IIRC, Neil did quite a bit of that some time ago, and he may have a script for it. Doing a binary search under SVN should be very easy, given that a revision number identifies the entire state of the repository. From brett at python.org Fri Jun 9 05:03:19 2006 From: brett at python.org (Brett Cannon) Date: Thu, 8 Jun 2006 20:03:19 -0700 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> Message-ID: On 6/8/06, Tim Peters wrote: > > [Tim] > >> FYI, here's the minimal set of failing tests: > >> > >> $ python_d ../Lib/test/regrtest.py test_file test_optparse > >> test_file > >> test_optparse > >> test test_optparse failed -- Traceback (most recent call last): > >> File "C:\Code\python\lib\test\test_optparse.py", line 1042, in > test_filetype_noexist > >> test_support.TESTFN) > >> File "C:\Code\python\lib\test\test_optparse.py", line 158, in > assertParseFail > >> self.assertFalse("expected parse failure") > >> AssertionError > > [Brett] > > Different type of failure as well; > > Not so. > > > if you look at the original failure it has to do with the help output > > having an extra newline. > > While if you look at the original failure ;-), you'll see that _both_ > failure modes occur. The one I showed above occurs when test_optparse > runs the first time; the one you're thinking of occurs when regrest > *re*runs test_optparse in verbose mode. The original HPPA log > contained both failures. Ah, my mistake. >> ... > >> I have no idea why any of this is true, but there's good and bad news: > >> reverting rev 46757 does _not_ make the problem go away. > > > Actually, that run had two checkins; there was also 46755. > > Build 992 on the W2k trunk buildbot had only 46757 in its blamelist, > and was the first failing test run there. > > > But when I ``svn update -r46754`` it still fails on my OS X laptop. > > What revision was your laptop at before the update? It could help a > lot to know the earliest revision at which this fails. No clue. I had not updated my local version in quite some time since most of my dev as of late has been at work. > So still ain't my fault. =) > > No, you're so argumentative today I'm starting to suspect it is ;-) Sorry, but at the moment Python is failing on ``make install`` when it runs compileall . ... > > >> As to why the failure only showed up recently, I'm not sure, but > >> test_file must run before test_optparse, and it looks like the problem > >> goes away if "too many"(!) other tests intervene. The Win2K buildbot > >> is unique in that test_file has been followed very soon by > >> test_optparse two builds in a row. > > > We don't have any mechanism in place to record when we find tests > failing in > > a row to always run them in that order until we fix it, do we? > > That's right -- none. If would be easy to check in a little temporary > tweak -- think I'll do that. > > > Nor do we have a script to just continually check out older revisions > > in svn, compile, and test until the tests do pass, huh? > > We don't, and I don't either. IIRC, Neil did quite a bit of that some > time ago, and he may have a script for it. Doing a binary search > under SVN should be very easy, given that a revision number identifies > the entire state of the repository. > That would be handy. Question is do we just want a progressive backtrack or an actual binary search that goes back a set number of revisions and then begins to creep back up in rev. numbers when it realizes it has gone back too far. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060608/c161a02b/attachment.htm From python-checkins at python.org Fri Jun 9 05:09:44 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 9 Jun 2006 05:09:44 +0200 (CEST) Subject: [Python-checkins] r46763 - python/trunk/Lib/test/regrtest.py Message-ID: <20060609030944.B835A1E4008@bag.python.org> Author: tim.peters Date: Fri Jun 9 05:09:42 2006 New Revision: 46763 Modified: python/trunk/Lib/test/regrtest.py Log: To boost morale :-), force test_optparse to run immediately after test_file until we can figure out how to fix it. (See python-dev; at the moment we don't even know which checkin caused the problem.) Modified: python/trunk/Lib/test/regrtest.py ============================================================================== --- python/trunk/Lib/test/regrtest.py (original) +++ python/trunk/Lib/test/regrtest.py Fri Jun 9 05:09:42 2006 @@ -344,6 +344,11 @@ tests = tests[:1] if randomize: random.shuffle(tests) + # XXX Temporary hack to force test_optparse to run immediately + # XXX after test_file. This should go away as soon as we fix + # XXX whatever it is that's causing that to fail. + tests.remove("test_file") + tests.insert(tests.index("test_optparse"), "test_file") if trace: import trace tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix], From tim.peters at gmail.com Fri Jun 9 05:15:59 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 8 Jun 2006 23:15:59 -0400 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> Message-ID: <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> ... [Tim] >> What revision was your laptop at before the update? It could help a >> lot to know the earliest revision at which this fails. [Brett] > No clue. I had not updated my local version in quite some time since most > of my dev as of late has been at work. A good clue is to look at the "Revsion: NNNNN" line from "svn info" output executed from the root of your checkout. Or if you have the Python executable handy: >>> import sys >>> sys.subversion ('CPython', 'trunk', '46762') No, I'm not making that up! >> ... >> Doing a binary search under SVN should be very easy, given that >> a revision number identifies the entire state of the repository. > That would be handy. Question is do we just want a progressive backtrack or > an actual binary search that goes back a set number of revisions and then > begins to creep back up in rev. numbers when it realizes it has gone back > too far. What we really want to do is solve the problem. If we're going to tie up my machine doing it, I want as few builds as theoretically possible. If we're going to tie up your machine, it's fine by me if it goes back one checkin at a time until 1991 :-) From buildbot at python.org Fri Jun 9 05:23:25 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:23:25 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060609032325.468BF1E4008@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/310 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 05:31:29 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:31:29 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060609033129.EC40C1E4008@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1075 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 05:32:04 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:32:04 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060609033204.C3E341E4008@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/978 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 05:32:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:32:36 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP-2 trunk Message-ID: <20060609033236.933931E400E@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-2%2520trunk/builds/585 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 05:37:12 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:37:12 +0000 Subject: [Python-checkins] buildbot warnings in x86 W2k trunk Message-ID: <20060609033712.DE4231E4008@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/995 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From brett at python.org Fri Jun 9 05:38:29 2006 From: brett at python.org (Brett Cannon) Date: Thu, 8 Jun 2006 20:38:29 -0700 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> Message-ID: On 6/8/06, Tim Peters wrote: > ... > > [Tim] > >> What revision was your laptop at before the update? It could help a > >> lot to know the earliest revision at which this fails. > > [Brett] > > No clue. I had not updated my local version in quite some time since > most > > of my dev as of late has been at work. > > A good clue is to look at the "Revsion: NNNNN" line from "svn info" > output executed from the root of your checkout. Or if you have the > Python executable handy: > > >>> import sys > >>> sys.subversion > ('CPython', 'trunk', '46762') > > No, I'm not making that up! Oh, I believe you. Issue is that I did a svn update when I got home today. I have another checkout that I never modify (to use as a reference checkout) that I have not updated since rev. 43738 and it passes the tests. That sure helps narrow it down, doesn't it. =) A quick check of rev. 46750 has the test passing as well. >> ... > >> Doing a binary search under SVN should be very easy, given that > >> a revision number identifies the entire state of the repository. > > > That would be handy. Question is do we just want a progressive > backtrack or > > an actual binary search that goes back a set number of revisions and > then > > begins to creep back up in rev. numbers when it realizes it has gone > back > > too far. > > What we really want to do is solve the problem. Of course. If we're going to tie > up my machine doing it, I want as few builds as theoretically > possible. If we're going to tie up your machine, it's fine by me if > it goes back one checkin at a time until 1991 :-) > =) On my slow machine, it might be another 15 years before we get to current on HEAD. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060608/039d9f85/attachment.html From tim.peters at gmail.com Fri Jun 9 05:42:28 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 8 Jun 2006 23:42:28 -0400 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> Message-ID: <1f7befae0606082042n7a178acdpb8bcfcd2de0aa787@mail.gmail.com> Well, this sure sucks. This is the earliest revision at which the tests fail: """ r46752 | georg.brandl | 2006-06-08 10:50:53 -0400 (Thu, 08 Jun 2006) | 3 lines Changed paths: M /python/trunk/Lib/test/test_file.py Convert test_file to unittest. """ If _that's_ not a reason for using doctest, I don't know what is ;-) From buildbot at python.org Fri Jun 9 05:43:51 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:43:51 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060609034351.B40B41E4008@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/691 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 05:50:11 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:50:11 +0000 Subject: [Python-checkins] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20060609035011.E14361E4008@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%2520trunk/builds/939 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 05:51:05 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:51:05 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060609035105.3F6761E4008@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/938 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 05:51:42 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 9 Jun 2006 05:51:42 +0200 (CEST) Subject: [Python-checkins] r46764 - python/trunk/Lib/test/test_file.py Message-ID: <20060609035142.4BD9F1E4008@bag.python.org> Author: tim.peters Date: Fri Jun 9 05:51:41 2006 New Revision: 46764 Modified: python/trunk/Lib/test/test_file.py Log: AutoFileTests.tearDown(): Removed mysterious undocumented try/except. Remove TESTFN. Throughout: used open() instead of file(), and wrapped long lines. Modified: python/trunk/Lib/test/test_file.py ============================================================================== --- python/trunk/Lib/test/test_file.py (original) +++ python/trunk/Lib/test/test_file.py Fri Jun 9 05:51:41 2006 @@ -11,14 +11,12 @@ # file tests for which a test file is automatically set up def setUp(self): - self.f = file(TESTFN, 'wb') + self.f = open(TESTFN, 'wb') def tearDown(self): - try: - if self.f: - self.f.close() - except IOError: - pass + if self.f: + self.f.close() + os.remove(TESTFN) def testWeakRefs(self): # verify weak references @@ -73,9 +71,11 @@ def testWritelinesNonString(self): # verify writelines with non-string object - class NonString: pass + class NonString: + pass - self.assertRaises(TypeError, self.f.writelines, [NonString(), NonString()]) + self.assertRaises(TypeError, self.f.writelines, + [NonString(), NonString()]) def testRepr(self): # verify repr works @@ -93,8 +93,8 @@ def testMethods(self): methods = ['fileno', 'flush', 'isatty', 'next', 'read', 'readinto', - 'readline', 'readlines', 'seek', 'tell', 'truncate', 'write', - 'xreadlines', '__iter__'] + 'readline', 'readlines', 'seek', 'tell', 'truncate', + 'write', 'xreadlines', '__iter__'] if sys.platform.startswith('atheos'): methods.remove('truncate') @@ -113,7 +113,7 @@ # check invalid mode strings for mode in ("", "aU", "wU+"): try: - f = file(TESTFN, mode) + f = open(TESTFN, mode) except ValueError: pass else: @@ -175,11 +175,11 @@ def bug801631(): # SF bug # "file.truncate fault on windows" - f = file(TESTFN, 'wb') + f = open(TESTFN, 'wb') f.write('12345678901') # 11 bytes f.close() - f = file(TESTFN,'rb+') + f = open(TESTFN,'rb+') data = f.read(5) if data != '12345': self.fail("Read on file opened for update failed %r" % data) @@ -201,14 +201,14 @@ os.unlink(TESTFN) def testIteration(self): - # Test the complex interaction when mixing file-iteration and the various - # read* methods. Ostensibly, the mixture could just be tested to work - # when it should work according to the Python language, instead of fail - # when it should fail according to the current CPython implementation. - # People don't always program Python the way they should, though, and the - # implemenation might change in subtle ways, so we explicitly test for - # errors, too; the test will just have to be updated when the - # implementation changes. + # Test the complex interaction when mixing file-iteration and the + # various read* methods. Ostensibly, the mixture could just be tested + # to work when it should work according to the Python language, + # instead of fail when it should fail according to the current CPython + # implementation. People don't always program Python the way they + # should, though, and the implemenation might change in subtle ways, + # so we explicitly test for errors, too; the test will just have to + # be updated when the implementation changes. dataoffset = 16384 filler = "ham\n" assert not dataoffset % len(filler), \ @@ -246,12 +246,13 @@ (methodname, args)) f.close() - # Test to see if harmless (by accident) mixing of read* and iteration - # still works. This depends on the size of the internal iteration - # buffer (currently 8192,) but we can test it in a flexible manner. - # Each line in the bag o' ham is 4 bytes ("h", "a", "m", "\n"), so - # 4096 lines of that should get us exactly on the buffer boundary for - # any power-of-2 buffersize between 4 and 16384 (inclusive). + # Test to see if harmless (by accident) mixing of read* and + # iteration still works. This depends on the size of the internal + # iteration buffer (currently 8192,) but we can test it in a + # flexible manner. Each line in the bag o' ham is 4 bytes + # ("h", "a", "m", "\n"), so 4096 lines of that should get us + # exactly on the buffer boundary for any power-of-2 buffersize + # between 4 and 16384 (inclusive). f = open(TESTFN) for i in range(nchunks): f.next() From buildbot at python.org Fri Jun 9 05:57:21 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 03:57:21 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060609035722.02D6F1E4008@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/647 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 06:00:18 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 04:00:18 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060609040018.9F8021E4008@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/607 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 06:02:06 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 9 Jun 2006 06:02:06 +0200 (CEST) Subject: [Python-checkins] r46765 - python/trunk/Lib/test/test_file.py Message-ID: <20060609040206.70B771E4008@bag.python.org> Author: tim.peters Date: Fri Jun 9 06:02:06 2006 New Revision: 46765 Modified: python/trunk/Lib/test/test_file.py Log: testUnicodeOpen(): I have no idea why, but making this test clean up after itself appears to fix the test failures when test_optparse follows test_file. test_main(): Get rid of TESTFN no matter what. That's also enough to fix the mystery failures. Doesn't hurt to fix them twice :-) Modified: python/trunk/Lib/test/test_file.py ============================================================================== --- python/trunk/Lib/test/test_file.py (original) +++ python/trunk/Lib/test/test_file.py Fri Jun 9 06:02:06 2006 @@ -135,6 +135,7 @@ f = open(unicode(TESTFN), "w") self.assert_(repr(f).startswith(" The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/686 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From brett at python.org Fri Jun 9 06:05:22 2006 From: brett at python.org (Brett Cannon) Date: Thu, 8 Jun 2006 21:05:22 -0700 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <1f7befae0606082042n7a178acdpb8bcfcd2de0aa787@mail.gmail.com> References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> <1f7befae0606082042n7a178acdpb8bcfcd2de0aa787@mail.gmail.com> Message-ID: On 6/8/06, Tim Peters wrote: > > Well, this sure sucks. This is the earliest revision at which the tests > fail: > > """ > r46752 | georg.brandl | 2006-06-08 10:50:53 -0400 (Thu, 08 Jun 2006) | 3 > lines > Changed paths: > M /python/trunk/Lib/test/test_file.py > > Convert test_file to unittest. > """ > > If _that's_ not a reason for using doctest, I don't know what is ;-) > That's hilarious! OK, that's it. I am definitely focusing my efforts for 2.6 in improving testing. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060608/074f232c/attachment.htm From buildbot at python.org Fri Jun 9 06:16:53 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 04:16:53 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060609041653.EEC001E4009@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/578 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From tim.peters at gmail.com Fri Jun 9 06:21:45 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 9 Jun 2006 00:21:45 -0400 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> <1f7befae0606082042n7a178acdpb8bcfcd2de0aa787@mail.gmail.com> Message-ID: <1f7befae0606082121u913571es82cdcd8687bf3d3e@mail.gmail.com> [Tim] >> Well, this sure sucks. This is the earliest revision at which the tests fail: >> >> """ >> r46752 | georg.brandl | 2006-06-08 10:50:53 -0400 (Thu, 08 Jun 2006) | 3 lines >> Changed paths: >> M /python/trunk/Lib/test/test_file.py >> >> Convert test_file to unittest. >> """ >> >> If _that's_ not a reason for using doctest, I don't know what is ;-) [Brett] > That's hilarious! OK, that's it. I am definitely focusing my efforts for > 2.6 in improving testing. LOL. Believe it or not, we wouldn't have had a problem here if doctest had been used instead -- but the reason isn't particularly sane ;-) Before the conversion, test_file.py ended with: try: bug801631() finally: os.unlink(TESTFN) so that TESTFN got removed _no matter what_. Some of the individual tests here are careless about cleaning up after themselves, and that last clause used to hide a multitude of lazy sins. A doctest would naturally also have ended with a "clean up the mess" block. After the conversion to unittest, there was no final cleanup block, and it just so happened that "testUnicodeOpen" has the alphabetically largest test name, so unittest runs it last, and testUnicodeOpen() was one of the lazy tests that didn't clean up after itself. What that has to do with test_optparse is left as an exercise for the reader ;-) From python-checkins at python.org Fri Jun 9 07:12:41 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 9 Jun 2006 07:12:41 +0200 (CEST) Subject: [Python-checkins] r46766 - python/trunk/Lib/test/regrtest.py Message-ID: <20060609051241.0634A1E4009@bag.python.org> Author: tim.peters Date: Fri Jun 9 07:12:40 2006 New Revision: 46766 Modified: python/trunk/Lib/test/regrtest.py Log: Remove the temporary hack to force test_optparse to run immediately after test_file. At least 8 buildbot boxes passed since the underlying problem got fixed, and they all failed before the fix, so there's no point to this anymore. Modified: python/trunk/Lib/test/regrtest.py ============================================================================== --- python/trunk/Lib/test/regrtest.py (original) +++ python/trunk/Lib/test/regrtest.py Fri Jun 9 07:12:40 2006 @@ -344,11 +344,6 @@ tests = tests[:1] if randomize: random.shuffle(tests) - # XXX Temporary hack to force test_optparse to run immediately - # XXX after test_file. This should go away as soon as we fix - # XXX whatever it is that's causing that to fail. - tests.remove("test_file") - tests.insert(tests.index("test_optparse"), "test_file") if trace: import trace tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix], From nnorwitz at gmail.com Fri Jun 9 07:39:45 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 8 Jun 2006 22:39:45 -0700 Subject: [Python-checkins] r46751 - python/trunk/Objects/fileobject.c In-Reply-To: <20060608145022.621EF1E4013@bag.python.org> References: <20060608145022.621EF1E4013@bag.python.org> Message-ID: On 6/8/06, georg.brandl wrote: > Author: georg.brandl > Date: Thu Jun 8 16:50:21 2006 > New Revision: 46751 > > Modified: > python/trunk/Objects/fileobject.c > Log: > Bug #1502805: don't alias file.__exit__ to file.close since the > latter can return something that's true. How about a test for this? n From nnorwitz at gmail.com Fri Jun 9 07:45:53 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 8 Jun 2006 22:45:53 -0700 Subject: [Python-checkins] r46757 - in python/trunk: Lib/test/test_types.py Misc/NEWS Modules/arraymodule.c Objects/bufferobject.c In-Reply-To: <20060608170049.6A6411E4011@bag.python.org> References: <20060608170049.6A6411E4011@bag.python.org> Message-ID: On 6/8/06, brett.cannon wrote: > Author: brett.cannon > Date: Thu Jun 8 19:00:45 2006 > New Revision: 46757 > > Modified: python/trunk/Objects/bufferobject.c > ============================================================================== > --- python/trunk/Objects/bufferobject.c (original) > +++ python/trunk/Objects/bufferobject.c Thu Jun 8 19:00:45 2006 > @@ -15,8 +15,16 @@ > } PyBufferObject; > > > +enum buffer_t { > + READBUFFER, > + WRITEBUFFER, > + CHARBUFFER, > + ANY_BUFFER, > +}; Why the inconsistent use of _? My preference is for all them to have _. n From python-checkins at python.org Fri Jun 9 07:54:19 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 9 Jun 2006 07:54:19 +0200 (CEST) Subject: [Python-checkins] r46767 - python/trunk/Lib/test/test_file.py Message-ID: <20060609055419.0D0E41E4009@bag.python.org> Author: neal.norwitz Date: Fri Jun 9 07:54:18 2006 New Revision: 46767 Modified: python/trunk/Lib/test/test_file.py Log: Fix grammar and reflow Modified: python/trunk/Lib/test/test_file.py ============================================================================== --- python/trunk/Lib/test/test_file.py (original) +++ python/trunk/Lib/test/test_file.py Fri Jun 9 07:54:18 2006 @@ -314,8 +314,8 @@ def test_main(): - # Historically, these tests have sloppy about removing TESTFN. So get - # rid of it no matter what. + # Historically, these tests have been sloppy about removing TESTFN. + # So get rid of it no matter what. try: run_unittest(AutoFileTests, OtherFileTests) finally: From buildbot at python.org Fri Jun 9 08:09:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 06:09:30 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060609060931.0DDC21E4009@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/376 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From g.brandl at gmx.net Fri Jun 9 08:16:10 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 09 Jun 2006 08:16:10 +0200 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <1f7befae0606082121u913571es82cdcd8687bf3d3e@mail.gmail.com> References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> <1f7befae0606082042n7a178acdpb8bcfcd2de0aa787@mail.gmail.com> <1f7befae0606082121u913571es82cdcd8687bf3d3e@mail.gmail.com> Message-ID: Tim Peters wrote: > Before the conversion, test_file.py ended with: > > try: > bug801631() > finally: > os.unlink(TESTFN) > > so that TESTFN got removed _no matter what_. Some of the individual > tests here are careless about cleaning up after themselves, and that > last clause used to hide a multitude of lazy sins. A doctest would > naturally also have ended with a "clean up the mess" block. > > After the conversion to unittest, there was no final cleanup block, > and it just so happened that "testUnicodeOpen" has the alphabetically > largest test name, so unittest runs it last, and testUnicodeOpen() was > one of the lazy tests that didn't clean up after itself. Okay, so it's my fault. Somehow I always believed that regrtest would remove the TESTFN by itself since any test could raise an unexpected exception in the middle and not get around to removing it itself. Georg From g.brandl at gmx.net Fri Jun 9 08:27:17 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 09 Jun 2006 08:27:17 +0200 Subject: [Python-checkins] r46751 - python/trunk/Objects/fileobject.c In-Reply-To: References: <20060608145022.621EF1E4013@bag.python.org> Message-ID: Neal Norwitz wrote: > On 6/8/06, georg.brandl wrote: >> Author: georg.brandl >> Date: Thu Jun 8 16:50:21 2006 >> New Revision: 46751 >> >> Modified: >> python/trunk/Objects/fileobject.c >> Log: >> Bug #1502805: don't alias file.__exit__ to file.close since the >> latter can return something that's true. > > How about a test for this? I thought about it (on which occasion I converted test_file to breakbuildbots^Wunittest), but to trigger this bug, the tested file must have something in its file->f_close member which returns a true value. popen results do that, but I do not know whether this is acceptable. Georg From nnorwitz at gmail.com Fri Jun 9 08:37:53 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 8 Jun 2006 23:37:53 -0700 Subject: [Python-checkins] r46751 - python/trunk/Objects/fileobject.c In-Reply-To: References: <20060608145022.621EF1E4013@bag.python.org> Message-ID: On 6/8/06, Georg Brandl wrote: > Neal Norwitz wrote: > > On 6/8/06, georg.brandl wrote: > >> Author: georg.brandl > >> Date: Thu Jun 8 16:50:21 2006 > >> New Revision: 46751 > >> > >> Modified: > >> python/trunk/Objects/fileobject.c > >> Log: > >> Bug #1502805: don't alias file.__exit__ to file.close since the > >> latter can return something that's true. > > > > How about a test for this? > > I thought about it (on which occasion I converted test_file to > breakbuildbots^Wunittest), but to trigger this bug, the tested > file must have something in its file->f_close member which > returns a true value. popen results do that, but I do not know > whether this is acceptable. Even a simple test that just verifies it works and handles conditions like what if it is called when the file is closed would be good enough for now. The coverage helps demonstrate the code doesn't leak refs. You can call __exit__ directly and verify the conditions without using the with statement or popen. n From buildbot at python.org Fri Jun 9 08:53:47 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 06:53:47 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060609065347.B992F1E4009@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/610 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From fredrik at pythonware.com Fri Jun 9 10:17:35 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 09 Jun 2006 10:17:35 +0200 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: <1f7befae0606082121u913571es82cdcd8687bf3d3e@mail.gmail.com> References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081820p27c1d827n95a4a59f365cdfeb@mail.gmail.com> <1f7befae0606081905n32af04beqf37236a2bffe4034@mail.gmail.com> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> <1f7befae0606082042n7a178acdpb8bcfcd2de0aa787@mail.gmail.com> <1f7befae0606082121u913571es82cdcd8687bf3d3e@mail.gmail.com> Message-ID: Tim Peters wrote: > After the conversion to unittest, there was no final cleanup block, > and it just so happened that "testUnicodeOpen" has the alphabetically > largest test name, so unittest runs it last, and testUnicodeOpen() was > one of the lazy tests that didn't clean up after itself. shouldn't tests be designed so that they can be run even if a previous run failed to clean up after itself ? I generally design tests using this structure: cleanup() # establish preconditions test() if test successful: cleanup() # don't litter From neal at metaslash.com Fri Jun 9 11:17:41 2006 From: neal at metaslash.com (Neal Norwitz) Date: Fri, 9 Jun 2006 05:17:41 -0400 Subject: [Python-checkins] Python Regression Test Failures doc (1) Message-ID: <20060609091741.GA20223@python.psfb.org> TEXINPUTS=/home/neal/python/trunk/Doc/commontex: python /home/neal/python/trunk/Doc/tools/mkhowto --html --about html/stdabout.dat --iconserver ../icons --favicon ../icons/pyfav.png --address "See About this document... for information on suggesting changes." --up-link ../index.html --up-title "Python Documentation Index" --global-module-index "../modindex.html" --dvips-safe --dir html/whatsnew --split 4 whatsnew/whatsnew25.tex *** Session transcript and error messages are in /home/neal/python/trunk/Doc/html/whatsnew/whatsnew25.how. *** Exited with status 1. The relevant lines from the transcript are: ------------------------------------------------------------------------ +++ latex whatsnew25 This is TeX, Version 3.14159 (Web2C 7.4.5) (/home/neal/python/trunk/Doc/whatsnew/whatsnew25.tex LaTeX2e <2001/06/01> Babel and hyphenation patterns for american, french, german, ngerman, n ohyphenation, loaded. (/home/neal/python/trunk/Doc/texinputs/howto.cls Document Class: howto 1998/02/25 Document class (Python HOWTO) (/home/neal/python/trunk/Doc/texinputs/pypaper.sty (/usr/share/texmf/tex/latex/psnfss/times.sty) Using Times instead of Computer Modern. ) (/usr/share/texmf/tex/latex/misc/fancybox.sty Style option: `fancybox' v1.3 <2000/09/19> (tvz) ) (/usr/share/texmf/tex/latex/base/article.cls Document Class: article 2001/04/21 v1.4e Standard LaTeX document class (/usr/share/texmf/tex/latex/base/size10.clo)) (/home/neal/python/trunk/Doc/texinputs/fancyhdr.sty) Using fancier footers than usual. (/home/neal/python/trunk/Doc/texinputs/python.sty (/usr/share/texmf/tex/latex/tools/longtable.sty) (/home/neal/python/trunk/Doc/texinputs/underscore.sty) (/usr/share/texmf/tex/latex/tools/verbatim.sty) (/usr/share/texmf/tex/latex/base/alltt.sty))) (/home/neal/python/trunk/Doc/texinputs/distutils.sty) No file whatsnew25.aux. (/usr/share/texmf/tex/latex/psnfss/ot1ptm.fd) (/usr/share/texmf/tex/latex/psnfss/ot1phv.fd) No file whatsnew25.toc. (/usr/share/texmf/tex/latex/psnfss/ot1pcr.fd) [1] [2]) Runaway argument? {update_wrapper(\var {wrapper, \var {wrapped})} function that helps y\ETC. ! File ended while scanning use of \function. \par <*> whatsnew25 ? ! Emergency stop. \par <*> whatsnew25 Output written on whatsnew25.dvi (2 pages, 6692 bytes). Transcript written on whatsnew25.log. *** Session transcript and error messages are in /home/neal/python/trunk/Doc/html/whatsnew/whatsnew25.how. *** Exited with status 1. +++ TEXINPUTS=/home/neal/python/trunk/Doc/whatsnew:/home/neal/python/trunk/Doc/commontex:/home/neal/python/trunk/Doc/paper-letter:/home/neal/python/trunk/Doc/texinputs: +++ latex whatsnew25 make: *** [html/whatsnew/whatsnew25.html] Error 1 From buildbot at python.org Fri Jun 9 11:44:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 09:44:20 +0000 Subject: [Python-checkins] buildbot warnings in MIPS Debian trunk Message-ID: <20060609094420.5F66D1E4009@bag.python.org> The Buildbot has detected a new failure of MIPS Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/MIPS%2520Debian%2520trunk/builds/201 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 12:03:17 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Fri, 9 Jun 2006 12:03:17 +0200 (CEST) Subject: [Python-checkins] r46768 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060609100317.A775E1E400A@bag.python.org> Author: mateusz.rukowicz Date: Fri Jun 9 12:03:15 2006 New Revision: 46768 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Switched to 'limbs' instead of 'digits', some functions still calculates digits - this will be removed. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Fri Jun 9 12:03:15 2006 @@ -168,6 +168,27 @@ } } +static void +_limb_fill(long *self, long ndigits, long x) +{ + //TODO +} + +static long +_limb_get_digit(long *self, long ndigits, long i) +{ + long pos = ndigits - i - 1; + long limb = pos / LOG; + pos %= LOG; + long tmp = self[limb]; + while(pos) + { + tmp/=10; + pos --; + } + return tmp%10; +} + /* helpful macros ************************************************************/ /* if we later decide to call this module "squeaky" */ @@ -374,7 +395,10 @@ res = _new_decimalobj(type, thing->ob_size, sign, 0); if (!res) return NULL; for (i = 0; i < thing->ob_size; i++) - res->digits[i] = thing->digits[i]; + res->digits[i] = thing->digits[i]; /* DELETE */ + + for (i = 0; i< res->limb_count;i++) + res->limbs[i] = thing->limbs[i]; return res; } @@ -478,6 +502,7 @@ if (res) { for (i = 0; i < ctx->prec; i++) res->digits[i] = 9; + _limb_fill(res->limbs, ctx->prec, 9); return res; } } @@ -490,6 +515,7 @@ if (res) { for (i = 0; i < ctx->prec; i++) res->digits[i] = 9; + _limb_fill(res->limbs, ctx->prec, 9); return res; } } @@ -666,10 +692,10 @@ i = 0; while(new->limbs[i] >= BASE) { + assert(i+1 < new->limb_count); new->limbs[i] -= BASE; new->limbs[i+1] ++; i++; - assert(i+1 < new->limb_count); } return new; } @@ -700,9 +726,9 @@ new->digits[i] = self->digits[i]; _limb_first_n_digits(self->limbs, self->ob_size, 0, new->limbs, prec); - if (!new) return NULL; for (i = prec; i < self->ob_size; i++) - if (self->digits[i] > 0) { + if(_limb_get_digit(self->limbs,self->ob_size, i) > 0){ /* SLOW */ +// if (self->digits[i] > 0) { new2 = _decimal_increment(new, 1, ctx); Py_DECREF(new); if (!new2) @@ -728,8 +754,9 @@ { decimalobject *new; assert(expdiff > 0); - if (self->ob_size > prec && self->digits[prec] >= 5) { - new = _decimal_increment(tmp, 1, ctx); +// if (self->ob_size > prec && self->digits[prec] >= 5) { + if(self->ob_size > prec && _limb_get_digit(self->limbs, self->ob_size, prec) >= 5){ /* SLOW */ + new = _decimal_increment(tmp, 1, ctx); Py_DECREF(tmp); if (!new) return NULL; if (new->ob_size > prec) { @@ -756,11 +783,11 @@ tmp->digits[i] = self->digits[i]; last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); - last = self->digits[prec]; assert(self->digits[prec] == last); if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { - if (self->digits[i] != 0) + if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ + // if (self->digits[i] != 0) return _do_round_half_up(self, prec, expdiff, ctx, tmp); } /* self ends in 5000...., so tmp is okay */ @@ -785,11 +812,13 @@ assert(last == self->digits[prec]); if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { - if (self->digits[i] != 0) + if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ +// if (self->digits[i] != 0) return _do_round_half_up(self, prec, expdiff, ctx, tmp); } - if ((self->digits[prec-1] & 1) == 0) - return tmp; +// if ((self->digits[prec-1] & 1) == 0) + if((_limb_get_digit(self->limbs, self->ob_size, i)&1) == 0) + return tmp; } return _do_round_half_up(self, prec, expdiff, ctx, tmp); } @@ -875,6 +904,7 @@ if (!new) return NULL; while (i--) new->digits[i] = 0; + _limb_fill(new->limbs, new->ob_size,0); if (handle_Rounded(ctx, NULL) != 0) { Py_DECREF(new); @@ -889,6 +919,7 @@ new->digits[0] = 0; for (i = 1; i < new->ob_size; i++) new->digits[i] = self->digits[i-1]; + _limb_first_n_digits(self->limbs, self->ob_size, -1, new->limbs, new->ob_size); prec = 1; } else if (prec < 0) { new = _NEW_decimalobj(2, self->sign, @@ -896,6 +927,7 @@ if (!new) return NULL; new->digits[0] = 0; new->digits[1] = 1; + new->limbs[0] = 1; prec = 1; } else { new = _decimal_get_copy(self); @@ -918,14 +950,17 @@ for (i = new->ob_size; i < new2->ob_size; i++) { new2->digits[i] = 0; } + + _limb_first_n_digits(new->limbs, new->ob_size, 0, new2->limbs, new2->ob_size); Py_DECREF(new); return new2; } /* Maybe all the lost digits are 0. */ for (i = expdiff; i < self->ob_size; i++) { - if (self->digits[i] > 0) - goto no_way; +// if (self->digits[i] > 0) + if(_limb_get_digit(self->limbs, self->ob_size, i) > 0) + goto no_way; } /* All lost digits are 0, so just clobber new */ new->ob_size = prec; @@ -996,6 +1031,7 @@ if (!ans) return NULL; ans->digits[0] = 0; + ans->limbs[0] = 0; return ans; } @@ -1013,6 +1049,7 @@ return NULL; ans->digits[0] = 0; ans->digits[1] = 1; + ans->limbs[0] = 1; digits = 1; } else { ans = _NEW_decimalobj(self->ob_size+1, self->sign, self->exp); @@ -1021,6 +1058,7 @@ for (i = 0; i < self->ob_size; i++) ans->digits[i+1] = self->digits[i]; ans->digits[0] = 0; + _limb_first_n_digits(self->limbs, self->ob_size, -1, ans->limbs, ans->ob_size); } tmp = _decimal_round(ans, digits, ctx, rounding); @@ -1807,7 +1845,7 @@ } -static PyObject * +static PyObject * /* TODO review that */ _do_decimal_str(decimalobject *d, contextobject *context, int engineering) { char *outbuf; @@ -1876,8 +1914,9 @@ } if (engineering) { /* eng has no leading zeros */ for (i = 0, j = 0; i < imax; i++) { - if (d->digits[j] == 0) { - j++; +// if (d->digits[j] == 0) { + if(_limb_get_digit(d->limbs, d->ob_size, j) == 0){ + j++; } else { break; } @@ -1889,8 +1928,9 @@ SANITY_CHECK(p); continue; } else { - p += sprintf(p, "%d", d->digits[j]); - SANITY_CHECK(p); +// p += sprintf(p, "%d", d->digits[j]); + p += sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, j)); /* SLOW */ + SANITY_CHECK(p); j++; } } @@ -2453,9 +2493,12 @@ long i; if (ISSPECIAL(self)) return 1; - for (i = 0; i < self->ob_size; i++) - if (self->digits[i] != 0) return 1; - return 0; +// for (i = 0; i < self->ob_size; i++) +// if (self->digits[i] != 0) return 1; + + for(i=0;ilimb_count;i++) + if(self->limbs[i] != 0) return 1; + return 0; } @@ -2877,7 +2920,7 @@ { item = PyTuple_GET_ITEM(digtup, i); long x; - if(PyInt_AsLong(item)) + if(PyInt_Check(item)) x = PyInt_AsLong(item); else if (PyLong_Check(item)) { x = PyLong_AsLong(item); From python-checkins at python.org Fri Jun 9 12:22:35 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 12:22:35 +0200 (CEST) Subject: [Python-checkins] r46769 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060609102235.BFF291E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 12:22:35 2006 New Revision: 46769 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Markup fix Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Fri Jun 9 12:22:35 2006 @@ -173,7 +173,7 @@ Another function in the \module{functools} module is the -\function{update_wrapper(\var{wrapper, \var{wrapped})} function that +\function{update_wrapper(\var{wrapper}, \var{wrapped})} function that helps you write well-behaved decorators. \function{update_wrapper()} copies the name, module, and docstring attribute to a wrapper function so that tracebacks inside the wrapped function are easier to From python-checkins at python.org Fri Jun 9 13:05:33 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Fri, 9 Jun 2006 13:05:33 +0200 (CEST) Subject: [Python-checkins] r46770 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060609110533.CBE871E4061@bag.python.org> Author: mateusz.rukowicz Date: Fri Jun 9 13:05:33 2006 New Revision: 46770 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Redo of _decimal_fromliteral, now it results in normalized Decimal. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Fri Jun 9 13:05:33 2006 @@ -2671,105 +2671,223 @@ static decimalobject * _decimal_fromliteral(PyTypeObject *type, char *str, long len, contextobject *ctx) { - long ipos = 0; /* start of integral digits */ - long dpos = -1; /* decimal point location */ - long dend = len; /* end of integral/decimal digits (last digit+1) */ - char *p = str; - char sign = 0; - decimalobject *new; - long exp = 0, i, ndigits; + char sign; + long exp = 0; + long ndigits; + long zero = 0; /* if zero */ + long digits_after_dot = 0; + char *p = str; + char *first_digit = 0; + char *dot = 0; + char *e = 0; + char *c; + long i; + decimalobject *new; - /* optional sign */ - if (*p == '+') { - ++p; - ipos = 1; - } else if (*p == '-') { - ++p; - ipos = 1; - sign = 1; - } + if(*p == '+') + p++; + else if(*p == '-') + { + p++; + sign = 1; + } + + if(*p == 'e' || *p == 'E') /* no digits befor E */ + goto err; + + /* now at *p is our integer with (possibly) dot */ + + for(c = p; c - str < len; c++) + { + if(!isdigit(*c)) + { + if(*c == 'e' || *c == 'E') + { + if(e) + goto err; /* we should have only one e */ + e = c; + } + + else if(*c == '.') + { + if(dot) + goto err; + dot = c; + } + + else if(*c == '-' || *c == '+') /* we should have sign only after e */ + { + if(c - e != 1) + goto err; + } + else /* we shouldn't have anything else */ + goto err; + } + else + { + if(dot) + digits_after_dot ++; + + if(!first_digit && *c != '0') + { + if(!e) /* it's digit of exponent */ + first_digit = c; + } + } + } /* we now are pretty sure, that string is properly formated */ + if(!first_digit) + { + zero = 1; + ndigits = 1; + } + else /* let's count digits */ + { + ndigits = 0; + for(c = first_digit; c - str < len; c++) + { + if(isdigit(*c)) + ndigits ++; + else + if(*c == 'e' || *c == 'E') + break; + + assert(isdigit(*c) || *c == '.'); + } + } + + if(e) /* pretty obvious */ + exp = atol(e+1); + exp -= digits_after_dot; + + new = _new_decimalobj(type, ndigits, sign, exp); + + for(i = 0; ilimb_count ;i++) + new->limbs[i] = 0; + + if(!first_digit) /* we are 0 */ + { + new->limbs[0] = 0; + return new; + } + + long mult = 1; /* now we just read integer till '\0' or 'e'/'E', dont care about '.' */ + long limb = 0; + + for(c = p; c - str < len; c++) + { + if(*c == 'e' || *c == 'E') break; + if(!isdigit(*c)) continue; - do { - if (*p == '.') { - dpos = p-str; - } else if (*p == 'e' || *p == 'E') { - dend = p-str; - if (dend == ipos || (dpos >= 0 && dend == ipos+1)) + assert(limb < new->limb_count); + new->limbs[limb] += mult * (*c - '0'); + + if(mult == BASE) + { + limb ++; + mult = 1; + } + } + + return new; +// long ipos = 0; /* start of integral digits */ +// long dpos = -1; /* decimal point location */ +// long dend = len; /* end of integral/decimal digits (last digit+1) */ +// char *p = str; +// char sign = 0; +// decimalobject *new; +// long exp = 0, i, ndigits; + + /* optional sign */ +// if (*p == '+') { +// ++p; +// ipos = 1; +// } else if (*p == '-') { +// ++p; +// ipos = 1; +// sign = 1; +// } + +// do { +// if (*p == '.') { +// dpos = p-str; +// } else if (*p == 'e' || *p == 'E') { +// dend = p-str; +// if (dend == ipos || (dpos >= 0 && dend == ipos+1)) /* no digits between sign and E */ - goto err; - ++p; - break; - } else if (!isdigit(*p)) { - goto err; - } - } while (*++p); +// goto err; +// ++p; +// break; +// } else if (!isdigit(*p)) { +// goto err; +// } +// } while (*++p); /* exponential part or end of string */ - if (*p == 0) { - if (ipos == dpos && dpos == dend-1) +// if (*p == 0) { +// if (ipos == dpos && dpos == dend-1) /* no digits at all */ - goto err; - goto calculate; - } else if (*p == '-' || *p == '+') { - ++p; - } +// goto err; +// goto calculate; +// } else if (*p == '-' || *p == '+') { +// ++p; +// } - do { +// do { /* optional exponential digits */ - if (!isdigit(*p)) - goto err; - } while (*++p); - - calculate: - if (dend < len) - exp = atol(str + dend + 1); - if (dpos >= 0) +// if (!isdigit(*p)) +// goto err; +// } while (*++p); + +// calculate: +// if (dend < len) +// exp = atol(str + dend + 1); +// if (dpos >= 0) /* specified fractional digits, reduce exp */ - exp -= dend - dpos - 1; +// exp -= dend - dpos - 1; /* If it's not a zero, strip leading 0s */ - for (p = str+ipos; p-str < dend; ++p) { - if (*p != '0') { +// for (p = str+ipos; p-str < dend; ++p) { +// if (*p != '0') { /* first nonzero digit */ - ipos = (p-str); - goto finish; - } - } +// ipos = (p-str); +// goto finish; +// } +// } /* it's a zero */ - dend = (dpos == ipos ? ipos+2 : ipos+1); +// dend = (dpos == ipos ? ipos+2 : ipos+1); - finish: - ndigits = dend - ipos - (dpos<0 ? 0 : 1); - new = _new_decimalobj(type, ndigits, sign, exp); - if (!new) - return NULL; - i = 0; - for (p = str+ipos; p-str < dend; ++p) { - if (*p == '.') continue; - new->digits[i] = *p - 48; - i++; - } +// finish: +// ndigits = dend - ipos - (dpos<0 ? 0 : 1); +// new = _new_decimalobj(type, ndigits, sign, exp); +// if (!new) +// return NULL; +// i = 0; +// for (p = str+ipos; p-str < dend; ++p) { +// if (*p == '.') continue; +// new->digits[i] = *p - 48; +// i++; +// } - long mult = 1; - long limb = 0; - for(i=0;i limb_count; i++) - new->limbs[i] = 0; +// long mult = 1; +// long limb = 0; +// for(i=0;i limb_count; i++) +// new->limbs[i] = 0; - for(i = dend - 1; i>= ipos; i--) - { - if(str[i] == '.') continue; - assert(limb < new->limb_count); - new->limbs[limb] += mult * (str[i] - '0'); - - mult *= 10; - if(mult == BASE) - { - mult = 1; - limb ++; - } - } +// for(i = dend - 1; i>= ipos; i--) +// { +// if(str[i] == '.') continue; +// assert(limb < new->limb_count); +// new->limbs[limb] += mult * (str[i] - '0'); +// +// mult *= 10; +// if(mult == BASE) +// { +// mult = 1; +// limb ++; +// } +// } - return new; + // return new; err: return handle_ConversionSyntax(type, ctx, "invalid literal for Decimal"); From steve at holdenweb.com Fri Jun 9 13:29:06 2006 From: steve at holdenweb.com (Steve Holden) Date: Fri, 09 Jun 2006 12:29:06 +0100 Subject: [Python-checkins] r46770 - sandbox/trunk/decimal-c/_decimal.c In-Reply-To: <20060609110533.CBE871E4061@bag.python.org> References: <20060609110533.CBE871E4061@bag.python.org> Message-ID: <44895B82.8030505@holdenweb.com> mateusz.rukowicz wrote: > Author: mateusz.rukowicz > Date: Fri Jun 9 13:05:33 2006 > New Revision: 46770 > > Modified: > sandbox/trunk/decimal-c/_decimal.c > Log: > Redo of _decimal_fromliteral, now it results in normalized Decimal. > > > Modified: sandbox/trunk/decimal-c/_decimal.c > ============================================================================== [...] > +// for(i = dend - 1; i>= ipos; i--) > +// { > +// if(str[i] == '.') continue; > +// assert(limb < new->limb_count); > +// new->limbs[limb] += mult * (str[i] - '0'); > +// > +// mult *= 10; > +// if(mult == BASE) > +// { > +// mult = 1; > +// limb ++; > +// } > +// } > I believe this form of comment is forbidden by PEP 7. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Love me, love my blog http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From python-checkins at python.org Fri Jun 9 13:37:48 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Fri, 9 Jun 2006 13:37:48 +0200 (CEST) Subject: [Python-checkins] r46771 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060609113748.0DF1C1E400A@bag.python.org> Author: mateusz.rukowicz Date: Fri Jun 9 13:37:47 2006 New Revision: 46771 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Bug fixed. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Fri Jun 9 13:37:47 2006 @@ -2678,6 +2678,7 @@ long digits_after_dot = 0; char *p = str; char *first_digit = 0; + char *last_digit = 0; char *dot = 0; char *e = 0; char *c; @@ -2740,13 +2741,16 @@ zero = 1; ndigits = 1; } - else /* let's count digits */ + else /* let's count digits and find by the way last digit*/ { ndigits = 0; for(c = first_digit; c - str < len; c++) { if(isdigit(*c)) + { ndigits ++; + last_digit = c; + } else if(*c == 'e' || *c == 'E') break; @@ -2773,7 +2777,7 @@ long mult = 1; /* now we just read integer till '\0' or 'e'/'E', dont care about '.' */ long limb = 0; - for(c = p; c - str < len; c++) + for(c = last_digit; c >= first_digit; c--) { if(*c == 'e' || *c == 'E') break; if(!isdigit(*c)) continue; @@ -2781,6 +2785,7 @@ assert(limb < new->limb_count); new->limbs[limb] += mult * (*c - '0'); + mult *= 10; if(mult == BASE) { limb ++; From fredrik at pythonware.com Fri Jun 9 13:57:27 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 09 Jun 2006 13:57:27 +0200 Subject: [Python-checkins] r46770 - sandbox/trunk/decimal-c/_decimal.c In-Reply-To: <44895B82.8030505@holdenweb.com> References: <20060609110533.CBE871E4061@bag.python.org> <44895B82.8030505@holdenweb.com> Message-ID: Steve Holden wrote: > I believe this form of comment is forbidden by PEP 7. it forbidden by ANSI/ISO C89, which is the C version used by Python. From python-checkins at python.org Fri Jun 9 15:11:52 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 15:11:52 +0200 (CEST) Subject: [Python-checkins] r46772 - sandbox/trunk/tests Message-ID: <20060609131152.D0A021E401F@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 15:11:52 2006 New Revision: 46772 Added: sandbox/trunk/tests/ Log: Create directory for tests being worked on From python-checkins at python.org Fri Jun 9 15:15:57 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 15:15:57 +0200 (CEST) Subject: [Python-checkins] r46773 - in python/trunk/Lib: test/test_sax.py xmlcore/sax/saxutils.py Message-ID: <20060609131557.E105F1E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 15:15:57 2006 New Revision: 46773 Modified: python/trunk/Lib/test/test_sax.py python/trunk/Lib/xmlcore/sax/saxutils.py Log: [Bug #1472827] Make saxutils.XMLGenerator handle \r\n\t in attribute values by escaping them properly. 2.4 bugfix candidate. Modified: python/trunk/Lib/test/test_sax.py ============================================================================== --- python/trunk/Lib/test/test_sax.py (original) +++ python/trunk/Lib/test/test_sax.py Fri Jun 9 15:15:57 2006 @@ -175,11 +175,14 @@ gen.endElement("e") gen.startElement("e", {"a": "'\""}) gen.endElement("e") + gen.startElement("e", {"a": "\n\r\t"}) + gen.endElement("e") gen.endElement("doc") gen.endDocument() - return result.getvalue() == start \ - + "" + return result.getvalue() == start + ("" + "" + "") def test_xmlgen_ignorable(): result = StringIO() Modified: python/trunk/Lib/xmlcore/sax/saxutils.py ============================================================================== --- python/trunk/Lib/xmlcore/sax/saxutils.py (original) +++ python/trunk/Lib/xmlcore/sax/saxutils.py Fri Jun 9 15:15:57 2006 @@ -68,6 +68,8 @@ the optional entities parameter. The keys and values must all be strings; each key will be replaced with its corresponding value. """ + entities = entities.copy() + entities.update({'\n': ' ', '\r': ' ', '\t':' '}) data = escape(data, entities) if '"' in data: if "'" in data: From python-checkins at python.org Fri Jun 9 15:17:15 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 15:17:15 +0200 (CEST) Subject: [Python-checkins] r46774 - sandbox/trunk/tests/test_sax.py Message-ID: <20060609131715.EEC701E4019@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 15:17:15 2006 New Revision: 46774 Added: sandbox/trunk/tests/test_sax.py (contents, props changed) Log: Start at rewriting test_sax to use doctest -- but it turns out to be a big job! Added: sandbox/trunk/tests/test_sax.py ============================================================================== --- (empty file) +++ sandbox/trunk/tests/test_sax.py Fri Jun 9 15:17:15 2006 @@ -0,0 +1,735 @@ +# regression test for SAX 2.0 -*- coding: iso-8859-1 -*- +# $Id$ + +from xmlcore.sax import make_parser, ContentHandler, \ + SAXException, SAXReaderNotAvailable, SAXParseException +try: + make_parser() +except SAXReaderNotAvailable: + # don't try to test this module if we cannot create a parser + raise ImportError("no XML parsers available") +from xmlcore.sax.saxutils import XMLGenerator, escape, unescape, quoteattr, \ + XMLFilterBase +from xmlcore.sax.expatreader import create_parser +from xmlcore.sax.xmlreader import InputSource, AttributesImpl, AttributesNSImpl +from cStringIO import StringIO +from test.test_support import verify, verbose, TestFailed, findfile, run_doctest +import os + +# ===== Utilities + +tests = 0 +failures = [] + +def confirm(outcome, name): + global tests + + tests = tests + 1 + if outcome: + if verbose: + print "Passed", name + else: + failures.append(name) + +def test_make_parser2(): + try: + # Creating parsers several times in a row should succeed. + # Testing this because there have been failures of this kind + # before. + from xmlcore.sax import make_parser + p = make_parser() + from xmlcore.sax import make_parser + p = make_parser() + from xmlcore.sax import make_parser + p = make_parser() + from xmlcore.sax import make_parser + p = make_parser() + from xmlcore.sax import make_parser + p = make_parser() + from xmlcore.sax import make_parser + p = make_parser() + except: + return 0 + else: + return p + + +# =========================================================================== +# +# saxutils tests +# +# =========================================================================== + +# ===== escape + +def test_escape_basic(): + return escape("Donald Duck & Co") == "Donald Duck & Co" + +def test_escape_all(): + return escape("") == "<Donald Duck & Co>" + +def test_escape_extra(): + return escape("Hei p� deg", {"�" : "å"}) == "Hei på deg" + +# ===== unescape + +def test_unescape_basic(): + return unescape("Donald Duck & Co") == "Donald Duck & Co" + +def test_unescape_all(): + return unescape("<Donald Duck & Co>") == "" + +def test_unescape_extra(): + return unescape("Hei p� deg", {"�" : "å"}) == "Hei på deg" + +def test_unescape_amp_extra(): + return unescape("&foo;", {"&foo;": "splat"}) == "&foo;" + +# ===== quoteattr + +def test_quoteattr_basic(): + return quoteattr("Donald Duck & Co") == '"Donald Duck & Co"' + +def test_single_quoteattr(): + return (quoteattr('Includes "double" quotes') + == '\'Includes "double" quotes\'') + +def test_double_quoteattr(): + return (quoteattr("Includes 'single' quotes") + == "\"Includes 'single' quotes\"") + +def test_single_double_quoteattr(): + return (quoteattr("Includes 'single' and \"double\" quotes") + == "\"Includes 'single' and "double" quotes\"") + +# ===== make_parser + +def test_make_parser(): + try: + # Creating a parser should succeed - it should fall back + # to the expatreader + p = make_parser(['xmlcore.parsers.no_such_parser']) + except: + return 0 + else: + return p + + +# ===== XMLGenerator + +start = '\n' + +def test_xmlgen_basic(): + result = StringIO() + gen = XMLGenerator(result) + gen.startDocument() + gen.startElement("doc", {}) + gen.endElement("doc") + gen.endDocument() + + return result.getvalue() == start + "" + +def test_xmlgen_content(): + result = StringIO() + gen = XMLGenerator(result) + + gen.startDocument() + gen.startElement("doc", {}) + gen.characters("huhei") + gen.endElement("doc") + gen.endDocument() + + return result.getvalue() == start + "huhei" + +def test_xmlgen_pi(): + result = StringIO() + gen = XMLGenerator(result) + + gen.startDocument() + gen.processingInstruction("test", "data") + gen.startElement("doc", {}) + gen.endElement("doc") + gen.endDocument() + + return result.getvalue() == start + "" + +def test_xmlgen_content_escape(): + result = StringIO() + gen = XMLGenerator(result) + + gen.startDocument() + gen.startElement("doc", {}) + gen.characters("<huhei&" + +def test_xmlgen_attr_escape(): + result = StringIO() + gen = XMLGenerator(result) + + gen.startDocument() + gen.startElement("doc", {"a": '"'}) + gen.startElement("e", {"a": "'"}) + gen.endElement("e") + gen.startElement("e", {"a": "'\""}) + gen.endElement("e") + gen.startElement("e", {"a": "\n\r\t"}) + gen.endElement("e") + gen.endElement("doc") + gen.endDocument() + + return result.getvalue() == start + ("" + "" + "") + +def test_xmlgen_ignorable(): + result = StringIO() + gen = XMLGenerator(result) + + gen.startDocument() + gen.startElement("doc", {}) + gen.ignorableWhitespace(" ") + gen.endElement("doc") + gen.endDocument() + + return result.getvalue() == start + " " + +ns_uri = "http://www.python.org/xml-ns/saxtest/" + +def test_xmlgen_ns(): + result = StringIO() + gen = XMLGenerator(result) + + gen.startDocument() + gen.startPrefixMapping("ns1", ns_uri) + gen.startElementNS((ns_uri, "doc"), "ns1:doc", {}) + # add an unqualified name + gen.startElementNS((None, "udoc"), None, {}) + gen.endElementNS((None, "udoc"), None) + gen.endElementNS((ns_uri, "doc"), "ns1:doc") + gen.endPrefixMapping("ns1") + gen.endDocument() + + return result.getvalue() == start + \ + ('' % + ns_uri) + +# ===== XMLFilterBase + +def test_filter_basic(): + result = StringIO() + gen = XMLGenerator(result) + filter = XMLFilterBase() + filter.setContentHandler(gen) + + filter.startDocument() + filter.startElement("doc", {}) + filter.characters("content") + filter.ignorableWhitespace(" ") + filter.endElement("doc") + filter.endDocument() + + return result.getvalue() == start + "content " + +# =========================================================================== +# +# expatreader tests +# +# =========================================================================== + +# ===== XMLReader support + +def test_expat_file(): + parser = create_parser() + result = StringIO() + xmlgen = XMLGenerator(result) + + parser.setContentHandler(xmlgen) + parser.parse(open(findfile("test"+os.extsep+"xml"))) + + return result.getvalue() == xml_test_out + +# ===== DTDHandler support + +class TestDTDHandler: + + def __init__(self): + self._notations = [] + self._entities = [] + + def notationDecl(self, name, publicId, systemId): + self._notations.append((name, publicId, systemId)) + + def unparsedEntityDecl(self, name, publicId, systemId, ndata): + self._entities.append((name, publicId, systemId, ndata)) + +def test_expat_dtdhandler(): + parser = create_parser() + handler = TestDTDHandler() + parser.setDTDHandler(handler) + + parser.feed('\n') + parser.feed(' \n') + parser.feed(']>\n') + parser.feed('') + parser.close() + + return handler._notations == [("GIF", "-//CompuServe//NOTATION Graphics Interchange Format 89a//EN", None)] and \ + handler._entities == [("img", None, "expat.gif", "GIF")] + +# ===== EntityResolver support + +class TestEntityResolver: + + def resolveEntity(self, publicId, systemId): + inpsrc = InputSource() + inpsrc.setByteStream(StringIO("")) + return inpsrc + +def test_expat_entityresolver(): + parser = create_parser() + parser.setEntityResolver(TestEntityResolver()) + result = StringIO() + parser.setContentHandler(XMLGenerator(result)) + + parser.feed('\n') + parser.feed(']>\n') + parser.feed('&test;') + parser.close() + + return result.getvalue() == start + "" + +# ===== Attributes support + +class AttrGatherer(ContentHandler): + + def startElement(self, name, attrs): + self._attrs = attrs + + def startElementNS(self, name, qname, attrs): + self._attrs = attrs + +def test_expat_attrs_empty(): + parser = create_parser() + gather = AttrGatherer() + parser.setContentHandler(gather) + + parser.feed("") + parser.close() + + return verify_empty_attrs(gather._attrs) + +def test_expat_attrs_wattr(): + parser = create_parser() + gather = AttrGatherer() + parser.setContentHandler(gather) + + parser.feed("") + parser.close() + + return verify_attrs_wattr(gather._attrs) + +def test_expat_nsattrs_empty(): + parser = create_parser(1) + gather = AttrGatherer() + parser.setContentHandler(gather) + + parser.feed("") + parser.close() + + return verify_empty_nsattrs(gather._attrs) + +def test_expat_nsattrs_wattr(): + parser = create_parser(1) + gather = AttrGatherer() + parser.setContentHandler(gather) + + parser.feed("" % ns_uri) + parser.close() + + attrs = gather._attrs + + return attrs.getLength() == 1 and \ + attrs.getNames() == [(ns_uri, "attr")] and \ + (attrs.getQNames() == [] or attrs.getQNames() == ["ns:attr"]) and \ + len(attrs) == 1 and \ + attrs.has_key((ns_uri, "attr")) and \ + attrs.keys() == [(ns_uri, "attr")] and \ + attrs.get((ns_uri, "attr")) == "val" and \ + attrs.get((ns_uri, "attr"), 25) == "val" and \ + attrs.items() == [((ns_uri, "attr"), "val")] and \ + attrs.values() == ["val"] and \ + attrs.getValue((ns_uri, "attr")) == "val" and \ + attrs[(ns_uri, "attr")] == "val" + +# ===== InputSource support + +xml_test_out = open(findfile("test"+os.extsep+"xml"+os.extsep+"out")).read() + +def test_expat_inpsource_filename(): + parser = create_parser() + result = StringIO() + xmlgen = XMLGenerator(result) + + parser.setContentHandler(xmlgen) + parser.parse(findfile("test"+os.extsep+"xml")) + + return result.getvalue() == xml_test_out + +def test_expat_inpsource_sysid(): + parser = create_parser() + result = StringIO() + xmlgen = XMLGenerator(result) + + parser.setContentHandler(xmlgen) + parser.parse(InputSource(findfile("test"+os.extsep+"xml"))) + + return result.getvalue() == xml_test_out + +def test_expat_inpsource_stream(): + parser = create_parser() + result = StringIO() + xmlgen = XMLGenerator(result) + + parser.setContentHandler(xmlgen) + inpsrc = InputSource() + inpsrc.setByteStream(open(findfile("test"+os.extsep+"xml"))) + parser.parse(inpsrc) + + return result.getvalue() == xml_test_out + +# ===== IncrementalParser support + +def test_expat_incremental(): + result = StringIO() + xmlgen = XMLGenerator(result) + parser = create_parser() + parser.setContentHandler(xmlgen) + + parser.feed("") + parser.feed("") + parser.close() + + return result.getvalue() == start + "" + +def test_expat_incremental_reset(): + result = StringIO() + xmlgen = XMLGenerator(result) + parser = create_parser() + parser.setContentHandler(xmlgen) + + parser.feed("") + parser.feed("text") + + result = StringIO() + xmlgen = XMLGenerator(result) + parser.setContentHandler(xmlgen) + parser.reset() + + parser.feed("") + parser.feed("text") + parser.feed("") + parser.close() + + return result.getvalue() == start + "text" + +# ===== Locator support + +def test_expat_locator_noinfo(): + result = StringIO() + xmlgen = XMLGenerator(result) + parser = create_parser() + parser.setContentHandler(xmlgen) + + parser.feed("") + parser.feed("") + parser.close() + + return parser.getSystemId() is None and \ + parser.getPublicId() is None and \ + parser.getLineNumber() == 1 + +def test_expat_locator_withinfo(): + result = StringIO() + xmlgen = XMLGenerator(result) + parser = create_parser() + parser.setContentHandler(xmlgen) + parser.parse(findfile("test.xml")) + + return parser.getSystemId() == findfile("test.xml") and \ + parser.getPublicId() is None + + +# =========================================================================== +# +# error reporting +# +# =========================================================================== + +def test_expat_inpsource_location(): + """ + >>> parser = create_parser() + >>> parser.setContentHandler(ContentHandler()) # do nothing + >>> source = InputSource() + >>> source.setByteStream(StringIO("")) #ill-formed + >>> name = "a file name" + >>> source.setSystemId(name) + >>> parser.parse(source) + Traceback (most recent call last): + ... + SAXParseException: a file name:1:9: not well-formed (invalid token) + >>> source.setByteStream(StringIO("")) #ill-formed + >>> try: + ... parser.parse(source) + ... except SAXParseException, e: + ... e.getSystemId() + 'a file name' + """ + +def test_expat_incomplete(): + """ + >>> parser = create_parser() + >>> parser.setContentHandler(ContentHandler()) # do nothing + >>> parser.parse(StringIO("")) + Traceback (most recent call last): + ... + SAXParseException: :1:5: no element found + + """ + +def test_sax_parse_exception_str(): + """ + pass various values from a locator to the SAXParseException to + make sure that the __str__() doesn't fall apart when None is + passed instead of an integer line and column number + + # use "normal" values for the locator: + >>> str(SAXParseException("message", None, DummyLocator(1, 1))) + 'sysid:1:1: message' + + # use None for the line number: + >>> str(SAXParseException("message", None, DummyLocator(None, 1))) + 'sysid:?:1: message' + + # use None for the column number: + >>> str(SAXParseException("message", None, DummyLocator(1, None))) + 'sysid:1:?: message' + + # use None for both: + >>> str(SAXParseException("message", None, DummyLocator(None, None))) + 'sysid:?:?: message' + """ + +class DummyLocator: + def __init__(self, lineno, colno): + self._lineno = lineno + self._colno = colno + + def getPublicId(self): + return "pubid" + + def getSystemId(self): + return "sysid" + + def getLineNumber(self): + return self._lineno + + def getColumnNumber(self): + return self._colno + +# =========================================================================== +# +# xmlreader tests +# +# =========================================================================== + +# ===== AttributesImpl + +def verify_empty_attrs(attrs): + try: + attrs.getValue("attr") + gvk = 0 + except KeyError: + gvk = 1 + + try: + attrs.getValueByQName("attr") + gvqk = 0 + except KeyError: + gvqk = 1 + + try: + attrs.getNameByQName("attr") + gnqk = 0 + except KeyError: + gnqk = 1 + + try: + attrs.getQNameByName("attr") + gqnk = 0 + except KeyError: + gqnk = 1 + + try: + attrs["attr"] + gik = 0 + except KeyError: + gik = 1 + + return attrs.getLength() == 0 and \ + attrs.getNames() == [] and \ + attrs.getQNames() == [] and \ + len(attrs) == 0 and \ + not attrs.has_key("attr") and \ + attrs.keys() == [] and \ + attrs.get("attrs") is None and \ + attrs.get("attrs", 25) == 25 and \ + attrs.items() == [] and \ + attrs.values() == [] and \ + gvk and gvqk and gnqk and gik and gqnk + +def verify_attrs_wattr(attrs): + return attrs.getLength() == 1 and \ + attrs.getNames() == ["attr"] and \ + attrs.getQNames() == ["attr"] and \ + len(attrs) == 1 and \ + attrs.has_key("attr") and \ + attrs.keys() == ["attr"] and \ + attrs.get("attr") == "val" and \ + attrs.get("attr", 25) == "val" and \ + attrs.items() == [("attr", "val")] and \ + attrs.values() == ["val"] and \ + attrs.getValue("attr") == "val" and \ + attrs.getValueByQName("attr") == "val" and \ + attrs.getNameByQName("attr") == "attr" and \ + attrs["attr"] == "val" and \ + attrs.getQNameByName("attr") == "attr" + +def test_attrs_empty(): + return verify_empty_attrs(AttributesImpl({})) + +def test_attrs_wattr(): + return verify_attrs_wattr(AttributesImpl({"attr" : "val"})) + +# ===== AttributesImpl + +def verify_empty_nsattrs(attrs): + try: + attrs.getValue((ns_uri, "attr")) + gvk = 0 + except KeyError: + gvk = 1 + + try: + attrs.getValueByQName("ns:attr") + gvqk = 0 + except KeyError: + gvqk = 1 + + try: + attrs.getNameByQName("ns:attr") + gnqk = 0 + except KeyError: + gnqk = 1 + + try: + attrs.getQNameByName((ns_uri, "attr")) + gqnk = 0 + except KeyError: + gqnk = 1 + + try: + attrs[(ns_uri, "attr")] + gik = 0 + except KeyError: + gik = 1 + + return attrs.getLength() == 0 and \ + attrs.getNames() == [] and \ + attrs.getQNames() == [] and \ + len(attrs) == 0 and \ + not attrs.has_key((ns_uri, "attr")) and \ + attrs.keys() == [] and \ + attrs.get((ns_uri, "attr")) is None and \ + attrs.get((ns_uri, "attr"), 25) == 25 and \ + attrs.items() == [] and \ + attrs.values() == [] and \ + gvk and gvqk and gnqk and gik and gqnk + +def test_nsattrs_empty(): + return verify_empty_nsattrs(AttributesNSImpl({}, {})) + +def test_nsattrs_wattr(): + """ + >>> attrs = AttributesNSImpl({(ns_uri, "attr") : "val"}, + ... {(ns_uri, "attr") : "ns:attr"}) + >>> attrs.getLength() + 1 + >>> attrs.getNames() + [('http://www.python.org/xml-ns/saxtest/', 'attr')] + >>> attrs.getQNames() + ['ns:attr'] + >>> len(attrs) + 1 + >>> attrs.has_key((ns_uri, 'attr')) + True + >>> attrs.get((ns_uri, "attr")) + 'val' + >>> attrs.get((ns_uri, "attr"), 25) + 'val' + >>> attrs.items() + [(('http://www.python.org/xml-ns/saxtest/', 'attr'), 'val')] + >>> attrs.values() + ['val'] + >>> attrs.getValue((ns_uri, "attr")) + 'val' + >>> attrs.getValueByQName("ns:attr") + 'val' + >>> attrs.getNameByQName("ns:attr" + ... ) + ('http://www.python.org/xml-ns/saxtest/', 'attr') + >>> attrs[(ns_uri, "attr")] + 'val' + >>> attrs.getQNameByName((ns_uri, "attr")) + 'ns:attr' + >>> attrs.getQNameByName((ns_uri, "bogus")) + Traceback (most recent call last): + File "", line 1, in + File "/home/amk/source/p/python/Lib/xmlcore/sax/xmlreader.py", line 366, in getQNameByName + return self._qnames[name] + KeyError: ('http://www.python.org/xml-ns/saxtest/', 'bogus') + """ + + +# ===== Main program +def main(): + import __main__ + items = __main__.__dict__.items() + items.sort() + for (name, value) in items: + if name.startswith("test_"): + if not value.__doc__: + print name, value + confirm(value(), name) + + # We delete the items variable so that the assignment to items above + # doesn't pick up the old value of items (which messes with attempts + # to find reference leaks). + del items + + if verbose: + print "%d tests, %d failures" % (tests, len(failures)) + if failures: + raise TestFailed("%d of %d tests failed: %s" + % (len(failures), tests, ", ".join(failures))) + + from test import test_sax + run_doctest(test_sax, verbosity=True) + + +if __name__ == '__main__': + main() From python-checkins at python.org Fri Jun 9 15:21:08 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 15:21:08 +0200 (CEST) Subject: [Python-checkins] r46775 - sandbox/trunk/tests/test_saxgen.py Message-ID: <20060609132108.E8A881E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 15:21:08 2006 New Revision: 46775 Added: sandbox/trunk/tests/test_saxgen.py (contents, props changed) Log: Then I tried writing a fresh set of tests for XMLGenerator using doctest. This test suite is complete, but duplicates the testing performed in test_sax.py. I'll leave this file out of the trunk for now. Added: sandbox/trunk/tests/test_saxgen.py ============================================================================== --- (empty file) +++ sandbox/trunk/tests/test_saxgen.py Fri Jun 9 15:21:08 2006 @@ -0,0 +1,57 @@ +"""Test of the saxutils.XMLGenerator class. + +$Id$ +""" + +import StringIO +from test import test_support +from xmlcore.sax.saxutils import XMLGenerator + +def test_basic (): + """ + >>> gen = XMLGenerator() + >>> gen.startDocument() + + >>> gen.startElement('root', dict(attr1='value1', attr2 = '<, >, and &')) + + >>> gen.characters('Paragraph containing <, >, and &') + Paragraph containing <, >, and & + >>> gen.ignorableWhitespace('\\n\\n') + + + >>> gen.processingInstruction('xml-stylesheet', 'http://example.com/css') + + >>> gen.endElement('root') + + """ + +def test_ns (): + """ + >>> gen = XMLGenerator() + >>> ns_uri = 'http://example.com/xbel' + >>> gen.startDocument() + + >>> gen.startPrefixMapping('xbel', ns_uri) + >>> gen.startElementNS((ns_uri, 'xbel'), 'xbel:xbel', + ... {(ns_uri, 'version'):'1.0'}) + + >>> gen.startElementNS((ns_uri, 'title'), 'xbel:title', {}) + + >>> gen.endElementNS((ns_uri, 'title'), 'xbel:title') + + >>> gen.startElementNS((None, 'about'), 'about', {}) + + >>> gen.endElementNS((None, 'about'), 'about') + + >>> gen.endElementNS((ns_uri, 'xbel'), 'xbel:xbel') + + >>> gen.endPrefixMapping('xbel') + """ + +def main(): + from test import test_saxgen + test_support.run_doctest(test_saxgen, verbosity=True) + + +if __name__ == '__main__': + main() From python-checkins at python.org Fri Jun 9 15:55:01 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 15:55:01 +0200 (CEST) Subject: [Python-checkins] r46776 - python/branches/release24-maint/Modules/zlibmodule.c Message-ID: <20060609135501.3B1E51E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 15:55:00 2006 New Revision: 46776 Modified: python/branches/release24-maint/Modules/zlibmodule.c Log: Check for error on module creation Modified: python/branches/release24-maint/Modules/zlibmodule.c ============================================================================== --- python/branches/release24-maint/Modules/zlibmodule.c (original) +++ python/branches/release24-maint/Modules/zlibmodule.c Fri Jun 9 15:55:00 2006 @@ -880,6 +880,8 @@ m = Py_InitModule4("zlib", zlib_methods, zlib_module_documentation, (PyObject*)NULL,PYTHON_API_VERSION); + if (m == NULL) + return; ZlibError = PyErr_NewException("zlib.error", NULL, NULL); if (ZlibError != NULL) { From python-checkins at python.org Fri Jun 9 16:00:16 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 16:00:16 +0200 (CEST) Subject: [Python-checkins] r46777 - in python/branches/release24-maint: Lib/test/test_sax.py Lib/xml/sax/saxutils.py Misc/NEWS Message-ID: <20060609140016.15AAD1E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 16:00:15 2006 New Revision: 46777 Modified: python/branches/release24-maint/Lib/test/test_sax.py python/branches/release24-maint/Lib/xml/sax/saxutils.py python/branches/release24-maint/Misc/NEWS Log: [Bug #1472827] Correctly escape newlines and tabs in attribute values in saxutils.XMLGenerator Modified: python/branches/release24-maint/Lib/test/test_sax.py ============================================================================== --- python/branches/release24-maint/Lib/test/test_sax.py (original) +++ python/branches/release24-maint/Lib/test/test_sax.py Fri Jun 9 16:00:15 2006 @@ -175,11 +175,14 @@ gen.endElement("e") gen.startElement("e", {"a": "'\""}) gen.endElement("e") + gen.startElement("e", {"a": "\n\r\t"}) + gen.endElement("e") gen.endElement("doc") gen.endDocument() - return result.getvalue() == start \ - + "" + return result.getvalue() == start + ("" + "" + "") def test_xmlgen_ignorable(): result = StringIO() Modified: python/branches/release24-maint/Lib/xml/sax/saxutils.py ============================================================================== --- python/branches/release24-maint/Lib/xml/sax/saxutils.py (original) +++ python/branches/release24-maint/Lib/xml/sax/saxutils.py Fri Jun 9 16:00:15 2006 @@ -68,6 +68,8 @@ the optional entities parameter. The keys and values must all be strings; each key will be replaced with its corresponding value. """ + entities = entities.copy() + entities.update({'\n': ' ', '\r': ' ', '\t':' '}) data = escape(data, entities) if '"' in data: if "'" in data: Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Fri Jun 9 16:00:15 2006 @@ -69,6 +69,10 @@ raise an exception some day. But dicts have been allowed, and "mostly worked", so support for them won't go away without warning. +- Bug #1472827: correctly escape newlines and tabs in attribute values in + the saxutils.XMLGenerator class. + + Tools/Demos ----------- From python-checkins at python.org Fri Jun 9 18:28:03 2006 From: python-checkins at python.org (kristjan.jonsson) Date: Fri, 9 Jun 2006 18:28:03 +0200 (CEST) Subject: [Python-checkins] r46778 - in python/trunk: Modules/posixmodule.c PC/pyconfig.h Message-ID: <20060609162803.84A941E400A@bag.python.org> Author: kristjan.jonsson Date: Fri Jun 9 18:28:01 2006 New Revision: 46778 Modified: python/trunk/Modules/posixmodule.c python/trunk/PC/pyconfig.h Log: Turn off warning about deprecated CRT functions on for VisualStudio .NET 2005. Make the definition #ARRAYSIZE conditional. VisualStudio .NET 2005 already has it defined using a better gimmick. Modified: python/trunk/Modules/posixmodule.c ============================================================================== --- python/trunk/Modules/posixmodule.c (original) +++ python/trunk/Modules/posixmodule.c Fri Jun 9 18:28:01 2006 @@ -1227,7 +1227,9 @@ #define ISSLASHA(c) ((c) == '\\' || (c) == '/') #define ISSLASHW(c) ((c) == L'\\' || (c) == L'/') +#ifndef ARRAYSIZE #define ARRAYSIZE(a) (sizeof(a) / sizeof(a[0])) +#endif static BOOL IsUNCRootA(char *path, int pathlen) Modified: python/trunk/PC/pyconfig.h ============================================================================== --- python/trunk/PC/pyconfig.h (original) +++ python/trunk/PC/pyconfig.h Fri Jun 9 18:28:01 2006 @@ -164,6 +164,12 @@ #define Py_IS_INFINITY(X) (!_finite(X) && !_isnan(X)) #define Py_IS_FINITE(X) _finite(X) +/* Turn off warnings about deprecated C runtime functions in + VisualStudio .NET 2005 */ +#if _MSC_VER >= 1400 && !defined _CRT_SECURE_NO_DEPRECATE +#define _CRT_SECURE_NO_DEPRECATE +#endif + #endif /* _MSC_VER */ /* define some ANSI types that are not defined in earlier Win headers */ From python-checkins at python.org Fri Jun 9 18:40:20 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 18:40:20 +0200 (CEST) Subject: [Python-checkins] r46779 - in python/trunk: Doc/lib/lib.tex Doc/lib/libwsgiref.tex Lib/test/test_wsgiref.py Lib/wsgiref Lib/wsgiref.egg-info Lib/wsgiref/__init__.py Lib/wsgiref/handlers.py Lib/wsgiref/headers.py Lib/wsgiref/simple_server.py Lib/wsgiref/util.py Lib/wsgiref/validate.py Message-ID: <20060609164020.091291E400A@bag.python.org> Author: phillip.eby Date: Fri Jun 9 18:40:18 2006 New Revision: 46779 Added: python/trunk/Doc/lib/libwsgiref.tex (contents, props changed) python/trunk/Lib/test/test_wsgiref.py (contents, props changed) python/trunk/Lib/wsgiref/ python/trunk/Lib/wsgiref.egg-info python/trunk/Lib/wsgiref/__init__.py (contents, props changed) python/trunk/Lib/wsgiref/handlers.py (contents, props changed) python/trunk/Lib/wsgiref/headers.py (contents, props changed) python/trunk/Lib/wsgiref/simple_server.py (contents, props changed) python/trunk/Lib/wsgiref/util.py (contents, props changed) python/trunk/Lib/wsgiref/validate.py (contents, props changed) Modified: python/trunk/Doc/lib/lib.tex Log: Import wsgiref into the stdlib, as of the external version 0.1-r2181. Modified: python/trunk/Doc/lib/lib.tex ============================================================================== --- python/trunk/Doc/lib/lib.tex (original) +++ python/trunk/Doc/lib/lib.tex Fri Jun 9 18:40:18 2006 @@ -292,6 +292,7 @@ \input{libwebbrowser} \input{libcgi} \input{libcgitb} +\input{libwsgiref} \input{liburllib} \input{liburllib2} \input{libhttplib} Added: python/trunk/Doc/lib/libwsgiref.tex ============================================================================== --- (empty file) +++ python/trunk/Doc/lib/libwsgiref.tex Fri Jun 9 18:40:18 2006 @@ -0,0 +1,779 @@ +\section{\module{wsgiref} --- WSGI Utilities and Reference +Implementation} +\declaremodule{}{wsgiref} +\moduleauthor{Phillip J. Eby}{pje at telecommunity.com} +\sectionauthor{Phillip J. Eby}{pje at telecommunity.com} +\modulesynopsis{WSGI Utilities and Reference Implementation} + +The Web Server Gateway Interface (WSGI) is a standard interface +between web server software and web applications written in Python. +Having a standard interface makes it easy to use an application +that supports WSGI with a number of different web servers. + +Only authors of web servers and programming frameworks need to know +every detail and corner case of the WSGI design. You don't need to +understand every detail of WSGI just to install a WSGI application or +to write a web application using an existing framework. + +\module{wsgiref} is a reference implementation of the WSGI specification +that can be used to add WSGI support to a web server or framework. It +provides utilities for manipulating WSGI environment variables and +response headers, base classes for implementing WSGI servers, a demo +HTTP server that serves WSGI applications, and a validation tool that +checks WSGI servers and applications for conformance to the +WSGI specification (\pep{333}). + +% XXX If you're just trying to write a web application... +% XXX should create a URL on python.org to point people to. + + + + + + + + + + + + + + +\subsection{\module{wsgiref.util} -- WSGI environment utilities} +\declaremodule{}{wsgiref.util} + +This module provides a variety of utility functions for working with +WSGI environments. A WSGI environment is a dictionary containing +HTTP request variables as described in \pep{333}. All of the functions +taking an \var{environ} parameter expect a WSGI-compliant dictionary to +be supplied; please see \pep{333} for a detailed specification. + +\begin{funcdesc}{guess_scheme}{environ} +Return a guess for whether \code{wsgi.url_scheme} should be ``http'' or +``https'', by checking for a \code{HTTPS} environment variable in the +\var{environ} dictionary. The return value is a string. + +This function is useful when creating a gateway that wraps CGI or a +CGI-like protocol such as FastCGI. Typically, servers providing such +protocols will include a \code{HTTPS} variable with a value of ``1'' +``yes'', or ``on'' when a request is received via SSL. So, this +function returns ``https'' if such a value is found, and ``http'' +otherwise. +\end{funcdesc} + +\begin{funcdesc}{request_uri}{environ \optional{, include_query=1}} +Return the full request URI, optionally including the query string, +using the algorithm found in the ``URL Reconstruction'' section of +\pep{333}. If \var{include_query} is false, the query string is +not included in the resulting URI. +\end{funcdesc} + +\begin{funcdesc}{application_uri}{environ} +Similar to \function{request_uri}, except that the \code{PATH_INFO} and +\code{QUERY_STRING} variables are ignored. The result is the base URI +of the application object addressed by the request. +\end{funcdesc} + +\begin{funcdesc}{shift_path_info}{environ} +Shift a single name from \code{PATH_INFO} to \code{SCRIPT_NAME} and +return the name. The \var{environ} dictionary is \emph{modified} +in-place; use a copy if you need to keep the original \code{PATH_INFO} +or \code{SCRIPT_NAME} intact. + +If there are no remaining path segments in \code{PATH_INFO}, \code{None} +is returned. + +Typically, this routine is used to process each portion of a request +URI path, for example to treat the path as a series of dictionary keys. +This routine modifies the passed-in environment to make it suitable for +invoking another WSGI application that is located at the target URI. +For example, if there is a WSGI application at \code{/foo}, and the +request URI path is \code{/foo/bar/baz}, and the WSGI application at +\code{/foo} calls \function{shift_path_info}, it will receive the string +``bar'', and the environment will be updated to be suitable for passing +to a WSGI application at \code{/foo/bar}. That is, \code{SCRIPT_NAME} +will change from \code{/foo} to \code{/foo/bar}, and \code{PATH_INFO} +will change from \code{/bar/baz} to \code{/baz}. + +When \code{PATH_INFO} is just a ``/'', this routine returns an empty +string and appends a trailing slash to \code{SCRIPT_NAME}, even though +empty path segments are normally ignored, and \code{SCRIPT_NAME} doesn't +normally end in a slash. This is intentional behavior, to ensure that +an application can tell the difference between URIs ending in \code{/x} +from ones ending in \code{/x/} when using this routine to do object +traversal. + +\end{funcdesc} + +\begin{funcdesc}{setup_testing_defaults}{environ} +Update \var{environ} with trivial defaults for testing purposes. + +This routine adds various parameters required for WSGI, including +\code{HTTP_HOST}, \code{SERVER_NAME}, \code{SERVER_PORT}, +\code{REQUEST_METHOD}, \code{SCRIPT_NAME}, \code{PATH_INFO}, and all of +the \pep{333}-defined \code{wsgi.*} variables. It only supplies default +values, and does not replace any existing settings for these variables. + +This routine is intended to make it easier for unit tests of WSGI +servers and applications to set up dummy environments. It should NOT +be used by actual WSGI servers or applications, since the data is fake! +\end{funcdesc} + + + +In addition to the environment functions above, the +\module{wsgiref.util} module also provides these miscellaneous +utilities: + +\begin{funcdesc}{is_hop_by_hop}{header_name} +Return true if 'header_name' is an HTTP/1.1 ``Hop-by-Hop'' header, as +defined by \rfc{2616}. +\end{funcdesc} + +\begin{classdesc}{FileWrapper}{filelike \optional{, blksize=8192}} +A wrapper to convert a file-like object to an iterator. The resulting +objects support both \method{__getitem__} and \method{__iter__} +iteration styles, for compatibility with Python 2.1 and Jython. +As the object is iterated over, the optional \var{blksize} parameter +will be repeatedly passed to the \var{filelike} object's \method{read()} +method to obtain strings to yield. When \method{read()} returns an +empty string, iteration is ended and is not resumable. + +If \var{filelike} has a \method{close()} method, the returned object +will also have a \method{close()} method, and it will invoke the +\var{filelike} object's \method{close()} method when called. +\end{classdesc} + + + + + + + + + + + + + + + + + + + +\subsection{\module{wsgiref.headers} -- WSGI response header tools} +\declaremodule{}{wsgiref.headers} + +This module provides a single class, \class{Headers}, for convenient +manipulation of WSGI response headers using a mapping-like interface. + +\begin{classdesc}{Headers}{headers} +Create a mapping-like object wrapping \var{headers}, which must be a +list of header name/value tuples as described in \pep{333}. Any changes +made to the new \class{Headers} object will directly update the +\var{headers} list it was created with. + +\class{Headers} objects support typical mapping operations including +\method{__getitem__}, \method{get}, \method{__setitem__}, +\method{setdefault}, \method{__delitem__}, \method{__contains__} and +\method{has_key}. For each of these methods, the key is the header name +(treated case-insensitively), and the value is the first value +associated with that header name. Setting a header deletes any existing +values for that header, then adds a new value at the end of the wrapped +header list. Headers' existing order is generally maintained, with new +headers added to the end of the wrapped list. + +Unlike a dictionary, \class{Headers} objects do not raise an error when +you try to get or delete a key that isn't in the wrapped header list. +Getting a nonexistent header just returns \code{None}, and deleting +a nonexistent header does nothing. + +\class{Headers} objects also support \method{keys()}, \method{values()}, +and \method{items()} methods. The lists returned by \method{keys()} +and \method{items()} can include the same key more than once if there +is a multi-valued header. The \code{len()} of a \class{Headers} object +is the same as the length of its \method{items()}, which is the same +as the length of the wrapped header list. In fact, the \method{items()} +method just returns a copy of the wrapped header list. + +Calling \code{str()} on a \class{Headers} object returns a formatted +string suitable for transmission as HTTP response headers. Each header +is placed on a line with its value, separated by a colon and a space. +Each line is terminated by a carriage return and line feed, and the +string is terminated with a blank line. + +In addition to their mapping interface and formatting features, +\class{Headers} objects also have the following methods for querying +and adding multi-valued headers, and for adding headers with MIME +parameters: + +\begin{methoddesc}{get_all}{name} +Return a list of all the values for the named header. + +The returned list will be sorted in the order they appeared in the +original header list or were added to this instance, and may contain +duplicates. Any fields deleted and re-inserted are always appended to +the header list. If no fields exist with the given name, returns an +empty list. +\end{methoddesc} + + +\begin{methoddesc}{add_header}{name, value, **_params} +Add a (possibly multi-valued) header, with optional MIME parameters +specified via keyword arguments. + +\var{name} is the header field to add. Keyword arguments can be used to +set MIME parameters for the header field. Each parameter must be a +string or \code{None}. Underscores in parameter names are converted to +dashes, since dashes are illegal in Python identifiers, but many MIME +parameter names include dashes. If the parameter value is a string, it +is added to the header value parameters in the form \code{name="value"}. +If it is \code{None}, only the parameter name is added. (This is used +for MIME parameters without a value.) Example usage: + +\begin{verbatim} +h.add_header('content-disposition', 'attachment', filename='bud.gif') +\end{verbatim} + +The above will add a header that looks like this: + +\begin{verbatim} +Content-Disposition: attachment; filename="bud.gif" +\end{verbatim} +\end{methoddesc} +\end{classdesc} + +\subsection{\module{wsgiref.simple_server} -- a simple WSGI HTTP server} +\declaremodule[wsgiref.simpleserver]{}{wsgiref.simple_server} + +This module implements a simple HTTP server (based on +\module{BaseHTTPServer}) that serves WSGI applications. Each server +instance serves a single WSGI application on a given host and port. If +you want to serve multiple applications on a single host and port, you +should create a WSGI application that parses \code{PATH_INFO} to select +which application to invoke for each request. (E.g., using the +\function{shift_path_info()} function from \module{wsgiref.util}.) + + +\begin{funcdesc}{make_server}{host, port, app +\optional{, server_class=\class{WSGIServer} \optional{, +handler_class=\class{WSGIRequestHandler}}}} +Create a new WSGI server listening on \var{host} and \var{port}, +accepting connections for \var{app}. The return value is an instance of +the supplied \var{server_class}, and will process requests using the +specified \var{handler_class}. \var{app} must be a WSGI application +object, as defined by \pep{333}. + +Example usage: +\begin{verbatim}from wsgiref.simple_server import make_server, demo_app + +httpd = make_server('', 8000, demo_app) +print "Serving HTTP on port 8000..." + +# Respond to requests until process is killed +httpd.serve_forever() + +# Alternative: serve one request, then exit +##httpd.handle_request() +\end{verbatim} + +\end{funcdesc} + + + + + + +\begin{funcdesc}{demo_app}{environ, start_response} +This function is a small but complete WSGI application that +returns a text page containing the message ``Hello world!'' +and a list of the key/value pairs provided in the +\var{environ} parameter. It's useful for verifying that a WSGI server +(such as \module{wsgiref.simple_server}) is able to run a simple WSGI +application correctly. +\end{funcdesc} + + +\begin{classdesc}{WSGIServer}{server_address, RequestHandlerClass} +Create a \class{WSGIServer} instance. \var{server_address} should be +a \code{(host,port)} tuple, and \var{RequestHandlerClass} should be +the subclass of \class{BaseHTTPServer.BaseHTTPRequestHandler} that will +be used to process requests. + +You do not normally need to call this constructor, as the +\function{make_server()} function can handle all the details for you. + +\class{WSGIServer} is a subclass +of \class{BaseHTTPServer.HTTPServer}, so all of its methods (such as +\method{serve_forever()} and \method{handle_request()}) are available. +\class{WSGIServer} also provides these WSGI-specific methods: + +\begin{methoddesc}{set_app}{application} +Sets the callable \var{application} as the WSGI application that will +receive requests. +\end{methoddesc} + +\begin{methoddesc}{get_app}{} +Returns the currently-set application callable. +\end{methoddesc} + +Normally, however, you do not need to use these additional methods, as +\method{set_app()} is normally called by \function{make_server()}, and +the \method{get_app()} exists mainly for the benefit of request handler +instances. +\end{classdesc} + + + +\begin{classdesc}{WSGIRequestHandler}{request, client_address, server} +Create an HTTP handler for the given \var{request} (i.e. a socket), +\var{client_address} (a \code{(\var{host},\var{port})} tuple), and +\var{server} (\class{WSGIServer} instance). + +You do not need to create instances of this class directly; they are +automatically created as needed by \class{WSGIServer} objects. You +can, however, subclass this class and supply it as a \var{handler_class} +to the \function{make_server()} function. Some possibly relevant +methods for overriding in subclasses: + +\begin{methoddesc}{get_environ}{} +Returns a dictionary containing the WSGI environment for a request. The +default implementation copies the contents of the \class{WSGIServer} +object's \member{base_environ} dictionary attribute and then adds +various headers derived from the HTTP request. Each call to this method +should return a new dictionary containing all of the relevant CGI +environment variables as specified in \pep{333}. +\end{methoddesc} + +\begin{methoddesc}{get_stderr}{} +Return the object that should be used as the \code{wsgi.errors} stream. +The default implementation just returns \code{sys.stderr}. +\end{methoddesc} + +\begin{methoddesc}{handle}{} +Process the HTTP request. The default implementation creates a handler +instance using a \module{wsgiref.handlers} class to implement the actual +WSGI application interface. +\end{methoddesc} + +\end{classdesc} + + + + + + + + + +\subsection{\module{wsgiref.validate} -- WSGI conformance checker} +\declaremodule{}{wsgiref.validate} +When creating new WSGI application objects, frameworks, servers, or +middleware, it can be useful to validate the new code's conformance +using \module{wsgiref.validate}. This module provides a function that +creates WSGI application objects that validate communications between +a WSGI server or gateway and a WSGI application object, to check both +sides for protocol conformance. + +Note that this utility does not guarantee complete \pep{333} compliance; +an absence of errors from this module does not necessarily mean that +errors do not exist. However, if this module does produce an error, +then it is virtually certain that either the server or application is +not 100\% compliant. + +This module is based on the \module{paste.lint} module from Ian +Bicking's ``Python Paste'' library. + +\begin{funcdesc}{validator}{application} +Wrap \var{application} and return a new WSGI application object. The +returned application will forward all requests to the original +\var{application}, and will check that both the \var{application} and +the server invoking it are conforming to the WSGI specification and to +RFC 2616. + +Any detected nonconformance results in an \exception{AssertionError} +being raised; note, however, that how these errors are handled is +server-dependent. For example, \module{wsgiref.simple_server} and other +servers based on \module{wsgiref.handlers} (that don't override the +error handling methods to do something else) will simply output a +message that an error has occurred, and dump the traceback to +\code{sys.stderr} or some other error stream. + +This wrapper may also generate output using the \module{warnings} module +to indicate behaviors that are questionable but which may not actually +be prohibited by \pep{333}. Unless they are suppressed using Python +command-line options or the \module{warnings} API, any such warnings +will be written to \code{sys.stderr} (\emph{not} \code{wsgi.errors}, +unless they happen to be the same object). +\end{funcdesc} + +\subsection{\module{wsgiref.handlers} -- server/gateway base classes} +\declaremodule{}{wsgiref.handlers} + +This module provides base handler classes for implementing WSGI servers +and gateways. These base classes handle most of the work of +communicating with a WSGI application, as long as they are given a +CGI-like environment, along with input, output, and error streams. + + +\begin{classdesc}{CGIHandler}{} +CGI-based invocation via \code{sys.stdin}, \code{sys.stdout}, +\code{sys.stderr} and \code{os.environ}. This is useful when you have +a WSGI application and want to run it as a CGI script. Simply invoke +\code{CGIHandler().run(app)}, where \code{app} is the WSGI application +object you wish to invoke. + +This class is a subclass of \class{BaseCGIHandler} that sets +\code{wsgi.run_once} to true, \code{wsgi.multithread} to false, and +\code{wsgi.multiprocess} to true, and always uses \module{sys} and +\module{os} to obtain the necessary CGI streams and environment. +\end{classdesc} + + +\begin{classdesc}{BaseCGIHandler}{stdin, stdout, stderr, environ +\optional{, multithread=True \optional{, multiprocess=False}}} + +Similar to \class{CGIHandler}, but instead of using the \module{sys} and +\module{os} modules, the CGI environment and I/O streams are specified +explicitly. The \var{multithread} and \var{multiprocess} values are +used to set the \code{wsgi.multithread} and \code{wsgi.multiprocess} +flags for any applications run by the handler instance. + +This class is a subclass of \class{SimpleHandler} intended for use with +software other than HTTP ``origin servers''. If you are writing a +gateway protocol implementation (such as CGI, FastCGI, SCGI, etc.) that +uses a \code{Status:} header to send an HTTP status, you probably want +to subclass this instead of \class{SimpleHandler}. +\end{classdesc} + + + +\begin{classdesc}{SimpleHandler}{stdin, stdout, stderr, environ +\optional{,multithread=True \optional{, multiprocess=False}}} + +Similar to \class{BaseCGIHandler}, but designed for use with HTTP origin +servers. If you are writing an HTTP server implementation, you will +probably want to subclass this instead of \class{BaseCGIHandler} + +This class is a subclass of \class{BaseHandler}. It overrides the +\method{__init__()}, \method{get_stdin()}, \method{get_stderr()}, +\method{add_cgi_vars()}, \method{_write()}, and \method{_flush()} +methods to support explicitly setting the environment and streams via +the constructor. The supplied environment and streams are stored in +the \member{stdin}, \member{stdout}, \member{stderr}, and +\member{environ} attributes. +\end{classdesc} + +\begin{classdesc}{BaseHandler}{} +This is an abstract base class for running WSGI applications. Each +instance will handle a single HTTP request, although in principle you +could create a subclass that was reusable for multiple requests. + +\class{BaseHandler} instances have only one method intended for external +use: + +\begin{methoddesc}{run}{app} +Run the specified WSGI application, \var{app}. +\end{methoddesc} + +All of the other \class{BaseHandler} methods are invoked by this method +in the process of running the application, and thus exist primarily to +allow customizing the process. + +The following methods MUST be overridden in a subclass: + +\begin{methoddesc}{_write}{data} +Buffer the string \var{data} for transmission to the client. It's okay +if this method actually transmits the data; \class{BaseHandler} +just separates write and flush operations for greater efficiency +when the underlying system actually has such a distinction. +\end{methoddesc} + +\begin{methoddesc}{_flush}{} +Force buffered data to be transmitted to the client. It's okay if this +method is a no-op (i.e., if \method{_write()} actually sends the data). +\end{methoddesc} + +\begin{methoddesc}{get_stdin}{} +Return an input stream object suitable for use as the \code{wsgi.input} +of the request currently being processed. +\end{methoddesc} + +\begin{methoddesc}{get_stderr}{} +Return an output stream object suitable for use as the +\code{wsgi.errors} of the request currently being processed. +\end{methoddesc} + +\begin{methoddesc}{add_cgi_vars}{} +Insert CGI variables for the current request into the \member{environ} +attribute. +\end{methoddesc} + +Here are some other methods and attributes you may wish to override. +This list is only a summary, however, and does not include every method +that can be overridden. You should consult the docstrings and source +code for additional information before attempting to create a customized +\class{BaseHandler} subclass. + + + + + + + + + + + + + + + + +Attributes and methods for customizing the WSGI environment: + +\begin{memberdesc}{wsgi_multithread} +The value to be used for the \code{wsgi.multithread} environment +variable. It defaults to true in \class{BaseHandler}, but may have +a different default (or be set by the constructor) in the other +subclasses. +\end{memberdesc} + +\begin{memberdesc}{wsgi_multiprocess} +The value to be used for the \code{wsgi.multiprocess} environment +variable. It defaults to true in \class{BaseHandler}, but may have +a different default (or be set by the constructor) in the other +subclasses. +\end{memberdesc} + +\begin{memberdesc}{wsgi_run_once} +The value to be used for the \code{wsgi.run_once} environment +variable. It defaults to false in \class{BaseHandler}, but +\class{CGIHandler} sets it to true by default. +\end{memberdesc} + +\begin{memberdesc}{os_environ} +The default environment variables to be included in every request's +WSGI environment. By default, this is a copy of \code{os.environ} at +the time that \module{wsgiref.handlers} was imported, but subclasses can +either create their own at the class or instance level. Note that the +dictionary should be considered read-only, since the default value is +shared between multiple classes and instances. +\end{memberdesc} + +\begin{memberdesc}{server_software} +If the \member{origin_server} attribute is set, this attribute's value +is used to set the default \code{SERVER_SOFTWARE} WSGI environment +variable, and also to set a default \code{Server:} header in HTTP +responses. It is ignored for handlers (such as \class{BaseCGIHandler} +and \class{CGIHandler}) that are not HTTP origin servers. +\end{memberdesc} + + + +\begin{methoddesc}{get_scheme}{} +Return the URL scheme being used for the current request. The default +implementation uses the \function{guess_scheme()} function from +\module{wsgiref.util} to guess whether the scheme should be ``http'' or +``https'', based on the current request's \member{environ} variables. +\end{methoddesc} + +\begin{methoddesc}{setup_environ}{} +Set the \member{environ} attribute to a fully-populated WSGI +environment. The default implementation uses all of the above methods +and attributes, plus the \method{get_stdin()}, \method{get_stderr()}, +and \method{add_cgi_vars()} methods and the \member{wsgi_file_wrapper} +attribute. It also inserts a \code{SERVER_SOFTWARE} key if not present, +as long as the \member{origin_server} attribute is a true value and the +\member{server_software} attribute is set. +\end{methoddesc} + + + + + + + + + + + + + + + + + + + + + + + + + +Methods and attributes for customizing exception handling: + +\begin{methoddesc}{log_exception}{exc_info} +Log the \var{exc_info} tuple in the server log. \var{exc_info} is a +\code{(\var{type}, \var{value}, \var{traceback})} tuple. The default +implementation simply writes the traceback to the request's +\code{wsgi.errors} stream and flushes it. Subclasses can override this +method to change the format or retarget the output, mail the traceback +to an administrator, or whatever other action may be deemed suitable. +\end{methoddesc} + +\begin{memberdesc}{traceback_limit} +The maximum number of frames to include in tracebacks output by the +default \method{log_exception()} method. If \code{None}, all frames +are included. +\end{memberdesc} + +\begin{methoddesc}{error_output}{environ, start_response} +This method is a WSGI application to generate an error page for the +user. It is only invoked if an error occurs before headers are sent +to the client. + +This method can access the current error information using +\code{sys.exc_info()}, and should pass that information to +\var{start_response} when calling it (as described in the ``Error +Handling'' section of \pep{333}). + +The default implementation just uses the \member{error_status}, +\member{error_headers}, and \member{error_body} attributes to generate +an output page. Subclasses can override this to produce more dynamic +error output. + +Note, however, that it's not recommended from a security perspective to +spit out diagnostics to any old user; ideally, you should have to do +something special to enable diagnostic output, which is why the default +implementation doesn't include any. +\end{methoddesc} + + + + +\begin{memberdesc}{error_status} +The HTTP status used for error responses. This should be a status +string as defined in \pep{333}; it defaults to a 500 code and message. +\end{memberdesc} + +\begin{memberdesc}{error_headers} +The HTTP headers used for error responses. This should be a list of +WSGI response headers (\code{(\var{name}, \var{value})} tuples), as +described in \pep{333}. The default list just sets the content type +to \code{text/plain}. +\end{memberdesc} + +\begin{memberdesc}{error_body} +The error response body. This should be an HTTP response body string. +It defaults to the plain text, ``A server error occurred. Please +contact the administrator.'' +\end{memberdesc} + + + + + + + + + + + + + + + + + + + + + + + + +Methods and attributes for \pep{333}'s ``Optional Platform-Specific File +Handling'' feature: + +\begin{memberdesc}{wsgi_file_wrapper} +A \code{wsgi.file_wrapper} factory, or \code{None}. The default value +of this attribute is the \class{FileWrapper} class from +\module{wsgiref.util}. +\end{memberdesc} + +\begin{methoddesc}{sendfile}{} +Override to implement platform-specific file transmission. This method +is called only if the application's return value is an instance of +the class specified by the \member{wsgi_file_wrapper} attribute. It +should return a true value if it was able to successfully transmit the +file, so that the default transmission code will not be executed. +The default implementation of this method just returns a false value. +\end{methoddesc} + + +Miscellaneous methods and attributes: + +\begin{memberdesc}{origin_server} +This attribute should be set to a true value if the handler's +\method{_write()} and \method{_flush()} are being used to communicate +directly to the client, rather than via a CGI-like gateway protocol that +wants the HTTP status in a special \code{Status:} header. + +This attribute's default value is true in \class{BaseHandler}, but +false in \class{BaseCGIHandler} and \class{CGIHandler}. +\end{memberdesc} + +\begin{memberdesc}{http_version} +If \member{origin_server} is true, this string attribute is used to +set the HTTP version of the response set to the client. It defaults to +\code{"1.0"}. +\end{memberdesc} + + + + + +\end{classdesc} + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Added: python/trunk/Lib/test/test_wsgiref.py ============================================================================== --- (empty file) +++ python/trunk/Lib/test/test_wsgiref.py Fri Jun 9 18:40:18 2006 @@ -0,0 +1,615 @@ +from __future__ import nested_scopes # Backward compat for 2.1 +from unittest import TestSuite, TestCase, makeSuite +from wsgiref.util import setup_testing_defaults +from wsgiref.headers import Headers +from wsgiref.handlers import BaseHandler, BaseCGIHandler +from wsgiref import util +from wsgiref.validate import validator +from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app +from wsgiref.simple_server import make_server +from StringIO import StringIO +from SocketServer import BaseServer +import re, sys + + +class MockServer(WSGIServer): + """Non-socket HTTP server""" + + def __init__(self, server_address, RequestHandlerClass): + BaseServer.__init__(self, server_address, RequestHandlerClass) + self.server_bind() + + def server_bind(self): + host, port = self.server_address + self.server_name = host + self.server_port = port + self.setup_environ() + + +class MockHandler(WSGIRequestHandler): + """Non-socket HTTP handler""" + def setup(self): + self.connection = self.request + self.rfile, self.wfile = self.connection + + def finish(self): + pass + + + + + +def hello_app(environ,start_response): + start_response("200 OK", [ + ('Content-Type','text/plain'), + ('Date','Mon, 05 Jun 2006 18:49:54 GMT') + ]) + return ["Hello, world!"] + +def run_amock(app=hello_app, data="GET / HTTP/1.0\n\n"): + server = make_server("", 80, app, MockServer, MockHandler) + inp, out, err, olderr = StringIO(data), StringIO(), StringIO(), sys.stderr + sys.stderr = err + + try: + server.finish_request((inp,out), ("127.0.0.1",8888)) + finally: + sys.stderr = olderr + + return out.getvalue(), err.getvalue() + + + + + + + + + + + + + + + + + + + + + + + +def compare_generic_iter(make_it,match): + """Utility to compare a generic 2.1/2.2+ iterator with an iterable + + If running under Python 2.2+, this tests the iterator using iter()/next(), + as well as __getitem__. 'make_it' must be a function returning a fresh + iterator to be tested (since this may test the iterator twice).""" + + it = make_it() + n = 0 + for item in match: + assert it[n]==item + n+=1 + try: + it[n] + except IndexError: + pass + else: + raise AssertionError("Too many items from __getitem__",it) + + try: + iter, StopIteration + except NameError: + pass + else: + # Only test iter mode under 2.2+ + it = make_it() + assert iter(it) is it + for item in match: + assert it.next()==item + try: + it.next() + except StopIteration: + pass + else: + raise AssertionError("Too many items from .next()",it) + + + + + + +class IntegrationTests(TestCase): + + def check_hello(self, out, has_length=True): + self.assertEqual(out, + "HTTP/1.0 200 OK\r\n" + "Server: WSGIServer/0.1 Python/"+sys.version.split()[0]+"\r\n" + "Content-Type: text/plain\r\n" + "Date: Mon, 05 Jun 2006 18:49:54 GMT\r\n" + + (has_length and "Content-Length: 13\r\n" or "") + + "\r\n" + "Hello, world!" + ) + + def test_plain_hello(self): + out, err = run_amock() + self.check_hello(out) + + def test_validated_hello(self): + out, err = run_amock(validator(hello_app)) + # the middleware doesn't support len(), so content-length isn't there + self.check_hello(out, has_length=False) + + def test_simple_validation_error(self): + def bad_app(environ,start_response): + start_response("200 OK", ('Content-Type','text/plain')) + return ["Hello, world!"] + out, err = run_amock(validator(bad_app)) + self.failUnless(out.endswith( + "A server error occurred. Please contact the administrator." + )) + self.assertEqual( + err.splitlines()[-2], + "AssertionError: Headers (('Content-Type', 'text/plain')) must" + " be of type list: " + ) + + + + + + +class UtilityTests(TestCase): + + def checkShift(self,sn_in,pi_in,part,sn_out,pi_out): + env = {'SCRIPT_NAME':sn_in,'PATH_INFO':pi_in} + util.setup_testing_defaults(env) + self.assertEqual(util.shift_path_info(env),part) + self.assertEqual(env['PATH_INFO'],pi_out) + self.assertEqual(env['SCRIPT_NAME'],sn_out) + return env + + def checkDefault(self, key, value, alt=None): + # Check defaulting when empty + env = {} + util.setup_testing_defaults(env) + if isinstance(value,StringIO): + self.failUnless(isinstance(env[key],StringIO)) + else: + self.assertEqual(env[key],value) + + # Check existing value + env = {key:alt} + util.setup_testing_defaults(env) + self.failUnless(env[key] is alt) + + def checkCrossDefault(self,key,value,**kw): + util.setup_testing_defaults(kw) + self.assertEqual(kw[key],value) + + def checkAppURI(self,uri,**kw): + util.setup_testing_defaults(kw) + self.assertEqual(util.application_uri(kw),uri) + + def checkReqURI(self,uri,query=1,**kw): + util.setup_testing_defaults(kw) + self.assertEqual(util.request_uri(kw,query),uri) + + + + + + + def checkFW(self,text,size,match): + + def make_it(text=text,size=size): + return util.FileWrapper(StringIO(text),size) + + compare_generic_iter(make_it,match) + + it = make_it() + self.failIf(it.filelike.closed) + + for item in it: + pass + + self.failIf(it.filelike.closed) + + it.close() + self.failUnless(it.filelike.closed) + + + def testSimpleShifts(self): + self.checkShift('','/', '', '/', '') + self.checkShift('','/x', 'x', '/x', '') + self.checkShift('/','', None, '/', '') + self.checkShift('/a','/x/y', 'x', '/a/x', '/y') + self.checkShift('/a','/x/', 'x', '/a/x', '/') + + + def testNormalizedShifts(self): + self.checkShift('/a/b', '/../y', '..', '/a', '/y') + self.checkShift('', '/../y', '..', '', '/y') + self.checkShift('/a/b', '//y', 'y', '/a/b/y', '') + self.checkShift('/a/b', '//y/', 'y', '/a/b/y', '/') + self.checkShift('/a/b', '/./y', 'y', '/a/b/y', '') + self.checkShift('/a/b', '/./y/', 'y', '/a/b/y', '/') + self.checkShift('/a/b', '///./..//y/.//', '..', '/a', '/y/') + self.checkShift('/a/b', '///', '', '/a/b/', '') + self.checkShift('/a/b', '/.//', '', '/a/b/', '') + self.checkShift('/a/b', '/x//', 'x', '/a/b/x', '/') + self.checkShift('/a/b', '/.', None, '/a/b', '') + + + def testDefaults(self): + for key, value in [ + ('SERVER_NAME','127.0.0.1'), + ('SERVER_PORT', '80'), + ('SERVER_PROTOCOL','HTTP/1.0'), + ('HTTP_HOST','127.0.0.1'), + ('REQUEST_METHOD','GET'), + ('SCRIPT_NAME',''), + ('PATH_INFO','/'), + ('wsgi.version', (1,0)), + ('wsgi.run_once', 0), + ('wsgi.multithread', 0), + ('wsgi.multiprocess', 0), + ('wsgi.input', StringIO("")), + ('wsgi.errors', StringIO()), + ('wsgi.url_scheme','http'), + ]: + self.checkDefault(key,value) + + + def testCrossDefaults(self): + self.checkCrossDefault('HTTP_HOST',"foo.bar",SERVER_NAME="foo.bar") + self.checkCrossDefault('wsgi.url_scheme',"https",HTTPS="on") + self.checkCrossDefault('wsgi.url_scheme',"https",HTTPS="1") + self.checkCrossDefault('wsgi.url_scheme',"https",HTTPS="yes") + self.checkCrossDefault('wsgi.url_scheme',"http",HTTPS="foo") + self.checkCrossDefault('SERVER_PORT',"80",HTTPS="foo") + self.checkCrossDefault('SERVER_PORT',"443",HTTPS="on") + + + def testGuessScheme(self): + self.assertEqual(util.guess_scheme({}), "http") + self.assertEqual(util.guess_scheme({'HTTPS':"foo"}), "http") + self.assertEqual(util.guess_scheme({'HTTPS':"on"}), "https") + self.assertEqual(util.guess_scheme({'HTTPS':"yes"}), "https") + self.assertEqual(util.guess_scheme({'HTTPS':"1"}), "https") + + + + + + def testAppURIs(self): + self.checkAppURI("http://127.0.0.1/") + self.checkAppURI("http://127.0.0.1/spam", SCRIPT_NAME="/spam") + self.checkAppURI("http://spam.example.com:2071/", + HTTP_HOST="spam.example.com:2071", SERVER_PORT="2071") + self.checkAppURI("http://spam.example.com/", + SERVER_NAME="spam.example.com") + self.checkAppURI("http://127.0.0.1/", + HTTP_HOST="127.0.0.1", SERVER_NAME="spam.example.com") + self.checkAppURI("https://127.0.0.1/", HTTPS="on") + self.checkAppURI("http://127.0.0.1:8000/", SERVER_PORT="8000", + HTTP_HOST=None) + + def testReqURIs(self): + self.checkReqURI("http://127.0.0.1/") + self.checkReqURI("http://127.0.0.1/spam", SCRIPT_NAME="/spam") + self.checkReqURI("http://127.0.0.1/spammity/spam", + SCRIPT_NAME="/spammity", PATH_INFO="/spam") + self.checkReqURI("http://127.0.0.1/spammity/spam?say=ni", + SCRIPT_NAME="/spammity", PATH_INFO="/spam",QUERY_STRING="say=ni") + self.checkReqURI("http://127.0.0.1/spammity/spam", 0, + SCRIPT_NAME="/spammity", PATH_INFO="/spam",QUERY_STRING="say=ni") + + def testFileWrapper(self): + self.checkFW("xyz"*50, 120, ["xyz"*40,"xyz"*10]) + + def testHopByHop(self): + for hop in ( + "Connection Keep-Alive Proxy-Authenticate Proxy-Authorization " + "TE Trailers Transfer-Encoding Upgrade" + ).split(): + for alt in hop, hop.title(), hop.upper(), hop.lower(): + self.failUnless(util.is_hop_by_hop(alt)) + + # Not comprehensive, just a few random header names + for hop in ( + "Accept Cache-Control Date Pragma Trailer Via Warning" + ).split(): + for alt in hop, hop.title(), hop.upper(), hop.lower(): + self.failIf(util.is_hop_by_hop(alt)) + +class HeaderTests(TestCase): + + def testMappingInterface(self): + test = [('x','y')] + self.assertEqual(len(Headers([])),0) + self.assertEqual(len(Headers(test[:])),1) + self.assertEqual(Headers(test[:]).keys(), ['x']) + self.assertEqual(Headers(test[:]).values(), ['y']) + self.assertEqual(Headers(test[:]).items(), test) + self.failIf(Headers(test).items() is test) # must be copy! + + h=Headers([]) + del h['foo'] # should not raise an error + + h['Foo'] = 'bar' + for m in h.has_key, h.__contains__, h.get, h.get_all, h.__getitem__: + self.failUnless(m('foo')) + self.failUnless(m('Foo')) + self.failUnless(m('FOO')) + self.failIf(m('bar')) + + self.assertEqual(h['foo'],'bar') + h['foo'] = 'baz' + self.assertEqual(h['FOO'],'baz') + self.assertEqual(h.get_all('foo'),['baz']) + + self.assertEqual(h.get("foo","whee"), "baz") + self.assertEqual(h.get("zoo","whee"), "whee") + self.assertEqual(h.setdefault("foo","whee"), "baz") + self.assertEqual(h.setdefault("zoo","whee"), "whee") + self.assertEqual(h["foo"],"baz") + self.assertEqual(h["zoo"],"whee") + + def testRequireList(self): + self.assertRaises(TypeError, Headers, "foo") + + + def testExtras(self): + h = Headers([]) + self.assertEqual(str(h),'\r\n') + + h.add_header('foo','bar',baz="spam") + self.assertEqual(h['foo'], 'bar; baz="spam"') + self.assertEqual(str(h),'foo: bar; baz="spam"\r\n\r\n') + + h.add_header('Foo','bar',cheese=None) + self.assertEqual(h.get_all('foo'), + ['bar; baz="spam"', 'bar; cheese']) + + self.assertEqual(str(h), + 'foo: bar; baz="spam"\r\n' + 'Foo: bar; cheese\r\n' + '\r\n' + ) + + +class ErrorHandler(BaseCGIHandler): + """Simple handler subclass for testing BaseHandler""" + + def __init__(self,**kw): + setup_testing_defaults(kw) + BaseCGIHandler.__init__( + self, StringIO(''), StringIO(), StringIO(), kw, + multithread=True, multiprocess=True + ) + +class TestHandler(ErrorHandler): + """Simple handler subclass for testing BaseHandler, w/error passthru""" + + def handle_error(self): + raise # for testing, we want to see what's happening + + + + + + + + + + + +class HandlerTests(TestCase): + + def checkEnvironAttrs(self, handler): + env = handler.environ + for attr in [ + 'version','multithread','multiprocess','run_once','file_wrapper' + ]: + if attr=='file_wrapper' and handler.wsgi_file_wrapper is None: + continue + self.assertEqual(getattr(handler,'wsgi_'+attr),env['wsgi.'+attr]) + + def checkOSEnviron(self,handler): + empty = {}; setup_testing_defaults(empty) + env = handler.environ + from os import environ + for k,v in environ.items(): + if not empty.has_key(k): + self.assertEqual(env[k],v) + for k,v in empty.items(): + self.failUnless(env.has_key(k)) + + def testEnviron(self): + h = TestHandler(X="Y") + h.setup_environ() + self.checkEnvironAttrs(h) + self.checkOSEnviron(h) + self.assertEqual(h.environ["X"],"Y") + + def testCGIEnviron(self): + h = BaseCGIHandler(None,None,None,{}) + h.setup_environ() + for key in 'wsgi.url_scheme', 'wsgi.input', 'wsgi.errors': + assert h.environ.has_key(key) + + def testScheme(self): + h=TestHandler(HTTPS="on"); h.setup_environ() + self.assertEqual(h.environ['wsgi.url_scheme'],'https') + h=TestHandler(); h.setup_environ() + self.assertEqual(h.environ['wsgi.url_scheme'],'http') + + + def testAbstractMethods(self): + h = BaseHandler() + for name in [ + '_flush','get_stdin','get_stderr','add_cgi_vars' + ]: + self.assertRaises(NotImplementedError, getattr(h,name)) + self.assertRaises(NotImplementedError, h._write, "test") + + + def testContentLength(self): + # Demo one reason iteration is better than write()... ;) + + def trivial_app1(e,s): + s('200 OK',[]) + return [e['wsgi.url_scheme']] + + def trivial_app2(e,s): + s('200 OK',[])(e['wsgi.url_scheme']) + return [] + + h = TestHandler() + h.run(trivial_app1) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "Content-Length: 4\r\n" + "\r\n" + "http") + + h = TestHandler() + h.run(trivial_app2) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "\r\n" + "http") + + + + + + + + def testBasicErrorOutput(self): + + def non_error_app(e,s): + s('200 OK',[]) + return [] + + def error_app(e,s): + raise AssertionError("This should be caught by handler") + + h = ErrorHandler() + h.run(non_error_app) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "Content-Length: 0\r\n" + "\r\n") + self.assertEqual(h.stderr.getvalue(),"") + + h = ErrorHandler() + h.run(error_app) + self.assertEqual(h.stdout.getvalue(), + "Status: %s\r\n" + "Content-Type: text/plain\r\n" + "Content-Length: %d\r\n" + "\r\n%s" % (h.error_status,len(h.error_body),h.error_body)) + + self.failUnless(h.stderr.getvalue().find("AssertionError")<>-1) + + def testErrorAfterOutput(self): + MSG = "Some output has been sent" + def error_app(e,s): + s("200 OK",[])(MSG) + raise AssertionError("This should be caught by handler") + + h = ErrorHandler() + h.run(error_app) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "\r\n"+MSG) + self.failUnless(h.stderr.getvalue().find("AssertionError")<>-1) + + + def testHeaderFormats(self): + + def non_error_app(e,s): + s('200 OK',[]) + return [] + + stdpat = ( + r"HTTP/%s 200 OK\r\n" + r"Date: \w{3}, [ 0123]\d \w{3} \d{4} \d\d:\d\d:\d\d GMT\r\n" + r"%s" r"Content-Length: 0\r\n" r"\r\n" + ) + shortpat = ( + "Status: 200 OK\r\n" "Content-Length: 0\r\n" "\r\n" + ) + + for ssw in "FooBar/1.0", None: + sw = ssw and "Server: %s\r\n" % ssw or "" + + for version in "1.0", "1.1": + for proto in "HTTP/0.9", "HTTP/1.0", "HTTP/1.1": + + h = TestHandler(SERVER_PROTOCOL=proto) + h.origin_server = False + h.http_version = version + h.server_software = ssw + h.run(non_error_app) + self.assertEqual(shortpat,h.stdout.getvalue()) + + h = TestHandler(SERVER_PROTOCOL=proto) + h.origin_server = True + h.http_version = version + h.server_software = ssw + h.run(non_error_app) + if proto=="HTTP/0.9": + self.assertEqual(h.stdout.getvalue(),"") + else: + self.failUnless( + re.match(stdpat%(version,sw), h.stdout.getvalue()), + (stdpat%(version,sw), h.stdout.getvalue()) + ) + +# This epilogue is needed for compatibility with the Python 2.5 regrtest module + +def test_main(): + import unittest + from test.test_support import run_suite + run_suite( + unittest.defaultTestLoader.loadTestsFromModule(sys.modules[__name__]) + ) + +if __name__ == "__main__": + test_main() + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +# the above lines intentionally left blank Added: python/trunk/Lib/wsgiref.egg-info ============================================================================== --- (empty file) +++ python/trunk/Lib/wsgiref.egg-info Fri Jun 9 18:40:18 2006 @@ -0,0 +1,8 @@ +Metadata-Version: 1.0 +Name: wsgiref +Version: 0.1 +Summary: WSGI (PEP 333) Reference Library +Author: Phillip J. Eby +Author-email: web-sig at python.org +License: PSF or ZPL +Platform: UNKNOWN Added: python/trunk/Lib/wsgiref/__init__.py ============================================================================== --- (empty file) +++ python/trunk/Lib/wsgiref/__init__.py Fri Jun 9 18:40:18 2006 @@ -0,0 +1,23 @@ +"""wsgiref -- a WSGI (PEP 333) Reference Library + +Current Contents: + +* util -- Miscellaneous useful functions and wrappers + +* headers -- Manage response headers + +* handlers -- base classes for server/gateway implementations + +* simple_server -- a simple BaseHTTPServer that supports WSGI + +* validate -- validation wrapper that sits between an app and a server + to detect errors in either + +To-Do: + +* cgi_gateway -- Run WSGI apps under CGI (pending a deployment standard) + +* cgi_wrapper -- Run CGI apps under WSGI + +* router -- a simple middleware component that handles URL traversal +""" Added: python/trunk/Lib/wsgiref/handlers.py ============================================================================== --- (empty file) +++ python/trunk/Lib/wsgiref/handlers.py Fri Jun 9 18:40:18 2006 @@ -0,0 +1,492 @@ +"""Base classes for server/gateway implementations""" + +from types import StringType +from util import FileWrapper, guess_scheme, is_hop_by_hop +from headers import Headers + +import sys, os, time + +__all__ = ['BaseHandler', 'SimpleHandler', 'BaseCGIHandler', 'CGIHandler'] + +try: + dict +except NameError: + def dict(items): + d = {} + for k,v in items: + d[k] = v + return d + +try: + True + False +except NameError: + True = not None + False = not True + + +# Weekday and month names for HTTP date/time formatting; always English! +_weekdayname = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] +_monthname = [None, # Dummy so we can use 1-based month numbers + "Jan", "Feb", "Mar", "Apr", "May", "Jun", + "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"] + +def format_date_time(timestamp): + year, month, day, hh, mm, ss, wd, y, z = time.gmtime(timestamp) + return "%s, %02d %3s %4d %02d:%02d:%02d GMT" % ( + _weekdayname[wd], day, _monthname[month], year, hh, mm, ss + ) + + + +class BaseHandler: + """Manage the invocation of a WSGI application""" + + # Configuration parameters; can override per-subclass or per-instance + wsgi_version = (1,0) + wsgi_multithread = True + wsgi_multiprocess = True + wsgi_run_once = False + + origin_server = True # We are transmitting direct to client + http_version = "1.0" # Version that should be used for response + server_software = None # String name of server software, if any + + # os_environ is used to supply configuration from the OS environment: + # by default it's a copy of 'os.environ' as of import time, but you can + # override this in e.g. your __init__ method. + os_environ = dict(os.environ.items()) + + # Collaborator classes + wsgi_file_wrapper = FileWrapper # set to None to disable + headers_class = Headers # must be a Headers-like class + + # Error handling (also per-subclass or per-instance) + traceback_limit = None # Print entire traceback to self.get_stderr() + error_status = "500 Dude, this is whack!" + error_headers = [('Content-Type','text/plain')] + error_body = "A server error occurred. Please contact the administrator." + + # State variables (don't mess with these) + status = result = None + headers_sent = False + headers = None + bytes_sent = 0 + + + + + + + + + def run(self, application): + """Invoke the application""" + # Note to self: don't move the close()! Asynchronous servers shouldn't + # call close() from finish_response(), so if you close() anywhere but + # the double-error branch here, you'll break asynchronous servers by + # prematurely closing. Async servers must return from 'run()' without + # closing if there might still be output to iterate over. + try: + self.setup_environ() + self.result = application(self.environ, self.start_response) + self.finish_response() + except: + try: + self.handle_error() + except: + # If we get an error handling an error, just give up already! + self.close() + raise # ...and let the actual server figure it out. + + + def setup_environ(self): + """Set up the environment for one request""" + + env = self.environ = self.os_environ.copy() + self.add_cgi_vars() + + env['wsgi.input'] = self.get_stdin() + env['wsgi.errors'] = self.get_stderr() + env['wsgi.version'] = self.wsgi_version + env['wsgi.run_once'] = self.wsgi_run_once + env['wsgi.url_scheme'] = self.get_scheme() + env['wsgi.multithread'] = self.wsgi_multithread + env['wsgi.multiprocess'] = self.wsgi_multiprocess + + if self.wsgi_file_wrapper is not None: + env['wsgi.file_wrapper'] = self.wsgi_file_wrapper + + if self.origin_server and self.server_software: + env.setdefault('SERVER_SOFTWARE',self.server_software) + + + def finish_response(self): + """Send any iterable data, then close self and the iterable + + Subclasses intended for use in asynchronous servers will + want to redefine this method, such that it sets up callbacks + in the event loop to iterate over the data, and to call + 'self.close()' once the response is finished. + """ + if not self.result_is_file() or not self.sendfile(): + for data in self.result: + self.write(data) + self.finish_content() + self.close() + + + def get_scheme(self): + """Return the URL scheme being used""" + return guess_scheme(self.environ) + + + def set_content_length(self): + """Compute Content-Length or switch to chunked encoding if possible""" + try: + blocks = len(self.result) + except (TypeError,AttributeError,NotImplementedError): + pass + else: + if blocks==1: + self.headers['Content-Length'] = str(self.bytes_sent) + return + # XXX Try for chunked encoding if origin server and client is 1.1 + + + def cleanup_headers(self): + """Make any necessary header changes or defaults + + Subclasses can extend this to add other defaults. + """ + if not self.headers.has_key('Content-Length'): + self.set_content_length() + + def start_response(self, status, headers,exc_info=None): + """'start_response()' callable as specified by PEP 333""" + + if exc_info: + try: + if self.headers_sent: + # Re-raise original exception if headers sent + raise exc_info[0], exc_info[1], exc_info[2] + finally: + exc_info = None # avoid dangling circular ref + elif self.headers is not None: + raise AssertionError("Headers already set!") + + assert type(status) is StringType,"Status must be a string" + assert len(status)>=4,"Status must be at least 4 characters" + assert int(status[:3]),"Status message must begin w/3-digit code" + assert status[3]==" ", "Status message must have a space after code" + if __debug__: + for name,val in headers: + assert type(name) is StringType,"Header names must be strings" + assert type(val) is StringType,"Header values must be strings" + assert not is_hop_by_hop(name),"Hop-by-hop headers not allowed" + self.status = status + self.headers = self.headers_class(headers) + return self.write + + + def send_preamble(self): + """Transmit version/status/date/server, via self._write()""" + if self.origin_server: + if self.client_is_modern(): + self._write('HTTP/%s %s\r\n' % (self.http_version,self.status)) + if not self.headers.has_key('Date'): + self._write( + 'Date: %s\r\n' % format_date_time(time.time()) + ) + if self.server_software and not self.headers.has_key('Server'): + self._write('Server: %s\r\n' % self.server_software) + else: + self._write('Status: %s\r\n' % self.status) + + def write(self, data): + """'write()' callable as specified by PEP 333""" + + assert type(data) is StringType,"write() argument must be string" + + if not self.status: + raise AssertionError("write() before start_response()") + + elif not self.headers_sent: + # Before the first output, send the stored headers + self.bytes_sent = len(data) # make sure we know content-length + self.send_headers() + else: + self.bytes_sent += len(data) + + # XXX check Content-Length and truncate if too many bytes written? + self._write(data) + self._flush() + + + def sendfile(self): + """Platform-specific file transmission + + Override this method in subclasses to support platform-specific + file transmission. It is only called if the application's + return iterable ('self.result') is an instance of + 'self.wsgi_file_wrapper'. + + This method should return a true value if it was able to actually + transmit the wrapped file-like object using a platform-specific + approach. It should return a false value if normal iteration + should be used instead. An exception can be raised to indicate + that transmission was attempted, but failed. + + NOTE: this method should call 'self.send_headers()' if + 'self.headers_sent' is false and it is going to attempt direct + transmission of the file. + """ + return False # No platform-specific transmission by default + + + def finish_content(self): + """Ensure headers and content have both been sent""" + if not self.headers_sent: + self.headers['Content-Length'] = "0" + self.send_headers() + else: + pass # XXX check if content-length was too short? + + def close(self): + """Close the iterable (if needed) and reset all instance vars + + Subclasses may want to also drop the client connection. + """ + try: + if hasattr(self.result,'close'): + self.result.close() + finally: + self.result = self.headers = self.status = self.environ = None + self.bytes_sent = 0; self.headers_sent = False + + + def send_headers(self): + """Transmit headers to the client, via self._write()""" + self.cleanup_headers() + self.headers_sent = True + if not self.origin_server or self.client_is_modern(): + self.send_preamble() + self._write(str(self.headers)) + + + def result_is_file(self): + """True if 'self.result' is an instance of 'self.wsgi_file_wrapper'""" + wrapper = self.wsgi_file_wrapper + return wrapper is not None and isinstance(self.result,wrapper) + + + def client_is_modern(self): + """True if client can accept status and headers""" + return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9' + + + def log_exception(self,exc_info): + """Log the 'exc_info' tuple in the server log + + Subclasses may override to retarget the output or change its format. + """ + try: + from traceback import print_exception + stderr = self.get_stderr() + print_exception( + exc_info[0], exc_info[1], exc_info[2], + self.traceback_limit, stderr + ) + stderr.flush() + finally: + exc_info = None + + def handle_error(self): + """Log current error, and send error output to client if possible""" + self.log_exception(sys.exc_info()) + if not self.headers_sent: + self.result = self.error_output(self.environ, self.start_response) + self.finish_response() + # XXX else: attempt advanced recovery techniques for HTML or text? + + def error_output(self, environ, start_response): + """WSGI mini-app to create error output + + By default, this just uses the 'error_status', 'error_headers', + and 'error_body' attributes to generate an output page. It can + be overridden in a subclass to dynamically generate diagnostics, + choose an appropriate message for the user's preferred language, etc. + + Note, however, that it's not recommended from a security perspective to + spit out diagnostics to any old user; ideally, you should have to do + something special to enable diagnostic output, which is why we don't + include any here! + """ + start_response(self.error_status,self.error_headers[:],sys.exc_info()) + return [self.error_body] + + + # Pure abstract methods; *must* be overridden in subclasses + + def _write(self,data): + """Override in subclass to buffer data for send to client + + It's okay if this method actually transmits the data; BaseHandler + just separates write and flush operations for greater efficiency + when the underlying system actually has such a distinction. + """ + raise NotImplementedError + + def _flush(self): + """Override in subclass to force sending of recent '_write()' calls + + It's okay if this method is a no-op (i.e., if '_write()' actually + sends the data. + """ + raise NotImplementedError + + def get_stdin(self): + """Override in subclass to return suitable 'wsgi.input'""" + raise NotImplementedError + + def get_stderr(self): + """Override in subclass to return suitable 'wsgi.errors'""" + raise NotImplementedError + + def add_cgi_vars(self): + """Override in subclass to insert CGI variables in 'self.environ'""" + raise NotImplementedError + + + + + + + + + + + +class SimpleHandler(BaseHandler): + """Handler that's just initialized with streams, environment, etc. + + This handler subclass is intended for synchronous HTTP/1.0 origin servers, + and handles sending the entire response output, given the correct inputs. + + Usage:: + + handler = SimpleHandler( + inp,out,err,env, multithread=False, multiprocess=True + ) + handler.run(app)""" + + def __init__(self,stdin,stdout,stderr,environ, + multithread=True, multiprocess=False + ): + self.stdin = stdin + self.stdout = stdout + self.stderr = stderr + self.base_env = environ + self.wsgi_multithread = multithread + self.wsgi_multiprocess = multiprocess + + def get_stdin(self): + return self.stdin + + def get_stderr(self): + return self.stderr + + def add_cgi_vars(self): + self.environ.update(self.base_env) + + def _write(self,data): + self.stdout.write(data) + self._write = self.stdout.write + + def _flush(self): + self.stdout.flush() + self._flush = self.stdout.flush + + +class BaseCGIHandler(SimpleHandler): + + """CGI-like systems using input/output/error streams and environ mapping + + Usage:: + + handler = BaseCGIHandler(inp,out,err,env) + handler.run(app) + + This handler class is useful for gateway protocols like ReadyExec and + FastCGI, that have usable input/output/error streams and an environment + mapping. It's also the base class for CGIHandler, which just uses + sys.stdin, os.environ, and so on. + + The constructor also takes keyword arguments 'multithread' and + 'multiprocess' (defaulting to 'True' and 'False' respectively) to control + the configuration sent to the application. It sets 'origin_server' to + False (to enable CGI-like output), and assumes that 'wsgi.run_once' is + False. + """ + + origin_server = False + + + + + + + + + + + + + + + + + + + +class CGIHandler(BaseCGIHandler): + + """CGI-based invocation via sys.stdin/stdout/stderr and os.environ + + Usage:: + + CGIHandler().run(app) + + The difference between this class and BaseCGIHandler is that it always + uses 'wsgi.run_once' of 'True', 'wsgi.multithread' of 'False', and + 'wsgi.multiprocess' of 'True'. It does not take any initialization + parameters, but always uses 'sys.stdin', 'os.environ', and friends. + + If you need to override any of these parameters, use BaseCGIHandler + instead. + """ + + wsgi_run_once = True + + def __init__(self): + BaseCGIHandler.__init__( + self, sys.stdin, sys.stdout, sys.stderr, dict(os.environ.items()), + multithread=False, multiprocess=True + ) + + + + + + + + + + + + + + + + + Added: python/trunk/Lib/wsgiref/headers.py ============================================================================== --- (empty file) +++ python/trunk/Lib/wsgiref/headers.py Fri Jun 9 18:40:18 2006 @@ -0,0 +1,205 @@ +"""Manage HTTP Response Headers + +Much of this module is red-handedly pilfered from email.Message in the stdlib, +so portions are Copyright (C) 2001,2002 Python Software Foundation, and were +written by Barry Warsaw. +""" + +from types import ListType, TupleType + +# Regular expression that matches `special' characters in parameters, the +# existance of which force quoting of the parameter value. +import re +tspecials = re.compile(r'[ <>@,;:\\"/\[\]\?=]') + +def _formatparam(param, value=None, quote=1): + """Convenience function to format and return a key=value pair. + + This will quote the value if needed or if quote is true. + """ + if value is not None and len(value) > 0: + if quote or tspecials.search(value): + value = value.replace('\\', '\\\\').replace('"', r'\"') + return '%s="%s"' % (param, value) + else: + return '%s=%s' % (param, value) + else: + return param + + + + + + + + + + + + + + +class Headers: + + """Manage a collection of HTTP response headers""" + + def __init__(self,headers): + if type(headers) is not ListType: + raise TypeError("Headers must be a list of name/value tuples") + self._headers = headers + + def __len__(self): + """Return the total number of headers, including duplicates.""" + return len(self._headers) + + def __setitem__(self, name, val): + """Set the value of a header.""" + del self[name] + self._headers.append((name, val)) + + def __delitem__(self,name): + """Delete all occurrences of a header, if present. + + Does *not* raise an exception if the header is missing. + """ + name = name.lower() + self._headers[:] = [kv for kv in self._headers if kv[0].lower()<>name] + + def __getitem__(self,name): + """Get the first header value for 'name' + + Return None if the header is missing instead of raising an exception. + + Note that if the header appeared multiple times, the first exactly which + occurrance gets returned is undefined. Use getall() to get all + the values matching a header field name. + """ + return self.get(name) + + + + + + def has_key(self, name): + """Return true if the message contains the header.""" + return self.get(name) is not None + + __contains__ = has_key + + + def get_all(self, name): + """Return a list of all the values for the named field. + + These will be sorted in the order they appeared in the original header + list or were added to this instance, and may contain duplicates. Any + fields deleted and re-inserted are always appended to the header list. + If no fields exist with the given name, returns an empty list. + """ + name = name.lower() + return [kv[1] for kv in self._headers if kv[0].lower()==name] + + + def get(self,name,default=None): + """Get the first header value for 'name', or return 'default'""" + name = name.lower() + for k,v in self._headers: + if k.lower()==name: + return v + return default + + + def keys(self): + """Return a list of all the header field names. + + These will be sorted in the order they appeared in the original header + list, or were added to this instance, and may contain duplicates. + Any fields deleted and re-inserted are always appended to the header + list. + """ + return [k for k, v in self._headers] + + + + + def values(self): + """Return a list of all header values. + + These will be sorted in the order they appeared in the original header + list, or were added to this instance, and may contain duplicates. + Any fields deleted and re-inserted are always appended to the header + list. + """ + return [v for k, v in self._headers] + + def items(self): + """Get all the header fields and values. + + These will be sorted in the order they were in the original header + list, or were added to this instance, and may contain duplicates. + Any fields deleted and re-inserted are always appended to the header + list. + """ + return self._headers[:] + + def __repr__(self): + return "Headers(%s)" % `self._headers` + + def __str__(self): + """str() returns the formatted headers, complete with end line, + suitable for direct HTTP transmission.""" + return '\r\n'.join(["%s: %s" % kv for kv in self._headers]+['','']) + + def setdefault(self,name,value): + """Return first matching header value for 'name', or 'value' + + If there is no header named 'name', add a new header with name 'name' + and value 'value'.""" + result = self.get(name) + if result is None: + self._headers.append((name,value)) + return value + else: + return result + + + def add_header(self, _name, _value, **_params): + """Extended header setting. + + _name is the header field to add. keyword arguments can be used to set + additional parameters for the header field, with underscores converted + to dashes. Normally the parameter will be added as key="value" unless + value is None, in which case only the key will be added. + + Example: + + h.add_header('content-disposition', 'attachment', filename='bud.gif') + + Note that unlike the corresponding 'email.Message' method, this does + *not* handle '(charset, language, value)' tuples: all values must be + strings or None. + """ + parts = [] + if _value is not None: + parts.append(_value) + for k, v in _params.items(): + if v is None: + parts.append(k.replace('_', '-')) + else: + parts.append(_formatparam(k.replace('_', '-'), v)) + self._headers.append((_name, "; ".join(parts))) + + + + + + + + + + + + + + + + Added: python/trunk/Lib/wsgiref/simple_server.py ============================================================================== --- (empty file) +++ python/trunk/Lib/wsgiref/simple_server.py Fri Jun 9 18:40:18 2006 @@ -0,0 +1,205 @@ +"""BaseHTTPServer that implements the Python WSGI protocol (PEP 333, rev 1.21) + +This is both an example of how WSGI can be implemented, and a basis for running +simple web applications on a local machine, such as might be done when testing +or debugging an application. It has not been reviewed for security issues, +however, and we strongly recommend that you use a "real" web server for +production use. + +For example usage, see the 'if __name__=="__main__"' block at the end of the +module. See also the BaseHTTPServer module docs for other API information. +""" + +from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer +import urllib, sys +from wsgiref.handlers import SimpleHandler + +__version__ = "0.1" +__all__ = ['WSGIServer', 'WSGIRequestHandler', 'demo_app', 'make_server'] + + +server_version = "WSGIServer/" + __version__ +sys_version = "Python/" + sys.version.split()[0] +software_version = server_version + ' ' + sys_version + + +class ServerHandler(SimpleHandler): + + server_software = software_version + + def close(self): + try: + self.request_handler.log_request( + self.status.split(' ',1)[0], self.bytes_sent + ) + finally: + SimpleHandler.close(self) + + + + + +class WSGIServer(HTTPServer): + + """BaseHTTPServer that implements the Python WSGI protocol""" + + application = None + + def server_bind(self): + """Override server_bind to store the server name.""" + HTTPServer.server_bind(self) + self.setup_environ() + + def setup_environ(self): + # Set up base environment + env = self.base_environ = {} + env['SERVER_NAME'] = self.server_name + env['GATEWAY_INTERFACE'] = 'CGI/1.1' + env['SERVER_PORT'] = str(self.server_port) + env['REMOTE_HOST']='' + env['CONTENT_LENGTH']='' + env['SCRIPT_NAME'] = '' + + def get_app(self): + return self.application + + def set_app(self,application): + self.application = application + + + + + + + + + + + + + + + +class WSGIRequestHandler(BaseHTTPRequestHandler): + + server_version = "WSGIServer/" + __version__ + + def get_environ(self): + env = self.server.base_environ.copy() + env['SERVER_PROTOCOL'] = self.request_version + env['REQUEST_METHOD'] = self.command + if '?' in self.path: + path,query = self.path.split('?',1) + else: + path,query = self.path,'' + + env['PATH_INFO'] = urllib.unquote(path) + env['QUERY_STRING'] = query + + host = self.address_string() + if host != self.client_address[0]: + env['REMOTE_HOST'] = host + env['REMOTE_ADDR'] = self.client_address[0] + + if self.headers.typeheader is None: + env['CONTENT_TYPE'] = self.headers.type + else: + env['CONTENT_TYPE'] = self.headers.typeheader + + length = self.headers.getheader('content-length') + if length: + env['CONTENT_LENGTH'] = length + + for h in self.headers.headers: + k,v = h.split(':',1) + k=k.replace('-','_').upper(); v=v.strip() + if k in env: + continue # skip content length, type,etc. + if 'HTTP_'+k in env: + env['HTTP_'+k] += ','+v # comma-separate multiple headers + else: + env['HTTP_'+k] = v + return env + + def get_stderr(self): + return sys.stderr + + def handle(self): + """Handle a single HTTP request""" + + self.raw_requestline = self.rfile.readline() + if not self.parse_request(): # An error code has been sent, just exit + return + + handler = ServerHandler( + self.rfile, self.wfile, self.get_stderr(), self.get_environ() + ) + handler.request_handler = self # backpointer for logging + handler.run(self.server.get_app()) + + + + + + + + + + + + + + + + + + + + + + + + + + +def demo_app(environ,start_response): + from StringIO import StringIO + stdout = StringIO() + print >>stdout, "Hello world!" + print >>stdout + h = environ.items(); h.sort() + for k,v in h: + print >>stdout, k,'=',`v` + start_response("200 OK", [('Content-Type','text/plain')]) + return [stdout.getvalue()] + + +def make_server( + host, port, app, server_class=WSGIServer, handler_class=WSGIRequestHandler +): + """Create a new WSGI server listening on `host` and `port` for `app`""" + server = server_class((host, port), handler_class) + server.set_app(app) + return server + + +if __name__ == '__main__': + server_address = ('', 8000) + httpd = make_server('', 8000, demo_app) + sa = httpd.socket.getsockname() + print "Serving HTTP on", sa[0], "port", sa[1], "..." + import webbrowser + webbrowser.open('http://localhost:8000/xyz?abc') + httpd.handle_request() # serve one request, then exit + + + + + + + + + + + + Added: python/trunk/Lib/wsgiref/util.py ============================================================================== --- (empty file) +++ python/trunk/Lib/wsgiref/util.py Fri Jun 9 18:40:18 2006 @@ -0,0 +1,205 @@ +"""Miscellaneous WSGI-related Utilities""" + +import posixpath + +__all__ = [ + 'FileWrapper', 'guess_scheme', 'application_uri', 'request_uri', + 'shift_path_info', 'setup_testing_defaults', +] + + +class FileWrapper: + """Wrapper to convert file-like objects to iterables""" + + def __init__(self, filelike, blksize=8192): + self.filelike = filelike + self.blksize = blksize + if hasattr(filelike,'close'): + self.close = filelike.close + + def __getitem__(self,key): + data = self.filelike.read(self.blksize) + if data: + return data + raise IndexError + + def __iter__(self): + return self + + def next(self): + data = self.filelike.read(self.blksize) + if data: + return data + raise StopIteration + + + + + + + + +def guess_scheme(environ): + """Return a guess for whether 'wsgi.url_scheme' should be 'http' or 'https' + """ + if environ.get("HTTPS") in ('yes','on','1'): + return 'https' + else: + return 'http' + +def application_uri(environ): + """Return the application's base URI (no PATH_INFO or QUERY_STRING)""" + url = environ['wsgi.url_scheme']+'://' + from urllib import quote + + if environ.get('HTTP_HOST'): + url += environ['HTTP_HOST'] + else: + url += environ['SERVER_NAME'] + + if environ['wsgi.url_scheme'] == 'https': + if environ['SERVER_PORT'] != '443': + url += ':' + environ['SERVER_PORT'] + else: + if environ['SERVER_PORT'] != '80': + url += ':' + environ['SERVER_PORT'] + + url += quote(environ.get('SCRIPT_NAME') or '/') + return url + +def request_uri(environ, include_query=1): + """Return the full request URI, optionally including the query string""" + url = application_uri(environ) + from urllib import quote + path_info = quote(environ.get('PATH_INFO','')) + if not environ.get('SCRIPT_NAME'): + url += path_info[1:] + else: + url += path_info + if include_query and environ.get('QUERY_STRING'): + url += '?' + environ['QUERY_STRING'] + return url + +def shift_path_info(environ): + """Shift a name from PATH_INFO to SCRIPT_NAME, returning it + + If there are no remaining path segments in PATH_INFO, return None. + Note: 'environ' is modified in-place; use a copy if you need to keep + the original PATH_INFO or SCRIPT_NAME. + + Note: when PATH_INFO is just a '/', this returns '' and appends a trailing + '/' to SCRIPT_NAME, even though empty path segments are normally ignored, + and SCRIPT_NAME doesn't normally end in a '/'. This is intentional + behavior, to ensure that an application can tell the difference between + '/x' and '/x/' when traversing to objects. + """ + path_info = environ.get('PATH_INFO','') + if not path_info: + return None + + path_parts = path_info.split('/') + path_parts[1:-1] = [p for p in path_parts[1:-1] if p and p<>'.'] + name = path_parts[1] + del path_parts[1] + + script_name = environ.get('SCRIPT_NAME','') + script_name = posixpath.normpath(script_name+'/'+name) + if script_name.endswith('/'): + script_name = script_name[:-1] + if not name and not script_name.endswith('/'): + script_name += '/' + + environ['SCRIPT_NAME'] = script_name + environ['PATH_INFO'] = '/'.join(path_parts) + + # Special case: '/.' on PATH_INFO doesn't get stripped, + # because we don't strip the last element of PATH_INFO + # if there's only one path part left. Instead of fixing this + # above, we fix it here so that PATH_INFO gets normalized to + # an empty string in the environ. + if name=='.': + name = None + return name + +def setup_testing_defaults(environ): + """Update 'environ' with trivial defaults for testing purposes + + This adds various parameters required for WSGI, including HTTP_HOST, + SERVER_NAME, SERVER_PORT, REQUEST_METHOD, SCRIPT_NAME, PATH_INFO, + and all of the wsgi.* variables. It only supplies default values, + and does not replace any existing settings for these variables. + + This routine is intended to make it easier for unit tests of WSGI + servers and applications to set up dummy environments. It should *not* + be used by actual WSGI servers or applications, since the data is fake! + """ + + environ.setdefault('SERVER_NAME','127.0.0.1') + environ.setdefault('SERVER_PROTOCOL','HTTP/1.0') + + environ.setdefault('HTTP_HOST',environ['SERVER_NAME']) + environ.setdefault('REQUEST_METHOD','GET') + + if 'SCRIPT_NAME' not in environ and 'PATH_INFO' not in environ: + environ.setdefault('SCRIPT_NAME','') + environ.setdefault('PATH_INFO','/') + + environ.setdefault('wsgi.version', (1,0)) + environ.setdefault('wsgi.run_once', 0) + environ.setdefault('wsgi.multithread', 0) + environ.setdefault('wsgi.multiprocess', 0) + + from StringIO import StringIO + environ.setdefault('wsgi.input', StringIO("")) + environ.setdefault('wsgi.errors', StringIO()) + environ.setdefault('wsgi.url_scheme',guess_scheme(environ)) + + if environ['wsgi.url_scheme']=='http': + environ.setdefault('SERVER_PORT', '80') + elif environ['wsgi.url_scheme']=='https': + environ.setdefault('SERVER_PORT', '443') + + + + +_hoppish = { + 'connection':1, 'keep-alive':1, 'proxy-authenticate':1, + 'proxy-authorization':1, 'te':1, 'trailers':1, 'transfer-encoding':1, + 'upgrade':1 +}.has_key + +def is_hop_by_hop(header_name): + """Return true if 'header_name' is an HTTP/1.1 "Hop-by-Hop" header""" + return _hoppish(header_name.lower()) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Added: python/trunk/Lib/wsgiref/validate.py ============================================================================== --- (empty file) +++ python/trunk/Lib/wsgiref/validate.py Fri Jun 9 18:40:18 2006 @@ -0,0 +1,429 @@ +# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org) +# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php +# Also licenced under the Apache License, 2.0: http://opensource.org/licenses/apache2.0.php +# Licensed to PSF under a Contributor Agreement +""" +Middleware to check for obedience to the WSGI specification. + +Some of the things this checks: + +* Signature of the application and start_response (including that + keyword arguments are not used). + +* Environment checks: + + - Environment is a dictionary (and not a subclass). + + - That all the required keys are in the environment: REQUEST_METHOD, + SERVER_NAME, SERVER_PORT, wsgi.version, wsgi.input, wsgi.errors, + wsgi.multithread, wsgi.multiprocess, wsgi.run_once + + - That HTTP_CONTENT_TYPE and HTTP_CONTENT_LENGTH are not in the + environment (these headers should appear as CONTENT_LENGTH and + CONTENT_TYPE). + + - Warns if QUERY_STRING is missing, as the cgi module acts + unpredictably in that case. + + - That CGI-style variables (that don't contain a .) have + (non-unicode) string values + + - That wsgi.version is a tuple + + - That wsgi.url_scheme is 'http' or 'https' (@@: is this too + restrictive?) + + - Warns if the REQUEST_METHOD is not known (@@: probably too + restrictive). + + - That SCRIPT_NAME and PATH_INFO are empty or start with / + + - That at least one of SCRIPT_NAME or PATH_INFO are set. + + - That CONTENT_LENGTH is a positive integer. + + - That SCRIPT_NAME is not '/' (it should be '', and PATH_INFO should + be '/'). + + - That wsgi.input has the methods read, readline, readlines, and + __iter__ + + - That wsgi.errors has the methods flush, write, writelines + +* The status is a string, contains a space, starts with an integer, + and that integer is in range (> 100). + +* That the headers is a list (not a subclass, not another kind of + sequence). + +* That the items of the headers are tuples of strings. + +* That there is no 'status' header (that is used in CGI, but not in + WSGI). + +* That the headers don't contain newlines or colons, end in _ or -, or + contain characters codes below 037. + +* That Content-Type is given if there is content (CGI often has a + default content type, but WSGI does not). + +* That no Content-Type is given when there is no content (@@: is this + too restrictive?) + +* That the exc_info argument to start_response is a tuple or None. + +* That all calls to the writer are with strings, and no other methods + on the writer are accessed. + +* That wsgi.input is used properly: + + - .read() is called with zero or one argument + + - That it returns a string + + - That readline, readlines, and __iter__ return strings + + - That .close() is not called + + - No other methods are provided + +* That wsgi.errors is used properly: + + - .write() and .writelines() is called with a string + + - That .close() is not called, and no other methods are provided. + +* The response iterator: + + - That it is not a string (it should be a list of a single string; a + string will work, but perform horribly). + + - That .next() returns a string + + - That the iterator is not iterated over until start_response has + been called (that can signal either a server or application + error). + + - That .close() is called (doesn't raise exception, only prints to + sys.stderr, because we only know it isn't called when the object + is garbage collected). +""" +__all__ = ['validator'] + + +import re +import sys +from types import DictType, StringType, TupleType, ListType +import warnings + +header_re = re.compile(r'^[a-zA-Z][a-zA-Z0-9\-_]*$') +bad_header_value_re = re.compile(r'[\000-\037]') + +class WSGIWarning(Warning): + """ + Raised in response to WSGI-spec-related warnings + """ + +def validator(application): + + """ + When applied between a WSGI server and a WSGI application, this + middleware will check for WSGI compliancy on a number of levels. + This middleware does not modify the request or response in any + way, but will throw an AssertionError if anything seems off + (except for a failure to close the application iterator, which + will be printed to stderr -- there's no way to throw an exception + at that point). + """ + + def lint_app(*args, **kw): + assert len(args) == 2, "Two arguments required" + assert not kw, "No keyword arguments allowed" + environ, start_response = args + + check_environ(environ) + + # We use this to check if the application returns without + # calling start_response: + start_response_started = [] + + def start_response_wrapper(*args, **kw): + assert len(args) == 2 or len(args) == 3, ( + "Invalid number of arguments: %s" % args) + assert not kw, "No keyword arguments allowed" + status = args[0] + headers = args[1] + if len(args) == 3: + exc_info = args[2] + else: + exc_info = None + + check_status(status) + check_headers(headers) + check_content_type(status, headers) + check_exc_info(exc_info) + + start_response_started.append(None) + return WriteWrapper(start_response(*args)) + + environ['wsgi.input'] = InputWrapper(environ['wsgi.input']) + environ['wsgi.errors'] = ErrorWrapper(environ['wsgi.errors']) + + iterator = application(environ, start_response_wrapper) + assert iterator is not None and iterator != False, ( + "The application must return an iterator, if only an empty list") + + check_iterator(iterator) + + return IteratorWrapper(iterator, start_response_started) + + return lint_app + +class InputWrapper: + + def __init__(self, wsgi_input): + self.input = wsgi_input + + def read(self, *args): + assert len(args) <= 1 + v = self.input.read(*args) + assert type(v) is type("") + return v + + def readline(self): + v = self.input.readline() + assert type(v) is type("") + return v + + def readlines(self, *args): + assert len(args) <= 1 + lines = self.input.readlines(*args) + assert type(lines) is type([]) + for line in lines: + assert type(line) is type("") + return lines + + def __iter__(self): + while 1: + line = self.readline() + if not line: + return + yield line + + def close(self): + assert 0, "input.close() must not be called" + +class ErrorWrapper: + + def __init__(self, wsgi_errors): + self.errors = wsgi_errors + + def write(self, s): + assert type(s) is type("") + self.errors.write(s) + + def flush(self): + self.errors.flush() + + def writelines(self, seq): + for line in seq: + self.write(line) + + def close(self): + assert 0, "errors.close() must not be called" + +class WriteWrapper: + + def __init__(self, wsgi_writer): + self.writer = wsgi_writer + + def __call__(self, s): + assert type(s) is type("") + self.writer(s) + +class PartialIteratorWrapper: + + def __init__(self, wsgi_iterator): + self.iterator = wsgi_iterator + + def __iter__(self): + # We want to make sure __iter__ is called + return IteratorWrapper(self.iterator) + +class IteratorWrapper: + + def __init__(self, wsgi_iterator, check_start_response): + self.original_iterator = wsgi_iterator + self.iterator = iter(wsgi_iterator) + self.closed = False + self.check_start_response = check_start_response + + def __iter__(self): + return self + + def next(self): + assert not self.closed, ( + "Iterator read after closed") + v = self.iterator.next() + if self.check_start_response is not None: + assert self.check_start_response, ( + "The application returns and we started iterating over its body, but start_response has not yet been called") + self.check_start_response = None + return v + + def close(self): + self.closed = True + if hasattr(self.original_iterator, 'close'): + self.original_iterator.close() + + def __del__(self): + if not self.closed: + sys.stderr.write( + "Iterator garbage collected without being closed") + assert self.closed, ( + "Iterator garbage collected without being closed") + +def check_environ(environ): + assert type(environ) is DictType, ( + "Environment is not of the right type: %r (environment: %r)" + % (type(environ), environ)) + + for key in ['REQUEST_METHOD', 'SERVER_NAME', 'SERVER_PORT', + 'wsgi.version', 'wsgi.input', 'wsgi.errors', + 'wsgi.multithread', 'wsgi.multiprocess', + 'wsgi.run_once']: + assert key in environ, ( + "Environment missing required key: %r" % key) + + for key in ['HTTP_CONTENT_TYPE', 'HTTP_CONTENT_LENGTH']: + assert key not in environ, ( + "Environment should not have the key: %s " + "(use %s instead)" % (key, key[5:])) + + if 'QUERY_STRING' not in environ: + warnings.warn( + 'QUERY_STRING is not in the WSGI environment; the cgi ' + 'module will use sys.argv when this variable is missing, ' + 'so application errors are more likely', + WSGIWarning) + + for key in environ.keys(): + if '.' in key: + # Extension, we don't care about its type + continue + assert type(environ[key]) is StringType, ( + "Environmental variable %s is not a string: %r (value: %r)" + % (type(environ[key]), environ[key])) + + assert type(environ['wsgi.version']) is TupleType, ( + "wsgi.version should be a tuple (%r)" % environ['wsgi.version']) + assert environ['wsgi.url_scheme'] in ('http', 'https'), ( + "wsgi.url_scheme unknown: %r" % environ['wsgi.url_scheme']) + + check_input(environ['wsgi.input']) + check_errors(environ['wsgi.errors']) + + # @@: these need filling out: + if environ['REQUEST_METHOD'] not in ( + 'GET', 'HEAD', 'POST', 'OPTIONS','PUT','DELETE','TRACE'): + warnings.warn( + "Unknown REQUEST_METHOD: %r" % environ['REQUEST_METHOD'], + WSGIWarning) + + assert (not environ.get('SCRIPT_NAME') + or environ['SCRIPT_NAME'].startswith('/')), ( + "SCRIPT_NAME doesn't start with /: %r" % environ['SCRIPT_NAME']) + assert (not environ.get('PATH_INFO') + or environ['PATH_INFO'].startswith('/')), ( + "PATH_INFO doesn't start with /: %r" % environ['PATH_INFO']) + if environ.get('CONTENT_LENGTH'): + assert int(environ['CONTENT_LENGTH']) >= 0, ( + "Invalid CONTENT_LENGTH: %r" % environ['CONTENT_LENGTH']) + + if not environ.get('SCRIPT_NAME'): + assert environ.has_key('PATH_INFO'), ( + "One of SCRIPT_NAME or PATH_INFO are required (PATH_INFO " + "should at least be '/' if SCRIPT_NAME is empty)") + assert environ.get('SCRIPT_NAME') != '/', ( + "SCRIPT_NAME cannot be '/'; it should instead be '', and " + "PATH_INFO should be '/'") + +def check_input(wsgi_input): + for attr in ['read', 'readline', 'readlines', '__iter__']: + assert hasattr(wsgi_input, attr), ( + "wsgi.input (%r) doesn't have the attribute %s" + % (wsgi_input, attr)) + +def check_errors(wsgi_errors): + for attr in ['flush', 'write', 'writelines']: + assert hasattr(wsgi_errors, attr), ( + "wsgi.errors (%r) doesn't have the attribute %s" + % (wsgi_errors, attr)) + +def check_status(status): + assert type(status) is StringType, ( + "Status must be a string (not %r)" % status) + # Implicitly check that we can turn it into an integer: + status_code = status.split(None, 1)[0] + assert len(status_code) == 3, ( + "Status codes must be three characters: %r" % status_code) + status_int = int(status_code) + assert status_int >= 100, "Status code is invalid: %r" % status_int + if len(status) < 4 or status[3] != ' ': + warnings.warn( + "The status string (%r) should be a three-digit integer " + "followed by a single space and a status explanation" + % status, WSGIWarning) + +def check_headers(headers): + assert type(headers) is ListType, ( + "Headers (%r) must be of type list: %r" + % (headers, type(headers))) + header_names = {} + for item in headers: + assert type(item) is TupleType, ( + "Individual headers (%r) must be of type tuple: %r" + % (item, type(item))) + assert len(item) == 2 + name, value = item + assert name.lower() != 'status', ( + "The Status header cannot be used; it conflicts with CGI " + "script, and HTTP status is not given through headers " + "(value: %r)." % value) + header_names[name.lower()] = None + assert '\n' not in name and ':' not in name, ( + "Header names may not contain ':' or '\\n': %r" % name) + assert header_re.search(name), "Bad header name: %r" % name + assert not name.endswith('-') and not name.endswith('_'), ( + "Names may not end in '-' or '_': %r" % name) + assert not bad_header_value_re.search(value), ( + "Bad header value: %r (bad char: %r)" + % (value, bad_header_value_re.search(value).group(0))) + +def check_content_type(status, headers): + code = int(status.split(None, 1)[0]) + # @@: need one more person to verify this interpretation of RFC 2616 + # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html + NO_MESSAGE_BODY = (204, 304) + for name, value in headers: + if name.lower() == 'content-type': + if code not in NO_MESSAGE_BODY: + return + assert 0, (("Content-Type header found in a %s response, " + "which must not return content.") % code) + if code not in NO_MESSAGE_BODY: + assert 0, "No Content-Type header found in headers (%s)" % headers + +def check_exc_info(exc_info): + assert exc_info is None or type(exc_info) is type(()), ( + "exc_info (%r) is not a tuple: %r" % (exc_info, type(exc_info))) + # More exc_info checks? + +def check_iterator(iterator): + # Technically a string is legal, which is why it's a really bad + # idea, because it may cause the response to be returned + # character-by-character + assert not isinstance(iterator, str), ( + "You should not return a string as your application iterator, " + "instead return a single-item list containing that string.") + From python-checkins at python.org Fri Jun 9 18:41:30 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Fri, 9 Jun 2006 18:41:30 +0200 (CEST) Subject: [Python-checkins] r46780 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060609164130.70A321E400A@bag.python.org> Author: mateusz.rukowicz Date: Fri Jun 9 18:41:29 2006 New Revision: 46780 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Rewrite _do_decimal_str which is a way easier to code and understand now, and actually works, some bug fixes also. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Fri Jun 9 18:41:29 2006 @@ -1852,6 +1852,7 @@ int buflen, i, imax, j; int leftdigits, dotplace, adjexp; int append_E = 0, append_adjexp = 0; + long extra_zeros=0; char *p, *end; PyObject *ret; @@ -1878,63 +1879,119 @@ *p++ = '-'; } - leftdigits = d->exp + d->ob_size; - if (engineering) { - dotplace = (leftdigits-1)%3+1; - adjexp = leftdigits -1 - (leftdigits-1)%3; - } else { - dotplace = 1; - adjexp = leftdigits - 1; - } - if (d->exp == 0) { - dotplace = -1; /* no dot */ - } else if (d->exp < 0 && adjexp >= 0) { - dotplace = leftdigits; - } else if (d->exp < 0 && adjexp >= -6) { - if (!engineering) { /* eng has no leading zeros */ - for (i = leftdigits; i < 0; i++) { - *p++ = '0'; - SANITY_CHECK(p); - } - *p++ = '0'; - } - } else { - if (d->ob_size <= dotplace) - dotplace = -1; + /* The rule is as follows, imagine you write integer: + * 123456789 we want to print it, without adding post-zeros + * or adding at most 6 pre-zeros that is + * 0.00000123456789, if none of this can be done, then, we put dot + * after first digit, and compute E *or* if engineering, then, we put + * dot after 1st 2nd or 3rd digit, and assure that E is divisible by 3 + * huh */ + + /* (1) dotplace = -adjexp + d->exp + d->ob_size */ + /* why is that like? well, d->exp moves dot right, d->ob_size moves dot right + * and adjexp moves dot left */ + adjexp = 0; + dotplace = d->exp + d->ob_size; + /* dotplace must be at most d->ob_size (no dot at all) and at last -5 (6 pre-zeros)*/ + if(dotplace >d->ob_size || dotplace <-5) + { + /* ok, we have to put dot after 1 digit, that is dotplace = 1 + * we compute adjexp from equation (1) */ + dotplace = 1; + adjexp = -dotplace + d->exp + d->ob_size; + } - if (adjexp) { - append_E = 1; - append_adjexp = 1; - } - } + if(engineering) /* we have to be sure, adjexp%3 == 0 */ + while(adjexp%3) + { + adjexp --; + dotplace ++; + extra_zeros ++; + } + + /* now all we have to do, is to put it to the string =] */ + + if(dotplace <= 0) + { + *p++ = '0'; + *p++ = '.'; + while(dotplace++ < 0) + *p++ = '0'; + dotplace = -1; + } + + for(i = 0;iob_size;i++) + { + if(dotplace == i) + *p++ = '.'; + p += sprintf(p, "%d", _limb_get_digit(d->limbs,d->ob_size,i)); + } + + while(extra_zeros --)*p++ = '0'; + + /* that was a way easier way =] */ + +// if (engineering) { +// dotplace = (leftdigits-1)%3+1; +// adjexp = leftdigits -1 - (leftdigits-1)%3; +// } else { +// dotplace = 1; +// adjexp = leftdigits - 1; +// } +// if (d->exp == 0) { +// dotplace = -1; /* no dot */ +// } else if (d->exp < 0 && adjexp >= 0) { +// dotplace = leftdigits; +// } else if (d->exp < 0 && adjexp >= -6) { +// if (!engineering) { /* eng has no leading zeros */ +// for (i = leftdigits; i < 0; i++) { +// *p++ = '0'; +// SANITY_CHECK(p); +// } +// *p++ = '0'; +// } +// } else { +// if (d->ob_size <= dotplace) +// dotplace = -1; + +// if (adjexp) { +// append_E = 1; +// append_adjexp = 1; +// } +// } - imax = d->ob_size; - if (dotplace > 0) { - imax++; - } - if (engineering) { /* eng has no leading zeros */ - for (i = 0, j = 0; i < imax; i++) { +// imax = d->ob_size; +// if (dotplace > 0) { +// imax++; +// } +// if (engineering) { /* eng has no leading zeros */ +// for (i = 0, j = 0; i < imax; i++) { // if (d->digits[j] == 0) { - if(_limb_get_digit(d->limbs, d->ob_size, j) == 0){ - j++; - } else { - break; - } - } - } - for (i = 0, j = 0; i < imax; i++) { - if (i == dotplace) { - *p++ = '.'; - SANITY_CHECK(p); - continue; - } else { +// if(_limb_get_digit(d->limbs, d->ob_size, j) == 0){ +// j++; +// } else { +// break; +// } +// } +// } +// for (i = 0, j = 0; i < imax; i++) { +// if (i == dotplace) { +// *p++ = '.'; +// SANITY_CHECK(p); +// continue; +// } else { // p += sprintf(p, "%d", d->digits[j]); - p += sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, j)); /* SLOW */ - SANITY_CHECK(p); - j++; - } - } +// p += sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, j)); /* SLOW */ +// SANITY_CHECK(p); +// j++; +// } +// } + if(adjexp) + { + append_E = 1; + append_adjexp = 1; + } if (append_E) { if (context->capitals) { *p++ = 'E'; @@ -1957,13 +2014,14 @@ SANITY_CHECK(p); /* if engineering and (just a sign | empty | starts with a dot or an E) */ - if (engineering && ((p - &outbuf[1] > (d->sign & 1)) || outbuf[1] == '.' || - outbuf[1] == 'E' || outbuf[1] == 'e')) { - outbuf[0] = '0'; - p = outbuf; - } else { - p = &outbuf[1]; - } +// if (engineering && ((p - &outbuf[1] > (d->sign & 1)) || outbuf[1] == '.' || +// outbuf[1] == 'E' || outbuf[1] == 'e')) { +// outbuf[0] = '0'; +// p = outbuf; +// } else { +// p = &outbuf[1]; +// } + p = &outbuf[1]; SANITY_CHECK(p); ret = PyString_FromString(p); PyMem_FREE(outbuf); @@ -2685,6 +2743,9 @@ long i; decimalobject *new; + if(!len) + goto err; /* empty string */ + if(*p == '+') p++; else if(*p == '-') @@ -2693,7 +2754,7 @@ sign = 1; } - if(*p == 'e' || *p == 'E') /* no digits befor E */ + if(*p == 'e' || *p == 'E') /* no digits before E */ goto err; /* now at *p is our integer with (possibly) dot */ @@ -2726,7 +2787,7 @@ } else { - if(dot) + if(dot && !e) digits_after_dot ++; if(!first_digit && *c != '0') @@ -2764,6 +2825,8 @@ exp -= digits_after_dot; new = _new_decimalobj(type, ndigits, sign, exp); + if(!new) + return NULL; for(i = 0; ilimb_count ;i++) new->limbs[i] = 0; From python-checkins at python.org Fri Jun 9 18:42:03 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 18:42:03 +0200 (CEST) Subject: [Python-checkins] r46781 - sandbox/trunk/setuptools/setuptools/command/easy_install.py Message-ID: <20060609164203.995D01E400C@bag.python.org> Author: phillip.eby Date: Fri Jun 9 18:42:03 2006 New Revision: 46781 Modified: sandbox/trunk/setuptools/setuptools/command/easy_install.py Log: Fix sometimes not detecting local packages installed outside of "site" directories. Modified: sandbox/trunk/setuptools/setuptools/command/easy_install.py ============================================================================== --- sandbox/trunk/setuptools/setuptools/command/easy_install.py (original) +++ sandbox/trunk/setuptools/setuptools/command/easy_install.py Fri Jun 9 18:42:03 2006 @@ -171,7 +171,7 @@ self.package_index = self.create_index( self.index_url, search_path = self.shadow_path, hosts=hosts ) - self.local_index = Environment(self.shadow_path) + self.local_index = Environment(self.shadow_path+sys.path) if self.find_links is not None: if isinstance(self.find_links, basestring): From python-checkins at python.org Fri Jun 9 18:44:16 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 18:44:16 +0200 (CEST) Subject: [Python-checkins] r46782 - in sandbox/branches/setuptools-0.6: EasyInstall.txt setuptools/command/easy_install.py Message-ID: <20060609164416.20FE81E400A@bag.python.org> Author: phillip.eby Date: Fri Jun 9 18:44:15 2006 New Revision: 46782 Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py Log: Fix sometimes not detecting local packages installed outside of "site" directories. (merge from trunk) Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt ============================================================================== --- sandbox/branches/setuptools-0.6/EasyInstall.txt (original) +++ sandbox/branches/setuptools-0.6/EasyInstall.txt Fri Jun 9 18:44:15 2006 @@ -1098,6 +1098,9 @@ 0.6b3 * Fix local --find-links eggs not being copied except with --always-copy. + * Fix sometimes not detecting local packages installed outside of "site" + directories. + 0.6b2 * Don't install or update a ``site.py`` patch when installing to a ``PYTHONPATH`` directory with ``--multi-version``, unless an Modified: sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py Fri Jun 9 18:44:15 2006 @@ -171,7 +171,7 @@ self.package_index = self.create_index( self.index_url, search_path = self.shadow_path, hosts=hosts ) - self.local_index = Environment(self.shadow_path) + self.local_index = Environment(self.shadow_path+sys.path) if self.find_links is not None: if isinstance(self.find_links, basestring): From python-checkins at python.org Fri Jun 9 18:44:41 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 18:44:41 +0200 (CEST) Subject: [Python-checkins] r46783 - python/trunk/Misc/NEWS Message-ID: <20060609164441.0E7BA1E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 18:44:40 2006 New Revision: 46783 Modified: python/trunk/Misc/NEWS Log: Add note about XMLGenerator bugfix Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Fri Jun 9 18:44:40 2006 @@ -191,6 +191,10 @@ - Patch #1470846: fix urllib2 ProxyBasicAuthHandler. +- Bug #1472827: correctly escape newlines and tabs in attribute values in + the saxutils.XMLGenerator class. + + Build ----- From python-checkins at python.org Fri Jun 9 18:46:51 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 18:46:51 +0200 (CEST) Subject: [Python-checkins] r46784 - python/trunk/Misc/NEWS Message-ID: <20060609164651.E769A1E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 18:46:51 2006 New Revision: 46784 Modified: python/trunk/Misc/NEWS Log: Add note about wsgiref Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Fri Jun 9 18:46:51 2006 @@ -136,6 +136,8 @@ Library ------- +- The wsgiref package has been added to the standard library. + - The functions update_wrapper() and wraps() have been added to the functools module. These make it easier to copy relevant metadata from the original function when writing wrapper functions. From brett at python.org Fri Jun 9 19:02:32 2006 From: brett at python.org (Brett Cannon) Date: Fri, 9 Jun 2006 10:02:32 -0700 Subject: [Python-checkins] r46757 - in python/trunk: Lib/test/test_types.py Misc/NEWS Modules/arraymodule.c Objects/bufferobject.c In-Reply-To: References: <20060608170049.6A6411E4011@bag.python.org> Message-ID: No good reason. =) I will change it. -Brett On 6/8/06, Neal Norwitz wrote: > > On 6/8/06, brett.cannon wrote: > > Author: brett.cannon > > Date: Thu Jun 8 19:00:45 2006 > > New Revision: 46757 > > > > Modified: python/trunk/Objects/bufferobject.c > > > ============================================================================== > > --- python/trunk/Objects/bufferobject.c (original) > > +++ python/trunk/Objects/bufferobject.c Thu Jun 8 19:00:45 2006 > > @@ -15,8 +15,16 @@ > > } PyBufferObject; > > > > > > +enum buffer_t { > > + READBUFFER, > > + WRITEBUFFER, > > + CHARBUFFER, > > + ANY_BUFFER, > > +}; > > Why the inconsistent use of _? My preference is for all them to have _. > > n > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060609/ff0dbf3e/attachment.html From python-checkins at python.org Fri Jun 9 19:05:48 2006 From: python-checkins at python.org (brett.cannon) Date: Fri, 9 Jun 2006 19:05:48 +0200 (CEST) Subject: [Python-checkins] r46785 - python/trunk/Objects/bufferobject.c Message-ID: <20060609170548.5ADCA1E4024@bag.python.org> Author: brett.cannon Date: Fri Jun 9 19:05:48 2006 New Revision: 46785 Modified: python/trunk/Objects/bufferobject.c Log: Fix inconsistency in naming within an enum. Modified: python/trunk/Objects/bufferobject.c ============================================================================== --- python/trunk/Objects/bufferobject.c (original) +++ python/trunk/Objects/bufferobject.c Fri Jun 9 19:05:48 2006 @@ -16,9 +16,9 @@ enum buffer_t { - READBUFFER, - WRITEBUFFER, - CHARBUFFER, + READ_BUFFER, + WRITE_BUFFER, + CHAR_BUFFER, ANY_BUFFER, }; @@ -40,13 +40,13 @@ "single-segment buffer object expected"); return 0; } - if ((buffer_type == READBUFFER) || + if ((buffer_type == READ_BUFFER) || ((buffer_type == ANY_BUFFER) && self->b_readonly)) proc = bp->bf_getreadbuffer; - else if ((buffer_type == WRITEBUFFER) || + else if ((buffer_type == WRITE_BUFFER) || (buffer_type == ANY_BUFFER)) proc = (readbufferproc)bp->bf_getwritebuffer; - else if (buffer_type == CHARBUFFER) { + else if (buffer_type == CHAR_BUFFER) { if (!PyType_HasFeature(self->ob_type, Py_TPFLAGS_HAVE_GETCHARBUFFER)) { PyErr_SetString(PyExc_TypeError, @@ -58,13 +58,13 @@ if (!proc) { char *buffer_type_name; switch (buffer_type) { - case READBUFFER: + case READ_BUFFER: buffer_type_name = "read"; break; - case WRITEBUFFER: + case WRITE_BUFFER: buffer_type_name = "write"; break; - case CHARBUFFER: + case CHAR_BUFFER: buffer_type_name = "char"; break; default: @@ -592,7 +592,7 @@ "accessing non-existent buffer segment"); return -1; } - if (!get_buf(self, pp, &size, READBUFFER)) + if (!get_buf(self, pp, &size, READ_BUFFER)) return -1; return size; } @@ -613,7 +613,7 @@ "accessing non-existent buffer segment"); return -1; } - if (!get_buf(self, pp, &size, WRITEBUFFER)) + if (!get_buf(self, pp, &size, WRITE_BUFFER)) return -1; return size; } @@ -640,7 +640,7 @@ "accessing non-existent buffer segment"); return -1; } - if (!get_buf(self, &ptr, &size, CHARBUFFER)) + if (!get_buf(self, &ptr, &size, CHAR_BUFFER)) return -1; *pp = (const char *)ptr; return size; From python-checkins at python.org Fri Jun 9 19:10:06 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 19:10:06 +0200 (CEST) Subject: [Python-checkins] r46786 - sandbox/trunk/wsgiref-docs Message-ID: <20060609171006.2AD6C1E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 19:10:05 2006 New Revision: 46786 Removed: sandbox/trunk/wsgiref-docs/ Log: Remove draft of wsgiref docs From python-checkins at python.org Fri Jun 9 19:47:01 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 9 Jun 2006 19:47:01 +0200 (CEST) Subject: [Python-checkins] r46787 - python/trunk/Lib/wsgiref/handlers.py python/trunk/Lib/wsgiref/headers.py python/trunk/Lib/wsgiref/simple_server.py python/trunk/Lib/wsgiref/util.py python/trunk/Lib/wsgiref/validate.py Message-ID: <20060609174701.237691E400C@bag.python.org> Author: tim.peters Date: Fri Jun 9 19:47:00 2006 New Revision: 46787 Modified: python/trunk/Lib/wsgiref/handlers.py python/trunk/Lib/wsgiref/headers.py python/trunk/Lib/wsgiref/simple_server.py python/trunk/Lib/wsgiref/util.py python/trunk/Lib/wsgiref/validate.py Log: Whitespace normalization. Modified: python/trunk/Lib/wsgiref/handlers.py ============================================================================== --- python/trunk/Lib/wsgiref/handlers.py (original) +++ python/trunk/Lib/wsgiref/handlers.py Fri Jun 9 19:47:00 2006 @@ -209,7 +209,7 @@ assert type(data) is StringType,"write() argument must be string" if not self.status: - raise AssertionError("write() before start_response()") + raise AssertionError("write() before start_response()") elif not self.headers_sent: # Before the first output, send the stored headers @@ -473,20 +473,3 @@ self, sys.stdin, sys.stdout, sys.stderr, dict(os.environ.items()), multithread=False, multiprocess=True ) - - - - - - - - - - - - - - - - - Modified: python/trunk/Lib/wsgiref/headers.py ============================================================================== --- python/trunk/Lib/wsgiref/headers.py (original) +++ python/trunk/Lib/wsgiref/headers.py Fri Jun 9 19:47:00 2006 @@ -187,19 +187,3 @@ else: parts.append(_formatparam(k.replace('_', '-'), v)) self._headers.append((_name, "; ".join(parts))) - - - - - - - - - - - - - - - - Modified: python/trunk/Lib/wsgiref/simple_server.py ============================================================================== --- python/trunk/Lib/wsgiref/simple_server.py (original) +++ python/trunk/Lib/wsgiref/simple_server.py Fri Jun 9 19:47:00 2006 @@ -191,15 +191,3 @@ import webbrowser webbrowser.open('http://localhost:8000/xyz?abc') httpd.handle_request() # serve one request, then exit - - - - - - - - - - - - Modified: python/trunk/Lib/wsgiref/util.py ============================================================================== --- python/trunk/Lib/wsgiref/util.py (original) +++ python/trunk/Lib/wsgiref/util.py Fri Jun 9 19:47:00 2006 @@ -59,10 +59,10 @@ if environ['wsgi.url_scheme'] == 'https': if environ['SERVER_PORT'] != '443': - url += ':' + environ['SERVER_PORT'] + url += ':' + environ['SERVER_PORT'] else: if environ['SERVER_PORT'] != '80': - url += ':' + environ['SERVER_PORT'] + url += ':' + environ['SERVER_PORT'] url += quote(environ.get('SCRIPT_NAME') or '/') return url @@ -171,35 +171,3 @@ def is_hop_by_hop(header_name): """Return true if 'header_name' is an HTTP/1.1 "Hop-by-Hop" header""" return _hoppish(header_name.lower()) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Modified: python/trunk/Lib/wsgiref/validate.py ============================================================================== --- python/trunk/Lib/wsgiref/validate.py (original) +++ python/trunk/Lib/wsgiref/validate.py Fri Jun 9 19:47:00 2006 @@ -426,4 +426,3 @@ assert not isinstance(iterator, str), ( "You should not return a string as your application iterator, " "instead return a single-item list containing that string.") - From python-checkins at python.org Fri Jun 9 19:55:10 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 19:55:10 +0200 (CEST) Subject: [Python-checkins] r46788 - sandbox/trunk/setuptools/ez_setup.py Message-ID: <20060609175510.AB8BC1E400D@bag.python.org> Author: phillip.eby Date: Fri Jun 9 19:55:10 2006 New Revision: 46788 Modified: sandbox/trunk/setuptools/ez_setup.py Log: Fix mysterious errors during initial setuptools install, caused by ez_setup trying to run easy_install twice, due to a code fallthru after deleting the egg from which it's running. Modified: sandbox/trunk/setuptools/ez_setup.py ============================================================================== --- sandbox/trunk/setuptools/ez_setup.py (original) +++ sandbox/trunk/setuptools/ez_setup.py Fri Jun 9 19:55:10 2006 @@ -139,7 +139,7 @@ egg = download_setuptools(version, to_dir=tmpdir, delay=0) sys.path.insert(0,egg) from setuptools.command.easy_install import main - main(list(argv)+[egg]) + return main(list(argv)+[egg]) # we're done here finally: shutil.rmtree(tmpdir) else: From python-checkins at python.org Fri Jun 9 19:56:34 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 19:56:34 +0200 (CEST) Subject: [Python-checkins] r46789 - sandbox/branches/setuptools-0.6/EasyInstall.txt sandbox/branches/setuptools-0.6/ez_setup.py Message-ID: <20060609175634.77A091E400E@bag.python.org> Author: phillip.eby Date: Fri Jun 9 19:56:34 2006 New Revision: 46789 Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt sandbox/branches/setuptools-0.6/ez_setup.py Log: Fix mysterious errors during initial setuptools install, caused by ez_setup trying to run easy_install twice, due to a code fallthru after deleting the egg from which it's running. (merge from trunk) Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt ============================================================================== --- sandbox/branches/setuptools-0.6/EasyInstall.txt (original) +++ sandbox/branches/setuptools-0.6/EasyInstall.txt Fri Jun 9 19:56:34 2006 @@ -1101,6 +1101,10 @@ * Fix sometimes not detecting local packages installed outside of "site" directories. + * Fix mysterious errors during initial ``setuptools`` install, caused by + ``ez_setup`` trying to run ``easy_install`` twice, due to a code fallthru + after deleting the egg from which it's running. + 0.6b2 * Don't install or update a ``site.py`` patch when installing to a ``PYTHONPATH`` directory with ``--multi-version``, unless an Modified: sandbox/branches/setuptools-0.6/ez_setup.py ============================================================================== --- sandbox/branches/setuptools-0.6/ez_setup.py (original) +++ sandbox/branches/setuptools-0.6/ez_setup.py Fri Jun 9 19:56:34 2006 @@ -14,7 +14,7 @@ This file can also be run as a script to install or upgrade setuptools. """ import sys -DEFAULT_VERSION = "0.6b3" +DEFAULT_VERSION = "0.6b2" DEFAULT_URL = "http://cheeseshop.python.org/packages/%s/s/setuptools/" % sys.version[:3] md5_data = { @@ -143,7 +143,7 @@ egg = download_setuptools(version, to_dir=tmpdir, delay=0) sys.path.insert(0,egg) from setuptools.command.easy_install import main - main(list(argv)+[egg]) + return main(list(argv)+[egg]) # we're done here finally: shutil.rmtree(tmpdir) else: From buildbot at python.org Fri Jun 9 20:10:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 18:10:14 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060609181015.23D981E401C@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1085 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 20:24:28 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 20:24:28 +0200 (CEST) Subject: [Python-checkins] r46790 - sandbox/trunk/setuptools/setuptools/command/build_py.py Message-ID: <20060609182428.7F9F81E400A@bag.python.org> Author: phillip.eby Date: Fri Jun 9 20:24:28 2006 New Revision: 46790 Modified: sandbox/trunk/setuptools/setuptools/command/build_py.py Log: Allow .py files found by the include_package_data option to be automatically included. Remove duplicate data file matches if both include_package_data and package_data are used to refer to the same files. Modified: sandbox/trunk/setuptools/setuptools/command/build_py.py ============================================================================== --- sandbox/trunk/setuptools/setuptools/command/build_py.py (original) +++ sandbox/trunk/setuptools/setuptools/command/build_py.py Fri Jun 9 20:24:28 2006 @@ -92,18 +92,18 @@ self.run_command('egg_info') ei_cmd = self.get_finalized_command('egg_info') for path in ei_cmd.filelist.files: - if path.endswith('.py'): - continue d,f = os.path.split(assert_relative(path)) prev = None + oldf = f while d and d!=prev and d not in src_dirs: prev = d d, df = os.path.split(d) f = os.path.join(df, f) if d in src_dirs: + if path.endswith('.py') and f==oldf: + continue # it's a module, not data mf.setdefault(src_dirs[d],[]).append(path) - def get_data_files(self): pass # kludge 2.4 for lazy computation if sys.version<"2.4": # Python 2.4 already has this code @@ -174,7 +174,11 @@ ) ) bad = dict.fromkeys(bad) - return [f for f in files if f not in bad] + seen = {} + return [ + f for f in files if f not in bad + and f not in seen and seen.setdefault(f,1) # ditch dupes + ] def assert_relative(path): @@ -190,3 +194,12 @@ setup.py directory, *never* absolute paths. """ % path ) + + + + + + + + + From python-checkins at python.org Fri Jun 9 20:26:46 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 20:26:46 +0200 (CEST) Subject: [Python-checkins] r46791 - in sandbox/branches/setuptools-0.6: setuptools.txt setuptools/command/build_py.py Message-ID: <20060609182646.597C61E4019@bag.python.org> Author: phillip.eby Date: Fri Jun 9 20:26:45 2006 New Revision: 46791 Modified: sandbox/branches/setuptools-0.6/setuptools.txt sandbox/branches/setuptools-0.6/setuptools/command/build_py.py Log: Allow .py files found by the include_package_data option to be automatically included. Remove duplicate data file matches if both include_package_data and package_data are used to refer to the same files. (merge from trunk) Modified: sandbox/branches/setuptools-0.6/setuptools.txt ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools.txt (original) +++ sandbox/branches/setuptools-0.6/setuptools.txt Fri Jun 9 20:26:45 2006 @@ -2501,6 +2501,11 @@ 0.6b3 * Fix bdist_egg not including files in .egg-info subdirectories. + * Allow .py files found by the include_package_data option to be + automatically included. Remove duplicate data file matches if both + include_package_data and package_data are used to refer to the same + files. + 0.6b1 * Strip ``module`` from the end of compiled extension modules when computing the name of a ``.py`` loader/wrapper. (Python's import machinery ignores Modified: sandbox/branches/setuptools-0.6/setuptools/command/build_py.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/command/build_py.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/command/build_py.py Fri Jun 9 20:26:45 2006 @@ -92,18 +92,18 @@ self.run_command('egg_info') ei_cmd = self.get_finalized_command('egg_info') for path in ei_cmd.filelist.files: - if path.endswith('.py'): - continue d,f = os.path.split(assert_relative(path)) prev = None + oldf = f while d and d!=prev and d not in src_dirs: prev = d d, df = os.path.split(d) f = os.path.join(df, f) if d in src_dirs: + if path.endswith('.py') and f==oldf: + continue # it's a module, not data mf.setdefault(src_dirs[d],[]).append(path) - def get_data_files(self): pass # kludge 2.4 for lazy computation if sys.version<"2.4": # Python 2.4 already has this code @@ -167,14 +167,18 @@ globs = (self.exclude_package_data.get('', []) + self.exclude_package_data.get(package, [])) bad = [] - for pattern in globs: + for pattern in globs: bad.extend( fnmatch.filter( files, os.path.join(src_dir, convert_path(pattern)) ) ) bad = dict.fromkeys(bad) - return [f for f in files if f not in bad] + seen = {} + return [ + f for f in files if f not in bad + and f not in seen and seen.setdefault(f,1) # ditch dupes + ] def assert_relative(path): @@ -199,7 +203,3 @@ - - - - From python-checkins at python.org Fri Jun 9 20:29:52 2006 From: python-checkins at python.org (georg.brandl) Date: Fri, 9 Jun 2006 20:29:52 +0200 (CEST) Subject: [Python-checkins] r46792 - python/trunk/Lib/test/test_file.py Message-ID: <20060609182952.B6F311E400D@bag.python.org> Author: georg.brandl Date: Fri Jun 9 20:29:52 2006 New Revision: 46792 Modified: python/trunk/Lib/test/test_file.py Log: Test file.__exit__. Modified: python/trunk/Lib/test/test_file.py ============================================================================== --- python/trunk/Lib/test/test_file.py (original) +++ python/trunk/Lib/test/test_file.py Fri Jun 9 20:29:52 2006 @@ -98,7 +98,9 @@ if sys.platform.startswith('atheos'): methods.remove('truncate') - self.f.close() + # __exit__ should close the file + self.f.__exit__(None, None, None) + self.assert_(self.f.closed) for methodname in methods: method = getattr(self.f, methodname) @@ -106,6 +108,14 @@ self.assertRaises(ValueError, method) self.assertRaises(ValueError, self.f.writelines, []) + # file is closed, __exit__ shouldn't do anything + self.assertEquals(self.f.__exit__(None, None, None), None) + # it must also return None if an exception was given + try: + 1/0 + except: + self.assertEquals(self.f.__exit__(*sys.exc_info()), None) + class OtherFileTests(unittest.TestCase): From python-checkins at python.org Fri Jun 9 20:37:08 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 20:37:08 +0200 (CEST) Subject: [Python-checkins] r46793 - sandbox/branches/setuptools-0.6/EasyInstall.txt sandbox/branches/setuptools-0.6/ez_setup.py sandbox/branches/setuptools-0.6/setuptools.txt Message-ID: <20060609183708.9CE141E4019@bag.python.org> Author: phillip.eby Date: Fri Jun 9 20:37:08 2006 New Revision: 46793 Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt sandbox/branches/setuptools-0.6/ez_setup.py sandbox/branches/setuptools-0.6/setuptools.txt Log: 0.6b3 release Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt ============================================================================== --- sandbox/branches/setuptools-0.6/EasyInstall.txt (original) +++ sandbox/branches/setuptools-0.6/EasyInstall.txt Fri Jun 9 20:37:08 2006 @@ -1096,7 +1096,8 @@ ============================ 0.6b3 - * Fix local --find-links eggs not being copied except with --always-copy. + * Fix local ``--find-links`` eggs not being copied except with + ``--always-copy``. * Fix sometimes not detecting local packages installed outside of "site" directories. Modified: sandbox/branches/setuptools-0.6/ez_setup.py ============================================================================== --- sandbox/branches/setuptools-0.6/ez_setup.py (original) +++ sandbox/branches/setuptools-0.6/ez_setup.py Fri Jun 9 20:37:08 2006 @@ -14,18 +14,16 @@ This file can also be run as a script to install or upgrade setuptools. """ import sys -DEFAULT_VERSION = "0.6b2" +DEFAULT_VERSION = "0.6b3" DEFAULT_URL = "http://cheeseshop.python.org/packages/%s/s/setuptools/" % sys.version[:3] md5_data = { - 'setuptools-0.6a10-py2.3.egg': '162d8357f1aff2b0349c6c247ee62987', - 'setuptools-0.6a10-py2.4.egg': '803a2d8db501c1ac3b5b6fb4e907f788', - 'setuptools-0.6a11-py2.3.egg': 'd12bf8e13aaeb25c91350c8d77f01a71', - 'setuptools-0.6a11-py2.4.egg': 'a95d5bc7a070aa1028bc4dcb5270b133', 'setuptools-0.6b1-py2.3.egg': '8822caf901250d848b996b7f25c6e6ca', 'setuptools-0.6b1-py2.4.egg': 'b79a8a403e4502fbb85ee3f1941735cb', 'setuptools-0.6b2-py2.3.egg': '5657759d8a6d8fc44070a9d07272d99b', 'setuptools-0.6b2-py2.4.egg': '4996a8d169d2be661fa32a6e52e4f82a', + 'setuptools-0.6b3dev_r46791-py2.3.egg': 'e765a29566575ffac5d81cdf0c6f8db9', + 'setuptools-0.6b3dev_r46791-py2.4.egg': 'd249c022ed029ad60d134bd998adc880', } import sys, os Modified: sandbox/branches/setuptools-0.6/setuptools.txt ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools.txt (original) +++ sandbox/branches/setuptools-0.6/setuptools.txt Fri Jun 9 20:37:08 2006 @@ -2499,11 +2499,11 @@ ---------------------------- 0.6b3 - * Fix bdist_egg not including files in .egg-info subdirectories. + * Fix ``bdist_egg`` not including files in subdirectories of ``.egg-info``. - * Allow .py files found by the include_package_data option to be + * Allow ``.py`` files found by the ``include_package_data`` option to be automatically included. Remove duplicate data file matches if both - include_package_data and package_data are used to refer to the same + ``include_package_data`` and ``package_data`` are used to refer to the same files. 0.6b1 From buildbot at python.org Fri Jun 9 20:37:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 18:37:14 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060609183714.C9C231E401F@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/614 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,brett.cannon,phillip.eby Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 20:40:46 2006 From: python-checkins at python.org (brett.cannon) Date: Fri, 9 Jun 2006 20:40:46 +0200 (CEST) Subject: [Python-checkins] r46794 - in python/trunk/Lib: msilib test/crashers wsgiref Message-ID: <20060609184046.F04F11E4014@bag.python.org> Author: brett.cannon Date: Fri Jun 9 20:40:46 2006 New Revision: 46794 Modified: python/trunk/Lib/msilib/ (props changed) python/trunk/Lib/test/crashers/ (props changed) python/trunk/Lib/wsgiref/ (props changed) Log: svn:ignore .pyc and .pyo files. From python-checkins at python.org Fri Jun 9 20:45:49 2006 From: python-checkins at python.org (georg.brandl) Date: Fri, 9 Jun 2006 20:45:49 +0200 (CEST) Subject: [Python-checkins] r46795 - in python/trunk: Doc/lib/libstdtypes.tex Lib/test/string_tests.py Misc/NEWS Objects/stringobject.c Objects/unicodeobject.c Message-ID: <20060609184549.B5ABC1E400A@bag.python.org> Author: georg.brandl Date: Fri Jun 9 20:45:48 2006 New Revision: 46795 Modified: python/trunk/Doc/lib/libstdtypes.tex python/trunk/Lib/test/string_tests.py python/trunk/Misc/NEWS python/trunk/Objects/stringobject.c python/trunk/Objects/unicodeobject.c Log: RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Modified: python/trunk/Doc/lib/libstdtypes.tex ============================================================================== --- python/trunk/Doc/lib/libstdtypes.tex (original) +++ python/trunk/Doc/lib/libstdtypes.tex Fri Jun 9 20:45:48 2006 @@ -618,8 +618,11 @@ \begin{methoddesc}[string]{endswith}{suffix\optional{, start\optional{, end}}} Return \code{True} if the string ends with the specified \var{suffix}, -otherwise return \code{False}. With optional \var{start}, test beginning at +otherwise return \code{False}. \var{suffix} can also be a tuple of +suffixes to look for. With optional \var{start}, test beginning at that position. With optional \var{end}, stop comparing at that position. + +\versionchanged[Accept tuples as \var{suffix}]{2.5} \end{methoddesc} \begin{methoddesc}[string]{expandtabs}{\optional{tabsize}} @@ -829,9 +832,12 @@ \begin{methoddesc}[string]{startswith}{prefix\optional{, start\optional{, end}}} Return \code{True} if string starts with the \var{prefix}, otherwise -return \code{False}. With optional \var{start}, test string beginning at +return \code{False}. \var{prefix} can also be a tuple of +suffixes to look for. With optional \var{start}, test string beginning at that position. With optional \var{end}, stop comparing string at that position. + +\versionchanged[Accept tuples as \var{prefix}]{2.5} \end{methoddesc} \begin{methoddesc}[string]{strip}{\optional{chars}} Modified: python/trunk/Lib/test/string_tests.py ============================================================================== --- python/trunk/Lib/test/string_tests.py (original) +++ python/trunk/Lib/test/string_tests.py Fri Jun 9 20:45:48 2006 @@ -819,6 +819,21 @@ self.checkraises(TypeError, 'hello', 'startswith') self.checkraises(TypeError, 'hello', 'startswith', 42) + # test tuple arguments + self.checkequal(True, 'hello', 'startswith', ('he', 'ha')) + self.checkequal(False, 'hello', 'startswith', ('lo', 'llo')) + self.checkequal(True, 'hello', 'startswith', ('hellox', 'hello')) + self.checkequal(False, 'hello', 'startswith', ()) + self.checkequal(True, 'helloworld', 'startswith', ('hellowo', + 'rld', 'lowo'), 3) + self.checkequal(False, 'helloworld', 'startswith', ('hellowo', 'ello', + 'rld'), 3) + self.checkequal(True, 'hello', 'startswith', ('lo', 'he'), 0, -1) + self.checkequal(False, 'hello', 'startswith', ('he', 'hel'), 0, 1) + self.checkequal(True, 'hello', 'startswith', ('he', 'hel'), 0, 2) + + self.checkraises(TypeError, 'hello', 'startswith', (42,)) + def test_endswith(self): self.checkequal(True, 'hello', 'endswith', 'lo') self.checkequal(False, 'hello', 'endswith', 'he') @@ -853,6 +868,21 @@ self.checkraises(TypeError, 'hello', 'endswith') self.checkraises(TypeError, 'hello', 'endswith', 42) + # test tuple arguments + self.checkequal(False, 'hello', 'endswith', ('he', 'ha')) + self.checkequal(True, 'hello', 'endswith', ('lo', 'llo')) + self.checkequal(True, 'hello', 'endswith', ('hellox', 'hello')) + self.checkequal(False, 'hello', 'endswith', ()) + self.checkequal(True, 'helloworld', 'endswith', ('hellowo', + 'rld', 'lowo'), 3) + self.checkequal(False, 'helloworld', 'endswith', ('hellowo', 'ello', + 'rld'), 3, -1) + self.checkequal(True, 'hello', 'endswith', ('hell', 'ell'), 0, -1) + self.checkequal(False, 'hello', 'endswith', ('he', 'hel'), 0, 1) + self.checkequal(True, 'hello', 'endswith', ('he', 'hell'), 0, 4) + + self.checkraises(TypeError, 'hello', 'endswith', (42,)) + def test___contains__(self): self.checkequal(True, '', '__contains__', '') # vereq('' in '', True) self.checkequal(True, 'abc', '__contains__', '') # vereq('' in 'abc', True) @@ -872,7 +902,7 @@ self.checkequal(u'abc', 'abc', '__getitem__', slice(0, 1000)) self.checkequal(u'a', 'abc', '__getitem__', slice(0, 1)) self.checkequal(u'', 'abc', '__getitem__', slice(0, 0)) - # FIXME What about negative indizes? This is handled differently by [] and __getitem__(slice) + # FIXME What about negative indices? This is handled differently by [] and __getitem__(slice) self.checkraises(TypeError, 'abc', '__getitem__', 'def') Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Fri Jun 9 20:45:48 2006 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- The string and unicode methods startswith() and endswith() now accept + a tuple of prefixes/suffixes to look for. Implements RFE #1491485. + - Buffer objects, at the C level, never used the char buffer implementation even when the char buffer for the wrapped object was explicitly requested (originally returned the read or write buffer). Modified: python/trunk/Objects/stringobject.c ============================================================================== --- python/trunk/Objects/stringobject.c (original) +++ python/trunk/Objects/stringobject.c Fri Jun 9 20:45:48 2006 @@ -3099,54 +3099,96 @@ /** End DALKE **/ +/* Matches the end (direction > 0) or start (direction < 0) of self + * against substr, using the start and end arguments. Returns + * -1 on error, 0 if not found and 1 if found. + */ +Py_LOCAL(int) +_string_tailmatch(PyStringObject *self, PyObject *substr, Py_ssize_t start, + Py_ssize_t end, int direction) +{ + Py_ssize_t len = PyString_GET_SIZE(self); + Py_ssize_t slen; + const char* sub; + const char* str; + + if (PyString_Check(substr)) { + sub = PyString_AS_STRING(substr); + slen = PyString_GET_SIZE(substr); + } +#ifdef Py_USING_UNICODE + else if (PyUnicode_Check(substr)) + return PyUnicode_Tailmatch((PyObject *)self, + substr, start, end, direction); +#endif + else if (PyObject_AsCharBuffer(substr, &sub, &slen)) + return -1; + str = PyString_AS_STRING(self); + + string_adjust_indices(&start, &end, len); + + if (direction < 0) { + /* startswith */ + if (start+slen > len) + return 0; + + if (end-start >= slen) + return ! memcmp(str+start, sub, slen); + else + return 0; + } else { + /* endswith */ + if (end-start < slen || start > len) + return 0; + + if (end-slen > start) + start = end - slen; + if (end-start >= slen) + return ! memcmp(str+start, sub, slen); + else + return 0; + } +} + + PyDoc_STRVAR(startswith__doc__, "S.startswith(prefix[, start[, end]]) -> bool\n\ \n\ Return True if S starts with the specified prefix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +prefix can also be a tuple of strings to try."); static PyObject * string_startswith(PyStringObject *self, PyObject *args) { - const char* str = PyString_AS_STRING(self); - Py_ssize_t len = PyString_GET_SIZE(self); - const char* prefix; - Py_ssize_t plen; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; PyObject *subobj; + int result; if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj, _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - if (PyString_Check(subobj)) { - prefix = PyString_AS_STRING(subobj); - plen = PyString_GET_SIZE(subobj); - } -#ifdef Py_USING_UNICODE - else if (PyUnicode_Check(subobj)) { - Py_ssize_t rc; - rc = PyUnicode_Tailmatch((PyObject *)self, - subobj, start, end, -1); - if (rc == -1) - return NULL; - else - return PyBool_FromLong((long) rc); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + result = _string_tailmatch(self, + PyTuple_GET_ITEM(subobj, i), + start, end, -1); + if (result == -1) + return NULL; + else if (result) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; } -#endif - else if (PyObject_AsCharBuffer(subobj, &prefix, &plen)) + result = _string_tailmatch(self, subobj, start, end, -1); + if (result == -1) return NULL; - - string_adjust_indices(&start, &end, len); - - if (start+plen > len) - return PyBool_FromLong(0); - - if (end-start >= plen) - return PyBool_FromLong(!memcmp(str+start, prefix, plen)); else - return PyBool_FromLong(0); + return PyBool_FromLong(result); } @@ -3155,51 +3197,39 @@ \n\ Return True if S ends with the specified suffix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +suffix can also be a tuple of strings to try."); static PyObject * string_endswith(PyStringObject *self, PyObject *args) { - const char* str = PyString_AS_STRING(self); - Py_ssize_t len = PyString_GET_SIZE(self); - const char* suffix; - Py_ssize_t slen; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; PyObject *subobj; + int result; if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj, _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - if (PyString_Check(subobj)) { - suffix = PyString_AS_STRING(subobj); - slen = PyString_GET_SIZE(subobj); - } -#ifdef Py_USING_UNICODE - else if (PyUnicode_Check(subobj)) { - Py_ssize_t rc; - rc = PyUnicode_Tailmatch((PyObject *)self, - subobj, start, end, +1); - if (rc == -1) - return NULL; - else - return PyBool_FromLong((long) rc); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + result = _string_tailmatch(self, + PyTuple_GET_ITEM(subobj, i), + start, end, +1); + if (result == -1) + return NULL; + else if (result) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; } -#endif - else if (PyObject_AsCharBuffer(subobj, &suffix, &slen)) + result = _string_tailmatch(self, subobj, start, end, +1); + if (result == -1) return NULL; - - string_adjust_indices(&start, &end, len); - - if (end-start < slen || start > len) - return PyBool_FromLong(0); - - if (end-slen > start) - start = end - slen; - if (end-start >= slen) - return PyBool_FromLong(!memcmp(str+start, suffix, slen)); else - return PyBool_FromLong(0); + return PyBool_FromLong(result); } Modified: python/trunk/Objects/unicodeobject.c ============================================================================== --- python/trunk/Objects/unicodeobject.c (original) +++ python/trunk/Objects/unicodeobject.c Fri Jun 9 20:45:48 2006 @@ -6667,29 +6667,44 @@ \n\ Return True if S starts with the specified prefix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +prefix can also be a tuple of strings to try."); static PyObject * unicode_startswith(PyUnicodeObject *self, PyObject *args) { + PyObject *subobj; PyUnicodeObject *substring; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - PyObject *result; + int result; - if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &substring, + if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj, _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - substring = (PyUnicodeObject *)PyUnicode_FromObject( - (PyObject *)substring); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + substring = (PyUnicodeObject *)PyUnicode_FromObject( + PyTuple_GET_ITEM(subobj, i)); + if (substring == NULL) + return NULL; + result = tailmatch(self, substring, start, end, -1); + Py_DECREF(substring); + if (result) { + Py_RETURN_TRUE; + } + } + /* nothing matched */ + Py_RETURN_FALSE; + } + substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj); if (substring == NULL) - return NULL; - - result = PyBool_FromLong(tailmatch(self, substring, start, end, -1)); - + return NULL; + result = tailmatch(self, substring, start, end, -1); Py_DECREF(substring); - return result; + return PyBool_FromLong(result); } @@ -6698,29 +6713,44 @@ \n\ Return True if S ends with the specified suffix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +suffix can also be a tuple of strings to try."); static PyObject * unicode_endswith(PyUnicodeObject *self, PyObject *args) { + PyObject *subobj; PyUnicodeObject *substring; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - PyObject *result; + int result; - if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &substring, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj, + _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - substring = (PyUnicodeObject *)PyUnicode_FromObject( - (PyObject *)substring); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + substring = (PyUnicodeObject *)PyUnicode_FromObject( + PyTuple_GET_ITEM(subobj, i)); + if (substring == NULL) + return NULL; + result = tailmatch(self, substring, start, end, +1); + Py_DECREF(substring); + if (result) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; + } + substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj); if (substring == NULL) - return NULL; - - result = PyBool_FromLong(tailmatch(self, substring, start, end, +1)); + return NULL; + result = tailmatch(self, substring, start, end, +1); Py_DECREF(substring); - return result; + return PyBool_FromLong(result); } From buildbot at python.org Fri Jun 9 20:54:18 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 18:54:18 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060609185418.3D5851E400A@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/695 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 20:58:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 18:58:02 +0000 Subject: [Python-checkins] buildbot failure in x86 W2k trunk Message-ID: <20060609185802.964491E4013@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/1007 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 20:59:19 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 20:59:19 +0200 (CEST) Subject: [Python-checkins] r46796 - sandbox/branches/setuptools-0.6/ez_setup.py sandbox/branches/setuptools-0.6/release.sh Message-ID: <20060609185919.4497D1E400C@bag.python.org> Author: phillip.eby Date: Fri Jun 9 20:59:18 2006 New Revision: 46796 Modified: sandbox/branches/setuptools-0.6/ez_setup.py sandbox/branches/setuptools-0.6/release.sh Log: 0.6b3 brown bag fix; update release script to remove revision tags before uploading! Modified: sandbox/branches/setuptools-0.6/ez_setup.py ============================================================================== --- sandbox/branches/setuptools-0.6/ez_setup.py (original) +++ sandbox/branches/setuptools-0.6/ez_setup.py Fri Jun 9 20:59:18 2006 @@ -22,8 +22,8 @@ 'setuptools-0.6b1-py2.4.egg': 'b79a8a403e4502fbb85ee3f1941735cb', 'setuptools-0.6b2-py2.3.egg': '5657759d8a6d8fc44070a9d07272d99b', 'setuptools-0.6b2-py2.4.egg': '4996a8d169d2be661fa32a6e52e4f82a', - 'setuptools-0.6b3dev_r46791-py2.3.egg': 'e765a29566575ffac5d81cdf0c6f8db9', - 'setuptools-0.6b3dev_r46791-py2.4.egg': 'd249c022ed029ad60d134bd998adc880', + 'setuptools-0.6b3-py2.3.egg': 'bb31c0fc7399a63579975cad9f5a0618', + 'setuptools-0.6b3-py2.4.egg': '38a8c6b3d6ecd22247f179f7da669fac', } import sys, os Modified: sandbox/branches/setuptools-0.6/release.sh ============================================================================== --- sandbox/branches/setuptools-0.6/release.sh (original) +++ sandbox/branches/setuptools-0.6/release.sh Fri Jun 9 20:59:18 2006 @@ -9,8 +9,11 @@ export VERSION="0.6b3" +wpython setup.py setopt -r -c egg_info -o tag_build && \ +wpython setup.py setopt -r setopt -r -c egg_info -o tag_svn_revision && \ wpython setup.py -q source && \ cpython setup.py -q binary && \ +svn revert setup.cfg python ez_setup.py --md5update dist/setuptools-$VERSION*-py2.?.egg && \ scp ez_setup.py virtual-python.py t3:web/PEAK/dist/ && \ cp ez_setup.py ~/projects/ez_setup/__init__.py && \ @@ -18,5 +21,5 @@ ~/projects/ez_setup/__init__.py && \ svn up ~/projects/*/ez_setup -# XXX update wiki pages from EasyInstall.txt, setuptools.txt, & +# XXX update wiki pages from EasyInstall.txt, setuptools.txt, & # pkg_resources.txt From python-checkins at python.org Fri Jun 9 21:02:30 2006 From: python-checkins at python.org (phillip.eby) Date: Fri, 9 Jun 2006 21:02:30 +0200 (CEST) Subject: [Python-checkins] r46797 - sandbox/branches/setuptools-0.6/release.sh Message-ID: <20060609190230.544201E400A@bag.python.org> Author: phillip.eby Date: Fri Jun 9 21:02:29 2006 New Revision: 46797 Modified: sandbox/branches/setuptools-0.6/release.sh Log: syntax error Modified: sandbox/branches/setuptools-0.6/release.sh ============================================================================== --- sandbox/branches/setuptools-0.6/release.sh (original) +++ sandbox/branches/setuptools-0.6/release.sh Fri Jun 9 21:02:29 2006 @@ -13,7 +13,7 @@ wpython setup.py setopt -r setopt -r -c egg_info -o tag_svn_revision && \ wpython setup.py -q source && \ cpython setup.py -q binary && \ -svn revert setup.cfg +svn revert setup.cfg && \ python ez_setup.py --md5update dist/setuptools-$VERSION*-py2.?.egg && \ scp ez_setup.py virtual-python.py t3:web/PEAK/dist/ && \ cp ez_setup.py ~/projects/ez_setup/__init__.py && \ From python-checkins at python.org Fri Jun 9 21:03:19 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 21:03:19 +0200 (CEST) Subject: [Python-checkins] r46798 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060609190319.C50211E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 21:03:16 2006 New Revision: 46798 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Describe startswith()/endswiith() change; add reminder about wsgiref Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Fri Jun 9 21:03:16 2006 @@ -2,6 +2,7 @@ \usepackage{distutils} % $Id$ +% wsgiref section % Fix XXX comments % Count up the patches and bugs @@ -1098,6 +1099,17 @@ (Implemented by Fredrik Lundh following a suggestion by Raymond Hettinger.) +\item The \method{startswith()} and \method{endswith()} methods +of string types now accept tuples of strings to check for. + +\begin{verbatim} +def is_image_file (filename): + return filename.endswith(('.gif', '.jpg', '.tiff')) +\end{verbatim} + +(Implemented by Georg Brandl following a suggestion by Tom Lynn.) +% RFE #1491485 + \item The \function{min()} and \function{max()} built-in functions gained a \code{key} keyword parameter analogous to the \code{key} argument for \method{sort()}. This parameter supplies a function that @@ -2015,6 +2027,11 @@ \end{seealso} +%====================================================================== +%\subsection{The wsgiref package\label{module-wsgiref}} + +% XXX write this + % ====================================================================== \section{Build and C API Changes\label{build-api}} From buildbot at python.org Fri Jun 9 21:08:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 19:08:30 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060609190830.D6DB31E400C@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/316 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: kristjan.jonsson Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 21:10:29 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 19:10:29 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060609191029.F01A01E400A@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/990 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Fri Jun 9 21:24:07 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 19:24:07 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060609192407.780A91E400A@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/657 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 9 21:24:45 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 9 Jun 2006 21:24:45 +0200 (CEST) Subject: [Python-checkins] r46799 - python/trunk/Lib/test/regrtest.py Message-ID: <20060609192445.769B71E400A@bag.python.org> Author: tim.peters Date: Fri Jun 9 21:24:44 2006 New Revision: 46799 Modified: python/trunk/Lib/test/regrtest.py Log: Implementing a happy idea from Georg Brandl: make runtest() try to clean up files and directories the tests often leave behind by mistake. This is the first time in history I don't have a bogus "db_home" directory after running the tests ;-) Also worked on runtest's docstring, to say something about all the arguments, and to document the non-obvious return values. New functions runtest_inner() and cleanup_test_droppings() in support of the above. Modified: python/trunk/Lib/test/regrtest.py ============================================================================== --- python/trunk/Lib/test/regrtest.py (original) +++ python/trunk/Lib/test/regrtest.py Fri Jun 9 21:24:44 2006 @@ -496,14 +496,30 @@ def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False): """Run a single test. + test -- the name of the test generate -- if true, generate output, instead of running the test - and comparing it to a previously created output file + and comparing it to a previously created output file verbose -- if true, print more messages quiet -- if true, don't print 'skipped' messages (probably redundant) testdir -- test directory + huntrleaks -- run multiple times to test for leaks; requires a debug + build; a triple corresponding to -R's three arguments + Return: + -2 test skipped because resource denied + -1 test skipped for some other reason + 0 test failed + 1 test passed """ + try: + return runtest_inner(test, generate, verbose, quiet, testdir, + huntrleaks) + finally: + cleanup_test_droppings(test, verbose) + +def runtest_inner(test, generate, verbose, quiet, + testdir=None, huntrleaks=False): test_support.unload(test) if not testdir: testdir = findtestdir() @@ -595,6 +611,37 @@ sys.stdout.flush() return 0 +def cleanup_test_droppings(testname, verbose): + import shutil + + # Try to clean up junk commonly left behind. While tests shouldn't leave + # any files or directories behind, when a test fails that can be tedious + # for it to arrange. The consequences can be especially nasty on Windows, + # since if a test leaves a file open, it cannot be deleted by name (while + # there's nothing we can do about that here either, we can display the + # name of the offending test, which is a real help). + for name in (test_support.TESTFN, + "db_home", + ): + if not os.path.exists(name): + continue + + if os.path.isdir(name): + kind, nuker = "directory", shutil.rmtree + elif os.path.isfile(name): + kind, nuker = "file", os.unlink + else: + raise SystemError("os.path says %r exists but is neither " + "directory nor file" % name) + + if verbose: + print "%r left behind %s %r" % (testname, kind, name) + try: + nuker(name) + except Exception, msg: + print >> sys.stderr, ("%r left behind %s %r and it couldn't be " + "removed: %s" % (testname, kind, name, msg)) + def dash_R(the_module, test, indirect_test, huntrleaks): # This code is hackish and inelegant, but it seems to do the job. import copy_reg From python-checkins at python.org Fri Jun 9 21:43:26 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 21:43:26 +0200 (CEST) Subject: [Python-checkins] r46800 - python/trunk/Lib/wsgiref/simple_server.py Message-ID: <20060609194326.04F8A1E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 21:43:25 2006 New Revision: 46800 Modified: python/trunk/Lib/wsgiref/simple_server.py Log: Remove unused variable Modified: python/trunk/Lib/wsgiref/simple_server.py ============================================================================== --- python/trunk/Lib/wsgiref/simple_server.py (original) +++ python/trunk/Lib/wsgiref/simple_server.py Fri Jun 9 21:43:25 2006 @@ -184,7 +184,6 @@ if __name__ == '__main__': - server_address = ('', 8000) httpd = make_server('', 8000, demo_app) sa = httpd.socket.getsockname() print "Serving HTTP on", sa[0], "port", sa[1], "..." From python-checkins at python.org Fri Jun 9 21:56:05 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 9 Jun 2006 21:56:05 +0200 (CEST) Subject: [Python-checkins] r46801 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060609195605.926231E400A@bag.python.org> Author: andrew.kuchling Date: Fri Jun 9 21:56:05 2006 New Revision: 46801 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Add some wsgiref text Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Fri Jun 9 21:56:05 2006 @@ -2030,7 +2030,42 @@ %====================================================================== %\subsection{The wsgiref package\label{module-wsgiref}} -% XXX write this +% XXX should this be in a PEP 333 section instead? +\begin{comment} + +The Web Server Gateway Interface (WSGI) v1.0 defines a standard +interface between web servers and Python web applications and is +described in \pep{333}. The \module{wsgiref} package is a reference +implementation of the WSGI specification. + +The package includes a basic HTTP server that will run a WSGI +application; this server is useful for debugging but isn't intended for +production use. + +% XXX structure of WSGI applications? +% XXX provide an example using Django or some other framework? + +\begin{verbatim} +from wsgiref import simple_server + +wsgi_app = ... + +host = '' +port = 8000 +httpd = make_server(host, port, wsgi_app) +httpd.serve_forever() +\end{verbatim} + + +\begin{seealso} + +\seepep{333}{Python Web Server Gateway Interface v1.0}{PEP written by +Phillip J. Eby.} + +\end{seealso} + + +\end{comment} % ====================================================================== \section{Build and C API Changes\label{build-api}} From python-checkins at python.org Fri Jun 9 21:56:10 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Fri, 9 Jun 2006 21:56:10 +0200 (CEST) Subject: [Python-checkins] r46802 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060609195610.E73AC1E400C@bag.python.org> Author: mateusz.rukowicz Date: Fri Jun 9 21:56:10 2006 New Revision: 46802 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Added _fix and _rescale, more bugs fixed. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Fri Jun 9 21:56:10 2006 @@ -777,7 +777,7 @@ long i, last; decimalobject *tmp; assert(expdiff > 0); - tmp = _NEW_decimalobj(prec, self->sign, self->exp - expdiff); + tmp = _NEW_decimalobj(prec, self->sign, self->exp + expdiff); if (!tmp) return NULL; for (i = 0; i < prec; i++) tmp->digits[i] = self->digits[i]; @@ -803,7 +803,7 @@ decimalobject *tmp; long i, last; assert(expdiff > 0); - tmp = _NEW_decimalobj(prec, self->sign, self->exp - expdiff); + tmp = _NEW_decimalobj(prec, self->sign, self->exp + expdiff); if (!tmp) return NULL; for (i = 0; i < prec; i++) tmp->digits[i] = self->digits[i]; @@ -829,7 +829,7 @@ { decimalobject *tmp; long i; - tmp = _NEW_decimalobj(prec, self->sign, self->exp - expdiff); + tmp = _NEW_decimalobj(prec, self->sign, self->exp + expdiff); if (!tmp) return NULL; for (i = 0; i < prec; i++) tmp->digits[i] = self->digits[i]; @@ -939,7 +939,7 @@ return new; else if (expdiff < 0) { /* we need to extend precision */ - new2 = _NEW_decimalobj(prec, new->sign, new->exp - expdiff); + new2 = _NEW_decimalobj(prec, new->sign, new->exp + expdiff); if (!new2) { Py_DECREF(new); return NULL; @@ -964,7 +964,7 @@ } /* All lost digits are 0, so just clobber new */ new->ob_size = prec; - new->exp -= expdiff; + new->exp += expdiff; if (handle_Rounded(ctx, NULL) != 0) { Py_DECREF(new); return NULL; @@ -1058,7 +1058,7 @@ for (i = 0; i < self->ob_size; i++) ans->digits[i+1] = self->digits[i]; ans->digits[0] = 0; - _limb_first_n_digits(self->limbs, self->ob_size, -1, ans->limbs, ans->ob_size); + _limb_first_n_digits(self->limbs, self->ob_size, 0, ans->limbs, ans->ob_size); } tmp = _decimal_round(ans, digits, ctx, rounding); @@ -1070,7 +1070,8 @@ /* We need one digit less, just clobber tmp. */ for (i = 0; i < tmp->ob_size-1; i++) tmp->digits[i] = tmp->digits[i+1]; - tmp->ob_size--; + _limb_cut_one_digit(tmp->limbs, tmp->ob_size); + tmp->ob_size--; } tmp->exp = exp; @@ -1089,6 +1090,30 @@ return tmp; } +static PyObject * +decimal_rescale(decimalobject *self, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"exp", "rounding", "context", "watchexp", 0}; + contextobject *ctx = NULL; + long exp; + int rounding = -1, watchexp = 1; + + if(!PyArg_ParseTupleAndKeywords(args,kwds,"l|iOi:_rescale",kwlist, + &exp, &rounding, &ctx, &watchexp)) + return NULL; + + if(ctx == NULL) + if(!(ctx = getcontext())) + return NULL; + + if(!PyDecimalContext_Check(ctx)){ + PyErr_SetString(PyExc_TypeError, "context must be Context object"); + return NULL; + } + + return _decimal_rescale(self,exp,ctx,rounding,watchexp); +} + /* Fix the exponents and return a copy with the exponents in bounds. * Only call if known not to be a special value. */ static decimalobject * @@ -1184,7 +1209,7 @@ Py_DECREF(ans); if (!rounded) return NULL; - ans = _fixexponents(self, ctx); + ans = _fixexponents(rounded, ctx); Py_DECREF(rounded); if (!ans) return NULL; @@ -1192,6 +1217,22 @@ return ans; } +static PyObject * +decimal_fix(decimalobject *self, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"context",0}; + contextobject *ctx; + + if(!PyArg_ParseTupleAndKeywords(args, kwds, "O:_fix", kwlist, + &ctx)) + return NULL; + + if(ctx == Py_None) + if(!(ctx = getcontext())) + return NULL; + + return _decimal_fix(self,ctx); +} /* convert something to a Decimal. On failure, either returns NotImplemented or raises an exception, depending on the raise argument. @@ -1845,7 +1886,7 @@ } -static PyObject * /* TODO review that */ +static PyObject * _do_decimal_str(decimalobject *d, contextobject *context, int engineering) { char *outbuf; @@ -2059,6 +2100,12 @@ } static PyMethodDef decimal_methods[] = { + {"_rescale", (PyCFunction)decimal_rescale, + METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("asdf")}, + {"_fix", (PyCFunction)decimal_fix, + METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("asdf")}, {"adjusted", (PyCFunction)decimal_adjusted, METH_NOARGS, PyDoc_STR("Return the adjusted exponent of self.")}, From python-checkins at python.org Fri Jun 9 21:59:12 2006 From: python-checkins at python.org (thomas.heller) Date: Fri, 9 Jun 2006 21:59:12 +0200 (CEST) Subject: [Python-checkins] r46803 - python/trunk/Modules/_ctypes/libffi/install-sh Message-ID: <20060609195912.4B2EE1E400A@bag.python.org> Author: thomas.heller Date: Fri Jun 9 21:59:11 2006 New Revision: 46803 Modified: python/trunk/Modules/_ctypes/libffi/install-sh (props changed) Log: set eol-style svn property From python-checkins at python.org Fri Jun 9 22:01:02 2006 From: python-checkins at python.org (thomas.heller) Date: Fri, 9 Jun 2006 22:01:02 +0200 (CEST) Subject: [Python-checkins] r46804 - python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S Message-ID: <20060609200102.3D8671E401F@bag.python.org> Author: thomas.heller Date: Fri Jun 9 22:01:01 2006 New Revision: 46804 Modified: python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S (props changed) Log: set eol-style svn property From tim.peters at gmail.com Fri Jun 9 22:01:08 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 9 Jun 2006 16:01:08 -0400 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk In-Reply-To: References: <20060608202043.D5CEC1E4004@bag.python.org> <1f7befae0606081956u110433c0pda3179ba4bc35410@mail.gmail.com> <1f7befae0606082015l63a85567lb4d5eeb4791deb2b@mail.gmail.com> <1f7befae0606082042n7a178acdpb8bcfcd2de0aa787@mail.gmail.com> <1f7befae0606082121u913571es82cdcd8687bf3d3e@mail.gmail.com> Message-ID: <1f7befae0606091301u687dde3t5a3571ecd2ff2638@mail.gmail.com> [Georg Brandl] > Okay, so it's my fault. After an hour or so of investigation, obviously ;-) You're not being yelled at, though! This was subtle, and the same weird failures didn't happen before only because of accidental details in how test_file.py happened to be coded before. > Somehow I always believed that regrtest would remove the TESTFN by itself > since any test could raise an unexpected exception in the middle and not get > around to removing it itself. Excellent idea! I checked that in. It took far less time to do it than I previously wasted trying to figure out why the bsddb tests always left behind a bogus "db_home" directory on my box. From python-checkins at python.org Fri Jun 9 22:43:50 2006 From: python-checkins at python.org (georg.brandl) Date: Fri, 9 Jun 2006 22:43:50 +0200 (CEST) Subject: [Python-checkins] r46805 - in python/trunk/Lib: _MozillaCookieJar.py difflib.py idlelib/EditorWindow.py idlelib/configHandler.py idlelib/configHelpSourceEdit.py test/test_compiler.py test/test_inspect.py test/test_tcl.py trace.py warnings.py webbrowser.py Message-ID: <20060609204350.18EE11E400A@bag.python.org> Author: georg.brandl Date: Fri Jun 9 22:43:48 2006 New Revision: 46805 Modified: python/trunk/Lib/_MozillaCookieJar.py python/trunk/Lib/difflib.py python/trunk/Lib/idlelib/EditorWindow.py python/trunk/Lib/idlelib/configHandler.py python/trunk/Lib/idlelib/configHelpSourceEdit.py python/trunk/Lib/test/test_compiler.py python/trunk/Lib/test/test_inspect.py python/trunk/Lib/test/test_tcl.py python/trunk/Lib/trace.py python/trunk/Lib/warnings.py python/trunk/Lib/webbrowser.py Log: Make use of new str.startswith/endswith semantics. Occurences in email and compiler were ignored due to backwards compat requirements. Modified: python/trunk/Lib/_MozillaCookieJar.py ============================================================================== --- python/trunk/Lib/_MozillaCookieJar.py (original) +++ python/trunk/Lib/_MozillaCookieJar.py Fri Jun 9 22:43:48 2006 @@ -63,8 +63,7 @@ if line.endswith("\n"): line = line[:-1] # skip comments and blank lines XXX what is $ for? - if (line.strip().startswith("#") or - line.strip().startswith("$") or + if (line.strip().startswith(("#", "$")) or line.strip() == ""): continue Modified: python/trunk/Lib/difflib.py ============================================================================== --- python/trunk/Lib/difflib.py (original) +++ python/trunk/Lib/difflib.py Fri Jun 9 22:43:48 2006 @@ -1422,8 +1422,7 @@ num_blanks_pending -= 1 yield _make_line(lines,'-',0), None, True continue - elif s.startswith('--?+') or s.startswith('--+') or \ - s.startswith('- '): + elif s.startswith(('--?+', '--+', '- ')): # in delete block and see a intraline change or unchanged line # coming: yield the delete line and then blanks from_line,to_line = _make_line(lines,'-',0), None @@ -1447,7 +1446,7 @@ num_blanks_pending += 1 yield None, _make_line(lines,'+',1), True continue - elif s.startswith('+ ') or s.startswith('+-'): + elif s.startswith(('+ ', '+-')): # will be leaving an add block: yield blanks then add line from_line, to_line = None, _make_line(lines,'+',1) num_blanks_to_yield,num_blanks_pending = num_blanks_pending+1,0 Modified: python/trunk/Lib/idlelib/EditorWindow.py ============================================================================== --- python/trunk/Lib/idlelib/EditorWindow.py (original) +++ python/trunk/Lib/idlelib/EditorWindow.py Fri Jun 9 22:43:48 2006 @@ -649,7 +649,7 @@ def __extra_help_callback(self, helpfile): "Create a callback with the helpfile value frozen at definition time" def display_extra_help(helpfile=helpfile): - if not (helpfile.startswith('www') or helpfile.startswith('http')): + if not helpfile.startswith(('www', 'http')): url = os.path.normpath(helpfile) if sys.platform[:3] == 'win': os.startfile(helpfile) Modified: python/trunk/Lib/idlelib/configHandler.py ============================================================================== --- python/trunk/Lib/idlelib/configHandler.py (original) +++ python/trunk/Lib/idlelib/configHandler.py Fri Jun 9 22:43:48 2006 @@ -406,7 +406,7 @@ names=extnNameList kbNameIndicies=[] for name in names: - if name.endswith('_bindings') or name.endswith('_cfgBindings'): + if name.endswith(('_bindings', '_cfgBindings')): kbNameIndicies.append(names.index(name)) kbNameIndicies.sort() kbNameIndicies.reverse() Modified: python/trunk/Lib/idlelib/configHelpSourceEdit.py ============================================================================== --- python/trunk/Lib/idlelib/configHelpSourceEdit.py (original) +++ python/trunk/Lib/idlelib/configHelpSourceEdit.py Fri Jun 9 22:43:48 2006 @@ -127,7 +127,7 @@ parent=self) self.entryPath.focus_set() pathOk = False - elif path.startswith('www.') or path.startswith('http'): + elif path.startswith(('www.', 'http')): pass else: if path[:5] == 'file:': @@ -146,8 +146,7 @@ self.path.get().strip()) if sys.platform == 'darwin': path = self.result[1] - if (path.startswith('www') or path.startswith('file:') - or path.startswith('http:')): + if path.startswith(('www', 'file:', 'http:')): pass else: # Mac Safari insists on using the URI form for local files Modified: python/trunk/Lib/test/test_compiler.py ============================================================================== --- python/trunk/Lib/test/test_compiler.py (original) +++ python/trunk/Lib/test/test_compiler.py Fri Jun 9 22:43:48 2006 @@ -62,7 +62,7 @@ def testLineNo(self): # Test that all nodes except Module have a correct lineno attribute. filename = __file__ - if filename.endswith(".pyc") or filename.endswith(".pyo"): + if filename.endswith((".pyc", ".pyo")): filename = filename[:-1] tree = compiler.parseFile(filename) self.check_lineno(tree) Modified: python/trunk/Lib/test/test_inspect.py ============================================================================== --- python/trunk/Lib/test/test_inspect.py (original) +++ python/trunk/Lib/test/test_inspect.py Fri Jun 9 22:43:48 2006 @@ -15,7 +15,7 @@ # isdatadescriptor modfile = mod.__file__ -if modfile.endswith('c') or modfile.endswith('o'): +if modfile.endswith(('c', 'o')): modfile = modfile[:-1] import __builtin__ Modified: python/trunk/Lib/test/test_tcl.py ============================================================================== --- python/trunk/Lib/test/test_tcl.py (original) +++ python/trunk/Lib/test/test_tcl.py Fri Jun 9 22:43:48 2006 @@ -130,10 +130,8 @@ import os old_display = None import sys - if (sys.platform.startswith('win') or - sys.platform.startswith('darwin') or - sys.platform.startswith('cygwin')): - return # no failure possible on windows? + if sys.platform.startswith(('win', 'darwin', 'cygwin')): + return # no failure possible on windows? if 'DISPLAY' in os.environ: old_display = os.environ['DISPLAY'] del os.environ['DISPLAY'] Modified: python/trunk/Lib/trace.py ============================================================================== --- python/trunk/Lib/trace.py (original) +++ python/trunk/Lib/trace.py Fri Jun 9 22:43:48 2006 @@ -285,7 +285,7 @@ if filename == "": continue - if filename.endswith(".pyc") or filename.endswith(".pyo"): + if filename.endswith((".pyc", ".pyo")): filename = filename[:-1] if coverdir is None: Modified: python/trunk/Lib/warnings.py ============================================================================== --- python/trunk/Lib/warnings.py (original) +++ python/trunk/Lib/warnings.py Fri Jun 9 22:43:48 2006 @@ -46,7 +46,7 @@ filename = globals.get('__file__') if filename: fnl = filename.lower() - if fnl.endswith(".pyc") or fnl.endswith(".pyo"): + if fnl.endswith((".pyc", ".pyo")): filename = filename[:-1] else: if module == "__main__": Modified: python/trunk/Lib/webbrowser.py ============================================================================== --- python/trunk/Lib/webbrowser.py (original) +++ python/trunk/Lib/webbrowser.py Fri Jun 9 22:43:48 2006 @@ -98,8 +98,7 @@ if sys.platform[:3] == "win": def _isexecutable(cmd): cmd = cmd.lower() - if os.path.isfile(cmd) and (cmd.endswith(".exe") or - cmd.endswith(".bat")): + if os.path.isfile(cmd) and cmd.endswith((".exe", ".bat")): return True for ext in ".exe", ".bat": if os.path.isfile(cmd + ext): From buildbot at python.org Fri Jun 9 23:07:39 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 21:07:39 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060609210739.4741D1E4017@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1092 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 10 00:00:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 22:00:02 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060609220002.CDE3C1E400A@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/318 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 00:31:24 2006 From: python-checkins at python.org (brett.cannon) Date: Sat, 10 Jun 2006 00:31:24 +0200 (CEST) Subject: [Python-checkins] r46806 - in python/trunk: Lib/test/crashers/infinite_rec_3.py Lib/test/test_descr.py Misc/NEWS Objects/abstract.c Message-ID: <20060609223124.4668F1E400D@bag.python.org> Author: brett.cannon Date: Sat Jun 10 00:31:23 2006 New Revision: 46806 Removed: python/trunk/Lib/test/crashers/infinite_rec_3.py Modified: python/trunk/Lib/test/test_descr.py python/trunk/Misc/NEWS python/trunk/Objects/abstract.c Log: An object with __call__ as an attribute, when called, will have that attribute checked for __call__ itself, and will continue to look until it finds an object without the attribute. This can lead to an infinite recursion. Closes bug #532646, again. Will be backported. Deleted: /python/trunk/Lib/test/crashers/infinite_rec_3.py ============================================================================== --- /python/trunk/Lib/test/crashers/infinite_rec_3.py Sat Jun 10 00:31:23 2006 +++ (empty file) @@ -1,9 +0,0 @@ - -# http://python.org/sf/1202533 - -class A(object): - pass -A.__call__ = A() - -if __name__ == '__main__': - A()() # segfault: infinite recursion in C Modified: python/trunk/Lib/test/test_descr.py ============================================================================== --- python/trunk/Lib/test/test_descr.py (original) +++ python/trunk/Lib/test/test_descr.py Sat Jun 10 00:31:23 2006 @@ -3171,6 +3171,21 @@ list.__init__(a, sequence=[0, 1, 2]) vereq(a, [0, 1, 2]) +def recursive__call__(): + if verbose: print ("Testing recursive __call__() by setting to instance of " + "class ...") + class A(object): + pass + + A.__call__ = A() + try: + A()() + except RuntimeError: + pass + else: + raise TestFailed("Recursion limit should have been reached for " + "__call__()") + def delhook(): if verbose: print "Testing __del__ hook..." log = [] @@ -4164,6 +4179,7 @@ buffer_inherit() str_of_str_subclass() kwdargs() + recursive__call__() delhook() hashinherit() strops() Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 10 00:31:23 2006 @@ -12,6 +12,12 @@ Core and builtins ----------------- +- Bug #532646: object.__call__() will continue looking for the __call__ + attribute on objects until one without one is found. This leads to recursion + when you take a class and set its __call__ attribute to an instance of the + class. Originally fixed for classic classes, but this fix is for new-style. + Removes the infinite_rec_3 crasher. + - The string and unicode methods startswith() and endswith() now accept a tuple of prefixes/suffixes to look for. Implements RFE #1491485. Modified: python/trunk/Objects/abstract.c ============================================================================== --- python/trunk/Objects/abstract.c (original) +++ python/trunk/Objects/abstract.c Sat Jun 10 00:31:23 2006 @@ -1790,7 +1790,15 @@ ternaryfunc call; if ((call = func->ob_type->tp_call) != NULL) { + /* slot_tp_call() will be called and ends up calling + PyObject_Call() if the object returned for __call__ has + __call__ itself defined upon it. This can be an infinite + recursion if you set __call__ in a class to an instance of + it. */ + if (Py_EnterRecursiveCall(" in __call__")) + return NULL; PyObject *result = (*call)(func, arg, kw); + Py_LeaveRecursiveCall(); if (result == NULL && !PyErr_Occurred()) PyErr_SetString( PyExc_SystemError, From buildbot at python.org Sat Jun 10 00:38:21 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 22:38:21 +0000 Subject: [Python-checkins] buildbot failure in x86 W2k trunk Message-ID: <20060609223821.C52FB1E400A@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/1013 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 00:38:30 2006 From: python-checkins at python.org (brett.cannon) Date: Sat, 10 Jun 2006 00:38:30 +0200 (CEST) Subject: [Python-checkins] r46807 - in python/branches/release24-maint: Lib/test/test_descr.py Misc/NEWS Objects/abstract.c Message-ID: <20060609223830.60D931E400A@bag.python.org> Author: brett.cannon Date: Sat Jun 10 00:38:29 2006 New Revision: 46807 Modified: python/branches/release24-maint/Lib/test/test_descr.py python/branches/release24-maint/Misc/NEWS python/branches/release24-maint/Objects/abstract.c Log: Backport of fix of bug #532646 for new-style classes. Modified: python/branches/release24-maint/Lib/test/test_descr.py ============================================================================== --- python/branches/release24-maint/Lib/test/test_descr.py (original) +++ python/branches/release24-maint/Lib/test/test_descr.py Sat Jun 10 00:38:29 2006 @@ -3116,6 +3116,21 @@ list.__init__(a, sequence=[0, 1, 2]) vereq(a, [0, 1, 2]) +def recursive__call__(): + if verbose: print ("Testing recursive __call__() by setting to instance of " + "class ...") + class A(object): + pass + + A.__call__ = A() + try: + A()() + except RuntimeError: + pass + else: + raise TestFailed("Recursion limit should have been reached for " + "__call__()") + def delhook(): if verbose: print "Testing __del__ hook..." log = [] @@ -4118,6 +4133,7 @@ buffer_inherit() str_of_str_subclass() kwdargs() + recursive__call__() delhook() hashinherit() strops() Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Sat Jun 10 00:38:29 2006 @@ -12,6 +12,10 @@ Core and builtins ----------------- +- Bug #532646: The object set to the __call__ attribute has its own __call__ + attribute checked; this continues until the attribute can no longer be found + or segfaulting. Recursion limit is now followed. + - Bug #1454485: Don't crash on Unicode characters <0. - Patch #1488312, Fix memory alignment problem on SPARC in unicode Modified: python/branches/release24-maint/Objects/abstract.c ============================================================================== --- python/branches/release24-maint/Objects/abstract.c (original) +++ python/branches/release24-maint/Objects/abstract.c Sat Jun 10 00:38:29 2006 @@ -1792,7 +1792,10 @@ ternaryfunc call; if ((call = func->ob_type->tp_call) != NULL) { + if (Py_EnterRecursiveCall(" in __call__")) + return NULL; PyObject *result = (*call)(func, arg, kw); + Py_LeaveRecursiveCall(); if (result == NULL && !PyErr_Occurred()) PyErr_SetString( PyExc_SystemError, From buildbot at python.org Sat Jun 10 00:39:03 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 22:39:03 +0000 Subject: [Python-checkins] buildbot failure in x86 XP trunk Message-ID: <20060609223903.22D811E400A@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/991 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon BUILD FAILED: failed compile sincerely, -The Buildbot From buildbot at python.org Sat Jun 10 00:40:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 22:40:55 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060609224055.670711E4010@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/618 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 10 00:43:40 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 22:43:40 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP-2 trunk Message-ID: <20060609224340.D07671E400A@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-2%2520trunk/builds/601 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Sat Jun 10 00:45:03 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 22:45:03 +0000 Subject: [Python-checkins] buildbot warnings in x86 W2k 2.4 Message-ID: <20060609224503.EBA741E400A@bag.python.org> The Buildbot has detected a new failure of x86 W2k 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%25202.4/builds/148 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 00:45:54 2006 From: python-checkins at python.org (brett.cannon) Date: Sat, 10 Jun 2006 00:45:54 +0200 (CEST) Subject: [Python-checkins] r46808 - python/trunk/Objects/abstract.c Message-ID: <20060609224554.7F8D41E4014@bag.python.org> Author: brett.cannon Date: Sat Jun 10 00:45:54 2006 New Revision: 46808 Modified: python/trunk/Objects/abstract.c Log: Fix bug introduced in rev. 46806 by not having variable declaration at the top of a block. Modified: python/trunk/Objects/abstract.c ============================================================================== --- python/trunk/Objects/abstract.c (original) +++ python/trunk/Objects/abstract.c Sat Jun 10 00:45:54 2006 @@ -1790,14 +1790,16 @@ ternaryfunc call; if ((call = func->ob_type->tp_call) != NULL) { + PyObject *result = NULL; /* slot_tp_call() will be called and ends up calling PyObject_Call() if the object returned for __call__ has __call__ itself defined upon it. This can be an infinite recursion if you set __call__ in a class to an instance of it. */ - if (Py_EnterRecursiveCall(" in __call__")) + if (Py_EnterRecursiveCall(" in __call__")) { return NULL; - PyObject *result = (*call)(func, arg, kw); + } + result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (result == NULL && !PyErr_Occurred()) PyErr_SetString( From buildbot at python.org Sat Jun 10 00:46:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 09 Jun 2006 22:46:14 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP 2.4 Message-ID: <20060609224614.DFB8C1E400A@bag.python.org> The Buildbot has detected a new failure of x86 XP 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%25202.4/builds/153 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 00:49:31 2006 From: python-checkins at python.org (brett.cannon) Date: Sat, 10 Jun 2006 00:49:31 +0200 (CEST) Subject: [Python-checkins] r46809 - python/branches/release24-maint/Objects/abstract.c Message-ID: <20060609224931.B77EC1E400A@bag.python.org> Author: brett.cannon Date: Sat Jun 10 00:49:31 2006 New Revision: 46809 Modified: python/branches/release24-maint/Objects/abstract.c Log: Fix bug introduced in rev. 46807 where variable was not declared at top of block. Modified: python/branches/release24-maint/Objects/abstract.c ============================================================================== --- python/branches/release24-maint/Objects/abstract.c (original) +++ python/branches/release24-maint/Objects/abstract.c Sat Jun 10 00:49:31 2006 @@ -1792,9 +1792,10 @@ ternaryfunc call; if ((call = func->ob_type->tp_call) != NULL) { + PyObject *result = NULL; if (Py_EnterRecursiveCall(" in __call__")) return NULL; - PyObject *result = (*call)(func, arg, kw); + result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (result == NULL && !PyErr_Occurred()) PyErr_SetString( From python-checkins at python.org Sat Jun 10 03:14:26 2006 From: python-checkins at python.org (matt.fleming) Date: Sat, 10 Jun 2006 03:14:26 +0200 (CEST) Subject: [Python-checkins] r46810 - in sandbox/trunk/pdb: README.txt mconnection.py mpdb.py test test/tcptest.py Message-ID: <20060610011426.C5C361E400A@bag.python.org> Author: matt.fleming Date: Sat Jun 10 03:14:26 2006 New Revision: 46810 Added: sandbox/trunk/pdb/test/ sandbox/trunk/pdb/test/tcptest.py Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Log: Added a unit test for the connection library. Worked a 'generic' output method into the MPdb class and changed the abstract classes for both the MClientConnectionInterface and MServerConnectionInterface so that they must implemented a read() and write() method on the connection. Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Sat Jun 10 03:14:26 2006 @@ -9,7 +9,7 @@ aims to correct this wish. -=[TODO]=- -* Write unittests +* Write more unit tests * sort out the namespace corruption - """ c:\soc\pdb\test.py(3)x() Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Sat Jun 10 03:14:26 2006 @@ -35,6 +35,18 @@ """ raise NotImplementedError, NotImplementedMessage + def write(self, msg): + """ This method is used to write to a debugger that is + connected to this server. + """ + raise NotImplementedError, NotImplementedMessage + + def read(self, bufsize=1024): + """ This method is reads a maximum of 'bufsize' bytes from a + connected debugger. + """ + raise NotImplementedError, NotImplementedMessage + class MClientConnectionInterface(object): """ This is the interface that a client connection should implement. """ @@ -42,6 +54,20 @@ """ This method is called by a debugger to connect to a server. """ raise NotImplementedError, NotImplementedMessage + + def write(self, msg): + """ This method is called by a debugger to write 'msg' to the + server it is connected to. + """ + raise NotImplementedError, NotImplementedMessage + + + def read(self, bufsize=1024): + """ This method reads a maximum of 'bufsize' bytes from the connection + to the server. + """ + raise NotImplementedError, NotImplementedMessage + def disconnect(self): """ This method is called by a debugger to disconnect from a server. @@ -67,6 +93,12 @@ def accept(self, debugger, addr): pass + def write(self, msg): + self.output.write(msg) + + def read(self, bufsize=1024): + return self.input.read() + def disconnect(self): self.output.close() self.input.close() @@ -94,15 +126,24 @@ self.listen() def listen(self): - self._sock.listen(5) + """ This method is not usually called except by this object's + setup() method. + """ + self._sock.listen(1) debugger, addr = self._sock.accept() self.accept(debugger, addr) def accept(self, debugger, addr): - self.output = debugger.makefile('w') + self.output = debugger + self.input = debugger + + def write(self, msg): + self.output.sendall(msg) + def read(self, bufsize=1024): + return self.input.recv(bufsize) + def disconnect(self): - self.output.flush() self.output.close() self._sock.close() @@ -117,7 +158,6 @@ self._dev = device self.input = None self.output = None - self.setup() def setup(self): """ Create our fileobject by opening the serial device for @@ -140,15 +180,20 @@ h, p = addr.split(':') self.host = h self.port = int(p) - self.setup() def setup(self): - """ Connect to the server. """ + """ Connect to the server. 'input' reads data from the + server. 'output' writes data to the server. + """ self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self._sock.connect((self.host, self.port)) - self.output = self._sock.makefile('w') + def write(self, msg): + self._sock.send(msg) + + def read(self, bufsize=1024): + return self._sock.recv(bufsize) + def disconnect(self): """ Close the socket to the server. """ - self.output.close() self._sock.close() Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Sat Jun 10 03:14:26 2006 @@ -4,7 +4,8 @@ # Student: Matthew J. Fleming # Mentor: Robert L. Bernstein """ -This module provides improvements over the Python Debugger (Pdb). +This module provides improvements over the Python Debugger (Pdb) by building +on the work done by Rocky Bernstein in The Extended Python Debugger. This module allows, - debugging of applications running in a separate process to the debugger @@ -15,16 +16,14 @@ import os from optparse import OptionParser -import pdb +import pydb import socket import sys import traceback -# Need custom safe Repr just like pdb -_saferepr = pdb._repr.repr line_prefix = '\n-> ' -class MPdb(pdb.Pdb): +class MPdb(pydb.Pdb): """ This class extends the command set and functionality of the Python debugger and provides support for, @@ -42,17 +41,15 @@ specify alternate input and output file objects; if not specified, sys.stdin and sys.stdout are used. """ - pdb.Pdb.__init__(self, completekey, stdin, stdout) + pydb.Pdb.__init__(self, completekey, stdin, stdout) self.prompt = '(MPdb)' def _rebind_input(self, new_input): """ This method rebinds the debugger's input to the object specified by 'new_input'. """ - self.raw_input = 1 self.stdin.flush() self.stdin = new_input - self.stdin.flush() def _rebind_output(self, new_output): """ This method rebinds the debugger's output to the object specified @@ -60,7 +57,21 @@ """ self.stdout.flush() self.stdout = new_output - self.stdout.flush() + + def msg_nocr(self, msg, out=None): + """Common routine for reporting messages (no carriage return). + Derived classed may want to override this to capture output. + """ + do_print = True + if self.logging: + if self.logging_fileobj is not None: + print >> self.logging_fileobj, msg, + do_print = not self.logging_redirect + if do_print: + if out is None: + out = self.stdout + print >> out, msg, + out.flush() # Debugger commands def do_attach(self, addr): @@ -88,9 +99,7 @@ List of target subcommands: target serial -- Use a remote computer via a serial line -target tcp -- Use a remote computer via a TCP connection -target udp -- Use a remote computer via a UDP connection -target xml -- Use a remote computer via the xmlrpc lib +target socket -- Use a remote computer via a socket connection """ cls, addr = args.split(' ') if '.' in cls: @@ -99,7 +108,7 @@ exec 'from ' + base + ' import ' + cls else: __import__(cls) - self.connection = eval(mod+'(addr)') + self.connection = eval(cls+'(addr)') self.connection.setup() # XXX currently this method doesn't do anything Added: sandbox/trunk/pdb/test/tcptest.py ============================================================================== --- (empty file) +++ sandbox/trunk/pdb/test/tcptest.py Sat Jun 10 03:14:26 2006 @@ -0,0 +1,41 @@ +#!/usr/bin/env python +import os +import sys +import socket +import time +import unittest + +__addr__ = 'localhost:8000' + +sys.path.append("..") +from mconnection import MServerConnectionTCP, MClientConnectionTCP + +class TestTCPConnections(unittest.TestCase): + def testClientConnectAndRead(self): + # We need to exercise finer control over the server's socket so we're + # not using the setup() method. + self.client = MClientConnectionTCP(__addr__) + self.server = MServerConnectionTCP(__addr__) + self.server._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.server._sock.bind((self.server.host, self.server.port)) + self.server._sock.listen(1) + pid = os.fork() + for i in range(30): + try: + self.client.setup() + break + except socket.error: + pass + time.sleep(5) + debugger, addr = self.server._sock.accept() + self.server.accept(debugger, addr) + self.server.write("good") + line = self.client.read() + self.assertEqual("good", line, "Could not read from server") + self.client.disconnect() + self.server.disconnect() + + + +if __name__ == '__main__': + unittest.main() From python-checkins at python.org Sat Jun 10 03:41:25 2006 From: python-checkins at python.org (brett.cannon) Date: Sat, 10 Jun 2006 03:41:25 +0200 (CEST) Subject: [Python-checkins] r46811 - peps/trunk/pep-0291.txt peps/trunk/pep-0360.txt Message-ID: <20060610014125.9DEB01E400A@bag.python.org> Author: brett.cannon Date: Sat Jun 10 03:41:25 2006 New Revision: 46811 Modified: peps/trunk/pep-0291.txt peps/trunk/pep-0360.txt Log: Add wsgiref information. Modified: peps/trunk/pep-0291.txt ============================================================================== --- peps/trunk/pep-0291.txt (original) +++ peps/trunk/pep-0291.txt Sat Jun 10 03:41:25 2006 @@ -98,6 +98,7 @@ pybench Marc-Andre Lemburg 1.5.2 [3] sre Fredrik Lundh 2.1 subprocess Peter Astrand 2.2 + wsgiref Phillip J. Eby 2.1 xml (PyXML) Martin v. Loewis 2.0 xmlrpclib Fredrik Lundh 2.1 Modified: peps/trunk/pep-0360.txt ============================================================================== --- peps/trunk/pep-0360.txt (original) +++ peps/trunk/pep-0360.txt Sat Jun 10 03:41:25 2006 @@ -124,6 +124,21 @@ deemed critical. +wsgiref +------- +:Web site: + None +:Standard library name: + wsgiref +:Contact Person: + Phillip J. Eby +:Synchronisation history: + * 0.1 (2.5) + +Bugs and patches should pass through the Web-SIG mailing list [#web-sig]_ +before being applied to HEAD. + + References ========== @@ -136,6 +151,9 @@ .. [#pysqlite-tracker] pysqlite tracker (http://pysqlite.org/) +.. [#web-sig] Web-SIG mailing list + (http://mail.python.org/mailman/listinfo/web-sig) + Copyright ========= From buildbot at python.org Sat Jun 10 05:54:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 03:54:20 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060610035420.F23531E400A@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/52 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 08:40:52 2006 From: python-checkins at python.org (georg.brandl) Date: Sat, 10 Jun 2006 08:40:52 +0200 (CEST) Subject: [Python-checkins] r46812 - in python/trunk: Lib/test/string_tests.py Objects/stringobject.c Objects/unicodeobject.c Message-ID: <20060610064052.387451E400C@bag.python.org> Author: georg.brandl Date: Sat Jun 10 08:40:50 2006 New Revision: 46812 Modified: python/trunk/Lib/test/string_tests.py python/trunk/Objects/stringobject.c python/trunk/Objects/unicodeobject.c Log: Apply perky's fix for #1503157: "/".join([u"", u""]) raising OverflowError. Also improve error message on overflow. Modified: python/trunk/Lib/test/string_tests.py ============================================================================== --- python/trunk/Lib/test/string_tests.py (original) +++ python/trunk/Lib/test/string_tests.py Sat Jun 10 08:40:50 2006 @@ -938,6 +938,8 @@ # test.test_string.StringTest.test_join) self.checkequal('a b c d', ' ', 'join', ['a', 'b', 'c', 'd']) self.checkequal('abcd', '', 'join', ('a', 'b', 'c', 'd')) + self.checkequal('bd', '', 'join', ('', 'b', '', 'd')) + self.checkequal('ac', '', 'join', ('a', '', 'c', '')) self.checkequal('w x y z', ' ', 'join', Sequence()) self.checkequal('abc', 'a', 'join', ('abc',)) self.checkequal('z', 'a', 'join', UserList(['z'])) Modified: python/trunk/Objects/stringobject.c ============================================================================== --- python/trunk/Objects/stringobject.c (original) +++ python/trunk/Objects/stringobject.c Sat Jun 10 08:40:50 2006 @@ -1788,7 +1788,7 @@ sz += seplen; if (sz < old_sz || sz > PY_SSIZE_T_MAX) { PyErr_SetString(PyExc_OverflowError, - "join() is too long for a Python string"); + "join() result is too long for a Python string"); Py_DECREF(seq); return NULL; } Modified: python/trunk/Objects/unicodeobject.c ============================================================================== --- python/trunk/Objects/unicodeobject.c (original) +++ python/trunk/Objects/unicodeobject.c Sat Jun 10 08:40:50 2006 @@ -4491,11 +4491,11 @@ /* Make sure we have enough space for the separator and the item. */ itemlen = PyUnicode_GET_SIZE(item); new_res_used = res_used + itemlen; - if (new_res_used <= 0) + if (new_res_used < 0) goto Overflow; if (i < seqlen - 1) { new_res_used += seplen; - if (new_res_used <= 0) + if (new_res_used < 0) goto Overflow; } if (new_res_used > res_alloc) { @@ -4536,7 +4536,7 @@ Overflow: PyErr_SetString(PyExc_OverflowError, - "join() is too long for a Python string"); + "join() result is too long for a Python string"); Py_DECREF(item); /* fall through */ From python-checkins at python.org Sat Jun 10 08:44:51 2006 From: python-checkins at python.org (georg.brandl) Date: Sat, 10 Jun 2006 08:44:51 +0200 (CEST) Subject: [Python-checkins] r46813 - python/branches/release24-maint/Lib/test/string_tests.py Message-ID: <20060610064451.60F981E400C@bag.python.org> Author: georg.brandl Date: Sat Jun 10 08:44:50 2006 New Revision: 46813 Modified: python/branches/release24-maint/Lib/test/string_tests.py Log: Add the new test from the #1503157 patch to the branch. Modified: python/branches/release24-maint/Lib/test/string_tests.py ============================================================================== --- python/branches/release24-maint/Lib/test/string_tests.py (original) +++ python/branches/release24-maint/Lib/test/string_tests.py Sat Jun 10 08:44:50 2006 @@ -584,6 +584,8 @@ # test.test_string.StringTest.test_join) self.checkequal('a b c d', ' ', 'join', ['a', 'b', 'c', 'd']) self.checkequal('abcd', '', 'join', ('a', 'b', 'c', 'd')) + self.checkequal('bd', '', 'join', ('', 'b', '', 'd')) + self.checkequal('ac', '', 'join', ('a', '', 'c', '')) self.checkequal('w x y z', ' ', 'join', Sequence()) self.checkequal('abc', 'a', 'join', ('abc',)) self.checkequal('z', 'a', 'join', UserList(['z'])) From buildbot at python.org Sat Jun 10 08:48:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 06:48:36 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060610064836.AA5DB1E400C@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/382 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 10 08:50:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 06:50:02 +0000 Subject: [Python-checkins] buildbot failure in amd64 gentoo trunk Message-ID: <20060610065002.EF4AF1E400C@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/998 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From buildbot at python.org Sat Jun 10 09:23:10 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 07:23:10 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060610072310.6B76D1E400C@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/620 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 09:46:49 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 10 Jun 2006 09:46:49 +0200 (CEST) Subject: [Python-checkins] r46814 - python/branches/release24-maint/Tools/buildbot/Makefile Message-ID: <20060610074649.B42B11E400D@bag.python.org> Author: martin.v.loewis Date: Sat Jun 10 09:46:49 2006 New Revision: 46814 Added: python/branches/release24-maint/Tools/buildbot/Makefile (contents, props changed) Log: Add cygwin makefile. Added: python/branches/release24-maint/Tools/buildbot/Makefile ============================================================================== --- (empty file) +++ python/branches/release24-maint/Tools/buildbot/Makefile Sat Jun 10 09:46:49 2006 @@ -0,0 +1,6 @@ +all: kill_python.exe + ./kill_python.exe + +kill_python.exe: kill_python.c + gcc -o kill_python.exe kill_python.c -lpsapi + From python-checkins at python.org Sat Jun 10 10:01:43 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 10 Jun 2006 10:01:43 +0200 (CEST) Subject: [Python-checkins] r46815 - python/branches/release24-maint/Tools/buildbot/kill_python.c Message-ID: <20060610080143.4F2211E400C@bag.python.org> Author: martin.v.loewis Date: Sat Jun 10 10:01:42 2006 New Revision: 46815 Added: python/branches/release24-maint/Tools/buildbot/kill_python.c (contents, props changed) Log: Add missing implementation file. Added: python/branches/release24-maint/Tools/buildbot/kill_python.c ============================================================================== --- (empty file) +++ python/branches/release24-maint/Tools/buildbot/kill_python.c Sat Jun 10 10:01:42 2006 @@ -0,0 +1,56 @@ +/* This program looks for processes which have build\PCbuild\python.exe + in their path and terminates them. */ +#include +#include +#include + +int main() +{ + DWORD pids[1024], cbNeeded; + int i, num_processes; + if (!EnumProcesses(pids, sizeof(pids), &cbNeeded)) { + printf("EnumProcesses failed\n"); + return 1; + } + num_processes = cbNeeded/sizeof(pids[0]); + for (i = 0; i < num_processes; i++) { + HANDLE hProcess; + char path[MAX_PATH]; + HMODULE mods[1024]; + int k, num_mods; + hProcess = OpenProcess(PROCESS_QUERY_INFORMATION + | PROCESS_VM_READ + | PROCESS_TERMINATE , + FALSE, pids[i]); + if (!hProcess) + /* process not accessible */ + continue; + if (!EnumProcessModules(hProcess, mods, sizeof(mods), &cbNeeded)) { + /* For unknown reasons, this sometimes returns ERROR_PARTIAL_COPY; + this apparently means we are not supposed to read the process. */ + if (GetLastError() == ERROR_PARTIAL_COPY) { + CloseHandle(hProcess); + continue; + } + printf("EnumProcessModules failed: %d\n", GetLastError()); + return 1; + } + if (!GetModuleFileNameEx(hProcess, NULL, path, sizeof(path))) { + printf("GetProcessImageFileName failed\n"); + return 1; + } + + _strlwr(path); + /* printf("%s\n", path); */ + if (strstr(path, "build\\pcbuild\\python_d.exe") != NULL) { + printf("Terminating %s (pid %d)\n", path, pids[i]); + if (!TerminateProcess(hProcess, 1)) { + printf("Termination failed: %d\n", GetLastError()); + return 1; + } + return 0; + } + + CloseHandle(hProcess); + } +} From buildbot at python.org Sat Jun 10 10:02:00 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 08:02:00 +0000 Subject: [Python-checkins] buildbot failure in x86 cygwin 2.4 Message-ID: <20060610080201.1C89A1E400C@bag.python.org> The Buildbot has detected a new failure of x86 cygwin 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520cygwin%25202.4/builds/105 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'Martin von Loewis': Test kill-python Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: BUILD FAILED: failed kill.python configure sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 10:07:25 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 10 Jun 2006 10:07:25 +0200 (CEST) Subject: [Python-checkins] r46816 - python/branches/release24-maint/Tools/buildbot/kill_python.c Message-ID: <20060610080725.9D41E1E400C@bag.python.org> Author: martin.v.loewis Date: Sat Jun 10 10:07:25 2006 New Revision: 46816 Modified: python/branches/release24-maint/Tools/buildbot/kill_python.c Log: Add build\\python.exe to the list of programs to be killed. Modified: python/branches/release24-maint/Tools/buildbot/kill_python.c ============================================================================== --- python/branches/release24-maint/Tools/buildbot/kill_python.c (original) +++ python/branches/release24-maint/Tools/buildbot/kill_python.c Sat Jun 10 10:07:25 2006 @@ -42,7 +42,8 @@ _strlwr(path); /* printf("%s\n", path); */ - if (strstr(path, "build\\pcbuild\\python_d.exe") != NULL) { + if ((strstr(path, "build\\pcbuild\\python_d.exe") != NULL) || + (strstr(path, "build\\python.exe") != NULL)) { printf("Terminating %s (pid %d)\n", path, pids[i]); if (!TerminateProcess(hProcess, 1)) { printf("Termination failed: %d\n", GetLastError()); From python-checkins at python.org Sat Jun 10 10:14:03 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 10 Jun 2006 10:14:03 +0200 (CEST) Subject: [Python-checkins] r46817 - python/trunk/Tools/buildbot/Makefile python/trunk/Tools/buildbot/kill_python.c Message-ID: <20060610081403.90C8F1E400C@bag.python.org> Author: martin.v.loewis Date: Sat Jun 10 10:14:03 2006 New Revision: 46817 Added: python/trunk/Tools/buildbot/Makefile (contents, props changed) Modified: python/trunk/Tools/buildbot/kill_python.c Log: Port cygwin kill_python changes from 2.4 branch. Added: python/trunk/Tools/buildbot/Makefile ============================================================================== --- (empty file) +++ python/trunk/Tools/buildbot/Makefile Sat Jun 10 10:14:03 2006 @@ -0,0 +1,6 @@ +all: kill_python.exe + ./kill_python.exe + +kill_python.exe: kill_python.c + gcc -o kill_python.exe kill_python.c -lpsapi + Modified: python/trunk/Tools/buildbot/kill_python.c ============================================================================== --- python/trunk/Tools/buildbot/kill_python.c (original) +++ python/trunk/Tools/buildbot/kill_python.c Sat Jun 10 10:14:03 2006 @@ -42,7 +42,8 @@ _strlwr(path); /* printf("%s\n", path); */ - if (strstr(path, "build\\pcbuild\\python_d.exe") != NULL) { + if ((strstr(path, "build\\pcbuild\\python_d.exe") != NULL) || + (strstr(path, "build\\python.exe") != NULL)) { printf("Terminating %s (pid %d)\n", path, pids[i]); if (!TerminateProcess(hProcess, 1)) { printf("Termination failed: %d\n", GetLastError()); From buildbot at python.org Sat Jun 10 10:39:58 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 08:39:58 +0000 Subject: [Python-checkins] buildbot warnings in x86 cygwin trunk Message-ID: <20060610083958.79E6D1E400C@bag.python.org> The Buildbot has detected a new failure of x86 cygwin trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520cygwin%2520trunk/builds/764 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'rebuild' button was pressed by '': Build Source Stamp: [branch trunk] HEAD Blamelist: Build Had Warnings: warnings kill.python test sincerely, -The Buildbot From buildbot at python.org Sat Jun 10 11:23:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 09:23:14 +0000 Subject: [Python-checkins] buildbot warnings in x86 cygwin 2.4 Message-ID: <20060610092314.45F421E400C@bag.python.org> The Buildbot has detected a new failure of x86 cygwin 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520cygwin%25202.4/builds/109 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 12:57:40 2006 From: python-checkins at python.org (armin.rigo) Date: Sat, 10 Jun 2006 12:57:40 +0200 (CEST) Subject: [Python-checkins] r46818 - python/trunk/Objects/dictobject.c Message-ID: <20060610105740.EBF081E401C@bag.python.org> Author: armin.rigo Date: Sat Jun 10 12:57:40 2006 New Revision: 46818 Modified: python/trunk/Objects/dictobject.c Log: SF bug #1503294. PyThreadState_GET() complains if the tstate is NULL, but only in debug mode. Modified: python/trunk/Objects/dictobject.c ============================================================================== --- python/trunk/Objects/dictobject.c (original) +++ python/trunk/Objects/dictobject.c Sat Jun 10 12:57:40 2006 @@ -561,7 +561,7 @@ /* We can arrive here with a NULL tstate during initialization: try running "python -Wi" for an example related to string interning. Let's just hope that no exception occurs then... */ - tstate = PyThreadState_GET(); + tstate = _PyThreadState_Current; if (tstate != NULL && tstate->curexc_type != NULL) { /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; From python-checkins at python.org Sat Jun 10 14:23:54 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 10 Jun 2006 14:23:54 +0200 (CEST) Subject: [Python-checkins] r46819 - in python/trunk: Include/Python.h Include/pyport.h Modules/_hotshot.c Modules/_localemodule.c Modules/arraymodule.c Modules/mmapmodule.c Modules/posixmodule.c Modules/selectmodule.c Modules/socketmodule.c Modules/timemodule.c Objects/fileobject.c PC/getpathp.c PC/os2emx/pyconfig.h PC/os2vacpp/pyconfig.h PC/pyconfig.h PC/winsound.c Python/ceval.c Python/dynload_win.c Python/mystrtoul.c Python/pythonrun.c Python/strtod.c Python/thread_nt.h RISCOS/pyconfig.h configure configure.in pyconfig.h.in Message-ID: <20060610122354.1E9031E4003@bag.python.org> Author: martin.v.loewis Date: Sat Jun 10 14:23:46 2006 New Revision: 46819 Modified: python/trunk/Include/Python.h python/trunk/Include/pyport.h python/trunk/Modules/_hotshot.c python/trunk/Modules/_localemodule.c python/trunk/Modules/arraymodule.c python/trunk/Modules/mmapmodule.c python/trunk/Modules/posixmodule.c python/trunk/Modules/selectmodule.c python/trunk/Modules/socketmodule.c python/trunk/Modules/timemodule.c python/trunk/Objects/fileobject.c python/trunk/PC/getpathp.c python/trunk/PC/os2emx/pyconfig.h python/trunk/PC/os2vacpp/pyconfig.h python/trunk/PC/pyconfig.h python/trunk/PC/winsound.c python/trunk/Python/ceval.c python/trunk/Python/dynload_win.c python/trunk/Python/mystrtoul.c python/trunk/Python/pythonrun.c python/trunk/Python/strtod.c python/trunk/Python/thread_nt.h python/trunk/RISCOS/pyconfig.h python/trunk/configure python/trunk/configure.in python/trunk/pyconfig.h.in Log: Patch #1495999: Part two of Windows CE changes. - update header checks, using autoconf - provide dummies for getenv, environ, and GetVersion - adjust MSC_VER check in socketmodule.c Modified: python/trunk/Include/Python.h ============================================================================== --- python/trunk/Include/Python.h (original) +++ python/trunk/Include/Python.h Sat Jun 10 14:23:46 2006 @@ -35,7 +35,7 @@ #endif #include -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif #include Modified: python/trunk/Include/pyport.h ============================================================================== --- python/trunk/Include/pyport.h (original) +++ python/trunk/Include/pyport.h Sat Jun 10 14:23:46 2006 @@ -240,10 +240,10 @@ * to your pyconfig.h. Python code beyond this should check HAVE_STAT and * HAVE_FSTAT instead. * Also - * #define DONT_HAVE_SYS_STAT_H - * if doesn't exist on your platform, and + * #define HAVE_SYS_STAT_H + * if exists on your platform, and * #define HAVE_STAT_H - * if does (don't look at me -- ths mess is inherited). + * if does. */ #ifndef DONT_HAVE_STAT #define HAVE_STAT @@ -258,7 +258,7 @@ #include "unixstuff.h" #endif -#ifndef DONT_HAVE_SYS_STAT_H +#ifdef HAVE_SYS_STAT_H #if defined(PYOS_OS2) && defined(PYCC_GCC) #include #endif Modified: python/trunk/Modules/_hotshot.c ============================================================================== --- python/trunk/Modules/_hotshot.c (original) +++ python/trunk/Modules/_hotshot.c Sat Jun 10 14:23:46 2006 @@ -14,7 +14,11 @@ */ #ifdef MS_WINDOWS #include + +#ifdef HAVE_DIRECT_H #include /* for getcwd() */ +#endif + typedef __int64 hs_time; #define GETTIMEOFDAY(P_HS_TIME) \ { LARGE_INTEGER _temp; \ Modified: python/trunk/Modules/_localemodule.c ============================================================================== --- python/trunk/Modules/_localemodule.c (original) +++ python/trunk/Modules/_localemodule.c Sat Jun 10 14:23:46 2006 @@ -16,7 +16,7 @@ #include #include -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif Modified: python/trunk/Modules/arraymodule.c ============================================================================== --- python/trunk/Modules/arraymodule.c (original) +++ python/trunk/Modules/arraymodule.c Sat Jun 10 14:23:46 2006 @@ -10,9 +10,9 @@ #ifdef STDC_HEADERS #include #else /* !STDC_HEADERS */ -#ifndef DONT_HAVE_SYS_TYPES_H +#ifdef HAVE_SYS_TYPES_H #include /* For size_t */ -#endif /* DONT_HAVE_SYS_TYPES_H */ +#endif /* HAVE_SYS_TYPES_H */ #endif /* !STDC_HEADERS */ struct arrayobject; /* Forward */ Modified: python/trunk/Modules/mmapmodule.c ============================================================================== --- python/trunk/Modules/mmapmodule.c (original) +++ python/trunk/Modules/mmapmodule.c Sat Jun 10 14:23:46 2006 @@ -50,7 +50,10 @@ #endif /* UNIX */ #include + +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ /* Prefer MAP_ANONYMOUS since MAP_ANON is deprecated according to man page. */ #if !defined(MAP_ANONYMOUS) && defined(MAP_ANON) Modified: python/trunk/Modules/posixmodule.c ============================================================================== --- python/trunk/Modules/posixmodule.c (original) +++ python/trunk/Modules/posixmodule.c Sat Jun 10 14:23:46 2006 @@ -64,14 +64,21 @@ #include "osdefs.h" #endif +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ + +#ifdef HAVE_SYS_STAT_H #include +#endif /* HAVE_SYS_STAT_H */ #ifdef HAVE_SYS_WAIT_H #include /* For WNOHANG */ #endif +#ifdef HAVE_SIGNAL_H #include +#endif #ifdef HAVE_FCNTL_H #include @@ -246,9 +253,15 @@ #endif #ifdef _MSC_VER +#ifdef HAVE_DIRECT_H #include +#endif +#ifdef HAVE_IO_H #include +#endif +#ifdef HAVE_PROCESS_H #include +#endif #include "osdefs.h" #define _WIN32_WINNT 0x0400 /* Needed for CryptoAPI on some systems */ #include Modified: python/trunk/Modules/selectmodule.c ============================================================================== --- python/trunk/Modules/selectmodule.c (original) +++ python/trunk/Modules/selectmodule.c Sat Jun 10 14:23:46 2006 @@ -36,7 +36,7 @@ extern void bzero(void *, int); #endif -#ifndef DONT_HAVE_SYS_TYPES_H +#ifdef HAVE_SYS_TYPES_H #include #endif Modified: python/trunk/Modules/socketmodule.c ============================================================================== --- python/trunk/Modules/socketmodule.c (original) +++ python/trunk/Modules/socketmodule.c Sat Jun 10 14:23:46 2006 @@ -234,7 +234,9 @@ #endif /* Generic includes */ +#ifdef HAVE_SYS_TYPES_H #include +#endif /* Generic socket object definitions and includes */ #define PySocket_BUILDING_SOCKET @@ -270,7 +272,9 @@ #else /* MS_WINDOWS includes */ -# include +# ifdef HAVE_FCNTL_H +# include +# endif #endif @@ -290,7 +294,7 @@ * _SS_ALIGNSIZE is defined in sys/socket.h by 6.5.21, * for example, but not by 6.5.10. */ -#elif defined(_MSC_VER) && _MSC_VER>1200 +#elif defined(_MSC_VER) && _MSC_VER>1201 /* Do not include addrinfo.h for MSVC7 or greater. 'addrinfo' and * EAI_* constants are defined in (the already included) ws2tcpip.h. */ Modified: python/trunk/Modules/timemodule.c ============================================================================== --- python/trunk/Modules/timemodule.c (original) +++ python/trunk/Modules/timemodule.c Sat Jun 10 14:23:46 2006 @@ -19,7 +19,9 @@ #include +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ #ifdef QUICKWIN #include Modified: python/trunk/Objects/fileobject.c ============================================================================== --- python/trunk/Objects/fileobject.c (original) +++ python/trunk/Objects/fileobject.c Sat Jun 10 14:23:46 2006 @@ -4,9 +4,9 @@ #include "Python.h" #include "structmember.h" -#ifndef DONT_HAVE_SYS_TYPES_H +#ifdef HAVE_SYS_TYPES_H #include -#endif /* DONT_HAVE_SYS_TYPES_H */ +#endif /* HAVE_SYS_TYPES_H */ #ifdef MS_WINDOWS #define fileno _fileno Modified: python/trunk/PC/getpathp.c ============================================================================== --- python/trunk/PC/getpathp.c (original) +++ python/trunk/PC/getpathp.c Sat Jun 10 14:23:46 2006 @@ -62,8 +62,14 @@ #include #endif +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ + +#ifdef HAVE_SYS_STAT_H #include +#endif /* HAVE_SYS_STAT_H */ + #include /* Search in some common locations for the associated Python libraries. Modified: python/trunk/PC/os2emx/pyconfig.h ============================================================================== --- python/trunk/PC/os2emx/pyconfig.h (original) +++ python/trunk/PC/os2emx/pyconfig.h Sat Jun 10 14:23:46 2006 @@ -254,15 +254,33 @@ /* Define if you have the waitpid function. */ #define HAVE_WAITPID 1 +/* Define if you have the header file. */ +#undef HAVE_CONIO_H + +/* Define if you have the header file. */ +#undef HAVE_DIRECT_H + /* Define if you have the header file. */ #define HAVE_DIRENT_H 1 +/* Define if you have the header file. */ +#define HAVE_ERRNO_H 1 + /* Define if you have the header file. */ #define HAVE_FCNTL_H 1 +/* Define if you have the header file. */ +#undef HAVE_IO_H + /* Define if you have the header file. */ #define HAVE_NCURSES_H 1 +/* Define to 1 if you have the header file. */ +#define HAVE_PROCESS_H 1 + +/* Define if you have the header file. */ +#define HAVE_SIGNAL_H 1 + /* Define if you have the header file. */ #define HAVE_SYS_FILE_H 1 @@ -272,12 +290,18 @@ /* Define if you have the header file. */ #define HAVE_SYS_SELECT_H 1 +/* Define if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + /* Define if you have the header file. */ #define HAVE_SYS_TIME_H 1 /* Define if you have the header file. */ #define HAVE_SYS_TIMES_H 1 +/* Define if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + /* Define if you have the header file. */ #define HAVE_SYS_UN_H 1 Modified: python/trunk/PC/os2vacpp/pyconfig.h ============================================================================== --- python/trunk/PC/os2vacpp/pyconfig.h (original) +++ python/trunk/PC/os2vacpp/pyconfig.h Sat Jun 10 14:23:46 2006 @@ -112,6 +112,10 @@ #define HAVE_HYPOT 1 /* hypot() */ #define HAVE_PUTENV 1 /* putenv() */ /* #define VA_LIST_IS_ARRAY 1 */ /* if va_list is an array of some kind */ +/* #define HAVE_CONIO_H 1 */ /* #include */ +#define HAVE_ERRNO_H 1 /* #include */ +#define HAVE_SYS_STAT_H 1 /* #include */ +#define HAVE_SYS_TYPES_H 1 /* #include */ /* Variable-Arguments/Prototypes */ #define HAVE_PROTOTYPES 1 /* VAC++ supports C Function Prototypes */ @@ -124,6 +128,7 @@ #define MALLOC_ZERO_RETURNS_NULL 1 /* Our malloc(0) returns a NULL ptr */ /* Signal Handling */ +#define HAVE_SIGNAL_H 1 /* signal.h */ #define RETSIGTYPE void /* Return type of handlers (int or void) */ /* #undef WANT_SIGFPE_HANDLER */ /* Handle SIGFPE (see Include/pyfpe.h) */ /* #define HAVE_ALARM 1 */ /* alarm() */ @@ -163,7 +168,9 @@ #define HAVE_SETVBUF 1 /* setvbuf() */ #define HAVE_GETCWD 1 /* getcwd() */ #define HAVE_PIPE 1 /* pipe() [OS/2-specific code added] */ +#define HAVE_IO_H 1 /* #include */ #define HAVE_FCNTL_H 1 /* #include */ +#define HAVE_DIRECT_H 1 /* #include */ /* #define HAVE_FLOCK 1 */ /* flock() */ /* #define HAVE_TRUNCATE 1 */ /* truncate() */ /* #define HAVE_FTRUNCATE 1 */ /* ftruncate() */ @@ -172,6 +179,7 @@ /* #define HAVE_OPENDIR 1 */ /* opendir() */ /* Process Operations */ +#define HAVE_PROCESS_H 1 /* #include */ #define HAVE_GETPID 1 /* getpid() */ #define HAVE_SYSTEM 1 /* system() */ #define HAVE_WAIT 1 /* wait() */ Modified: python/trunk/PC/pyconfig.h ============================================================================== --- python/trunk/PC/pyconfig.h (original) +++ python/trunk/PC/pyconfig.h Sat Jun 10 14:23:46 2006 @@ -78,8 +78,15 @@ #endif #ifdef MS_WINCE -#define DONT_HAVE_SYS_STAT_H -#define DONT_HAVE_ERRNO_H +/* Python uses GetVersion() to distinguish between + * Windows NT and 9x/ME where OS Unicode support is concerned. + * Windows CE supports Unicode in the same way as NT so we + * define the missing GetVersion() accordingly. + */ +#define GetVersion() (4) +/* Windows CE does not support environment variables */ +#define getenv(v) (NULL) +#define environ (NULL) #endif /* Compiler specific defines */ @@ -356,6 +363,16 @@ /* Define to empty if the keyword does not work. */ /* #define const */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_CONIO_H 1 +#endif + +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_DIRECT_H 1 +#endif + /* Define if you have dirent.h. */ /* #define DIRENT 1 */ @@ -561,11 +578,26 @@ /* Define if you have the header file. */ /* #undef HAVE_DLFCN_H */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_ERRNO_H 1 +#endif + /* Define if you have the header file. */ #ifndef MS_WINCE #define HAVE_FCNTL_H 1 #endif +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_PROCESS_H 1 +#endif + +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_SIGNAL_H 1 +#endif + /* Define if you have the prototypes. */ #define HAVE_STDARG_PROTOTYPES @@ -581,12 +613,22 @@ /* Define if you have the header file. */ /* #define HAVE_SYS_SELECT_H 1 */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_SYS_STAT_H 1 +#endif + /* Define if you have the header file. */ /* #define HAVE_SYS_TIME_H 1 */ /* Define if you have the header file. */ /* #define HAVE_SYS_TIMES_H 1 */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_SYS_TYPES_H 1 +#endif + /* Define if you have the header file. */ /* #define HAVE_SYS_UN_H 1 */ Modified: python/trunk/PC/winsound.c ============================================================================== --- python/trunk/PC/winsound.c (original) +++ python/trunk/PC/winsound.c Sat Jun 10 14:23:46 2006 @@ -37,7 +37,9 @@ #include #include +#ifdef HAVE_CONIO_H #include /* port functions on Win9x */ +#endif #include PyDoc_STRVAR(sound_playsound_doc, Modified: python/trunk/Python/ceval.c ============================================================================== --- python/trunk/Python/ceval.c (original) +++ python/trunk/Python/ceval.c Sat Jun 10 14:23:46 2006 @@ -205,7 +205,7 @@ #ifdef WITH_THREAD -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif #include "pythread.h" Modified: python/trunk/Python/dynload_win.c ============================================================================== --- python/trunk/Python/dynload_win.c (original) +++ python/trunk/Python/dynload_win.c Sat Jun 10 14:23:46 2006 @@ -2,7 +2,9 @@ /* Support for dynamic loading of extension modules */ #include +#ifdef HAVE_DIRECT_H #include +#endif #include #include "Python.h" Modified: python/trunk/Python/mystrtoul.c ============================================================================== --- python/trunk/Python/mystrtoul.c (original) +++ python/trunk/Python/mystrtoul.c Sat Jun 10 14:23:46 2006 @@ -17,7 +17,7 @@ #include -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif Modified: python/trunk/Python/pythonrun.c ============================================================================== --- python/trunk/Python/pythonrun.c (original) +++ python/trunk/Python/pythonrun.c Sat Jun 10 14:23:46 2006 @@ -17,7 +17,9 @@ #include "eval.h" #include "marshal.h" +#ifdef HAVE_SIGNAL_H #include +#endif #ifdef HAVE_LANGINFO_H #include Modified: python/trunk/Python/strtod.c ============================================================================== --- python/trunk/Python/strtod.c (original) +++ python/trunk/Python/strtod.c Sat Jun 10 14:23:46 2006 @@ -54,7 +54,7 @@ extern double atof(const char *); /* Only called when result known to be ok */ -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif extern int errno; Modified: python/trunk/Python/thread_nt.h ============================================================================== --- python/trunk/Python/thread_nt.h (original) +++ python/trunk/Python/thread_nt.h Sat Jun 10 14:23:46 2006 @@ -5,7 +5,9 @@ #include #include +#ifdef HAVE_PROCESS_H #include +#endif typedef struct NRMUTEX { LONG owned ; Modified: python/trunk/RISCOS/pyconfig.h ============================================================================== --- python/trunk/RISCOS/pyconfig.h (original) +++ python/trunk/RISCOS/pyconfig.h Sat Jun 10 14:23:46 2006 @@ -553,6 +553,9 @@ /* Define if you have the waitpid function. */ #undef HAVE_WAITPID +/* Define if you have the header file. */ +#undef HAVE_CONIO_H + /* Define if you have the header file. */ #undef HAVE_DB_H @@ -562,18 +565,27 @@ /* Define if you have the header file. */ #undef HAVE_DB_185_H +/* Define if you have the header file. */ +#undef HAVE_DIRECT_H + /* Define if you have the header file. */ #undef HAVE_DIRENT_H /* Define if you have the header file. */ #undef HAVE_DLFCN_H +/* Define if you have the header file. */ +#define HAVE_ERRNO_H 1 + /* Define if you have the header file. */ #undef HAVE_FCNTL_H /* Define if you have the header file. */ #undef HAVE_GDBM_NDBM_H +/* Define if you have the header file. */ +#undef HAVE_IO_H + /* Define if you have the header file. */ #undef HAVE_LANGINFO_H @@ -595,12 +607,18 @@ /* Define if you have the header file. */ #undef HAVE_POLL_H +/* Define if you have the header file. */ +#undef HAVE_PROCESS_H + /* Define if you have the header file. */ #undef HAVE_PTHREAD_H /* Define if you have the header file. */ #undef HAVE_PTY_H +/* Define if you have the header file. */ +#define HAVE_SIGNAL_H + /* Define if you have the header file. */ #undef HAVE_SYS_AUDIOIO_H @@ -634,12 +652,18 @@ /* Define if you have the header file. */ #undef HAVE_SYS_SOCKET_H +/* Define if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + /* Define if you have the header file. */ #undef HAVE_SYS_TIME_H /* Define if you have the header file. */ #undef HAVE_SYS_TIMES_H +/* Define if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + /* Define if you have the header file. */ #undef HAVE_SYS_UN_H @@ -688,7 +712,6 @@ #define DONT_HAVE_FSTAT 1 #define DONT_HAVE_STAT 1 -#undef DONT_HAVE_SYS_STAT_H #define PLATFORM "riscos" Modified: python/trunk/configure ============================================================================== --- python/trunk/configure (original) +++ python/trunk/configure Sat Jun 10 14:23:46 2006 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 46700 . +# From configure.in Revision: 46720 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59 for python 2.5. # @@ -4623,14 +4623,24 @@ -for ac_header in asm/types.h curses.h dlfcn.h fcntl.h grp.h \ -shadow.h langinfo.h libintl.h ncurses.h poll.h pthread.h \ -stropts.h termios.h thread.h \ + + + + + + + + +for ac_header in asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +fcntl.h grp.h \ +shadow.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ +signal.h stropts.h termios.h thread.h \ unistd.h utime.h \ sys/audioio.h sys/bsdtty.h sys/file.h sys/loadavg.h sys/lock.h sys/mkdev.h \ sys/modem.h \ -sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/time.h \ -sys/times.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ +sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/stat.h \ +sys/time.h \ +sys/times.h sys/types.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ bluetooth/bluetooth.h do @@ -22046,9 +22056,10 @@ # 1. Remove the extension, and $U if already installed. ac_i=`echo "$ac_i" | sed 's/\$U\././;s/\.o$//;s/\.obj$//'` - # 2. Add them. - ac_libobjs="$ac_libobjs $ac_i\$U.$ac_objext" - ac_ltlibobjs="$ac_ltlibobjs $ac_i"'$U.lo' + # 2. Prepend LIBOBJDIR. When used with automake>=1.10 LIBOBJDIR + # will be set to the directory where LIBOBJS objects are built. + ac_libobjs="$ac_libobjs \${LIBOBJDIR}$ac_i\$U.$ac_objext" + ac_ltlibobjs="$ac_ltlibobjs \${LIBOBJDIR}$ac_i"'$U.lo' done LIBOBJS=$ac_libobjs Modified: python/trunk/configure.in ============================================================================== --- python/trunk/configure.in (original) +++ python/trunk/configure.in Sat Jun 10 14:23:46 2006 @@ -1066,14 +1066,16 @@ # checks for header files AC_HEADER_STDC -AC_CHECK_HEADERS(asm/types.h curses.h dlfcn.h fcntl.h grp.h \ -shadow.h langinfo.h libintl.h ncurses.h poll.h pthread.h \ -stropts.h termios.h thread.h \ +AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +fcntl.h grp.h \ +shadow.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ +signal.h stropts.h termios.h thread.h \ unistd.h utime.h \ sys/audioio.h sys/bsdtty.h sys/file.h sys/loadavg.h sys/lock.h sys/mkdev.h \ sys/modem.h \ -sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/time.h \ -sys/times.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ +sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/stat.h \ +sys/time.h \ +sys/times.h sys/types.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ bluetooth/bluetooth.h) AC_HEADER_DIRENT Modified: python/trunk/pyconfig.h.in ============================================================================== --- python/trunk/pyconfig.h.in (original) +++ python/trunk/pyconfig.h.in Sat Jun 10 14:23:46 2006 @@ -73,6 +73,9 @@ /* Define to 1 if you have the `confstr' function. */ #undef HAVE_CONFSTR +/* Define to 1 if you have the header file. */ +#undef HAVE_CONIO_H + /* Define to 1 if you have the `ctermid' function. */ #undef HAVE_CTERMID @@ -91,6 +94,9 @@ /* Define if we have /dev/ptmx. */ #undef HAVE_DEV_PTMX +/* Define to 1 if you have the header file. */ +#undef HAVE_DIRECT_H + /* Define to 1 if you have the header file, and it defines `DIR'. */ #undef HAVE_DIRENT_H @@ -107,6 +113,9 @@ /* Defined when any dynamic module loading is enabled. */ #undef HAVE_DYNAMIC_LOADING +/* Define to 1 if you have the header file. */ +#undef HAVE_ERRNO_H + /* Define to 1 if you have the `execv' function. */ #undef HAVE_EXECV @@ -248,6 +257,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_INTTYPES_H +/* Define to 1 if you have the header file. */ +#undef HAVE_IO_H + /* Define to 1 if you have the `kill' function. */ #undef HAVE_KILL @@ -353,6 +365,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_POLL_H +/* Define to 1 if you have the header file. */ +#undef HAVE_PROCESS_H + /* Define if your compiler supports function prototype */ #undef HAVE_PROTOTYPES @@ -446,6 +461,9 @@ /* Define to 1 if you have the `siginterrupt' function. */ #undef HAVE_SIGINTERRUPT +/* Define to 1 if you have the header file. */ +#undef HAVE_SIGNAL_H + /* Define to 1 if you have the `sigrelse' function. */ #undef HAVE_SIGRELSE From python-checkins at python.org Sat Jun 10 16:09:11 2006 From: python-checkins at python.org (skip.montanaro) Date: Sat, 10 Jun 2006 16:09:11 +0200 (CEST) Subject: [Python-checkins] r46820 - python/trunk/Lib/Queue.py Message-ID: <20060610140911.CE6541E4003@bag.python.org> Author: skip.montanaro Date: Sat Jun 10 16:09:11 2006 New Revision: 46820 Modified: python/trunk/Lib/Queue.py Log: document the class, not its initializer Modified: python/trunk/Lib/Queue.py ============================================================================== --- python/trunk/Lib/Queue.py (original) +++ python/trunk/Lib/Queue.py Sat Jun 10 16:09:11 2006 @@ -14,11 +14,11 @@ pass class Queue: - def __init__(self, maxsize=0): - """Initialize a queue object with a given maximum size. + """Create a queue object with a given maximum size. - If maxsize is <= 0, the queue size is infinite. - """ + If maxsize is <= 0, the queue size is infinite. + """ + def __init__(self, maxsize=0): try: import threading except ImportError: From python-checkins at python.org Sat Jun 10 18:40:02 2006 From: python-checkins at python.org (greg.ward) Date: Sat, 10 Jun 2006 18:40:02 +0200 (CEST) Subject: [Python-checkins] r46821 - python/trunk/Doc/lib/liboptparse.tex Message-ID: <20060610164002.4A1C91E400C@bag.python.org> Author: greg.ward Date: Sat Jun 10 18:40:01 2006 New Revision: 46821 Modified: python/trunk/Doc/lib/liboptparse.tex Log: Sync with Optik docs (rev 518): * restore "Extending optparse" section * document ALWAYS_TYPED_ACTIONS (SF #1449311) Modified: python/trunk/Doc/lib/liboptparse.tex ============================================================================== --- python/trunk/Doc/lib/liboptparse.tex (original) +++ python/trunk/Doc/lib/liboptparse.tex Sat Jun 10 18:40:01 2006 @@ -1,3 +1,5 @@ +% THIS FILE IS AUTO-GENERATED! DO NOT EDIT! +% (Your changes will be lost the next time it is generated.) \section{\module{optparse} --- More powerful command line option parser} \declaremodule{standard}{optparse} \moduleauthor{Greg Ward}{gward at python.net} @@ -306,7 +308,7 @@ Actions tell \module{optparse} what to do when it encounters an option on the command line. There is a fixed set of actions hard-coded into \module{optparse}; -adding new actions is an advanced topic covered in section~\ref{optparse-extending}, Extending \module{optparse}. +adding new actions is an advanced topic covered in section~\ref{optparse-extending-optparse}, Extending \module{optparse}. Most actions tell \module{optparse} to store a value in some variable{---}for example, take a string from the command line and store it in an attribute of \code{options}. @@ -371,7 +373,7 @@ string: the default destination for \code{"-f"} is \code{f}. \module{optparse} also includes built-in \code{long} and \code{complex} types. Adding -types is covered in section~\ref{optparse-extending}, Extending \module{optparse}. +types is covered in section~\ref{optparse-extending-optparse}, Extending \module{optparse}. \subsubsection{Handling boolean (flag) options\label{optparse-handling-boolean-options}} @@ -566,7 +568,7 @@ parser = OptionParser(usage="%prog [-f] [-q]", version="%prog 1.0") \end{verbatim} -Note that \code{"{\%}prog"} is expanded just like it is in \code{usage}. Apart +\code{"{\%}prog"} is expanded just like it is in \code{usage}. Apart from that, \code{version} can contain anything you like. When you supply it, \module{optparse} automatically adds a \code{"-{}-version"} option to your parser. If it encounters this option on the command line, it expands your @@ -659,7 +661,7 @@ if __name__ == "__main__": main() \end{verbatim} -% $Id: tutorial.txt 505 2005-07-22 01:52:40Z gward $ +% $Id: tutorial.txt 515 2006-06-10 15:37:45Z gward $ \subsection{Reference Guide\label{optparse-reference-guide}} @@ -1146,7 +1148,7 @@ \module{optparse} has six built-in option types: \code{string}, \code{int}, \code{long}, \code{choice}, \code{float} and \code{complex}. If you need to add new option -types, see section~\ref{optparse-extending}, Extending \module{optparse}. +types, see section~\ref{optparse-extending-optparse}, Extending \module{optparse}. Arguments to string options are not checked or converted in any way: the text on the command line is stored in the destination (or passed to the @@ -1681,3 +1683,206 @@ \code{"-c"}. Fixing this is left as an exercise for the reader. % $Id: callbacks.txt 415 2004-09-30 02:26:17Z greg $ + +\subsection{Extending \module{optparse}\label{optparse-extending-optparse}} + +Since the two major controlling factors in how \module{optparse} interprets +command-line options are the action and type of each option, the most +likely direction of extension is to add new actions and new types. + + +\subsubsection{Adding new types\label{optparse-adding-new-types}} + +To add new types, you need to define your own subclass of \module{optparse}'s Option +class. This class has a couple of attributes that define \module{optparse}'s types: +\member{TYPES} and \member{TYPE{\_}CHECKER}. + +\member{TYPES} is a tuple of type names; in your subclass, simply define a new +tuple \member{TYPES} that builds on the standard one. + +\member{TYPE{\_}CHECKER} is a dictionary mapping type names to type-checking +functions. A type-checking function has the following signature: +\begin{verbatim} +def check_mytype(option, opt, value) +\end{verbatim} + +where \code{option} is an \class{Option} instance, \code{opt} is an option string +(e.g., \code{"-f"}), and \code{value} is the string from the command line that +must be checked and converted to your desired type. \code{check{\_}mytype()} +should return an object of the hypothetical type \code{mytype}. The value +returned by a type-checking function will wind up in the OptionValues +instance returned by \method{OptionParser.parse{\_}args()}, or be passed to a +callback as the \code{value} parameter. + +Your type-checking function should raise OptionValueError if it +encounters any problems. OptionValueError takes a single string +argument, which is passed as-is to OptionParser's \method{error()} method, +which in turn prepends the program name and the string \code{"error:"} and +prints everything to stderr before terminating the process. + +Here's a silly example that demonstrates adding a \code{complex} option +type to parse Python-style complex numbers on the command line. (This +is even sillier than it used to be, because \module{optparse} 1.3 added built-in +support for complex numbers, but never mind.) + +First, the necessary imports: +\begin{verbatim} +from copy import copy +from optparse import Option, OptionValueError +\end{verbatim} + +You need to define your type-checker first, since it's referred to later +(in the \member{TYPE{\_}CHECKER} class attribute of your Option subclass): +\begin{verbatim} +def check_complex(option, opt, value): + try: + return complex(value) + except ValueError: + raise OptionValueError( + "option %s: invalid complex value: %r" % (opt, value)) +\end{verbatim} + +Finally, the Option subclass: +\begin{verbatim} +class MyOption (Option): + TYPES = Option.TYPES + ("complex",) + TYPE_CHECKER = copy(Option.TYPE_CHECKER) + TYPE_CHECKER["complex"] = check_complex +\end{verbatim} + +(If we didn't make a \function{copy()} of \member{Option.TYPE{\_}CHECKER}, we would end +up modifying the \member{TYPE{\_}CHECKER} attribute of \module{optparse}'s Option class. +This being Python, nothing stops you from doing that except good manners +and common sense.) + +That's it! Now you can write a script that uses the new option type +just like any other \module{optparse}-based script, except you have to instruct your +OptionParser to use MyOption instead of Option: +\begin{verbatim} +parser = OptionParser(option_class=MyOption) +parser.add_option("-c", type="complex") +\end{verbatim} + +Alternately, you can build your own option list and pass it to +OptionParser; if you don't use \method{add{\_}option()} in the above way, you +don't need to tell OptionParser which option class to use: +\begin{verbatim} +option_list = [MyOption("-c", action="store", type="complex", dest="c")] +parser = OptionParser(option_list=option_list) +\end{verbatim} + + +\subsubsection{Adding new actions\label{optparse-adding-new-actions}} + +Adding new actions is a bit trickier, because you have to understand +that \module{optparse} has a couple of classifications for actions: +\begin{description} +\item[``store'' actions] +actions that result in \module{optparse} storing a value to an attribute of the +current OptionValues instance; these options require a \member{dest} +attribute to be supplied to the Option constructor +\item[``typed'' actions] +actions that take a value from the command line and expect it to be +of a certain type; or rather, a string that can be converted to a +certain type. These options require a \member{type} attribute to the +Option constructor. +\end{description} + +These are overlapping sets: some default ``store'' actions are \code{store}, +\code{store{\_}const}, \code{append}, and \code{count}, while the default ``typed'' +actions are \code{store}, \code{append}, and \code{callback}. + +When you add an action, you need to categorize it by listing it in at +least one of the following class attributes of Option (all are lists of +strings): +\begin{description} +\item[\member{ACTIONS}] +all actions must be listed in ACTIONS +\item[\member{STORE{\_}ACTIONS}] +``store'' actions are additionally listed here +\item[\member{TYPED{\_}ACTIONS}] +``typed'' actions are additionally listed here +\item[\code{ALWAYS{\_}TYPED{\_}ACTIONS}] +actions that always take a type (i.e. whose options always take a +value) are additionally listed here. The only effect of this is +that \module{optparse} assigns the default type, \code{string}, to options with no +explicit type whose action is listed in \code{ALWAYS{\_}TYPED{\_}ACTIONS}. +\end{description} + +In order to actually implement your new action, you must override +Option's \method{take{\_}action()} method and add a case that recognizes your +action. + +For example, let's add an \code{extend} action. This is similar to the +standard \code{append} action, but instead of taking a single value from +the command-line and appending it to an existing list, \code{extend} will +take multiple values in a single comma-delimited string, and extend an +existing list with them. That is, if \code{"-{}-names"} is an \code{extend} +option of type \code{string}, the command line +\begin{verbatim} +--names=foo,bar --names blah --names ding,dong +\end{verbatim} + +would result in a list +\begin{verbatim} +["foo", "bar", "blah", "ding", "dong"] +\end{verbatim} + +Again we define a subclass of Option: +\begin{verbatim} +class MyOption (Option): + + ACTIONS = Option.ACTIONS + ("extend",) + STORE_ACTIONS = Option.STORE_ACTIONS + ("extend",) + TYPED_ACTIONS = Option.TYPED_ACTIONS + ("extend",) + ALWAYS_TYPED_ACTIONS = Option.ALWAYS_TYPED_ACTIONS + ("extend",) + + def take_action(self, action, dest, opt, value, values, parser): + if action == "extend": + lvalue = value.split(",") + values.ensure_value(dest, []).extend(lvalue) + else: + Option.take_action( + self, action, dest, opt, value, values, parser) +\end{verbatim} + +Features of note: +\begin{itemize} +\item {} +\code{extend} both expects a value on the command-line and stores that +value somewhere, so it goes in both \member{STORE{\_}ACTIONS} and +\member{TYPED{\_}ACTIONS} + +\item {} +to ensure that \module{optparse} assigns the default type of \code{string} to +\code{extend} actions, we put the \code{extend} action in +\code{ALWAYS{\_}TYPED{\_}ACTIONS} as well + +\item {} +\method{MyOption.take{\_}action()} implements just this one new action, and +passes control back to \method{Option.take{\_}action()} for the standard +\module{optparse} actions + +\item {} +\code{values} is an instance of the optparse{\_}parser.Values class, +which provides the very useful \method{ensure{\_}value()} method. +\method{ensure{\_}value()} is essentially \function{getattr()} with a safety valve; +it is called as +\begin{verbatim} +values.ensure_value(attr, value) +\end{verbatim} + +If the \code{attr} attribute of \code{values} doesn't exist or is None, then +ensure{\_}value() first sets it to \code{value}, and then returns 'value. +This is very handy for actions like \code{extend}, \code{append}, and +\code{count}, all of which accumulate data in a variable and expect that +variable to be of a certain type (a list for the first two, an integer +for the latter). Using \method{ensure{\_}value()} means that scripts using +your action don't have to worry about setting a default value for the +option destinations in question; they can just leave the default as +None and \method{ensure{\_}value()} will take care of getting it right when +it's needed. + +\end{itemize} +% $Id: extending.txt 517 2006-06-10 16:18:11Z gward $ + From python-checkins at python.org Sat Jun 10 18:46:02 2006 From: python-checkins at python.org (skip.montanaro) Date: Sat, 10 Jun 2006 18:46:02 +0200 (CEST) Subject: [Python-checkins] r46822 - python/branches/release24-maint/Doc/lib/libcsv.tex Message-ID: <20060610164602.B3A7A1E4003@bag.python.org> Author: skip.montanaro Date: Sat Jun 10 18:46:02 2006 New Revision: 46822 Modified: python/branches/release24-maint/Doc/lib/libcsv.tex Log: backport wording fix Modified: python/branches/release24-maint/Doc/lib/libcsv.tex ============================================================================== --- python/branches/release24-maint/Doc/lib/libcsv.tex (original) +++ python/branches/release24-maint/Doc/lib/libcsv.tex Sat Jun 10 18:46:02 2006 @@ -174,7 +174,7 @@ The \class{Sniffer} class is used to deduce the format of a CSV file. \end{classdesc} -The \class{Sniffer} class provides a single method: +The \class{Sniffer} class provides two methods: \begin{methoddesc}{sniff}{sample\optional{,delimiters=None}} Analyze the given \var{sample} and return a \class{Dialect} subclass From buildbot at python.org Sat Jun 10 18:59:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 16:59:30 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060610165930.CA0F21E4003@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/625 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: greg.ward Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 19:56:57 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sat, 10 Jun 2006 19:56:57 +0200 (CEST) Subject: [Python-checkins] r46823 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060610175657.54CBF1E400C@bag.python.org> Author: mateusz.rukowicz Date: Sat Jun 10 19:56:54 2006 New Revision: 46823 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Huge improvement, *many* bugs fixed, adding and substracting now works. Improved Contexts, roundings, and almost every function. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sat Jun 10 19:56:54 2006 @@ -171,7 +171,30 @@ static void _limb_fill(long *self, long ndigits, long x) { - //TODO + long full_limb = 0; + long mult = 1; + long i; + while(mult != BASE) + { + full_limb += x * mult; + mult *= 10; + } + + long limbs = ndigits / LOG; + + for(i = 0;i olimbs ? 1 : -1; + + for(i = limbs-1; i>=0 ;i--) + if(self[i] != other[i]) + return self[i] > other[i] ? 1 : -1; + return 0; +} + +static long +_limb_add_core(long *big, long bsize, long *small, long ssize, long *out) +{ + long max = bsize + 1; + long limbs_total = (max + LOG -1)/LOG; + long limbs_copy = (bsize + LOG -1)/LOG; + long limbs_add = (ssize + LOG -1)/LOG; + long i; + + for(i =0 ;i= BASE) + { + out[i] -= BASE; + out[i+1] ++; + } + } + + /* there may be some extra digit at max, check this out */ + if(_limb_get_digit(out, max, 0) > 0) + return max; + else + return max-1; + +} + + +static long +_limb_add(long *self, long ssize, long *other, long osize, long *out) +{ + if(ssize > osize) + return _limb_add_core(self, ssize, other, osize, out); + else + return _limb_add_core(other, osize, self, ssize, out); +} + +static long +_limb_sub(long *big, long bsize, long *small, long ssize, long *out) +{ + long blimbs = (bsize + LOG -1)/LOG; + long slimbs = (ssize + LOG -1)/LOG; + long i; + + for(i=0;isign == 0 ? SIGN_POSNAN : SIGN_NEGNAN); - res = _new_decimalobj(type, thing->ob_size, sign, 0); +// sign = ((thing->sign&1 == 0) ? SIGN_POSNAN : SIGN_NEGNAN); + if(thing->sign&1) /* neg */ + sign = SIGN_NEGNAN; + else + sign = SIGN_POSNAN; + + res = _new_decimalobj(type, thing->ob_size, sign, 0); if (!res) return NULL; for (i = 0; i < thing->ob_size; i++) res->digits[i] = thing->digits[i]; /* DELETE */ @@ -657,31 +776,31 @@ if (!new) return NULL; /* determine if we need a new digit */ - for (spot = 0; spot < self->ob_size; spot++) { - if (self->digits[spot] != 9) { - spot = -1; - break; - } - } - if (spot == -1) { +// for (spot = 0; spot < self->ob_size; spot++) { +// if (self->digits[spot] != 9) { +// spot = -1; +// break; +// } +// } +// if (spot == -1) { /* no new digit needed */ - new->ob_size--; - for (spot = 0; spot < self->ob_size; spot++) - new->digits[spot] = self->digits[spot]; - } else { - for (spot = 0; spot < self->ob_size; spot++) - new->digits[spot+1] = self->digits[spot]; - new->digits[0] = 0; - } +// new->ob_size--; +// for (spot = 0; spot < self->ob_size; spot++) +// new->digits[spot] = self->digits[spot]; +// } else { +// for (spot = 0; spot < self->ob_size; spot++) +// new->digits[spot+1] = self->digits[spot]; +// new->digits[0] = 0; +// } - spot = new->ob_size-1; - new->digits[spot]++; - while (new->digits[spot] == 10) { - new->digits[spot] = 0; - spot--; - assert(spot >= 0); - new->digits[spot]++; - } +// spot = new->ob_size-1; +// new->digits[spot]++; +// while (new->digits[spot] == 10) { +// new->digits[spot] = 0; +// spot--; + // assert(spot >= 0); +// new->digits[spot]++; +// } for(i=0;ilimb_count;i++) new->limbs[i] = self->limbs[i]; @@ -697,6 +816,8 @@ new->limbs[i+1] ++; i++; } + if(_limb_get_digit(new->limbs, new->ob_size, 0) == 0) + new->ob_size --; return new; } @@ -705,7 +826,7 @@ _round_down(decimalobject *self, long prec, long expdiff, contextobject *ctx) { long i; - decimalobject *new = _NEW_decimalobj(prec, self->sign, self->exp - expdiff); + decimalobject *new = _NEW_decimalobj(prec, self->sign, self->exp + expdiff); if (!new) return NULL; for (i = 0; i < prec; i++) new->digits[i] = self->digits[i]; @@ -719,7 +840,7 @@ _round_up(decimalobject *self, long prec, long expdiff, contextobject *ctx) { /* XXX temporary solution with limbs */ long i; - decimalobject *new = _NEW_decimalobj(prec, self->sign, self->exp - expdiff); + decimalobject *new = _NEW_decimalobj(prec, self->sign, self->exp + expdiff); decimalobject *new2 = NULL; if (!new) return NULL; for (i = 0; i < prec; i++) @@ -783,7 +904,7 @@ tmp->digits[i] = self->digits[i]; last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); - assert(self->digits[prec] == last); +// assert(self->digits[prec] == last); if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ @@ -808,8 +929,6 @@ for (i = 0; i < prec; i++) tmp->digits[i] = self->digits[i]; last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); - last = self->digits[prec]; - assert(last == self->digits[prec]); if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ @@ -817,7 +936,7 @@ return _do_round_half_up(self, prec, expdiff, ctx, tmp); } // if ((self->digits[prec-1] & 1) == 0) - if((_limb_get_digit(self->limbs, self->ob_size, i)&1) == 0) + if((_limb_get_digit(self->limbs, self->ob_size, prec-1)&1) == 0) return tmp; } return _do_round_half_up(self, prec, expdiff, ctx, tmp); @@ -839,7 +958,7 @@ /* Round up (regardless of sign) */ static decimalobject * -_round_floor(decimalobject *self, long prec, long expdiff, contextobject *ctx) +_round_ceiling(decimalobject *self, long prec, long expdiff, contextobject *ctx) { assert(self->sign <= 1); if (self->sign > 0) @@ -850,7 +969,7 @@ /* Round down (regardless of sign) */ static decimalobject * -_round_ceiling(decimalobject *self, long prec, long expdiff, contextobject *ctx) +_round_floor(decimalobject *self, long prec, long expdiff, contextobject *ctx) { assert(self->sign <= 1); if (self->sign > 0) @@ -957,14 +1076,19 @@ } /* Maybe all the lost digits are 0. */ - for (i = expdiff; i < self->ob_size; i++) { + for (i = self->ob_size - expdiff; i < self->ob_size; i++) { // if (self->digits[i] > 0) if(_limb_get_digit(self->limbs, self->ob_size, i) > 0) goto no_way; } /* All lost digits are 0, so just clobber new */ - new->ob_size = prec; - new->exp += expdiff; +// new->ob_size = prec; + while(new->ob_size > prec) + { + _limb_cut_one_digit(new->limbs, new->ob_size); /* VERY SLOW */ + new->ob_size --; + } + new->exp += expdiff; if (handle_Rounded(ctx, NULL) != 0) { Py_DECREF(new); return NULL; @@ -974,15 +1098,18 @@ no_way: /* Now rounding starts. We still own "new". */ rnd_func = round_funcs[rounding]; - if (prec != ctx->prec) { - ctx2 = (contextobject *)context_copy(ctx); - if (!ctx2) { - Py_DECREF(new); - return NULL; - } - ctx2->prec = prec; - ctx = ctx2; - } +// if (prec != ctx->prec) { +// ctx2 = (contextobject *)context_copy(ctx); +// if (!ctx2) { +// Py_DECREF(new); +// return NULL; +// } +// ctx2->prec = prec; +// ctx = ctx2; + /* }*/ /* XXX here is quite subtle bug - we copy context, and set flags in copied context + after that, they are lost, not sure if we really need function above, + I'll comment it */ + /* ctx2 is NULL if the original context is used, because that one * doesn't have to be DECREF'd. */ new2 = rnd_func(new, prec, expdiff, ctx); @@ -1002,7 +1129,7 @@ return NULL; } -/* Default values: rounding=-1 (use context), watchexp=1 */ +/* Default values: rounding=-1 (use context), watchexp=1 */ /* TODO TODO TODO */ static decimalobject * _decimal_rescale(decimalobject *self, long exp, contextobject *ctx, int rounding, int watchexp) @@ -1058,7 +1185,9 @@ for (i = 0; i < self->ob_size; i++) ans->digits[i+1] = self->digits[i]; ans->digits[0] = 0; - _limb_first_n_digits(self->limbs, self->ob_size, 0, ans->limbs, ans->ob_size); + for(i=0;ilimb_count;i++) + ans->limbs[i] = 0; + _limb_first_n_digits(self->limbs, self->ob_size, 0, ans->limbs, ans->ob_size-1); } tmp = _decimal_round(ans, digits, ctx, rounding); @@ -1066,11 +1195,12 @@ if (!tmp) return NULL; - if (tmp->digits[0] == 0 && tmp->ob_size > 1) { - /* We need one digit less, just clobber tmp. */ +// if (tmp->digits[0] == 0 && tmp->ob_size > 1) { + if(_limb_get_digit(tmp -> limbs, tmp->ob_size, 0) == 0 && tmp->ob_size > 1){ + /* We need one digit less, just clobber tmp. */ for (i = 0; i < tmp->ob_size-1; i++) tmp->digits[i] = tmp->digits[i+1]; - _limb_cut_one_digit(tmp->limbs, tmp->ob_size); +// _limb_cut_one_digit(tmp->limbs, tmp->ob_size); tmp->ob_size--; } tmp->exp = exp; @@ -1809,12 +1939,19 @@ SANITY_CHECK(p); /* check for digits != {0, } */ - if (d->ob_size != 1 || d->digits[0] != 0) { - for (i = 0; i < d->ob_size; i++) { - p += sprintf(p, "%d", d->digits[i]); - SANITY_CHECK(p); - } - } +// if (d->ob_size != 1 || d->digits[0] != 0) { +// for (i = 0; i < d->ob_size; i++) { +// p += sprintf(p, "%d", d->digits[i]); +// SANITY_CHECK(p); +// } + if(d->ob_size != 1 || _limb_get_digit(d->limbs, d->ob_size, 0) != 0) + { + for(i=0;i< d->ob_size;i++) + { + p+= sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, i)); /* SLOW */ + SANITY_CHECK(p); + } + } } else { /* infinity */ p += sprintf(p, "%s", "Infinity"); } @@ -2213,7 +2350,9 @@ { int shouldround, sign, negativezero = 0; long exp, oexp; - decimalobject *res, *res2; + long prec,cmp; + decimalobject *res, *res2, *ret, *o1, *o2; + prec = ctx->prec; if (ISSPECIAL(self) || ISSPECIAL(other)) { decimalobject *nan; @@ -2249,12 +2388,13 @@ res = _new_decimalobj(self->ob_type, 1, sign, exp); if (!res) return NULL; res->digits[0] = 0; + res->limbs[0] = 0; return res; } if (!decimal_nonzero(self)) { oexp = other->exp - ctx->prec - 1; exp = (exp > oexp ? exp : oexp); - res = _decimal_rescale(other, exp, ctx, 0, 1); + res = _decimal_rescale(other, exp, ctx, 0, 0); if (!res) return NULL; if (shouldround) { res2 = _decimal_fix(res, ctx); @@ -2267,7 +2407,7 @@ if (!decimal_nonzero(other)) { oexp = self->exp - ctx->prec - 1; exp = (exp > oexp ? exp : oexp); - res = _decimal_rescale(self, exp, ctx, 0, 1); + res = _decimal_rescale(self, exp, ctx, 0, 0); if (!res) return NULL; if (shouldround) { res2 = _decimal_fix(res, ctx); @@ -2278,8 +2418,161 @@ } } - /* XXX */ - Py_RETURN_NONE; + decimalobject *tmp; /* we borrow refference */ + decimalobject *oother; + + long numdigits = self->exp - other->exp; + + if(numdigits < 0) + { + numdigits *= -1; + tmp = other; + oother = self; + } + else + { + tmp = self; + oother = other; + } + + /* we borrow refference */ + if(shouldround && numdigits > prec + 1) + { + if(numdigits > (oother->ob_size + prec + 1 - tmp->ob_size)) + { + long extend = prec + 2 - tmp->ob_size; + if(extend <= 0) + extend = 1; + o1 = _NEW_decimalobj(tmp->ob_size + extend, tmp->sign, tmp->exp - extend); + if(!o1) + return NULL; + + _limb_first_n_digits(tmp->limbs, tmp->ob_size, 0, o1->limbs, o1->ob_size); + o2 = _NEW_decimalobj(1, oother->sign, o1->exp); + if(!o2) + { + Py_XDECREF(o1); + return NULL; + } + o2->limbs[0] =1; + goto calc; + } + } + + o1 = _NEW_decimalobj(tmp->ob_size + numdigits, tmp->sign, tmp->exp - numdigits); + _limb_first_n_digits(tmp->limbs, tmp->ob_size, 0, o1->limbs, o1->ob_size); + + if(!o1) + { + return NULL; + } + o2 = oother; + Py_INCREF(o2); + + +calc: + + assert(o1->exp == o2->exp); + exp = o1->exp; + cmp = _limb_compare(o1->limbs, o1->limb_count, o2->limbs, o2->limb_count); + + if(o1->sign != o2->sign) + { + if(!cmp) + { + Py_DECREF(o1); + Py_DECREF(o2); + ret = _NEW_decimalobj(1, negativezero, exp); + if(!ret) + return NULL; + ret->limbs[0] = 0; + if(ret->exp < ETINY(ctx)) + { + ret->exp = ETINY(ctx); + if(handle_Clamped(ctx, NULL) != 0) + { + Py_DECREF(ret); + return NULL; + } + } + + return ret; + } + else + { + long max_size; + if(cmp == -1) + { + ret = o1; + o1 = o2; + o2 = ret; + ret = 0; + } + /* o1 > o2 */ + max_size = o1->ob_size > o2->ob_size ? o1->ob_size : o2->ob_size; + ret = _NEW_decimalobj(max_size, o1->sign, o1->exp); + if(!ret) + { + Py_DECREF(o1); + Py_DECREF(o2); + return NULL; + } + + max_size = _limb_sub(o1->limbs, o1->ob_size, o2->limbs, o2->ob_size, ret->limbs); + ret->ob_size = max_size; + ret->limb_count = (max_size + LOG -1)/LOG; + + } + } + else + { + long max_size; + max_size = o1->ob_size > o2->ob_size ? o1->ob_size : o2->ob_size; + ret = _NEW_decimalobj(max_size + 1, o1->sign, o1->exp); /* we may have extra digit */ + if(!ret) + { + Py_DECREF(o1); + Py_DECREF(o2); + return NULL; + } + + max_size = _limb_add(o1->limbs, o1->ob_size, o2->limbs, o2->ob_size, ret->limbs); + ret->ob_size = max_size; + ret->limb_count = (max_size + LOG -1)/LOG; +// Py_DECREF(o1); +// Py_DECREF(o2); +// if(shouldround) +// { +// fixed = _decimal_fix(ret, ctx); +// Py_DECREF(ret); +// if(!fixed) +// { +// return NULL; +// } +// return fixed; +// } +// return ret; + } + + Py_DECREF(o1); + Py_DECREF(o2); + + if(shouldround) + { + decimalobject *fixed; + fixed = _decimal_fix(ret,ctx); + Py_DECREF(ret); + if(!fixed) + return NULL; + return fixed; + } + + return ret; + +error: + Py_XDECREF(o1); + Py_XDECREF(o2); + return NULL; } DECIMAL_SPECIAL_2FUNC(decimal_add) @@ -2693,10 +2986,14 @@ return NULL; /* remove sign */ if (str[0] == '+') + { str++; + len --; + } else if (str[0] == '-') { str++; literalsign = 1; + len --; } if (tolower(str[0]) == 'n' && tolower(str[1]) == 'a' && @@ -2734,7 +3031,24 @@ return NULL; for (i = 0; i < size; i++) new->digits[i] = *(start+i) -48; - return new; + for(i=0;ilimb_count;i++) + new->limbs[i] = 0; + + long mult = 1; + long limb = 0; + for(i = size -1; i>=0; i--) + { + assert(limb < new->limb_count); + new->limbs[limb] += mult * (start[i] - '0'); + mult *= 10; + if(mult == BASE) + { + mult = 1; + limb ++; + } + } + + return new; } @@ -3370,6 +3684,7 @@ static PyMemberDef _decimal_members[] = { {"_sign", T_INT, offsetof(decimalobject, sign), 0}, {"_exp", T_LONG, offsetof(decimalobject, exp), 0}, + {"_size", T_LONG, offsetof(decimalobject, ob_size), 0}, {NULL} }; @@ -3668,8 +3983,13 @@ static PyObject * context_clear_flags(contextobject *self) { - self->flags = 0; - Py_RETURN_NONE; +// self->flags = 0; +// PyDict_Clear(self->flags); + int i; + for(i = 0;iflags,i,0); + + Py_RETURN_NONE; } @@ -3810,6 +4130,32 @@ be created by macros. */ static PyObject * +context_apply(contextobject *self, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"a", 0}; + decimalobject *a; + + if(!PyArg_ParseTupleAndKeywords(args, kwds, "O", kwlist, &a)) + return NULL; + + if(!PyDecimal_Check(a)) + { + PyErr_SetString(PyExc_ValueError, "a must be Decimal object"); + return NULL; + } + + decimalobject *tmp = _decimal_fix(a, self); + if(!tmp) + return NULL; + + PyObject *ret = decimal_str(tmp); + Py_DECREF(tmp); + + return ret; + +} + +static PyObject * context_power(contextobject *self, PyObject *args) { PyObject *a, *b, *c; @@ -4227,6 +4573,8 @@ /* XXX: all that PyCFunction casting might not be necessary */ static PyMethodDef context_methods[] = { + {"_apply", (PyCFunction)context_apply, + METH_VARARGS | METH_KEYWORDS}, {"clear_flags", (PyCFunction)context_clear_flags, METH_NOARGS}, {"copy", (PyCFunction)context_copy, @@ -4457,6 +4805,11 @@ _flags = PyDict_New(); if (!_flags) goto err; + for(i=0; i < NUMSIGNALS; i++) /* XXX don't know if it's ok! */ + { + _set_flag(_flags,i,0); + } + if (pyflags == NULL) { /* don't copy flags from default context */ } else if (PyDict_Check(pyflags)) { From python-checkins at python.org Sat Jun 10 21:51:48 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 21:51:48 +0200 (CEST) Subject: [Python-checkins] r46824 - in python/trunk/Modules/_ctypes: _ctypes.c callproc.c ctypes.h libffi/src/x86/darwin.S libffi_msvc/mingwin32.S stgdict.c Message-ID: <20060610195148.874F81E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 21:51:46 2006 New Revision: 46824 Removed: python/trunk/Modules/_ctypes/libffi_msvc/mingwin32.S Modified: python/trunk/Modules/_ctypes/_ctypes.c python/trunk/Modules/_ctypes/callproc.c python/trunk/Modules/_ctypes/ctypes.h python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S python/trunk/Modules/_ctypes/stgdict.c Log: Upgrade to ctypes version 0.9.9.7. Summary of changes: - support for 'variable sized' data - support for anonymous structure/union fields - fix severe bug with certain arrays or structures containing more than 256 fields Modified: python/trunk/Modules/_ctypes/_ctypes.c ============================================================================== --- python/trunk/Modules/_ctypes/_ctypes.c (original) +++ python/trunk/Modules/_ctypes/_ctypes.c Sat Jun 10 21:51:46 2006 @@ -342,6 +342,14 @@ static PyObject * CDataType_repeat(PyObject *self, Py_ssize_t length) { + if (length < 0) + return PyErr_Format(PyExc_ValueError, +#if (PY_VERSION_HEX < 0x02050000) + "Array length must be >= 0, not %d", +#else + "Array length must be >= 0, not %zd", +#endif + length); return CreateArrayType(self, length); } @@ -1809,23 +1817,62 @@ } static PyObject * -unique_key(CDataObject *target, int index) +unique_key(CDataObject *target, Py_ssize_t index) { - char string[256]; /* XXX is that enough? */ + char string[256]; char *cp = string; - *cp++ = index + '0'; + size_t bytes_left; + + assert(sizeof(string) - 1 > sizeof(Py_ssize_t) * 2); +#if (PY_VERSION_HEX < 0x02050000) + cp += sprintf(cp, "%x", index); +#else +#ifdef MS_WIN32 +/* MSVC does not understand the 'z' size specifier */ + cp += sprintf(cp, "%Ix", index); +#else + cp += sprintf(cp, "%zx", index); +#endif +#endif while (target->b_base) { - *cp++ = target->b_index + '0'; + bytes_left = sizeof(string) - (cp - string) - 1; + /* Hex format needs 2 characters per byte */ + if (bytes_left < sizeof(Py_ssize_t) * 2) { + PyErr_SetString(PyExc_ValueError, + "ctypes object structure too deep"); + return NULL; + } +#if (PY_VERSION_HEX < 0x02050000) + cp += sprintf(cp, ":%x", (int)target->b_index); +#else +#ifdef MS_WIN32 + cp += sprintf(cp, ":%Ix", (size_t)target->b_index); +#else + cp += sprintf(cp, ":%zx", (size_t)target->b_index); +#endif +#endif target = target->b_base; } return PyString_FromStringAndSize(string, cp-string); } -/* Keep a reference to 'keep' in the 'target', at index 'index' */ + /* - * KeepRef travels the target's b_base pointer down to the root, - * building a sequence of indexes during the path. The indexes, which are a - * couple of small integers, are used to build a byte string usable as - * key int the root object's _objects dict. + * Keep a reference to 'keep' in the 'target', at index 'index'. + * + * If 'keep' is None, do nothing. + * + * Otherwise create a dictionary (if it does not yet exist) id the root + * objects 'b_objects' item, which will store the 'keep' object under a unique + * key. + * + * The unique_key helper travels the target's b_base pointer down to the root, + * building a string containing hex-formatted indexes found during traversal, + * separated by colons. + * + * The index tuple is used as a key into the root object's b_objects dict. + * + * Note: This function steals a refcount of the third argument, even if it + * fails! */ static int KeepRef(CDataObject *target, Py_ssize_t index, PyObject *keep) @@ -1846,6 +1893,10 @@ return 0; } key = unique_key(target, index); + if (key == NULL) { + Py_DECREF(keep); + return -1; + } result = PyDict_SetItem(ob->b_objects, key, keep); Py_DECREF(key); Py_DECREF(keep); @@ -2611,11 +2662,11 @@ *(void **)self->b_ptr = address; + Py_INCREF((PyObject *)dll); /* for KeepRef */ if (-1 == KeepRef((CDataObject *)self, 0, dll)) { Py_DECREF((PyObject *)self); return NULL; } - Py_INCREF((PyObject *)dll); /* for KeepRef above */ Py_INCREF(self); self->callable = (PyObject *)self; @@ -2751,11 +2802,11 @@ correctly... */ + Py_INCREF((PyObject *)self); /* for KeepRef */ if (-1 == KeepRef((CDataObject *)self, 0, (PyObject *)self)) { Py_DECREF((PyObject *)self); return NULL; } - Py_INCREF((PyObject *)self); /* for KeepRef above */ return (PyObject *)self; } @@ -3520,7 +3571,7 @@ int offset, size; StgDictObject *stgdict; - if (index < 0 || index >= self->b_length) { + if (self->b_length == 0 || index < 0 || (self->b_length > 1 && index >= self->b_length)) { PyErr_SetString(PyExc_IndexError, "invalid index"); return NULL; @@ -3549,11 +3600,11 @@ if (ilow < 0) ilow = 0; - else if (ilow > self->b_length) + else if (ilow > self->b_length && self->b_length != 1) ilow = self->b_length; if (ihigh < ilow) ihigh = ilow; - else if (ihigh > self->b_length) + else if (ihigh > self->b_length && self->b_length != 1) ihigh = self->b_length; len = ihigh - ilow; @@ -3596,7 +3647,8 @@ } stgdict = PyObject_stgdict((PyObject *)self); - if (index < 0 || index >= stgdict->length) { + if (self->b_length == 0 || index < 0 + || (self->b_length > 1 && index >= self->b_length)) { PyErr_SetString(PyExc_IndexError, "invalid index"); return -1; @@ -3623,17 +3675,19 @@ if (ilow < 0) ilow = 0; - else if (ilow > self->b_length) + else if (ilow > self->b_length && self->b_length != 1) ilow = self->b_length; + if (ihigh < 0) ihigh = 0; + if (ihigh < ilow) ihigh = ilow; - else if (ihigh > self->b_length) + else if (ihigh > self->b_length && self->b_length != 1) ihigh = self->b_length; len = PySequence_Length(value); - if (len != ihigh - ilow) { + if (self->b_length != 1 && len != ihigh - ilow) { PyErr_SetString(PyExc_ValueError, "Can only assign sequence of same size"); return -1; @@ -4020,7 +4074,8 @@ Pointer_item(PyObject *_self, Py_ssize_t index) { CDataObject *self = (CDataObject *)_self; - int size, offset; + int size; + Py_ssize_t offset; StgDictObject *stgdict, *itemdict; PyObject *proto; @@ -4030,9 +4085,9 @@ return NULL; } - stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); + assert(stgdict->proto); proto = stgdict->proto; /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ @@ -4040,7 +4095,7 @@ size = itemdict->size; offset = index * itemdict->size; - return CData_get(stgdict->proto, stgdict->getfunc, (PyObject *)self, + return CData_get(proto, stgdict->getfunc, (PyObject *)self, index, size, (*(char **)self->b_ptr) + offset); } @@ -4049,7 +4104,9 @@ { CDataObject *self = (CDataObject *)_self; int size; - StgDictObject *stgdict; + Py_ssize_t offset; + StgDictObject *stgdict, *itemdict; + PyObject *proto; if (value == NULL) { PyErr_SetString(PyExc_TypeError, @@ -4064,16 +4121,17 @@ } stgdict = PyObject_stgdict((PyObject *)self); - if (index != 0) { - PyErr_SetString(PyExc_IndexError, - "invalid index"); - return -1; - } - size = stgdict->size / stgdict->length; + assert(stgdict); + assert(stgdict->proto); - /* XXXXX Make sure proto is NOT NULL! */ - return CData_set((PyObject *)self, stgdict->proto, stgdict->setfunc, value, - index, size, *(void **)self->b_ptr); + proto = stgdict->proto; + /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ + itemdict = PyType_stgdict(proto); + size = itemdict->size; + offset = index * itemdict->size; + + return CData_set((PyObject *)self, proto, stgdict->setfunc, value, + index, size, (*(char **)self->b_ptr) + offset); } static PyObject * @@ -4090,8 +4148,8 @@ stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); return CData_FromBaseObj(stgdict->proto, - (PyObject *)self, 0, - *(void **)self->b_ptr); + (PyObject *)self, 0, + *(void **)self->b_ptr); } static int @@ -4439,7 +4497,7 @@ } static PyObject * -cast(void *ptr, PyObject *ctype) +cast(void *ptr, PyObject *src, PyObject *ctype) { CDataObject *result; if (0 == cast_check_pointertype(ctype)) @@ -4447,6 +4505,36 @@ result = (CDataObject *)PyObject_CallFunctionObjArgs(ctype, NULL); if (result == NULL) return NULL; + + /* + The casted objects '_objects' member: + + It must certainly contain the source objects one. + It must contain the source object itself. + */ + if (CDataObject_Check(src)) { + CDataObject *obj = (CDataObject *)src; + /* CData_GetContainer will initialize src.b_objects, we need + this so it can be shared */ + CData_GetContainer(obj); + /* But we need a dictionary! */ + if (obj->b_objects == Py_None) { + Py_DECREF(Py_None); + obj->b_objects = PyDict_New(); + } + Py_INCREF(obj->b_objects); + result->b_objects = obj->b_objects; + if (result->b_objects) { + PyObject *index = PyLong_FromVoidPtr((void *)src); + int rc; + if (index == NULL) + return NULL; + rc = PyDict_SetItem(result->b_objects, index, src); + Py_DECREF(index); + if (rc == -1) + return NULL; + } + } /* Should we assert that result is a pointer type? */ memcpy(result->b_ptr, &ptr, sizeof(void *)); return (PyObject *)result; @@ -4581,7 +4669,7 @@ #endif PyModule_AddObject(m, "FUNCFLAG_CDECL", PyInt_FromLong(FUNCFLAG_CDECL)); PyModule_AddObject(m, "FUNCFLAG_PYTHONAPI", PyInt_FromLong(FUNCFLAG_PYTHONAPI)); - PyModule_AddStringConstant(m, "__version__", "0.9.9.6"); + PyModule_AddStringConstant(m, "__version__", "0.9.9.7"); PyModule_AddObject(m, "_memmove_addr", PyLong_FromVoidPtr(memmove)); PyModule_AddObject(m, "_memset_addr", PyLong_FromVoidPtr(memset)); Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Sat Jun 10 21:51:46 2006 @@ -1444,7 +1444,64 @@ } #endif +static PyObject * +resize(PyObject *self, PyObject *args) +{ + CDataObject *obj; + StgDictObject *dict; + Py_ssize_t size; + + if (!PyArg_ParseTuple(args, +#if (PY_VERSION_HEX < 0x02050000) + "Oi:resize", +#else + "On:resize", +#endif + (PyObject *)&obj, &size)) + return NULL; + + dict = PyObject_stgdict((PyObject *)obj); + if (dict == NULL) { + PyErr_SetString(PyExc_TypeError, + "excepted ctypes instance"); + return NULL; + } + if (size < dict->size) { + PyErr_Format(PyExc_ValueError, + "minimum size is %d", dict->size); + return NULL; + } + if (obj->b_needsfree == 0) { + PyErr_Format(PyExc_ValueError, + "Memory cannot be resized because this object doesn't own it"); + return NULL; + } + if (size <= sizeof(obj->b_value)) { + /* internal default buffer is large enough */ + obj->b_size = size; + goto done; + } + if (obj->b_size <= sizeof(obj->b_value)) { + /* We are currently using the objects default buffer, but it + isn't large enough any more. */ + void *ptr = PyMem_Malloc(size); + if (ptr == NULL) + return PyErr_NoMemory(); + memset(ptr, 0, size); + memmove(ptr, obj->b_ptr, obj->b_size); + obj->b_ptr = ptr; + obj->b_size = size; + } else { + obj->b_ptr = PyMem_Realloc(obj->b_ptr, size); + obj->b_size = size; + } + done: + Py_INCREF(Py_None); + return Py_None; +} + PyMethodDef module_methods[] = { + {"resize", resize, METH_VARARGS, "Resize the memory buffer of a ctypes instance"}, #ifdef CTYPES_UNICODE {"set_conversion_mode", set_conversion_mode, METH_VARARGS, set_conversion_mode_doc}, #endif Modified: python/trunk/Modules/_ctypes/ctypes.h ============================================================================== --- python/trunk/Modules/_ctypes/ctypes.h (original) +++ python/trunk/Modules/_ctypes/ctypes.h Sat Jun 10 21:51:46 2006 @@ -59,7 +59,7 @@ Py_ssize_t b_length; /* number of references we need */ Py_ssize_t b_index; /* index of this object into base's b_object list */ - PyObject *b_objects; /* list of references we need to keep */ + PyObject *b_objects; /* dictionary of references we need to keep, or Py_None */ union value b_value; }; @@ -181,6 +181,7 @@ PyObject *proto; /* a type or NULL */ GETFUNC getfunc; /* getter function if proto is NULL */ SETFUNC setfunc; /* setter function if proto is NULL */ + int anonymous; } CFieldObject; /* A subclass of PyDictObject, used as the instance dictionary of ctypes Modified: python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S ============================================================================== --- python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S (original) +++ python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S Sat Jun 10 21:51:46 2006 @@ -193,3 +193,198 @@ #endif /* ifndef __x86_64__ */ #endif /* defined __i386__ */ +#ifdef __i386__ +/* ----------------------------------------------------------------------- + darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. + + X86 Foreign Function Interface + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + ----------------------------------------------------------------------- */ + +/* + * This file is based on sysv.S and then hacked up by Ronald who hasn't done + * assembly programming in 8 years. + */ + +#ifndef __x86_64__ + +#define LIBFFI_ASM +#include +#include + +.text + +.globl _ffi_prep_args + +.align 4 +.globl _ffi_call_SYSV + +_ffi_call_SYSV: +.LFB1: + pushl %ebp +.LCFI0: + movl %esp,%ebp +.LCFI1: + /* Make room for all of the new args. */ + movl 16(%ebp),%ecx + subl %ecx,%esp + + movl %esp,%eax + + /* Place all of the ffi_prep_args in position */ + pushl 12(%ebp) + pushl %eax + call *8(%ebp) + + /* Return stack to previous state and call the function */ + addl $8,%esp + + call *28(%ebp) + + /* Remove the space we pushed for the args */ + movl 16(%ebp),%ecx + addl %ecx,%esp + + /* Load %ecx with the return type code */ + movl 20(%ebp),%ecx + + /* If the return value pointer is NULL, assume no return value. */ + cmpl $0,24(%ebp) + jne retint + + /* Even if there is no space for the return value, we are + obliged to handle floating-point values. */ + cmpl $FFI_TYPE_FLOAT,%ecx + jne noretval + fstp %st(0) + + jmp epilogue + +retint: + cmpl $FFI_TYPE_INT,%ecx + jne retfloat + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + jmp epilogue + +retfloat: + cmpl $FFI_TYPE_FLOAT,%ecx + jne retdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstps (%ecx) + jmp epilogue + +retdouble: + cmpl $FFI_TYPE_DOUBLE,%ecx + jne retlongdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpl (%ecx) + jmp epilogue + +retlongdouble: + cmpl $FFI_TYPE_LONGDOUBLE,%ecx + jne retint64 + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpt (%ecx) + jmp epilogue + +retint64: + cmpl $FFI_TYPE_SINT64,%ecx + jne retstruct + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + movl %edx,4(%ecx) + +retstruct: + /* Nothing to do! */ + +noretval: +epilogue: + movl %ebp,%esp + popl %ebp + ret +.LFE1: +.ffi_call_SYSV_end: +#if 0 + .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV +#endif + +#if 0 + .section .eh_frame,EH_FRAME_FLAGS, at progbits +.Lframe1: + .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ +.LSCIE1: + .long 0x0 /* CIE Identifier Tag */ + .byte 0x1 /* CIE Version */ +#ifdef __PIC__ + .ascii "zR\0" /* CIE Augmentation */ +#else + .ascii "\0" /* CIE Augmentation */ +#endif + .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ + .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ + .byte 0x8 /* CIE RA Column */ +#ifdef __PIC__ + .byte 0x1 /* .uleb128 0x1; Augmentation size */ + .byte 0x1b /* FDE Encoding (pcrel sdata4) */ +#endif + .byte 0xc /* DW_CFA_def_cfa */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x88 /* DW_CFA_offset, column 0x8 */ + .byte 0x1 /* .uleb128 0x1 */ + .align 4 +.LECIE1: +.LSFDE1: + .long .LEFDE1-.LASFDE1 /* FDE Length */ +.LASFDE1: + .long .LASFDE1-.Lframe1 /* FDE CIE offset */ +#ifdef __PIC__ + .long .LFB1-. /* FDE initial location */ +#else + .long .LFB1 /* FDE initial location */ +#endif + .long .LFE1-.LFB1 /* FDE address range */ +#ifdef __PIC__ + .byte 0x0 /* .uleb128 0x0; Augmentation size */ +#endif + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI0-.LFB1 + .byte 0xe /* DW_CFA_def_cfa_offset */ + .byte 0x8 /* .uleb128 0x8 */ + .byte 0x85 /* DW_CFA_offset, column 0x5 */ + .byte 0x2 /* .uleb128 0x2 */ + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI1-.LCFI0 + .byte 0xd /* DW_CFA_def_cfa_register */ + .byte 0x5 /* .uleb128 0x5 */ + .align 4 +.LEFDE1: +#endif + +#endif /* ifndef __x86_64__ */ + +#endif /* defined __i386__ */ Deleted: /python/trunk/Modules/_ctypes/libffi_msvc/mingwin32.S ============================================================================== --- /python/trunk/Modules/_ctypes/libffi_msvc/mingwin32.S Sat Jun 10 21:51:46 2006 +++ (empty file) @@ -1,228 +0,0 @@ -/* ----------------------------------------------------------------------- - win32.S - Copyright (c) 1996, 1998, 2001, 2002 Red Hat, Inc. - Copyright (c) 2001 John Beniton - Copyright (c) 2002 Ranjit Mathew - - - X86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#define LIBFFI_ASM -#include -#include - -.text - -.globl ffi_prep_args - - # This assumes we are using gas. - .balign 16 -.globl _ffi_call_SYSV - -_ffi_call_SYSV: - pushl %ebp - movl %esp,%ebp - - # Make room for all of the new args. - movl 16(%ebp),%ecx - subl %ecx,%esp - - movl %esp,%eax - - # Place all of the ffi_prep_args in position - pushl 12(%ebp) - pushl %eax - call *8(%ebp) - - # Return stack to previous state and call the function - addl $8,%esp - - # FIXME: Align the stack to a 128-bit boundary to avoid - # potential performance hits. - - call *28(%ebp) - - # Remove the space we pushed for the args - movl 16(%ebp),%ecx - addl %ecx,%esp - - # Load %ecx with the return type code - movl 20(%ebp),%ecx - - # If the return value pointer is NULL, assume no return value. - cmpl $0,24(%ebp) - jne retint - - # Even if there is no space for the return value, we are - # obliged to handle floating-point values. - cmpl $2,%ecx # Float_type - jne noretval - fstp %st(0) - - jmp epilogue - -retint: - cmpl $1,%ecx # Int_type - jne retfloat - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - jmp epilogue - -retfloat: - cmpl $2,%ecx # Float_type - jne retdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstps (%ecx) - jmp epilogue - -retdouble: - cmpl $3,%ecx # Double_type - jne retlongdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpl (%ecx) - jmp epilogue - -retlongdouble: - cmpl $4,%ecx # Longdouble_type - jne retint64 - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpt (%ecx) - jmp epilogue - -retint64: - cmpl $12,%ecx # SINT64_type - jne retstruct - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - movl %edx,4(%ecx) - -retstruct: - # Nothing to do! - -noretval: -epilogue: - movl %ebp,%esp - popl %ebp - ret - -.ffi_call_SYSV_end: - - # This assumes we are using gas. - .balign 16 -.globl _ffi_call_STDCALL - -_ffi_call_STDCALL: - pushl %ebp - movl %esp,%ebp - - # Make room for all of the new args. - movl 16(%ebp),%ecx - subl %ecx,%esp - - movl %esp,%eax - - # Place all of the ffi_prep_args in position - pushl 12(%ebp) - pushl %eax - call *8(%ebp) - - # Return stack to previous state and call the function - addl $8,%esp - - # FIXME: Align the stack to a 128-bit boundary to avoid - # potential performance hits. - - call *28(%ebp) - - # stdcall functions pop arguments off the stack themselves - - # Load %ecx with the return type code - movl 20(%ebp),%ecx - - # If the return value pointer is NULL, assume no return value. - cmpl $0,24(%ebp) - jne sc_retint - - # Even if there is no space for the return value, we are - # obliged to handle floating-point values. - cmpl $2,%ecx # Float_type - jne sc_noretval - fstp %st(0) - - jmp sc_epilogue - -sc_retint: - cmpl $1,%ecx # Int_type - jne sc_retfloat - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - jmp sc_epilogue - -sc_retfloat: - cmpl $2,%ecx # Float_type - jne sc_retdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstps (%ecx) - jmp sc_epilogue - -sc_retdouble: - cmpl $2,%ecx # Double_type - jne sc_retlongdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpl (%ecx) - jmp sc_epilogue - -sc_retlongdouble: - cmpl $4,%ecx # Longdouble_type - jne sc_retint64 - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpt (%ecx) - jmp sc_epilogue - -sc_retint64: - cmpl $12,%ecx # SINT64_Type - jne sc_retstruct - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - movl %edx,4(%ecx) - -sc_retstruct: - # Nothing to do! - -sc_noretval: -sc_epilogue: - movl %ebp,%esp - popl %ebp - ret - -.ffi_call_STDCALL_end: - Modified: python/trunk/Modules/_ctypes/stgdict.c ============================================================================== --- python/trunk/Modules/_ctypes/stgdict.c (original) +++ python/trunk/Modules/_ctypes/stgdict.c Sat Jun 10 21:51:46 2006 @@ -142,30 +142,129 @@ return PyType_stgdict((PyObject *)self->ob_type); } -#if 0 -/* work in progress: anonymous structure fields */ -int -GetFields(PyObject *desc, int *pindex, int *psize, int *poffset, int *palign, int pack); +/* descr is the descriptor for a field marked as anonymous. Get all the + _fields_ descriptors from descr->proto, create new descriptors with offset + and index adjusted, and stuff them into type. + */ +static int +MakeFields(PyObject *type, CFieldObject *descr, + Py_ssize_t index, Py_ssize_t offset) +{ + Py_ssize_t i; + PyObject *fields; + PyObject *fieldlist; + + fields = PyObject_GetAttrString(descr->proto, "_fields_"); + if (fields == NULL) + return -1; + fieldlist = PySequence_Fast(fields, "_fields_ must be a sequence"); + Py_DECREF(fields); + if (fieldlist == NULL) + return -1; + + for (i = 0; i < PySequence_Fast_GET_SIZE(fieldlist); ++i) { + PyObject *pair = PySequence_Fast_GET_ITEM(fieldlist, i); /* borrowed */ + PyObject *fname, *ftype; + CFieldObject *fdescr; + CFieldObject *new_descr; + // Convert to PyArg_UnpackTuple... + if (!PyArg_ParseTuple(pair, "OO", &fname, &ftype)) { + Py_DECREF(fieldlist); + return -1; + } + fdescr = (CFieldObject *)PyObject_GetAttr(descr->proto, fname); + if (fdescr == NULL) { + Py_DECREF(fieldlist); + return -1; + } + if (fdescr->ob_type != &CField_Type) { + PyErr_SetString(PyExc_TypeError, "unexpected type"); + Py_DECREF(fdescr); + Py_DECREF(fieldlist); + return -1; + } + if (fdescr->anonymous) { + int rc = MakeFields(type, fdescr, + index + fdescr->index, + offset + fdescr->offset); + Py_DECREF(fdescr); + if (rc == -1) { + Py_DECREF(fieldlist); + return -1; + } + continue; + } + new_descr = (CFieldObject *)PyObject_CallObject((PyObject *)&CField_Type, NULL); + assert(new_descr->ob_type == &CField_Type); + if (new_descr == NULL) { + Py_DECREF(fdescr); + Py_DECREF(fieldlist); + return -1; + } + new_descr->size = fdescr->size; + new_descr->offset = fdescr->offset + offset; + new_descr->index = fdescr->index + index; + new_descr->proto = fdescr->proto; + Py_XINCREF(new_descr->proto); + new_descr->getfunc = fdescr->getfunc; + new_descr->setfunc = fdescr->setfunc; + + Py_DECREF(fdescr); + + if (-1 == PyObject_SetAttr(type, fname, (PyObject *)new_descr)) { + Py_DECREF(fieldlist); + Py_DECREF(new_descr); + return -1; + } + Py_DECREF(new_descr); + } + Py_DECREF(fieldlist); + return 0; +} +/* Iterate over the names in the type's _anonymous_ attribute, if present, + */ +static int +MakeAnonFields(PyObject *type) { - int i; - PyObject *tuples = PyObject_GetAttrString(desc, "_fields_"); - if (tuples == NULL) - return -1; - if (!PyTuple_Check(tuples)) - return -1; /* leak */ - for (i = 0; i < PyTuple_GET_SIZE(tuples); ++i) { - char *fname; - PyObject *dummy; - CFieldObject *field; - PyObject *pair = PyTuple_GET_ITEM(tuples, i); - if (!PyArg_ParseTuple(pair, "sO", &fname, &dummy)) - return -1; /* leak */ - field = PyObject_GetAttrString(desc, fname); - Py_DECREF(field); + PyObject *anon; + PyObject *anon_names; + Py_ssize_t i; + + anon = PyObject_GetAttrString(type, "_anonymous_"); + if (anon == NULL) { + PyErr_Clear(); + return 0; } + anon_names = PySequence_Fast(anon, "_anonymous_ must be a sequence"); + Py_DECREF(anon); + if (anon_names == NULL) + return -1; + + for (i = 0; i < PySequence_Fast_GET_SIZE(anon_names); ++i) { + PyObject *fname = PySequence_Fast_GET_ITEM(anon_names, i); /* borrowed */ + CFieldObject *descr = (CFieldObject *)PyObject_GetAttr(type, fname); + if (descr == NULL) { + Py_DECREF(anon_names); + return -1; + } + assert(descr->ob_type == &CField_Type); + descr->anonymous = 1; + + /* descr is in the field descriptor. */ + if (-1 == MakeFields(type, (CFieldObject *)descr, + ((CFieldObject *)descr)->index, + ((CFieldObject *)descr)->offset)) { + Py_DECREF(descr); + Py_DECREF(anon_names); + return -1; + } + Py_DECREF(descr); + } + + Py_DECREF(anon_names); + return 0; } -#endif /* Retrive the (optional) _pack_ attribute from a type, the _fields_ attribute, @@ -368,5 +467,5 @@ stgdict->size = size; stgdict->align = total_align; stgdict->length = len; /* ADD ffi_ofs? */ - return 0; + return MakeAnonFields(type); } From python-checkins at python.org Sat Jun 10 21:55:37 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 21:55:37 +0200 (CEST) Subject: [Python-checkins] r46825 - in python/trunk/Lib/ctypes: __init__.py test/test_anon.py test/test_cast.py test/test_keeprefs.py test/test_objects.py test/test_slicing.py test/test_structures.py test/test_varsize_struct.py Message-ID: <20060610195537.860A91E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 21:55:36 2006 New Revision: 46825 Added: python/trunk/Lib/ctypes/test/test_anon.py (contents, props changed) python/trunk/Lib/ctypes/test/test_objects.py (contents, props changed) python/trunk/Lib/ctypes/test/test_varsize_struct.py (contents, props changed) Modified: python/trunk/Lib/ctypes/__init__.py python/trunk/Lib/ctypes/test/test_cast.py python/trunk/Lib/ctypes/test/test_keeprefs.py python/trunk/Lib/ctypes/test/test_slicing.py python/trunk/Lib/ctypes/test/test_structures.py Log: Upgrade to ctypes version 0.9.9.7. Summary of changes: - support for 'variable sized' data - support for anonymous structure/union fields - fix severe bug with certain arrays or structures containing more than 256 fields Modified: python/trunk/Lib/ctypes/__init__.py ============================================================================== --- python/trunk/Lib/ctypes/__init__.py (original) +++ python/trunk/Lib/ctypes/__init__.py Sat Jun 10 21:55:36 2006 @@ -1,9 +1,8 @@ """create and manipulate C data types in Python""" import os as _os, sys as _sys -from itertools import chain as _chain -__version__ = "0.9.9.6" +__version__ = "0.9.9.7" from _ctypes import Union, Structure, Array from _ctypes import _Pointer @@ -111,7 +110,7 @@ elif _os.name == "posix": from _ctypes import dlopen as _dlopen -from _ctypes import sizeof, byref, addressof, alignment +from _ctypes import sizeof, byref, addressof, alignment, resize from _ctypes import _SimpleCData class py_object(_SimpleCData): @@ -293,7 +292,7 @@ return "<%s '%s', handle %x at %x>" % \ (self.__class__.__name__, self._name, (self._handle & (_sys.maxint*2 + 1)), - id(self)) + id(self) & (_sys.maxint*2 + 1)) def __getattr__(self, name): if name.startswith('__') and name.endswith('__'): @@ -419,12 +418,10 @@ _restype_ = restype _flags_ = _FUNCFLAG_CDECL | _FUNCFLAG_PYTHONAPI return CFunctionType -_cast = PYFUNCTYPE(py_object, c_void_p, py_object)(_cast_addr) +_cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr) def cast(obj, typ): - result = _cast(obj, typ) - result.__keepref = obj - return result + return _cast(obj, obj, typ) _string_at = CFUNCTYPE(py_object, c_void_p, c_int)(_string_at_addr) def string_at(ptr, size=0): Added: python/trunk/Lib/ctypes/test/test_anon.py ============================================================================== --- (empty file) +++ python/trunk/Lib/ctypes/test/test_anon.py Sat Jun 10 21:55:36 2006 @@ -0,0 +1,60 @@ +import unittest +from ctypes import * + +class AnonTest(unittest.TestCase): + + def test_anon(self): + class ANON(Union): + _fields_ = [("a", c_int), + ("b", c_int)] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + + self.failUnlessEqual(ANON.a.offset, 0) + self.failUnlessEqual(ANON.b.offset, 0) + + def test_anon_nonseq(self): + # TypeError: _anonymous_ must be a sequence + self.failUnlessRaises(TypeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], "_anonymous_": 42})) + + def test_anon_nonmember(self): + # AttributeError: type object 'Name' has no attribute 'x' + self.failUnlessRaises(AttributeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], + "_anonymous_": ["x"]})) + + def test_nested(self): + class ANON_S(Structure): + _fields_ = [("a", c_int)] + + class ANON_U(Union): + _fields_ = [("_", ANON_S), + ("b", c_int)] + _anonymous_ = ["_"] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON_U), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.x.offset, 0) + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + self.failUnlessEqual(Y._.offset, sizeof(c_int)) + self.failUnlessEqual(Y.y.offset, sizeof(c_int) * 2) + +if __name__ == "__main__": + unittest.main() Modified: python/trunk/Lib/ctypes/test/test_cast.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_cast.py (original) +++ python/trunk/Lib/ctypes/test/test_cast.py Sat Jun 10 21:55:36 2006 @@ -30,17 +30,32 @@ ptr = cast(address, POINTER(c_int)) self.failUnlessEqual([ptr[i] for i in range(3)], [42, 17, 2]) + def test_p2a_objects(self): + array = (c_char_p * 5)() + self.failUnlessEqual(array._objects, None) + array[0] = "foo bar" + self.failUnlessEqual(array._objects, {'0': "foo bar"}) - def test_ptr2array(self): - array = (c_int * 3)(42, 17, 2) + p = cast(array, POINTER(c_char_p)) + # array and p share a common _objects attribute + self.failUnless(p._objects is array._objects) + self.failUnlessEqual(array._objects, {'0': "foo bar", id(array): array}) + p[0] = "spam spam" + self.failUnlessEqual(p._objects, {'0': "spam spam", id(array): array}) + self.failUnless(array._objects is p._objects) + p[1] = "foo bar" + self.failUnlessEqual(p._objects, {'1': 'foo bar', '0': "spam spam", id(array): array}) + self.failUnless(array._objects is p._objects) - from sys import getrefcount - - before = getrefcount(array) - ptr = cast(array, POINTER(c_int)) - self.failUnlessEqual(getrefcount(array), before + 1) - del ptr - self.failUnlessEqual(getrefcount(array), before) + def test_other(self): + p = cast((c_int * 4)(1, 2, 3, 4), POINTER(c_int)) + self.failUnlessEqual(p[:4], [1,2, 3, 4]) + c_int() + self.failUnlessEqual(p[:4], [1, 2, 3, 4]) + p[2] = 96 + self.failUnlessEqual(p[:4], [1, 2, 96, 4]) + c_int() + self.failUnlessEqual(p[:4], [1, 2, 96, 4]) if __name__ == "__main__": unittest.main() Modified: python/trunk/Lib/ctypes/test/test_keeprefs.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_keeprefs.py (original) +++ python/trunk/Lib/ctypes/test/test_keeprefs.py Sat Jun 10 21:55:36 2006 @@ -61,6 +61,8 @@ r.ul.x = 22 r.ul.y = 44 self.assertEquals(r._objects, {'0': {}}) + r.lr = POINT() + self.assertEquals(r._objects, {'0': {}, '1': {}}) class ArrayTestCase(unittest.TestCase): def test_cint_array(self): @@ -86,9 +88,10 @@ self.assertEquals(x._objects, {'1': {}}) class PointerTestCase(unittest.TestCase): - def X_test_p_cint(self): - x = pointer(c_int(42)) - print x._objects + def test_p_cint(self): + i = c_int(42) + x = pointer(i) + self.failUnlessEqual(x._objects, {'1': i}) class DeletePointerTestCase(unittest.TestCase): def X_test(self): Added: python/trunk/Lib/ctypes/test/test_objects.py ============================================================================== --- (empty file) +++ python/trunk/Lib/ctypes/test/test_objects.py Sat Jun 10 21:55:36 2006 @@ -0,0 +1,66 @@ +r''' +This tests the '_objects' attribute of ctypes instances. '_objects' +holds references to objects that must be kept alive as long as the +ctypes instance, to make sure that the memory buffer is valid. + +WARNING: The '_objects' attribute is exposed ONLY for debugging ctypes itself, +it MUST NEVER BE MODIFIED! + +'_objects' is initialized to a dictionary on first use, before that it +is None. + +Here is an array of string pointers: + +>>> from ctypes import * +>>> array = (c_char_p * 5)() +>>> print array._objects +None +>>> + +The memory block stores pointers to strings, and the strings itself +assigned from Python must be kept. + +>>> array[4] = 'foo bar' +>>> array._objects +{'4': 'foo bar'} +>>> array[4] +'foo bar' +>>> + +It gets more complicated when the ctypes instance itself is contained +in a 'base' object. + +>>> class X(Structure): +... _fields_ = [("x", c_int), ("y", c_int), ("array", c_char_p * 5)] +... +>>> x = X() +>>> print x._objects +None +>>> + +The'array' attribute of the 'x' object shares part of the memory buffer +of 'x' ('_b_base_' is either None, or the root object owning the memory block): + +>>> print x.array._b_base_ # doctest: +ELLIPSIS + +>>> + +>>> x.array[0] = 'spam spam spam' +>>> x._objects +{'0:2': 'spam spam spam'} +>>> x.array._b_base_._objects +{'0:2': 'spam spam spam'} +>>> + +''' + +import unittest, doctest + +import ctypes.test.test_objects + +class TestCase(unittest.TestCase): + def test(self): + doctest.testmod(ctypes.test.test_objects) + +if __name__ == '__main__': + doctest.testmod(ctypes.test.test_objects) Modified: python/trunk/Lib/ctypes/test/test_slicing.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_slicing.py (original) +++ python/trunk/Lib/ctypes/test/test_slicing.py Sat Jun 10 21:55:36 2006 @@ -35,7 +35,7 @@ self.assertRaises(ValueError, setslice, a, 0, 5, range(32)) def test_char_ptr(self): - s = "abcdefghijklmnopqrstuvwxyz\0" + s = "abcdefghijklmnopqrstuvwxyz" dll = CDLL(_ctypes_test.__file__) dll.my_strdup.restype = POINTER(c_char) @@ -50,9 +50,31 @@ dll.my_strdup.restype = POINTER(c_byte) res = dll.my_strdup(s) - self.failUnlessEqual(res[:len(s)-1], range(ord("a"), ord("z")+1)) + self.failUnlessEqual(res[:len(s)], range(ord("a"), ord("z")+1)) dll.my_free(res) + def test_char_ptr_with_free(self): + dll = CDLL(_ctypes_test.__file__) + s = "abcdefghijklmnopqrstuvwxyz" + + class allocated_c_char_p(c_char_p): + pass + + dll.my_free.restype = None + def errcheck(result, func, args): + retval = result.value + dll.my_free(result) + return retval + + dll.my_strdup.restype = allocated_c_char_p + dll.my_strdup.errcheck = errcheck + try: + res = dll.my_strdup(s) + self.failUnlessEqual(res, s) + finally: + del dll.my_strdup.errcheck + + def test_char_array(self): s = "abcdefghijklmnopqrstuvwxyz\0" Modified: python/trunk/Lib/ctypes/test/test_structures.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_structures.py (original) +++ python/trunk/Lib/ctypes/test/test_structures.py Sat Jun 10 21:55:36 2006 @@ -138,8 +138,8 @@ self.failUnlessEqual(X.y.size, sizeof(c_char)) # readonly - self.assertRaises(AttributeError, setattr, X.x, "offset", 92) - self.assertRaises(AttributeError, setattr, X.x, "size", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "offset", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "size", 92) class X(Union): _fields_ = [("x", c_int), @@ -152,8 +152,8 @@ self.failUnlessEqual(X.y.size, sizeof(c_char)) # readonly - self.assertRaises(AttributeError, setattr, X.x, "offset", 92) - self.assertRaises(AttributeError, setattr, X.x, "size", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "offset", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "size", 92) # XXX Should we check nested data types also? # offset is always relative to the class... @@ -298,7 +298,7 @@ "expected string or Unicode object, int found") else: self.failUnlessEqual(msg, - "(Phone) TypeError: " + "(Phone) exceptions.TypeError: " "expected string or Unicode object, int found") cls, msg = self.get_except(Person, "Someone", ("a", "b", "c")) @@ -307,7 +307,7 @@ self.failUnlessEqual(msg, "(Phone) : too many initializers") else: - self.failUnlessEqual(msg, "(Phone) ValueError: too many initializers") + self.failUnlessEqual(msg, "(Phone) exceptions.ValueError: too many initializers") def get_except(self, func, *args): Added: python/trunk/Lib/ctypes/test/test_varsize_struct.py ============================================================================== --- (empty file) +++ python/trunk/Lib/ctypes/test/test_varsize_struct.py Sat Jun 10 21:55:36 2006 @@ -0,0 +1,115 @@ +from ctypes import * +import unittest + +class VarSizeTest(unittest.TestCase): + def test_resize(self): + class X(Structure): + _fields_ = [("item", c_int), + ("array", c_int * 1)] + + self.failUnlessEqual(sizeof(X), sizeof(c_int) * 2) + x = X() + x.item = 42 + x.array[0] = 100 + self.failUnlessEqual(sizeof(x), sizeof(c_int) * 2) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for 10 additional items + new_size = sizeof(X) + sizeof(c_int) * 9 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + def test_array_invalid_length(self): + # cannot create arrays with non-positive size + self.failUnlessRaises(ValueError, lambda: c_int * -1) + self.failUnlessRaises(ValueError, lambda: c_int * -3) + + def test_zerosized_array(self): + array = (c_int * 0)() + # accessing elements of zero-sized arrays raise IndexError + self.failUnlessRaises(IndexError, array.__setitem__, 0, None) + self.failUnlessRaises(IndexError, array.__getitem__, 0) + self.failUnlessRaises(IndexError, array.__setitem__, 1, None) + self.failUnlessRaises(IndexError, array.__getitem__, 1) + self.failUnlessRaises(IndexError, array.__setitem__, -1, None) + self.failUnlessRaises(IndexError, array.__getitem__, -1) + + def test_varsized_array(self): + array = (c_int * 20)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) + + # no range checking is done on arrays with size == 1 + varsize_array = (c_int * 1).from_address(addressof(array)) + + # __getitem__ + self.failUnlessEqual(varsize_array[0], 20) + self.failUnlessEqual(varsize_array[1], 21) + self.failUnlessEqual(varsize_array[2], 22) + self.failUnlessEqual(varsize_array[3], 23) + self.failUnlessEqual(varsize_array[4], 24) + self.failUnlessEqual(varsize_array[5], 25) + self.failUnlessEqual(varsize_array[6], 26) + self.failUnlessEqual(varsize_array[7], 27) + self.failUnlessEqual(varsize_array[8], 28) + self.failUnlessEqual(varsize_array[9], 29) + + # still, normal sequence of length one behaviour: + self.failUnlessEqual(varsize_array[-1], 20) + self.failUnlessRaises(IndexError, lambda: varsize_array[-2]) + # except for this one, which will raise MemoryError + self.failUnlessRaises(MemoryError, lambda: varsize_array[:]) + + # __setitem__ + varsize_array[0] = 100 + varsize_array[1] = 101 + varsize_array[2] = 102 + varsize_array[3] = 103 + varsize_array[4] = 104 + varsize_array[5] = 105 + varsize_array[6] = 106 + varsize_array[7] = 107 + varsize_array[8] = 108 + varsize_array[9] = 109 + + for i in range(10): + self.failUnlessEqual(varsize_array[i], i + 100) + self.failUnlessEqual(array[i], i + 100) + + # __getslice__ + self.failUnlessEqual(varsize_array[0:10], range(100, 110)) + self.failUnlessEqual(varsize_array[1:9], range(101, 109)) + self.failUnlessEqual(varsize_array[1:-1], []) + + # __setslice__ + varsize_array[0:10] = range(1000, 1010) + self.failUnlessEqual(varsize_array[0:10], range(1000, 1010)) + + varsize_array[1:9] = range(1001, 1009) + self.failUnlessEqual(varsize_array[1:9], range(1001, 1009)) + + def test_vararray_is_sane(self): + array = (c_int * 15)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) + + varsize_array = (c_int * 1).from_address(addressof(array)) + varsize_array[:] = [1, 2, 3, 4, 5] + + self.failUnlessEqual(array[:], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) + self.failUnlessEqual(varsize_array[0:10], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29]) + + array[:5] = [10, 11, 12, 13, 14] + self.failUnlessEqual(array[:], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) + self.failUnlessEqual(varsize_array[0:10], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29]) + +if __name__ == "__main__": + unittest.main() From python-checkins at python.org Sat Jun 10 22:01:34 2006 From: python-checkins at python.org (fred.drake) Date: Sat, 10 Jun 2006 22:01:34 +0200 (CEST) Subject: [Python-checkins] r46826 - python/trunk/Doc/ref/ref4.tex Message-ID: <20060610200134.6A3271E4020@bag.python.org> Author: fred.drake Date: Sat Jun 10 22:01:34 2006 New Revision: 46826 Modified: python/trunk/Doc/ref/ref4.tex Log: SF patch #1303595: improve description of __builtins__, explaining how it varies between __main__ and other modules, and strongly suggest not touching it but using __builtin__ if absolutely necessary Modified: python/trunk/Doc/ref/ref4.tex ============================================================================== --- python/trunk/Doc/ref/ref4.tex (original) +++ python/trunk/Doc/ref/ref4.tex Sat Jun 10 22:01:34 2006 @@ -97,10 +97,20 @@ The built-in namespace associated with the execution of a code block is actually found by looking up the name \code{__builtins__} in its global namespace; this should be a dictionary or a module (in the -latter case the module's dictionary is used). Normally, the -\code{__builtins__} namespace is the dictionary of the built-in module -\module{__builtin__} (note: no `s'). If it isn't, restricted -execution\indexii{restricted}{execution} mode is in effect. +latter case the module's dictionary is used). By default, when in the +\module{__main__} module, \code{__builtins__} is the built-in module +\module{__builtin__} (note: no `s'); when in any other module, +\code{__builtins__} is an alias for the dictionary of the +\module{__builtin__} module itself. \code{__builtins__} can be set +to a user-created dictionary to create a weak form of restricted +execution\indexii{restricted}{execution}. + +\begin{notice} + Users should not touch \code{__builtins__}; it is strictly an + implementation detail. Users wanting to override values in the + built-in namespace should \keyword{import} the \module{__builtin__} + (no `s') module and modify its attributes appropriately. +\end{notice} The namespace for a module is automatically created the first time a module is imported. The main module for a script is always called From python-checkins at python.org Sat Jun 10 22:02:59 2006 From: python-checkins at python.org (fred.drake) Date: Sat, 10 Jun 2006 22:02:59 +0200 (CEST) Subject: [Python-checkins] r46827 - python/trunk/Doc/ACKS Message-ID: <20060610200259.3B1CF1E400E@bag.python.org> Author: fred.drake Date: Sat Jun 10 22:02:58 2006 New Revision: 46827 Modified: python/trunk/Doc/ACKS Log: credit for SF patch #1303595 Modified: python/trunk/Doc/ACKS ============================================================================== --- python/trunk/Doc/ACKS (original) +++ python/trunk/Doc/ACKS Sat Jun 10 22:02:58 2006 @@ -190,6 +190,7 @@ Mats Wichmann Gerry Wiener Timothy Wild +Collin Winter Blake Winton Dan Wolfe Steven Work From python-checkins at python.org Sat Jun 10 22:04:07 2006 From: python-checkins at python.org (fred.drake) Date: Sat, 10 Jun 2006 22:04:07 +0200 (CEST) Subject: [Python-checkins] r46828 - python/branches/release24-maint/Doc/ref/ref4.tex Message-ID: <20060610200407.392A31E400C@bag.python.org> Author: fred.drake Date: Sat Jun 10 22:04:06 2006 New Revision: 46828 Modified: python/branches/release24-maint/Doc/ref/ref4.tex Log: SF patch #1303595: improve description of __builtins__, explaining how it varies between __main__ and other modules, and strongly suggest not touching it but using __builtin__ if absolutely necessary Modified: python/branches/release24-maint/Doc/ref/ref4.tex ============================================================================== --- python/branches/release24-maint/Doc/ref/ref4.tex (original) +++ python/branches/release24-maint/Doc/ref/ref4.tex Sat Jun 10 22:04:06 2006 @@ -97,10 +97,20 @@ The built-in namespace associated with the execution of a code block is actually found by looking up the name \code{__builtins__} in its global namespace; this should be a dictionary or a module (in the -latter case the module's dictionary is used). Normally, the -\code{__builtins__} namespace is the dictionary of the built-in module -\module{__builtin__} (note: no `s'). If it isn't, restricted -execution\indexii{restricted}{execution} mode is in effect. +latter case the module's dictionary is used). By default, when in the +\module{__main__} module, \code{__builtins__} is the built-in module +\module{__builtin__} (note: no `s'); when in any other module, +\code{__builtins__} is an alias for the dictionary of the +\module{__builtin__} module itself. \code{__builtins__} can be set +to a user-created dictionary to create a weak form of restricted +execution\indexii{restricted}{execution}. + +\begin{notice} + Users should not touch \code{__builtins__}; it is strictly an + implementation detail. Users wanting to override values in the + built-in namespace should \keyword{import} the \module{__builtin__} + (no `s') module and modify its attributes appropriately. +\end{notice} The namespace for a module is automatically created the first time a module is imported. The main module for a script is always called From python-checkins at python.org Sat Jun 10 22:04:26 2006 From: python-checkins at python.org (fred.drake) Date: Sat, 10 Jun 2006 22:04:26 +0200 (CEST) Subject: [Python-checkins] r46829 - python/branches/release24-maint/Doc/ACKS Message-ID: <20060610200426.29AE31E400C@bag.python.org> Author: fred.drake Date: Sat Jun 10 22:04:25 2006 New Revision: 46829 Modified: python/branches/release24-maint/Doc/ACKS Log: credit for SF patch #1303595 Modified: python/branches/release24-maint/Doc/ACKS ============================================================================== --- python/branches/release24-maint/Doc/ACKS (original) +++ python/branches/release24-maint/Doc/ACKS Sat Jun 10 22:04:25 2006 @@ -187,6 +187,7 @@ Mats Wichmann Gerry Wiener Timothy Wild +Collin Winter Blake Winton Dan Wolfe Steven Work From python-checkins at python.org Sat Jun 10 22:08:22 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 22:08:22 +0200 (CEST) Subject: [Python-checkins] r46830 - in external/ctypes: ANNOUNCE BUGS ChangeLog MANIFEST.in TODO ctypes/__init__.py ctypes/test/test_anon.py ctypes/test/test_cast.py ctypes/test/test_cfuncs.py ctypes/test/test_find.py ctypes/test/test_keeprefs.py ctypes/test/test_loading.py ctypes/test/test_objects.py ctypes/test/test_pointers.py ctypes/test/test_python_api.py ctypes/test/test_slicing.py ctypes/test/test_structures.py ctypes/test/test_varsize_struct.py docs/anatomy.txt docs/manual/callbacks.txt docs/manual/ctypes.txt docs/manual/docs.txt docs/manual/filter.py docs/manual/functions.txt docs/manual/libctypes.txt docs/manual/libraries.txt docs/manual/make.bat docs/manual/manual.html docs/manual/manual.txt docs/manual/mkpydoc.py docs/manual/reference.txt docs/manual/simple_types.txt docs/manual/struct_union.txt docs/manual/table.txt docs/manual/test-tutorial.py docs/manual/tutorial.txt docs/manual/utilities.txt docs/test-anatomy.py env.bat setup.py source/_ctypes.c source/_ctypes_test.c source/callbacks.c source/callproc.c source/cfield.c source/ctypes.h source/libffi/configure source/libffi/configure.ac source/libffi/fficonfig.h.in source/libffi/fficonfig.py.in source/libffi/src/darwin source/libffi/src/darwin/ffitarget.h source/libffi/src/powerpc/darwin.S source/libffi/src/powerpc/darwin_closure.S source/libffi/src/powerpc/ffi_darwin.c source/libffi/src/prep_cif.c source/libffi/src/x86/darwin.S source/libffi/src/x86/ffi_darwin.c source/libffi/src/x86/ffitarget.h source/stgdict.c Message-ID: <20060610200822.661CA1E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 22:08:12 2006 New Revision: 46830 Added: external/ctypes/BUGS external/ctypes/TODO external/ctypes/ctypes/test/test_anon.py external/ctypes/ctypes/test/test_objects.py external/ctypes/ctypes/test/test_varsize_struct.py external/ctypes/docs/anatomy.txt external/ctypes/docs/manual/ctypes.txt external/ctypes/docs/manual/filter.py external/ctypes/docs/test-anatomy.py external/ctypes/source/libffi/src/darwin/ external/ctypes/source/libffi/src/darwin/ffitarget.h external/ctypes/source/libffi/src/x86/darwin.S external/ctypes/source/libffi/src/x86/ffi_darwin.c Removed: external/ctypes/docs/manual/callbacks.txt external/ctypes/docs/manual/docs.txt external/ctypes/docs/manual/functions.txt external/ctypes/docs/manual/libctypes.txt external/ctypes/docs/manual/libraries.txt external/ctypes/docs/manual/manual.html external/ctypes/docs/manual/manual.txt external/ctypes/docs/manual/simple_types.txt external/ctypes/docs/manual/struct_union.txt external/ctypes/docs/manual/table.txt external/ctypes/docs/manual/utilities.txt Modified: external/ctypes/ANNOUNCE external/ctypes/ChangeLog external/ctypes/MANIFEST.in external/ctypes/ctypes/__init__.py external/ctypes/ctypes/test/test_cast.py external/ctypes/ctypes/test/test_cfuncs.py external/ctypes/ctypes/test/test_find.py external/ctypes/ctypes/test/test_keeprefs.py external/ctypes/ctypes/test/test_loading.py external/ctypes/ctypes/test/test_pointers.py external/ctypes/ctypes/test/test_python_api.py external/ctypes/ctypes/test/test_slicing.py external/ctypes/ctypes/test/test_structures.py external/ctypes/docs/manual/make.bat external/ctypes/docs/manual/mkpydoc.py external/ctypes/docs/manual/reference.txt external/ctypes/docs/manual/test-tutorial.py external/ctypes/docs/manual/tutorial.txt external/ctypes/env.bat external/ctypes/setup.py external/ctypes/source/_ctypes.c external/ctypes/source/_ctypes_test.c external/ctypes/source/callbacks.c external/ctypes/source/callproc.c external/ctypes/source/cfield.c external/ctypes/source/ctypes.h external/ctypes/source/libffi/configure external/ctypes/source/libffi/configure.ac external/ctypes/source/libffi/fficonfig.h.in external/ctypes/source/libffi/fficonfig.py.in external/ctypes/source/libffi/src/powerpc/darwin.S external/ctypes/source/libffi/src/powerpc/darwin_closure.S external/ctypes/source/libffi/src/powerpc/ffi_darwin.c external/ctypes/source/libffi/src/prep_cif.c external/ctypes/source/libffi/src/x86/ffitarget.h external/ctypes/source/stgdict.c Log: Import ctypes version 0.9.9.7. For changes see the ChangeLog file. Modified: external/ctypes/ANNOUNCE ============================================================================== --- external/ctypes/ANNOUNCE (original) +++ external/ctypes/ANNOUNCE Sat Jun 10 22:08:12 2006 @@ -1,5 +1,5 @@ -ctypes 0.9.9.6 released - Apr 20, 2006 -===================================== +ctypes 0.9.9.7 released - June 9, 2006 +====================================== Overview @@ -17,6 +17,30 @@ libffi supports this platform. +Changes in 0.9.9.7 + + Fixes for 64-bit big endian machines. + + It is now possible to read and write any index of pointer + instances (up to 0.9.9.6, reading was allowed for any index, but + writing only for index zero). + + Support for unnamed (anonymous) structure fields has been added, + the field names must be listed in the '_anonymous_' class + variable. Fields of unnamed structure fields can be accessed + directly from the parent structure. + + Enabled darwin/x86 support for libffi (Bob Ippolito and Ronald + Oussuren). + + Added support for variable sized data structures: array + types with exactly 1 element don't do bounds checking any more, + and a resize function can resize the internal memory buffer of + ctypes instances. + + Fixed a bug with certain array or structure types that contained + more than 256 elements. + Changes in 0.9.9.6 The most important change is that the old way to load libraries Added: external/ctypes/BUGS ============================================================================== --- (empty file) +++ external/ctypes/BUGS Sat Jun 10 22:08:12 2006 @@ -0,0 +1,2 @@ +- Memory leak in the _pointer_type_cache dictionary since types are + made immortal. Modified: external/ctypes/ChangeLog ============================================================================== --- external/ctypes/ChangeLog (original) +++ external/ctypes/ChangeLog Sat Jun 10 22:08:12 2006 @@ -1,3 +1,105 @@ +2006-06-10 Thomas Heller + + * Tagged release_0_9_9_7. + + * Fix build on MIPS for libffi. + +2006-06-09 Thomas Heller + + * Correctly support nested anonymous structures/union fields. + + * Remove the restriction that pointer item assignments only work + with index of zero. Another fix for the cast function to keep + needed objects alive. + + * A casted object does now reference the source objects '_objects' + member. + + * It is now possible to assign arbitrary indexes on pointer + instances. + +2006-06-08 Thomas Heller + + * Merged in changes done in Python trunk: Fixes for 64-bit big + endian machines. + +2006-06-02 Thomas Heller + + * Fixed slice assignment to variable-sized arrays, with test. + +2006-06-01 Thomas Heller + + * Added support for unnamed structure fields, they must be listed + in the '_anonymous_' class variable. + +2006-05-31 Thomas Heller + + * Merged in changes done in Python trunk, by Bob Ippolito and + Ronald Oussuren: enable darwin/x86 support for libffi and hence + ctypes (doesn't yet support --enable-universalsdk) + +2006-05-23 Thomas Heller + + * ctypes\__init__.py: Fixed a possible FutureWarning when id() + returned an integer less than zero or larger than sys.maxint, in + CDLL.__repr__. + +2006-05-19 Thomas Heller + + * docs\anatomy.txt: Add some documentation, also containing + doctest testcases, about some ctypes object internals. + + * docs\manual\tutorial.txt: Document variable-sized data types. + + * source\_ctypes.c: Allow zero-sized arrays again, although + accessing elements will always raise IndexError. + + * source\_ctypes.c: Change the way unique_key calculates the keys + into the b_objects array. The previous way was broken for arrays + or structures having more than 256 components, which led to + strange errors. + +2006-05-18 Thomas Heller + + * source\callproc.c: Fix compiler warning about comparison of + signed and unsigned values. + + * source\_ctypes.c: Correct refcount issues in the (unlikely) case + that KeepRef fails. + +2006-05-12 Thomas Heller + + * Bug fix for the cast function, so that it better keeps needed + objects alive. Prevent creation of array types with zero or + negative sizes. + + * Implemented support for variable sized data structures: array + types with exactly 1 element don't do bounds checking any more, + and a resize function can resize the internal memory buffer of + ctypes instances. + + * Some fixes to the test-suite. Free allocated memory, remove + duplicate tests, don't run unreliable tests by default. + +2006-04-28 Thomas Heller + + * docs/manual/tutorial.txt: End each code sample with '>>>', so + that doctest also works on the LaTeX version of the tutorial + (otherwise some tests would break because doctest would see + '\end{verbatim}' as expected output. + + * docs/manual/test-tutorial.py: Python 2.5a2 formats exceptions in + a different way (I'm not sure if this will be fixed or not). Add + doctest.IGNORE_EXCEPTION_DETAIL. + +2006-04-26 Thomas Heller + + * Tagged release_0_9_9_6, and released it. + +2006-04-25 Thomas Heller + + * Fix compiler warnings on Darwin, patch by Brett Canon. + 2006-04-19 Thomas Heller * docs\manual\tutorial.txt: Updates to the tutorial. Modified: external/ctypes/MANIFEST.in ============================================================================== --- external/ctypes/MANIFEST.in (original) +++ external/ctypes/MANIFEST.in Sat Jun 10 22:08:12 2006 @@ -1,5 +1,5 @@ # standard files -include ACKS ANNOUNCE ChangeLog MANIFEST MANIFEST.in LICENSE.txt NEWS.txt README* +include ACKS ANNOUNCE BUGS ChangeLog MANIFEST MANIFEST.in LICENSE.txt NEWS.txt README* # support files for development include ctypes-dev.el @@ -25,6 +25,7 @@ include wince/_ctypes.vcw wince/_ctypes.vcp wince/_ctypes_test.vcp # docs/manual +include docs/anatomy.txt include docs/manual/tutorial.txt include docs/manual/tutorial.html include docs/manual/mkpydoc.py Added: external/ctypes/TODO ============================================================================== --- (empty file) +++ external/ctypes/TODO Sat Jun 10 22:08:12 2006 @@ -0,0 +1,12 @@ +to-do list for ctypes +--------------------- + +- Docs, docs, docs. + +- Integrate newer libffi sources (HPPA architecture). + +- Finish the MingW32 port. + +- Port to Win64 (AMD). + +- Remove the _pointer_type_cache leak. Modified: external/ctypes/ctypes/__init__.py ============================================================================== --- external/ctypes/ctypes/__init__.py (original) +++ external/ctypes/ctypes/__init__.py Sat Jun 10 22:08:12 2006 @@ -1,7 +1,6 @@ """create and manipulate C data types in Python""" import os as _os, sys as _sys -from itertools import chain as _chain # special developer support to use ctypes from the CVS sandbox, # without installing it @@ -11,7 +10,7 @@ execfile(_magicfile) del _magicfile -__version__ = "0.9.9.6" +__version__ = "0.9.9.7" from _ctypes import Union, Structure, Array from _ctypes import _Pointer @@ -119,7 +118,7 @@ elif _os.name == "posix": from _ctypes import dlopen as _dlopen -from _ctypes import sizeof, byref, addressof, alignment +from _ctypes import sizeof, byref, addressof, alignment, resize from _ctypes import _SimpleCData class py_object(_SimpleCData): @@ -301,7 +300,7 @@ return "<%s '%s', handle %x at %x>" % \ (self.__class__.__name__, self._name, (self._handle & (_sys.maxint*2 + 1)), - id(self)) + id(self) & (_sys.maxint*2 + 1)) def __getattr__(self, name): if name.startswith('__') and name.endswith('__'): @@ -427,12 +426,10 @@ _restype_ = restype _flags_ = _FUNCFLAG_CDECL | _FUNCFLAG_PYTHONAPI return CFunctionType -_cast = PYFUNCTYPE(py_object, c_void_p, py_object)(_cast_addr) +_cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr) def cast(obj, typ): - result = _cast(obj, typ) - result.__keepref = obj - return result + return _cast(obj, obj, typ) _string_at = CFUNCTYPE(py_object, c_void_p, c_int)(_string_at_addr) def string_at(ptr, size=0): Added: external/ctypes/ctypes/test/test_anon.py ============================================================================== --- (empty file) +++ external/ctypes/ctypes/test/test_anon.py Sat Jun 10 22:08:12 2006 @@ -0,0 +1,60 @@ +import unittest +from ctypes import * + +class AnonTest(unittest.TestCase): + + def test_anon(self): + class ANON(Union): + _fields_ = [("a", c_int), + ("b", c_int)] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + + self.failUnlessEqual(ANON.a.offset, 0) + self.failUnlessEqual(ANON.b.offset, 0) + + def test_anon_nonseq(self): + # TypeError: _anonymous_ must be a sequence + self.failUnlessRaises(TypeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], "_anonymous_": 42})) + + def test_anon_nonmember(self): + # AttributeError: type object 'Name' has no attribute 'x' + self.failUnlessRaises(AttributeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], + "_anonymous_": ["x"]})) + + def test_nested(self): + class ANON_S(Structure): + _fields_ = [("a", c_int)] + + class ANON_U(Union): + _fields_ = [("_", ANON_S), + ("b", c_int)] + _anonymous_ = ["_"] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON_U), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.x.offset, 0) + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + self.failUnlessEqual(Y._.offset, sizeof(c_int)) + self.failUnlessEqual(Y.y.offset, sizeof(c_int) * 2) + +if __name__ == "__main__": + unittest.main() Modified: external/ctypes/ctypes/test/test_cast.py ============================================================================== --- external/ctypes/ctypes/test/test_cast.py (original) +++ external/ctypes/ctypes/test/test_cast.py Sat Jun 10 22:08:12 2006 @@ -30,17 +30,32 @@ ptr = cast(address, POINTER(c_int)) self.failUnlessEqual([ptr[i] for i in range(3)], [42, 17, 2]) + def test_p2a_objects(self): + array = (c_char_p * 5)() + self.failUnlessEqual(array._objects, None) + array[0] = "foo bar" + self.failUnlessEqual(array._objects, {'0': "foo bar"}) - def test_ptr2array(self): - array = (c_int * 3)(42, 17, 2) + p = cast(array, POINTER(c_char_p)) + # array and p share a common _objects attribute + self.failUnless(p._objects is array._objects) + self.failUnlessEqual(array._objects, {'0': "foo bar", id(array): array}) + p[0] = "spam spam" + self.failUnlessEqual(p._objects, {'0': "spam spam", id(array): array}) + self.failUnless(array._objects is p._objects) + p[1] = "foo bar" + self.failUnlessEqual(p._objects, {'1': 'foo bar', '0': "spam spam", id(array): array}) + self.failUnless(array._objects is p._objects) - from sys import getrefcount - - before = getrefcount(array) - ptr = cast(array, POINTER(c_int)) - self.failUnlessEqual(getrefcount(array), before + 1) - del ptr - self.failUnlessEqual(getrefcount(array), before) + def test_other(self): + p = cast((c_int * 4)(1, 2, 3, 4), POINTER(c_int)) + self.failUnlessEqual(p[:4], [1,2, 3, 4]) + c_int() + self.failUnlessEqual(p[:4], [1, 2, 3, 4]) + p[2] = 96 + self.failUnlessEqual(p[:4], [1, 2, 96, 4]) + c_int() + self.failUnlessEqual(p[:4], [1, 2, 96, 4]) if __name__ == "__main__": unittest.main() Modified: external/ctypes/ctypes/test/test_cfuncs.py ============================================================================== --- external/ctypes/ctypes/test/test_cfuncs.py (original) +++ external/ctypes/ctypes/test/test_cfuncs.py Sat Jun 10 22:08:12 2006 @@ -40,41 +40,49 @@ def test_short(self): self._dll.tf_h.restype = c_short + self._dll.tf_h.argtypes = (c_short,) self.failUnlessEqual(self._dll.tf_h(-32766), -10922) self.failUnlessEqual(self.S(), -32766) def test_short_plus(self): self._dll.tf_bh.restype = c_short + self._dll.tf_bh.argtypes = (c_byte, c_short) self.failUnlessEqual(self._dll.tf_bh(0, -32766), -10922) self.failUnlessEqual(self.S(), -32766) def test_ushort(self): self._dll.tf_H.restype = c_ushort + self._dll.tf_H.argtypes = (c_ushort,) self.failUnlessEqual(self._dll.tf_H(65535), 21845) self.failUnlessEqual(self.U(), 65535) def test_ushort_plus(self): self._dll.tf_bH.restype = c_ushort + self._dll.tf_bH.argtypes = (c_byte, c_ushort) self.failUnlessEqual(self._dll.tf_bH(0, 65535), 21845) self.failUnlessEqual(self.U(), 65535) def test_int(self): self._dll.tf_i.restype = c_int + self._dll.tf_i.argtypes = (c_int,) self.failUnlessEqual(self._dll.tf_i(-2147483646), -715827882) self.failUnlessEqual(self.S(), -2147483646) def test_int_plus(self): self._dll.tf_bi.restype = c_int + self._dll.tf_bi.argtypes = (c_byte, c_int) self.failUnlessEqual(self._dll.tf_bi(0, -2147483646), -715827882) self.failUnlessEqual(self.S(), -2147483646) def test_uint(self): self._dll.tf_I.restype = c_uint + self._dll.tf_I.argtypes = (c_uint,) self.failUnlessEqual(self._dll.tf_I(4294967295), 1431655765) self.failUnlessEqual(self.U(), 4294967295) def test_uint_plus(self): self._dll.tf_bI.restype = c_uint + self._dll.tf_bI.argtypes = (c_byte, c_uint) self.failUnlessEqual(self._dll.tf_bI(0, 4294967295), 1431655765) self.failUnlessEqual(self.U(), 4294967295) Modified: external/ctypes/ctypes/test/test_find.py ============================================================================== --- external/ctypes/ctypes/test/test_find.py (original) +++ external/ctypes/ctypes/test/test_find.py Sat Jun 10 22:08:12 2006 @@ -39,9 +39,23 @@ if lib_glu: self.glu = CDLL(lib_glu, RTLD_GLOBAL) if lib_glut: - self.glut = CDLL(lib_glut) + # On some systems, additional libraries seem to be + # required, loading glut fails with + # "OSError: /usr/lib/libglut.so.3: undefined symbol: XGetExtensionVersion" + # I cannot figure out how to repair the test on these + # systems (red hat), so we ignore it when the glut or gle + # libraries cannot be loaded. See also: + # https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1478253&group_id=5470 + # http://mail.python.org/pipermail/python-dev/2006-May/064789.html + try: + self.glut = CDLL(lib_glut) + except OSError: + pass if lib_gle: - self.gle = CDLL(lib_gle) + try: + self.gle = CDLL(lib_gle) + except OSError: + pass if lib_gl: def test_gl(self): Modified: external/ctypes/ctypes/test/test_keeprefs.py ============================================================================== --- external/ctypes/ctypes/test/test_keeprefs.py (original) +++ external/ctypes/ctypes/test/test_keeprefs.py Sat Jun 10 22:08:12 2006 @@ -61,6 +61,8 @@ r.ul.x = 22 r.ul.y = 44 self.assertEquals(r._objects, {'0': {}}) + r.lr = POINT() + self.assertEquals(r._objects, {'0': {}, '1': {}}) class ArrayTestCase(unittest.TestCase): def test_cint_array(self): @@ -86,9 +88,10 @@ self.assertEquals(x._objects, {'1': {}}) class PointerTestCase(unittest.TestCase): - def X_test_p_cint(self): - x = pointer(c_int(42)) - print x._objects + def test_p_cint(self): + i = c_int(42) + x = pointer(i) + self.failUnlessEqual(x._objects, {'1': i}) class DeletePointerTestCase(unittest.TestCase): def X_test(self): Modified: external/ctypes/ctypes/test/test_loading.py ============================================================================== --- external/ctypes/ctypes/test/test_loading.py (original) +++ external/ctypes/ctypes/test/test_loading.py Sat Jun 10 22:08:12 2006 @@ -51,8 +51,9 @@ if os.name in ("nt", "ce"): def test_load_library(self): - print find_library("kernel32") - print find_library("user32") + if is_resource_enabled("printing"): + print find_library("kernel32") + print find_library("user32") if os.name == "nt": windll.kernel32.GetModuleHandleW Added: external/ctypes/ctypes/test/test_objects.py ============================================================================== --- (empty file) +++ external/ctypes/ctypes/test/test_objects.py Sat Jun 10 22:08:12 2006 @@ -0,0 +1,66 @@ +r''' +This tests the '_objects' attribute of ctypes instances. '_objects' +holds references to objects that must be kept alive as long as the +ctypes instance, to make sure that the memory buffer is valid. + +WARNING: The '_objects' attribute is exposed ONLY for debugging ctypes itself, +it MUST NEVER BE MODIFIED! + +'_objects' is initialized to a dictionary on first use, before that it +is None. + +Here is an array of string pointers: + +>>> from ctypes import * +>>> array = (c_char_p * 5)() +>>> print array._objects +None +>>> + +The memory block stores pointers to strings, and the strings itself +assigned from Python must be kept. + +>>> array[4] = 'foo bar' +>>> array._objects +{'4': 'foo bar'} +>>> array[4] +'foo bar' +>>> + +It gets more complicated when the ctypes instance itself is contained +in a 'base' object. + +>>> class X(Structure): +... _fields_ = [("x", c_int), ("y", c_int), ("array", c_char_p * 5)] +... +>>> x = X() +>>> print x._objects +None +>>> + +The'array' attribute of the 'x' object shares part of the memory buffer +of 'x' ('_b_base_' is either None, or the root object owning the memory block): + +>>> print x.array._b_base_ # doctest: +ELLIPSIS + +>>> + +>>> x.array[0] = 'spam spam spam' +>>> x._objects +{'0:2': 'spam spam spam'} +>>> x.array._b_base_._objects +{'0:2': 'spam spam spam'} +>>> + +''' + +import unittest, doctest + +import ctypes.test.test_objects + +class TestCase(unittest.TestCase): + def test(self): + doctest.testmod(ctypes.test.test_objects) + +if __name__ == '__main__': + doctest.testmod(ctypes.test.test_objects) Modified: external/ctypes/ctypes/test/test_pointers.py ============================================================================== --- external/ctypes/ctypes/test/test_pointers.py (original) +++ external/ctypes/ctypes/test/test_pointers.py Sat Jun 10 22:08:12 2006 @@ -133,28 +133,7 @@ self.failUnlessEqual(p[0], 42) self.failUnlessEqual(p.contents.value, 42) - def test_incomplete(self): - lpcell = POINTER("cell") - class cell(Structure): - _fields_ = [("value", c_int), - ("next", lpcell)] - SetPointerType(lpcell, cell) - - # Make a structure containing a pointer to itself: - c = cell() - c.value = 42 - c.next = pointer(c) - - result = [] - for i in range(8): - result.append(c.value) - c = c.next[0] - self.failUnlessEqual(result, [42] * 8) - - from ctypes import _pointer_type_cache - del _pointer_type_cache[cell] - - def test_charpp( self ): + def test_charpp(self): """Test that a character pointer-to-pointer is correctly passed""" dll = CDLL(_ctypes_test.__file__) func = dll._testfunc_c_p_p Modified: external/ctypes/ctypes/test/test_python_api.py ============================================================================== --- external/ctypes/ctypes/test/test_python_api.py (original) +++ external/ctypes/ctypes/test/test_python_api.py Sat Jun 10 22:08:12 2006 @@ -1,5 +1,6 @@ from ctypes import * import unittest, sys +from ctypes.test import is_resource_enabled ################################################################ # This section should be moved into ctypes\__init__.py, when it's ready. @@ -33,20 +34,24 @@ del pyob self.failUnlessEqual(grc(s), refcnt) - def test_PyInt_Long(self): - ref42 = grc(42) - pythonapi.PyInt_FromLong.restype = py_object - self.failUnlessEqual(pythonapi.PyInt_FromLong(42), 42) - - self.failUnlessEqual(grc(42), ref42) - - pythonapi.PyInt_AsLong.argtypes = (py_object,) - pythonapi.PyInt_AsLong.restype = c_long - - res = pythonapi.PyInt_AsLong(42) - self.failUnlessEqual(grc(res), ref42 + 1) - del res - self.failUnlessEqual(grc(42), ref42) + if is_resource_enabled("refcount"): + # This test is unreliable, because it is possible that code in + # unittest changes the refcount of the '42' integer. So, it + # is disabled by default. + def test_PyInt_Long(self): + ref42 = grc(42) + pythonapi.PyInt_FromLong.restype = py_object + self.failUnlessEqual(pythonapi.PyInt_FromLong(42), 42) + + self.failUnlessEqual(grc(42), ref42) + + pythonapi.PyInt_AsLong.argtypes = (py_object,) + pythonapi.PyInt_AsLong.restype = c_long + + res = pythonapi.PyInt_AsLong(42) + self.failUnlessEqual(grc(res), ref42 + 1) + del res + self.failUnlessEqual(grc(42), ref42) def test_PyObj_FromPtr(self): s = "abc def ghi jkl" Modified: external/ctypes/ctypes/test/test_slicing.py ============================================================================== --- external/ctypes/ctypes/test/test_slicing.py (original) +++ external/ctypes/ctypes/test/test_slicing.py Sat Jun 10 22:08:12 2006 @@ -35,20 +35,45 @@ self.assertRaises(ValueError, setslice, a, 0, 5, range(32)) def test_char_ptr(self): - s = "abcdefghijklmnopqrstuvwxyz\0" + s = "abcdefghijklmnopqrstuvwxyz" dll = CDLL(_ctypes_test.__file__) dll.my_strdup.restype = POINTER(c_char) + dll.my_free.restype = None res = dll.my_strdup(s) self.failUnlessEqual(res[:len(s)], s) import operator self.assertRaises(TypeError, operator.setslice, res, 0, 5, u"abcde") + dll.my_free(res) dll.my_strdup.restype = POINTER(c_byte) res = dll.my_strdup(s) - self.failUnlessEqual(res[:len(s)-1], range(ord("a"), ord("z")+1)) + self.failUnlessEqual(res[:len(s)], range(ord("a"), ord("z")+1)) + dll.my_free(res) + + def test_char_ptr_with_free(self): + dll = CDLL(_ctypes_test.__file__) + s = "abcdefghijklmnopqrstuvwxyz" + + class allocated_c_char_p(c_char_p): + pass + + dll.my_free.restype = None + def errcheck(result, func, args): + retval = result.value + dll.my_free(result) + return retval + + dll.my_strdup.restype = allocated_c_char_p + dll.my_strdup.errcheck = errcheck + try: + res = dll.my_strdup(s) + self.failUnlessEqual(res, s) + finally: + del dll.my_strdup.errcheck + def test_char_array(self): s = "abcdefghijklmnopqrstuvwxyz\0" @@ -68,12 +93,14 @@ dll = CDLL(_ctypes_test.__file__) dll.my_wcsdup.restype = POINTER(c_wchar) dll.my_wcsdup.argtypes = POINTER(c_wchar), + dll.my_free.restype = None res = dll.my_wcsdup(s) self.failUnlessEqual(res[:len(s)], s) import operator self.assertRaises(TypeError, operator.setslice, res, 0, 5, u"abcde") + dll.my_free(res) if sizeof(c_wchar) == sizeof(c_short): dll.my_wcsdup.restype = POINTER(c_short) @@ -81,8 +108,11 @@ dll.my_wcsdup.restype = POINTER(c_int) elif sizeof(c_wchar) == sizeof(c_long): dll.my_wcsdup.restype = POINTER(c_long) + else: + return res = dll.my_wcsdup(s) self.failUnlessEqual(res[:len(s)-1], range(ord("a"), ord("z")+1)) + dll.my_free(res) ################################################################ Modified: external/ctypes/ctypes/test/test_structures.py ============================================================================== --- external/ctypes/ctypes/test/test_structures.py (original) +++ external/ctypes/ctypes/test/test_structures.py Sat Jun 10 22:08:12 2006 @@ -138,8 +138,8 @@ self.failUnlessEqual(X.y.size, sizeof(c_char)) # readonly - self.assertRaises(TypeError, setattr, X.x, "offset", 92) - self.assertRaises(TypeError, setattr, X.x, "size", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "offset", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "size", 92) class X(Union): _fields_ = [("x", c_int), @@ -152,8 +152,8 @@ self.failUnlessEqual(X.y.size, sizeof(c_char)) # readonly - self.assertRaises(TypeError, setattr, X.x, "offset", 92) - self.assertRaises(TypeError, setattr, X.x, "size", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "offset", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "size", 92) # XXX Should we check nested data types also? # offset is always relative to the class... @@ -294,7 +294,7 @@ # In Python 2.5, Exception is a new-style class, and the repr changed if issubclass(Exception, object): self.failUnlessEqual(msg, - "(Phone) : " + "(Phone) : " "expected string or Unicode object, int found") else: self.failUnlessEqual(msg, @@ -305,7 +305,7 @@ self.failUnlessEqual(cls, RuntimeError) if issubclass(Exception, object): self.failUnlessEqual(msg, - "(Phone) : too many initializers") + "(Phone) : too many initializers") else: self.failUnlessEqual(msg, "(Phone) exceptions.ValueError: too many initializers") Added: external/ctypes/ctypes/test/test_varsize_struct.py ============================================================================== --- (empty file) +++ external/ctypes/ctypes/test/test_varsize_struct.py Sat Jun 10 22:08:12 2006 @@ -0,0 +1,115 @@ +from ctypes import * +import unittest + +class VarSizeTest(unittest.TestCase): + def test_resize(self): + class X(Structure): + _fields_ = [("item", c_int), + ("array", c_int * 1)] + + self.failUnlessEqual(sizeof(X), sizeof(c_int) * 2) + x = X() + x.item = 42 + x.array[0] = 100 + self.failUnlessEqual(sizeof(x), sizeof(c_int) * 2) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for 10 additional items + new_size = sizeof(X) + sizeof(c_int) * 9 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + def test_array_invalid_length(self): + # cannot create arrays with non-positive size + self.failUnlessRaises(ValueError, lambda: c_int * -1) + self.failUnlessRaises(ValueError, lambda: c_int * -3) + + def test_zerosized_array(self): + array = (c_int * 0)() + # accessing elements of zero-sized arrays raise IndexError + self.failUnlessRaises(IndexError, array.__setitem__, 0, None) + self.failUnlessRaises(IndexError, array.__getitem__, 0) + self.failUnlessRaises(IndexError, array.__setitem__, 1, None) + self.failUnlessRaises(IndexError, array.__getitem__, 1) + self.failUnlessRaises(IndexError, array.__setitem__, -1, None) + self.failUnlessRaises(IndexError, array.__getitem__, -1) + + def test_varsized_array(self): + array = (c_int * 20)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) + + # no range checking is done on arrays with size == 1 + varsize_array = (c_int * 1).from_address(addressof(array)) + + # __getitem__ + self.failUnlessEqual(varsize_array[0], 20) + self.failUnlessEqual(varsize_array[1], 21) + self.failUnlessEqual(varsize_array[2], 22) + self.failUnlessEqual(varsize_array[3], 23) + self.failUnlessEqual(varsize_array[4], 24) + self.failUnlessEqual(varsize_array[5], 25) + self.failUnlessEqual(varsize_array[6], 26) + self.failUnlessEqual(varsize_array[7], 27) + self.failUnlessEqual(varsize_array[8], 28) + self.failUnlessEqual(varsize_array[9], 29) + + # still, normal sequence of length one behaviour: + self.failUnlessEqual(varsize_array[-1], 20) + self.failUnlessRaises(IndexError, lambda: varsize_array[-2]) + # except for this one, which will raise MemoryError + self.failUnlessRaises(MemoryError, lambda: varsize_array[:]) + + # __setitem__ + varsize_array[0] = 100 + varsize_array[1] = 101 + varsize_array[2] = 102 + varsize_array[3] = 103 + varsize_array[4] = 104 + varsize_array[5] = 105 + varsize_array[6] = 106 + varsize_array[7] = 107 + varsize_array[8] = 108 + varsize_array[9] = 109 + + for i in range(10): + self.failUnlessEqual(varsize_array[i], i + 100) + self.failUnlessEqual(array[i], i + 100) + + # __getslice__ + self.failUnlessEqual(varsize_array[0:10], range(100, 110)) + self.failUnlessEqual(varsize_array[1:9], range(101, 109)) + self.failUnlessEqual(varsize_array[1:-1], []) + + # __setslice__ + varsize_array[0:10] = range(1000, 1010) + self.failUnlessEqual(varsize_array[0:10], range(1000, 1010)) + + varsize_array[1:9] = range(1001, 1009) + self.failUnlessEqual(varsize_array[1:9], range(1001, 1009)) + + def test_vararray_is_sane(self): + array = (c_int * 15)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) + + varsize_array = (c_int * 1).from_address(addressof(array)) + varsize_array[:] = [1, 2, 3, 4, 5] + + self.failUnlessEqual(array[:], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) + self.failUnlessEqual(varsize_array[0:10], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29]) + + array[:5] = [10, 11, 12, 13, 14] + self.failUnlessEqual(array[:], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) + self.failUnlessEqual(varsize_array[0:10], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29]) + +if __name__ == "__main__": + unittest.main() Added: external/ctypes/docs/anatomy.txt ============================================================================== --- (empty file) +++ external/ctypes/docs/anatomy.txt Sat Jun 10 22:08:12 2006 @@ -0,0 +1,233 @@ +ctypes objects anatomy +====================== + +This article describes some of the internals of ctypes types and +ctypes instances. + +The object structure +-------------------- + +Definition of the structure that each ctypes instance has:: + + struct tagCDataObject { + PyObject_HEAD /* Standard Python object fields */ + char *b_ptr; /* pointer to memory block */ + int b_needsfree; /* does the object own its memory block? */ + CDataObject *b_base; /* pointer to base object or NULL */ + Py_ssize_t b_size; /* size of memory block in bytes */ + Py_ssize_t b_length; /* number of fields of this object */ + Py_ssize_t b_index; /* index of this object into the base + objects b_object list */ + PyObject *b_objects; /* references we need to keep */ + union value b_value; /* a small default buffer */ + }; + +Here is what the fields contain: + +- ``PyObject_HEAD`` + + Standard for every Python object. + +- ``b_ptr`` + + Pointer to the memory block that the object uses. + +- ``b_needsfree`` + + A flag which is nonzero if the object does owns the memory block, + nonzero otherwise. + +- ``b_base`` + + If the object does not own the memory block, this is the 'root' object + that owns the memory block. Otherwise it is NULL. + +- ``b_size`` + + Size of memory block in bytes. + +- ``b_length`` + + ... + +- ``b_index`` + + If b_base is not NULL, this is the index of this object in the + 'root' object. + +- ``b_objects`` + + This is a pointer containing a Python object which contains other + objects that have to be kept alive as long as this object lives. + Either ``None``, or a dictionary. + +- ``b_value`` + + A default memory block which can be used by small objects to avoid + a PyMem_Malloc calls. + +The memory block +---------------- + +Basically, a ctypes object instance has a memory block containing C +compatible data, plus the ``b_objects`` Python object pointer. The +latter is used to keep referenced objects alive that are referenced by +pointers in the memory block. + +Consider an array of 4 string pointers, defined by this C code:: + + char *string_array[4]; + +The ctypes definition is:: + + >>> from ctypes import * + >>> string_array = (c_char_p * 4)() + >>> + +The memory block of ``string_array`` is initialized to all zeros, +and retrieving items returns 4 ``None`` objects:: + + >>> string_array[:] + [None, None, None, None] + >>> + +We can assign Python strings to the items, and get them back:: + + >>> string_array[0] = "foo bar"; string_array[1] = "spam, spam, spam" + >>> string_array[0:2] + ['foo bar', 'spam, spam, spam'] + >>> + +The memory block contains the *pointers* to the strings (ctypes +objects implement the buffer interface, so we can use the following +snippet to examine the buffer contents):: + + >>> print repr(str(buffer(string_array))) # doctest: +SKIP + '\x94\xb7\xbd\x00\xfc\x80\xbf\x00\x00\x00\x00\x00\x00\x00\x00\x00' + >>> + +The strings themselves must also be kept in memory, otherwise the +pointers in the memory block would access invalid or freed memory. +The are stored in a dictionary in the``b_objects`` field of the +``tagCDataObject`` structure defined above. This field is exposed as +attribute named ``_objects`` to Python, but you should be aware that +this object is only exposed for debugging (and understanding) of +ctypes objects, you should *never* modify it:: + + >>> string_array._objects + {'1': 'spam, spam, spam', '0': 'foo bar'} + >>> + +The ``b_objects`` dictionary stores these needed objects as values, at +a key that is calculated from the item index or field index. + +Not all ctypes objects have to keep references, simple types like +integers or floats for example can happily live in the memory block, +without any other needs. In this case ``b_objects`` contains +``Py_None``, and the dictionary is never created:: + + >>> int_array = (c_int * 3)() + >>> int_array[:] = 1, 2, 3 + >>> print int_array._objects + None + >>> + +XXX remove this? + + >>> s_a = (c_char_p * 5210)() + >>> s_a[0] = "zero" + >>> s_a[255] = "ali baba" + >>> s_a[257] = "forty robbers" + >>> s_a[1] = "spam and eggs" + >>> s_a._objects + {'1': 'spam and eggs', '0': 'zero', '101': 'forty robbers', 'ff': 'ali baba'} + >>> + +The important thing to keep in mind is that it must be possible to +'reconstruct' the whole ctypes object from the memory block and the +``b_objects`` pointer. + + +What happens if a ctypes object is stored in another ctypes object +field? Define a structure which has a field storing a string array:: + + >>> class Container(Structure): + ... _fields_ = [("count", c_uint), + ... ("strings", c_char_p * 4)] + ... + >>> + >>> container = Container() + >>> container.strings = string_array + >>> container._objects + {'1': {'1': 'spam, spam, spam', '0': 'foo bar'}} + >>> + +As we can see, the ``string_array`` ``b_objects`` dictionary has been +inserted into the ``container`` ``b_objects`` dictionary at index 1, +because 1 is the index of the ``strings`` field. The contents of the +``string_array`` memory block has been copied into the ``container`` +memory block as well. + +Again, things get slighlty more complicated when we use a structure +containing a pointer field instead of an array field. In this case, +the memory block of the pointer object must be kept alive in addition +to the pointer ``b_object`` dictionary. Here is what ctypes does:: + + >>> class PointerContainer(Structure): + ... _fields_ = [("count", c_uint), + ... ("string_p", POINTER(c_char_p))] + ... + >>> pc = PointerContainer() + >>> pc.string_p = string_array + >>> pc._objects + {'1': ({'1': 'spam, spam, spam', '0': 'foo bar'}, )} + >>> + +So, assigning an array instance to a pointer type field stores a tuple +containing the arrays ``b_objects`` dictionary plus the array object +itself. Of course, in this case the memory block of ``string_array`` +is NOT copied into the ``PointerContainer`` memory block, only the +address is copied. + +What happens if we retrieve the string pointer from the +PointerContainer instance? ctypes doesn't do OOR (original object +return), ctypes returns a new object on each attribute access:: + + >>> pc.string_p is pc.string_p + False + >>> + + >>> print pc.string_p._objects + None + >>> print pc.string_p._b_base_ + + >>> + + >>> other = PointerContainer() + >>> other.string_p = pc.string_p + >>> print other._objects + {'1': {'1': ({'1': 'spam, spam, spam', '0': 'foo bar'}, )}} + >>> + + >>> x = Container() + >>> x.strings = (c_char_p * 4)() + >>> print x._objects + {'1': {}} + >>> x.strings[2] = "python ctypes" + >>> print x._objects + {'1': {}, '2:1': 'python ctypes'} + >>> + + >>> class X(Structure): + ... _fields_ = [("a", c_int), + ... ("b", c_int), + ... ("c", c_int), + ... ("d", c_int), + ... ("container", Container)] + ... + >>> x = X() + >>> x.container.strings = (c_char_p * 4)() + >>> x.container.strings[1] = "foobar.org" + >>> x._objects + {'1:4': {}, '1:1:4': 'foobar.org'} + >>> \ No newline at end of file Deleted: /external/ctypes/docs/manual/callbacks.txt ============================================================================== --- /external/ctypes/docs/manual/callbacks.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,31 +0,0 @@ -Callback functions -++++++++++++++++++ - -ctypes is able to create C callable functions from Python callables. -This is useful because sometimes library functions need a callback -function parameter; the ``qsort`` C function is such an example. - -Callback functions are created by first creating a function prototype -with a call to ``CFUNCTYPE`` or ``WINFUNCTYPE``, specifying the return -type and the argument types that the callback function will receive. - -Calling the prototype with a single Python callable will create and -return a C-callable function pointer or callback function. Note that -this allows using prototypes as decorators creating callback -functions (Windows example):: - - @WINFUNCTYPE(BOOL, HWND, LPARAM) - def enumwindowsproc(hwnd, lParam): - .... - return True - -When a Python exception is raised in the Python callable, the return -value of the C callable function is undefined. - -Important note: You must keep a reference to the callback AS LONG as -foreign code will call it! Segfaults will result if the callback is -cleaned up by Python's garbage collector and external code then -tries to call it. - -Callback objects can also be called from Python - this may be useful -for debugging. Added: external/ctypes/docs/manual/ctypes.txt ============================================================================== --- (empty file) +++ external/ctypes/docs/manual/ctypes.txt Sat Jun 10 22:08:12 2006 @@ -0,0 +1,11 @@ +:Module: ctypes +:Summary: A foreign function library for Python. +:Module Type: standard +:Author: Thomas Heller +:Synopsis: A foreign function library for Python. +:Version Added: 2.5 + +``ctypes`` is a foreign function library for Python. + +.. include:: tutorial.txt +.. include:: reference.txt Deleted: /external/ctypes/docs/manual/docs.txt ============================================================================== --- /external/ctypes/docs/manual/docs.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,7 +0,0 @@ - -.. include:: libraries.txt -.. include:: functions.txt -.. include:: callbacks.txt -.. include:: simple_types.txt -.. include:: struct_union.txt -.. include:: utilities.txt Added: external/ctypes/docs/manual/filter.py ============================================================================== --- (empty file) +++ external/ctypes/docs/manual/filter.py Sat Jun 10 22:08:12 2006 @@ -0,0 +1,18 @@ +import sys + +TOKENS = """: funcdesc +: excdesc +: vardesc +: classdesc +: methoddesc +: memberdesc +: classdesc*""".splitlines() + +for line in open(sys.argv[1], "r"): + for token in TOKENS: + line = line.rstrip() + if line.endswith(token): + print line[:-len(token)] + break + else: + print line Deleted: /external/ctypes/docs/manual/functions.txt ============================================================================== --- /external/ctypes/docs/manual/functions.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,135 +0,0 @@ - -Foreign functions -+++++++++++++++++ - -Functions exported from loaded shared libraries (foreign functions) -can be accessed in two ways. The easiest way is to retrieve them as -attributes of library objects by name:: - - libc = cdll.find("c") # posix - libc = cdll.msvcrt # windows - # attribute access - atoi = libc.atoi - # alternative indexing notation - atoi = libc["atoi"] - -This creates an instance of a foreign function object, using the -calling convention specified by the library object ``cdll``, bound to -the C library ``atoi`` function. The C function is assumed to return -an integer (which is correct for ``atoi``), and the argument types are -not specified (``atoi`` expects a single ``char *`` argument). - -If the library function returns a type different from ``int``, the -``restype`` attribute can be set to a ctypes type that describes the -return type, or to ``None`` meaning no return value (``void``). - -The optional ``argtypes`` attribute can be set to a sequence of ctypes -types that the function expects. - -If needed, the function can (as in C) be called with more arguments -than the length of the argtypes sequence. - -The optional ``errcheck`` attribute can be set to a Python callable, -which can be used to validate and/or process the library function's return -value. ``errcheck`` will be called with three arguments, after the -library function has returned:: - - errcheck(retval, function, arguments) - -``retval`` is the value that the library function returned, converted -according to ``restype``. ``function`` is the ctypes function object -(libc.atoi in this case), and ``arguments`` is a tuple containing the -arguments that have been used to call ``function``. ``errcheck`` -should validate the library function result, raise an error if it -detects a failure, or return the needed return value otherwise. - -Function prototypes -------------------- - -Another way to access a function exported from shared libraries is to -first create a prototype by calling a factory function, specifying the -return type and the argument types. The factory function itself -specifies the calling convention: ``CFUNCTYPE`` uses the standard C -calling convention, ``WINFUNCTYPE`` (Windows only) uses the stdcall -calling convention. The factory function must be called with the -return type plus the argument types. For the C ``atoi`` function one -would use ``CFUNCTYPE(c_int, c_char_p)``. - -This returns a function prototype, which is a ctypes type representing -all functions that are compatible with the calling convention, return -type, and argument types. - -The ``CFUNCTYPE`` and ``WINFUNCTYPE`` factory functions cache and -reuse the types they create in internal caches, so is is cheap to call -them over and over with the same or different arguments. - -An instance of this function prototype, bound to a foreign library -function, can be created by calling the prototype with the name of the -function as string, and a loaded library:: - - proto = CFUNCTYPE(c_int, c_char_p) - atoi = proto("atoi", libc) - -Parameter flags ---------------- - -It is possible to specify a third argument ``paramflags`` when calling -the prototype. This is used to specify additional information for -each argument: direction of data transfer, the name, and a default -value. - -A tuple with the same length as ``argtypes`` (the second argument in -the prototype call) must be used. Each item in this tuple must be a -tuple, having either one, two, or three items. - -The first item is the direction flag, an integer specifying if this is -an input (use ``1``) or an output (use ``2``) parameter. The optional -second item is a string containing the parameter name, the optional -third item is a default value for the parameter. - -If parameter names are specified, the function object created can be -called with named arguments in the usual way. Arguments with default -values do not need to be specified when the function is called. - -``out`` parameter types must be pointer types. When the function -object is called, ctypes will automatically create empty instances of -them, pass them to the library function, retrieve the value from them, -and return the value, if there is exactly one ``out`` parameter, or a -tuple of values, if there is more than one ``out`` parameter. The -original foreign function return value is lost in this case (but see -below for how it can be retrieved). - -If ``paramflags`` have been used in the prototype call, and an -``errcheck`` attribute is also present, the ``errcheck`` callable will -be called with a fourth parameter ``outargs``:: - - errcheck(retval, function, arguments, outargs) - -``outargs`` is a tuple containing all the ``out`` parameters that -ctypes has created. Without the ``errcheck`` function ctypes would -retrieve the values contained in these pointer objects, and return -them. The ``errcheck`` function can let ctypes continue this -processing by returning the ``outargs`` tuple. It could also return -something else, or raise an error if it detects that the library -function has failed. - -COM methods (Windows only) --------------------------- - -XXX Should this be left undocumented? Mentioned for completeness. - -The prototypes created by ``WINFUNCTYPE`` can be called with a -positive small integer ``index``, a string ``name``, an optional -``paramflags`` tuple, and a optional ``iid`` parameter. - -This creates a function object wrapping a COM method. ``index`` is -the index into the COM object's virtual function table, ``name`` is -the name of the COM method (only useful for debugging), ``paramflags`` -has the same meaning as for normal function objects, and ``iid`` is a -string or buffer containing the interface id of the COM interface -this method belongs to. ``iid`` is used to get extended COM error -information in case the method returns a FAILED ''HRESULT`` value. - -Note that COM methods expect an additional first argument that is NOT -listed in the prototypes ``argtypes`` when they are called: this must -be the integer address of a COM interface pointer. Deleted: /external/ctypes/docs/manual/libctypes.txt ============================================================================== --- /external/ctypes/docs/manual/libctypes.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,4 +0,0 @@ -Blah blah short overview of ctypes. - -.. include:: tutorial.txt -.. include:: reference.txt \ No newline at end of file Deleted: /external/ctypes/docs/manual/libraries.txt ============================================================================== --- /external/ctypes/docs/manual/libraries.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,169 +0,0 @@ - -Shared Libraries, DLLs -++++++++++++++++++++++ - -Shared libraries are accessed when compiling/linking a program, and -when the program is run. The purpose of the ``find`` method is to -locate a library in a way similar to what the compiler does (on -platforms with several versions of a shared library the most recent -should be loaded), while ``load`` acts like when a program is run, and -uses the runtime loader directly. ``load_version`` works like ``load`` -but tries to be platform independent (for cases where this makes -sense). Loading via attribute access is a shorthand notation -especially usefull for interactive use, it is equivalent to calling -``load_version`` with no version specified. - - -class LibraryLoader -------------------- - -Instances of ``LibraryLoader`` are used to load shared libraries. -Usually there is no need to create a libraryloader, instead one of the -predefined loaders should be used. - -Libraryloaders have the following methods: - -``find(name, mode=None)`` - - Try to find a library, load and return it. ``name`` is the - library name without any prefix like ``lib``, suffix like ``.so``, - ``.dylib`` or version number (this is the form used for the posix - linker option ``-l``). - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function, ignored on Windows. - - On Windows, this method does exactly the same as the ``load`` - method. - - On other platforms, this function might call other programs like - the compiler to find the library. When using ctypes to write a - shared library wrapping, consider using ``load_version`` or - ``load`` instead. - -``load_version(name, version=None, mode=None)`` - - Build a system dependent filename from ``name`` and optionally - ``version``, then load and return it. ``name`` is the library - name without any prefix like ``lib`` and suffix like ``.so`` or - ``.dylib``. This method should be used if a library is available - on different platforms, using the particular naming convention of - each platform. - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function, ignored on Windows. - - Example: calling ``loader.load_version('z', '1.1.3')`` would - possibly load ``/usr/lib/libz.1.1.3.dylib`` on Mac OS X, or - ``/lib/libz.so.1.1.3`` on a Linux system. - -``load(libname, mode=None)`` - - Load and return the library with the given libname. On most - systems ``libname`` is the filename of the shared library; when - it's not a pathname it will be searched in a system dependent list - of locations (on many systems additional search paths can be - specified by an environment variable). Sometimes the file - extension (like ``.dll`` on Windows) can be omitted. - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function. ``RTLD_LOCAL`` and ``RTLD_GLOBAL`` are - typical values. On Windows, ``mode`` is ignored. - -``load_library(pathname, mode=None`` - - Load and return the library with the given pathname. This method - passes the ``pathname`` directly to the underlying ``dlopen`` or - ``LoadLibrary`` function. - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function. ``RTLD_LOCAL`` and ``RTLD_GLOBAL`` are - typical values. On Windows, ``mode`` is ignored. - -``LoadLibrary(pathname, mode=None)`` - - This is an alias for the ``load_library`` method documented above, - maintained for backwards comatibility only. - - -Libaries can also be loaded by accessing them as attributes of the -loader instance, internally this calls ``load_version`` without -specifying ``version`` or ``mode``. Obviously this only works for -libraries with names that are valid Python identifiers, and when the -name does not start with a ``_`` character. - -Predefined library loaders --------------------------- - -ctypes provides some LibraryLoader instances, the differences between -them are the calling conventions the functions will use and the -default return type of the functions. All these loaders use the -``RTLD_LOCAL`` mode flag. - -Functions can be accessed as named attributes of loaded libraries. - -On Windows, structured exception handling is used around the function -call to protect Python from crashing in case you pass invalid -parameters to the function. - -``cdll`` - - Functions provided by libraries loaded using the ``cdll`` loader - will be called with the standard C calling convention, and have a - default return type of ``int``. ctypes releases the Python global - interpreter lock (GIL) just before calling the foreign function, - and reacquires it before returing, so other threads are able to - run. - -``windll`` - - Windows only. Functions provided by libraries loaded by - ``windll`` will be called using the Windows ``__stdcall`` calling - convention. ctypes can detect when the wrong number - of parameters has been passed to the function call by examining - the stack pointer before and after the function call. If the - wrong parameter count was used, an exception is raised (although - the function really *has* been called). The return value of the - function is lost in this case. Again, the GIL is released during - the duration of the function call. - -``oledll`` - - Windows only. ``oledll`` behaves in the same way as ``windll``, - except that the called function is expected to return a - ``HRESULT`` value. These are long values containing error or - success codes. In case the function returns an error ``HRESULT`` - value, a ``WindowsError`` is raised. The GIL is released during the - duration of function call. - -``pydll`` - - This loader allows to call functions in libraries using the - *Python* calling convention, for example Python C API functions. - The GIL is *not* released during the function call, and the state - of the Python error flag is examined after the function returns. - If the error flag is set, the Python exception is raised. - - ctypes provides a prefabricated instance of ``pydll`` exposing the - Python C api as the ``pythonapi`` symbol, you should however make - sure to set the correct ``restype`` for the functions you use. - -Library objects ---------------- - -The library loaders create instances of ``CDLL``, ``WinDLL``, -``OleDLL``, or ``PyDLL`` classes. You can, however, also load a -library by constructing one of these classes by calling the -constructor with the pathname of the library and an optional ``mode`` -argument as described in the previous section. - -Library objects implement ``__getattr__`` and ``__getitem__`` methods -that allow to access foreign functions by attribute access or -indexing. The latter is useful if the name of the function is not a -valid Python identifier, or clashes with special Python method names -that start and end with two underscore characters. - -Library objects have two private attributes: ``_name`` is the pathname -of the library, ``_handle`` is the handle to the library that -``dlopen`` has returned. - Modified: external/ctypes/docs/manual/make.bat ============================================================================== --- external/ctypes/docs/manual/make.bat (original) +++ external/ctypes/docs/manual/make.bat Sat Jun 10 22:08:12 2006 @@ -1,3 +1,3 @@ -c:\python24\scripts\rst2html.py manual.txt manual.html -c:\python24\scripts\rst2html.py docs.txt docs.html -c:\python24\scripts\rst2html.py tutorial.txt tutorial.html +filter.py tutorial.txt | c:\python24\python c:\python24\scripts\rst2html.py > tutorial.html +filter.py reference.txt | c:\python24\python c:\python24\scripts\rst2html.py > reference.html +filter.py ctypes.txt | c:\python24\python c:\python24\scripts\rst2html.py > ctypes.html Deleted: /external/ctypes/docs/manual/manual.html ============================================================================== --- /external/ctypes/docs/manual/manual.html Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,950 +0,0 @@ - - - - - - -ctypes manual - - - -
-
ctypes manual
-
(work in progress, $Revision: 1.5 $)
-
-
Contents
-
-
Shared Libraries, DLLs
-
class LibraryLoader
-
Predefined library loaders
-
Library objects
-
-
-
Foreign functions
-
Function prototypes
-
Parameter flags
-
COM methods (Windows only)
-
-
-
Callback functions
-
Simple types
-
Class attributes of simple types
-
Class methods of simple types
-
Instance attributes of simple types
-
Numeric types
-
Character types
-
Pointer types
-
String types
-
-
-
Structure and union types
-
Defining field names and types
-
Packing fields
-
Bit fields
-
Recursive data types
-
Byte order
-
Builtin functions
-
Deprecated functions
-
-
-
-
-
-
Shared Libraries, DLLs
-
Shared libraries are accessed when compiling/linking a program, and -when the program is run. The purpose of the find method is to -locate a library in a way similar to what the compiler does (on -platforms with several versions of a shared library the most recent -should be loaded), while load acts like when a program is run, and -uses the runtime loader directly. load_version works like load -but tries to be platform independent (for cases where this makes -sense). Loading via attribute access is a shorthand notation -especially usefull for interactive use, it is equivalent to calling -load_version with no version specified.
-
-
class LibraryLoader
-
Instances of LibraryLoader are used to load shared libraries. -Usually there is no need to create a libraryloader, instead one of the -predefined loaders should be used.
-
Libraryloaders have the following methods:
-
find(name, mode=None)
-
-
Try to find a library, load and return it. name is the -library name without any prefix like lib, suffix like .so, -.dylib or version number (this is the form used for the posix -linker option -l).
-
mode allows to override the default flags passed to the -dlopen() function, ignored on Windows.
-
On Windows, this method does exactly the same as the load -method.
-
On other platforms, this function might call other programs like -the compiler to find the library. When using ctypes to write a -shared library wrapping, consider using load_version or -load instead.
-
-
load_version(name, version=None, mode=None)
-
-
Build a system dependent filename from name and optionally -version, then load and return it. name is the library -name without any prefix like lib and suffix like .so or -.dylib. This method should be used if a library is available -on different platforms, using the particular naming convention of -each platform.
-
mode allows to override the default flags passed to the -dlopen() function, ignored on Windows.
-
Example: calling loader.load_version('z', '1.1.3') would -possibly load /usr/lib/libz.1.1.3.dylib on Mac OS X, or -/lib/libz.so.1.1.3 on a Linux system.
-
-
load(libname, mode=None)
-
-
Load and return the library with the given libname. On most -systems libname is the filename of the shared library; when -it's not a pathname it will be searched in a system dependent list -of locations (on many systems additional search paths can be -specified by an environment variable). Sometimes the file -extension (like .dll on Windows) can be omitted.
-
mode allows to override the default flags passed to the -dlopen() function. RTLD_LOCAL and RTLD_GLOBAL are -typical values. On Windows, mode is ignored.
-
-
load_library(pathname, mode=None
-
-
Load and return the library with the given pathname. This method -passes the pathname directly to the underlying dlopen or -LoadLibrary function.
-
mode allows to override the default flags passed to the -dlopen() function. RTLD_LOCAL and RTLD_GLOBAL are -typical values. On Windows, mode is ignored.
-
-
LoadLibrary(pathname, mode=None)
-
-This is an alias for the load_library method documented above, -maintained for backwards comatibility only.
-
Libaries can also be loaded by accessing them as attributes of the -loader instance, internally this calls load_version without -specifying version or mode. Obviously this only works for -libraries with names that are valid Python identifiers, and when the -name does not start with a _ character.
-
-
-
Predefined library loaders
-
ctypes provides some LibraryLoader instances, the differences between -them are the calling conventions the functions will use and the -default return type of the functions. All these loaders use the -RTLD_LOCAL mode flag.
-
Functions can be accessed as named attributes of loaded libraries.
-
On Windows, structured exception handling is used around the function -call to protect Python from crashing in case you pass invalid -parameters to the function.
-
cdll
-
-Functions provided by libraries loaded using the cdll loader -will be called with the standard C calling convention, and have a -default return type of int. ctypes releases the Python global -interpreter lock (GIL) just before calling the foreign function, -and reacquires it before returing, so other threads are able to -run.
-
windll
-
-Windows only. Functions provided by libraries loaded by -windll will be called using the Windows __stdcall calling -convention. ctypes can detect when the wrong number -of parameters has been passed to the function call by examining -the stack pointer before and after the function call. If the -wrong parameter count was used, an exception is raised (although -the function really has been called). The return value of the -function is lost in this case. Again, the GIL is released during -the duration of the function call.
-
oledll
-
-Windows only. oledll behaves in the same way as windll, -except that the called function is expected to return a -HRESULT value. These are long values containing error or -success codes. In case the function returns an error HRESULT -value, a WindowsError is raised. The GIL is released during the -duration of function call.
-
pydll
-
-
This loader allows to call functions in libraries using the -Python calling convention, for example Python C API functions. -The GIL is not released during the function call, and the state -of the Python error flag is examined after the function returns. -If the error flag is set, the Python exception is raised.
-
ctypes provides a prefabricated instance of pydll exposing the -Python C api as the pythonapi symbol, you should however make -sure to set the correct restype for the functions you use.
-
-
-
-
Library objects
-
The library loaders create instances of CDLL, WinDLL, -OleDLL, or PyDLL classes. You can, however, also load a -library by constructing one of these classes by calling the -constructor with the pathname of the library and an optional mode -argument as described in the previous section.
-
Library objects implement __getattr__ and __getitem__ methods -that allow to access foreign functions by attribute access or -indexing. The latter is useful if the name of the function is not a -valid Python identifier, or clashes with special Python method names -that start and end with two underscore characters.
-
Library objects have two private attributes: _name is the pathname -of the library, _handle is the handle to the library that -dlopen has returned.
-
-
-
-
Foreign functions
-
Functions exported from loaded shared libraries (foreign functions) -can be accessed in two ways. The easiest way is to retrieve them as -attributes of library objects by name:
-
-libc = cdll.find("c") # posix -libc = cdll.msvcrt # windows -# attribute access -atoi = libc.atoi -# alternative indexing notation -atoi = libc["atoi"] -
-
This creates an instance of a foreign function object, using the -calling convention specified by the library object cdll, bound to -the C library atoi function. The C function is assumed to return -an integer (which is correct for atoi), and the argument types are -not specified (atoi expects a single char * argument).
-
If the library function returns a type different from int, the -restype attribute can be set to a ctypes type that describes the -return type, or to None meaning no return value (void).
-
The optional argtypes attribute can be set to a sequence of ctypes -types that the function expects.
-
If needed, the function can (as in C) be called with more arguments -than the length of the argtypes sequence.
-
The optional errcheck attribute can be set to a Python callable, -which can be used to validate and/or process the library function's return -value. errcheck will be called with three arguments, after the -library function has returned:
-
-errcheck(retval, function, arguments) -
-
retval is the value that the library function returned, converted -according to restype. function is the ctypes function object -(libc.atoi in this case), and arguments is a tuple containing the -arguments that have been used to call function. errcheck -should validate the library function result, raise an error if it -detects a failure, or return the needed return value otherwise.
-
-
Function prototypes
-
Another way to access a function exported from shared libraries is to -first create a prototype by calling a factory function, specifying the -return type and the argument types. The factory function itself -specifies the calling convention: CFUNCTYPE uses the standard C -calling convention, WINFUNCTYPE (Windows only) uses the stdcall -calling convention. The factory function must be called with the -return type plus the argument types. For the C atoi function one -would use CFUNCTYPE(c_int, c_char_p).
-
This returns a function prototype, which is a ctypes type representing -all functions that are compatible with the calling convention, return -type, and argument types.
-
The CFUNCTYPE and WINFUNCTYPE factory functions cache and -reuse the types they create in internal caches, so is is cheap to call -them over and over with the same or different arguments.
-
An instance of this function prototype, bound to a foreign library -function, can be created by calling the prototype with the name of the -function as string, and a loaded library:
-
-proto = CFUNCTYPE(c_int, c_char_p) -atoi = proto("atoi", libc) -
-
-
-
Parameter flags
-
It is possible to specify a third argument paramflags when calling -the prototype. This is used to specify additional information for -each argument: direction of data transfer, the name, and a default -value.
-
A tuple with the same length as argtypes (the second argument in -the prototype call) must be used. Each item in this tuple must be a -tuple, having either one, two, or three items.
-
The first item is the direction flag, an integer specifying if this is -an input (use 1) or an output (use 2) parameter. The optional -second item is a string containing the parameter name, the optional -third item is a default value for the parameter.
-
If parameter names are specified, the function object created can be -called with named arguments in the usual way. Arguments with default -values do not need to be specified when the function is called.
-
out parameter types must be pointer types. When the function -object is called, ctypes will automatically create empty instances of -them, pass them to the library function, retrieve the value from them, -and return the value, if there is exactly one out parameter, or a -tuple of values, if there is more than one out parameter. The -original foreign function return value is lost in this case (but see -below for how it can be retrieved).
-
If paramflags have been used in the prototype call, and an -errcheck attribute is also present, the errcheck callable will -be called with a fourth parameter outargs:
-
-errcheck(retval, function, arguments, outargs) -
-
outargs is a tuple containing all the out parameters that -ctypes has created. Without the errcheck function ctypes would -retrieve the values contained in these pointer objects, and return -them. The errcheck function can let ctypes continue this -processing by returning the outargs tuple. It could also return -something else, or raise an error if it detects that the library -function has failed.
-
-
-
COM methods (Windows only)
-
XXX Should this be left undocumented? Mentioned for completeness.
-
The prototypes created by WINFUNCTYPE can be called with a -positive small integer index, a string name, an optional -paramflags tuple, and a optional iid parameter.
-
This creates a function object wrapping a COM method. index is -the index into the COM object's virtual function table, name is -the name of the COM method (only useful for debugging), paramflags -has the same meaning as for normal function objects, and iid is a -string or buffer containing the interface id of the COM interface -this method belongs to. iid is used to get extended COM error -information in case the method returns a FAILED ''HRESULT`` value.
-
Note that COM methods expect an additional first argument that is NOT -listed in the prototypes argtypes when they are called: this must -be the integer address of a COM interface pointer.
-
-
-
-
Callback functions
-
ctypes is able to create C callable functions from Python callables. -This is useful because sometimes library functions need a callback -function parameter; the qsort C function is such an example.
-
Callback functions are created by first creating a function prototype -with a call to CFUNCTYPE or WINFUNCTYPE, specifying the return -type and the argument types that the callback function will receive.
-
Calling the prototype with a single Python callable will create and -return a C-callable function pointer or callback function. Note that -this allows using prototypes as decorators creating callback -functions (Windows example):
-
-@WINFUNCTYPE(BOOL, HWND, LPARAM) -def enumwindowsproc(hwnd, lParam): - .... - return True -
-
When a Python exception is raised in the Python callable, the return -value of the C callable function is undefined.
-
Important note: You must keep a reference to the callback AS LONG as -foreign code will call it! Segfaults will result if the callback is -cleaned up by Python's garbage collector and external code then -tries to call it.
-
Callback objects can also be called from Python - this may be useful -for debugging.
-
-
-
Simple types
-
Simple types have some special behaviour: When they are accessed as -structure or union fields, items of array instances, or as foreign -function return values, they are transparently converted from and to -the native Python types int, long, string, and unicode.
-
This is not the case for subclasses of simple data types, so while a -c_void_p type is transparently converted from and to Python -integer or long, a subclass of c_void_p is not converted. This -allows you to define new behaviour almost completely.
-
-
Class attributes of simple types
-
__ctype_be__, __ctype_le__
-
-If the type supports different byte order (pointer types do NOT -support this), __ctype_be__ and __ctype_le__ are types -with bug endian and little endian byte order. For example, -c_int.__ctype_be__ is an integer type with the memory block in -big endian byte order.
-
_type_
-
-Implementation artifact: the typecode for this type, a single -character string code compatible to what the struct module uses. -Additional characters are used for types that the struct module -does not support.
-
-
-
Class methods of simple types
-
(To be exact, these are not class methods, instead these are methods -of the metaclass. The most prominent difference to classmethods is -that you can call these methods on the class, but not on the instance -of the simple type.)
-
__ctypes_from_outparam__
-
-TBD
-
from_address
-
-TBD
-
from_param
-
-
This class method is used to adapt function parameters. If a type -is specified in a function's argtypes sequence, in a function call -the from_param(arg) method will be called with the actual -argument, and the result will be passed to the foreign function -call as a parameter.
-
from_param usually returns an internal object that you cannot -use in Python code - it only makes sense to pass this object to -foreign functions.
-
On one hand, from_param is a performance optimization - it -allows you to pass Python integers to function calls expecting a -c_int argument type, without having to create a full-featured -c_int instance.
-
On the other hand, from_param can adapt other objects to -parameters. XXX explain the automatic byref call for byref -arguments.
-
-
in_dll
-
-TBD
-
-
-
Instance attributes of simple types
-
value
-
-Allows to get or set the current value of the object. For simple -types, this is always a native Python object like integer, long, -string, unicode, or None.
-
_objects (never modify this)
-
-Implementation artifact: a Python object keeping references to -other objects which must be kept alive. Never modify anything on -the returned object. XXX Should probably not be exposed.
-
_b_base_ (readonly)
-
-Implementation artifact: the base object owning the memory block -(if any).
-
_b_needsfree_ (readonly)
-
-Implementation artifact: does this object have to free its memory -block on destruction.
-
_as_parameter_ (readonly)
-
-Implementation artifact (?): how to pass this object as a function -parameter.
-
-
-
Numeric types
-
Integer types are c_byte, c_short, c_int, c_long, -c_longlong and their unsigned variants c_ubyte, c_ushort, -c_uint, c_ulong and c_ulonglong, floating point types are -c_float and c_double.
-
The constructor and the from_param class method accept a Python -integer for integer types, a Python float for floating point types.
-
On 32-bit platforms where sizeof(int) == sizeof(long), c_int is an -alias for c_long, on 64-bit platforms where sizeof(long) == -sizeof(long long), c_long is an alias for c_longlong.
-
-
-
Character types
-
Character types are c_char and c_wchar, representing the C -char and wchar_t types.
-
The constructor and the from_param class method accept a single -character Python string or unicode string. Conversion between string -and unicode, if needed, is done according to the ctypes -encoding/decoding rules.
-
-
-
Pointer types
-
The only simple pointer type is c_void_p, which represents the C -void * data type. c_void_p can also be written as -POINTER(None).
-
The constructor accepts one optional argument, which must be an -integer or long (interpreted as an address), or None.
-
The from_param class method accepts everything that could be used -as a pointer. XXX Should accept objects using the buffer interface as -well.
-
The value attribute accepts and returns None or integer.
-
XXX Shouldn't the constructor accept the same types as from_param?
-
-
-
String types
-
ctypes has the c_char_p and c_wchar_p types which represent -const pointers to zero terminated strings in C: const char * and -const wchar_t *. Since strings and Unicode instances are -immutable, these types should be considered readonly: do not pass them -to functions that write into the buffer.
-
The constructor accepts one optional argument, which must be a Python -or unicode string, an integer, or None.
-
The from_param class method accepts a string or a Unicode string, -as well as None. Conversion between string and Unicode, if -needed, is done according to the ctypes encoding/decoding rules.
-
XXX Why does the constructor accept an integer, and from_param doesn't?
-
-
-
-
Structure and union types
-
ctypes provides the abstract base classes Structure and Union -to define structure and union types. Subclasses must at least define -a _fields_ attribute.
-
-
Defining field names and types
-
_fields_ must be a sequence of tuples. The first item of each -tuple is a string specifying the name of the structure/union field. -The second item must by a ctypes type.
-
A descriptor will be created for each field, allowing you to access the -field's contents from instances. Accessed from the class, the fields -expose readonly .offset and .size attributes. offset is -the byte-offset of the field from the beginning of the -structure/union, size is the number of bytes the field contains.
-
A simple example is a POINT structure containing integer fields named -x and y:
-
-class Point(Structure): - _fields_ = [("x", c_int), - ("y", c_int)] -
-
-
-
Packing fields
-
Normally fields are aligned in the same way as the host's C compiler -would do it. This native alignment can be overridden by setting a -_pack_ attribute in the type. It must be a small positive integer -which is the maximum field alignment.
-
-
-
Bit fields
-
Integer fields support bit sizes. The bit-size must be specified as -the third item of the _fields_ tuple. Bit fields are constructed -in the same way the host's C compiler does it. For bit fields, the -field descriptor's .size attribute contains the number of bits in -the high word, and the bit offset from the beginning of the structure in -the low word. XXX is that correct?
-
-
-
Recursive data types
-
To define recursive types, it is possible to assign the _fields_ -value after the class statement. Here is an example of a linked -list data structure, which contains a pointer to itself:
-
-class Node(Structure): - pass -Node._fields_ = [("next", POINTER(Node)), - ("value", ...)] -
-
_fields_ must be set, and cannot be changed, after the type is -used for the first time.
-
-
-
Byte order
-
It is possible to create Structure and Union types using non-native -byte order by using the BigEndianStructure, -LittleEndianStructure, BigEndianUnion, and -LittleEndianUnion base classes. Structures and Unions with -non-native byte order do not support pointer fields.
-
-
-
Builtin functions
-
addressof(object)
-
-Returns the address of a ctypes instance as an integer.
-
alignment(type_or_object)
-
-Returns the alignment requirements in bytes of a ctypes type or -instance.
-
byref(object)
-
-Returns a light-weight pointer to a ctypes instance. The returned -object can only be used as function call parameter. Behaves the -same as calling pointer(object), but is a lot faster. Same as -&object in C.
-
cast(object, typ)
-
-This function is similar to the cast operator in C. Returns a new -instance of type which shares the memory block of object. -typ must be a pointer type.
-
CFUNCTYPE(restype, *argtypes)
-
-Create a function prototype using the C calling convention.
-
create_string_buffer(init, size=None)
-
-
Convenience function to create a mutable character buffer.
-
init must be a string. If size is supplied it must be a -positive integer that specifies the size of the buffer, otherwise -the length of the init string is used. -This function returns a ctypes array of characters c_char.
-
-
create_unicode_buffer(init, size=None)
-
-
Convenience function to create a mutable unicode buffer.
-
init must be a unicode string. If size is supplied it -must be a positive integer that specifies the number of characters -in the buffer, otherwise the length of the init string is -used. This function returns a ctypes array of characters c_wchar.
-
-
DllCanUnloadNow(), DllGetClassObject(rclsid, riid, ppv) (Windows only)
-
-Functions used to implement COM servers.
-
FormatError([code]) (Windows only)
-
-Returns a textual description of the error code, or the last error -code set by Windows.
-
GetLastError() (Windows only)
-
-Returns the last error code set by Windows.
-
memmove(dst, src, count)
-
-Same as the standard C memmove library function: copies -count bytes from src to dst. dst and src must -be integers or anything else that can be converted into a pointer.
-
memset(dst, c, count)
-
-Same as the standard C memset function. Fills the memory block -at address dst with count bytes of value c. dst must be -an integer specifying an address, or a ctypes instance.
-
pointer(object)
-
-
This function creates a new pointer instance, pointing to the -supplied argument which must be an instance of a ctypes type. The -return pointer is of type POINTER(type(object)). If you have -a ctypes instance, and you want to pass the address of it to a -function call, you should use byref(object) instead which is -much faster.
-
NULL pointer instances are boolean``False``, so to check for a -NULL pointer do this:
-
-# assuming ptr is in ctypes pointer instance -if ptr: - print "Non-NULL pointer instance" -else: - print "NULL pointer instance" -
-
-
POINTER(cls)
-
-
This factory function creates and returns a new ctypes type. -Pointer types are cached, so calling this function is cheap.
-
To create a NULL pointer instance, call the created type -without an argument:
-
-null_ptr = POINTER(c_int)() -
-
-
set_conversion_mode(encoding, errors)
-
-This function sets the encoding/decoding rules which are used when -ctypes has to convert between unicode and byte strings. It -returns the previous encoding, as well as a tuple of any errors. -If not set, default conversions are used: -On Windows, msbc, ignore , on other systems, ascii, strict.
-
sizeof(type_or_object)
-
-Returns the size in bytes of a ctypes type or instance memory -buffer. Does the same as the C sizeof() function.
-
string_at(addr[, size])
-
-This function does the same as the Python PyString_FromString / -PyString_FromStringAndSize C api functions.
-
WinError(code=None, descr=None)
-
-
XXX This is probably the worst named thing in ctypes!
-
This function creates a WindowsError instance. If code is -not specified, GetLastError() is called to determine the error -code. If descr is not specified, FormatError is called to -get a textual description of the error.
-
-
WINFUNCTYPE(restype, *argtypes) (Windows only)
-
-Create a function prototype using the __stdcall calling convention -(on Windows), or using the C calling convention (on Windows CE).
-
wstring_at(addr[, size])
-
-This function does the same as the Python PyUnicode_FromWideString -C api function. If size is not specified, wcslen is used -to determine the string length.
-
-
-
Deprecated functions
-
These deprecated functions are still supported for backwards -comatibility, they should not be used for new code:
-
c_buffer(init, size=None)
-
-Deprecated. Use create_string_buffer() instead.
-
ARRAY(cls, len)
-
-Deprecated. Use cls * len instead.
-
SetPointerType(pointer_class, cls)
-
-Deprecated.
-
-
-
- - Deleted: /external/ctypes/docs/manual/manual.txt ============================================================================== --- /external/ctypes/docs/manual/manual.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,9 +0,0 @@ -============= -ctypes manual -============= - -(work in progress, $Revision: 1.3 $) - -.. contents:: - -.. include:: docs.txt Modified: external/ctypes/docs/manual/mkpydoc.py ============================================================================== --- external/ctypes/docs/manual/mkpydoc.py (original) +++ external/ctypes/docs/manual/mkpydoc.py Sat Jun 10 22:08:12 2006 @@ -1,12 +1,11 @@ -#!/usr/bin/env python -# Convert the ctypes docs to LaTeX for use in Python docs +#!/usr/bin/python + +# Convert the reStructuredText docs to LaTeX for use in Python docs # This script is a hacked version taken from the Optik SVN repository. import sys, os import re -from popen2 import popen2 -from glob import glob import rfc822 from distutils.dep_util import newer_group, newer from docutils.core import Publisher @@ -42,15 +41,7 @@ class PyLaTeXTranslator(LaTeXTranslator): remap_title = { } - - # XXX need to factor this out - module_name = "ctypes" - module_summary = "A foreign function library for Python." - module_type = "standard" - module_author = "Thomas Heller" - module_author_email = "theller at python.net" - module_synopsis = ("A foreign function library for Python.") - version_added = "2.5" + roman = (None,None,"ii","iii","iv","v") refuri_override = { "reference" : "reference-guide", @@ -59,10 +50,14 @@ def __init__(self, document): LaTeXTranslator.__init__(self, document) + self.label_prefix = "" + self.docinfo = {} self.head_prefix = [] self.head = [] self.body_prefix = [] self.in_title = False + self.in_anydesc = False # _title is different if it is a funcdesc + self.admonition_stack = [] # Disable a bunch of methods from the base class. empty_method = lambda self: None @@ -72,19 +67,10 @@ 'field_name'): setattr(self, 'visit_' + nodetype, empty_method) setattr(self, 'depart_' + nodetype, empty_method) - - self.head_prefix = [ - "\\section{\\module{%(module_name)s} --- %(module_summary)s}\n" - "\\declaremodule{%(module_type)s}{%(module_name)s}\n" - "\\moduleauthor{%(module_author)s}{%(module_author_email)s}\n" - "\\modulesynopsis{%(module_synopsis)s}\n" - "\\versionadded{%(version_added)s}\n" - % vars(self.__class__) - ] - # TODO definitions get from latexwriter - # TODO definitions must be guarded if multiple modules are included + self.head_prefix = [] + # definitions must be guarded if multiple modules are included self.definitions = [ - "\\newlength{\\locallinewidth}\n" + "\\ifx\\locallinewidth\\undefined\\newlength{\\locallinewidth}\\fi\n" "\\setlength{\\locallinewidth}{\\linewidth}\n" ] def astext(self): @@ -95,14 +81,16 @@ self.body + self.body_suffix) + def set_label_prefix(self, text): + self.label_prefix = text.replace(" ","-") + def generate_section_label(self, title): title = title.lower() title = re.sub(r'$[^$]*\)', '', title) title = re.sub(r'[^\w\s\-]', '', title) title = re.sub(r'\b(the|an?|and|your|are)\b', '', title) title = re.sub(r'(example \d+).*', r'\1', title) -## title = title.replace("optik", "optparse") - return "ctypes-" + "-".join(title.split()) + return self.label_prefix + "-" + "-".join(title.split()) def visit_document(self, node): pass @@ -111,32 +99,66 @@ pass def visit_docinfo(self, node): - #print "visit_docinfo: %r" % node - self.docinfo = [] + pass def depart_docinfo(self, node): - #print "depart_docinfo: %r" % node - self.body = self.docinfo + self.body - self.docinfo = None + # module and summary are mandatory + self.body.append( + "\\section{\\module{%(module)s} --- %(summary)s}\n" + % self.docinfo ) + if self.docinfo.has_key("moduletype"): + self.body.append( + "\\declaremodule{%(moduletype)s}{%(module)s}\n" + % self.docinfo ) + if self.docinfo.has_key("moduleauthor"): + self.body.append( + "\\moduleauthor{%(moduleauthor)s}{%(moduleauthoremail)s}\n" + % self.docinfo ) + if self.docinfo.has_key("synopsis"): + self.body.append( + "\\modulesynopsis{%(synopsis)s}\n" + % self.docinfo ) + if self.docinfo.has_key("release"): + self.body.append( "\\release{%(release)s}\n" % self.docinfo ) + if self.docinfo.has_key("shortversion"): + self.body.append( "\\setshortversion{%(shortversion)s}\n" + % self.docinfo ) + if self.docinfo.has_key("sectionauthor"): + self.body.append( + "\\sectionauthor{%(sectionauthor)s}{%(sectionauthoremail)s}\n" + % self.docinfo ) + if self.docinfo.has_key("versionadded"): + self.body.append( + "\\versionadded{%(versionadded)s}\n" + % self.docinfo ) def visit_docinfo_item(self, node, name): - #print "visit_docinfo_item: node=%r, name=%r" % (node, name) if name == "author": - (name, email) = rfc822.parseaddr(node.astext()) - self.docinfo.append("\\sectionauthor{%s}{%s}\n" % (name, email)) + (ename, email) = rfc822.parseaddr(node.astext()) + self.docinfo["moduleauthor"] = ename + self.docinfo["moduleauthoremail"] = email raise nodes.SkipNode def depart_docinfo_item(self, node): pass - #def visit_field(self, node): - # (name, value) = (node[0].astext(), node[1].astext()) - # print "visit_field: node=%r (name=%r, value=%r)" % (node, name, value) - # if self.docinfo is not None: - # if name == "VersionAdded": - # self.docinfo.append("\\versionadded{%s}\n" % value) - # raise nodes.SkipNode - + def visit_field(self, node): + if isinstance(node.parent, nodes.docinfo): + name = node[0].astext().lower().replace(" ","") + if name == "moduleauthor": + (ename, email) = rfc822.parseaddr(node[1].astext()) + self.docinfo["moduleauthor"] = ename + self.docinfo["moduleauthoremail"] = email + elif name in ("author", "sectionauthor") : + (ename, email) = rfc822.parseaddr(node[1].astext()) + self.docinfo["sectionauthor"] = ename + self.docinfo["sectionauthoremail"] = email + else: + if name == "module": + self.set_label_prefix(node[1].astext()) + self.docinfo[name] = node[1].astext() + raise nodes.SkipNode + _quoted_string_re = re.compile(r'\"[^\"]*\"') _short_opt_string_re = re.compile(r'-[a-zA-Z]') _long_opt_string_re = re.compile(r'--[a-zA-Z-]+') @@ -147,7 +169,6 @@ def visit_literal(self, node): assert isinstance(node[0], nodes.Text) text = node[0].data -#### text = re.sub(r'optik(\.[a-z]+)?\.', 'optparse.', text) if self.in_title: cmd = None elif self._quoted_string_re.match(text): @@ -171,6 +192,93 @@ if cmd is not None: self.body.append('\\%s{' % cmd) + # use topics for special environments + def visit_topic(self, node): + classes = node.get('classes', ['topic', ]) + if classes[0] in ('datadesc', 'datadescni', 'excdesc', 'classdesc*', + 'csimplemacrodesc', 'ctypedesc', 'memberdesc', + 'memberdescni', 'cvardesc', 'excclassdesc', + 'funcdesc', 'funcdescni', 'methoddesc', + 'methoddescni', 'cmemberdesc', 'classdesc', + 'cfuncdesc'): + self.body.append('\n\\begin{%s}' % classes[0]) + self.context.append('\\end{%s}\n' % classes[0]) + self.in_anydesc = classes[0] + else: + self.context.append('') + + def depart_topic(self, node): + self.in_anydesc = False + self.body.append(self.context.pop()) + + # use definition lists for special environments + # + # definition_list + # defintion_list_item + # term + # classifier + # definition + # paragraph ? + def visit_definition_list(self, node): + pass + + def depart_definition_list(self, node): + pass + + def visit_definition_list_item(self, node): + self._dl_term = [] + + def depart_definition_list_item(self, node): + try: + self.body.append(self.context.pop()) + except: + self.body.append("% WARN definition list without classifier\n") + + + def visit_term(self, node): + self._dl_term.append(node.astext()) + raise nodes.SkipNode + + def depart_term(self, node): + pass + + def visit_classifier(self, node): + # TODO here it should be decided if it is latex or python + classifier = node.astext() + + if classifier in ('datadesc', 'datadescni', 'excdesc', 'classdesc*', + 'csimplemacrodesc', 'ctypedesc', 'memberdesc', + 'memberdescni', 'cvardesc', 'excclassdesc', + 'funcdesc', 'funcdescni', 'methoddesc', + 'methoddescni', 'cmemberdesc', 'classdesc', + 'cfuncdesc'): + pass + else: + classifier = 'datadescni' + self.body.append('\n\\begin{%s}' % classifier) + self.in_anydesc = classifier + self.body.append(self.anydesc_title(self._dl_term.pop())) + self.context.append('\\end{%s}\n' % classifier) + self.in_anydesc = None + raise nodes.SkipNode + + def depart_classifier(self, node): + pass + + def visit_definition(self, node): + if len(self._dl_term)>0: + # no classifier, fake it (maybe make a plain latex description). + classifier = 'datadescni' + self.body.append('\n\\begin{%s}' % classifier) + self.in_anydesc = classifier + self.body.append(self.anydesc_title(self._dl_term.pop())) + self.context.append('\\end{%s}\n' % classifier) + self.in_anydesc = None + + def depart_definition(self, node): + pass + + def depart_literal(self, node): if not self.in_title: self.body.append('}') @@ -184,11 +292,52 @@ self.verbatim = 0 self.body.append("\n\\end{verbatim}\n") + def anydesc_title(self, title): + """Returns the title for xxxdesc environments.""" + def markup_optional_parameters(s): + return s.replace('[','\\optional{').replace(']','}') + def with_params(s): + return markup_optional_parameters( + '{%s}' % s.replace('(','}{').replace(')','')) + def with_tag_or_typename(s, braces): + # "name", "tag name", "name(params)", "type name(params)" + param_pos = s.find("(") + blank_pos = s.find(" ") + if ((blank_pos>0 and param_pos<0) + or (blank_pos>0 and blank_pos %s" % (title, label) section_name = self.d_class.section(self.section_level + 1) self.body.append("\n\n\\%s{" % section_name) self.context.append("\\label{%s}}\n" % label) @@ -197,40 +346,41 @@ def depart_title(self, node): self.in_title = False self.body.append(self.context.pop()) - + def visit_target(self, node): pass def depart_target(self, node): pass - def bookmark(self, node): - pass - - def visit_definition(self, node): - pass - - def depart_definition(self, node): - pass - - def visit_definition_list_item(self, node): - pass + def visit_admonition(self, node, name=''): + self.admonition_stack.append(name) + if name in ('note', 'warning'): + self.body.append('\\begin{notice}[%s]' % name) + else: + LaTeXTranslator.visit_admonition(self, node, name) + def depart_admonition(self, node=None): + name = self.admonition_stack.pop() + if name=="note": + self.body.append('\\end{notice}\n') + else: + LaTeXTranslator.depart_admonition(self, node) - def depart_definition_list_item(self, node): + def bookmark(self, node): pass def visit_reference(self, node): if node.has_key('refuri'): refuri = node['refuri'] basename = os.path.splitext(refuri)[0] - label = "optparse-" + self.refuri_override.get(basename, basename) + label = self.label_prefix + "-" + self.refuri_override.get(basename, basename) print "got refuri=%r, label=%r" % (refuri, label) elif node.has_key('refid'): label = self.generate_section_label(node['refid']) print "got refid=%r, label=%r" % (node['refid'], label) else: print "warning: unhandled reference: node=%r" % node - LaTeXTranslator.visit_reference(self, node) + LaTeXTranslator.visit_reference(self, node) self.body.append("section~\\ref{%s}, " % label) raise nodes.SkipDeparture @@ -247,12 +397,10 @@ text = self._em_dash_re.sub(u"\u2014", text) text = self._quoted_phrase_re.sub(u"\u201C\\1\u201D", text) text = re.sub(r'\bdocument\b', "section", text) -#### text = re.sub(r'optik(\.[a-z]+)?', 'optparse', text) text = self.encode(text) # A couple of transformations are easiest if they go direct # to LaTeX, so do them *after* encode(). -## text = text.replace("Optik", "\\module{optparse}") text = text.replace("UNIX", "\\UNIX{}") self.body.append(text) @@ -260,24 +408,44 @@ def depart_Text(self, node): pass + # table handling + # TODO move table handling into latex2e writer Table class. + def visit_table(self, node): + self.active_table.open() + def depart_table(self, node): + self.body.append('\\end{table%s}\n' % + (self.roman[len(self.active_table._col_specs)]) ) + # TODO use roman to map name ? only i ... iv is supported + self.active_table.close() + def visit_thead(self, node): + self.body.append('\\begin{table%s}{l%s}{textrm}\n' % + (self.roman[len(self.active_table._col_specs)], + '|l'*(len(self.active_table._col_specs)-1) + ) ) + self.active_table.set('preamble written',1) + def depart_thead(self, node): + pass + def visit_row(self, node): + if not isinstance(node.parent, nodes.thead): + self.body.append('\\line%s' % + (self.roman[len(self.active_table._col_specs)], ) + ) + def depart_row(self, node): + # CAUTION: latex2html stuffs content outside of {} into paragraphs + # before the table. + pass + def visit_entry(self, node): + if node.has_key('morerows') or node.has_key('morecols'): + raise NotImplementedError('Cells spanning rows or columns are not' + ' supported.') + # CAUTION: latex2html needs ``\lineii{`` the brace must follow + # immediately + self.body.append('{') + def depart_entry(self, node): + self.body.append('}\n') -def concatenate_sources(sources, target): - print "concatenating source files to %s" % target - outdir = os.path.dirname(target) - if not os.path.isdir(outdir): - os.makedirs(outdir) - outfile = open(target, "wt") - for filename in sources: - file = open(filename, "rt") - for line in file: - outfile.write(line) - outfile.write("\n\n") - file.close() - outfile.close() - def convert(infilename, outfilename): - print "converting %s to %s" % (infilename, outfilename) pub = Publisher() pub.set_components('standalone', # reader @@ -291,8 +459,7 @@ pub.publish() def main(): - convert("tutorial.txt", "tutorial.tex") - convert("tutorial.txt", "../../../trunk/Doc/lib/libctypes.tex") + convert(sys.argv[1], sys.argv[2]) if missing: mod = open("missing.py", "w") mod.write("# possible markups:\n") Modified: external/ctypes/docs/manual/reference.txt ============================================================================== --- external/ctypes/docs/manual/reference.txt (original) +++ external/ctypes/docs/manual/reference.txt Sat Jun 10 22:08:12 2006 @@ -1,746 +1,552 @@ -ctypes manual -+++++++++++++ +ctypes reference +================ +Loading shared libraries +------------------------ -Shared Libraries, DLLs ----------------------- - -Shared libraries are accessed when compiling/linking a program, and -when the program is run. The purpose of the ``find`` method is to -locate a library in a way similar to what the compiler does (on -platforms with several versions of a shared library the most recent -should be loaded), while ``load`` acts like when a program is run, and -uses the runtime loader directly. ``load_version`` works like ``load`` -but tries to be platform independent (for cases where this makes -sense). Loading via attribute access is a shorthand notation -especially usefull for interactive use, it is equivalent to calling -``load_version`` with no version specified. - - -class LibraryLoader -................... - -Instances of ``LibraryLoader`` are used to load shared libraries. -Usually there is no need to create a libraryloader, instead one of the -predefined loaders should be used. - -Libraryloaders have the following methods: - -``find(name, mode=None)`` - - Try to find a library, load and return it. ``name`` is the - library name without any prefix like ``lib``, suffix like ``.so``, - ``.dylib`` or version number (this is the form used for the posix - linker option ``-l``). - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function, ignored on Windows. - - On Windows, this method does exactly the same as the ``load`` - method. - - On other platforms, this function might call other programs like - the compiler to find the library. When using ctypes to write a - shared library wrapping, consider using ``load_version`` or - ``load`` instead. - -``load_version(name, version=None, mode=None)`` - - Build a system dependent filename from ``name`` and optionally - ``version``, then load and return it. ``name`` is the library - name without any prefix like ``lib`` and suffix like ``.so`` or - ``.dylib``. This method should be used if a library is available - on different platforms, using the particular naming convention of - each platform. - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function, ignored on Windows. - - Example: calling ``loader.load_version('z', '1.1.3')`` would - possibly load ``/usr/lib/libz.1.1.3.dylib`` on Mac OS X, or - ``/lib/libz.so.1.1.3`` on a Linux system. - -``load(libname, mode=None)`` - - Load and return the library with the given libname. On most - systems ``libname`` is the filename of the shared library; when - it's not a pathname it will be searched in a system dependent list - of locations (on many systems additional search paths can be - specified by an environment variable). Sometimes the file - extension (like ``.dll`` on Windows) can be omitted. - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function. ``RTLD_LOCAL`` and ``RTLD_GLOBAL`` are - typical values. On Windows, ``mode`` is ignored. - -``load_library(pathname, mode=None`` - - Load and return the library with the given pathname. This method - passes the ``pathname`` directly to the underlying ``dlopen`` or - ``LoadLibrary`` function. - - ``mode`` allows to override the default flags passed to the - ``dlopen()`` function. ``RTLD_LOCAL`` and ``RTLD_GLOBAL`` are - typical values. On Windows, ``mode`` is ignored. - -``LoadLibrary(pathname, mode=None)`` - - This is an alias for the ``load_library`` method documented above, - maintained for backwards comatibility only. - - -Libaries can also be loaded by accessing them as attributes of the -loader instance, internally this calls ``load_version`` without -specifying ``version`` or ``mode``. Obviously this only works for -libraries with names that are valid Python identifiers, and when the -name does not start with a ``_`` character. - -Predefined library loaders -.......................... - -ctypes provides some LibraryLoader instances, the differences between -them are the calling conventions the functions will use and the -default return type of the functions. All these loaders use the -``RTLD_LOCAL`` mode flag. - -Functions can be accessed as named attributes of loaded libraries. - -On Windows, structured exception handling is used around the function -call to protect Python from crashing in case you pass invalid -parameters to the function. - -``cdll`` - - Functions provided by libraries loaded using the ``cdll`` loader - will be called with the standard C calling convention, and have a - default return type of ``int``. ctypes releases the Python global - interpreter lock (GIL) just before calling the foreign function, - and reacquires it before returing, so other threads are able to - run. - -``windll`` - - Windows only. Functions provided by libraries loaded by - ``windll`` will be called using the Windows ``__stdcall`` calling - convention. ctypes can detect when the wrong number - of parameters has been passed to the function call by examining - the stack pointer before and after the function call. If the - wrong parameter count was used, an exception is raised (although - the function really *has* been called). The return value of the - function is lost in this case. Again, the GIL is released during - the duration of the function call. - -``oledll`` - - Windows only. ``oledll`` behaves in the same way as ``windll``, - except that the called function is expected to return a - ``HRESULT`` value. These are long values containing error or - success codes. In case the function returns an error ``HRESULT`` - value, a ``WindowsError`` is raised. The GIL is released during the - duration of function call. - -``pydll`` - - This loader allows to call functions in libraries using the - *Python* calling convention, for example Python C API functions. - The GIL is *not* released during the function call, and the state - of the Python error flag is examined after the function returns. - If the error flag is set, the Python exception is raised. - - ctypes provides a prefabricated instance of ``pydll`` exposing the - Python C api as the ``pythonapi`` symbol, you should however make - sure to set the correct ``restype`` for the functions you use. - -Library objects -............... - -The library loaders create instances of ``CDLL``, ``WinDLL``, -``OleDLL``, or ``PyDLL`` classes. You can, however, also load a -library by constructing one of these classes by calling the -constructor with the pathname of the library and an optional ``mode`` -argument as described in the previous section. - -Library objects implement ``__getattr__`` and ``__getitem__`` methods -that allow to access foreign functions by attribute access or -indexing. The latter is useful if the name of the function is not a -valid Python identifier, or clashes with special Python method names -that start and end with two underscore characters. - -Library objects have two private attributes: ``_name`` is the pathname -of the library, ``_handle`` is the handle to the library that -``dlopen`` has returned. - - -Foreign functions ------------------ - -Functions exported from loaded shared libraries (foreign functions) -can be accessed in two ways. The easiest way is to retrieve them as -attributes of library objects by name:: - - libc = cdll.find("c") # posix - libc = cdll.msvcrt # windows - # attribute access - atoi = libc.atoi - # alternative indexing notation - atoi = libc["atoi"] - -This creates an instance of a foreign function object, using the -calling convention specified by the library object ``cdll``, bound to -the C library ``atoi`` function. The C function is assumed to return -an integer (which is correct for ``atoi``), and the argument types are -not specified (``atoi`` expects a single ``char *`` argument). - -If the library function returns a type different from ``int``, the -``restype`` attribute can be set to a ctypes type that describes the -return type, or to ``None`` meaning no return value (``void``). - -The optional ``argtypes`` attribute can be set to a sequence of ctypes -types that the function expects. - -If needed, the function can (as in C) be called with more arguments -than the length of the argtypes sequence. - -The optional ``errcheck`` attribute can be set to a Python callable, -which can be used to validate and/or process the library function's return -value. ``errcheck`` will be called with three arguments, after the -library function has returned:: - - errcheck(retval, function, arguments) - -``retval`` is the value that the library function returned, converted -according to ``restype``. ``function`` is the ctypes function object -(libc.atoi in this case), and ``arguments`` is a tuple containing the -arguments that have been used to call ``function``. ``errcheck`` -should validate the library function result, raise an error if it -detects a failure, or return the needed return value otherwise. - -Function prototypes -................... - -Another way to access a function exported from shared libraries is to -first create a prototype by calling a factory function, specifying the -return type and the argument types. The factory function itself -specifies the calling convention: ``CFUNCTYPE`` uses the standard C -calling convention, ``WINFUNCTYPE`` (Windows only) uses the stdcall -calling convention. The factory function must be called with the -return type plus the argument types. For the C ``atoi`` function one -would use ``CFUNCTYPE(c_int, c_char_p)``. - -This returns a function prototype, which is a ctypes type representing -all functions that are compatible with the calling convention, return -type, and argument types. - -The ``CFUNCTYPE`` and ``WINFUNCTYPE`` factory functions cache and -reuse the types they create in internal caches, so is is cheap to call -them over and over with the same or different arguments. - -An instance of this function prototype, bound to a foreign library -function, can be created by calling the prototype with the name of the -function as string, and a loaded library:: - - proto = CFUNCTYPE(c_int, c_char_p) - atoi = proto("atoi", libc) - -Parameter flags -............... - -It is possible to specify a third argument ``paramflags`` when calling -the prototype. This is used to specify additional information for -each argument: direction of data transfer, the name, and a default -value. - -A tuple with the same length as ``argtypes`` (the second argument in -the prototype call) must be used. Each item in this tuple must be a -tuple, having either one, two, or three items. - -The first item is the direction flag, an integer specifying if this is -an input (use ``1``) or an output (use ``2``) parameter. The optional -second item is a string containing the parameter name, the optional -third item is a default value for the parameter. - -If parameter names are specified, the function object created can be -called with named arguments in the usual way. Arguments with default -values do not need to be specified when the function is called. - -``out`` parameter types must be pointer types. When the function -object is called, ctypes will automatically create empty instances of -them, pass them to the library function, retrieve the value from them, -and return the value, if there is exactly one ``out`` parameter, or a -tuple of values, if there is more than one ``out`` parameter. The -original foreign function return value is lost in this case (but see -below for how it can be retrieved). - -If ``paramflags`` have been used in the prototype call, and an -``errcheck`` attribute is also present, the ``errcheck`` callable will -be called with a fourth parameter ``outargs``:: - - errcheck(retval, function, arguments, outargs) - -``outargs`` is a tuple containing all the ``out`` parameters that -ctypes has created. Without the ``errcheck`` function ctypes would -retrieve the values contained in these pointer objects, and return -them. The ``errcheck`` function can let ctypes continue this -processing by returning the ``outargs`` tuple. It could also return -something else, or raise an error if it detects that the library -function has failed. - -Callback functions -.................. - -ctypes is able to create C callable functions from Python callables. -This is useful because sometimes library functions need a callback -function parameter; the ``qsort`` C function is such an example. - -Callback functions are created by first creating a function prototype -with a call to ``CFUNCTYPE`` or ``WINFUNCTYPE``, specifying the return -type and the argument types that the callback function will receive. - -Calling the prototype with a single Python callable will create and -return a C-callable function pointer or callback function. Note that -this allows using prototypes as decorators creating callback -functions (Windows example):: - - @WINFUNCTYPE(BOOL, HWND, LPARAM) - def enumwindowsproc(hwnd, lParam): - .... - return True - -When a Python exception is raised in the Python callable, the return -value of the C callable function is undefined. - -Important note: You must keep a reference to the callback AS LONG as -foreign code will call it! Segfaults will result if the callback is -cleaned up by Python's garbage collector and external code then -tries to call it. - -Callback objects can also be called from Python - this may be useful -for debugging. - - -COM methods (Windows only) -.......................... - -XXX Should this be left undocumented? Mentioned for completeness. - -The prototypes created by ``WINFUNCTYPE`` can be called with a -positive small integer ``index``, a string ``name``, an optional -``paramflags`` tuple, and a optional ``iid`` parameter. - -This creates a function object wrapping a COM method. ``index`` is -the index into the COM object's virtual function table, ``name`` is -the name of the COM method (only useful for debugging), ``paramflags`` -has the same meaning as for normal function objects, and ``iid`` is a -string or buffer containing the interface id of the COM interface -this method belongs to. ``iid`` is used to get extended COM error -information in case the method returns a FAILED ''HRESULT`` value. - -Note that COM methods expect an additional first argument that is NOT -listed in the prototypes ``argtypes`` when they are called: this must -be the integer address of a COM interface pointer. - - -Simple types ------------- - -Simple types have some special behaviour: When they are accessed as -structure or union fields, items of array instances, or as foreign -function return values, they are transparently converted from and to -the native Python types int, long, string, and unicode. - -This is *not* the case for subclasses of simple data types, so while a -``c_void_p`` type is transparently converted from and to Python -integer or long, a subclass of c_void_p is *not* converted. This -allows you to define new behaviour almost completely. - -Class attributes of simple types -................................ - -``__ctype__be__``, ``__ctype_le__`` - - If the type supports different byte order (pointer types do NOT - support this), ``__ctype_be__`` and ``__ctype_le__`` are types - with bug endian and little endian byte order. For example, - ``c_int.__ctype_be__`` is an integer type with the memory block in - big endian byte order. - -``_type_`` - - Implementation artifact: the typecode for this type, a single - character string code compatible to what the ``struct`` module uses. - Additional characters are used for types that the ``struct`` module - does not support. - - -Class methods of simple types -............................. - -(To be exact, these are not class methods, instead these are methods -of the metaclass. The most prominent difference to classmethods is -that you can call these methods on the class, but not on the instance -of the simple type.) - -``__ctypes_from_outparam__`` - - TBD - -``from_address`` - - TBD - -``from_param`` - - This class method is used to adapt function parameters. If a type - is specified in a function's argtypes sequence, in a function call - the ``from_param(arg)`` method will be called with the actual - argument, and the result will be passed to the foreign function - call as a parameter. - - ``from_param`` usually returns an internal object that you cannot - use in Python code - it only makes sense to pass this object to - foreign functions. - - On one hand, ``from_param`` is a performance optimization - it - allows you to pass Python integers to function calls expecting a - ``c_int`` argument type, without having to create a full-featured - ``c_int`` instance. - - On the other hand, ``from_param`` can adapt other objects to - parameters. XXX explain the automatic ``byref`` call for byref - arguments. - -``in_dll`` - - TBD - -Instance attributes of simple types -................................... - -``value`` - - Allows to get or set the current value of the object. For simple - types, this is always a native Python object like integer, long, - string, unicode, or None. - -``_objects`` (never modify this) - - Implementation artifact: a Python object keeping references to - other objects which must be kept alive. Never modify anything on - the returned object. XXX Should probably not be exposed. - -``_b_base_`` (readonly) - - Implementation artifact: the base object owning the memory block - (if any). - -``_b_needsfree_`` (readonly) - - Implementation artifact: does this object have to free its memory - block on destruction. - -``_as_parameter_`` (readonly) - - Implementation artifact (?): how to pass this object as a function - parameter. - - -Numeric types -............. - -Integer types are ``c_byte``, ``c_short``, ``c_int``, ``c_long``, -``c_longlong`` and their unsigned variants ``c_ubyte``, ``c_ushort``, -``c_uint``, ``c_ulong`` and ``c_ulonglong``, floating point types are -``c_float`` and ``c_double``. - -The constructor and the ``from_param`` class method accept a Python -integer for integer types, a Python float for floating point types. - -On 32-bit platforms where sizeof(int) == sizeof(long), ``c_int`` is an -alias for ``c_long``, on 64-bit platforms where sizeof(long) == -sizeof(long long), ``c_long`` is an alias for ``c_longlong``. - -Character types -............... - -Character types are ``c_char`` and ``c_wchar``, representing the C -``char`` and ``wchar_t`` types. - -The constructor and the ``from_param`` class method accept a single -character Python string or unicode string. Conversion between string -and unicode, if needed, is done according to the ctypes -encoding/decoding rules. - - -Pointer types -............. - -The only simple pointer type is ``c_void_p``, which represents the C -``void *`` data type. ``c_void_p`` can also be written as -``POINTER(None)``. - -The constructor accepts one optional argument, which must be an -integer or long (interpreted as an address), or ``None``. - -The ``from_param`` class method accepts everything that could be used -as a pointer. XXX Should accept objects using the buffer interface as -well. - -The ``value`` attribute accepts and returns None or integer. - -XXX Shouldn't the constructor accept the same types as from_param? - - -String types -............ - -ctypes has the ``c_char_p`` and ``c_wchar_p`` types which represent -const pointers to zero terminated strings in C: ``const char *`` and -``const wchar_t *``. Since strings and Unicode instances are -immutable, these types should be considered readonly: do not pass them -to functions that write into the buffer. - -The constructor accepts one optional argument, which must be a Python -or unicode string, an integer, or ``None``. - -The ``from_param`` class method accepts a string or a Unicode string, -as well as ``None``. Conversion between string and Unicode, if -needed, is done according to the ctypes encoding/decoding rules. - -XXX Why does the constructor accept an integer, and from_param doesn't? - -Structure and union types -------------------------- - -ctypes provides the abstract base classes ``Structure`` and ``Union`` -to define structure and union types. Subclasses must at least define -a ``_fields_`` attribute. - -Defining field names and types -.............................. - -``_fields_`` must be a sequence of tuples. The first item of each -tuple is a string specifying the name of the structure/union field. -The second item must by a ctypes type. - -A descriptor will be created for each field, allowing you to access the -field's contents from instances. Accessed from the class, the fields -expose readonly ``.offset`` and ``.size`` attributes. ``offset`` is -the byte-offset of the field from the beginning of the -structure/union, ``size`` is the number of bytes the field contains. +``LibraryLoader(dlltype)`` : classdesc + Class which loads shared libraries. -A simple example is a POINT structure containing integer fields named -``x`` and ``y``:: +``LoadLibrary(name, mode=RTLD_LOCAL, handle=None)`` : methoddesc + Load a shared library. - class Point(Structure): - _fields_ = [("x", c_int), - ("y", c_int)] +``CDLL(name, mode=RTLD_LOCAL, handle=None)`` : classdesc + XXX +``cdll`` : vardesc + XXX +``OleDLL(name, mode=RTLD_LOCAL, handle=None)`` : funcdesc + XXX -Field alignment -............... +``oledll`` : vardesc + XXX -Normally fields are aligned in the same way as the host's C compiler -would do it. This native alignment can be overridden by setting a -``_pack_`` attribute in the type. It must be a small positive integer -which is the maximum field alignment. +``py_object`` : classdesc* + XXX -Bit fields -.......... +``PyDLL(name, mode=RTLD_LOCAL, handle=None)`` : funcdesc + XXX -Integer fields support bit sizes. The bit-size must be specified as -the third item of the ``_fields_`` tuple. Bit fields are constructed -in the same way the host's C compiler does it. For bit fields, the -field descriptor's ``.size`` attribute contains the number of bits in -the high word, and the bit offset from the beginning of the structure in -the low word. XXX is that correct? +``pydll`` : vardesc + XXX -Recursive data types -.................... +``RTLD_GLOBAL`` : vardesc + XXX -To define recursive types, it is possible to assign the ``_fields_`` -value *after* the class statement. Here is an example of a linked -list data structure, which contains a pointer to itself:: +``RTLD_LOCAL`` : vardesc + XXX - class Node(Structure): - pass - Node._fields_ = [("next", POINTER(Node)), - ("value", ...)] +``WinDLL(name, mode=RTLD_LOCAL, handle=None)`` : funcdesc + XXX -``_fields_`` must be set, and cannot be changed, after the type is -used for the first time. +``windll`` : vardesc + XXX -Byte order -.......... +``pythonapi()`` : vardesc + XXX -It is possible to create Structure and Union types using non-native -byte order by using the ``BigEndianStructure``, -``LittleEndianStructure``, ``BigEndianUnion``, and -``LittleEndianUnion`` base classes. Structures and Unions with -non-native byte order do *not* support pointer fields. - - -Builtin functions +Foreign functions ----------------- -``addressof(object)`` - - Returns the address of a ctypes instance as an integer. - -``alignment(type_or_object)`` - - Returns the alignment requirements in bytes of a ctypes type or - instance. - -``byref(object)`` - - Returns a light-weight pointer to a ctypes instance. The returned - object can only be used as function call parameter. Behaves the - same as calling ``pointer(object)``, but is a lot faster. Same as - ``&object`` in C. - -``cast(object, typ)`` - - This function is similar to the cast operator in C. Returns a new - instance of ``type`` which shares the memory block of ``object``. - ``typ`` must be a pointer type. - -``CFUNCTYPE(restype, *argtypes)`` - - Create a function prototype using the C calling convention. - - -``create_string_buffer(init, size=None)`` - - Convenience function to create a mutable character buffer. - - ``init`` must be a string. If ``size`` is supplied it must be a - positive integer that specifies the size of the buffer, otherwise - the length of the ``init`` string is used. - This function returns a ctypes array of characters ``c_char``. - -``create_unicode_buffer(init, size=None)`` - - Convenience function to create a mutable unicode buffer. - - ``init`` must be a unicode string. If ``size`` is supplied it - must be a positive integer that specifies the number of characters - in the buffer, otherwise the length of the ``init`` string is - used. This function returns a ctypes array of characters ``c_wchar``. - - -``DllCanUnloadNow()``, ``DllGetClassObject(rclsid, riid, ppv)`` (Windows only) - - Functions used to implement COM servers. - - -``FormatError([code])`` (Windows only) - - Returns a textual description of the error code, or the last error - code set by Windows. - -``GetLastError()`` (Windows only) - - Returns the last error code set by Windows. - -``memmove(dst, src, count)`` +The ultimate goal of ``ctypes`` is to call functions in shared +libraries, aka as foreign functions. Foreign function instances can +be created by retrieving them as attributes of loaded shared +libraries, or by instantiating a *function prototype*. + +By default, functions got as attributes of loaded shared libraries +accept any arguments, and have a return type of ``c_int``. + +Function prototypes are created by factory functions. + +They are created by calling one of the following factory functions: + +``CFUNCTYPE(restype, *argtypes)`` : funcdesc + This is a factory function that returns a function prototype. The + function prototype describes a function that has a result type of + ``restype``, and accepts arguments as specified by + ``argtypes``. The function prototype can be used to construct + several kinds of functions, depending on how the prototype is + called. + + The prototypes returned by ``CFUNCTYPE`` or ``PYFUNCTYPE`` create + functions that use the standard C calling convention, prototypes + returned from ``WINFUNCTYPE`` (on Windows) use the ``__stdcall`` + calling convention. + + Functions created by calling the ``CFUNCTYPE`` and ``WINFUNCTYPE`` + prototypes release the Python GIL before entering the foreign + function, and acquire it back after leaving the function code. - Same as the standard C ``memmove`` library function: copies - ``count`` bytes from ``src`` to ``dst``. ``dst`` and ``src`` must - be integers or anything else that can be converted into a pointer. - -``memset(dst, c, count)`` - - Same as the standard C ``memset`` function. Fills the memory block - at address ``dst`` with ``count`` bytes of value ``c``. ``dst`` must be - an integer specifying an address, or a ctypes instance. - - -``pointer(object)`` - - This function creates a new pointer instance, pointing to the - supplied argument which must be an instance of a ctypes type. The - return pointer is of type ``POINTER(type(object))``. If you have - a ctypes instance, and you want to pass the address of it to a - function call, you should use ``byref(object)`` instead which is - much faster. - - NULL pointer instances are boolean``False``, so to check for a - NULL pointer do this:: - - # assuming ptr is in ctypes pointer instance - if ptr: - print "Non-NULL pointer instance" - else: - print "NULL pointer instance" - - -``POINTER(cls)`` - - This factory function creates and returns a new ctypes type. - Pointer types are cached, so calling this function is cheap. - - To create a ``NULL`` pointer instance, call the created type - without an argument:: +``WINFUNCTYPE(restype, *argtypes)`` : funcdesc + TBD - null_ptr = POINTER(c_int)() +``PYFUNCTYPE(restype, *argtypes)`` : funcdesc + TBD +``ArgumentError()`` : excdesc + This exception is raised when a foreign function call cannot + convert one of the passed arguments. -``set_conversion_mode(encoding, errors)`` - This function sets the encoding/decoding rules which are used when - ctypes has to convert between unicode and byte strings. It - returns the previous encoding, as well as a tuple of any errors. - If not set, default conversions are used: - On Windows, ``msbc, ignore`` , on other systems, ``ascii, strict``. +Utility functions +----------------- -``sizeof(type_or_object)`` +``addressof(obj)`` : funcdesc + Returns the address of the memory buffer as integer. ``obj`` must + be an instance of a ctypes type. + +``alignment(obj_or_type)`` : funcdesc + Returns the alignment requirements of a ctypes type. + ``obj_or_type`` must be a ctypes type or instance. + +``byref(obj)`` : funcdesc + Returns a light-weight pointer to ``obj``, which must be an + instance of a ctypes type. The returned object can only be used as + a foreign function call parameter. It behaves similar to + ``pointer(obj)``, but the construction is a lot faster. + +``cast(obj, type)`` : funcdesc + This function is similar to the cast operator in C. It returns a + new instance of ``type`` which points to the same memory block as + ``obj``. ``type`` must be a pointer type, and ``obj`` must be an + object that can be interpreted as a pointer. + +``create_string_buffer(init_or_size[, size])`` : funcdesc + This function creates a mutable character buffer. The returned + object is a ctypes array of ``c_char``. + + ``init_or_size`` must be an integer which specifies the size of + the array, or a string which will be used to initialize the array + items. + + If a string is specified as first argument, the buffer is made one + item larger than the length of the string so that the last element + in the array is a NUL termination character. An integer can be + passed as second argument which allows to specify the size of the + array if the length of the string should not be used. + + If the first parameter is a unicode string, it is converted into + an 8-bit string according to ctypes conversion rules. + +``create_unicode_buffer(init_or_size[, size])`` : funcdesc + This function creates a mutable unicode character buffer. The + returned object is a ctypes array of ``c_wchar``. + + ``init_or_size`` must be an integer which specifies the size of + the array, or a unicode string which will be used to initialize + the array items. + + If a unicode string is specified as first argument, the buffer is + made one item larger than the length of the string so that the + last element in the array is a NUL termination character. An + integer can be passed as second argument which allows to specify + the size of the array if the length of the string should not be + used. + + If the first parameter is a 8-bit string, it is converted into an + unicode string according to ctypes conversion rules. + +``DllCanUnloadNow()`` : funcdesc + Windows only: This function is a hook which allows to implement + inprocess COM servers with ctypes. It is called from the + DllCanUnloadNow function that the _ctypes extension dll exports. + +``DllGetClassObject()`` : funcdesc + Windows only: This function is a hook which allows to implement + inprocess COM servers with ctypes. It is called from the + DllGetClassObject function that the ``_ctypes`` extension dll exports. + +``FormatError([code])`` : funcdesc + Windows only: Returns a textual description of the error code. If + no error code is specified, the last error code is used by calling + the Windows api function GetLastError. + +``GetLastError()`` : funcdesc + Windows only: Returns the last error code set by Windows in the + calling thread. + +``memmove(dst, src, count)`` : funcdesc + Same as the standard C memmove library function: copies ``count`` + bytes from ``src`` to ``dst``. ``dst`` and ``src`` must be + integers or ctypes instances that can be converted to pointers. + +``memset(dst, c, count)`` : funcdesc + Same as the standard C memset library function: fills the memory + block at address ``dst`` with ``count`` bytes of value + ``c``. ``dst`` must be an integer specifying an address, or a + ctypes instance. + +``POINTER(type)`` : funcdesc + This factory function creates and returns a new ctypes pointer + type. Pointer types are cached an reused internally, so calling + this function repeatedly is cheap. type must be a ctypes type. + +``pointer(obj)`` : funcdesc + This function creates a new pointer instance, pointing to + ``obj``. The returned object is of the type POINTER(type(obj)). + + Note: If you just want to pass a pointer to an object to a foreign + function call, you should use ``byref(obj)`` which is much faster. + +``resize(obj, size)`` : funcdesc + This function resizes the internal memory buffer of obj, which + must be an instance of a ctypes type. It is not possible to make + the buffer smaller than the native size of the objects type, as + given by sizeof(type(obj)), but it is possible to enlarge the + buffer. + +``set_conversion_mode(encoding, errors)`` : funcdesc + This function sets the rules that ctypes objects use when + converting between 8-bit strings and unicode strings. encoding + must be a string specifying an encoding, like ``'utf-8'`` or + ``'mbcs'``, errors must be a string specifying the error handling + on encoding/decoding errors. Examples of possible values are + ``"strict"``, ``"replace"``, or ``"ignore"``. + + set_conversion_mode returns a 2-tuple containing the previous + conversion rules. On windows, the initial conversion rules are + ``('mbcs', 'ignore')``, on other systems ``('ascii', 'strict')``. +``sizeof(obj_or_type)`` : funcdesc Returns the size in bytes of a ctypes type or instance memory - buffer. Does the same as the C sizeof() function. - + buffer. Does the same as the C ``sizeof()`` function. -``string_at(addr[, size])`` - - This function does the same as the Python ``PyString_FromString`` / - ``PyString_FromStringAndSize`` C api functions. - -``WinError(code=None, descr=None)`` - - XXX This is probably the worst named thing in ctypes! - - This function creates a ``WindowsError`` instance. If ``code`` is - not specified, GetLastError() is called to determine the error - code. If ``descr`` is not specified, ``FormatError`` is called to +``string_at(address[, size])`` : funcdesc + This function returns the string starting at memory address + address. If size is specified, it is used as size, otherwise the + string is assumed to be zero-terminated. + +``WinError(code=None, descr=None)`` : funcdesc + Windows only: this function is probably the worst-named thing in + ctypes. It creates an instance of WindowsError. If ``code`` is not + specified, ``GetLastError`` is called to determine the error + code. If ``descr`` is not spcified, ``FormatError`` is called to get a textual description of the error. -``WINFUNCTYPE(restype, *argtypes)`` (Windows only) - - Create a function prototype using the __stdcall calling convention - (on Windows), or using the C calling convention (on Windows CE). - -``wstring_at(addr[, size])`` - - This function does the same as the Python ``PyUnicode_FromWideString`` - C api function. If ``size`` is not specified, ``wcslen`` is used - to determine the string length. +``wstring_at(address)`` : funcdesc + This function returns the wide character string starting at memory + address ``address`` as unicode string. If ``size`` is specified, + it is used as the number of characters of the string, otherwise + the string is assumed to be zero-terminated. + + +Data types +---------- + +``_CData`` : classdesc* + This non-public class is the common base class of all ctypes data + types. Among other things, all ctypes type instances contain a + memory block that hold C compatible data; the address of the + memory block is returned by the ``addressof()`` helper function. + Another instance variable is exposed as ``_objects``; this + contains other Python objects that need to be kept alive in case + the memory block contains pointers. + +Common methods of ctypes data types, these are all class methods (to +be exact, they are methods of the metaclass): + +``from_address(address)`` : methoddesc + This method returns a ctypes type instance using the memory + specified by address. + +``from_param(obj)`` : methoddesc + This method adapts obj to a ctypes type. + +``in_dll(name, library)`` : methoddesc + This method returns a ctypes type instance exported by a shared + library. ``name`` is the name of the symbol that exports the data, + ``library`` is the loaded shared library. + +Common instance variables of ctypes data types: + +``_b_base_`` : memberdesc + Sometimes ctypes data instances do not own the memory block they + contain, instead they share part of the memory block of a base + object. The ``_b_base_`` readonly member is the root ctypes + object that owns the memory block. + +``_b_needsfree_`` : memberdesc + This readonly variable is true when the ctypes data instance has + allocated the memory block itself, false otherwise. + +``_objects`` : memberdesc + This member is either ``None`` or a dictionary containing Python + objects that need to be kept alive so that the memory block + contents is kept valid. This object is only exposed for + debugging; never modify the contents of this dictionary. +Fundamental data types +---------------------- -Deprecated functions -.................... - -These deprecated functions are still supported for backwards -comatibility, they should not be used for new code: - -``c_buffer(init, size=None)`` - - Deprecated. Use ``create_string_buffer()`` instead. - -``ARRAY(cls, len)`` - - Deprecated. Use ``cls * len`` instead. +``_SimpleCData`` : classdesc* + This non-public class is the base class of all fundamental ctypes + data types. It is mentioned here because it contains the common + attributes of the fundamental ctypes data types. ``_SimpleCData`` + is a subclass of ``_CData``, so it inherits their methods and + attributes. + +Instances have a single attribute: + +``value`` : memberdesc + This attribute contains the actual value of the instance. For + integer and pointer types, it is an integer, for character types, + it is a single character string, for character pointer types it + is a Python string or unicode string. + + When the ``value`` attribute is retrieved from a ctypes instance, + usually a new object is returned each time. ``ctypes`` does *not* + implement original object return, always a new object is + constructed. The same is true for all other ctypes object + instances. + +Fundamental data types, when returned as foreign function call +results, or, for example, by retrieving structure field members or +array items, are transparently converted to native Python types. In +other words, if a foreign function has a ``restype`` of ``c_char_p``, +you will always receive a Python string, *not* a ``c_char_p`` +instance. + +Subclasses of fundamental data types do *not* inherit this behaviour. +So, if a foreign functions ``restype`` is a subclass of ``c_void_p``, +you will receive an instance of this subclass from the function call. +Of course, you can get the value of the pointer by accessing the +``value`` attribute. + +These are the fundamental ctypes data types: + +``c_byte`` : classdesc* + Represents the C signed char datatype, and interprets the value as + small integer. The constructor accepts an optional integer + initializer; no overflow checking is done. + +``c_char`` : classdesc* + Represents the C char datatype, and interprets the value as a single + character. The constructor accepts an optional string initializer, + the length of the string must be exactly one character. + +``c_char_p`` : classdesc* + Represents the C char * datatype, which must be a pointer to a + zero-terminated string. The constructor accepts an integer + address, or a string. + +``c_double`` : classdesc* + Represents the C double datatype. The constructor accepts an + optional float initializer. + +``c_float`` : classdesc* + Represents the C double datatype. The constructor accepts an + optional float initializer. + +``c_int`` : classdesc* + Represents the C signed int datatype. The constructor accepts an + optional integer initializer; no overflow checking is done. On + platforms where ``sizeof(int) == sizeof(long)`` it is an alias to + ``c_long``. + +``c_int8`` : classdesc* + Represents the C 8-bit ``signed int`` datatype. Usually an alias for + ``c_byte``. + +``c_int16`` : classdesc* + Represents the C 16-bit signed int datatype. Usually an alias for + ``c_short``. + +``c_int32`` : classdesc* + Represents the C 32-bit signed int datatype. Usually an alias for + ``c_int``. + +``c_int64`` : classdesc* + Represents the C 64-bit ``signed int`` datatype. Usually an alias + for ``c_longlong``. + +``c_long`` : classdesc* + Represents the C ``signed long`` datatype. The constructor accepts an + optional integer initializer; no overflow checking is done. + +``c_longlong`` : classdesc* + Represents the C ``signed long long`` datatype. The constructor accepts + an optional integer initializer; no overflow checking is done. + +``c_short`` : classdesc* + Represents the C ``signed short`` datatype. The constructor accepts an + optional integer initializer; no overflow checking is done. + +``c_size_t`` : classdesc* + Represents the C ``size_t`` datatype. + +``c_ubyte`` : classdesc* + Represents the C ``unsigned char`` datatype, it interprets the + value as small integer. The constructor accepts an optional + integer initializer; no overflow checking is done. + +``c_uint`` : classdesc* + Represents the C ``unsigned int`` datatype. The constructor accepts an + optional integer initializer; no overflow checking is done. On + platforms where ``sizeof(int) == sizeof(long)`` it is an alias for + ``c_ulong``. + +``c_uint8`` : classdesc* + Represents the C 8-bit unsigned int datatype. Usually an alias for + ``c_ubyte``. + +``c_uint16`` : classdesc* + Represents the C 16-bit unsigned int datatype. Usually an alias for + ``c_ushort``. + +``c_uint32`` : classdesc* + Represents the C 32-bit unsigned int datatype. Usually an alias for + ``c_uint``. + +``c_uint64`` : classdesc* + Represents the C 64-bit unsigned int datatype. Usually an alias for + ``c_ulonglong``. + +``c_ulong`` : classdesc* + Represents the C ``unsigned long`` datatype. The constructor accepts an + optional integer initializer; no overflow checking is done. + +``c_ulonglong`` : classdesc* + Represents the C ``unsigned long long`` datatype. The constructor + accepts an optional integer initializer; no overflow checking is + done. + +``c_ushort`` : classdesc* + Represents the C ``unsigned short`` datatype. The constructor accepts an + optional integer initializer; no overflow checking is done. + +``c_void_p`` : classdesc* + Represents the C ``void *`` type. The value is represented as + integer. The constructor accepts an optional integer initializer. + +``c_wchar`` : classdesc* + Represents the C ``wchar_t`` datatype, and interprets the value as a + single character unicode string. The constructor accepts an + optional string initializer, the length of the string must be + exactly one character. + +``c_wchar_p`` : classdesc* + Represents the C ``wchar_t *`` datatype, which must be a pointer to + a zero-terminated wide character string. The constructor accepts + an integer address, or a string. + +``HRESULT`` : classdesc* + Windows only: Represents a ``HRESULT`` value, which contains success + or error information for a function or method call. + +Structured data types +--------------------- + +``Union(*args, **kw)`` : classdesc + Abstract base class for unions in native byte order. + +``BigEndianStructure(*args, **kw)`` : classdesc + Abstract base class for structures in *big endian* byte order. + +``LittleEndianStructure(*args, **kw)`` : classdesc + Abstract base class for structures in *little endian* byte order. + +Structures with non-native byte order cannot contain pointer type +fields, or any other data types containing pointer type fields. + +``Structure(*args, **kw)`` : classdesc + Abstract base class for structures in *native* byte order. + +Concrete structure and union types must be created by subclassing one +of these types, and at least define a ``_fields_`` class variable. +``ctypes`` will create descriptors which allow reading and writing the +fields by direct attribute accesses. These are the + +``_fields_`` : memberdesc + A sequence defining the structure fields. The items must be + 2-tuples or 3-tuples. The first item is the name of the field, + the second item specifies the type of the field; it can be any + ctypes data type. + + For integer type fields, a third optional item can be given. It + must be a small positive integer defining the bit width of the + field. + + Field names must be unique within one structure or union. This is + not checked, only one field can be accessed when names are + repeated. + + It is possible to define the ``_fields_`` class variable *after* + the class statement that defines the Structure subclass, this + allows to create data types that directly or indirectly reference + themselves:: + + class List(Structure): + pass + List._fields_ = [("pnext", POINTER(List)), + ... + ] + + The ``_fields_`` class variable must, however, be defined before + the type is first used (an instance is created, ``sizeof()`` is + called on it, and so on). Later assignments to the ``_fields_`` + class variable will raise an AttributeError. + + Structure and union subclass constructors accept both positional + and named arguments. Positional arguments are used to initialize + the fields in the same order as they appear in the ``_fields_`` + definition, named arguments are used to initialize the fields with + the corresponding name. + + It is possible to defined sub-subclasses of structure types, they + inherit the fields of the base class plus the ``_fields_`` defined + in the sub-subclass, if any. + + +``_pack_`` : memberdesc + An optional small integer that allows to override the alignment of + structure fields in the instance. ``_pack_`` must already be + defined when ``_fields_`` is assigned, otherwise it will have no + effect. + +``_anonymous_`` : memberdesc + An optional sequence that lists the names of unnamed (anonymous) + fields. ``_anonymous_`` must be already defined when ``_fields_`` + is assigned, otherwise it will have no effect. + + The fields listed in this variable must be structure or union type + fields. ``ctypes`` will create descriptors in the structure type + that allows to access the nested fields directly, without the need + to create the structure or union field. + + Here is an example type (Windows):: + + class _U(Union): + _fields_ = [("lptdesc", POINTER(TYPEDESC)), + ("lpadesc", POINTER(ARRAYDESC)), + ("hreftype", HREFTYPE)] + + class TYPEDESC(Structure): + _fields_ = [("u", _U), + ("vt", VARTYPE)] + + _anonymous_ = ("u",) + + The ``TYPEDESC`` structure describes a COM data type, the ``vt`` + field specifies which one of the union fields is valid. Since the + ``u`` field is defined as anonymous field, it is now possible to + access the members directly off the TYPEDESC instance. + ``td.lptdesc`` and ``td.u.lptdesc`` are equivalent, but the former + is faster since it does not need to create a temporary ``_U`` + instance:: + + td = TYPEDESC() + td.vt = VT_PTR + td.lptdesc = POINTER(some_type) + td.u.lptdesc = POINTER(some_type) + +It is possible to defined sub-subclasses of structures, they inherit +the fields of the base class. If the subclass definition has a +separate``_fields_`` variable, the fields specified in this are +appended to the fields of the base class. -``SetPointerType(pointer_class, cls)`` +Arrays and pointers +------------------- - Deprecated. +XXX Deleted: /external/ctypes/docs/manual/simple_types.txt ============================================================================== --- /external/ctypes/docs/manual/simple_types.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,167 +0,0 @@ -Simple types -++++++++++++ - -Simple types have some special behaviour: When they are accessed as -structure or union fields, items of array instances, or as foreign -function return values, they are transparently converted from and to -the native Python types int, long, string, and unicode. - -This is *not* the case for subclasses of simple data types, so while a -``c_void_p`` type is transparently converted from and to Python -integer or long, a subclass of c_void_p is *not* converted. This -allows you to define new behaviour almost completely. - -Class attributes of simple types --------------------------------- - -``__ctype_be__``, ``__ctype_le__`` - - If the type supports different byte order (pointer types do NOT - support this), ``__ctype_be__`` and ``__ctype_le__`` are types - with bug endian and little endian byte order. For example, - ``c_int.__ctype_be__`` is an integer type with the memory block in - big endian byte order. - -``_type_`` - - Implementation artifact: the typecode for this type, a single - character string code compatible to what the ``struct`` module uses. - Additional characters are used for types that the ``struct`` module - does not support. - - -Class methods of simple types ------------------------------ - -(To be exact, these are not class methods, instead these are methods -of the metaclass. The most prominent difference to classmethods is -that you can call these methods on the class, but not on the instance -of the simple type.) - -``__ctypes_from_outparam__`` - - TBD - -``from_address`` - - TBD - -``from_param`` - - This class method is used to adapt function parameters. If a type - is specified in a function's argtypes sequence, in a function call - the ``from_param(arg)`` method will be called with the actual - argument, and the result will be passed to the foreign function - call as a parameter. - - ``from_param`` usually returns an internal object that you cannot - use in Python code - it only makes sense to pass this object to - foreign functions. - - On one hand, ``from_param`` is a performance optimization - it - allows you to pass Python integers to function calls expecting a - ``c_int`` argument type, without having to create a full-featured - ``c_int`` instance. - - On the other hand, ``from_param`` can adapt other objects to - parameters. XXX explain the automatic ``byref`` call for byref - arguments. - -``in_dll`` - - TBD - -Instance attributes of simple types ------------------------------------ - -``value`` - - Allows to get or set the current value of the object. For simple - types, this is always a native Python object like integer, long, - string, unicode, or None. - -``_objects`` (never modify this) - - Implementation artifact: a Python object keeping references to - other objects which must be kept alive. Never modify anything on - the returned object. XXX Should probably not be exposed. - -``_b_base_`` (readonly) - - Implementation artifact: the base object owning the memory block - (if any). - -``_b_needsfree_`` (readonly) - - Implementation artifact: does this object have to free its memory - block on destruction. - -``_as_parameter_`` (readonly) - - Implementation artifact (?): how to pass this object as a function - parameter. - - -Numeric types -------------- - -Integer types are ``c_byte``, ``c_short``, ``c_int``, ``c_long``, -``c_longlong`` and their unsigned variants ``c_ubyte``, ``c_ushort``, -``c_uint``, ``c_ulong`` and ``c_ulonglong``, floating point types are -``c_float`` and ``c_double``. - -The constructor and the ``from_param`` class method accept a Python -integer for integer types, a Python float for floating point types. - -On 32-bit platforms where sizeof(int) == sizeof(long), ``c_int`` is an -alias for ``c_long``, on 64-bit platforms where sizeof(long) == -sizeof(long long), ``c_long`` is an alias for ``c_longlong``. - -Character types ---------------- - -Character types are ``c_char`` and ``c_wchar``, representing the C -``char`` and ``wchar_t`` types. - -The constructor and the ``from_param`` class method accept a single -character Python string or unicode string. Conversion between string -and unicode, if needed, is done according to the ctypes -encoding/decoding rules. - - -Pointer types -------------- - -The only simple pointer type is ``c_void_p``, which represents the C -``void *`` data type. ``c_void_p`` can also be written as -``POINTER(None)``. - -The constructor accepts one optional argument, which must be an -integer or long (interpreted as an address), or ``None``. - -The ``from_param`` class method accepts everything that could be used -as a pointer. XXX Should accept objects using the buffer interface as -well. - -The ``value`` attribute accepts and returns None or integer. - -XXX Shouldn't the constructor accept the same types as from_param? - - -String types ------------- - -ctypes has the ``c_char_p`` and ``c_wchar_p`` types which represent -const pointers to zero terminated strings in C: ``const char *`` and -``const wchar_t *``. Since strings and Unicode instances are -immutable, these types should be considered readonly: do not pass them -to functions that write into the buffer. - -The constructor accepts one optional argument, which must be a Python -or unicode string, an integer, or ``None``. - -The ``from_param`` class method accepts a string or a Unicode string, -as well as ``None``. Conversion between string and Unicode, if -needed, is done according to the ctypes encoding/decoding rules. - -XXX Why does the constructor accept an integer, and from_param doesn't? \ No newline at end of file Deleted: /external/ctypes/docs/manual/struct_union.txt ============================================================================== --- /external/ctypes/docs/manual/struct_union.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,70 +0,0 @@ -Structure and union types -+++++++++++++++++++++++++ - -ctypes provides the abstract base classes ``Structure`` and ``Union`` -to define structure and union types. Subclasses must at least define -a ``_fields_`` attribute. - -Defining field names and types ------------------------------- - -``_fields_`` must be a sequence of tuples. The first item of each -tuple is a string specifying the name of the structure/union field. -The second item must by a ctypes type. - -A descriptor will be created for each field, allowing you to access the -field's contents from instances. Accessed from the class, the fields -expose readonly ``.offset`` and ``.size`` attributes. ``offset`` is -the byte-offset of the field from the beginning of the -structure/union, ``size`` is the number of bytes the field contains. - -A simple example is a POINT structure containing integer fields named -``x`` and ``y``:: - - class Point(Structure): - _fields_ = [("x", c_int), - ("y", c_int)] - - - -Packing fields --------------- - -Normally fields are aligned in the same way as the host's C compiler -would do it. This native alignment can be overridden by setting a -``_pack_`` attribute in the type. It must be a small positive integer -which is the maximum field alignment. - -Bit fields ----------- - -Integer fields support bit sizes. The bit-size must be specified as -the third item of the ``_fields_`` tuple. Bit fields are constructed -in the same way the host's C compiler does it. For bit fields, the -field descriptor's ``.size`` attribute contains the number of bits in -the high word, and the bit offset from the beginning of the structure in -the low word. XXX is that correct? - -Recursive data types --------------------- - -To define recursive types, it is possible to assign the ``_fields_`` -value *after* the class statement. Here is an example of a linked -list data structure, which contains a pointer to itself:: - - class Node(Structure): - pass - Node._fields_ = [("next", POINTER(Node)), - ("value", ...)] - -``_fields_`` must be set, and cannot be changed, after the type is -used for the first time. - -Byte order ----------- - -It is possible to create Structure and Union types using non-native -byte order by using the ``BigEndianStructure``, -``LittleEndianStructure``, ``BigEndianUnion``, and -``LittleEndianUnion`` base classes. Structures and Unions with -non-native byte order do *not* support pointer fields. Deleted: /external/ctypes/docs/manual/table.txt ============================================================================== --- /external/ctypes/docs/manual/table.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,41 +0,0 @@ - +---------------+-----------------------+-----------+ - |ctypes' type |C type |Python type| - +===============+=======================+===========+ - |``c_char`` |``char`` |character | - +---------------+-----------------------+-----------+ - |``c_byte`` |``char`` |integer | - +---------------+-----------------------+-----------+ - |``c_ubyte`` |``unsigned char`` |integer | - +---------------+-----------------------+-----------+ - |``c_short`` |``short`` |integer | - +---------------+-----------------------+-----------+ - |``c_ushort`` |``unsigned short`` |integer | - +---------------+-----------------------+-----------+ - |``c_int`` |``int`` |integer | - +---------------+-----------------------+-----------+ - |``c_uint`` |``unsigned int`` |integer | - +---------------+-----------------------+-----------+ - |``c_long`` |``long`` |integer | - +---------------+-----------------------+-----------+ - |``c_ulong`` |``unsigned long`` |long | - +---------------+-----------------------+-----------+ - |``c_longlong`` |``__int64`` or |long | - | |``long long`` | | - +---------------+-----------------------+-----------+ - |``c_ulonglong``|``unsigned __int64`` or|long | - | |``unsigned long long`` | | - +---------------+-----------------------+-----------+ - |``c_float`` |``float`` |float | - +---------------+-----------------------+-----------+ - |``c_double`` |``double`` |float | - +---------------+-----------------------+-----------+ - |``c_char_p`` |``char *`` |string or | - | |(NUL terminated) |``None`` | - +---------------+-----------------------+-----------+ - |``c_wchar_p`` |``wchar_t *`` |unicode or | - | |(NUL terminated) |``None`` | - +---------------+-----------------------+-----------+ - |``c_void_p`` |``void *`` |integer or | - | | |``None`` | - +---------------+-----------------------+-----------+ - Modified: external/ctypes/docs/manual/test-tutorial.py ============================================================================== --- external/ctypes/docs/manual/test-tutorial.py (original) +++ external/ctypes/docs/manual/test-tutorial.py Sat Jun 10 22:08:12 2006 @@ -5,6 +5,7 @@ # handle platform specific issues WINDOWS = doctest.register_optionflag("WINDOWS") LINUX = doctest.register_optionflag("LINUX") +OSX = doctest.register_optionflag("OSX") SKIP = doctest.register_optionflag("SKIP") # handle size specific issues @@ -20,6 +21,8 @@ examples.remove(ex) elif LINUX in ex.options and not sys.platform.startswith("linux"): examples.remove(ex) + elif OSX in ex.options and not sys.platform.startswith("darwin"): + examples.remove(ex) elif SKIP in ex.options: examples.remove(ex) elif "printf(" in ex.source: @@ -38,5 +41,11 @@ doctest.DocTestRunner = MyDocTestRunner if __name__ == "__main__": + # Python 2.5a2 formats exceptions differently than before, so we + # add IGNORE_EXCEPTION_DETAIL. I do not know if this will be + # fixed or not. (Is apparently fixed). + # + # ctypes may give localized error messages from the operating + # syetem, so we need IGNORE_EXCEPTION_DETAIL anyway. doctest.testfile("tutorial.txt", - optionflags=doctest.ELLIPSIS) + optionflags=doctest.ELLIPSIS | doctest.IGNORE_EXCEPTION_DETAIL) Modified: external/ctypes/docs/manual/tutorial.txt ============================================================================== --- external/ctypes/docs/manual/tutorial.txt (original) +++ external/ctypes/docs/manual/tutorial.txt Sat Jun 10 22:08:12 2006 @@ -1,5 +1,3 @@ -``ctypes`` is a foreign function library for Python. - ctypes tutorial =============== @@ -59,6 +57,12 @@ XXX Add section for Mac OS X. +Finding shared libraries +------------------------ + +XXX Add description of ctypes.util.find_library (once I really +understand it enough to describe it). + Accessing functions from loaded dlls ------------------------------------ @@ -168,8 +172,8 @@ Before we move on calling functions with other parameter types, we have to learn more about ``ctypes`` data types. -Simple data types ------------------ +Fundamental data types +---------------------- ``ctypes`` defines a number of primitive C compatible data types : @@ -254,6 +258,7 @@ c_char_p('Hi, there') >>> print s # first string is unchanged Hello, World + >>> You should be careful, however, not to pass them to functions expecting pointers to mutable memory. If you need mutable memory @@ -442,7 +447,7 @@ >>> GetModuleHandle.restype = ValidHandle # doctest: +WINDOWS >>> GetModuleHandle(None) # doctest: +WINDOWS 486539264 - >>> GetModuleHandle("something silly") # doctest: +WINDOWS +IGNORE_EXCEPTION_DETAIL + >>> GetModuleHandle("something silly") # doctest: +WINDOWS Traceback (most recent call last): File "", line 1, in ? File "", line 3, in ValidHandle @@ -606,6 +611,7 @@ >>> >>> print len(MyStruct().point_array) 4 + >>> Instances are created in the usual way, by calling the class:: @@ -632,54 +638,64 @@ Pointers -------- +XXX Rewrite this section. Normally one only uses indexing, not the .contents +attribute! +List some recipes with pointers. bool(ptr), POINTER(tp)(), ...? + Pointer instances are created by calling the ``pointer`` function on a ``ctypes`` type:: - >>> from ctypes import * - >>> i = c_int(42) - >>> pi = pointer(i) - >>> - -XXX XXX Not correct: use indexing, not the contents atribute + >>> from ctypes import * + >>> i = c_int(42) + >>> pi = pointer(i) + >>> Pointer instances have a ``contents`` attribute which returns the -ctypes' type pointed to, the ``c_int(42)`` in the above case:: +object to which the pointer points, the ``i`` object above:: - >>> pi.contents - c_long(42) - >>> + >>> pi.contents + c_long(42) + >>> + +Note that ``ctypes`` does not have OOR (original object return), it +constructs a new, equivalent object each time you retrieve an +attribute:: + + >>> pi.contents is i + False + >>> pi.contents is pi.contents + False + >>> Assigning another ``c_int`` instance to the pointer's contents attribute would cause the pointer to point to the memory location where this is stored:: - >>> pi.contents = c_int(99) - >>> pi.contents - c_long(99) - >>> + >>> pi.contents = c_int(99) + >>> pi.contents + c_long(99) + >>> Pointer instances can also be indexed with integers:: - >>> pi[0] - 99 - >>> + >>> pi[0] + 99 + >>> -XXX What is this??? Assigning to an integer index changes the pointed to value:: - >>> i2 = pi[0] - >>> i2 - 99 - >>> pi[0] = 22 - >>> i2 - 99 - >>> + >>> print i + c_long(99) + >>> pi[0] = 22 + >>> print i + c_long(22) + >>> It is also possible to use indexes different from 0, but you must know -what you're doing when you use this: You access or change arbitrary -memory locations when you do this. Generally you only use this feature -if you receive a pointer from a C function, and you *know* that the -pointer actually points to an array instead of a single item. +what you're doing, just as in C: You can access or change arbitrary +memory locations. Generally you only use this feature if you receive a +pointer from a C function, and you *know* that the pointer actually +points to an array instead of a single item. Pointer classes/types --------------------- @@ -692,7 +708,7 @@ >>> PI = POINTER(c_int) >>> PI - >>> PI(42) # doctest: +IGNORE_EXCEPTION_DETAIL + >>> PI(42) Traceback (most recent call last): File "", line 1, in ? TypeError: expected c_long instead of int @@ -700,6 +716,78 @@ >>> +Type conversions +---------------- + +Usually, ctypes does strict type checking. This means, if you have +``POINTER(c_int)`` in the ``argtypes`` list of a function or in the +``_fields_`` of a structure definition, only instances of exactly the +same type are accepted. There are some exceptions to this rule, where +ctypes accepts other objects. For example, you can pass compatible +array instances instead of pointer types. So, for ``POINTER(c_int)``, +ctypes accepts an array of c_int values:: + + >>> class Bar(Structure): + ... _fields_ = [("count", c_int), ("values", POINTER(c_int))] + ... + >>> bar = Bar() + >>> print bar._objects + None + >>> bar.values = (c_int * 3)(1, 2, 3) + >>> print bar._objects + {'1': ({}, )} + >>> bar.count = 3 + >>> for i in range(bar.count): + ... print bar.values[i] + ... + 1 + 2 + 3 + >>> + +To set a POINTER type field to ``NULL``, you can assign ``None``:: + + >>> bar.values = None + >>> + +XXX list other conversions... + +Sometimes you have instances of incompatible types. In ``C``, you can +cast one type into another type. ``ctypes`` provides a ``cast`` +function which can be used in the same way. The Bar structure defined +above accepts ``POINTER(c_int)`` pointers or ``c_int`` arrays for its +``values`` field, but not instances of other types:: + + >>> bar.values = (c_byte * 4)() + Traceback (most recent call last): + File "", line 1, in ? + TypeError: incompatible types, c_byte_Array_4 instance instead of LP_c_long instance + >>> + +For these cases, the ``cast`` function is handy. + +The ``cast`` function can be used to cast a ctypes instance into a +pointer to a different ctypes data type. ``cast`` takes two +parameters, a ctypes object that is or can be converted to a pointer +of some kind, and a ctypes pointer type. It returns an instance of +the second argument, which references the same memory block as the +first argument:: + + >>> a = (c_byte * 4)() + >>> cast(a, POINTER(c_int)) + + >>> + +So, ``cast`` can be used to assign to the ``values`` field of ``Bar`` +the structure:: + + >>> bar = Bar() + >>> bar.values = cast((c_byte * 4)(), POINTER(c_int)) + >>> print bar.values[0] + 0 + >>> + + Incomplete Types ---------------- @@ -1010,6 +1098,7 @@ >>> rc.a, rc.b = rc.b, rc.a >>> print rc.a.x, rc.a.y, rc.b.x, rc.b.y 3 4 3 4 + >>> Hm. We certainly expected the last statement to print ``3 4 1 2``. What happended? Here are the steps of the ``rc.a, rc.b = rc.b, rc.a`` @@ -1018,6 +1107,7 @@ >>> temp0, temp1 = rc.b, rc.a >>> rc.a = temp0 >>> rc.b = temp1 + >>> Note that ``temp0`` and ``temp1`` are objects still using the internal buffer of the ``rc`` object above. So executing ``rc.a = temp0`` @@ -1046,6 +1136,76 @@ the object itself, instead the ``contents`` of the object is stored. Accessing the contents again constructs a new Python each time! +Variable-sized data types +------------------------- + +``ctypes`` provides some support for variable-sized arrays and +structures (this was added in version 0.9.9.7). + +The ``resize`` function can be used to resize the memory buffer of an +existing ctypes object. The function takes the object as first +argument, and the requested size in bytes as the second argument. The +memory block cannot be made smaller than the natural memory block +specified by the objects type, a ``ValueError`` is raised if this is +tried:: + + >>> short_array = (c_short * 4)() + >>> print sizeof(short_array) + 8 + >>> resize(short_array, 4) + Traceback (most recent call last): + ... + ValueError: minimum size is 8 + >>> resize(short_array, 32) + >>> sizeof(short_array) + 32 + >>> sizeof(type(short_array)) + 8 + >>> + +This is nice and fine, but how would one access the additional +elements contained in this array? Since the type still only knows +about 4 elements, we get errors accessing other elements:: + + >>> short_array[:] + [0, 0, 0, 0] + >>> short_array[7] + Traceback (most recent call last): + ... + IndexError: invalid index + >>> + +The solution is to use 1-element arrays; as a special case ctypes does +no bounds checking on them:: + + >>> short_array = (c_short * 1)() + >>> print sizeof(short_array) + 2 + >>> resize(short_array, 32) + >>> sizeof(short_array) + 32 + >>> sizeof(type(short_array)) + 2 + >>> short_array[0:8] + [0, 0, 0, 0, 0, 0, 0, 0] + >>> short_array[7] = 42 + >>> short_array[0:8] + [0, 0, 0, 0, 0, 0, 0, 42] + >>> + +Using 1-element arrays as variable sized fields in structures works as +well, but they should be used as the last field in the structure +definition. This example shows a definition from the Windows header +files:: + + class SP_DEVICE_INTERFACE_DETAIL_DATA(Structure): + _fields_ = [("cbSize", c_int), + ("DevicePath", c_char * 1)] + +Another way to use variable-sized data types with ``ctypes`` is to use +the dynamic nature of Python, and (re-)define the data type after the +required size is already known, on a case by case basis. + Bugs, ToDo and non-implemented things ------------------------------------- Deleted: /external/ctypes/docs/manual/utilities.txt ============================================================================== --- /external/ctypes/docs/manual/utilities.txt Sat Jun 10 22:08:12 2006 +++ (empty file) @@ -1,164 +0,0 @@ -Builtin functions ------------------ - -``addressof(object)`` - - Returns the address of a ctypes instance as an integer. - -``alignment(type_or_object)`` - - Returns the alignment requirements in bytes of a ctypes type or - instance. - -``byref(object)`` - - Returns a light-weight pointer to a ctypes instance. The returned - object can only be used as function call parameter. Behaves the - same as calling ``pointer(object)``, but is a lot faster. Same as - ``&object`` in C. - -``cast(object, typ)`` - - This function is similar to the cast operator in C. Returns a new - instance of ``type`` which shares the memory block of ``object``. - ``typ`` must be a pointer type. - -``CFUNCTYPE(restype, *argtypes)`` - - Create a function prototype using the C calling convention. - - -``create_string_buffer(init, size=None)`` - - Convenience function to create a mutable character buffer. - - ``init`` must be a string. If ``size`` is supplied it must be a - positive integer that specifies the size of the buffer, otherwise - the length of the ``init`` string is used. - This function returns a ctypes array of characters ``c_char``. - -``create_unicode_buffer(init, size=None)`` - - Convenience function to create a mutable unicode buffer. - - ``init`` must be a unicode string. If ``size`` is supplied it - must be a positive integer that specifies the number of characters - in the buffer, otherwise the length of the ``init`` string is - used. This function returns a ctypes array of characters ``c_wchar``. - - -``DllCanUnloadNow()``, ``DllGetClassObject(rclsid, riid, ppv)`` (Windows only) - - Functions used to implement COM servers. - - -``FormatError([code])`` (Windows only) - - Returns a textual description of the error code, or the last error - code set by Windows. - -``GetLastError()`` (Windows only) - - Returns the last error code set by Windows. - -``memmove(dst, src, count)`` - - Same as the standard C ``memmove`` library function: copies - ``count`` bytes from ``src`` to ``dst``. ``dst`` and ``src`` must - be integers or anything else that can be converted into a pointer. - -``memset(dst, c, count)`` - - Same as the standard C ``memset`` function. Fills the memory block - at address ``dst`` with ``count`` bytes of value ``c``. ``dst`` must be - an integer specifying an address, or a ctypes instance. - - -``pointer(object)`` - - This function creates a new pointer instance, pointing to the - supplied argument which must be an instance of a ctypes type. The - return pointer is of type ``POINTER(type(object))``. If you have - a ctypes instance, and you want to pass the address of it to a - function call, you should use ``byref(object)`` instead which is - much faster. - - NULL pointer instances are boolean``False``, so to check for a - NULL pointer do this:: - - # assuming ptr is in ctypes pointer instance - if ptr: - print "Non-NULL pointer instance" - else: - print "NULL pointer instance" - - -``POINTER(cls)`` - - This factory function creates and returns a new ctypes type. - Pointer types are cached, so calling this function is cheap. - - To create a ``NULL`` pointer instance, call the created type - without an argument:: - - null_ptr = POINTER(c_int)() - - -``set_conversion_mode(encoding, errors)`` - - This function sets the encoding/decoding rules which are used when - ctypes has to convert between unicode and byte strings. It - returns the previous encoding, as well as a tuple of any errors. - If not set, default conversions are used: - On Windows, ``msbc, ignore`` , on other systems, ``ascii, strict``. - - -``sizeof(type_or_object)`` - - Returns the size in bytes of a ctypes type or instance memory - buffer. Does the same as the C sizeof() function. - - -``string_at(addr[, size])`` - - This function does the same as the Python ``PyString_FromString`` / - ``PyString_FromStringAndSize`` C api functions. - -``WinError(code=None, descr=None)`` - - XXX This is probably the worst named thing in ctypes! - - This function creates a ``WindowsError`` instance. If ``code`` is - not specified, GetLastError() is called to determine the error - code. If ``descr`` is not specified, ``FormatError`` is called to - get a textual description of the error. - -``WINFUNCTYPE(restype, *argtypes)`` (Windows only) - - Create a function prototype using the __stdcall calling convention - (on Windows), or using the C calling convention (on Windows CE). - -``wstring_at(addr[, size])`` - - This function does the same as the Python ``PyUnicode_FromWideString`` - C api function. If ``size`` is not specified, ``wcslen`` is used - to determine the string length. - - -Deprecated functions --------------------- - -These deprecated functions are still supported for backwards -comatibility, they should not be used for new code: - -``c_buffer(init, size=None)`` - - Deprecated. Use ``create_string_buffer()`` instead. - -``ARRAY(cls, len)`` - - Deprecated. Use ``cls * len`` instead. - -``SetPointerType(pointer_class, cls)`` - - Deprecated. Added: external/ctypes/docs/test-anatomy.py ============================================================================== --- (empty file) +++ external/ctypes/docs/test-anatomy.py Sat Jun 10 22:08:12 2006 @@ -0,0 +1,45 @@ +#!/usr/bin/env python +import sys +import doctest + +# handle platform specific issues +WINDOWS = doctest.register_optionflag("WINDOWS") +LINUX = doctest.register_optionflag("LINUX") +SKIP = doctest.register_optionflag("SKIP") + +# handle size specific issues +import ctypes +c_int_name = ctypes.c_int.__name__ + +base = doctest.DocTestRunner +class MyDocTestRunner(base): + def run(self, test, compileflags=None, out=None, clear_globs=True): + examples = test.examples[:] + for ex in test.examples: + if WINDOWS in ex.options and sys.platform != "win32": + examples.remove(ex) + elif LINUX in ex.options and not sys.platform.startswith("linux"): + examples.remove(ex) + elif SKIP in ex.options: + examples.remove(ex) +## elif "printf(" in ex.source: +## # handle that doctest doesn't catch printf's output +## lines = ex.want.splitlines() +## try: +## int(lines[-1]) +## except ValueError: +## pass +## else: +## ex.want = "\n".join(lines[1:]) + "\n" +## else: + ex.want = ex.want.replace("c_long", c_int_name) + test.examples = examples + return base.run(self, test, compileflags, out, clear_globs) +doctest.DocTestRunner = MyDocTestRunner + +if __name__ == "__main__": + # Python 2.5a2 formats exceptions differently than before, so we + # add IGNORE_EXCEPTION_DETAIL. I do not know if this will be + # fixed or not. + doctest.testfile("anatomy.txt", + optionflags=doctest.ELLIPSIS | doctest.IGNORE_EXCEPTION_DETAIL) Modified: external/ctypes/env.bat ============================================================================== --- external/ctypes/env.bat (original) +++ external/ctypes/env.bat Sat Jun 10 22:08:12 2006 @@ -1,2 +1,2 @@ @echo off -for %%i in (.) do set PYTHONPATH=%%~fi \ No newline at end of file +for %%i in (.) do set PYTHONPATH=%%~fi;%%~fi\codegen;%%~fi\codegen\scripts Modified: external/ctypes/setup.py ============================================================================== --- external/ctypes/setup.py (original) +++ external/ctypes/setup.py Sat Jun 10 22:08:12 2006 @@ -1,6 +1,6 @@ #!/usr/bin/env python # -# $Id: setup.py,v 1.140 2006/04/14 15:49:33 theller Exp $ +# $Id: setup.py,v 1.141 2006/05/19 20:06:31 theller Exp $ # # @@ -11,7 +11,7 @@ LIBFFI_SOURCES='source/libffi' -__version__ = "0.9.9.6" +__version__ = "0.9.9.7" ################################################################ Modified: external/ctypes/source/_ctypes.c ============================================================================== --- external/ctypes/source/_ctypes.c (original) +++ external/ctypes/source/_ctypes.c Sat Jun 10 22:08:12 2006 @@ -342,6 +342,14 @@ static PyObject * CDataType_repeat(PyObject *self, Py_ssize_t length) { + if (length < 0) + return PyErr_Format(PyExc_ValueError, +#if (PY_VERSION_HEX < 0x02050000) + "Array length must be >= 0, not %d", +#else + "Array length must be >= 0, not %zd", +#endif + length); return CreateArrayType(self, length); } @@ -1809,23 +1817,62 @@ } static PyObject * -unique_key(CDataObject *target, int index) +unique_key(CDataObject *target, Py_ssize_t index) { - char string[256]; /* XXX is that enough? */ + char string[256]; char *cp = string; - *cp++ = index + '0'; + size_t bytes_left; + + assert(sizeof(string) - 1 > sizeof(Py_ssize_t) * 2); +#if (PY_VERSION_HEX < 0x02050000) + cp += sprintf(cp, "%x", index); +#else +#ifdef MS_WIN32 +/* MSVC does not understand the 'z' size specifier */ + cp += sprintf(cp, "%Ix", index); +#else + cp += sprintf(cp, "%zx", index); +#endif +#endif while (target->b_base) { - *cp++ = target->b_index + '0'; + bytes_left = sizeof(string) - (cp - string) - 1; + /* Hex format needs 2 characters per byte */ + if (bytes_left < sizeof(Py_ssize_t) * 2) { + PyErr_SetString(PyExc_ValueError, + "ctypes object structure too deep"); + return NULL; + } +#if (PY_VERSION_HEX < 0x02050000) + cp += sprintf(cp, ":%x", (int)target->b_index); +#else +#ifdef MS_WIN32 + cp += sprintf(cp, ":%Ix", (size_t)target->b_index); +#else + cp += sprintf(cp, ":%zx", (size_t)target->b_index); +#endif +#endif target = target->b_base; } return PyString_FromStringAndSize(string, cp-string); } -/* Keep a reference to 'keep' in the 'target', at index 'index' */ + /* - * KeepRef travels the target's b_base pointer down to the root, - * building a sequence of indexes during the path. The indexes, which are a - * couple of small integers, are used to build a byte string usable as - * key int the root object's _objects dict. + * Keep a reference to 'keep' in the 'target', at index 'index'. + * + * If 'keep' is None, do nothing. + * + * Otherwise create a dictionary (if it does not yet exist) id the root + * objects 'b_objects' item, which will store the 'keep' object under a unique + * key. + * + * The unique_key helper travels the target's b_base pointer down to the root, + * building a string containing hex-formatted indexes found during traversal, + * separated by colons. + * + * The index tuple is used as a key into the root object's b_objects dict. + * + * Note: This function steals a refcount of the third argument, even if it + * fails! */ static int KeepRef(CDataObject *target, Py_ssize_t index, PyObject *keep) @@ -1846,6 +1893,10 @@ return 0; } key = unique_key(target, index); + if (key == NULL) { + Py_DECREF(keep); + return -1; + } result = PyDict_SetItem(ob->b_objects, key, keep); Py_DECREF(key); Py_DECREF(keep); @@ -2611,11 +2662,11 @@ *(void **)self->b_ptr = address; + Py_INCREF((PyObject *)dll); /* for KeepRef */ if (-1 == KeepRef((CDataObject *)self, 0, dll)) { Py_DECREF((PyObject *)self); return NULL; } - Py_INCREF((PyObject *)dll); /* for KeepRef above */ Py_INCREF(self); self->callable = (PyObject *)self; @@ -2751,11 +2802,11 @@ correctly... */ + Py_INCREF((PyObject *)self); /* for KeepRef */ if (-1 == KeepRef((CDataObject *)self, 0, (PyObject *)self)) { Py_DECREF((PyObject *)self); return NULL; } - Py_INCREF((PyObject *)self); /* for KeepRef above */ return (PyObject *)self; } @@ -3520,7 +3571,7 @@ int offset, size; StgDictObject *stgdict; - if (index < 0 || index >= self->b_length) { + if (self->b_length == 0 || index < 0 || (self->b_length > 1 && index >= self->b_length)) { PyErr_SetString(PyExc_IndexError, "invalid index"); return NULL; @@ -3549,11 +3600,11 @@ if (ilow < 0) ilow = 0; - else if (ilow > self->b_length) + else if (ilow > self->b_length && self->b_length != 1) ilow = self->b_length; if (ihigh < ilow) ihigh = ilow; - else if (ihigh > self->b_length) + else if (ihigh > self->b_length && self->b_length != 1) ihigh = self->b_length; len = ihigh - ilow; @@ -3596,7 +3647,8 @@ } stgdict = PyObject_stgdict((PyObject *)self); - if (index < 0 || index >= stgdict->length) { + if (self->b_length == 0 || index < 0 + || (self->b_length > 1 && index >= self->b_length)) { PyErr_SetString(PyExc_IndexError, "invalid index"); return -1; @@ -3623,17 +3675,19 @@ if (ilow < 0) ilow = 0; - else if (ilow > self->b_length) + else if (ilow > self->b_length && self->b_length != 1) ilow = self->b_length; + if (ihigh < 0) ihigh = 0; + if (ihigh < ilow) ihigh = ilow; - else if (ihigh > self->b_length) + else if (ihigh > self->b_length && self->b_length != 1) ihigh = self->b_length; len = PySequence_Length(value); - if (len != ihigh - ilow) { + if (self->b_length != 1 && len != ihigh - ilow) { PyErr_SetString(PyExc_ValueError, "Can only assign sequence of same size"); return -1; @@ -4020,7 +4074,8 @@ Pointer_item(PyObject *_self, Py_ssize_t index) { CDataObject *self = (CDataObject *)_self; - int size, offset; + int size; + Py_ssize_t offset; StgDictObject *stgdict, *itemdict; PyObject *proto; @@ -4030,9 +4085,9 @@ return NULL; } - stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); + assert(stgdict->proto); proto = stgdict->proto; /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ @@ -4040,7 +4095,7 @@ size = itemdict->size; offset = index * itemdict->size; - return CData_get(stgdict->proto, stgdict->getfunc, (PyObject *)self, + return CData_get(proto, stgdict->getfunc, (PyObject *)self, index, size, (*(char **)self->b_ptr) + offset); } @@ -4049,7 +4104,9 @@ { CDataObject *self = (CDataObject *)_self; int size; - StgDictObject *stgdict; + Py_ssize_t offset; + StgDictObject *stgdict, *itemdict; + PyObject *proto; if (value == NULL) { PyErr_SetString(PyExc_TypeError, @@ -4064,16 +4121,17 @@ } stgdict = PyObject_stgdict((PyObject *)self); - if (index != 0) { - PyErr_SetString(PyExc_IndexError, - "invalid index"); - return -1; - } - size = stgdict->size / stgdict->length; + assert(stgdict); + assert(stgdict->proto); - /* XXXXX Make sure proto is NOT NULL! */ - return CData_set((PyObject *)self, stgdict->proto, stgdict->setfunc, value, - index, size, *(void **)self->b_ptr); + proto = stgdict->proto; + /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ + itemdict = PyType_stgdict(proto); + size = itemdict->size; + offset = index * itemdict->size; + + return CData_set((PyObject *)self, proto, stgdict->setfunc, value, + index, size, (*(char **)self->b_ptr) + offset); } static PyObject * @@ -4090,8 +4148,8 @@ stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); return CData_FromBaseObj(stgdict->proto, - (PyObject *)self, 0, - *(void **)self->b_ptr); + (PyObject *)self, 0, + *(void **)self->b_ptr); } static int @@ -4439,7 +4497,7 @@ } static PyObject * -cast(void *ptr, PyObject *ctype) +cast(void *ptr, PyObject *src, PyObject *ctype) { CDataObject *result; if (0 == cast_check_pointertype(ctype)) @@ -4447,6 +4505,36 @@ result = (CDataObject *)PyObject_CallFunctionObjArgs(ctype, NULL); if (result == NULL) return NULL; + + /* + The casted objects '_objects' member: + + It must certainly contain the source objects one. + It must contain the source object itself. + */ + if (CDataObject_Check(src)) { + CDataObject *obj = (CDataObject *)src; + /* CData_GetContainer will initialize src.b_objects, we need + this so it can be shared */ + CData_GetContainer(obj); + /* But we need a dictionary! */ + if (obj->b_objects == Py_None) { + Py_DECREF(Py_None); + obj->b_objects = PyDict_New(); + } + Py_INCREF(obj->b_objects); + result->b_objects = obj->b_objects; + if (result->b_objects) { + PyObject *index = PyLong_FromVoidPtr((void *)src); + int rc; + if (index == NULL) + return NULL; + rc = PyDict_SetItem(result->b_objects, index, src); + Py_DECREF(index); + if (rc == -1) + return NULL; + } + } /* Should we assert that result is a pointer type? */ memcpy(result->b_ptr, &ptr, sizeof(void *)); return (PyObject *)result; @@ -4581,7 +4669,7 @@ #endif PyModule_AddObject(m, "FUNCFLAG_CDECL", PyInt_FromLong(FUNCFLAG_CDECL)); PyModule_AddObject(m, "FUNCFLAG_PYTHONAPI", PyInt_FromLong(FUNCFLAG_PYTHONAPI)); - PyModule_AddStringConstant(m, "__version__", "0.9.9.6"); + PyModule_AddStringConstant(m, "__version__", "0.9.9.7"); PyModule_AddObject(m, "_memmove_addr", PyLong_FromVoidPtr(memmove)); PyModule_AddObject(m, "_memset_addr", PyLong_FromVoidPtr(memset)); Modified: external/ctypes/source/_ctypes_test.c ============================================================================== --- external/ctypes/source/_ctypes_test.c (original) +++ external/ctypes/source/_ctypes_test.c Sat Jun 10 22:08:12 2006 @@ -96,6 +96,11 @@ return dst; } +EXPORT(void)my_free(void *ptr) +{ + free(ptr); +} + #ifdef HAVE_WCHAR_H EXPORT(wchar_t *) my_wcsdup(wchar_t *src) { @@ -199,11 +204,6 @@ return 0; } -EXPORT(void) my_free(void *p) -{ - printf("my_free got %p\n", p); -} - typedef struct { char *name; char *value; Modified: external/ctypes/source/callbacks.c ============================================================================== --- external/ctypes/source/callbacks.c (original) +++ external/ctypes/source/callbacks.c Sat Jun 10 22:08:12 2006 @@ -199,45 +199,16 @@ result = PyObject_CallObject(callable, arglist); CHECK("'calling callback function'", result); - if ((restype != &ffi_type_void) - && result && result != Py_None) { /* XXX What is returned for Py_None ? */ - /* another big endian hack */ - union { - char c; - short s; - int i; - long l; - } r; + if ((restype != &ffi_type_void) && result && result != Py_None) { PyObject *keep; assert(setfunc); - switch (restype->size) { - case 1: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.c; - break; - case SIZEOF_SHORT: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.s; - break; - case SIZEOF_INT: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.i; - break; -#if (SIZEOF_LONG != SIZEOF_INT) - case SIZEOF_LONG: - keep = setfunc(&r, result, 0); - CHECK("'converting callback result'", keep); - *(ffi_arg *)mem = r.l; - break; +#ifdef WORDS_BIGENDIAN + /* See the corresponding code in callproc.c, around line 961 */ + if (restype->type != FFI_TYPE_FLOAT && restype->size < sizeof(ffi_arg)) + mem = (char *)mem + sizeof(ffi_arg) - restype->size; #endif - default: - keep = setfunc(mem, result, 0); - CHECK("'converting callback result'", keep); - break; - } + keep = setfunc(mem, result, 0); + CHECK("'converting callback result'", keep); /* keep is an object we have to keep alive so that the result stays valid. If there is no such object, the setfunc will have returned Py_None. Modified: external/ctypes/source/callproc.c ============================================================================== --- external/ctypes/source/callproc.c (original) +++ external/ctypes/source/callproc.c Sat Jun 10 22:08:12 2006 @@ -964,7 +964,14 @@ address cannot simply be used as result pointer, instead we must adjust the pointer value: */ - if (rtype->size < sizeof(ffi_arg)) + /* + XXX I should find out and clarify why this is needed at all, + especially why adjusting for ffi_type_float must be avoided on + 64-bit platforms. + */ + if (rtype->type != FFI_TYPE_FLOAT + && rtype->type != FFI_TYPE_STRUCT + && rtype->size < sizeof(ffi_arg)) resbuf = (char *)resbuf + sizeof(ffi_arg) - rtype->size; #endif @@ -1437,7 +1444,64 @@ } #endif +static PyObject * +resize(PyObject *self, PyObject *args) +{ + CDataObject *obj; + StgDictObject *dict; + Py_ssize_t size; + + if (!PyArg_ParseTuple(args, +#if (PY_VERSION_HEX < 0x02050000) + "Oi:resize", +#else + "On:resize", +#endif + (PyObject *)&obj, &size)) + return NULL; + + dict = PyObject_stgdict((PyObject *)obj); + if (dict == NULL) { + PyErr_SetString(PyExc_TypeError, + "excepted ctypes instance"); + return NULL; + } + if (size < dict->size) { + PyErr_Format(PyExc_ValueError, + "minimum size is %d", dict->size); + return NULL; + } + if (obj->b_needsfree == 0) { + PyErr_Format(PyExc_ValueError, + "Memory cannot be resized because this object doesn't own it"); + return NULL; + } + if (size <= sizeof(obj->b_value)) { + /* internal default buffer is large enough */ + obj->b_size = size; + goto done; + } + if (obj->b_size <= sizeof(obj->b_value)) { + /* We are currently using the objects default buffer, but it + isn't large enough any more. */ + void *ptr = PyMem_Malloc(size); + if (ptr == NULL) + return PyErr_NoMemory(); + memset(ptr, 0, size); + memmove(ptr, obj->b_ptr, obj->b_size); + obj->b_ptr = ptr; + obj->b_size = size; + } else { + obj->b_ptr = PyMem_Realloc(obj->b_ptr, size); + obj->b_size = size; + } + done: + Py_INCREF(Py_None); + return Py_None; +} + PyMethodDef module_methods[] = { + {"resize", resize, METH_VARARGS, "Resize the memory buffer of a ctypes instance"}, #ifdef CTYPES_UNICODE {"set_conversion_mode", set_conversion_mode, METH_VARARGS, set_conversion_mode_doc}, #endif Modified: external/ctypes/source/cfield.c ============================================================================== --- external/ctypes/source/cfield.c (original) +++ external/ctypes/source/cfield.c Sat Jun 10 22:08:12 2006 @@ -1,5 +1,4 @@ #include "Python.h" -#include "structmember.h" #include #ifdef MS_WIN32 @@ -208,14 +207,30 @@ self->index, self->size, src->b_ptr + self->offset); } -static PyMemberDef CField_members[] = { - { "offset", T_UINT, - offsetof(CFieldObject, offset), READONLY, - "offset in bytes of this field"}, - { "size", T_UINT, - offsetof(CFieldObject, size), READONLY, - "size in bytes of this field"}, - { NULL }, +static PyObject * +CField_get_offset(PyObject *self, void *data) +{ +#if (PY_VERSION_HEX < 0x02050000) + return PyInt_FromLong(((CFieldObject *)self)->offset); +#else + return PyInt_FromSsize_t(((CFieldObject *)self)->offset); +#endif +} + +static PyObject * +CField_get_size(PyObject *self, void *data) +{ +#if (PY_VERSION_HEX < 0x02050000) + return PyInt_FromLong(((CFieldObject *)self)->size); +#else + return PyInt_FromSsize_t(((CFieldObject *)self)->size); +#endif +} + +static PyGetSetDef CField_getset[] = { + { "offset", CField_get_offset, NULL, "offset in bytes of this field" }, + { "size", CField_get_size, NULL, "size in bytes of this field" }, + { NULL, NULL, NULL, NULL }, }; static int @@ -298,8 +313,8 @@ 0, /* tp_iter */ 0, /* tp_iternext */ 0, /* tp_methods */ - CField_members, /* tp_members */ - 0, /* tp_getset */ + 0, /* tp_members */ + CField_getset, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ (descrgetfunc)CField_get, /* tp_descr_get */ Modified: external/ctypes/source/ctypes.h ============================================================================== --- external/ctypes/source/ctypes.h (original) +++ external/ctypes/source/ctypes.h Sat Jun 10 22:08:12 2006 @@ -59,7 +59,7 @@ Py_ssize_t b_length; /* number of references we need */ Py_ssize_t b_index; /* index of this object into base's b_object list */ - PyObject *b_objects; /* list of references we need to keep */ + PyObject *b_objects; /* dictionary of references we need to keep, or Py_None */ union value b_value; }; @@ -181,6 +181,7 @@ PyObject *proto; /* a type or NULL */ GETFUNC getfunc; /* getter function if proto is NULL */ SETFUNC setfunc; /* setter function if proto is NULL */ + int anonymous; } CFieldObject; /* A subclass of PyDictObject, used as the instance dictionary of ctypes Modified: external/ctypes/source/libffi/configure ============================================================================== --- external/ctypes/source/libffi/configure (original) +++ external/ctypes/source/libffi/configure Sat Jun 10 22:08:12 2006 @@ -310,7 +310,7 @@ # include #endif" -ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS build build_cpu build_vendor build_os host host_cpu host_vendor host_os target target_cpu target_vendor target_os CC ac_ct_CC EXEEXT OBJEXT CFLAGS CPP CPPFLAGS EGREP ALLOCA HAVE_LONG_DOUBLE TARGET TARGETDIR LIBOBJS LTLIBOBJS' +ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS build build_cpu build_vendor build_os host host_cpu host_vendor host_os target target_cpu target_vendor target_os CC ac_ct_CC EXEEXT OBJEXT CFLAGS CPP CPPFLAGS EGREP ALLOCA HAVE_LONG_DOUBLE TARGET TARGETDIR MKTARGET LIBOBJS LTLIBOBJS' ac_subst_files='' # Initialize some variables set by options. @@ -3483,6 +3483,7 @@ TARGETDIR="unknown" case "$host" in +i*86-*-darwin*) TARGET=X86_DARWIN; TARGETDIR=x86;; i*86-*-linux*) TARGET=X86; TARGETDIR=x86;; i*86-*-gnu*) TARGET=X86; TARGETDIR=x86;; i*86-*-solaris2.1[0-9]*) TARGET=X86_64; TARGETDIR=x86;; @@ -3534,6 +3535,8 @@ { (exit 1); exit 1; }; } fi +MKTARGET=$TARGET + case x$TARGET in xMIPS*) TARGET=MIPS ;; *) ;; @@ -5241,6 +5244,9 @@ esac + + + if test x$TARGET = xSPARC; then echo "$as_me:$LINENO: checking assembler and linker support unaligned pc related relocs" >&5 echo $ECHO_N "checking assembler and linker support unaligned pc related relocs... $ECHO_C" >&6 @@ -5457,6 +5463,7 @@ + cat >>confdefs.h <<\_ACEOF #define FFI_NO_RAW_API 1 _ACEOF @@ -5467,7 +5474,15 @@ ac_config_commands="$ac_config_commands src" - ac_config_links="$ac_config_links include/ffitarget.h:src/$TARGETDIR/ffitarget.h" +TARGETINCDIR=$TARGETDIR +case $host in +*-*-darwin*) + TARGETINCDIR="darwin" + ;; +esac + + + ac_config_links="$ac_config_links include/ffitarget.h:src/$TARGETINCDIR/ffitarget.h" ac_config_links="$ac_config_links include/ffi_common.h:include/ffi_common.h" @@ -6014,7 +6029,7 @@ # Handling of arguments. "include/ffi.h" ) CONFIG_FILES="$CONFIG_FILES include/ffi.h" ;; "fficonfig.py" ) CONFIG_FILES="$CONFIG_FILES fficonfig.py" ;; - "include/ffitarget.h" ) CONFIG_LINKS="$CONFIG_LINKS include/ffitarget.h:src/$TARGETDIR/ffitarget.h" ;; + "include/ffitarget.h" ) CONFIG_LINKS="$CONFIG_LINKS include/ffitarget.h:src/$TARGETINCDIR/ffitarget.h" ;; "include/ffi_common.h" ) CONFIG_LINKS="$CONFIG_LINKS include/ffi_common.h:include/ffi_common.h" ;; "include" ) CONFIG_COMMANDS="$CONFIG_COMMANDS include" ;; "src" ) CONFIG_COMMANDS="$CONFIG_COMMANDS src" ;; @@ -6129,6 +6144,7 @@ s, at HAVE_LONG_DOUBLE@,$HAVE_LONG_DOUBLE,;t t s, at TARGET@,$TARGET,;t t s, at TARGETDIR@,$TARGETDIR,;t t +s, at MKTARGET@,$MKTARGET,;t t s, at LIBOBJS@,$LIBOBJS,;t t s, at LTLIBOBJS@,$LTLIBOBJS,;t t CEOF Modified: external/ctypes/source/libffi/configure.ac ============================================================================== --- external/ctypes/source/libffi/configure.ac (original) +++ external/ctypes/source/libffi/configure.ac Sat Jun 10 22:08:12 2006 @@ -21,6 +21,7 @@ TARGETDIR="unknown" case "$host" in +i*86-*-darwin*) TARGET=X86_DARWIN; TARGETDIR=x86;; i*86-*-linux*) TARGET=X86; TARGETDIR=x86;; i*86-*-gnu*) TARGET=X86; TARGETDIR=x86;; i*86-*-solaris2.1[[0-9]]*) TARGET=X86_64; TARGETDIR=x86;; @@ -70,6 +71,12 @@ AC_MSG_ERROR(["libffi has not been ported to $host."]) fi +dnl libffi changes TARGET for MIPS to define a such macro in the header +dnl while MIPS_IRIX or MIPS_LINUX is separatedly used to decide which +dnl files will be compiled. So, we need to keep the original decision +dnl of TARGET to use in fficonfig.py.in. +MKTARGET=$TARGET + case x$TARGET in xMIPS*) TARGET=MIPS ;; *) ;; @@ -93,6 +100,24 @@ AC_SUBST(HAVE_LONG_DOUBLE) AC_C_BIGENDIAN +AH_VERBATIM([WORDS_BIGENDIAN], +[ +/* Define to 1 if your processor stores words with the most significant byte + first (like Motorola and SPARC, unlike Intel and VAX). + + The block below does compile-time checking for endianness on platforms + that use GCC and therefore allows compiling fat binaries on OSX by using + '-arch ppc -arch i386' as the compile flags. The phrasing was choosen + such that the configure-result is used on systems that don't use GCC. +*/ +#ifdef __BIG_ENDIAN__ +#define WORDS_BIGENDIAN 1 +#else +#ifndef __LITTLE_ENDIAN__ +#undef WORDS_BIGENDIAN +#endif +#endif]) + if test x$TARGET = xSPARC; then AC_CACHE_CHECK([assembler and linker support unaligned pc related relocs], @@ -183,6 +208,7 @@ AC_SUBST(TARGET) AC_SUBST(TARGETDIR) +AC_SUBST(MKTARGET) AC_SUBST(SHELL) @@ -194,7 +220,15 @@ test -d src/$TARGETDIR || mkdir src/$TARGETDIR ], [TARGETDIR="$TARGETDIR"]) -AC_CONFIG_LINKS(include/ffitarget.h:src/$TARGETDIR/ffitarget.h) +TARGETINCDIR=$TARGETDIR +case $host in +*-*-darwin*) + TARGETINCDIR="darwin" + ;; +esac + + +AC_CONFIG_LINKS(include/ffitarget.h:src/$TARGETINCDIR/ffitarget.h) AC_CONFIG_LINKS(include/ffi_common.h:include/ffi_common.h) AC_CONFIG_FILES(include/ffi.h fficonfig.py) Modified: external/ctypes/source/libffi/fficonfig.h.in ============================================================================== --- external/ctypes/source/libffi/fficonfig.h.in (original) +++ external/ctypes/source/libffi/fficonfig.h.in Sat Jun 10 22:08:12 2006 @@ -114,9 +114,22 @@ /* Define to 1 if you have the ANSI C header files. */ #undef STDC_HEADERS + /* Define to 1 if your processor stores words with the most significant byte - first (like Motorola and SPARC, unlike Intel and VAX). */ + first (like Motorola and SPARC, unlike Intel and VAX). + + The block below does compile-time checking for endianness on platforms + that use GCC and therefore allows compiling fat binaries on OSX by using + '-arch ppc -arch i386' as the compile flags. The phrasing was choosen + such that the configure-result is used on systems that don't use GCC. +*/ +#ifdef __BIG_ENDIAN__ +#define WORDS_BIGENDIAN 1 +#else +#ifndef __LITTLE_ENDIAN__ #undef WORDS_BIGENDIAN +#endif +#endif #ifdef HAVE_HIDDEN_VISIBILITY_ATTRIBUTE Modified: external/ctypes/source/libffi/fficonfig.py.in ============================================================================== --- external/ctypes/source/libffi/fficonfig.py.in (original) +++ external/ctypes/source/libffi/fficonfig.py.in Sat Jun 10 22:08:12 2006 @@ -6,6 +6,7 @@ 'MIPS_IRIX': ['src/mips/ffi.c', 'src/mips/o32.S', 'src/mips/n32.S'], 'MIPS_LINUX': ['src/mips/ffi.c', 'src/mips/o32.S'], 'X86': ['src/x86/ffi.c', 'src/x86/sysv.S'], + 'X86_DARWIN': ['src/x86/ffi_darwin.c', 'src/x86/darwin.S'], 'X86_WIN32': ['src/x86/ffi.c', 'src/x86/win32.S'], 'SPARC': ['src/sparc/ffi.c', 'src/sparc/v8.S', 'src/sparc/v9.S'], 'ALPHA': ['src/alpha/ffi.c', 'src/alpha/osf.S'], @@ -26,8 +27,19 @@ 'PA': ['src/pa/linux.S', 'src/pa/ffi.c'], } +# Build all darwin related files on all supported darwin architectures, this +# makes it easier to build universal binaries. +if 1: + all_darwin = ('X86_DARWIN', 'POWERPC_DARWIN') + all_darwin_files = [] + for pn in all_darwin: + all_darwin_files.extend(ffi_platforms[pn]) + for pn in all_darwin: + ffi_platforms[pn] = all_darwin_files + del all_darwin, all_darwin_files, pn + ffi_srcdir = '@srcdir@' -ffi_sources += ffi_platforms['@TARGET@'] +ffi_sources += ffi_platforms['@MKTARGET@'] ffi_sources = [os.path.join('@srcdir@', f) for f in ffi_sources] ffi_cflags = '@CFLAGS@' Added: external/ctypes/source/libffi/src/darwin/ffitarget.h ============================================================================== --- (empty file) +++ external/ctypes/source/libffi/src/darwin/ffitarget.h Sat Jun 10 22:08:12 2006 @@ -0,0 +1,25 @@ +/* + * This file is for MacOSX only. Dispatch to the right architecture include + * file based on the current archictecture (instead of relying on a symlink + * created by configure). This makes is possible to build a univeral binary + * of ctypes in one go. + */ +#if defined(__i386__) + +#ifndef X86_DARWIN +#define X86_DARWIN +#endif +#undef POWERPC_DARWIN + +#include "../src/x86/ffitarget.h" + +#elif defined(__ppc__) + +#ifndef POWERPC_DARWIN +#define POWERPC_DARWIN +#endif +#undef X86_DARWIN + +#include "../src/powerpc/ffitarget.h" + +#endif Modified: external/ctypes/source/libffi/src/powerpc/darwin.S ============================================================================== --- external/ctypes/source/libffi/src/powerpc/darwin.S (original) +++ external/ctypes/source/libffi/src/powerpc/darwin.S Sat Jun 10 22:08:12 2006 @@ -1,3 +1,4 @@ +#ifdef __ppc__ /* ----------------------------------------------------------------------- darwin.S - Copyright (c) 2000 John Hornkvist Copyright (c) 2004 Free Software Foundation, Inc. @@ -243,3 +244,4 @@ .align LOG2_GPR_BYTES LLFB0$non_lazy_ptr: .g_long LFB0 +#endif Modified: external/ctypes/source/libffi/src/powerpc/darwin_closure.S ============================================================================== --- external/ctypes/source/libffi/src/powerpc/darwin_closure.S (original) +++ external/ctypes/source/libffi/src/powerpc/darwin_closure.S Sat Jun 10 22:08:12 2006 @@ -1,3 +1,4 @@ +#ifdef __ppc__ /* ----------------------------------------------------------------------- darwin_closure.S - Copyright (c) 2002, 2003, 2004, Free Software Foundation, Inc. based on ppc_closure.S @@ -315,3 +316,4 @@ .align LOG2_GPR_BYTES LLFB1$non_lazy_ptr: .g_long LFB1 +#endif Modified: external/ctypes/source/libffi/src/powerpc/ffi_darwin.c ============================================================================== --- external/ctypes/source/libffi/src/powerpc/ffi_darwin.c (original) +++ external/ctypes/source/libffi/src/powerpc/ffi_darwin.c Sat Jun 10 22:08:12 2006 @@ -1,3 +1,4 @@ +#ifdef __ppc__ /* ----------------------------------------------------------------------- ffi.c - Copyright (c) 1998 Geoffrey Keating @@ -380,18 +381,18 @@ extern void ffi_call_AIX(/*@out@*/ extended_cif *, unsigned, unsigned, /*@out@*/ unsigned *, - void (*fn)(), - void (*fn2)()); + void (*fn)(void), + void (*fn2)(extended_cif *, unsigned *const)); extern void ffi_call_DARWIN(/*@out@*/ extended_cif *, unsigned, unsigned, /*@out@*/ unsigned *, - void (*fn)(), - void (*fn2)()); + void (*fn)(void), + void (*fn2)(extended_cif *, unsigned *const)); /*@=declundef@*/ /*@=exportheader@*/ void ffi_call(/*@dependent@*/ ffi_cif *cif, - void (*fn)(), + void (*fn)(void), /*@out@*/ void *rvalue, /*@dependent@*/ void **avalue) { @@ -767,3 +768,4 @@ /* Tell ffi_closure_ASM to perform return type promotions. */ return cif->rtype->type; } +#endif Modified: external/ctypes/source/libffi/src/prep_cif.c ============================================================================== --- external/ctypes/source/libffi/src/prep_cif.c (original) +++ external/ctypes/source/libffi/src/prep_cif.c Sat Jun 10 22:08:12 2006 @@ -55,11 +55,29 @@ /* Perform a sanity check on the argument type */ FFI_ASSERT_VALID_TYPE(*ptr); +#ifdef POWERPC_DARWIN + { + int curalign; + + curalign = (*ptr)->alignment; + if (ptr != &(arg->elements[0])) { + if (curalign > 4 && curalign != 16) { + curalign = 4; + } + } + arg->size = ALIGN(arg->size, curalign); + arg->size += (*ptr)->size; + + arg->alignment = (arg->alignment > curalign) ? + arg->alignment : curalign; + } +#else arg->size = ALIGN(arg->size, (*ptr)->alignment); arg->size += (*ptr)->size; arg->alignment = (arg->alignment > (*ptr)->alignment) ? arg->alignment : (*ptr)->alignment; +#endif ptr++; } @@ -89,6 +107,19 @@ /* Perform machine independent ffi_cif preparation, then call machine dependent routine. */ +#ifdef X86_DARWIN +static inline int struct_on_stack(int size) +{ + if (size > 8) return 1; + /* This is not what the ABI says, but is what is really implemented */ + switch (size) { + case 1: case 2: case 4: case 8: return 0; + } + return 1; +} +#endif + + ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, ffi_abi abi, unsigned int nargs, /*@dependent@*/ /*@out@*/ /*@partial@*/ ffi_type *rtype, @@ -124,6 +155,10 @@ #ifdef SPARC && (cif->abi != FFI_V9 || cif->rtype->size > 32) #endif +#ifdef X86_DARWIN + + && (struct_on_stack(cif->rtype->size)) +#endif ) bytes = STACK_ARG_SIZE(sizeof(void*)); #endif @@ -139,7 +174,16 @@ check after the initialization. */ FFI_ASSERT_VALID_TYPE(*ptr); -#if !defined __x86_64__ && !defined S390 && !defined PA +#if defined(X86_DARWIN) + { + int align = (*ptr)->alignment; + if (align > 4) align = 4; + if ((align - 1) & bytes) + bytes = ALIGN(bytes, align); + bytes += STACK_ARG_SIZE((*ptr)->size); + } + +#elif !defined __x86_64__ && !defined S390 && !defined PA #ifdef SPARC if (((*ptr)->type == FFI_TYPE_STRUCT && ((*ptr)->size > 16 || cif->abi != FFI_V9)) Added: external/ctypes/source/libffi/src/x86/darwin.S ============================================================================== --- (empty file) +++ external/ctypes/source/libffi/src/x86/darwin.S Sat Jun 10 22:08:12 2006 @@ -0,0 +1,390 @@ +#ifdef __i386__ +/* ----------------------------------------------------------------------- + darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. + + X86 Foreign Function Interface + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + ----------------------------------------------------------------------- */ + +/* + * This file is based on sysv.S and then hacked up by Ronald who hasn't done + * assembly programming in 8 years. + */ + +#ifndef __x86_64__ + +#define LIBFFI_ASM +#include +#include + +.text + +.globl _ffi_prep_args + +.align 4 +.globl _ffi_call_SYSV + +_ffi_call_SYSV: +.LFB1: + pushl %ebp +.LCFI0: + movl %esp,%ebp +.LCFI1: + /* Make room for all of the new args. */ + movl 16(%ebp),%ecx + subl %ecx,%esp + + movl %esp,%eax + + /* Place all of the ffi_prep_args in position */ + pushl 12(%ebp) + pushl %eax + call *8(%ebp) + + /* Return stack to previous state and call the function */ + addl $8,%esp + + call *28(%ebp) + + /* Remove the space we pushed for the args */ + movl 16(%ebp),%ecx + addl %ecx,%esp + + /* Load %ecx with the return type code */ + movl 20(%ebp),%ecx + + /* If the return value pointer is NULL, assume no return value. */ + cmpl $0,24(%ebp) + jne retint + + /* Even if there is no space for the return value, we are + obliged to handle floating-point values. */ + cmpl $FFI_TYPE_FLOAT,%ecx + jne noretval + fstp %st(0) + + jmp epilogue + +retint: + cmpl $FFI_TYPE_INT,%ecx + jne retfloat + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + jmp epilogue + +retfloat: + cmpl $FFI_TYPE_FLOAT,%ecx + jne retdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstps (%ecx) + jmp epilogue + +retdouble: + cmpl $FFI_TYPE_DOUBLE,%ecx + jne retlongdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpl (%ecx) + jmp epilogue + +retlongdouble: + cmpl $FFI_TYPE_LONGDOUBLE,%ecx + jne retint64 + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpt (%ecx) + jmp epilogue + +retint64: + cmpl $FFI_TYPE_SINT64,%ecx + jne retstruct + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + movl %edx,4(%ecx) + +retstruct: + /* Nothing to do! */ + +noretval: +epilogue: + movl %ebp,%esp + popl %ebp + ret +.LFE1: +.ffi_call_SYSV_end: +#if 0 + .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV +#endif + +#if 0 + .section .eh_frame,EH_FRAME_FLAGS, at progbits +.Lframe1: + .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ +.LSCIE1: + .long 0x0 /* CIE Identifier Tag */ + .byte 0x1 /* CIE Version */ +#ifdef __PIC__ + .ascii "zR\0" /* CIE Augmentation */ +#else + .ascii "\0" /* CIE Augmentation */ +#endif + .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ + .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ + .byte 0x8 /* CIE RA Column */ +#ifdef __PIC__ + .byte 0x1 /* .uleb128 0x1; Augmentation size */ + .byte 0x1b /* FDE Encoding (pcrel sdata4) */ +#endif + .byte 0xc /* DW_CFA_def_cfa */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x88 /* DW_CFA_offset, column 0x8 */ + .byte 0x1 /* .uleb128 0x1 */ + .align 4 +.LECIE1: +.LSFDE1: + .long .LEFDE1-.LASFDE1 /* FDE Length */ +.LASFDE1: + .long .LASFDE1-.Lframe1 /* FDE CIE offset */ +#ifdef __PIC__ + .long .LFB1-. /* FDE initial location */ +#else + .long .LFB1 /* FDE initial location */ +#endif + .long .LFE1-.LFB1 /* FDE address range */ +#ifdef __PIC__ + .byte 0x0 /* .uleb128 0x0; Augmentation size */ +#endif + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI0-.LFB1 + .byte 0xe /* DW_CFA_def_cfa_offset */ + .byte 0x8 /* .uleb128 0x8 */ + .byte 0x85 /* DW_CFA_offset, column 0x5 */ + .byte 0x2 /* .uleb128 0x2 */ + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI1-.LCFI0 + .byte 0xd /* DW_CFA_def_cfa_register */ + .byte 0x5 /* .uleb128 0x5 */ + .align 4 +.LEFDE1: +#endif + +#endif /* ifndef __x86_64__ */ + +#endif /* defined __i386__ */ +#ifdef __i386__ +/* ----------------------------------------------------------------------- + darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. + + X86 Foreign Function Interface + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + ----------------------------------------------------------------------- */ + +/* + * This file is based on sysv.S and then hacked up by Ronald who hasn't done + * assembly programming in 8 years. + */ + +#ifndef __x86_64__ + +#define LIBFFI_ASM +#include +#include + +.text + +.globl _ffi_prep_args + +.align 4 +.globl _ffi_call_SYSV + +_ffi_call_SYSV: +.LFB1: + pushl %ebp +.LCFI0: + movl %esp,%ebp +.LCFI1: + /* Make room for all of the new args. */ + movl 16(%ebp),%ecx + subl %ecx,%esp + + movl %esp,%eax + + /* Place all of the ffi_prep_args in position */ + pushl 12(%ebp) + pushl %eax + call *8(%ebp) + + /* Return stack to previous state and call the function */ + addl $8,%esp + + call *28(%ebp) + + /* Remove the space we pushed for the args */ + movl 16(%ebp),%ecx + addl %ecx,%esp + + /* Load %ecx with the return type code */ + movl 20(%ebp),%ecx + + /* If the return value pointer is NULL, assume no return value. */ + cmpl $0,24(%ebp) + jne retint + + /* Even if there is no space for the return value, we are + obliged to handle floating-point values. */ + cmpl $FFI_TYPE_FLOAT,%ecx + jne noretval + fstp %st(0) + + jmp epilogue + +retint: + cmpl $FFI_TYPE_INT,%ecx + jne retfloat + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + jmp epilogue + +retfloat: + cmpl $FFI_TYPE_FLOAT,%ecx + jne retdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstps (%ecx) + jmp epilogue + +retdouble: + cmpl $FFI_TYPE_DOUBLE,%ecx + jne retlongdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpl (%ecx) + jmp epilogue + +retlongdouble: + cmpl $FFI_TYPE_LONGDOUBLE,%ecx + jne retint64 + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpt (%ecx) + jmp epilogue + +retint64: + cmpl $FFI_TYPE_SINT64,%ecx + jne retstruct + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + movl %edx,4(%ecx) + +retstruct: + /* Nothing to do! */ + +noretval: +epilogue: + movl %ebp,%esp + popl %ebp + ret +.LFE1: +.ffi_call_SYSV_end: +#if 0 + .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV +#endif + +#if 0 + .section .eh_frame,EH_FRAME_FLAGS, at progbits +.Lframe1: + .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ +.LSCIE1: + .long 0x0 /* CIE Identifier Tag */ + .byte 0x1 /* CIE Version */ +#ifdef __PIC__ + .ascii "zR\0" /* CIE Augmentation */ +#else + .ascii "\0" /* CIE Augmentation */ +#endif + .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ + .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ + .byte 0x8 /* CIE RA Column */ +#ifdef __PIC__ + .byte 0x1 /* .uleb128 0x1; Augmentation size */ + .byte 0x1b /* FDE Encoding (pcrel sdata4) */ +#endif + .byte 0xc /* DW_CFA_def_cfa */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x88 /* DW_CFA_offset, column 0x8 */ + .byte 0x1 /* .uleb128 0x1 */ + .align 4 +.LECIE1: +.LSFDE1: + .long .LEFDE1-.LASFDE1 /* FDE Length */ +.LASFDE1: + .long .LASFDE1-.Lframe1 /* FDE CIE offset */ +#ifdef __PIC__ + .long .LFB1-. /* FDE initial location */ +#else + .long .LFB1 /* FDE initial location */ +#endif + .long .LFE1-.LFB1 /* FDE address range */ +#ifdef __PIC__ + .byte 0x0 /* .uleb128 0x0; Augmentation size */ +#endif + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI0-.LFB1 + .byte 0xe /* DW_CFA_def_cfa_offset */ + .byte 0x8 /* .uleb128 0x8 */ + .byte 0x85 /* DW_CFA_offset, column 0x5 */ + .byte 0x2 /* .uleb128 0x2 */ + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI1-.LCFI0 + .byte 0xd /* DW_CFA_def_cfa_register */ + .byte 0x5 /* .uleb128 0x5 */ + .align 4 +.LEFDE1: +#endif + +#endif /* ifndef __x86_64__ */ + +#endif /* defined __i386__ */ Added: external/ctypes/source/libffi/src/x86/ffi_darwin.c ============================================================================== --- (empty file) +++ external/ctypes/source/libffi/src/x86/ffi_darwin.c Sat Jun 10 22:08:12 2006 @@ -0,0 +1,610 @@ +# ifdef __i386__ +/* ----------------------------------------------------------------------- + ffi.c - Copyright (c) 1996, 1998, 1999, 2001 Red Hat, Inc. + Copyright (c) 2002 Ranjit Mathew + Copyright (c) 2002 Bo Thorsen + Copyright (c) 2002 Roger Sayle + + x86 Foreign Function Interface + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + ----------------------------------------------------------------------- */ + +#ifndef __x86_64__ + +#include +#include + +#include + +/* ffi_prep_args is called by the assembly routine once stack space + has been allocated for the function's arguments */ + +/*@-exportheader@*/ +void ffi_prep_args(char *stack, extended_cif *ecif); + +static inline int retval_on_stack(ffi_type* tp) +{ + if (tp->type == FFI_TYPE_STRUCT) { + int sz = tp->size; + if (sz > 8) { + return 1; + } + switch (sz) { + case 1: case 2: case 4: case 8: return 0; + default: return 1; + } + } + return 0; +} + + +void ffi_prep_args(char *stack, extended_cif *ecif) +/*@=exportheader@*/ +{ + register unsigned int i; + register void **p_argv; + register char *argp; + register ffi_type **p_arg; + + argp = stack; + + if (retval_on_stack(ecif->cif->rtype)) { + *(void **) argp = ecif->rvalue; + argp += 4; + } + + + p_argv = ecif->avalue; + + for (i = ecif->cif->nargs, p_arg = ecif->cif->arg_types; + i != 0; + i--, p_arg++) + { + size_t z; + + /* Align if necessary */ + if ((sizeof(int) - 1) & (unsigned) argp) + argp = (char *) ALIGN(argp, sizeof(int)); + + z = (*p_arg)->size; + if (z < sizeof(int)) + { + z = sizeof(int); + switch ((*p_arg)->type) + { + case FFI_TYPE_SINT8: + *(signed int *) argp = (signed int)*(SINT8 *)(* p_argv); + break; + + case FFI_TYPE_UINT8: + *(unsigned int *) argp = (unsigned int)*(UINT8 *)(* p_argv); + break; + + case FFI_TYPE_SINT16: + *(signed int *) argp = (signed int)*(SINT16 *)(* p_argv); + break; + + case FFI_TYPE_UINT16: + *(unsigned int *) argp = (unsigned int)*(UINT16 *)(* p_argv); + break; + + case FFI_TYPE_SINT32: + *(signed int *) argp = (signed int)*(SINT32 *)(* p_argv); + break; + + case FFI_TYPE_UINT32: + *(unsigned int *) argp = (unsigned int)*(UINT32 *)(* p_argv); + break; + + case FFI_TYPE_STRUCT: + *(unsigned int *) argp = (unsigned int)*(UINT32 *)(* p_argv); + break; + + default: + FFI_ASSERT(0); + } + } + else + { + memcpy(argp, *p_argv, z); + } + p_argv++; + argp += z; + } + + return; +} + +/* Perform machine dependent cif processing */ +ffi_status ffi_prep_cif_machdep(ffi_cif *cif) +{ + /* Set the return type flag */ + switch (cif->rtype->type) + { + case FFI_TYPE_VOID: +#if !defined(X86_WIN32) + case FFI_TYPE_STRUCT: +#endif + case FFI_TYPE_SINT64: + case FFI_TYPE_FLOAT: + case FFI_TYPE_DOUBLE: + case FFI_TYPE_LONGDOUBLE: + cif->flags = (unsigned) cif->rtype->type; + break; + + case FFI_TYPE_UINT64: + cif->flags = FFI_TYPE_SINT64; + break; + +#if defined X86_WIN32 + + case FFI_TYPE_STRUCT: + if (cif->rtype->size == 1) + { + cif->flags = FFI_TYPE_SINT8; /* same as char size */ + } + else if (cif->rtype->size == 2) + { + cif->flags = FFI_TYPE_SINT16; /* same as short size */ + } + else if (cif->rtype->size == 4) + { + cif->flags = FFI_TYPE_INT; /* same as int type */ + } + else if (cif->rtype->size == 8) + { + cif->flags = FFI_TYPE_SINT64; /* same as int64 type */ + } + else + { + cif->flags = FFI_TYPE_STRUCT; + } + break; +#endif + + default: + cif->flags = FFI_TYPE_INT; + break; + } + + /* Darwin: The stack needs to be aligned to a multiple of 16 bytes */ +#if 0 + cif->bytes = (cif->bytes + 15) & ~0xF; +#endif + + return FFI_OK; +} + +/*@-declundef@*/ +/*@-exportheader@*/ +extern void ffi_call_SYSV(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)(void)); +/*@=declundef@*/ +/*@=exportheader@*/ + +#ifdef X86_WIN32 +/*@-declundef@*/ +/*@-exportheader@*/ +extern void ffi_call_STDCALL(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)(void)); +/*@=declundef@*/ +/*@=exportheader@*/ +#endif /* X86_WIN32 */ + +void ffi_call(/*@dependent@*/ ffi_cif *cif, + void (*fn)(), + /*@out@*/ void *rvalue, + /*@dependent@*/ void **avalue) +{ + extended_cif ecif; + int flags; + + ecif.cif = cif; + ecif.avalue = avalue; + + /* If the return value is a struct and we don't have a return */ + /* value address then we need to make one */ + + if ((rvalue == NULL) && retval_on_stack(cif->rtype)) + { + /*@-sysunrecog@*/ + ecif.rvalue = alloca(cif->rtype->size); + /*@=sysunrecog@*/ + } + else + ecif.rvalue = rvalue; + + flags = cif->flags; + if (flags == FFI_TYPE_STRUCT) { + if (cif->rtype->size == 8) { + flags = FFI_TYPE_SINT64; + } else if (cif->rtype->size == 4) { + flags = FFI_TYPE_INT; + } else if (cif->rtype->size == 2) { + flags = FFI_TYPE_INT; + } else if (cif->rtype->size == 1) { + flags = FFI_TYPE_INT; + } + } + + + switch (cif->abi) + { + case FFI_SYSV: + /*@-usedef@*/ + /* To avoid changing the assembly code make sure the size of the argument + * block is a multiple of 16. Then add 8 to compensate for local variables + * in ffi_call_SYSV. + */ + ffi_call_SYSV(ffi_prep_args, &ecif, ALIGN(cif->bytes, 16) +8, + flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#ifdef X86_WIN32 + case FFI_STDCALL: + /*@-usedef@*/ + ffi_call_STDCALL(ffi_prep_args, &ecif, cif->bytes, + cif->flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#endif /* X86_WIN32 */ + default: + FFI_ASSERT(0); + break; + } +} + + +/** private members **/ + +static void ffi_prep_incoming_args_SYSV (char *stack, void **ret, + void** args, ffi_cif* cif); +static void ffi_closure_SYSV (ffi_closure *) + __attribute__ ((regparm(1))); +#if !FFI_NO_RAW_API +static void ffi_closure_raw_SYSV (ffi_raw_closure *) + __attribute__ ((regparm(1))); +#endif + +/* This function is jumped to by the trampoline */ + +static void +ffi_closure_SYSV (closure) + ffi_closure *closure; +{ + // this is our return value storage + long double res; + + // our various things... + ffi_cif *cif; + void **arg_area; + unsigned short rtype; + void *resp = (void*)&res; + void *args = __builtin_dwarf_cfa (); + + cif = closure->cif; + arg_area = (void**) alloca (cif->nargs * sizeof (void*)); + + /* this call will initialize ARG_AREA, such that each + * element in that array points to the corresponding + * value on the stack; and if the function returns + * a structure, it will re-set RESP to point to the + * structure return address. */ + + ffi_prep_incoming_args_SYSV(args, (void**)&resp, arg_area, cif); + + (closure->fun) (cif, resp, arg_area, closure->user_data); + + rtype = cif->flags; + + if (!retval_on_stack(cif->rtype) && cif->flags == FFI_TYPE_STRUCT) { + if (cif->rtype->size == 8) { + rtype = FFI_TYPE_SINT64; + } else { + rtype = FFI_TYPE_INT; + } + } + + /* now, do a generic return based on the value of rtype */ + if (rtype == FFI_TYPE_INT) + { + asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); + } + else if (rtype == FFI_TYPE_FLOAT) + { + asm ("flds (%0)" : : "r" (resp) : "st" ); + } + else if (rtype == FFI_TYPE_DOUBLE) + { + asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_LONGDOUBLE) + { + asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_SINT64) + { + asm ("movl 0(%0),%%eax;" + "movl 4(%0),%%edx" + : : "r"(resp) + : "eax", "edx"); + } +#ifdef X86_WIN32 + else if (rtype == FFI_TYPE_SINT8) /* 1-byte struct */ + { + asm ("movsbl (%0),%%eax" : : "r" (resp) : "eax"); + } + else if (rtype == FFI_TYPE_SINT16) /* 2-bytes struct */ + { + asm ("movswl (%0),%%eax" : : "r" (resp) : "eax"); + } +#endif +} + +/*@-exportheader@*/ +static void +ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, + void **avalue, ffi_cif *cif) +/*@=exportheader@*/ +{ + register unsigned int i; + register void **p_argv; + register char *argp; + register ffi_type **p_arg; + + argp = stack; + + if (retval_on_stack(cif->rtype)) { + *rvalue = *(void **) argp; + argp += 4; + } + + p_argv = avalue; + + for (i = cif->nargs, p_arg = cif->arg_types; (i != 0); i--, p_arg++) + { + size_t z; + + /* Align if necessary */ + if ((sizeof(int) - 1) & (unsigned) argp) { + argp = (char *) ALIGN(argp, sizeof(int)); + } + + z = (*p_arg)->size; + + /* because we're little endian, this is what it turns into. */ + + *p_argv = (void*) argp; + + p_argv++; + argp += z; + } + + return; +} + +/* How to make a trampoline. Derived from gcc/config/i386/i386.c. */ + +#define FFI_INIT_TRAMPOLINE(TRAMP,FUN,CTX) \ +({ unsigned char *__tramp = (unsigned char*)(TRAMP); \ + unsigned int __fun = (unsigned int)(FUN); \ + unsigned int __ctx = (unsigned int)(CTX); \ + unsigned int __dis = __fun - ((unsigned int) __tramp + FFI_TRAMPOLINE_SIZE); \ + *(unsigned char*) &__tramp[0] = 0xb8; \ + *(unsigned int*) &__tramp[1] = __ctx; /* movl __ctx, %eax */ \ + *(unsigned char *) &__tramp[5] = 0xe9; \ + *(unsigned int*) &__tramp[6] = __dis; /* jmp __fun */ \ + }) + + +/* the cif must already be prep'ed */ + +ffi_status +ffi_prep_closure (ffi_closure* closure, + ffi_cif* cif, + void (*fun)(ffi_cif*,void*,void**,void*), + void *user_data) +{ + FFI_ASSERT (cif->abi == FFI_SYSV); + + FFI_INIT_TRAMPOLINE (&closure->tramp[0], \ + &ffi_closure_SYSV, \ + (void*)closure); + + closure->cif = cif; + closure->user_data = user_data; + closure->fun = fun; + + return FFI_OK; +} + +/* ------- Native raw API support -------------------------------- */ + +#if !FFI_NO_RAW_API + +static void +ffi_closure_raw_SYSV (closure) + ffi_raw_closure *closure; +{ + // this is our return value storage + long double res; + + // our various things... + ffi_raw *raw_args; + ffi_cif *cif; + unsigned short rtype; + void *resp = (void*)&res; + + /* get the cif */ + cif = closure->cif; + + /* the SYSV/X86 abi matches the RAW API exactly, well.. almost */ + raw_args = (ffi_raw*) __builtin_dwarf_cfa (); + + (closure->fun) (cif, resp, raw_args, closure->user_data); + + rtype = cif->flags; + + /* now, do a generic return based on the value of rtype */ + if (rtype == FFI_TYPE_INT) + { + asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); + } + else if (rtype == FFI_TYPE_FLOAT) + { + asm ("flds (%0)" : : "r" (resp) : "st" ); + } + else if (rtype == FFI_TYPE_DOUBLE) + { + asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_LONGDOUBLE) + { + asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_SINT64) + { + asm ("movl 0(%0),%%eax; movl 4(%0),%%edx" + : : "r"(resp) + : "eax", "edx"); + } +} + + + + +ffi_status +ffi_prep_raw_closure (ffi_raw_closure* closure, + ffi_cif* cif, + void (*fun)(ffi_cif*,void*,ffi_raw*,void*), + void *user_data) +{ + int i; + + FFI_ASSERT (cif->abi == FFI_SYSV); + + // we currently don't support certain kinds of arguments for raw + // closures. This should be implemented by a separate assembly language + // routine, since it would require argument processing, something we + // don't do now for performance. + + for (i = cif->nargs-1; i >= 0; i--) + { + FFI_ASSERT (cif->arg_types[i]->type != FFI_TYPE_STRUCT); + FFI_ASSERT (cif->arg_types[i]->type != FFI_TYPE_LONGDOUBLE); + } + + + FFI_INIT_TRAMPOLINE (&closure->tramp[0], &ffi_closure_raw_SYSV, + (void*)closure); + + closure->cif = cif; + closure->user_data = user_data; + closure->fun = fun; + + return FFI_OK; +} + +static void +ffi_prep_args_raw(char *stack, extended_cif *ecif) +{ + memcpy (stack, ecif->avalue, ecif->cif->bytes); +} + +/* we borrow this routine from libffi (it must be changed, though, to + * actually call the function passed in the first argument. as of + * libffi-1.20, this is not the case.) + */ + +extern void +ffi_call_SYSV(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)()); + +#ifdef X86_WIN32 +extern void +ffi_call_STDCALL(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)()); +#endif /* X86_WIN32 */ + +void +ffi_raw_call(/*@dependent@*/ ffi_cif *cif, + void (*fn)(), + /*@out@*/ void *rvalue, + /*@dependent@*/ ffi_raw *fake_avalue) +{ + extended_cif ecif; + void **avalue = (void **)fake_avalue; + + ecif.cif = cif; + ecif.avalue = avalue; + + /* If the return value is a struct and we don't have a return */ + /* value address then we need to make one */ + + if ((rvalue == NULL) && retval_on_stack(cif->rtype)) + { + /*@-sysunrecog@*/ + ecif.rvalue = alloca(cif->rtype->size); + /*@=sysunrecog@*/ + } + else + ecif.rvalue = rvalue; + + + switch (cif->abi) + { + case FFI_SYSV: + /*@-usedef@*/ + ffi_call_SYSV(ffi_prep_args_raw, &ecif, cif->bytes, + cif->flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#ifdef X86_WIN32 + case FFI_STDCALL: + /*@-usedef@*/ + ffi_call_STDCALL(ffi_prep_args_raw, &ecif, cif->bytes, + cif->flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#endif /* X86_WIN32 */ + default: + FFI_ASSERT(0); + break; + } +} + +#endif + +#endif /* __x86_64__ */ + +#endif /* __i386__ */ Modified: external/ctypes/source/libffi/src/x86/ffitarget.h ============================================================================== --- external/ctypes/source/libffi/src/x86/ffitarget.h (original) +++ external/ctypes/source/libffi/src/x86/ffitarget.h Sat Jun 10 22:08:12 2006 @@ -51,7 +51,7 @@ #endif /* ---- Intel x86 and AMD x86-64 - */ -#if !defined(X86_WIN32) && (defined(__i386__) || defined(__x86_64__)) +#if !defined(X86_WIN32) && (defined(__i386__) || defined(__x86_64__)) FFI_SYSV, FFI_UNIX64, /* Unix variants all use the same ABI for x86-64 */ #ifdef __i386__ Modified: external/ctypes/source/stgdict.c ============================================================================== --- external/ctypes/source/stgdict.c (original) +++ external/ctypes/source/stgdict.c Sat Jun 10 22:08:12 2006 @@ -142,30 +142,129 @@ return PyType_stgdict((PyObject *)self->ob_type); } -#if 0 -/* work in progress: anonymous structure fields */ -int -GetFields(PyObject *desc, int *pindex, int *psize, int *poffset, int *palign, int pack); +/* descr is the descriptor for a field marked as anonymous. Get all the + _fields_ descriptors from descr->proto, create new descriptors with offset + and index adjusted, and stuff them into type. + */ +static int +MakeFields(PyObject *type, CFieldObject *descr, + Py_ssize_t index, Py_ssize_t offset) +{ + Py_ssize_t i; + PyObject *fields; + PyObject *fieldlist; + + fields = PyObject_GetAttrString(descr->proto, "_fields_"); + if (fields == NULL) + return -1; + fieldlist = PySequence_Fast(fields, "_fields_ must be a sequence"); + Py_DECREF(fields); + if (fieldlist == NULL) + return -1; + + for (i = 0; i < PySequence_Fast_GET_SIZE(fieldlist); ++i) { + PyObject *pair = PySequence_Fast_GET_ITEM(fieldlist, i); /* borrowed */ + PyObject *fname, *ftype; + CFieldObject *fdescr; + CFieldObject *new_descr; + // Convert to PyArg_UnpackTuple... + if (!PyArg_ParseTuple(pair, "OO", &fname, &ftype)) { + Py_DECREF(fieldlist); + return -1; + } + fdescr = (CFieldObject *)PyObject_GetAttr(descr->proto, fname); + if (fdescr == NULL) { + Py_DECREF(fieldlist); + return -1; + } + if (fdescr->ob_type != &CField_Type) { + PyErr_SetString(PyExc_TypeError, "unexpected type"); + Py_DECREF(fdescr); + Py_DECREF(fieldlist); + return -1; + } + if (fdescr->anonymous) { + int rc = MakeFields(type, fdescr, + index + fdescr->index, + offset + fdescr->offset); + Py_DECREF(fdescr); + if (rc == -1) { + Py_DECREF(fieldlist); + return -1; + } + continue; + } + new_descr = (CFieldObject *)PyObject_CallObject((PyObject *)&CField_Type, NULL); + assert(new_descr->ob_type == &CField_Type); + if (new_descr == NULL) { + Py_DECREF(fdescr); + Py_DECREF(fieldlist); + return -1; + } + new_descr->size = fdescr->size; + new_descr->offset = fdescr->offset + offset; + new_descr->index = fdescr->index + index; + new_descr->proto = fdescr->proto; + Py_XINCREF(new_descr->proto); + new_descr->getfunc = fdescr->getfunc; + new_descr->setfunc = fdescr->setfunc; + + Py_DECREF(fdescr); + + if (-1 == PyObject_SetAttr(type, fname, (PyObject *)new_descr)) { + Py_DECREF(fieldlist); + Py_DECREF(new_descr); + return -1; + } + Py_DECREF(new_descr); + } + Py_DECREF(fieldlist); + return 0; +} +/* Iterate over the names in the type's _anonymous_ attribute, if present, + */ +static int +MakeAnonFields(PyObject *type) { - int i; - PyObject *tuples = PyObject_GetAttrString(desc, "_fields_"); - if (tuples == NULL) - return -1; - if (!PyTuple_Check(tuples)) - return -1; /* leak */ - for (i = 0; i < PyTuple_GET_SIZE(tuples); ++i) { - char *fname; - PyObject *dummy; - CFieldObject *field; - PyObject *pair = PyTuple_GET_ITEM(tuples, i); - if (!PyArg_ParseTuple(pair, "sO", &fname, &dummy)) - return -1; /* leak */ - field = PyObject_GetAttrString(desc, fname); - Py_DECREF(field); + PyObject *anon; + PyObject *anon_names; + Py_ssize_t i; + + anon = PyObject_GetAttrString(type, "_anonymous_"); + if (anon == NULL) { + PyErr_Clear(); + return 0; } + anon_names = PySequence_Fast(anon, "_anonymous_ must be a sequence"); + Py_DECREF(anon); + if (anon_names == NULL) + return -1; + + for (i = 0; i < PySequence_Fast_GET_SIZE(anon_names); ++i) { + PyObject *fname = PySequence_Fast_GET_ITEM(anon_names, i); /* borrowed */ + CFieldObject *descr = (CFieldObject *)PyObject_GetAttr(type, fname); + if (descr == NULL) { + Py_DECREF(anon_names); + return -1; + } + assert(descr->ob_type == &CField_Type); + descr->anonymous = 1; + + /* descr is in the field descriptor. */ + if (-1 == MakeFields(type, (CFieldObject *)descr, + ((CFieldObject *)descr)->index, + ((CFieldObject *)descr)->offset)) { + Py_DECREF(descr); + Py_DECREF(anon_names); + return -1; + } + Py_DECREF(descr); + } + + Py_DECREF(anon_names); + return 0; } -#endif /* Retrive the (optional) _pack_ attribute from a type, the _fields_ attribute, @@ -368,5 +467,5 @@ stgdict->size = size; stgdict->align = total_align; stgdict->length = len; /* ADD ffi_ofs? */ - return 0; + return MakeAnonFields(type); } From python-checkins at python.org Sat Jun 10 22:29:38 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 22:29:38 +0200 (CEST) Subject: [Python-checkins] r46831 - python/trunk/Doc/lib/lib.tex python/trunk/Doc/lib/libctypes.tex python/trunk/Doc/lib/libctypesref.tex Message-ID: <20060610202938.732BE1E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 22:29:34 2006 New Revision: 46831 Removed: python/trunk/Doc/lib/libctypesref.tex Modified: python/trunk/Doc/lib/lib.tex python/trunk/Doc/lib/libctypes.tex Log: New docs for ctypes. Modified: python/trunk/Doc/lib/lib.tex ============================================================================== --- python/trunk/Doc/lib/lib.tex (original) +++ python/trunk/Doc/lib/lib.tex Sat Jun 10 22:29:34 2006 @@ -245,7 +245,6 @@ \input{libplatform} \input{liberrno} \input{libctypes} -\input{libctypesref} \input{libsomeos} % Optional Operating System Services \input{libselect} Modified: python/trunk/Doc/lib/libctypes.tex ============================================================================== --- python/trunk/Doc/lib/libctypes.tex (original) +++ python/trunk/Doc/lib/libctypes.tex Sat Jun 10 22:29:34 2006 @@ -1,4 +1,4 @@ -\newlength{\locallinewidth} +\ifx\locallinewidth\undefined\newlength{\locallinewidth}\fi \setlength{\locallinewidth}{\linewidth} \section{\module{ctypes} --- A foreign function library for Python.} \declaremodule{standard}{ctypes} @@ -70,6 +70,12 @@ XXX Add section for Mac OS X. +\subsubsection{Finding shared libraries\label{ctypes-finding-shared-libraries}} + +XXX Add description of ctypes.util.find{\_}library (once I really +understand it enough to describe it). + + \subsubsection{Accessing functions from loaded dlls\label{ctypes-accessing-functions-from-loaded-dlls}} Functions are accessed as attributes of dll objects: @@ -186,158 +192,172 @@ have to learn more about \code{ctypes} data types. -\subsubsection{Simple data types\label{ctypes-simple-data-types}} +\subsubsection{Fundamental data types\label{ctypes-fundamental-data-types}} \code{ctypes} defines a number of primitive C compatible data types : \begin{quote} - -\begin{longtable}[c]{|p{0.19\locallinewidth}|p{0.28\locallinewidth}|p{0.14\locallinewidth}|} -\hline -\textbf{ +\begin{tableiii}{l|l|l}{textrm} +{ ctypes type -} & \textbf{ +} +{ C type -} & \textbf{ +} +{ Python type -} \\ -\hline -\endhead - +} +\lineiii{ \class{c{\_}char} - & +} +{ \code{char} - & +} +{ character - \\ -\hline - +} +\lineiii{ \class{c{\_}byte} - & +} +{ \code{char} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}ubyte} - & +} +{ \code{unsigned char} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}short} - & +} +{ \code{short} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}ushort} - & +} +{ \code{unsigned short} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}int} - & +} +{ \code{int} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}uint} - & +} +{ \code{unsigned int} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}long} - & +} +{ \code{long} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}ulong} - & +} +{ \code{unsigned long} - & +} +{ long - \\ -\hline - +} +\lineiii{ \class{c{\_}longlong} - & +} +{ \code{{\_}{\_}int64} or \code{long long} - & +} +{ long - \\ -\hline - +} +\lineiii{ \class{c{\_}ulonglong} - & +} +{ \code{unsigned {\_}{\_}int64} or \code{unsigned long long} - & +} +{ long - \\ -\hline - +} +\lineiii{ \class{c{\_}float} - & +} +{ \code{float} - & +} +{ float - \\ -\hline - +} +\lineiii{ \class{c{\_}double} - & +} +{ \code{double} - & +} +{ float - \\ -\hline - +} +\lineiii{ \class{c{\_}char{\_}p} - & +} +{ \code{char *} (NUL terminated) - & +} +{ string or \code{None} - \\ -\hline - +} +\lineiii{ \class{c{\_}wchar{\_}p} - & +} +{ \code{wchar{\_}t *} (NUL terminated) - & +} +{ unicode or \code{None} - \\ -\hline - +} +\lineiii{ \class{c{\_}void{\_}p} - & +} +{ \code{void *} - & +} +{ integer or \code{None} - \\ -\hline -\end{longtable} +} +\end{tableiii} \end{quote} All these types can be created by calling them with an optional @@ -380,6 +400,7 @@ c_char_p('Hi, there') >>> print s # first string is unchanged Hello, World +>>> \end{verbatim} You should be careful, however, not to pass them to functions @@ -575,7 +596,7 @@ >>> GetModuleHandle.restype = ValidHandle # doctest: +WINDOWS >>> GetModuleHandle(None) # doctest: +WINDOWS 486539264 ->>> GetModuleHandle("something silly") # doctest: +WINDOWS +IGNORE_EXCEPTION_DETAIL +>>> GetModuleHandle("something silly") # doctest: +WINDOWS Traceback (most recent call last): File "", line 1, in ? File "", line 3, in ValidHandle @@ -744,6 +765,7 @@ >>> >>> print len(MyStruct().point_array) 4 +>>> \end{verbatim} Instances are created in the usual way, by calling the class: @@ -772,6 +794,10 @@ \subsubsection{Pointers\label{ctypes-pointers}} +XXX Rewrite this section. Normally one only uses indexing, not the .contents +attribute! +List some recipes with pointers. bool(ptr), POINTER(tp)(), ...? + Pointer instances are created by calling the \code{pointer} function on a \code{ctypes} type: \begin{verbatim} @@ -781,16 +807,25 @@ >>> \end{verbatim} -XXX XXX Not correct: use indexing, not the contents atribute - Pointer instances have a \code{contents} attribute which returns the -ctypes' type pointed to, the \code{c{\_}int(42)} in the above case: +object to which the pointer points, the \code{i} object above: \begin{verbatim} >>> pi.contents c_long(42) >>> \end{verbatim} +Note that \code{ctypes} does not have OOR (original object return), it +constructs a new, equivalent object each time you retrieve an +attribute: +\begin{verbatim} +>>> pi.contents is i +False +>>> pi.contents is pi.contents +False +>>> +\end{verbatim} + Assigning another \class{c{\_}int} instance to the pointer's contents attribute would cause the pointer to point to the memory location where this is stored: @@ -808,23 +843,21 @@ >>> \end{verbatim} -XXX What is this??? Assigning to an integer index changes the pointed to value: \begin{verbatim} ->>> i2 = pi[0] ->>> i2 -99 +>>> print i +c_long(99) >>> pi[0] = 22 ->>> i2 -99 +>>> print i +c_long(22) >>> \end{verbatim} It is also possible to use indexes different from 0, but you must know -what you're doing when you use this: You access or change arbitrary -memory locations when you do this. Generally you only use this feature -if you receive a pointer from a C function, and you \emph{know} that the -pointer actually points to an array instead of a single item. +what you're doing, just as in C: You can access or change arbitrary +memory locations. Generally you only use this feature if you receive a +pointer from a C function, and you \emph{know} that the pointer actually +points to an array instead of a single item. \subsubsection{Pointer classes/types\label{ctypes-pointer-classestypes}} @@ -837,7 +870,7 @@ >>> PI = POINTER(c_int) >>> PI ->>> PI(42) # doctest: +IGNORE_EXCEPTION_DETAIL +>>> PI(42) Traceback (most recent call last): File "", line 1, in ? TypeError: expected c_long instead of int @@ -847,6 +880,82 @@ \end{verbatim} +\subsubsection{Type conversions\label{ctypes-type-conversions}} + +Usually, ctypes does strict type checking. This means, if you have +\code{POINTER(c{\_}int)} in the \member{argtypes} list of a function or in the +\member{{\_}fields{\_}} of a structure definition, only instances of exactly the +same type are accepted. There are some exceptions to this rule, where +ctypes accepts other objects. For example, you can pass compatible +array instances instead of pointer types. So, for \code{POINTER(c{\_}int)}, +ctypes accepts an array of c{\_}int values: +\begin{verbatim} +>>> class Bar(Structure): +... _fields_ = [("count", c_int), ("values", POINTER(c_int))] +... +>>> bar = Bar() +>>> print bar._objects +None +>>> bar.values = (c_int * 3)(1, 2, 3) +>>> print bar._objects +{'1': ({}, )} +>>> bar.count = 3 +>>> for i in range(bar.count): +... print bar.values[i] +... +1 +2 +3 +>>> +\end{verbatim} + +To set a POINTER type field to \code{NULL}, you can assign \code{None}: +\begin{verbatim} +>>> bar.values = None +>>> +\end{verbatim} + +XXX list other conversions... + +Sometimes you have instances of incompatible types. In \code{C}, you can +cast one type into another type. \code{ctypes} provides a \code{cast} +function which can be used in the same way. The Bar structure defined +above accepts \code{POINTER(c{\_}int)} pointers or \class{c{\_}int} arrays for its +\code{values} field, but not instances of other types: +\begin{verbatim} +>>> bar.values = (c_byte * 4)() +Traceback (most recent call last): + File "", line 1, in ? +TypeError: incompatible types, c_byte_Array_4 instance instead of LP_c_long instance +>>> +\end{verbatim} + +For these cases, the \code{cast} function is handy. + +The \code{cast} function can be used to cast a ctypes instance into a +pointer to a different ctypes data type. \code{cast} takes two +parameters, a ctypes object that is or can be converted to a pointer +of some kind, and a ctypes pointer type. It returns an instance of +the second argument, which references the same memory block as the +first argument: +\begin{verbatim} +>>> a = (c_byte * 4)() +>>> cast(a, POINTER(c_int)) + +>>> +\end{verbatim} + +So, \code{cast} can be used to assign to the \code{values} field of \code{Bar} +the structure: +\begin{verbatim} +>>> bar = Bar() +>>> bar.values = cast((c_byte * 4)(), POINTER(c_int)) +>>> print bar.values[0] +0 +>>> +\end{verbatim} + + \subsubsection{Incomplete Types\label{ctypes-incomplete-types}} \emph{Incomplete Types} are structures, unions or arrays whose members are @@ -1175,6 +1284,7 @@ >>> rc.a, rc.b = rc.b, rc.a >>> print rc.a.x, rc.a.y, rc.b.x, rc.b.y 3 4 3 4 +>>> \end{verbatim} Hm. We certainly expected the last statement to print \code{3 4 1 2}. @@ -1184,6 +1294,7 @@ >>> temp0, temp1 = rc.b, rc.a >>> rc.a = temp0 >>> rc.b = temp1 +>>> \end{verbatim} Note that \code{temp0} and \code{temp1} are objects still using the internal @@ -1214,6 +1325,80 @@ Accessing the contents again constructs a new Python each time! +\subsubsection{Variable-sized data types\label{ctypes-variable-sized-data-types}} + +\code{ctypes} provides some support for variable-sized arrays and +structures (this was added in version 0.9.9.7). + +The \code{resize} function can be used to resize the memory buffer of an +existing ctypes object. The function takes the object as first +argument, and the requested size in bytes as the second argument. The +memory block cannot be made smaller than the natural memory block +specified by the objects type, a \code{ValueError} is raised if this is +tried: +\begin{verbatim} +>>> short_array = (c_short * 4)() +>>> print sizeof(short_array) +8 +>>> resize(short_array, 4) +Traceback (most recent call last): + ... +ValueError: minimum size is 8 +>>> resize(short_array, 32) +>>> sizeof(short_array) +32 +>>> sizeof(type(short_array)) +8 +>>> +\end{verbatim} + +This is nice and fine, but how would one access the additional +elements contained in this array? Since the type still only knows +about 4 elements, we get errors accessing other elements: +\begin{verbatim} +>>> short_array[:] +[0, 0, 0, 0] +>>> short_array[7] +Traceback (most recent call last): + ... +IndexError: invalid index +>>> +\end{verbatim} + +The solution is to use 1-element arrays; as a special case ctypes does +no bounds checking on them: +\begin{verbatim} +>>> short_array = (c_short * 1)() +>>> print sizeof(short_array) +2 +>>> resize(short_array, 32) +>>> sizeof(short_array) +32 +>>> sizeof(type(short_array)) +2 +>>> short_array[0:8] +[0, 0, 0, 0, 0, 0, 0, 0] +>>> short_array[7] = 42 +>>> short_array[0:8] +[0, 0, 0, 0, 0, 0, 0, 42] +>>> +\end{verbatim} + +Using 1-element arrays as variable sized fields in structures works as +well, but they should be used as the last field in the structure +definition. This example shows a definition from the Windows header +files: +\begin{verbatim} +class SP_DEVICE_INTERFACE_DETAIL_DATA(Structure): + _fields_ = [("cbSize", c_int), + ("DevicePath", c_char * 1)] +\end{verbatim} + +Another way to use variable-sized data types with \code{ctypes} is to use +the dynamic nature of Python, and (re-)define the data type after the +required size is already known, on a case by case basis. + + \subsubsection{Bugs, ToDo and non-implemented things\label{ctypes-bugs-todo-non-implemented-things}} Enumeration types are not implemented. You can do it easily yourself, @@ -1224,3 +1409,636 @@ % compile-command: "make.bat" % End: + +\subsection{ctypes reference\label{ctypes-ctypes-reference}} + + +\subsubsection{Loading shared libraries\label{ctypes-loading-shared-libraries}} + +\begin{classdesc}{LibraryLoader}{dlltype} +Class which loads shared libraries. +\end{classdesc} + +\begin{methoddesc}{LoadLibrary}{name, mode=RTLD_LOCAL, handle=None} +Load a shared library. +\end{methoddesc} + +\begin{classdesc}{CDLL}{name, mode=RTLD_LOCAL, handle=None} +XXX +\end{classdesc} + +\begin{datadescni}{cdll} +XXX +\end{datadescni} + +\begin{funcdesc}{OleDLL}{name, mode=RTLD_LOCAL, handle=None} +XXX +\end{funcdesc} + +\begin{datadescni}{oledll} +XXX +\end{datadescni} + +\begin{classdesc*}{py_object} +XXX +\end{classdesc*} + +\begin{funcdesc}{PyDLL}{name, mode=RTLD_LOCAL, handle=None} +XXX +\end{funcdesc} + +\begin{datadescni}{pydll} +XXX +\end{datadescni} + +\begin{datadescni}{RTLD_GLOBAL} +XXX +\end{datadescni} + +\begin{datadescni}{RTLD_LOCAL} +XXX +\end{datadescni} + +\begin{funcdesc}{WinDLL}{name, mode=RTLD_LOCAL, handle=None} +XXX +\end{funcdesc} + +\begin{datadescni}{windll} +XXX +\end{datadescni} + +\begin{datadescni}{pythonapi()} +XXX +\end{datadescni} + + +\subsubsection{Foreign functions\label{ctypes-foreign-functions}} + +The ultimate goal of \code{ctypes} is to call functions in shared +libraries, aka as foreign functions. Foreign function instances can +be created by retrieving them as attributes of loaded shared +libraries, or by instantiating a \emph{function prototype}. + +By default, functions got as attributes of loaded shared libraries +accept any arguments, and have a return type of \class{c{\_}int}. + +Function prototypes are created by factory functions. + +They are created by calling one of the following factory functions: + +\begin{funcdesc}{CFUNCTYPE}{restype, *argtypes} +This is a factory function that returns a function prototype. The +function prototype describes a function that has a result type of +\member{restype}, and accepts arguments as specified by +\member{argtypes}. The function prototype can be used to construct +several kinds of functions, depending on how the prototype is +called. + +The prototypes returned by \function{CFUNCTYPE} or \code{PYFUNCTYPE} create +functions that use the standard C calling convention, prototypes +returned from \function{WINFUNCTYPE} (on Windows) use the \code{{\_}{\_}stdcall} +calling convention. + +Functions created by calling the \function{CFUNCTYPE} and \function{WINFUNCTYPE} +prototypes release the Python GIL before entering the foreign +function, and acquire it back after leaving the function code. +\end{funcdesc} + +\begin{funcdesc}{WINFUNCTYPE}{restype, *argtypes} +TBD +\end{funcdesc} + +\begin{funcdesc}{PYFUNCTYPE}{restype, *argtypes} +TBD +\end{funcdesc} + +\begin{excdesc}{ArgumentError()} +This exception is raised when a foreign function call cannot +convert one of the passed arguments. +\end{excdesc} + + +\subsubsection{Utility functions\label{ctypes-utility-functions}} + +\begin{funcdesc}{addressof}{obj} +Returns the address of the memory buffer as integer. \code{obj} must +be an instance of a ctypes type. +\end{funcdesc} + +\begin{funcdesc}{alignment}{obj_or_type} +Returns the alignment requirements of a ctypes type. +\code{obj{\_}or{\_}type} must be a ctypes type or instance. +\end{funcdesc} + +\begin{funcdesc}{byref}{obj} +Returns a light-weight pointer to \code{obj}, which must be an +instance of a ctypes type. The returned object can only be used as +a foreign function call parameter. It behaves similar to +\code{pointer(obj)}, but the construction is a lot faster. +\end{funcdesc} + +\begin{funcdesc}{cast}{obj, type} +This function is similar to the cast operator in C. It returns a +new instance of \code{type} which points to the same memory block as +\code{obj}. \code{type} must be a pointer type, and \code{obj} must be an +object that can be interpreted as a pointer. +\end{funcdesc} + +\begin{funcdesc}{create_string_buffer}{init_or_size\optional{, size}} +This function creates a mutable character buffer. The returned +object is a ctypes array of \class{c{\_}char}. + +\code{init{\_}or{\_}size} must be an integer which specifies the size of +the array, or a string which will be used to initialize the array +items. + +If a string is specified as first argument, the buffer is made one +item larger than the length of the string so that the last element +in the array is a NUL termination character. An integer can be +passed as second argument which allows to specify the size of the +array if the length of the string should not be used. + +If the first parameter is a unicode string, it is converted into +an 8-bit string according to ctypes conversion rules. +\end{funcdesc} + +\begin{funcdesc}{create_unicode_buffer}{init_or_size\optional{, size}} +This function creates a mutable unicode character buffer. The +returned object is a ctypes array of \class{c{\_}wchar}. + +\code{init{\_}or{\_}size} must be an integer which specifies the size of +the array, or a unicode string which will be used to initialize +the array items. + +If a unicode string is specified as first argument, the buffer is +made one item larger than the length of the string so that the +last element in the array is a NUL termination character. An +integer can be passed as second argument which allows to specify +the size of the array if the length of the string should not be +used. + +If the first parameter is a 8-bit string, it is converted into an +unicode string according to ctypes conversion rules. +\end{funcdesc} + +\begin{funcdesc}{DllCanUnloadNow}{} +Windows only: This function is a hook which allows to implement +inprocess COM servers with ctypes. It is called from the +DllCanUnloadNow function that the {\_}ctypes extension dll exports. +\end{funcdesc} + +\begin{funcdesc}{DllGetClassObject}{} +Windows only: This function is a hook which allows to implement +inprocess COM servers with ctypes. It is called from the +DllGetClassObject function that the \code{{\_}ctypes} extension dll exports. +\end{funcdesc} + +\begin{funcdesc}{FormatError}{\optional{code}} +Windows only: Returns a textual description of the error code. If +no error code is specified, the last error code is used by calling +the Windows api function GetLastError. +\end{funcdesc} + +\begin{funcdesc}{GetLastError}{} +Windows only: Returns the last error code set by Windows in the +calling thread. +\end{funcdesc} + +\begin{funcdesc}{memmove}{dst, src, count} +Same as the standard C memmove library function: copies \var{count} +bytes from \code{src} to \var{dst}. \var{dst} and \code{src} must be +integers or ctypes instances that can be converted to pointers. +\end{funcdesc} + +\begin{funcdesc}{memset}{dst, c, count} +Same as the standard C memset library function: fills the memory +block at address \var{dst} with \var{count} bytes of value +\var{c}. \var{dst} must be an integer specifying an address, or a +ctypes instance. +\end{funcdesc} + +\begin{funcdesc}{POINTER}{type} +This factory function creates and returns a new ctypes pointer +type. Pointer types are cached an reused internally, so calling +this function repeatedly is cheap. type must be a ctypes type. +\end{funcdesc} + +\begin{funcdesc}{pointer}{obj} +This function creates a new pointer instance, pointing to +\code{obj}. The returned object is of the type POINTER(type(obj)). + +Note: If you just want to pass a pointer to an object to a foreign +function call, you should use \code{byref(obj)} which is much faster. +\end{funcdesc} + +\begin{funcdesc}{resize}{obj, size} +This function resizes the internal memory buffer of obj, which +must be an instance of a ctypes type. It is not possible to make +the buffer smaller than the native size of the objects type, as +given by sizeof(type(obj)), but it is possible to enlarge the +buffer. +\end{funcdesc} + +\begin{funcdesc}{set_conversion_mode}{encoding, errors} +This function sets the rules that ctypes objects use when +converting between 8-bit strings and unicode strings. encoding +must be a string specifying an encoding, like \code{'utf-8'} or +\code{'mbcs'}, errors must be a string specifying the error handling +on encoding/decoding errors. Examples of possible values are +\code{"strict"}, \code{"replace"}, or \code{"ignore"}. + +set{\_}conversion{\_}mode returns a 2-tuple containing the previous +conversion rules. On windows, the initial conversion rules are +\code{('mbcs', 'ignore')}, on other systems \code{('ascii', 'strict')}. +\end{funcdesc} + +\begin{funcdesc}{sizeof}{obj_or_type} +Returns the size in bytes of a ctypes type or instance memory +buffer. Does the same as the C \code{sizeof()} function. +\end{funcdesc} + +\begin{funcdesc}{string_at}{address\optional{, size}} +This function returns the string starting at memory address +address. If size is specified, it is used as size, otherwise the +string is assumed to be zero-terminated. +\end{funcdesc} + +\begin{funcdesc}{WinError}{code=None, descr=None} +Windows only: this function is probably the worst-named thing in +ctypes. It creates an instance of WindowsError. If \var{code} is not +specified, \code{GetLastError} is called to determine the error +code. If \code{descr} is not spcified, \function{FormatError} is called to +get a textual description of the error. +\end{funcdesc} + +\begin{funcdesc}{wstring_at}{address} +This function returns the wide character string starting at memory +address \code{address} as unicode string. If \code{size} is specified, +it is used as the number of characters of the string, otherwise +the string is assumed to be zero-terminated. +\end{funcdesc} + + +\subsubsection{Data types\label{ctypes-data-types}} + +\begin{classdesc*}{_CData} +This non-public class is the common base class of all ctypes data +types. Among other things, all ctypes type instances contain a +memory block that hold C compatible data; the address of the +memory block is returned by the \code{addressof()} helper function. +Another instance variable is exposed as \member{{\_}objects}; this +contains other Python objects that need to be kept alive in case +the memory block contains pointers. +\end{classdesc*} + +Common methods of ctypes data types, these are all class methods (to +be exact, they are methods of the metaclass): + +\begin{methoddesc}{from_address}{address} +This method returns a ctypes type instance using the memory +specified by address. +\end{methoddesc} + +\begin{methoddesc}{from_param}{obj} +This method adapts obj to a ctypes type. +\end{methoddesc} + +\begin{methoddesc}{in_dll}{name, library} +This method returns a ctypes type instance exported by a shared +library. \var{name} is the name of the symbol that exports the data, +\code{library} is the loaded shared library. +\end{methoddesc} + +Common instance variables of ctypes data types: + +\begin{memberdesc}{_b_base_} +Sometimes ctypes data instances do not own the memory block they +contain, instead they share part of the memory block of a base +object. The \member{{\_}b{\_}base{\_}} readonly member is the root ctypes +object that owns the memory block. +\end{memberdesc} + +\begin{memberdesc}{_b_needsfree_} +This readonly variable is true when the ctypes data instance has +allocated the memory block itself, false otherwise. +\end{memberdesc} + +\begin{memberdesc}{_objects} +This member is either \code{None} or a dictionary containing Python +objects that need to be kept alive so that the memory block +contents is kept valid. This object is only exposed for +debugging; never modify the contents of this dictionary. +\end{memberdesc} + + +\subsubsection{Fundamental data types\label{ctypes-fundamental-data-types}} + +\begin{classdesc*}{_SimpleCData} +This non-public class is the base class of all fundamental ctypes +data types. It is mentioned here because it contains the common +attributes of the fundamental ctypes data types. \code{{\_}SimpleCData} +is a subclass of \code{{\_}CData}, so it inherits their methods and +attributes. +\end{classdesc*} + +Instances have a single attribute: + +\begin{memberdesc}{value} +This attribute contains the actual value of the instance. For +integer and pointer types, it is an integer, for character types, +it is a single character string, for character pointer types it +is a Python string or unicode string. + +When the \code{value} attribute is retrieved from a ctypes instance, +usually a new object is returned each time. \code{ctypes} does \emph{not} +implement original object return, always a new object is +constructed. The same is true for all other ctypes object +instances. +\end{memberdesc} + +Fundamental data types, when returned as foreign function call +results, or, for example, by retrieving structure field members or +array items, are transparently converted to native Python types. In +other words, if a foreign function has a \member{restype} of \class{c{\_}char{\_}p}, +you will always receive a Python string, \emph{not} a \class{c{\_}char{\_}p} +instance. + +Subclasses of fundamental data types do \emph{not} inherit this behaviour. +So, if a foreign functions \member{restype} is a subclass of \class{c{\_}void{\_}p}, +you will receive an instance of this subclass from the function call. +Of course, you can get the value of the pointer by accessing the +\code{value} attribute. + +These are the fundamental ctypes data types: + +\begin{classdesc*}{c_byte} +Represents the C signed char datatype, and interprets the value as +small integer. The constructor accepts an optional integer +initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_char} +Represents the C char datatype, and interprets the value as a single +character. The constructor accepts an optional string initializer, +the length of the string must be exactly one character. +\end{classdesc*} + +\begin{classdesc*}{c_char_p} +Represents the C char * datatype, which must be a pointer to a +zero-terminated string. The constructor accepts an integer +address, or a string. +\end{classdesc*} + +\begin{classdesc*}{c_double} +Represents the C double datatype. The constructor accepts an +optional float initializer. +\end{classdesc*} + +\begin{classdesc*}{c_float} +Represents the C double datatype. The constructor accepts an +optional float initializer. +\end{classdesc*} + +\begin{classdesc*}{c_int} +Represents the C signed int datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. On +platforms where \code{sizeof(int) == sizeof(long)} it is an alias to +\class{c{\_}long}. +\end{classdesc*} + +\begin{classdesc*}{c_int8} +Represents the C 8-bit \code{signed int} datatype. Usually an alias for +\class{c{\_}byte}. +\end{classdesc*} + +\begin{classdesc*}{c_int16} +Represents the C 16-bit signed int datatype. Usually an alias for +\class{c{\_}short}. +\end{classdesc*} + +\begin{classdesc*}{c_int32} +Represents the C 32-bit signed int datatype. Usually an alias for +\class{c{\_}int}. +\end{classdesc*} + +\begin{classdesc*}{c_int64} +Represents the C 64-bit \code{signed int} datatype. Usually an alias +for \class{c{\_}longlong}. +\end{classdesc*} + +\begin{classdesc*}{c_long} +Represents the C \code{signed long} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_longlong} +Represents the C \code{signed long long} datatype. The constructor accepts +an optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_short} +Represents the C \code{signed short} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_size_t} +Represents the C \code{size{\_}t} datatype. +\end{classdesc*} + +\begin{classdesc*}{c_ubyte} +Represents the C \code{unsigned char} datatype, it interprets the +value as small integer. The constructor accepts an optional +integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_uint} +Represents the C \code{unsigned int} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. On +platforms where \code{sizeof(int) == sizeof(long)} it is an alias for +\class{c{\_}ulong}. +\end{classdesc*} + +\begin{classdesc*}{c_uint8} +Represents the C 8-bit unsigned int datatype. Usually an alias for +\class{c{\_}ubyte}. +\end{classdesc*} + +\begin{classdesc*}{c_uint16} +Represents the C 16-bit unsigned int datatype. Usually an alias for +\class{c{\_}ushort}. +\end{classdesc*} + +\begin{classdesc*}{c_uint32} +Represents the C 32-bit unsigned int datatype. Usually an alias for +\class{c{\_}uint}. +\end{classdesc*} + +\begin{classdesc*}{c_uint64} +Represents the C 64-bit unsigned int datatype. Usually an alias for +\class{c{\_}ulonglong}. +\end{classdesc*} + +\begin{classdesc*}{c_ulong} +Represents the C \code{unsigned long} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_ulonglong} +Represents the C \code{unsigned long long} datatype. The constructor +accepts an optional integer initializer; no overflow checking is +done. +\end{classdesc*} + +\begin{classdesc*}{c_ushort} +Represents the C \code{unsigned short} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_void_p} +Represents the C \code{void *} type. The value is represented as +integer. The constructor accepts an optional integer initializer. +\end{classdesc*} + +\begin{classdesc*}{c_wchar} +Represents the C \code{wchar{\_}t} datatype, and interprets the value as a +single character unicode string. The constructor accepts an +optional string initializer, the length of the string must be +exactly one character. +\end{classdesc*} + +\begin{classdesc*}{c_wchar_p} +Represents the C \code{wchar{\_}t *} datatype, which must be a pointer to +a zero-terminated wide character string. The constructor accepts +an integer address, or a string. +\end{classdesc*} + +\begin{classdesc*}{HRESULT} +Windows only: Represents a \class{HRESULT} value, which contains success +or error information for a function or method call. +\end{classdesc*} + + +\subsubsection{Structured data types\label{ctypes-structured-data-types}} + +\begin{classdesc}{Union}{*args, **kw} +Abstract base class for unions in native byte order. +\end{classdesc} + +\begin{classdesc}{BigEndianStructure}{*args, **kw} +Abstract base class for structures in \emph{big endian} byte order. +\end{classdesc} + +\begin{classdesc}{LittleEndianStructure}{*args, **kw} +Abstract base class for structures in \emph{little endian} byte order. +\end{classdesc} + +Structures with non-native byte order cannot contain pointer type +fields, or any other data types containing pointer type fields. + +\begin{classdesc}{Structure}{*args, **kw} +Abstract base class for structures in \emph{native} byte order. +\end{classdesc} + +Concrete structure and union types must be created by subclassing one +of these types, and at least define a \member{{\_}fields{\_}} class variable. +\code{ctypes} will create descriptors which allow reading and writing the +fields by direct attribute accesses. These are the + +\begin{memberdesc}{_fields_} +A sequence defining the structure fields. The items must be +2-tuples or 3-tuples. The first item is the name of the field, +the second item specifies the type of the field; it can be any +ctypes data type. + +For integer type fields, a third optional item can be given. It +must be a small positive integer defining the bit width of the +field. + +Field names must be unique within one structure or union. This is +not checked, only one field can be accessed when names are +repeated. + +It is possible to define the \member{{\_}fields{\_}} class variable \emph{after} +the class statement that defines the Structure subclass, this +allows to create data types that directly or indirectly reference +themselves: +\begin{verbatim} +class List(Structure): + pass +List._fields_ = [("pnext", POINTER(List)), + ... + ] +\end{verbatim} + +The \member{{\_}fields{\_}} class variable must, however, be defined before +the type is first used (an instance is created, \code{sizeof()} is +called on it, and so on). Later assignments to the \member{{\_}fields{\_}} +class variable will raise an AttributeError. + +Structure and union subclass constructors accept both positional +and named arguments. Positional arguments are used to initialize +the fields in the same order as they appear in the \member{{\_}fields{\_}} +definition, named arguments are used to initialize the fields with +the corresponding name. + +It is possible to defined sub-subclasses of structure types, they +inherit the fields of the base class plus the \member{{\_}fields{\_}} defined +in the sub-subclass, if any. +\end{memberdesc} + +\begin{memberdesc}{_pack_} +An optional small integer that allows to override the alignment of +structure fields in the instance. \member{{\_}pack{\_}} must already be +defined when \member{{\_}fields{\_}} is assigned, otherwise it will have no +effect. +\end{memberdesc} + +\begin{memberdesc}{_anonymous_} +An optional sequence that lists the names of unnamed (anonymous) +fields. \code{{\_}anonymous{\_}} must be already defined when \member{{\_}fields{\_}} +is assigned, otherwise it will have no effect. + +The fields listed in this variable must be structure or union type +fields. \code{ctypes} will create descriptors in the structure type +that allows to access the nested fields directly, without the need +to create the structure or union field. + +Here is an example type (Windows): +\begin{verbatim} +class _U(Union): + _fields_ = [("lptdesc", POINTER(TYPEDESC)), + ("lpadesc", POINTER(ARRAYDESC)), + ("hreftype", HREFTYPE)] + +class TYPEDESC(Structure): + _fields_ = [("u", _U), + ("vt", VARTYPE)] + + _anonymous_ = ("u",) +\end{verbatim} + +The \code{TYPEDESC} structure describes a COM data type, the \code{vt} +field specifies which one of the union fields is valid. Since the +\code{u} field is defined as anonymous field, it is now possible to +access the members directly off the TYPEDESC instance. +\code{td.lptdesc} and \code{td.u.lptdesc} are equivalent, but the former +is faster since it does not need to create a temporary \code{{\_}U} +instance: +\begin{verbatim} +td = TYPEDESC() +td.vt = VT_PTR +td.lptdesc = POINTER(some_type) +td.u.lptdesc = POINTER(some_type) +\end{verbatim} +\end{memberdesc} + +It is possible to defined sub-subclasses of structures, they inherit +the fields of the base class. If the subclass definition has a +separate``{\_}fields{\_}`` variable, the fields specified in this are +appended to the fields of the base class. + + +\subsubsection{Arrays and pointers\label{ctypes-arrays-pointers}} + +XXX + Deleted: /python/trunk/Doc/lib/libctypesref.tex ============================================================================== --- /python/trunk/Doc/lib/libctypesref.tex Sat Jun 10 22:29:34 2006 +++ (empty file) @@ -1,457 +0,0 @@ -\subsection{ctypes reference\label{ctypes-reference}} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% functions -\subsubsection{ctypes functions} - -\begin{funcdesc}{addressof}{obj} -Returns the address of the memory buffer as integer. \var{obj} must -be an instance of a ctypes type. -\end{funcdesc} - -\begin{funcdesc}{alignment}{obj_or_type} -Returns the alignment requirements of a ctypes type. -\var{obj_or_type} must be a ctypes type or an instance. -\end{funcdesc} - -\begin{excclassdesc}{ArgumentError}{} -This exception is raised when a foreign function call cannot convert -one of the passed arguments. -\end{excclassdesc} - -\begin{funcdesc}{byref}{obj} -Returns a light-weight pointer to \var{obj}, which must be an instance -of a ctypes type. The returned object can only be used as a foreign -function call parameter. It behaves similar to \code{pointer(obj)}, -but the construction is a lot faster. -\end{funcdesc} - -\begin{funcdesc}{cast}{obj, type} -This function is similar to the cast operator in C. It returns a new -instance of \var{type} which points to the same memory block as -\code{obj}. \code{type} must be a pointer type, and \code{obj} - must be an object that can be interpreted as a pointer. -\end{funcdesc} - -% XXX separate section for CFUNCTYPE, WINFUNCTYPE, PYFUNCTYPE? - -\begin{funcdesc}{CFUNCTYPE}{restype, *argtypes} -This is a factory function that returns a function prototype. The -function prototype describes a function that has a result type of -\code{restype}, and accepts arguments as specified by \code{argtypes}. -The function prototype can be used to construct several kinds of -functions, depending on how the prototype is called. - -The prototypes returned by \code{CFUNCTYPE} or \code{PYFUNCTYPE} -create functions that use the standard C calling convention, -prototypes returned from \code{WINFUNCTYPE} (on Windows) use the -\code{__stdcall} calling convention. - -Functions created by calling the \code{CFUNCTYPE} and -\code{WINFUNCTYPE} prototypes release the Python GIL -before entering the foreign function, and acquire it back after -leaving the function code. - -% XXX differences between CFUNCTYPE / WINFUNCTYPE / PYFUNCTYPE - -\end{funcdesc} - -\begin{funcdesc}{create_string_buffer}{init_or_size\optional{, size}} -This function creates a mutable character buffer. The returned object -is a ctypes array of \code{c_char}. - -\var{init_or_size} must be an integer which specifies the size of the -array, or a string which will be used to initialize the array items. - -If a string is specified as first argument, the buffer is made one -item larger than the length of the string so that the last element in -the array is a NUL termination character. An integer can be passed as -second argument which allows to specify the size of the array if the -length of the string should not be used. - -If the first parameter is a unicode string, it is converted into an -8-bit string according to ctypes conversion rules. -\end{funcdesc} - -\begin{funcdesc}{create_unicode_buffer}{init_or_size\optional{, size}} -This function creates a mutable unicode character buffer. The -returned object is a ctypes array of \code{c_wchar}. - -\var{init_or_size} must be an integer which specifies the size of the -array, or a unicode string which will be used to initialize the array -items. - -If a unicode string is specified as first argument, the buffer is made -one item larger than the length of the string so that the last element -in the array is a NUL termination character. An integer can be passed -as second argument which allows to specify the size of the array if -the length of the string should not be used. - -If the first parameter is a 8-bit string, it is converted into an -unicode string according to ctypes conversion rules. -\end{funcdesc} - -\begin{funcdesc}{DllCanUnloadNow}{} -Windows only: This function is a hook which allows to implement -inprocess COM servers with ctypes. It is called from the -\code{DllCanUnloadNow} function that the \code{_ctypes} -extension dll exports. -\end{funcdesc} - -\begin{funcdesc}{DllGetClassObject}{} -Windows only: This function is a hook which allows to implement -inprocess COM servers with ctypes. It is called from the -\code{DllGetClassObject} function that the \code{_ctypes} -extension dll exports. -\end{funcdesc} - -\begin{funcdesc}{FormatError}{\optional{code}} -Windows only: Returns a textual description of the error code. If no -error code is specified, the last error code is used by calling the -Windows api function \code{GetLastError}. -\end{funcdesc} - -\begin{funcdesc}{GetLastError}{} -Windows only: Returns the last error code set by Windows in the -calling thread. -\end{funcdesc} - -\begin{funcdesc}{memmove}{dst, src, count} -Same as the standard C \code{memmove} library function: copies -\var{count} bytes from \code{src} to \code{dst}. \code{dst} and -\code{src} must be integers or ctypes instances that can be converted to pointers. -\end{funcdesc} - -\begin{funcdesc}{memset}{dst, c, count} -Same as the standard C \code{memset} library function: fills the -memory clock at address \code{dst} with \var{count} bytes of value -\var{c}. \var{dst} must be an integer specifying an address, or a ctypes instance. -\end{funcdesc} - -\begin{funcdesc}{POINTER}{type} -This factory function creates and returns a new ctypes pointer type. -Pointer types are cached an reused internally, so calling this -function repeatedly is cheap. \var{type} must be a ctypes type. -\end{funcdesc} - -\begin{funcdesc}{pointer}{obj} -This function creates a new pointer instance, pointing to \var{obj}. -The returned object is of the type \code{POINTER(type(obj))}. - -Note: If you just want to pass a pointer to an object to a foreign -function call, you should use \code{byref(obj)} which is much faster. -\end{funcdesc} - -\begin{funcdesc}{PYFUNCTYPE}{restype, *argtypes} -\end{funcdesc} - -\begin{funcdesc}{pythonapi}{} -\end{funcdesc} - -\begin{funcdesc}{resize}{obj, size} -This function resizes the internal memory buffer of \var{obj}, which -must be an instance of a ctypes type. It is not possible to make the -buffer smaller than the native size of the objects type, as given by -\code{sizeof(type(obj))}, but it is possible to enlarge the buffer. -\end{funcdesc} - -\begin{funcdesc}{set_conversion_mode}{encoding, errors} -This function sets the rules that ctypes objects use when converting -between 8-bit strings and unicode strings. \var{encoding} must be a -string specifying an encoding, like 'utf-8' or 'mbcs', \var{errors} -must be a string specifying the error handling on encoding/decoding -errors. Examples of possible values are ``strict'', ``replace'', or -``ignore''. - -\code{set_conversion_mode} returns a 2-tuple containing the previous -conversion rules. On windows, the initial conversion rules are -\code{('mbcs', 'ignore')}, on other systems \code{('ascii', 'strict')}. -\end{funcdesc} - -\begin{funcdesc}{sizeof}{obj_or_type} -Returns the size in bytes of a ctypes type or instance memory buffer. -Does the same as the C sizeof() function. -\end{funcdesc} - -\begin{funcdesc}{string_at}{address\optional{size}} -This function returns the string starting at memory address -\var{address}. If \var{size} is specified, it is used as size, -otherwise the string is assumed to be zero-terminated. -\end{funcdesc} - -\begin{funcdesc}{WinError}{code=None, descr=None} -Windows only: this function is probably the worst-named thing in -ctypes. It creates an instance of \code{WindowsError}. If \var{code} -is not specified, \code{GetLastError} is called to determine the error -code. If \var{descr} is not spcified, \var{FormatError} is called to -get a textual description of the error. -\end{funcdesc} - -\begin{funcdesc}{WINFUNCTYPE}{restype, *argtypes} -\end{funcdesc} - -\begin{funcdesc}{wstring_at}{address} -This function returns the wide character string starting at memory -address \var{address} as unicode string. If \var{size} is specified, -it is used as size, otherwise the string is assumed to be -zero-terminated. -\end{funcdesc} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% data types -\subsubsection{data types} - -ctypes defines a lot of C compatible datatypes, and also allows to -define your own types. Among other things, a ctypes type instance -holds a memory block that contains C compatible data. - -\begin{classdesc}{_ctypes._CData}{} -This non-public class is the base class of all ctypes data types. It -is mentioned here because it contains the common methods of the ctypes -data types. -\end{classdesc} - -Common methods of ctypes data types, these are all class methods (to -be exact, they are methods of the metaclass): - -\begin{methoddesc}{from_address}{address} -This method returns a ctypes type instance using the memory specified -by \code{address}. -\end{methoddesc} - -\begin{methoddesc}{from_param}{obj} -This method adapts \code{obj} to a ctypes type. -\end{methoddesc} - -\begin{methoddesc}{in_dll}{name, library} -This method returns a ctypes type instance exported by a shared -library. \var{name} is the name of the symbol that exports the data, -\var{library} is the loaded shared library. -\end{methoddesc} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% simple data types -\subsubsection{simple data types} - -\begin{classdesc}{_ctypes._SimpleCData}{} -This non-public class is the base class of all ctypes data types. It -is mentioned here because it contains the common attributes of the -ctypes data types. -\end{classdesc} - -\begin{memberdesc}{value} -This attribute contains the actual value of the instance. For integer -types, it is an integer. -\end{memberdesc} - -Here are the simple ctypes data types: - -\begin{classdesc}{c_byte}{\optional{value}} -Represents a C \code{signed char} datatype, and interprets the value -as small integer. The constructor accepts an optional integer -initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_char}{\optional{value}} -Represents a C \code{char} datatype, and interprets the value as a -single character. The constructor accepts an optional string -initializer, the length of the string must be exactly one character. -\end{classdesc} - -\begin{classdesc}{c_char_p}{\optional{value}} -Represents a C \code{char *} datatype, which must be a pointer to a -zero-terminated string. The constructor accepts an integer address, -or a string. -% XXX Explain the difference to POINTER(c_char) -\end{classdesc} - -\begin{classdesc}{c_double}{\optional{value}} -Represents a C \code{double} datatype. The constructor accepts an -optional float initializer. -\end{classdesc} - -\begin{classdesc}{c_float}{\optional{value}} -Represents a C \code{double} datatype. The constructor accepts an -optional float initializer. -\end{classdesc} - -\begin{classdesc}{c_int}{\optional{value}} -Represents a C \code{signed int} datatype. The constructor accepts an -optional integer initializer; no overflow checking is done. On -platforms where \code{sizeof(int) == sizeof(long)} \var{c_int} is an -alias to \var{c_long}. -\end{classdesc} - -\begin{classdesc}{c_int16}{\optional{value}} -Represents a C 16-bit \code{signed int} datatype. Usually an alias -for \var{c_short}. -\end{classdesc} - -\begin{classdesc}{c_int32}{\optional{value}} -Represents a C 32-bit \code{signed int} datatype. Usually an alias -for \code{c_int}. -\end{classdesc} - -\begin{classdesc}{c_int64}{\optional{value}} -Represents a C 64-bit \code{signed int} datatype. Usually an alias -for \code{c_longlong}. -\end{classdesc} - -\begin{classdesc}{c_int8}{\optional{value}} -Represents a C 8-bit \code{signed int} datatype. Usually an alias for \code{c_byte}. -\end{classdesc} - -\begin{classdesc}{c_long}{\optional{value}} -Represents a C \code{signed long} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_longlong}{\optional{value}} -Represents a C \code{signed long long} datatype. The constructor -accepts an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_short}{\optional{value}} -Represents a C \code{signed short} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_size_t}{\optional{value}} -Represents a C \code{size_t} datatype. -\end{classdesc} - -\begin{classdesc}{c_ubyte}{\optional{value}} -Represents a C \code{unsigned char} datatype, and interprets the value -as small integer. The constructor accepts an optional integer -initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_uint}{\optional{value}} -Represents a C \code{unsigned int} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. On -platforms where \code{sizeof(int) == sizeof(long)} \var{c_int} is an -alias to \var{c_long}. -\end{classdesc} - -\begin{classdesc}{c_uint16}{\optional{value}} -Represents a C 16-bit \code{unsigned int} datatype. Usually an alias -for \code{c_ushort}. -\end{classdesc} - -\begin{classdesc}{c_uint32}{\optional{value}} -Represents a C 32-bit \code{unsigned int} datatype. Usually an alias -for \code{c_uint}. -\end{classdesc} - -\begin{classdesc}{c_uint64}{\optional{value}} -Represents a C 64-bit \code{unsigned int} datatype. Usually an alias -for \code{c_ulonglong}. -\end{classdesc} - -\begin{classdesc}{c_uint8}{\optional{value}} -Represents a C 8-bit \code{unsigned int} datatype. Usually an alias -for \code{c_ubyte}. -\end{classdesc} - -\begin{classdesc}{c_ulong}{\optional{value}} -Represents a C \code{unsigned long} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_ulonglong}{\optional{value}} -Represents a C \code{unsigned long long} datatype. The constructor -accepts an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_ushort}{\optional{value}} -Represents a C \code{unsigned short} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_void_p}{\optional{value}} -Represents a C \code{void *} type. The value is represented as -integer. The constructor accepts an optional integer initializer. -\end{classdesc} - -\begin{classdesc}{c_wchar}{\optional{value}} -Represents a C \code{wchar_t} datatype, and interprets the value as a -single character unicode string. The constructor accepts an optional -string initializer, the length of the string must be exactly one -character. -\end{classdesc} - -\begin{classdesc}{c_wchar_p}{\optional{value}} -Represents a C \code{wchar_t *} datatype, which must be a pointer to a -zero-terminated wide character string. The constructor accepts an -integer address, or a string. -% XXX Explain the difference to POINTER(c_wchar) -\end{classdesc} - -\begin{classdesc}{HRESULT}{} -Windows only: Represents a \code{HRESULT} value, which contains -success or error information for a function or method call. -\end{classdesc} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% structured data types -\subsubsection{structured data types} - -\begin{classdesc}{BigEndianStructure}{} -\end{classdesc} - -\begin{classdesc}{LittleEndianStructure}{} -\end{classdesc} - -\begin{classdesc}{Structure}{} -Base class for Structure data types. - -\end{classdesc} - -\begin{classdesc}{Union}{} -\end{classdesc} - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% libraries -\subsubsection{libraries} - -\begin{classdesc}{CDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{cdll} -\end{datadesc} - -\begin{classdesc}{LibraryLoader}{dlltype} - -\begin{memberdesc}{LoadLibrary}{name, mode=RTLD_LOCAL, handle=None} -\end{memberdesc} - -\end{classdesc} - -\begin{classdesc}{OleDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{oledll} -\end{datadesc} - -\begin{classdesc}{py_object}{} -\end{classdesc} - -\begin{classdesc}{PyDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{pydll}{} -\end{datadesc} - -\begin{datadesc}{RTLD_GLOBAL} -\end{datadesc} - -\begin{datadesc}{RTLD_LOCAL} -\end{datadesc} - -\begin{classdesc}{WinDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{windll} -\end{datadesc} - From python-checkins at python.org Sat Jun 10 22:32:16 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 22:32:16 +0200 (CEST) Subject: [Python-checkins] r46832 - in external/ctypes: ctypes/test/test_anon.py ctypes/test/test_objects.py ctypes/test/test_varsize_struct.py docs/anatomy.txt docs/manual/ctypes.txt docs/manual/filter.py docs/test-anatomy.py source/libffi/src/darwin/ffitarget.h source/libffi/src/x86/darwin.S source/libffi/src/x86/ffi_darwin.c Message-ID: <20060610203216.33A021E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 22:32:12 2006 New Revision: 46832 Modified: external/ctypes/ctypes/test/test_anon.py (contents, props changed) external/ctypes/ctypes/test/test_objects.py (contents, props changed) external/ctypes/ctypes/test/test_varsize_struct.py (contents, props changed) external/ctypes/docs/anatomy.txt (contents, props changed) external/ctypes/docs/manual/ctypes.txt (contents, props changed) external/ctypes/docs/manual/filter.py (contents, props changed) external/ctypes/docs/test-anatomy.py (contents, props changed) external/ctypes/source/libffi/src/darwin/ffitarget.h (contents, props changed) external/ctypes/source/libffi/src/x86/darwin.S (contents, props changed) external/ctypes/source/libffi/src/x86/ffi_darwin.c (contents, props changed) Log: Set svn:eol-style property Modified: external/ctypes/ctypes/test/test_anon.py ============================================================================== --- external/ctypes/ctypes/test/test_anon.py (original) +++ external/ctypes/ctypes/test/test_anon.py Sat Jun 10 22:32:12 2006 @@ -1,60 +1,60 @@ -import unittest -from ctypes import * - -class AnonTest(unittest.TestCase): - - def test_anon(self): - class ANON(Union): - _fields_ = [("a", c_int), - ("b", c_int)] - - class Y(Structure): - _fields_ = [("x", c_int), - ("_", ANON), - ("y", c_int)] - _anonymous_ = ["_"] - - self.failUnlessEqual(Y.a.offset, sizeof(c_int)) - self.failUnlessEqual(Y.b.offset, sizeof(c_int)) - - self.failUnlessEqual(ANON.a.offset, 0) - self.failUnlessEqual(ANON.b.offset, 0) - - def test_anon_nonseq(self): - # TypeError: _anonymous_ must be a sequence - self.failUnlessRaises(TypeError, - lambda: type(Structure)("Name", - (Structure,), - {"_fields_": [], "_anonymous_": 42})) - - def test_anon_nonmember(self): - # AttributeError: type object 'Name' has no attribute 'x' - self.failUnlessRaises(AttributeError, - lambda: type(Structure)("Name", - (Structure,), - {"_fields_": [], - "_anonymous_": ["x"]})) - - def test_nested(self): - class ANON_S(Structure): - _fields_ = [("a", c_int)] - - class ANON_U(Union): - _fields_ = [("_", ANON_S), - ("b", c_int)] - _anonymous_ = ["_"] - - class Y(Structure): - _fields_ = [("x", c_int), - ("_", ANON_U), - ("y", c_int)] - _anonymous_ = ["_"] - - self.failUnlessEqual(Y.x.offset, 0) - self.failUnlessEqual(Y.a.offset, sizeof(c_int)) - self.failUnlessEqual(Y.b.offset, sizeof(c_int)) - self.failUnlessEqual(Y._.offset, sizeof(c_int)) - self.failUnlessEqual(Y.y.offset, sizeof(c_int) * 2) - -if __name__ == "__main__": - unittest.main() +import unittest +from ctypes import * + +class AnonTest(unittest.TestCase): + + def test_anon(self): + class ANON(Union): + _fields_ = [("a", c_int), + ("b", c_int)] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + + self.failUnlessEqual(ANON.a.offset, 0) + self.failUnlessEqual(ANON.b.offset, 0) + + def test_anon_nonseq(self): + # TypeError: _anonymous_ must be a sequence + self.failUnlessRaises(TypeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], "_anonymous_": 42})) + + def test_anon_nonmember(self): + # AttributeError: type object 'Name' has no attribute 'x' + self.failUnlessRaises(AttributeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], + "_anonymous_": ["x"]})) + + def test_nested(self): + class ANON_S(Structure): + _fields_ = [("a", c_int)] + + class ANON_U(Union): + _fields_ = [("_", ANON_S), + ("b", c_int)] + _anonymous_ = ["_"] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON_U), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.x.offset, 0) + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + self.failUnlessEqual(Y._.offset, sizeof(c_int)) + self.failUnlessEqual(Y.y.offset, sizeof(c_int) * 2) + +if __name__ == "__main__": + unittest.main() Modified: external/ctypes/ctypes/test/test_objects.py ============================================================================== --- external/ctypes/ctypes/test/test_objects.py (original) +++ external/ctypes/ctypes/test/test_objects.py Sat Jun 10 22:32:12 2006 @@ -1,66 +1,66 @@ -r''' -This tests the '_objects' attribute of ctypes instances. '_objects' -holds references to objects that must be kept alive as long as the -ctypes instance, to make sure that the memory buffer is valid. - -WARNING: The '_objects' attribute is exposed ONLY for debugging ctypes itself, -it MUST NEVER BE MODIFIED! - -'_objects' is initialized to a dictionary on first use, before that it -is None. - -Here is an array of string pointers: - ->>> from ctypes import * ->>> array = (c_char_p * 5)() ->>> print array._objects -None ->>> - -The memory block stores pointers to strings, and the strings itself -assigned from Python must be kept. - ->>> array[4] = 'foo bar' ->>> array._objects -{'4': 'foo bar'} ->>> array[4] -'foo bar' ->>> - -It gets more complicated when the ctypes instance itself is contained -in a 'base' object. - ->>> class X(Structure): -... _fields_ = [("x", c_int), ("y", c_int), ("array", c_char_p * 5)] -... ->>> x = X() ->>> print x._objects -None ->>> - -The'array' attribute of the 'x' object shares part of the memory buffer -of 'x' ('_b_base_' is either None, or the root object owning the memory block): - ->>> print x.array._b_base_ # doctest: +ELLIPSIS - ->>> - ->>> x.array[0] = 'spam spam spam' ->>> x._objects -{'0:2': 'spam spam spam'} ->>> x.array._b_base_._objects -{'0:2': 'spam spam spam'} ->>> - -''' - -import unittest, doctest - -import ctypes.test.test_objects - -class TestCase(unittest.TestCase): - def test(self): - doctest.testmod(ctypes.test.test_objects) - -if __name__ == '__main__': - doctest.testmod(ctypes.test.test_objects) +r''' +This tests the '_objects' attribute of ctypes instances. '_objects' +holds references to objects that must be kept alive as long as the +ctypes instance, to make sure that the memory buffer is valid. + +WARNING: The '_objects' attribute is exposed ONLY for debugging ctypes itself, +it MUST NEVER BE MODIFIED! + +'_objects' is initialized to a dictionary on first use, before that it +is None. + +Here is an array of string pointers: + +>>> from ctypes import * +>>> array = (c_char_p * 5)() +>>> print array._objects +None +>>> + +The memory block stores pointers to strings, and the strings itself +assigned from Python must be kept. + +>>> array[4] = 'foo bar' +>>> array._objects +{'4': 'foo bar'} +>>> array[4] +'foo bar' +>>> + +It gets more complicated when the ctypes instance itself is contained +in a 'base' object. + +>>> class X(Structure): +... _fields_ = [("x", c_int), ("y", c_int), ("array", c_char_p * 5)] +... +>>> x = X() +>>> print x._objects +None +>>> + +The'array' attribute of the 'x' object shares part of the memory buffer +of 'x' ('_b_base_' is either None, or the root object owning the memory block): + +>>> print x.array._b_base_ # doctest: +ELLIPSIS + +>>> + +>>> x.array[0] = 'spam spam spam' +>>> x._objects +{'0:2': 'spam spam spam'} +>>> x.array._b_base_._objects +{'0:2': 'spam spam spam'} +>>> + +''' + +import unittest, doctest + +import ctypes.test.test_objects + +class TestCase(unittest.TestCase): + def test(self): + doctest.testmod(ctypes.test.test_objects) + +if __name__ == '__main__': + doctest.testmod(ctypes.test.test_objects) Modified: external/ctypes/ctypes/test/test_varsize_struct.py ============================================================================== --- external/ctypes/ctypes/test/test_varsize_struct.py (original) +++ external/ctypes/ctypes/test/test_varsize_struct.py Sat Jun 10 22:32:12 2006 @@ -1,115 +1,115 @@ -from ctypes import * -import unittest - -class VarSizeTest(unittest.TestCase): - def test_resize(self): - class X(Structure): - _fields_ = [("item", c_int), - ("array", c_int * 1)] - - self.failUnlessEqual(sizeof(X), sizeof(c_int) * 2) - x = X() - x.item = 42 - x.array[0] = 100 - self.failUnlessEqual(sizeof(x), sizeof(c_int) * 2) - - # make room for one additional item - new_size = sizeof(X) + sizeof(c_int) * 1 - resize(x, new_size) - self.failUnlessEqual(sizeof(x), new_size) - self.failUnlessEqual((x.item, x.array[0]), (42, 100)) - - # make room for 10 additional items - new_size = sizeof(X) + sizeof(c_int) * 9 - resize(x, new_size) - self.failUnlessEqual(sizeof(x), new_size) - self.failUnlessEqual((x.item, x.array[0]), (42, 100)) - - # make room for one additional item - new_size = sizeof(X) + sizeof(c_int) * 1 - resize(x, new_size) - self.failUnlessEqual(sizeof(x), new_size) - self.failUnlessEqual((x.item, x.array[0]), (42, 100)) - - def test_array_invalid_length(self): - # cannot create arrays with non-positive size - self.failUnlessRaises(ValueError, lambda: c_int * -1) - self.failUnlessRaises(ValueError, lambda: c_int * -3) - - def test_zerosized_array(self): - array = (c_int * 0)() - # accessing elements of zero-sized arrays raise IndexError - self.failUnlessRaises(IndexError, array.__setitem__, 0, None) - self.failUnlessRaises(IndexError, array.__getitem__, 0) - self.failUnlessRaises(IndexError, array.__setitem__, 1, None) - self.failUnlessRaises(IndexError, array.__getitem__, 1) - self.failUnlessRaises(IndexError, array.__setitem__, -1, None) - self.failUnlessRaises(IndexError, array.__getitem__, -1) - - def test_varsized_array(self): - array = (c_int * 20)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) - - # no range checking is done on arrays with size == 1 - varsize_array = (c_int * 1).from_address(addressof(array)) - - # __getitem__ - self.failUnlessEqual(varsize_array[0], 20) - self.failUnlessEqual(varsize_array[1], 21) - self.failUnlessEqual(varsize_array[2], 22) - self.failUnlessEqual(varsize_array[3], 23) - self.failUnlessEqual(varsize_array[4], 24) - self.failUnlessEqual(varsize_array[5], 25) - self.failUnlessEqual(varsize_array[6], 26) - self.failUnlessEqual(varsize_array[7], 27) - self.failUnlessEqual(varsize_array[8], 28) - self.failUnlessEqual(varsize_array[9], 29) - - # still, normal sequence of length one behaviour: - self.failUnlessEqual(varsize_array[-1], 20) - self.failUnlessRaises(IndexError, lambda: varsize_array[-2]) - # except for this one, which will raise MemoryError - self.failUnlessRaises(MemoryError, lambda: varsize_array[:]) - - # __setitem__ - varsize_array[0] = 100 - varsize_array[1] = 101 - varsize_array[2] = 102 - varsize_array[3] = 103 - varsize_array[4] = 104 - varsize_array[5] = 105 - varsize_array[6] = 106 - varsize_array[7] = 107 - varsize_array[8] = 108 - varsize_array[9] = 109 - - for i in range(10): - self.failUnlessEqual(varsize_array[i], i + 100) - self.failUnlessEqual(array[i], i + 100) - - # __getslice__ - self.failUnlessEqual(varsize_array[0:10], range(100, 110)) - self.failUnlessEqual(varsize_array[1:9], range(101, 109)) - self.failUnlessEqual(varsize_array[1:-1], []) - - # __setslice__ - varsize_array[0:10] = range(1000, 1010) - self.failUnlessEqual(varsize_array[0:10], range(1000, 1010)) - - varsize_array[1:9] = range(1001, 1009) - self.failUnlessEqual(varsize_array[1:9], range(1001, 1009)) - - def test_vararray_is_sane(self): - array = (c_int * 15)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) - - varsize_array = (c_int * 1).from_address(addressof(array)) - varsize_array[:] = [1, 2, 3, 4, 5] - - self.failUnlessEqual(array[:], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) - self.failUnlessEqual(varsize_array[0:10], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29]) - - array[:5] = [10, 11, 12, 13, 14] - self.failUnlessEqual(array[:], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) - self.failUnlessEqual(varsize_array[0:10], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29]) - -if __name__ == "__main__": - unittest.main() +from ctypes import * +import unittest + +class VarSizeTest(unittest.TestCase): + def test_resize(self): + class X(Structure): + _fields_ = [("item", c_int), + ("array", c_int * 1)] + + self.failUnlessEqual(sizeof(X), sizeof(c_int) * 2) + x = X() + x.item = 42 + x.array[0] = 100 + self.failUnlessEqual(sizeof(x), sizeof(c_int) * 2) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for 10 additional items + new_size = sizeof(X) + sizeof(c_int) * 9 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + def test_array_invalid_length(self): + # cannot create arrays with non-positive size + self.failUnlessRaises(ValueError, lambda: c_int * -1) + self.failUnlessRaises(ValueError, lambda: c_int * -3) + + def test_zerosized_array(self): + array = (c_int * 0)() + # accessing elements of zero-sized arrays raise IndexError + self.failUnlessRaises(IndexError, array.__setitem__, 0, None) + self.failUnlessRaises(IndexError, array.__getitem__, 0) + self.failUnlessRaises(IndexError, array.__setitem__, 1, None) + self.failUnlessRaises(IndexError, array.__getitem__, 1) + self.failUnlessRaises(IndexError, array.__setitem__, -1, None) + self.failUnlessRaises(IndexError, array.__getitem__, -1) + + def test_varsized_array(self): + array = (c_int * 20)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) + + # no range checking is done on arrays with size == 1 + varsize_array = (c_int * 1).from_address(addressof(array)) + + # __getitem__ + self.failUnlessEqual(varsize_array[0], 20) + self.failUnlessEqual(varsize_array[1], 21) + self.failUnlessEqual(varsize_array[2], 22) + self.failUnlessEqual(varsize_array[3], 23) + self.failUnlessEqual(varsize_array[4], 24) + self.failUnlessEqual(varsize_array[5], 25) + self.failUnlessEqual(varsize_array[6], 26) + self.failUnlessEqual(varsize_array[7], 27) + self.failUnlessEqual(varsize_array[8], 28) + self.failUnlessEqual(varsize_array[9], 29) + + # still, normal sequence of length one behaviour: + self.failUnlessEqual(varsize_array[-1], 20) + self.failUnlessRaises(IndexError, lambda: varsize_array[-2]) + # except for this one, which will raise MemoryError + self.failUnlessRaises(MemoryError, lambda: varsize_array[:]) + + # __setitem__ + varsize_array[0] = 100 + varsize_array[1] = 101 + varsize_array[2] = 102 + varsize_array[3] = 103 + varsize_array[4] = 104 + varsize_array[5] = 105 + varsize_array[6] = 106 + varsize_array[7] = 107 + varsize_array[8] = 108 + varsize_array[9] = 109 + + for i in range(10): + self.failUnlessEqual(varsize_array[i], i + 100) + self.failUnlessEqual(array[i], i + 100) + + # __getslice__ + self.failUnlessEqual(varsize_array[0:10], range(100, 110)) + self.failUnlessEqual(varsize_array[1:9], range(101, 109)) + self.failUnlessEqual(varsize_array[1:-1], []) + + # __setslice__ + varsize_array[0:10] = range(1000, 1010) + self.failUnlessEqual(varsize_array[0:10], range(1000, 1010)) + + varsize_array[1:9] = range(1001, 1009) + self.failUnlessEqual(varsize_array[1:9], range(1001, 1009)) + + def test_vararray_is_sane(self): + array = (c_int * 15)(20, 21, 22, 23, 24, 25, 26, 27, 28, 29) + + varsize_array = (c_int * 1).from_address(addressof(array)) + varsize_array[:] = [1, 2, 3, 4, 5] + + self.failUnlessEqual(array[:], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) + self.failUnlessEqual(varsize_array[0:10], [1, 2, 3, 4, 5, 25, 26, 27, 28, 29]) + + array[:5] = [10, 11, 12, 13, 14] + self.failUnlessEqual(array[:], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) + self.failUnlessEqual(varsize_array[0:10], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29]) + +if __name__ == "__main__": + unittest.main() Modified: external/ctypes/docs/anatomy.txt ============================================================================== --- external/ctypes/docs/anatomy.txt (original) +++ external/ctypes/docs/anatomy.txt Sat Jun 10 22:32:12 2006 @@ -1,233 +1,233 @@ -ctypes objects anatomy -====================== - -This article describes some of the internals of ctypes types and -ctypes instances. - -The object structure --------------------- - -Definition of the structure that each ctypes instance has:: - - struct tagCDataObject { - PyObject_HEAD /* Standard Python object fields */ - char *b_ptr; /* pointer to memory block */ - int b_needsfree; /* does the object own its memory block? */ - CDataObject *b_base; /* pointer to base object or NULL */ - Py_ssize_t b_size; /* size of memory block in bytes */ - Py_ssize_t b_length; /* number of fields of this object */ - Py_ssize_t b_index; /* index of this object into the base - objects b_object list */ - PyObject *b_objects; /* references we need to keep */ - union value b_value; /* a small default buffer */ - }; - -Here is what the fields contain: - -- ``PyObject_HEAD`` - - Standard for every Python object. - -- ``b_ptr`` - - Pointer to the memory block that the object uses. - -- ``b_needsfree`` - - A flag which is nonzero if the object does owns the memory block, - nonzero otherwise. - -- ``b_base`` - - If the object does not own the memory block, this is the 'root' object - that owns the memory block. Otherwise it is NULL. - -- ``b_size`` - - Size of memory block in bytes. - -- ``b_length`` - - ... - -- ``b_index`` - - If b_base is not NULL, this is the index of this object in the - 'root' object. - -- ``b_objects`` - - This is a pointer containing a Python object which contains other - objects that have to be kept alive as long as this object lives. - Either ``None``, or a dictionary. - -- ``b_value`` - - A default memory block which can be used by small objects to avoid - a PyMem_Malloc calls. - -The memory block ----------------- - -Basically, a ctypes object instance has a memory block containing C -compatible data, plus the ``b_objects`` Python object pointer. The -latter is used to keep referenced objects alive that are referenced by -pointers in the memory block. - -Consider an array of 4 string pointers, defined by this C code:: - - char *string_array[4]; - -The ctypes definition is:: - - >>> from ctypes import * - >>> string_array = (c_char_p * 4)() - >>> - -The memory block of ``string_array`` is initialized to all zeros, -and retrieving items returns 4 ``None`` objects:: - - >>> string_array[:] - [None, None, None, None] - >>> - -We can assign Python strings to the items, and get them back:: - - >>> string_array[0] = "foo bar"; string_array[1] = "spam, spam, spam" - >>> string_array[0:2] - ['foo bar', 'spam, spam, spam'] - >>> - -The memory block contains the *pointers* to the strings (ctypes -objects implement the buffer interface, so we can use the following -snippet to examine the buffer contents):: - - >>> print repr(str(buffer(string_array))) # doctest: +SKIP - '\x94\xb7\xbd\x00\xfc\x80\xbf\x00\x00\x00\x00\x00\x00\x00\x00\x00' - >>> - -The strings themselves must also be kept in memory, otherwise the -pointers in the memory block would access invalid or freed memory. -The are stored in a dictionary in the``b_objects`` field of the -``tagCDataObject`` structure defined above. This field is exposed as -attribute named ``_objects`` to Python, but you should be aware that -this object is only exposed for debugging (and understanding) of -ctypes objects, you should *never* modify it:: - - >>> string_array._objects - {'1': 'spam, spam, spam', '0': 'foo bar'} - >>> - -The ``b_objects`` dictionary stores these needed objects as values, at -a key that is calculated from the item index or field index. - -Not all ctypes objects have to keep references, simple types like -integers or floats for example can happily live in the memory block, -without any other needs. In this case ``b_objects`` contains -``Py_None``, and the dictionary is never created:: - - >>> int_array = (c_int * 3)() - >>> int_array[:] = 1, 2, 3 - >>> print int_array._objects - None - >>> - -XXX remove this? - - >>> s_a = (c_char_p * 5210)() - >>> s_a[0] = "zero" - >>> s_a[255] = "ali baba" - >>> s_a[257] = "forty robbers" - >>> s_a[1] = "spam and eggs" - >>> s_a._objects - {'1': 'spam and eggs', '0': 'zero', '101': 'forty robbers', 'ff': 'ali baba'} - >>> - -The important thing to keep in mind is that it must be possible to -'reconstruct' the whole ctypes object from the memory block and the -``b_objects`` pointer. - - -What happens if a ctypes object is stored in another ctypes object -field? Define a structure which has a field storing a string array:: - - >>> class Container(Structure): - ... _fields_ = [("count", c_uint), - ... ("strings", c_char_p * 4)] - ... - >>> - >>> container = Container() - >>> container.strings = string_array - >>> container._objects - {'1': {'1': 'spam, spam, spam', '0': 'foo bar'}} - >>> - -As we can see, the ``string_array`` ``b_objects`` dictionary has been -inserted into the ``container`` ``b_objects`` dictionary at index 1, -because 1 is the index of the ``strings`` field. The contents of the -``string_array`` memory block has been copied into the ``container`` -memory block as well. - -Again, things get slighlty more complicated when we use a structure -containing a pointer field instead of an array field. In this case, -the memory block of the pointer object must be kept alive in addition -to the pointer ``b_object`` dictionary. Here is what ctypes does:: - - >>> class PointerContainer(Structure): - ... _fields_ = [("count", c_uint), - ... ("string_p", POINTER(c_char_p))] - ... - >>> pc = PointerContainer() - >>> pc.string_p = string_array - >>> pc._objects - {'1': ({'1': 'spam, spam, spam', '0': 'foo bar'}, )} - >>> - -So, assigning an array instance to a pointer type field stores a tuple -containing the arrays ``b_objects`` dictionary plus the array object -itself. Of course, in this case the memory block of ``string_array`` -is NOT copied into the ``PointerContainer`` memory block, only the -address is copied. - -What happens if we retrieve the string pointer from the -PointerContainer instance? ctypes doesn't do OOR (original object -return), ctypes returns a new object on each attribute access:: - - >>> pc.string_p is pc.string_p - False - >>> - - >>> print pc.string_p._objects - None - >>> print pc.string_p._b_base_ - - >>> - - >>> other = PointerContainer() - >>> other.string_p = pc.string_p - >>> print other._objects - {'1': {'1': ({'1': 'spam, spam, spam', '0': 'foo bar'}, )}} - >>> - - >>> x = Container() - >>> x.strings = (c_char_p * 4)() - >>> print x._objects - {'1': {}} - >>> x.strings[2] = "python ctypes" - >>> print x._objects - {'1': {}, '2:1': 'python ctypes'} - >>> - - >>> class X(Structure): - ... _fields_ = [("a", c_int), - ... ("b", c_int), - ... ("c", c_int), - ... ("d", c_int), - ... ("container", Container)] - ... - >>> x = X() - >>> x.container.strings = (c_char_p * 4)() - >>> x.container.strings[1] = "foobar.org" - >>> x._objects - {'1:4': {}, '1:1:4': 'foobar.org'} +ctypes objects anatomy +====================== + +This article describes some of the internals of ctypes types and +ctypes instances. + +The object structure +-------------------- + +Definition of the structure that each ctypes instance has:: + + struct tagCDataObject { + PyObject_HEAD /* Standard Python object fields */ + char *b_ptr; /* pointer to memory block */ + int b_needsfree; /* does the object own its memory block? */ + CDataObject *b_base; /* pointer to base object or NULL */ + Py_ssize_t b_size; /* size of memory block in bytes */ + Py_ssize_t b_length; /* number of fields of this object */ + Py_ssize_t b_index; /* index of this object into the base + objects b_object list */ + PyObject *b_objects; /* references we need to keep */ + union value b_value; /* a small default buffer */ + }; + +Here is what the fields contain: + +- ``PyObject_HEAD`` + + Standard for every Python object. + +- ``b_ptr`` + + Pointer to the memory block that the object uses. + +- ``b_needsfree`` + + A flag which is nonzero if the object does owns the memory block, + nonzero otherwise. + +- ``b_base`` + + If the object does not own the memory block, this is the 'root' object + that owns the memory block. Otherwise it is NULL. + +- ``b_size`` + + Size of memory block in bytes. + +- ``b_length`` + + ... + +- ``b_index`` + + If b_base is not NULL, this is the index of this object in the + 'root' object. + +- ``b_objects`` + + This is a pointer containing a Python object which contains other + objects that have to be kept alive as long as this object lives. + Either ``None``, or a dictionary. + +- ``b_value`` + + A default memory block which can be used by small objects to avoid + a PyMem_Malloc calls. + +The memory block +---------------- + +Basically, a ctypes object instance has a memory block containing C +compatible data, plus the ``b_objects`` Python object pointer. The +latter is used to keep referenced objects alive that are referenced by +pointers in the memory block. + +Consider an array of 4 string pointers, defined by this C code:: + + char *string_array[4]; + +The ctypes definition is:: + + >>> from ctypes import * + >>> string_array = (c_char_p * 4)() + >>> + +The memory block of ``string_array`` is initialized to all zeros, +and retrieving items returns 4 ``None`` objects:: + + >>> string_array[:] + [None, None, None, None] + >>> + +We can assign Python strings to the items, and get them back:: + + >>> string_array[0] = "foo bar"; string_array[1] = "spam, spam, spam" + >>> string_array[0:2] + ['foo bar', 'spam, spam, spam'] + >>> + +The memory block contains the *pointers* to the strings (ctypes +objects implement the buffer interface, so we can use the following +snippet to examine the buffer contents):: + + >>> print repr(str(buffer(string_array))) # doctest: +SKIP + '\x94\xb7\xbd\x00\xfc\x80\xbf\x00\x00\x00\x00\x00\x00\x00\x00\x00' + >>> + +The strings themselves must also be kept in memory, otherwise the +pointers in the memory block would access invalid or freed memory. +The are stored in a dictionary in the``b_objects`` field of the +``tagCDataObject`` structure defined above. This field is exposed as +attribute named ``_objects`` to Python, but you should be aware that +this object is only exposed for debugging (and understanding) of +ctypes objects, you should *never* modify it:: + + >>> string_array._objects + {'1': 'spam, spam, spam', '0': 'foo bar'} + >>> + +The ``b_objects`` dictionary stores these needed objects as values, at +a key that is calculated from the item index or field index. + +Not all ctypes objects have to keep references, simple types like +integers or floats for example can happily live in the memory block, +without any other needs. In this case ``b_objects`` contains +``Py_None``, and the dictionary is never created:: + + >>> int_array = (c_int * 3)() + >>> int_array[:] = 1, 2, 3 + >>> print int_array._objects + None + >>> + +XXX remove this? + + >>> s_a = (c_char_p * 5210)() + >>> s_a[0] = "zero" + >>> s_a[255] = "ali baba" + >>> s_a[257] = "forty robbers" + >>> s_a[1] = "spam and eggs" + >>> s_a._objects + {'1': 'spam and eggs', '0': 'zero', '101': 'forty robbers', 'ff': 'ali baba'} + >>> + +The important thing to keep in mind is that it must be possible to +'reconstruct' the whole ctypes object from the memory block and the +``b_objects`` pointer. + + +What happens if a ctypes object is stored in another ctypes object +field? Define a structure which has a field storing a string array:: + + >>> class Container(Structure): + ... _fields_ = [("count", c_uint), + ... ("strings", c_char_p * 4)] + ... + >>> + >>> container = Container() + >>> container.strings = string_array + >>> container._objects + {'1': {'1': 'spam, spam, spam', '0': 'foo bar'}} + >>> + +As we can see, the ``string_array`` ``b_objects`` dictionary has been +inserted into the ``container`` ``b_objects`` dictionary at index 1, +because 1 is the index of the ``strings`` field. The contents of the +``string_array`` memory block has been copied into the ``container`` +memory block as well. + +Again, things get slighlty more complicated when we use a structure +containing a pointer field instead of an array field. In this case, +the memory block of the pointer object must be kept alive in addition +to the pointer ``b_object`` dictionary. Here is what ctypes does:: + + >>> class PointerContainer(Structure): + ... _fields_ = [("count", c_uint), + ... ("string_p", POINTER(c_char_p))] + ... + >>> pc = PointerContainer() + >>> pc.string_p = string_array + >>> pc._objects + {'1': ({'1': 'spam, spam, spam', '0': 'foo bar'}, )} + >>> + +So, assigning an array instance to a pointer type field stores a tuple +containing the arrays ``b_objects`` dictionary plus the array object +itself. Of course, in this case the memory block of ``string_array`` +is NOT copied into the ``PointerContainer`` memory block, only the +address is copied. + +What happens if we retrieve the string pointer from the +PointerContainer instance? ctypes doesn't do OOR (original object +return), ctypes returns a new object on each attribute access:: + + >>> pc.string_p is pc.string_p + False + >>> + + >>> print pc.string_p._objects + None + >>> print pc.string_p._b_base_ + + >>> + + >>> other = PointerContainer() + >>> other.string_p = pc.string_p + >>> print other._objects + {'1': {'1': ({'1': 'spam, spam, spam', '0': 'foo bar'}, )}} + >>> + + >>> x = Container() + >>> x.strings = (c_char_p * 4)() + >>> print x._objects + {'1': {}} + >>> x.strings[2] = "python ctypes" + >>> print x._objects + {'1': {}, '2:1': 'python ctypes'} + >>> + + >>> class X(Structure): + ... _fields_ = [("a", c_int), + ... ("b", c_int), + ... ("c", c_int), + ... ("d", c_int), + ... ("container", Container)] + ... + >>> x = X() + >>> x.container.strings = (c_char_p * 4)() + >>> x.container.strings[1] = "foobar.org" + >>> x._objects + {'1:4': {}, '1:1:4': 'foobar.org'} >>> \ No newline at end of file Modified: external/ctypes/docs/manual/ctypes.txt ============================================================================== --- external/ctypes/docs/manual/ctypes.txt (original) +++ external/ctypes/docs/manual/ctypes.txt Sat Jun 10 22:32:12 2006 @@ -1,11 +1,11 @@ -:Module: ctypes -:Summary: A foreign function library for Python. -:Module Type: standard -:Author: Thomas Heller -:Synopsis: A foreign function library for Python. -:Version Added: 2.5 - -``ctypes`` is a foreign function library for Python. - -.. include:: tutorial.txt -.. include:: reference.txt +:Module: ctypes +:Summary: A foreign function library for Python. +:Module Type: standard +:Author: Thomas Heller +:Synopsis: A foreign function library for Python. +:Version Added: 2.5 + +``ctypes`` is a foreign function library for Python. + +.. include:: tutorial.txt +.. include:: reference.txt Modified: external/ctypes/docs/manual/filter.py ============================================================================== --- external/ctypes/docs/manual/filter.py (original) +++ external/ctypes/docs/manual/filter.py Sat Jun 10 22:32:12 2006 @@ -1,18 +1,18 @@ -import sys - -TOKENS = """: funcdesc -: excdesc -: vardesc -: classdesc -: methoddesc -: memberdesc -: classdesc*""".splitlines() - -for line in open(sys.argv[1], "r"): - for token in TOKENS: - line = line.rstrip() - if line.endswith(token): - print line[:-len(token)] - break - else: - print line +import sys + +TOKENS = """: funcdesc +: excdesc +: vardesc +: classdesc +: methoddesc +: memberdesc +: classdesc*""".splitlines() + +for line in open(sys.argv[1], "r"): + for token in TOKENS: + line = line.rstrip() + if line.endswith(token): + print line[:-len(token)] + break + else: + print line Modified: external/ctypes/docs/test-anatomy.py ============================================================================== --- external/ctypes/docs/test-anatomy.py (original) +++ external/ctypes/docs/test-anatomy.py Sat Jun 10 22:32:12 2006 @@ -1,45 +1,45 @@ -#!/usr/bin/env python -import sys -import doctest - -# handle platform specific issues -WINDOWS = doctest.register_optionflag("WINDOWS") -LINUX = doctest.register_optionflag("LINUX") -SKIP = doctest.register_optionflag("SKIP") - -# handle size specific issues -import ctypes -c_int_name = ctypes.c_int.__name__ - -base = doctest.DocTestRunner -class MyDocTestRunner(base): - def run(self, test, compileflags=None, out=None, clear_globs=True): - examples = test.examples[:] - for ex in test.examples: - if WINDOWS in ex.options and sys.platform != "win32": - examples.remove(ex) - elif LINUX in ex.options and not sys.platform.startswith("linux"): - examples.remove(ex) - elif SKIP in ex.options: - examples.remove(ex) -## elif "printf(" in ex.source: -## # handle that doctest doesn't catch printf's output -## lines = ex.want.splitlines() -## try: -## int(lines[-1]) -## except ValueError: -## pass -## else: -## ex.want = "\n".join(lines[1:]) + "\n" -## else: - ex.want = ex.want.replace("c_long", c_int_name) - test.examples = examples - return base.run(self, test, compileflags, out, clear_globs) -doctest.DocTestRunner = MyDocTestRunner - -if __name__ == "__main__": - # Python 2.5a2 formats exceptions differently than before, so we - # add IGNORE_EXCEPTION_DETAIL. I do not know if this will be - # fixed or not. - doctest.testfile("anatomy.txt", - optionflags=doctest.ELLIPSIS | doctest.IGNORE_EXCEPTION_DETAIL) +#!/usr/bin/env python +import sys +import doctest + +# handle platform specific issues +WINDOWS = doctest.register_optionflag("WINDOWS") +LINUX = doctest.register_optionflag("LINUX") +SKIP = doctest.register_optionflag("SKIP") + +# handle size specific issues +import ctypes +c_int_name = ctypes.c_int.__name__ + +base = doctest.DocTestRunner +class MyDocTestRunner(base): + def run(self, test, compileflags=None, out=None, clear_globs=True): + examples = test.examples[:] + for ex in test.examples: + if WINDOWS in ex.options and sys.platform != "win32": + examples.remove(ex) + elif LINUX in ex.options and not sys.platform.startswith("linux"): + examples.remove(ex) + elif SKIP in ex.options: + examples.remove(ex) +## elif "printf(" in ex.source: +## # handle that doctest doesn't catch printf's output +## lines = ex.want.splitlines() +## try: +## int(lines[-1]) +## except ValueError: +## pass +## else: +## ex.want = "\n".join(lines[1:]) + "\n" +## else: + ex.want = ex.want.replace("c_long", c_int_name) + test.examples = examples + return base.run(self, test, compileflags, out, clear_globs) +doctest.DocTestRunner = MyDocTestRunner + +if __name__ == "__main__": + # Python 2.5a2 formats exceptions differently than before, so we + # add IGNORE_EXCEPTION_DETAIL. I do not know if this will be + # fixed or not. + doctest.testfile("anatomy.txt", + optionflags=doctest.ELLIPSIS | doctest.IGNORE_EXCEPTION_DETAIL) Modified: external/ctypes/source/libffi/src/darwin/ffitarget.h ============================================================================== --- external/ctypes/source/libffi/src/darwin/ffitarget.h (original) +++ external/ctypes/source/libffi/src/darwin/ffitarget.h Sat Jun 10 22:32:12 2006 @@ -1,25 +1,25 @@ -/* - * This file is for MacOSX only. Dispatch to the right architecture include - * file based on the current archictecture (instead of relying on a symlink - * created by configure). This makes is possible to build a univeral binary - * of ctypes in one go. - */ -#if defined(__i386__) - -#ifndef X86_DARWIN -#define X86_DARWIN -#endif -#undef POWERPC_DARWIN - -#include "../src/x86/ffitarget.h" - -#elif defined(__ppc__) - -#ifndef POWERPC_DARWIN -#define POWERPC_DARWIN -#endif -#undef X86_DARWIN - -#include "../src/powerpc/ffitarget.h" - -#endif +/* + * This file is for MacOSX only. Dispatch to the right architecture include + * file based on the current archictecture (instead of relying on a symlink + * created by configure). This makes is possible to build a univeral binary + * of ctypes in one go. + */ +#if defined(__i386__) + +#ifndef X86_DARWIN +#define X86_DARWIN +#endif +#undef POWERPC_DARWIN + +#include "../src/x86/ffitarget.h" + +#elif defined(__ppc__) + +#ifndef POWERPC_DARWIN +#define POWERPC_DARWIN +#endif +#undef X86_DARWIN + +#include "../src/powerpc/ffitarget.h" + +#endif Modified: external/ctypes/source/libffi/src/x86/darwin.S ============================================================================== --- external/ctypes/source/libffi/src/x86/darwin.S (original) +++ external/ctypes/source/libffi/src/x86/darwin.S Sat Jun 10 22:32:12 2006 @@ -1,390 +1,390 @@ -#ifdef __i386__ -/* ----------------------------------------------------------------------- - darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. - - X86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -/* - * This file is based on sysv.S and then hacked up by Ronald who hasn't done - * assembly programming in 8 years. - */ - -#ifndef __x86_64__ - -#define LIBFFI_ASM -#include -#include - -.text - -.globl _ffi_prep_args - -.align 4 -.globl _ffi_call_SYSV - -_ffi_call_SYSV: -.LFB1: - pushl %ebp -.LCFI0: - movl %esp,%ebp -.LCFI1: - /* Make room for all of the new args. */ - movl 16(%ebp),%ecx - subl %ecx,%esp - - movl %esp,%eax - - /* Place all of the ffi_prep_args in position */ - pushl 12(%ebp) - pushl %eax - call *8(%ebp) - - /* Return stack to previous state and call the function */ - addl $8,%esp - - call *28(%ebp) - - /* Remove the space we pushed for the args */ - movl 16(%ebp),%ecx - addl %ecx,%esp - - /* Load %ecx with the return type code */ - movl 20(%ebp),%ecx - - /* If the return value pointer is NULL, assume no return value. */ - cmpl $0,24(%ebp) - jne retint - - /* Even if there is no space for the return value, we are - obliged to handle floating-point values. */ - cmpl $FFI_TYPE_FLOAT,%ecx - jne noretval - fstp %st(0) - - jmp epilogue - -retint: - cmpl $FFI_TYPE_INT,%ecx - jne retfloat - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - jmp epilogue - -retfloat: - cmpl $FFI_TYPE_FLOAT,%ecx - jne retdouble - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstps (%ecx) - jmp epilogue - -retdouble: - cmpl $FFI_TYPE_DOUBLE,%ecx - jne retlongdouble - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstpl (%ecx) - jmp epilogue - -retlongdouble: - cmpl $FFI_TYPE_LONGDOUBLE,%ecx - jne retint64 - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstpt (%ecx) - jmp epilogue - -retint64: - cmpl $FFI_TYPE_SINT64,%ecx - jne retstruct - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - movl %edx,4(%ecx) - -retstruct: - /* Nothing to do! */ - -noretval: -epilogue: - movl %ebp,%esp - popl %ebp - ret -.LFE1: -.ffi_call_SYSV_end: -#if 0 - .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV -#endif - -#if 0 - .section .eh_frame,EH_FRAME_FLAGS, at progbits -.Lframe1: - .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ -.LSCIE1: - .long 0x0 /* CIE Identifier Tag */ - .byte 0x1 /* CIE Version */ -#ifdef __PIC__ - .ascii "zR\0" /* CIE Augmentation */ -#else - .ascii "\0" /* CIE Augmentation */ -#endif - .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ - .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ - .byte 0x8 /* CIE RA Column */ -#ifdef __PIC__ - .byte 0x1 /* .uleb128 0x1; Augmentation size */ - .byte 0x1b /* FDE Encoding (pcrel sdata4) */ -#endif - .byte 0xc /* DW_CFA_def_cfa */ - .byte 0x4 /* .uleb128 0x4 */ - .byte 0x4 /* .uleb128 0x4 */ - .byte 0x88 /* DW_CFA_offset, column 0x8 */ - .byte 0x1 /* .uleb128 0x1 */ - .align 4 -.LECIE1: -.LSFDE1: - .long .LEFDE1-.LASFDE1 /* FDE Length */ -.LASFDE1: - .long .LASFDE1-.Lframe1 /* FDE CIE offset */ -#ifdef __PIC__ - .long .LFB1-. /* FDE initial location */ -#else - .long .LFB1 /* FDE initial location */ -#endif - .long .LFE1-.LFB1 /* FDE address range */ -#ifdef __PIC__ - .byte 0x0 /* .uleb128 0x0; Augmentation size */ -#endif - .byte 0x4 /* DW_CFA_advance_loc4 */ - .long .LCFI0-.LFB1 - .byte 0xe /* DW_CFA_def_cfa_offset */ - .byte 0x8 /* .uleb128 0x8 */ - .byte 0x85 /* DW_CFA_offset, column 0x5 */ - .byte 0x2 /* .uleb128 0x2 */ - .byte 0x4 /* DW_CFA_advance_loc4 */ - .long .LCFI1-.LCFI0 - .byte 0xd /* DW_CFA_def_cfa_register */ - .byte 0x5 /* .uleb128 0x5 */ - .align 4 -.LEFDE1: -#endif - -#endif /* ifndef __x86_64__ */ - -#endif /* defined __i386__ */ -#ifdef __i386__ -/* ----------------------------------------------------------------------- - darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. - - X86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -/* - * This file is based on sysv.S and then hacked up by Ronald who hasn't done - * assembly programming in 8 years. - */ - -#ifndef __x86_64__ - -#define LIBFFI_ASM -#include -#include - -.text - -.globl _ffi_prep_args - -.align 4 -.globl _ffi_call_SYSV - -_ffi_call_SYSV: -.LFB1: - pushl %ebp -.LCFI0: - movl %esp,%ebp -.LCFI1: - /* Make room for all of the new args. */ - movl 16(%ebp),%ecx - subl %ecx,%esp - - movl %esp,%eax - - /* Place all of the ffi_prep_args in position */ - pushl 12(%ebp) - pushl %eax - call *8(%ebp) - - /* Return stack to previous state and call the function */ - addl $8,%esp - - call *28(%ebp) - - /* Remove the space we pushed for the args */ - movl 16(%ebp),%ecx - addl %ecx,%esp - - /* Load %ecx with the return type code */ - movl 20(%ebp),%ecx - - /* If the return value pointer is NULL, assume no return value. */ - cmpl $0,24(%ebp) - jne retint - - /* Even if there is no space for the return value, we are - obliged to handle floating-point values. */ - cmpl $FFI_TYPE_FLOAT,%ecx - jne noretval - fstp %st(0) - - jmp epilogue - -retint: - cmpl $FFI_TYPE_INT,%ecx - jne retfloat - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - jmp epilogue - -retfloat: - cmpl $FFI_TYPE_FLOAT,%ecx - jne retdouble - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstps (%ecx) - jmp epilogue - -retdouble: - cmpl $FFI_TYPE_DOUBLE,%ecx - jne retlongdouble - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstpl (%ecx) - jmp epilogue - -retlongdouble: - cmpl $FFI_TYPE_LONGDOUBLE,%ecx - jne retint64 - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstpt (%ecx) - jmp epilogue - -retint64: - cmpl $FFI_TYPE_SINT64,%ecx - jne retstruct - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - movl %edx,4(%ecx) - -retstruct: - /* Nothing to do! */ - -noretval: -epilogue: - movl %ebp,%esp - popl %ebp - ret -.LFE1: -.ffi_call_SYSV_end: -#if 0 - .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV -#endif - -#if 0 - .section .eh_frame,EH_FRAME_FLAGS, at progbits -.Lframe1: - .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ -.LSCIE1: - .long 0x0 /* CIE Identifier Tag */ - .byte 0x1 /* CIE Version */ -#ifdef __PIC__ - .ascii "zR\0" /* CIE Augmentation */ -#else - .ascii "\0" /* CIE Augmentation */ -#endif - .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ - .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ - .byte 0x8 /* CIE RA Column */ -#ifdef __PIC__ - .byte 0x1 /* .uleb128 0x1; Augmentation size */ - .byte 0x1b /* FDE Encoding (pcrel sdata4) */ -#endif - .byte 0xc /* DW_CFA_def_cfa */ - .byte 0x4 /* .uleb128 0x4 */ - .byte 0x4 /* .uleb128 0x4 */ - .byte 0x88 /* DW_CFA_offset, column 0x8 */ - .byte 0x1 /* .uleb128 0x1 */ - .align 4 -.LECIE1: -.LSFDE1: - .long .LEFDE1-.LASFDE1 /* FDE Length */ -.LASFDE1: - .long .LASFDE1-.Lframe1 /* FDE CIE offset */ -#ifdef __PIC__ - .long .LFB1-. /* FDE initial location */ -#else - .long .LFB1 /* FDE initial location */ -#endif - .long .LFE1-.LFB1 /* FDE address range */ -#ifdef __PIC__ - .byte 0x0 /* .uleb128 0x0; Augmentation size */ -#endif - .byte 0x4 /* DW_CFA_advance_loc4 */ - .long .LCFI0-.LFB1 - .byte 0xe /* DW_CFA_def_cfa_offset */ - .byte 0x8 /* .uleb128 0x8 */ - .byte 0x85 /* DW_CFA_offset, column 0x5 */ - .byte 0x2 /* .uleb128 0x2 */ - .byte 0x4 /* DW_CFA_advance_loc4 */ - .long .LCFI1-.LCFI0 - .byte 0xd /* DW_CFA_def_cfa_register */ - .byte 0x5 /* .uleb128 0x5 */ - .align 4 -.LEFDE1: -#endif - -#endif /* ifndef __x86_64__ */ - -#endif /* defined __i386__ */ +#ifdef __i386__ +/* ----------------------------------------------------------------------- + darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. + + X86 Foreign Function Interface + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + ----------------------------------------------------------------------- */ + +/* + * This file is based on sysv.S and then hacked up by Ronald who hasn't done + * assembly programming in 8 years. + */ + +#ifndef __x86_64__ + +#define LIBFFI_ASM +#include +#include + +.text + +.globl _ffi_prep_args + +.align 4 +.globl _ffi_call_SYSV + +_ffi_call_SYSV: +.LFB1: + pushl %ebp +.LCFI0: + movl %esp,%ebp +.LCFI1: + /* Make room for all of the new args. */ + movl 16(%ebp),%ecx + subl %ecx,%esp + + movl %esp,%eax + + /* Place all of the ffi_prep_args in position */ + pushl 12(%ebp) + pushl %eax + call *8(%ebp) + + /* Return stack to previous state and call the function */ + addl $8,%esp + + call *28(%ebp) + + /* Remove the space we pushed for the args */ + movl 16(%ebp),%ecx + addl %ecx,%esp + + /* Load %ecx with the return type code */ + movl 20(%ebp),%ecx + + /* If the return value pointer is NULL, assume no return value. */ + cmpl $0,24(%ebp) + jne retint + + /* Even if there is no space for the return value, we are + obliged to handle floating-point values. */ + cmpl $FFI_TYPE_FLOAT,%ecx + jne noretval + fstp %st(0) + + jmp epilogue + +retint: + cmpl $FFI_TYPE_INT,%ecx + jne retfloat + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + jmp epilogue + +retfloat: + cmpl $FFI_TYPE_FLOAT,%ecx + jne retdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstps (%ecx) + jmp epilogue + +retdouble: + cmpl $FFI_TYPE_DOUBLE,%ecx + jne retlongdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpl (%ecx) + jmp epilogue + +retlongdouble: + cmpl $FFI_TYPE_LONGDOUBLE,%ecx + jne retint64 + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpt (%ecx) + jmp epilogue + +retint64: + cmpl $FFI_TYPE_SINT64,%ecx + jne retstruct + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + movl %edx,4(%ecx) + +retstruct: + /* Nothing to do! */ + +noretval: +epilogue: + movl %ebp,%esp + popl %ebp + ret +.LFE1: +.ffi_call_SYSV_end: +#if 0 + .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV +#endif + +#if 0 + .section .eh_frame,EH_FRAME_FLAGS, at progbits +.Lframe1: + .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ +.LSCIE1: + .long 0x0 /* CIE Identifier Tag */ + .byte 0x1 /* CIE Version */ +#ifdef __PIC__ + .ascii "zR\0" /* CIE Augmentation */ +#else + .ascii "\0" /* CIE Augmentation */ +#endif + .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ + .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ + .byte 0x8 /* CIE RA Column */ +#ifdef __PIC__ + .byte 0x1 /* .uleb128 0x1; Augmentation size */ + .byte 0x1b /* FDE Encoding (pcrel sdata4) */ +#endif + .byte 0xc /* DW_CFA_def_cfa */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x88 /* DW_CFA_offset, column 0x8 */ + .byte 0x1 /* .uleb128 0x1 */ + .align 4 +.LECIE1: +.LSFDE1: + .long .LEFDE1-.LASFDE1 /* FDE Length */ +.LASFDE1: + .long .LASFDE1-.Lframe1 /* FDE CIE offset */ +#ifdef __PIC__ + .long .LFB1-. /* FDE initial location */ +#else + .long .LFB1 /* FDE initial location */ +#endif + .long .LFE1-.LFB1 /* FDE address range */ +#ifdef __PIC__ + .byte 0x0 /* .uleb128 0x0; Augmentation size */ +#endif + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI0-.LFB1 + .byte 0xe /* DW_CFA_def_cfa_offset */ + .byte 0x8 /* .uleb128 0x8 */ + .byte 0x85 /* DW_CFA_offset, column 0x5 */ + .byte 0x2 /* .uleb128 0x2 */ + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI1-.LCFI0 + .byte 0xd /* DW_CFA_def_cfa_register */ + .byte 0x5 /* .uleb128 0x5 */ + .align 4 +.LEFDE1: +#endif + +#endif /* ifndef __x86_64__ */ + +#endif /* defined __i386__ */ +#ifdef __i386__ +/* ----------------------------------------------------------------------- + darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. + + X86 Foreign Function Interface + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + ----------------------------------------------------------------------- */ + +/* + * This file is based on sysv.S and then hacked up by Ronald who hasn't done + * assembly programming in 8 years. + */ + +#ifndef __x86_64__ + +#define LIBFFI_ASM +#include +#include + +.text + +.globl _ffi_prep_args + +.align 4 +.globl _ffi_call_SYSV + +_ffi_call_SYSV: +.LFB1: + pushl %ebp +.LCFI0: + movl %esp,%ebp +.LCFI1: + /* Make room for all of the new args. */ + movl 16(%ebp),%ecx + subl %ecx,%esp + + movl %esp,%eax + + /* Place all of the ffi_prep_args in position */ + pushl 12(%ebp) + pushl %eax + call *8(%ebp) + + /* Return stack to previous state and call the function */ + addl $8,%esp + + call *28(%ebp) + + /* Remove the space we pushed for the args */ + movl 16(%ebp),%ecx + addl %ecx,%esp + + /* Load %ecx with the return type code */ + movl 20(%ebp),%ecx + + /* If the return value pointer is NULL, assume no return value. */ + cmpl $0,24(%ebp) + jne retint + + /* Even if there is no space for the return value, we are + obliged to handle floating-point values. */ + cmpl $FFI_TYPE_FLOAT,%ecx + jne noretval + fstp %st(0) + + jmp epilogue + +retint: + cmpl $FFI_TYPE_INT,%ecx + jne retfloat + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + jmp epilogue + +retfloat: + cmpl $FFI_TYPE_FLOAT,%ecx + jne retdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstps (%ecx) + jmp epilogue + +retdouble: + cmpl $FFI_TYPE_DOUBLE,%ecx + jne retlongdouble + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpl (%ecx) + jmp epilogue + +retlongdouble: + cmpl $FFI_TYPE_LONGDOUBLE,%ecx + jne retint64 + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + fstpt (%ecx) + jmp epilogue + +retint64: + cmpl $FFI_TYPE_SINT64,%ecx + jne retstruct + /* Load %ecx with the pointer to storage for the return value */ + movl 24(%ebp),%ecx + movl %eax,0(%ecx) + movl %edx,4(%ecx) + +retstruct: + /* Nothing to do! */ + +noretval: +epilogue: + movl %ebp,%esp + popl %ebp + ret +.LFE1: +.ffi_call_SYSV_end: +#if 0 + .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV +#endif + +#if 0 + .section .eh_frame,EH_FRAME_FLAGS, at progbits +.Lframe1: + .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ +.LSCIE1: + .long 0x0 /* CIE Identifier Tag */ + .byte 0x1 /* CIE Version */ +#ifdef __PIC__ + .ascii "zR\0" /* CIE Augmentation */ +#else + .ascii "\0" /* CIE Augmentation */ +#endif + .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ + .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ + .byte 0x8 /* CIE RA Column */ +#ifdef __PIC__ + .byte 0x1 /* .uleb128 0x1; Augmentation size */ + .byte 0x1b /* FDE Encoding (pcrel sdata4) */ +#endif + .byte 0xc /* DW_CFA_def_cfa */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x4 /* .uleb128 0x4 */ + .byte 0x88 /* DW_CFA_offset, column 0x8 */ + .byte 0x1 /* .uleb128 0x1 */ + .align 4 +.LECIE1: +.LSFDE1: + .long .LEFDE1-.LASFDE1 /* FDE Length */ +.LASFDE1: + .long .LASFDE1-.Lframe1 /* FDE CIE offset */ +#ifdef __PIC__ + .long .LFB1-. /* FDE initial location */ +#else + .long .LFB1 /* FDE initial location */ +#endif + .long .LFE1-.LFB1 /* FDE address range */ +#ifdef __PIC__ + .byte 0x0 /* .uleb128 0x0; Augmentation size */ +#endif + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI0-.LFB1 + .byte 0xe /* DW_CFA_def_cfa_offset */ + .byte 0x8 /* .uleb128 0x8 */ + .byte 0x85 /* DW_CFA_offset, column 0x5 */ + .byte 0x2 /* .uleb128 0x2 */ + .byte 0x4 /* DW_CFA_advance_loc4 */ + .long .LCFI1-.LCFI0 + .byte 0xd /* DW_CFA_def_cfa_register */ + .byte 0x5 /* .uleb128 0x5 */ + .align 4 +.LEFDE1: +#endif + +#endif /* ifndef __x86_64__ */ + +#endif /* defined __i386__ */ Modified: external/ctypes/source/libffi/src/x86/ffi_darwin.c ============================================================================== --- external/ctypes/source/libffi/src/x86/ffi_darwin.c (original) +++ external/ctypes/source/libffi/src/x86/ffi_darwin.c Sat Jun 10 22:32:12 2006 @@ -1,610 +1,610 @@ -# ifdef __i386__ -/* ----------------------------------------------------------------------- - ffi.c - Copyright (c) 1996, 1998, 1999, 2001 Red Hat, Inc. - Copyright (c) 2002 Ranjit Mathew - Copyright (c) 2002 Bo Thorsen - Copyright (c) 2002 Roger Sayle - - x86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#ifndef __x86_64__ - -#include -#include - -#include - -/* ffi_prep_args is called by the assembly routine once stack space - has been allocated for the function's arguments */ - -/*@-exportheader@*/ -void ffi_prep_args(char *stack, extended_cif *ecif); - -static inline int retval_on_stack(ffi_type* tp) -{ - if (tp->type == FFI_TYPE_STRUCT) { - int sz = tp->size; - if (sz > 8) { - return 1; - } - switch (sz) { - case 1: case 2: case 4: case 8: return 0; - default: return 1; - } - } - return 0; -} - - -void ffi_prep_args(char *stack, extended_cif *ecif) -/*@=exportheader@*/ -{ - register unsigned int i; - register void **p_argv; - register char *argp; - register ffi_type **p_arg; - - argp = stack; - - if (retval_on_stack(ecif->cif->rtype)) { - *(void **) argp = ecif->rvalue; - argp += 4; - } - - - p_argv = ecif->avalue; - - for (i = ecif->cif->nargs, p_arg = ecif->cif->arg_types; - i != 0; - i--, p_arg++) - { - size_t z; - - /* Align if necessary */ - if ((sizeof(int) - 1) & (unsigned) argp) - argp = (char *) ALIGN(argp, sizeof(int)); - - z = (*p_arg)->size; - if (z < sizeof(int)) - { - z = sizeof(int); - switch ((*p_arg)->type) - { - case FFI_TYPE_SINT8: - *(signed int *) argp = (signed int)*(SINT8 *)(* p_argv); - break; - - case FFI_TYPE_UINT8: - *(unsigned int *) argp = (unsigned int)*(UINT8 *)(* p_argv); - break; - - case FFI_TYPE_SINT16: - *(signed int *) argp = (signed int)*(SINT16 *)(* p_argv); - break; - - case FFI_TYPE_UINT16: - *(unsigned int *) argp = (unsigned int)*(UINT16 *)(* p_argv); - break; - - case FFI_TYPE_SINT32: - *(signed int *) argp = (signed int)*(SINT32 *)(* p_argv); - break; - - case FFI_TYPE_UINT32: - *(unsigned int *) argp = (unsigned int)*(UINT32 *)(* p_argv); - break; - - case FFI_TYPE_STRUCT: - *(unsigned int *) argp = (unsigned int)*(UINT32 *)(* p_argv); - break; - - default: - FFI_ASSERT(0); - } - } - else - { - memcpy(argp, *p_argv, z); - } - p_argv++; - argp += z; - } - - return; -} - -/* Perform machine dependent cif processing */ -ffi_status ffi_prep_cif_machdep(ffi_cif *cif) -{ - /* Set the return type flag */ - switch (cif->rtype->type) - { - case FFI_TYPE_VOID: -#if !defined(X86_WIN32) - case FFI_TYPE_STRUCT: -#endif - case FFI_TYPE_SINT64: - case FFI_TYPE_FLOAT: - case FFI_TYPE_DOUBLE: - case FFI_TYPE_LONGDOUBLE: - cif->flags = (unsigned) cif->rtype->type; - break; - - case FFI_TYPE_UINT64: - cif->flags = FFI_TYPE_SINT64; - break; - -#if defined X86_WIN32 - - case FFI_TYPE_STRUCT: - if (cif->rtype->size == 1) - { - cif->flags = FFI_TYPE_SINT8; /* same as char size */ - } - else if (cif->rtype->size == 2) - { - cif->flags = FFI_TYPE_SINT16; /* same as short size */ - } - else if (cif->rtype->size == 4) - { - cif->flags = FFI_TYPE_INT; /* same as int type */ - } - else if (cif->rtype->size == 8) - { - cif->flags = FFI_TYPE_SINT64; /* same as int64 type */ - } - else - { - cif->flags = FFI_TYPE_STRUCT; - } - break; -#endif - - default: - cif->flags = FFI_TYPE_INT; - break; - } - - /* Darwin: The stack needs to be aligned to a multiple of 16 bytes */ -#if 0 - cif->bytes = (cif->bytes + 15) & ~0xF; -#endif - - return FFI_OK; -} - -/*@-declundef@*/ -/*@-exportheader@*/ -extern void ffi_call_SYSV(void (*)(char *, extended_cif *), - /*@out@*/ extended_cif *, - unsigned, unsigned, - /*@out@*/ unsigned *, - void (*fn)(void)); -/*@=declundef@*/ -/*@=exportheader@*/ - -#ifdef X86_WIN32 -/*@-declundef@*/ -/*@-exportheader@*/ -extern void ffi_call_STDCALL(void (*)(char *, extended_cif *), - /*@out@*/ extended_cif *, - unsigned, unsigned, - /*@out@*/ unsigned *, - void (*fn)(void)); -/*@=declundef@*/ -/*@=exportheader@*/ -#endif /* X86_WIN32 */ - -void ffi_call(/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ void **avalue) -{ - extended_cif ecif; - int flags; - - ecif.cif = cif; - ecif.avalue = avalue; - - /* If the return value is a struct and we don't have a return */ - /* value address then we need to make one */ - - if ((rvalue == NULL) && retval_on_stack(cif->rtype)) - { - /*@-sysunrecog@*/ - ecif.rvalue = alloca(cif->rtype->size); - /*@=sysunrecog@*/ - } - else - ecif.rvalue = rvalue; - - flags = cif->flags; - if (flags == FFI_TYPE_STRUCT) { - if (cif->rtype->size == 8) { - flags = FFI_TYPE_SINT64; - } else if (cif->rtype->size == 4) { - flags = FFI_TYPE_INT; - } else if (cif->rtype->size == 2) { - flags = FFI_TYPE_INT; - } else if (cif->rtype->size == 1) { - flags = FFI_TYPE_INT; - } - } - - - switch (cif->abi) - { - case FFI_SYSV: - /*@-usedef@*/ - /* To avoid changing the assembly code make sure the size of the argument - * block is a multiple of 16. Then add 8 to compensate for local variables - * in ffi_call_SYSV. - */ - ffi_call_SYSV(ffi_prep_args, &ecif, ALIGN(cif->bytes, 16) +8, - flags, ecif.rvalue, fn); - /*@=usedef@*/ - break; -#ifdef X86_WIN32 - case FFI_STDCALL: - /*@-usedef@*/ - ffi_call_STDCALL(ffi_prep_args, &ecif, cif->bytes, - cif->flags, ecif.rvalue, fn); - /*@=usedef@*/ - break; -#endif /* X86_WIN32 */ - default: - FFI_ASSERT(0); - break; - } -} - - -/** private members **/ - -static void ffi_prep_incoming_args_SYSV (char *stack, void **ret, - void** args, ffi_cif* cif); -static void ffi_closure_SYSV (ffi_closure *) - __attribute__ ((regparm(1))); -#if !FFI_NO_RAW_API -static void ffi_closure_raw_SYSV (ffi_raw_closure *) - __attribute__ ((regparm(1))); -#endif - -/* This function is jumped to by the trampoline */ - -static void -ffi_closure_SYSV (closure) - ffi_closure *closure; -{ - // this is our return value storage - long double res; - - // our various things... - ffi_cif *cif; - void **arg_area; - unsigned short rtype; - void *resp = (void*)&res; - void *args = __builtin_dwarf_cfa (); - - cif = closure->cif; - arg_area = (void**) alloca (cif->nargs * sizeof (void*)); - - /* this call will initialize ARG_AREA, such that each - * element in that array points to the corresponding - * value on the stack; and if the function returns - * a structure, it will re-set RESP to point to the - * structure return address. */ - - ffi_prep_incoming_args_SYSV(args, (void**)&resp, arg_area, cif); - - (closure->fun) (cif, resp, arg_area, closure->user_data); - - rtype = cif->flags; - - if (!retval_on_stack(cif->rtype) && cif->flags == FFI_TYPE_STRUCT) { - if (cif->rtype->size == 8) { - rtype = FFI_TYPE_SINT64; - } else { - rtype = FFI_TYPE_INT; - } - } - - /* now, do a generic return based on the value of rtype */ - if (rtype == FFI_TYPE_INT) - { - asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); - } - else if (rtype == FFI_TYPE_FLOAT) - { - asm ("flds (%0)" : : "r" (resp) : "st" ); - } - else if (rtype == FFI_TYPE_DOUBLE) - { - asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_LONGDOUBLE) - { - asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_SINT64) - { - asm ("movl 0(%0),%%eax;" - "movl 4(%0),%%edx" - : : "r"(resp) - : "eax", "edx"); - } -#ifdef X86_WIN32 - else if (rtype == FFI_TYPE_SINT8) /* 1-byte struct */ - { - asm ("movsbl (%0),%%eax" : : "r" (resp) : "eax"); - } - else if (rtype == FFI_TYPE_SINT16) /* 2-bytes struct */ - { - asm ("movswl (%0),%%eax" : : "r" (resp) : "eax"); - } -#endif -} - -/*@-exportheader@*/ -static void -ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, - void **avalue, ffi_cif *cif) -/*@=exportheader@*/ -{ - register unsigned int i; - register void **p_argv; - register char *argp; - register ffi_type **p_arg; - - argp = stack; - - if (retval_on_stack(cif->rtype)) { - *rvalue = *(void **) argp; - argp += 4; - } - - p_argv = avalue; - - for (i = cif->nargs, p_arg = cif->arg_types; (i != 0); i--, p_arg++) - { - size_t z; - - /* Align if necessary */ - if ((sizeof(int) - 1) & (unsigned) argp) { - argp = (char *) ALIGN(argp, sizeof(int)); - } - - z = (*p_arg)->size; - - /* because we're little endian, this is what it turns into. */ - - *p_argv = (void*) argp; - - p_argv++; - argp += z; - } - - return; -} - -/* How to make a trampoline. Derived from gcc/config/i386/i386.c. */ - -#define FFI_INIT_TRAMPOLINE(TRAMP,FUN,CTX) \ -({ unsigned char *__tramp = (unsigned char*)(TRAMP); \ - unsigned int __fun = (unsigned int)(FUN); \ - unsigned int __ctx = (unsigned int)(CTX); \ - unsigned int __dis = __fun - ((unsigned int) __tramp + FFI_TRAMPOLINE_SIZE); \ - *(unsigned char*) &__tramp[0] = 0xb8; \ - *(unsigned int*) &__tramp[1] = __ctx; /* movl __ctx, %eax */ \ - *(unsigned char *) &__tramp[5] = 0xe9; \ - *(unsigned int*) &__tramp[6] = __dis; /* jmp __fun */ \ - }) - - -/* the cif must already be prep'ed */ - -ffi_status -ffi_prep_closure (ffi_closure* closure, - ffi_cif* cif, - void (*fun)(ffi_cif*,void*,void**,void*), - void *user_data) -{ - FFI_ASSERT (cif->abi == FFI_SYSV); - - FFI_INIT_TRAMPOLINE (&closure->tramp[0], \ - &ffi_closure_SYSV, \ - (void*)closure); - - closure->cif = cif; - closure->user_data = user_data; - closure->fun = fun; - - return FFI_OK; -} - -/* ------- Native raw API support -------------------------------- */ - -#if !FFI_NO_RAW_API - -static void -ffi_closure_raw_SYSV (closure) - ffi_raw_closure *closure; -{ - // this is our return value storage - long double res; - - // our various things... - ffi_raw *raw_args; - ffi_cif *cif; - unsigned short rtype; - void *resp = (void*)&res; - - /* get the cif */ - cif = closure->cif; - - /* the SYSV/X86 abi matches the RAW API exactly, well.. almost */ - raw_args = (ffi_raw*) __builtin_dwarf_cfa (); - - (closure->fun) (cif, resp, raw_args, closure->user_data); - - rtype = cif->flags; - - /* now, do a generic return based on the value of rtype */ - if (rtype == FFI_TYPE_INT) - { - asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); - } - else if (rtype == FFI_TYPE_FLOAT) - { - asm ("flds (%0)" : : "r" (resp) : "st" ); - } - else if (rtype == FFI_TYPE_DOUBLE) - { - asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_LONGDOUBLE) - { - asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_SINT64) - { - asm ("movl 0(%0),%%eax; movl 4(%0),%%edx" - : : "r"(resp) - : "eax", "edx"); - } -} - - - - -ffi_status -ffi_prep_raw_closure (ffi_raw_closure* closure, - ffi_cif* cif, - void (*fun)(ffi_cif*,void*,ffi_raw*,void*), - void *user_data) -{ - int i; - - FFI_ASSERT (cif->abi == FFI_SYSV); - - // we currently don't support certain kinds of arguments for raw - // closures. This should be implemented by a separate assembly language - // routine, since it would require argument processing, something we - // don't do now for performance. - - for (i = cif->nargs-1; i >= 0; i--) - { - FFI_ASSERT (cif->arg_types[i]->type != FFI_TYPE_STRUCT); - FFI_ASSERT (cif->arg_types[i]->type != FFI_TYPE_LONGDOUBLE); - } - - - FFI_INIT_TRAMPOLINE (&closure->tramp[0], &ffi_closure_raw_SYSV, - (void*)closure); - - closure->cif = cif; - closure->user_data = user_data; - closure->fun = fun; - - return FFI_OK; -} - -static void -ffi_prep_args_raw(char *stack, extended_cif *ecif) -{ - memcpy (stack, ecif->avalue, ecif->cif->bytes); -} - -/* we borrow this routine from libffi (it must be changed, though, to - * actually call the function passed in the first argument. as of - * libffi-1.20, this is not the case.) - */ - -extern void -ffi_call_SYSV(void (*)(char *, extended_cif *), - /*@out@*/ extended_cif *, - unsigned, unsigned, - /*@out@*/ unsigned *, - void (*fn)()); - -#ifdef X86_WIN32 -extern void -ffi_call_STDCALL(void (*)(char *, extended_cif *), - /*@out@*/ extended_cif *, - unsigned, unsigned, - /*@out@*/ unsigned *, - void (*fn)()); -#endif /* X86_WIN32 */ - -void -ffi_raw_call(/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ ffi_raw *fake_avalue) -{ - extended_cif ecif; - void **avalue = (void **)fake_avalue; - - ecif.cif = cif; - ecif.avalue = avalue; - - /* If the return value is a struct and we don't have a return */ - /* value address then we need to make one */ - - if ((rvalue == NULL) && retval_on_stack(cif->rtype)) - { - /*@-sysunrecog@*/ - ecif.rvalue = alloca(cif->rtype->size); - /*@=sysunrecog@*/ - } - else - ecif.rvalue = rvalue; - - - switch (cif->abi) - { - case FFI_SYSV: - /*@-usedef@*/ - ffi_call_SYSV(ffi_prep_args_raw, &ecif, cif->bytes, - cif->flags, ecif.rvalue, fn); - /*@=usedef@*/ - break; -#ifdef X86_WIN32 - case FFI_STDCALL: - /*@-usedef@*/ - ffi_call_STDCALL(ffi_prep_args_raw, &ecif, cif->bytes, - cif->flags, ecif.rvalue, fn); - /*@=usedef@*/ - break; -#endif /* X86_WIN32 */ - default: - FFI_ASSERT(0); - break; - } -} - -#endif - -#endif /* __x86_64__ */ - -#endif /* __i386__ */ +# ifdef __i386__ +/* ----------------------------------------------------------------------- + ffi.c - Copyright (c) 1996, 1998, 1999, 2001 Red Hat, Inc. + Copyright (c) 2002 Ranjit Mathew + Copyright (c) 2002 Bo Thorsen + Copyright (c) 2002 Roger Sayle + + x86 Foreign Function Interface + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + ----------------------------------------------------------------------- */ + +#ifndef __x86_64__ + +#include +#include + +#include + +/* ffi_prep_args is called by the assembly routine once stack space + has been allocated for the function's arguments */ + +/*@-exportheader@*/ +void ffi_prep_args(char *stack, extended_cif *ecif); + +static inline int retval_on_stack(ffi_type* tp) +{ + if (tp->type == FFI_TYPE_STRUCT) { + int sz = tp->size; + if (sz > 8) { + return 1; + } + switch (sz) { + case 1: case 2: case 4: case 8: return 0; + default: return 1; + } + } + return 0; +} + + +void ffi_prep_args(char *stack, extended_cif *ecif) +/*@=exportheader@*/ +{ + register unsigned int i; + register void **p_argv; + register char *argp; + register ffi_type **p_arg; + + argp = stack; + + if (retval_on_stack(ecif->cif->rtype)) { + *(void **) argp = ecif->rvalue; + argp += 4; + } + + + p_argv = ecif->avalue; + + for (i = ecif->cif->nargs, p_arg = ecif->cif->arg_types; + i != 0; + i--, p_arg++) + { + size_t z; + + /* Align if necessary */ + if ((sizeof(int) - 1) & (unsigned) argp) + argp = (char *) ALIGN(argp, sizeof(int)); + + z = (*p_arg)->size; + if (z < sizeof(int)) + { + z = sizeof(int); + switch ((*p_arg)->type) + { + case FFI_TYPE_SINT8: + *(signed int *) argp = (signed int)*(SINT8 *)(* p_argv); + break; + + case FFI_TYPE_UINT8: + *(unsigned int *) argp = (unsigned int)*(UINT8 *)(* p_argv); + break; + + case FFI_TYPE_SINT16: + *(signed int *) argp = (signed int)*(SINT16 *)(* p_argv); + break; + + case FFI_TYPE_UINT16: + *(unsigned int *) argp = (unsigned int)*(UINT16 *)(* p_argv); + break; + + case FFI_TYPE_SINT32: + *(signed int *) argp = (signed int)*(SINT32 *)(* p_argv); + break; + + case FFI_TYPE_UINT32: + *(unsigned int *) argp = (unsigned int)*(UINT32 *)(* p_argv); + break; + + case FFI_TYPE_STRUCT: + *(unsigned int *) argp = (unsigned int)*(UINT32 *)(* p_argv); + break; + + default: + FFI_ASSERT(0); + } + } + else + { + memcpy(argp, *p_argv, z); + } + p_argv++; + argp += z; + } + + return; +} + +/* Perform machine dependent cif processing */ +ffi_status ffi_prep_cif_machdep(ffi_cif *cif) +{ + /* Set the return type flag */ + switch (cif->rtype->type) + { + case FFI_TYPE_VOID: +#if !defined(X86_WIN32) + case FFI_TYPE_STRUCT: +#endif + case FFI_TYPE_SINT64: + case FFI_TYPE_FLOAT: + case FFI_TYPE_DOUBLE: + case FFI_TYPE_LONGDOUBLE: + cif->flags = (unsigned) cif->rtype->type; + break; + + case FFI_TYPE_UINT64: + cif->flags = FFI_TYPE_SINT64; + break; + +#if defined X86_WIN32 + + case FFI_TYPE_STRUCT: + if (cif->rtype->size == 1) + { + cif->flags = FFI_TYPE_SINT8; /* same as char size */ + } + else if (cif->rtype->size == 2) + { + cif->flags = FFI_TYPE_SINT16; /* same as short size */ + } + else if (cif->rtype->size == 4) + { + cif->flags = FFI_TYPE_INT; /* same as int type */ + } + else if (cif->rtype->size == 8) + { + cif->flags = FFI_TYPE_SINT64; /* same as int64 type */ + } + else + { + cif->flags = FFI_TYPE_STRUCT; + } + break; +#endif + + default: + cif->flags = FFI_TYPE_INT; + break; + } + + /* Darwin: The stack needs to be aligned to a multiple of 16 bytes */ +#if 0 + cif->bytes = (cif->bytes + 15) & ~0xF; +#endif + + return FFI_OK; +} + +/*@-declundef@*/ +/*@-exportheader@*/ +extern void ffi_call_SYSV(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)(void)); +/*@=declundef@*/ +/*@=exportheader@*/ + +#ifdef X86_WIN32 +/*@-declundef@*/ +/*@-exportheader@*/ +extern void ffi_call_STDCALL(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)(void)); +/*@=declundef@*/ +/*@=exportheader@*/ +#endif /* X86_WIN32 */ + +void ffi_call(/*@dependent@*/ ffi_cif *cif, + void (*fn)(), + /*@out@*/ void *rvalue, + /*@dependent@*/ void **avalue) +{ + extended_cif ecif; + int flags; + + ecif.cif = cif; + ecif.avalue = avalue; + + /* If the return value is a struct and we don't have a return */ + /* value address then we need to make one */ + + if ((rvalue == NULL) && retval_on_stack(cif->rtype)) + { + /*@-sysunrecog@*/ + ecif.rvalue = alloca(cif->rtype->size); + /*@=sysunrecog@*/ + } + else + ecif.rvalue = rvalue; + + flags = cif->flags; + if (flags == FFI_TYPE_STRUCT) { + if (cif->rtype->size == 8) { + flags = FFI_TYPE_SINT64; + } else if (cif->rtype->size == 4) { + flags = FFI_TYPE_INT; + } else if (cif->rtype->size == 2) { + flags = FFI_TYPE_INT; + } else if (cif->rtype->size == 1) { + flags = FFI_TYPE_INT; + } + } + + + switch (cif->abi) + { + case FFI_SYSV: + /*@-usedef@*/ + /* To avoid changing the assembly code make sure the size of the argument + * block is a multiple of 16. Then add 8 to compensate for local variables + * in ffi_call_SYSV. + */ + ffi_call_SYSV(ffi_prep_args, &ecif, ALIGN(cif->bytes, 16) +8, + flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#ifdef X86_WIN32 + case FFI_STDCALL: + /*@-usedef@*/ + ffi_call_STDCALL(ffi_prep_args, &ecif, cif->bytes, + cif->flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#endif /* X86_WIN32 */ + default: + FFI_ASSERT(0); + break; + } +} + + +/** private members **/ + +static void ffi_prep_incoming_args_SYSV (char *stack, void **ret, + void** args, ffi_cif* cif); +static void ffi_closure_SYSV (ffi_closure *) + __attribute__ ((regparm(1))); +#if !FFI_NO_RAW_API +static void ffi_closure_raw_SYSV (ffi_raw_closure *) + __attribute__ ((regparm(1))); +#endif + +/* This function is jumped to by the trampoline */ + +static void +ffi_closure_SYSV (closure) + ffi_closure *closure; +{ + // this is our return value storage + long double res; + + // our various things... + ffi_cif *cif; + void **arg_area; + unsigned short rtype; + void *resp = (void*)&res; + void *args = __builtin_dwarf_cfa (); + + cif = closure->cif; + arg_area = (void**) alloca (cif->nargs * sizeof (void*)); + + /* this call will initialize ARG_AREA, such that each + * element in that array points to the corresponding + * value on the stack; and if the function returns + * a structure, it will re-set RESP to point to the + * structure return address. */ + + ffi_prep_incoming_args_SYSV(args, (void**)&resp, arg_area, cif); + + (closure->fun) (cif, resp, arg_area, closure->user_data); + + rtype = cif->flags; + + if (!retval_on_stack(cif->rtype) && cif->flags == FFI_TYPE_STRUCT) { + if (cif->rtype->size == 8) { + rtype = FFI_TYPE_SINT64; + } else { + rtype = FFI_TYPE_INT; + } + } + + /* now, do a generic return based on the value of rtype */ + if (rtype == FFI_TYPE_INT) + { + asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); + } + else if (rtype == FFI_TYPE_FLOAT) + { + asm ("flds (%0)" : : "r" (resp) : "st" ); + } + else if (rtype == FFI_TYPE_DOUBLE) + { + asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_LONGDOUBLE) + { + asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_SINT64) + { + asm ("movl 0(%0),%%eax;" + "movl 4(%0),%%edx" + : : "r"(resp) + : "eax", "edx"); + } +#ifdef X86_WIN32 + else if (rtype == FFI_TYPE_SINT8) /* 1-byte struct */ + { + asm ("movsbl (%0),%%eax" : : "r" (resp) : "eax"); + } + else if (rtype == FFI_TYPE_SINT16) /* 2-bytes struct */ + { + asm ("movswl (%0),%%eax" : : "r" (resp) : "eax"); + } +#endif +} + +/*@-exportheader@*/ +static void +ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, + void **avalue, ffi_cif *cif) +/*@=exportheader@*/ +{ + register unsigned int i; + register void **p_argv; + register char *argp; + register ffi_type **p_arg; + + argp = stack; + + if (retval_on_stack(cif->rtype)) { + *rvalue = *(void **) argp; + argp += 4; + } + + p_argv = avalue; + + for (i = cif->nargs, p_arg = cif->arg_types; (i != 0); i--, p_arg++) + { + size_t z; + + /* Align if necessary */ + if ((sizeof(int) - 1) & (unsigned) argp) { + argp = (char *) ALIGN(argp, sizeof(int)); + } + + z = (*p_arg)->size; + + /* because we're little endian, this is what it turns into. */ + + *p_argv = (void*) argp; + + p_argv++; + argp += z; + } + + return; +} + +/* How to make a trampoline. Derived from gcc/config/i386/i386.c. */ + +#define FFI_INIT_TRAMPOLINE(TRAMP,FUN,CTX) \ +({ unsigned char *__tramp = (unsigned char*)(TRAMP); \ + unsigned int __fun = (unsigned int)(FUN); \ + unsigned int __ctx = (unsigned int)(CTX); \ + unsigned int __dis = __fun - ((unsigned int) __tramp + FFI_TRAMPOLINE_SIZE); \ + *(unsigned char*) &__tramp[0] = 0xb8; \ + *(unsigned int*) &__tramp[1] = __ctx; /* movl __ctx, %eax */ \ + *(unsigned char *) &__tramp[5] = 0xe9; \ + *(unsigned int*) &__tramp[6] = __dis; /* jmp __fun */ \ + }) + + +/* the cif must already be prep'ed */ + +ffi_status +ffi_prep_closure (ffi_closure* closure, + ffi_cif* cif, + void (*fun)(ffi_cif*,void*,void**,void*), + void *user_data) +{ + FFI_ASSERT (cif->abi == FFI_SYSV); + + FFI_INIT_TRAMPOLINE (&closure->tramp[0], \ + &ffi_closure_SYSV, \ + (void*)closure); + + closure->cif = cif; + closure->user_data = user_data; + closure->fun = fun; + + return FFI_OK; +} + +/* ------- Native raw API support -------------------------------- */ + +#if !FFI_NO_RAW_API + +static void +ffi_closure_raw_SYSV (closure) + ffi_raw_closure *closure; +{ + // this is our return value storage + long double res; + + // our various things... + ffi_raw *raw_args; + ffi_cif *cif; + unsigned short rtype; + void *resp = (void*)&res; + + /* get the cif */ + cif = closure->cif; + + /* the SYSV/X86 abi matches the RAW API exactly, well.. almost */ + raw_args = (ffi_raw*) __builtin_dwarf_cfa (); + + (closure->fun) (cif, resp, raw_args, closure->user_data); + + rtype = cif->flags; + + /* now, do a generic return based on the value of rtype */ + if (rtype == FFI_TYPE_INT) + { + asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); + } + else if (rtype == FFI_TYPE_FLOAT) + { + asm ("flds (%0)" : : "r" (resp) : "st" ); + } + else if (rtype == FFI_TYPE_DOUBLE) + { + asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_LONGDOUBLE) + { + asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); + } + else if (rtype == FFI_TYPE_SINT64) + { + asm ("movl 0(%0),%%eax; movl 4(%0),%%edx" + : : "r"(resp) + : "eax", "edx"); + } +} + + + + +ffi_status +ffi_prep_raw_closure (ffi_raw_closure* closure, + ffi_cif* cif, + void (*fun)(ffi_cif*,void*,ffi_raw*,void*), + void *user_data) +{ + int i; + + FFI_ASSERT (cif->abi == FFI_SYSV); + + // we currently don't support certain kinds of arguments for raw + // closures. This should be implemented by a separate assembly language + // routine, since it would require argument processing, something we + // don't do now for performance. + + for (i = cif->nargs-1; i >= 0; i--) + { + FFI_ASSERT (cif->arg_types[i]->type != FFI_TYPE_STRUCT); + FFI_ASSERT (cif->arg_types[i]->type != FFI_TYPE_LONGDOUBLE); + } + + + FFI_INIT_TRAMPOLINE (&closure->tramp[0], &ffi_closure_raw_SYSV, + (void*)closure); + + closure->cif = cif; + closure->user_data = user_data; + closure->fun = fun; + + return FFI_OK; +} + +static void +ffi_prep_args_raw(char *stack, extended_cif *ecif) +{ + memcpy (stack, ecif->avalue, ecif->cif->bytes); +} + +/* we borrow this routine from libffi (it must be changed, though, to + * actually call the function passed in the first argument. as of + * libffi-1.20, this is not the case.) + */ + +extern void +ffi_call_SYSV(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)()); + +#ifdef X86_WIN32 +extern void +ffi_call_STDCALL(void (*)(char *, extended_cif *), + /*@out@*/ extended_cif *, + unsigned, unsigned, + /*@out@*/ unsigned *, + void (*fn)()); +#endif /* X86_WIN32 */ + +void +ffi_raw_call(/*@dependent@*/ ffi_cif *cif, + void (*fn)(), + /*@out@*/ void *rvalue, + /*@dependent@*/ ffi_raw *fake_avalue) +{ + extended_cif ecif; + void **avalue = (void **)fake_avalue; + + ecif.cif = cif; + ecif.avalue = avalue; + + /* If the return value is a struct and we don't have a return */ + /* value address then we need to make one */ + + if ((rvalue == NULL) && retval_on_stack(cif->rtype)) + { + /*@-sysunrecog@*/ + ecif.rvalue = alloca(cif->rtype->size); + /*@=sysunrecog@*/ + } + else + ecif.rvalue = rvalue; + + + switch (cif->abi) + { + case FFI_SYSV: + /*@-usedef@*/ + ffi_call_SYSV(ffi_prep_args_raw, &ecif, cif->bytes, + cif->flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#ifdef X86_WIN32 + case FFI_STDCALL: + /*@-usedef@*/ + ffi_call_STDCALL(ffi_prep_args_raw, &ecif, cif->bytes, + cif->flags, ecif.rvalue, fn); + /*@=usedef@*/ + break; +#endif /* X86_WIN32 */ + default: + FFI_ASSERT(0); + break; + } +} + +#endif + +#endif /* __x86_64__ */ + +#endif /* __i386__ */ From buildbot at python.org Sat Jun 10 22:48:06 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 20:48:06 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060610204806.880761E400C@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/707 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 22:50:17 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sat, 10 Jun 2006 22:50:17 +0200 (CEST) Subject: [Python-checkins] r46833 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060610205017.219AB1E400C@bag.python.org> Author: mateusz.rukowicz Date: Sat Jun 10 22:50:16 2006 New Revision: 46833 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Another bugs fixed, compare now works well. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sat Jun 10 22:50:16 2006 @@ -321,8 +321,10 @@ #define GETNAN(op) ( ( (op)->sign == SIGN_POSNAN || (op)->sign == SIGN_NEGNAN ) ? \ 1 : ( ( (op)->sign == SIGN_POSSNAN || (op)->sign == SIGN_NEGSNAN ) ? 2 : 0 ) ) -#define ISINF(op) ((op)->sign == SIGN_POSINF || (op)->sign == SIGN_NEGINF) - +//#define ISINF(op) ((op)->sign == SIGN_POSINF || (op)->sign == SIGN_NEGINF) +#define ISINF(op) ( ( (op)->sign == SIGN_POSINF || (op)->sign == SIGN_NEGINF) ? \ + ((op)->sign == SIGN_POSINF ? 1 : -1) : 0 ) + /* Exponent calculations */ /* XXX: overflow checking? */ @@ -380,6 +382,7 @@ #define S_UNDERFLOW 6 #define S_SUBNORMAL 7 +/* WTF ?? */ /* Other conditions. If they occur and raise an exception, it will be InvalidOperation. Also, context->flags, ->traps and ->ignored will only contain entries for the signals defined above. @@ -1476,6 +1479,14 @@ /* comparisons with NaN always report self>other */ return 1; } + + if(ISINF(self) || ISINF(other)) + { + if(ISINF(self) == ISINF(other)) + return 0; + else + return ISINF(self) > ISINF(other) ? 1 : -1 ; + } } /* If both are 0, sign comparison isn't correct. */ @@ -1515,12 +1526,14 @@ next: /* Adjusted sizes are not equal. */ - if (adj1 > adj2 && self->digits[0] != 0) - return 1 - 2*(self->sign & 1); /* 0 -> 1, 1 -> -1 */ - else if (adj1 < adj2 && other->digits[0] != 0) - return 1 - 2*(other->sign & 1); +// if (adj1 > adj2 && self->digits[0] != 0) + if(adj1 > adj2 && _limb_get_digit(self->limbs, self->ob_size, 0) != 0) + return 1 - 2*(self->sign & 1); /* 0 -> 1, 1 -> -1 */ +// else if (adj1 < adj2 && other->digits[0] != 0) + else if(adj1 < adj2 && _limb_get_digit(other->limbs, other->ob_size, 0) != 0) + return -1 + 2*(other->sign & 1); - ctx2 = context_shallow_copy(ctx); + ctx2 = context_copy(ctx); if (!ctx2) return 0; /* error */ ctx2->rounding = ROUND_UP; /* away from 0 */ @@ -1531,7 +1544,7 @@ ans = _do_decimal_subtract(self, other, ctx2); if (!ans) return 0; - if (decimal_nonzero(ans)) + if (!decimal_nonzero(ans)) i = 0; else if (ans->sign & 1) i = -1; @@ -1968,6 +1981,7 @@ char *p, *end; int roundexp, i; + end = &outbuf[FORMAT_SIZE-1]; p = outbuf; if (d->sign & 1) { *p++ = '-'; @@ -1975,15 +1989,25 @@ } if (d->exp < 0 && d->exp >= -6) { /* figure out the right number of digits */ - i = sprintf(p, "0.%0*d", abs(d->exp), 0); +// i = sprintf(p, "0.%0*d", abs(d->exp), 0); /* and write those as zeros */ - for (; i > 0; i--) { - *p++ = '0'; - } +// for (; i > 0; i--) { +// *p++ = '0'; +// } + i = 0; + *p++ = '0'; + *p++ = '.'; + for(;iexp);i++) + *p++ = '0'; SANITY_CHECK(p); } else { - roundexp = ((d->exp - 1)/3 + 1) * 3; /* nearest gt mult of 3 */ - if (roundexp != d->exp) { +// roundexp = ((d->exp - 1)/3 + 1) * 3; /* nearest gt mult of 3 */ + roundexp = d->exp; + while(roundexp%3) + roundexp ++; + if (roundexp != d->exp) { + *p++ = '0'; + *p++ = '.'; for (i = 0; i < (roundexp - d->exp); i++) *p++ = '0'; SANITY_CHECK(p); @@ -2005,6 +2029,8 @@ SANITY_CHECK(p); } } + *p++ = 0; + p = &outbuf[0]; return PyString_FromString(p); } @@ -2084,11 +2110,14 @@ { adjexp --; dotplace ++; - extra_zeros ++; +// extra_zeros ++; } /* now all we have to do, is to put it to the string =] */ + if(dotplace > d->ob_size) + extra_zeros = dotplace - d->ob_size; + if(dotplace <= 0) { *p++ = '0'; @@ -2705,7 +2734,7 @@ static decimalobject * _do_decimal_negative(decimalobject *self, contextobject *ctx) { - int sign = 0; + int sign = self->sign; decimalobject *tmp, *res; if (ISSPECIAL(self)) { @@ -3100,6 +3129,7 @@ char *last_digit = 0; char *dot = 0; char *e = 0; + int any_digit = 0; char *c; long i; decimalobject *new; @@ -3133,7 +3163,7 @@ else if(*c == '.') { - if(dot) + if(dot || e) goto err; dot = c; } @@ -3148,6 +3178,8 @@ } else { + if(!e) + any_digit = 1; if(dot && !e) digits_after_dot ++; @@ -3158,6 +3190,8 @@ } } } /* we now are pretty sure, that string is properly formated */ + if(!any_digit) + goto err; if(!first_digit) { zero = 1; @@ -3180,9 +3214,21 @@ assert(isdigit(*c) || *c == '.'); } } + + if(!ndigits) + { + goto err; + } if(e) /* pretty obvious */ - exp = atol(e+1); + { + int ok; + ok = sscanf(e+1,"%d", &exp); + if(ok!=1) + { + goto err; + } + } exp -= digits_after_dot; new = _new_decimalobj(type, ndigits, sign, exp); @@ -3583,7 +3629,11 @@ return decimal_from_long(type, x); } - ctx = getcontext(); + if(context) + ctx = context; + else + ctx = getcontext(); + if (!ctx) return NULL; if (PyLong_Check(value)) @@ -4135,7 +4185,7 @@ static char *kwlist[] = {"a", 0}; decimalobject *a; - if(!PyArg_ParseTupleAndKeywords(args, kwds, "O", kwlist, &a)) + if(!PyArg_ParseTupleAndKeywords(args, kwds, "O:_apply", kwlist, &a)) return NULL; if(!PyDecimal_Check(a)) @@ -4264,12 +4314,13 @@ static PyObject * -context_to_sci_string(contextobject *self, PyObject *args) +context_to_sci_string(contextobject *self, PyObject *args, PyObject *kwds) { PyObject *a, *res; decimalobject *dec_a = NULL; + static char *kwlist[] = {"a", 0}; - if (!PyArg_ParseTuple(args, "O:to_sci_string", &a)) + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O:to_sci_string", kwlist, &a)) return NULL; dec_a = (decimalobject *)_convert_to_decimal( @@ -4277,8 +4328,7 @@ if (dec_a == NULL) return NULL; - /* XXX: default third argument? */ - res = _do_decimal_str(dec_a, self, -1); + res = _do_decimal_str(dec_a, self, 0); Py_DECREF(dec_a); return res; } @@ -4627,7 +4677,7 @@ {"to_integral", (PyCFunction)context_to_integral, METH_O}, {"to_sci_string", (PyCFunction)context_to_sci_string, - METH_O}, + METH_VARARGS | METH_KEYWORDS}, {"_ignore_all_flags", (PyCFunction)context_ignore_all_flags, METH_NOARGS}, {"_ignore_flags", (PyCFunction)context_ignore_flags, From buildbot at python.org Sat Jun 10 22:55:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 20:55:02 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable 2.4 Message-ID: <20060610205502.7F5871E400C@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%25202.4/builds/97 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 10 23:07:20 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 23:07:20 +0200 (CEST) Subject: [Python-checkins] r46834 - python/trunk/Modules/_ctypes/_ctypes.c Message-ID: <20060610210720.829DF1E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 23:07:19 2006 New Revision: 46834 Modified: python/trunk/Modules/_ctypes/_ctypes.c Log: Fix a wrong printf format. Modified: python/trunk/Modules/_ctypes/_ctypes.c ============================================================================== --- python/trunk/Modules/_ctypes/_ctypes.c (original) +++ python/trunk/Modules/_ctypes/_ctypes.c Sat Jun 10 23:07:19 2006 @@ -1827,12 +1827,7 @@ #if (PY_VERSION_HEX < 0x02050000) cp += sprintf(cp, "%x", index); #else -#ifdef MS_WIN32 -/* MSVC does not understand the 'z' size specifier */ - cp += sprintf(cp, "%Ix", index); -#else - cp += sprintf(cp, "%zx", index); -#endif + cp += sprintf(cp, "%x", Py_SAFE_DOWNCAST(index, Py_ssize_t, int)); #endif while (target->b_base) { bytes_left = sizeof(string) - (cp - string) - 1; From python-checkins at python.org Sat Jun 10 23:17:58 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 23:17:58 +0200 (CEST) Subject: [Python-checkins] r46835 - python/trunk/Modules/_ctypes/_ctypes.c Message-ID: <20060610211758.ECA251E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 23:17:58 2006 New Revision: 46835 Modified: python/trunk/Modules/_ctypes/_ctypes.c Log: Fix the second occurrence of the problematic printf format. Modified: python/trunk/Modules/_ctypes/_ctypes.c ============================================================================== --- python/trunk/Modules/_ctypes/_ctypes.c (original) +++ python/trunk/Modules/_ctypes/_ctypes.c Sat Jun 10 23:17:58 2006 @@ -1840,11 +1840,7 @@ #if (PY_VERSION_HEX < 0x02050000) cp += sprintf(cp, ":%x", (int)target->b_index); #else -#ifdef MS_WIN32 - cp += sprintf(cp, ":%Ix", (size_t)target->b_index); -#else - cp += sprintf(cp, ":%zx", (size_t)target->b_index); -#endif + cp += sprintf(cp, ":%x", Py_SAFE_DOWNCAST(target->b_index, Py_ssize_t, int)); #endif target = target->b_base; } From python-checkins at python.org Sat Jun 10 23:30:16 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sat, 10 Jun 2006 23:30:16 +0200 (CEST) Subject: [Python-checkins] r46836 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060610213016.6E73C1E401B@bag.python.org> Author: mateusz.rukowicz Date: Sat Jun 10 23:30:15 2006 New Revision: 46836 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Max now works and passes every test. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sat Jun 10 23:30:15 2006 @@ -1604,7 +1604,9 @@ _do_decimal_max(decimalobject *self, decimalobject *other, contextobject *ctx) { - decimalobject *ans, *cmp; + decimalobject *ans; + int cmp; + contextobject *ctx2; if (ISSPECIAL(self) || ISSPECIAL(other)) { decimalobject *nan; @@ -1623,7 +1625,44 @@ return nan; } - /* XXX */ + ans = self; /* let's assume */ + if(!ctx) + { + ctx = getcontext(); + } + + /* we have to give getcontext() to _do_decimal_compare */ + ctx2 = getcontext(); + if(!ctx2) + return NULL; + + cmp = _do_real_decimal_compare(self, other,ctx2); + if(PyErr_Occurred()) + return NULL; + + if(!cmp) + { + if(self->sign != other->sign) + { + if(self->sign) + ans = other; + } + else + { + if(self->exp < other->exp && !self->sign) + ans = other; + else if(self->exp > other->exp && self->sign) + ans = other; + } + } + else if(cmp == -1) + ans = other; + + if(ctx ->rounding_dec == ALWAYS_ROUND) + return _decimal_fix(ans, ctx); + + Py_INCREF(ans); + return ans; } DECIMAL_BINARY_FUNC(max) From nnorwitz at gmail.com Sat Jun 10 23:32:51 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 10 Jun 2006 14:32:51 -0700 Subject: [Python-checkins] r46824 - in python/trunk/Modules/_ctypes: _ctypes.c callproc.c ctypes.h libffi/src/x86/darwin.S libffi_msvc/mingwin32.S stgdict.c In-Reply-To: <20060610195148.874F81E400C@bag.python.org> References: <20060610195148.874F81E400C@bag.python.org> Message-ID: 2 issues, see below On 6/10/06, thomas.heller wrote: > Author: thomas.heller > Date: Sat Jun 10 21:51:46 2006 > New Revision: 46824 > > Modified: python/trunk/Modules/_ctypes/callproc.c > ============================================================================== > --- python/trunk/Modules/_ctypes/callproc.c (original) > +++ python/trunk/Modules/_ctypes/callproc.c Sat Jun 10 21:51:46 2006 > @@ -1444,7 +1444,64 @@ > } > #endif > > +static PyObject * > +resize(PyObject *self, PyObject *args) > +{ ... > + if (obj->b_size <= sizeof(obj->b_value)) { > + /* We are currently using the objects default buffer, but it > + isn't large enough any more. */ > + void *ptr = PyMem_Malloc(size); > + if (ptr == NULL) > + return PyErr_NoMemory(); > + memset(ptr, 0, size); > + memmove(ptr, obj->b_ptr, obj->b_size); > + obj->b_ptr = ptr; > + obj->b_size = size; > + } else { > + obj->b_ptr = PyMem_Realloc(obj->b_ptr, size); > + obj->b_size = size; > + } Realloc can fail. If it did this would leak memory. > Modified: python/trunk/Modules/_ctypes/stgdict.c > ============================================================================== > --- python/trunk/Modules/_ctypes/stgdict.c (original) > +++ python/trunk/Modules/_ctypes/stgdict.c Sat Jun 10 21:51:46 2006 > +static int > +MakeFields(PyObject *type, CFieldObject *descr, > + Py_ssize_t index, Py_ssize_t offset) > +{ ... > + // Convert to PyArg_UnpackTuple... > + if (!PyArg_ParseTuple(pair, "OO", &fname, &ftype)) { > + Py_DECREF(fieldlist); > + return -1; > + } Should use /* */ comments instead of //. From python-checkins at python.org Sat Jun 10 23:56:04 2006 From: python-checkins at python.org (thomas.heller) Date: Sat, 10 Jun 2006 23:56:04 +0200 (CEST) Subject: [Python-checkins] r46837 - python/trunk/Modules/_ctypes/stgdict.c Message-ID: <20060610215604.454661E400C@bag.python.org> Author: thomas.heller Date: Sat Jun 10 23:56:03 2006 New Revision: 46837 Modified: python/trunk/Modules/_ctypes/stgdict.c Log: Don't use C++ comment. Modified: python/trunk/Modules/_ctypes/stgdict.c ============================================================================== --- python/trunk/Modules/_ctypes/stgdict.c (original) +++ python/trunk/Modules/_ctypes/stgdict.c Sat Jun 10 23:56:03 2006 @@ -167,7 +167,7 @@ PyObject *fname, *ftype; CFieldObject *fdescr; CFieldObject *new_descr; - // Convert to PyArg_UnpackTuple... + /* Convert to PyArg_UnpackTuple... */ if (!PyArg_ParseTuple(pair, "OO", &fname, &ftype)) { Py_DECREF(fieldlist); return -1; From python-checkins at python.org Sun Jun 11 00:01:50 2006 From: python-checkins at python.org (thomas.heller) Date: Sun, 11 Jun 2006 00:01:50 +0200 (CEST) Subject: [Python-checkins] r46838 - python/trunk/Modules/_ctypes/callproc.c Message-ID: <20060610220150.D93881E4013@bag.python.org> Author: thomas.heller Date: Sun Jun 11 00:01:50 2006 New Revision: 46838 Modified: python/trunk/Modules/_ctypes/callproc.c Log: Handle failure of PyMem_Realloc. Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Sun Jun 11 00:01:50 2006 @@ -1492,7 +1492,10 @@ obj->b_ptr = ptr; obj->b_size = size; } else { - obj->b_ptr = PyMem_Realloc(obj->b_ptr, size); + void * ptr = PyMem_Realloc(obj->b_ptr, size); + if (ptr == NULL) + return PyErr_NoMemory(); + obj->b_ptr = ptr; obj->b_size = size; } done: From theller at python.net Sun Jun 11 00:03:15 2006 From: theller at python.net (Thomas Heller) Date: Sun, 11 Jun 2006 00:03:15 +0200 Subject: [Python-checkins] r46824 - in python/trunk/Modules/_ctypes: _ctypes.c callproc.c ctypes.h libffi/src/x86/darwin.S libffi_msvc/mingwin32.S stgdict.c In-Reply-To: References: <20060610195148.874F81E400C@bag.python.org> Message-ID: Neal Norwitz wrote: > 2 issues, see below Should be fixed now. Thanks for the review. From python-checkins at python.org Sun Jun 11 00:38:14 2006 From: python-checkins at python.org (skip.montanaro) Date: Sun, 11 Jun 2006 00:38:14 +0200 (CEST) Subject: [Python-checkins] r46839 - python/trunk/Objects/bufferobject.c Message-ID: <20060610223814.1D8C21E400C@bag.python.org> Author: skip.montanaro Date: Sun Jun 11 00:38:13 2006 New Revision: 46839 Modified: python/trunk/Objects/bufferobject.c Log: Suppress warning on MacOSX about possible use before set of proc. Modified: python/trunk/Objects/bufferobject.c ============================================================================== --- python/trunk/Objects/bufferobject.c (original) +++ python/trunk/Objects/bufferobject.c Sun Jun 11 00:38:13 2006 @@ -33,7 +33,7 @@ } else { Py_ssize_t count, offset; - readbufferproc proc; + readbufferproc proc = 0; PyBufferProcs *bp = self->b_base->ob_type->tp_as_buffer; if ((*bp->bf_getsegcount)(self->b_base, NULL) != 1) { PyErr_SetString(PyExc_TypeError, From python-checkins at python.org Sun Jun 11 00:51:46 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 11 Jun 2006 00:51:46 +0200 (CEST) Subject: [Python-checkins] r46840 - python/trunk/Lib/random.py Message-ID: <20060610225146.2CD491E400D@bag.python.org> Author: tim.peters Date: Sun Jun 11 00:51:45 2006 New Revision: 46840 Modified: python/trunk/Lib/random.py Log: shuffle() doscstring: Removed warning about sequence length versus generator period. While this was a real weakness of the older WH generator for lists with just a few dozen elements, and so could potentially bite the naive ;-), the Twister should show excellent behavior up to at least 600 elements. Module docstring: reflowed some jarringly short lines. Modified: python/trunk/Lib/random.py ============================================================================== --- python/trunk/Lib/random.py (original) +++ python/trunk/Lib/random.py Sun Jun 11 00:51:45 2006 @@ -29,13 +29,12 @@ General notes on the underlying Mersenne Twister core generator: * The period is 2**19937-1. -* It is one of the most extensively tested generators in existence -* Without a direct way to compute N steps forward, the - semantics of jumpahead(n) are weakened to simply jump - to another distant state and rely on the large period - to avoid overlapping sequences. -* The random() method is implemented in C, executes in - a single Python step, and is, therefore, threadsafe. +* It is one of the most extensively tested generators in existence. +* Without a direct way to compute N steps forward, the semantics of + jumpahead(n) are weakened to simply jump to another distant state and rely + on the large period to avoid overlapping sequences. +* The random() method is implemented in C, executes in a single Python step, + and is, therefore, threadsafe. """ @@ -253,11 +252,6 @@ Optional arg random is a 0-argument function returning a random float in [0.0, 1.0); by default, the standard random.random. - - Note that for even rather small len(x), the total number of - permutations of x is larger than the period of most random number - generators; this implies that "most" permutations of a long - sequence can never be generated. """ if random is None: From python-checkins at python.org Sun Jun 11 00:54:19 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 11 Jun 2006 00:54:19 +0200 (CEST) Subject: [Python-checkins] r46841 - python/branches/release24-maint/Lib/random.py Message-ID: <20060610225419.9ECA71E400C@bag.python.org> Author: tim.peters Date: Sun Jun 11 00:54:19 2006 New Revision: 46841 Modified: python/branches/release24-maint/Lib/random.py Log: Merge rev 46840 from trunk. shuffle() doscstring: Removed warning about sequence length versus generator period. While this was a real weakness of the older WH generator for lists with just a few dozen elements, and so could potentially bite the naive ;-), the Twister should show excellent behavior up to at least 600 elements. Modified: python/branches/release24-maint/Lib/random.py ============================================================================== --- python/branches/release24-maint/Lib/random.py (original) +++ python/branches/release24-maint/Lib/random.py Sun Jun 11 00:54:19 2006 @@ -29,13 +29,12 @@ General notes on the underlying Mersenne Twister core generator: * The period is 2**19937-1. -* It is one of the most extensively tested generators in existence -* Without a direct way to compute N steps forward, the - semantics of jumpahead(n) are weakened to simply jump - to another distant state and rely on the large period - to avoid overlapping sequences. -* The random() method is implemented in C, executes in - a single Python step, and is, therefore, threadsafe. +* It is one of the most extensively tested generators in existence. +* Without a direct way to compute N steps forward, the semantics of + jumpahead(n) are weakened to simply jump to another distant state and rely + on the large period to avoid overlapping sequences. +* The random() method is implemented in C, executes in a single Python step, + and is, therefore, threadsafe. """ @@ -253,11 +252,6 @@ Optional arg random is a 0-argument function returning a random float in [0.0, 1.0); by default, the standard random.random. - - Note that for even rather small len(x), the total number of - permutations of x is larger than the period of most random number - generators; this implies that "most" permutations of a long - sequence can never be generated. """ if random is None: From buildbot at python.org Sun Jun 11 01:32:56 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 23:32:56 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian 2.4 Message-ID: <20060610233256.4B9831E400D@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%25202.4/builds/25 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: fred.drake,martin.v.loewis,skip.montanaro Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 01:33:54 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sun, 11 Jun 2006 01:33:54 +0200 (CEST) Subject: [Python-checkins] r46842 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060610233354.461EC1E400C@bag.python.org> Author: mateusz.rukowicz Date: Sun Jun 11 01:33:53 2006 New Revision: 46842 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Multiplying and min works. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sun Jun 11 01:33:53 2006 @@ -301,6 +301,59 @@ return left_digits; } +/* temporary solution, will be speeded up */ +static long +_limb_multiply_core(long *first, long flimbs, long *second, long slimbs, long *out) +{ + long max_limbs = flimbs + slimbs; + long i,j; + + for(i = 0;i= BASE) + { + out[i+j+1] += out[i+j]/BASE; + out[i+j] %= BASE; + } + } + + for(i = 0;i= BASE) + { + assert(i+1 < max_limbs); + out[i+1] += out[i] / BASE; + out[i] %= BASE; + } + } + + while(!out[max_limbs -1] && max_limbs > 1) max_limbs --; + return max_limbs; +} + +static long +_limb_multiply(long *first, long fsize, long *second, long ssize, long *out) +{ + long used_limbs; + long flimbs, slimbs; + long digits_at_most; + flimbs = (fsize + LOG - 1)/LOG; + slimbs = (ssize + LOG - 1)/LOG; + used_limbs = _limb_multiply_core(first, flimbs, second, slimbs, out); + + digits_at_most = used_limbs * LOG; + if(!digits_at_most) + return 1; + + while(_limb_get_digit(out, digits_at_most, 0) == 0 && digits_at_most >1) digits_at_most --; + + return digits_at_most; +} /* helpful macros ************************************************************/ @@ -1540,6 +1593,8 @@ if (context_ignore_all_flags(ctx2) == NULL) return 0; /* error */ + Py_XDECREF(ctx2); + /* XXX: WTF? */ ans = _do_decimal_subtract(self, other, ctx2); if (!ans) return 0; @@ -1671,8 +1726,71 @@ _do_decimal_min(decimalobject *self, decimalobject *other, contextobject *ctx) { - /* XXX */ - Py_RETURN_NONE; + decimalobject *ans; + long cmp; + decimalobject *ctx2; + + if(ISSPECIAL(self) || ISSPECIAL(other)) + { + decimalobject *nan; + int sn = GETNAN(self); + int so = GETNAN(other); + + if(sn || so) + { + if(so == 1 && sn != 2) + { + Py_INCREF(self); + return self; + } + else if (sn == 1 && so != 2) + { + Py_INCREF(other); + return other; + } + } + + if(_check_nans(self, other, ctx, &nan) != 0) + return nan; + } + + ans = self; + if(!ctx) + ctx = getcontext(); + if(!ctx) + return NULL; + + ctx2 = getcontext(); + if(!ctx2) + return NULL; + cmp = _do_real_decimal_compare(self,other,ctx2); + + if(PyErr_Occurred()) + return NULL; + + if(!cmp) + { + if(self->sign != other->sign) + { + if(!self->sign) + ans = other; + } + else{ + if(self->exp > other->exp && !self->sign) + ans = other; + else if (self->exp < other->exp && self->sign) + ans = other; + } + } + else if(cmp == 1) + ans = other; + + if(ctx->rounding_dec == ALWAYS_ROUND) + return _decimal_fix(ans, ctx); + + Py_INCREF(ans); + return ans; + } DECIMAL_BINARY_FUNC(min) @@ -1697,14 +1815,16 @@ new = _NEW_decimalobj(1, dup->sign, 0); Py_DECREF(dup); if (!new) return NULL; - new->digits[0] = 0; + new->limbs[0] = 0; return new; } /* strip trailing 0s from dup */ - while (dup->digits[dup->ob_size - 1] == 0) { - dup->ob_size--; - dup->exp++; +// while (dup->digits[dup->ob_size - 1] == 0) { + while(_limb_get_digit(dup->limbs, dup->ob_size, dup->ob_size -1) == 0){ + _limb_cut_one_digit(dup->limbs,dup->ob_size); + dup->ob_size --; + dup->exp ++; } return dup; } @@ -1776,7 +1896,7 @@ static PyObject * _do_decimal_same_quantum(decimalobject *self, decimalobject *other) { - if (ISSPECIAL(self) && ISSPECIAL(other)) { + if (ISSPECIAL(self) || ISSPECIAL(other)) { if (GETNAN(self) || GETNAN(other)) { if (GETNAN(self) && GETNAN(other)) Py_RETURN_TRUE; @@ -1870,6 +1990,8 @@ &rounding, &ctx)) return NULL; ENSURE_CONTEXT("to_integral", ctx); + if(rounding == -1) + rounding = ctx->rounding; if (!VALID_ROUND(rounding)) { PyErr_SetString(PyExc_ValueError, "invalid rounding value"); return NULL; @@ -2673,12 +2795,103 @@ DECIMAL_SPECIAL_2FUNC(decimal_subtract) +/* it's just so straightforward, wish adding was the same ;P */ static decimalobject * _do_decimal_multiply(decimalobject *self, decimalobject *other, contextobject *ctx) { - /* XXX */ - Py_RETURN_NONE; + long resultsign; + long resultexp; + long shouldround; + long i; + long max_limbs; + long size; + decimalobject *ans; + + resultsign = (self->sign&1) ^ (other->sign&1); + + if(ISSPECIAL(self) || ISSPECIAL(other)) + { + decimalobject *nan; + if(_check_nans(self, other, ctx, &nan)) + return nan; + + if(ISINF(self)) + { + if(decimal_nonzero(other)) + { + ans = _NEW_decimalobj(1, resultsign ? SIGN_NEGINF : SIGN_POSINF, 0); + return ans; + } + return handle_InvalidOperation(self->ob_type, ctx, "(+-)INF * 0", NULL); + } + + if(ISINF(other)) + { + if(decimal_nonzero(self)) + { + ans = _NEW_decimalobj(1, resultsign ? SIGN_NEGINF : SIGN_POSINF, 0); + return ans; + } + return handle_InvalidOperation(self->ob_type, ctx, "0 * (+-)INF", NULL); + } + } + + resultexp = self->exp + other->exp; + shouldround = ctx->rounding_dec == ALWAYS_ROUND; + + if(!decimal_nonzero(self) || !decimal_nonzero(other)) + { + decimalobject *ret; + ans = _NEW_decimalobj(1, resultsign, resultexp); + if(!ans) + return NULL; + + ans->limbs[0] = 0; + goto done; + } + + if(self->ob_size == 1 && self->limbs[0] == 1) + { + ans = _NEW_decimalobj(other->ob_size, resultsign, resultexp); + if(!ans) + return NULL; + + for(i = 0; ilimb_count;i++) + ans->limbs[i] = other->limbs[i]; + goto done; + } + + if(other->ob_size == 1 && other->limbs[0] == 1) + { + ans = _NEW_decimalobj(self->ob_size, resultsign, resultexp); + if(!ans) + return NULL; + + for(i = 0; ilimb_count; i++) + ans->limbs[i] = self->limbs[i]; + + goto done; + } + + max_limbs = (self->ob_size + LOG -1)/ LOG + (other->ob_size + LOG -1)/LOG; + ans = _NEW_decimalobj(max_limbs * LOG, resultsign, resultexp); + if(!ans) + return NULL; + + size = _limb_multiply(self->limbs, self->ob_size, other->limbs, other->ob_size, ans->limbs); + + ans->ob_size = size; + + +done: + if(shouldround) + { + decimalobject *fixed = _decimal_fix(ans, ctx); + Py_DECREF(ans); + return fixed; + } + return ans; } DECIMAL_SPECIAL_2FUNC(decimal_multiply) @@ -4333,12 +4546,13 @@ static PyObject * -context_to_integral(contextobject *self, PyObject *args) +context_to_integral(contextobject *self, PyObject *args, PyObject *kwds) { + static char *kwlist[] = {"a", 0}; PyObject *a, *res; decimalobject *dec_a = NULL; - if (!PyArg_ParseTuple(args, "O:to_integral", &a)) + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O:to_integral", kwlist, &a)) return NULL; dec_a = (decimalobject *)_convert_to_decimal( @@ -4714,7 +4928,7 @@ {"to_eng_string", (PyCFunction)context_to_eng_string, METH_O}, {"to_integral", (PyCFunction)context_to_integral, - METH_O}, + METH_VARARGS | METH_KEYWORDS}, {"to_sci_string", (PyCFunction)context_to_sci_string, METH_VARARGS | METH_KEYWORDS}, {"_ignore_all_flags", (PyCFunction)context_ignore_all_flags, From buildbot at python.org Sun Jun 11 01:54:12 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 10 Jun 2006 23:54:12 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060610235412.253251E400C@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/627 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake,thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 02:20:35 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sun, 11 Jun 2006 02:20:35 +0200 (CEST) Subject: [Python-checkins] r46843 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060611002035.AC8C41E401D@bag.python.org> Author: mateusz.rukowicz Date: Sun Jun 11 02:20:34 2006 New Revision: 46843 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Some bugs repaired. Added conversion of self, that is now 1 + CD("5") works. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sun Jun 11 02:20:34 2006 @@ -1646,7 +1646,7 @@ if (PyErr_Occurred()) return NULL; - if ( (res == 0 && (op == Py_EQ || op == Py_LE || op == Py_GT)) || + if ( (res == 0 && (op == Py_EQ || op == Py_LE || op == Py_GT || op == Py_GE)) || (res == -1 && (op == Py_NE || op == Py_LT || op == Py_LE)) || (res == 1 && (op == Py_NE || op == Py_GT || op == Py_GE))) Py_RETURN_TRUE; @@ -2072,12 +2072,15 @@ return NULL; for (i = 0; i < self->ob_size; i++) { - d = PyInt_FromLong(self->digits[i]); + d = PyInt_FromLong(_limb_get_digit(self->limbs, self->ob_size, i)); /* SLOW */ if (!d) return NULL; PyTuple_SET_ITEM(digits, i, d); } - res = Py_BuildValue("(bOn)", self->sign % 2, digits, self->exp); + if(!ISINF(self)) + res = Py_BuildValue("(bOn)", self->sign % 2, digits, self->exp); + else + res = Py_BuildValue("(bOO)", self->sign % 2, digits, PyString_FromString("F")); Py_DECREF(digits); return res; } @@ -2509,19 +2512,33 @@ in PyNumberMethods slots. As this can take any objects, but other code in this module wants to call some of those methods too, this wrapper is a convenience */ +/* Actually, self might be possibly not decimal I assume that self or other + * will be =] */ #define DECIMAL_SPECIAL_2FUNC(func) \ static PyObject * \ func (PyObject *self, PyObject *other) { \ decimalobject *res; \ + decimalobject *to_decref; \ contextobject *ctx = getcontext(); \ if (!ctx) return NULL; \ + if(PyDecimal_Check(self)) \ + { \ other = _convert_to_decimal(self->ob_type, other, ctx, 0); \ - if (other == NULL || other == Py_NotImplemented) return other; \ + if (other == NULL || other == Py_NotImplemented) return other; \ + to_decref = other; \ + } \ + else \ + { \ + assert(PyDecimal_Check(other)); \ + self = _convert_to_decimal(other->ob_type, self, ctx, 0); \ + to_decref = self; \ + if(self == NULL || self == Py_NotImplemented) return self; \ + } \ res = _do_##func((decimalobject *)self, \ (decimalobject *)other, ctx); \ - Py_DECREF(other); \ + Py_DECREF(to_decref); \ return (PyObject *)res; \ - } + } /* Same for one-argument methods. */ #define DECIMAL_SPECIAL_1FUNC(func) \ @@ -2538,6 +2555,8 @@ _do_decimal_add(decimalobject *self, decimalobject *other, contextobject *ctx) { + assert(PyDecimal_Check(other)); + assert(PyDecimal_Check(self)); int shouldround, sign, negativezero = 0; long exp, oexp; long prec,cmp; From python-checkins at python.org Sun Jun 11 02:40:50 2006 From: python-checkins at python.org (greg.ward) Date: Sun, 11 Jun 2006 02:40:50 +0200 (CEST) Subject: [Python-checkins] r46844 - in python/trunk: Doc/lib/libtextwrap.tex Lib/test/test_textwrap.py Lib/textwrap.py Misc/NEWS Message-ID: <20060611004050.9B3CA1E400D@bag.python.org> Author: greg.ward Date: Sun Jun 11 02:40:49 2006 New Revision: 46844 Modified: python/trunk/Doc/lib/libtextwrap.tex python/trunk/Lib/test/test_textwrap.py python/trunk/Lib/textwrap.py python/trunk/Misc/NEWS Log: Bug #1361643: fix textwrap.dedent() so it handles tabs appropriately, i.e. do *not* expand tabs, but treat them as whitespace that is not equivalent to spaces. Add a couple of test cases. Clarify docs. Modified: python/trunk/Doc/lib/libtextwrap.tex ============================================================================== --- python/trunk/Doc/lib/libtextwrap.tex (original) +++ python/trunk/Doc/lib/libtextwrap.tex Sun Jun 11 02:40:49 2006 @@ -47,12 +47,17 @@ left of the text. \begin{funcdesc}{dedent}{text} -Remove any whitespace that can be uniformly removed from the left -of every line in \var{text}. +Remove any common leading whitespace from every line in \var{text}. -This is typically used to make triple-quoted strings line up with -the left edge of screen/whatever, while still presenting it in the -source code in indented form. +This can be used to make triple-quoted strings line up with the left +edge of the display, while still presenting them in the source code +in indented form. + +Note that tabs and spaces are both treated as whitespace, but they are +not equal: the lines \code{" {} hello"} and \code{"\textbackslash{}thello"} +are considered to have no common leading whitespace. (This behaviour is +new in Python 2.5; older versions of this module incorrectly expanded +tabs before searching for common leading whitespace.) For example: \begin{verbatim} Modified: python/trunk/Lib/test/test_textwrap.py ============================================================================== --- python/trunk/Lib/test/test_textwrap.py (original) +++ python/trunk/Lib/test/test_textwrap.py Sun Jun 11 02:40:49 2006 @@ -460,38 +460,42 @@ # of IndentTestCase! class DedentTestCase(unittest.TestCase): + def assertUnchanged(self, text): + """assert that dedent() has no effect on 'text'""" + self.assertEquals(text, dedent(text)) + def test_dedent_nomargin(self): # No lines indented. text = "Hello there.\nHow are you?\nOh good, I'm glad." - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) # Similar, with a blank line. text = "Hello there.\n\nBoo!" - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) # Some lines indented, but overall margin is still zero. text = "Hello there.\n This is indented." - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) # Again, add a blank line. text = "Hello there.\n\n Boo!\n" - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) def test_dedent_even(self): # All lines indented by two spaces. text = " Hello there.\n How are ya?\n Oh good." expect = "Hello there.\nHow are ya?\nOh good." - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Same, with blank lines. text = " Hello there.\n\n How are ya?\n Oh good.\n" expect = "Hello there.\n\nHow are ya?\nOh good.\n" - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Now indent one of the blank lines. text = " Hello there.\n \n How are ya?\n Oh good.\n" expect = "Hello there.\n\nHow are ya?\nOh good.\n" - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) def test_dedent_uneven(self): # Lines indented unevenly. @@ -505,18 +509,53 @@ while 1: return foo ''' - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Uneven indentation with a blank line. text = " Foo\n Bar\n\n Baz\n" expect = "Foo\n Bar\n\n Baz\n" - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Uneven indentation with a whitespace-only line. text = " Foo\n Bar\n \n Baz\n" expect = "Foo\n Bar\n\n Baz\n" - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) + # dedent() should not mangle internal tabs + def test_dedent_preserve_internal_tabs(self): + text = " hello\tthere\n how are\tyou?" + expect = "hello\tthere\nhow are\tyou?" + self.assertEquals(expect, dedent(text)) + + # make sure that it preserves tabs when it's not making any + # changes at all + self.assertEquals(expect, dedent(expect)) + + # dedent() should not mangle tabs in the margin (i.e. + # tabs and spaces both count as margin, but are *not* + # considered equivalent) + def test_dedent_preserve_margin_tabs(self): + text = " hello there\n\thow are you?" + self.assertUnchanged(text) + + # same effect even if we have 8 spaces + text = " hello there\n\thow are you?" + self.assertUnchanged(text) + + # dedent() only removes whitespace that can be uniformly removed! + text = "\thello there\n\thow are you?" + expect = "hello there\nhow are you?" + self.assertEquals(expect, dedent(text)) + + text = " \thello there\n \thow are you?" + self.assertEquals(expect, dedent(text)) + + text = " \t hello there\n \t how are you?" + self.assertEquals(expect, dedent(text)) + + text = " \thello there\n \t how are you?" + expect = "hello there\n how are you?" + self.assertEquals(expect, dedent(text)) def test_main(): Modified: python/trunk/Lib/textwrap.py ============================================================================== --- python/trunk/Lib/textwrap.py (original) +++ python/trunk/Lib/textwrap.py Sun Jun 11 02:40:49 2006 @@ -317,41 +317,58 @@ # -- Loosely related functionality ------------------------------------- -def dedent(text): - """dedent(text : string) -> string +_whitespace_only_re = re.compile('^[ \t]+$', re.MULTILINE) +_leading_whitespace_re = re.compile('(^[ \t]*)(?:[^ \t\n])', re.MULTILINE) - Remove any whitespace than can be uniformly removed from the left - of every line in `text`. +def dedent(text): + """Remove any common leading whitespace from every line in `text`. - This can be used e.g. to make triple-quoted strings line up with - the left edge of screen/whatever, while still presenting it in the - source code in indented form. - - For example: - - def test(): - # end first line with \ to avoid the empty line! - s = '''\ - hello - world - ''' - print repr(s) # prints ' hello\n world\n ' - print repr(dedent(s)) # prints 'hello\n world\n' + This can be used to make triple-quoted strings line up with the left + edge of the display, while still presenting them in the source code + in indented form. + + Note that tabs and spaces are both treated as whitespace, but they + are not equal: the lines " hello" and "\thello" are + considered to have no common leading whitespace. (This behaviour is + new in Python 2.5; older versions of this module incorrectly + expanded tabs before searching for common leading whitespace.) """ - lines = text.expandtabs().split('\n') + # Look for the longest leading string of spaces and tabs common to + # all lines. margin = None - for line in lines: - content = line.lstrip() - if not content: - continue - indent = len(line) - len(content) + text = _whitespace_only_re.sub('', text) + indents = _leading_whitespace_re.findall(text) + for indent in indents: if margin is None: margin = indent - else: - margin = min(margin, indent) - if margin is not None and margin > 0: - for i in range(len(lines)): - lines[i] = lines[i][margin:] + # Current line more deeply indented than previous winner: + # no change (previous winner is still on top). + elif indent.startswith(margin): + pass + + # Current line consistent with and no deeper than previous winner: + # it's the new winner. + elif margin.startswith(indent): + margin = indent + + # Current line and previous winner have no common whitespace: + # there is no margin. + else: + margin = "" + break - return '\n'.join(lines) + # sanity check (testing/debugging only) + if 0 and margin: + for line in text.split("\n"): + assert not line or line.startswith(margin), \ + "line = %r, margin = %r" % (line, margin) + + if margin: + text = re.sub(r'(?m)^' + margin, '', text) + return text + +if __name__ == "__main__": + #print dedent("\tfoo\n\tbar") + #print dedent(" \thello there\n \t how are you?") + print dedent("Hello there.\n This is indented.") Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sun Jun 11 02:40:49 2006 @@ -145,6 +145,9 @@ Library ------- +- Bug #1361643: fix textwrap.dedent() so it handles tabs appropriately; + clarify docs. + - The wsgiref package has been added to the standard library. - The functions update_wrapper() and wraps() have been added to the functools From python-checkins at python.org Sun Jun 11 02:59:07 2006 From: python-checkins at python.org (talin) Date: Sun, 11 Jun 2006 02:59:07 +0200 (CEST) Subject: [Python-checkins] r46845 - peps/trunk/pep-3101.txt Message-ID: <20060611005907.0A2651E400C@bag.python.org> Author: talin Date: Sun Jun 11 02:59:06 2006 New Revision: 46845 Modified: peps/trunk/pep-3101.txt Log: Lots of changes - added specification for conversions, error handling, complex field specs and general cleanup. Modified: peps/trunk/pep-3101.txt ============================================================================== --- peps/trunk/pep-3101.txt (original) +++ peps/trunk/pep-3101.txt Sun Jun 11 02:59:06 2006 @@ -8,7 +8,7 @@ Content-Type: text/plain Created: 16-Apr-2006 Python-Version: 3.0 -Post-History: 28-Apr-2006 +Post-History: 28-Apr-2006, 6-May-2006, 10-Jun-2006 Abstract @@ -48,7 +48,7 @@ Specification - The specification will consist of 4 parts: + The specification will consist of the following parts: - Specification of a new formatting method to be added to the built-in string class. @@ -60,6 +60,26 @@ - Specification of an API for user-defined formatting classes. + - Specification of how formatting errors are handled. + + Note on string encodings: Since this PEP is being targeted + at Python 3.0, it is assumed that all strings are unicode strings, + and that the use of the word 'string' in the context of this + document will generally refer to a Python 3.0 string, which is + the same as Python 2.x unicode object. + + If it should happen that this functionality is backported to + the 2.x series, then it will be necessary to handle both regular + string as well as unicode objects. All of the function call + interfaces described in this PEP can be used for both strings + and unicode objects, and in all cases there is sufficient + information to be able to properly deduce the output string + type (in other words, there is no need for two separate APIs). + In all cases, the type of the template string dominates - that + is, the result of the conversion will always result in an object + that contains the same representation of characters as the + input template string. + String Methods @@ -75,9 +95,6 @@ identified by its keyword name, so in the above example, 'c' is used to refer to the third argument. - The result of the format call is an object of the same type - (string or unicode) as the format string. - Format Strings @@ -90,32 +107,59 @@ "My name is Fred" - Braces can be escaped using a backslash: + Braces can be escaped by doubling: - "My name is {0} :-\{\}".format('Fred') + "My name is {0} :-{{}}".format('Fred') Which would produce: "My name is Fred :-{}" - + The element within the braces is called a 'field'. Fields consist of a 'field name', which can either be simple or compound, and an optional 'conversion specifier'. + + +Simple and Compound Field Names Simple field names are either names or numbers. If numbers, they must be valid base-10 integers; if names, they must be valid Python identifiers. A number is used to identify a positional argument, while a name is used to identify a keyword argument. + + A compound field name is a combination of multiple simple field + names in an expression: - Compound names are a sequence of simple names seperated by - periods: + "My name is {0.name}".format(file('out.txt')) + + This example shows the use of the 'getattr' or 'dot' operator + in a field expression. The dot operator allows an attribute of + an input value to be specified as the field value. + + The types of expressions that can be used in a compound name + have been deliberately limited in order to prevent potential + security exploits resulting from the ability to place arbitrary + Python expressions inside of strings. Only two operators are + supported, the '.' (getattr) operator, and the '[]' (getitem) + operator. + + An example of the 'getitem' syntax: + + "My name is {0[name]}".format(dict(name='Fred')) + + It should be noted that the use of 'getitem' within a string is + much more limited than its normal use. In the above example, the + string 'name' really is the literal string 'name', not a variable + named 'name'. The rules for parsing an item key are the same as + for parsing a simple name - in other words, if it looks like a + number, then its treated as a number, if it looks like an + identifier, then it is used as a string. + + It is not possible to specify arbitrary dictionary keys from + within a format string. - "My name is {0.name} :-\{\}".format(dict(name='Fred')) - Compound names can be used to access specific dictionary entries, - array elements, or object attributes. In the above example, the - '{0.name}' field refers to the dictionary entry 'name' within - positional argument 0. +Conversion Specifiers Each field can also specify an optional set of 'conversion specifiers' which can be used to adjust the format of that field. @@ -129,53 +173,135 @@ built-in types will recognize a standard set of conversion specifiers. - The conversion specifier consists of a sequence of zero or more - characters, each of which can consist of any printable character - except for a non-escaped '}'. - - Conversion specifiers can themselves contain replacement fields; - this will be described in a later section. Except for this - replacement, the format() method does not attempt to intepret the - conversion specifiers in any way; it merely passes all of the - characters between the first colon ':' and the matching right - brace ('}') to the various underlying formatters (described - later.) + Conversion specifiers can themselves contain replacement fields. + For example, a field whose field width it itself a parameter + could be specified via: + + "{0:{1}}".format(a, b, c) + + Note that the doubled '}' at the end, which would normally be + escaped, is not escaped in this case. The reason is because + the '{{' and '}}' syntax for escapes is only applied when used + *outside* of a format field. Within a format field, the brace + characters always have their normal meaning. + + The syntax for conversion specifiers is open-ended, since except + than doing field replacements, the format() method does not + attempt to interpret them in any way; it merely passes all of the + characters between the first colon and the matching brace to + the various underlying formatter methods. Standard Conversion Specifiers - For most built-in types, the conversion specifiers will be the - same or similar to the existing conversion specifiers used with - the '%' operator. Thus, instead of '%02.2x", you will say - '{0:02.2x}'. - - There are a few differences however: - - - The trailing letter is optional - you don't need to say '2.2d', - you can instead just say '2.2'. If the letter is omitted, a - default will be assumed based on the type of the argument. - The defaults will be as follows: - - string or unicode object: 's' - integer: 'd' - floating-point number: 'f' - all other types: 's' - - - Variable field width specifiers use a nested version of the {} - syntax, allowing the width specifier to be either a positional - or keyword argument: - - "{0:{1}.{2}d}".format(a, b, c) - - - The support for length modifiers (which are ignored by Python - anyway) is dropped. - - For non-built-in types, the conversion specifiers will be specific - to that type. An example is the 'datetime' class, whose - conversion specifiers are identical to the arguments to the - strftime() function: + If an object does not define its own conversion specifiers, a + standard set of conversion specifiers are used. These are similar + in concept to the conversion specifiers used by the existing '%' + operator, however there are also a number of significant + differences. The standard conversion specifiers fall into three + major categories: string conversions, integer conversions and + floating point conversions. + + The general form of a standard conversion specifier is: + + [[fill]align][sign][width][.precision][type] + + The brackets ([]) indicate an optional field. + + Then the optional align flag can be one of the following: + + '<' - Forces the field to be left-aligned within the available + space (This is the default.) + '>' - Forces the field to be right-aligned within the + available space. + '=' - Forces the padding to be placed between immediately + after the sign, if any. This is used for printing fields + in the form '+000000120'. + + Note that unless a minimum field width is defined, the field + width will always be the same size as the data to fill it, so + that the alignment option has no meaning in this case. + + The optional 'fill' character defines the character to be used to + pad the field to the minimum width. The alignment flag must be + supplied if the character is a number other than 0 (otherwise the + character would be interpreted as part of the field width + specifier). A zero fill character without an alignment flag + implies an alignment type of '='. + + The 'sign' field can be one of the following: + + '+' - indicates that a sign should be used for both + positive as well as negative numbers + '-' - indicates that a sign should be used only for negative + numbers (this is the default behaviour) + ' ' - indicates that a leading space should be used on + positive numbers + '()' - indicates that negative numbers should be surrounded + by parentheses + + 'width' is a decimal integer defining the minimum field width. If + not specified, then the field width will be determined by the + content. + + The 'precision' field is a decimal number indicating how many + digits should be displayed after the decimal point. + + Finally, the 'type' determines how the data should be presented. + If the type field is absent, an appropriate type will be assigned + based on the value to be formatted ('d' for integers and longs, + 'g' for floats, and 's' for everything else.) + + The available string conversion types are: + + 's' - String format. Invokes str() on the object. + This is the default conversion specifier type. + 'r' - Repr format. Invokes repr() on the object. + + There are several integer conversion types. All invoke int() on + the object before attempting to format it. + + The available integer conversion types are: + + 'b' - Binary. Outputs the number in base 2. + 'c' - Character. Converts the integer to the corresponding + unicode character before printing. + 'd' - Decimal Integer. Outputs the number in base 10. + 'o' - Octal format. Outputs the number in base 8. + 'x' - Hex format. Outputs the number in base 16, using lower- + case letters for the digits above 9. + 'X' - Hex format. Outputs the number in base 16, using upper- + case letters for the digits above 9. + + There are several floating point conversion types. All invoke + float() on the object before attempting to format it. + + The available floating point conversion types are: + + 'e' - Exponent notation. Prints the number in scientific + notation using the letter 'e' to indicate the exponent. + 'E' - Exponent notation. Same as 'e' except it uses an upper + case 'E' as the separator character. + 'f' - Fixed point. Displays the number as a fixed-point + number. + 'F' - Fixed point. Same as 'f'. + 'g' - General format. This prints the number as a fixed-point + number, unless the number is too large, in which case + it switches to 'e' exponent notation. + 'G' - General format. Same as 'g' except switches to 'E' + if the number gets to large. + 'n' - Number. This is the same as 'g', except that it uses the + current locale setting to insert the appropriate + number separator characters. + '%' - Percentage. Multiplies the number by 100 and displays + in fixed ('f') format, followed by a percent sign. + + Objects are able to define their own conversion specifiers to + replace the standard ones. An example is the 'datetime' class, + whose conversion specifiers might look something like the + arguments to the strftime() function: - "Today is: {0:%a %b %d %H:%M:%S %Y}".format(datetime.now()) + "Today is: {0:a b d H:M:S Y}".format(datetime.now()) Controlling Formatting @@ -224,19 +350,22 @@ API for such an application-specific formatter is up to the application; here are several possible examples: - cell_format( "The total is: {0}", total ) + cell_format("The total is: {0}", total) - TemplateString( "The total is: {0}" ).format( total ) + TemplateString("The total is: {0}").format(total) Creating an application-specific formatter is relatively straight- forward. The string and unicode classes will have a class method called 'cformat' that does all the actual work of formatting; The built-in format() method is just a wrapper that calls cformat. + + The type signature for the cFormat function is as follows: + + cformat(template, format_hook, args, kwargs) The parameters to the cformat function are: - -- The format string (or unicode; the same function handles - both.) + -- The format template string. -- A callable 'format hook', which is called once per field -- A tuple containing the positional arguments -- A dict containing the keyword arguments @@ -251,7 +380,7 @@ will attempt to call the field format hook with the following arguments: - format_hook(value, conversion, buffer) + format_hook(value, conversion) The 'value' field corresponds to the value being formatted, which was retrieved from the arguments using the field name. @@ -260,20 +389,49 @@ field, which will be either a string or unicode object, depending on the type of the original format string. - The 'buffer' argument is a Python array object, either a byte - array or unicode character array. The buffer object will contain - the partially constructed string; the field hook is free to modify - the contents of this buffer if needed. - The field_hook will be called once per field. The field_hook may take one of two actions: + + 1) Return a string or unicode object that is the result + of the formatting operation. - 1) Return False, indicating that the field_hook will not + 2) Return None, indicating that the field_hook will not process this field and the default formatting should be used. This decision should be based on the type of the value object, and the contents of the conversion string. - 2) Append the formatted field to the buffer, and return True. + +Error handling + + The string formatting system has two error handling modes, which + are controlled by the value of a class variable: + + string.strict_format_errors = True + + The 'strict_format_errors' flag defaults to False, or 'lenient' + mode. Setting it to True enables 'strict' mode. The current mode + determines how errors are handled, depending on the type of the + error. + + The types of errors that can occur are: + + 1) Reference to a missing or invalid argument from within a + field specifier. In strict mode, this will raise an exception. + In lenient mode, this will cause the value of the field to be + replaced with the string '?name?', where 'name' will be the + type of error (KeyError, IndexError, or AttributeError). + + So for example: + + >>> string.strict_format_errors = False + >>> print 'Item 2 of argument 0 is: {0[2]}'.format( [0,1] ) + "Item 2 of argument 0 is: ?IndexError?" + + 2) Unused argument. In strict mode, this will raise an exception. + In lenient mode, this will be ignored. + + 3) Exception raised by underlying formatter. These exceptions + are always passed through, regardless of the current mode. Alternate Syntax @@ -325,22 +483,14 @@ Some specific aspects of the syntax warrant additional comments: - 1) The use of the backslash character for escapes. A few people - suggested doubling the brace characters to indicate a literal - brace rather than using backslash as an escape character. This is - also the convention used in the .Net libraries. Here's how the - previously-given example would look with this convention: - - "My name is {0} :-{{}}".format('Fred') - - One problem with this syntax is that it conflicts with the use of - nested braces to allow parameterization of the conversion - specifiers: - - "{0:{1}.{2}}".format(a, b, c) - - (There are alternative solutions, but they are too long to go - into here.) + 1) Backslash character for escapes. The original version of + this PEP used backslash rather than doubling to escape a bracket. + This worked because backslashes in Python string literals that + don't conform to a standard backslash sequence such as '\n' + are left unmodified. However, this caused a certain amount + of confusion, and led to potential situations of multiple + recursive escapes, i.e. '\\\\{' to place a literal backslash + in front of a bracket. 2) The use of the colon character (':') as a separator for conversion specifiers. This was chosen simply because that's From python-checkins at python.org Sun Jun 11 03:08:41 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sun, 11 Jun 2006 03:08:41 +0200 (CEST) Subject: [Python-checkins] r46846 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060611010841.034C91E400C@bag.python.org> Author: mateusz.rukowicz Date: Sun Jun 11 03:08:39 2006 New Revision: 46846 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Some minor changes. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sun Jun 11 03:08:39 2006 @@ -3120,8 +3120,9 @@ buf[0] = (self->sign & 1 ? '-' : '+'); for (i = 0; i < max; i++) { if (i < self->ob_size) - buf[i+1] = self->digits[i] + 48; - else +// buf[i+1] = self->digits[i] + 48; + buf[i+1] = _limb_get_digit(self->limbs, self->ob_size, i) + '0'; /* SLOW */ + else buf[i+1] = '0'; } buf[max+1] = 0; @@ -3163,8 +3164,9 @@ for (i = 0; i < max; i++) { if (i < self->ob_size) - res = res * 10 + self->digits[i]; - else +// res = res * 10 + self->digits[i]; + res = res * 10 + _limb_get_digit(self->limbs, self->ob_size, i); /* SLOW */ + else res *= 10; } if (self->sign & 1) res = -res; @@ -4018,7 +4020,7 @@ tup = PyTuple_New(self->ob_size); if (!tup) return NULL; for (i = 0; i < self->ob_size; i++) - PyTuple_SET_ITEM(tup, i, PyInt_FromLong(self->digits[i])); + PyTuple_SET_ITEM(tup, i, PyInt_FromLong(_limb_get_digit(self->limbs, self->ob_size, i))); /* SLOW */ return tup; } @@ -5090,6 +5092,7 @@ &pyignored)) return NULL; + if (pytraps == NULL) { _traps = PyDict_Copy(PyDecimal_DefaultContext->traps); if (!_traps) goto err; @@ -5201,8 +5204,9 @@ else capitals = capitals & 1; - if (rounding_dec == -1) - rounding_dec = PyDecimal_DefaultContext->rounding_dec; +// if (rounding_dec == -1) + if(rounding_dec != NEVER_ROUND && rounding_dec != ALWAYS_ROUND) /* XXX????*/ + rounding_dec = PyDecimal_DefaultContext->rounding_dec; if (rounding_dec != NEVER_ROUND && rounding_dec != ALWAYS_ROUND) { PyErr_SetString(PyExc_ValueError, "_rounding_decision must be one of " From buildbot at python.org Sun Jun 11 03:41:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 01:41:20 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060611014120.30EF41E400C@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/153 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,fred.drake,greg.ward,martin.v.loewis,skip.montanaro,thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 04:05:13 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sun, 11 Jun 2006 04:05:13 +0200 (CEST) Subject: [Python-checkins] r46847 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060611020513.D0BC51E400F@bag.python.org> Author: mateusz.rukowicz Date: Sun Jun 11 04:05:13 2006 New Revision: 46847 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Added =0; ;-) without this, when compiling with -O *nothing* works properly =]. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sun Jun 11 04:05:13 2006 @@ -3392,7 +3392,7 @@ static decimalobject * _decimal_fromliteral(PyTypeObject *type, char *str, long len, contextobject *ctx) { - char sign; + char sign = 0; long exp = 0; long ndigits; long zero = 0; /* if zero */ From python-checkins at python.org Sun Jun 11 04:12:37 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sun, 11 Jun 2006 04:12:37 +0200 (CEST) Subject: [Python-checkins] r46848 - sandbox/trunk/decimal-c/Makefile Message-ID: <20060611021237.DA1E71E400C@bag.python.org> Author: mateusz.rukowicz Date: Sun Jun 11 04:12:37 2006 New Revision: 46848 Modified: sandbox/trunk/decimal-c/Makefile Log: This is how it should look like. Modified: sandbox/trunk/decimal-c/Makefile ============================================================================== --- sandbox/trunk/decimal-c/Makefile (original) +++ sandbox/trunk/decimal-c/Makefile Sun Jun 11 04:12:37 2006 @@ -3,7 +3,7 @@ PYTHON_25=../../python/python PYTH_DIR=../../python -all: _decimal.o _decimal.so run +all: _decimal.so run module: $(PYTHON_25) setup.py build cp build/lib*/_decimal.so . @@ -17,8 +17,11 @@ debug: _decimal.so gdb $(PYTHON_25) -_decimal.o: _decimal.c decimal.h - gcc -pthread -fno-strict-aliasing -DNDEBUG -g -Wall -Wstrict-prototypes -fPIC -I${PYTH_DIR}/Include -I${PYTH_DIR} -c _decimal.c -o _decimal.o -_decimal.so: _decimal.o - gcc -pthread -shared _decimal.o -o _decimal.so - +#_decimal.o: _decimal.c decimal.h +# gcc -pthread -fno-strict-aliasing -g -O1 -DNEDEBUG -Wall -Wstrict-prototypes -fPIC -I${PYTH_DIR}/Include -I${PYTH_DIR} -c _decimal.c -o _decimal.o +#_decimal.so: _decimal.o +# gcc -pthread -shared _decimal.o -g -o _decimal.so + +_decimal.so: + $(PYTHON_25) setup.py build + cp build/lib*/_decimal.so . From python-checkins at python.org Sun Jun 11 04:42:36 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sun, 11 Jun 2006 04:42:36 +0200 (CEST) Subject: [Python-checkins] r46849 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060611024236.EBBE91E400C@bag.python.org> Author: mateusz.rukowicz Date: Sun Jun 11 04:42:35 2006 New Revision: 46849 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Get rid of some warnings. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sun Jun 11 04:42:35 2006 @@ -655,7 +655,7 @@ static decimalobject * handle_Overflow(PyTypeObject *type, contextobject *ctx, char *expl, long lsign) { - decimalobject *res; + decimalobject *res = 0; long i; HANDLE_ERROR(ctx, S_OVERFLOW, expl, NULL); @@ -813,7 +813,7 @@ static decimalobject * _decimal_increment(decimalobject *self, int round, contextobject *ctx) { - long spot, i; + long i; decimalobject *new; if (ISSPECIAL(self)) { @@ -1297,7 +1297,7 @@ return NULL; } - return _decimal_rescale(self,exp,ctx,rounding,watchexp); + return (PyObject*)_decimal_rescale(self,exp,ctx,rounding,watchexp); } /* Fix the exponents and return a copy with the exponents in bounds. @@ -1413,11 +1413,11 @@ &ctx)) return NULL; - if(ctx == Py_None) + if((PyObject*)ctx == Py_None) if(!(ctx = getcontext())) return NULL; - return _decimal_fix(self,ctx); + return (PyObject*)_decimal_fix(self,ctx); } /* convert something to a Decimal. On failure, either returns NotImplemented or raises an exception, depending @@ -1639,7 +1639,7 @@ other = (decimalobject *)_convert_to_decimal(self->ob_type, (PyObject *)other, ctx, 0); - if (other == Py_NotImplemented) return other; + if ((PyObject*)other == Py_NotImplemented) return (PyObject*)other; res = _do_real_decimal_compare(self, other, ctx); Py_DECREF(other); @@ -1727,8 +1727,8 @@ contextobject *ctx) { decimalobject *ans; - long cmp; - decimalobject *ctx2; + int cmp; + contextobject *ctx2; if(ISSPECIAL(self) || ISSPECIAL(other)) { @@ -2217,8 +2217,8 @@ _do_decimal_str(decimalobject *d, contextobject *context, int engineering) { char *outbuf; - int buflen, i, imax, j; - int leftdigits, dotplace, adjexp; + int buflen, i; + int dotplace, adjexp; int append_E = 0, append_adjexp = 0; long extra_zeros=0; char *p, *end; @@ -2412,7 +2412,7 @@ if (prec == -1) prec = ctx->prec; if (rnd == -1) rnd = ctx->rounding; - return _decimal_round(self, prec, ctx, rnd); + return (PyObject*)_decimal_round(self, prec, ctx, rnd); } static PyObject * @@ -2426,7 +2426,7 @@ ENSURE_CONTEXT("abs", ctx); if (round == -1) round = ctx->rounding; - return _do_decimal_absolute(self, ctx, round); + return (PyObject*)_do_decimal_absolute(self, ctx, round); } static PyMethodDef decimal_methods[] = { @@ -2518,19 +2518,19 @@ static PyObject * \ func (PyObject *self, PyObject *other) { \ decimalobject *res; \ - decimalobject *to_decref; \ + PyObject *to_decref; \ contextobject *ctx = getcontext(); \ if (!ctx) return NULL; \ if(PyDecimal_Check(self)) \ { \ - other = _convert_to_decimal(self->ob_type, other, ctx, 0); \ - if (other == NULL || other == Py_NotImplemented) return other; \ + other = (PyObject*)_convert_to_decimal(self->ob_type, other, ctx, 0); \ + if (other == NULL || other == Py_NotImplemented) return other; \ to_decref = other; \ } \ else \ { \ assert(PyDecimal_Check(other)); \ - self = _convert_to_decimal(other->ob_type, self, ctx, 0); \ + self = (PyObject*)_convert_to_decimal(other->ob_type, self, ctx, 0); \ to_decref = self; \ if(self == NULL || self == Py_NotImplemented) return self; \ } \ @@ -2778,7 +2778,6 @@ return ret; -error: Py_XDECREF(o1); Py_XDECREF(o2); return NULL; @@ -3883,6 +3882,12 @@ return (PyObject *)new; } + if(context) + if(!PyDecimalContext_Check(context)) + { + PyErr_SetString(PyExc_TypeError, "context must be Context type"); + return NULL; + } if (PyDecimal_Check(value)) return (PyObject *)_decimal_get_copy((decimalobject *)value); @@ -3903,7 +3908,7 @@ } if(context) - ctx = context; + ctx = (contextobject*) context; else ctx = getcontext(); @@ -4835,7 +4840,7 @@ static PyObject * context_raise_error(contextobject *self, PyObject *args, PyObject *kwds) { - static char *kwlist[] = {"explanation", 0}; +/* static char *kwlist[] = {"explanation", 0}; */ PyObject *condition = NULL; PyObject *explanation = NULL; PyObject *rest = NULL; From buildbot at python.org Sun Jun 11 06:02:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 04:02:17 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060611040218.01D871E400C@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/629 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: greg.ward Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 07:44:18 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 07:44:18 +0200 (CEST) Subject: [Python-checkins] r46850 - python/trunk/Modules/_bsddb.c Message-ID: <20060611054418.A20AE1E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 07:44:18 2006 New Revision: 46850 Modified: python/trunk/Modules/_bsddb.c Log: Fix Coverity # 146. newDBSequenceObject would deref dbobj, so it can't be NULL. We know it's not NULL from the ParseTuple and DbObject_Check will verify it's not NULL. Modified: python/trunk/Modules/_bsddb.c ============================================================================== --- python/trunk/Modules/_bsddb.c (original) +++ python/trunk/Modules/_bsddb.c Sun Jun 11 07:44:18 2006 @@ -5560,15 +5560,13 @@ static PyObject* DBSequence_construct(PyObject* self, PyObject* args, PyObject* kwargs) { - PyObject* dbobj = NULL; + PyObject* dbobj; int flags = 0; static char* kwnames[] = { "db", "flags", NULL}; if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|i:DBSequence", kwnames, &dbobj, &flags)) return NULL; - if (dbobj == Py_None) - dbobj = NULL; - else if (dbobj && !DBObject_Check(dbobj)) { + if (!DBObject_Check(dbobj)) { makeTypeError("DB", dbobj); return NULL; } From python-checkins at python.org Sun Jun 11 07:45:26 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 07:45:26 +0200 (CEST) Subject: [Python-checkins] r46851 - python/trunk/Modules/_struct.c Message-ID: <20060611054526.4CF7F1E4012@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 07:45:25 2006 New Revision: 46851 Modified: python/trunk/Modules/_struct.c Log: Wrap some long lines Top/Bottom factor out some common expressions Add a XXX comment about widing offset. Modified: python/trunk/Modules/_struct.c ============================================================================== --- python/trunk/Modules/_struct.c (original) +++ python/trunk/Modules/_struct.c Sun Jun 11 07:45:25 2006 @@ -218,7 +218,8 @@ get_wrapped_long(PyObject *v, long *p) { if (get_long(v, p) < 0) { - if (PyLong_Check(v) && PyErr_ExceptionMatches(PyExc_OverflowError)) { + if (PyLong_Check(v) && + PyErr_ExceptionMatches(PyExc_OverflowError)) { PyObject *wrapped; long x; PyErr_Clear(); @@ -1396,23 +1397,17 @@ const char *res = startfrom + code->offset; if (e->format == 's') { v = PyString_FromStringAndSize(res, code->size); - if (v == NULL) - goto fail; - PyTuple_SET_ITEM(result, i++, v); } else if (e->format == 'p') { Py_ssize_t n = *(unsigned char*)res; if (n >= code->size) n = code->size - 1; v = PyString_FromStringAndSize(res + 1, n); - if (v == NULL) - goto fail; - PyTuple_SET_ITEM(result, i++, v); } else { v = e->unpack(res, e); - if (v == NULL) - goto fail; - PyTuple_SET_ITEM(result, i++, v); } + if (v == NULL) + goto fail; + PyTuple_SET_ITEM(result, i++, v); } return result; @@ -1438,7 +1433,8 @@ if (inputstr == NULL || !PyString_Check(inputstr) || PyString_GET_SIZE(inputstr) != soself->s_size) { PyErr_Format(StructError, - "unpack requires a string argument of length %zd", soself->s_size); + "unpack requires a string argument of length %zd", + soself->s_size); return NULL; } return s_unpack_internal(soself, PyString_AS_STRING(inputstr)); @@ -1504,17 +1500,18 @@ s_pack_internal(PyStructObject *soself, PyObject *args, int offset, char* buf) { formatcode *code; + /* XXX(nnorwitz): why does i need to be a local? can we use + the offset parameter or do we need the wider width? */ Py_ssize_t i; memset(buf, '\0', soself->s_size); i = offset; for (code = soself->s_codes; code->fmtdef != NULL; code++) { Py_ssize_t n; - PyObject *v; + PyObject *v = PyTuple_GET_ITEM(args, i++); const formatdef *e = code->fmtdef; char *res = buf + code->offset; if (e->format == 's') { - v = PyTuple_GET_ITEM(args, i++); if (!PyString_Check(v)) { PyErr_SetString(StructError, "argument for 's' must be a string"); @@ -1526,7 +1523,6 @@ if (n > 0) memcpy(res, PyString_AS_STRING(v), n); } else if (e->format == 'p') { - v = PyTuple_GET_ITEM(args, i++); if (!PyString_Check(v)) { PyErr_SetString(StructError, "argument for 'p' must be a string"); @@ -1541,7 +1537,6 @@ n = 255; *res = Py_SAFE_DOWNCAST(n, Py_ssize_t, unsigned char); } else { - v = PyTuple_GET_ITEM(args, i++); if (e->pack(res, v, e) < 0) { if (PyLong_Check(v) && PyErr_ExceptionMatches(PyExc_OverflowError)) PyErr_SetString(StructError, From python-checkins at python.org Sun Jun 11 07:45:47 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 07:45:47 +0200 (CEST) Subject: [Python-checkins] r46852 - python/trunk/Doc/lib/libwsgiref.tex Message-ID: <20060611054547.4C8FF1E4014@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 07:45:47 2006 New Revision: 46852 Modified: python/trunk/Doc/lib/libwsgiref.tex Log: Add versionadded to doc Modified: python/trunk/Doc/lib/libwsgiref.tex ============================================================================== --- python/trunk/Doc/lib/libwsgiref.tex (original) +++ python/trunk/Doc/lib/libwsgiref.tex Sun Jun 11 07:45:47 2006 @@ -5,6 +5,8 @@ \sectionauthor{Phillip J. Eby}{pje at telecommunity.com} \modulesynopsis{WSGI Utilities and Reference Implementation} +\versionadded{2.5} + The Web Server Gateway Interface (WSGI) is a standard interface between web server software and web applications written in Python. Having a standard interface makes it easy to use an application From python-checkins at python.org Sun Jun 11 07:47:15 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 07:47:15 +0200 (CEST) Subject: [Python-checkins] r46853 - python/trunk/Objects/stringobject.c Message-ID: <20060611054715.32F241E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 07:47:14 2006 New Revision: 46853 Modified: python/trunk/Objects/stringobject.c Log: Update doc to make it agree with code. Bottom factor out some common code. Modified: python/trunk/Objects/stringobject.c ============================================================================== --- python/trunk/Objects/stringobject.c (original) +++ python/trunk/Objects/stringobject.c Sun Jun 11 07:47:14 2006 @@ -3099,7 +3099,7 @@ /** End DALKE **/ -/* Matches the end (direction > 0) or start (direction < 0) of self +/* Matches the end (direction >= 0) or start (direction < 0) of self * against substr, using the start and end arguments. Returns * -1 on error, 0 if not found and 1 if found. */ @@ -3131,11 +3131,6 @@ /* startswith */ if (start+slen > len) return 0; - - if (end-start >= slen) - return ! memcmp(str+start, sub, slen); - else - return 0; } else { /* endswith */ if (end-start < slen || start > len) @@ -3143,11 +3138,10 @@ if (end-slen > start) start = end - slen; - if (end-start >= slen) - return ! memcmp(str+start, sub, slen); - else - return 0; } + if (end-start >= slen) + return ! memcmp(str+start, sub, slen); + return 0; } From python-checkins at python.org Sun Jun 11 07:48:15 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 07:48:15 +0200 (CEST) Subject: [Python-checkins] r46854 - python/trunk/Objects/frameobject.c Message-ID: <20060611054815.125151E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 07:48:14 2006 New Revision: 46854 Modified: python/trunk/Objects/frameobject.c Log: f_code can't be NULL based on Frame_New and other code that derefs it. So there doesn't seem to be much point to checking here. Modified: python/trunk/Objects/frameobject.c ============================================================================== --- python/trunk/Objects/frameobject.c (original) +++ python/trunk/Objects/frameobject.c Sun Jun 11 07:48:14 2006 @@ -425,7 +425,7 @@ Py_CLEAR(f->f_exc_traceback); co = f->f_code; - if (co != NULL && co->co_zombieframe == NULL) + if (co->co_zombieframe == NULL) co->co_zombieframe = f; else if (numfree < MAXFREELIST) { ++numfree; @@ -435,7 +435,7 @@ else PyObject_GC_Del(f); - Py_XDECREF(co); + Py_DECREF(co); Py_TRASHCAN_SAFE_END(f) } From buildbot at python.org Sun Jun 11 09:00:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 07:00:55 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060611070055.8E8781E401C@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/603 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 09:26:28 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 09:26:28 +0200 (CEST) Subject: [Python-checkins] r46855 - in python/trunk/Lib: pkgutil.py wsgiref/validate.py Message-ID: <20060611072628.58B351E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 09:26:27 2006 New Revision: 46855 Modified: python/trunk/Lib/pkgutil.py python/trunk/Lib/wsgiref/validate.py Log: Fix errors found by pychecker Modified: python/trunk/Lib/pkgutil.py ============================================================================== --- python/trunk/Lib/pkgutil.py (original) +++ python/trunk/Lib/pkgutil.py Sun Jun 11 09:26:27 2006 @@ -208,6 +208,7 @@ def _reopen(self): if self.file and self.file.closed: + mod_type = self.etc[2] if mod_type==imp.PY_SOURCE: self.file = open(self.filename, 'rU') elif mod_type in (imp.PY_COMPILED, imp.C_EXTENSION): Modified: python/trunk/Lib/wsgiref/validate.py ============================================================================== --- python/trunk/Lib/wsgiref/validate.py (original) +++ python/trunk/Lib/wsgiref/validate.py Sun Jun 11 09:26:27 2006 @@ -248,7 +248,7 @@ def __iter__(self): # We want to make sure __iter__ is called - return IteratorWrapper(self.iterator) + return IteratorWrapper(self.iterator, None) class IteratorWrapper: @@ -313,7 +313,7 @@ continue assert type(environ[key]) is StringType, ( "Environmental variable %s is not a string: %r (value: %r)" - % (type(environ[key]), environ[key])) + % (key, type(environ[key]), environ[key])) assert type(environ['wsgi.version']) is TupleType, ( "wsgi.version should be a tuple (%r)" % environ['wsgi.version']) From python-checkins at python.org Sun Jun 11 09:26:51 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 09:26:51 +0200 (CEST) Subject: [Python-checkins] r46856 - python/trunk/Lib/test/regrtest.py Message-ID: <20060611072651.4D6121E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 09:26:50 2006 New Revision: 46856 Modified: python/trunk/Lib/test/regrtest.py Log: warnings was imported at module scope, no need to import again Modified: python/trunk/Lib/test/regrtest.py ============================================================================== --- python/trunk/Lib/test/regrtest.py (original) +++ python/trunk/Lib/test/regrtest.py Sun Jun 11 09:26:50 2006 @@ -684,7 +684,7 @@ def dash_R_cleanup(fs, ps, pic): import gc, copy_reg - import _strptime, linecache, warnings, dircache + import _strptime, linecache, dircache import urlparse, urllib, urllib2, mimetypes, doctest import struct, filecmp from distutils.dir_util import _path_created From python-checkins at python.org Sun Jun 11 09:27:56 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 09:27:56 +0200 (CEST) Subject: [Python-checkins] r46857 - python/trunk/Lib/pstats.py Message-ID: <20060611072756.A26681E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 09:27:56 2006 New Revision: 46857 Modified: python/trunk/Lib/pstats.py Log: Fix errors found by pychecker. I think these changes are correct, but I'm not sure. Could someone who knows how this module works test it? It can at least start on the cmd line. Modified: python/trunk/Lib/pstats.py ============================================================================== --- python/trunk/Lib/pstats.py (original) +++ python/trunk/Lib/pstats.py Sun Jun 11 09:27:56 2006 @@ -548,8 +548,10 @@ self.prompt = "% " if profile is not None: self.stats = Stats(profile) + self.stream = self.stats.stream else: self.stats = None + self.stream = sys.stdout def generic(self, fn, line): args = line.split() @@ -667,14 +669,15 @@ return None import sys - print >> self.stream, "Welcome to the profile statistics browser." if len(sys.argv) > 1: initprofile = sys.argv[1] else: initprofile = None try: - ProfileBrowser(initprofile).cmdloop() - print >> self.stream, "Goodbye." + browser = ProfileBrowser(initprofile) + print >> browser.stream, "Welcome to the profile statistics browser." + browser.cmdloop() + print >> browser.stream, "Goodbye." except KeyboardInterrupt: pass From fredrik at pythonware.com Sun Jun 11 09:56:45 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 11 Jun 2006 09:56:45 +0200 Subject: [Python-checkins] r46853 - python/trunk/Objects/stringobject.c In-Reply-To: <20060611054715.32F241E400C@bag.python.org> References: <20060611054715.32F241E400C@bag.python.org> Message-ID: neal.norwitz wrote: > Update doc to make it agree with code. not that it matters much in this case, but please don't document im- plementation shortcuts: for the helpers that take direction arguments, >0 is forward and <0 is backwards and that's it. From python-checkins at python.org Sun Jun 11 10:35:15 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 10:35:15 +0200 (CEST) Subject: [Python-checkins] r46858 - in python/trunk/Lib: bsddb/dbrecio.py bsddb/dbtables.py compiler/symbols.py compiler/transformer.py ctypes/util.py Message-ID: <20060611083515.52E991E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 10:35:14 2006 New Revision: 46858 Modified: python/trunk/Lib/bsddb/dbrecio.py python/trunk/Lib/bsddb/dbtables.py python/trunk/Lib/compiler/symbols.py python/trunk/Lib/compiler/transformer.py python/trunk/Lib/ctypes/util.py Log: Fix errors found by pychecker Modified: python/trunk/Lib/bsddb/dbrecio.py ============================================================================== --- python/trunk/Lib/bsddb/dbrecio.py (original) +++ python/trunk/Lib/bsddb/dbrecio.py Sun Jun 11 10:35:14 2006 @@ -75,7 +75,7 @@ dlen = newpos - self.pos - r = self.db.get(key, txn=self.txn, dlen=dlen, doff=self.pos) + r = self.db.get(self.key, txn=self.txn, dlen=dlen, doff=self.pos) self.pos = newpos return r @@ -121,7 +121,7 @@ "Negative size not allowed") elif size < self.pos: self.pos = size - self.db.put(key, "", txn=self.txn, dlen=self.len-size, doff=size) + self.db.put(self.key, "", txn=self.txn, dlen=self.len-size, doff=size) def write(self, s): if self.closed: @@ -131,7 +131,7 @@ self.buflist.append('\0'*(self.pos - self.len)) self.len = self.pos newpos = self.pos + len(s) - self.db.put(key, s, txn=self.txn, dlen=len(s), doff=self.pos) + self.db.put(self.key, s, txn=self.txn, dlen=len(s), doff=self.pos) self.pos = newpos def writelines(self, list): Modified: python/trunk/Lib/bsddb/dbtables.py ============================================================================== --- python/trunk/Lib/bsddb/dbtables.py (original) +++ python/trunk/Lib/bsddb/dbtables.py Sun Jun 11 10:35:14 2006 @@ -32,6 +32,12 @@ # For Python 2.3 from bsddb.db import * +# XXX(nnorwitz): is this correct? DBIncompleteError is conditional in _bsddb.c +try: + DBIncompleteError +except NameError: + class DBIncompleteError(Exception): + pass class TableDBError(StandardError): pass Modified: python/trunk/Lib/compiler/symbols.py ============================================================================== --- python/trunk/Lib/compiler/symbols.py (original) +++ python/trunk/Lib/compiler/symbols.py Sun Jun 11 10:35:14 2006 @@ -191,7 +191,7 @@ self.add_param('[outmost-iterable]') def get_names(self): - keys = Scope.get_names() + keys = Scope.get_names(self) return keys class LambdaScope(FunctionScope): Modified: python/trunk/Lib/compiler/transformer.py ============================================================================== --- python/trunk/Lib/compiler/transformer.py (original) +++ python/trunk/Lib/compiler/transformer.py Sun Jun 11 10:35:14 2006 @@ -729,8 +729,6 @@ def atom(self, nodelist): return self._atom_dispatch[nodelist[0][0]](nodelist) - n.lineno = nodelist[0][2] - return n def atom_lpar(self, nodelist): if nodelist[1][0] == token.RPAR: Modified: python/trunk/Lib/ctypes/util.py ============================================================================== --- python/trunk/Lib/ctypes/util.py (original) +++ python/trunk/Lib/ctypes/util.py Sun Jun 11 10:35:14 2006 @@ -1,5 +1,4 @@ import sys, os -import ctypes # find_library(name) returns the pathname of a library, or None. if os.name == "nt": @@ -41,7 +40,7 @@ elif os.name == "posix": # Andreas Degert's find functions, using gcc, /sbin/ldconfig, objdump - import re, tempfile + import re, tempfile, errno def _findLib_gcc(name): expr = '[^\s]*lib%s\.[^\s]*' % name From buildbot at python.org Sun Jun 11 10:35:48 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 08:35:48 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060611083548.5856C1E400C@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/631 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 16:33:37 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 11 Jun 2006 16:33:37 +0200 (CEST) Subject: [Python-checkins] r46859 - python/trunk/Lib/idlelib/Bindings.py python/trunk/Lib/idlelib/EditorWindow.py python/trunk/Lib/idlelib/PyShell.py python/trunk/Lib/idlelib/ZoomHeight.py python/trunk/Lib/idlelib/buildapp.py python/trunk/Lib/idlelib/configHandler.py python/trunk/Lib/idlelib/keybindingDialog.py python/trunk/Lib/idlelib/macosxSupport.py Message-ID: <20060611143337.E21E31E4007@bag.python.org> Author: ronald.oussoren Date: Sun Jun 11 16:33:36 2006 New Revision: 46859 Added: python/trunk/Lib/idlelib/macosxSupport.py Removed: python/trunk/Lib/idlelib/buildapp.py Modified: python/trunk/Lib/idlelib/Bindings.py python/trunk/Lib/idlelib/EditorWindow.py python/trunk/Lib/idlelib/PyShell.py python/trunk/Lib/idlelib/ZoomHeight.py python/trunk/Lib/idlelib/configHandler.py python/trunk/Lib/idlelib/keybindingDialog.py Log: This patch improves the L&F of IDLE on OSX. The changes are conditionalized on being in an IDLE.app bundle on darwin. This does a slight reorganisation of the menus and adds support for file-open events. Modified: python/trunk/Lib/idlelib/Bindings.py ============================================================================== --- python/trunk/Lib/idlelib/Bindings.py (original) +++ python/trunk/Lib/idlelib/Bindings.py Sun Jun 11 16:33:36 2006 @@ -80,6 +80,32 @@ ]), ] +import sys +if sys.platform == 'darwin' and '.app' in sys.executable: + # Running as a proper MacOS application bundle. This block restructures + # the menus a little to make them conform better to the HIG. + + quitItem = menudefs[0][1][-1] + closeItem = menudefs[0][1][-2] + + # Remove the last 3 items of the file menu: a separator, close window and + # quit. Close window will be reinserted just above the save item, where + # it should be according to the HIG. Quit is in the application menu. + del menudefs[0][1][-3:] + menudefs[0][1].insert(6, closeItem) + + # Remove the 'About' entry from the help menu, it is in the application + # menu + del menudefs[-1][1][0:2] + + menudefs.insert(0, + ('application', [ + ('About IDLE', '<>'), + None, + ('_Preferences....', '<>'), + ])) + + default_keydefs = idleConf.GetCurrentKeySet() del sys Modified: python/trunk/Lib/idlelib/EditorWindow.py ============================================================================== --- python/trunk/Lib/idlelib/EditorWindow.py (original) +++ python/trunk/Lib/idlelib/EditorWindow.py Sun Jun 11 16:33:36 2006 @@ -17,6 +17,7 @@ import PyParse from configHandler import idleConf import aboutDialog, textView, configDialog +import macosxSupport # The default tab setting for a Text widget, in average-width characters. TK_TABWIDTH_DEFAULT = 8 @@ -66,9 +67,18 @@ 'Python%d%d.chm' % sys.version_info[:2]) if os.path.isfile(chmfile): dochome = chmfile + + elif macosxSupport.runningAsOSXApp(): + # documentation is stored inside the python framework + dochome = os.path.join(sys.prefix, + 'Resources/English.lproj/Documentation/index.html') + dochome = os.path.normpath(dochome) if os.path.isfile(dochome): EditorWindow.help_url = dochome + if sys.platform == 'darwin': + # Safari requires real file:-URLs + EditorWindow.help_url = 'file://' + EditorWindow.help_url else: EditorWindow.help_url = "http://www.python.org/doc/current" currentTheme=idleConf.CurrentTheme() @@ -278,6 +288,10 @@ def set_status_bar(self): self.status_bar = self.MultiStatusBar(self.top) + if macosxSupport.runningAsOSXApp(): + # Insert some padding to avoid obscuring some of the statusbar + # by the resize widget. + self.status_bar.set_label('_padding1', ' ', side=RIGHT) self.status_bar.set_label('column', 'Col: ?', side=RIGHT) self.status_bar.set_label('line', 'Ln: ?', side=RIGHT) self.status_bar.pack(side=BOTTOM, fill=X) @@ -301,6 +315,11 @@ ("help", "_Help"), ] + if macosxSupport.runningAsOSXApp(): + del menu_specs[-3] + menu_specs[-2] = ("windows", "_Window") + + def createmenubar(self): mbar = self.menubar self.menudict = menudict = {} @@ -308,6 +327,12 @@ underline, label = prepstr(label) menudict[name] = menu = Menu(mbar, name=name) mbar.add_cascade(label=label, menu=menu, underline=underline) + + if sys.platform == 'darwin' and '.framework' in sys.executable: + # Insert the application menu + menudict['application'] = menu = Menu(mbar, name='apple') + mbar.add_cascade(label='IDLE', menu=menu) + self.fill_menus() self.base_helpmenu_length = self.menudict['help'].index(END) self.reset_help_menu_entries() Modified: python/trunk/Lib/idlelib/PyShell.py ============================================================================== --- python/trunk/Lib/idlelib/PyShell.py (original) +++ python/trunk/Lib/idlelib/PyShell.py Sun Jun 11 16:33:36 2006 @@ -11,6 +11,7 @@ import threading import traceback import types +import macosxSupport import linecache from code import InteractiveInterpreter @@ -777,6 +778,11 @@ ("help", "_Help"), ] + if macosxSupport.runningAsOSXApp(): + del menu_specs[-3] + menu_specs[-2] = ("windows", "_Window") + + # New classes from IdleHistory import History @@ -1371,9 +1377,12 @@ enable_shell = enable_shell or not edit_start # start editor and/or shell windows: root = Tk(className="Idle") + fixwordbreaks(root) root.withdraw() flist = PyShellFileList(root) + macosxSupport.setupApp(root, flist) + if enable_edit: if not (cmd or script): for filename in args: @@ -1381,8 +1390,17 @@ if not args: flist.new() if enable_shell: - if not flist.open_shell(): + shell = flist.open_shell() + if not shell: return # couldn't open shell + + if macosxSupport.runningAsOSXApp() and flist.dict: + # On OSX: when the user has double-clicked on a file that causes + # IDLE to be launched the shell window will open just in front of + # the file she wants to see. Lower the interpreter window when + # there are open files. + shell.top.lower() + shell = flist.pyshell # handle remaining options: if debug: @@ -1403,6 +1421,7 @@ elif script: shell.interp.prepend_syspath(script) shell.interp.execfile(script) + root.mainloop() root.destroy() Modified: python/trunk/Lib/idlelib/ZoomHeight.py ============================================================================== --- python/trunk/Lib/idlelib/ZoomHeight.py (original) +++ python/trunk/Lib/idlelib/ZoomHeight.py Sun Jun 11 16:33:36 2006 @@ -2,6 +2,7 @@ import re import sys +import macosxSupport class ZoomHeight: @@ -29,6 +30,14 @@ if sys.platform == 'win32': newy = 0 newheight = newheight - 72 + + elif macosxSupport.runningAsOSXApp(): + # The '88' below is a magic number that avoids placing the bottom + # of the window below the panel on my machine. I don't know how + # to calculate the correct value for this with tkinter. + newy = 22 + newheight = newheight - newy - 88 + else: #newy = 24 newy = 0 Deleted: /python/trunk/Lib/idlelib/buildapp.py ============================================================================== --- /python/trunk/Lib/idlelib/buildapp.py Sun Jun 11 16:33:36 2006 +++ (empty file) @@ -1,17 +0,0 @@ -# -# After running python setup.py install, run this program from the command -# line like so: -# -# % python2.3 buildapp.py build -# -# A double-clickable IDLE application will be created in the build/ directory. -# - -from bundlebuilder import buildapp - -buildapp( - name="IDLE", - mainprogram="idle.py", - argv_emulation=1, - iconfile="Icons/idle.icns", -) Modified: python/trunk/Lib/idlelib/configHandler.py ============================================================================== --- python/trunk/Lib/idlelib/configHandler.py (original) +++ python/trunk/Lib/idlelib/configHandler.py Sun Jun 11 16:33:36 2006 @@ -20,6 +20,7 @@ import os import sys import string +import macosxSupport from ConfigParser import ConfigParser, NoOptionError, NoSectionError class InvalidConfigType(Exception): pass @@ -495,7 +496,18 @@ return binding def GetCurrentKeySet(self): - return self.GetKeySet(self.CurrentKeys()) + result = self.GetKeySet(self.CurrentKeys()) + + if macosxSupport.runningAsOSXApp(): + # We're using AquaTk, replace all keybingings that use the + # Alt key by ones that use the Option key because the former + # don't work reliably. + for k, v in result.items(): + v2 = [ x.replace(' Author: greg.ward Date: Sun Jun 11 16:42:41 2006 New Revision: 46860 Modified: python/trunk/Doc/lib/liboptparse.tex python/trunk/Misc/NEWS Log: SF #1366250: optparse docs: fix inconsistency in variable name; minor tweaks. Modified: python/trunk/Doc/lib/liboptparse.tex ============================================================================== --- python/trunk/Doc/lib/liboptparse.tex (original) +++ python/trunk/Doc/lib/liboptparse.tex Sun Jun 11 16:42:41 2006 @@ -1197,16 +1197,16 @@ where the input parameters are \begin{description} \item[\code{args}] -the list of arguments to process (\code{sys.argv{[}1:]} by default) +the list of arguments to process (default: \code{sys.argv{[}1:]}) \item[\code{options}] -object to store option arguments in (a new instance of -optparse.Values by default) +object to store option arguments in (default: a new instance of +optparse.Values) \end{description} and the return values are \begin{description} \item[\code{options}] -the same object as was passed in as \code{options}, or the new +the same object that was passed in as \code{options}, or the optparse.Values instance created by \module{optparse} \item[\code{args}] the leftover positional arguments after all options have been @@ -1214,9 +1214,9 @@ \end{description} The most common usage is to supply neither keyword argument. If you -supply a \code{values} object, it will be repeatedly modified with a -\code{setattr()} call for every option argument written to an option -destination, and finally returned by \method{parse{\_}args()}. +supply \code{options}, it will be modified with repeated \code{setattr()} +calls (roughly one for every option argument stored to an option +destination) and returned by \method{parse{\_}args()}. If \method{parse{\_}args()} encounters any errors in the argument list, it calls the OptionParser's \method{error()} method with an appropriate end-user error Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sun Jun 11 16:42:41 2006 @@ -145,6 +145,8 @@ Library ------- +- Bug #1366250: minor optparse documentation error. + - Bug #1361643: fix textwrap.dedent() so it handles tabs appropriately; clarify docs. From python-checkins at python.org Sun Jun 11 18:24:12 2006 From: python-checkins at python.org (greg.ward) Date: Sun, 11 Jun 2006 18:24:12 +0200 (CEST) Subject: [Python-checkins] r46861 - in python/trunk: Lib/optparse.py Lib/test/test_optparse.py Misc/NEWS Message-ID: <20060611162412.826DB1E4007@bag.python.org> Author: greg.ward Date: Sun Jun 11 18:24:11 2006 New Revision: 46861 Modified: python/trunk/Lib/optparse.py python/trunk/Lib/test/test_optparse.py python/trunk/Misc/NEWS Log: Bug #1498146: fix optparse to handle Unicode strings in option help, description, and epilog. Modified: python/trunk/Lib/optparse.py ============================================================================== --- python/trunk/Lib/optparse.py (original) +++ python/trunk/Lib/optparse.py Sun Jun 11 18:24:11 2006 @@ -16,7 +16,7 @@ # Python developers: please do not make changes to this file, since # it is automatically generated from the Optik source code. -__version__ = "1.5.1" +__version__ = "1.5.1+" __all__ = ['Option', 'SUPPRESS_HELP', @@ -75,8 +75,8 @@ # This file was generated from: -# Id: option_parser.py 509 2006-04-20 00:58:24Z gward -# Id: option.py 509 2006-04-20 00:58:24Z gward +# Id: option_parser.py 522 2006-06-11 16:22:03Z gward +# Id: option.py 522 2006-06-11 16:22:03Z gward # Id: help.py 509 2006-04-20 00:58:24Z gward # Id: errors.py 509 2006-04-20 00:58:24Z gward @@ -256,7 +256,7 @@ text_width, initial_indent=indent, subsequent_indent=indent) - + def format_description(self, description): if description: return self._format_text(description) + "\n" @@ -1214,7 +1214,7 @@ """ Declare that you are done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by - it) can be garbage-collected promptly. After calling destroy(), the + it) can be garbage-collected promptly. After calling destroy(), the OptionParser is unusable. """ OptionContainer.destroy(self) @@ -1629,6 +1629,10 @@ result.append(self.format_epilog(formatter)) return "".join(result) + # used by test suite + def _get_encoding(self, file): + return getattr(file, "encoding", sys.getdefaultencoding()) + def print_help(self, file=None): """print_help(file : file = stdout) @@ -1637,7 +1641,8 @@ """ if file is None: file = sys.stdout - file.write(self.format_help()) + encoding = self._get_encoding(file) + file.write(self.format_help().encode(encoding, "replace")) # class OptionParser Modified: python/trunk/Lib/test/test_optparse.py ============================================================================== --- python/trunk/Lib/test/test_optparse.py (original) +++ python/trunk/Lib/test/test_optparse.py Sun Jun 11 18:24:11 2006 @@ -15,7 +15,7 @@ import types import unittest -from cStringIO import StringIO +from StringIO import StringIO from pprint import pprint from test import test_support @@ -164,15 +164,23 @@ expected_error=None): """Assert the parser prints the expected output on stdout.""" save_stdout = sys.stdout + encoding = getattr(save_stdout, 'encoding', None) try: try: sys.stdout = StringIO() + if encoding: + sys.stdout.encoding = encoding self.parser.parse_args(cmdline_args) finally: output = sys.stdout.getvalue() sys.stdout = save_stdout except InterceptedError, err: + self.assert_( + type(output) is types.StringType, + "expected output to be an ordinary string, not %r" + % type(output)) + if output != expected_output: self.fail("expected: \n'''\n" + expected_output + "'''\nbut got \n'''\n" + output + "'''") @@ -1456,6 +1464,10 @@ return InterceptingOptionParser(option_list=options) def assertHelpEquals(self, expected_output): + if type(expected_output) is types.UnicodeType: + encoding = self.parser._get_encoding(sys.stdout) + expected_output = expected_output.encode(encoding, "replace") + save_argv = sys.argv[:] try: # Make optparse believe bar.py is being executed. @@ -1486,6 +1498,27 @@ self.parser = self.make_parser(60) self.assertHelpEquals(_expected_help_short_lines) + def test_help_unicode(self): + self.parser = InterceptingOptionParser(usage=SUPPRESS_USAGE) + self.parser.add_option("-a", action="store_true", help=u"ol\u00E9!") + expect = u"""\ +Options: + -h, --help show this help message and exit + -a ol\u00E9! +""" + self.assertHelpEquals(expect) + + def test_help_unicode_description(self): + self.parser = InterceptingOptionParser(usage=SUPPRESS_USAGE, + description=u"ol\u00E9!") + expect = u"""\ +ol\u00E9! + +Options: + -h, --help show this help message and exit +""" + self.assertHelpEquals(expect) + def test_help_description_groups(self): self.parser.set_description( "This is the program description for %prog. %prog has " Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sun Jun 11 18:24:11 2006 @@ -145,6 +145,9 @@ Library ------- +- Bug #1498146: fix optparse to handle Unicode strings in option help, + description, and epilog. + - Bug #1366250: minor optparse documentation error. - Bug #1361643: fix textwrap.dedent() so it handles tabs appropriately; From python-checkins at python.org Sun Jun 11 19:04:24 2006 From: python-checkins at python.org (thomas.heller) Date: Sun, 11 Jun 2006 19:04:24 +0200 (CEST) Subject: [Python-checkins] r46862 - python/trunk/Modules/_ctypes/callproc.c Message-ID: <20060611170424.4D4DF1E4007@bag.python.org> Author: thomas.heller Date: Sun Jun 11 19:04:22 2006 New Revision: 46862 Modified: python/trunk/Modules/_ctypes/callproc.c Log: Release the GIL during COM method calls, to avoid deadlocks in Python coded COM objects. Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Sun Jun 11 19:04:22 2006 @@ -804,14 +804,20 @@ PyObject *obj; TCHAR *text; + /* We absolutely have to release the GIL during COM method calls, + otherwise we may get a deadlock! + */ + Py_BEGIN_ALLOW_THREADS + hr = pIunk->lpVtbl->QueryInterface(pIunk, &IID_ISupportErrorInfo, (void **)&psei); if (FAILED(hr)) goto failed; + hr = psei->lpVtbl->InterfaceSupportsErrorInfo(psei, riid); psei->lpVtbl->Release(psei); - if (FAILED(hr)) goto failed; + hr = GetErrorInfo(0, &pei); if (hr != S_OK) goto failed; @@ -822,9 +828,10 @@ pei->lpVtbl->GetHelpFile(pei, &helpfile); pei->lpVtbl->GetSource(pei, &source); + pei->lpVtbl->Release(pei); + failed: - if (pei) - pei->lpVtbl->Release(pei); + Py_END_ALLOW_THREADS progid = NULL; ProgIDFromCLSID(&guid, &progid); From buildbot at python.org Sun Jun 11 19:40:13 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 17:40:13 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060611174013.BB2681E400E@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/608 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: greg.ward Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sun Jun 11 20:33:19 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 18:33:19 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060611183319.B3EAF1E4007@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/636 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 21:42:52 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 11 Jun 2006 21:42:52 +0200 (CEST) Subject: [Python-checkins] r46863 - in python/trunk/Lib: ctypes/test/test_anon.py ctypes/test/test_varsize_struct.py idlelib/Bindings.py idlelib/PyShell.py idlelib/configHandler.py idlelib/macosxSupport.py optparse.py test/test_textwrap.py textwrap.py Message-ID: <20060611194252.8AEFB1E4007@bag.python.org> Author: tim.peters Date: Sun Jun 11 21:42:51 2006 New Revision: 46863 Modified: python/trunk/Lib/ctypes/test/test_anon.py python/trunk/Lib/ctypes/test/test_varsize_struct.py python/trunk/Lib/idlelib/Bindings.py python/trunk/Lib/idlelib/PyShell.py python/trunk/Lib/idlelib/configHandler.py python/trunk/Lib/idlelib/macosxSupport.py python/trunk/Lib/optparse.py python/trunk/Lib/test/test_textwrap.py python/trunk/Lib/textwrap.py Log: Whitespace normalization. Modified: python/trunk/Lib/ctypes/test/test_anon.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_anon.py (original) +++ python/trunk/Lib/ctypes/test/test_anon.py Sun Jun 11 21:42:51 2006 @@ -49,7 +49,7 @@ ("_", ANON_U), ("y", c_int)] _anonymous_ = ["_"] - + self.failUnlessEqual(Y.x.offset, 0) self.failUnlessEqual(Y.a.offset, sizeof(c_int)) self.failUnlessEqual(Y.b.offset, sizeof(c_int)) Modified: python/trunk/Lib/ctypes/test/test_varsize_struct.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_varsize_struct.py (original) +++ python/trunk/Lib/ctypes/test/test_varsize_struct.py Sun Jun 11 21:42:51 2006 @@ -110,6 +110,6 @@ array[:5] = [10, 11, 12, 13, 14] self.failUnlessEqual(array[:], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29, 0, 0, 0, 0, 0]) self.failUnlessEqual(varsize_array[0:10], [10, 11, 12, 13, 14, 25, 26, 27, 28, 29]) - + if __name__ == "__main__": unittest.main() Modified: python/trunk/Lib/idlelib/Bindings.py ============================================================================== --- python/trunk/Lib/idlelib/Bindings.py (original) +++ python/trunk/Lib/idlelib/Bindings.py Sun Jun 11 21:42:51 2006 @@ -94,7 +94,7 @@ del menudefs[0][1][-3:] menudefs[0][1].insert(6, closeItem) - # Remove the 'About' entry from the help menu, it is in the application + # Remove the 'About' entry from the help menu, it is in the application # menu del menudefs[-1][1][0:2] Modified: python/trunk/Lib/idlelib/PyShell.py ============================================================================== --- python/trunk/Lib/idlelib/PyShell.py (original) +++ python/trunk/Lib/idlelib/PyShell.py Sun Jun 11 21:42:51 2006 @@ -1396,8 +1396,8 @@ if macosxSupport.runningAsOSXApp() and flist.dict: # On OSX: when the user has double-clicked on a file that causes - # IDLE to be launched the shell window will open just in front of - # the file she wants to see. Lower the interpreter window when + # IDLE to be launched the shell window will open just in front of + # the file she wants to see. Lower the interpreter window when # there are open files. shell.top.lower() Modified: python/trunk/Lib/idlelib/configHandler.py ============================================================================== --- python/trunk/Lib/idlelib/configHandler.py (original) +++ python/trunk/Lib/idlelib/configHandler.py Sun Jun 11 21:42:51 2006 @@ -500,7 +500,7 @@ if macosxSupport.runningAsOSXApp(): # We're using AquaTk, replace all keybingings that use the - # Alt key by ones that use the Option key because the former + # Alt key by ones that use the Option key because the former # don't work reliably. for k, v in result.items(): v2 = [ x.replace(' Author: tim.peters Date: Sun Jun 11 21:43:49 2006 New Revision: 46864 Modified: python/trunk/Lib/idlelib/macosxSupport.py (contents, props changed) Log: Add missing svn:eol-style property to text files. Modified: python/trunk/Lib/idlelib/macosxSupport.py ============================================================================== --- python/trunk/Lib/idlelib/macosxSupport.py (original) +++ python/trunk/Lib/idlelib/macosxSupport.py Sun Jun 11 21:43:49 2006 @@ -1,36 +1,36 @@ -""" -A number of function that enhance IDLE on MacOSX when it used as a normal -GUI application (as opposed to an X11 application). -""" -import sys - -def runningAsOSXApp(): - """ Returns True iff running from the IDLE.app bundle on OSX """ - return (sys.platform == 'darwin' and 'IDLE.app' in sys.argv[0]) - -def addOpenEventSupport(root, flist): - """ - This ensures that the application will respont to open AppleEvents, which - makes is feaseable to use IDLE as the default application for python files. - """ - def doOpenFile(*args): - for fn in args: - flist.open(fn) - - # The command below is a hook in aquatk that is called whenever the app - # receives a file open event. The callback can have multiple arguments, - # one for every file that should be opened. - root.createcommand("::tk::mac::OpenDocument", doOpenFile) - -def hideTkConsole(root): - root.tk.call('console', 'hide') - - -def setupApp(root, flist): - """ - Perform setup for the OSX application bundle. - """ - if not runningAsOSXApp(): return - - hideTkConsole(root) - addOpenEventSupport(root, flist) +""" +A number of function that enhance IDLE on MacOSX when it used as a normal +GUI application (as opposed to an X11 application). +""" +import sys + +def runningAsOSXApp(): + """ Returns True iff running from the IDLE.app bundle on OSX """ + return (sys.platform == 'darwin' and 'IDLE.app' in sys.argv[0]) + +def addOpenEventSupport(root, flist): + """ + This ensures that the application will respont to open AppleEvents, which + makes is feaseable to use IDLE as the default application for python files. + """ + def doOpenFile(*args): + for fn in args: + flist.open(fn) + + # The command below is a hook in aquatk that is called whenever the app + # receives a file open event. The callback can have multiple arguments, + # one for every file that should be opened. + root.createcommand("::tk::mac::OpenDocument", doOpenFile) + +def hideTkConsole(root): + root.tk.call('console', 'hide') + + +def setupApp(root, flist): + """ + Perform setup for the OSX application bundle. + """ + if not runningAsOSXApp(): return + + hideTkConsole(root) + addOpenEventSupport(root, flist) From python-checkins at python.org Sun Jun 11 21:45:57 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 11 Jun 2006 21:45:57 +0200 (CEST) Subject: [Python-checkins] r46865 - python/trunk/Makefile.pre.in Message-ID: <20060611194557.D79B31E4014@bag.python.org> Author: ronald.oussoren Date: Sun Jun 11 21:45:57 2006 New Revision: 46865 Modified: python/trunk/Makefile.pre.in Log: Remove message about using make frameworkinstall, that's no longer necesssary Modified: python/trunk/Makefile.pre.in ============================================================================== --- python/trunk/Makefile.pre.in (original) +++ python/trunk/Makefile.pre.in Sun Jun 11 21:45:57 2006 @@ -653,12 +653,6 @@ # Install the interpreter with $(VERSION) affixed # This goes into $(exec_prefix) altbininstall: $(BUILDPYTHON) - @if test "$(PYTHONFRAMEWORKDIR)" != no-framework; then \ - if test ! -f $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/Current/Resources/Info.plist; then \ - echo 'Framework build: use "make frameworkinstall" in stead of "make install"'; \ - exit 1; \ - fi; \ - fi @for i in $(BINDIR) $(LIBDIR); \ do \ if test ! -d $(DESTDIR)$$i; then \ From python-checkins at python.org Sun Jun 11 22:23:30 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 11 Jun 2006 22:23:30 +0200 (CEST) Subject: [Python-checkins] r46866 - python/trunk/Mac/Makefile.in Message-ID: <20060611202330.3E1421E4007@bag.python.org> Author: ronald.oussoren Date: Sun Jun 11 22:23:29 2006 New Revision: 46866 Modified: python/trunk/Mac/Makefile.in Log: Use configure to substitute the correct prefix instead of hardcoding Modified: python/trunk/Mac/Makefile.in ============================================================================== --- python/trunk/Mac/Makefile.in (original) +++ python/trunk/Mac/Makefile.in Sun Jun 11 22:23:29 2006 @@ -5,7 +5,7 @@ VERSION=@VERSION@ builddir = .. srcdir=@srcdir@ -prefix=/Library/Frameworks/Python.framework/Versions/$(VERSION) +prefix=@prefix@ LIBDEST=$(prefix)/lib/python$(VERSION) RUNSHARED=@RUNSHARED@ BUILDEXE=@BUILDEXEEXT@ From python-checkins at python.org Sun Jun 11 22:24:45 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 11 Jun 2006 22:24:45 +0200 (CEST) Subject: [Python-checkins] r46867 - in python/trunk/Mac: BuildScript/build-installer.py Tools/fixapplepython23.py Message-ID: <20060611202445.F1F5D1E4007@bag.python.org> Author: ronald.oussoren Date: Sun Jun 11 22:24:45 2006 New Revision: 46867 Modified: python/trunk/Mac/BuildScript/build-installer.py python/trunk/Mac/Tools/fixapplepython23.py Log: - Change fixapplepython23.py to ensure that it will run with /usr/bin/python on intel macs. - Fix some minor problems in the installer for OSX Modified: python/trunk/Mac/BuildScript/build-installer.py ============================================================================== --- python/trunk/Mac/BuildScript/build-installer.py (original) +++ python/trunk/Mac/BuildScript/build-installer.py Sun Jun 11 22:24:45 2006 @@ -236,7 +236,7 @@ Mac OS X 10.3 to ensure that you can build new python extensions using that copy of python after installing this version of python. - """ + """, postflight="../Tools/fixapplepython23.py", topdir="/Library/Frameworks/Python.framework", source="/empty-dir", @@ -686,6 +686,9 @@ data = data.replace('$MACOSX_DEPLOYMENT_TARGET', '10.3 or later') data = data.replace('$ARCHITECTURES', "i386, ppc") data = data.replace('$INSTALL_SIZE', installSize()) + + # This one is not handy as a template variable + data = data.replace('$PYTHONFRAMEWORKINSTALLDIR', '/Library/Frameworks/Python.framework') fp = open(outPath, 'wb') fp.write(data) fp.close() @@ -703,7 +706,10 @@ def packageFromRecipe(targetDir, recipe): curdir = os.getcwd() try: - pkgname = recipe['name'] + # The major version (such as 2.5) is included in the pacakge name + # because haveing two version of python installed at the same time is + # common. + pkgname = '%s-%s'%(recipe['name'], getVersion()) srcdir = recipe.get('source') pkgroot = recipe.get('topdir', srcdir) postflight = recipe.get('postflight') @@ -804,7 +810,7 @@ IFPkgFlagComponentDirectory="Contents/Packages", IFPkgFlagPackageList=[ dict( - IFPkgFlagPackageLocation='%s.pkg'%(item['name']), + IFPkgFlagPackageLocation='%s-%s.pkg'%(item['name'], getVersion()), IFPkgFlagPackageSelection='selected' ) for item in PKG_RECIPES @@ -812,6 +818,7 @@ IFPkgFormatVersion=0.10000000149011612, IFPkgFlagBackgroundScaling="proportional", IFPkgFlagBackgroundAlignment="left", + IFPkgFlagAuthorizationAction="RootAuthorization", ) writePlist(pl, path) @@ -859,7 +866,7 @@ else: patchFile(os.path.join('resources', fn), os.path.join(rsrcDir, fn)) - shutil.copy("../../../LICENSE", os.path.join(rsrcDir, 'License.txt')) + shutil.copy("../../LICENSE", os.path.join(rsrcDir, 'License.txt')) def installSize(clear=False, _saved=[]): @@ -1005,7 +1012,7 @@ patchFile('resources/ReadMe.txt', os.path.join(WORKDIR, 'installer', 'ReadMe.txt')) # Ditto for the license file. - shutil.copy('../../../LICENSE', os.path.join(WORKDIR, 'installer', 'License.txt')) + shutil.copy('../../LICENSE', os.path.join(WORKDIR, 'installer', 'License.txt')) fp = open(os.path.join(WORKDIR, 'installer', 'Build.txt'), 'w') print >> fp, "# BUILD INFO" Modified: python/trunk/Mac/Tools/fixapplepython23.py ============================================================================== --- python/trunk/Mac/Tools/fixapplepython23.py (original) +++ python/trunk/Mac/Tools/fixapplepython23.py Sun Jun 11 22:24:45 2006 @@ -94,9 +94,19 @@ else: do_apply = True # First check OS version + if sys.byteorder == 'little': + # All intel macs are fine + print "fixapplypython23: no fix is needed on MacOSX on Intel" + sys.exit(0) + if gestalt.gestalt('sysv') < 0x1030: print 'fixapplepython23: no fix needed on MacOSX < 10.3' sys.exit(0) + + if gestalt.gestalt('sysv') >= 0x1040: + print 'fixapplepython23: no fix needed on MacOSX >= 10.4' + sys.exit(0) + # Test that a framework Python is indeed installed if not os.path.exists(MAKEFILE): print 'fixapplepython23: Python framework does not appear to be installed (?), nothing fixed' From python-checkins at python.org Sun Jun 11 22:25:57 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 22:25:57 +0200 (CEST) Subject: [Python-checkins] r46868 - python/trunk/Lib/test/test_timeout.py python/trunk/Lib/test/test_urllib2net.py python/trunk/Lib/test/test_urllibnet.py Message-ID: <20060611202557.7F6B41E400C@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 22:25:56 2006 New Revision: 46868 Modified: python/trunk/Lib/test/test_timeout.py python/trunk/Lib/test/test_urllib2net.py python/trunk/Lib/test/test_urllibnet.py Log: Try to fix several networking tests. The problem is that if hosts have a search path setup, some of these hosts resolve to the wrong address. By appending a period to the hostname, the hostname should only resolve to what we want it to resolve to. Hopefully this doesn't break different bots. Modified: python/trunk/Lib/test/test_timeout.py ============================================================================== --- python/trunk/Lib/test/test_timeout.py (original) +++ python/trunk/Lib/test/test_timeout.py Sun Jun 11 22:25:56 2006 @@ -100,7 +100,7 @@ def setUp(self): self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.addr_remote = ('www.python.org', 80) + self.addr_remote = ('www.python.org.', 80) self.addr_local = ('127.0.0.1', 25339) def tearDown(self): Modified: python/trunk/Lib/test/test_urllib2net.py ============================================================================== --- python/trunk/Lib/test/test_urllib2net.py (original) +++ python/trunk/Lib/test/test_urllib2net.py Sun Jun 11 22:25:56 2006 @@ -123,7 +123,7 @@ # domain will be spared to serve its defined # purpose. # urllib2.urlopen, "http://www.sadflkjsasadf.com/") - urllib2.urlopen, "http://www.python.invalid/") + urllib2.urlopen, "http://www.python.invalid./") class OtherNetworkTests(unittest.TestCase): Modified: python/trunk/Lib/test/test_urllibnet.py ============================================================================== --- python/trunk/Lib/test/test_urllibnet.py (original) +++ python/trunk/Lib/test/test_urllibnet.py Sun Jun 11 22:25:56 2006 @@ -110,7 +110,7 @@ # domain will be spared to serve its defined # purpose. # urllib.urlopen, "http://www.sadflkjsasadf.com/") - urllib.urlopen, "http://www.python.invalid/") + urllib.urlopen, "http://www.python.invalid./") class urlretrieveNetworkTests(unittest.TestCase): """Tests urllib.urlretrieve using the network.""" From python-checkins at python.org Sun Jun 11 22:42:02 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 22:42:02 +0200 (CEST) Subject: [Python-checkins] r46869 - python/trunk/Lib/test/test_urllib2net.py Message-ID: <20060611204202.A1D971E4007@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 22:42:02 2006 New Revision: 46869 Modified: python/trunk/Lib/test/test_urllib2net.py Log: Try to fix another networking test. The problem is that if hosts have a search path setup, some of these hosts resolve to the wrong address. By appending a period to the hostname, the hostname should only resolve to what we want it to resolve to. Hopefully this doesn't break different bots. Also add more info to failure message to aid debugging test failure. Modified: python/trunk/Lib/test/test_urllib2net.py ============================================================================== --- python/trunk/Lib/test/test_urllib2net.py (original) +++ python/trunk/Lib/test/test_urllib2net.py Sun Jun 11 22:42:02 2006 @@ -160,8 +160,8 @@ "urllib2$") urls = [ # Thanks to Fred for finding these! - 'gopher://gopher.lib.ncsu.edu/11/library/stacks/Alex', - 'gopher://gopher.vt.edu:10010/10/33', + 'gopher://gopher.lib.ncsu.edu./11/library/stacks/Alex', + 'gopher://gopher.vt.edu.:10010/10/33', ] self._test_urls(urls, self._extra_handlers()) @@ -239,7 +239,9 @@ except (IOError, socket.error, OSError), err: debug(err) if expected_err: - self.assert_(isinstance(err, expected_err)) + msg = ("Didn't get expected error(s) %s for %s %s, got %s" % + (expected_err, url, req, err)) + self.assert_(isinstance(err, expected_err), msg) else: buf = f.read() f.close() @@ -259,7 +261,6 @@ return handlers - def test_main(): test_support.requires("network") test_support.run_unittest(URLTimeoutTest, urlopenNetworkTests, From python-checkins at python.org Sun Jun 11 22:46:46 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 22:46:46 +0200 (CEST) Subject: [Python-checkins] r46870 - python/trunk/Lib/test/test_urllib2net.py Message-ID: <20060611204646.D3B9D1E4007@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 22:46:46 2006 New Revision: 46870 Modified: python/trunk/Lib/test/test_urllib2net.py Log: Fix test on PPC64 buildbot. It raised an IOError (really an URLError which derives from an IOError). That seems valid. Env Error includes both OSError and IOError, so this seems like a reasonable fix. Modified: python/trunk/Lib/test/test_urllib2net.py ============================================================================== --- python/trunk/Lib/test/test_urllib2net.py (original) +++ python/trunk/Lib/test/test_urllib2net.py Sun Jun 11 22:46:46 2006 @@ -176,7 +176,7 @@ # XXX bug, should raise URLError #('file://nonsensename/etc/passwd', None, urllib2.URLError) - ('file://nonsensename/etc/passwd', None, (OSError, socket.error)) + ('file://nonsensename/etc/passwd', None, (EnvironmentError, socket.error)) ] self._test_urls(urls, self._extra_handlers()) finally: From python-checkins at python.org Sun Jun 11 22:53:00 2006 From: python-checkins at python.org (tim.peters) Date: Sun, 11 Jun 2006 22:53:00 +0200 (CEST) Subject: [Python-checkins] r46871 - python/trunk/Lib/test/test_wsgiref.py Message-ID: <20060611205300.276101E400F@bag.python.org> Author: tim.peters Date: Sun Jun 11 22:52:59 2006 New Revision: 46871 Modified: python/trunk/Lib/test/test_wsgiref.py Log: compare_generic_iter(): Fixed the failure of test_wsgiref's testFileWrapper when running with -O. test_simple_validation_error still fails under -O. That appears to be because wsgiref's validate.py uses `assert` statements all over the place to check arguments for sanity. That should all be changed (it's not a logical error in the software if a user passes bogus arguments, so this isn't a reasonable use for `assert` -- checking external preconditions should generally raise ValueError or TypeError instead, as appropriate). Modified: python/trunk/Lib/test/test_wsgiref.py ============================================================================== --- python/trunk/Lib/test/test_wsgiref.py (original) +++ python/trunk/Lib/test/test_wsgiref.py Sun Jun 11 22:52:59 2006 @@ -80,7 +80,7 @@ -def compare_generic_iter(make_it,match): +def compare_generic_iter(test, make_it, match): """Utility to compare a generic 2.1/2.2+ iterator with an iterable If running under Python 2.2+, this tests the iterator using iter()/next(), @@ -90,7 +90,7 @@ it = make_it() n = 0 for item in match: - assert it[n]==item + test.assertEqual(it[n], item) n+=1 try: it[n] @@ -106,15 +106,10 @@ else: # Only test iter mode under 2.2+ it = make_it() - assert iter(it) is it + test.assert_(iter(it) is it) for item in match: - assert it.next()==item - try: - it.next() - except StopIteration: - pass - else: - raise AssertionError("Too many items from .next()",it) + test.assertEqual(it.next(), item) + test.assertRaises(StopIteration, it.next) @@ -208,7 +203,7 @@ def make_it(text=text,size=size): return util.FileWrapper(StringIO(text),size) - compare_generic_iter(make_it,match) + compare_generic_iter(self, make_it, match) it = make_it() self.failIf(it.filelike.closed) @@ -440,7 +435,7 @@ h = BaseCGIHandler(None,None,None,{}) h.setup_environ() for key in 'wsgi.url_scheme', 'wsgi.input', 'wsgi.errors': - assert h.environ.has_key(key) + self.assert_(h.environ.has_key(key)) def testScheme(self): h=TestHandler(HTTPS="on"); h.setup_environ() From buildbot at python.org Sun Jun 11 23:16:07 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 21:16:07 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060611211607.2BEBD1E4007@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1119 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 11 23:38:39 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 11 Jun 2006 23:38:39 +0200 (CEST) Subject: [Python-checkins] r46872 - python/trunk/Lib/test/test_socket.py Message-ID: <20060611213839.614081E4007@bag.python.org> Author: neal.norwitz Date: Sun Jun 11 23:38:38 2006 New Revision: 46872 Modified: python/trunk/Lib/test/test_socket.py Log: Get test to pass on S/390. Shout if you think this change is incorrect. Modified: python/trunk/Lib/test/test_socket.py ============================================================================== --- python/trunk/Lib/test/test_socket.py (original) +++ python/trunk/Lib/test/test_socket.py Sun Jun 11 23:38:38 2006 @@ -447,7 +447,12 @@ sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.bind(("0.0.0.0", PORT+1)) name = sock.getsockname() - self.assertEqual(name, ("0.0.0.0", PORT+1)) + # XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate + # it reasonable to get the host's addr in addition to 0.0.0.0. + # At least for eCos. This is required for the S/390 to pass. + my_ip_addr = socket.gethostbyname(socket.gethostname()) + self.assert_(name[0] in ("0.0.0.0", my_ip_addr), '%s invalid' % name[0]) + self.assertEqual(name[1], PORT+1) def testGetSockOpt(self): # Testing getsockopt() From s.percivall at chello.se Mon Jun 12 00:19:59 2006 From: s.percivall at chello.se (Simon Percivall) Date: Mon, 12 Jun 2006 00:19:59 +0200 Subject: [Python-checkins] r46824 - in python/trunk/Modules/_ctypes: _ctypes.c callproc.c ctypes.h libffi/src/x86/darwin.S libffi_msvc/mingwin32.S stgdict.c In-Reply-To: <20060610195148.874F81E400C@bag.python.org> References: <20060610195148.874F81E400C@bag.python.org> Message-ID: <7DBF0A0B-AE7E-46EB-B27C-E757115A95EE@chello.se> On 10 jun 2006, at 21.51, thomas.heller wrote: > Author: thomas.heller > Date: Sat Jun 10 21:51:46 2006 > New Revision: 46824 > > Removed: > python/trunk/Modules/_ctypes/libffi_msvc/mingwin32.S > Modified: > python/trunk/Modules/_ctypes/_ctypes.c > python/trunk/Modules/_ctypes/callproc.c > python/trunk/Modules/_ctypes/ctypes.h > python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S > python/trunk/Modules/_ctypes/stgdict.c > Log: > Upgrade to ctypes version 0.9.9.7. > > > > Modified: python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S > ====================================================================== > ======== > --- python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S (original) > +++ python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S Sat Jun 10 > 21:51:46 2006 > @@ -193,3 +193,198 @@ > #endif /* ifndef __x86_64__ */ > > #endif /* defined __i386__ */ > +#ifdef __i386__ > > The darwin.S checkin seems to be an oopsie!. The entire contents is duplicated in the file, leading to compile error. //Simon From buildbot at python.org Mon Jun 12 01:03:22 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 11 Jun 2006 23:03:22 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060611230322.D14001E4007@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/337 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 12 04:05:56 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:05:56 +0200 (CEST) Subject: [Python-checkins] r46873 - python/trunk/Modules/_bsddb.c Message-ID: <20060612020556.CAAD91E4007@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:05:55 2006 New Revision: 46873 Modified: python/trunk/Modules/_bsddb.c Log: Cleanup Py_ssize_t a little (get rid of second #ifdef) Modified: python/trunk/Modules/_bsddb.c ============================================================================== --- python/trunk/Modules/_bsddb.c (original) +++ python/trunk/Modules/_bsddb.c Mon Jun 12 04:05:55 2006 @@ -103,7 +103,8 @@ #if (PY_VERSION_HEX < 0x02050000) -#define Py_ssize_t int +typedef int Py_ssize_t; +typedef inquiry lenfunc; #endif #ifdef WITH_THREAD @@ -5153,11 +5154,7 @@ static PyMappingMethods DB_mapping = { -#if (PY_VERSION_HEX < 0x02050000) (inquiry)DB_length, /*mp_length*/ -#else - (lenfunc)DB_length, /*mp_length*/ -#endif (binaryfunc)DB_subscript, /*mp_subscript*/ (objobjargproc)DB_ass_sub, /*mp_ass_subscript*/ }; From python-checkins at python.org Mon Jun 12 04:06:18 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:06:18 +0200 (CEST) Subject: [Python-checkins] r46874 - python/trunk/Modules/_elementtree.c Message-ID: <20060612020618.597331E400C@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:06:17 2006 New Revision: 46874 Modified: python/trunk/Modules/_elementtree.c Log: Fix some Py_ssize_t issues Modified: python/trunk/Modules/_elementtree.c ============================================================================== --- python/trunk/Modules/_elementtree.c (original) +++ python/trunk/Modules/_elementtree.c Mon Jun 12 04:06:17 2006 @@ -717,7 +717,8 @@ LOCAL(int) checkpath(PyObject* tag) { - int i, check = 1; + Py_ssize_t i; + int check = 1; /* check if a tag contains an xpath character */ @@ -1180,7 +1181,7 @@ element_setslice(PyObject* self_, Py_ssize_t start, Py_ssize_t end, PyObject* item) { ElementObject* self = (ElementObject*) self_; - int i, new, old; + Py_ssize_t i, new, old; PyObject* recycle = NULL; if (!self->extra) @@ -1426,7 +1427,7 @@ PyObject* data; /* data collector (string or list), or NULL */ PyObject* stack; /* element stack */ - int index; /* current stack size (0=empty) */ + Py_ssize_t index; /* current stack size (0=empty) */ /* element tracing */ PyObject* events; /* list of events, or NULL if not collecting */ @@ -1606,7 +1607,7 @@ PyString_CheckExact(data) && PyString_GET_SIZE(data) == 1) { /* expat often generates single character data sections; handle the most common case by resizing the existing string... */ - int size = PyString_GET_SIZE(self->data); + Py_ssize_t size = PyString_GET_SIZE(self->data); if (_PyString_Resize(&self->data, size + 1) < 0) return NULL; PyString_AS_STRING(self->data)[size] = PyString_AS_STRING(data)[0]; @@ -2456,7 +2457,7 @@ { /* activate element event reporting */ - int i; + Py_ssize_t i; TreeBuilderObject* target; PyObject* events; /* event collector */ From python-checkins at python.org Mon Jun 12 04:06:43 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:06:43 +0200 (CEST) Subject: [Python-checkins] r46875 - python/trunk/Modules/_tkinter.c python/trunk/Modules/bz2module.c python/trunk/Modules/dlmodule.c Message-ID: <20060612020643.68B3B1E400C@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:06:42 2006 New Revision: 46875 Modified: python/trunk/Modules/_tkinter.c python/trunk/Modules/bz2module.c python/trunk/Modules/dlmodule.c Log: Fix some Py_ssize_t issues Modified: python/trunk/Modules/_tkinter.c ============================================================================== --- python/trunk/Modules/_tkinter.c (original) +++ python/trunk/Modules/_tkinter.c Mon Jun 12 04:06:42 2006 @@ -932,12 +932,13 @@ #ifdef Py_USING_UNICODE else if (PyUnicode_Check(value)) { Py_UNICODE *inbuf = PyUnicode_AS_UNICODE(value); - int size = PyUnicode_GET_SIZE(value); + Py_ssize_t size = PyUnicode_GET_SIZE(value); /* This #ifdef assumes that Tcl uses UCS-2. See TCL_UTF_MAX test above. */ #if defined(Py_UNICODE_WIDE) && TCL_UTF_MAX == 3 Tcl_UniChar *outbuf; - int i; + Py_ssize_t i; + assert(size < size * sizeof(Tcl_UniChar)); outbuf = (Tcl_UniChar*)ckalloc(size * sizeof(Tcl_UniChar)); if (!outbuf) { PyErr_NoMemory(); Modified: python/trunk/Modules/bz2module.c ============================================================================== --- python/trunk/Modules/bz2module.c (original) +++ python/trunk/Modules/bz2module.c Mon Jun 12 04:06:42 2006 @@ -1570,7 +1570,7 @@ } } - _PyString_Resize(&ret, (int)(BZS_TOTAL_OUT(bzs) - totalout)); + _PyString_Resize(&ret, (Py_ssize_t)(BZS_TOTAL_OUT(bzs) - totalout)); RELEASE_LOCK(self); return ret; @@ -1636,7 +1636,7 @@ } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)(BZS_TOTAL_OUT(bzs) - totalout)); + _PyString_Resize(&ret, (Py_ssize_t)(BZS_TOTAL_OUT(bzs) - totalout)); RELEASE_LOCK(self); return ret; @@ -1860,7 +1860,7 @@ } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)(BZS_TOTAL_OUT(bzs) - totalout)); + _PyString_Resize(&ret, (Py_ssize_t)(BZS_TOTAL_OUT(bzs) - totalout)); RELEASE_LOCK(self); return ret; @@ -2069,7 +2069,7 @@ } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)BZS_TOTAL_OUT(bzs)); + _PyString_Resize(&ret, (Py_ssize_t)BZS_TOTAL_OUT(bzs)); BZ2_bzCompressEnd(bzs); return ret; @@ -2148,7 +2148,7 @@ } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)BZS_TOTAL_OUT(bzs)); + _PyString_Resize(&ret, (Py_ssize_t)BZS_TOTAL_OUT(bzs)); BZ2_bzDecompressEnd(bzs); return ret; Modified: python/trunk/Modules/dlmodule.c ============================================================================== --- python/trunk/Modules/dlmodule.c (original) +++ python/trunk/Modules/dlmodule.c Mon Jun 12 04:06:42 2006 @@ -77,8 +77,8 @@ long, long, long, long, long); long alist[10]; long res; - int i; - int n = PyTuple_Size(args); + Py_ssize_t i; + Py_ssize_t n = PyTuple_Size(args); if (n < 1) { PyErr_SetString(PyExc_TypeError, "at least a name is needed"); return NULL; From python-checkins at python.org Mon Jun 12 04:07:24 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:07:24 +0200 (CEST) Subject: [Python-checkins] r46876 - python/trunk/Lib/UserString.py Message-ID: <20060612020724.DB3801E4007@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:07:24 2006 New Revision: 46876 Modified: python/trunk/Lib/UserString.py Log: Cleanup: Remove import of types to get StringTypes, we can just use basestring. Modified: python/trunk/Lib/UserString.py ============================================================================== --- python/trunk/Lib/UserString.py (original) +++ python/trunk/Lib/UserString.py Mon Jun 12 04:07:24 2006 @@ -5,14 +5,13 @@ Note: string objects have grown methods in Python 1.6 This module requires Python 1.6 or later. """ -from types import StringTypes import sys __all__ = ["UserString","MutableString"] class UserString: def __init__(self, seq): - if isinstance(seq, StringTypes): + if isinstance(seq, basestring): self.data = seq elif isinstance(seq, UserString): self.data = seq.data[:] @@ -43,12 +42,12 @@ def __add__(self, other): if isinstance(other, UserString): return self.__class__(self.data + other.data) - elif isinstance(other, StringTypes): + elif isinstance(other, basestring): return self.__class__(self.data + other) else: return self.__class__(self.data + str(other)) def __radd__(self, other): - if isinstance(other, StringTypes): + if isinstance(other, basestring): return self.__class__(other + self.data) else: return self.__class__(str(other) + self.data) @@ -163,7 +162,7 @@ start = max(start, 0); end = max(end, 0) if isinstance(sub, UserString): self.data = self.data[:start]+sub.data+self.data[end:] - elif isinstance(sub, StringTypes): + elif isinstance(sub, basestring): self.data = self.data[:start]+sub+self.data[end:] else: self.data = self.data[:start]+str(sub)+self.data[end:] @@ -175,7 +174,7 @@ def __iadd__(self, other): if isinstance(other, UserString): self.data += other.data - elif isinstance(other, StringTypes): + elif isinstance(other, basestring): self.data += other else: self.data += str(other) From python-checkins at python.org Mon Jun 12 04:07:57 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:07:57 +0200 (CEST) Subject: [Python-checkins] r46877 - python/trunk/Parser/tokenizer.c Message-ID: <20060612020757.DFBA71E4007@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:07:57 2006 New Revision: 46877 Modified: python/trunk/Parser/tokenizer.c Log: Don't truncate if size_t is bigger than uint Modified: python/trunk/Parser/tokenizer.c ============================================================================== --- python/trunk/Parser/tokenizer.c (original) +++ python/trunk/Parser/tokenizer.c Mon Jun 12 04:07:57 2006 @@ -1232,7 +1232,7 @@ do { *tp++ = c = tok_nextc(tok); } while (c != EOF && c != '\n' && - (unsigned int)(tp - cbuf + 1) < sizeof(cbuf)); + (size_t)(tp - cbuf + 1) < sizeof(cbuf)); *tp = '\0'; for (cp = tabforms; cp < tabforms + sizeof(tabforms)/sizeof(tabforms[0]); From python-checkins at python.org Mon Jun 12 04:08:41 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:08:41 +0200 (CEST) Subject: [Python-checkins] r46878 - python/trunk/Objects/listobject.c Message-ID: <20060612020841.97E311E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:08:41 2006 New Revision: 46878 Modified: python/trunk/Objects/listobject.c Log: Don't leak the list object if there's an error allocating the item storage. Backport candidate Modified: python/trunk/Objects/listobject.c ============================================================================== --- python/trunk/Objects/listobject.c (original) +++ python/trunk/Objects/listobject.c Mon Jun 12 04:08:41 2006 @@ -108,8 +108,10 @@ op->ob_item = NULL; else { op->ob_item = (PyObject **) PyMem_MALLOC(nbytes); - if (op->ob_item == NULL) + if (op->ob_item == NULL) { + Py_DECREF(op); return PyErr_NoMemory(); + } memset(op->ob_item, 0, nbytes); } op->ob_size = size; From python-checkins at python.org Mon Jun 12 04:09:04 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:09:04 +0200 (CEST) Subject: [Python-checkins] r46879 - python/trunk/configure python/trunk/configure.in Message-ID: <20060612020904.18E551E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:09:03 2006 New Revision: 46879 Modified: python/trunk/configure python/trunk/configure.in Log: Fix typo. Backport if anyone cares. :-) Modified: python/trunk/configure ============================================================================== --- python/trunk/configure (original) +++ python/trunk/configure Mon Jun 12 04:09:03 2006 @@ -1587,7 +1587,7 @@ # On Mac OS X 10.4, defining _POSIX_C_SOURCE or _XOPEN_SOURCE # disables platform specific features beyond repair. # On Mac OS X 10.3, defining _POSIX_C_SOURCE or _XOPEN_SOURCE - # has no effect, don't bother defineing them + # has no effect, don't bother defining them Darwin/[78].*) define_xopen_source=no ;; Modified: python/trunk/configure.in ============================================================================== --- python/trunk/configure.in (original) +++ python/trunk/configure.in Mon Jun 12 04:09:03 2006 @@ -235,7 +235,7 @@ # On Mac OS X 10.4, defining _POSIX_C_SOURCE or _XOPEN_SOURCE # disables platform specific features beyond repair. # On Mac OS X 10.3, defining _POSIX_C_SOURCE or _XOPEN_SOURCE - # has no effect, don't bother defineing them + # has no effect, don't bother defining them Darwin/@<:@78@:>@.*) define_xopen_source=no ;; From python-checkins at python.org Mon Jun 12 04:09:35 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:09:35 +0200 (CEST) Subject: [Python-checkins] r46880 - python/trunk/Python/compile.c Message-ID: <20060612020935.098CB1E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:09:34 2006 New Revision: 46880 Modified: python/trunk/Python/compile.c Log: Fix indentation of case and a Py_ssize_t issue. Modified: python/trunk/Python/compile.c ============================================================================== --- python/trunk/Python/compile.c (original) +++ python/trunk/Python/compile.c Mon Jun 12 04:09:34 2006 @@ -3665,7 +3665,7 @@ assert(s->kind == AugAssign_kind); switch (e->kind) { - case Attribute_kind: + case Attribute_kind: auge = Attribute(e->v.Attribute.value, e->v.Attribute.attr, AugLoad, e->lineno, e->col_offset, c->c_arena); if (auge == NULL) @@ -4195,7 +4195,7 @@ assemble_emit(struct assembler *a, struct instr *i) { int size, arg = 0, ext = 0; - int len = PyString_GET_SIZE(a->a_bytecode); + Py_ssize_t len = PyString_GET_SIZE(a->a_bytecode); char *code; size = instrsize(i); From python-checkins at python.org Mon Jun 12 04:11:19 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:11:19 +0200 (CEST) Subject: [Python-checkins] r46881 - in python/trunk: Include/frameobject.h Misc/NEWS Objects/frameobject.c Python/ceval.c Message-ID: <20060612021119.12BC11E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:11:18 2006 New Revision: 46881 Modified: python/trunk/Include/frameobject.h python/trunk/Misc/NEWS python/trunk/Objects/frameobject.c python/trunk/Python/ceval.c Log: Get rid of f_restricted too. Doc the other 4 ints that were already removed at the NeedForSpeed sprint. Modified: python/trunk/Include/frameobject.h ============================================================================== --- python/trunk/Include/frameobject.h (original) +++ python/trunk/Include/frameobject.h Mon Jun 12 04:11:18 2006 @@ -41,8 +41,6 @@ /* As of 2.3 f_lineno is only valid when tracing is active (i.e. when f_trace is set) -- at other times use PyCode_Addr2Line instead. */ int f_lineno; /* Current line number */ - int f_restricted; /* Flag set if restricted operations - in this scope */ int f_iblock; /* index in f_blockstack */ PyTryBlock f_blockstack[CO_MAXBLOCKS]; /* for try and loop blocks */ PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */ @@ -54,6 +52,8 @@ PyAPI_DATA(PyTypeObject) PyFrame_Type; #define PyFrame_Check(op) ((op)->ob_type == &PyFrame_Type) +#define PyFrame_IsRestricted(f) \ + ((f)->f_builtins != (f)->f_tstate->interp->builtins) PyAPI_FUNC(PyFrameObject *) PyFrame_New(PyThreadState *, PyCodeObject *, PyObject *, PyObject *); Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Mon Jun 12 04:11:18 2006 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Removed 5 integers from C frame objects (PyFrameObject). + f_nlocals, f_ncells, f_nfreevars, f_stack_size, f_restricted. + - Bug #532646: object.__call__() will continue looking for the __call__ attribute on objects until one without one is found. This leads to recursion when you take a class and set its __call__ attribute to an instance of the Modified: python/trunk/Objects/frameobject.c ============================================================================== --- python/trunk/Objects/frameobject.c (original) +++ python/trunk/Objects/frameobject.c Mon Jun 12 04:11:18 2006 @@ -20,7 +20,6 @@ {"f_builtins", T_OBJECT, OFF(f_builtins),RO}, {"f_globals", T_OBJECT, OFF(f_globals), RO}, {"f_lasti", T_INT, OFF(f_lasti), RO}, - {"f_restricted",T_INT, OFF(f_restricted),RO}, {"f_exc_type", T_OBJECT, OFF(f_exc_type)}, {"f_exc_value", T_OBJECT, OFF(f_exc_value)}, {"f_exc_traceback", T_OBJECT, OFF(f_exc_traceback)}, @@ -341,11 +340,18 @@ return 0; } +static PyObject * +frame_getrestricted(PyFrameObject *f, void *closure) +{ + return PyBool_FromLong(PyFrame_IsRestricted(f)); +} + static PyGetSetDef frame_getsetlist[] = { {"f_locals", (getter)frame_getlocals, NULL, NULL}, {"f_lineno", (getter)frame_getlineno, (setter)frame_setlineno, NULL}, {"f_trace", (getter)frame_gettrace, (setter)frame_settrace, NULL}, + {"f_restricted",(getter)frame_getrestricted,NULL, NULL}, {0} }; @@ -664,7 +670,6 @@ f->f_lasti = -1; f->f_lineno = code->co_firstlineno; - f->f_restricted = (builtins != tstate->interp->builtins); f->f_iblock = 0; f->f_stacktop = f->f_valuestack; Modified: python/trunk/Python/ceval.c ============================================================================== --- python/trunk/Python/ceval.c (original) +++ python/trunk/Python/ceval.c Mon Jun 12 04:11:18 2006 @@ -3354,7 +3354,7 @@ PyEval_GetRestricted(void) { PyFrameObject *current_frame = PyEval_GetFrame(); - return current_frame == NULL ? 0 : current_frame->f_restricted; + return current_frame == NULL ? 0 : PyFrame_IsRestricted(current_frame); } int From python-checkins at python.org Mon Jun 12 04:13:22 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:13:22 +0200 (CEST) Subject: [Python-checkins] r46882 - in python/trunk: Lib/test/test_socket.py Lib/test/test_socket_ssl.py Lib/test/test_socketserver.py Lib/test/test_support.py Misc/NEWS Message-ID: <20060612021322.3B7DA1E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:13:21 2006 New Revision: 46882 Modified: python/trunk/Lib/test/test_socket.py python/trunk/Lib/test/test_socket_ssl.py python/trunk/Lib/test/test_socketserver.py python/trunk/Lib/test/test_support.py python/trunk/Misc/NEWS Log: Fix the socket tests so they can be run concurrently. Backport candidate Modified: python/trunk/Lib/test/test_socket.py ============================================================================== --- python/trunk/Lib/test/test_socket.py (original) +++ python/trunk/Lib/test/test_socket.py Mon Jun 12 04:13:21 2006 @@ -21,7 +21,8 @@ def setUp(self): self.serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - self.serv.bind((HOST, PORT)) + global PORT + PORT = test_support.bind_port(self.serv, HOST, PORT) self.serv.listen(1) def tearDown(self): @@ -33,7 +34,8 @@ def setUp(self): self.serv = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - self.serv.bind((HOST, PORT)) + global PORT + PORT = test_support.bind_port(self.serv, HOST, PORT) def tearDown(self): self.serv.close() Modified: python/trunk/Lib/test/test_socket_ssl.py ============================================================================== --- python/trunk/Lib/test/test_socket_ssl.py (original) +++ python/trunk/Lib/test/test_socket_ssl.py Mon Jun 12 04:13:21 2006 @@ -71,7 +71,7 @@ return # Some random port to connect to. - PORT = 9934 + PORT = [9934] listener_ready = threading.Event() listener_gone = threading.Event() @@ -82,7 +82,7 @@ # know the socket is gone. def listener(): s = socket.socket() - s.bind(('', PORT)) + PORT[0] = test_support.bind_port(s, '', PORT[0]) s.listen(5) listener_ready.set() s.accept() @@ -92,7 +92,7 @@ def connector(): listener_ready.wait() s = socket.socket() - s.connect(('localhost', PORT)) + s.connect(('localhost', PORT[0])) listener_gone.wait() try: ssl_sock = socket.ssl(s) Modified: python/trunk/Lib/test/test_socketserver.py ============================================================================== --- python/trunk/Lib/test/test_socketserver.py (original) +++ python/trunk/Lib/test/test_socketserver.py Mon Jun 12 04:13:21 2006 @@ -6,6 +6,7 @@ from SocketServer import * import socket +import errno import select import time import threading @@ -77,6 +78,11 @@ pass if verbose: print "thread: creating server" svr = svrcls(self.__addr, self.__hdlrcls) + # pull the address out of the server in case it changed + # this can happen if another process is using the port + addr = getattr(svr, 'server_address') + if addr: + self.__addr = addr if verbose: print "thread: serving three times" svr.serve_a_few() if verbose: print "thread: done" @@ -136,7 +142,25 @@ t.join() if verbose: print "done" -tcpservers = [TCPServer, ThreadingTCPServer] +class ForgivingTCPServer(TCPServer): + # prevent errors if another process is using the port we want + def server_bind(self): + host, default_port = self.server_address + # this code shamelessly stolen from test.test_support + # the ports were changed to protect the innocent + import sys + for port in [default_port, 3434, 8798, 23833]: + try: + self.server_address = host, port + TCPServer.server_bind(self) + break + except socket.error, (err, msg): + if err != errno.EADDRINUSE: + raise + print >>sys.__stderr__, \ + ' WARNING: failed to listen on port %d, trying another' % port + +tcpservers = [ForgivingTCPServer, ThreadingTCPServer] if hasattr(os, 'fork') and os.name not in ('os2',): tcpservers.append(ForkingTCPServer) udpservers = [UDPServer, ThreadingUDPServer] Modified: python/trunk/Lib/test/test_support.py ============================================================================== --- python/trunk/Lib/test/test_support.py (original) +++ python/trunk/Lib/test/test_support.py Mon Jun 12 04:13:21 2006 @@ -89,6 +89,24 @@ msg = "Use of the `%s' resource not enabled" % resource raise ResourceDenied(msg) +def bind_port(sock, host='', preferred_port=54321): + """Try to bind the sock to a port. If we are running multiple + tests and we don't try multiple ports, the test can fails. This + makes the test more robust.""" + + import socket, errno + # some random ports that hopefully no one is listening on. + for port in [preferred_port, 9907, 10243, 32999]: + try: + sock.bind((host, port)) + return port + except socket.error, (err, msg): + if err != errno.EADDRINUSE: + raise + print >>sys.__stderr__, \ + ' WARNING: failed to listen on port %d, trying another' % port + raise TestFailed, 'unable to find port to listen on' + FUZZ = 1e-6 def fcmp(x, y): # fuzzy comparison function Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Mon Jun 12 04:13:21 2006 @@ -12,6 +12,8 @@ Core and builtins ----------------- +- Fix the socket tests so they can be run concurrently. + - Removed 5 integers from C frame objects (PyFrameObject). f_nlocals, f_ncells, f_nfreevars, f_stack_size, f_restricted. From python-checkins at python.org Mon Jun 12 04:16:11 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 04:16:11 +0200 (CEST) Subject: [Python-checkins] r46883 - python/trunk/Python/ceval.c Message-ID: <20060612021611.0B2291E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 04:16:10 2006 New Revision: 46883 Modified: python/trunk/Python/ceval.c Log: i and j are initialized below when used. No need to do it twice Modified: python/trunk/Python/ceval.c ============================================================================== --- python/trunk/Python/ceval.c (original) +++ python/trunk/Python/ceval.c Mon Jun 12 04:16:10 2006 @@ -2766,7 +2766,7 @@ /* Allocate and initialize storage for cell vars, and copy free vars into frame. This isn't too efficient right now. */ if (PyTuple_GET_SIZE(co->co_cellvars)) { - int i = 0, j = 0, nargs, found; + int i, j, nargs, found; char *cellname, *argname; PyObject *c; From python-checkins at python.org Mon Jun 12 05:05:07 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 05:05:07 +0200 (CEST) Subject: [Python-checkins] r46884 - python/trunk/Lib/doctest.py Message-ID: <20060612030507.C7A311E4012@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 05:05:03 2006 New Revision: 46884 Modified: python/trunk/Lib/doctest.py Log: Remove unused import Modified: python/trunk/Lib/doctest.py ============================================================================== --- python/trunk/Lib/doctest.py (original) +++ python/trunk/Lib/doctest.py Mon Jun 12 05:05:03 2006 @@ -95,7 +95,7 @@ import __future__ -import sys, traceback, inspect, linecache, os, re, types +import sys, traceback, inspect, linecache, os, re import unittest, difflib, pdb, tempfile import warnings from StringIO import StringIO From python-checkins at python.org Mon Jun 12 05:05:41 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 05:05:41 +0200 (CEST) Subject: [Python-checkins] r46885 - python/trunk/Modules/_sre.c python/trunk/Modules/sre.h Message-ID: <20060612030541.67CFC1E401B@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 05:05:40 2006 New Revision: 46885 Modified: python/trunk/Modules/_sre.c python/trunk/Modules/sre.h Log: Impl ssize_t Modified: python/trunk/Modules/_sre.c ============================================================================== --- python/trunk/Modules/_sre.c (original) +++ python/trunk/Modules/_sre.c Mon Jun 12 05:05:40 2006 @@ -39,6 +39,8 @@ static char copyright[] = " SRE 2.2.2 Copyright (c) 1997-2002 by Secret Labs AB "; +#define PY_SSIZE_T_CLEAN + #include "Python.h" #include "structmember.h" /* offsetof */ @@ -261,9 +263,9 @@ } static int -data_stack_grow(SRE_STATE* state, int size) +data_stack_grow(SRE_STATE* state, Py_ssize_t size) { - int minsize, cursize; + Py_ssize_t minsize, cursize; minsize = state->data_stack_base+size; cursize = state->data_stack_size; if (cursize < minsize) { @@ -335,7 +337,7 @@ { /* check if pointer is at given position */ - int thisp, thatp; + Py_ssize_t thisp, thatp; switch (at) { @@ -476,7 +478,7 @@ case SRE_OP_BIGCHARSET: /* <256 blockindices> */ { - int count, block; + Py_ssize_t count, block; count = *(set++); if (sizeof(SRE_CODE) == 2) { @@ -510,15 +512,15 @@ } } -LOCAL(int) SRE_MATCH(SRE_STATE* state, SRE_CODE* pattern); +LOCAL(Py_ssize_t) SRE_MATCH(SRE_STATE* state, SRE_CODE* pattern); -LOCAL(int) -SRE_COUNT(SRE_STATE* state, SRE_CODE* pattern, int maxcount) +LOCAL(Py_ssize_t) +SRE_COUNT(SRE_STATE* state, SRE_CODE* pattern, Py_ssize_t maxcount) { SRE_CODE chr; SRE_CHAR* ptr = (SRE_CHAR *)state->ptr; SRE_CHAR* end = (SRE_CHAR *)state->end; - int i; + Py_ssize_t i; /* adjust end */ if (maxcount < end - ptr && maxcount != 65535) @@ -608,7 +610,7 @@ SRE_CHAR* end = state->end; SRE_CHAR* ptr = state->ptr; - int i; + Py_ssize_t i; /* check minimal length */ if (pattern[3] && (end - ptr) < pattern[3]) @@ -785,13 +787,13 @@ while (0) /* gcc doesn't like labels at end of scopes */ \ typedef struct { - int last_ctx_pos; - int jump; + Py_ssize_t last_ctx_pos; + Py_ssize_t jump; SRE_CHAR* ptr; SRE_CODE* pattern; - int count; - int lastmark; - int lastindex; + Py_ssize_t count; + Py_ssize_t lastmark; + Py_ssize_t lastindex; union { SRE_CODE chr; SRE_REPEAT* rep; @@ -800,13 +802,13 @@ /* check if string matches the given pattern. returns <0 for error, 0 for failure, and 1 for success */ -LOCAL(int) +LOCAL(Py_ssize_t) SRE_MATCH(SRE_STATE* state, SRE_CODE* pattern) { SRE_CHAR* end = (SRE_CHAR *)state->end; - int alloc_pos, ctx_pos = -1; - int i, ret = 0; - int jump; + Py_ssize_t alloc_pos, ctx_pos = -1; + Py_ssize_t i, ret = 0; + Py_ssize_t jump; SRE_MATCH_CONTEXT* ctx; SRE_MATCH_CONTEXT* nextctx; @@ -851,7 +853,7 @@ state->mark array. If it is increased by more than 1, the intervening marks must be set to NULL to signal that these marks have not been encountered. */ - int j = state->lastmark + 1; + Py_ssize_t j = state->lastmark + 1; while (j < i) state->mark[j++] = NULL; state->lastmark = i; @@ -1037,7 +1039,7 @@ string. check if the rest of the pattern matches, and backtrack if not. */ - if (ctx->count < (int) ctx->pattern[1]) + if (ctx->count < (Py_ssize_t) ctx->pattern[1]) RETURN_FAILURE; if (ctx->pattern[ctx->pattern[0]] == SRE_OP_SUCCESS) { @@ -1053,12 +1055,12 @@ the rest of the pattern cannot possibly match */ ctx->u.chr = ctx->pattern[ctx->pattern[0]+1]; for (;;) { - while (ctx->count >= (int) ctx->pattern[1] && + while (ctx->count >= (Py_ssize_t) ctx->pattern[1] && (ctx->ptr >= end || *ctx->ptr != ctx->u.chr)) { ctx->ptr--; ctx->count--; } - if (ctx->count < (int) ctx->pattern[1]) + if (ctx->count < (Py_ssize_t) ctx->pattern[1]) break; state->ptr = ctx->ptr; DO_JUMP(JUMP_REPEAT_ONE_1, jump_repeat_one_1, @@ -1076,7 +1078,7 @@ } else { /* general case */ - while (ctx->count >= (int) ctx->pattern[1]) { + while (ctx->count >= (Py_ssize_t) ctx->pattern[1]) { state->ptr = ctx->ptr; DO_JUMP(JUMP_REPEAT_ONE_2, jump_repeat_one_2, ctx->pattern+ctx->pattern[0]); @@ -1116,7 +1118,7 @@ ret = SRE_COUNT(state, ctx->pattern+3, ctx->pattern[1]); RETURN_ON_ERROR(ret); DATA_LOOKUP_AT(SRE_MATCH_CONTEXT, ctx, ctx_pos); - if (ret < (int) ctx->pattern[1]) + if (ret < (Py_ssize_t) ctx->pattern[1]) /* didn't match minimum number of times */ RETURN_FAILURE; /* advance past minimum matches of repeat */ @@ -1132,8 +1134,8 @@ } else { /* general case */ LASTMARK_SAVE(); - while ((int)ctx->pattern[2] == 65535 - || ctx->count <= (int)ctx->pattern[2]) { + while ((Py_ssize_t)ctx->pattern[2] == 65535 + || ctx->count <= (Py_ssize_t)ctx->pattern[2]) { state->ptr = ctx->ptr; DO_JUMP(JUMP_MIN_REPEAT_ONE,jump_min_repeat_one, ctx->pattern+ctx->pattern[0]); @@ -1312,7 +1314,7 @@ ctx->ptr, ctx->pattern[0])); i = ctx->pattern[0]; { - int groupref = i+i; + Py_ssize_t groupref = i+i; if (groupref >= state->lastmark) { RETURN_FAILURE; } else { @@ -1336,7 +1338,7 @@ ctx->ptr, ctx->pattern[0])); i = ctx->pattern[0]; { - int groupref = i+i; + Py_ssize_t groupref = i+i; if (groupref >= state->lastmark) { RETURN_FAILURE; } else { @@ -1361,7 +1363,7 @@ /* codeyes codeno ... */ i = ctx->pattern[0]; { - int groupref = i+i; + Py_ssize_t groupref = i+i; if (groupref >= state->lastmark) { ctx->pattern += ctx->pattern[1]; break; @@ -1474,14 +1476,14 @@ return ret; /* should never get here */ } -LOCAL(int) +LOCAL(Py_ssize_t) SRE_SEARCH(SRE_STATE* state, SRE_CODE* pattern) { SRE_CHAR* ptr = (SRE_CHAR *)state->start; SRE_CHAR* end = (SRE_CHAR *)state->end; - int status = 0; - int prefix_len = 0; - int prefix_skip = 0; + Py_ssize_t status = 0; + Py_ssize_t prefix_len = 0; + Py_ssize_t prefix_skip = 0; SRE_CODE* prefix = NULL; SRE_CODE* charset = NULL; SRE_CODE* overlap = NULL; @@ -1523,7 +1525,7 @@ if (prefix_len > 1) { /* pattern starts with a known prefix. use the overlap table to skip forward as fast as we possibly can */ - int i = 0; + Py_ssize_t i = 0; end = (SRE_CHAR *)state->end; while (ptr < end) { for (;;) { @@ -1604,7 +1606,7 @@ } LOCAL(int) -SRE_LITERAL_TEMPLATE(SRE_CHAR* ptr, int len) +SRE_LITERAL_TEMPLATE(SRE_CHAR* ptr, Py_ssize_t len) { /* check if given string is a literal template (i.e. no escapes) */ while (len-- > 0) @@ -1625,7 +1627,7 @@ static PyObject * sre_codesize(PyObject* self, PyObject *unused) { - return Py_BuildValue("i", sizeof(SRE_CODE)); + return Py_BuildValue("l", sizeof(SRE_CODE)); } static PyObject * @@ -1660,14 +1662,15 @@ } static void* -getstring(PyObject* string, int* p_length, int* p_charsize) +getstring(PyObject* string, Py_ssize_t* p_length, int* p_charsize) { /* given a python object, return a data pointer, a length (in characters), and a character size. return NULL if the object is not a string (or not compatible) */ PyBufferProcs *buffer; - int size, bytes, charsize; + Py_ssize_t size, bytes; + int charsize; void* ptr; #if defined(HAVE_UNICODE) @@ -1706,7 +1709,7 @@ if (PyString_Check(string) || bytes == size) charsize = 1; #if defined(HAVE_UNICODE) - else if (bytes == (int) (size * sizeof(Py_UNICODE))) + else if (bytes == (Py_ssize_t) (size * sizeof(Py_UNICODE))) charsize = sizeof(Py_UNICODE); #endif else { @@ -1726,11 +1729,11 @@ LOCAL(PyObject*) state_init(SRE_STATE* state, PatternObject* pattern, PyObject* string, - int start, int end) + Py_ssize_t start, Py_ssize_t end) { /* prepare state object */ - int length; + Py_ssize_t length; int charsize; void* ptr; @@ -1792,9 +1795,9 @@ (((char*)(member) - (char*)(state)->beginning) / (state)->charsize) LOCAL(PyObject*) -state_getslice(SRE_STATE* state, int index, PyObject* string, int empty) +state_getslice(SRE_STATE* state, Py_ssize_t index, PyObject* string, int empty) { - int i, j; + Py_ssize_t i, j; index = (index - 1) * 2; @@ -1854,10 +1857,10 @@ int status; PyObject* string; - int start = 0; - int end = INT_MAX; + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|ii:match", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:match", kwlist, &string, &start, &end)) return NULL; @@ -1891,10 +1894,10 @@ int status; PyObject* string; - int start = 0; - int end = INT_MAX; + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|ii:search", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:search", kwlist, &string, &start, &end)) return NULL; @@ -2029,13 +2032,13 @@ SRE_STATE state; PyObject* list; int status; - int i, b, e; + Py_ssize_t i, b, e; PyObject* string; - int start = 0; - int end = INT_MAX; + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; static char* kwlist[] = { "source", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|ii:findall", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:findall", kwlist, &string, &start, &end)) return NULL; @@ -2154,18 +2157,18 @@ PyObject* list; PyObject* item; int status; - int n; - int i; + Py_ssize_t n; + Py_ssize_t i; void* last; PyObject* string; - int maxsplit = 0; + Py_ssize_t maxsplit = 0; static char* kwlist[] = { "source", "maxsplit", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|i:split", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|n:split", kwlist, &string, &maxsplit)) return NULL; - string = state_init(&state, self, string, 0, INT_MAX); + string = state_init(&state, self, string, 0, PY_SSIZE_T_MAX); if (!string) return NULL; @@ -2259,7 +2262,7 @@ static PyObject* pattern_subx(PatternObject* self, PyObject* ptemplate, PyObject* string, - int count, int subn) + Py_ssize_t count, Py_ssize_t subn) { SRE_STATE state; PyObject* list; @@ -2269,8 +2272,9 @@ PyObject* match; void* ptr; int status; - int n; - int i, b, e; + Py_ssize_t n; + Py_ssize_t i, b, e; + int bint; int filter_is_callable; if (PyCallable_Check(ptemplate)) { @@ -2281,7 +2285,8 @@ } else { /* if not callable, check if it's a literal string */ int literal; - ptr = getstring(ptemplate, &n, &b); + ptr = getstring(ptemplate, &n, &bint); + b = bint; if (ptr) { if (b == 1) { literal = sre_literal_template((unsigned char *)ptr, n); @@ -2310,7 +2315,7 @@ } } - string = state_init(&state, self, string, 0, INT_MAX); + string = state_init(&state, self, string, 0, PY_SSIZE_T_MAX); if (!string) { Py_DECREF(filter); return NULL; @@ -2443,9 +2448,9 @@ { PyObject* ptemplate; PyObject* string; - int count = 0; + Py_ssize_t count = 0; static char* kwlist[] = { "repl", "string", "count", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|i:sub", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|n:sub", kwlist, &ptemplate, &string, &count)) return NULL; @@ -2457,9 +2462,9 @@ { PyObject* ptemplate; PyObject* string; - int count = 0; + Py_ssize_t count = 0; static char* kwlist[] = { "repl", "string", "count", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|i:subn", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|n:subn", kwlist, &ptemplate, &string, &count)) return NULL; @@ -2642,15 +2647,15 @@ /* "compile" pattern descriptor to pattern object */ PatternObject* self; - int i, n; + Py_ssize_t i, n; PyObject* pattern; int flags = 0; PyObject* code; - int groups = 0; + Py_ssize_t groups = 0; PyObject* groupindex = NULL; PyObject* indexgroup = NULL; - if (!PyArg_ParseTuple(args, "OiO!|iOO", &pattern, &flags, + if (!PyArg_ParseTuple(args, "OiO!|nOO", &pattern, &flags, &PyList_Type, &code, &groups, &groupindex, &indexgroup)) return NULL; @@ -2711,7 +2716,7 @@ } static PyObject* -match_getslice_by_index(MatchObject* self, int index, PyObject* def) +match_getslice_by_index(MatchObject* self, Py_ssize_t index, PyObject* def) { if (index < 0 || index >= self->groups) { /* raise IndexError if we were given a bad group number */ @@ -2735,21 +2740,21 @@ ); } -static int +static Py_ssize_t match_getindex(MatchObject* self, PyObject* index) { - int i; + Py_ssize_t i; if (PyInt_Check(index)) - return (int) PyInt_AS_LONG(index); + return PyInt_AsSsize_t(index); i = -1; if (self->pattern->groupindex) { index = PyObject_GetItem(self->pattern->groupindex, index); if (index) { - if (PyInt_Check(index)) - i = (int) PyInt_AS_LONG(index); + if (PyInt_Check(index) || PyLong_Check(index)) + i = PyInt_AsSsize_t(index); Py_DECREF(index); } else PyErr_Clear(); @@ -2778,7 +2783,7 @@ match_group(MatchObject* self, PyObject* args) { PyObject* result; - int i, size; + Py_ssize_t i, size; size = PyTuple_GET_SIZE(args); @@ -2813,7 +2818,7 @@ match_groups(MatchObject* self, PyObject* args, PyObject* kw) { PyObject* result; - int index; + Py_ssize_t index; PyObject* def = Py_None; static char* kwlist[] = { "default", NULL }; @@ -2842,7 +2847,7 @@ { PyObject* result; PyObject* keys; - int index; + Py_ssize_t index; PyObject* def = Py_None; static char* kwlist[] = { "default", NULL }; @@ -2888,7 +2893,7 @@ static PyObject* match_start(MatchObject* self, PyObject* args) { - int index; + Py_ssize_t index; PyObject* index_ = Py_False; /* zero */ if (!PyArg_UnpackTuple(args, "start", 0, 1, &index_)) @@ -2911,7 +2916,7 @@ static PyObject* match_end(MatchObject* self, PyObject* args) { - int index; + Py_ssize_t index; PyObject* index_ = Py_False; /* zero */ if (!PyArg_UnpackTuple(args, "end", 0, 1, &index_)) @@ -2932,7 +2937,7 @@ } LOCAL(PyObject*) -_pair(int i1, int i2) +_pair(Py_ssize_t i1, Py_ssize_t i2) { PyObject* pair; PyObject* item; @@ -2941,12 +2946,12 @@ if (!pair) return NULL; - item = PyInt_FromLong(i1); + item = PyInt_FromSsize_t(i1); if (!item) goto error; PyTuple_SET_ITEM(pair, 0, item); - item = PyInt_FromLong(i2); + item = PyInt_FromSsize_t(i2); if (!item) goto error; PyTuple_SET_ITEM(pair, 1, item); @@ -2961,7 +2966,7 @@ static PyObject* match_span(MatchObject* self, PyObject* args) { - int index; + Py_ssize_t index; PyObject* index_ = Py_False; /* zero */ if (!PyArg_UnpackTuple(args, "span", 0, 1, &index_)) @@ -2986,7 +2991,7 @@ { PyObject* regs; PyObject* item; - int index; + Py_ssize_t index; regs = PyTuple_New(self->groups); if (!regs) @@ -3012,7 +3017,7 @@ { #ifdef USE_BUILTIN_COPY MatchObject* copy; - int slots, offset; + Py_ssize_t slots, offset; slots = 2 * (self->pattern->groups+1); @@ -3029,7 +3034,7 @@ Py_XINCREF(self->regs); memcpy((char*) copy + offset, (char*) self + offset, - sizeof(MatchObject) + slots * sizeof(int) - offset); + sizeof(MatchObject) + slots * sizeof(Py_ssize_t) - offset); return (PyObject*) copy; #else @@ -3144,7 +3149,7 @@ statichere PyTypeObject Match_Type = { PyObject_HEAD_INIT(NULL) 0, "_" SRE_MODULE ".SRE_Match", - sizeof(MatchObject), sizeof(int), + sizeof(MatchObject), sizeof(Py_ssize_t), (destructor)match_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ (getattrfunc)match_getattr /*tp_getattr*/ @@ -3156,7 +3161,7 @@ /* create match object (from state object) */ MatchObject* match; - int i, j; + Py_ssize_t i, j; char* base; int n; @@ -3329,9 +3334,9 @@ ScannerObject* self; PyObject* string; - int start = 0; - int end = INT_MAX; - if (!PyArg_ParseTuple(args, "O|ii:scanner", &string, &start, &end)) + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; + if (!PyArg_ParseTuple(args, "O|nn:scanner", &string, &start, &end)) return NULL; /* create scanner object */ Modified: python/trunk/Modules/sre.h ============================================================================== --- python/trunk/Modules/sre.h (original) +++ python/trunk/Modules/sre.h Mon Jun 12 05:05:40 2006 @@ -23,7 +23,7 @@ typedef struct { PyObject_VAR_HEAD - int groups; /* must be first! */ + Py_ssize_t groups; /* must be first! */ PyObject* groupindex; PyObject* indexgroup; /* compatibility */ @@ -31,7 +31,7 @@ int flags; /* flags used when compiling pattern source */ PyObject *weakreflist; /* List of weak references */ /* pattern code */ - int codesize; + Py_ssize_t codesize; SRE_CODE code[1]; } PatternObject; @@ -42,10 +42,10 @@ PyObject* string; /* link to the target string (must be first) */ PyObject* regs; /* cached list of matching spans */ PatternObject* pattern; /* link to the regex (pattern) object */ - int pos, endpos; /* current target slice */ - int lastindex; /* last index marker seen by the engine (-1 if none) */ - int groups; /* number of groups (start/end marks) */ - int mark[1]; + Py_ssize_t pos, endpos; /* current target slice */ + Py_ssize_t lastindex; /* last index marker seen by the engine (-1 if none) */ + Py_ssize_t groups; /* number of groups (start/end marks) */ + Py_ssize_t mark[1]; } MatchObject; typedef unsigned int (*SRE_TOLOWER_HOOK)(unsigned int ch); @@ -54,7 +54,7 @@ #define SRE_MARK_SIZE 200 typedef struct SRE_REPEAT_T { - int count; + Py_ssize_t count; SRE_CODE* pattern; /* points to REPEAT operator arguments */ void* last_ptr; /* helper to check for infinite loops */ struct SRE_REPEAT_T *prev; /* points to previous repeat context */ @@ -68,17 +68,17 @@ void* end; /* end of original string */ /* attributes for the match object */ PyObject* string; - int pos, endpos; + Py_ssize_t pos, endpos; /* character size */ int charsize; /* registers */ - int lastindex; - int lastmark; + Py_ssize_t lastindex; + Py_ssize_t lastmark; void* mark[SRE_MARK_SIZE]; /* dynamically allocated stuff */ char* data_stack; - unsigned int data_stack_size; - unsigned int data_stack_base; + size_t data_stack_size; + size_t data_stack_base; /* current repeat context */ SRE_REPEAT *repeat; /* hooks */ From python-checkins at python.org Mon Jun 12 05:33:10 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 12 Jun 2006 05:33:10 +0200 (CEST) Subject: [Python-checkins] r46886 - in python/trunk: Lib/test/test_zlib.py Misc/ACKS Modules/zlibmodule.c configure configure.in pyconfig.h.in Message-ID: <20060612033310.91FBA1E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 12 05:33:09 2006 New Revision: 46886 Modified: python/trunk/Lib/test/test_zlib.py python/trunk/Misc/ACKS python/trunk/Modules/zlibmodule.c python/trunk/configure python/trunk/configure.in python/trunk/pyconfig.h.in Log: Patch #1503046, Conditional compilation of zlib.(de)compressobj.copy copy is only in newer versions of zlib. This should allow zlibmodule to work with older versions like the Tru64 buildbot. Modified: python/trunk/Lib/test/test_zlib.py ============================================================================== --- python/trunk/Lib/test/test_zlib.py (original) +++ python/trunk/Lib/test/test_zlib.py Mon Jun 12 05:33:09 2006 @@ -302,63 +302,65 @@ dco = zlib.decompressobj() self.assertEqual(dco.flush(), "") # Returns nothing - def test_compresscopy(self): - # Test copying a compression object - data0 = HAMLET_SCENE - data1 = HAMLET_SCENE.swapcase() - c0 = zlib.compressobj(zlib.Z_BEST_COMPRESSION) - bufs0 = [] - bufs0.append(c0.compress(data0)) - - c1 = c0.copy() - bufs1 = bufs0[:] - - bufs0.append(c0.compress(data0)) - bufs0.append(c0.flush()) - s0 = ''.join(bufs0) - - bufs1.append(c1.compress(data1)) - bufs1.append(c1.flush()) - s1 = ''.join(bufs1) - - self.assertEqual(zlib.decompress(s0),data0+data0) - self.assertEqual(zlib.decompress(s1),data0+data1) - - def test_badcompresscopy(self): - # Test copying a compression object in an inconsistent state - c = zlib.compressobj() - c.compress(HAMLET_SCENE) - c.flush() - self.assertRaises(ValueError, c.copy) - - def test_decompresscopy(self): - # Test copying a decompression object - data = HAMLET_SCENE - comp = zlib.compress(data) - - d0 = zlib.decompressobj() - bufs0 = [] - bufs0.append(d0.decompress(comp[:32])) - - d1 = d0.copy() - bufs1 = bufs0[:] - - bufs0.append(d0.decompress(comp[32:])) - s0 = ''.join(bufs0) - - bufs1.append(d1.decompress(comp[32:])) - s1 = ''.join(bufs1) - - self.assertEqual(s0,s1) - self.assertEqual(s0,data) - - def test_baddecompresscopy(self): - # Test copying a compression object in an inconsistent state - data = zlib.compress(HAMLET_SCENE) - d = zlib.decompressobj() - d.decompress(data) - d.flush() - self.assertRaises(ValueError, d.copy) + if hasattr(zlib.compressobj(), "copy"): + def test_compresscopy(self): + # Test copying a compression object + data0 = HAMLET_SCENE + data1 = HAMLET_SCENE.swapcase() + c0 = zlib.compressobj(zlib.Z_BEST_COMPRESSION) + bufs0 = [] + bufs0.append(c0.compress(data0)) + + c1 = c0.copy() + bufs1 = bufs0[:] + + bufs0.append(c0.compress(data0)) + bufs0.append(c0.flush()) + s0 = ''.join(bufs0) + + bufs1.append(c1.compress(data1)) + bufs1.append(c1.flush()) + s1 = ''.join(bufs1) + + self.assertEqual(zlib.decompress(s0),data0+data0) + self.assertEqual(zlib.decompress(s1),data0+data1) + + def test_badcompresscopy(self): + # Test copying a compression object in an inconsistent state + c = zlib.compressobj() + c.compress(HAMLET_SCENE) + c.flush() + self.assertRaises(ValueError, c.copy) + + if hasattr(zlib.decompressobj(), "copy"): + def test_decompresscopy(self): + # Test copying a decompression object + data = HAMLET_SCENE + comp = zlib.compress(data) + + d0 = zlib.decompressobj() + bufs0 = [] + bufs0.append(d0.decompress(comp[:32])) + + d1 = d0.copy() + bufs1 = bufs0[:] + + bufs0.append(d0.decompress(comp[32:])) + s0 = ''.join(bufs0) + + bufs1.append(d1.decompress(comp[32:])) + s1 = ''.join(bufs1) + + self.assertEqual(s0,s1) + self.assertEqual(s0,data) + + def test_baddecompresscopy(self): + # Test copying a compression object in an inconsistent state + data = zlib.compress(HAMLET_SCENE) + d = zlib.decompressobj() + d.decompress(data) + d.flush() + self.assertRaises(ValueError, d.copy) def genblock(seed, length, step=1024, generator=random): """length-byte stream of random data from a seed (in step-byte blocks).""" Modified: python/trunk/Misc/ACKS ============================================================================== --- python/trunk/Misc/ACKS (original) +++ python/trunk/Misc/ACKS Mon Jun 12 05:33:09 2006 @@ -24,6 +24,7 @@ Jason Asbahr David Ascher Peter �strand +Chris AtLee John Aycock Donovan Baarda Attila Babo Modified: python/trunk/Modules/zlibmodule.c ============================================================================== --- python/trunk/Modules/zlibmodule.c (original) +++ python/trunk/Modules/zlibmodule.c Mon Jun 12 05:33:09 2006 @@ -653,6 +653,7 @@ return RetVal; } +#ifdef HAVE_ZLIB_COPY PyDoc_STRVAR(comp_copy__doc__, "copy() -- Return a copy of the compression object."); @@ -754,6 +755,7 @@ Py_XDECREF(retval); return NULL; } +#endif PyDoc_STRVAR(decomp_flush__doc__, "flush( [length] ) -- Return a string containing any remaining\n" @@ -827,8 +829,10 @@ comp_compress__doc__}, {"flush", (binaryfunc)PyZlib_flush, METH_VARARGS, comp_flush__doc__}, +#ifdef HAVE_ZLIB_COPY {"copy", (PyCFunction)PyZlib_copy, METH_NOARGS, comp_copy__doc__}, +#endif {NULL, NULL} }; @@ -838,8 +842,10 @@ decomp_decompress__doc__}, {"flush", (binaryfunc)PyZlib_unflush, METH_VARARGS, decomp_flush__doc__}, +#ifdef HAVE_ZLIB_COPY {"copy", (PyCFunction)PyZlib_uncopy, METH_NOARGS, decomp_copy__doc__}, +#endif {NULL, NULL} }; Modified: python/trunk/configure ============================================================================== --- python/trunk/configure (original) +++ python/trunk/configure Mon Jun 12 05:33:09 2006 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 46720 . +# From configure.in Revision: 46879 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59 for python 2.5. # @@ -14980,6 +14980,79 @@ fi +echo "$as_me:$LINENO: checking for inflateCopy in -lz" >&5 +echo $ECHO_N "checking for inflateCopy in -lz... $ECHO_C" >&6 +if test "${ac_cv_lib_z_inflateCopy+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + ac_check_lib_save_LIBS=$LIBS +LIBS="-lz $LIBS" +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ + +/* Override any gcc2 internal prototype to avoid an error. */ +#ifdef __cplusplus +extern "C" +#endif +/* We use char because int might match the return type of a gcc2 + builtin and then its argument prototype would still apply. */ +char inflateCopy (); +int +main () +{ +inflateCopy (); + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext conftest$ac_exeext +if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 + (eval $ac_link) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest$ac_exeext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_cv_lib_z_inflateCopy=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +ac_cv_lib_z_inflateCopy=no +fi +rm -f conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext +LIBS=$ac_check_lib_save_LIBS +fi +echo "$as_me:$LINENO: result: $ac_cv_lib_z_inflateCopy" >&5 +echo "${ECHO_T}$ac_cv_lib_z_inflateCopy" >&6 +if test $ac_cv_lib_z_inflateCopy = yes; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_ZLIB_COPY 1 +_ACEOF + +fi + + echo "$as_me:$LINENO: checking for hstrerror" >&5 echo $ECHO_N "checking for hstrerror... $ECHO_C" >&6 cat >conftest.$ac_ext <<_ACEOF @@ -22056,10 +22129,9 @@ # 1. Remove the extension, and $U if already installed. ac_i=`echo "$ac_i" | sed 's/\$U\././;s/\.o$//;s/\.obj$//'` - # 2. Prepend LIBOBJDIR. When used with automake>=1.10 LIBOBJDIR - # will be set to the directory where LIBOBJS objects are built. - ac_libobjs="$ac_libobjs \${LIBOBJDIR}$ac_i\$U.$ac_objext" - ac_ltlibobjs="$ac_ltlibobjs \${LIBOBJDIR}$ac_i"'$U.lo' + # 2. Add them. + ac_libobjs="$ac_libobjs $ac_i\$U.$ac_objext" + ac_ltlibobjs="$ac_ltlibobjs $ac_i"'$U.lo' done LIBOBJS=$ac_libobjs Modified: python/trunk/configure.in ============================================================================== --- python/trunk/configure.in (original) +++ python/trunk/configure.in Mon Jun 12 05:33:09 2006 @@ -2351,6 +2351,9 @@ AC_CHECK_LIB(resolv, inet_aton) ) +dnl Check if system zlib has *Copy() functions +AC_CHECK_LIB(z, inflateCopy, AC_DEFINE(HAVE_ZLIB_COPY, 1, Define if the zlib library has inflateCopy)) + AC_MSG_CHECKING(for hstrerror) AC_TRY_LINK([ #include "confdefs.h" Modified: python/trunk/pyconfig.h.in ============================================================================== --- python/trunk/pyconfig.h.in (original) +++ python/trunk/pyconfig.h.in Mon Jun 12 05:33:09 2006 @@ -710,6 +710,9 @@ */ #undef HAVE_WORKING_TZSET +/* Define if the zlib library has inflateCopy */ +#undef HAVE_ZLIB_COPY + /* Define to 1 if you have the `_getpty' function. */ #undef HAVE__GETPTY From buildbot at python.org Mon Jun 12 05:52:29 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 03:52:29 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060612035229.F0D711E4005@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/393 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 12 06:04:33 2006 From: python-checkins at python.org (phillip.eby) Date: Mon, 12 Jun 2006 06:04:33 +0200 (CEST) Subject: [Python-checkins] r46887 - in python/trunk/Lib: test/test_wsgiref.py wsgiref.egg-info wsgiref/handlers.py wsgiref/headers.py wsgiref/simple_server.py wsgiref/util.py wsgiref/validate.py Message-ID: <20060612040433.C9FFF1E4002@bag.python.org> Author: phillip.eby Date: Mon Jun 12 06:04:32 2006 New Revision: 46887 Modified: python/trunk/Lib/test/test_wsgiref.py python/trunk/Lib/wsgiref.egg-info python/trunk/Lib/wsgiref/handlers.py python/trunk/Lib/wsgiref/headers.py python/trunk/Lib/wsgiref/simple_server.py python/trunk/Lib/wsgiref/util.py python/trunk/Lib/wsgiref/validate.py Log: Sync w/external release 0.1.2. Please see PEP 360 before making changes to external packages. Modified: python/trunk/Lib/test/test_wsgiref.py ============================================================================== --- python/trunk/Lib/test/test_wsgiref.py (original) +++ python/trunk/Lib/test/test_wsgiref.py Mon Jun 12 06:04:32 2006 @@ -80,7 +80,7 @@ -def compare_generic_iter(test, make_it, match): +def compare_generic_iter(make_it,match): """Utility to compare a generic 2.1/2.2+ iterator with an iterable If running under Python 2.2+, this tests the iterator using iter()/next(), @@ -90,7 +90,7 @@ it = make_it() n = 0 for item in match: - test.assertEqual(it[n], item) + if not it[n]==item: raise AssertionError n+=1 try: it[n] @@ -106,10 +106,15 @@ else: # Only test iter mode under 2.2+ it = make_it() - test.assert_(iter(it) is it) + if not iter(it) is it: raise AssertionError for item in match: - test.assertEqual(it.next(), item) - test.assertRaises(StopIteration, it.next) + if not it.next()==item: raise AssertionError + try: + it.next() + except StopIteration: + pass + else: + raise AssertionError("Too many items from .next()",it) @@ -203,7 +208,7 @@ def make_it(text=text,size=size): return util.FileWrapper(StringIO(text),size) - compare_generic_iter(self, make_it, match) + compare_generic_iter(make_it,match) it = make_it() self.failIf(it.filelike.closed) Modified: python/trunk/Lib/wsgiref.egg-info ============================================================================== --- python/trunk/Lib/wsgiref.egg-info (original) +++ python/trunk/Lib/wsgiref.egg-info Mon Jun 12 06:04:32 2006 @@ -1,6 +1,6 @@ Metadata-Version: 1.0 Name: wsgiref -Version: 0.1 +Version: 0.1.2 Summary: WSGI (PEP 333) Reference Library Author: Phillip J. Eby Author-email: web-sig at python.org Modified: python/trunk/Lib/wsgiref/handlers.py ============================================================================== --- python/trunk/Lib/wsgiref/handlers.py (original) +++ python/trunk/Lib/wsgiref/handlers.py Mon Jun 12 06:04:32 2006 @@ -473,3 +473,20 @@ self, sys.stdin, sys.stdout, sys.stderr, dict(os.environ.items()), multithread=False, multiprocess=True ) + + + + + + + + + + + + + + + + +# Modified: python/trunk/Lib/wsgiref/headers.py ============================================================================== --- python/trunk/Lib/wsgiref/headers.py (original) +++ python/trunk/Lib/wsgiref/headers.py Mon Jun 12 06:04:32 2006 @@ -187,3 +187,19 @@ else: parts.append(_formatparam(k.replace('_', '-'), v)) self._headers.append((_name, "; ".join(parts))) + + + + + + + + + + + + + + + +# Modified: python/trunk/Lib/wsgiref/simple_server.py ============================================================================== --- python/trunk/Lib/wsgiref/simple_server.py (original) +++ python/trunk/Lib/wsgiref/simple_server.py Mon Jun 12 06:04:32 2006 @@ -190,3 +190,16 @@ import webbrowser webbrowser.open('http://localhost:8000/xyz?abc') httpd.handle_request() # serve one request, then exit + + + + + + + + + + + + +# Modified: python/trunk/Lib/wsgiref/util.py ============================================================================== --- python/trunk/Lib/wsgiref/util.py (original) +++ python/trunk/Lib/wsgiref/util.py Mon Jun 12 06:04:32 2006 @@ -171,3 +171,35 @@ def is_hop_by_hop(header_name): """Return true if 'header_name' is an HTTP/1.1 "Hop-by-Hop" header""" return _hoppish(header_name.lower()) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +# Modified: python/trunk/Lib/wsgiref/validate.py ============================================================================== --- python/trunk/Lib/wsgiref/validate.py (original) +++ python/trunk/Lib/wsgiref/validate.py Mon Jun 12 06:04:32 2006 @@ -124,6 +124,10 @@ Raised in response to WSGI-spec-related warnings """ +def assert_(cond, *args): + if not cond: + raise AssertionError(*args) + def validator(application): """ @@ -137,8 +141,8 @@ """ def lint_app(*args, **kw): - assert len(args) == 2, "Two arguments required" - assert not kw, "No keyword arguments allowed" + assert_(len(args) == 2, "Two arguments required") + assert_(not kw, "No keyword arguments allowed") environ, start_response = args check_environ(environ) @@ -148,9 +152,9 @@ start_response_started = [] def start_response_wrapper(*args, **kw): - assert len(args) == 2 or len(args) == 3, ( - "Invalid number of arguments: %s" % args) - assert not kw, "No keyword arguments allowed" + assert_(len(args) == 2 or len(args) == 3, ( + "Invalid number of arguments: %s" % (args,))) + assert_(not kw, "No keyword arguments allowed") status = args[0] headers = args[1] if len(args) == 3: @@ -170,7 +174,7 @@ environ['wsgi.errors'] = ErrorWrapper(environ['wsgi.errors']) iterator = application(environ, start_response_wrapper) - assert iterator is not None and iterator != False, ( + assert_(iterator is not None and iterator != False, "The application must return an iterator, if only an empty list") check_iterator(iterator) @@ -185,22 +189,22 @@ self.input = wsgi_input def read(self, *args): - assert len(args) <= 1 + assert_(len(args) <= 1) v = self.input.read(*args) - assert type(v) is type("") + assert_(type(v) is type("")) return v def readline(self): v = self.input.readline() - assert type(v) is type("") + assert_(type(v) is type("")) return v def readlines(self, *args): - assert len(args) <= 1 + assert_(len(args) <= 1) lines = self.input.readlines(*args) - assert type(lines) is type([]) + assert_(type(lines) is type([])) for line in lines: - assert type(line) is type("") + assert_(type(line) is type("")) return lines def __iter__(self): @@ -211,7 +215,7 @@ yield line def close(self): - assert 0, "input.close() must not be called" + assert_(0, "input.close() must not be called") class ErrorWrapper: @@ -219,7 +223,7 @@ self.errors = wsgi_errors def write(self, s): - assert type(s) is type("") + assert_(type(s) is type("")) self.errors.write(s) def flush(self): @@ -230,7 +234,7 @@ self.write(line) def close(self): - assert 0, "errors.close() must not be called" + assert_(0, "errors.close() must not be called") class WriteWrapper: @@ -238,7 +242,7 @@ self.writer = wsgi_writer def __call__(self, s): - assert type(s) is type("") + assert_(type(s) is type("")) self.writer(s) class PartialIteratorWrapper: @@ -262,11 +266,11 @@ return self def next(self): - assert not self.closed, ( + assert_(not self.closed, "Iterator read after closed") v = self.iterator.next() if self.check_start_response is not None: - assert self.check_start_response, ( + assert_(self.check_start_response, "The application returns and we started iterating over its body, but start_response has not yet been called") self.check_start_response = None return v @@ -280,11 +284,11 @@ if not self.closed: sys.stderr.write( "Iterator garbage collected without being closed") - assert self.closed, ( + assert_(self.closed, "Iterator garbage collected without being closed") def check_environ(environ): - assert type(environ) is DictType, ( + assert_(type(environ) is DictType, "Environment is not of the right type: %r (environment: %r)" % (type(environ), environ)) @@ -292,11 +296,11 @@ 'wsgi.version', 'wsgi.input', 'wsgi.errors', 'wsgi.multithread', 'wsgi.multiprocess', 'wsgi.run_once']: - assert key in environ, ( - "Environment missing required key: %r" % key) + assert_(key in environ, + "Environment missing required key: %r" % (key,)) for key in ['HTTP_CONTENT_TYPE', 'HTTP_CONTENT_LENGTH']: - assert key not in environ, ( + assert_(key not in environ, "Environment should not have the key: %s " "(use %s instead)" % (key, key[5:])) @@ -311,13 +315,13 @@ if '.' in key: # Extension, we don't care about its type continue - assert type(environ[key]) is StringType, ( + assert_(type(environ[key]) is StringType, "Environmental variable %s is not a string: %r (value: %r)" % (key, type(environ[key]), environ[key])) - assert type(environ['wsgi.version']) is TupleType, ( - "wsgi.version should be a tuple (%r)" % environ['wsgi.version']) - assert environ['wsgi.url_scheme'] in ('http', 'https'), ( + assert_(type(environ['wsgi.version']) is TupleType, + "wsgi.version should be a tuple (%r)" % (environ['wsgi.version'],)) + assert_(environ['wsgi.url_scheme'] in ('http', 'https'), "wsgi.url_scheme unknown: %r" % environ['wsgi.url_scheme']) check_input(environ['wsgi.input']) @@ -330,45 +334,45 @@ "Unknown REQUEST_METHOD: %r" % environ['REQUEST_METHOD'], WSGIWarning) - assert (not environ.get('SCRIPT_NAME') - or environ['SCRIPT_NAME'].startswith('/')), ( + assert_(not environ.get('SCRIPT_NAME') + or environ['SCRIPT_NAME'].startswith('/'), "SCRIPT_NAME doesn't start with /: %r" % environ['SCRIPT_NAME']) - assert (not environ.get('PATH_INFO') - or environ['PATH_INFO'].startswith('/')), ( + assert_(not environ.get('PATH_INFO') + or environ['PATH_INFO'].startswith('/'), "PATH_INFO doesn't start with /: %r" % environ['PATH_INFO']) if environ.get('CONTENT_LENGTH'): - assert int(environ['CONTENT_LENGTH']) >= 0, ( + assert_(int(environ['CONTENT_LENGTH']) >= 0, "Invalid CONTENT_LENGTH: %r" % environ['CONTENT_LENGTH']) if not environ.get('SCRIPT_NAME'): - assert environ.has_key('PATH_INFO'), ( + assert_(environ.has_key('PATH_INFO'), "One of SCRIPT_NAME or PATH_INFO are required (PATH_INFO " "should at least be '/' if SCRIPT_NAME is empty)") - assert environ.get('SCRIPT_NAME') != '/', ( + assert_(environ.get('SCRIPT_NAME') != '/', "SCRIPT_NAME cannot be '/'; it should instead be '', and " "PATH_INFO should be '/'") def check_input(wsgi_input): for attr in ['read', 'readline', 'readlines', '__iter__']: - assert hasattr(wsgi_input, attr), ( + assert_(hasattr(wsgi_input, attr), "wsgi.input (%r) doesn't have the attribute %s" % (wsgi_input, attr)) def check_errors(wsgi_errors): for attr in ['flush', 'write', 'writelines']: - assert hasattr(wsgi_errors, attr), ( + assert_(hasattr(wsgi_errors, attr), "wsgi.errors (%r) doesn't have the attribute %s" % (wsgi_errors, attr)) def check_status(status): - assert type(status) is StringType, ( + assert_(type(status) is StringType, "Status must be a string (not %r)" % status) # Implicitly check that we can turn it into an integer: status_code = status.split(None, 1)[0] - assert len(status_code) == 3, ( + assert_(len(status_code) == 3, "Status codes must be three characters: %r" % status_code) status_int = int(status_code) - assert status_int >= 100, "Status code is invalid: %r" % status_int + assert_(status_int >= 100, "Status code is invalid: %r" % status_int) if len(status) < 4 or status[3] != ' ': warnings.warn( "The status string (%r) should be a three-digit integer " @@ -376,28 +380,28 @@ % status, WSGIWarning) def check_headers(headers): - assert type(headers) is ListType, ( + assert_(type(headers) is ListType, "Headers (%r) must be of type list: %r" % (headers, type(headers))) header_names = {} for item in headers: - assert type(item) is TupleType, ( + assert_(type(item) is TupleType, "Individual headers (%r) must be of type tuple: %r" % (item, type(item))) - assert len(item) == 2 + assert_(len(item) == 2) name, value = item - assert name.lower() != 'status', ( + assert_(name.lower() != 'status', "The Status header cannot be used; it conflicts with CGI " "script, and HTTP status is not given through headers " "(value: %r)." % value) header_names[name.lower()] = None - assert '\n' not in name and ':' not in name, ( + assert_('\n' not in name and ':' not in name, "Header names may not contain ':' or '\\n': %r" % name) - assert header_re.search(name), "Bad header name: %r" % name - assert not name.endswith('-') and not name.endswith('_'), ( + assert_(header_re.search(name), "Bad header name: %r" % name) + assert_(not name.endswith('-') and not name.endswith('_'), "Names may not end in '-' or '_': %r" % name) - assert not bad_header_value_re.search(value), ( - "Bad header value: %r (bad char: %r)" + if bad_header_value_re.search(value): + assert_(0, "Bad header value: %r (bad char: %r)" % (value, bad_header_value_re.search(value).group(0))) def check_content_type(status, headers): @@ -409,13 +413,13 @@ if name.lower() == 'content-type': if code not in NO_MESSAGE_BODY: return - assert 0, (("Content-Type header found in a %s response, " + assert_(0, ("Content-Type header found in a %s response, " "which must not return content.") % code) if code not in NO_MESSAGE_BODY: - assert 0, "No Content-Type header found in headers (%s)" % headers + assert_(0, "No Content-Type header found in headers (%s)" % headers) def check_exc_info(exc_info): - assert exc_info is None or type(exc_info) is type(()), ( + assert_(exc_info is None or type(exc_info) is type(()), "exc_info (%r) is not a tuple: %r" % (exc_info, type(exc_info))) # More exc_info checks? @@ -423,6 +427,6 @@ # Technically a string is legal, which is why it's a really bad # idea, because it may cause the response to be returned # character-by-character - assert not isinstance(iterator, str), ( + assert_(not isinstance(iterator, str), "You should not return a string as your application iterator, " "instead return a single-item list containing that string.") From python-checkins at python.org Mon Jun 12 06:26:37 2006 From: python-checkins at python.org (martin.v.loewis) Date: Mon, 12 Jun 2006 06:26:37 +0200 (CEST) Subject: [Python-checkins] r46888 - python/trunk/Modules/_bsddb.c Message-ID: <20060612042637.8FE1C1E4002@bag.python.org> Author: martin.v.loewis Date: Mon Jun 12 06:26:31 2006 New Revision: 46888 Modified: python/trunk/Modules/_bsddb.c Log: Get rid of function pointer cast. Modified: python/trunk/Modules/_bsddb.c ============================================================================== --- python/trunk/Modules/_bsddb.c (original) +++ python/trunk/Modules/_bsddb.c Mon Jun 12 06:26:31 2006 @@ -104,7 +104,6 @@ #if (PY_VERSION_HEX < 0x02050000) typedef int Py_ssize_t; -typedef inquiry lenfunc; #endif #ifdef WITH_THREAD @@ -2682,12 +2681,13 @@ /*-------------------------------------------------------------- */ /* Mapping and Dictionary-like access routines */ -Py_ssize_t DB_length(DBObject* self) +Py_ssize_t DB_length(PyObject* _self) { int err; Py_ssize_t size = 0; int flags = 0; void* sp; + DBObject* self = (DBObject*)_self; if (self->db == NULL) { PyObject *t = Py_BuildValue("(is)", 0, "DB object has been closed"); @@ -5154,7 +5154,7 @@ static PyMappingMethods DB_mapping = { - (inquiry)DB_length, /*mp_length*/ + DB_length, /*mp_length*/ (binaryfunc)DB_subscript, /*mp_subscript*/ (objobjargproc)DB_ass_sub, /*mp_ass_subscript*/ }; From buildbot at python.org Mon Jun 12 06:34:52 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 04:34:52 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060612043452.716E41E4002@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/686 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 12 08:05:57 2006 From: python-checkins at python.org (thomas.heller) Date: Mon, 12 Jun 2006 08:05:57 +0200 (CEST) Subject: [Python-checkins] r46889 - python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S Message-ID: <20060612060557.9343B1E4002@bag.python.org> Author: thomas.heller Date: Mon Jun 12 08:05:57 2006 New Revision: 46889 Modified: python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S Log: I don't know how that happend, but the entire file contents was duplicated. Thanks to Simon Percivall for the heads up. Modified: python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S ============================================================================== --- python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S (original) +++ python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S Mon Jun 12 08:05:57 2006 @@ -193,198 +193,3 @@ #endif /* ifndef __x86_64__ */ #endif /* defined __i386__ */ -#ifdef __i386__ -/* ----------------------------------------------------------------------- - darwin.S - Copyright (c) 1996, 1998, 2001, 2002, 2003 Red Hat, Inc. - - X86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -/* - * This file is based on sysv.S and then hacked up by Ronald who hasn't done - * assembly programming in 8 years. - */ - -#ifndef __x86_64__ - -#define LIBFFI_ASM -#include -#include - -.text - -.globl _ffi_prep_args - -.align 4 -.globl _ffi_call_SYSV - -_ffi_call_SYSV: -.LFB1: - pushl %ebp -.LCFI0: - movl %esp,%ebp -.LCFI1: - /* Make room for all of the new args. */ - movl 16(%ebp),%ecx - subl %ecx,%esp - - movl %esp,%eax - - /* Place all of the ffi_prep_args in position */ - pushl 12(%ebp) - pushl %eax - call *8(%ebp) - - /* Return stack to previous state and call the function */ - addl $8,%esp - - call *28(%ebp) - - /* Remove the space we pushed for the args */ - movl 16(%ebp),%ecx - addl %ecx,%esp - - /* Load %ecx with the return type code */ - movl 20(%ebp),%ecx - - /* If the return value pointer is NULL, assume no return value. */ - cmpl $0,24(%ebp) - jne retint - - /* Even if there is no space for the return value, we are - obliged to handle floating-point values. */ - cmpl $FFI_TYPE_FLOAT,%ecx - jne noretval - fstp %st(0) - - jmp epilogue - -retint: - cmpl $FFI_TYPE_INT,%ecx - jne retfloat - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - jmp epilogue - -retfloat: - cmpl $FFI_TYPE_FLOAT,%ecx - jne retdouble - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstps (%ecx) - jmp epilogue - -retdouble: - cmpl $FFI_TYPE_DOUBLE,%ecx - jne retlongdouble - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstpl (%ecx) - jmp epilogue - -retlongdouble: - cmpl $FFI_TYPE_LONGDOUBLE,%ecx - jne retint64 - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - fstpt (%ecx) - jmp epilogue - -retint64: - cmpl $FFI_TYPE_SINT64,%ecx - jne retstruct - /* Load %ecx with the pointer to storage for the return value */ - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - movl %edx,4(%ecx) - -retstruct: - /* Nothing to do! */ - -noretval: -epilogue: - movl %ebp,%esp - popl %ebp - ret -.LFE1: -.ffi_call_SYSV_end: -#if 0 - .size ffi_call_SYSV,.ffi_call_SYSV_end-ffi_call_SYSV -#endif - -#if 0 - .section .eh_frame,EH_FRAME_FLAGS, at progbits -.Lframe1: - .long .LECIE1-.LSCIE1 /* Length of Common Information Entry */ -.LSCIE1: - .long 0x0 /* CIE Identifier Tag */ - .byte 0x1 /* CIE Version */ -#ifdef __PIC__ - .ascii "zR\0" /* CIE Augmentation */ -#else - .ascii "\0" /* CIE Augmentation */ -#endif - .byte 0x1 /* .uleb128 0x1; CIE Code Alignment Factor */ - .byte 0x7c /* .sleb128 -4; CIE Data Alignment Factor */ - .byte 0x8 /* CIE RA Column */ -#ifdef __PIC__ - .byte 0x1 /* .uleb128 0x1; Augmentation size */ - .byte 0x1b /* FDE Encoding (pcrel sdata4) */ -#endif - .byte 0xc /* DW_CFA_def_cfa */ - .byte 0x4 /* .uleb128 0x4 */ - .byte 0x4 /* .uleb128 0x4 */ - .byte 0x88 /* DW_CFA_offset, column 0x8 */ - .byte 0x1 /* .uleb128 0x1 */ - .align 4 -.LECIE1: -.LSFDE1: - .long .LEFDE1-.LASFDE1 /* FDE Length */ -.LASFDE1: - .long .LASFDE1-.Lframe1 /* FDE CIE offset */ -#ifdef __PIC__ - .long .LFB1-. /* FDE initial location */ -#else - .long .LFB1 /* FDE initial location */ -#endif - .long .LFE1-.LFB1 /* FDE address range */ -#ifdef __PIC__ - .byte 0x0 /* .uleb128 0x0; Augmentation size */ -#endif - .byte 0x4 /* DW_CFA_advance_loc4 */ - .long .LCFI0-.LFB1 - .byte 0xe /* DW_CFA_def_cfa_offset */ - .byte 0x8 /* .uleb128 0x8 */ - .byte 0x85 /* DW_CFA_offset, column 0x5 */ - .byte 0x2 /* .uleb128 0x2 */ - .byte 0x4 /* DW_CFA_advance_loc4 */ - .long .LCFI1-.LCFI0 - .byte 0xd /* DW_CFA_def_cfa_register */ - .byte 0x5 /* .uleb128 0x5 */ - .align 4 -.LEFDE1: -#endif - -#endif /* ifndef __x86_64__ */ - -#endif /* defined __i386__ */ From theller at python.net Mon Jun 12 08:16:58 2006 From: theller at python.net (Thomas Heller) Date: Mon, 12 Jun 2006 08:16:58 +0200 Subject: [Python-checkins] r46824 - in python/trunk/Modules/_ctypes: _ctypes.c callproc.c ctypes.h libffi/src/x86/darwin.S libffi_msvc/mingwin32.S stgdict.c In-Reply-To: <7DBF0A0B-AE7E-46EB-B27C-E757115A95EE@chello.se> References: <20060610195148.874F81E400C@bag.python.org> <7DBF0A0B-AE7E-46EB-B27C-E757115A95EE@chello.se> Message-ID: Simon Percivall wrote: > On 10 jun 2006, at 21.51, thomas.heller wrote: >> Author: thomas.heller >> Date: Sat Jun 10 21:51:46 2006 >> New Revision: 46824 >> >> Removed: >> python/trunk/Modules/_ctypes/libffi_msvc/mingwin32.S >> Modified: >> python/trunk/Modules/_ctypes/_ctypes.c >> python/trunk/Modules/_ctypes/callproc.c >> python/trunk/Modules/_ctypes/ctypes.h >> python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S >> python/trunk/Modules/_ctypes/stgdict.c >> Log: >> Upgrade to ctypes version 0.9.9.7. >> >> >> >> Modified: python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S >> ====================================================================== >> ======== >> --- python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S (original) >> +++ python/trunk/Modules/_ctypes/libffi/src/x86/darwin.S Sat Jun 10 >> 21:51:46 2006 >> @@ -193,3 +193,198 @@ >> #endif /* ifndef __x86_64__ */ >> >> #endif /* defined __i386__ */ >> +#ifdef __i386__ >> >> > > The darwin.S checkin seems to be an oopsie!. The entire contents > is duplicated in the file, leading to compile error. > Thanks for the heads-up, should be fixed now. I have no idea why this happend - IIRC, I only did run svneol.py on the file. Thomas From buildbot at python.org Mon Jun 12 08:20:58 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 06:20:58 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060612062058.29F5E1E4002@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/340 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,phillip.eby Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 12 08:43:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 06:43:20 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060612064320.F30D71E4005@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/641 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis,neal.norwitz,phillip.eby Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 12 10:19:38 2006 From: python-checkins at python.org (nick.coghlan) Date: Mon, 12 Jun 2006 10:19:38 +0200 (CEST) Subject: [Python-checkins] r46890 - python/trunk/Lib/site.py Message-ID: <20060612081938.3C7641E4013@bag.python.org> Author: nick.coghlan Date: Mon Jun 12 10:19:37 2006 New Revision: 46890 Modified: python/trunk/Lib/site.py Log: Fix site module docstring to match the code Modified: python/trunk/Lib/site.py ============================================================================== --- python/trunk/Lib/site.py (original) +++ python/trunk/Lib/site.py Mon Jun 12 10:19:37 2006 @@ -13,8 +13,8 @@ This will append site-specific paths to the module search path. On Unix, it starts with sys.prefix and sys.exec_prefix (if different) and appends lib/python/site-packages as well as lib/site-python. -On other platforms (mainly Mac and Windows), it uses just sys.prefix -(and sys.exec_prefix, if different, but this is unlikely). The +On other platforms (mainly Mac and Windows), it tries each of the +prefixes directly, as well as with lib/site-packages appended. The resulting directories, if they exist, are appended to sys.path, and also inspected for path configuration files. From python-checkins at python.org Mon Jun 12 10:23:03 2006 From: python-checkins at python.org (nick.coghlan) Date: Mon, 12 Jun 2006 10:23:03 +0200 (CEST) Subject: [Python-checkins] r46891 - python/trunk/Lib/site.py Message-ID: <20060612082303.397981E4002@bag.python.org> Author: nick.coghlan Date: Mon Jun 12 10:23:02 2006 New Revision: 46891 Modified: python/trunk/Lib/site.py Log: Fix site module docstring to match the code for Mac OSX, too Modified: python/trunk/Lib/site.py ============================================================================== --- python/trunk/Lib/site.py (original) +++ python/trunk/Lib/site.py Mon Jun 12 10:23:02 2006 @@ -11,9 +11,10 @@ works). This will append site-specific paths to the module search path. On -Unix, it starts with sys.prefix and sys.exec_prefix (if different) and -appends lib/python/site-packages as well as lib/site-python. -On other platforms (mainly Mac and Windows), it tries each of the +Unix (including Mac OSX), it starts with sys.prefix and +sys.exec_prefix (if different) and appends +lib/python/site-packages as well as lib/site-python. +On other platforms (such as Windows), it tries each of the prefixes directly, as well as with lib/site-packages appended. The resulting directories, if they exist, are appended to sys.path, and also inspected for path configuration files. From python-checkins at python.org Mon Jun 12 10:27:13 2006 From: python-checkins at python.org (nick.coghlan) Date: Mon, 12 Jun 2006 10:27:13 +0200 (CEST) Subject: [Python-checkins] r46892 - python/trunk/Doc/lib/libsite.tex Message-ID: <20060612082713.E69091E4005@bag.python.org> Author: nick.coghlan Date: Mon Jun 12 10:27:13 2006 New Revision: 46892 Modified: python/trunk/Doc/lib/libsite.tex Log: The site module documentation also described the Windows behaviour incorrectly. Modified: python/trunk/Doc/lib/libsite.tex ============================================================================== --- python/trunk/Doc/lib/libsite.tex (original) +++ python/trunk/Doc/lib/libsite.tex Mon Jun 12 10:27:13 2006 @@ -16,12 +16,13 @@ It starts by constructing up to four directories from a head and a tail part. For the head part, it uses \code{sys.prefix} and \code{sys.exec_prefix}; empty heads are skipped. For -the tail part, it uses the empty string (on Windows) or -\file{lib/python\shortversion/site-packages} (on \UNIX{} and Macintosh) -and then \file{lib/site-python}. For each of the distinct -head-tail combinations, it sees if it refers to an existing directory, -and if so, adds it to \code{sys.path} and also inspects the newly added -path for configuration files. +the tail part, it uses the empty string and then +\file{lib/site-packages} (on Windows) or +\file{lib/python\shortversion/site-packages} and then +\file{lib/site-python} (on \UNIX{} and Macintosh). For each of the +distinct head-tail combinations, it sees if it refers to an existing +directory, and if so, adds it to \code{sys.path} and also inspects +the newly added path for configuration files. \indexii{site-python}{directory} \indexii{site-packages}{directory} From buildbot at python.org Mon Jun 12 10:47:56 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 08:47:56 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060612084756.3895F1E4002@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/618 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: nick.coghlan Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Mon Jun 12 12:17:11 2006 From: python-checkins at python.org (nick.coghlan) Date: Mon, 12 Jun 2006 12:17:11 +0200 (CEST) Subject: [Python-checkins] r46893 - python/trunk/Modules/main.c Message-ID: <20060612101711.8D7321E4002@bag.python.org> Author: nick.coghlan Date: Mon Jun 12 12:17:11 2006 New Revision: 46893 Modified: python/trunk/Modules/main.c Log: Make the -m switch conform to the documentation of sys.path by behaving like the -c switch Modified: python/trunk/Modules/main.c ============================================================================== --- python/trunk/Modules/main.c (original) +++ python/trunk/Modules/main.c Mon Jun 12 12:17:11 2006 @@ -462,9 +462,10 @@ } if (module != NULL) { - /* Backup _PyOS_optind and force sys.arv[0] = module */ + /* Backup _PyOS_optind and force sys.argv[0] = '-c' + so that PySys_SetArgv correctly sets sys.path[0] to ''*/ _PyOS_optind--; - argv[_PyOS_optind] = module; + argv[_PyOS_optind] = "-c"; } PySys_SetArgv(argc-_PyOS_optind, argv+_PyOS_optind); From buildbot at python.org Mon Jun 12 12:39:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 10:39:28 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060612103928.645991E4002@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1129 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 12 12:39:45 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 10:39:45 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060612103945.68AEB1E4002@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1033 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 12 13:11:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 11:11:02 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060612111102.714FF1E4002@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/729 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot From ncoghlan at gmail.com Mon Jun 12 14:19:22 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 12 Jun 2006 22:19:22 +1000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk In-Reply-To: <20060612111102.714FF1E4002@bag.python.org> References: <20060612111102.714FF1E4002@bag.python.org> Message-ID: <448D5BCA.2060901@gmail.com> buildbot at python.org wrote: > The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. > Full details are available at: > http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/729 > > Buildbot URL: http://www.python.org/dev/buildbot/all/ > > Build Reason: > Build Source Stamp: [branch trunk] HEAD > Blamelist: nick.coghlan > > Build Had Warnings: warnings test The buildbots may be blaming me, but I don't see how any of the recent bot failures could be my fault - all I checked in was a slight change to the effect the -m switch has on sys.path[0] (it now follows the sys.path documentation and sets it to '', the same as -c does). The test failures reported since then were: test_signal on the tru64 alpha test_timeout on one of the gentoo boxes test_asynchat on a different gentoo box Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python-checkins at python.org Mon Jun 12 17:45:13 2006 From: python-checkins at python.org (kristjan.jonsson) Date: Mon, 12 Jun 2006 17:45:13 +0200 (CEST) Subject: [Python-checkins] r46894 - in python/trunk: Modules/timemodule.c Objects/exceptions.c Objects/fileobject.c Message-ID: <20060612154513.619FB1E4013@bag.python.org> Author: kristjan.jonsson Date: Mon Jun 12 17:45:12 2006 New Revision: 46894 Modified: python/trunk/Modules/timemodule.c python/trunk/Objects/exceptions.c python/trunk/Objects/fileobject.c Log: Fix the CRT argument error handling for VisualStudio .NET 2005. Install a CRT error handler and disable the assertion for debug builds. This causes CRT to set errno to EINVAL. This update fixes crash cases in the test suite where the default CRT error handler would cause process exit. Modified: python/trunk/Modules/timemodule.c ============================================================================== --- python/trunk/Modules/timemodule.c (original) +++ python/trunk/Modules/timemodule.c Mon Jun 12 17:45:12 2006 @@ -467,6 +467,14 @@ return ret; } free(outbuf); +#if defined _MSC_VER && _MSC_VER >= 1400 + /* VisualStudio .NET 2005 does this properly */ + if (buflen == 0 && errno == EINVAL) { + PyErr_SetString(PyExc_ValueError, "Invalid format string"); + return 0; + } +#endif + } } Modified: python/trunk/Objects/exceptions.c ============================================================================== --- python/trunk/Objects/exceptions.c (original) +++ python/trunk/Objects/exceptions.c Mon Jun 12 17:45:12 2006 @@ -1967,6 +1967,29 @@ if (PyDict_SetItemString(bdict, # TYPE, PyExc_ ## TYPE)) \ Py_FatalError("Module dictionary insertion problem."); +#if defined _MSC_VER && _MSC_VER >= 1400 +/* crt variable checking in VisualStudio .NET 2005 */ +#include + +static int prevCrtReportMode; +static _invalid_parameter_handler prevCrtHandler; + +/* Invalid parameter handler. Sets a ValueError exception */ +static void +InvalidParameterHandler( + const wchar_t * expression, + const wchar_t * function, + const wchar_t * file, + unsigned int line, + uintptr_t pReserved) +{ + /* Do nothing, allow execution to continue. Usually this + * means that the CRT will set errno to EINVAL + */ +} +#endif + + PyMODINIT_FUNC _PyExc_Init(void) { @@ -2096,6 +2119,13 @@ Py_FatalError("Cannot pre-allocate MemoryError instance\n"); Py_DECREF(bltinmod); + +#if defined _MSC_VER && _MSC_VER >= 1400 + /* Set CRT argument error handler */ + prevCrtHandler = _set_invalid_parameter_handler(InvalidParameterHandler); + /* turn off assertions in debug mode */ + prevCrtReportMode = _CrtSetReportMode(_CRT_ASSERT, 0); +#endif } void @@ -2103,4 +2133,9 @@ { Py_XDECREF(PyExc_MemoryErrorInst); PyExc_MemoryErrorInst = NULL; +#if defined _MSC_VER && _MSC_VER >= 1400 + /* reset CRT error handling */ + _set_invalid_parameter_handler(prevCrtHandler); + _CrtSetReportMode(_CRT_ASSERT, prevCrtReportMode); +#endif } Modified: python/trunk/Objects/fileobject.c ============================================================================== --- python/trunk/Objects/fileobject.c (original) +++ python/trunk/Objects/fileobject.c Mon Jun 12 17:45:12 2006 @@ -241,13 +241,15 @@ } if (f->f_fp == NULL) { -#ifdef _MSC_VER +#if defined _MSC_VER && _MSC_VER < 1400 /* MSVC 6 (Microsoft) leaves errno at 0 for bad mode strings, * across all Windows flavors. When it sets EINVAL varies * across Windows flavors, the exact conditions aren't * documented, and the answer lies in the OS's implementation * of Win32's CreateFile function (whose source is secret). * Seems the best we can do is map EINVAL to ENOENT. + * Starting with Visual Studio .NET 2005, EINVAL is correctly + * set by our CRT error handler (set in exceptions.c.) */ if (errno == 0) /* bad mode string */ errno = EINVAL; From buildbot at python.org Mon Jun 12 17:55:13 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 15:55:13 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060612155513.8645A1E401B@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/645 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: kristjan.jonsson Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 12 19:27:35 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 17:27:35 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060612172735.63ADF1E4002@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/344 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: kristjan.jonsson Build Had Warnings: warnings test sincerely, -The Buildbot From nnorwitz at gmail.com Mon Jun 12 19:37:36 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 12 Jun 2006 10:37:36 -0700 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk In-Reply-To: <448D5BCA.2060901@gmail.com> References: <20060612111102.714FF1E4002@bag.python.org> <448D5BCA.2060901@gmail.com> Message-ID: For known problems like the Tru64 one: http://wiki.python.org/moin/BuildBot timeout is flakey depending on the host. Haven't seen asynchat problem before. If it continues, I'll try to look at it tonight. On 6/12/06, Nick Coghlan wrote: > buildbot at python.org wrote: > > The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. > > Full details are available at: > > http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/729 > > > > Buildbot URL: http://www.python.org/dev/buildbot/all/ > > > > Build Reason: > > Build Source Stamp: [branch trunk] HEAD > > Blamelist: nick.coghlan > > > > Build Had Warnings: warnings test > > The buildbots may be blaming me, but I don't see how any of the recent bot > failures could be my fault - all I checked in was a slight change to the > effect the -m switch has on sys.path[0] (it now follows the sys.path > documentation and sets it to '', the same as -c does). > > The test failures reported since then were: > test_signal on the tru64 alpha > test_timeout on one of the gentoo boxes > test_asynchat on a different gentoo box > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From jimjjewett at gmail.com Mon Jun 12 20:06:58 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 12 Jun 2006 14:06:58 -0400 Subject: [Python-checkins] r46768 - sandbox/trunk/decimal-c/_decimal.c In-Reply-To: <20060609100317.A775E1E400A@bag.python.org> References: <20060609100317.A775E1E400A@bag.python.org> Message-ID: It might be time to be more explicit in the documentation about what a "limb" is. >From a quick scan, it seems to be a group of 3 digits that are handled together for space efficiency. But that doesn't quite make sense -- why 3-digit (=10 bit) limbs instead of 4-digit (same storage requirement) or 9digit (since you're using C longs anyway). There may be good reasons involving delayed normalization, but they should be documented. Or it may be that I'm way off base, in which case the real explanation should be documeted. -jJ On 6/9/06, mateusz.rukowicz wrote: > Author: mateusz.rukowicz > Date: Fri Jun 9 12:03:15 2006 > New Revision: 46768 > > Modified: > sandbox/trunk/decimal-c/_decimal.c > Log: > Switched to 'limbs' instead of 'digits', some functions still calculates digits - this will be removed. > > > Modified: sandbox/trunk/decimal-c/_decimal.c > ============================================================================== > --- sandbox/trunk/decimal-c/_decimal.c (original) > +++ sandbox/trunk/decimal-c/_decimal.c Fri Jun 9 12:03:15 2006 > @@ -168,6 +168,27 @@ > } > } > > +static void > +_limb_fill(long *self, long ndigits, long x) > +{ > + //TODO > +} > + > +static long > +_limb_get_digit(long *self, long ndigits, long i) > +{ > + long pos = ndigits - i - 1; > + long limb = pos / LOG; > + pos %= LOG; > + long tmp = self[limb]; > + while(pos) > + { > + tmp/=10; > + pos --; > + } > + return tmp%10; > +} > + > /* helpful macros ************************************************************/ > > /* if we later decide to call this module "squeaky" */ > @@ -374,7 +395,10 @@ > res = _new_decimalobj(type, thing->ob_size, sign, 0); > if (!res) return NULL; > for (i = 0; i < thing->ob_size; i++) > - res->digits[i] = thing->digits[i]; > + res->digits[i] = thing->digits[i]; /* DELETE */ > + > + for (i = 0; i< res->limb_count;i++) > + res->limbs[i] = thing->limbs[i]; > return res; > } > > @@ -478,6 +502,7 @@ > if (res) { > for (i = 0; i < ctx->prec; i++) > res->digits[i] = 9; > + _limb_fill(res->limbs, ctx->prec, 9); > return res; > } > } > @@ -490,6 +515,7 @@ > if (res) { > for (i = 0; i < ctx->prec; i++) > res->digits[i] = 9; > + _limb_fill(res->limbs, ctx->prec, 9); > return res; > } > } > @@ -666,10 +692,10 @@ > i = 0; > while(new->limbs[i] >= BASE) > { > + assert(i+1 < new->limb_count); > new->limbs[i] -= BASE; > new->limbs[i+1] ++; > i++; > - assert(i+1 < new->limb_count); > } > return new; > } > @@ -700,9 +726,9 @@ > new->digits[i] = self->digits[i]; > _limb_first_n_digits(self->limbs, self->ob_size, 0, new->limbs, prec); > > - if (!new) return NULL; > for (i = prec; i < self->ob_size; i++) > - if (self->digits[i] > 0) { > + if(_limb_get_digit(self->limbs,self->ob_size, i) > 0){ /* SLOW */ > +// if (self->digits[i] > 0) { > new2 = _decimal_increment(new, 1, ctx); > Py_DECREF(new); > if (!new2) > @@ -728,8 +754,9 @@ > { > decimalobject *new; > assert(expdiff > 0); > - if (self->ob_size > prec && self->digits[prec] >= 5) { > - new = _decimal_increment(tmp, 1, ctx); > +// if (self->ob_size > prec && self->digits[prec] >= 5) { > + if(self->ob_size > prec && _limb_get_digit(self->limbs, self->ob_size, prec) >= 5){ /* SLOW */ > + new = _decimal_increment(tmp, 1, ctx); > Py_DECREF(tmp); > if (!new) return NULL; > if (new->ob_size > prec) { > @@ -756,11 +783,11 @@ > tmp->digits[i] = self->digits[i]; > > last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); > - last = self->digits[prec]; > assert(self->digits[prec] == last); > if (last == 5) { > for (i = prec+1; i < self->ob_size; i++) { > - if (self->digits[i] != 0) > + if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ > + // if (self->digits[i] != 0) > return _do_round_half_up(self, prec, expdiff, ctx, tmp); > } > /* self ends in 5000...., so tmp is okay */ > @@ -785,11 +812,13 @@ > assert(last == self->digits[prec]); > if (last == 5) { > for (i = prec+1; i < self->ob_size; i++) { > - if (self->digits[i] != 0) > + if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ > +// if (self->digits[i] != 0) > return _do_round_half_up(self, prec, expdiff, ctx, tmp); > } > - if ((self->digits[prec-1] & 1) == 0) > - return tmp; > +// if ((self->digits[prec-1] & 1) == 0) > + if((_limb_get_digit(self->limbs, self->ob_size, i)&1) == 0) > + return tmp; > } > return _do_round_half_up(self, prec, expdiff, ctx, tmp); > } > @@ -875,6 +904,7 @@ > if (!new) return NULL; > while (i--) > new->digits[i] = 0; > + _limb_fill(new->limbs, new->ob_size,0); > > if (handle_Rounded(ctx, NULL) != 0) { > Py_DECREF(new); > @@ -889,6 +919,7 @@ > new->digits[0] = 0; > for (i = 1; i < new->ob_size; i++) > new->digits[i] = self->digits[i-1]; > + _limb_first_n_digits(self->limbs, self->ob_size, -1, new->limbs, new->ob_size); > prec = 1; > } else if (prec < 0) { > new = _NEW_decimalobj(2, self->sign, > @@ -896,6 +927,7 @@ > if (!new) return NULL; > new->digits[0] = 0; > new->digits[1] = 1; > + new->limbs[0] = 1; > prec = 1; > } else { > new = _decimal_get_copy(self); > @@ -918,14 +950,17 @@ > for (i = new->ob_size; i < new2->ob_size; i++) { > new2->digits[i] = 0; > } > + > + _limb_first_n_digits(new->limbs, new->ob_size, 0, new2->limbs, new2->ob_size); > Py_DECREF(new); > return new2; > } > > /* Maybe all the lost digits are 0. */ > for (i = expdiff; i < self->ob_size; i++) { > - if (self->digits[i] > 0) > - goto no_way; > +// if (self->digits[i] > 0) > + if(_limb_get_digit(self->limbs, self->ob_size, i) > 0) > + goto no_way; > } > /* All lost digits are 0, so just clobber new */ > new->ob_size = prec; > @@ -996,6 +1031,7 @@ > if (!ans) > return NULL; > ans->digits[0] = 0; > + ans->limbs[0] = 0; > return ans; > } > > @@ -1013,6 +1049,7 @@ > return NULL; > ans->digits[0] = 0; > ans->digits[1] = 1; > + ans->limbs[0] = 1; > digits = 1; > } else { > ans = _NEW_decimalobj(self->ob_size+1, self->sign, self->exp); > @@ -1021,6 +1058,7 @@ > for (i = 0; i < self->ob_size; i++) > ans->digits[i+1] = self->digits[i]; > ans->digits[0] = 0; > + _limb_first_n_digits(self->limbs, self->ob_size, -1, ans->limbs, ans->ob_size); > } > > tmp = _decimal_round(ans, digits, ctx, rounding); > @@ -1807,7 +1845,7 @@ > } > > > -static PyObject * > +static PyObject * /* TODO review that */ > _do_decimal_str(decimalobject *d, contextobject *context, int engineering) > { > char *outbuf; > @@ -1876,8 +1914,9 @@ > } > if (engineering) { /* eng has no leading zeros */ > for (i = 0, j = 0; i < imax; i++) { > - if (d->digits[j] == 0) { > - j++; > +// if (d->digits[j] == 0) { > + if(_limb_get_digit(d->limbs, d->ob_size, j) == 0){ > + j++; > } else { > break; > } > @@ -1889,8 +1928,9 @@ > SANITY_CHECK(p); > continue; > } else { > - p += sprintf(p, "%d", d->digits[j]); > - SANITY_CHECK(p); > +// p += sprintf(p, "%d", d->digits[j]); > + p += sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, j)); /* SLOW */ > + SANITY_CHECK(p); > j++; > } > } > @@ -2453,9 +2493,12 @@ > long i; > if (ISSPECIAL(self)) > return 1; > - for (i = 0; i < self->ob_size; i++) > - if (self->digits[i] != 0) return 1; > - return 0; > +// for (i = 0; i < self->ob_size; i++) > +// if (self->digits[i] != 0) return 1; > + > + for(i=0;ilimb_count;i++) > + if(self->limbs[i] != 0) return 1; > + return 0; > } > > > @@ -2877,7 +2920,7 @@ > { > item = PyTuple_GET_ITEM(digtup, i); > long x; > - if(PyInt_AsLong(item)) > + if(PyInt_Check(item)) > x = PyInt_AsLong(item); > else if (PyLong_Check(item)) { > x = PyLong_AsLong(item); > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From guido at python.org Mon Jun 12 20:33:52 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Jun 2006 11:33:52 -0700 Subject: [Python-checkins] r46887 - in python/trunk/Lib: test/test_wsgiref.py wsgiref.egg-info wsgiref/handlers.py wsgiref/headers.py wsgiref/simple_server.py wsgiref/util.py wsgiref/validate.py In-Reply-To: <20060612040433.C9FFF1E4002@bag.python.org> References: <20060612040433.C9FFF1E4002@bag.python.org> Message-ID: Phillip, I recall that you specifically promised me that the odd whitespace usage would not be carried over into the Python standard library. Please revert that part if this checkin. --Guido On 6/11/06, phillip.eby wrote: > Author: phillip.eby > Date: Mon Jun 12 06:04:32 2006 > New Revision: 46887 > > Modified: > python/trunk/Lib/test/test_wsgiref.py > python/trunk/Lib/wsgiref.egg-info > python/trunk/Lib/wsgiref/handlers.py > python/trunk/Lib/wsgiref/headers.py > python/trunk/Lib/wsgiref/simple_server.py > python/trunk/Lib/wsgiref/util.py > python/trunk/Lib/wsgiref/validate.py > Log: > Sync w/external release 0.1.2. Please see PEP 360 before making changes to external packages. > > > Modified: python/trunk/Lib/test/test_wsgiref.py > ============================================================================== > --- python/trunk/Lib/test/test_wsgiref.py (original) > +++ python/trunk/Lib/test/test_wsgiref.py Mon Jun 12 06:04:32 2006 > @@ -80,7 +80,7 @@ > > > > -def compare_generic_iter(test, make_it, match): > +def compare_generic_iter(make_it,match): > """Utility to compare a generic 2.1/2.2+ iterator with an iterable > > If running under Python 2.2+, this tests the iterator using iter()/next(), > @@ -90,7 +90,7 @@ > it = make_it() > n = 0 > for item in match: > - test.assertEqual(it[n], item) > + if not it[n]==item: raise AssertionError > n+=1 > try: > it[n] > @@ -106,10 +106,15 @@ > else: > # Only test iter mode under 2.2+ > it = make_it() > - test.assert_(iter(it) is it) > + if not iter(it) is it: raise AssertionError > for item in match: > - test.assertEqual(it.next(), item) > - test.assertRaises(StopIteration, it.next) > + if not it.next()==item: raise AssertionError > + try: > + it.next() > + except StopIteration: > + pass > + else: > + raise AssertionError("Too many items from .next()",it) > > > > @@ -203,7 +208,7 @@ > def make_it(text=text,size=size): > return util.FileWrapper(StringIO(text),size) > > - compare_generic_iter(self, make_it, match) > + compare_generic_iter(make_it,match) > > it = make_it() > self.failIf(it.filelike.closed) > > Modified: python/trunk/Lib/wsgiref.egg-info > ============================================================================== > --- python/trunk/Lib/wsgiref.egg-info (original) > +++ python/trunk/Lib/wsgiref.egg-info Mon Jun 12 06:04:32 2006 > @@ -1,6 +1,6 @@ > Metadata-Version: 1.0 > Name: wsgiref > -Version: 0.1 > +Version: 0.1.2 > Summary: WSGI (PEP 333) Reference Library > Author: Phillip J. Eby > Author-email: web-sig at python.org > > Modified: python/trunk/Lib/wsgiref/handlers.py > ============================================================================== > --- python/trunk/Lib/wsgiref/handlers.py (original) > +++ python/trunk/Lib/wsgiref/handlers.py Mon Jun 12 06:04:32 2006 > @@ -473,3 +473,20 @@ > self, sys.stdin, sys.stdout, sys.stderr, dict(os.environ.items()), > multithread=False, multiprocess=True > ) > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > +# > > Modified: python/trunk/Lib/wsgiref/headers.py > ============================================================================== > --- python/trunk/Lib/wsgiref/headers.py (original) > +++ python/trunk/Lib/wsgiref/headers.py Mon Jun 12 06:04:32 2006 > @@ -187,3 +187,19 @@ > else: > parts.append(_formatparam(k.replace('_', '-'), v)) > self._headers.append((_name, "; ".join(parts))) > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > +# > > Modified: python/trunk/Lib/wsgiref/simple_server.py > ============================================================================== > --- python/trunk/Lib/wsgiref/simple_server.py (original) > +++ python/trunk/Lib/wsgiref/simple_server.py Mon Jun 12 06:04:32 2006 > @@ -190,3 +190,16 @@ > import webbrowser > webbrowser.open('http://localhost:8000/xyz?abc') > httpd.handle_request() # serve one request, then exit > + > + > + > + > + > + > + > + > + > + > + > + > +# > > Modified: python/trunk/Lib/wsgiref/util.py > ============================================================================== > --- python/trunk/Lib/wsgiref/util.py (original) > +++ python/trunk/Lib/wsgiref/util.py Mon Jun 12 06:04:32 2006 > @@ -171,3 +171,35 @@ > def is_hop_by_hop(header_name): > """Return true if 'header_name' is an HTTP/1.1 "Hop-by-Hop" header""" > return _hoppish(header_name.lower()) > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > +# > > Modified: python/trunk/Lib/wsgiref/validate.py > ============================================================================== > --- python/trunk/Lib/wsgiref/validate.py (original) > +++ python/trunk/Lib/wsgiref/validate.py Mon Jun 12 06:04:32 2006 > @@ -124,6 +124,10 @@ > Raised in response to WSGI-spec-related warnings > """ > > +def assert_(cond, *args): > + if not cond: > + raise AssertionError(*args) > + > def validator(application): > > """ > @@ -137,8 +141,8 @@ > """ > > def lint_app(*args, **kw): > - assert len(args) == 2, "Two arguments required" > - assert not kw, "No keyword arguments allowed" > + assert_(len(args) == 2, "Two arguments required") > + assert_(not kw, "No keyword arguments allowed") > environ, start_response = args > > check_environ(environ) > @@ -148,9 +152,9 @@ > start_response_started = [] > > def start_response_wrapper(*args, **kw): > - assert len(args) == 2 or len(args) == 3, ( > - "Invalid number of arguments: %s" % args) > - assert not kw, "No keyword arguments allowed" > + assert_(len(args) == 2 or len(args) == 3, ( > + "Invalid number of arguments: %s" % (args,))) > + assert_(not kw, "No keyword arguments allowed") > status = args[0] > headers = args[1] > if len(args) == 3: > @@ -170,7 +174,7 @@ > environ['wsgi.errors'] = ErrorWrapper(environ['wsgi.errors']) > > iterator = application(environ, start_response_wrapper) > - assert iterator is not None and iterator != False, ( > + assert_(iterator is not None and iterator != False, > "The application must return an iterator, if only an empty list") > > check_iterator(iterator) > @@ -185,22 +189,22 @@ > self.input = wsgi_input > > def read(self, *args): > - assert len(args) <= 1 > + assert_(len(args) <= 1) > v = self.input.read(*args) > - assert type(v) is type("") > + assert_(type(v) is type("")) > return v > > def readline(self): > v = self.input.readline() > - assert type(v) is type("") > + assert_(type(v) is type("")) > return v > > def readlines(self, *args): > - assert len(args) <= 1 > + assert_(len(args) <= 1) > lines = self.input.readlines(*args) > - assert type(lines) is type([]) > + assert_(type(lines) is type([])) > for line in lines: > - assert type(line) is type("") > + assert_(type(line) is type("")) > return lines > > def __iter__(self): > @@ -211,7 +215,7 @@ > yield line > > def close(self): > - assert 0, "input.close() must not be called" > + assert_(0, "input.close() must not be called") > > class ErrorWrapper: > > @@ -219,7 +223,7 @@ > self.errors = wsgi_errors > > def write(self, s): > - assert type(s) is type("") > + assert_(type(s) is type("")) > self.errors.write(s) > > def flush(self): > @@ -230,7 +234,7 @@ > self.write(line) > > def close(self): > - assert 0, "errors.close() must not be called" > + assert_(0, "errors.close() must not be called") > > class WriteWrapper: > > @@ -238,7 +242,7 @@ > self.writer = wsgi_writer > > def __call__(self, s): > - assert type(s) is type("") > + assert_(type(s) is type("")) > self.writer(s) > > class PartialIteratorWrapper: > @@ -262,11 +266,11 @@ > return self > > def next(self): > - assert not self.closed, ( > + assert_(not self.closed, > "Iterator read after closed") > v = self.iterator.next() > if self.check_start_response is not None: > - assert self.check_start_response, ( > + assert_(self.check_start_response, > "The application returns and we started iterating over its body, but start_response has not yet been called") > self.check_start_response = None > return v > @@ -280,11 +284,11 @@ > if not self.closed: > sys.stderr.write( > "Iterator garbage collected without being closed") > - assert self.closed, ( > + assert_(self.closed, > "Iterator garbage collected without being closed") > > def check_environ(environ): > - assert type(environ) is DictType, ( > + assert_(type(environ) is DictType, > "Environment is not of the right type: %r (environment: %r)" > % (type(environ), environ)) > > @@ -292,11 +296,11 @@ > 'wsgi.version', 'wsgi.input', 'wsgi.errors', > 'wsgi.multithread', 'wsgi.multiprocess', > 'wsgi.run_once']: > - assert key in environ, ( > - "Environment missing required key: %r" % key) > + assert_(key in environ, > + "Environment missing required key: %r" % (key,)) > > for key in ['HTTP_CONTENT_TYPE', 'HTTP_CONTENT_LENGTH']: > - assert key not in environ, ( > + assert_(key not in environ, > "Environment should not have the key: %s " > "(use %s instead)" % (key, key[5:])) > > @@ -311,13 +315,13 @@ > if '.' in key: > # Extension, we don't care about its type > continue > - assert type(environ[key]) is StringType, ( > + assert_(type(environ[key]) is StringType, > "Environmental variable %s is not a string: %r (value: %r)" > % (key, type(environ[key]), environ[key])) > > - assert type(environ['wsgi.version']) is TupleType, ( > - "wsgi.version should be a tuple (%r)" % environ['wsgi.version']) > - assert environ['wsgi.url_scheme'] in ('http', 'https'), ( > + assert_(type(environ['wsgi.version']) is TupleType, > + "wsgi.version should be a tuple (%r)" % (environ['wsgi.version'],)) > + assert_(environ['wsgi.url_scheme'] in ('http', 'https'), > "wsgi.url_scheme unknown: %r" % environ['wsgi.url_scheme']) > > check_input(environ['wsgi.input']) > @@ -330,45 +334,45 @@ > "Unknown REQUEST_METHOD: %r" % environ['REQUEST_METHOD'], > WSGIWarning) > > - assert (not environ.get('SCRIPT_NAME') > - or environ['SCRIPT_NAME'].startswith('/')), ( > + assert_(not environ.get('SCRIPT_NAME') > + or environ['SCRIPT_NAME'].startswith('/'), > "SCRIPT_NAME doesn't start with /: %r" % environ['SCRIPT_NAME']) > - assert (not environ.get('PATH_INFO') > - or environ['PATH_INFO'].startswith('/')), ( > + assert_(not environ.get('PATH_INFO') > + or environ['PATH_INFO'].startswith('/'), > "PATH_INFO doesn't start with /: %r" % environ['PATH_INFO']) > if environ.get('CONTENT_LENGTH'): > - assert int(environ['CONTENT_LENGTH']) >= 0, ( > + assert_(int(environ['CONTENT_LENGTH']) >= 0, > "Invalid CONTENT_LENGTH: %r" % environ['CONTENT_LENGTH']) > > if not environ.get('SCRIPT_NAME'): > - assert environ.has_key('PATH_INFO'), ( > + assert_(environ.has_key('PATH_INFO'), > "One of SCRIPT_NAME or PATH_INFO are required (PATH_INFO " > "should at least be '/' if SCRIPT_NAME is empty)") > - assert environ.get('SCRIPT_NAME') != '/', ( > + assert_(environ.get('SCRIPT_NAME') != '/', > "SCRIPT_NAME cannot be '/'; it should instead be '', and " > "PATH_INFO should be '/'") > > def check_input(wsgi_input): > for attr in ['read', 'readline', 'readlines', '__iter__']: > - assert hasattr(wsgi_input, attr), ( > + assert_(hasattr(wsgi_input, attr), > "wsgi.input (%r) doesn't have the attribute %s" > % (wsgi_input, attr)) > > def check_errors(wsgi_errors): > for attr in ['flush', 'write', 'writelines']: > - assert hasattr(wsgi_errors, attr), ( > + assert_(hasattr(wsgi_errors, attr), > "wsgi.errors (%r) doesn't have the attribute %s" > % (wsgi_errors, attr)) > > def check_status(status): > - assert type(status) is StringType, ( > + assert_(type(status) is StringType, > "Status must be a string (not %r)" % status) > # Implicitly check that we can turn it into an integer: > status_code = status.split(None, 1)[0] > - assert len(status_code) == 3, ( > + assert_(len(status_code) == 3, > "Status codes must be three characters: %r" % status_code) > status_int = int(status_code) > - assert status_int >= 100, "Status code is invalid: %r" % status_int > + assert_(status_int >= 100, "Status code is invalid: %r" % status_int) > if len(status) < 4 or status[3] != ' ': > warnings.warn( > "The status string (%r) should be a three-digit integer " > @@ -376,28 +380,28 @@ > % status, WSGIWarning) > > def check_headers(headers): > - assert type(headers) is ListType, ( > + assert_(type(headers) is ListType, > "Headers (%r) must be of type list: %r" > % (headers, type(headers))) > header_names = {} > for item in headers: > - assert type(item) is TupleType, ( > + assert_(type(item) is TupleType, > "Individual headers (%r) must be of type tuple: %r" > % (item, type(item))) > - assert len(item) == 2 > + assert_(len(item) == 2) > name, value = item > - assert name.lower() != 'status', ( > + assert_(name.lower() != 'status', > "The Status header cannot be used; it conflicts with CGI " > "script, and HTTP status is not given through headers " > "(value: %r)." % value) > header_names[name.lower()] = None > - assert '\n' not in name and ':' not in name, ( > + assert_('\n' not in name and ':' not in name, > "Header names may not contain ':' or '\\n': %r" % name) > - assert header_re.search(name), "Bad header name: %r" % name > - assert not name.endswith('-') and not name.endswith('_'), ( > + assert_(header_re.search(name), "Bad header name: %r" % name) > + assert_(not name.endswith('-') and not name.endswith('_'), > "Names may not end in '-' or '_': %r" % name) > - assert not bad_header_value_re.search(value), ( > - "Bad header value: %r (bad char: %r)" > + if bad_header_value_re.search(value): > + assert_(0, "Bad header value: %r (bad char: %r)" > % (value, bad_header_value_re.search(value).group(0))) > > def check_content_type(status, headers): > @@ -409,13 +413,13 @@ > if name.lower() == 'content-type': > if code not in NO_MESSAGE_BODY: > return > - assert 0, (("Content-Type header found in a %s response, " > + assert_(0, ("Content-Type header found in a %s response, " > "which must not return content.") % code) > if code not in NO_MESSAGE_BODY: > - assert 0, "No Content-Type header found in headers (%s)" % headers > + assert_(0, "No Content-Type header found in headers (%s)" % headers) > > def check_exc_info(exc_info): > - assert exc_info is None or type(exc_info) is type(()), ( > + assert_(exc_info is None or type(exc_info) is type(()), > "exc_info (%r) is not a tuple: %r" % (exc_info, type(exc_info))) > # More exc_info checks? > > @@ -423,6 +427,6 @@ > # Technically a string is legal, which is why it's a really bad > # idea, because it may cause the response to be returned > # character-by-character > - assert not isinstance(iterator, str), ( > + assert_(not isinstance(iterator, str), > "You should not return a string as your application iterator, " > "instead return a single-item list containing that string.") > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python-checkins at python.org Mon Jun 12 20:38:37 2006 From: python-checkins at python.org (matt.fleming) Date: Mon, 12 Jun 2006 20:38:37 +0200 (CEST) Subject: [Python-checkins] r46895 - in sandbox/trunk/pdb: mconnection.py mpdb.py test/tcptest.py Message-ID: <20060612183837.C57821E4002@bag.python.org> Author: matt.fleming Date: Mon Jun 12 20:38:36 2006 New Revision: 46895 Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/test/tcptest.py Log: Patched with some changes from Rocky (my mentor), the pdbserver command (think gdbserver) now works better and allows a mpdb 'client' to connect and play with the session. Also changed the interface for the connection classes to work with lines instead of bytes. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Mon Jun 12 20:38:36 2006 @@ -7,31 +7,21 @@ class MServerConnectionInterface(object): """ This is an abstract class that specifies the interface a server - connection class must implement. + connection class must implement. If a target is given, we'll + set up a connection on that target """ - def accept(self, console, addr): - """ This method is called when a connection from a debugger - is accepted by this server. - """ - raise NotImplementedError, NotImplementedMessage + def connect(self, target): + """Use this to set the target. It can also be specified + on the initialization.""" + raise NotImplementedError, NotImplementedMessage def disconnect(self): - """ This method is called to disconnect any debuggers that - are connected and to stop accepting any more connections from - debuggers. - """ + """ This method is called to disconnect connections.""" raise NotImplementedError, NotImplementedMessage - def listen(self): - """ This method is called when a server is initially - configured to listen for connections from debuggers. - """ - raise NotImplementedError, NotImplementedMessage - - def setup(self): - """ This is the only method that is called by the debugger. - It must handle setting up a connection to listen on and accept - a debugger connection. + def readline(self): + """ This method reads a line of data of maximum length 'bufsize' + from the connected debugger. """ raise NotImplementedError, NotImplementedMessage @@ -41,158 +31,135 @@ """ raise NotImplementedError, NotImplementedMessage - def read(self, bufsize=1024): - """ This method is reads a maximum of 'bufsize' bytes from a - connected debugger. - """ - raise NotImplementedError, NotImplementedMessage - class MClientConnectionInterface(object): """ This is the interface that a client connection should implement. """ - def setup(self): - """ This method is called by a debugger to connect to a server. """ + def connect(self, target): + """ This method is called to connect to a target. """ raise NotImplementedError, NotImplementedMessage - - def write(self, msg): - """ This method is called by a debugger to write 'msg' to the - server it is connected to. - """ + def disconnect(self): + """ This method use to disconnect a target.""" raise NotImplementedError, NotImplementedMessage - - - def read(self, bufsize=1024): - """ This method reads a maximum of 'bufsize' bytes from the connection - to the server. + + def readline(self, bufsize): + """ This method reads a line of data of maxium length 'bufisze' + from a connected target. """ raise NotImplementedError, NotImplementedMessage - def disconnect(self): - """ This method is called by a debugger to disconnect from a - server. + def write(self, msg): + """ This method is called write 'msg' to the connected target. """ raise NotImplementedError, NotImplementedMessage - + + +### This might go in a different file class MServerConnectionSerial(MServerConnectionInterface): - """ This is a server connection class that allows a debugger - to connect via a serial line. A serial device must be specified - (e.g. /dev/ttyS0, COM1, etc.). + + """ This server connection class that allows a connection to a + target via a serial line. """ - def __init__(self, device): - self._dev = device + + def __init__(self): MServerConnectionInterface.__init__(self) + self.input = None + self.output = None - def setup(self): + def connect(self, device): + """ Create our fileobject by opening the serial device for + this connection. A serial device must be specified, + (e.g. /dev/ttyS0, /dev/ttya, COM1, etc.). + """ + self._dev = device self.output = open(self._dev, 'w') self.input = open(self._dev, 'r') - def listen(self): - pass + def disconnect(self): + """ Close the serial device. """ + self.output.close() + self.input.close() - def accept(self, debugger, addr): - pass + def readline(self, bufsize=2048): + line = self.input.recv(bufsize) + return line def write(self, msg): self.output.write(msg) - def read(self, bufsize=1024): - return self.input.read() - - def disconnect(self): - self.output.close() - self.input.close() +MClientConnectionSerial = MServerConnectionSerial +### This might go in a different file class MServerConnectionTCP(MServerConnectionInterface): - """ This is an implementation of a server class that uses the TCP - protocol as its means of communication. Debuggers wishing to connect - to this target must use this syntax for the server command, - - `target tcp hostname:port - + """This is an implementation of a server class that uses the TCP + protocol as its means of communication. """ - def __init__(self, addr): - """ 'addr' specifies the hostname and port combination of - the server. - """ + def __init__(self): + self.listening = False + self.output = self.input = None MServerConnectionInterface.__init__(self) + + def connect(self, addr): + """Set to allow a connection from a client. 'addr' specifies + the hostname and port combination of the server. + """ h,p = addr.split(':') self.host = h self.port = int(p) - - def setup(self): - self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self._sock.bind((self.host, self.port)) - self.listen() - - def listen(self): - """ This method is not usually called except by this object's - setup() method. - """ - self._sock.listen(1) - debugger, addr = self._sock.accept() - self.accept(debugger, addr) - - def accept(self, debugger, addr): - self.output = debugger - self.input = debugger + if not self.listening: + self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + self._sock.bind((self.host, self.port)) + self._sock.listen(1) + self.listening = True + self.output, addr = self._sock.accept() + self.input = self.output - def write(self, msg): - self.output.sendall(msg) - - def read(self, bufsize=1024): - return self.input.recv(bufsize) - def disconnect(self): self.output.close() self._sock.close() + self.listening = False -class MClientConnectionSerial(MClientConnectionInterface): - """ A class that allows a connection to be made from a debugger - to a server via a serial line. Specify the serial device it is - connected to (e.g. /dev/ttya). - """ - def __init__(self, device): - """ Specify the serial device. """ - MClientConnectionInterface.__init__(self) - self._dev = device - self.input = None - self.output = None - - def setup(self): - """ Create our fileobject by opening the serial device for - this connection. - """ - self.output = open(self._dev, "w") + def readline(self, bufsize=2048): + line = self.input.recv(bufsize) + if line[-1].isalpha(): line += '\n' + return line - def disconnect(self): - """ Close the serial device. """ - self.output.close() + def flush(self): + pass + + def write(self, msg): + self.output.sendall(msg) class MClientConnectionTCP(MClientConnectionInterface): """ A class that allows a connection to be made from a debugger - to a server via TCP. Specify the address of the server - (e.g. host:2020). + to a server via TCP. """ - def __init__(self, addr): + def __init__(self): """ Specify the address to connection to. """ MClientConnectionInterface.__init__(self) + + def connect(self, addr): + """Connect to the server. 'input' reads data from the + server. 'output' writes data to the server. Specify the + address of the server (e.g. host:2020). """ + h, p = addr.split(':') self.host = h self.port = int(p) - - def setup(self): - """ Connect to the server. 'input' reads data from the - server. 'output' writes data to the server. - """ self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self._sock.connect((self.host, self.port)) def write(self, msg): - self._sock.send(msg) + self._sock.sendall(msg) + + def readline(self, bufsize=2048): + line = self._sock.recv(bufsize) + return line + - def read(self, bufsize=1024): - return self._sock.recv(bufsize) + def flush(self): + pass def disconnect(self): """ Close the socket to the server. """ Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Mon Jun 12 20:38:36 2006 @@ -43,12 +43,15 @@ """ pydb.Pdb.__init__(self, completekey, stdin, stdout) self.prompt = '(MPdb)' + self.target = 'local' # local connections by default + self.connection = None def _rebind_input(self, new_input): """ This method rebinds the debugger's input to the object specified by 'new_input'. """ self.stdin.flush() + self.use_rawinput = False self.stdin = new_input def _rebind_output(self, new_output): @@ -58,9 +61,23 @@ self.stdout.flush() self.stdout = new_output + def remote_onecmd(self, line): + """ All commands in 'line' are sent across this object's connection + instance variable. + """ + self.connection.write(line) + ret = self.connection.readline() + # The output from the command that we've just sent to the server + # is returned along with the prompt of that server. So we keep reading + # until we find our prompt. + while "(MPdb)" not in ret: + ret += self.connection.readline() + self.msg_nocr(ret) + return + def msg_nocr(self, msg, out=None): - """Common routine for reporting messages (no carriage return). - Derived classed may want to override this to capture output. + """Common routine for reporting messages. Derived classes may want + to override this to capture output. """ do_print = True if self.logging: @@ -71,7 +88,6 @@ if out is None: out = self.stdout print >> out, msg, - out.flush() # Debugger commands def do_attach(self, addr): @@ -86,7 +102,6 @@ (see the "directory" command). You can also use the "file" command to specify the program, and to load its symbol table. """ - def do_target(self, args): """ Connect to a target machine or process. The first argument is the type or protocol of the target machine @@ -99,19 +114,60 @@ List of target subcommands: target serial -- Use a remote computer via a serial line -target socket -- Use a remote computer via a socket connection +target tcp -- Use a remote computer via a socket connection """ - cls, addr = args.split(' ') - if '.' in cls: - base = cls[:cls.rfind('.')] - cls = cls[cls.rfind('.')+1:] - exec 'from ' + base + ' import ' + cls + target, addr = args.split(' ') + if self.target == 'remote': + self.msg('Already connected to a remote machine.') + return + if target == 'tcp': + # TODO: need to save state of current debug session + if self.connection: self.connection.disconnect() + try: + from mconnection import MClientConnectionTCP + self.connection = MClientConnectionTCP() + except ImportError: + self.msg('Could not import MClientConnectionTCP') + return + elif target == 'serial': + if self.connection: self.connection.disconnect() + try: + from mconnection import MClientConnectionSerial + self.connection = MClientConnectionSerial() + except ImportError: + self.msg('Could not import MClientConnectionSerial') + return else: - __import__(cls) - self.connection = eval(cls+'(addr)') - self.connection.setup() - # XXX currently this method doesn't do anything - + if '.' in target: + if self.connection: self.connection.disconnect() + # We dynamically load the class for the connection + base = cls[:cls.rfind('.')] + cls = cls[cls.rfind('.')+1:] + exec 'from ' + base + ' import ' + cls + else: + try: + __import__(target) + except ImportError: + self.msg('Unknown target type') + return + self.connection = eval(target+'()') + self.connection.connect(addr) + # This interpreter no longer interprets commands but sends + # them straight across this object's connection to a server. + self.prompt = "" # Get our prompt from the server now + line = self.connection.readline() + self.msg_nocr(line) + self.msg = self.connection.write + self.onecmd = self.remote_onecmd + self.target = 'remote' + + def do_detach(self, args): + """ Detach a process or file previously attached. +If a process, it is no longer traced, and it continues its execution. If +you were debugging a file, the file is closed and Pdb no longer accesses it. +""" + pass + def do_pdbserver(self, args): """ Allow a debugger to connect to this session. The first argument is the type or protocol that is used for this connection @@ -120,24 +176,37 @@ The next argument is protocol specific arguments (e.g. hostname and port number for a TCP connection, or a serial device for a serial line connection). The next argument is the filename of the script that is -being debugged. The rest of the arguments are passed to the script file -and are optional. For more information on the arguments for a +being debugged. The rest of the arguments are passed as arguments to the +script file and are optional. For more information on the arguments for a particular protocol, type `help pdbserver ' followed by the protocol name. The syntax for this command is, `pdbserver ConnectionClass comm scriptfile [args ...]' """ - cls, comm, scriptfile_and_args = args.split(' ') - if '.' in cls: - base = cls[:cls.rfind('.')] - cls = cls[cls.rfind('.')+1:] - exec 'from ' + base + ' import ' + cls + target, comm, scriptfile_and_args = args.split(' ') + if self.target == 'remote': + self.msg('Already connected remotely') + return + if target == 'tcp': + try: + from mconnection import MServerConnectionTCP + self.connection = MServerConnectionTCP() + except ImportError: + self.msg('Could not load MServerConnectionTCP class') + return else: - __import__(cls) - self.connection = eval(cls+'(comm)') - self.connection.setup() - self._rebind_output(self.connection.output) + if '.' in target: + base = target[:target.rfind('.')] + target = target[target.rfind('.')+1:] + exec 'from ' + base + ' import ' + target + else: + __import__(target) + self.connection = eval(target+'()') + self.connection.connect(comm) + self.target = 'remote' + self._rebind_output(self.connection) + self._rebind_input(self.connection) def main(options): opts = options[0] @@ -170,6 +239,7 @@ mainpyfile + " will be restarted" # Utility functions + # Parse arguments def parse_opts(): parser = OptionParser() Modified: sandbox/trunk/pdb/test/tcptest.py ============================================================================== --- sandbox/trunk/pdb/test/tcptest.py (original) +++ sandbox/trunk/pdb/test/tcptest.py Mon Jun 12 20:38:36 2006 @@ -12,30 +12,26 @@ class TestTCPConnections(unittest.TestCase): def testClientConnectAndRead(self): - # We need to exercise finer control over the server's socket so we're - # not using the setup() method. - self.client = MClientConnectionTCP(__addr__) - self.server = MServerConnectionTCP(__addr__) - self.server._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.server._sock.bind((self.server.host, self.server.port)) - self.server._sock.listen(1) pid = os.fork() - for i in range(30): - try: - self.client.setup() - break - except socket.error: - pass - time.sleep(5) - debugger, addr = self.server._sock.accept() - self.server.accept(debugger, addr) - self.server.write("good") - line = self.client.read() - self.assertEqual("good", line, "Could not read from server") - self.client.disconnect() - self.server.disconnect() + if pid == 0: + # Child + self.server = MServerConnectionTCP() + self.server.connect(__addr__) + self.server.write("good") + self.server.disconnect() + else: + # Parent + self.client = MClientConnectionTCP() + for i in range(30): + try: + self.client.connect(__addr__) + break + except socket.error: + pass + time.sleep(5) + line = self.client.readline() + self.assertEqual("good", line, "Could not read from server") + self.client.disconnect() - - if __name__ == '__main__': unittest.main() From python-checkins at python.org Mon Jun 12 21:13:41 2006 From: python-checkins at python.org (tim.peters) Date: Mon, 12 Jun 2006 21:13:41 +0200 (CEST) Subject: [Python-checkins] r46896 - peps/trunk/pep-0237.txt Message-ID: <20060612191341.2B4D11E4006@bag.python.org> Author: tim.peters Date: Mon Jun 12 21:13:38 2006 New Revision: 46896 Modified: peps/trunk/pep-0237.txt Log: Record pronouncement about trailing 'L' on hex/oct output. Modified: peps/trunk/pep-0237.txt ============================================================================== --- peps/trunk/pep-0237.txt (original) +++ peps/trunk/pep-0237.txt Mon Jun 12 21:13:38 2006 @@ -207,6 +207,7 @@ C. The trailing 'L' is dropped from repr(), and made illegal on input. (If possible, the 'long' type completely disappears.) + The trailing 'L' is also dropped from hex() and oct(). Phase A will be implemented in Python 2.2. @@ -315,6 +316,13 @@ These issues, previously open, have been resolved. + - hex() and oct() applied to longs will continue to produce a + trailing 'L' until Python 3000. The original text above wasn't + clear about this, but since it didn't happen in Python 2.4 it + was thought better to leave it alone. BDFL pronouncement here: + + http://mail.python.org/pipermail/python-dev/2006-June/065918.html + - What to do about sys.maxint? Leave it in, since it is still relevant whenever the distinction between short and long ints is still relevant (e.g. when inspecting the type of a value). From mateusz.rukowicz at vp.pl Mon Jun 12 21:55:43 2006 From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz) Date: Mon, 12 Jun 2006 21:55:43 +0200 Subject: [Python-checkins] r46768 - sandbox/trunk/decimal-c/_decimal.c In-Reply-To: References: <20060609100317.A775E1E400A@bag.python.org> Message-ID: <448DC6BF.9090706@vp.pl> Hi, Jim Jewett wrote: > It might be time to be more explicit in the documentation about what a > "limb" is. > Ok, I will do that. >> From a quick scan, it seems to be a group of 3 digits that are handled > > together for space efficiency. But that doesn't quite make sense -- > why 3-digit (=10 bit) limbs instead of 4-digit (same storage > requirement) or 9digit (since you're using C longs anyway). There may > be good reasons involving delayed normalization, but they should be > documented. Or it may be that I'm way off base, in which case the > real explanation should be documeted. > > -jJ > At the moment there is a group of 3 digits, to make debugin easier, later I will change it to higher value. I must appologize - I wasn't aware of existing this mailing list :S. I will take all your advices into account. Also I will make it C89 compatible soon, and repair indenting. Sorry for that. Regards, Mateusz Rukowicz. From neal at metaslash.com Mon Jun 12 22:12:30 2006 From: neal at metaslash.com (Neal Norwitz) Date: Mon, 12 Jun 2006 16:12:30 -0400 Subject: [Python-checkins] Python Regression Test Failures basics (2) Message-ID: <20060612201230.GA18272@python.psfb.org> case $MAKEFLAGS in \ *-s*) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py -q build;; \ *) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py build;; \ esac running build running build_ext db.h: found (4, 1) in /usr/include db lib: using (4, 1) db-4.1 sqlite: found /usr/include/sqlite3.h /usr/include/sqlite3.h: version 3.2.1 INFO: Can't locate Tcl/Tk libs and/or headers running build_scripts [34085 refs] ./python -E -c 'import sys ; from distutils.util import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform [8674 refs] find ./Lib -name '*.py[co]' -print | xargs rm -f ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat Exception in thread Thread-1: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') Exception in thread Thread-2: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') test test_asynchat failed -- errors occurred in test.test_asynchat.TestAsynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8679 refs] [8679 refs] [8679 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9027 refs] [9027 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8890 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] this bit of output is from a test of stdout in a different process ... [8674 refs] [8674 refs] [8890 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8674 refs] [8675 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8674 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 285 tests OK. 1 test failed: test_asynchat 30 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [452054 refs] make: [test] Error 1 (ignored) ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat Exception in thread Thread-1: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') Exception in thread Thread-2: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') test test_asynchat failed -- errors occurred in test.test_asynchat.TestAsynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8679 refs] [8679 refs] [8679 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9027 refs] [9027 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8890 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] this bit of output is from a test of stdout in a different process ... [8674 refs] [8674 refs] [8890 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8674 refs] [8675 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8674 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 285 tests OK. 1 test failed: test_asynchat 30 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [451585 refs] make: *** [test] Error 1 From python-checkins at python.org Mon Jun 12 22:39:26 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Mon, 12 Jun 2006 22:39:26 +0200 (CEST) Subject: [Python-checkins] r46897 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060612203926.7B8BB1E4002@bag.python.org> Author: mateusz.rukowicz Date: Mon Jun 12 22:39:25 2006 New Revision: 46897 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Now code is C89 compatible, added little explanation of what 'limb' is. Indentation fixes cooming soon. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Mon Jun 12 22:39:25 2006 @@ -70,6 +70,14 @@ #include /* integer arithmetic */ +/* Little explanation: + * Digits are grouped into limbs, that is, every limb keeps LOG digits. + * Arithmetic perfomred on those limbs are very the same that in base 10 + * but insetad of dealing "digits" 0-9, we deal with "digits" 0-(BASE -1) + * Note, that first limb is least significant. + * Further in code when I say "limbs" I mean one calculation unit, and + * when I say "digit" I mean digit between 0-9 (limb consists of LOG digits). + */ /* takes new_digits from self[], starting from start_at digit, *digits* not limbs */ /* returns digit after that we returned (useful for rounding) */ @@ -154,6 +162,7 @@ return last_digit; } +/* cuts least significant digit */ static void _limb_cut_one_digit(long *self, long ndigits) { @@ -374,7 +383,6 @@ #define GETNAN(op) ( ( (op)->sign == SIGN_POSNAN || (op)->sign == SIGN_NEGNAN ) ? \ 1 : ( ( (op)->sign == SIGN_POSSNAN || (op)->sign == SIGN_NEGSNAN ) ? 2 : 0 ) ) -//#define ISINF(op) ((op)->sign == SIGN_POSINF || (op)->sign == SIGN_NEGINF) #define ISINF(op) ( ( (op)->sign == SIGN_POSINF || (op)->sign == SIGN_NEGINF) ? \ ((op)->sign == SIGN_POSINF ? 1 : -1) : 0 ) @@ -561,7 +569,6 @@ } assert(PyDecimal_Check(thing)); /* we want to return a NaN, but keep diagnostics */ -// sign = ((thing->sign&1 == 0) ? SIGN_POSNAN : SIGN_NEGNAN); if(thing->sign&1) /* neg */ sign = SIGN_NEGNAN; else @@ -831,35 +838,10 @@ if (!new) return NULL; - /* determine if we need a new digit */ -// for (spot = 0; spot < self->ob_size; spot++) { -// if (self->digits[spot] != 9) { -// spot = -1; -// break; -// } -// } -// if (spot == -1) { - /* no new digit needed */ -// new->ob_size--; -// for (spot = 0; spot < self->ob_size; spot++) -// new->digits[spot] = self->digits[spot]; -// } else { -// for (spot = 0; spot < self->ob_size; spot++) -// new->digits[spot+1] = self->digits[spot]; -// new->digits[0] = 0; -// } - -// spot = new->ob_size-1; -// new->digits[spot]++; -// while (new->digits[spot] == 10) { -// new->digits[spot] = 0; -// spot--; - // assert(spot >= 0); -// new->digits[spot]++; -// } for(i=0;ilimb_count;i++) new->limbs[i] = self->limbs[i]; + if(self->limb_count != new->limb_count) new->limbs[new->limb_count - 1] = 0; /* we have new limb */ @@ -894,7 +876,7 @@ /* Round away from 0. */ static decimalobject * _round_up(decimalobject *self, long prec, long expdiff, contextobject *ctx) -{ /* XXX temporary solution with limbs */ +{ long i; decimalobject *new = _NEW_decimalobj(prec, self->sign, self->exp + expdiff); decimalobject *new2 = NULL; @@ -905,7 +887,6 @@ for (i = prec; i < self->ob_size; i++) if(_limb_get_digit(self->limbs,self->ob_size, i) > 0){ /* SLOW */ -// if (self->digits[i] > 0) { new2 = _decimal_increment(new, 1, ctx); Py_DECREF(new); if (!new2) @@ -931,7 +912,6 @@ { decimalobject *new; assert(expdiff > 0); -// if (self->ob_size > prec && self->digits[prec] >= 5) { if(self->ob_size > prec && _limb_get_digit(self->limbs, self->ob_size, prec) >= 5){ /* SLOW */ new = _decimal_increment(tmp, 1, ctx); Py_DECREF(tmp); @@ -960,11 +940,9 @@ tmp->digits[i] = self->digits[i]; last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); -// assert(self->digits[prec] == last); if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ - // if (self->digits[i] != 0) return _do_round_half_up(self, prec, expdiff, ctx, tmp); } /* self ends in 5000...., so tmp is okay */ @@ -988,10 +966,8 @@ if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ -// if (self->digits[i] != 0) return _do_round_half_up(self, prec, expdiff, ctx, tmp); } -// if ((self->digits[prec-1] & 1) == 0) if((_limb_get_digit(self->limbs, self->ob_size, prec-1)&1) == 0) return tmp; } @@ -1133,12 +1109,11 @@ /* Maybe all the lost digits are 0. */ for (i = self->ob_size - expdiff; i < self->ob_size; i++) { -// if (self->digits[i] > 0) if(_limb_get_digit(self->limbs, self->ob_size, i) > 0) goto no_way; } /* All lost digits are 0, so just clobber new */ -// new->ob_size = prec; + while(new->ob_size > prec) { _limb_cut_one_digit(new->limbs, new->ob_size); /* VERY SLOW */ @@ -1154,15 +1129,16 @@ no_way: /* Now rounding starts. We still own "new". */ rnd_func = round_funcs[rounding]; -// if (prec != ctx->prec) { -// ctx2 = (contextobject *)context_copy(ctx); -// if (!ctx2) { -// Py_DECREF(new); -// return NULL; -// } -// ctx2->prec = prec; -// ctx = ctx2; - /* }*/ /* XXX here is quite subtle bug - we copy context, and set flags in copied context + /* + if (prec != ctx->prec) { + ctx2 = (contextobject *)context_copy(ctx); + if (!ctx2) { + Py_DECREF(new); + return NULL; + } + ctx2->prec = prec; + ctx = ctx2; + }*/ /* XXX here is quite subtle bug - we copy context, and set flags in copied context after that, they are lost, not sure if we really need function above, I'll comment it */ @@ -1251,12 +1227,10 @@ if (!tmp) return NULL; -// if (tmp->digits[0] == 0 && tmp->ob_size > 1) { if(_limb_get_digit(tmp -> limbs, tmp->ob_size, 0) == 0 && tmp->ob_size > 1){ /* We need one digit less, just clobber tmp. */ for (i = 0; i < tmp->ob_size-1; i++) - tmp->digits[i] = tmp->digits[i+1]; -// _limb_cut_one_digit(tmp->limbs, tmp->ob_size); + tmp->digits[i] = tmp->digits[i+1]; /* TODO make last digit 0 */ tmp->ob_size--; } tmp->exp = exp; @@ -1579,10 +1553,8 @@ next: /* Adjusted sizes are not equal. */ -// if (adj1 > adj2 && self->digits[0] != 0) if(adj1 > adj2 && _limb_get_digit(self->limbs, self->ob_size, 0) != 0) return 1 - 2*(self->sign & 1); /* 0 -> 1, 1 -> -1 */ -// else if (adj1 < adj2 && other->digits[0] != 0) else if(adj1 < adj2 && _limb_get_digit(other->limbs, other->ob_size, 0) != 0) return -1 + 2*(other->sign & 1); @@ -1595,7 +1567,6 @@ Py_XDECREF(ctx2); - /* XXX: WTF? */ ans = _do_decimal_subtract(self, other, ctx2); if (!ans) return 0; @@ -1820,7 +1791,6 @@ } /* strip trailing 0s from dup */ -// while (dup->digits[dup->ob_size - 1] == 0) { while(_limb_get_digit(dup->limbs, dup->ob_size, dup->ob_size -1) == 0){ _limb_cut_one_digit(dup->limbs,dup->ob_size); dup->ob_size --; @@ -2116,11 +2086,6 @@ SANITY_CHECK(p); /* check for digits != {0, } */ -// if (d->ob_size != 1 || d->digits[0] != 0) { -// for (i = 0; i < d->ob_size; i++) { -// p += sprintf(p, "%d", d->digits[i]); -// SANITY_CHECK(p); -// } if(d->ob_size != 1 || _limb_get_digit(d->limbs, d->ob_size, 0) != 0) { for(i=0;i< d->ob_size;i++) @@ -2152,20 +2117,14 @@ SANITY_CHECK(p); } if (d->exp < 0 && d->exp >= -6) { - /* figure out the right number of digits */ -// i = sprintf(p, "0.%0*d", abs(d->exp), 0); - /* and write those as zeros */ -// for (; i > 0; i--) { -// *p++ = '0'; -// } + i = 0; *p++ = '0'; *p++ = '.'; - for(;iexp);i++) + for (;iexp);i++) *p++ = '0'; SANITY_CHECK(p); } else { -// roundexp = ((d->exp - 1)/3 + 1) * 3; /* nearest gt mult of 3 */ roundexp = d->exp; while(roundexp%3) roundexp ++; @@ -2274,7 +2233,6 @@ { adjexp --; dotplace ++; -// extra_zeros ++; } /* now all we have to do, is to put it to the string =] */ @@ -2302,61 +2260,6 @@ /* that was a way easier way =] */ -// if (engineering) { -// dotplace = (leftdigits-1)%3+1; -// adjexp = leftdigits -1 - (leftdigits-1)%3; -// } else { -// dotplace = 1; -// adjexp = leftdigits - 1; -// } -// if (d->exp == 0) { -// dotplace = -1; /* no dot */ -// } else if (d->exp < 0 && adjexp >= 0) { -// dotplace = leftdigits; -// } else if (d->exp < 0 && adjexp >= -6) { -// if (!engineering) { /* eng has no leading zeros */ -// for (i = leftdigits; i < 0; i++) { -// *p++ = '0'; -// SANITY_CHECK(p); -// } -// *p++ = '0'; -// } -// } else { -// if (d->ob_size <= dotplace) -// dotplace = -1; - -// if (adjexp) { -// append_E = 1; -// append_adjexp = 1; -// } -// } - -// imax = d->ob_size; -// if (dotplace > 0) { -// imax++; -// } -// if (engineering) { /* eng has no leading zeros */ -// for (i = 0, j = 0; i < imax; i++) { -// if (d->digits[j] == 0) { -// if(_limb_get_digit(d->limbs, d->ob_size, j) == 0){ -// j++; -// } else { -// break; -// } -// } -// } -// for (i = 0, j = 0; i < imax; i++) { -// if (i == dotplace) { -// *p++ = '.'; -// SANITY_CHECK(p); -// continue; -// } else { -// p += sprintf(p, "%d", d->digits[j]); -// p += sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, j)); /* SLOW */ -// SANITY_CHECK(p); -// j++; -// } -// } if(adjexp) { @@ -2384,14 +2287,6 @@ *p++ = 0; SANITY_CHECK(p); - /* if engineering and (just a sign | empty | starts with a dot or an E) */ -// if (engineering && ((p - &outbuf[1] > (d->sign & 1)) || outbuf[1] == '.' || -// outbuf[1] == 'E' || outbuf[1] == 'e')) { -// outbuf[0] = '0'; -// p = outbuf; -// } else { -// p = &outbuf[1]; -// } p = &outbuf[1]; SANITY_CHECK(p); ret = PyString_FromString(p); @@ -2748,19 +2643,20 @@ max_size = _limb_add(o1->limbs, o1->ob_size, o2->limbs, o2->ob_size, ret->limbs); ret->ob_size = max_size; ret->limb_count = (max_size + LOG -1)/LOG; -// Py_DECREF(o1); -// Py_DECREF(o2); -// if(shouldround) -// { -// fixed = _decimal_fix(ret, ctx); -// Py_DECREF(ret); -// if(!fixed) -// { -// return NULL; -// } -// return fixed; -// } -// return ret; +/* + Py_DECREF(o1); + Py_DECREF(o2); + if(shouldround) + { + fixed = _decimal_fix(ret, ctx); + Py_DECREF(ret); + if(!fixed) + { + return NULL; + } + return fixed; + } + return ret; */ } Py_DECREF(o1); @@ -3119,7 +3015,6 @@ buf[0] = (self->sign & 1 ? '-' : '+'); for (i = 0; i < max; i++) { if (i < self->ob_size) -// buf[i+1] = self->digits[i] + 48; buf[i+1] = _limb_get_digit(self->limbs, self->ob_size, i) + '0'; /* SLOW */ else buf[i+1] = '0'; @@ -3163,7 +3058,6 @@ for (i = 0; i < max; i++) { if (i < self->ob_size) -// res = res * 10 + self->digits[i]; res = res * 10 + _limb_get_digit(self->limbs, self->ob_size, i); /* SLOW */ else res *= 10; @@ -3192,8 +3086,6 @@ long i; if (ISSPECIAL(self)) return 1; -// for (i = 0; i < self->ob_size; i++) -// if (self->digits[i] != 0) return 1; for(i=0;ilimb_count;i++) if(self->limbs[i] != 0) return 1; @@ -3536,105 +3428,6 @@ } return new; -// long ipos = 0; /* start of integral digits */ -// long dpos = -1; /* decimal point location */ -// long dend = len; /* end of integral/decimal digits (last digit+1) */ -// char *p = str; -// char sign = 0; -// decimalobject *new; -// long exp = 0, i, ndigits; - - /* optional sign */ -// if (*p == '+') { -// ++p; -// ipos = 1; -// } else if (*p == '-') { -// ++p; -// ipos = 1; -// sign = 1; -// } - -// do { -// if (*p == '.') { -// dpos = p-str; -// } else if (*p == 'e' || *p == 'E') { -// dend = p-str; -// if (dend == ipos || (dpos >= 0 && dend == ipos+1)) - /* no digits between sign and E */ -// goto err; -// ++p; -// break; -// } else if (!isdigit(*p)) { -// goto err; -// } -// } while (*++p); - - /* exponential part or end of string */ -// if (*p == 0) { -// if (ipos == dpos && dpos == dend-1) - /* no digits at all */ -// goto err; -// goto calculate; -// } else if (*p == '-' || *p == '+') { -// ++p; -// } - -// do { - /* optional exponential digits */ -// if (!isdigit(*p)) -// goto err; -// } while (*++p); - -// calculate: -// if (dend < len) -// exp = atol(str + dend + 1); -// if (dpos >= 0) - /* specified fractional digits, reduce exp */ -// exp -= dend - dpos - 1; - - /* If it's not a zero, strip leading 0s */ -// for (p = str+ipos; p-str < dend; ++p) { -// if (*p != '0') { - /* first nonzero digit */ -// ipos = (p-str); -// goto finish; -// } -// } - /* it's a zero */ -// dend = (dpos == ipos ? ipos+2 : ipos+1); - -// finish: -// ndigits = dend - ipos - (dpos<0 ? 0 : 1); -// new = _new_decimalobj(type, ndigits, sign, exp); -// if (!new) -// return NULL; -// i = 0; -// for (p = str+ipos; p-str < dend; ++p) { -// if (*p == '.') continue; -// new->digits[i] = *p - 48; -// i++; -// } - -// long mult = 1; -// long limb = 0; -// for(i=0;i limb_count; i++) -// new->limbs[i] = 0; - -// for(i = dend - 1; i>= ipos; i--) -// { -// if(str[i] == '.') continue; -// assert(limb < new->limb_count); -// new->limbs[limb] += mult * (str[i] - '0'); -// -// mult *= 10; -// if(mult == BASE) -// { -// mult = 1; -// limb ++; -// } -// } - - // return new; err: return handle_ConversionSyntax(type, ctx, "invalid literal for Decimal"); @@ -3773,7 +3566,7 @@ } new->digits[i] = x; - } //this loop will go out soon XXX + } /* this loop will go out soon XXX */ for(i = 0;i < new->limb_count;i++) new->limbs[i] = 0; @@ -3781,7 +3574,7 @@ long mult = 1; long limb = 0; new->limbs[0] = 0; - for(i = new->ob_size-1; i>=0; i--) //limb[0] keeps least significant limb + for(i = new->ob_size-1; i>=0; i--) /* limb[0] keeps least significant limb */ { item = PyTuple_GET_ITEM(digtup, i); long x; @@ -4311,8 +4104,6 @@ static PyObject * context_clear_flags(contextobject *self) { -// self->flags = 0; -// PyDict_Clear(self->flags); int i; for(i = 0;iflags,i,0); @@ -5209,7 +5000,7 @@ else capitals = capitals & 1; -// if (rounding_dec == -1) +/* if (rounding_dec == -1) */ if(rounding_dec != NEVER_ROUND && rounding_dec != ALWAYS_ROUND) /* XXX????*/ rounding_dec = PyDecimal_DefaultContext->rounding_dec; From python-checkins at python.org Mon Jun 12 22:45:51 2006 From: python-checkins at python.org (thomas.heller) Date: Mon, 12 Jun 2006 22:45:51 +0200 (CEST) Subject: [Python-checkins] r46898 - peps/trunk/pep-0360.txt Message-ID: <20060612204551.A58E01E4024@bag.python.org> Author: thomas.heller Date: Mon Jun 12 22:45:51 2006 New Revision: 46898 Modified: peps/trunk/pep-0360.txt Log: Remove 'ctypes' from the list of externally maintained modules, the version in Python SVN will from now on be the upstream version. Modified: peps/trunk/pep-0360.txt ============================================================================== --- peps/trunk/pep-0360.txt (original) +++ peps/trunk/pep-0360.txt Mon Jun 12 22:45:51 2006 @@ -42,23 +42,6 @@ release). -ctypes ------- - -:Web site: - http://starship.python.net/crew/theller/ctypes/ -:Standard library name: - ctypes -:Contact person: - Thomas Heller -:Synchronisation history: - * 0.9.9.6 (2.5) - -Bugs can be reported to either the Python tracker [#python-tracker]_ -or the ctypes tracker [#ctypes-tracker]_ and assigned to -Thomas Heller. - - ElementTree ----------- @@ -145,9 +128,6 @@ .. [#python-tracker] Python tracker (http://sourceforge.net/tracker/?group_id=5470) -.. [#ctypes-tracker] ctypes tracker - (http://sourceforge.net/tracker/?group_id=71702) - .. [#pysqlite-tracker] pysqlite tracker (http://pysqlite.org/) From python-checkins at python.org Mon Jun 12 22:56:50 2006 From: python-checkins at python.org (thomas.heller) Date: Mon, 12 Jun 2006 22:56:50 +0200 (CEST) Subject: [Python-checkins] r46899 - in python/trunk: Lib/ctypes/__init__.py Lib/ctypes/_endian.py Lib/ctypes/macholib/__init__.py Lib/ctypes/macholib/dyld.py Lib/ctypes/macholib/dylib.py Lib/ctypes/macholib/framework.py Lib/ctypes/util.py Lib/ctypes/wintypes.py Modules/_ctypes/_ctypes.c Modules/_ctypes/_ctypes_test.c Modules/_ctypes/callbacks.c Modules/_ctypes/callproc.c Modules/_ctypes/cfield.c Modules/_ctypes/ctypes.h Modules/_ctypes/ctypes_dlfcn.h Modules/_ctypes/malloc_closure.c Modules/_ctypes/stgdict.c Message-ID: <20060612205650.3A5091E4005@bag.python.org> Author: thomas.heller Date: Mon Jun 12 22:56:48 2006 New Revision: 46899 Modified: python/trunk/Lib/ctypes/__init__.py python/trunk/Lib/ctypes/_endian.py python/trunk/Lib/ctypes/macholib/__init__.py python/trunk/Lib/ctypes/macholib/dyld.py python/trunk/Lib/ctypes/macholib/dylib.py python/trunk/Lib/ctypes/macholib/framework.py python/trunk/Lib/ctypes/util.py python/trunk/Lib/ctypes/wintypes.py python/trunk/Modules/_ctypes/_ctypes.c python/trunk/Modules/_ctypes/_ctypes_test.c python/trunk/Modules/_ctypes/callbacks.c python/trunk/Modules/_ctypes/callproc.c python/trunk/Modules/_ctypes/cfield.c python/trunk/Modules/_ctypes/ctypes.h python/trunk/Modules/_ctypes/ctypes_dlfcn.h python/trunk/Modules/_ctypes/malloc_closure.c python/trunk/Modules/_ctypes/stgdict.c Log: Add pep-291 compatibility markers. Modified: python/trunk/Lib/ctypes/__init__.py ============================================================================== --- python/trunk/Lib/ctypes/__init__.py (original) +++ python/trunk/Lib/ctypes/__init__.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """create and manipulate C data types in Python""" import os as _os, sys as _sys Modified: python/trunk/Lib/ctypes/_endian.py ============================================================================== --- python/trunk/Lib/ctypes/_endian.py (original) +++ python/trunk/Lib/ctypes/_endian.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### import sys from ctypes import * Modified: python/trunk/Lib/ctypes/macholib/__init__.py ============================================================================== --- python/trunk/Lib/ctypes/macholib/__init__.py (original) +++ python/trunk/Lib/ctypes/macholib/__init__.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ Enough Mach-O to make your head spin. Modified: python/trunk/Lib/ctypes/macholib/dyld.py ============================================================================== --- python/trunk/Lib/ctypes/macholib/dyld.py (original) +++ python/trunk/Lib/ctypes/macholib/dyld.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ dyld emulation """ Modified: python/trunk/Lib/ctypes/macholib/dylib.py ============================================================================== --- python/trunk/Lib/ctypes/macholib/dylib.py (original) +++ python/trunk/Lib/ctypes/macholib/dylib.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ Generic dylib path manipulation """ Modified: python/trunk/Lib/ctypes/macholib/framework.py ============================================================================== --- python/trunk/Lib/ctypes/macholib/framework.py (original) +++ python/trunk/Lib/ctypes/macholib/framework.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ Generic framework path manipulation """ Modified: python/trunk/Lib/ctypes/util.py ============================================================================== --- python/trunk/Lib/ctypes/util.py (original) +++ python/trunk/Lib/ctypes/util.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### import sys, os # find_library(name) returns the pathname of a library, or None. Modified: python/trunk/Lib/ctypes/wintypes.py ============================================================================== --- python/trunk/Lib/ctypes/wintypes.py (original) +++ python/trunk/Lib/ctypes/wintypes.py Mon Jun 12 22:56:48 2006 @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### # XXX This module needs cleanup. from ctypes import * Modified: python/trunk/Modules/_ctypes/_ctypes.c ============================================================================== --- python/trunk/Modules/_ctypes/_ctypes.c (original) +++ python/trunk/Modules/_ctypes/_ctypes.c Mon Jun 12 22:56:48 2006 @@ -1,3 +1,8 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + + /* ToDo: Modified: python/trunk/Modules/_ctypes/_ctypes_test.c ============================================================================== --- python/trunk/Modules/_ctypes/_ctypes_test.c (original) +++ python/trunk/Modules/_ctypes/_ctypes_test.c Mon Jun 12 22:56:48 2006 @@ -1,3 +1,8 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + + #include /* Modified: python/trunk/Modules/_ctypes/callbacks.c ============================================================================== --- python/trunk/Modules/_ctypes/callbacks.c (original) +++ python/trunk/Modules/_ctypes/callbacks.c Mon Jun 12 22:56:48 2006 @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include "Python.h" #include "compile.h" /* required only for 2.3, as it seems */ #include "frameobject.h" Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Mon Jun 12 22:56:48 2006 @@ -1,3 +1,8 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + + /* * History: First version dated from 3/97, derived from my SCMLIB version * for win16. Modified: python/trunk/Modules/_ctypes/cfield.c ============================================================================== --- python/trunk/Modules/_ctypes/cfield.c (original) +++ python/trunk/Modules/_ctypes/cfield.c Mon Jun 12 22:56:48 2006 @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include "Python.h" #include Modified: python/trunk/Modules/_ctypes/ctypes.h ============================================================================== --- python/trunk/Modules/_ctypes/ctypes.h (original) +++ python/trunk/Modules/_ctypes/ctypes.h Mon Jun 12 22:56:48 2006 @@ -1,4 +1,6 @@ -/******************************************************************/ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ #if (PY_VERSION_HEX < 0x02050000) typedef int Py_ssize_t; Modified: python/trunk/Modules/_ctypes/ctypes_dlfcn.h ============================================================================== --- python/trunk/Modules/_ctypes/ctypes_dlfcn.h (original) +++ python/trunk/Modules/_ctypes/ctypes_dlfcn.h Mon Jun 12 22:56:48 2006 @@ -1,4 +1,6 @@ -/******************************************************************/ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ #ifndef _CTYPES_DLFCN_H_ #define _CTYPES_DLFCN_H_ Modified: python/trunk/Modules/_ctypes/malloc_closure.c ============================================================================== --- python/trunk/Modules/_ctypes/malloc_closure.c (original) +++ python/trunk/Modules/_ctypes/malloc_closure.c Mon Jun 12 22:56:48 2006 @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include #include #ifdef MS_WIN32 Modified: python/trunk/Modules/_ctypes/stgdict.c ============================================================================== --- python/trunk/Modules/_ctypes/stgdict.c (original) +++ python/trunk/Modules/_ctypes/stgdict.c Mon Jun 12 22:56:48 2006 @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include "Python.h" #include #ifdef MS_WIN32 From neal at metaslash.com Mon Jun 12 23:08:06 2006 From: neal at metaslash.com (Neal Norwitz) Date: Mon, 12 Jun 2006 17:08:06 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (80) Message-ID: <20060612210806.GA386@python.psfb.org> test_builtin leaked [8, 8, 8] references test_exceptions leaked [40, 40, 40] references test_StringIO leaked [4, 4, 4] references test_base64 leaked [14, 14, 14] references test_cfgparser leaked [12, 12, 12] references test_cgi leaked [20, 20, 20] references test_codeccallbacks leaked [108, 108, 108] references test_codecs leaked [38, 38, 38] references test_coercion leaked [106, 106, 106] references test_complex leaked [10, 10, 10] references test_contextlib leaked [10, 10, 10] references test_cookie leaked [22, 22, 22] references test_cookielib leaked [4, 4, 4] references test_copy leaked [4, 4, 4] references test_copy_reg leaked [22, 22, 22] references test_csv leaked [14, 14, 14] references test_ctypes leaked [8, 8, 8] references test_decimal leaked [10679, 9236, 9985] references test_descr leaked [36, 36, 36] references test_dict leaked [13, 13, 13] references test_distutils leaked [8, 8, 8] references test_doctest leaked [157, 157, 157] references test_doctest2 leaked [2, 2, 2] references test_email leaked [12, 12, 12] references test_email_renamed leaked [12, 12, 12] references test_fileinput leaked [6, 6, 6] references test_fork1 leaked [37, 37, 37] references test_generators leaked [12, 12, 12] references test_getargs2 leaked [14, 14, 14] references test_global leaked [6, 6, 6] references test_import leaked [2, 2, 2] references test_inspect leaked [11, 11, 11] references test_list leaked [4, 4, 4] references test_mailbox leaked [656, 656, 656] references test_minidom leaked [4, 4, 4] references test_operator leaked [16, 16, 16] references test_os leaked [15, 15, 15] references test_pep292 leaked [37, 37, 37] references test_pep352 leaked [48, 48, 48] references test_pickletools leaked [12, 12, 12] references test_posix leaked [8, 8, 8] references test_profilehooks leaked [1, 1, 1] references test_pyexpat leaked [2, 2, 2] references test_queue leaked [32, 34, 34] references test_random leaked [18, 18, 18] references test_repr leaked [2, 2, 2] references test_runpy leaked [8, 8, 8] references test_scope leaked [14, 14, 14] references test_sgmllib leaked [2, 2, 2] references test_shelve leaked [6, 6, 6] references test_socket leaked [4250, 4594, 4088] references test_socket_ssl leaked [1, 4, 4] references test_socketserver leaked [135, 135, 135] references test_sqlite leaked [31, 14, 58] references test_strop leaked [88, 88, 88] references test_strptime leaked [10, 10, 10] references test_struct leaked [3782, 3890, 3854] references test_subprocess leaked [9, 9, 9] references test_syntax leaked [2, 2, 2] references test_sys leaked [2, 2, 2] references test_tarfile leaked [452, 452, 452] references test_textwrap leaked [4, 4, 4] references test_thread leaked [53, 52, 52] references test_threadedtempfile leaked [35, 37, 35] references test_threading leaked [38, 39, 37] references test_threadsignals leaked [0, 0, -8] references test_tokenize leaked [4, 4, 4] references test_traceback leaked [1, 1, 1] references test_urllib2 leaked [3, 3, 3] references test_userdict leaked [29, 29, 29] references test_userlist leaked [4, 4, 4] references test_uu leaked [6, 6, 6] references test_wait3 leaked [37, 37, 37] references test_wait4 leaked [33, 37, 37] references test_warnings leaked [32, 32, 32] references test_weakref leaked [2, 2, 2] references test_with leaked [12, 12, 12] references test_wsgiref leaked [6, 6, 6] references test_xml_etree leaked [2, 2, 2] references test_xrange leaked [2, 2, 2] references From python-checkins at python.org Mon Jun 12 23:28:02 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Mon, 12 Jun 2006 23:28:02 +0200 (CEST) Subject: [Python-checkins] r46900 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060612212802.D13A61E4002@bag.python.org> Author: mateusz.rukowicz Date: Mon Jun 12 23:28:00 2006 New Revision: 46900 Modified: sandbox/trunk/decimal-c/_decimal.c Log: There is no tabs anymore in code, nor white spaces at end of lines. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Mon Jun 12 23:28:00 2006 @@ -69,6 +69,13 @@ #include "decimal.h" #include +/* Important note: + * The code is meant to work well and without errors, not yet to be efficent. + * I will start optimizing code, when I will be quite sure, everything + * works fine. Real slow code, that should be replaced have comment + * "SLOW". */ + + /* integer arithmetic */ /* Little explanation: * Digits are grouped into limbs, that is, every limb keeps LOG digits. @@ -81,287 +88,287 @@ /* takes new_digits from self[], starting from start_at digit, *digits* not limbs */ /* returns digit after that we returned (useful for rounding) */ -static long -_limb_first_n_digits(long *self, long ndigits, long start_at, long *new, long new_digits) +static long +_limb_first_n_digits(long *self, long ndigits, long start_at, long *new, long new_digits) { - long start_pos = ndigits - (start_at + new_digits); /* where we start counting from left */ - long pos = 0; /* we are here */ - long self_limb = 0; /* we start at first digit of first limb */ - long self_mult = 1; - long new_limb = 0; - long new_mult = 1; - long last_digit = 0; /* digit from start_pos -1 */ - long i; - - for(i = 0;i olimbs ? 1 : -1; - - for(i = limbs-1; i>=0 ;i--) - if(self[i] != other[i]) - return self[i] > other[i] ? 1 : -1; - return 0; + long i; + if(limbs != olimbs) + return limbs > olimbs ? 1 : -1; + + for(i = limbs-1; i>=0 ;i--) + if(self[i] != other[i]) + return self[i] > other[i] ? 1 : -1; + return 0; } static long _limb_add_core(long *big, long bsize, long *small, long ssize, long *out) { - long max = bsize + 1; - long limbs_total = (max + LOG -1)/LOG; - long limbs_copy = (bsize + LOG -1)/LOG; - long limbs_add = (ssize + LOG -1)/LOG; - long i; - - for(i =0 ;i= BASE) - { - out[i] -= BASE; - out[i+1] ++; - } - } - - /* there may be some extra digit at max, check this out */ - if(_limb_get_digit(out, max, 0) > 0) - return max; - else - return max-1; - + long max = bsize + 1; + long limbs_total = (max + LOG -1)/LOG; + long limbs_copy = (bsize + LOG -1)/LOG; + long limbs_add = (ssize + LOG -1)/LOG; + long i; + + for(i =0 ;i= BASE) + { + out[i] -= BASE; + out[i+1] ++; + } + } + + /* there may be some extra digit at max, check this out */ + if(_limb_get_digit(out, max, 0) > 0) + return max; + else + return max-1; + } static long _limb_add(long *self, long ssize, long *other, long osize, long *out) { - if(ssize > osize) - return _limb_add_core(self, ssize, other, osize, out); - else - return _limb_add_core(other, osize, self, ssize, out); + if(ssize > osize) + return _limb_add_core(self, ssize, other, osize, out); + else + return _limb_add_core(other, osize, self, ssize, out); } static long _limb_sub(long *big, long bsize, long *small, long ssize, long *out) { - long blimbs = (bsize + LOG -1)/LOG; - long slimbs = (ssize + LOG -1)/LOG; - long i; - - for(i=0;i= BASE) - { - out[i+j+1] += out[i+j]/BASE; - out[i+j] %= BASE; - } - } - - for(i = 0;i= BASE) - { - assert(i+1 < max_limbs); - out[i+1] += out[i] / BASE; - out[i] %= BASE; - } - } + long max_limbs = flimbs + slimbs; + long i,j; + + for(i = 0;i 1) max_limbs --; - return max_limbs; + for(i = 0;i= BASE) + { + out[i+j+1] += out[i+j]/BASE; + out[i+j] %= BASE; + } + } + + for(i = 0;i= BASE) + { + assert(i+1 < max_limbs); + out[i+1] += out[i] / BASE; + out[i] %= BASE; + } + } + + while(!out[max_limbs -1] && max_limbs > 1) max_limbs --; + return max_limbs; } static long _limb_multiply(long *first, long fsize, long *second, long ssize, long *out) { - long used_limbs; - long flimbs, slimbs; - long digits_at_most; - flimbs = (fsize + LOG - 1)/LOG; - slimbs = (ssize + LOG - 1)/LOG; - used_limbs = _limb_multiply_core(first, flimbs, second, slimbs, out); - - digits_at_most = used_limbs * LOG; - if(!digits_at_most) - return 1; + long used_limbs; + long flimbs, slimbs; + long digits_at_most; + flimbs = (fsize + LOG - 1)/LOG; + slimbs = (ssize + LOG - 1)/LOG; + used_limbs = _limb_multiply_core(first, flimbs, second, slimbs, out); + + digits_at_most = used_limbs * LOG; + if(!digits_at_most) + return 1; - while(_limb_get_digit(out, digits_at_most, 0) == 0 && digits_at_most >1) digits_at_most --; + while(_limb_get_digit(out, digits_at_most, 0) == 0 && digits_at_most >1) digits_at_most --; - return digits_at_most; + return digits_at_most; } @@ -383,9 +390,9 @@ #define GETNAN(op) ( ( (op)->sign == SIGN_POSNAN || (op)->sign == SIGN_NEGNAN ) ? \ 1 : ( ( (op)->sign == SIGN_POSSNAN || (op)->sign == SIGN_NEGSNAN ) ? 2 : 0 ) ) -#define ISINF(op) ( ( (op)->sign == SIGN_POSINF || (op)->sign == SIGN_NEGINF) ? \ - ((op)->sign == SIGN_POSINF ? 1 : -1) : 0 ) - +#define ISINF(op) ( ( (op)->sign == SIGN_POSINF || (op)->sign == SIGN_NEGINF) ? \ + ((op)->sign == SIGN_POSINF ? 1 : -1) : 0 ) + /* Exponent calculations */ /* XXX: overflow checking? */ @@ -569,18 +576,18 @@ } assert(PyDecimal_Check(thing)); /* we want to return a NaN, but keep diagnostics */ - if(thing->sign&1) /* neg */ - sign = SIGN_NEGNAN; - else - sign = SIGN_POSNAN; + if(thing->sign&1) /* neg */ + sign = SIGN_NEGNAN; + else + sign = SIGN_POSNAN; - res = _new_decimalobj(type, thing->ob_size, sign, 0); + res = _new_decimalobj(type, thing->ob_size, sign, 0); if (!res) return NULL; for (i = 0; i < thing->ob_size; i++) - res->digits[i] = thing->digits[i]; /* DELETE */ - - for (i = 0; i< res->limb_count;i++) - res->limbs[i] = thing->limbs[i]; + res->digits[i] = thing->digits[i]; /* DELETE */ + + for (i = 0; i< res->limb_count;i++) + res->limbs[i] = thing->limbs[i]; return res; } @@ -684,7 +691,7 @@ if (res) { for (i = 0; i < ctx->prec; i++) res->digits[i] = 9; - _limb_fill(res->limbs, ctx->prec, 9); + _limb_fill(res->limbs, ctx->prec, 9); return res; } } @@ -697,7 +704,7 @@ if (res) { for (i = 0; i < ctx->prec; i++) res->digits[i] = 9; - _limb_fill(res->limbs, ctx->prec, 9); + _limb_fill(res->limbs, ctx->prec, 9); return res; } } @@ -729,9 +736,9 @@ { decimalobject *new; char *arr = NULL; - long *arr2 = NULL; - long limb_c = ndigits / LOG; - limb_c += (ndigits % LOG) > 0; + long *arr2 = NULL; + long limb_c = ndigits / LOG; + limb_c += (ndigits % LOG) > 0; if (ndigits > LONG_MAX) { PyErr_NoMemory(); @@ -744,12 +751,12 @@ goto err; } - arr2 = PyObject_MALLOC(limb_c * sizeof(long)); + arr2 = PyObject_MALLOC(limb_c * sizeof(long)); - if(!arr2){ - PyErr_NoMemory(); - goto err; - } + if(!arr2){ + PyErr_NoMemory(); + goto err; + } new = (decimalobject *)type->tp_alloc(type, 0); if (new == NULL) goto err; @@ -757,13 +764,13 @@ new->exp = exp; new->ob_size = ndigits; new->digits = arr; - new->limb_count = limb_c; - new->limbs = arr2; + new->limb_count = limb_c; + new->limbs = arr2; return new; err: if (arr) PyObject_FREE(arr); - if (arr2) PyObject_FREE(arr2); + if (arr2) PyObject_FREE(arr2); return NULL; } @@ -839,23 +846,23 @@ if (!new) return NULL; - for(i=0;ilimb_count;i++) - new->limbs[i] = self->limbs[i]; - - if(self->limb_count != new->limb_count) - new->limbs[new->limb_count - 1] = 0; /* we have new limb */ - - new->limbs[0] ++; - i = 0; - while(new->limbs[i] >= BASE) - { - assert(i+1 < new->limb_count); - new->limbs[i] -= BASE; - new->limbs[i+1] ++; - i++; - } - if(_limb_get_digit(new->limbs, new->ob_size, 0) == 0) - new->ob_size --; + for(i=0;ilimb_count;i++) + new->limbs[i] = self->limbs[i]; + + if(self->limb_count != new->limb_count) + new->limbs[new->limb_count - 1] = 0; /* we have new limb */ + + new->limbs[0] ++; + i = 0; + while(new->limbs[i] >= BASE) + { + assert(i+1 < new->limb_count); + new->limbs[i] -= BASE; + new->limbs[i+1] ++; + i++; + } + if(_limb_get_digit(new->limbs, new->ob_size, 0) == 0) + new->ob_size --; return new; } @@ -869,36 +876,36 @@ for (i = 0; i < prec; i++) new->digits[i] = self->digits[i]; - _limb_first_n_digits(self->limbs, self->ob_size, 0, new->limbs, prec); + _limb_first_n_digits(self->limbs, self->ob_size, 0, new->limbs, prec); return new; } /* Round away from 0. */ static decimalobject * -_round_up(decimalobject *self, long prec, long expdiff, contextobject *ctx) -{ +_round_up(decimalobject *self, long prec, long expdiff, contextobject *ctx) +{ long i; decimalobject *new = _NEW_decimalobj(prec, self->sign, self->exp + expdiff); decimalobject *new2 = NULL; if (!new) return NULL; for (i = 0; i < prec; i++) new->digits[i] = self->digits[i]; - _limb_first_n_digits(self->limbs, self->ob_size, 0, new->limbs, prec); - + _limb_first_n_digits(self->limbs, self->ob_size, 0, new->limbs, prec); + for (i = prec; i < self->ob_size; i++) - if(_limb_get_digit(self->limbs,self->ob_size, i) > 0){ /* SLOW */ + if(_limb_get_digit(self->limbs,self->ob_size, i) > 0){ /* SLOW */ new2 = _decimal_increment(new, 1, ctx); Py_DECREF(new); if (!new2) return NULL; if (new2->ob_size > prec) { - _limb_cut_one_digit(new2->limbs,new2->ob_size); + _limb_cut_one_digit(new2->limbs,new2->ob_size); new2->ob_size--; new2->exp++; } return new2; } - return new; + return new; } /* Actually round half up. Returns a new reference, either on tmp @@ -912,12 +919,12 @@ { decimalobject *new; assert(expdiff > 0); - if(self->ob_size > prec && _limb_get_digit(self->limbs, self->ob_size, prec) >= 5){ /* SLOW */ - new = _decimal_increment(tmp, 1, ctx); + if(self->ob_size > prec && _limb_get_digit(self->limbs, self->ob_size, prec) >= 5){ /* SLOW */ + new = _decimal_increment(tmp, 1, ctx); Py_DECREF(tmp); if (!new) return NULL; if (new->ob_size > prec) { - _limb_cut_one_digit(new->limbs,new->ob_size); + _limb_cut_one_digit(new->limbs,new->ob_size); new->ob_size--; new->exp++; } @@ -939,10 +946,10 @@ for (i = 0; i < prec; i++) tmp->digits[i] = self->digits[i]; - last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); + last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { - if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ + if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ return _do_round_half_up(self, prec, expdiff, ctx, tmp); } /* self ends in 5000...., so tmp is okay */ @@ -962,14 +969,14 @@ if (!tmp) return NULL; for (i = 0; i < prec; i++) tmp->digits[i] = self->digits[i]; - last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); + last = _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); if (last == 5) { for (i = prec+1; i < self->ob_size; i++) { - if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ + if(_limb_get_digit(self->limbs, self->ob_size, i) != 0) /* SLOW */ return _do_round_half_up(self, prec, expdiff, ctx, tmp); } - if((_limb_get_digit(self->limbs, self->ob_size, prec-1)&1) == 0) - return tmp; + if((_limb_get_digit(self->limbs, self->ob_size, prec-1)&1) == 0) + return tmp; } return _do_round_half_up(self, prec, expdiff, ctx, tmp); } @@ -984,7 +991,7 @@ if (!tmp) return NULL; for (i = 0; i < prec; i++) tmp->digits[i] = self->digits[i]; - _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); + _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, prec); return _do_round_half_up(self, prec, expdiff, ctx, tmp); } @@ -1055,7 +1062,7 @@ if (!new) return NULL; while (i--) new->digits[i] = 0; - _limb_fill(new->limbs, new->ob_size,0); + _limb_fill(new->limbs, new->ob_size,0); if (handle_Rounded(ctx, NULL) != 0) { Py_DECREF(new); @@ -1070,7 +1077,7 @@ new->digits[0] = 0; for (i = 1; i < new->ob_size; i++) new->digits[i] = self->digits[i-1]; - _limb_first_n_digits(self->limbs, self->ob_size, -1, new->limbs, new->ob_size); + _limb_first_n_digits(self->limbs, self->ob_size, -1, new->limbs, new->ob_size); prec = 1; } else if (prec < 0) { new = _NEW_decimalobj(2, self->sign, @@ -1078,7 +1085,7 @@ if (!new) return NULL; new->digits[0] = 0; new->digits[1] = 1; - new->limbs[0] = 1; + new->limbs[0] = 1; prec = 1; } else { new = _decimal_get_copy(self); @@ -1102,24 +1109,24 @@ new2->digits[i] = 0; } - _limb_first_n_digits(new->limbs, new->ob_size, 0, new2->limbs, new2->ob_size); + _limb_first_n_digits(new->limbs, new->ob_size, 0, new2->limbs, new2->ob_size); Py_DECREF(new); return new2; } /* Maybe all the lost digits are 0. */ for (i = self->ob_size - expdiff; i < self->ob_size; i++) { - if(_limb_get_digit(self->limbs, self->ob_size, i) > 0) - goto no_way; + if(_limb_get_digit(self->limbs, self->ob_size, i) > 0) + goto no_way; } /* All lost digits are 0, so just clobber new */ - while(new->ob_size > prec) - { - _limb_cut_one_digit(new->limbs, new->ob_size); /* VERY SLOW */ - new->ob_size --; - } - new->exp += expdiff; + while(new->ob_size > prec) + { + _limb_cut_one_digit(new->limbs, new->ob_size); /* VERY SLOW */ + new->ob_size --; + } + new->exp += expdiff; if (handle_Rounded(ctx, NULL) != 0) { Py_DECREF(new); return NULL; @@ -1129,7 +1136,7 @@ no_way: /* Now rounding starts. We still own "new". */ rnd_func = round_funcs[rounding]; - /* + /* if (prec != ctx->prec) { ctx2 = (contextobject *)context_copy(ctx); if (!ctx2) { @@ -1138,10 +1145,10 @@ } ctx2->prec = prec; ctx = ctx2; - }*/ /* XXX here is quite subtle bug - we copy context, and set flags in copied context - after that, they are lost, not sure if we really need function above, - I'll comment it */ - + }*/ /* XXX here is quite subtle bug - we copy context, and set flags in copied context + after that, they are lost, not sure if we really need function above, + I'll comment it */ + /* ctx2 is NULL if the original context is used, because that one * doesn't have to be DECREF'd. */ new2 = rnd_func(new, prec, expdiff, ctx); @@ -1161,7 +1168,7 @@ return NULL; } -/* Default values: rounding=-1 (use context), watchexp=1 */ /* TODO TODO TODO */ +/* Default values: rounding=-1 (use context), watchexp=1 */ static decimalobject * _decimal_rescale(decimalobject *self, long exp, contextobject *ctx, int rounding, int watchexp) @@ -1190,7 +1197,7 @@ if (!ans) return NULL; ans->digits[0] = 0; - ans->limbs[0] = 0; + ans->limbs[0] = 0; return ans; } @@ -1208,7 +1215,7 @@ return NULL; ans->digits[0] = 0; ans->digits[1] = 1; - ans->limbs[0] = 1; + ans->limbs[0] = 1; digits = 1; } else { ans = _NEW_decimalobj(self->ob_size+1, self->sign, self->exp); @@ -1217,9 +1224,9 @@ for (i = 0; i < self->ob_size; i++) ans->digits[i+1] = self->digits[i]; ans->digits[0] = 0; - for(i=0;ilimb_count;i++) - ans->limbs[i] = 0; - _limb_first_n_digits(self->limbs, self->ob_size, 0, ans->limbs, ans->ob_size-1); + for(i=0;ilimb_count;i++) + ans->limbs[i] = 0; + _limb_first_n_digits(self->limbs, self->ob_size, 0, ans->limbs, ans->ob_size-1); } tmp = _decimal_round(ans, digits, ctx, rounding); @@ -1227,11 +1234,11 @@ if (!tmp) return NULL; - if(_limb_get_digit(tmp -> limbs, tmp->ob_size, 0) == 0 && tmp->ob_size > 1){ - /* We need one digit less, just clobber tmp. */ + if(_limb_get_digit(tmp -> limbs, tmp->ob_size, 0) == 0 && tmp->ob_size > 1){ + /* We need one digit less, just clobber tmp. */ for (i = 0; i < tmp->ob_size-1; i++) - tmp->digits[i] = tmp->digits[i+1]; /* TODO make last digit 0 */ - tmp->ob_size--; + tmp->digits[i] = tmp->digits[i+1]; /* TODO make last digit 0 */ + tmp->ob_size--; } tmp->exp = exp; @@ -1253,25 +1260,25 @@ static PyObject * decimal_rescale(decimalobject *self, PyObject *args, PyObject *kwds) { - static char *kwlist[] = {"exp", "rounding", "context", "watchexp", 0}; - contextobject *ctx = NULL; - long exp; - int rounding = -1, watchexp = 1; - - if(!PyArg_ParseTupleAndKeywords(args,kwds,"l|iOi:_rescale",kwlist, - &exp, &rounding, &ctx, &watchexp)) - return NULL; - - if(ctx == NULL) - if(!(ctx = getcontext())) - return NULL; - - if(!PyDecimalContext_Check(ctx)){ - PyErr_SetString(PyExc_TypeError, "context must be Context object"); - return NULL; - } + static char *kwlist[] = {"exp", "rounding", "context", "watchexp", 0}; + contextobject *ctx = NULL; + long exp; + int rounding = -1, watchexp = 1; - return (PyObject*)_decimal_rescale(self,exp,ctx,rounding,watchexp); + if(!PyArg_ParseTupleAndKeywords(args,kwds,"l|iOi:_rescale",kwlist, + &exp, &rounding, &ctx, &watchexp)) + return NULL; + + if(ctx == NULL) + if(!(ctx = getcontext())) + return NULL; + + if(!PyDecimalContext_Check(ctx)){ + PyErr_SetString(PyExc_TypeError, "context must be Context object"); + return NULL; + } + + return (PyObject*)_decimal_rescale(self,exp,ctx,rounding,watchexp); } /* Fix the exponents and return a copy with the exponents in bounds. @@ -1288,8 +1295,8 @@ long Etiny = ETINY(ctx); if (self->exp < Etiny) { if (!decimal_nonzero(self)) { - ans = _decimal_get_copy(self); - if (!ans) + ans = _decimal_get_copy(self); + if (!ans) return NULL; ans->exp = Etiny; if (handle_Clamped(ctx, NULL) != 0) @@ -1380,18 +1387,18 @@ static PyObject * decimal_fix(decimalobject *self, PyObject *args, PyObject *kwds) { - static char *kwlist[] = {"context",0}; - contextobject *ctx; + static char *kwlist[] = {"context",0}; + contextobject *ctx; - if(!PyArg_ParseTupleAndKeywords(args, kwds, "O:_fix", kwlist, - &ctx)) - return NULL; - - if((PyObject*)ctx == Py_None) - if(!(ctx = getcontext())) - return NULL; + if(!PyArg_ParseTupleAndKeywords(args, kwds, "O:_fix", kwlist, + &ctx)) + return NULL; - return (PyObject*)_decimal_fix(self,ctx); + if((PyObject*)ctx == Py_None) + if(!(ctx = getcontext())) + return NULL; + + return (PyObject*)_decimal_fix(self,ctx); } /* convert something to a Decimal. On failure, either returns NotImplemented or raises an exception, depending @@ -1507,13 +1514,13 @@ return 1; } - if(ISINF(self) || ISINF(other)) - { - if(ISINF(self) == ISINF(other)) - return 0; - else - return ISINF(self) > ISINF(other) ? 1 : -1 ; - } + if(ISINF(self) || ISINF(other)) + { + if(ISINF(self) == ISINF(other)) + return 0; + else + return ISINF(self) > ISINF(other) ? 1 : -1 ; + } } /* If both are 0, sign comparison isn't correct. */ @@ -1553,10 +1560,10 @@ next: /* Adjusted sizes are not equal. */ - if(adj1 > adj2 && _limb_get_digit(self->limbs, self->ob_size, 0) != 0) - return 1 - 2*(self->sign & 1); /* 0 -> 1, 1 -> -1 */ - else if(adj1 < adj2 && _limb_get_digit(other->limbs, other->ob_size, 0) != 0) - return -1 + 2*(other->sign & 1); + if(adj1 > adj2 && _limb_get_digit(self->limbs, self->ob_size, 0) != 0) + return 1 - 2*(self->sign & 1); /* 0 -> 1, 1 -> -1 */ + else if(adj1 < adj2 && _limb_get_digit(other->limbs, other->ob_size, 0) != 0) + return -1 + 2*(other->sign & 1); ctx2 = context_copy(ctx); if (!ctx2) return 0; /* error */ @@ -1565,7 +1572,7 @@ if (context_ignore_all_flags(ctx2) == NULL) return 0; /* error */ - Py_XDECREF(ctx2); + Py_XDECREF(ctx2); ans = _do_decimal_subtract(self, other, ctx2); if (!ans) return 0; @@ -1631,8 +1638,8 @@ contextobject *ctx) { decimalobject *ans; - int cmp; - contextobject *ctx2; + int cmp; + contextobject *ctx2; if (ISSPECIAL(self) || ISSPECIAL(other)) { decimalobject *nan; @@ -1651,44 +1658,44 @@ return nan; } - ans = self; /* let's assume */ - if(!ctx) - { - ctx = getcontext(); - } - - /* we have to give getcontext() to _do_decimal_compare */ - ctx2 = getcontext(); - if(!ctx2) - return NULL; - - cmp = _do_real_decimal_compare(self, other,ctx2); - if(PyErr_Occurred()) - return NULL; - - if(!cmp) - { - if(self->sign != other->sign) - { - if(self->sign) - ans = other; - } - else - { - if(self->exp < other->exp && !self->sign) - ans = other; - else if(self->exp > other->exp && self->sign) - ans = other; - } - } - else if(cmp == -1) - ans = other; + ans = self; /* let's assume */ + if(!ctx) + { + ctx = getcontext(); + } + + /* we have to give getcontext() to _do_decimal_compare */ + ctx2 = getcontext(); + if(!ctx2) + return NULL; + + cmp = _do_real_decimal_compare(self, other,ctx2); + if(PyErr_Occurred()) + return NULL; - if(ctx ->rounding_dec == ALWAYS_ROUND) - return _decimal_fix(ans, ctx); + if(!cmp) + { + if(self->sign != other->sign) + { + if(self->sign) + ans = other; + } + else + { + if(self->exp < other->exp && !self->sign) + ans = other; + else if(self->exp > other->exp && self->sign) + ans = other; + } + } + else if(cmp == -1) + ans = other; - Py_INCREF(ans); - return ans; + if(ctx ->rounding_dec == ALWAYS_ROUND) + return _decimal_fix(ans, ctx); + + Py_INCREF(ans); + return ans; } DECIMAL_BINARY_FUNC(max) @@ -1697,70 +1704,70 @@ _do_decimal_min(decimalobject *self, decimalobject *other, contextobject *ctx) { - decimalobject *ans; - int cmp; - contextobject *ctx2; - - if(ISSPECIAL(self) || ISSPECIAL(other)) - { - decimalobject *nan; - int sn = GETNAN(self); - int so = GETNAN(other); - - if(sn || so) - { - if(so == 1 && sn != 2) - { - Py_INCREF(self); - return self; - } - else if (sn == 1 && so != 2) - { - Py_INCREF(other); - return other; - } - } - - if(_check_nans(self, other, ctx, &nan) != 0) - return nan; - } - - ans = self; - if(!ctx) - ctx = getcontext(); - if(!ctx) - return NULL; - - ctx2 = getcontext(); - if(!ctx2) - return NULL; - cmp = _do_real_decimal_compare(self,other,ctx2); - - if(PyErr_Occurred()) - return NULL; - - if(!cmp) - { - if(self->sign != other->sign) - { - if(!self->sign) - ans = other; - } - else{ - if(self->exp > other->exp && !self->sign) - ans = other; - else if (self->exp < other->exp && self->sign) - ans = other; - } - } - else if(cmp == 1) - ans = other; - - if(ctx->rounding_dec == ALWAYS_ROUND) - return _decimal_fix(ans, ctx); - - Py_INCREF(ans); - return ans; + decimalobject *ans; + int cmp; + contextobject *ctx2; + + if(ISSPECIAL(self) || ISSPECIAL(other)) + { + decimalobject *nan; + int sn = GETNAN(self); + int so = GETNAN(other); + + if(sn || so) + { + if(so == 1 && sn != 2) + { + Py_INCREF(self); + return self; + } + else if (sn == 1 && so != 2) + { + Py_INCREF(other); + return other; + } + } + + if(_check_nans(self, other, ctx, &nan) != 0) + return nan; + } + + ans = self; + if(!ctx) + ctx = getcontext(); + if(!ctx) + return NULL; + + ctx2 = getcontext(); + if(!ctx2) + return NULL; + cmp = _do_real_decimal_compare(self,other,ctx2); + + if(PyErr_Occurred()) + return NULL; + + if(!cmp) + { + if(self->sign != other->sign) + { + if(!self->sign) + ans = other; + } + else{ + if(self->exp > other->exp && !self->sign) + ans = other; + else if (self->exp < other->exp && self->sign) + ans = other; + } + } + else if(cmp == 1) + ans = other; + + if(ctx->rounding_dec == ALWAYS_ROUND) + return _decimal_fix(ans, ctx); + + Py_INCREF(ans); + return ans; } DECIMAL_BINARY_FUNC(min) @@ -1786,15 +1793,15 @@ new = _NEW_decimalobj(1, dup->sign, 0); Py_DECREF(dup); if (!new) return NULL; - new->limbs[0] = 0; + new->limbs[0] = 0; return new; } /* strip trailing 0s from dup */ - while(_limb_get_digit(dup->limbs, dup->ob_size, dup->ob_size -1) == 0){ - _limb_cut_one_digit(dup->limbs,dup->ob_size); - dup->ob_size --; - dup->exp ++; + while(_limb_get_digit(dup->limbs, dup->ob_size, dup->ob_size -1) == 0){ + _limb_cut_one_digit(dup->limbs,dup->ob_size); + dup->ob_size --; + dup->exp ++; } return dup; } @@ -1960,8 +1967,8 @@ &rounding, &ctx)) return NULL; ENSURE_CONTEXT("to_integral", ctx); - if(rounding == -1) - rounding = ctx->rounding; + if(rounding == -1) + rounding = ctx->rounding; if (!VALID_ROUND(rounding)) { PyErr_SetString(PyExc_ValueError, "invalid rounding value"); return NULL; @@ -2014,8 +2021,8 @@ return NULL; for (i = 0; i < self->ob_size; i++) new->digits[i] = self->digits[i]; - for (i = 0;i < new->limb_count;i++) - new->limbs[i] = self->limbs[i]; + for (i = 0;i < new->limb_count;i++) + new->limbs[i] = self->limbs[i]; return new; } @@ -2042,15 +2049,15 @@ return NULL; for (i = 0; i < self->ob_size; i++) { - d = PyInt_FromLong(_limb_get_digit(self->limbs, self->ob_size, i)); /* SLOW */ + d = PyInt_FromLong(_limb_get_digit(self->limbs, self->ob_size, i)); /* SLOW */ if (!d) return NULL; PyTuple_SET_ITEM(digits, i, d); } - if(!ISINF(self)) - res = Py_BuildValue("(bOn)", self->sign % 2, digits, self->exp); - else - res = Py_BuildValue("(bOO)", self->sign % 2, digits, PyString_FromString("F")); + if(!ISINF(self)) + res = Py_BuildValue("(bOn)", self->sign % 2, digits, self->exp); + else + res = Py_BuildValue("(bOO)", self->sign % 2, digits, PyString_FromString("F")); Py_DECREF(digits); return res; } @@ -2086,14 +2093,14 @@ SANITY_CHECK(p); /* check for digits != {0, } */ - if(d->ob_size != 1 || _limb_get_digit(d->limbs, d->ob_size, 0) != 0) - { - for(i=0;i< d->ob_size;i++) - { - p+= sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, i)); /* SLOW */ - SANITY_CHECK(p); - } - } + if(d->ob_size != 1 || _limb_get_digit(d->limbs, d->ob_size, 0) != 0) + { + for(i=0;i< d->ob_size;i++) + { + p+= sprintf(p, "%d", _limb_get_digit(d->limbs, d->ob_size, i)); /* SLOW */ + SANITY_CHECK(p); + } + } } else { /* infinity */ p += sprintf(p, "%s", "Infinity"); } @@ -2110,27 +2117,27 @@ char *p, *end; int roundexp, i; - end = &outbuf[FORMAT_SIZE-1]; + end = &outbuf[FORMAT_SIZE-1]; p = outbuf; if (d->sign & 1) { *p++ = '-'; SANITY_CHECK(p); } if (d->exp < 0 && d->exp >= -6) { - - i = 0; - *p++ = '0'; - *p++ = '.'; - for (;iexp);i++) - *p++ = '0'; + + i = 0; + *p++ = '0'; + *p++ = '.'; + for (;iexp);i++) + *p++ = '0'; SANITY_CHECK(p); } else { - roundexp = d->exp; - while(roundexp%3) - roundexp ++; - if (roundexp != d->exp) { - *p++ = '0'; - *p++ = '.'; + roundexp = d->exp; + while(roundexp%3) + roundexp ++; + if (roundexp != d->exp) { + *p++ = '0'; + *p++ = '.'; for (i = 0; i < (roundexp - d->exp); i++) *p++ = '0'; SANITY_CHECK(p); @@ -2152,8 +2159,8 @@ SANITY_CHECK(p); } } - *p++ = 0; - p = &outbuf[0]; + *p++ = 0; + p = &outbuf[0]; return PyString_FromString(p); } @@ -2172,14 +2179,14 @@ } -static PyObject * +static PyObject * _do_decimal_str(decimalobject *d, contextobject *context, int engineering) { char *outbuf; int buflen, i; int dotplace, adjexp; int append_E = 0, append_adjexp = 0; - long extra_zeros=0; + long extra_zeros=0; char *p, *end; PyObject *ret; @@ -2206,66 +2213,66 @@ *p++ = '-'; } - /* The rule is as follows, imagine you write integer: - * 123456789 we want to print it, without adding post-zeros - * or adding at most 6 pre-zeros that is - * 0.00000123456789, if none of this can be done, then, we put dot - * after first digit, and compute E *or* if engineering, then, we put - * dot after 1st 2nd or 3rd digit, and assure that E is divisible by 3 - * huh */ - - /* (1) dotplace = -adjexp + d->exp + d->ob_size */ - /* why is that like? well, d->exp moves dot right, d->ob_size moves dot right - * and adjexp moves dot left */ - adjexp = 0; - dotplace = d->exp + d->ob_size; - /* dotplace must be at most d->ob_size (no dot at all) and at last -5 (6 pre-zeros)*/ - if(dotplace >d->ob_size || dotplace <-5) - { - /* ok, we have to put dot after 1 digit, that is dotplace = 1 - * we compute adjexp from equation (1) */ - dotplace = 1; - adjexp = -dotplace + d->exp + d->ob_size; - } - - if(engineering) /* we have to be sure, adjexp%3 == 0 */ - while(adjexp%3) - { - adjexp --; - dotplace ++; - } - - /* now all we have to do, is to put it to the string =] */ - - if(dotplace > d->ob_size) - extra_zeros = dotplace - d->ob_size; - - if(dotplace <= 0) - { - *p++ = '0'; - *p++ = '.'; - while(dotplace++ < 0) - *p++ = '0'; - dotplace = -1; - } - - for(i = 0;iob_size;i++) - { - if(dotplace == i) - *p++ = '.'; - p += sprintf(p, "%d", _limb_get_digit(d->limbs,d->ob_size,i)); - } - - while(extra_zeros --)*p++ = '0'; - - /* that was a way easier way =] */ - - - if(adjexp) - { - append_E = 1; - append_adjexp = 1; - } + /* The rule is as follows, imagine you write integer: + * 123456789 we want to print it, without adding post-zeros + * or adding at most 6 pre-zeros that is + * 0.00000123456789, if none of this can be done, then, we put dot + * after first digit, and compute E *or* if engineering, then, we put + * dot after 1st 2nd or 3rd digit, and assure that E is divisible by 3 + * huh */ + + /* (1) dotplace = -adjexp + d->exp + d->ob_size */ + /* why is that like? well, d->exp moves dot right, d->ob_size moves dot right + * and adjexp moves dot left */ + adjexp = 0; + dotplace = d->exp + d->ob_size; + /* dotplace must be at most d->ob_size (no dot at all) and at last -5 (6 pre-zeros)*/ + if(dotplace >d->ob_size || dotplace <-5) + { + /* ok, we have to put dot after 1 digit, that is dotplace = 1 + * we compute adjexp from equation (1) */ + dotplace = 1; + adjexp = -dotplace + d->exp + d->ob_size; + } + + if(engineering) /* we have to be sure, adjexp%3 == 0 */ + while(adjexp%3) + { + adjexp --; + dotplace ++; + } + + /* now all we have to do, is to put it to the string =] */ + + if(dotplace > d->ob_size) + extra_zeros = dotplace - d->ob_size; + + if(dotplace <= 0) + { + *p++ = '0'; + *p++ = '.'; + while(dotplace++ < 0) + *p++ = '0'; + dotplace = -1; + } + + for(i = 0;iob_size;i++) + { + if(dotplace == i) + *p++ = '.'; + p += sprintf(p, "%d", _limb_get_digit(d->limbs,d->ob_size,i)); + } + + while(extra_zeros --)*p++ = '0'; + + /* that was a way easier way =] */ + + + if(adjexp) + { + append_E = 1; + append_adjexp = 1; + } if (append_E) { if (context->capitals) { *p++ = 'E'; @@ -2287,7 +2294,7 @@ *p++ = 0; SANITY_CHECK(p); - p = &outbuf[1]; + p = &outbuf[1]; SANITY_CHECK(p); ret = PyString_FromString(p); PyMem_FREE(outbuf); @@ -2325,12 +2332,12 @@ } static PyMethodDef decimal_methods[] = { - {"_rescale", (PyCFunction)decimal_rescale, - METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("asdf")}, - {"_fix", (PyCFunction)decimal_fix, - METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("asdf")}, + {"_rescale", (PyCFunction)decimal_rescale, + METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("asdf")}, + {"_fix", (PyCFunction)decimal_fix, + METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("asdf")}, {"adjusted", (PyCFunction)decimal_adjusted, METH_NOARGS, PyDoc_STR("Return the adjusted exponent of self.")}, @@ -2413,27 +2420,27 @@ static PyObject * \ func (PyObject *self, PyObject *other) { \ decimalobject *res; \ - PyObject *to_decref; \ + PyObject *to_decref; \ contextobject *ctx = getcontext(); \ if (!ctx) return NULL; \ - if(PyDecimal_Check(self)) \ - { \ + if(PyDecimal_Check(self)) \ + { \ other = (PyObject*)_convert_to_decimal(self->ob_type, other, ctx, 0); \ - if (other == NULL || other == Py_NotImplemented) return other; \ - to_decref = other; \ - } \ - else \ - { \ - assert(PyDecimal_Check(other)); \ - self = (PyObject*)_convert_to_decimal(other->ob_type, self, ctx, 0); \ - to_decref = self; \ - if(self == NULL || self == Py_NotImplemented) return self; \ - } \ + if (other == NULL || other == Py_NotImplemented) return other; \ + to_decref = other; \ + } \ + else \ + { \ + assert(PyDecimal_Check(other)); \ + self = (PyObject*)_convert_to_decimal(other->ob_type, self, ctx, 0); \ + to_decref = self; \ + if(self == NULL || self == Py_NotImplemented) return self; \ + } \ res = _do_##func((decimalobject *)self, \ (decimalobject *)other, ctx); \ Py_DECREF(to_decref); \ return (PyObject *)res; \ - } + } /* Same for one-argument methods. */ #define DECIMAL_SPECIAL_1FUNC(func) \ @@ -2450,13 +2457,13 @@ _do_decimal_add(decimalobject *self, decimalobject *other, contextobject *ctx) { - assert(PyDecimal_Check(other)); - assert(PyDecimal_Check(self)); + assert(PyDecimal_Check(other)); + assert(PyDecimal_Check(self)); int shouldround, sign, negativezero = 0; long exp, oexp; - long prec,cmp; + long prec,cmp; decimalobject *res, *res2, *ret, *o1, *o2; - prec = ctx->prec; + prec = ctx->prec; if (ISSPECIAL(self) || ISSPECIAL(other)) { decimalobject *nan; @@ -2492,7 +2499,7 @@ res = _new_decimalobj(self->ob_type, 1, sign, exp); if (!res) return NULL; res->digits[0] = 0; - res->limbs[0] = 0; + res->limbs[0] = 0; return res; } if (!decimal_nonzero(self)) { @@ -2522,161 +2529,161 @@ } } - decimalobject *tmp; /* we borrow refference */ - decimalobject *oother; + decimalobject *tmp; /* we borrow refference */ + decimalobject *oother; - long numdigits = self->exp - other->exp; + long numdigits = self->exp - other->exp; - if(numdigits < 0) - { - numdigits *= -1; - tmp = other; - oother = self; - } - else - { - tmp = self; - oother = other; - } - - /* we borrow refference */ - if(shouldround && numdigits > prec + 1) - { - if(numdigits > (oother->ob_size + prec + 1 - tmp->ob_size)) - { - long extend = prec + 2 - tmp->ob_size; - if(extend <= 0) - extend = 1; - o1 = _NEW_decimalobj(tmp->ob_size + extend, tmp->sign, tmp->exp - extend); - if(!o1) - return NULL; - - _limb_first_n_digits(tmp->limbs, tmp->ob_size, 0, o1->limbs, o1->ob_size); - o2 = _NEW_decimalobj(1, oother->sign, o1->exp); - if(!o2) - { - Py_XDECREF(o1); - return NULL; - } - o2->limbs[0] =1; - goto calc; - } - } - - o1 = _NEW_decimalobj(tmp->ob_size + numdigits, tmp->sign, tmp->exp - numdigits); - _limb_first_n_digits(tmp->limbs, tmp->ob_size, 0, o1->limbs, o1->ob_size); - - if(!o1) - { - return NULL; - } - o2 = oother; - Py_INCREF(o2); + if(numdigits < 0) + { + numdigits *= -1; + tmp = other; + oother = self; + } + else + { + tmp = self; + oother = other; + } + + /* we borrow refference */ + if(shouldround && numdigits > prec + 1) + { + if(numdigits > (oother->ob_size + prec + 1 - tmp->ob_size)) + { + long extend = prec + 2 - tmp->ob_size; + if(extend <= 0) + extend = 1; + o1 = _NEW_decimalobj(tmp->ob_size + extend, tmp->sign, tmp->exp - extend); + if(!o1) + return NULL; + + _limb_first_n_digits(tmp->limbs, tmp->ob_size, 0, o1->limbs, o1->ob_size); + o2 = _NEW_decimalobj(1, oother->sign, o1->exp); + if(!o2) + { + Py_XDECREF(o1); + return NULL; + } + o2->limbs[0] =1; + goto calc; + } + } + + o1 = _NEW_decimalobj(tmp->ob_size + numdigits, tmp->sign, tmp->exp - numdigits); + _limb_first_n_digits(tmp->limbs, tmp->ob_size, 0, o1->limbs, o1->ob_size); + + if(!o1) + { + return NULL; + } + o2 = oother; + Py_INCREF(o2); calc: - assert(o1->exp == o2->exp); - exp = o1->exp; - cmp = _limb_compare(o1->limbs, o1->limb_count, o2->limbs, o2->limb_count); - - if(o1->sign != o2->sign) - { - if(!cmp) - { - Py_DECREF(o1); - Py_DECREF(o2); - ret = _NEW_decimalobj(1, negativezero, exp); - if(!ret) - return NULL; - ret->limbs[0] = 0; - if(ret->exp < ETINY(ctx)) - { - ret->exp = ETINY(ctx); - if(handle_Clamped(ctx, NULL) != 0) - { - Py_DECREF(ret); - return NULL; - } - } - - return ret; - } - else - { - long max_size; - if(cmp == -1) - { - ret = o1; - o1 = o2; - o2 = ret; - ret = 0; - } - /* o1 > o2 */ - max_size = o1->ob_size > o2->ob_size ? o1->ob_size : o2->ob_size; - ret = _NEW_decimalobj(max_size, o1->sign, o1->exp); - if(!ret) - { - Py_DECREF(o1); - Py_DECREF(o2); - return NULL; - } - - max_size = _limb_sub(o1->limbs, o1->ob_size, o2->limbs, o2->ob_size, ret->limbs); - ret->ob_size = max_size; - ret->limb_count = (max_size + LOG -1)/LOG; - - } - } - else - { - long max_size; - max_size = o1->ob_size > o2->ob_size ? o1->ob_size : o2->ob_size; - ret = _NEW_decimalobj(max_size + 1, o1->sign, o1->exp); /* we may have extra digit */ - if(!ret) - { - Py_DECREF(o1); - Py_DECREF(o2); - return NULL; - } - - max_size = _limb_add(o1->limbs, o1->ob_size, o2->limbs, o2->ob_size, ret->limbs); - ret->ob_size = max_size; - ret->limb_count = (max_size + LOG -1)/LOG; + assert(o1->exp == o2->exp); + exp = o1->exp; + cmp = _limb_compare(o1->limbs, o1->limb_count, o2->limbs, o2->limb_count); + + if(o1->sign != o2->sign) + { + if(!cmp) + { + Py_DECREF(o1); + Py_DECREF(o2); + ret = _NEW_decimalobj(1, negativezero, exp); + if(!ret) + return NULL; + ret->limbs[0] = 0; + if(ret->exp < ETINY(ctx)) + { + ret->exp = ETINY(ctx); + if(handle_Clamped(ctx, NULL) != 0) + { + Py_DECREF(ret); + return NULL; + } + } + + return ret; + } + else + { + long max_size; + if(cmp == -1) + { + ret = o1; + o1 = o2; + o2 = ret; + ret = 0; + } + /* o1 > o2 */ + max_size = o1->ob_size > o2->ob_size ? o1->ob_size : o2->ob_size; + ret = _NEW_decimalobj(max_size, o1->sign, o1->exp); + if(!ret) + { + Py_DECREF(o1); + Py_DECREF(o2); + return NULL; + } + + max_size = _limb_sub(o1->limbs, o1->ob_size, o2->limbs, o2->ob_size, ret->limbs); + ret->ob_size = max_size; + ret->limb_count = (max_size + LOG -1)/LOG; + + } + } + else + { + long max_size; + max_size = o1->ob_size > o2->ob_size ? o1->ob_size : o2->ob_size; + ret = _NEW_decimalobj(max_size + 1, o1->sign, o1->exp); /* we may have extra digit */ + if(!ret) + { + Py_DECREF(o1); + Py_DECREF(o2); + return NULL; + } + + max_size = _limb_add(o1->limbs, o1->ob_size, o2->limbs, o2->ob_size, ret->limbs); + ret->ob_size = max_size; + ret->limb_count = (max_size + LOG -1)/LOG; /* - Py_DECREF(o1); - Py_DECREF(o2); - if(shouldround) - { - fixed = _decimal_fix(ret, ctx); - Py_DECREF(ret); - if(!fixed) - { - return NULL; - } - return fixed; - } - return ret; */ - } - - Py_DECREF(o1); - Py_DECREF(o2); - - if(shouldround) - { - decimalobject *fixed; - fixed = _decimal_fix(ret,ctx); - Py_DECREF(ret); - if(!fixed) - return NULL; - return fixed; - } - - return ret; - - Py_XDECREF(o1); - Py_XDECREF(o2); - return NULL; + Py_DECREF(o1); + Py_DECREF(o2); + if(shouldround) + { + fixed = _decimal_fix(ret, ctx); + Py_DECREF(ret); + if(!fixed) + { + return NULL; + } + return fixed; + } + return ret; */ + } + + Py_DECREF(o1); + Py_DECREF(o2); + + if(shouldround) + { + decimalobject *fixed; + fixed = _decimal_fix(ret,ctx); + Py_DECREF(ret); + if(!fixed) + return NULL; + return fixed; + } + + return ret; + + Py_XDECREF(o1); + Py_XDECREF(o2); + return NULL; } DECIMAL_SPECIAL_2FUNC(decimal_add) @@ -2714,98 +2721,98 @@ _do_decimal_multiply(decimalobject *self, decimalobject *other, contextobject *ctx) { - long resultsign; - long resultexp; - long shouldround; - long i; - long max_limbs; - long size; - decimalobject *ans; - - resultsign = (self->sign&1) ^ (other->sign&1); - - if(ISSPECIAL(self) || ISSPECIAL(other)) - { - decimalobject *nan; - if(_check_nans(self, other, ctx, &nan)) - return nan; - - if(ISINF(self)) - { - if(decimal_nonzero(other)) - { - ans = _NEW_decimalobj(1, resultsign ? SIGN_NEGINF : SIGN_POSINF, 0); - return ans; - } - return handle_InvalidOperation(self->ob_type, ctx, "(+-)INF * 0", NULL); - } - - if(ISINF(other)) - { - if(decimal_nonzero(self)) - { - ans = _NEW_decimalobj(1, resultsign ? SIGN_NEGINF : SIGN_POSINF, 0); - return ans; - } - return handle_InvalidOperation(self->ob_type, ctx, "0 * (+-)INF", NULL); - } - } - - resultexp = self->exp + other->exp; - shouldround = ctx->rounding_dec == ALWAYS_ROUND; - - if(!decimal_nonzero(self) || !decimal_nonzero(other)) - { - decimalobject *ret; - ans = _NEW_decimalobj(1, resultsign, resultexp); - if(!ans) - return NULL; - - ans->limbs[0] = 0; - goto done; - } - - if(self->ob_size == 1 && self->limbs[0] == 1) - { - ans = _NEW_decimalobj(other->ob_size, resultsign, resultexp); - if(!ans) - return NULL; - - for(i = 0; ilimb_count;i++) - ans->limbs[i] = other->limbs[i]; - goto done; - } - - if(other->ob_size == 1 && other->limbs[0] == 1) - { - ans = _NEW_decimalobj(self->ob_size, resultsign, resultexp); - if(!ans) - return NULL; - - for(i = 0; ilimb_count; i++) - ans->limbs[i] = self->limbs[i]; - - goto done; - } - - max_limbs = (self->ob_size + LOG -1)/ LOG + (other->ob_size + LOG -1)/LOG; - ans = _NEW_decimalobj(max_limbs * LOG, resultsign, resultexp); - if(!ans) - return NULL; + long resultsign; + long resultexp; + long shouldround; + long i; + long max_limbs; + long size; + decimalobject *ans; - size = _limb_multiply(self->limbs, self->ob_size, other->limbs, other->ob_size, ans->limbs); + resultsign = (self->sign&1) ^ (other->sign&1); + + if(ISSPECIAL(self) || ISSPECIAL(other)) + { + decimalobject *nan; + if(_check_nans(self, other, ctx, &nan)) + return nan; - ans->ob_size = size; - + if(ISINF(self)) + { + if(decimal_nonzero(other)) + { + ans = _NEW_decimalobj(1, resultsign ? SIGN_NEGINF : SIGN_POSINF, 0); + return ans; + } + return handle_InvalidOperation(self->ob_type, ctx, "(+-)INF * 0", NULL); + } + + if(ISINF(other)) + { + if(decimal_nonzero(self)) + { + ans = _NEW_decimalobj(1, resultsign ? SIGN_NEGINF : SIGN_POSINF, 0); + return ans; + } + return handle_InvalidOperation(self->ob_type, ctx, "0 * (+-)INF", NULL); + } + } + + resultexp = self->exp + other->exp; + shouldround = ctx->rounding_dec == ALWAYS_ROUND; + + if(!decimal_nonzero(self) || !decimal_nonzero(other)) + { + decimalobject *ret; + ans = _NEW_decimalobj(1, resultsign, resultexp); + if(!ans) + return NULL; + + ans->limbs[0] = 0; + goto done; + } + + if(self->ob_size == 1 && self->limbs[0] == 1) + { + ans = _NEW_decimalobj(other->ob_size, resultsign, resultexp); + if(!ans) + return NULL; + + for(i = 0; ilimb_count;i++) + ans->limbs[i] = other->limbs[i]; + goto done; + } + + if(other->ob_size == 1 && other->limbs[0] == 1) + { + ans = _NEW_decimalobj(self->ob_size, resultsign, resultexp); + if(!ans) + return NULL; + + for(i = 0; ilimb_count; i++) + ans->limbs[i] = self->limbs[i]; + + goto done; + } + + max_limbs = (self->ob_size + LOG -1)/ LOG + (other->ob_size + LOG -1)/LOG; + ans = _NEW_decimalobj(max_limbs * LOG, resultsign, resultexp); + if(!ans) + return NULL; + + size = _limb_multiply(self->limbs, self->ob_size, other->limbs, other->ob_size, ans->limbs); + + ans->ob_size = size; + done: - if(shouldround) - { - decimalobject *fixed = _decimal_fix(ans, ctx); - Py_DECREF(ans); - return fixed; - } - return ans; + if(shouldround) + { + decimalobject *fixed = _decimal_fix(ans, ctx); + Py_DECREF(ans); + return fixed; + } + return ans; } DECIMAL_SPECIAL_2FUNC(decimal_multiply) @@ -2869,13 +2876,13 @@ } -static PyObject * -decimal_power(PyObject *self, PyObject *other, PyObject *modulo) { - decimalobject *res; +static PyObject * +decimal_power(PyObject *self, PyObject *other, PyObject *modulo) { + decimalobject *res; contextobject *ctx = getcontext(); - if (!ctx) return NULL; + if (!ctx) return NULL; - other = _convert_to_decimal(self->ob_type, other, ctx, 0); + other = _convert_to_decimal(self->ob_type, other, ctx, 0); if (other == NULL || other == Py_NotImplemented) return other; if (modulo == Py_None) { @@ -2887,10 +2894,10 @@ return modulo; } } - res = _do_decimal_power((decimalobject *)self, + res = _do_decimal_power((decimalobject *)self, (decimalobject *)other, (decimalobject *)modulo, - ctx); + ctx); Py_DECREF(other); Py_DECREF(modulo); return (PyObject *)res; @@ -3015,8 +3022,8 @@ buf[0] = (self->sign & 1 ? '-' : '+'); for (i = 0; i < max; i++) { if (i < self->ob_size) - buf[i+1] = _limb_get_digit(self->limbs, self->ob_size, i) + '0'; /* SLOW */ - else + buf[i+1] = _limb_get_digit(self->limbs, self->ob_size, i) + '0'; /* SLOW */ + else buf[i+1] = '0'; } buf[max+1] = 0; @@ -3058,8 +3065,8 @@ for (i = 0; i < max; i++) { if (i < self->ob_size) - res = res * 10 + _limb_get_digit(self->limbs, self->ob_size, i); /* SLOW */ - else + res = res * 10 + _limb_get_digit(self->limbs, self->ob_size, i); /* SLOW */ + else res *= 10; } if (self->sign & 1) res = -res; @@ -3087,9 +3094,9 @@ if (ISSPECIAL(self)) return 1; - for(i=0;ilimb_count;i++) - if(self->limbs[i] != 0) return 1; - return 0; + for(i=0;ilimb_count;i++) + if(self->limbs[i] != 0) return 1; + return 0; } @@ -3162,7 +3169,7 @@ decimal_dealloc(decimalobject *d) { PyObject_FREE(d->digits); - PyObject_FREE(d->limbs); + PyObject_FREE(d->limbs); d->ob_type->tp_free(d); } @@ -3179,14 +3186,14 @@ return NULL; /* remove sign */ if (str[0] == '+') - { + { str++; - len --; - } + len --; + } else if (str[0] == '-') { str++; literalsign = 1; - len --; + len --; } if (tolower(str[0]) == 'n' && tolower(str[1]) == 'a' && @@ -3224,24 +3231,24 @@ return NULL; for (i = 0; i < size; i++) new->digits[i] = *(start+i) -48; - for(i=0;ilimb_count;i++) - new->limbs[i] = 0; + for(i=0;ilimb_count;i++) + new->limbs[i] = 0; - long mult = 1; - long limb = 0; - for(i = size -1; i>=0; i--) - { - assert(limb < new->limb_count); - new->limbs[limb] += mult * (start[i] - '0'); - mult *= 10; - if(mult == BASE) - { - mult = 1; - limb ++; - } - } + long mult = 1; + long limb = 0; + for(i = size -1; i>=0; i--) + { + assert(limb < new->limb_count); + new->limbs[limb] += mult * (start[i] - '0'); + mult *= 10; + if(mult == BASE) + { + mult = 1; + limb ++; + } + } - return new; + return new; } @@ -3273,7 +3280,7 @@ if (!new) return NULL; new->digits[0] = 0; - new->limbs[0] = 0; + new->limbs[0] = 0; return new; } return NULL; @@ -3283,151 +3290,151 @@ static decimalobject * _decimal_fromliteral(PyTypeObject *type, char *str, long len, contextobject *ctx) { - char sign = 0; - long exp = 0; - long ndigits; - long zero = 0; /* if zero */ - long digits_after_dot = 0; - char *p = str; - char *first_digit = 0; - char *last_digit = 0; - char *dot = 0; - char *e = 0; - int any_digit = 0; - char *c; - long i; - decimalobject *new; - - if(!len) - goto err; /* empty string */ - - if(*p == '+') - p++; - else if(*p == '-') - { - p++; - sign = 1; - } - - if(*p == 'e' || *p == 'E') /* no digits before E */ - goto err; - - /* now at *p is our integer with (possibly) dot */ - - for(c = p; c - str < len; c++) - { - if(!isdigit(*c)) - { - if(*c == 'e' || *c == 'E') - { - if(e) - goto err; /* we should have only one e */ - e = c; - } - - else if(*c == '.') - { - if(dot || e) - goto err; - dot = c; - } - - else if(*c == '-' || *c == '+') /* we should have sign only after e */ - { - if(c - e != 1) - goto err; - } - else /* we shouldn't have anything else */ - goto err; - } - else - { - if(!e) - any_digit = 1; - if(dot && !e) - digits_after_dot ++; - - if(!first_digit && *c != '0') - { - if(!e) /* it's digit of exponent */ - first_digit = c; - } - } - } /* we now are pretty sure, that string is properly formated */ - if(!any_digit) - goto err; - if(!first_digit) - { - zero = 1; - ndigits = 1; - } - else /* let's count digits and find by the way last digit*/ - { - ndigits = 0; - for(c = first_digit; c - str < len; c++) - { - if(isdigit(*c)) - { - ndigits ++; - last_digit = c; - } - else - if(*c == 'e' || *c == 'E') - break; - - assert(isdigit(*c) || *c == '.'); - } - } - - if(!ndigits) - { - goto err; - } - - if(e) /* pretty obvious */ - { - int ok; - ok = sscanf(e+1,"%d", &exp); - if(ok!=1) - { - goto err; - } - } - exp -= digits_after_dot; - - new = _new_decimalobj(type, ndigits, sign, exp); - if(!new) - return NULL; - - for(i = 0; ilimb_count ;i++) - new->limbs[i] = 0; - - if(!first_digit) /* we are 0 */ - { - new->limbs[0] = 0; - return new; - } - - long mult = 1; /* now we just read integer till '\0' or 'e'/'E', dont care about '.' */ - long limb = 0; - - for(c = last_digit; c >= first_digit; c--) - { - if(*c == 'e' || *c == 'E') break; - if(!isdigit(*c)) continue; - - assert(limb < new->limb_count); - new->limbs[limb] += mult * (*c - '0'); - - mult *= 10; - if(mult == BASE) - { - limb ++; - mult = 1; - } - } - - return new; + char sign = 0; + long exp = 0; + long ndigits; + long zero = 0; /* if zero */ + long digits_after_dot = 0; + char *p = str; + char *first_digit = 0; + char *last_digit = 0; + char *dot = 0; + char *e = 0; + int any_digit = 0; + char *c; + long i; + decimalobject *new; + + if(!len) + goto err; /* empty string */ + + if(*p == '+') + p++; + else if(*p == '-') + { + p++; + sign = 1; + } + + if(*p == 'e' || *p == 'E') /* no digits before E */ + goto err; + + /* now at *p is our integer with (possibly) dot */ + + for(c = p; c - str < len; c++) + { + if(!isdigit(*c)) + { + if(*c == 'e' || *c == 'E') + { + if(e) + goto err; /* we should have only one e */ + e = c; + } + + else if(*c == '.') + { + if(dot || e) + goto err; + dot = c; + } + + else if(*c == '-' || *c == '+') /* we should have sign only after e */ + { + if(c - e != 1) + goto err; + } + else /* we shouldn't have anything else */ + goto err; + } + else + { + if(!e) + any_digit = 1; + if(dot && !e) + digits_after_dot ++; + + if(!first_digit && *c != '0') + { + if(!e) /* it's digit of exponent */ + first_digit = c; + } + } + } /* we now are pretty sure, that string is properly formated */ + if(!any_digit) + goto err; + if(!first_digit) + { + zero = 1; + ndigits = 1; + } + else /* let's count digits and find by the way last digit*/ + { + ndigits = 0; + for(c = first_digit; c - str < len; c++) + { + if(isdigit(*c)) + { + ndigits ++; + last_digit = c; + } + else + if(*c == 'e' || *c == 'E') + break; + + assert(isdigit(*c) || *c == '.'); + } + } + + if(!ndigits) + { + goto err; + } + + if(e) /* pretty obvious */ + { + int ok; + ok = sscanf(e+1,"%d", &exp); + if(ok!=1) + { + goto err; + } + } + exp -= digits_after_dot; + + new = _new_decimalobj(type, ndigits, sign, exp); + if(!new) + return NULL; + + for(i = 0; ilimb_count ;i++) + new->limbs[i] = 0; + + if(!first_digit) /* we are 0 */ + { + new->limbs[0] = 0; + return new; + } + + long mult = 1; /* now we just read integer till '\0' or 'e'/'E', dont care about '.' */ + long limb = 0; + + for(c = last_digit; c >= first_digit; c--) + { + if(*c == 'e' || *c == 'E') break; + if(!isdigit(*c)) continue; + + assert(limb < new->limb_count); + new->limbs[limb] += mult * (*c - '0'); + + mult *= 10; + if(mult == BASE) + { + limb ++; + mult = 1; + } + } + + return new; err: return handle_ConversionSyntax(type, ctx, "invalid literal for Decimal"); @@ -3469,8 +3476,8 @@ if (value == 0) { new = _new_decimalobj(type, 1, 0, 0); if (!new) return NULL; - new->digits[0] = 0; - new->limbs[0] = 0; + new->digits[0] = 0; + new->limbs[0] = 0; return (PyObject *)new; } else if (value < 0) { value = -value; @@ -3483,8 +3490,8 @@ v /= 10; } - v = value; - + v = value; + new = _new_decimalobj(type, ndigits, (neg ? SIGN_NEG : SIGN_POS), 0); if (!new) return NULL; while (value) { @@ -3493,12 +3500,12 @@ i++; } - for(i=0; i < new->limb_count; i++) - { - new->limbs[i] = v % BASE; - v /= BASE; - } - + for(i=0; i < new->limb_count; i++) + { + new->limbs[i] = v % BASE; + v /= BASE; + } + return (PyObject *)new; } @@ -3546,7 +3553,7 @@ new = _new_decimalobj(type, PyTuple_GET_SIZE(digtup), sign, exp); for (i = 0; i < new->ob_size; i++) { item = PyTuple_GET_ITEM(digtup, i); - long x; + long x; if (PyInt_Check(item)) { x = PyInt_AsLong(item); } else if (PyLong_Check(item)) { @@ -3558,55 +3565,55 @@ "must be composed of non negative integer elements."); goto err; } - - if(x < 0 || x > 9) - { - PyErr_Format(PyExc_ValueError, "Invalid digit: %ld", x); - goto err; - } - - new->digits[i] = x; - } /* this loop will go out soon XXX */ - - for(i = 0;i < new->limb_count;i++) - new->limbs[i] = 0; - - long mult = 1; - long limb = 0; - new->limbs[0] = 0; - for(i = new->ob_size-1; i>=0; i--) /* limb[0] keeps least significant limb */ - { - item = PyTuple_GET_ITEM(digtup, i); - long x; - if(PyInt_Check(item)) - x = PyInt_AsLong(item); - else if (PyLong_Check(item)) { - x = PyLong_AsLong(item); - if(x == -1 && PyErr_Occurred()) - goto err; - } - else { - PyErr_SetString(PyExc_ValueError, "The second value in the tuple " - "must be composed of non negative integer elements."); - goto err; - } - - if(x < 0 || x > 9) - { - PyErr_Format(PyExc_ValueError, "Invalid digit: %ld", x); - goto err; - } - - assert(limb < new->limb_count); - new->limbs[limb] += mult * x; - mult *= 10; - - if(mult == BASE) /* we already used LOG digits of one limb */ - { - mult = 1; - limb ++; - } - } + + if(x < 0 || x > 9) + { + PyErr_Format(PyExc_ValueError, "Invalid digit: %ld", x); + goto err; + } + + new->digits[i] = x; + } /* this loop will go out soon XXX */ + + for(i = 0;i < new->limb_count;i++) + new->limbs[i] = 0; + + long mult = 1; + long limb = 0; + new->limbs[0] = 0; + for(i = new->ob_size-1; i>=0; i--) /* limb[0] keeps least significant limb */ + { + item = PyTuple_GET_ITEM(digtup, i); + long x; + if(PyInt_Check(item)) + x = PyInt_AsLong(item); + else if (PyLong_Check(item)) { + x = PyLong_AsLong(item); + if(x == -1 && PyErr_Occurred()) + goto err; + } + else { + PyErr_SetString(PyExc_ValueError, "The second value in the tuple " + "must be composed of non negative integer elements."); + goto err; + } + + if(x < 0 || x > 9) + { + PyErr_Format(PyExc_ValueError, "Invalid digit: %ld", x); + goto err; + } + + assert(limb < new->limb_count); + new->limbs[limb] += mult * x; + mult *= 10; + + if(mult == BASE) /* we already used LOG digits of one limb */ + { + mult = 1; + limb ++; + } + } Py_DECREF(digtup); Py_DECREF(tup); @@ -3671,16 +3678,16 @@ decimalobject *new = _new_decimalobj(type, 1, 0, 0); if (!new) return NULL; new->digits[0] = 0; - new->limbs[0] = 0; + new->limbs[0] = 0; return (PyObject *)new; } - if(context) - if(!PyDecimalContext_Check(context)) - { - PyErr_SetString(PyExc_TypeError, "context must be Context type"); - return NULL; - } + if(context) + if(!PyDecimalContext_Check(context)) + { + PyErr_SetString(PyExc_TypeError, "context must be Context type"); + return NULL; + } if (PyDecimal_Check(value)) return (PyObject *)_decimal_get_copy((decimalobject *)value); @@ -3700,10 +3707,10 @@ return decimal_from_long(type, x); } - if(context) - ctx = (contextobject*) context; - else - ctx = getcontext(); + if(context) + ctx = (contextobject*) context; + else + ctx = getcontext(); if (!ctx) return NULL; @@ -3805,7 +3812,7 @@ static PyMemberDef _decimal_members[] = { {"_sign", T_INT, offsetof(decimalobject, sign), 0}, {"_exp", T_LONG, offsetof(decimalobject, exp), 0}, - {"_size", T_LONG, offsetof(decimalobject, ob_size), 0}, + {"_size", T_LONG, offsetof(decimalobject, ob_size), 0}, {NULL} }; @@ -3818,7 +3825,7 @@ tup = PyTuple_New(self->ob_size); if (!tup) return NULL; for (i = 0; i < self->ob_size; i++) - PyTuple_SET_ITEM(tup, i, PyInt_FromLong(_limb_get_digit(self->limbs, self->ob_size, i))); /* SLOW */ + PyTuple_SET_ITEM(tup, i, PyInt_FromLong(_limb_get_digit(self->limbs, self->ob_size, i))); /* SLOW */ return tup; } @@ -3864,13 +3871,13 @@ static PyObject * _decimal_get_limbs(decimalobject *self) { - PyObject *tup; - long i; - tup = PyTuple_New(self->limb_count); - if(!tup) return NULL; - for(i = 0; i< self->limb_count; i++) - PyTuple_SET_ITEM(tup, i, PyInt_FromLong(self->limbs[i])); - return tup; + PyObject *tup; + long i; + tup = PyTuple_New(self->limb_count); + if(!tup) return NULL; + for(i = 0; i< self->limb_count; i++) + PyTuple_SET_ITEM(tup, i, PyInt_FromLong(self->limbs[i])); + return tup; } static PyObject * @@ -3884,7 +3891,7 @@ static PyGetSetDef _decimal_getset[] = { - {"_limbs", (getter)_decimal_get_limbs, 0}, + {"_limbs", (getter)_decimal_get_limbs, 0}, {"_int", (getter)_decimal_get_int, (setter)_decimal_set_int}, {"_is_special", (getter)_decimal_is_special, 0}, {NULL} @@ -4068,7 +4075,7 @@ t = traps; Py_INCREF(t); } - if (!t) goto err; + if (!t) goto err; if (!ignored) { i = PyDict_New(); @@ -4104,11 +4111,11 @@ static PyObject * context_clear_flags(contextobject *self) { - int i; - for(i = 0;iflags,i,0); - - Py_RETURN_NONE; + int i; + for(i = 0;iflags,i,0); + + Py_RETURN_NONE; } @@ -4251,43 +4258,43 @@ static PyObject * context_apply(contextobject *self, PyObject *args, PyObject *kwds) { - static char *kwlist[] = {"a", 0}; - decimalobject *a; + static char *kwlist[] = {"a", 0}; + decimalobject *a; - if(!PyArg_ParseTupleAndKeywords(args, kwds, "O:_apply", kwlist, &a)) - return NULL; + if(!PyArg_ParseTupleAndKeywords(args, kwds, "O:_apply", kwlist, &a)) + return NULL; - if(!PyDecimal_Check(a)) - { - PyErr_SetString(PyExc_ValueError, "a must be Decimal object"); - return NULL; - } - - decimalobject *tmp = _decimal_fix(a, self); - if(!tmp) - return NULL; + if(!PyDecimal_Check(a)) + { + PyErr_SetString(PyExc_ValueError, "a must be Decimal object"); + return NULL; + } + + decimalobject *tmp = _decimal_fix(a, self); + if(!tmp) + return NULL; - PyObject *ret = decimal_str(tmp); - Py_DECREF(tmp); + PyObject *ret = decimal_str(tmp); + Py_DECREF(tmp); - return ret; - + return ret; + } - + static PyObject * context_power(contextobject *self, PyObject *args) { - PyObject *a, *b, *c; - decimalobject *dec_a = NULL, *dec_b = NULL, *dec_c = NULL, *res; + PyObject *a, *b, *c; + decimalobject *dec_a = NULL, *dec_b = NULL, *dec_c = NULL, *res; if (!PyArg_ParseTuple(args, "OO|O:power", &a, &b, &c)) return NULL; - dec_a = (decimalobject *)_convert_to_decimal( - &PyDecimal_DecimalType, a, self, 1); + dec_a = (decimalobject *)_convert_to_decimal( + &PyDecimal_DecimalType, a, self, 1); if (dec_a == NULL) - return NULL; + return NULL; - dec_b = (decimalobject *)_convert_to_decimal( - &PyDecimal_DecimalType, b, self, 1); + dec_b = (decimalobject *)_convert_to_decimal( + &PyDecimal_DecimalType, b, self, 1); if (dec_b == NULL) { Py_DECREF(dec_a); return NULL; @@ -4300,11 +4307,11 @@ Py_DECREF(dec_b); return NULL; } - res = _do_decimal_power(dec_a, dec_b, dec_c, self); - Py_DECREF(dec_a); + res = _do_decimal_power(dec_a, dec_b, dec_c, self); + Py_DECREF(dec_a); Py_DECREF(dec_b); Py_DECREF(dec_c); - return (PyObject *)res; + return (PyObject *)res; } @@ -4365,7 +4372,7 @@ static PyObject * context_to_integral(contextobject *self, PyObject *args, PyObject *kwds) { - static char *kwlist[] = {"a", 0}; + static char *kwlist[] = {"a", 0}; PyObject *a, *res; decimalobject *dec_a = NULL; @@ -4388,7 +4395,7 @@ { PyObject *a, *res; decimalobject *dec_a = NULL; - static char *kwlist[] = {"a", 0}; + static char *kwlist[] = {"a", 0}; if (!PyArg_ParseTupleAndKeywords(args, kwds, "O:to_sci_string", kwlist, &a)) return NULL; @@ -4693,8 +4700,8 @@ /* XXX: all that PyCFunction casting might not be necessary */ static PyMethodDef context_methods[] = { - {"_apply", (PyCFunction)context_apply, - METH_VARARGS | METH_KEYWORDS}, + {"_apply", (PyCFunction)context_apply, + METH_VARARGS | METH_KEYWORDS}, {"clear_flags", (PyCFunction)context_clear_flags, METH_NOARGS}, {"copy", (PyCFunction)context_copy, @@ -4926,10 +4933,10 @@ _flags = PyDict_New(); if (!_flags) goto err; - for(i=0; i < NUMSIGNALS; i++) /* XXX don't know if it's ok! */ - { - _set_flag(_flags,i,0); - } + for(i=0; i < NUMSIGNALS; i++) /* XXX don't know if it's ok! */ + { + _set_flag(_flags,i,0); + } if (pyflags == NULL) { /* don't copy flags from default context */ @@ -5001,8 +5008,8 @@ capitals = capitals & 1; /* if (rounding_dec == -1) */ - if(rounding_dec != NEVER_ROUND && rounding_dec != ALWAYS_ROUND) /* XXX????*/ - rounding_dec = PyDecimal_DefaultContext->rounding_dec; + if(rounding_dec != NEVER_ROUND && rounding_dec != ALWAYS_ROUND) /* XXX????*/ + rounding_dec = PyDecimal_DefaultContext->rounding_dec; if (rounding_dec != NEVER_ROUND && rounding_dec != ALWAYS_ROUND) { PyErr_SetString(PyExc_ValueError, "_rounding_decision must be one of " @@ -5137,7 +5144,7 @@ static PyMethodDef module_methods[] = { {"getcontext", (PyCFunction)module_getcontext, METH_NOARGS}, {"setcontext", (PyCFunction)module_setcontext, METH_VARARGS | METH_KEYWORDS}, - {NULL, NULL} + {NULL, NULL} }; @@ -5166,7 +5173,7 @@ PyMODINIT_FUNC INITFUNC_NAME(void) { - PyObject *m; /* a module object */ + PyObject *m; /* a module object */ PyObject *tup; contextobject *ctx; PyObject *traps; From neal at metaslash.com Mon Jun 12 23:29:28 2006 From: neal at metaslash.com (Neal Norwitz) Date: Mon, 12 Jun 2006 17:29:28 -0400 Subject: [Python-checkins] Python Regression Test Failures all (1) Message-ID: <20060612212928.GA4827@python.psfb.org> test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat Exception in thread Thread-1: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') Exception in thread Thread-2: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') test test_asynchat failed -- errors occurred in test.test_asynchat.TestAsynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler testCompileLibrary still working, be patient... test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8679 refs] [8679 refs] [8679 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9027 refs] [9027 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socketserver test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8890 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] this bit of output is from a test of stdout in a different process ... [8674 refs] [8674 refs] [8890 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8674 refs] [8675 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8674 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllibnet test_urlparse test_userdict test_userlist test_userstring test_uu test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 291 tests OK. 1 test failed: test_asynchat 21 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl test_gl test_imgfile test_ioctl test_macfs test_macostools test_nis test_pep277 test_plistlib test_scriptpackages test_startfile test_sunaudiodev test_tcl test_unicode_file test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [707800 refs] From buildbot at python.org Mon Jun 12 23:34:41 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 12 Jun 2006 21:34:41 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP trunk Message-ID: <20060612213441.DD8591E4002@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/1026 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 01:47:54 2006 From: python-checkins at python.org (ka-ping.yee) Date: Tue, 13 Jun 2006 01:47:54 +0200 (CEST) Subject: [Python-checkins] r46901 - in python/trunk/Lib: test/test_uuid.py uuid.py Message-ID: <20060612234754.2A0B61E4002@bag.python.org> Author: ka-ping.yee Date: Tue Jun 13 01:47:52 2006 New Revision: 46901 Added: python/trunk/Lib/test/test_uuid.py python/trunk/Lib/uuid.py Log: Add the uuid module. This module has been tested so far on Windows XP (Python 2.4 and 2.5a2), Mac OS X (Python 2.3, 2.4, and 2.5a2), and Linux (Python 2.4 and 2.5a2). Added: python/trunk/Lib/test/test_uuid.py ============================================================================== --- (empty file) +++ python/trunk/Lib/test/test_uuid.py Tue Jun 13 01:47:52 2006 @@ -0,0 +1,396 @@ +from unittest import TestCase, main +import uuid + +def importable(name): + try: + __import__(name) + return True + except: + return False + +class TestUUID(TestCase): + last_node = None + + def test_UUID(self): + equal = self.assertEqual + ascending = [] + for (string, curly, hex, bytes, fields, integer, urn, + time, clock_seq, variant, version) in [ + ('00000000-0000-0000-0000-000000000000', + '{00000000-0000-0000-0000-000000000000}', + '00000000000000000000000000000000', + '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', + (0, 0, 0, 0, 0, 0), + 0, + 'urn:uuid:00000000-0000-0000-0000-000000000000', + 0, 0, uuid.RESERVED_NCS, None), + ('00010203-0405-0607-0809-0a0b0c0d0e0f', + '{00010203-0405-0607-0809-0a0b0c0d0e0f}', + '000102030405060708090a0b0c0d0e0f', + '\0\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\x0d\x0e\x0f', + (0x00010203L, 0x0405, 0x0607, 8, 9, 0x0a0b0c0d0e0fL), + 0x000102030405060708090a0b0c0d0e0fL, + 'urn:uuid:00010203-0405-0607-0809-0a0b0c0d0e0f', + 0x607040500010203L, 0x809, uuid.RESERVED_NCS, None), + ('02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + '{02d9e6d5-9467-382e-8f9b-9300a64ac3cd}', + '02d9e6d59467382e8f9b9300a64ac3cd', + '\x02\xd9\xe6\xd5\x94\x67\x38\x2e\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', + (0x02d9e6d5L, 0x9467, 0x382e, 0x8f, 0x9b, 0x9300a64ac3cdL), + 0x02d9e6d59467382e8f9b9300a64ac3cdL, + 'urn:uuid:02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + 0x82e946702d9e6d5L, 0xf9b, uuid.RFC_4122, 3), + ('12345678-1234-5678-1234-567812345678', + '{12345678-1234-5678-1234-567812345678}', + '12345678123456781234567812345678', + '\x12\x34\x56\x78'*4, + (0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678), + 0x12345678123456781234567812345678, + 'urn:uuid:12345678-1234-5678-1234-567812345678', + 0x678123412345678L, 0x1234, uuid.RESERVED_NCS, None), + ('6ba7b810-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b810-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8109dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x10\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b810L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8109dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b810-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b810L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b811-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b811-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8119dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x11\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b811L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8119dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b811-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b811L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b812-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b812-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8129dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x12\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b812L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8129dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b812-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b812L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b814-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b814-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8149dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x14\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b814L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8149dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b814-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b814L, 0xb4, uuid.RFC_4122, 1), + ('7d444840-9dc0-11d1-b245-5ffdce74fad2', + '{7d444840-9dc0-11d1-b245-5ffdce74fad2}', + '7d4448409dc011d1b2455ffdce74fad2', + '\x7d\x44\x48\x40\x9d\xc0\x11\xd1\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', + (0x7d444840L, 0x9dc0, 0x11d1, 0xb2, 0x45, 0x5ffdce74fad2L), + 0x7d4448409dc011d1b2455ffdce74fad2L, + 'urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2', + 0x1d19dc07d444840L, 0x3245, uuid.RFC_4122, 1), + ('e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + '{e902893a-9d22-3c7e-a7b8-d6e313b71d9f}', + 'e902893a9d223c7ea7b8d6e313b71d9f', + '\xe9\x02\x89\x3a\x9d\x22\x3c\x7e\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', + (0xe902893aL, 0x9d22, 0x3c7e, 0xa7, 0xb8, 0xd6e313b71d9fL), + 0xe902893a9d223c7ea7b8d6e313b71d9fL, + 'urn:uuid:e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + 0xc7e9d22e902893aL, 0x27b8, uuid.RFC_4122, 3), + ('eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + '{eb424026-6f54-4ef8-a4d0-bb658a1fc6cf}', + 'eb4240266f544ef8a4d0bb658a1fc6cf', + '\xeb\x42\x40\x26\x6f\x54\x4e\xf8\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', + (0xeb424026L, 0x6f54, 0x4ef8, 0xa4, 0xd0, 0xbb658a1fc6cfL), + 0xeb4240266f544ef8a4d0bb658a1fc6cfL, + 'urn:uuid:eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + 0xef86f54eb424026L, 0x24d0, uuid.RFC_4122, 4), + ('f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + '{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}', + 'f81d4fae7dec11d0a76500a0c91e6bf6', + '\xf8\x1d\x4f\xae\x7d\xec\x11\xd0\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', + (0xf81d4faeL, 0x7dec, 0x11d0, 0xa7, 0x65, 0x00a0c91e6bf6L), + 0xf81d4fae7dec11d0a76500a0c91e6bf6L, + 'urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + 0x1d07decf81d4faeL, 0x2765, uuid.RFC_4122, 1), + ('fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + '{fffefdfc-fffe-fffe-fffe-fffefdfcfbfa}', + 'fffefdfcfffefffefffefffefdfcfbfa', + '\xff\xfe\xfd\xfc\xff\xfe\xff\xfe\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', + (0xfffefdfcL, 0xfffe, 0xfffe, 0xff, 0xfe, 0xfffefdfcfbfaL), + 0xfffefdfcfffefffefffefffefdfcfbfaL, + 'urn:uuid:fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + 0xffefffefffefdfcL, 0x3ffe, uuid.RESERVED_FUTURE, None), + ('ffffffff-ffff-ffff-ffff-ffffffffffff', + '{ffffffff-ffff-ffff-ffff-ffffffffffff}', + 'ffffffffffffffffffffffffffffffff', + '\xff'*16, + (0xffffffffL, 0xffffL, 0xffffL, 0xff, 0xff, 0xffffffffffffL), + 0xffffffffffffffffffffffffffffffffL, + 'urn:uuid:ffffffff-ffff-ffff-ffff-ffffffffffff', + 0xfffffffffffffffL, 0x3fff, uuid.RESERVED_FUTURE, None), + ]: + equivalents = [] + # Construct each UUID in several different ways. + for u in [uuid.UUID(string), uuid.UUID(curly), uuid.UUID(hex), + uuid.UUID(bytes=bytes), uuid.UUID(fields=fields), + uuid.UUID(int=integer), uuid.UUID(urn)]: + # Test all conversions and properties of the UUID object. + equal(str(u), string) + equal(int(u), integer) + equal(u.bytes, bytes) + equal(u.fields, fields) + equal(u.time_low, fields[0]) + equal(u.time_mid, fields[1]) + equal(u.time_hi_version, fields[2]) + equal(u.clock_seq_hi_variant, fields[3]) + equal(u.clock_seq_low, fields[4]) + equal(u.node, fields[5]) + equal(u.hex, hex) + equal(u.int, integer) + equal(u.urn, urn) + equal(u.time, time) + equal(u.clock_seq, clock_seq) + equal(u.variant, variant) + equal(u.version, version) + equivalents.append(u) + + # Different construction methods should give the same UUID. + for u in equivalents: + for v in equivalents: + equal(u, v) + ascending.append(u) + + # Test comparison of UUIDs. + for i in range(len(ascending)): + for j in range(len(ascending)): + equal(cmp(i, j), cmp(ascending[i], ascending[j])) + + # Test sorting of UUIDs (above list is in ascending order). + resorted = ascending[:] + resorted.reverse() + resorted.sort() + equal(ascending, resorted) + + def test_exceptions(self): + badvalue = lambda f: self.assertRaises(ValueError, f) + badtype = lambda f: self.assertRaises(TypeError, f) + + # Badly formed hex strings. + badvalue(lambda: uuid.UUID('')) + badvalue(lambda: uuid.UUID('abc')) + badvalue(lambda: uuid.UUID('1234567812345678123456781234567')) + badvalue(lambda: uuid.UUID('123456781234567812345678123456789')) + badvalue(lambda: uuid.UUID('123456781234567812345678z2345678')) + + # Badly formed bytes. + badvalue(lambda: uuid.UUID(bytes='abc')) + badvalue(lambda: uuid.UUID(bytes='\0'*15)) + badvalue(lambda: uuid.UUID(bytes='\0'*17)) + + # Badly formed fields. + badvalue(lambda: uuid.UUID(fields=(1,))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5, 6, 7))) + + # Field values out of range. + badvalue(lambda: uuid.UUID(fields=(-1, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0x100000000L, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, -1, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0x10000L, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, -1, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0x10000L, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, -1, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0x100L, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, -1, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0x100L, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, -1))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, 0x1000000000000L))) + + # Version number out of range. + badvalue(lambda: uuid.UUID('00'*16, version=0)) + badvalue(lambda: uuid.UUID('00'*16, version=6)) + + # Integer value out of range. + badvalue(lambda: uuid.UUID(int=-1)) + badvalue(lambda: uuid.UUID(int=1<<128L)) + + # Must supply exactly one of hex, bytes, fields, int. + h, b, f, i = '00'*16, '\0'*16, (0, 0, 0, 0, 0, 0), 0 + uuid.UUID(h) + uuid.UUID(hex=h) + uuid.UUID(bytes=b) + uuid.UUID(fields=f) + uuid.UUID(int=i) + + # Wrong number of arguments (positional). + badtype(lambda: uuid.UUID()) + badtype(lambda: uuid.UUID(h, b)) + badtype(lambda: uuid.UUID(h, b, f)) + badtype(lambda: uuid.UUID(h, b, f, i)) + + # Duplicate arguments (named). + badtype(lambda: uuid.UUID(hex=h, bytes=b)) + badtype(lambda: uuid.UUID(hex=h, fields=f)) + badtype(lambda: uuid.UUID(hex=h, int=i)) + badtype(lambda: uuid.UUID(bytes=b, fields=f)) + badtype(lambda: uuid.UUID(bytes=b, int=i)) + badtype(lambda: uuid.UUID(fields=f, int=i)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i, fields=f)) + + # Duplicate arguments (positional and named). + badtype(lambda: uuid.UUID(h, hex=h)) + badtype(lambda: uuid.UUID(h, bytes=b)) + badtype(lambda: uuid.UUID(h, fields=f)) + badtype(lambda: uuid.UUID(h, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i, fields=f)) + + # Immutability. + u = uuid.UUID(h) + badtype(lambda: setattr(u, 'hex', h)) + badtype(lambda: setattr(u, 'bytes', b)) + badtype(lambda: setattr(u, 'fields', f)) + badtype(lambda: setattr(u, 'int', i)) + + def check_node(self, node, source=''): + individual_group_bit = (node >> 40L) & 1 + universal_local_bit = (node >> 40L) & 2 + message = "%012x doesn't look like a real MAC address" % node + self.assertEqual(individual_group_bit, 0, message) + self.assertEqual(universal_local_bit, 0, message) + self.assertNotEqual(node, 0, message) + self.assertNotEqual(node, 0xffffffffffffL, message) + self.assert_(0 <= node, message) + self.assert_(node < 1<<48L, message) + + import sys + if source: + sys.stderr.write('(%s: %012x)' % (source, node)) + if TestUUID.last_node: + self.assertEqual(TestUUID.last_node, node, 'inconsistent node IDs') + else: + TestUUID.last_node = node + + def test_ifconfig_getnode(self): + import os + if os.name == 'posix': + self.check_node(uuid._ifconfig_getnode(), 'ifconfig') + + def test_ipconfig_getnode(self): + import os + if os.name == 'nt': + self.check_node(uuid._ipconfig_getnode(), 'ipconfig') + + def test_netbios_getnode(self): + if importable('win32wnet') and importable('netbios'): + self.check_node(uuid._netbios_getnode(), 'netbios') + + def test_random_getnode(self): + node = uuid._random_getnode() + self.assert_(0 <= node) + self.assert_(node < 1<<48L) + + def test_unixdll_getnode(self): + import os + if importable('ctypes') and os.name == 'posix': + self.check_node(uuid._unixdll_getnode(), 'unixdll') + + def test_windll_getnode(self): + import os + if importable('ctypes') and os.name == 'nt': + self.check_node(uuid._windll_getnode(), 'windll') + + def test_getnode(self): + self.check_node(uuid.getnode()) + + # Test it again to ensure consistency. + self.check_node(uuid.getnode()) + + def test_uuid1(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 1. + for u in [uuid.uuid1() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 1) + + # Make sure the supplied node ID appears in the UUID. + u = uuid.uuid1(0) + equal(u.node, 0) + u = uuid.uuid1(0x123456789abc) + equal(u.node, 0x123456789abc) + u = uuid.uuid1(0xffffffffffff) + equal(u.node, 0xffffffffffff) + + # Make sure the supplied clock sequence appears in the UUID. + u = uuid.uuid1(0x123456789abc, 0) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | u.clock_seq_low, 0) + u = uuid.uuid1(0x123456789abc, 0x1234) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x1234) + u = uuid.uuid1(0x123456789abc, 0x3fff) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x3fff) + + def test_uuid3(self): + equal = self.assertEqual + + # Test some known version-3 UUIDs. + for u, v in [(uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org'), + '6fa459ea-ee8a-3ca4-894e-db77e160355e'), + (uuid.uuid3(uuid.NAMESPACE_URL, 'http://python.org/'), + '9fe8e8c4-aaa8-32a9-a55c-4535a88b748d'), + (uuid.uuid3(uuid.NAMESPACE_OID, '1.3.6.1'), + 'dd1a1cef-13d5-368a-ad82-eca71acd4cd1'), + (uuid.uuid3(uuid.NAMESPACE_X500, 'c=ca'), + '658d3002-db6b-3040-a1d1-8ddd7d189a4d'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 3) + equal(u, uuid.UUID(v)) + equal(str(u), v) + + def test_uuid4(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 4. + for u in [uuid.uuid4() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 4) + + def test_uuid5(self): + equal = self.assertEqual + + # Test some known version-5 UUIDs. + for u, v in [(uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org'), + '886313e1-3b8a-5372-9b90-0c9aee199e5d'), + (uuid.uuid5(uuid.NAMESPACE_URL, 'http://python.org/'), + '4c565f0d-3f5a-5890-b41b-20cf47701c5e'), + (uuid.uuid5(uuid.NAMESPACE_OID, '1.3.6.1'), + '1447fa61-5277-5fef-a9b3-fbc6e44f4af3'), + (uuid.uuid5(uuid.NAMESPACE_X500, 'c=ca'), + 'cc957dd1-a972-5349-98cd-874190002798'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 5) + equal(u, uuid.UUID(v)) + equal(str(u), v) + +if __name__ == '__main__': + main() Added: python/trunk/Lib/uuid.py ============================================================================== --- (empty file) +++ python/trunk/Lib/uuid.py Tue Jun 13 01:47:52 2006 @@ -0,0 +1,477 @@ +r"""UUID objects (universally unique identifiers) according to RFC 4122. + +This module provides immutable UUID objects (class UUID) and the functions +uuid1(), uuid3(), uuid4(), uuid5() for generating version 1, 3, 4, and 5 +UUIDs as specified in RFC 4122. + +If all you want is a unique ID, you should probably call uuid1() or uuid4(). +Note that uuid1() may compromise privacy since it creates a UUID containing +the computer's network address. uuid4() creates a random UUID. + +Typical usage: + + >>> import uuid + + # make a UUID based on the host ID and current time + >>> uuid.uuid1() + UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + + # make a UUID using an MD5 hash of a namespace UUID and a name + >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') + UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + + # make a random UUID + >>> uuid.uuid4() + UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + + # make a UUID using a SHA-1 hash of a namespace UUID and a name + >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') + UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + + # make a UUID from a string of hex digits (braces and hyphens ignored) + >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') + + # convert a UUID to a string of hex digits in standard form + >>> str(x) + '00010203-0405-0607-0809-0a0b0c0d0e0f' + + # get the raw 16 bytes of the UUID + >>> x.bytes + '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + + # make a UUID from a 16-byte string + >>> uuid.UUID(bytes=x.bytes) + UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') + +This module works with Python 2.3 or higher.""" + +__author__ = 'Ka-Ping Yee ' +__date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') +__version__ = '$Revision: 1.30 $'.split()[1] + +RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, RESERVED_FUTURE = [ + 'reserved for NCS compatibility', 'specified in RFC 4122', + 'reserved for Microsoft compatibility', 'reserved for future definition'] + +class UUID(object): + """Instances of the UUID class represent UUIDs as specified in RFC 4122. + UUID objects are immutable, hashable, and usable as dictionary keys. + Converting a UUID to a string with str() yields something in the form + '12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts + four possible forms: a similar string of hexadecimal digits, or a + string of 16 raw bytes as an argument named 'bytes', or a tuple of + six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and + 48-bit values respectively) as an argument named 'fields', or a single + 128-bit integer as an argument named 'int'. + + UUIDs have these read-only attributes: + + bytes the UUID as a 16-byte string + + fields a tuple of the six integer fields of the UUID, + which are also available as six individual attributes + and two derived attributes: + + time_low the first 32 bits of the UUID + time_mid the next 16 bits of the UUID + time_hi_version the next 16 bits of the UUID + clock_seq_hi_variant the next 8 bits of the UUID + clock_seq_low the next 8 bits of the UUID + node the last 48 bits of the UUID + + time the 60-bit timestamp + clock_seq the 14-bit sequence number + + hex the UUID as a 32-character hexadecimal string + + int the UUID as a 128-bit integer + + urn the UUID as a URN as specified in RFC 4122 + + variant the UUID variant (one of the constants RESERVED_NCS, + RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE) + + version the UUID version number (1 through 5, meaningful only + when the variant is RFC_4122) + """ + + def __init__(self, hex=None, bytes=None, fields=None, int=None, + version=None): + r"""Create a UUID from either a string of 32 hexadecimal digits, + a string of 16 bytes as the 'bytes' argument, a tuple of six + integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, + 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as + the 'fields' argument, or a single 128-bit integer as the 'int' + argument. When a string of hex digits is given, curly braces, + hyphens, and a URN prefix are all optional. For example, these + expressions all yield the same UUID: + + UUID('{12345678-1234-5678-1234-567812345678}') + UUID('12345678123456781234567812345678') + UUID('urn:uuid:12345678-1234-5678-1234-567812345678') + UUID(bytes='\x12\x34\x56\x78'*4) + UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) + UUID(int=0x12345678123456781234567812345678) + + Exactly one of 'hex', 'bytes', 'fields', or 'int' must be given. + The 'version' argument is optional; if given, the resulting UUID + will have its variant and version number set according to RFC 4122, + overriding bits in the given 'hex', 'bytes', 'fields', or 'int'. + """ + + if [hex, bytes, fields, int].count(None) != 3: + raise TypeError('need just one of hex, bytes, fields, or int') + if hex is not None: + hex = hex.replace('urn:', '').replace('uuid:', '') + hex = hex.strip('{}').replace('-', '') + if len(hex) != 32: + raise ValueError('badly formed hexadecimal UUID string') + int = long(hex, 16) + if bytes is not None: + if len(bytes) != 16: + raise ValueError('bytes is not a 16-char string') + int = long(('%02x'*16) % tuple(map(ord, bytes)), 16) + if fields is not None: + if len(fields) != 6: + raise ValueError('fields is not a 6-tuple') + (time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node) = fields + if not 0 <= time_low < 1<<32L: + raise ValueError('field 1 out of range (need a 32-bit value)') + if not 0 <= time_mid < 1<<16L: + raise ValueError('field 2 out of range (need a 16-bit value)') + if not 0 <= time_hi_version < 1<<16L: + raise ValueError('field 3 out of range (need a 16-bit value)') + if not 0 <= clock_seq_hi_variant < 1<<8L: + raise ValueError('field 4 out of range (need an 8-bit value)') + if not 0 <= clock_seq_low < 1<<8L: + raise ValueError('field 5 out of range (need an 8-bit value)') + if not 0 <= node < 1<<48L: + raise ValueError('field 6 out of range (need a 48-bit value)') + clock_seq = (clock_seq_hi_variant << 8L) | clock_seq_low + int = ((time_low << 96L) | (time_mid << 80L) | + (time_hi_version << 64L) | (clock_seq << 48L) | node) + if int is not None: + if not 0 <= int < 1<<128L: + raise ValueError('int is out of range (need a 128-bit value)') + if version is not None: + if not 1 <= version <= 5: + raise ValueError('illegal version number') + # Set the variant to RFC 4122. + int &= ~(0xc000 << 48L) + int |= 0x8000 << 48L + # Set the version number. + int &= ~(0xf000 << 64L) + int |= version << 76L + self.__dict__['int'] = int + + def __cmp__(self, other): + if isinstance(other, UUID): + return cmp(self.int, other.int) + return NotImplemented + + def __hash__(self): + return hash(self.int) + + def __int__(self): + return self.int + + def __repr__(self): + return 'UUID(%r)' % str(self) + + def __setattr__(self, name, value): + raise TypeError('UUID objects are immutable') + + def __str__(self): + hex = '%032x' % self.int + return '%s-%s-%s-%s-%s' % ( + hex[:8], hex[8:12], hex[12:16], hex[16:20], hex[20:]) + + def get_bytes(self): + bytes = '' + for shift in range(0, 128, 8): + bytes = chr((self.int >> shift) & 0xff) + bytes + return bytes + + bytes = property(get_bytes) + + def get_fields(self): + return (self.time_low, self.time_mid, self.time_hi_version, + self.clock_seq_hi_variant, self.clock_seq_low, self.node) + + fields = property(get_fields) + + def get_time_low(self): + return self.int >> 96L + + time_low = property(get_time_low) + + def get_time_mid(self): + return (self.int >> 80L) & 0xffff + + time_mid = property(get_time_mid) + + def get_time_hi_version(self): + return (self.int >> 64L) & 0xffff + + time_hi_version = property(get_time_hi_version) + + def get_clock_seq_hi_variant(self): + return (self.int >> 56L) & 0xff + + clock_seq_hi_variant = property(get_clock_seq_hi_variant) + + def get_clock_seq_low(self): + return (self.int >> 48L) & 0xff + + clock_seq_low = property(get_clock_seq_low) + + def get_time(self): + return (((self.time_hi_version & 0x0fffL) << 48L) | + (self.time_mid << 32L) | self.time_low) + + time = property(get_time) + + def get_clock_seq(self): + return (((self.clock_seq_hi_variant & 0x3fL) << 8L) | + self.clock_seq_low) + + clock_seq = property(get_clock_seq) + + def get_node(self): + return self.int & 0xffffffffffff + + node = property(get_node) + + def get_hex(self): + return '%032x' % self.int + + hex = property(get_hex) + + def get_urn(self): + return 'urn:uuid:' + str(self) + + urn = property(get_urn) + + def get_variant(self): + if not self.int & (0x8000 << 48L): + return RESERVED_NCS + elif not self.int & (0x4000 << 48L): + return RFC_4122 + elif not self.int & (0x2000 << 48L): + return RESERVED_MICROSOFT + else: + return RESERVED_FUTURE + + variant = property(get_variant) + + def get_version(self): + # The version bits are only meaningful for RFC 4122 UUIDs. + if self.variant == RFC_4122: + return int((self.int >> 76L) & 0xf) + + version = property(get_version) + +def _ifconfig_getnode(): + """Get the hardware address on Unix by running ifconfig.""" + import os + for dir in ['', '/sbin/', '/usr/sbin']: + try: + pipe = os.popen(os.path.join(dir, 'ifconfig')) + except IOError: + continue + for line in pipe: + words = line.lower().split() + for i in range(len(words)): + if words[i] in ['hwaddr', 'ether']: + return int(words[i + 1].replace(':', ''), 16) + +def _ipconfig_getnode(): + """Get the hardware address on Windows by running ipconfig.exe.""" + import os, re + dirs = ['', r'c:\windows\system32', r'c:\winnt\system32'] + try: + import ctypes + buffer = ctypes.create_string_buffer(300) + ctypes.windll.kernel32.GetSystemDirectoryA(buffer, 300) + dirs.insert(0, buffer.value.decode('mbcs')) + except: + pass + for dir in dirs: + try: + pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all') + except IOError: + continue + for line in pipe: + value = line.split(':')[-1].strip().lower() + if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): + return int(value.replace('-', ''), 16) + +def _netbios_getnode(): + """Get the hardware address on Windows using NetBIOS calls. + See http://support.microsoft.com/kb/118623 for details.""" + import win32wnet, netbios + ncb = netbios.NCB() + ncb.Command = netbios.NCBENUM + ncb.Buffer = adapters = netbios.LANA_ENUM() + adapters._pack() + if win32wnet.Netbios(ncb) != 0: + return + adapters._unpack() + for i in range(adapters.length): + ncb.Reset() + ncb.Command = netbios.NCBRESET + ncb.Lana_num = ord(adapters.lana[i]) + if win32wnet.Netbios(ncb) != 0: + continue + ncb.Reset() + ncb.Command = netbios.NCBASTAT + ncb.Lana_num = ord(adapters.lana[i]) + ncb.Callname = '*'.ljust(16) + ncb.Buffer = status = netbios.ADAPTER_STATUS() + if win32wnet.Netbios(ncb) != 0: + continue + status._unpack() + bytes = map(ord, status.adapter_address) + return ((bytes[0]<<40L) + (bytes[1]<<32L) + (bytes[2]<<24L) + + (bytes[3]<<16L) + (bytes[4]<<8L) + bytes[5]) + +# Thanks to Thomas Heller for ctypes and for his help with its use here. + +# If ctypes is available, use it to find system routines for UUID generation. +_uuid_generate_random = _uuid_generate_time = _UuidCreate = None +try: + import ctypes, ctypes.util + _buffer = ctypes.create_string_buffer(16) + + # The uuid_generate_* routines are provided by libuuid on at least + # Linux and FreeBSD, and provided by libc on Mac OS X. + for libname in ['uuid', 'c']: + try: + lib = ctypes.CDLL(ctypes.util.find_library(libname)) + except: + continue + if hasattr(lib, 'uuid_generate_random'): + _uuid_generate_random = lib.uuid_generate_random + if hasattr(lib, 'uuid_generate_time'): + _uuid_generate_time = lib.uuid_generate_time + + # On Windows prior to 2000, UuidCreate gives a UUID containing the + # hardware address. On Windows 2000 and later, UuidCreate makes a + # random UUID and UuidCreateSequential gives a UUID containing the + # hardware address. These routines are provided by the RPC runtime. + try: + lib = ctypes.windll.rpcrt4 + except: + lib = None + _UuidCreate = getattr(lib, 'UuidCreateSequential', + getattr(lib, 'UuidCreate', None)) +except: + pass + +def _unixdll_getnode(): + """Get the hardware address on Unix using ctypes.""" + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw).node + +def _windll_getnode(): + """Get the hardware address on Windows using ctypes.""" + if _UuidCreate(_buffer) == 0: + return UUID(bytes=_buffer.raw).node + +def _random_getnode(): + """Get a random node ID, with eighth bit set as suggested by RFC 4122.""" + import random + return random.randrange(0, 1<<48L) | 0x010000000000L + +_node = None + +def getnode(): + """Get the hardware address as a 48-bit integer. The first time this + runs, it may launch a separate program, which could be quite slow. If + all attempts to obtain the hardware address fail, we choose a random + 48-bit number with its eighth bit set to 1 as recommended in RFC 4122.""" + + global _node + if _node is not None: + return _node + + import sys + if sys.platform == 'win32': + getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] + else: + getters = [_unixdll_getnode, _ifconfig_getnode] + + for getter in getters + [_random_getnode]: + try: + _node = getter() + except: + continue + if _node is not None: + return _node + +def uuid1(node=None, clock_seq=None): + """Generate a UUID from a host ID, sequence number, and the current time. + If 'node' is not given, getnode() is used to obtain the hardware + address. If 'clock_seq' is given, it is used as the sequence number; + otherwise a random 14-bit sequence number is chosen.""" + + # When the system provides a version-1 UUID generator, use it (but don't + # use UuidCreate here because its UUIDs don't conform to RFC 4122). + if _uuid_generate_time and node is clock_seq is None: + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw) + + import time + nanoseconds = int(time.time() * 1e9) + # 0x01b21dd213814000 is the number of 100-ns intervals between the + # UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00. + timestamp = int(nanoseconds/100) + 0x01b21dd213814000L + if clock_seq is None: + import random + clock_seq = random.randrange(1<<14L) # instead of stable storage + time_low = timestamp & 0xffffffffL + time_mid = (timestamp >> 32L) & 0xffffL + time_hi_version = (timestamp >> 48L) & 0x0fffL + clock_seq_low = clock_seq & 0xffL + clock_seq_hi_variant = (clock_seq >> 8L) & 0x3fL + if node is None: + node = getnode() + return UUID(fields=(time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node), version=1) + +def uuid3(namespace, name): + """Generate a UUID from the MD5 hash of a namespace UUID and a name.""" + import md5 + hash = md5.md5(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=3) + +def uuid4(): + """Generate a random UUID.""" + + # When the system provides a version-4 UUID generator, use it. + if _uuid_generate_random: + _uuid_generate_random(_buffer) + return UUID(bytes=_buffer.raw) + + # Otherwise, get randomness from urandom or the 'random' module. + try: + import os + return UUID(bytes=os.urandom(16), version=4) + except: + import random + bytes = [chr(random.randrange(256)) for i in range(16)] + return UUID(bytes=bytes, version=4) + +def uuid5(namespace, name): + """Generate a UUID from the SHA-1 hash of a namespace UUID and a name.""" + import sha + hash = sha.sha(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=5) + +# The following standard UUIDs are for use with uuid3() or uuid5(). + +NAMESPACE_DNS = UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_URL = UUID('6ba7b811-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_OID = UUID('6ba7b812-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_X500 = UUID('6ba7b814-9dad-11d1-80b4-00c04fd430c8') From python-checkins at python.org Tue Jun 13 02:30:03 2006 From: python-checkins at python.org (tim.peters) Date: Tue, 13 Jun 2006 02:30:03 +0200 (CEST) Subject: [Python-checkins] r46902 - in python/trunk/Lib: test/test_uuid.py uuid.py Message-ID: <20060613003003.2F8CD1E4002@bag.python.org> Author: tim.peters Date: Tue Jun 13 02:30:01 2006 New Revision: 46902 Modified: python/trunk/Lib/test/test_uuid.py python/trunk/Lib/uuid.py Log: Whitespace normalization. Modified: python/trunk/Lib/test/test_uuid.py ============================================================================== --- python/trunk/Lib/test/test_uuid.py (original) +++ python/trunk/Lib/test/test_uuid.py Tue Jun 13 02:30:01 2006 @@ -1,396 +1,396 @@ -from unittest import TestCase, main -import uuid - -def importable(name): - try: - __import__(name) - return True - except: - return False - -class TestUUID(TestCase): - last_node = None - - def test_UUID(self): - equal = self.assertEqual - ascending = [] - for (string, curly, hex, bytes, fields, integer, urn, - time, clock_seq, variant, version) in [ - ('00000000-0000-0000-0000-000000000000', - '{00000000-0000-0000-0000-000000000000}', - '00000000000000000000000000000000', - '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', - (0, 0, 0, 0, 0, 0), - 0, - 'urn:uuid:00000000-0000-0000-0000-000000000000', - 0, 0, uuid.RESERVED_NCS, None), - ('00010203-0405-0607-0809-0a0b0c0d0e0f', - '{00010203-0405-0607-0809-0a0b0c0d0e0f}', - '000102030405060708090a0b0c0d0e0f', - '\0\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\x0d\x0e\x0f', - (0x00010203L, 0x0405, 0x0607, 8, 9, 0x0a0b0c0d0e0fL), - 0x000102030405060708090a0b0c0d0e0fL, - 'urn:uuid:00010203-0405-0607-0809-0a0b0c0d0e0f', - 0x607040500010203L, 0x809, uuid.RESERVED_NCS, None), - ('02d9e6d5-9467-382e-8f9b-9300a64ac3cd', - '{02d9e6d5-9467-382e-8f9b-9300a64ac3cd}', - '02d9e6d59467382e8f9b9300a64ac3cd', - '\x02\xd9\xe6\xd5\x94\x67\x38\x2e\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', - (0x02d9e6d5L, 0x9467, 0x382e, 0x8f, 0x9b, 0x9300a64ac3cdL), - 0x02d9e6d59467382e8f9b9300a64ac3cdL, - 'urn:uuid:02d9e6d5-9467-382e-8f9b-9300a64ac3cd', - 0x82e946702d9e6d5L, 0xf9b, uuid.RFC_4122, 3), - ('12345678-1234-5678-1234-567812345678', - '{12345678-1234-5678-1234-567812345678}', - '12345678123456781234567812345678', - '\x12\x34\x56\x78'*4, - (0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678), - 0x12345678123456781234567812345678, - 'urn:uuid:12345678-1234-5678-1234-567812345678', - 0x678123412345678L, 0x1234, uuid.RESERVED_NCS, None), - ('6ba7b810-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b810-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8109dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x10\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b810L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8109dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b810-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b810L, 0xb4, uuid.RFC_4122, 1), - ('6ba7b811-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b811-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8119dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x11\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b811L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8119dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b811-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b811L, 0xb4, uuid.RFC_4122, 1), - ('6ba7b812-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b812-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8129dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x12\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b812L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8129dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b812-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b812L, 0xb4, uuid.RFC_4122, 1), - ('6ba7b814-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b814-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8149dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x14\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b814L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8149dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b814-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b814L, 0xb4, uuid.RFC_4122, 1), - ('7d444840-9dc0-11d1-b245-5ffdce74fad2', - '{7d444840-9dc0-11d1-b245-5ffdce74fad2}', - '7d4448409dc011d1b2455ffdce74fad2', - '\x7d\x44\x48\x40\x9d\xc0\x11\xd1\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', - (0x7d444840L, 0x9dc0, 0x11d1, 0xb2, 0x45, 0x5ffdce74fad2L), - 0x7d4448409dc011d1b2455ffdce74fad2L, - 'urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2', - 0x1d19dc07d444840L, 0x3245, uuid.RFC_4122, 1), - ('e902893a-9d22-3c7e-a7b8-d6e313b71d9f', - '{e902893a-9d22-3c7e-a7b8-d6e313b71d9f}', - 'e902893a9d223c7ea7b8d6e313b71d9f', - '\xe9\x02\x89\x3a\x9d\x22\x3c\x7e\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', - (0xe902893aL, 0x9d22, 0x3c7e, 0xa7, 0xb8, 0xd6e313b71d9fL), - 0xe902893a9d223c7ea7b8d6e313b71d9fL, - 'urn:uuid:e902893a-9d22-3c7e-a7b8-d6e313b71d9f', - 0xc7e9d22e902893aL, 0x27b8, uuid.RFC_4122, 3), - ('eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', - '{eb424026-6f54-4ef8-a4d0-bb658a1fc6cf}', - 'eb4240266f544ef8a4d0bb658a1fc6cf', - '\xeb\x42\x40\x26\x6f\x54\x4e\xf8\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', - (0xeb424026L, 0x6f54, 0x4ef8, 0xa4, 0xd0, 0xbb658a1fc6cfL), - 0xeb4240266f544ef8a4d0bb658a1fc6cfL, - 'urn:uuid:eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', - 0xef86f54eb424026L, 0x24d0, uuid.RFC_4122, 4), - ('f81d4fae-7dec-11d0-a765-00a0c91e6bf6', - '{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}', - 'f81d4fae7dec11d0a76500a0c91e6bf6', - '\xf8\x1d\x4f\xae\x7d\xec\x11\xd0\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', - (0xf81d4faeL, 0x7dec, 0x11d0, 0xa7, 0x65, 0x00a0c91e6bf6L), - 0xf81d4fae7dec11d0a76500a0c91e6bf6L, - 'urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6', - 0x1d07decf81d4faeL, 0x2765, uuid.RFC_4122, 1), - ('fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', - '{fffefdfc-fffe-fffe-fffe-fffefdfcfbfa}', - 'fffefdfcfffefffefffefffefdfcfbfa', - '\xff\xfe\xfd\xfc\xff\xfe\xff\xfe\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', - (0xfffefdfcL, 0xfffe, 0xfffe, 0xff, 0xfe, 0xfffefdfcfbfaL), - 0xfffefdfcfffefffefffefffefdfcfbfaL, - 'urn:uuid:fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', - 0xffefffefffefdfcL, 0x3ffe, uuid.RESERVED_FUTURE, None), - ('ffffffff-ffff-ffff-ffff-ffffffffffff', - '{ffffffff-ffff-ffff-ffff-ffffffffffff}', - 'ffffffffffffffffffffffffffffffff', - '\xff'*16, - (0xffffffffL, 0xffffL, 0xffffL, 0xff, 0xff, 0xffffffffffffL), - 0xffffffffffffffffffffffffffffffffL, - 'urn:uuid:ffffffff-ffff-ffff-ffff-ffffffffffff', - 0xfffffffffffffffL, 0x3fff, uuid.RESERVED_FUTURE, None), - ]: - equivalents = [] - # Construct each UUID in several different ways. - for u in [uuid.UUID(string), uuid.UUID(curly), uuid.UUID(hex), - uuid.UUID(bytes=bytes), uuid.UUID(fields=fields), - uuid.UUID(int=integer), uuid.UUID(urn)]: - # Test all conversions and properties of the UUID object. - equal(str(u), string) - equal(int(u), integer) - equal(u.bytes, bytes) - equal(u.fields, fields) - equal(u.time_low, fields[0]) - equal(u.time_mid, fields[1]) - equal(u.time_hi_version, fields[2]) - equal(u.clock_seq_hi_variant, fields[3]) - equal(u.clock_seq_low, fields[4]) - equal(u.node, fields[5]) - equal(u.hex, hex) - equal(u.int, integer) - equal(u.urn, urn) - equal(u.time, time) - equal(u.clock_seq, clock_seq) - equal(u.variant, variant) - equal(u.version, version) - equivalents.append(u) - - # Different construction methods should give the same UUID. - for u in equivalents: - for v in equivalents: - equal(u, v) - ascending.append(u) - - # Test comparison of UUIDs. - for i in range(len(ascending)): - for j in range(len(ascending)): - equal(cmp(i, j), cmp(ascending[i], ascending[j])) - - # Test sorting of UUIDs (above list is in ascending order). - resorted = ascending[:] - resorted.reverse() - resorted.sort() - equal(ascending, resorted) - - def test_exceptions(self): - badvalue = lambda f: self.assertRaises(ValueError, f) - badtype = lambda f: self.assertRaises(TypeError, f) - - # Badly formed hex strings. - badvalue(lambda: uuid.UUID('')) - badvalue(lambda: uuid.UUID('abc')) - badvalue(lambda: uuid.UUID('1234567812345678123456781234567')) - badvalue(lambda: uuid.UUID('123456781234567812345678123456789')) - badvalue(lambda: uuid.UUID('123456781234567812345678z2345678')) - - # Badly formed bytes. - badvalue(lambda: uuid.UUID(bytes='abc')) - badvalue(lambda: uuid.UUID(bytes='\0'*15)) - badvalue(lambda: uuid.UUID(bytes='\0'*17)) - - # Badly formed fields. - badvalue(lambda: uuid.UUID(fields=(1,))) - badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5))) - badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5, 6, 7))) - - # Field values out of range. - badvalue(lambda: uuid.UUID(fields=(-1, 0, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0x100000000L, 0, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, -1, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0x10000L, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, -1, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0x10000L, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, -1, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0x100L, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, -1, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0x100L, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, -1))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, 0x1000000000000L))) - - # Version number out of range. - badvalue(lambda: uuid.UUID('00'*16, version=0)) - badvalue(lambda: uuid.UUID('00'*16, version=6)) - - # Integer value out of range. - badvalue(lambda: uuid.UUID(int=-1)) - badvalue(lambda: uuid.UUID(int=1<<128L)) - - # Must supply exactly one of hex, bytes, fields, int. - h, b, f, i = '00'*16, '\0'*16, (0, 0, 0, 0, 0, 0), 0 - uuid.UUID(h) - uuid.UUID(hex=h) - uuid.UUID(bytes=b) - uuid.UUID(fields=f) - uuid.UUID(int=i) - - # Wrong number of arguments (positional). - badtype(lambda: uuid.UUID()) - badtype(lambda: uuid.UUID(h, b)) - badtype(lambda: uuid.UUID(h, b, f)) - badtype(lambda: uuid.UUID(h, b, f, i)) - - # Duplicate arguments (named). - badtype(lambda: uuid.UUID(hex=h, bytes=b)) - badtype(lambda: uuid.UUID(hex=h, fields=f)) - badtype(lambda: uuid.UUID(hex=h, int=i)) - badtype(lambda: uuid.UUID(bytes=b, fields=f)) - badtype(lambda: uuid.UUID(bytes=b, int=i)) - badtype(lambda: uuid.UUID(fields=f, int=i)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(hex=h, fields=f, int=i)) - badtype(lambda: uuid.UUID(bytes=b, int=i, fields=f)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i, fields=f)) - - # Duplicate arguments (positional and named). - badtype(lambda: uuid.UUID(h, hex=h)) - badtype(lambda: uuid.UUID(h, bytes=b)) - badtype(lambda: uuid.UUID(h, fields=f)) - badtype(lambda: uuid.UUID(h, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b)) - badtype(lambda: uuid.UUID(h, hex=h, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, int=i)) - badtype(lambda: uuid.UUID(h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(h, fields=f, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, fields=f, int=i)) - badtype(lambda: uuid.UUID(h, bytes=b, int=i, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i, fields=f)) - - # Immutability. - u = uuid.UUID(h) - badtype(lambda: setattr(u, 'hex', h)) - badtype(lambda: setattr(u, 'bytes', b)) - badtype(lambda: setattr(u, 'fields', f)) - badtype(lambda: setattr(u, 'int', i)) - - def check_node(self, node, source=''): - individual_group_bit = (node >> 40L) & 1 - universal_local_bit = (node >> 40L) & 2 - message = "%012x doesn't look like a real MAC address" % node - self.assertEqual(individual_group_bit, 0, message) - self.assertEqual(universal_local_bit, 0, message) - self.assertNotEqual(node, 0, message) - self.assertNotEqual(node, 0xffffffffffffL, message) - self.assert_(0 <= node, message) - self.assert_(node < 1<<48L, message) - - import sys - if source: - sys.stderr.write('(%s: %012x)' % (source, node)) - if TestUUID.last_node: - self.assertEqual(TestUUID.last_node, node, 'inconsistent node IDs') - else: - TestUUID.last_node = node - - def test_ifconfig_getnode(self): - import os - if os.name == 'posix': - self.check_node(uuid._ifconfig_getnode(), 'ifconfig') - - def test_ipconfig_getnode(self): - import os - if os.name == 'nt': - self.check_node(uuid._ipconfig_getnode(), 'ipconfig') - - def test_netbios_getnode(self): - if importable('win32wnet') and importable('netbios'): - self.check_node(uuid._netbios_getnode(), 'netbios') - - def test_random_getnode(self): - node = uuid._random_getnode() - self.assert_(0 <= node) - self.assert_(node < 1<<48L) - - def test_unixdll_getnode(self): - import os - if importable('ctypes') and os.name == 'posix': - self.check_node(uuid._unixdll_getnode(), 'unixdll') - - def test_windll_getnode(self): - import os - if importable('ctypes') and os.name == 'nt': - self.check_node(uuid._windll_getnode(), 'windll') - - def test_getnode(self): - self.check_node(uuid.getnode()) - - # Test it again to ensure consistency. - self.check_node(uuid.getnode()) - - def test_uuid1(self): - equal = self.assertEqual - - # Make sure uuid4() generates UUIDs that are actually version 1. - for u in [uuid.uuid1() for i in range(10)]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 1) - - # Make sure the supplied node ID appears in the UUID. - u = uuid.uuid1(0) - equal(u.node, 0) - u = uuid.uuid1(0x123456789abc) - equal(u.node, 0x123456789abc) - u = uuid.uuid1(0xffffffffffff) - equal(u.node, 0xffffffffffff) - - # Make sure the supplied clock sequence appears in the UUID. - u = uuid.uuid1(0x123456789abc, 0) - equal(u.node, 0x123456789abc) - equal(((u.clock_seq_hi_variant & 0x3f) << 8) | u.clock_seq_low, 0) - u = uuid.uuid1(0x123456789abc, 0x1234) - equal(u.node, 0x123456789abc) - equal(((u.clock_seq_hi_variant & 0x3f) << 8) | - u.clock_seq_low, 0x1234) - u = uuid.uuid1(0x123456789abc, 0x3fff) - equal(u.node, 0x123456789abc) - equal(((u.clock_seq_hi_variant & 0x3f) << 8) | - u.clock_seq_low, 0x3fff) - - def test_uuid3(self): - equal = self.assertEqual - - # Test some known version-3 UUIDs. - for u, v in [(uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org'), - '6fa459ea-ee8a-3ca4-894e-db77e160355e'), - (uuid.uuid3(uuid.NAMESPACE_URL, 'http://python.org/'), - '9fe8e8c4-aaa8-32a9-a55c-4535a88b748d'), - (uuid.uuid3(uuid.NAMESPACE_OID, '1.3.6.1'), - 'dd1a1cef-13d5-368a-ad82-eca71acd4cd1'), - (uuid.uuid3(uuid.NAMESPACE_X500, 'c=ca'), - '658d3002-db6b-3040-a1d1-8ddd7d189a4d'), - ]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 3) - equal(u, uuid.UUID(v)) - equal(str(u), v) - - def test_uuid4(self): - equal = self.assertEqual - - # Make sure uuid4() generates UUIDs that are actually version 4. - for u in [uuid.uuid4() for i in range(10)]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 4) - - def test_uuid5(self): - equal = self.assertEqual - - # Test some known version-5 UUIDs. - for u, v in [(uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org'), - '886313e1-3b8a-5372-9b90-0c9aee199e5d'), - (uuid.uuid5(uuid.NAMESPACE_URL, 'http://python.org/'), - '4c565f0d-3f5a-5890-b41b-20cf47701c5e'), - (uuid.uuid5(uuid.NAMESPACE_OID, '1.3.6.1'), - '1447fa61-5277-5fef-a9b3-fbc6e44f4af3'), - (uuid.uuid5(uuid.NAMESPACE_X500, 'c=ca'), - 'cc957dd1-a972-5349-98cd-874190002798'), - ]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 5) - equal(u, uuid.UUID(v)) - equal(str(u), v) - -if __name__ == '__main__': - main() +from unittest import TestCase, main +import uuid + +def importable(name): + try: + __import__(name) + return True + except: + return False + +class TestUUID(TestCase): + last_node = None + + def test_UUID(self): + equal = self.assertEqual + ascending = [] + for (string, curly, hex, bytes, fields, integer, urn, + time, clock_seq, variant, version) in [ + ('00000000-0000-0000-0000-000000000000', + '{00000000-0000-0000-0000-000000000000}', + '00000000000000000000000000000000', + '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', + (0, 0, 0, 0, 0, 0), + 0, + 'urn:uuid:00000000-0000-0000-0000-000000000000', + 0, 0, uuid.RESERVED_NCS, None), + ('00010203-0405-0607-0809-0a0b0c0d0e0f', + '{00010203-0405-0607-0809-0a0b0c0d0e0f}', + '000102030405060708090a0b0c0d0e0f', + '\0\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\x0d\x0e\x0f', + (0x00010203L, 0x0405, 0x0607, 8, 9, 0x0a0b0c0d0e0fL), + 0x000102030405060708090a0b0c0d0e0fL, + 'urn:uuid:00010203-0405-0607-0809-0a0b0c0d0e0f', + 0x607040500010203L, 0x809, uuid.RESERVED_NCS, None), + ('02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + '{02d9e6d5-9467-382e-8f9b-9300a64ac3cd}', + '02d9e6d59467382e8f9b9300a64ac3cd', + '\x02\xd9\xe6\xd5\x94\x67\x38\x2e\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', + (0x02d9e6d5L, 0x9467, 0x382e, 0x8f, 0x9b, 0x9300a64ac3cdL), + 0x02d9e6d59467382e8f9b9300a64ac3cdL, + 'urn:uuid:02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + 0x82e946702d9e6d5L, 0xf9b, uuid.RFC_4122, 3), + ('12345678-1234-5678-1234-567812345678', + '{12345678-1234-5678-1234-567812345678}', + '12345678123456781234567812345678', + '\x12\x34\x56\x78'*4, + (0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678), + 0x12345678123456781234567812345678, + 'urn:uuid:12345678-1234-5678-1234-567812345678', + 0x678123412345678L, 0x1234, uuid.RESERVED_NCS, None), + ('6ba7b810-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b810-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8109dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x10\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b810L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8109dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b810-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b810L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b811-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b811-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8119dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x11\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b811L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8119dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b811-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b811L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b812-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b812-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8129dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x12\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b812L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8129dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b812-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b812L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b814-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b814-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8149dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x14\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b814L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8149dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b814-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b814L, 0xb4, uuid.RFC_4122, 1), + ('7d444840-9dc0-11d1-b245-5ffdce74fad2', + '{7d444840-9dc0-11d1-b245-5ffdce74fad2}', + '7d4448409dc011d1b2455ffdce74fad2', + '\x7d\x44\x48\x40\x9d\xc0\x11\xd1\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', + (0x7d444840L, 0x9dc0, 0x11d1, 0xb2, 0x45, 0x5ffdce74fad2L), + 0x7d4448409dc011d1b2455ffdce74fad2L, + 'urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2', + 0x1d19dc07d444840L, 0x3245, uuid.RFC_4122, 1), + ('e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + '{e902893a-9d22-3c7e-a7b8-d6e313b71d9f}', + 'e902893a9d223c7ea7b8d6e313b71d9f', + '\xe9\x02\x89\x3a\x9d\x22\x3c\x7e\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', + (0xe902893aL, 0x9d22, 0x3c7e, 0xa7, 0xb8, 0xd6e313b71d9fL), + 0xe902893a9d223c7ea7b8d6e313b71d9fL, + 'urn:uuid:e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + 0xc7e9d22e902893aL, 0x27b8, uuid.RFC_4122, 3), + ('eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + '{eb424026-6f54-4ef8-a4d0-bb658a1fc6cf}', + 'eb4240266f544ef8a4d0bb658a1fc6cf', + '\xeb\x42\x40\x26\x6f\x54\x4e\xf8\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', + (0xeb424026L, 0x6f54, 0x4ef8, 0xa4, 0xd0, 0xbb658a1fc6cfL), + 0xeb4240266f544ef8a4d0bb658a1fc6cfL, + 'urn:uuid:eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + 0xef86f54eb424026L, 0x24d0, uuid.RFC_4122, 4), + ('f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + '{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}', + 'f81d4fae7dec11d0a76500a0c91e6bf6', + '\xf8\x1d\x4f\xae\x7d\xec\x11\xd0\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', + (0xf81d4faeL, 0x7dec, 0x11d0, 0xa7, 0x65, 0x00a0c91e6bf6L), + 0xf81d4fae7dec11d0a76500a0c91e6bf6L, + 'urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + 0x1d07decf81d4faeL, 0x2765, uuid.RFC_4122, 1), + ('fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + '{fffefdfc-fffe-fffe-fffe-fffefdfcfbfa}', + 'fffefdfcfffefffefffefffefdfcfbfa', + '\xff\xfe\xfd\xfc\xff\xfe\xff\xfe\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', + (0xfffefdfcL, 0xfffe, 0xfffe, 0xff, 0xfe, 0xfffefdfcfbfaL), + 0xfffefdfcfffefffefffefffefdfcfbfaL, + 'urn:uuid:fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + 0xffefffefffefdfcL, 0x3ffe, uuid.RESERVED_FUTURE, None), + ('ffffffff-ffff-ffff-ffff-ffffffffffff', + '{ffffffff-ffff-ffff-ffff-ffffffffffff}', + 'ffffffffffffffffffffffffffffffff', + '\xff'*16, + (0xffffffffL, 0xffffL, 0xffffL, 0xff, 0xff, 0xffffffffffffL), + 0xffffffffffffffffffffffffffffffffL, + 'urn:uuid:ffffffff-ffff-ffff-ffff-ffffffffffff', + 0xfffffffffffffffL, 0x3fff, uuid.RESERVED_FUTURE, None), + ]: + equivalents = [] + # Construct each UUID in several different ways. + for u in [uuid.UUID(string), uuid.UUID(curly), uuid.UUID(hex), + uuid.UUID(bytes=bytes), uuid.UUID(fields=fields), + uuid.UUID(int=integer), uuid.UUID(urn)]: + # Test all conversions and properties of the UUID object. + equal(str(u), string) + equal(int(u), integer) + equal(u.bytes, bytes) + equal(u.fields, fields) + equal(u.time_low, fields[0]) + equal(u.time_mid, fields[1]) + equal(u.time_hi_version, fields[2]) + equal(u.clock_seq_hi_variant, fields[3]) + equal(u.clock_seq_low, fields[4]) + equal(u.node, fields[5]) + equal(u.hex, hex) + equal(u.int, integer) + equal(u.urn, urn) + equal(u.time, time) + equal(u.clock_seq, clock_seq) + equal(u.variant, variant) + equal(u.version, version) + equivalents.append(u) + + # Different construction methods should give the same UUID. + for u in equivalents: + for v in equivalents: + equal(u, v) + ascending.append(u) + + # Test comparison of UUIDs. + for i in range(len(ascending)): + for j in range(len(ascending)): + equal(cmp(i, j), cmp(ascending[i], ascending[j])) + + # Test sorting of UUIDs (above list is in ascending order). + resorted = ascending[:] + resorted.reverse() + resorted.sort() + equal(ascending, resorted) + + def test_exceptions(self): + badvalue = lambda f: self.assertRaises(ValueError, f) + badtype = lambda f: self.assertRaises(TypeError, f) + + # Badly formed hex strings. + badvalue(lambda: uuid.UUID('')) + badvalue(lambda: uuid.UUID('abc')) + badvalue(lambda: uuid.UUID('1234567812345678123456781234567')) + badvalue(lambda: uuid.UUID('123456781234567812345678123456789')) + badvalue(lambda: uuid.UUID('123456781234567812345678z2345678')) + + # Badly formed bytes. + badvalue(lambda: uuid.UUID(bytes='abc')) + badvalue(lambda: uuid.UUID(bytes='\0'*15)) + badvalue(lambda: uuid.UUID(bytes='\0'*17)) + + # Badly formed fields. + badvalue(lambda: uuid.UUID(fields=(1,))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5, 6, 7))) + + # Field values out of range. + badvalue(lambda: uuid.UUID(fields=(-1, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0x100000000L, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, -1, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0x10000L, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, -1, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0x10000L, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, -1, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0x100L, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, -1, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0x100L, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, -1))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, 0x1000000000000L))) + + # Version number out of range. + badvalue(lambda: uuid.UUID('00'*16, version=0)) + badvalue(lambda: uuid.UUID('00'*16, version=6)) + + # Integer value out of range. + badvalue(lambda: uuid.UUID(int=-1)) + badvalue(lambda: uuid.UUID(int=1<<128L)) + + # Must supply exactly one of hex, bytes, fields, int. + h, b, f, i = '00'*16, '\0'*16, (0, 0, 0, 0, 0, 0), 0 + uuid.UUID(h) + uuid.UUID(hex=h) + uuid.UUID(bytes=b) + uuid.UUID(fields=f) + uuid.UUID(int=i) + + # Wrong number of arguments (positional). + badtype(lambda: uuid.UUID()) + badtype(lambda: uuid.UUID(h, b)) + badtype(lambda: uuid.UUID(h, b, f)) + badtype(lambda: uuid.UUID(h, b, f, i)) + + # Duplicate arguments (named). + badtype(lambda: uuid.UUID(hex=h, bytes=b)) + badtype(lambda: uuid.UUID(hex=h, fields=f)) + badtype(lambda: uuid.UUID(hex=h, int=i)) + badtype(lambda: uuid.UUID(bytes=b, fields=f)) + badtype(lambda: uuid.UUID(bytes=b, int=i)) + badtype(lambda: uuid.UUID(fields=f, int=i)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i, fields=f)) + + # Duplicate arguments (positional and named). + badtype(lambda: uuid.UUID(h, hex=h)) + badtype(lambda: uuid.UUID(h, bytes=b)) + badtype(lambda: uuid.UUID(h, fields=f)) + badtype(lambda: uuid.UUID(h, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i, fields=f)) + + # Immutability. + u = uuid.UUID(h) + badtype(lambda: setattr(u, 'hex', h)) + badtype(lambda: setattr(u, 'bytes', b)) + badtype(lambda: setattr(u, 'fields', f)) + badtype(lambda: setattr(u, 'int', i)) + + def check_node(self, node, source=''): + individual_group_bit = (node >> 40L) & 1 + universal_local_bit = (node >> 40L) & 2 + message = "%012x doesn't look like a real MAC address" % node + self.assertEqual(individual_group_bit, 0, message) + self.assertEqual(universal_local_bit, 0, message) + self.assertNotEqual(node, 0, message) + self.assertNotEqual(node, 0xffffffffffffL, message) + self.assert_(0 <= node, message) + self.assert_(node < 1<<48L, message) + + import sys + if source: + sys.stderr.write('(%s: %012x)' % (source, node)) + if TestUUID.last_node: + self.assertEqual(TestUUID.last_node, node, 'inconsistent node IDs') + else: + TestUUID.last_node = node + + def test_ifconfig_getnode(self): + import os + if os.name == 'posix': + self.check_node(uuid._ifconfig_getnode(), 'ifconfig') + + def test_ipconfig_getnode(self): + import os + if os.name == 'nt': + self.check_node(uuid._ipconfig_getnode(), 'ipconfig') + + def test_netbios_getnode(self): + if importable('win32wnet') and importable('netbios'): + self.check_node(uuid._netbios_getnode(), 'netbios') + + def test_random_getnode(self): + node = uuid._random_getnode() + self.assert_(0 <= node) + self.assert_(node < 1<<48L) + + def test_unixdll_getnode(self): + import os + if importable('ctypes') and os.name == 'posix': + self.check_node(uuid._unixdll_getnode(), 'unixdll') + + def test_windll_getnode(self): + import os + if importable('ctypes') and os.name == 'nt': + self.check_node(uuid._windll_getnode(), 'windll') + + def test_getnode(self): + self.check_node(uuid.getnode()) + + # Test it again to ensure consistency. + self.check_node(uuid.getnode()) + + def test_uuid1(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 1. + for u in [uuid.uuid1() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 1) + + # Make sure the supplied node ID appears in the UUID. + u = uuid.uuid1(0) + equal(u.node, 0) + u = uuid.uuid1(0x123456789abc) + equal(u.node, 0x123456789abc) + u = uuid.uuid1(0xffffffffffff) + equal(u.node, 0xffffffffffff) + + # Make sure the supplied clock sequence appears in the UUID. + u = uuid.uuid1(0x123456789abc, 0) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | u.clock_seq_low, 0) + u = uuid.uuid1(0x123456789abc, 0x1234) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x1234) + u = uuid.uuid1(0x123456789abc, 0x3fff) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x3fff) + + def test_uuid3(self): + equal = self.assertEqual + + # Test some known version-3 UUIDs. + for u, v in [(uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org'), + '6fa459ea-ee8a-3ca4-894e-db77e160355e'), + (uuid.uuid3(uuid.NAMESPACE_URL, 'http://python.org/'), + '9fe8e8c4-aaa8-32a9-a55c-4535a88b748d'), + (uuid.uuid3(uuid.NAMESPACE_OID, '1.3.6.1'), + 'dd1a1cef-13d5-368a-ad82-eca71acd4cd1'), + (uuid.uuid3(uuid.NAMESPACE_X500, 'c=ca'), + '658d3002-db6b-3040-a1d1-8ddd7d189a4d'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 3) + equal(u, uuid.UUID(v)) + equal(str(u), v) + + def test_uuid4(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 4. + for u in [uuid.uuid4() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 4) + + def test_uuid5(self): + equal = self.assertEqual + + # Test some known version-5 UUIDs. + for u, v in [(uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org'), + '886313e1-3b8a-5372-9b90-0c9aee199e5d'), + (uuid.uuid5(uuid.NAMESPACE_URL, 'http://python.org/'), + '4c565f0d-3f5a-5890-b41b-20cf47701c5e'), + (uuid.uuid5(uuid.NAMESPACE_OID, '1.3.6.1'), + '1447fa61-5277-5fef-a9b3-fbc6e44f4af3'), + (uuid.uuid5(uuid.NAMESPACE_X500, 'c=ca'), + 'cc957dd1-a972-5349-98cd-874190002798'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 5) + equal(u, uuid.UUID(v)) + equal(str(u), v) + +if __name__ == '__main__': + main() Modified: python/trunk/Lib/uuid.py ============================================================================== --- python/trunk/Lib/uuid.py (original) +++ python/trunk/Lib/uuid.py Tue Jun 13 02:30:01 2006 @@ -1,477 +1,477 @@ -r"""UUID objects (universally unique identifiers) according to RFC 4122. - -This module provides immutable UUID objects (class UUID) and the functions -uuid1(), uuid3(), uuid4(), uuid5() for generating version 1, 3, 4, and 5 -UUIDs as specified in RFC 4122. - -If all you want is a unique ID, you should probably call uuid1() or uuid4(). -Note that uuid1() may compromise privacy since it creates a UUID containing -the computer's network address. uuid4() creates a random UUID. - -Typical usage: - - >>> import uuid - - # make a UUID based on the host ID and current time - >>> uuid.uuid1() - UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') - - # make a UUID using an MD5 hash of a namespace UUID and a name - >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') - UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') - - # make a random UUID - >>> uuid.uuid4() - UUID('16fd2706-8baf-433b-82eb-8c7fada847da') - - # make a UUID using a SHA-1 hash of a namespace UUID and a name - >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') - UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') - - # make a UUID from a string of hex digits (braces and hyphens ignored) - >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') - - # convert a UUID to a string of hex digits in standard form - >>> str(x) - '00010203-0405-0607-0809-0a0b0c0d0e0f' - - # get the raw 16 bytes of the UUID - >>> x.bytes - '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' - - # make a UUID from a 16-byte string - >>> uuid.UUID(bytes=x.bytes) - UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') - -This module works with Python 2.3 or higher.""" - -__author__ = 'Ka-Ping Yee ' -__date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') -__version__ = '$Revision: 1.30 $'.split()[1] - -RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, RESERVED_FUTURE = [ - 'reserved for NCS compatibility', 'specified in RFC 4122', - 'reserved for Microsoft compatibility', 'reserved for future definition'] - -class UUID(object): - """Instances of the UUID class represent UUIDs as specified in RFC 4122. - UUID objects are immutable, hashable, and usable as dictionary keys. - Converting a UUID to a string with str() yields something in the form - '12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts - four possible forms: a similar string of hexadecimal digits, or a - string of 16 raw bytes as an argument named 'bytes', or a tuple of - six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and - 48-bit values respectively) as an argument named 'fields', or a single - 128-bit integer as an argument named 'int'. - - UUIDs have these read-only attributes: - - bytes the UUID as a 16-byte string - - fields a tuple of the six integer fields of the UUID, - which are also available as six individual attributes - and two derived attributes: - - time_low the first 32 bits of the UUID - time_mid the next 16 bits of the UUID - time_hi_version the next 16 bits of the UUID - clock_seq_hi_variant the next 8 bits of the UUID - clock_seq_low the next 8 bits of the UUID - node the last 48 bits of the UUID - - time the 60-bit timestamp - clock_seq the 14-bit sequence number - - hex the UUID as a 32-character hexadecimal string - - int the UUID as a 128-bit integer - - urn the UUID as a URN as specified in RFC 4122 - - variant the UUID variant (one of the constants RESERVED_NCS, - RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE) - - version the UUID version number (1 through 5, meaningful only - when the variant is RFC_4122) - """ - - def __init__(self, hex=None, bytes=None, fields=None, int=None, - version=None): - r"""Create a UUID from either a string of 32 hexadecimal digits, - a string of 16 bytes as the 'bytes' argument, a tuple of six - integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, - 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as - the 'fields' argument, or a single 128-bit integer as the 'int' - argument. When a string of hex digits is given, curly braces, - hyphens, and a URN prefix are all optional. For example, these - expressions all yield the same UUID: - - UUID('{12345678-1234-5678-1234-567812345678}') - UUID('12345678123456781234567812345678') - UUID('urn:uuid:12345678-1234-5678-1234-567812345678') - UUID(bytes='\x12\x34\x56\x78'*4) - UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) - UUID(int=0x12345678123456781234567812345678) - - Exactly one of 'hex', 'bytes', 'fields', or 'int' must be given. - The 'version' argument is optional; if given, the resulting UUID - will have its variant and version number set according to RFC 4122, - overriding bits in the given 'hex', 'bytes', 'fields', or 'int'. - """ - - if [hex, bytes, fields, int].count(None) != 3: - raise TypeError('need just one of hex, bytes, fields, or int') - if hex is not None: - hex = hex.replace('urn:', '').replace('uuid:', '') - hex = hex.strip('{}').replace('-', '') - if len(hex) != 32: - raise ValueError('badly formed hexadecimal UUID string') - int = long(hex, 16) - if bytes is not None: - if len(bytes) != 16: - raise ValueError('bytes is not a 16-char string') - int = long(('%02x'*16) % tuple(map(ord, bytes)), 16) - if fields is not None: - if len(fields) != 6: - raise ValueError('fields is not a 6-tuple') - (time_low, time_mid, time_hi_version, - clock_seq_hi_variant, clock_seq_low, node) = fields - if not 0 <= time_low < 1<<32L: - raise ValueError('field 1 out of range (need a 32-bit value)') - if not 0 <= time_mid < 1<<16L: - raise ValueError('field 2 out of range (need a 16-bit value)') - if not 0 <= time_hi_version < 1<<16L: - raise ValueError('field 3 out of range (need a 16-bit value)') - if not 0 <= clock_seq_hi_variant < 1<<8L: - raise ValueError('field 4 out of range (need an 8-bit value)') - if not 0 <= clock_seq_low < 1<<8L: - raise ValueError('field 5 out of range (need an 8-bit value)') - if not 0 <= node < 1<<48L: - raise ValueError('field 6 out of range (need a 48-bit value)') - clock_seq = (clock_seq_hi_variant << 8L) | clock_seq_low - int = ((time_low << 96L) | (time_mid << 80L) | - (time_hi_version << 64L) | (clock_seq << 48L) | node) - if int is not None: - if not 0 <= int < 1<<128L: - raise ValueError('int is out of range (need a 128-bit value)') - if version is not None: - if not 1 <= version <= 5: - raise ValueError('illegal version number') - # Set the variant to RFC 4122. - int &= ~(0xc000 << 48L) - int |= 0x8000 << 48L - # Set the version number. - int &= ~(0xf000 << 64L) - int |= version << 76L - self.__dict__['int'] = int - - def __cmp__(self, other): - if isinstance(other, UUID): - return cmp(self.int, other.int) - return NotImplemented - - def __hash__(self): - return hash(self.int) - - def __int__(self): - return self.int - - def __repr__(self): - return 'UUID(%r)' % str(self) - - def __setattr__(self, name, value): - raise TypeError('UUID objects are immutable') - - def __str__(self): - hex = '%032x' % self.int - return '%s-%s-%s-%s-%s' % ( - hex[:8], hex[8:12], hex[12:16], hex[16:20], hex[20:]) - - def get_bytes(self): - bytes = '' - for shift in range(0, 128, 8): - bytes = chr((self.int >> shift) & 0xff) + bytes - return bytes - - bytes = property(get_bytes) - - def get_fields(self): - return (self.time_low, self.time_mid, self.time_hi_version, - self.clock_seq_hi_variant, self.clock_seq_low, self.node) - - fields = property(get_fields) - - def get_time_low(self): - return self.int >> 96L - - time_low = property(get_time_low) - - def get_time_mid(self): - return (self.int >> 80L) & 0xffff - - time_mid = property(get_time_mid) - - def get_time_hi_version(self): - return (self.int >> 64L) & 0xffff - - time_hi_version = property(get_time_hi_version) - - def get_clock_seq_hi_variant(self): - return (self.int >> 56L) & 0xff - - clock_seq_hi_variant = property(get_clock_seq_hi_variant) - - def get_clock_seq_low(self): - return (self.int >> 48L) & 0xff - - clock_seq_low = property(get_clock_seq_low) - - def get_time(self): - return (((self.time_hi_version & 0x0fffL) << 48L) | - (self.time_mid << 32L) | self.time_low) - - time = property(get_time) - - def get_clock_seq(self): - return (((self.clock_seq_hi_variant & 0x3fL) << 8L) | - self.clock_seq_low) - - clock_seq = property(get_clock_seq) - - def get_node(self): - return self.int & 0xffffffffffff - - node = property(get_node) - - def get_hex(self): - return '%032x' % self.int - - hex = property(get_hex) - - def get_urn(self): - return 'urn:uuid:' + str(self) - - urn = property(get_urn) - - def get_variant(self): - if not self.int & (0x8000 << 48L): - return RESERVED_NCS - elif not self.int & (0x4000 << 48L): - return RFC_4122 - elif not self.int & (0x2000 << 48L): - return RESERVED_MICROSOFT - else: - return RESERVED_FUTURE - - variant = property(get_variant) - - def get_version(self): - # The version bits are only meaningful for RFC 4122 UUIDs. - if self.variant == RFC_4122: - return int((self.int >> 76L) & 0xf) - - version = property(get_version) - -def _ifconfig_getnode(): - """Get the hardware address on Unix by running ifconfig.""" - import os - for dir in ['', '/sbin/', '/usr/sbin']: - try: - pipe = os.popen(os.path.join(dir, 'ifconfig')) - except IOError: - continue - for line in pipe: - words = line.lower().split() - for i in range(len(words)): - if words[i] in ['hwaddr', 'ether']: - return int(words[i + 1].replace(':', ''), 16) - -def _ipconfig_getnode(): - """Get the hardware address on Windows by running ipconfig.exe.""" - import os, re - dirs = ['', r'c:\windows\system32', r'c:\winnt\system32'] - try: - import ctypes - buffer = ctypes.create_string_buffer(300) - ctypes.windll.kernel32.GetSystemDirectoryA(buffer, 300) - dirs.insert(0, buffer.value.decode('mbcs')) - except: - pass - for dir in dirs: - try: - pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all') - except IOError: - continue - for line in pipe: - value = line.split(':')[-1].strip().lower() - if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): - return int(value.replace('-', ''), 16) - -def _netbios_getnode(): - """Get the hardware address on Windows using NetBIOS calls. - See http://support.microsoft.com/kb/118623 for details.""" - import win32wnet, netbios - ncb = netbios.NCB() - ncb.Command = netbios.NCBENUM - ncb.Buffer = adapters = netbios.LANA_ENUM() - adapters._pack() - if win32wnet.Netbios(ncb) != 0: - return - adapters._unpack() - for i in range(adapters.length): - ncb.Reset() - ncb.Command = netbios.NCBRESET - ncb.Lana_num = ord(adapters.lana[i]) - if win32wnet.Netbios(ncb) != 0: - continue - ncb.Reset() - ncb.Command = netbios.NCBASTAT - ncb.Lana_num = ord(adapters.lana[i]) - ncb.Callname = '*'.ljust(16) - ncb.Buffer = status = netbios.ADAPTER_STATUS() - if win32wnet.Netbios(ncb) != 0: - continue - status._unpack() - bytes = map(ord, status.adapter_address) - return ((bytes[0]<<40L) + (bytes[1]<<32L) + (bytes[2]<<24L) + - (bytes[3]<<16L) + (bytes[4]<<8L) + bytes[5]) - -# Thanks to Thomas Heller for ctypes and for his help with its use here. - -# If ctypes is available, use it to find system routines for UUID generation. -_uuid_generate_random = _uuid_generate_time = _UuidCreate = None -try: - import ctypes, ctypes.util - _buffer = ctypes.create_string_buffer(16) - - # The uuid_generate_* routines are provided by libuuid on at least - # Linux and FreeBSD, and provided by libc on Mac OS X. - for libname in ['uuid', 'c']: - try: - lib = ctypes.CDLL(ctypes.util.find_library(libname)) - except: - continue - if hasattr(lib, 'uuid_generate_random'): - _uuid_generate_random = lib.uuid_generate_random - if hasattr(lib, 'uuid_generate_time'): - _uuid_generate_time = lib.uuid_generate_time - - # On Windows prior to 2000, UuidCreate gives a UUID containing the - # hardware address. On Windows 2000 and later, UuidCreate makes a - # random UUID and UuidCreateSequential gives a UUID containing the - # hardware address. These routines are provided by the RPC runtime. - try: - lib = ctypes.windll.rpcrt4 - except: - lib = None - _UuidCreate = getattr(lib, 'UuidCreateSequential', - getattr(lib, 'UuidCreate', None)) -except: - pass - -def _unixdll_getnode(): - """Get the hardware address on Unix using ctypes.""" - _uuid_generate_time(_buffer) - return UUID(bytes=_buffer.raw).node - -def _windll_getnode(): - """Get the hardware address on Windows using ctypes.""" - if _UuidCreate(_buffer) == 0: - return UUID(bytes=_buffer.raw).node - -def _random_getnode(): - """Get a random node ID, with eighth bit set as suggested by RFC 4122.""" - import random - return random.randrange(0, 1<<48L) | 0x010000000000L - -_node = None - -def getnode(): - """Get the hardware address as a 48-bit integer. The first time this - runs, it may launch a separate program, which could be quite slow. If - all attempts to obtain the hardware address fail, we choose a random - 48-bit number with its eighth bit set to 1 as recommended in RFC 4122.""" - - global _node - if _node is not None: - return _node - - import sys - if sys.platform == 'win32': - getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] - else: - getters = [_unixdll_getnode, _ifconfig_getnode] - - for getter in getters + [_random_getnode]: - try: - _node = getter() - except: - continue - if _node is not None: - return _node - -def uuid1(node=None, clock_seq=None): - """Generate a UUID from a host ID, sequence number, and the current time. - If 'node' is not given, getnode() is used to obtain the hardware - address. If 'clock_seq' is given, it is used as the sequence number; - otherwise a random 14-bit sequence number is chosen.""" - - # When the system provides a version-1 UUID generator, use it (but don't - # use UuidCreate here because its UUIDs don't conform to RFC 4122). - if _uuid_generate_time and node is clock_seq is None: - _uuid_generate_time(_buffer) - return UUID(bytes=_buffer.raw) - - import time - nanoseconds = int(time.time() * 1e9) - # 0x01b21dd213814000 is the number of 100-ns intervals between the - # UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00. - timestamp = int(nanoseconds/100) + 0x01b21dd213814000L - if clock_seq is None: - import random - clock_seq = random.randrange(1<<14L) # instead of stable storage - time_low = timestamp & 0xffffffffL - time_mid = (timestamp >> 32L) & 0xffffL - time_hi_version = (timestamp >> 48L) & 0x0fffL - clock_seq_low = clock_seq & 0xffL - clock_seq_hi_variant = (clock_seq >> 8L) & 0x3fL - if node is None: - node = getnode() - return UUID(fields=(time_low, time_mid, time_hi_version, - clock_seq_hi_variant, clock_seq_low, node), version=1) - -def uuid3(namespace, name): - """Generate a UUID from the MD5 hash of a namespace UUID and a name.""" - import md5 - hash = md5.md5(namespace.bytes + name).digest() - return UUID(bytes=hash[:16], version=3) - -def uuid4(): - """Generate a random UUID.""" - - # When the system provides a version-4 UUID generator, use it. - if _uuid_generate_random: - _uuid_generate_random(_buffer) - return UUID(bytes=_buffer.raw) - - # Otherwise, get randomness from urandom or the 'random' module. - try: - import os - return UUID(bytes=os.urandom(16), version=4) - except: - import random - bytes = [chr(random.randrange(256)) for i in range(16)] - return UUID(bytes=bytes, version=4) - -def uuid5(namespace, name): - """Generate a UUID from the SHA-1 hash of a namespace UUID and a name.""" - import sha - hash = sha.sha(namespace.bytes + name).digest() - return UUID(bytes=hash[:16], version=5) - -# The following standard UUIDs are for use with uuid3() or uuid5(). - -NAMESPACE_DNS = UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8') -NAMESPACE_URL = UUID('6ba7b811-9dad-11d1-80b4-00c04fd430c8') -NAMESPACE_OID = UUID('6ba7b812-9dad-11d1-80b4-00c04fd430c8') -NAMESPACE_X500 = UUID('6ba7b814-9dad-11d1-80b4-00c04fd430c8') +r"""UUID objects (universally unique identifiers) according to RFC 4122. + +This module provides immutable UUID objects (class UUID) and the functions +uuid1(), uuid3(), uuid4(), uuid5() for generating version 1, 3, 4, and 5 +UUIDs as specified in RFC 4122. + +If all you want is a unique ID, you should probably call uuid1() or uuid4(). +Note that uuid1() may compromise privacy since it creates a UUID containing +the computer's network address. uuid4() creates a random UUID. + +Typical usage: + + >>> import uuid + + # make a UUID based on the host ID and current time + >>> uuid.uuid1() + UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + + # make a UUID using an MD5 hash of a namespace UUID and a name + >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') + UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + + # make a random UUID + >>> uuid.uuid4() + UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + + # make a UUID using a SHA-1 hash of a namespace UUID and a name + >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') + UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + + # make a UUID from a string of hex digits (braces and hyphens ignored) + >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') + + # convert a UUID to a string of hex digits in standard form + >>> str(x) + '00010203-0405-0607-0809-0a0b0c0d0e0f' + + # get the raw 16 bytes of the UUID + >>> x.bytes + '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + + # make a UUID from a 16-byte string + >>> uuid.UUID(bytes=x.bytes) + UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') + +This module works with Python 2.3 or higher.""" + +__author__ = 'Ka-Ping Yee ' +__date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') +__version__ = '$Revision: 1.30 $'.split()[1] + +RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, RESERVED_FUTURE = [ + 'reserved for NCS compatibility', 'specified in RFC 4122', + 'reserved for Microsoft compatibility', 'reserved for future definition'] + +class UUID(object): + """Instances of the UUID class represent UUIDs as specified in RFC 4122. + UUID objects are immutable, hashable, and usable as dictionary keys. + Converting a UUID to a string with str() yields something in the form + '12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts + four possible forms: a similar string of hexadecimal digits, or a + string of 16 raw bytes as an argument named 'bytes', or a tuple of + six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and + 48-bit values respectively) as an argument named 'fields', or a single + 128-bit integer as an argument named 'int'. + + UUIDs have these read-only attributes: + + bytes the UUID as a 16-byte string + + fields a tuple of the six integer fields of the UUID, + which are also available as six individual attributes + and two derived attributes: + + time_low the first 32 bits of the UUID + time_mid the next 16 bits of the UUID + time_hi_version the next 16 bits of the UUID + clock_seq_hi_variant the next 8 bits of the UUID + clock_seq_low the next 8 bits of the UUID + node the last 48 bits of the UUID + + time the 60-bit timestamp + clock_seq the 14-bit sequence number + + hex the UUID as a 32-character hexadecimal string + + int the UUID as a 128-bit integer + + urn the UUID as a URN as specified in RFC 4122 + + variant the UUID variant (one of the constants RESERVED_NCS, + RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE) + + version the UUID version number (1 through 5, meaningful only + when the variant is RFC_4122) + """ + + def __init__(self, hex=None, bytes=None, fields=None, int=None, + version=None): + r"""Create a UUID from either a string of 32 hexadecimal digits, + a string of 16 bytes as the 'bytes' argument, a tuple of six + integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, + 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as + the 'fields' argument, or a single 128-bit integer as the 'int' + argument. When a string of hex digits is given, curly braces, + hyphens, and a URN prefix are all optional. For example, these + expressions all yield the same UUID: + + UUID('{12345678-1234-5678-1234-567812345678}') + UUID('12345678123456781234567812345678') + UUID('urn:uuid:12345678-1234-5678-1234-567812345678') + UUID(bytes='\x12\x34\x56\x78'*4) + UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) + UUID(int=0x12345678123456781234567812345678) + + Exactly one of 'hex', 'bytes', 'fields', or 'int' must be given. + The 'version' argument is optional; if given, the resulting UUID + will have its variant and version number set according to RFC 4122, + overriding bits in the given 'hex', 'bytes', 'fields', or 'int'. + """ + + if [hex, bytes, fields, int].count(None) != 3: + raise TypeError('need just one of hex, bytes, fields, or int') + if hex is not None: + hex = hex.replace('urn:', '').replace('uuid:', '') + hex = hex.strip('{}').replace('-', '') + if len(hex) != 32: + raise ValueError('badly formed hexadecimal UUID string') + int = long(hex, 16) + if bytes is not None: + if len(bytes) != 16: + raise ValueError('bytes is not a 16-char string') + int = long(('%02x'*16) % tuple(map(ord, bytes)), 16) + if fields is not None: + if len(fields) != 6: + raise ValueError('fields is not a 6-tuple') + (time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node) = fields + if not 0 <= time_low < 1<<32L: + raise ValueError('field 1 out of range (need a 32-bit value)') + if not 0 <= time_mid < 1<<16L: + raise ValueError('field 2 out of range (need a 16-bit value)') + if not 0 <= time_hi_version < 1<<16L: + raise ValueError('field 3 out of range (need a 16-bit value)') + if not 0 <= clock_seq_hi_variant < 1<<8L: + raise ValueError('field 4 out of range (need an 8-bit value)') + if not 0 <= clock_seq_low < 1<<8L: + raise ValueError('field 5 out of range (need an 8-bit value)') + if not 0 <= node < 1<<48L: + raise ValueError('field 6 out of range (need a 48-bit value)') + clock_seq = (clock_seq_hi_variant << 8L) | clock_seq_low + int = ((time_low << 96L) | (time_mid << 80L) | + (time_hi_version << 64L) | (clock_seq << 48L) | node) + if int is not None: + if not 0 <= int < 1<<128L: + raise ValueError('int is out of range (need a 128-bit value)') + if version is not None: + if not 1 <= version <= 5: + raise ValueError('illegal version number') + # Set the variant to RFC 4122. + int &= ~(0xc000 << 48L) + int |= 0x8000 << 48L + # Set the version number. + int &= ~(0xf000 << 64L) + int |= version << 76L + self.__dict__['int'] = int + + def __cmp__(self, other): + if isinstance(other, UUID): + return cmp(self.int, other.int) + return NotImplemented + + def __hash__(self): + return hash(self.int) + + def __int__(self): + return self.int + + def __repr__(self): + return 'UUID(%r)' % str(self) + + def __setattr__(self, name, value): + raise TypeError('UUID objects are immutable') + + def __str__(self): + hex = '%032x' % self.int + return '%s-%s-%s-%s-%s' % ( + hex[:8], hex[8:12], hex[12:16], hex[16:20], hex[20:]) + + def get_bytes(self): + bytes = '' + for shift in range(0, 128, 8): + bytes = chr((self.int >> shift) & 0xff) + bytes + return bytes + + bytes = property(get_bytes) + + def get_fields(self): + return (self.time_low, self.time_mid, self.time_hi_version, + self.clock_seq_hi_variant, self.clock_seq_low, self.node) + + fields = property(get_fields) + + def get_time_low(self): + return self.int >> 96L + + time_low = property(get_time_low) + + def get_time_mid(self): + return (self.int >> 80L) & 0xffff + + time_mid = property(get_time_mid) + + def get_time_hi_version(self): + return (self.int >> 64L) & 0xffff + + time_hi_version = property(get_time_hi_version) + + def get_clock_seq_hi_variant(self): + return (self.int >> 56L) & 0xff + + clock_seq_hi_variant = property(get_clock_seq_hi_variant) + + def get_clock_seq_low(self): + return (self.int >> 48L) & 0xff + + clock_seq_low = property(get_clock_seq_low) + + def get_time(self): + return (((self.time_hi_version & 0x0fffL) << 48L) | + (self.time_mid << 32L) | self.time_low) + + time = property(get_time) + + def get_clock_seq(self): + return (((self.clock_seq_hi_variant & 0x3fL) << 8L) | + self.clock_seq_low) + + clock_seq = property(get_clock_seq) + + def get_node(self): + return self.int & 0xffffffffffff + + node = property(get_node) + + def get_hex(self): + return '%032x' % self.int + + hex = property(get_hex) + + def get_urn(self): + return 'urn:uuid:' + str(self) + + urn = property(get_urn) + + def get_variant(self): + if not self.int & (0x8000 << 48L): + return RESERVED_NCS + elif not self.int & (0x4000 << 48L): + return RFC_4122 + elif not self.int & (0x2000 << 48L): + return RESERVED_MICROSOFT + else: + return RESERVED_FUTURE + + variant = property(get_variant) + + def get_version(self): + # The version bits are only meaningful for RFC 4122 UUIDs. + if self.variant == RFC_4122: + return int((self.int >> 76L) & 0xf) + + version = property(get_version) + +def _ifconfig_getnode(): + """Get the hardware address on Unix by running ifconfig.""" + import os + for dir in ['', '/sbin/', '/usr/sbin']: + try: + pipe = os.popen(os.path.join(dir, 'ifconfig')) + except IOError: + continue + for line in pipe: + words = line.lower().split() + for i in range(len(words)): + if words[i] in ['hwaddr', 'ether']: + return int(words[i + 1].replace(':', ''), 16) + +def _ipconfig_getnode(): + """Get the hardware address on Windows by running ipconfig.exe.""" + import os, re + dirs = ['', r'c:\windows\system32', r'c:\winnt\system32'] + try: + import ctypes + buffer = ctypes.create_string_buffer(300) + ctypes.windll.kernel32.GetSystemDirectoryA(buffer, 300) + dirs.insert(0, buffer.value.decode('mbcs')) + except: + pass + for dir in dirs: + try: + pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all') + except IOError: + continue + for line in pipe: + value = line.split(':')[-1].strip().lower() + if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): + return int(value.replace('-', ''), 16) + +def _netbios_getnode(): + """Get the hardware address on Windows using NetBIOS calls. + See http://support.microsoft.com/kb/118623 for details.""" + import win32wnet, netbios + ncb = netbios.NCB() + ncb.Command = netbios.NCBENUM + ncb.Buffer = adapters = netbios.LANA_ENUM() + adapters._pack() + if win32wnet.Netbios(ncb) != 0: + return + adapters._unpack() + for i in range(adapters.length): + ncb.Reset() + ncb.Command = netbios.NCBRESET + ncb.Lana_num = ord(adapters.lana[i]) + if win32wnet.Netbios(ncb) != 0: + continue + ncb.Reset() + ncb.Command = netbios.NCBASTAT + ncb.Lana_num = ord(adapters.lana[i]) + ncb.Callname = '*'.ljust(16) + ncb.Buffer = status = netbios.ADAPTER_STATUS() + if win32wnet.Netbios(ncb) != 0: + continue + status._unpack() + bytes = map(ord, status.adapter_address) + return ((bytes[0]<<40L) + (bytes[1]<<32L) + (bytes[2]<<24L) + + (bytes[3]<<16L) + (bytes[4]<<8L) + bytes[5]) + +# Thanks to Thomas Heller for ctypes and for his help with its use here. + +# If ctypes is available, use it to find system routines for UUID generation. +_uuid_generate_random = _uuid_generate_time = _UuidCreate = None +try: + import ctypes, ctypes.util + _buffer = ctypes.create_string_buffer(16) + + # The uuid_generate_* routines are provided by libuuid on at least + # Linux and FreeBSD, and provided by libc on Mac OS X. + for libname in ['uuid', 'c']: + try: + lib = ctypes.CDLL(ctypes.util.find_library(libname)) + except: + continue + if hasattr(lib, 'uuid_generate_random'): + _uuid_generate_random = lib.uuid_generate_random + if hasattr(lib, 'uuid_generate_time'): + _uuid_generate_time = lib.uuid_generate_time + + # On Windows prior to 2000, UuidCreate gives a UUID containing the + # hardware address. On Windows 2000 and later, UuidCreate makes a + # random UUID and UuidCreateSequential gives a UUID containing the + # hardware address. These routines are provided by the RPC runtime. + try: + lib = ctypes.windll.rpcrt4 + except: + lib = None + _UuidCreate = getattr(lib, 'UuidCreateSequential', + getattr(lib, 'UuidCreate', None)) +except: + pass + +def _unixdll_getnode(): + """Get the hardware address on Unix using ctypes.""" + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw).node + +def _windll_getnode(): + """Get the hardware address on Windows using ctypes.""" + if _UuidCreate(_buffer) == 0: + return UUID(bytes=_buffer.raw).node + +def _random_getnode(): + """Get a random node ID, with eighth bit set as suggested by RFC 4122.""" + import random + return random.randrange(0, 1<<48L) | 0x010000000000L + +_node = None + +def getnode(): + """Get the hardware address as a 48-bit integer. The first time this + runs, it may launch a separate program, which could be quite slow. If + all attempts to obtain the hardware address fail, we choose a random + 48-bit number with its eighth bit set to 1 as recommended in RFC 4122.""" + + global _node + if _node is not None: + return _node + + import sys + if sys.platform == 'win32': + getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] + else: + getters = [_unixdll_getnode, _ifconfig_getnode] + + for getter in getters + [_random_getnode]: + try: + _node = getter() + except: + continue + if _node is not None: + return _node + +def uuid1(node=None, clock_seq=None): + """Generate a UUID from a host ID, sequence number, and the current time. + If 'node' is not given, getnode() is used to obtain the hardware + address. If 'clock_seq' is given, it is used as the sequence number; + otherwise a random 14-bit sequence number is chosen.""" + + # When the system provides a version-1 UUID generator, use it (but don't + # use UuidCreate here because its UUIDs don't conform to RFC 4122). + if _uuid_generate_time and node is clock_seq is None: + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw) + + import time + nanoseconds = int(time.time() * 1e9) + # 0x01b21dd213814000 is the number of 100-ns intervals between the + # UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00. + timestamp = int(nanoseconds/100) + 0x01b21dd213814000L + if clock_seq is None: + import random + clock_seq = random.randrange(1<<14L) # instead of stable storage + time_low = timestamp & 0xffffffffL + time_mid = (timestamp >> 32L) & 0xffffL + time_hi_version = (timestamp >> 48L) & 0x0fffL + clock_seq_low = clock_seq & 0xffL + clock_seq_hi_variant = (clock_seq >> 8L) & 0x3fL + if node is None: + node = getnode() + return UUID(fields=(time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node), version=1) + +def uuid3(namespace, name): + """Generate a UUID from the MD5 hash of a namespace UUID and a name.""" + import md5 + hash = md5.md5(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=3) + +def uuid4(): + """Generate a random UUID.""" + + # When the system provides a version-4 UUID generator, use it. + if _uuid_generate_random: + _uuid_generate_random(_buffer) + return UUID(bytes=_buffer.raw) + + # Otherwise, get randomness from urandom or the 'random' module. + try: + import os + return UUID(bytes=os.urandom(16), version=4) + except: + import random + bytes = [chr(random.randrange(256)) for i in range(16)] + return UUID(bytes=bytes, version=4) + +def uuid5(namespace, name): + """Generate a UUID from the SHA-1 hash of a namespace UUID and a name.""" + import sha + hash = sha.sha(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=5) + +# The following standard UUIDs are for use with uuid3() or uuid5(). + +NAMESPACE_DNS = UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_URL = UUID('6ba7b811-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_OID = UUID('6ba7b812-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_X500 = UUID('6ba7b814-9dad-11d1-80b4-00c04fd430c8') From python-checkins at python.org Tue Jun 13 02:30:51 2006 From: python-checkins at python.org (tim.peters) Date: Tue, 13 Jun 2006 02:30:51 +0200 (CEST) Subject: [Python-checkins] r46903 - in python/trunk/Lib: test/test_uuid.py uuid.py Message-ID: <20060613003051.815671E4002@bag.python.org> Author: tim.peters Date: Tue Jun 13 02:30:50 2006 New Revision: 46903 Modified: python/trunk/Lib/test/test_uuid.py (contents, props changed) python/trunk/Lib/uuid.py (contents, props changed) Log: Added missing svn:eol-style property to text files. Modified: python/trunk/Lib/test/test_uuid.py ============================================================================== --- python/trunk/Lib/test/test_uuid.py (original) +++ python/trunk/Lib/test/test_uuid.py Tue Jun 13 02:30:50 2006 @@ -1,396 +1,396 @@ -from unittest import TestCase, main -import uuid - -def importable(name): - try: - __import__(name) - return True - except: - return False - -class TestUUID(TestCase): - last_node = None - - def test_UUID(self): - equal = self.assertEqual - ascending = [] - for (string, curly, hex, bytes, fields, integer, urn, - time, clock_seq, variant, version) in [ - ('00000000-0000-0000-0000-000000000000', - '{00000000-0000-0000-0000-000000000000}', - '00000000000000000000000000000000', - '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', - (0, 0, 0, 0, 0, 0), - 0, - 'urn:uuid:00000000-0000-0000-0000-000000000000', - 0, 0, uuid.RESERVED_NCS, None), - ('00010203-0405-0607-0809-0a0b0c0d0e0f', - '{00010203-0405-0607-0809-0a0b0c0d0e0f}', - '000102030405060708090a0b0c0d0e0f', - '\0\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\x0d\x0e\x0f', - (0x00010203L, 0x0405, 0x0607, 8, 9, 0x0a0b0c0d0e0fL), - 0x000102030405060708090a0b0c0d0e0fL, - 'urn:uuid:00010203-0405-0607-0809-0a0b0c0d0e0f', - 0x607040500010203L, 0x809, uuid.RESERVED_NCS, None), - ('02d9e6d5-9467-382e-8f9b-9300a64ac3cd', - '{02d9e6d5-9467-382e-8f9b-9300a64ac3cd}', - '02d9e6d59467382e8f9b9300a64ac3cd', - '\x02\xd9\xe6\xd5\x94\x67\x38\x2e\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', - (0x02d9e6d5L, 0x9467, 0x382e, 0x8f, 0x9b, 0x9300a64ac3cdL), - 0x02d9e6d59467382e8f9b9300a64ac3cdL, - 'urn:uuid:02d9e6d5-9467-382e-8f9b-9300a64ac3cd', - 0x82e946702d9e6d5L, 0xf9b, uuid.RFC_4122, 3), - ('12345678-1234-5678-1234-567812345678', - '{12345678-1234-5678-1234-567812345678}', - '12345678123456781234567812345678', - '\x12\x34\x56\x78'*4, - (0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678), - 0x12345678123456781234567812345678, - 'urn:uuid:12345678-1234-5678-1234-567812345678', - 0x678123412345678L, 0x1234, uuid.RESERVED_NCS, None), - ('6ba7b810-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b810-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8109dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x10\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b810L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8109dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b810-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b810L, 0xb4, uuid.RFC_4122, 1), - ('6ba7b811-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b811-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8119dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x11\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b811L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8119dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b811-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b811L, 0xb4, uuid.RFC_4122, 1), - ('6ba7b812-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b812-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8129dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x12\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b812L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8129dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b812-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b812L, 0xb4, uuid.RFC_4122, 1), - ('6ba7b814-9dad-11d1-80b4-00c04fd430c8', - '{6ba7b814-9dad-11d1-80b4-00c04fd430c8}', - '6ba7b8149dad11d180b400c04fd430c8', - '\x6b\xa7\xb8\x14\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', - (0x6ba7b814L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), - 0x6ba7b8149dad11d180b400c04fd430c8L, - 'urn:uuid:6ba7b814-9dad-11d1-80b4-00c04fd430c8', - 0x1d19dad6ba7b814L, 0xb4, uuid.RFC_4122, 1), - ('7d444840-9dc0-11d1-b245-5ffdce74fad2', - '{7d444840-9dc0-11d1-b245-5ffdce74fad2}', - '7d4448409dc011d1b2455ffdce74fad2', - '\x7d\x44\x48\x40\x9d\xc0\x11\xd1\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', - (0x7d444840L, 0x9dc0, 0x11d1, 0xb2, 0x45, 0x5ffdce74fad2L), - 0x7d4448409dc011d1b2455ffdce74fad2L, - 'urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2', - 0x1d19dc07d444840L, 0x3245, uuid.RFC_4122, 1), - ('e902893a-9d22-3c7e-a7b8-d6e313b71d9f', - '{e902893a-9d22-3c7e-a7b8-d6e313b71d9f}', - 'e902893a9d223c7ea7b8d6e313b71d9f', - '\xe9\x02\x89\x3a\x9d\x22\x3c\x7e\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', - (0xe902893aL, 0x9d22, 0x3c7e, 0xa7, 0xb8, 0xd6e313b71d9fL), - 0xe902893a9d223c7ea7b8d6e313b71d9fL, - 'urn:uuid:e902893a-9d22-3c7e-a7b8-d6e313b71d9f', - 0xc7e9d22e902893aL, 0x27b8, uuid.RFC_4122, 3), - ('eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', - '{eb424026-6f54-4ef8-a4d0-bb658a1fc6cf}', - 'eb4240266f544ef8a4d0bb658a1fc6cf', - '\xeb\x42\x40\x26\x6f\x54\x4e\xf8\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', - (0xeb424026L, 0x6f54, 0x4ef8, 0xa4, 0xd0, 0xbb658a1fc6cfL), - 0xeb4240266f544ef8a4d0bb658a1fc6cfL, - 'urn:uuid:eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', - 0xef86f54eb424026L, 0x24d0, uuid.RFC_4122, 4), - ('f81d4fae-7dec-11d0-a765-00a0c91e6bf6', - '{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}', - 'f81d4fae7dec11d0a76500a0c91e6bf6', - '\xf8\x1d\x4f\xae\x7d\xec\x11\xd0\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', - (0xf81d4faeL, 0x7dec, 0x11d0, 0xa7, 0x65, 0x00a0c91e6bf6L), - 0xf81d4fae7dec11d0a76500a0c91e6bf6L, - 'urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6', - 0x1d07decf81d4faeL, 0x2765, uuid.RFC_4122, 1), - ('fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', - '{fffefdfc-fffe-fffe-fffe-fffefdfcfbfa}', - 'fffefdfcfffefffefffefffefdfcfbfa', - '\xff\xfe\xfd\xfc\xff\xfe\xff\xfe\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', - (0xfffefdfcL, 0xfffe, 0xfffe, 0xff, 0xfe, 0xfffefdfcfbfaL), - 0xfffefdfcfffefffefffefffefdfcfbfaL, - 'urn:uuid:fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', - 0xffefffefffefdfcL, 0x3ffe, uuid.RESERVED_FUTURE, None), - ('ffffffff-ffff-ffff-ffff-ffffffffffff', - '{ffffffff-ffff-ffff-ffff-ffffffffffff}', - 'ffffffffffffffffffffffffffffffff', - '\xff'*16, - (0xffffffffL, 0xffffL, 0xffffL, 0xff, 0xff, 0xffffffffffffL), - 0xffffffffffffffffffffffffffffffffL, - 'urn:uuid:ffffffff-ffff-ffff-ffff-ffffffffffff', - 0xfffffffffffffffL, 0x3fff, uuid.RESERVED_FUTURE, None), - ]: - equivalents = [] - # Construct each UUID in several different ways. - for u in [uuid.UUID(string), uuid.UUID(curly), uuid.UUID(hex), - uuid.UUID(bytes=bytes), uuid.UUID(fields=fields), - uuid.UUID(int=integer), uuid.UUID(urn)]: - # Test all conversions and properties of the UUID object. - equal(str(u), string) - equal(int(u), integer) - equal(u.bytes, bytes) - equal(u.fields, fields) - equal(u.time_low, fields[0]) - equal(u.time_mid, fields[1]) - equal(u.time_hi_version, fields[2]) - equal(u.clock_seq_hi_variant, fields[3]) - equal(u.clock_seq_low, fields[4]) - equal(u.node, fields[5]) - equal(u.hex, hex) - equal(u.int, integer) - equal(u.urn, urn) - equal(u.time, time) - equal(u.clock_seq, clock_seq) - equal(u.variant, variant) - equal(u.version, version) - equivalents.append(u) - - # Different construction methods should give the same UUID. - for u in equivalents: - for v in equivalents: - equal(u, v) - ascending.append(u) - - # Test comparison of UUIDs. - for i in range(len(ascending)): - for j in range(len(ascending)): - equal(cmp(i, j), cmp(ascending[i], ascending[j])) - - # Test sorting of UUIDs (above list is in ascending order). - resorted = ascending[:] - resorted.reverse() - resorted.sort() - equal(ascending, resorted) - - def test_exceptions(self): - badvalue = lambda f: self.assertRaises(ValueError, f) - badtype = lambda f: self.assertRaises(TypeError, f) - - # Badly formed hex strings. - badvalue(lambda: uuid.UUID('')) - badvalue(lambda: uuid.UUID('abc')) - badvalue(lambda: uuid.UUID('1234567812345678123456781234567')) - badvalue(lambda: uuid.UUID('123456781234567812345678123456789')) - badvalue(lambda: uuid.UUID('123456781234567812345678z2345678')) - - # Badly formed bytes. - badvalue(lambda: uuid.UUID(bytes='abc')) - badvalue(lambda: uuid.UUID(bytes='\0'*15)) - badvalue(lambda: uuid.UUID(bytes='\0'*17)) - - # Badly formed fields. - badvalue(lambda: uuid.UUID(fields=(1,))) - badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5))) - badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5, 6, 7))) - - # Field values out of range. - badvalue(lambda: uuid.UUID(fields=(-1, 0, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0x100000000L, 0, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, -1, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0x10000L, 0, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, -1, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0x10000L, 0, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, -1, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0x100L, 0, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, -1, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0x100L, 0))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, -1))) - badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, 0x1000000000000L))) - - # Version number out of range. - badvalue(lambda: uuid.UUID('00'*16, version=0)) - badvalue(lambda: uuid.UUID('00'*16, version=6)) - - # Integer value out of range. - badvalue(lambda: uuid.UUID(int=-1)) - badvalue(lambda: uuid.UUID(int=1<<128L)) - - # Must supply exactly one of hex, bytes, fields, int. - h, b, f, i = '00'*16, '\0'*16, (0, 0, 0, 0, 0, 0), 0 - uuid.UUID(h) - uuid.UUID(hex=h) - uuid.UUID(bytes=b) - uuid.UUID(fields=f) - uuid.UUID(int=i) - - # Wrong number of arguments (positional). - badtype(lambda: uuid.UUID()) - badtype(lambda: uuid.UUID(h, b)) - badtype(lambda: uuid.UUID(h, b, f)) - badtype(lambda: uuid.UUID(h, b, f, i)) - - # Duplicate arguments (named). - badtype(lambda: uuid.UUID(hex=h, bytes=b)) - badtype(lambda: uuid.UUID(hex=h, fields=f)) - badtype(lambda: uuid.UUID(hex=h, int=i)) - badtype(lambda: uuid.UUID(bytes=b, fields=f)) - badtype(lambda: uuid.UUID(bytes=b, int=i)) - badtype(lambda: uuid.UUID(fields=f, int=i)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(hex=h, fields=f, int=i)) - badtype(lambda: uuid.UUID(bytes=b, int=i, fields=f)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i, fields=f)) - - # Duplicate arguments (positional and named). - badtype(lambda: uuid.UUID(h, hex=h)) - badtype(lambda: uuid.UUID(h, bytes=b)) - badtype(lambda: uuid.UUID(h, fields=f)) - badtype(lambda: uuid.UUID(h, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b)) - badtype(lambda: uuid.UUID(h, hex=h, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, int=i)) - badtype(lambda: uuid.UUID(h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(h, fields=f, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, fields=f, int=i)) - badtype(lambda: uuid.UUID(h, bytes=b, int=i, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i, fields=f)) - - # Immutability. - u = uuid.UUID(h) - badtype(lambda: setattr(u, 'hex', h)) - badtype(lambda: setattr(u, 'bytes', b)) - badtype(lambda: setattr(u, 'fields', f)) - badtype(lambda: setattr(u, 'int', i)) - - def check_node(self, node, source=''): - individual_group_bit = (node >> 40L) & 1 - universal_local_bit = (node >> 40L) & 2 - message = "%012x doesn't look like a real MAC address" % node - self.assertEqual(individual_group_bit, 0, message) - self.assertEqual(universal_local_bit, 0, message) - self.assertNotEqual(node, 0, message) - self.assertNotEqual(node, 0xffffffffffffL, message) - self.assert_(0 <= node, message) - self.assert_(node < 1<<48L, message) - - import sys - if source: - sys.stderr.write('(%s: %012x)' % (source, node)) - if TestUUID.last_node: - self.assertEqual(TestUUID.last_node, node, 'inconsistent node IDs') - else: - TestUUID.last_node = node - - def test_ifconfig_getnode(self): - import os - if os.name == 'posix': - self.check_node(uuid._ifconfig_getnode(), 'ifconfig') - - def test_ipconfig_getnode(self): - import os - if os.name == 'nt': - self.check_node(uuid._ipconfig_getnode(), 'ipconfig') - - def test_netbios_getnode(self): - if importable('win32wnet') and importable('netbios'): - self.check_node(uuid._netbios_getnode(), 'netbios') - - def test_random_getnode(self): - node = uuid._random_getnode() - self.assert_(0 <= node) - self.assert_(node < 1<<48L) - - def test_unixdll_getnode(self): - import os - if importable('ctypes') and os.name == 'posix': - self.check_node(uuid._unixdll_getnode(), 'unixdll') - - def test_windll_getnode(self): - import os - if importable('ctypes') and os.name == 'nt': - self.check_node(uuid._windll_getnode(), 'windll') - - def test_getnode(self): - self.check_node(uuid.getnode()) - - # Test it again to ensure consistency. - self.check_node(uuid.getnode()) - - def test_uuid1(self): - equal = self.assertEqual - - # Make sure uuid4() generates UUIDs that are actually version 1. - for u in [uuid.uuid1() for i in range(10)]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 1) - - # Make sure the supplied node ID appears in the UUID. - u = uuid.uuid1(0) - equal(u.node, 0) - u = uuid.uuid1(0x123456789abc) - equal(u.node, 0x123456789abc) - u = uuid.uuid1(0xffffffffffff) - equal(u.node, 0xffffffffffff) - - # Make sure the supplied clock sequence appears in the UUID. - u = uuid.uuid1(0x123456789abc, 0) - equal(u.node, 0x123456789abc) - equal(((u.clock_seq_hi_variant & 0x3f) << 8) | u.clock_seq_low, 0) - u = uuid.uuid1(0x123456789abc, 0x1234) - equal(u.node, 0x123456789abc) - equal(((u.clock_seq_hi_variant & 0x3f) << 8) | - u.clock_seq_low, 0x1234) - u = uuid.uuid1(0x123456789abc, 0x3fff) - equal(u.node, 0x123456789abc) - equal(((u.clock_seq_hi_variant & 0x3f) << 8) | - u.clock_seq_low, 0x3fff) - - def test_uuid3(self): - equal = self.assertEqual - - # Test some known version-3 UUIDs. - for u, v in [(uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org'), - '6fa459ea-ee8a-3ca4-894e-db77e160355e'), - (uuid.uuid3(uuid.NAMESPACE_URL, 'http://python.org/'), - '9fe8e8c4-aaa8-32a9-a55c-4535a88b748d'), - (uuid.uuid3(uuid.NAMESPACE_OID, '1.3.6.1'), - 'dd1a1cef-13d5-368a-ad82-eca71acd4cd1'), - (uuid.uuid3(uuid.NAMESPACE_X500, 'c=ca'), - '658d3002-db6b-3040-a1d1-8ddd7d189a4d'), - ]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 3) - equal(u, uuid.UUID(v)) - equal(str(u), v) - - def test_uuid4(self): - equal = self.assertEqual - - # Make sure uuid4() generates UUIDs that are actually version 4. - for u in [uuid.uuid4() for i in range(10)]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 4) - - def test_uuid5(self): - equal = self.assertEqual - - # Test some known version-5 UUIDs. - for u, v in [(uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org'), - '886313e1-3b8a-5372-9b90-0c9aee199e5d'), - (uuid.uuid5(uuid.NAMESPACE_URL, 'http://python.org/'), - '4c565f0d-3f5a-5890-b41b-20cf47701c5e'), - (uuid.uuid5(uuid.NAMESPACE_OID, '1.3.6.1'), - '1447fa61-5277-5fef-a9b3-fbc6e44f4af3'), - (uuid.uuid5(uuid.NAMESPACE_X500, 'c=ca'), - 'cc957dd1-a972-5349-98cd-874190002798'), - ]: - equal(u.variant, uuid.RFC_4122) - equal(u.version, 5) - equal(u, uuid.UUID(v)) - equal(str(u), v) - -if __name__ == '__main__': - main() +from unittest import TestCase, main +import uuid + +def importable(name): + try: + __import__(name) + return True + except: + return False + +class TestUUID(TestCase): + last_node = None + + def test_UUID(self): + equal = self.assertEqual + ascending = [] + for (string, curly, hex, bytes, fields, integer, urn, + time, clock_seq, variant, version) in [ + ('00000000-0000-0000-0000-000000000000', + '{00000000-0000-0000-0000-000000000000}', + '00000000000000000000000000000000', + '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', + (0, 0, 0, 0, 0, 0), + 0, + 'urn:uuid:00000000-0000-0000-0000-000000000000', + 0, 0, uuid.RESERVED_NCS, None), + ('00010203-0405-0607-0809-0a0b0c0d0e0f', + '{00010203-0405-0607-0809-0a0b0c0d0e0f}', + '000102030405060708090a0b0c0d0e0f', + '\0\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\x0d\x0e\x0f', + (0x00010203L, 0x0405, 0x0607, 8, 9, 0x0a0b0c0d0e0fL), + 0x000102030405060708090a0b0c0d0e0fL, + 'urn:uuid:00010203-0405-0607-0809-0a0b0c0d0e0f', + 0x607040500010203L, 0x809, uuid.RESERVED_NCS, None), + ('02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + '{02d9e6d5-9467-382e-8f9b-9300a64ac3cd}', + '02d9e6d59467382e8f9b9300a64ac3cd', + '\x02\xd9\xe6\xd5\x94\x67\x38\x2e\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', + (0x02d9e6d5L, 0x9467, 0x382e, 0x8f, 0x9b, 0x9300a64ac3cdL), + 0x02d9e6d59467382e8f9b9300a64ac3cdL, + 'urn:uuid:02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + 0x82e946702d9e6d5L, 0xf9b, uuid.RFC_4122, 3), + ('12345678-1234-5678-1234-567812345678', + '{12345678-1234-5678-1234-567812345678}', + '12345678123456781234567812345678', + '\x12\x34\x56\x78'*4, + (0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678), + 0x12345678123456781234567812345678, + 'urn:uuid:12345678-1234-5678-1234-567812345678', + 0x678123412345678L, 0x1234, uuid.RESERVED_NCS, None), + ('6ba7b810-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b810-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8109dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x10\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b810L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8109dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b810-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b810L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b811-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b811-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8119dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x11\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b811L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8119dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b811-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b811L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b812-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b812-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8129dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x12\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b812L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8129dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b812-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b812L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b814-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b814-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8149dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x14\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b814L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8149dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b814-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b814L, 0xb4, uuid.RFC_4122, 1), + ('7d444840-9dc0-11d1-b245-5ffdce74fad2', + '{7d444840-9dc0-11d1-b245-5ffdce74fad2}', + '7d4448409dc011d1b2455ffdce74fad2', + '\x7d\x44\x48\x40\x9d\xc0\x11\xd1\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', + (0x7d444840L, 0x9dc0, 0x11d1, 0xb2, 0x45, 0x5ffdce74fad2L), + 0x7d4448409dc011d1b2455ffdce74fad2L, + 'urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2', + 0x1d19dc07d444840L, 0x3245, uuid.RFC_4122, 1), + ('e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + '{e902893a-9d22-3c7e-a7b8-d6e313b71d9f}', + 'e902893a9d223c7ea7b8d6e313b71d9f', + '\xe9\x02\x89\x3a\x9d\x22\x3c\x7e\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', + (0xe902893aL, 0x9d22, 0x3c7e, 0xa7, 0xb8, 0xd6e313b71d9fL), + 0xe902893a9d223c7ea7b8d6e313b71d9fL, + 'urn:uuid:e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + 0xc7e9d22e902893aL, 0x27b8, uuid.RFC_4122, 3), + ('eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + '{eb424026-6f54-4ef8-a4d0-bb658a1fc6cf}', + 'eb4240266f544ef8a4d0bb658a1fc6cf', + '\xeb\x42\x40\x26\x6f\x54\x4e\xf8\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', + (0xeb424026L, 0x6f54, 0x4ef8, 0xa4, 0xd0, 0xbb658a1fc6cfL), + 0xeb4240266f544ef8a4d0bb658a1fc6cfL, + 'urn:uuid:eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + 0xef86f54eb424026L, 0x24d0, uuid.RFC_4122, 4), + ('f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + '{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}', + 'f81d4fae7dec11d0a76500a0c91e6bf6', + '\xf8\x1d\x4f\xae\x7d\xec\x11\xd0\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', + (0xf81d4faeL, 0x7dec, 0x11d0, 0xa7, 0x65, 0x00a0c91e6bf6L), + 0xf81d4fae7dec11d0a76500a0c91e6bf6L, + 'urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + 0x1d07decf81d4faeL, 0x2765, uuid.RFC_4122, 1), + ('fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + '{fffefdfc-fffe-fffe-fffe-fffefdfcfbfa}', + 'fffefdfcfffefffefffefffefdfcfbfa', + '\xff\xfe\xfd\xfc\xff\xfe\xff\xfe\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', + (0xfffefdfcL, 0xfffe, 0xfffe, 0xff, 0xfe, 0xfffefdfcfbfaL), + 0xfffefdfcfffefffefffefffefdfcfbfaL, + 'urn:uuid:fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + 0xffefffefffefdfcL, 0x3ffe, uuid.RESERVED_FUTURE, None), + ('ffffffff-ffff-ffff-ffff-ffffffffffff', + '{ffffffff-ffff-ffff-ffff-ffffffffffff}', + 'ffffffffffffffffffffffffffffffff', + '\xff'*16, + (0xffffffffL, 0xffffL, 0xffffL, 0xff, 0xff, 0xffffffffffffL), + 0xffffffffffffffffffffffffffffffffL, + 'urn:uuid:ffffffff-ffff-ffff-ffff-ffffffffffff', + 0xfffffffffffffffL, 0x3fff, uuid.RESERVED_FUTURE, None), + ]: + equivalents = [] + # Construct each UUID in several different ways. + for u in [uuid.UUID(string), uuid.UUID(curly), uuid.UUID(hex), + uuid.UUID(bytes=bytes), uuid.UUID(fields=fields), + uuid.UUID(int=integer), uuid.UUID(urn)]: + # Test all conversions and properties of the UUID object. + equal(str(u), string) + equal(int(u), integer) + equal(u.bytes, bytes) + equal(u.fields, fields) + equal(u.time_low, fields[0]) + equal(u.time_mid, fields[1]) + equal(u.time_hi_version, fields[2]) + equal(u.clock_seq_hi_variant, fields[3]) + equal(u.clock_seq_low, fields[4]) + equal(u.node, fields[5]) + equal(u.hex, hex) + equal(u.int, integer) + equal(u.urn, urn) + equal(u.time, time) + equal(u.clock_seq, clock_seq) + equal(u.variant, variant) + equal(u.version, version) + equivalents.append(u) + + # Different construction methods should give the same UUID. + for u in equivalents: + for v in equivalents: + equal(u, v) + ascending.append(u) + + # Test comparison of UUIDs. + for i in range(len(ascending)): + for j in range(len(ascending)): + equal(cmp(i, j), cmp(ascending[i], ascending[j])) + + # Test sorting of UUIDs (above list is in ascending order). + resorted = ascending[:] + resorted.reverse() + resorted.sort() + equal(ascending, resorted) + + def test_exceptions(self): + badvalue = lambda f: self.assertRaises(ValueError, f) + badtype = lambda f: self.assertRaises(TypeError, f) + + # Badly formed hex strings. + badvalue(lambda: uuid.UUID('')) + badvalue(lambda: uuid.UUID('abc')) + badvalue(lambda: uuid.UUID('1234567812345678123456781234567')) + badvalue(lambda: uuid.UUID('123456781234567812345678123456789')) + badvalue(lambda: uuid.UUID('123456781234567812345678z2345678')) + + # Badly formed bytes. + badvalue(lambda: uuid.UUID(bytes='abc')) + badvalue(lambda: uuid.UUID(bytes='\0'*15)) + badvalue(lambda: uuid.UUID(bytes='\0'*17)) + + # Badly formed fields. + badvalue(lambda: uuid.UUID(fields=(1,))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5, 6, 7))) + + # Field values out of range. + badvalue(lambda: uuid.UUID(fields=(-1, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0x100000000L, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, -1, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0x10000L, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, -1, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0x10000L, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, -1, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0x100L, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, -1, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0x100L, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, -1))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, 0x1000000000000L))) + + # Version number out of range. + badvalue(lambda: uuid.UUID('00'*16, version=0)) + badvalue(lambda: uuid.UUID('00'*16, version=6)) + + # Integer value out of range. + badvalue(lambda: uuid.UUID(int=-1)) + badvalue(lambda: uuid.UUID(int=1<<128L)) + + # Must supply exactly one of hex, bytes, fields, int. + h, b, f, i = '00'*16, '\0'*16, (0, 0, 0, 0, 0, 0), 0 + uuid.UUID(h) + uuid.UUID(hex=h) + uuid.UUID(bytes=b) + uuid.UUID(fields=f) + uuid.UUID(int=i) + + # Wrong number of arguments (positional). + badtype(lambda: uuid.UUID()) + badtype(lambda: uuid.UUID(h, b)) + badtype(lambda: uuid.UUID(h, b, f)) + badtype(lambda: uuid.UUID(h, b, f, i)) + + # Duplicate arguments (named). + badtype(lambda: uuid.UUID(hex=h, bytes=b)) + badtype(lambda: uuid.UUID(hex=h, fields=f)) + badtype(lambda: uuid.UUID(hex=h, int=i)) + badtype(lambda: uuid.UUID(bytes=b, fields=f)) + badtype(lambda: uuid.UUID(bytes=b, int=i)) + badtype(lambda: uuid.UUID(fields=f, int=i)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i, fields=f)) + + # Duplicate arguments (positional and named). + badtype(lambda: uuid.UUID(h, hex=h)) + badtype(lambda: uuid.UUID(h, bytes=b)) + badtype(lambda: uuid.UUID(h, fields=f)) + badtype(lambda: uuid.UUID(h, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i, fields=f)) + + # Immutability. + u = uuid.UUID(h) + badtype(lambda: setattr(u, 'hex', h)) + badtype(lambda: setattr(u, 'bytes', b)) + badtype(lambda: setattr(u, 'fields', f)) + badtype(lambda: setattr(u, 'int', i)) + + def check_node(self, node, source=''): + individual_group_bit = (node >> 40L) & 1 + universal_local_bit = (node >> 40L) & 2 + message = "%012x doesn't look like a real MAC address" % node + self.assertEqual(individual_group_bit, 0, message) + self.assertEqual(universal_local_bit, 0, message) + self.assertNotEqual(node, 0, message) + self.assertNotEqual(node, 0xffffffffffffL, message) + self.assert_(0 <= node, message) + self.assert_(node < 1<<48L, message) + + import sys + if source: + sys.stderr.write('(%s: %012x)' % (source, node)) + if TestUUID.last_node: + self.assertEqual(TestUUID.last_node, node, 'inconsistent node IDs') + else: + TestUUID.last_node = node + + def test_ifconfig_getnode(self): + import os + if os.name == 'posix': + self.check_node(uuid._ifconfig_getnode(), 'ifconfig') + + def test_ipconfig_getnode(self): + import os + if os.name == 'nt': + self.check_node(uuid._ipconfig_getnode(), 'ipconfig') + + def test_netbios_getnode(self): + if importable('win32wnet') and importable('netbios'): + self.check_node(uuid._netbios_getnode(), 'netbios') + + def test_random_getnode(self): + node = uuid._random_getnode() + self.assert_(0 <= node) + self.assert_(node < 1<<48L) + + def test_unixdll_getnode(self): + import os + if importable('ctypes') and os.name == 'posix': + self.check_node(uuid._unixdll_getnode(), 'unixdll') + + def test_windll_getnode(self): + import os + if importable('ctypes') and os.name == 'nt': + self.check_node(uuid._windll_getnode(), 'windll') + + def test_getnode(self): + self.check_node(uuid.getnode()) + + # Test it again to ensure consistency. + self.check_node(uuid.getnode()) + + def test_uuid1(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 1. + for u in [uuid.uuid1() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 1) + + # Make sure the supplied node ID appears in the UUID. + u = uuid.uuid1(0) + equal(u.node, 0) + u = uuid.uuid1(0x123456789abc) + equal(u.node, 0x123456789abc) + u = uuid.uuid1(0xffffffffffff) + equal(u.node, 0xffffffffffff) + + # Make sure the supplied clock sequence appears in the UUID. + u = uuid.uuid1(0x123456789abc, 0) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | u.clock_seq_low, 0) + u = uuid.uuid1(0x123456789abc, 0x1234) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x1234) + u = uuid.uuid1(0x123456789abc, 0x3fff) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x3fff) + + def test_uuid3(self): + equal = self.assertEqual + + # Test some known version-3 UUIDs. + for u, v in [(uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org'), + '6fa459ea-ee8a-3ca4-894e-db77e160355e'), + (uuid.uuid3(uuid.NAMESPACE_URL, 'http://python.org/'), + '9fe8e8c4-aaa8-32a9-a55c-4535a88b748d'), + (uuid.uuid3(uuid.NAMESPACE_OID, '1.3.6.1'), + 'dd1a1cef-13d5-368a-ad82-eca71acd4cd1'), + (uuid.uuid3(uuid.NAMESPACE_X500, 'c=ca'), + '658d3002-db6b-3040-a1d1-8ddd7d189a4d'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 3) + equal(u, uuid.UUID(v)) + equal(str(u), v) + + def test_uuid4(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 4. + for u in [uuid.uuid4() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 4) + + def test_uuid5(self): + equal = self.assertEqual + + # Test some known version-5 UUIDs. + for u, v in [(uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org'), + '886313e1-3b8a-5372-9b90-0c9aee199e5d'), + (uuid.uuid5(uuid.NAMESPACE_URL, 'http://python.org/'), + '4c565f0d-3f5a-5890-b41b-20cf47701c5e'), + (uuid.uuid5(uuid.NAMESPACE_OID, '1.3.6.1'), + '1447fa61-5277-5fef-a9b3-fbc6e44f4af3'), + (uuid.uuid5(uuid.NAMESPACE_X500, 'c=ca'), + 'cc957dd1-a972-5349-98cd-874190002798'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 5) + equal(u, uuid.UUID(v)) + equal(str(u), v) + +if __name__ == '__main__': + main() Modified: python/trunk/Lib/uuid.py ============================================================================== --- python/trunk/Lib/uuid.py (original) +++ python/trunk/Lib/uuid.py Tue Jun 13 02:30:50 2006 @@ -1,477 +1,477 @@ -r"""UUID objects (universally unique identifiers) according to RFC 4122. - -This module provides immutable UUID objects (class UUID) and the functions -uuid1(), uuid3(), uuid4(), uuid5() for generating version 1, 3, 4, and 5 -UUIDs as specified in RFC 4122. - -If all you want is a unique ID, you should probably call uuid1() or uuid4(). -Note that uuid1() may compromise privacy since it creates a UUID containing -the computer's network address. uuid4() creates a random UUID. - -Typical usage: - - >>> import uuid - - # make a UUID based on the host ID and current time - >>> uuid.uuid1() - UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') - - # make a UUID using an MD5 hash of a namespace UUID and a name - >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') - UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') - - # make a random UUID - >>> uuid.uuid4() - UUID('16fd2706-8baf-433b-82eb-8c7fada847da') - - # make a UUID using a SHA-1 hash of a namespace UUID and a name - >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') - UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') - - # make a UUID from a string of hex digits (braces and hyphens ignored) - >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') - - # convert a UUID to a string of hex digits in standard form - >>> str(x) - '00010203-0405-0607-0809-0a0b0c0d0e0f' - - # get the raw 16 bytes of the UUID - >>> x.bytes - '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' - - # make a UUID from a 16-byte string - >>> uuid.UUID(bytes=x.bytes) - UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') - -This module works with Python 2.3 or higher.""" - -__author__ = 'Ka-Ping Yee ' -__date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') -__version__ = '$Revision: 1.30 $'.split()[1] - -RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, RESERVED_FUTURE = [ - 'reserved for NCS compatibility', 'specified in RFC 4122', - 'reserved for Microsoft compatibility', 'reserved for future definition'] - -class UUID(object): - """Instances of the UUID class represent UUIDs as specified in RFC 4122. - UUID objects are immutable, hashable, and usable as dictionary keys. - Converting a UUID to a string with str() yields something in the form - '12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts - four possible forms: a similar string of hexadecimal digits, or a - string of 16 raw bytes as an argument named 'bytes', or a tuple of - six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and - 48-bit values respectively) as an argument named 'fields', or a single - 128-bit integer as an argument named 'int'. - - UUIDs have these read-only attributes: - - bytes the UUID as a 16-byte string - - fields a tuple of the six integer fields of the UUID, - which are also available as six individual attributes - and two derived attributes: - - time_low the first 32 bits of the UUID - time_mid the next 16 bits of the UUID - time_hi_version the next 16 bits of the UUID - clock_seq_hi_variant the next 8 bits of the UUID - clock_seq_low the next 8 bits of the UUID - node the last 48 bits of the UUID - - time the 60-bit timestamp - clock_seq the 14-bit sequence number - - hex the UUID as a 32-character hexadecimal string - - int the UUID as a 128-bit integer - - urn the UUID as a URN as specified in RFC 4122 - - variant the UUID variant (one of the constants RESERVED_NCS, - RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE) - - version the UUID version number (1 through 5, meaningful only - when the variant is RFC_4122) - """ - - def __init__(self, hex=None, bytes=None, fields=None, int=None, - version=None): - r"""Create a UUID from either a string of 32 hexadecimal digits, - a string of 16 bytes as the 'bytes' argument, a tuple of six - integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, - 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as - the 'fields' argument, or a single 128-bit integer as the 'int' - argument. When a string of hex digits is given, curly braces, - hyphens, and a URN prefix are all optional. For example, these - expressions all yield the same UUID: - - UUID('{12345678-1234-5678-1234-567812345678}') - UUID('12345678123456781234567812345678') - UUID('urn:uuid:12345678-1234-5678-1234-567812345678') - UUID(bytes='\x12\x34\x56\x78'*4) - UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) - UUID(int=0x12345678123456781234567812345678) - - Exactly one of 'hex', 'bytes', 'fields', or 'int' must be given. - The 'version' argument is optional; if given, the resulting UUID - will have its variant and version number set according to RFC 4122, - overriding bits in the given 'hex', 'bytes', 'fields', or 'int'. - """ - - if [hex, bytes, fields, int].count(None) != 3: - raise TypeError('need just one of hex, bytes, fields, or int') - if hex is not None: - hex = hex.replace('urn:', '').replace('uuid:', '') - hex = hex.strip('{}').replace('-', '') - if len(hex) != 32: - raise ValueError('badly formed hexadecimal UUID string') - int = long(hex, 16) - if bytes is not None: - if len(bytes) != 16: - raise ValueError('bytes is not a 16-char string') - int = long(('%02x'*16) % tuple(map(ord, bytes)), 16) - if fields is not None: - if len(fields) != 6: - raise ValueError('fields is not a 6-tuple') - (time_low, time_mid, time_hi_version, - clock_seq_hi_variant, clock_seq_low, node) = fields - if not 0 <= time_low < 1<<32L: - raise ValueError('field 1 out of range (need a 32-bit value)') - if not 0 <= time_mid < 1<<16L: - raise ValueError('field 2 out of range (need a 16-bit value)') - if not 0 <= time_hi_version < 1<<16L: - raise ValueError('field 3 out of range (need a 16-bit value)') - if not 0 <= clock_seq_hi_variant < 1<<8L: - raise ValueError('field 4 out of range (need an 8-bit value)') - if not 0 <= clock_seq_low < 1<<8L: - raise ValueError('field 5 out of range (need an 8-bit value)') - if not 0 <= node < 1<<48L: - raise ValueError('field 6 out of range (need a 48-bit value)') - clock_seq = (clock_seq_hi_variant << 8L) | clock_seq_low - int = ((time_low << 96L) | (time_mid << 80L) | - (time_hi_version << 64L) | (clock_seq << 48L) | node) - if int is not None: - if not 0 <= int < 1<<128L: - raise ValueError('int is out of range (need a 128-bit value)') - if version is not None: - if not 1 <= version <= 5: - raise ValueError('illegal version number') - # Set the variant to RFC 4122. - int &= ~(0xc000 << 48L) - int |= 0x8000 << 48L - # Set the version number. - int &= ~(0xf000 << 64L) - int |= version << 76L - self.__dict__['int'] = int - - def __cmp__(self, other): - if isinstance(other, UUID): - return cmp(self.int, other.int) - return NotImplemented - - def __hash__(self): - return hash(self.int) - - def __int__(self): - return self.int - - def __repr__(self): - return 'UUID(%r)' % str(self) - - def __setattr__(self, name, value): - raise TypeError('UUID objects are immutable') - - def __str__(self): - hex = '%032x' % self.int - return '%s-%s-%s-%s-%s' % ( - hex[:8], hex[8:12], hex[12:16], hex[16:20], hex[20:]) - - def get_bytes(self): - bytes = '' - for shift in range(0, 128, 8): - bytes = chr((self.int >> shift) & 0xff) + bytes - return bytes - - bytes = property(get_bytes) - - def get_fields(self): - return (self.time_low, self.time_mid, self.time_hi_version, - self.clock_seq_hi_variant, self.clock_seq_low, self.node) - - fields = property(get_fields) - - def get_time_low(self): - return self.int >> 96L - - time_low = property(get_time_low) - - def get_time_mid(self): - return (self.int >> 80L) & 0xffff - - time_mid = property(get_time_mid) - - def get_time_hi_version(self): - return (self.int >> 64L) & 0xffff - - time_hi_version = property(get_time_hi_version) - - def get_clock_seq_hi_variant(self): - return (self.int >> 56L) & 0xff - - clock_seq_hi_variant = property(get_clock_seq_hi_variant) - - def get_clock_seq_low(self): - return (self.int >> 48L) & 0xff - - clock_seq_low = property(get_clock_seq_low) - - def get_time(self): - return (((self.time_hi_version & 0x0fffL) << 48L) | - (self.time_mid << 32L) | self.time_low) - - time = property(get_time) - - def get_clock_seq(self): - return (((self.clock_seq_hi_variant & 0x3fL) << 8L) | - self.clock_seq_low) - - clock_seq = property(get_clock_seq) - - def get_node(self): - return self.int & 0xffffffffffff - - node = property(get_node) - - def get_hex(self): - return '%032x' % self.int - - hex = property(get_hex) - - def get_urn(self): - return 'urn:uuid:' + str(self) - - urn = property(get_urn) - - def get_variant(self): - if not self.int & (0x8000 << 48L): - return RESERVED_NCS - elif not self.int & (0x4000 << 48L): - return RFC_4122 - elif not self.int & (0x2000 << 48L): - return RESERVED_MICROSOFT - else: - return RESERVED_FUTURE - - variant = property(get_variant) - - def get_version(self): - # The version bits are only meaningful for RFC 4122 UUIDs. - if self.variant == RFC_4122: - return int((self.int >> 76L) & 0xf) - - version = property(get_version) - -def _ifconfig_getnode(): - """Get the hardware address on Unix by running ifconfig.""" - import os - for dir in ['', '/sbin/', '/usr/sbin']: - try: - pipe = os.popen(os.path.join(dir, 'ifconfig')) - except IOError: - continue - for line in pipe: - words = line.lower().split() - for i in range(len(words)): - if words[i] in ['hwaddr', 'ether']: - return int(words[i + 1].replace(':', ''), 16) - -def _ipconfig_getnode(): - """Get the hardware address on Windows by running ipconfig.exe.""" - import os, re - dirs = ['', r'c:\windows\system32', r'c:\winnt\system32'] - try: - import ctypes - buffer = ctypes.create_string_buffer(300) - ctypes.windll.kernel32.GetSystemDirectoryA(buffer, 300) - dirs.insert(0, buffer.value.decode('mbcs')) - except: - pass - for dir in dirs: - try: - pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all') - except IOError: - continue - for line in pipe: - value = line.split(':')[-1].strip().lower() - if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): - return int(value.replace('-', ''), 16) - -def _netbios_getnode(): - """Get the hardware address on Windows using NetBIOS calls. - See http://support.microsoft.com/kb/118623 for details.""" - import win32wnet, netbios - ncb = netbios.NCB() - ncb.Command = netbios.NCBENUM - ncb.Buffer = adapters = netbios.LANA_ENUM() - adapters._pack() - if win32wnet.Netbios(ncb) != 0: - return - adapters._unpack() - for i in range(adapters.length): - ncb.Reset() - ncb.Command = netbios.NCBRESET - ncb.Lana_num = ord(adapters.lana[i]) - if win32wnet.Netbios(ncb) != 0: - continue - ncb.Reset() - ncb.Command = netbios.NCBASTAT - ncb.Lana_num = ord(adapters.lana[i]) - ncb.Callname = '*'.ljust(16) - ncb.Buffer = status = netbios.ADAPTER_STATUS() - if win32wnet.Netbios(ncb) != 0: - continue - status._unpack() - bytes = map(ord, status.adapter_address) - return ((bytes[0]<<40L) + (bytes[1]<<32L) + (bytes[2]<<24L) + - (bytes[3]<<16L) + (bytes[4]<<8L) + bytes[5]) - -# Thanks to Thomas Heller for ctypes and for his help with its use here. - -# If ctypes is available, use it to find system routines for UUID generation. -_uuid_generate_random = _uuid_generate_time = _UuidCreate = None -try: - import ctypes, ctypes.util - _buffer = ctypes.create_string_buffer(16) - - # The uuid_generate_* routines are provided by libuuid on at least - # Linux and FreeBSD, and provided by libc on Mac OS X. - for libname in ['uuid', 'c']: - try: - lib = ctypes.CDLL(ctypes.util.find_library(libname)) - except: - continue - if hasattr(lib, 'uuid_generate_random'): - _uuid_generate_random = lib.uuid_generate_random - if hasattr(lib, 'uuid_generate_time'): - _uuid_generate_time = lib.uuid_generate_time - - # On Windows prior to 2000, UuidCreate gives a UUID containing the - # hardware address. On Windows 2000 and later, UuidCreate makes a - # random UUID and UuidCreateSequential gives a UUID containing the - # hardware address. These routines are provided by the RPC runtime. - try: - lib = ctypes.windll.rpcrt4 - except: - lib = None - _UuidCreate = getattr(lib, 'UuidCreateSequential', - getattr(lib, 'UuidCreate', None)) -except: - pass - -def _unixdll_getnode(): - """Get the hardware address on Unix using ctypes.""" - _uuid_generate_time(_buffer) - return UUID(bytes=_buffer.raw).node - -def _windll_getnode(): - """Get the hardware address on Windows using ctypes.""" - if _UuidCreate(_buffer) == 0: - return UUID(bytes=_buffer.raw).node - -def _random_getnode(): - """Get a random node ID, with eighth bit set as suggested by RFC 4122.""" - import random - return random.randrange(0, 1<<48L) | 0x010000000000L - -_node = None - -def getnode(): - """Get the hardware address as a 48-bit integer. The first time this - runs, it may launch a separate program, which could be quite slow. If - all attempts to obtain the hardware address fail, we choose a random - 48-bit number with its eighth bit set to 1 as recommended in RFC 4122.""" - - global _node - if _node is not None: - return _node - - import sys - if sys.platform == 'win32': - getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] - else: - getters = [_unixdll_getnode, _ifconfig_getnode] - - for getter in getters + [_random_getnode]: - try: - _node = getter() - except: - continue - if _node is not None: - return _node - -def uuid1(node=None, clock_seq=None): - """Generate a UUID from a host ID, sequence number, and the current time. - If 'node' is not given, getnode() is used to obtain the hardware - address. If 'clock_seq' is given, it is used as the sequence number; - otherwise a random 14-bit sequence number is chosen.""" - - # When the system provides a version-1 UUID generator, use it (but don't - # use UuidCreate here because its UUIDs don't conform to RFC 4122). - if _uuid_generate_time and node is clock_seq is None: - _uuid_generate_time(_buffer) - return UUID(bytes=_buffer.raw) - - import time - nanoseconds = int(time.time() * 1e9) - # 0x01b21dd213814000 is the number of 100-ns intervals between the - # UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00. - timestamp = int(nanoseconds/100) + 0x01b21dd213814000L - if clock_seq is None: - import random - clock_seq = random.randrange(1<<14L) # instead of stable storage - time_low = timestamp & 0xffffffffL - time_mid = (timestamp >> 32L) & 0xffffL - time_hi_version = (timestamp >> 48L) & 0x0fffL - clock_seq_low = clock_seq & 0xffL - clock_seq_hi_variant = (clock_seq >> 8L) & 0x3fL - if node is None: - node = getnode() - return UUID(fields=(time_low, time_mid, time_hi_version, - clock_seq_hi_variant, clock_seq_low, node), version=1) - -def uuid3(namespace, name): - """Generate a UUID from the MD5 hash of a namespace UUID and a name.""" - import md5 - hash = md5.md5(namespace.bytes + name).digest() - return UUID(bytes=hash[:16], version=3) - -def uuid4(): - """Generate a random UUID.""" - - # When the system provides a version-4 UUID generator, use it. - if _uuid_generate_random: - _uuid_generate_random(_buffer) - return UUID(bytes=_buffer.raw) - - # Otherwise, get randomness from urandom or the 'random' module. - try: - import os - return UUID(bytes=os.urandom(16), version=4) - except: - import random - bytes = [chr(random.randrange(256)) for i in range(16)] - return UUID(bytes=bytes, version=4) - -def uuid5(namespace, name): - """Generate a UUID from the SHA-1 hash of a namespace UUID and a name.""" - import sha - hash = sha.sha(namespace.bytes + name).digest() - return UUID(bytes=hash[:16], version=5) - -# The following standard UUIDs are for use with uuid3() or uuid5(). - -NAMESPACE_DNS = UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8') -NAMESPACE_URL = UUID('6ba7b811-9dad-11d1-80b4-00c04fd430c8') -NAMESPACE_OID = UUID('6ba7b812-9dad-11d1-80b4-00c04fd430c8') -NAMESPACE_X500 = UUID('6ba7b814-9dad-11d1-80b4-00c04fd430c8') +r"""UUID objects (universally unique identifiers) according to RFC 4122. + +This module provides immutable UUID objects (class UUID) and the functions +uuid1(), uuid3(), uuid4(), uuid5() for generating version 1, 3, 4, and 5 +UUIDs as specified in RFC 4122. + +If all you want is a unique ID, you should probably call uuid1() or uuid4(). +Note that uuid1() may compromise privacy since it creates a UUID containing +the computer's network address. uuid4() creates a random UUID. + +Typical usage: + + >>> import uuid + + # make a UUID based on the host ID and current time + >>> uuid.uuid1() + UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + + # make a UUID using an MD5 hash of a namespace UUID and a name + >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') + UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + + # make a random UUID + >>> uuid.uuid4() + UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + + # make a UUID using a SHA-1 hash of a namespace UUID and a name + >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') + UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + + # make a UUID from a string of hex digits (braces and hyphens ignored) + >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') + + # convert a UUID to a string of hex digits in standard form + >>> str(x) + '00010203-0405-0607-0809-0a0b0c0d0e0f' + + # get the raw 16 bytes of the UUID + >>> x.bytes + '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + + # make a UUID from a 16-byte string + >>> uuid.UUID(bytes=x.bytes) + UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') + +This module works with Python 2.3 or higher.""" + +__author__ = 'Ka-Ping Yee ' +__date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') +__version__ = '$Revision: 1.30 $'.split()[1] + +RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, RESERVED_FUTURE = [ + 'reserved for NCS compatibility', 'specified in RFC 4122', + 'reserved for Microsoft compatibility', 'reserved for future definition'] + +class UUID(object): + """Instances of the UUID class represent UUIDs as specified in RFC 4122. + UUID objects are immutable, hashable, and usable as dictionary keys. + Converting a UUID to a string with str() yields something in the form + '12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts + four possible forms: a similar string of hexadecimal digits, or a + string of 16 raw bytes as an argument named 'bytes', or a tuple of + six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and + 48-bit values respectively) as an argument named 'fields', or a single + 128-bit integer as an argument named 'int'. + + UUIDs have these read-only attributes: + + bytes the UUID as a 16-byte string + + fields a tuple of the six integer fields of the UUID, + which are also available as six individual attributes + and two derived attributes: + + time_low the first 32 bits of the UUID + time_mid the next 16 bits of the UUID + time_hi_version the next 16 bits of the UUID + clock_seq_hi_variant the next 8 bits of the UUID + clock_seq_low the next 8 bits of the UUID + node the last 48 bits of the UUID + + time the 60-bit timestamp + clock_seq the 14-bit sequence number + + hex the UUID as a 32-character hexadecimal string + + int the UUID as a 128-bit integer + + urn the UUID as a URN as specified in RFC 4122 + + variant the UUID variant (one of the constants RESERVED_NCS, + RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE) + + version the UUID version number (1 through 5, meaningful only + when the variant is RFC_4122) + """ + + def __init__(self, hex=None, bytes=None, fields=None, int=None, + version=None): + r"""Create a UUID from either a string of 32 hexadecimal digits, + a string of 16 bytes as the 'bytes' argument, a tuple of six + integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, + 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as + the 'fields' argument, or a single 128-bit integer as the 'int' + argument. When a string of hex digits is given, curly braces, + hyphens, and a URN prefix are all optional. For example, these + expressions all yield the same UUID: + + UUID('{12345678-1234-5678-1234-567812345678}') + UUID('12345678123456781234567812345678') + UUID('urn:uuid:12345678-1234-5678-1234-567812345678') + UUID(bytes='\x12\x34\x56\x78'*4) + UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) + UUID(int=0x12345678123456781234567812345678) + + Exactly one of 'hex', 'bytes', 'fields', or 'int' must be given. + The 'version' argument is optional; if given, the resulting UUID + will have its variant and version number set according to RFC 4122, + overriding bits in the given 'hex', 'bytes', 'fields', or 'int'. + """ + + if [hex, bytes, fields, int].count(None) != 3: + raise TypeError('need just one of hex, bytes, fields, or int') + if hex is not None: + hex = hex.replace('urn:', '').replace('uuid:', '') + hex = hex.strip('{}').replace('-', '') + if len(hex) != 32: + raise ValueError('badly formed hexadecimal UUID string') + int = long(hex, 16) + if bytes is not None: + if len(bytes) != 16: + raise ValueError('bytes is not a 16-char string') + int = long(('%02x'*16) % tuple(map(ord, bytes)), 16) + if fields is not None: + if len(fields) != 6: + raise ValueError('fields is not a 6-tuple') + (time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node) = fields + if not 0 <= time_low < 1<<32L: + raise ValueError('field 1 out of range (need a 32-bit value)') + if not 0 <= time_mid < 1<<16L: + raise ValueError('field 2 out of range (need a 16-bit value)') + if not 0 <= time_hi_version < 1<<16L: + raise ValueError('field 3 out of range (need a 16-bit value)') + if not 0 <= clock_seq_hi_variant < 1<<8L: + raise ValueError('field 4 out of range (need an 8-bit value)') + if not 0 <= clock_seq_low < 1<<8L: + raise ValueError('field 5 out of range (need an 8-bit value)') + if not 0 <= node < 1<<48L: + raise ValueError('field 6 out of range (need a 48-bit value)') + clock_seq = (clock_seq_hi_variant << 8L) | clock_seq_low + int = ((time_low << 96L) | (time_mid << 80L) | + (time_hi_version << 64L) | (clock_seq << 48L) | node) + if int is not None: + if not 0 <= int < 1<<128L: + raise ValueError('int is out of range (need a 128-bit value)') + if version is not None: + if not 1 <= version <= 5: + raise ValueError('illegal version number') + # Set the variant to RFC 4122. + int &= ~(0xc000 << 48L) + int |= 0x8000 << 48L + # Set the version number. + int &= ~(0xf000 << 64L) + int |= version << 76L + self.__dict__['int'] = int + + def __cmp__(self, other): + if isinstance(other, UUID): + return cmp(self.int, other.int) + return NotImplemented + + def __hash__(self): + return hash(self.int) + + def __int__(self): + return self.int + + def __repr__(self): + return 'UUID(%r)' % str(self) + + def __setattr__(self, name, value): + raise TypeError('UUID objects are immutable') + + def __str__(self): + hex = '%032x' % self.int + return '%s-%s-%s-%s-%s' % ( + hex[:8], hex[8:12], hex[12:16], hex[16:20], hex[20:]) + + def get_bytes(self): + bytes = '' + for shift in range(0, 128, 8): + bytes = chr((self.int >> shift) & 0xff) + bytes + return bytes + + bytes = property(get_bytes) + + def get_fields(self): + return (self.time_low, self.time_mid, self.time_hi_version, + self.clock_seq_hi_variant, self.clock_seq_low, self.node) + + fields = property(get_fields) + + def get_time_low(self): + return self.int >> 96L + + time_low = property(get_time_low) + + def get_time_mid(self): + return (self.int >> 80L) & 0xffff + + time_mid = property(get_time_mid) + + def get_time_hi_version(self): + return (self.int >> 64L) & 0xffff + + time_hi_version = property(get_time_hi_version) + + def get_clock_seq_hi_variant(self): + return (self.int >> 56L) & 0xff + + clock_seq_hi_variant = property(get_clock_seq_hi_variant) + + def get_clock_seq_low(self): + return (self.int >> 48L) & 0xff + + clock_seq_low = property(get_clock_seq_low) + + def get_time(self): + return (((self.time_hi_version & 0x0fffL) << 48L) | + (self.time_mid << 32L) | self.time_low) + + time = property(get_time) + + def get_clock_seq(self): + return (((self.clock_seq_hi_variant & 0x3fL) << 8L) | + self.clock_seq_low) + + clock_seq = property(get_clock_seq) + + def get_node(self): + return self.int & 0xffffffffffff + + node = property(get_node) + + def get_hex(self): + return '%032x' % self.int + + hex = property(get_hex) + + def get_urn(self): + return 'urn:uuid:' + str(self) + + urn = property(get_urn) + + def get_variant(self): + if not self.int & (0x8000 << 48L): + return RESERVED_NCS + elif not self.int & (0x4000 << 48L): + return RFC_4122 + elif not self.int & (0x2000 << 48L): + return RESERVED_MICROSOFT + else: + return RESERVED_FUTURE + + variant = property(get_variant) + + def get_version(self): + # The version bits are only meaningful for RFC 4122 UUIDs. + if self.variant == RFC_4122: + return int((self.int >> 76L) & 0xf) + + version = property(get_version) + +def _ifconfig_getnode(): + """Get the hardware address on Unix by running ifconfig.""" + import os + for dir in ['', '/sbin/', '/usr/sbin']: + try: + pipe = os.popen(os.path.join(dir, 'ifconfig')) + except IOError: + continue + for line in pipe: + words = line.lower().split() + for i in range(len(words)): + if words[i] in ['hwaddr', 'ether']: + return int(words[i + 1].replace(':', ''), 16) + +def _ipconfig_getnode(): + """Get the hardware address on Windows by running ipconfig.exe.""" + import os, re + dirs = ['', r'c:\windows\system32', r'c:\winnt\system32'] + try: + import ctypes + buffer = ctypes.create_string_buffer(300) + ctypes.windll.kernel32.GetSystemDirectoryA(buffer, 300) + dirs.insert(0, buffer.value.decode('mbcs')) + except: + pass + for dir in dirs: + try: + pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all') + except IOError: + continue + for line in pipe: + value = line.split(':')[-1].strip().lower() + if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): + return int(value.replace('-', ''), 16) + +def _netbios_getnode(): + """Get the hardware address on Windows using NetBIOS calls. + See http://support.microsoft.com/kb/118623 for details.""" + import win32wnet, netbios + ncb = netbios.NCB() + ncb.Command = netbios.NCBENUM + ncb.Buffer = adapters = netbios.LANA_ENUM() + adapters._pack() + if win32wnet.Netbios(ncb) != 0: + return + adapters._unpack() + for i in range(adapters.length): + ncb.Reset() + ncb.Command = netbios.NCBRESET + ncb.Lana_num = ord(adapters.lana[i]) + if win32wnet.Netbios(ncb) != 0: + continue + ncb.Reset() + ncb.Command = netbios.NCBASTAT + ncb.Lana_num = ord(adapters.lana[i]) + ncb.Callname = '*'.ljust(16) + ncb.Buffer = status = netbios.ADAPTER_STATUS() + if win32wnet.Netbios(ncb) != 0: + continue + status._unpack() + bytes = map(ord, status.adapter_address) + return ((bytes[0]<<40L) + (bytes[1]<<32L) + (bytes[2]<<24L) + + (bytes[3]<<16L) + (bytes[4]<<8L) + bytes[5]) + +# Thanks to Thomas Heller for ctypes and for his help with its use here. + +# If ctypes is available, use it to find system routines for UUID generation. +_uuid_generate_random = _uuid_generate_time = _UuidCreate = None +try: + import ctypes, ctypes.util + _buffer = ctypes.create_string_buffer(16) + + # The uuid_generate_* routines are provided by libuuid on at least + # Linux and FreeBSD, and provided by libc on Mac OS X. + for libname in ['uuid', 'c']: + try: + lib = ctypes.CDLL(ctypes.util.find_library(libname)) + except: + continue + if hasattr(lib, 'uuid_generate_random'): + _uuid_generate_random = lib.uuid_generate_random + if hasattr(lib, 'uuid_generate_time'): + _uuid_generate_time = lib.uuid_generate_time + + # On Windows prior to 2000, UuidCreate gives a UUID containing the + # hardware address. On Windows 2000 and later, UuidCreate makes a + # random UUID and UuidCreateSequential gives a UUID containing the + # hardware address. These routines are provided by the RPC runtime. + try: + lib = ctypes.windll.rpcrt4 + except: + lib = None + _UuidCreate = getattr(lib, 'UuidCreateSequential', + getattr(lib, 'UuidCreate', None)) +except: + pass + +def _unixdll_getnode(): + """Get the hardware address on Unix using ctypes.""" + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw).node + +def _windll_getnode(): + """Get the hardware address on Windows using ctypes.""" + if _UuidCreate(_buffer) == 0: + return UUID(bytes=_buffer.raw).node + +def _random_getnode(): + """Get a random node ID, with eighth bit set as suggested by RFC 4122.""" + import random + return random.randrange(0, 1<<48L) | 0x010000000000L + +_node = None + +def getnode(): + """Get the hardware address as a 48-bit integer. The first time this + runs, it may launch a separate program, which could be quite slow. If + all attempts to obtain the hardware address fail, we choose a random + 48-bit number with its eighth bit set to 1 as recommended in RFC 4122.""" + + global _node + if _node is not None: + return _node + + import sys + if sys.platform == 'win32': + getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] + else: + getters = [_unixdll_getnode, _ifconfig_getnode] + + for getter in getters + [_random_getnode]: + try: + _node = getter() + except: + continue + if _node is not None: + return _node + +def uuid1(node=None, clock_seq=None): + """Generate a UUID from a host ID, sequence number, and the current time. + If 'node' is not given, getnode() is used to obtain the hardware + address. If 'clock_seq' is given, it is used as the sequence number; + otherwise a random 14-bit sequence number is chosen.""" + + # When the system provides a version-1 UUID generator, use it (but don't + # use UuidCreate here because its UUIDs don't conform to RFC 4122). + if _uuid_generate_time and node is clock_seq is None: + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw) + + import time + nanoseconds = int(time.time() * 1e9) + # 0x01b21dd213814000 is the number of 100-ns intervals between the + # UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00. + timestamp = int(nanoseconds/100) + 0x01b21dd213814000L + if clock_seq is None: + import random + clock_seq = random.randrange(1<<14L) # instead of stable storage + time_low = timestamp & 0xffffffffL + time_mid = (timestamp >> 32L) & 0xffffL + time_hi_version = (timestamp >> 48L) & 0x0fffL + clock_seq_low = clock_seq & 0xffL + clock_seq_hi_variant = (clock_seq >> 8L) & 0x3fL + if node is None: + node = getnode() + return UUID(fields=(time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node), version=1) + +def uuid3(namespace, name): + """Generate a UUID from the MD5 hash of a namespace UUID and a name.""" + import md5 + hash = md5.md5(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=3) + +def uuid4(): + """Generate a random UUID.""" + + # When the system provides a version-4 UUID generator, use it. + if _uuid_generate_random: + _uuid_generate_random(_buffer) + return UUID(bytes=_buffer.raw) + + # Otherwise, get randomness from urandom or the 'random' module. + try: + import os + return UUID(bytes=os.urandom(16), version=4) + except: + import random + bytes = [chr(random.randrange(256)) for i in range(16)] + return UUID(bytes=bytes, version=4) + +def uuid5(namespace, name): + """Generate a UUID from the SHA-1 hash of a namespace UUID and a name.""" + import sha + hash = sha.sha(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=5) + +# The following standard UUIDs are for use with uuid3() or uuid5(). + +NAMESPACE_DNS = UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_URL = UUID('6ba7b811-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_OID = UUID('6ba7b812-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_X500 = UUID('6ba7b814-9dad-11d1-80b4-00c04fd430c8') From buildbot at python.org Tue Jun 13 03:32:25 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 01:32:25 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060613013225.9A8EA1E4002@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/346 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ka-ping.yee Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 05:01:47 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Tue, 13 Jun 2006 05:01:47 +0200 (CEST) Subject: [Python-checkins] r46904 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060613030147.91F581E4002@bag.python.org> Author: mateusz.rukowicz Date: Tue Jun 13 05:01:44 2006 New Revision: 46904 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Real dividing now works and passes all tests, integer not yet implemented. Commented 2 functions causing sigsegv. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Tue Jun 13 05:01:44 2006 @@ -86,8 +86,33 @@ * when I say "digit" I mean digit between 0-9 (limb consists of LOG digits). */ +/* NOTE: there is a little inconcistency with size arguments, sometimes they + * say how many limbs, sometimes how many digits, it's going to change XXX*/ + /* takes new_digits from self[], starting from start_at digit, *digits* not limbs */ /* returns digit after that we returned (useful for rounding) */ + +/* just returns how many significant limbs */ +static long +_limb_size(long *self, long limbs) +{ + while(!self[limbs-1] && limbs >1) limbs --; + return limbs; +} + +/* just like binary <<, but base BASE not 2 */ +static void +_limb_move_left(long *self, long limbs, long count) +{ + assert(count >= 0); + long i; + for (i = limbs - 1; i - count >= 0; i--) + self[i] = self[i-count]; + + for (i = count-1; i >= 0; i--) + self[i] = 0; +} + static long _limb_first_n_digits(long *self, long ndigits, long start_at, long *new, long new_digits) { @@ -216,6 +241,8 @@ static long _limb_get_digit(long *self, long ndigits, long i) { + if(i >= ndigits) + return 0; long pos = ndigits - i - 1; long limb = pos / LOG; pos %= LOG; @@ -228,6 +255,22 @@ return tmp%10; } +/* returns how many significant digits */ +static long +_limb_size_s(long *self, long size) +{ + long limbs = (size + LOG -1)/ LOG; + long slimbs; + slimbs = _limb_size(self, limbs); + long size_at_most = slimbs * LOG; + + while(_limb_get_digit(self, size_at_most, 0) == 0 && size_at_most > 1) size_at_most --; + + return size_at_most; + +} + +/* compares, self and other must be normalized */ static int _limb_compare(long *self, long limbs, long *other, long olimbs) { @@ -241,6 +284,18 @@ return 0; } +/* just like above, but no need to proper sizes + * (may be extra zeros at most significant positions) */ +static int +_limb_compare_un(long *self, long slimbs, long *other, long olimbs) +{ + long n_slimbs, n_olimbs; + n_slimbs = _limb_size(self, slimbs); + n_olimbs = _limb_size(other, olimbs); + + return _limb_compare(self, n_slimbs, other, n_olimbs); +} + static long _limb_add_core(long *big, long bsize, long *small, long ssize, long *out) { @@ -286,6 +341,8 @@ return _limb_add_core(other, osize, self, ssize, out); } +/* substracts small from big, and stores result in out, sizes are in + * digits, not limbs */ static long _limb_sub(long *big, long bsize, long *small, long ssize, long *out) { @@ -317,6 +374,29 @@ return left_digits; } + +/* substracts small from big and stores result in big, sizes are in limbs + * returns how many limbs left */ +static long +_limb_sub_sl(long *big, long blimbs, long *small, long slimbs) +{ + long i; + for(i = 0; i < slimbs; i++) + big[i] -= small[i]; + + for(i = 0; i < blimbs; i++) { + if (big[i] < 0) + { + assert(i + 1 < blimbs); + big[i] += BASE; + big[i+1] --; + } + } + + return _limb_size(big, blimbs); + + +} /* temporary solution, will be speeded up */ static long _limb_multiply_core(long *first, long flimbs, long *second, long slimbs, long *out) @@ -371,6 +451,56 @@ return digits_at_most; } +/* performs dividing, returns relative position of dot to last limb, rest has + * at most as much elements as second + 1, it calculates prec *significant* + * limbs, first must not be 0 */ +/* XXX it's naive dividing, very slow */ +static long +_limb_divide(long *first, long flimbs, long *second, long slimbs, + long *out, long prec, long *rest) +{ + long rlimbs = 1; /* significant limbs of rest */ + long is_significant = 0; /* tells whether non_zero limb has already showed up */ + long out_pos = prec - 1; /* where we write result starts at prec - 1 stops at 0*/ + long new_pos = flimbs - 2; /* where is new digit to add at the end of rest, + if new_pos<0 add 0 */ + + rest[0] = first[flimbs-1]; + + while(1) { + long candidate = 0; + long cmp; + + while ((cmp = _limb_compare_un(rest, rlimbs, second, slimbs)) >= 0) { + candidate ++; + rlimbs = _limb_sub_sl(rest, rlimbs, second, slimbs); + } + + if (candidate) + is_significant = 1; + + if (is_significant) { + out[out_pos] = candidate; + out_pos --; + } + + if (out_pos < 0) + break; + + assert(rlimbs <= slimbs +1); + _limb_move_left(rest, slimbs+1, 1); + + if(new_pos >= 0) + rest[0] = first[new_pos]; + else + rest[0] = 0; + + rlimbs = _limb_size(rest, slimbs + 1); + new_pos --; + } + + return new_pos; +} /* helpful macros ************************************************************/ @@ -2331,13 +2461,427 @@ return (PyObject*)_do_decimal_absolute(self, ctx, round); } + +static PyObject * +_do_decimal__divide(decimalobject *self, decimalobject *other, int divmod, contextobject *ctx) +{ + int sign = (self->sign&1) ^ (other->sign&1); + int self_is_zero; + int other_is_zero; + int shouldround; + PyObject *ans; + + if (ISSPECIAL(self) || ISSPECIAL(other)) { + decimalobject *nan; + if (_check_nans(self, other, ctx, &nan)) { + if (divmod) { + ans = Py_BuildValue("(OO)", nan, nan); + Py_XDECREF(nan); + } + else + ans = (PyObject*) nan; + + return ans; + } + + if (ISINF(self) && ISINF(other)) { + if (divmod) { + decimalobject *ans1, *ans2; + ans1 = handle_InvalidOperation(self->ob_type, ctx, + "(+-) INF // (+-) INF", NULL); + if (!ans1) + return NULL; + ans2 = handle_InvalidOperation(self->ob_type, ctx, + "(+-) INF % (+-) INF", NULL); + + if (!ans2) { + Py_DECREF(ans1); + return NULL; + } + + ans = Py_BuildValue("(OO)", ans1, ans2); + Py_DECREF(ans1); + Py_DECREF(ans2); + } + else { + ans = (PyObject*)handle_InvalidOperation(self->ob_type, ctx, + "(+- INF / (+-) INF", NULL); + } + + return ans; + } + + if (ISINF(self)) { + decimalobject *inf; + int inf_sign; + + inf_sign = sign ? SIGN_NEGINF : SIGN_POSINF; + inf = _NEW_decimalobj(1, inf_sign, 0); + + if (!inf) + return NULL; + + inf->limbs[0] = 0; + + if (divmod == 1 || divmod == 3) { + decimalobject *ans2; + + ans2 = handle_InvalidOperation(self->ob_type, ctx, "INF % x", NULL); + if (!ans2) { + Py_DECREF(inf); + return NULL; + } + + ans = Py_BuildValue("(OO)", inf, ans2); + + Py_DECREF(inf); + Py_DECREF(ans2); + } + + else if (divmod == 2) { + decimalobject *nan; + nan = _NEW_decimalobj(1, SIGN_POSNAN, 0); + if (!nan) { + Py_DECREF(inf); + return NULL; + } + nan->limbs[0] = 0; + + ans = Py_BuildValue("(OO)", inf, nan); + Py_DECREF(inf); + Py_DECREF(nan); + } + /* divmod == 0 */ + else { + ans = (PyObject*) inf; + } + + return ans; + } + + if (ISINF(other)) { + if (divmod) { + decimalobject *ans1, *ans2; + ans1 = _NEW_decimalobj(1, sign, 0); + + if (!ans1) + return NULL; + + ans2 = _decimal_get_copy(self); + if (!ans2) { + Py_DECREF(ans1); + return NULL; + } + + ans = Py_BuildValue("(OO)", ans1, ans2); + Py_DECREF(ans1); + Py_DECREF(ans2); + } + else { + decimalobject *ret; + if (handle_Clamped(ctx, NULL)) + return NULL; + + ret = _NEW_decimalobj(1, sign, ETINY(ctx)); + if (!ret) + return NULL; + ret->limbs[0] = 0; + ans = (PyObject *) ret; + } + + return ans; + } + } + + self_is_zero = !decimal_nonzero(self); + other_is_zero = !decimal_nonzero(other); + + if (self_is_zero && other_is_zero) { + if (divmod) { + return handle_DivisionUndefined(self->ob_type, ctx, "0 / 0", 1); + } + return handle_DivisionUndefined(self->ob_type, ctx, "0 / 0", 0); + } + + if (self_is_zero) { + long exp; + if (divmod) { + decimalobject *ans1, *ans2; + + ans2 = _decimal_get_copy(self); + if (!ans2) + return NULL; + ans2->exp = self->exp < other->exp ? self->exp : other->exp; + + ans1 = _NEW_decimalobj(1, sign, 0); + if (!ans1) { + Py_DECREF(ans2); + return NULL; + } + ans = Py_BuildValue("(OO)", ans1, ans2); + + Py_DECREF(ans1); + Py_DECREF(ans2); + return ans; + } + + exp = self->exp - other->exp; + + if (exp < ETINY(ctx)) { + exp = ETINY(ctx); + if (handle_Clamped(ctx, "0e-x / y")) + return NULL; + } + + if (exp > ctx->Emax) { + exp = ctx->Emax; + if (handle_Clamped(ctx, "0e+x / y")) + return NULL; + } + + { + decimalobject *ret; + ret = _NEW_decimalobj(1, sign, exp); + if (!ret) + return NULL; + ret->limbs[0] = 0; + + ans = (PyObject*) ret; + return ans; + } + + } + + if (other_is_zero) { + if (divmod) { + return handle_DivisionByZero(self->ob_type, ctx, "divmod(x,0)", sign, 1); + } + return handle_DivisionByZero(self->ob_type, ctx, "x / 0", sign, 0); + } + + /* now fun starts, answer isnt one of 0 NaN or INF */ + + shouldround = ctx->rounding_dec == ALWAYS_ROUND; + + /* if we divide into ints, we'll check if self < other */ + if (divmod) { + decimalobject *abs1, *abs2; + int cmp; + abs1 = _do_decimal_absolute(self, ctx, 0); + if (!abs1) + return NULL; + + abs2 = _do_decimal_absolute(other, ctx, 0); + if (!abs2) { + Py_DECREF(abs1); + return NULL; + } + + cmp = _do_real_decimal_compare(self, other, ctx); + + Py_DECREF(abs1); + Py_DECREF(abs2); + if (PyErr_Occurred()) { + return NULL; + } + + if (cmp == -1) { + decimalobject *ans1, *ans2; + + ans1 = _NEW_decimalobj(1, sign, 0); + if (!ans1) + return NULL; + ans1->limbs[0] = 0; + + if (divmod == 1 || divmod == 3) { + decimalobject *tmp; + long exp; + exp = self->exp < other->exp ? self->exp : other->exp; + tmp = _decimal_rescale(self, exp, ctx, -1, 0); + + if (!tmp) { + Py_DECREF(ans1); + return NULL; + } + + if (shouldround) { + ans2 = _decimal_fix(tmp, ctx); + if (!ans2) { + Py_DECREF(tmp); + Py_DECREF(ans1); + return NULL; + } + Py_DECREF(tmp); + } + else { + ans2 = tmp; + } + + } + + else if (divmod == 2) { + ans2 = _decimal_get_copy(self); + if (!ans2) + { + Py_DECREF(ans1); + return NULL; + } + } + + ans = Py_BuildValue("(OO)", ans1, ans2); + Py_DECREF(ans1); + Py_DECREF(ans2); + + return ans; + } + } + + + if (divmod == 0) { + long prec_needed; /* how many significant digits we need */ + long significant_limbs; /* how many significant limbs we need to achieve prec */ + decimalobject *result; + long *remainder; + long exp = self->exp - other->exp; + long expdiff; + long rlimbs; + long old_size; + long adj1, adj2, adjusted = 0; + long i; + long max_size; + + + if (!shouldround) + Py_RETURN_NONE; /* TODO */ + + prec_needed = ctx->prec+1; + /* we need (prec_needed + LOG -1)/ LOG rounded up limbs, because + * it may happen, that first limb is between 1 and 9, so we'll + * get (significant_limbs-1) * LOG + 1 digits */ + significant_limbs = (prec_needed + LOG -1) / LOG; + significant_limbs += ((prec_needed + LOG -1) % LOG) != 0; + + remainder = (long*) PyMem_MALLOC((other->limb_count+1) * sizeof(long)); + memset(remainder, 0, sizeof(long) * (other->limb_count+1)); + if (!remainder) { + PyErr_NoMemory(); + return NULL; + } + + /* we create room for additional limb, in case remainder != 0 + * then, we will just move results->limbs one left, and set first + * limb 1, to make rounding working properly */ + result = _NEW_decimalobj(significant_limbs * LOG + LOG, sign, 0); + for (i = 0; i < result->limb_count; i++) + result->limbs[0] = 0; + result->ob_size -= LOG; + result->limb_count --; + if (!result) { + PyMem_FREE(remainder); + return NULL; + } + + expdiff = _limb_divide(self->limbs, self->limb_count, + other->limbs, other->limb_count, result->limbs, + significant_limbs, remainder); + result->limbs[significant_limbs] = 0; + + exp += expdiff * LOG + LOG; + + rlimbs = _limb_size(remainder, other->limb_count + 1); + /* non-zero */ + if (!(rlimbs == 1 && remainder[0] == 0)) { + exp -= LOG; + result->limb_count ++; + result->ob_size += LOG; + _limb_move_left(result->limbs, result->limb_count, 1); + result->limbs[0] = 1; + } + + old_size = result->ob_size; + result->ob_size = _limb_size_s(result->limbs, result->ob_size); + result->limb_count = (result->ob_size + LOG -1)/ LOG; + + result->exp = exp; + + max_size = self->ob_size > other->ob_size ? self->ob_size : other->ob_size; + + /* adjusted is computed just like in the python code */ + adjusted = self->ob_size - other->ob_size + 1; + for (i = 0;i < max_size ;i++) { + long x1,x2; + x1 = _limb_get_digit(self->limbs, self->ob_size, i); + x2 = _limb_get_digit(other->limbs, other->ob_size, i); + if (x2 > x1) { + adjusted --; + break; + } + } + assert(self->ob_size == _limb_size_s(self->limbs, self->ob_size)); + assert(other->ob_size == _limb_size_s(other->limbs, other->ob_size)); + /* to be compatible with python implementation, result int must + * have more than adjusted digits =] */ + + + while (result->ob_size > adjusted && + _limb_get_digit(result->limbs, result->ob_size, result->ob_size -1) == 0) { + _limb_cut_one_digit(result->limbs, result->ob_size); + result->ob_size --; + result->exp ++; + } + + + + if (shouldround) { + decimalobject *fixed; + fixed = _decimal_fix(result, ctx); + Py_DECREF(result); + ans = (PyObject*) fixed; + } + else { + ans = (PyObject*) result; + } + PyMem_FREE(remainder); + return ans; + } + + Py_RETURN_NONE; +} + +static PyObject * +decimal__divide(decimalobject *self, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"other", "divmod", "context", 0}; + contextobject *ctx = NULL; + int divmod = 0; + PyObject *other; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|iO:_divide", kwlist, + &other, &divmod, &ctx)) + return NULL; + + if (!ctx) + if (!(ctx = getcontext())) + return NULL; + + if (!PyDecimalContext_Check(ctx)) { + PyErr_SetString(PyExc_TypeError, "context must be Context object"); + return NULL; + } + + return (PyObject*) _do_decimal__divide(self, other, divmod, ctx); +} + static PyMethodDef decimal_methods[] = { + {"_divide", (PyCFunction)decimal__divide, + METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("TODO")}, {"_rescale", (PyCFunction)decimal_rescale, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("asdf")}, + PyDoc_STR("asdf")}, /* TODO */ {"_fix", (PyCFunction)decimal_fix, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("asdf")}, + PyDoc_STR("asdf")}, /* TODO */ {"adjusted", (PyCFunction)decimal_adjusted, METH_NOARGS, PyDoc_STR("Return the adjusted exponent of self.")}, @@ -2821,8 +3365,7 @@ _do_decimal_divide(decimalobject *self, decimalobject *other, contextobject *ctx) { - /* XXX */ - Py_RETURN_NONE; + return _do_decimal__divide(self, other, 0, ctx); } DECIMAL_SPECIAL_2FUNC(decimal_divide) @@ -2878,7 +3421,7 @@ static PyObject * decimal_power(PyObject *self, PyObject *other, PyObject *modulo) { - decimalobject *res; +/* decimalobject *res; contextobject *ctx = getcontext(); if (!ctx) return NULL; @@ -2900,7 +3443,8 @@ ctx); Py_DECREF(other); Py_DECREF(modulo); - return (PyObject *)res; + return (PyObject *)res;*/ + Py_RETURN_NONE; } @@ -4284,7 +4828,7 @@ static PyObject * context_power(contextobject *self, PyObject *args) { - PyObject *a, *b, *c; +/* PyObject *a, *b, *c; decimalobject *dec_a = NULL, *dec_b = NULL, *dec_c = NULL, *res; if (!PyArg_ParseTuple(args, "OO|O:power", &a, &b, &c)) return NULL; @@ -4311,7 +4855,8 @@ Py_DECREF(dec_a); Py_DECREF(dec_b); Py_DECREF(dec_c); - return (PyObject *)res; + return (PyObject *)res;*/ + Py_RETURN_NONE; } From python-checkins at python.org Tue Jun 13 05:30:08 2006 From: python-checkins at python.org (tim.peters) Date: Tue, 13 Jun 2006 05:30:08 +0200 (CEST) Subject: [Python-checkins] r46905 - python/trunk/Lib/difflib.py Message-ID: <20060613033008.561D31E4002@bag.python.org> Author: tim.peters Date: Tue Jun 13 05:30:07 2006 New Revision: 46905 Modified: python/trunk/Lib/difflib.py Log: get_matching_blocks(): rewrote code & comments so they match; added more comments about why it's this way at all; and removed what looked like needless expense (sorting (i, j, k) triples directly should give exactly the same order as sorting (i, (i, j, k)) pairs). Modified: python/trunk/Lib/difflib.py ============================================================================== --- python/trunk/Lib/difflib.py (original) +++ python/trunk/Lib/difflib.py Tue Jun 13 05:30:07 2006 @@ -474,30 +474,30 @@ if self.matching_blocks is not None: return self.matching_blocks la, lb = len(self.a), len(self.b) + self.matching_blocks = matching_blocks = [] - indexed_blocks = [] + # This is most naturally expressed as a recursive algorithm, but + # at least one user bumped into extreme use cases that exceeded + # the recursion limit on their box. So, now we maintain a list + # ('queue`) of blocks we still need to look at, and append partial + # results to `matching_blocks` in a loop; the matches are sorted + # at the end. queue = [(0, la, 0, lb)] while queue: - # builds list of matching blocks covering a[alo:ahi] and - # b[blo:bhi], appending them in increasing order to answer alo, ahi, blo, bhi = queue.pop() - + i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi) # a[alo:i] vs b[blo:j] unknown # a[i:i+k] same as b[j:j+k] # a[i+k:ahi] vs b[j+k:bhi] unknown - i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi) - - if k: + if k: # if k is 0, there was no matching block + matching_blocks.append(x) if alo < i and blo < j: queue.append((alo, i, blo, j)) - indexed_blocks.append((i, x)) if i+k < ahi and j+k < bhi: queue.append((i+k, ahi, j+k, bhi)) - indexed_blocks.sort() - - self.matching_blocks = [elem[1] for elem in indexed_blocks] - self.matching_blocks.append( (la, lb, 0) ) - return self.matching_blocks + matching_blocks.sort() + matching_blocks.append( (la, lb, 0) ) + return matching_blocks def get_opcodes(self): """Return list of 5-tuples describing how to turn a into b. From neal at metaslash.com Tue Jun 13 05:58:18 2006 From: neal at metaslash.com (Neal Norwitz) Date: Mon, 12 Jun 2006 23:58:18 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (80) Message-ID: <20060613035818.GA15597@python.psfb.org> test_builtin leaked [8, 8, 8] references test_exceptions leaked [40, 40, 40] references test_StringIO leaked [4, 4, 4] references test_base64 leaked [14, 14, 14] references test_cfgparser leaked [12, 12, 12] references test_cgi leaked [20, 20, 20] references test_codeccallbacks leaked [108, 108, 108] references test_codecs leaked [38, 38, 38] references test_coercion leaked [106, 106, 106] references test_complex leaked [10, 10, 10] references test_contextlib leaked [10, 10, 10] references test_cookie leaked [22, 22, 22] references test_cookielib leaked [4, 4, 4] references test_copy leaked [4, 4, 4] references test_copy_reg leaked [22, 22, 22] references test_csv leaked [14, 14, 14] references test_ctypes leaked [8, 8, 8] references test_decimal leaked [10679, 9236, 9985] references test_descr leaked [36, 36, 36] references test_dict leaked [13, 13, 13] references test_distutils leaked [8, 8, 8] references test_doctest leaked [157, 157, 157] references test_doctest2 leaked [2, 2, 2] references test_email leaked [12, 12, 12] references test_email_renamed leaked [12, 12, 12] references test_fileinput leaked [6, 6, 6] references test_fork1 leaked [37, 37, 37] references test_generators leaked [12, 12, 12] references test_getargs2 leaked [14, 14, 14] references test_global leaked [6, 6, 6] references test_import leaked [2, 2, 2] references test_inspect leaked [11, 11, 11] references test_list leaked [4, 4, 4] references test_mailbox leaked [656, 656, 656] references test_minidom leaked [4, 4, 4] references test_operator leaked [16, 16, 16] references test_os leaked [15, 15, 15] references test_pep292 leaked [37, 37, 37] references test_pep352 leaked [48, 48, 48] references test_pickletools leaked [12, 12, 12] references test_posix leaked [8, 8, 8] references test_profilehooks leaked [1, 1, 1] references test_pyexpat leaked [2, 2, 2] references test_queue leaked [32, 34, 34] references test_random leaked [18, 18, 18] references test_repr leaked [2, 2, 2] references test_runpy leaked [8, 8, 8] references test_scope leaked [14, 14, 14] references test_sgmllib leaked [2, 2, 2] references test_shelve leaked [6, 6, 6] references test_socket leaked [4250, 4594, 4088] references test_socket_ssl leaked [1, 4, 4] references test_socketserver leaked [135, 135, 135] references test_sqlite leaked [31, 14, 58] references test_strop leaked [88, 88, 88] references test_strptime leaked [10, 10, 10] references test_struct leaked [3782, 3890, 3854] references test_subprocess leaked [9, 9, 9] references test_syntax leaked [2, 2, 2] references test_sys leaked [2, 2, 2] references test_tarfile leaked [452, 452, 452] references test_textwrap leaked [4, 4, 4] references test_thread leaked [53, 52, 52] references test_threadedtempfile leaked [35, 37, 35] references test_threading leaked [38, 39, 37] references test_threadsignals leaked [0, 0, -8] references test_tokenize leaked [4, 4, 4] references test_traceback leaked [1, 1, 1] references test_urllib2 leaked [3, 3, 3] references test_userdict leaked [29, 29, 29] references test_userlist leaked [4, 4, 4] references test_uu leaked [6, 6, 6] references test_wait3 leaked [37, 37, 37] references test_wait4 leaked [33, 37, 37] references test_warnings leaked [32, 32, 32] references test_weakref leaked [2, 2, 2] references test_with leaked [12, 12, 12] references test_wsgiref leaked [6, 6, 6] references test_xml_etree leaked [2, 2, 2] references test_xrange leaked [2, 2, 2] references From python-checkins at python.org Tue Jun 13 06:08:54 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 13 Jun 2006 06:08:54 +0200 (CEST) Subject: [Python-checkins] r46906 - python/trunk/Lib/test/test_asynchat.py Message-ID: <20060613040854.4C76E1E4002@bag.python.org> Author: neal.norwitz Date: Tue Jun 13 06:08:53 2006 New Revision: 46906 Modified: python/trunk/Lib/test/test_asynchat.py Log: Don't fail if another process is listening on our port. Modified: python/trunk/Lib/test/test_asynchat.py ============================================================================== --- python/trunk/Lib/test/test_asynchat.py (original) +++ python/trunk/Lib/test/test_asynchat.py Tue Jun 13 06:08:53 2006 @@ -13,7 +13,8 @@ def run(self): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - sock.bind((HOST, PORT)) + global PORT + PORT = test_support.bind_port(sock, HOST, PORT) sock.listen(1) conn, client = sock.accept() buffer = "" From neal at metaslash.com Tue Jun 13 06:13:09 2006 From: neal at metaslash.com (Neal Norwitz) Date: Tue, 13 Jun 2006 00:13:09 -0400 Subject: [Python-checkins] Python Regression Test Failures all (1) Message-ID: <20060613041309.GA17329@python.psfb.org> test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat Exception in thread Thread-1: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') Exception in thread Thread-2: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/test/test_asynchat.py", line 16, in run sock.bind((HOST, PORT)) File "", line 1, in bind error: (98, 'Address already in use') test test_asynchat failed -- errors occurred in test.test_asynchat.TestAsynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8679 refs] [8679 refs] [8679 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9027 refs] [9027 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socketserver test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8890 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] [8674 refs] this bit of output is from a test of stdout in a different process ... [8674 refs] [8674 refs] [8890 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8674 refs] [8675 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8674 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllibnet test_urlparse test_userdict test_userlist test_userstring test_uu test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 291 tests OK. 1 test failed: test_asynchat 21 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl test_gl test_imgfile test_ioctl test_macfs test_macostools test_nis test_pep277 test_plistlib test_scriptpackages test_startfile test_sunaudiodev test_tcl test_unicode_file test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [706515 refs] From neal at metaslash.com Tue Jun 13 06:18:58 2006 From: neal at metaslash.com (Neal Norwitz) Date: Tue, 13 Jun 2006 00:18:58 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (80) Message-ID: <20060613041858.GA28697@python.psfb.org> test_builtin leaked [8, 8, 8] references test_exceptions leaked [40, 40, 40] references test_StringIO leaked [4, 4, 4] references test_base64 leaked [14, 14, 14] references test_cfgparser leaked [12, 12, 12] references test_cgi leaked [20, 20, 20] references test_codeccallbacks leaked [108, 108, 108] references test_codecs leaked [38, 38, 38] references test_coercion leaked [106, 106, 106] references test_complex leaked [10, 10, 10] references test_contextlib leaked [10, 10, 10] references test_cookie leaked [22, 22, 22] references test_cookielib leaked [4, 4, 4] references test_copy leaked [4, 4, 4] references test_copy_reg leaked [22, 22, 22] references test_csv leaked [14, 14, 14] references test_ctypes leaked [8, 8, 8] references test_decimal leaked [10679, 9236, 9985] references test_descr leaked [36, 36, 36] references test_dict leaked [13, 13, 13] references test_distutils leaked [8, 8, 8] references test_doctest leaked [157, 157, 157] references test_doctest2 leaked [2, 2, 2] references test_email leaked [12, 12, 12] references test_email_renamed leaked [12, 12, 12] references test_fileinput leaked [6, 6, 6] references test_fork1 leaked [37, 37, 37] references test_generators leaked [12, 12, 12] references test_getargs2 leaked [14, 14, 14] references test_global leaked [6, 6, 6] references test_import leaked [2, 2, 2] references test_inspect leaked [11, 11, 11] references test_list leaked [4, 4, 4] references test_mailbox leaked [656, 656, 656] references test_minidom leaked [4, 4, 4] references test_operator leaked [16, 16, 16] references test_os leaked [15, 15, 15] references test_pep292 leaked [37, 37, 37] references test_pep352 leaked [48, 48, 48] references test_pickletools leaked [12, 12, 12] references test_posix leaked [8, 8, 8] references test_profilehooks leaked [1, 1, 1] references test_pyexpat leaked [2, 2, 2] references test_queue leaked [32, 34, 34] references test_random leaked [18, 18, 18] references test_repr leaked [2, 2, 2] references test_runpy leaked [8, 8, 8] references test_scope leaked [14, 14, 14] references test_sgmllib leaked [2, 2, 2] references test_shelve leaked [6, 6, 6] references test_socket leaked [4250, 4594, 4088] references test_socket_ssl leaked [1, 4, 4] references test_socketserver leaked [135, 135, 135] references test_sqlite leaked [31, 14, 58] references test_strop leaked [88, 88, 88] references test_strptime leaked [10, 10, 10] references test_struct leaked [3782, 3890, 3854] references test_subprocess leaked [9, 9, 9] references test_syntax leaked [2, 2, 2] references test_sys leaked [2, 2, 2] references test_tarfile leaked [452, 452, 452] references test_textwrap leaked [4, 4, 4] references test_thread leaked [53, 52, 52] references test_threadedtempfile leaked [35, 37, 35] references test_threading leaked [38, 39, 37] references test_threadsignals leaked [0, 0, -8] references test_tokenize leaked [4, 4, 4] references test_traceback leaked [1, 1, 1] references test_urllib2 leaked [3, 3, 3] references test_userdict leaked [29, 29, 29] references test_userlist leaked [4, 4, 4] references test_uu leaked [6, 6, 6] references test_wait3 leaked [37, 37, 37] references test_wait4 leaked [33, 37, 37] references test_warnings leaked [32, 32, 32] references test_weakref leaked [2, 2, 2] references test_with leaked [12, 12, 12] references test_wsgiref leaked [6, 6, 6] references test_xml_etree leaked [2, 2, 2] references test_xrange leaked [2, 2, 2] references From nnorwitz at gmail.com Tue Jun 13 06:25:06 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 12 Jun 2006 21:25:06 -0700 Subject: [Python-checkins] r46904 - sandbox/trunk/decimal-c/_decimal.c In-Reply-To: <20060613030147.91F581E4002@bag.python.org> References: <20060613030147.91F581E4002@bag.python.org> Message-ID: On 6/12/06, mateusz.rukowicz wrote: > > Real dividing now works and passes all tests, integer not yet implemented. Commented 2 functions causing sigsegv. > static PyObject * > context_power(contextobject *self, PyObject *args) > { > - PyObject *a, *b, *c; > +/* PyObject *a, *b, *c; > decimalobject *dec_a = NULL, *dec_b = NULL, *dec_c = NULL, *res; > if (!PyArg_ParseTuple(args, "OO|O:power", &a, &b, &c)) > return NULL; > @@ -4311,7 +4855,8 @@ > Py_DECREF(dec_a); > Py_DECREF(dec_b); > Py_DECREF(dec_c); > - return (PyObject *)res; > + return (PyObject *)res;*/ > + Py_RETURN_NONE; > } In C/C++ the easier way to "comment out" code like this is to use #if 0. It is easy to search for since it won't be confused with a real comment. It also handles embedded comments just fine; #if 0 code that wouldn't even compile otherwise /* comments are fine in here too */ #endif n From buildbot at python.org Tue Jun 13 07:26:18 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 05:26:18 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060613052619.057851E4002@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/650 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 07:27:14 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 13 Jun 2006 07:27:14 +0200 (CEST) Subject: [Python-checkins] r46907 - peps/trunk/pep-0356.txt Message-ID: <20060613052714.565311E4002@bag.python.org> Author: neal.norwitz Date: Tue Jun 13 07:27:13 2006 New Revision: 46907 Modified: peps/trunk/pep-0356.txt Log: The schedule is mostly fixed, except for personnel availability. uuid and wsgiref were added to the stdlib. Reflow long lines. Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Tue Jun 13 07:27:13 2006 @@ -32,13 +32,8 @@ Release Schedule - Note that this schedule is completely tentative. The number of alphas, - betas and release candidates will be determined as the release process - unfolds. - alpha 1: April 5, 2006 [completed] alpha 2: April 27, 2006 [completed] - alpha 3: ------------- [skipped] beta 1: June 14, 2006 [planned] beta 2: July 6, 2006 [planned] rc 1: July 27, 2006 [planned] @@ -83,6 +78,10 @@ - pysqlite + - uuid + + - wsgiref + Other notable features - Added support for reading shadow passwords (http://python.org/sf/579435) @@ -102,8 +101,9 @@ - Add new icons for Windows with the new Python logo? - - New utilities in functools to help write wrapper functions that support naive - introspection (e.g. having f.__name__ return the original function name). + - New utilities in functools to help write wrapper functions that + support naive introspection (e.g. having f.__name__ return + the original function name). Possible features for 2.5 @@ -112,10 +112,6 @@ - Modules under consideration for inclusion: - - wsgiref to the standard library - SVN repository: svn://svn.eby-sarna.com/svnroot/wsgiref - (Owner: Phillip Eby) - - Upgrade pyexpat to use expat 2.0? (Patch by Trent Mick, #1462338) From buildbot at python.org Tue Jun 13 07:28:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 05:28:02 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060613052802.B70661E4002@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/169 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 07:37:00 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 05:37:00 +0000 Subject: [Python-checkins] buildbot failure in sparc Ubuntu dapper trunk Message-ID: <20060613053700.EC12C1E4002@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/400 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,tim.peters BUILD FAILED: failed svn sincerely, -The Buildbot From nnorwitz at gmail.com Tue Jun 13 08:58:25 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 12 Jun 2006 23:58:25 -0700 Subject: [Python-checkins] r46795 - in python/trunk: Doc/lib/libstdtypes.tex Lib/test/string_tests.py Misc/NEWS Objects/stringobject.c Objects/unicodeobject.c In-Reply-To: <20060609184549.B5ABC1E400A@bag.python.org> References: <20060609184549.B5ABC1E400A@bag.python.org> Message-ID: On 6/9/06, georg.brandl wrote: > Author: georg.brandl > Date: Fri Jun 9 20:45:48 2006 > New Revision: 46795 > > Log: > RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. What's the reason to not support any sequence and only support tuples? Are there any other APIs that only support tuples rather than all sequences? n From tim.peters at gmail.com Tue Jun 13 09:24:41 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 13 Jun 2006 03:24:41 -0400 Subject: [Python-checkins] r46795 - in python/trunk: Doc/lib/libstdtypes.tex Lib/test/string_tests.py Misc/NEWS Objects/stringobject.c Objects/unicodeobject.c In-Reply-To: References: <20060609184549.B5ABC1E400A@bag.python.org> Message-ID: <1f7befae0606130024x4eff304fo7efc585cb3ddfdcc@mail.gmail.com> [georg.brandl] >> Author: georg.brandl >> Date: Fri Jun 9 20:45:48 2006 >> New Revision: 46795 >> >> Log: >> RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. [Neal Norwitz] > What's the reason to not support any sequence and only support tuples? It can't support any sequence, else e.g. s.endswith(".py") would be ambiguous. > Are there any other APIs that only support tuples rather than all > sequences? Oh, who cares -- it would be a relief to leave _something_ simple <0.7 wink>. It's a bit of a stretch, but, e.g., >>> isinstance(1, [int, str]) Traceback (most recent call last): File "", line 1, in TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types is restricted to tuple containers; ditto issubclass. They're similar in that most expected uses involve small, constant collections. Given that {start,end}swidth can't support all sequences regardless, restricting it to tuples is OK by me, and was clearly sufficient for the uses made of this already in the standard library. From nnorwitz at gmail.com Tue Jun 13 09:38:42 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 13 Jun 2006 00:38:42 -0700 Subject: [Python-checkins] [Python-Dev] r46795 - in python/trunk: Doc/lib/libstdtypes.tex Lib/test/string_tests.py Misc/NEWS Objects/stringobject.c Objects/unicodeobject.c In-Reply-To: <1f7befae0606130024x4eff304fo7efc585cb3ddfdcc@mail.gmail.com> References: <20060609184549.B5ABC1E400A@bag.python.org> <1f7befae0606130024x4eff304fo7efc585cb3ddfdcc@mail.gmail.com> Message-ID: On 6/13/06, Tim Peters wrote: > [georg.brandl] > >> Author: georg.brandl > >> Date: Fri Jun 9 20:45:48 2006 > >> New Revision: 46795 > >> > >> Log: > >> RFE #1491485: str/unicode.endswith()/startswith() now accept a > tuple as first argument. > > [Neal Norwitz] > > What's the reason to not support any sequence and only support tuples? > > It can't support any sequence, else e.g. s.endswith(".py") would be ambiguous. Good point, I was really just thinking of lists though. > > Are there any other APIs that only support tuples rather than all > > sequences? > > Oh, who cares -- it would be a relief to leave _something_ simple <0.7 > wink>. :-) I was thinking about a use case like this: supported_suffixes = ['.py', '.pyc', '.pyo'] if sys.platform[:3] == 'win': supported_suffixes.append('.pyw') if pathname.endswith(supported_suffixes): # ... I realize you could just throw a tuple(supported_suffixes). I don't know that I've ever needed that specific use case. I don't think it's too common. Just thinking about avoiding newbie surprises. > Given that {start,end}swidth can't support all sequences regardless, > restricting it to tuples is OK by me, and was clearly sufficient for > the uses made of this already in the standard library. I'm ok with this either way. It's easy enough to add later, but hard to take away. n From python-checkins at python.org Tue Jun 13 10:28:20 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 13 Jun 2006 10:28:20 +0200 (CEST) Subject: [Python-checkins] r46908 - python/trunk/Objects/unicodeobject.c Message-ID: <20060613082820.253171E4005@bag.python.org> Author: neal.norwitz Date: Tue Jun 13 10:28:19 2006 New Revision: 46908 Modified: python/trunk/Objects/unicodeobject.c Log: Initialize the type object so pychecker can't crash the interpreter. Modified: python/trunk/Objects/unicodeobject.c ============================================================================== --- python/trunk/Objects/unicodeobject.c (original) +++ python/trunk/Objects/unicodeobject.c Tue Jun 13 10:28:19 2006 @@ -7792,6 +7792,8 @@ bloom_linebreak = make_bloom_mask( linebreak, sizeof(linebreak) / sizeof(linebreak[0]) ); + + PyType_Ready(&EncodingMapType); } /* Finalize the Unicode implementation */ From python-checkins at python.org Tue Jun 13 10:41:07 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 13 Jun 2006 10:41:07 +0200 (CEST) Subject: [Python-checkins] r46909 - python/trunk/Lib/test/test_codecs.py Message-ID: <20060613084107.AA6931E4006@bag.python.org> Author: neal.norwitz Date: Tue Jun 13 10:41:06 2006 New Revision: 46909 Modified: python/trunk/Lib/test/test_codecs.py Log: Verify the crash due to EncodingMap not initialized does not return Modified: python/trunk/Lib/test/test_codecs.py ============================================================================== --- python/trunk/Lib/test/test_codecs.py (original) +++ python/trunk/Lib/test/test_codecs.py Tue Jun 13 10:41:06 2006 @@ -1166,6 +1166,12 @@ encoder = codecs.getencoder(encoding) self.assertRaises(TypeError, encoder) + def test_encoding_map_type_initialized(self): + from encodings import cp1140 + # This used to crash, we are only verifying there's no crash. + table_type = type(cp1140.encoding_table) + self.assertEqual(table_type, table_type) + class BasicStrTest(unittest.TestCase): def test_basics(self): s = "abc123" From python-checkins at python.org Tue Jun 13 10:56:14 2006 From: python-checkins at python.org (thomas.heller) Date: Tue, 13 Jun 2006 10:56:14 +0200 (CEST) Subject: [Python-checkins] r46910 - python/trunk/Lib/ctypes/wintypes.py Message-ID: <20060613085614.815091E4005@bag.python.org> Author: thomas.heller Date: Tue Jun 13 10:56:14 2006 New Revision: 46910 Modified: python/trunk/Lib/ctypes/wintypes.py Log: Add some windows datatypes that were missing from this file, and add the aliases defined in windows header files for the structures. Modified: python/trunk/Lib/ctypes/wintypes.py ============================================================================== --- python/trunk/Lib/ctypes/wintypes.py (original) +++ python/trunk/Lib/ctypes/wintypes.py Tue Jun 13 10:56:14 2006 @@ -1,63 +1,107 @@ ###################################################################### # This file should be kept compatible with Python 2.3, see PEP 291. # ###################################################################### -# XXX This module needs cleanup. +# The most useful windows datatypes from ctypes import * -DWORD = c_ulong -WORD = c_ushort BYTE = c_byte +WORD = c_ushort +DWORD = c_ulong + +BOOLEAN = BYTE +BOOL = c_long +VARIANT_BOOL = c_short ULONG = c_ulong LONG = c_long +# in the windows header files, these are structures. LARGE_INTEGER = c_longlong ULARGE_INTEGER = c_ulonglong +LPCOLESTR = LPOLESTR = OLESTR = c_wchar_p +LPCWSTR = LPWSTR = c_wchar_p +LPCSTR = LPSTR = c_char_p +WPARAM = c_uint +LPARAM = c_long + +ATOM = WORD +LANGID = WORD + +COLORREF = DWORD +LGRPID = DWORD +LCTYPE = DWORD + +LCID = DWORD + +################################################################ +# HANDLE types HANDLE = c_ulong # in the header files: void * -HWND = HANDLE +HACCEL = HANDLE +HBITMAP = HANDLE +HBRUSH = HANDLE +HCOLORSPACE = HANDLE HDC = HANDLE -HMODULE = HANDLE +HDESK = HANDLE +HDWP = HANDLE +HENHMETAFILE = HANDLE +HFONT = HANDLE +HGDIOBJ = HANDLE +HGLOBAL = HANDLE +HHOOK = HANDLE +HICON = HANDLE HINSTANCE = HANDLE -HRGN = HANDLE -HTASK = HANDLE HKEY = HANDLE -HPEN = HANDLE -HGDIOBJ = HANDLE +HKL = HANDLE +HLOCAL = HANDLE HMENU = HANDLE +HMETAFILE = HANDLE +HMODULE = HANDLE +HMONITOR = HANDLE +HPALETTE = HANDLE +HPEN = HANDLE +HRGN = HANDLE +HRSRC = HANDLE +HSTR = HANDLE +HTASK = HANDLE +HWINSTA = HANDLE +HWND = HANDLE +SC_HANDLE = HANDLE +SERVICE_STATUS_HANDLE = HANDLE -LCID = DWORD - -WPARAM = c_uint -LPARAM = c_long - -BOOL = c_long -VARIANT_BOOL = c_short - -LPCOLESTR = LPOLESTR = OLESTR = c_wchar_p -LPCWSTR = LPWSTR = c_wchar_p - -LPCSTR = LPSTR = c_char_p +################################################################ +# Some important structure definitions class RECT(Structure): _fields_ = [("left", c_long), ("top", c_long), ("right", c_long), ("bottom", c_long)] -RECTL = RECT +tagRECT = _RECTL = RECTL = RECT + +class _SMALL_RECT(Structure): + _fields_ = [('Left', c_short), + ('Top', c_short), + ('Right', c_short), + ('Bottom', c_short)] +SMALL_RECT = _SMALL_RECT + +class _COORD(Structure): + _fields_ = [('X', c_short), + ('Y', c_short)] class POINT(Structure): _fields_ = [("x", c_long), ("y", c_long)] -POINTL = POINT +tagPOINT = _POINTL = POINTL = POINT class SIZE(Structure): _fields_ = [("cx", c_long), ("cy", c_long)] -SIZEL = SIZE +tagSIZE = SIZEL = SIZE def RGB(red, green, blue): return red + (green << 8) + (blue << 16) @@ -65,6 +109,7 @@ class FILETIME(Structure): _fields_ = [("dwLowDateTime", DWORD), ("dwHighDateTime", DWORD)] +_FILETIME = FILETIME class MSG(Structure): _fields_ = [("hWnd", HWND), @@ -73,6 +118,7 @@ ("lParam", LPARAM), ("time", DWORD), ("pt", POINT)] +tagMSG = MSG MAX_PATH = 260 class WIN32_FIND_DATAA(Structure): From mwh at python.net Tue Jun 13 11:24:35 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 13 Jun 2006 10:24:35 +0100 Subject: [Python-checkins] Python Regression Test Failures refleak (80) In-Reply-To: <20060613035818.GA15597@python.psfb.org> (Neal Norwitz's message of "Mon, 12 Jun 2006 23:58:18 -0400") References: <20060613035818.GA15597@python.psfb.org> Message-ID: <2mwtblo1bg.fsf@starship.python.net> Neal Norwitz writes: > test_builtin leaked [8, 8, 8] references > test_exceptions leaked [40, 40, 40] references So who broke refcounting then? This is pretty new and must be somewhere fairly core, hopefully someone remembers changing something there? I don't remember any specific checkins though. Cheers, mwh -- On a scale of One to AWESOME, twisted.web is PRETTY ABSTRACT!!!! -- from Twisted.Quotes From python-checkins at python.org Tue Jun 13 11:40:15 2006 From: python-checkins at python.org (thomas.heller) Date: Tue, 13 Jun 2006 11:40:15 +0200 (CEST) Subject: [Python-checkins] r46911 - python/trunk/Lib/ctypes/wintypes.py Message-ID: <20060613094015.21A481E4005@bag.python.org> Author: thomas.heller Date: Tue Jun 13 11:40:14 2006 New Revision: 46911 Modified: python/trunk/Lib/ctypes/wintypes.py Log: Add back WCHAR, UINT, DOUBLE, _LARGE_INTEGER, _ULARGE_INTEGER. VARIANT_BOOL is a special _ctypes data type, not c_short. Modified: python/trunk/Lib/ctypes/wintypes.py ============================================================================== --- python/trunk/Lib/ctypes/wintypes.py (original) +++ python/trunk/Lib/ctypes/wintypes.py Tue Jun 13 11:40:14 2006 @@ -9,16 +9,26 @@ WORD = c_ushort DWORD = c_ulong +WCHAR = c_wchar +UINT = c_uint + +DOUBLE = c_double + BOOLEAN = BYTE BOOL = c_long -VARIANT_BOOL = c_short + +from ctypes import _SimpleCData +class VARIANT_BOOL(_SimpleCData): + _type_ = "v" + def __repr__(self): + return "%s(%r)" % (self.__class__.__name__, self.value) ULONG = c_ulong LONG = c_long # in the windows header files, these are structures. -LARGE_INTEGER = c_longlong -ULARGE_INTEGER = c_ulonglong +_LARGE_INTEGER = LARGE_INTEGER = c_longlong +_ULARGE_INTEGER = ULARGE_INTEGER = c_ulonglong LPCOLESTR = LPOLESTR = OLESTR = c_wchar_p LPCWSTR = LPWSTR = c_wchar_p From buildbot at python.org Tue Jun 13 13:14:26 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 11:14:26 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060613111426.8494B1E4005@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/749 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 13:19:57 2006 From: python-checkins at python.org (ronald.oussoren) Date: Tue, 13 Jun 2006 13:19:57 +0200 (CEST) Subject: [Python-checkins] r46912 - python/trunk/Lib/linecache.py Message-ID: <20060613111957.7EB561E4005@bag.python.org> Author: ronald.oussoren Date: Tue Jun 13 13:19:56 2006 New Revision: 46912 Modified: python/trunk/Lib/linecache.py Log: Linecache contains support for PEP302 loaders, but fails to deal with loaders that return None to indicate that the module is valid but no source is available. This patch fixes that. Modified: python/trunk/Lib/linecache.py ============================================================================== --- python/trunk/Lib/linecache.py (original) +++ python/trunk/Lib/linecache.py Tue Jun 13 13:19:56 2006 @@ -94,6 +94,10 @@ except (ImportError, IOError): pass else: + if data is None: + # No luck, the PEP302 loader cannot find the source + # for this module. + return [] cache[filename] = ( len(data), None, [line+'\n' for line in data.splitlines()], fullname From python-checkins at python.org Tue Jun 13 13:57:06 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 13 Jun 2006 13:57:06 +0200 (CEST) Subject: [Python-checkins] r46913 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060613115706.8F3ED1E4005@bag.python.org> Author: andrew.kuchling Date: Tue Jun 13 13:57:04 2006 New Revision: 46913 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Mention uuid module Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 13 13:57:04 2006 @@ -1636,6 +1636,37 @@ by some specifications, so it's still available as \member{unicodedata.ucd_3_2_0}. +\item New module: the \module{uuid} module generates +universally unique identifiers (UUIDs) according to \rfc{4122}. The +RFC defines several different UUID versions that are generated from a +starting string, from system properties, or purely randomly. This +module contains a \class{UUID} class and +functions named \function{uuid1()}, +\function{uuid3()}, \function{uuid4()}, and +\function{uuid5()} to generate different versions of UUID. (Version 2 UUIDs +are not specified in \rfc{4122} and are not supported by this module.) + +\begin{verbatim} +>>> import uuid +>>> # make a UUID based on the host ID and current time +>>> uuid.uuid1() +UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + +>>> # make a UUID using an MD5 hash of a namespace UUID and a name +>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') +UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + +>>> # make a random UUID +>>> uuid.uuid4() +UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + +>>> # make a UUID using a SHA-1 hash of a namespace UUID and a name +>>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') +UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') +\end{verbatim} + +(Contributed by Ka-Ping Yee.) + \item The \module{webbrowser} module received a number of enhancements. It's now usable as a script with \code{python -m webbrowser}, taking a From python-checkins at python.org Tue Jun 13 14:00:33 2006 From: python-checkins at python.org (andrew.macintyre) Date: Tue, 13 Jun 2006 14:00:33 +0200 (CEST) Subject: [Python-checkins] r46914 - in python/branches/aimacintyre-sf1454481: Doc/lib/libthread.tex Doc/lib/libthreading.tex Lib/test/output/test_thread Lib/test/test_thread.py Modules/threadmodule.c Python/thread.c Python/thread_nt.h Python/thread_os2.h Python/thread_pthread.h Message-ID: <20060613120033.25BB11E4005@bag.python.org> Author: andrew.macintyre Date: Tue Jun 13 14:00:30 2006 New Revision: 46914 Modified: python/branches/aimacintyre-sf1454481/Doc/lib/libthread.tex python/branches/aimacintyre-sf1454481/Doc/lib/libthreading.tex python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py python/branches/aimacintyre-sf1454481/Modules/threadmodule.c python/branches/aimacintyre-sf1454481/Python/thread.c python/branches/aimacintyre-sf1454481/Python/thread_nt.h python/branches/aimacintyre-sf1454481/Python/thread_os2.h python/branches/aimacintyre-sf1454481/Python/thread_pthread.h Log: Rework of SF #1454481 to address issues on Solaris, with error handling and other cleanup Modified: python/branches/aimacintyre-sf1454481/Doc/lib/libthread.tex ============================================================================== --- python/branches/aimacintyre-sf1454481/Doc/lib/libthread.tex (original) +++ python/branches/aimacintyre-sf1454481/Doc/lib/libthread.tex Tue Jun 13 14:00:30 2006 @@ -79,16 +79,17 @@ optional \var{size} argument specifies the stack size to be used for subsequently created threads, and must be 0 (use platform or configured default) or a positive integer value of at least 32,768 (32kB). -If changing the thread stack size is unsupported, or the specified size -is invalid, a RuntimeWarning is issued and the stack size is unmodified. -32kB is currently the minimum supported stack size value, to guarantee -sufficient stack space for the interpreter itself. -Note that some platforms may have particular restrictions on values for -the stack size, such as requiring allocation in multiples of the system -memory page size - platform documentation should be referred to for more -information (4kB pages are common; using multiples of 4096 for the -stack size is the suggested approach in the absence of more specific -information). +If changing the thread stack size is unsupported, a \exception{ThreadError} +is raised. If the specified stack size is invalid, a \exception{ValueError} +is raised and the stack size is unmodified. 32kB is currently the minimum +supported stack size value to guarantee sufficient stack space for the +interpreter itself. Note that some platforms may have particular +restrictions on values for the stack size, such as requiring a minimum +stack size > 32kB or requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for +more information (4kB pages are common; using multiples of 4096 for +the stack size is the suggested approach in the absence of more +specific information). Availability: Windows, systems with \POSIX{} threads. \versionadded{2.5} \end{funcdesc} Modified: python/branches/aimacintyre-sf1454481/Doc/lib/libthreading.tex ============================================================================== --- python/branches/aimacintyre-sf1454481/Doc/lib/libthreading.tex (original) +++ python/branches/aimacintyre-sf1454481/Doc/lib/libthreading.tex Tue Jun 13 14:00:30 2006 @@ -130,16 +130,17 @@ optional \var{size} argument specifies the stack size to be used for subsequently created threads, and must be 0 (use platform or configured default) or a positive integer value of at least 32,768 (32kB). -If changing the thread stack size is unsupported, or the specified size -is invalid, a RuntimeWarning is issued and the stack size is unmodified. -32kB is currently the minimum supported stack size value, to guarantee -sufficient stack space for the interpreter itself. -Note that some platforms may have particular restrictions on values for -the stack size, such as requiring allocation in multiples of the system -memory page size - platform documentation should be referred to for more -information (4kB pages are common; using multiples of 4096 for the -stack size is the suggested approach in the absence of more specific -information). +If changing the thread stack size is unsupported, a \exception{ThreadError} +is raised. If the specified stack size is invalid, a \exception{ValueError} +is raised and the stack size is unmodified. 32kB is currently the minimum +supported stack size value to guarantee sufficient stack space for the +interpreter itself. Note that some platforms may have particular +restrictions on values for the stack size, such as requiring a minimum +stack size > 32kB or requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for +more information (4kB pages are common; using multiples of 4096 for +the stack size is the suggested approach in the absence of more +specific information). Availability: Windows, systems with \POSIX{} threads. \versionadded{2.5} \end{funcdesc} Modified: python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread Tue Jun 13 14:00:30 2006 @@ -6,6 +6,10 @@ all tasks done *** Changing thread stack size *** +caught expected ValueError setting stack_size(4096) +successfully set stack_size(32768) +successfully set stack_size(4194304) +successfully set stack_size(0) trying stack_size = 32768 waiting for all tasks to complete all tasks done Modified: python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py Tue Jun 13 14:00:30 2006 @@ -128,25 +128,33 @@ from os import name as os_name if os_name in ("nt", "os2", "posix"): - for tss, ok in ((4096, 0), (32768, 1), (0x400000, 1), (0, 1)): - if ok: - failed = lambda s, e: s != e - fail_msg = "stack_size(%d) failed - should succeed" - else: - failed = lambda s, e: s == e - fail_msg = "stack_size(%d) succeeded - should fail" - thread.stack_size(tss) - if failed(thread.stack_size(), tss): - raise ValueError, fail_msg % tss - - for tss in (32768, 0x400000): - print 'trying stack_size = %d' % tss - next_ident = 0 - for i in range(numtasks): - newtask() + tss_supported = 1 + try: + thread.stack_size(4096) + except ValueError: + print 'caught expected ValueError setting stack_size(4096)' + except thread.ThreadError: + tss_supported = 0 + print 'platform does not support changing thread stack size' - print 'waiting for all tasks to complete' - done.acquire() - print 'all tasks done' + if tss_supported: + failed = lambda s, e: s != e + fail_msg = "stack_size(%d) failed - should succeed" + for tss in (32768, 0x400000, 0): + thread.stack_size(tss) + if failed(thread.stack_size(), tss): + raise ValueError, fail_msg % tss + print 'successfully set stack_size(%d)' % tss - thread.stack_size(0) + for tss in (32768, 0x400000): + print 'trying stack_size = %d' % tss + next_ident = 0 + for i in range(numtasks): + newtask() + + print 'waiting for all tasks to complete' + done.acquire() + print 'all tasks done' + + # reset stack size to default + thread.stack_size(0) Modified: python/branches/aimacintyre-sf1454481/Modules/threadmodule.c ============================================================================== --- python/branches/aimacintyre-sf1454481/Modules/threadmodule.c (original) +++ python/branches/aimacintyre-sf1454481/Modules/threadmodule.c Tue Jun 13 14:00:30 2006 @@ -589,27 +589,36 @@ static PyObject * thread_stack_size(PyObject *self, PyObject *args) { - size_t old_size, new_size; + size_t old_size; + Py_ssize_t new_size = 0; PyObject *set_size = NULL; + int rc; - if (!PyArg_UnpackTuple(args, "stack_size", 0, 1, &set_size)) + if (!PyArg_ParseTuple(args, "|n:stack_size", &new_size)) return NULL; + if (new_size < 0) { + PyErr_SetString(PyExc_ValueError, + "size must be 0 or a positive value"); + return NULL; + } + old_size = PyThread_get_stacksize(); - if (set_size != NULL) { - if (PyInt_Check(set_size)) - new_size = (size_t) PyInt_AsLong(set_size); - else { - PyErr_SetString(PyExc_TypeError, - "size must be an integer"); - return NULL; - } - if (PyThread_set_stacksize(new_size)) - return NULL; + rc = PyThread_set_stacksize((size_t) new_size); + if (rc == -1) { + PyErr_Format(PyExc_ValueError, + "size not valid: %zd bytes", + new_size); + return NULL; + } + if (rc == -2) { + PyErr_SetString(ThreadError, + "setting stack size not supported"); + return NULL; } - return PyInt_FromLong((long) old_size); + return PyInt_FromSsize_t((Py_ssize_t) old_size); } PyDoc_STRVAR(stack_size_doc, @@ -618,18 +627,19 @@ Return the thread stack size used when creating new threads. The\n\ optional size argument specifies the stack size (in bytes) to be used\n\ for subsequently created threads, and must be 0 (use platform or\n\ -configured default) or a positive integer value of at least 32,768 (32kB).\n\ -If changing the thread stack size is unsupported, or the specified size\n\ -is invalid, a RuntimeWarning is issued and the stack size is unmodified.\n\ -32kB is currently the minimum supported stack size value, to guarantee\n\ +configured default) or a positive integer value of at least 32,768 (32k).\n\ +If changing the thread stack size is unsupported, a ThreadError\n\ +exception is raised. If the specified size is invalid, a ValueError\n\ +exception is raised, and the stack size is unmodified. 32k bytes\n\ + currently the minimum supported stack size value to guarantee\n\ sufficient stack space for the interpreter itself.\n\ \n\ Note that some platforms may have particular restrictions on values for\n\ -the stack size, such as requiring allocation in multiples of the system\n\ -memory page size - platform documentation should be referred to for more\n\ -information (4kB pages are common; using multiples of 4096 for the\n\ -stack size is the suggested approach in the absence of more specific\n\ -information)."); +the stack size, such as requiring a minimum stack size larger than 32kB or\n\ +requiring allocation in multiples of the system memory page size\n\ +- platform documentation should be referred to for more information\n\ +(4kB pages are common; using multiples of 4096 for the stack size is\n\ +the suggested approach in the absence of more specific information)."); static PyMethodDef thread_methods[] = { {"start_new_thread", (PyCFunction)thread_PyThread_start_new_thread, Modified: python/branches/aimacintyre-sf1454481/Python/thread.c ============================================================================== --- python/branches/aimacintyre-sf1454481/Python/thread.c (original) +++ python/branches/aimacintyre-sf1454481/Python/thread.c Tue Jun 13 14:00:30 2006 @@ -99,26 +99,6 @@ or the size specified by the THREAD_STACK_SIZE macro. */ static size_t _pythread_stacksize = 0; -size_t -PyThread_get_stacksize(void) -{ - return _pythread_stacksize; -} - -static int -_pythread_unsupported_set_stacksize(size_t size) -{ - return PyErr_Warn(PyExc_RuntimeWarning, - "setting thread stack size not supported on " - "this platform"); -} - -/* Only platforms with THREAD_SET_STACKSIZE() defined in - pthread_.h, overriding this default definition, - will support changing the stack size. - Return 1 if an exception is pending, 0 otherwise. */ -#define THREAD_SET_STACKSIZE(x) _pythread_unsupported_set_stacksize(x) - #ifdef SGI_THREADS #include "thread_sgi.h" #endif @@ -174,12 +154,26 @@ #endif */ -/* use appropriate thread stack size setting routine. - Return 1 if an exception is pending, 0 otherwise. */ +/* return the current thread stack size */ +size_t +PyThread_get_stacksize(void) +{ + return _pythread_stacksize; +} + +/* Only platforms defining a THREAD_SET_STACKSIZE() macro + in thread_.h support changing the stack size. + Return 0 if stack size is valid, + -1 if stack size value is invalid, + -2 if setting stack size is not supported. */ int PyThread_set_stacksize(size_t size) { +#if defined(THREAD_SET_STACKSIZE) return THREAD_SET_STACKSIZE(size); +#else + return -2; +#endif } #ifndef Py_HAVE_NATIVE_TLS Modified: python/branches/aimacintyre-sf1454481/Python/thread_nt.h ============================================================================== --- python/branches/aimacintyre-sf1454481/Python/thread_nt.h (original) +++ python/branches/aimacintyre-sf1454481/Python/thread_nt.h Tue Jun 13 14:00:30 2006 @@ -319,7 +319,7 @@ #define THREAD_MAX_STACKSIZE 0x10000000 /* 256MB */ /* set the thread stack size. - * Return 1 if an exception is pending, 0 otherwise. + * Return 0 if size is valid, -1 otherwise. */ static int _pythread_nt_set_stacksize(size_t size) @@ -335,15 +335,8 @@ _pythread_stacksize = size; return 0; } - else { - char warning[128]; - snprintf(warning, - 128, - "thread stack size of %#x bytes not supported on Win32", - size); - return PyErr_Warn(PyExc_RuntimeWarning, warning); - } + + return -1; } -#undef THREAD_SET_STACKSIZE #define THREAD_SET_STACKSIZE(x) _pythread_nt_set_stacksize(x) Modified: python/branches/aimacintyre-sf1454481/Python/thread_os2.h ============================================================================== --- python/branches/aimacintyre-sf1454481/Python/thread_os2.h (original) +++ python/branches/aimacintyre-sf1454481/Python/thread_os2.h Tue Jun 13 14:00:30 2006 @@ -286,7 +286,7 @@ #define THREAD_MAX_STACKSIZE 0x2000000 /* 32MB */ /* set the thread stack size. - * Return 1 if an exception is pending, 0 otherwise. + * Return 0 if size is valid, -1 otherwise. */ static int _pythread_os2_set_stacksize(size_t size) @@ -302,15 +302,8 @@ _pythread_stacksize = size; return 0; } - else { - char warning[128]; - snprintf(warning, - 128, - "thread stack size of %#x bytes not supported on OS/2", - size); - return PyErr_Warn(PyExc_RuntimeWarning, warning); - } + + return -1; } -#undef THREAD_SET_STACKSIZE #define THREAD_SET_STACKSIZE(x) _pythread_os2_set_stacksize(x) Modified: python/branches/aimacintyre-sf1454481/Python/thread_pthread.h ============================================================================== --- python/branches/aimacintyre-sf1454481/Python/thread_pthread.h (original) +++ python/branches/aimacintyre-sf1454481/Python/thread_pthread.h Tue Jun 13 14:00:30 2006 @@ -20,10 +20,6 @@ #endif /* for safety, ensure a viable minimum stacksize */ #define THREAD_STACK_MIN 0x8000 /* 32kB */ -#if THREAD_STACK_MIN < PTHREAD_STACK_MIN -#undef THREAD_STACK_MIN -#define THREAD_STACK_MIN PTHREAD_STACK_MIN -#endif #else /* !_POSIX_THREAD_ATTR_STACKSIZE */ #ifdef THREAD_STACK_SIZE #error "THREAD_STACK_SIZE defined but _POSIX_THREAD_ATTR_STACKSIZE undefined" @@ -165,7 +161,8 @@ PyThread_init_thread(); #if defined(THREAD_STACK_SIZE) || defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) - pthread_attr_init(&attrs); + if (pthread_attr_init(&attrs) != 0) + return -1; #endif #if defined(THREAD_STACK_SIZE) tss = (_pythread_stacksize != 0) ? _pythread_stacksize @@ -491,31 +488,46 @@ #endif /* USE_SEMAPHORES */ /* set the thread stack size. - * Return 1 if an exception is pending, 0 otherwise. + * Return 0 if size is valid, -1 if size is invalid, + * -2 if setting stack size is not supported. */ static int _pythread_pthread_set_stacksize(size_t size) { +#if defined(THREAD_STACK_SIZE) + pthread_attr_t attrs; + size_t tss_min; + int rc = 0; +#endif + /* set to default */ if (size == 0) { _pythread_stacksize = 0; return 0; } - /* valid range? */ - if (size >= THREAD_STACK_MIN) { - _pythread_stacksize = size; - return 0; - } - else { - char warning[128]; - snprintf(warning, - 128, - "thread stack size of %#x bytes not supported", - size); - return PyErr_Warn(PyExc_RuntimeWarning, warning); +#if defined(THREAD_STACK_SIZE) +#if defined(PTHREAD_STACK_MIN) + tss_min = PTHREAD_STACK_MIN > THREAD_STACK_MIN ? PTHREAD_STACK_MIN + : THREAD_STACK_MIN; +#else + tss_min = THREAD_STACK_MIN; +#endif + if (size >= tss_min) { + /* validate stack size by setting thread attribute */ + if (pthread_attr_init(&attrs) == 0) { + rc = pthread_attr_setstacksize(&attrs, size); + pthread_attr_destroy(&attrs); + if (rc == 0) { + _pythread_stacksize = size; + return 0; + } + } } + return -1; +#else + return -2; +#endif } -#undef THREAD_SET_STACKSIZE #define THREAD_SET_STACKSIZE(x) _pythread_pthread_set_stacksize(x) From python-checkins at python.org Tue Jun 13 14:02:12 2006 From: python-checkins at python.org (walter.doerwald) Date: Tue, 13 Jun 2006 14:02:12 +0200 (CEST) Subject: [Python-checkins] r46915 - python/trunk/Lib/encodings/utf_8_sig.py python/trunk/Lib/encodings/uu_codec.py Message-ID: <20060613120212.EF6571E4005@bag.python.org> Author: walter.doerwald Date: Tue Jun 13 14:02:12 2006 New Revision: 46915 Modified: python/trunk/Lib/encodings/utf_8_sig.py python/trunk/Lib/encodings/uu_codec.py Log: Fix passing errors to the encoder and decoder functions. Modified: python/trunk/Lib/encodings/utf_8_sig.py ============================================================================== --- python/trunk/Lib/encodings/utf_8_sig.py (original) +++ python/trunk/Lib/encodings/utf_8_sig.py Tue Jun 13 14:02:12 2006 @@ -30,9 +30,9 @@ def encode(self, input, final=False): if self.first: self.first = False - return codecs.BOM_UTF8 + codecs.utf_8_encode(input, errors)[0] + return codecs.BOM_UTF8 + codecs.utf_8_encode(input, self.errors)[0] else: - return codecs.utf_8_encode(input, errors)[0] + return codecs.utf_8_encode(input, self.errors)[0] def reset(self): codecs.IncrementalEncoder.reset(self) Modified: python/trunk/Lib/encodings/uu_codec.py ============================================================================== --- python/trunk/Lib/encodings/uu_codec.py (original) +++ python/trunk/Lib/encodings/uu_codec.py Tue Jun 13 14:02:12 2006 @@ -102,11 +102,11 @@ class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return uu_encode(input, errors)[0] + return uu_encode(input, self.errors)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): - return uu_decode(input, errors)[0] + return uu_decode(input, self.errors)[0] class StreamWriter(Codec,codecs.StreamWriter): pass From python-checkins at python.org Tue Jun 13 14:03:57 2006 From: python-checkins at python.org (andrew.macintyre) Date: Tue, 13 Jun 2006 14:03:57 +0200 (CEST) Subject: [Python-checkins] r46916 - python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py Message-ID: <20060613120357.6FC241E4005@bag.python.org> Author: andrew.macintyre Date: Tue Jun 13 14:03:57 2006 New Revision: 46916 Modified: python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py Log: reduce size of large stack to avoid ulimit settings Modified: python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py Tue Jun 13 14:03:57 2006 @@ -93,11 +93,11 @@ self.test_various_ops() threading.stack_size(0) - # run with a large thread stack size (16MB) + # run with a large thread stack size (4MB) def test_various_ops_large_stack(self): if verbose: - print 'with 16MB thread stack size...' - threading.stack_size(0x1000000) + print 'with 4MB thread stack size...' + threading.stack_size(0x400000) self.test_various_ops() threading.stack_size(0) From python-checkins at python.org Tue Jun 13 14:04:44 2006 From: python-checkins at python.org (walter.doerwald) Date: Tue, 13 Jun 2006 14:04:44 +0200 (CEST) Subject: [Python-checkins] r46917 - python/trunk/Lib/encodings/punycode.py Message-ID: <20060613120444.22F971E4005@bag.python.org> Author: walter.doerwald Date: Tue Jun 13 14:04:43 2006 New Revision: 46917 Modified: python/trunk/Lib/encodings/punycode.py Log: errors is an attribute in the incremental decoder not an argument. Modified: python/trunk/Lib/encodings/punycode.py ============================================================================== --- python/trunk/Lib/encodings/punycode.py (original) +++ python/trunk/Lib/encodings/punycode.py Tue Jun 13 14:04:43 2006 @@ -214,9 +214,9 @@ class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): - if errors not in ('strict', 'replace', 'ignore'): - raise UnicodeError, "Unsupported error handling "+errors - return punycode_decode(input, errors) + if self.errors not in ('strict', 'replace', 'ignore'): + raise UnicodeError, "Unsupported error handling "+self.errors + return punycode_decode(input, self.errors) class StreamWriter(Codec,codecs.StreamWriter): pass From buildbot at python.org Tue Jun 13 14:06:49 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 12:06:49 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20060613120649.362401E4006@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/171 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,thomas.heller BUILD FAILED: failed compile sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 14:27:52 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 12:27:52 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060613122752.B666F1E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/350 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 14:40:00 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 12:40:00 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo 2.4 Message-ID: <20060613124000.3D6621E4005@bag.python.org> The Buildbot has detected a new failure of x86 gentoo 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%25202.4/builds/170 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'Andrew MacIntyre': test fixes for SF1454481 Build Source Stamp: [branch branches/aimacintyre-sf1454481] 46916 Blamelist: Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 15:07:47 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 13:07:47 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD 2.4 Message-ID: <20060613130747.B8EFF1E4005@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%25202.4/builds/115 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'Andrew MacIntyre': test fixes for SF1454481 Build Source Stamp: [branch branches/aimacintyre-sf1454481] 46916 Blamelist: Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 15:27:27 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 13:27:27 +0000 Subject: [Python-checkins] buildbot failure in x86 XP 2.4 Message-ID: <20060613132727.52AAB1E4006@bag.python.org> The Buildbot has detected a new failure of x86 XP 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%25202.4/builds/161 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'Andrew MacIntyre': test fixes for SF1454481 Build Source Stamp: [branch branches/aimacintyre-sf1454481] 46916 Blamelist: BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 15:44:44 2006 From: python-checkins at python.org (andrew.macintyre) Date: Tue, 13 Jun 2006 15:44:44 +0200 (CEST) Subject: [Python-checkins] r46918 - in python/branches/aimacintyre-sf1454481/Lib/test: output/test_thread test_thread.py test_threading.py Message-ID: <20060613134444.855991E4005@bag.python.org> Author: andrew.macintyre Date: Tue Jun 13 15:44:43 2006 New Revision: 46918 Modified: python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py Log: use 1MB stack for testing, as some Linux buildbots don't allow 4MB thread stacks... Modified: python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread Tue Jun 13 15:44:43 2006 @@ -8,11 +8,11 @@ *** Changing thread stack size *** caught expected ValueError setting stack_size(4096) successfully set stack_size(32768) -successfully set stack_size(4194304) +successfully set stack_size(1048576) successfully set stack_size(0) trying stack_size = 32768 waiting for all tasks to complete all tasks done -trying stack_size = 4194304 +trying stack_size = 1048576 waiting for all tasks to complete all tasks done Modified: python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py Tue Jun 13 15:44:43 2006 @@ -140,13 +140,13 @@ if tss_supported: failed = lambda s, e: s != e fail_msg = "stack_size(%d) failed - should succeed" - for tss in (32768, 0x400000, 0): + for tss in (32768, 0x100000, 0): thread.stack_size(tss) if failed(thread.stack_size(), tss): raise ValueError, fail_msg % tss print 'successfully set stack_size(%d)' % tss - for tss in (32768, 0x400000): + for tss in (32768, 0x100000): print 'trying stack_size = %d' % tss next_ident = 0 for i in range(numtasks): Modified: python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py Tue Jun 13 15:44:43 2006 @@ -93,11 +93,11 @@ self.test_various_ops() threading.stack_size(0) - # run with a large thread stack size (4MB) + # run with a large thread stack size (1MB) def test_various_ops_large_stack(self): if verbose: - print 'with 4MB thread stack size...' - threading.stack_size(0x400000) + print 'with 1MB thread stack size...' + threading.stack_size(0x100000) self.test_various_ops() threading.stack_size(0) From buildbot at python.org Tue Jun 13 16:00:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 14:00:55 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060613140055.960661E401A@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/172 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 16:11:53 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 14:11:53 +0000 Subject: [Python-checkins] buildbot failure in x86 W2k 2.4 Message-ID: <20060613141153.D69C81E4010@bag.python.org> The Buildbot has detected a new failure of x86 W2k 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%25202.4/builds/157 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'Andrew MacIntyre': test fixes for SF1454481 Build Source Stamp: [branch branches/aimacintyre-sf1454481] 46918 Blamelist: BUILD FAILED: failed compile sincerely, -The Buildbot From g.brandl at gmx.net Tue Jun 13 16:17:20 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 13 Jun 2006 16:17:20 +0200 Subject: [Python-checkins] Python Regression Test Failures refleak (80) In-Reply-To: <2mwtblo1bg.fsf@starship.python.net> References: <20060613035818.GA15597@python.psfb.org> <2mwtblo1bg.fsf@starship.python.net> Message-ID: Michael Hudson wrote: > Neal Norwitz writes: > >> test_builtin leaked [8, 8, 8] references >> test_exceptions leaked [40, 40, 40] references > > So who broke refcounting then? This is pretty new and must be > somewhere fairly core, hopefully someone remembers changing something > there? I don't remember any specific checkins though. FWIW, it would perhaps be a good idea for the refleak mail to include something like sys.subversion. Georg From buildbot at python.org Tue Jun 13 16:44:00 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 14:44:00 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060613144401.1133B1E4006@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/630 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,walter.doerwald Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 17:04:38 2006 From: python-checkins at python.org (andrew.macintyre) Date: Tue, 13 Jun 2006 17:04:38 +0200 (CEST) Subject: [Python-checkins] r46919 - in python/trunk: Doc/lib/libthread.tex Doc/lib/libthreading.tex Include/pythread.h Lib/dummy_thread.py Lib/test/output/test_thread Lib/test/test_thread.py Lib/test/test_threading.py Lib/threading.py Misc/NEWS Modules/threadmodule.c Python/thread.c Python/thread_nt.h Python/thread_os2.h Python/thread_pthread.h Message-ID: <20060613150438.CF5371E4005@bag.python.org> Author: andrew.macintyre Date: Tue Jun 13 17:04:24 2006 New Revision: 46919 Modified: python/trunk/Doc/lib/libthread.tex python/trunk/Doc/lib/libthreading.tex python/trunk/Include/pythread.h python/trunk/Lib/dummy_thread.py python/trunk/Lib/test/output/test_thread python/trunk/Lib/test/test_thread.py python/trunk/Lib/test/test_threading.py python/trunk/Lib/threading.py python/trunk/Misc/NEWS python/trunk/Modules/threadmodule.c python/trunk/Python/thread.c python/trunk/Python/thread_nt.h python/trunk/Python/thread_os2.h python/trunk/Python/thread_pthread.h Log: Patch #1454481: Make thread stack size runtime tunable. Heavily revised, comprising revisions: 46640 - original trunk revision (backed out in r46655) 46647 - markup fix (backed out in r46655) 46692:46918 merged from branch aimacintyre-sf1454481 branch tested on buildbots (Windows buildbots had problems not related to these changes). Modified: python/trunk/Doc/lib/libthread.tex ============================================================================== --- python/trunk/Doc/lib/libthread.tex (original) +++ python/trunk/Doc/lib/libthread.tex Tue Jun 13 17:04:24 2006 @@ -74,6 +74,26 @@ another thread is created. \end{funcdesc} +\begin{funcdesc}{stack_size}{\optional{size}} +Return the thread stack size used when creating new threads. The +optional \var{size} argument specifies the stack size to be used for +subsequently created threads, and must be 0 (use platform or +configured default) or a positive integer value of at least 32,768 (32kB). +If changing the thread stack size is unsupported, a \exception{ThreadError} +is raised. If the specified stack size is invalid, a \exception{ValueError} +is raised and the stack size is unmodified. 32kB is currently the minimum +supported stack size value to guarantee sufficient stack space for the +interpreter itself. Note that some platforms may have particular +restrictions on values for the stack size, such as requiring a minimum +stack size > 32kB or requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for +more information (4kB pages are common; using multiples of 4096 for +the stack size is the suggested approach in the absence of more +specific information). +Availability: Windows, systems with \POSIX{} threads. +\versionadded{2.5} +\end{funcdesc} + Lock objects have the following methods: Modified: python/trunk/Doc/lib/libthreading.tex ============================================================================== --- python/trunk/Doc/lib/libthreading.tex (original) +++ python/trunk/Doc/lib/libthreading.tex Tue Jun 13 17:04:24 2006 @@ -125,6 +125,26 @@ \versionadded{2.3} \end{funcdesc} +\begin{funcdesc}{stack_size}{\optional{size}} +Return the thread stack size used when creating new threads. The +optional \var{size} argument specifies the stack size to be used for +subsequently created threads, and must be 0 (use platform or +configured default) or a positive integer value of at least 32,768 (32kB). +If changing the thread stack size is unsupported, a \exception{ThreadError} +is raised. If the specified stack size is invalid, a \exception{ValueError} +is raised and the stack size is unmodified. 32kB is currently the minimum +supported stack size value to guarantee sufficient stack space for the +interpreter itself. Note that some platforms may have particular +restrictions on values for the stack size, such as requiring a minimum +stack size > 32kB or requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for +more information (4kB pages are common; using multiples of 4096 for +the stack size is the suggested approach in the absence of more +specific information). +Availability: Windows, systems with \POSIX{} threads. +\versionadded{2.5} +\end{funcdesc} + Detailed interfaces for the objects are documented below. The design of this module is loosely based on Java's threading model. Modified: python/trunk/Include/pythread.h ============================================================================== --- python/trunk/Include/pythread.h (original) +++ python/trunk/Include/pythread.h Tue Jun 13 17:04:24 2006 @@ -25,6 +25,9 @@ #define NOWAIT_LOCK 0 PyAPI_FUNC(void) PyThread_release_lock(PyThread_type_lock); +PyAPI_FUNC(size_t) PyThread_get_stacksize(void); +PyAPI_FUNC(int) PyThread_set_stacksize(size_t); + #ifndef NO_EXIT_PROG PyAPI_FUNC(void) PyThread_exit_prog(int); PyAPI_FUNC(void) PyThread__PyThread_exit_prog(int); Modified: python/trunk/Lib/dummy_thread.py ============================================================================== --- python/trunk/Lib/dummy_thread.py (original) +++ python/trunk/Lib/dummy_thread.py Tue Jun 13 17:04:24 2006 @@ -20,6 +20,7 @@ 'interrupt_main', 'LockType'] import traceback as _traceback +import warnings class error(Exception): """Dummy implementation of thread.error.""" @@ -75,6 +76,13 @@ """Dummy implementation of thread.allocate_lock().""" return LockType() +def stack_size(size=None): + """Dummy implementation of thread.stack_size().""" + if size is not None: + msg = "setting thread stack size not supported on this platform" + warnings.warn(msg, RuntimeWarning) + return 0 + class LockType(object): """Class implementing dummy implementation of thread.LockType. Modified: python/trunk/Lib/test/output/test_thread ============================================================================== --- python/trunk/Lib/test/output/test_thread (original) +++ python/trunk/Lib/test/output/test_thread Tue Jun 13 17:04:24 2006 @@ -4,3 +4,15 @@ *** Barrier Test *** all tasks done + +*** Changing thread stack size *** +caught expected ValueError setting stack_size(4096) +successfully set stack_size(32768) +successfully set stack_size(1048576) +successfully set stack_size(0) +trying stack_size = 32768 +waiting for all tasks to complete +all tasks done +trying stack_size = 1048576 +waiting for all tasks to complete +all tasks done Modified: python/trunk/Lib/test/test_thread.py ============================================================================== --- python/trunk/Lib/test/test_thread.py (original) +++ python/trunk/Lib/test/test_thread.py Tue Jun 13 17:04:24 2006 @@ -115,3 +115,46 @@ thread.start_new_thread(task2, (i,)) done.acquire() print 'all tasks done' + +# not all platforms support changing thread stack size +print '\n*** Changing thread stack size ***' +if thread.stack_size() != 0: + raise ValueError, "initial stack_size not 0" + +thread.stack_size(0) +if thread.stack_size() != 0: + raise ValueError, "stack_size not reset to default" + +from os import name as os_name +if os_name in ("nt", "os2", "posix"): + + tss_supported = 1 + try: + thread.stack_size(4096) + except ValueError: + print 'caught expected ValueError setting stack_size(4096)' + except thread.ThreadError: + tss_supported = 0 + print 'platform does not support changing thread stack size' + + if tss_supported: + failed = lambda s, e: s != e + fail_msg = "stack_size(%d) failed - should succeed" + for tss in (32768, 0x100000, 0): + thread.stack_size(tss) + if failed(thread.stack_size(), tss): + raise ValueError, fail_msg % tss + print 'successfully set stack_size(%d)' % tss + + for tss in (32768, 0x100000): + print 'trying stack_size = %d' % tss + next_ident = 0 + for i in range(numtasks): + newtask() + + print 'waiting for all tasks to complete' + done.acquire() + print 'all tasks done' + + # reset stack size to default + thread.stack_size(0) Modified: python/trunk/Lib/test/test_threading.py ============================================================================== --- python/trunk/Lib/test/test_threading.py (original) +++ python/trunk/Lib/test/test_threading.py Tue Jun 13 17:04:24 2006 @@ -85,6 +85,22 @@ print 'all tasks done' self.assertEqual(numrunning.get(), 0) + # run with a minimum thread stack size (32kB) + def test_various_ops_small_stack(self): + if verbose: + print 'with 32kB thread stack size...' + threading.stack_size(0x8000) + self.test_various_ops() + threading.stack_size(0) + + # run with a large thread stack size (1MB) + def test_various_ops_large_stack(self): + if verbose: + print 'with 1MB thread stack size...' + threading.stack_size(0x100000) + self.test_various_ops() + threading.stack_size(0) + def test_foreign_thread(self): # Check that a "foreign" thread can use the threading module. def f(mutex): Modified: python/trunk/Lib/threading.py ============================================================================== --- python/trunk/Lib/threading.py (original) +++ python/trunk/Lib/threading.py Tue Jun 13 17:04:24 2006 @@ -15,7 +15,7 @@ # Rename some stuff so "from threading import *" is safe __all__ = ['activeCount', 'Condition', 'currentThread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', - 'Timer', 'setprofile', 'settrace', 'local'] + 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'] _start_new_thread = thread.start_new_thread _allocate_lock = thread.allocate_lock @@ -713,6 +713,8 @@ _active_limbo_lock.release() return active +from thread import stack_size + # Create the main thread object _MainThread() Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Tue Jun 13 17:04:24 2006 @@ -112,6 +112,9 @@ - Patch #1435422: zlib's compress and decompress objects now have a copy() method. +- Patch #1454481: thread stack size is now tunable at runtime for thread + enabled builds on Windows and systems with Posix threads support. + - On Win32, os.listdir now supports arbitrarily-long Unicode path names (up to the system limit of 32K characters). Modified: python/trunk/Modules/threadmodule.c ============================================================================== --- python/trunk/Modules/threadmodule.c (original) +++ python/trunk/Modules/threadmodule.c Tue Jun 13 17:04:24 2006 @@ -586,6 +586,61 @@ be relied upon, and the number should be seen purely as a magic cookie.\n\ A thread's identity may be reused for another thread after it exits."); +static PyObject * +thread_stack_size(PyObject *self, PyObject *args) +{ + size_t old_size; + Py_ssize_t new_size = 0; + PyObject *set_size = NULL; + int rc; + + if (!PyArg_ParseTuple(args, "|n:stack_size", &new_size)) + return NULL; + + if (new_size < 0) { + PyErr_SetString(PyExc_ValueError, + "size must be 0 or a positive value"); + return NULL; + } + + old_size = PyThread_get_stacksize(); + + rc = PyThread_set_stacksize((size_t) new_size); + if (rc == -1) { + PyErr_Format(PyExc_ValueError, + "size not valid: %zd bytes", + new_size); + return NULL; + } + if (rc == -2) { + PyErr_SetString(ThreadError, + "setting stack size not supported"); + return NULL; + } + + return PyInt_FromSsize_t((Py_ssize_t) old_size); +} + +PyDoc_STRVAR(stack_size_doc, +"stack_size([size]) -> size\n\ +\n\ +Return the thread stack size used when creating new threads. The\n\ +optional size argument specifies the stack size (in bytes) to be used\n\ +for subsequently created threads, and must be 0 (use platform or\n\ +configured default) or a positive integer value of at least 32,768 (32k).\n\ +If changing the thread stack size is unsupported, a ThreadError\n\ +exception is raised. If the specified size is invalid, a ValueError\n\ +exception is raised, and the stack size is unmodified. 32k bytes\n\ + currently the minimum supported stack size value to guarantee\n\ +sufficient stack space for the interpreter itself.\n\ +\n\ +Note that some platforms may have particular restrictions on values for\n\ +the stack size, such as requiring a minimum stack size larger than 32kB or\n\ +requiring allocation in multiples of the system memory page size\n\ +- platform documentation should be referred to for more information\n\ +(4kB pages are common; using multiples of 4096 for the stack size is\n\ +the suggested approach in the absence of more specific information)."); + static PyMethodDef thread_methods[] = { {"start_new_thread", (PyCFunction)thread_PyThread_start_new_thread, METH_VARARGS, @@ -605,6 +660,9 @@ METH_NOARGS, interrupt_doc}, {"get_ident", (PyCFunction)thread_get_ident, METH_NOARGS, get_ident_doc}, + {"stack_size", (PyCFunction)thread_stack_size, + METH_VARARGS, + stack_size_doc}, #ifndef NO_EXIT_PROG {"exit_prog", (PyCFunction)thread_PyThread_exit_prog, METH_VARARGS}, Modified: python/trunk/Python/thread.c ============================================================================== --- python/trunk/Python/thread.c (original) +++ python/trunk/Python/thread.c Tue Jun 13 17:04:24 2006 @@ -95,6 +95,11 @@ PyThread__init_thread(); } +/* Support for runtime thread stack size tuning. + A value of 0 means using the platform's default stack size + or the size specified by the THREAD_STACK_SIZE macro. */ +static size_t _pythread_stacksize = 0; + #ifdef SGI_THREADS #include "thread_sgi.h" #endif @@ -150,6 +155,28 @@ #endif */ +/* return the current thread stack size */ +size_t +PyThread_get_stacksize(void) +{ + return _pythread_stacksize; +} + +/* Only platforms defining a THREAD_SET_STACKSIZE() macro + in thread_.h support changing the stack size. + Return 0 if stack size is valid, + -1 if stack size value is invalid, + -2 if setting stack size is not supported. */ +int +PyThread_set_stacksize(size_t size) +{ +#if defined(THREAD_SET_STACKSIZE) + return THREAD_SET_STACKSIZE(size); +#else + return -2; +#endif +} + #ifndef Py_HAVE_NATIVE_TLS /* If the platform has not supplied a platform specific TLS implementation, provide our own. Modified: python/trunk/Python/thread_nt.h ============================================================================== --- python/trunk/Python/thread_nt.h (original) +++ python/trunk/Python/thread_nt.h Tue Jun 13 17:04:24 2006 @@ -196,7 +196,7 @@ if (obj.done == NULL) return -1; - rv = _beginthread(bootstrap, 0, &obj); /* use default stack size */ + rv = _beginthread(bootstrap, _pythread_stacksize, &obj); if (rv == (Py_uintptr_t)-1) { /* I've seen errno == EAGAIN here, which means "there are * too many threads". @@ -335,3 +335,30 @@ if (!(aLock && LeaveNonRecursiveMutex((PNRMUTEX) aLock))) dprintf(("%ld: Could not PyThread_release_lock(%p) error: %l\n", PyThread_get_thread_ident(), aLock, GetLastError())); } + +/* minimum/maximum thread stack sizes supported */ +#define THREAD_MIN_STACKSIZE 0x8000 /* 32kB */ +#define THREAD_MAX_STACKSIZE 0x10000000 /* 256MB */ + +/* set the thread stack size. + * Return 0 if size is valid, -1 otherwise. + */ +static int +_pythread_nt_set_stacksize(size_t size) +{ + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + + /* valid range? */ + if (size >= THREAD_MIN_STACKSIZE && size < THREAD_MAX_STACKSIZE) { + _pythread_stacksize = size; + return 0; + } + + return -1; +} + +#define THREAD_SET_STACKSIZE(x) _pythread_nt_set_stacksize(x) Modified: python/trunk/Python/thread_os2.h ============================================================================== --- python/trunk/Python/thread_os2.h (original) +++ python/trunk/Python/thread_os2.h Tue Jun 13 17:04:24 2006 @@ -14,10 +14,13 @@ long PyThread_get_thread_ident(void); #endif +/* default thread stack size of 64kB */ #if !defined(THREAD_STACK_SIZE) #define THREAD_STACK_SIZE 0x10000 #endif +#define OS2_STACKSIZE(x) (x ? x : THREAD_STACK_SIZE) + /* * Initialization of the C package, should not be needed. */ @@ -35,7 +38,10 @@ int aThread; int success = 0; - aThread = _beginthread(func, NULL, THREAD_STACK_SIZE, arg); + aThread = _beginthread(func, + NULL, + OS2_STACKSIZE(_pythread_stacksize), + arg); if (aThread == -1) { success = -1; @@ -275,3 +281,30 @@ DosExitCritSec(); #endif } + +/* minimum/maximum thread stack sizes supported */ +#define THREAD_MIN_STACKSIZE 0x8000 /* 32kB */ +#define THREAD_MAX_STACKSIZE 0x2000000 /* 32MB */ + +/* set the thread stack size. + * Return 0 if size is valid, -1 otherwise. + */ +static int +_pythread_os2_set_stacksize(size_t size) +{ + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + + /* valid range? */ + if (size >= THREAD_MIN_STACKSIZE && size < THREAD_MAX_STACKSIZE) { + _pythread_stacksize = size; + return 0; + } + + return -1; +} + +#define THREAD_SET_STACKSIZE(x) _pythread_os2_set_stacksize(x) Modified: python/trunk/Python/thread_pthread.h ============================================================================== --- python/trunk/Python/thread_pthread.h (original) +++ python/trunk/Python/thread_pthread.h Tue Jun 13 17:04:24 2006 @@ -12,6 +12,20 @@ #endif #include +/* The POSIX spec requires that use of pthread_attr_setstacksize + be conditional on _POSIX_THREAD_ATTR_STACKSIZE being defined. */ +#ifdef _POSIX_THREAD_ATTR_STACKSIZE +#ifndef THREAD_STACK_SIZE +#define THREAD_STACK_SIZE 0 /* use default stack size */ +#endif +/* for safety, ensure a viable minimum stacksize */ +#define THREAD_STACK_MIN 0x8000 /* 32kB */ +#else /* !_POSIX_THREAD_ATTR_STACKSIZE */ +#ifdef THREAD_STACK_SIZE +#error "THREAD_STACK_SIZE defined but _POSIX_THREAD_ATTR_STACKSIZE undefined" +#endif +#endif + /* The POSIX spec says that implementations supporting the sem_* family of functions must indicate this by defining _POSIX_SEMAPHORES. */ @@ -138,15 +152,27 @@ #if defined(THREAD_STACK_SIZE) || defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) pthread_attr_t attrs; #endif +#if defined(THREAD_STACK_SIZE) + size_t tss; +#endif + dprintf(("PyThread_start_new_thread called\n")); if (!initialized) PyThread_init_thread(); #if defined(THREAD_STACK_SIZE) || defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) - pthread_attr_init(&attrs); + if (pthread_attr_init(&attrs) != 0) + return -1; #endif -#ifdef THREAD_STACK_SIZE - pthread_attr_setstacksize(&attrs, THREAD_STACK_SIZE); +#if defined(THREAD_STACK_SIZE) + tss = (_pythread_stacksize != 0) ? _pythread_stacksize + : THREAD_STACK_SIZE; + if (tss != 0) { + if (pthread_attr_setstacksize(&attrs, tss) != 0) { + pthread_attr_destroy(&attrs); + return -1; + } + } #endif #if defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) pthread_attr_setscope(&attrs, PTHREAD_SCOPE_SYSTEM); @@ -460,3 +486,48 @@ } #endif /* USE_SEMAPHORES */ + +/* set the thread stack size. + * Return 0 if size is valid, -1 if size is invalid, + * -2 if setting stack size is not supported. + */ +static int +_pythread_pthread_set_stacksize(size_t size) +{ +#if defined(THREAD_STACK_SIZE) + pthread_attr_t attrs; + size_t tss_min; + int rc = 0; +#endif + + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + +#if defined(THREAD_STACK_SIZE) +#if defined(PTHREAD_STACK_MIN) + tss_min = PTHREAD_STACK_MIN > THREAD_STACK_MIN ? PTHREAD_STACK_MIN + : THREAD_STACK_MIN; +#else + tss_min = THREAD_STACK_MIN; +#endif + if (size >= tss_min) { + /* validate stack size by setting thread attribute */ + if (pthread_attr_init(&attrs) == 0) { + rc = pthread_attr_setstacksize(&attrs, size); + pthread_attr_destroy(&attrs); + if (rc == 0) { + _pythread_stacksize = size; + return 0; + } + } + } + return -1; +#else + return -2; +#endif +} + +#define THREAD_SET_STACKSIZE(x) _pythread_pthread_set_stacksize(x) From buildbot at python.org Tue Jun 13 17:26:29 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 15:26:29 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060613152629.DE1FD1E401E@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/741 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 17:55:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 15:55:37 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060613155537.705631E4005@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/703 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre Build Had Warnings: warnings test sincerely, -The Buildbot From brett at python.org Tue Jun 13 18:03:10 2006 From: brett at python.org (Brett Cannon) Date: Tue, 13 Jun 2006 09:03:10 -0700 Subject: [Python-checkins] Python Regression Test Failures refleak (80) In-Reply-To: References: <20060613035818.GA15597@python.psfb.org> <2mwtblo1bg.fsf@starship.python.net> Message-ID: I checked the other day and the first report came with rev. **896. You can change the output by modifying Misc/build.sh . Unfortunately it is a shell script, but that just means using ``svnversion .`` . -Brett On 6/13/06, Georg Brandl wrote: > > Michael Hudson wrote: > > Neal Norwitz writes: > > > >> test_builtin leaked [8, 8, 8] references > >> test_exceptions leaked [40, 40, 40] references > > > > So who broke refcounting then? This is pretty new and must be > > somewhere fairly core, hopefully someone remembers changing something > > there? I don't remember any specific checkins though. > > FWIW, it would perhaps be a good idea for the refleak mail > to include something like sys.subversion. > > Georg > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060613/ed0fb4cd/attachment.htm From python-checkins at python.org Tue Jun 13 18:06:56 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 13 Jun 2006 18:06:56 +0200 (CEST) Subject: [Python-checkins] r46920 - python/trunk/Modules/threadmodule.c Message-ID: <20060613160656.A458B1E4005@bag.python.org> Author: brett.cannon Date: Tue Jun 13 18:06:55 2006 New Revision: 46920 Modified: python/trunk/Modules/threadmodule.c Log: Remove unused variable. Modified: python/trunk/Modules/threadmodule.c ============================================================================== --- python/trunk/Modules/threadmodule.c (original) +++ python/trunk/Modules/threadmodule.c Tue Jun 13 18:06:55 2006 @@ -591,7 +591,6 @@ { size_t old_size; Py_ssize_t new_size = 0; - PyObject *set_size = NULL; int rc; if (!PyArg_ParseTuple(args, "|n:stack_size", &new_size)) From python-checkins at python.org Tue Jun 13 18:41:42 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 13 Jun 2006 18:41:42 +0200 (CEST) Subject: [Python-checkins] r46921 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060613164142.3BEC61E4006@bag.python.org> Author: andrew.kuchling Date: Tue Jun 13 18:41:41 2006 New Revision: 46921 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Add ability to set stack size Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 13 18:41:41 2006 @@ -1631,6 +1631,15 @@ % patch 918101 (Contributed by Lars Gust\"abel.) +\item The \module{threading} module now lets you set the stack size +used when new threads are created. The +\function{stack_size(\optional{\var{size}})} function returns the +currently configured stack size, and supplying the optional \var{size} +parameter sets a new value. Not all platforms support changing the +stack size, but Windows, POSIX threading, and OS/2 all do. +(Contributed by Andrew MacIntyre.) +% Patch 1454481 + \item The \module{unicodedata} module has been updated to use version 4.1.0 of the Unicode character database. Version 3.2.0 is required by some specifications, so it's still available as From python-checkins at python.org Tue Jun 13 18:45:50 2006 From: python-checkins at python.org (marc-andre.lemburg) Date: Tue, 13 Jun 2006 18:45:50 +0200 (CEST) Subject: [Python-checkins] r46922 - in external/pybench-2.0: Arithmetic.py Calls.py CommandLine.py Constructs.py Dict.py Exceptions.py Imports.py Instances.py LICENSE Lists.py Lookups.py NewInstances.py Numbers.py README Setup.py Strings.py Tuples.py Unicode.py clockres.py package package/__init__.py package/submodule.py platform.py pybench.py systimes.py Message-ID: <20060613164550.846321E4005@bag.python.org> Author: marc-andre.lemburg Date: Tue Jun 13 18:45:48 2006 New Revision: 46922 Added: external/pybench-2.0/ external/pybench-2.0/Arithmetic.py (contents, props changed) external/pybench-2.0/Calls.py (contents, props changed) external/pybench-2.0/CommandLine.py (contents, props changed) external/pybench-2.0/Constructs.py (contents, props changed) external/pybench-2.0/Dict.py (contents, props changed) external/pybench-2.0/Exceptions.py (contents, props changed) external/pybench-2.0/Imports.py (contents, props changed) external/pybench-2.0/Instances.py (contents, props changed) external/pybench-2.0/LICENSE (contents, props changed) external/pybench-2.0/Lists.py (contents, props changed) external/pybench-2.0/Lookups.py (contents, props changed) external/pybench-2.0/NewInstances.py (contents, props changed) external/pybench-2.0/Numbers.py (contents, props changed) external/pybench-2.0/README (contents, props changed) external/pybench-2.0/Setup.py (contents, props changed) external/pybench-2.0/Strings.py (contents, props changed) external/pybench-2.0/Tuples.py (contents, props changed) external/pybench-2.0/Unicode.py (contents, props changed) external/pybench-2.0/clockres.py (contents, props changed) external/pybench-2.0/package/ external/pybench-2.0/package/__init__.py (contents, props changed) external/pybench-2.0/package/submodule.py (contents, props changed) external/pybench-2.0/platform.py (contents, props changed) external/pybench-2.0/pybench.py (contents, props changed) external/pybench-2.0/systimes.py (contents, props changed) Log: Import pybench 2.0. Added: external/pybench-2.0/Arithmetic.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Arithmetic.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,777 @@ +from pybench import Test + +class SimpleIntegerArithmetic(Test): + + version = 2.0 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 120000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SimpleFloatArithmetic(Test): + + version = 2.0 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 120000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SimpleIntFloatArithmetic(Test): + + version = 2.0 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 120000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class SimpleLongArithmetic(Test): + + version = 2.0 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 60000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SimpleComplexArithmetic(Test): + + version = 2.0 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/Calls.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Calls.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,504 @@ +from pybench import Test + +class PythonFunctionCalls(Test): + + version = 2.0 + operations = 5*(1+4+4+2) + rounds = 60000 + + def test(self): + + global f,f1,g,h + + # define functions + def f(): + pass + + def f1(x): + pass + + def g(a,b,c): + return a,b,c + + def h(a,b,c,d=1,e=2,f=3): + return d,e,f + + # do calls + for i in xrange(self.rounds): + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + def calibrate(self): + + global f,f1,g,h + + # define functions + def f(): + pass + + def f1(x): + pass + + def g(a,b,c): + return a,b,c + + def h(a,b,c,d=1,e=2,f=3): + return d,e,f + + # do calls + for i in xrange(self.rounds): + pass + +### + +class BuiltinFunctionCalls(Test): + + version = 2.0 + operations = 5*(2+5+5+5) + rounds = 60000 + + def test(self): + + # localize functions + f0 = globals + f1 = hash + f2 = cmp + f3 = range + + # do calls + for i in xrange(self.rounds): + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + def calibrate(self): + + # localize functions + f0 = dir + f1 = hash + f2 = range + f3 = range + + # do calls + for i in xrange(self.rounds): + pass + +### + +class PythonMethodCalls(Test): + + version = 2.0 + operations = 5*(6 + 5 + 4) + rounds = 30000 + + def test(self): + + class c: + + x = 2 + s = 'string' + + def f(self): + + return self.x + + def j(self,a,b): + + self.y = a + self.t = b + return self.y + + def k(self,a,b,c=3): + + self.y = a + self.s = b + self.t = c + + o = c() + + for i in xrange(self.rounds): + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + def calibrate(self): + + class c: + + x = 2 + s = 'string' + + def f(self): + + return self.x + + def j(self,a,b): + + self.y = a + self.t = b + + def k(self,a,b,c=3): + + self.y = a + self.s = b + self.t = c + + o = c + + for i in xrange(self.rounds): + pass + +### + +class Recursion(Test): + + version = 2.0 + operations = 5 + rounds = 100000 + + def test(self): + + global f + + def f(x): + + if x > 1: + return f(x-1) + return 1 + + for i in xrange(self.rounds): + f(10) + f(10) + f(10) + f(10) + f(10) + + def calibrate(self): + + global f + + def f(x): + + if x > 0: + return f(x-1) + return 1 + + for i in xrange(self.rounds): + pass + + +### Test to make Fredrik happy... + +if __name__ == '__main__': + import timeit + if 0: + timeit.TestClass = PythonFunctionCalls + timeit.main(['-s', 'test = TestClass(); test.rounds = 1000', + 'test.test()']) + else: + setup = """\ +global f,f1,g,h + +# define functions +def f(): + pass + +def f1(x): + pass + +def g(a,b,c): + return a,b,c + +def h(a,b,c,d=1,e=2,f=3): + return d,e,f + +i = 1 +""" + test = """\ +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) +""" + + timeit.main(['-s', setup, + test]) + + Added: external/pybench-2.0/CommandLine.py ============================================================================== --- (empty file) +++ external/pybench-2.0/CommandLine.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,634 @@ +""" CommandLine - Get and parse command line options + + NOTE: This still is very much work in progress !!! + + Different version are likely to be incompatible. + + TODO: + + * Incorporate the changes made by (see Inbox) + * Add number range option using srange() + +""" + +__copyright__ = """\ +Copyright (c), 1997-2006, Marc-Andre Lemburg (mal at lemburg.com) +Copyright (c), 2000-2006, eGenix.com Software GmbH (info at egenix.com) +See the documentation for further information on copyrights, +or contact the author. All Rights Reserved. +""" + +__version__ = '1.2' + +import sys, getopt, string, glob, os, re, exceptions, traceback + +### Helpers + +def _getopt_flags(options): + + """ Convert the option list to a getopt flag string and long opt + list + + """ + s = [] + l = [] + for o in options: + if o.prefix == '-': + # short option + s.append(o.name) + if o.takes_argument: + s.append(':') + else: + # long option + if o.takes_argument: + l.append(o.name+'=') + else: + l.append(o.name) + return string.join(s,''),l + +def invisible_input(prompt='>>> '): + + """ Get raw input from a terminal without echoing the characters to + the terminal, e.g. for password queries. + + """ + import getpass + entry = getpass.getpass(prompt) + if entry is None: + raise KeyboardInterrupt + return entry + +def fileopen(name, mode='wb', encoding=None): + + """ Open a file using mode. + + Default mode is 'wb' meaning to open the file for writing in + binary mode. If encoding is given, I/O to and from the file is + transparently encoded using the given encoding. + + Files opened for writing are chmod()ed to 0600. + + """ + if name == 'stdout': + return sys.stdout + elif name == 'stderr': + return sys.stderr + elif name == 'stdin': + return sys.stdin + else: + if encoding is not None: + import codecs + f = codecs.open(name, mode, encoding) + else: + f = open(name, mode) + if 'w' in mode: + os.chmod(name, 0600) + return f + +def option_dict(options): + + """ Return a dictionary mapping option names to Option instances. + """ + d = {} + for option in options: + d[option.name] = option + return d + +# Alias +getpasswd = invisible_input + +_integerRE = re.compile('\s*(-?\d+)\s*$') +_integerRangeRE = re.compile('\s*(-?\d+)\s*-\s*(-?\d+)\s*$') + +def srange(s, + + split=string.split,integer=_integerRE, + integerRange=_integerRangeRE): + + """ Converts a textual representation of integer numbers and ranges + to a Python list. + + Supported formats: 2,3,4,2-10,-1 - -3, 5 - -2 + + Values are appended to the created list in the order specified + in the string. + + """ + l = [] + append = l.append + for entry in split(s,','): + m = integer.match(entry) + if m: + append(int(m.groups()[0])) + continue + m = integerRange.match(entry) + if m: + start,end = map(int,m.groups()) + l[len(l):] = range(start,end+1) + return l + +def abspath(path, + + expandvars=os.path.expandvars,expanduser=os.path.expanduser, + join=os.path.join,getcwd=os.getcwd): + + """ Return the corresponding absolute path for path. + + path is expanded in the usual shell ways before + joining it with the current working directory. + + """ + try: + path = expandvars(path) + except AttributeError: + pass + try: + path = expanduser(path) + except AttributeError: + pass + return join(getcwd(), path) + +### Option classes + +class Option: + + """ Option base class. Takes no argument. + + """ + default = None + helptext = '' + prefix = '-' + takes_argument = 0 + has_default = 0 + tab = 15 + + def __init__(self,name,help=None): + + if not name[:1] == '-': + raise TypeError,'option names must start with "-"' + if name[1:2] == '-': + self.prefix = '--' + self.name = name[2:] + else: + self.name = name[1:] + if help: + self.help = help + + def __str__(self): + + o = self + name = o.prefix + o.name + if o.takes_argument: + name = name + ' arg' + if len(name) > self.tab: + name = name + '\n' + ' ' * (self.tab + 1 + len(o.prefix)) + else: + name = '%-*s ' % (self.tab, name) + description = o.help + if o.has_default: + description = description + ' (%s)' % o.default + return '%s %s' % (name, description) + +class ArgumentOption(Option): + + """ Option that takes an argument. + + An optional default argument can be given. + + """ + def __init__(self,name,help=None,default=None): + + # Basemethod + Option.__init__(self,name,help) + + if default is not None: + self.default = default + self.has_default = 1 + self.takes_argument = 1 + +class SwitchOption(Option): + + """ Options that can be on or off. Has an optional default value. + + """ + def __init__(self,name,help=None,default=None): + + # Basemethod + Option.__init__(self,name,help) + + if default is not None: + self.default = default + self.has_default = 1 + +### Application baseclass + +class Application: + + """ Command line application interface with builtin argument + parsing. + + """ + # Options the program accepts (Option instances) + options = [] + + # Standard settings; these are appended to options in __init__ + preset_options = [SwitchOption('-v', + 'generate verbose output'), + SwitchOption('-h', + 'show this help text'), + SwitchOption('--help', + 'show this help text'), + SwitchOption('--debug', + 'enable debugging'), + SwitchOption('--copyright', + 'show copyright'), + SwitchOption('--examples', + 'show examples of usage')] + + # The help layout looks like this: + # [header] - defaults to '' + # + # [synopsis] - formatted as ' %s' % self.synopsis + # + # options: + # [options] - formatted from self.options + # + # [version] - formatted as 'Version:\n %s' % self.version, if given + # + # [about] - defaults to '' + # + # Note: all fields that do not behave as template are formatted + # using the instances dictionary as substitution namespace, + # e.g. %(name)s will be replaced by the applications name. + # + + # Header (default to program name) + header = '' + + # Name (defaults to program name) + name = '' + + # Synopsis (%(name)s is replaced by the program name) + synopsis = '%(name)s [option] files...' + + # Version (optional) + version = '' + + # General information printed after the possible options (optional) + about = '' + + # Examples of usage to show when the --examples option is given (optional) + examples = '' + + # Copyright to show + copyright = __copyright__ + + # Apply file globbing ? + globbing = 1 + + # Generate debug output ? + debug = 0 + + # Generate verbose output ? + verbose = 0 + + # Internal errors to catch + InternalError = exceptions.Exception + + # Instance variables: + values = None # Dictionary of passed options (or default values) + # indexed by the options name, e.g. '-h' + files = None # List of passed filenames + optionlist = None # List of passed options + + def __init__(self,argv=None): + + # Setup application specs + if argv is None: + argv = sys.argv + self.filename = os.path.split(argv[0])[1] + if not self.name: + self.name = os.path.split(self.filename)[1] + else: + self.name = self.name + if not self.header: + self.header = self.name + else: + self.header = self.header + + # Init .arguments list + self.arguments = argv[1:] + + # Setup Option mapping + self.option_map = option_dict(self.options) + + # Append preset options + for option in self.preset_options: + if not self.option_map.has_key(option.name): + self.add_option(option) + + # Init .files list + self.files = [] + + # Start Application + try: + # Process startup + rc = self.startup() + if rc is not None: + raise SystemExit,rc + + # Parse command line + rc = self.parse() + if rc is not None: + raise SystemExit,rc + + # Start application + rc = self.main() + if rc is None: + rc = 0 + + except SystemExit,rc: + pass + + except KeyboardInterrupt: + print + print '* User Break' + print + rc = 1 + + except self.InternalError: + print + print '* Internal Error (use --debug to display the traceback)' + if self.debug: + print + traceback.print_exc(20, sys.stdout) + elif self.verbose: + print ' %s: %s' % sys.exc_info()[:2] + print + rc = 1 + + raise SystemExit,rc + + def add_option(self, option): + + """ Add a new Option instance to the Application dynamically. + + Note that this has to be done *before* .parse() is being + executed. + + """ + self.options.append(option) + self.option_map[option.name] = option + + def startup(self): + + """ Set user defined instance variables. + + If this method returns anything other than None, the + process is terminated with the return value as exit code. + + """ + return None + + def exit(self, rc=0): + + """ Exit the program. + + rc is used as exit code and passed back to the calling + program. It defaults to 0 which usually means: OK. + + """ + raise SystemExit, rc + + def parse(self): + + """ Parse the command line and fill in self.values and self.files. + + After having parsed the options, the remaining command line + arguments are interpreted as files and passed to .handle_files() + for processing. + + As final step the option handlers are called in the order + of the options given on the command line. + + """ + # Parse arguments + self.values = values = {} + for o in self.options: + if o.has_default: + values[o.prefix+o.name] = o.default + else: + values[o.prefix+o.name] = 0 + flags,lflags = _getopt_flags(self.options) + try: + optlist,files = getopt.getopt(self.arguments,flags,lflags) + if self.globbing: + l = [] + for f in files: + gf = glob.glob(f) + if not gf: + l.append(f) + else: + l[len(l):] = gf + files = l + self.optionlist = optlist + self.files = files + self.files + except getopt.error,why: + self.help(why) + sys.exit(1) + + # Call file handler + rc = self.handle_files(self.files) + if rc is not None: + sys.exit(rc) + + # Call option handlers + for optionname, value in optlist: + + # Try to convert value to integer + try: + value = string.atoi(value) + except ValueError: + pass + + # Find handler and call it (or count the number of option + # instances on the command line) + handlername = 'handle' + string.replace(optionname, '-', '_') + try: + handler = getattr(self, handlername) + except AttributeError: + if value == '': + # count the number of occurances + if values.has_key(optionname): + values[optionname] = values[optionname] + 1 + else: + values[optionname] = 1 + else: + values[optionname] = value + else: + rc = handler(value) + if rc is not None: + raise SystemExit, rc + + # Apply final file check (for backward compatibility) + rc = self.check_files(self.files) + if rc is not None: + sys.exit(rc) + + def check_files(self,filelist): + + """ Apply some user defined checks on the files given in filelist. + + This may modify filelist in place. A typical application + is checking that at least n files are given. + + If this method returns anything other than None, the + process is terminated with the return value as exit code. + + """ + return None + + def help(self,note=''): + + self.print_header() + if self.synopsis: + print 'Synopsis:' + # To remain backward compatible: + try: + synopsis = self.synopsis % self.name + except (NameError, KeyError, TypeError): + synopsis = self.synopsis % self.__dict__ + print ' ' + synopsis + print + self.print_options() + if self.version: + print 'Version:' + print ' %s' % self.version + print + if self.about: + print string.strip(self.about % self.__dict__) + print + if note: + print '-'*72 + print 'Note:',note + print + + def notice(self,note): + + print '-'*72 + print 'Note:',note + print '-'*72 + print + + def print_header(self): + + print '-'*72 + print self.header % self.__dict__ + print '-'*72 + print + + def print_options(self): + + options = self.options + print 'Options and default settings:' + if not options: + print ' None' + return + long = filter(lambda x: x.prefix == '--', options) + short = filter(lambda x: x.prefix == '-', options) + items = short + long + for o in options: + print ' ',o + print + + # + # Example handlers: + # + # If a handler returns anything other than None, processing stops + # and the return value is passed to sys.exit() as argument. + # + + # File handler + def handle_files(self,files): + + """ This may process the files list in place. + """ + return None + + # Short option handler + def handle_h(self,arg): + + self.help() + return 0 + + def handle_v(self, value): + + """ Turn on verbose output. + """ + self.verbose = 1 + + # Handlers for long options have two underscores in their name + def handle__help(self,arg): + + self.help() + return 0 + + def handle__debug(self,arg): + + self.debug = 1 + # We don't want to catch internal errors: + self.InternalError = None + + def handle__copyright(self,arg): + + self.print_header() + print string.strip(self.copyright % self.__dict__) + print + return 0 + + def handle__examples(self,arg): + + self.print_header() + if self.examples: + print 'Examples:' + print + print string.strip(self.examples % self.__dict__) + print + else: + print 'No examples available.' + print + return 0 + + def main(self): + + """ Override this method as program entry point. + + The return value is passed to sys.exit() as argument. If + it is None, 0 is assumed (meaning OK). Unhandled + exceptions are reported with exit status code 1 (see + __init__ for further details). + + """ + return None + +# Alias +CommandLine = Application + +def _test(): + + class MyApplication(Application): + header = 'Test Application' + version = __version__ + options = [Option('-v','verbose')] + + def handle_v(self,arg): + print 'VERBOSE, Yeah !' + + cmd = MyApplication() + if not cmd.values['-h']: + cmd.help() + print 'files:',cmd.files + print 'Bye...' + +if __name__ == '__main__': + _test() Added: external/pybench-2.0/Constructs.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Constructs.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,564 @@ +from pybench import Test + +class IfThenElse(Test): + + version = 2.0 + operations = 30*3 # hard to say... + rounds = 150000 + + def test(self): + + a,b,c = 1,2,3 + for i in xrange(self.rounds): + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + def calibrate(self): + + a,b,c = 1,2,3 + for i in xrange(self.rounds): + pass + +class NestedForLoops(Test): + + version = 2.0 + operations = 1000*10*5 + rounds = 300 + + def test(self): + + l1 = range(1000) + l2 = range(10) + l3 = range(5) + for i in xrange(self.rounds): + for i in l1: + for j in l2: + for k in l3: + pass + + def calibrate(self): + + l1 = range(1000) + l2 = range(10) + l3 = range(5) + for i in xrange(self.rounds): + pass + +class ForLoops(Test): + + version = 2.0 + operations = 5 * 5 + rounds = 10000 + + def test(self): + + l1 = range(100) + for i in xrange(self.rounds): + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + def calibrate(self): + + l1 = range(1000) + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/Dict.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Dict.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,504 @@ +from pybench import Test + +class DictCreation(Test): + + version = 2.0 + operations = 5*(5 + 5) + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class DictWithStringKeys(Test): + + version = 2.0 + operations = 5*(6 + 6) + rounds = 200000 + + def test(self): + + d = {} + + for i in xrange(self.rounds): + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + def calibrate(self): + + d = {} + + for i in xrange(self.rounds): + pass + +class DictWithFloatKeys(Test): + + version = 2.0 + operations = 5*(6 + 6) + rounds = 150000 + + def test(self): + + d = {} + + for i in xrange(self.rounds): + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + def calibrate(self): + + d = {} + + for i in xrange(self.rounds): + pass + +class DictWithIntegerKeys(Test): + + version = 2.0 + operations = 5*(6 + 6) + rounds = 200000 + + def test(self): + + d = {} + + for i in xrange(self.rounds): + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + def calibrate(self): + + d = {} + + for i in xrange(self.rounds): + pass + +class SimpleDictManipulation(Test): + + version = 2.0 + operations = 5*(6 + 6 + 6 + 6) + rounds = 100000 + + def test(self): + + d = {} + has_key = d.has_key + + for i in xrange(self.rounds): + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + def calibrate(self): + + d = {} + has_key = d.has_key + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/Exceptions.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Exceptions.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,699 @@ +from pybench import Test + +class TryRaiseExcept(Test): + + version = 2.0 + operations = 2 + 3 + 3 + rounds = 80000 + + def test(self): + + error = ValueError + + for i in xrange(self.rounds): + try: + raise error + except: + pass + try: + raise error + except: + pass + try: + raise error,"something" + except: + pass + try: + raise error,"something" + except: + pass + try: + raise error,"something" + except: + pass + try: + raise error("something") + except: + pass + try: + raise error("something") + except: + pass + try: + raise error("something") + except: + pass + + def calibrate(self): + + error = ValueError + + for i in xrange(self.rounds): + pass + + +class TryExcept(Test): + + version = 2.0 + operations = 15 * 10 + rounds = 150000 + + def test(self): + + for i in xrange(self.rounds): + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +### Test to make Fredrik happy... + +if __name__ == '__main__': + import timeit + timeit.TestClass = TryRaiseExcept + timeit.main(['-s', 'test = TestClass(); test.rounds = 1000', + 'test.test()']) Added: external/pybench-2.0/Imports.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Imports.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,138 @@ +from pybench import Test + +# First imports: +import os +import package.submodule + +class SecondImport(Test): + + version = 2.0 + operations = 5 * 5 + rounds = 40000 + + def test(self): + + for i in xrange(self.rounds): + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class SecondPackageImport(Test): + + version = 2.0 + operations = 5 * 5 + rounds = 40000 + + def test(self): + + for i in xrange(self.rounds): + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SecondSubmoduleImport(Test): + + version = 2.0 + operations = 5 * 5 + rounds = 40000 + + def test(self): + + for i in xrange(self.rounds): + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + def calibrate(self): + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/Instances.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Instances.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,66 @@ +from pybench import Test + +class CreateInstances(Test): + + version = 2.0 + operations = 3 + 7 + 4 + rounds = 80000 + + def test(self): + + class c: + pass + + class d: + def __init__(self,a,b,c): + self.a = a + self.b = b + self.c = c + + class e: + def __init__(self,a,b,c=4): + self.a = a + self.b = b + self.c = c + self.d = a + self.e = b + self.f = c + + for i in xrange(self.rounds): + o = c() + o1 = c() + o2 = c() + p = d(i,i,3) + p1 = d(i,i,3) + p2 = d(i,3,3) + p3 = d(3,i,3) + p4 = d(i,i,i) + p5 = d(3,i,3) + p6 = d(i,i,i) + q = e(i,i,3) + q1 = e(i,i,3) + q2 = e(i,i,3) + q3 = e(i,i) + + def calibrate(self): + + class c: + pass + + class d: + def __init__(self,a,b,c): + self.a = a + self.b = b + self.c = c + + class e: + def __init__(self,a,b,c=4): + self.a = a + self.b = b + self.c = c + self.d = a + self.e = b + self.f = c + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/LICENSE ============================================================================== --- (empty file) +++ external/pybench-2.0/LICENSE Tue Jun 13 18:45:48 2006 @@ -0,0 +1,25 @@ +pybench License +--------------- + +This copyright notice and license applies to all files in the pybench +directory of the pybench distribution. + +Copyright (c), 1997-2006, Marc-Andre Lemburg (mal at lemburg.com) +Copyright (c), 2000-2006, eGenix.com Software GmbH (info at egenix.com) + + All Rights Reserved. + +Permission to use, copy, modify, and distribute this software and its +documentation for any purpose and without fee or royalty is hereby +granted, provided that the above copyright notice appear in all copies +and that both that copyright notice and this permission notice appear +in supporting documentation or portions thereof, including +modifications, that you make. + +THE AUTHOR MARC-ANDRE LEMBURG DISCLAIMS ALL WARRANTIES WITH REGARD TO +THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND +FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, +INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING +FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, +NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION +WITH THE USE OR PERFORMANCE OF THIS SOFTWARE ! Added: external/pybench-2.0/Lists.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Lists.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,295 @@ +from pybench import Test + +class SimpleListManipulation(Test): + + version = 2.0 + operations = 5* (6 + 6 + 6) + rounds = 130000 + + def test(self): + + l = [] + append = l.append + + for i in xrange(self.rounds): + + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + if len(l) > 10000: + # cut down the size + del l[:] + + def calibrate(self): + + l = [] + append = l.append + + for i in xrange(self.rounds): + pass + +class ListSlicing(Test): + + version = 2.0 + operations = 25*(3+1+2+1) + rounds = 800 + + def test(self): + + n = range(100) + r = range(25) + + for i in xrange(self.rounds): + + l = n[:] + + for j in r: + + m = l[50:] + m = l[:25] + m = l[50:55] + l[:3] = n + m = l[:-1] + m = l[1:] + l[-1:] = n + + def calibrate(self): + + n = range(100) + r = range(25) + + for i in xrange(self.rounds): + for j in r: + pass + +class SmallLists(Test): + + version = 2.0 + operations = 5*(1+ 6 + 6 + 3 + 1) + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + + l = [] + + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + def calibrate(self): + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/Lookups.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Lookups.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,945 @@ +from pybench import Test + +class SpecialClassAttribute(Test): + + version = 2.0 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + + for i in xrange(self.rounds): + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + def calibrate(self): + + class c: + pass + + for i in xrange(self.rounds): + pass + +class NormalClassAttribute(Test): + + version = 2.0 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + + for i in xrange(self.rounds): + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + def calibrate(self): + + class c: + pass + + for i in xrange(self.rounds): + pass + +class SpecialInstanceAttribute(Test): + + version = 2.0 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + def calibrate(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + pass + +class NormalInstanceAttribute(Test): + + version = 2.0 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + def calibrate(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + pass + +class BuiltinMethodLookup(Test): + + version = 2.0 + operations = 5*(3*5 + 3*5) + rounds = 70000 + + def test(self): + + l = [] + d = {} + + for i in xrange(self.rounds): + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + def calibrate(self): + + l = [] + d = {} + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/NewInstances.py ============================================================================== --- (empty file) +++ external/pybench-2.0/NewInstances.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,75 @@ +from pybench import Test + +# Check for new-style class support: +try: + class c(object): + pass +except NameError: + raise ImportError + +### + +class CreateNewInstances(Test): + + version = 2.0 + operations = 3 + 7 + 4 + rounds = 60000 + + def test(self): + + class c(object): + pass + + class d(object): + def __init__(self,a,b,c): + self.a = a + self.b = b + self.c = c + + class e(object): + def __init__(self,a,b,c=4): + self.a = a + self.b = b + self.c = c + self.d = a + self.e = b + self.f = c + + for i in xrange(self.rounds): + o = c() + o1 = c() + o2 = c() + p = d(i,i,3) + p1 = d(i,i,3) + p2 = d(i,3,3) + p3 = d(3,i,3) + p4 = d(i,i,i) + p5 = d(3,i,3) + p6 = d(i,i,i) + q = e(i,i,3) + q1 = e(i,i,3) + q2 = e(i,i,3) + q3 = e(i,i) + + def calibrate(self): + + class c(object): + pass + + class d(object): + def __init__(self,a,b,c): + self.a = a + self.b = b + self.c = c + + class e(object): + def __init__(self,a,b,c=4): + self.a = a + self.b = b + self.c = c + self.d = a + self.e = b + self.f = c + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/Numbers.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Numbers.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,784 @@ +from pybench import Test + +class CompareIntegers(Test): + + version = 2.0 + operations = 30 * 5 + rounds = 120000 + + def test(self): + + for i in xrange(self.rounds): + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class CompareFloats(Test): + + version = 2.0 + operations = 30 * 5 + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class CompareFloatsIntegers(Test): + + version = 2.0 + operations = 30 * 5 + rounds = 60000 + + def test(self): + + for i in xrange(self.rounds): + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class CompareLongs(Test): + + version = 2.0 + operations = 30 * 5 + rounds = 70000 + + def test(self): + + for i in xrange(self.rounds): + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + def calibrate(self): + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/README ============================================================================== --- (empty file) +++ external/pybench-2.0/README Tue Jun 13 18:45:48 2006 @@ -0,0 +1,368 @@ +________________________________________________________________________ + +PYBENCH - A Python Benchmark Suite +________________________________________________________________________ + + Extendable suite of of low-level benchmarks for measuring + the performance of the Python implementation + (interpreter, compiler or VM). + +pybench is a collection of tests that provides a standardized way to +measure the performance of Python implementations. It takes a very +close look at different aspects of Python programs and let's you +decide which factors are more important to you than others, rather +than wrapping everything up in one number, like the other performance +tests do (e.g. pystone which is included in the Python Standard +Library). + +pybench has been used in the past by several Python developers to +track down performance bottlenecks or to demonstrate the impact of +optimizations and new features in Python. + +The command line interface for pybench is the file pybench.py. Run +this script with option '--help' to get a listing of the possible +options. Without options, pybench will simply execute the benchmark +and then print out a report to stdout. + + +Micro-Manual +------------ + +Run 'pybench.py -h' to see the help screen. Run 'pybench.py' to run +the benchmark suite using default settings and 'pybench.py -f ' +to have it store the results in a file too. + +It is usually a good idea to run pybench.py multiple times to see +whether the environment, timers and benchmark run-times are suitable +for doing benchmark tests. + +You can use the comparison feature of pybench.py ('pybench.py -c +') to check how well the system behaves in comparison to a +reference run. + +If the differences are well below 10% for each test, then you have a +system that is good for doing benchmark testings. Of you get random +differences of more than 10% or significant differences between the +values for minimum and average time, then you likely have some +background processes running which cause the readings to become +inconsistent. Examples include: web-browsers, email clients, RSS +readers, music players, backup programs, etc. + +If you are only interested in a few tests of the whole suite, you can +use the filtering option, e.g. 'pybench.py -t string' will only +run/show the tests that have 'string' in their name. + +This is the current output of pybench.py --help: + +""" +------------------------------------------------------------------------ +PYBENCH - a benchmark test suite for Python interpreters/compilers. +------------------------------------------------------------------------ + +Synopsis: + pybench.py [option] files... + +Options and default settings: + -n arg number of rounds (10) + -f arg save benchmark to file arg () + -c arg compare benchmark with the one in file arg () + -s arg show benchmark in file arg, then exit () + -w arg set warp factor to arg (10) + -t arg run only tests with names matching arg () + -C arg set the number of calibration runs to arg (20) + -d hide noise in comparisons (0) + -v verbose output (not recommended) (0) + --with-gc enable garbage collection (0) + --with-syscheck use default sys check interval (0) + --timer arg use given timer (time.time) + -h show this help text + --help show this help text + --debug enable debugging + --copyright show copyright + --examples show examples of usage + +Version: + 2.0 + +The normal operation is to run the suite and display the +results. Use -f to save them for later reuse or comparisons. + +Available timers: + + time.time + time.clock + systimes.processtime + +Examples: + +python2.1 pybench.py -f p21.pybench +python2.5 pybench.py -f p25.pybench +python pybench.py -s p25.pybench -c p21.pybench +""" + +License +------- + +See LICENSE file. + + +Sample output +------------- + +""" +------------------------------------------------------------------------------- +PYBENCH 2.0 +------------------------------------------------------------------------------- +* using Python 2.4.2 +* disabled garbage collection +* system check interval set to maximum: 2147483647 +* using timer: time.time + +Calibrating tests. Please wait... + +Running 10 round(s) of the suite at warp factor 10: + +* Round 1 done in 6.388 seconds. +* Round 2 done in 6.485 seconds. +* Round 3 done in 6.786 seconds. +... +* Round 10 done in 6.546 seconds. + +------------------------------------------------------------------------------- +Benchmark: 2006-06-12 12:09:25 +------------------------------------------------------------------------------- + + Rounds: 10 + Warp: 10 + Timer: time.time + + Machine Details: + Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64 + Processor: x86_64 + + Python: + Executable: /usr/local/bin/python + Version: 2.4.2 + Compiler: GCC 3.3.4 (pre 3.3.5 20040809) + Bits: 64bit + Build: Oct 1 2005 15:24:35 (#1) + Unicode: UCS2 + + +Test minimum average operation overhead +------------------------------------------------------------------------------- + BuiltinFunctionCalls: 126ms 145ms 0.28us 0.274ms + BuiltinMethodLookup: 124ms 130ms 0.12us 0.316ms + CompareFloats: 109ms 110ms 0.09us 0.361ms + CompareFloatsIntegers: 100ms 104ms 0.12us 0.271ms + CompareIntegers: 137ms 138ms 0.08us 0.542ms + CompareInternedStrings: 124ms 127ms 0.08us 1.367ms + CompareLongs: 100ms 104ms 0.10us 0.316ms + CompareStrings: 111ms 115ms 0.12us 0.929ms + CompareUnicode: 108ms 128ms 0.17us 0.693ms + ConcatStrings: 142ms 155ms 0.31us 0.562ms + ConcatUnicode: 119ms 127ms 0.42us 0.384ms + CreateInstances: 123ms 128ms 1.14us 0.367ms + CreateNewInstances: 121ms 126ms 1.49us 0.335ms + CreateStringsWithConcat: 130ms 135ms 0.14us 0.916ms + CreateUnicodeWithConcat: 130ms 135ms 0.34us 0.361ms + DictCreation: 108ms 109ms 0.27us 0.361ms + DictWithFloatKeys: 149ms 153ms 0.17us 0.678ms + DictWithIntegerKeys: 124ms 126ms 0.11us 0.915ms + DictWithStringKeys: 114ms 117ms 0.10us 0.905ms + ForLoops: 110ms 111ms 4.46us 0.063ms + IfThenElse: 118ms 119ms 0.09us 0.685ms + ListSlicing: 116ms 120ms 8.59us 0.103ms + NestedForLoops: 125ms 137ms 0.09us 0.019ms + NormalClassAttribute: 124ms 136ms 0.11us 0.457ms + NormalInstanceAttribute: 110ms 117ms 0.10us 0.454ms + PythonFunctionCalls: 107ms 113ms 0.34us 0.271ms + PythonMethodCalls: 140ms 149ms 0.66us 0.141ms + Recursion: 156ms 166ms 3.32us 0.452ms + SecondImport: 112ms 118ms 1.18us 0.180ms + SecondPackageImport: 118ms 127ms 1.27us 0.180ms + SecondSubmoduleImport: 140ms 151ms 1.51us 0.180ms + SimpleComplexArithmetic: 128ms 139ms 0.16us 0.361ms + SimpleDictManipulation: 134ms 136ms 0.11us 0.452ms + SimpleFloatArithmetic: 110ms 113ms 0.09us 0.571ms + SimpleIntFloatArithmetic: 106ms 111ms 0.08us 0.548ms + SimpleIntegerArithmetic: 106ms 109ms 0.08us 0.544ms + SimpleListManipulation: 103ms 113ms 0.10us 0.587ms + SimpleLongArithmetic: 112ms 118ms 0.18us 0.271ms + SmallLists: 105ms 116ms 0.17us 0.366ms + SmallTuples: 108ms 128ms 0.24us 0.406ms + SpecialClassAttribute: 119ms 136ms 0.11us 0.453ms + SpecialInstanceAttribute: 143ms 155ms 0.13us 0.454ms + StringMappings: 115ms 121ms 0.48us 0.405ms + StringPredicates: 120ms 129ms 0.18us 2.064ms + StringSlicing: 111ms 127ms 0.23us 0.781ms + TryExcept: 125ms 126ms 0.06us 0.681ms + TryRaiseExcept: 133ms 137ms 2.14us 0.361ms + TupleSlicing: 117ms 120ms 0.46us 0.066ms + UnicodeMappings: 156ms 160ms 4.44us 0.429ms + UnicodePredicates: 117ms 121ms 0.22us 2.487ms + UnicodeProperties: 115ms 153ms 0.38us 2.070ms + UnicodeSlicing: 126ms 129ms 0.26us 0.689ms +------------------------------------------------------------------------------- +Totals: 6283ms 6673ms +""" +________________________________________________________________________ + +Writing New Tests +________________________________________________________________________ + +pybench tests are simple modules defining one or more pybench.Test +subclasses. + +Writing a test essentially boils down to providing two methods: +.test() which runs .rounds number of .operations test operations each +and .calibrate() which does the same except that it doesn't actually +execute the operations. + + +Here's an example: +------------------ + +from pybench import Test + +class IntegerCounting(Test): + + # Version number of the test as float (x.yy); this is important + # for comparisons of benchmark runs - tests with unequal version + # number will not get compared. + version = 1.0 + + # The number of abstract operations done in each round of the + # test. An operation is the basic unit of what you want to + # measure. The benchmark will output the amount of run-time per + # operation. Note that in order to raise the measured timings + # significantly above noise level, it is often required to repeat + # sets of operations more than once per test round. The measured + # overhead per test round should be less than 1 second. + operations = 20 + + # Number of rounds to execute per test run. This should be + # adjusted to a figure that results in a test run-time of between + # 1-2 seconds (at warp 1). + rounds = 100000 + + def test(self): + + """ Run the test. + + The test needs to run self.rounds executing + self.operations number of operations each. + + """ + # Init the test + a = 1 + + # Run test rounds + # + # NOTE: Use xrange() for all test loops unless you want to face + # a 20MB process ! + # + for i in xrange(self.rounds): + + # Repeat the operations per round to raise the run-time + # per operation significantly above the noise level of the + # for-loop overhead. + + # Execute 20 operations (a += 1): + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + a += 1 + + def calibrate(self): + + """ Calibrate the test. + + This method should execute everything that is needed to + setup and run the test - except for the actual operations + that you intend to measure. pybench uses this method to + measure the test implementation overhead. + + """ + # Init the test + a = 1 + + # Run test rounds (without actually doing any operation) + for i in xrange(self.rounds): + + # Skip the actual execution of the operations, since we + # only want to measure the test's administration overhead. + pass + +Registering a new test module +----------------------------- + +To register a test module with pybench, the classes need to be +imported into the pybench.Setup module. pybench will then scan all the +symbols defined in that module for subclasses of pybench.Test and +automatically add them to the benchmark suite. + + +Breaking Comparability +---------------------- + +If a change is made to any individual test that means it is no +longer strictly comparable with previous runs, the '.version' class +variable should be updated. Therefafter, comparisons with previous +versions of the test will list as "n/a" to reflect the change. + + +Version History +--------------- + + 2.0: rewrote parts of pybench which resulted in more repeatable + timings: + - made timer a parameter + - changed the platform default timer to use high-resolution + timers rather than process timers (which have a much lower + resolution) + - added option to select timer + - added process time timer (using systimes.py) + - changed to use min() as timing estimator (average + is still taken as well to provide an idea of the difference) + - garbage collection is turned off per default + - sys check interval is set to the highest possible value + - calibration is now a separate step and done using + a different strategy that allows measuring the test + overhead more accurately + - modified the tests to each give a run-time of between + 100-200ms using warp 10 + - changed default warp factor to 10 (from 20) + - compared results with timeit.py and confirmed measurements + - bumped all test versions to 2.0 + - updated platform.py to the latest version + - changed the output format a bit to make it look + nicer + - refactored the APIs somewhat + 1.3+: Steve Holden added the NewInstances test and the filtering + option during the NeedForSpeed sprint; this also triggered a long + discussion on how to improve benchmark timing and finally + resulted in the release of 2.0 + 1.3: initial checkin into the Python SVN repository + + +Have fun, +-- +Marc-Andre Lemburg +mal at lemburg.com Added: external/pybench-2.0/Setup.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Setup.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,39 @@ +#!python + +# Setup file for pybench +# +# This file has to import all tests to be run; it is executed as +# Python source file, so you can do all kinds of manipulations here +# rather than having to edit the tests themselves. +# +# Note: Please keep this module compatible to Python 1.5.2. +# +# Tests may include features in later Python versions, but these +# should then be embedded in try-except clauses in this configuration +# module. + +# Defaults +Number_of_rounds = 10 +Warp_factor = 10 + +# Import tests +from Arithmetic import * +from Calls import * +from Constructs import * +from Lookups import * +from Instances import * +try: + from NewInstances import * +except ImportError: + pass +from Lists import * +from Tuples import * +from Dict import * +from Exceptions import * +from Imports import * +from Strings import * +from Numbers import * +try: + from Unicode import * +except (ImportError, SyntaxError): + pass Added: external/pybench-2.0/Strings.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Strings.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,562 @@ +from pybench import Test +from string import join + +class ConcatStrings(Test): + + version = 2.0 + operations = 10 * 5 + rounds = 100000 + + def test(self): + + # Make sure the strings are *not* interned + s = join(map(str,range(100))) + t = join(map(str,range(1,101))) + + for i in xrange(self.rounds): + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + def calibrate(self): + + s = join(map(str,range(100))) + t = join(map(str,range(1,101))) + + for i in xrange(self.rounds): + pass + + +class CompareStrings(Test): + + version = 2.0 + operations = 10 * 5 + rounds = 200000 + + def test(self): + + # Make sure the strings are *not* interned + s = join(map(str,range(10))) + t = join(map(str,range(10))) + "abc" + + for i in xrange(self.rounds): + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + def calibrate(self): + + s = join(map(str,range(10))) + t = join(map(str,range(10))) + "abc" + + for i in xrange(self.rounds): + pass + + +class CompareInternedStrings(Test): + + version = 2.0 + operations = 10 * 5 + rounds = 300000 + + def test(self): + + # Make sure the strings *are* interned + s = intern(join(map(str,range(10)))) + t = s + + for i in xrange(self.rounds): + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + def calibrate(self): + + s = intern(join(map(str,range(10)))) + t = s + + for i in xrange(self.rounds): + pass + + +class CreateStringsWithConcat(Test): + + version = 2.0 + operations = 10 * 5 + rounds = 200000 + + def test(self): + + for i in xrange(self.rounds): + s = 'om' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class StringSlicing(Test): + + version = 2.0 + operations = 5 * 7 + rounds = 160000 + + def test(self): + + s = join(map(str,range(100))) + + for i in xrange(self.rounds): + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + def calibrate(self): + + s = join(map(str,range(100))) + + for i in xrange(self.rounds): + pass + +### String methods + +if hasattr('', 'lower'): + + class StringMappings(Test): + + version = 2.0 + operations = 3 * (5 + 4 + 2 + 1) + rounds = 70000 + + def test(self): + + s = join(map(chr,range(20)),'') + t = join(map(chr,range(50)),'') + u = join(map(chr,range(100)),'') + v = join(map(chr,range(256)),'') + + for i in xrange(self.rounds): + + s.lower() + s.lower() + s.lower() + s.lower() + s.lower() + + s.upper() + s.upper() + s.upper() + s.upper() + s.upper() + + s.title() + s.title() + s.title() + s.title() + s.title() + + t.lower() + t.lower() + t.lower() + t.lower() + + t.upper() + t.upper() + t.upper() + t.upper() + + t.title() + t.title() + t.title() + t.title() + + u.lower() + u.lower() + + u.upper() + u.upper() + + u.title() + u.title() + + v.lower() + + v.upper() + + v.title() + + def calibrate(self): + + s = join(map(chr,range(20)),'') + t = join(map(chr,range(50)),'') + u = join(map(chr,range(100)),'') + v = join(map(chr,range(256)),'') + + for i in xrange(self.rounds): + pass + + class StringPredicates(Test): + + version = 2.0 + operations = 10 * 7 + rounds = 100000 + + def test(self): + + data = ('abc', '123', ' ', '\xe4\xf6\xfc', '\xdf'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + def calibrate(self): + + data = ('abc', '123', ' ', '\u1234\u2345\u3456', '\uFFFF'*10) + data = ('abc', '123', ' ', '\xe4\xf6\xfc', '\xdf'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] Added: external/pybench-2.0/Tuples.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Tuples.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,360 @@ +from pybench import Test + +class TupleSlicing(Test): + + version = 2.0 + operations = 3 * 25 * 10 * 7 + rounds = 500 + + def test(self): + + r = range(25) + t = tuple(range(100)) + + for i in xrange(self.rounds): + + for j in r: + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + def calibrate(self): + + r = range(25) + t = tuple(range(100)) + + for i in xrange(self.rounds): + for j in r: + pass + +class SmallTuples(Test): + + version = 2.0 + operations = 5*(1 + 3 + 6 + 2) + rounds = 90000 + + def test(self): + + for i in xrange(self.rounds): + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + def calibrate(self): + + for i in xrange(self.rounds): + pass Added: external/pybench-2.0/Unicode.py ============================================================================== --- (empty file) +++ external/pybench-2.0/Unicode.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,542 @@ +try: + unicode +except NameError: + raise ImportError + +from pybench import Test +from string import join + +class ConcatUnicode(Test): + + version = 2.0 + operations = 10 * 5 + rounds = 60000 + + def test(self): + + # Make sure the strings are *not* interned + s = unicode(join(map(str,range(100)))) + t = unicode(join(map(str,range(1,101)))) + + for i in xrange(self.rounds): + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + def calibrate(self): + + s = unicode(join(map(str,range(100)))) + t = unicode(join(map(str,range(1,101)))) + + for i in xrange(self.rounds): + pass + + +class CompareUnicode(Test): + + version = 2.0 + operations = 10 * 5 + rounds = 150000 + + def test(self): + + # Make sure the strings are *not* interned + s = unicode(join(map(str,range(10)))) + t = unicode(join(map(str,range(10))) + "abc") + + for i in xrange(self.rounds): + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + def calibrate(self): + + s = unicode(join(map(str,range(10)))) + t = unicode(join(map(str,range(10))) + "abc") + + for i in xrange(self.rounds): + pass + + +class CreateUnicodeWithConcat(Test): + + version = 2.0 + operations = 10 * 5 + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + s = u'om' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class UnicodeSlicing(Test): + + version = 2.0 + operations = 5 * 7 + rounds = 140000 + + def test(self): + + s = unicode(join(map(str,range(100)))) + + for i in xrange(self.rounds): + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + def calibrate(self): + + s = unicode(join(map(str,range(100)))) + + for i in xrange(self.rounds): + pass + +### String methods + +class UnicodeMappings(Test): + + version = 2.0 + operations = 3 * (5 + 4 + 2 + 1) + rounds = 10000 + + def test(self): + + s = join(map(unichr,range(20)),'') + t = join(map(unichr,range(100)),'') + u = join(map(unichr,range(500)),'') + v = join(map(unichr,range(1000)),'') + + for i in xrange(self.rounds): + + s.lower() + s.lower() + s.lower() + s.lower() + s.lower() + + s.upper() + s.upper() + s.upper() + s.upper() + s.upper() + + s.title() + s.title() + s.title() + s.title() + s.title() + + t.lower() + t.lower() + t.lower() + t.lower() + + t.upper() + t.upper() + t.upper() + t.upper() + + t.title() + t.title() + t.title() + t.title() + + u.lower() + u.lower() + + u.upper() + u.upper() + + u.title() + u.title() + + v.lower() + + v.upper() + + v.title() + + def calibrate(self): + + s = join(map(unichr,range(20)),'') + t = join(map(unichr,range(100)),'') + u = join(map(unichr,range(500)),'') + v = join(map(unichr,range(1000)),'') + + for i in xrange(self.rounds): + pass + +class UnicodePredicates(Test): + + version = 2.0 + operations = 5 * 9 + rounds = 120000 + + def test(self): + + data = (u'abc', u'123', u' ', u'\u1234\u2345\u3456', u'\uFFFF'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + def calibrate(self): + + data = (u'abc', u'123', u' ', u'\u1234\u2345\u3456', u'\uFFFF'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] + +try: + import unicodedata +except ImportError: + pass +else: + class UnicodeProperties(Test): + + version = 2.0 + operations = 5 * 8 + rounds = 100000 + + def test(self): + + data = (u'a', u'1', u' ', u'\u1234', u'\uFFFF') + len_data = len(data) + digit = unicodedata.digit + numeric = unicodedata.numeric + decimal = unicodedata.decimal + category = unicodedata.category + bidirectional = unicodedata.bidirectional + decomposition = unicodedata.decomposition + mirrored = unicodedata.mirrored + combining = unicodedata.combining + + for i in xrange(self.rounds): + + c = data[i % len_data] + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + def calibrate(self): + + data = (u'a', u'1', u' ', u'\u1234', u'\uFFFF') + len_data = len(data) + digit = unicodedata.digit + numeric = unicodedata.numeric + decimal = unicodedata.decimal + category = unicodedata.category + bidirectional = unicodedata.bidirectional + decomposition = unicodedata.decomposition + mirrored = unicodedata.mirrored + combining = unicodedata.combining + + for i in xrange(self.rounds): + + c = data[i % len_data] Added: external/pybench-2.0/clockres.py ============================================================================== --- (empty file) +++ external/pybench-2.0/clockres.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,44 @@ +#!/usr/bin/env python + +""" clockres - calculates the resolution in seconds of a given timer. + + Copyright (c) 2006, Marc-Andre Lemburg (mal at egenix.com). See the + documentation for further information on copyrights, or contact + the author. All Rights Reserved. + +""" +import time + +TEST_TIME = 1.0 + +def clockres(timer): + d = {} + wallclock = time.time + start = wallclock() + stop = wallclock() + TEST_TIME + spin_loops = range(1000) + while 1: + now = wallclock() + if now >= stop: + break + for i in spin_loops: + d[timer()] = 1 + values = d.keys() + values.sort() + min_diff = TEST_TIME + for i in range(len(values) - 1): + diff = values[i+1] - values[i] + if diff < min_diff: + min_diff = diff + return min_diff + +if __name__ == '__main__': + print 'Clock resolution of various timer implementations:' + print 'time.clock: %10.3fus' % (clockres(time.clock) * 1e6) + print 'time.time: %10.3fus' % (clockres(time.time) * 1e6) + try: + import systimes + print 'systimes.processtime: %10.3fus' % (clockres(systimes.processtime) * 1e6) + except ImportError: + pass + Added: external/pybench-2.0/package/__init__.py ============================================================================== Added: external/pybench-2.0/package/submodule.py ============================================================================== Added: external/pybench-2.0/platform.py ============================================================================== --- (empty file) +++ external/pybench-2.0/platform.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,1254 @@ +#!/usr/bin/env python + +""" This module tries to retrieve as much platform-identifying data as + possible. It makes this information available via function APIs. + + If called from the command line, it prints the platform + information concatenated as single string to stdout. The output + format is useable as part of a filename. + +""" +# This module is maintained by Marc-Andre Lemburg . +# If you find problems, please submit bug reports/patches via the +# Python SourceForge Project Page and assign them to "lemburg". +# +# Note: Please keep this module compatible to Python 1.5.2. +# +# Still needed: +# * more support for WinCE +# * support for MS-DOS (PythonDX ?) +# * support for Amiga and other still unsupported platforms running Python +# * support for additional Linux distributions +# +# Many thanks to all those who helped adding platform-specific +# checks (in no particular order): +# +# Charles G Waldman, David Arnold, Gordon McMillan, Ben Darnell, +# Jeff Bauer, Cliff Crawford, Ivan Van Laningham, Josef +# Betancourt, Randall Hopper, Karl Putland, John Farrell, Greg +# Andruk, Just van Rossum, Thomas Heller, Mark R. Levinson, Mark +# Hammond, Bill Tutt, Hans Nowak, Uwe Zessin (OpenVMS support), +# Colin Kong, Trent Mick, Guido van Rossum +# +# History: +# +# +# +# 1.0.3 - added normalization of Windows system name +# 1.0.2 - added more Windows support +# 1.0.1 - reformatted to make doc.py happy +# 1.0.0 - reformatted a bit and checked into Python CVS +# 0.8.0 - added sys.version parser and various new access +# APIs (python_version(), python_compiler(), etc.) +# 0.7.2 - fixed architecture() to use sizeof(pointer) where available +# 0.7.1 - added support for Caldera OpenLinux +# 0.7.0 - some fixes for WinCE; untabified the source file +# 0.6.2 - support for OpenVMS - requires version 1.5.2-V006 or higher and +# vms_lib.getsyi() configured +# 0.6.1 - added code to prevent 'uname -p' on platforms which are +# known not to support it +# 0.6.0 - fixed win32_ver() to hopefully work on Win95,98,NT and Win2k; +# did some cleanup of the interfaces - some APIs have changed +# 0.5.5 - fixed another type in the MacOS code... should have +# used more coffee today ;-) +# 0.5.4 - fixed a few typos in the MacOS code +# 0.5.3 - added experimental MacOS support; added better popen() +# workarounds in _syscmd_ver() -- still not 100% elegant +# though +# 0.5.2 - fixed uname() to return '' instead of 'unknown' in all +# return values (the system uname command tends to return +# 'unknown' instead of just leaving the field emtpy) +# 0.5.1 - included code for slackware dist; added exception handlers +# to cover up situations where platforms don't have os.popen +# (e.g. Mac) or fail on socket.gethostname(); fixed libc +# detection RE +# 0.5.0 - changed the API names referring to system commands to *syscmd*; +# added java_ver(); made syscmd_ver() a private +# API (was system_ver() in previous versions) -- use uname() +# instead; extended the win32_ver() to also return processor +# type information +# 0.4.0 - added win32_ver() and modified the platform() output for WinXX +# 0.3.4 - fixed a bug in _follow_symlinks() +# 0.3.3 - fixed popen() and "file" command invokation bugs +# 0.3.2 - added architecture() API and support for it in platform() +# 0.3.1 - fixed syscmd_ver() RE to support Windows NT +# 0.3.0 - added system alias support +# 0.2.3 - removed 'wince' again... oh well. +# 0.2.2 - added 'wince' to syscmd_ver() supported platforms +# 0.2.1 - added cache logic and changed the platform string format +# 0.2.0 - changed the API to use functions instead of module globals +# since some action take too long to be run on module import +# 0.1.0 - first release +# +# You can always get the latest version of this module at: +# +# http://www.egenix.com/files/python/platform.py +# +# If that URL should fail, try contacting the author. + +__copyright__ = """ + Copyright (c) 1999-2000, Marc-Andre Lemburg; mailto:mal at lemburg.com + Copyright (c) 2000-2003, eGenix.com Software GmbH; mailto:info at egenix.com + + Permission to use, copy, modify, and distribute this software and its + documentation for any purpose and without fee or royalty is hereby granted, + provided that the above copyright notice appear in all copies and that + both that copyright notice and this permission notice appear in + supporting documentation or portions thereof, including modifications, + that you make. + + EGENIX.COM SOFTWARE GMBH DISCLAIMS ALL WARRANTIES WITH REGARD TO + THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND + FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, + INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING + FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, + NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION + WITH THE USE OR PERFORMANCE OF THIS SOFTWARE ! + +""" + +__version__ = '1.0.4' + +import sys,string,os,re + +### Platform specific APIs + +_libc_search = re.compile(r'(__libc_init)' + '|' + '(GLIBC_([0-9.]+))' + '|' + '(libc(_\w+)?\.so(?:\.(\d[0-9.]*))?)') + +def libc_ver(executable=sys.executable,lib='',version='', + + chunksize=2048): + + """ Tries to determine the libc version that the file executable + (which defaults to the Python interpreter) is linked against. + + Returns a tuple of strings (lib,version) which default to the + given parameters in case the lookup fails. + + Note that the function has intimate knowledge of how different + libc versions add symbols to the executable and thus is probably + only useable for executables compiled using gcc. + + The file is read and scanned in chunks of chunksize bytes. + + """ + f = open(executable,'rb') + binary = f.read(chunksize) + pos = 0 + while 1: + m = _libc_search.search(binary,pos) + if not m: + binary = f.read(chunksize) + if not binary: + break + pos = 0 + continue + libcinit,glibc,glibcversion,so,threads,soversion = m.groups() + if libcinit and not lib: + lib = 'libc' + elif glibc: + if lib != 'glibc': + lib = 'glibc' + version = glibcversion + elif glibcversion > version: + version = glibcversion + elif so: + if lib != 'glibc': + lib = 'libc' + if soversion > version: + version = soversion + if threads and version[-len(threads):] != threads: + version = version + threads + pos = m.end() + f.close() + return lib,version + +def _dist_try_harder(distname,version,id): + + """ Tries some special tricks to get the distribution + information in case the default method fails. + + Currently supports older SuSE Linux, Caldera OpenLinux and + Slackware Linux distributions. + + """ + if os.path.exists('/var/adm/inst-log/info'): + # SuSE Linux stores distribution information in that file + info = open('/var/adm/inst-log/info').readlines() + distname = 'SuSE' + for line in info: + tv = string.split(line) + if len(tv) == 2: + tag,value = tv + else: + continue + if tag == 'MIN_DIST_VERSION': + version = string.strip(value) + elif tag == 'DIST_IDENT': + values = string.split(value,'-') + id = values[2] + return distname,version,id + + if os.path.exists('/etc/.installed'): + # Caldera OpenLinux has some infos in that file (thanks to Colin Kong) + info = open('/etc/.installed').readlines() + for line in info: + pkg = string.split(line,'-') + if len(pkg) >= 2 and pkg[0] == 'OpenLinux': + # XXX does Caldera support non Intel platforms ? If yes, + # where can we find the needed id ? + return 'OpenLinux',pkg[1],id + + if os.path.isdir('/usr/lib/setup'): + # Check for slackware verson tag file (thanks to Greg Andruk) + verfiles = os.listdir('/usr/lib/setup') + for n in range(len(verfiles)-1, -1, -1): + if verfiles[n][:14] != 'slack-version-': + del verfiles[n] + if verfiles: + verfiles.sort() + distname = 'slackware' + version = verfiles[-1][14:] + return distname,version,id + + return distname,version,id + +_release_filename = re.compile(r'(\w+)[-_](release|version)') +_release_version = re.compile(r'([\d.]+)[^(]*(?:$(.+)$)?') + +# Note:In supported_dists below we need 'fedora' before 'redhat' as in +# Fedora redhat-release is a link to fedora-release. + +def dist(distname='',version='',id='', + + supported_dists=('SuSE', 'debian', 'fedora', 'redhat', 'mandrake')): + + """ Tries to determine the name of the Linux OS distribution name. + + The function first looks for a distribution release file in + /etc and then reverts to _dist_try_harder() in case no + suitable files are found. + + Returns a tuple (distname,version,id) which default to the + args given as parameters. + + """ + try: + etc = os.listdir('/etc') + except os.error: + # Probably not a Unix system + return distname,version,id + for file in etc: + m = _release_filename.match(file) + if m: + _distname,dummy = m.groups() + if _distname in supported_dists: + distname = _distname + break + else: + return _dist_try_harder(distname,version,id) + f = open('/etc/'+file,'r') + firstline = f.readline() + f.close() + m = _release_version.search(firstline) + if m: + _version,_id = m.groups() + if _version: + version = _version + if _id: + id = _id + else: + # Unkown format... take the first two words + l = string.split(string.strip(firstline)) + if l: + version = l[0] + if len(l) > 1: + id = l[1] + return distname,version,id + +class _popen: + + """ Fairly portable (alternative) popen implementation. + + This is mostly needed in case os.popen() is not available, or + doesn't work as advertised, e.g. in Win9X GUI programs like + PythonWin or IDLE. + + Writing to the pipe is currently not supported. + + """ + tmpfile = '' + pipe = None + bufsize = None + mode = 'r' + + def __init__(self,cmd,mode='r',bufsize=None): + + if mode != 'r': + raise ValueError,'popen()-emulation only supports read mode' + import tempfile + self.tmpfile = tmpfile = tempfile.mktemp() + os.system(cmd + ' > %s' % tmpfile) + self.pipe = open(tmpfile,'rb') + self.bufsize = bufsize + self.mode = mode + + def read(self): + + return self.pipe.read() + + def readlines(self): + + if self.bufsize is not None: + return self.pipe.readlines() + + def close(self, + + remove=os.unlink,error=os.error): + + if self.pipe: + rc = self.pipe.close() + else: + rc = 255 + if self.tmpfile: + try: + remove(self.tmpfile) + except error: + pass + return rc + + # Alias + __del__ = close + +def popen(cmd, mode='r', bufsize=None): + + """ Portable popen() interface. + """ + # Find a working popen implementation preferring win32pipe.popen + # over os.popen over _popen + popen = None + if os.environ.get('OS','') == 'Windows_NT': + # On NT win32pipe should work; on Win9x it hangs due to bugs + # in the MS C lib (see MS KnowledgeBase article Q150956) + try: + import win32pipe + except ImportError: + pass + else: + popen = win32pipe.popen + if popen is None: + if hasattr(os,'popen'): + popen = os.popen + # Check whether it works... it doesn't in GUI programs + # on Windows platforms + if sys.platform == 'win32': # XXX Others too ? + try: + popen('') + except os.error: + popen = _popen + else: + popen = _popen + if bufsize is None: + return popen(cmd,mode) + else: + return popen(cmd,mode,bufsize) + +def _norm_version(version,build=''): + + """ Normalize the version and build strings and return a single + version string using the format major.minor.build (or patchlevel). + """ + l = string.split(version,'.') + if build: + l.append(build) + try: + ints = map(int,l) + except ValueError: + strings = l + else: + strings = map(str,ints) + version = string.join(strings[:3],'.') + return version + +_ver_output = re.compile(r'(?:([\w ]+) ([\w.]+) ' + '.*' + 'Version ([\d.]+))') + +def _syscmd_ver(system='',release='',version='', + + supported_platforms=('win32','win16','dos','os2')): + + """ Tries to figure out the OS version used and returns + a tuple (system,release,version). + + It uses the "ver" shell command for this which is known + to exists on Windows, DOS and OS/2. XXX Others too ? + + In case this fails, the given parameters are used as + defaults. + + """ + if sys.platform not in supported_platforms: + return system,release,version + + # Try some common cmd strings + for cmd in ('ver','command /c ver','cmd /c ver'): + try: + pipe = popen(cmd) + info = pipe.read() + if pipe.close(): + raise os.error,'command failed' + # XXX How can I supress shell errors from being written + # to stderr ? + except os.error,why: + #print 'Command %s failed: %s' % (cmd,why) + continue + except IOError,why: + #print 'Command %s failed: %s' % (cmd,why) + continue + else: + break + else: + return system,release,version + + # Parse the output + info = string.strip(info) + m = _ver_output.match(info) + if m: + system,release,version = m.groups() + # Strip trailing dots from version and release + if release[-1] == '.': + release = release[:-1] + if version[-1] == '.': + version = version[:-1] + # Normalize the version and build strings (eliminating additional + # zeros) + version = _norm_version(version) + return system,release,version + +def _win32_getvalue(key,name,default=''): + + """ Read a value for name from the registry key. + + In case this fails, default is returned. + + """ + from win32api import RegQueryValueEx + try: + return RegQueryValueEx(key,name) + except: + return default + +def win32_ver(release='',version='',csd='',ptype=''): + + """ Get additional version information from the Windows Registry + and return a tuple (version,csd,ptype) referring to version + number, CSD level and OS type (multi/single + processor). + + As a hint: ptype returns 'Uniprocessor Free' on single + processor NT machines and 'Multiprocessor Free' on multi + processor machines. The 'Free' refers to the OS version being + free of debugging code. It could also state 'Checked' which + means the OS version uses debugging code, i.e. code that + checks arguments, ranges, etc. (Thomas Heller). + + Note: this function only works if Mark Hammond's win32 + package is installed and obviously only runs on Win32 + compatible platforms. + + """ + # XXX Is there any way to find out the processor type on WinXX ? + # XXX Is win32 available on Windows CE ? + # + # Adapted from code posted by Karl Putland to comp.lang.python. + # + # The mappings between reg. values and release names can be found + # here: http://msdn.microsoft.com/library/en-us/sysinfo/base/osversioninfo_str.asp + + # Import the needed APIs + try: + import win32api + except ImportError: + return release,version,csd,ptype + from win32api import RegQueryValueEx,RegOpenKeyEx,RegCloseKey,GetVersionEx + from win32con import HKEY_LOCAL_MACHINE,VER_PLATFORM_WIN32_NT,\ + VER_PLATFORM_WIN32_WINDOWS + + # Find out the registry key and some general version infos + maj,min,buildno,plat,csd = GetVersionEx() + version = '%i.%i.%i' % (maj,min,buildno & 0xFFFF) + if csd[:13] == 'Service Pack ': + csd = 'SP' + csd[13:] + if plat == VER_PLATFORM_WIN32_WINDOWS: + regkey = 'SOFTWARE\\Microsoft\\Windows\\CurrentVersion' + # Try to guess the release name + if maj == 4: + if min == 0: + release = '95' + elif min == 10: + release = '98' + elif min == 90: + release = 'Me' + else: + release = 'postMe' + elif maj == 5: + release = '2000' + elif plat == VER_PLATFORM_WIN32_NT: + regkey = 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion' + if maj <= 4: + release = 'NT' + elif maj == 5: + if min == 0: + release = '2000' + elif min == 1: + release = 'XP' + elif min == 2: + release = '2003Server' + else: + release = 'post2003' + else: + if not release: + # E.g. Win3.1 with win32s + release = '%i.%i' % (maj,min) + return release,version,csd,ptype + + # Open the registry key + try: + keyCurVer = RegOpenKeyEx(HKEY_LOCAL_MACHINE,regkey) + # Get a value to make sure the key exists... + RegQueryValueEx(keyCurVer,'SystemRoot') + except: + return release,version,csd,ptype + + # Parse values + #subversion = _win32_getvalue(keyCurVer, + # 'SubVersionNumber', + # ('',1))[0] + #if subversion: + # release = release + subversion # 95a, 95b, etc. + build = _win32_getvalue(keyCurVer, + 'CurrentBuildNumber', + ('',1))[0] + ptype = _win32_getvalue(keyCurVer, + 'CurrentType', + (ptype,1))[0] + + # Normalize version + version = _norm_version(version,build) + + # Close key + RegCloseKey(keyCurVer) + return release,version,csd,ptype + +def _mac_ver_lookup(selectors,default=None): + + from gestalt import gestalt + import MacOS + l = [] + append = l.append + for selector in selectors: + try: + append(gestalt(selector)) + except (RuntimeError, MacOS.Error): + append(default) + return l + +def _bcd2str(bcd): + + return hex(bcd)[2:] + +def mac_ver(release='',versioninfo=('','',''),machine=''): + + """ Get MacOS version information and return it as tuple (release, + versioninfo, machine) with versioninfo being a tuple (version, + dev_stage, non_release_version). + + Entries which cannot be determined are set to the paramter values + which default to ''. All tuple entries are strings. + + Thanks to Mark R. Levinson for mailing documentation links and + code examples for this function. Documentation for the + gestalt() API is available online at: + + http://www.rgaros.nl/gestalt/ + + """ + # Check whether the version info module is available + try: + import gestalt + import MacOS + except ImportError: + return release,versioninfo,machine + # Get the infos + sysv,sysu,sysa = _mac_ver_lookup(('sysv','sysu','sysa')) + # Decode the infos + if sysv: + major = (sysv & 0xFF00) >> 8 + minor = (sysv & 0x00F0) >> 4 + patch = (sysv & 0x000F) + release = '%s.%i.%i' % (_bcd2str(major),minor,patch) + if sysu: + major = int((sysu & 0xFF000000L) >> 24) + minor = (sysu & 0x00F00000) >> 20 + bugfix = (sysu & 0x000F0000) >> 16 + stage = (sysu & 0x0000FF00) >> 8 + nonrel = (sysu & 0x000000FF) + version = '%s.%i.%i' % (_bcd2str(major),minor,bugfix) + nonrel = _bcd2str(nonrel) + stage = {0x20:'development', + 0x40:'alpha', + 0x60:'beta', + 0x80:'final'}.get(stage,'') + versioninfo = (version,stage,nonrel) + if sysa: + machine = {0x1: '68k', + 0x2: 'PowerPC', + 0xa: 'i386'}.get(sysa,'') + return release,versioninfo,machine + +def _java_getprop(name,default): + + from java.lang import System + try: + return System.getProperty(name) + except: + return default + +def java_ver(release='',vendor='',vminfo=('','',''),osinfo=('','','')): + + """ Version interface for Jython. + + Returns a tuple (release,vendor,vminfo,osinfo) with vminfo being + a tuple (vm_name,vm_release,vm_vendor) and osinfo being a + tuple (os_name,os_version,os_arch). + + Values which cannot be determined are set to the defaults + given as parameters (which all default to ''). + + """ + # Import the needed APIs + try: + import java.lang + except ImportError: + return release,vendor,vminfo,osinfo + + vendor = _java_getprop('java.vendor',vendor) + release = _java_getprop('java.version',release) + vm_name,vm_release,vm_vendor = vminfo + vm_name = _java_getprop('java.vm.name',vm_name) + vm_vendor = _java_getprop('java.vm.vendor',vm_vendor) + vm_release = _java_getprop('java.vm.version',vm_release) + vminfo = vm_name,vm_release,vm_vendor + os_name,os_version,os_arch = osinfo + os_arch = _java_getprop('java.os.arch',os_arch) + os_name = _java_getprop('java.os.name',os_name) + os_version = _java_getprop('java.os.version',os_version) + osinfo = os_name,os_version,os_arch + + return release,vendor,vminfo,osinfo + +### System name aliasing + +def system_alias(system,release,version): + + """ Returns (system,release,version) aliased to common + marketing names used for some systems. + + It also does some reordering of the information in some cases + where it would otherwise cause confusion. + + """ + if system == 'Rhapsody': + # Apple's BSD derivative + # XXX How can we determine the marketing release number ? + return 'MacOS X Server',system+release,version + + elif system == 'SunOS': + # Sun's OS + if release < '5': + # These releases use the old name SunOS + return system,release,version + # Modify release (marketing release = SunOS release - 3) + l = string.split(release,'.') + if l: + try: + major = int(l[0]) + except ValueError: + pass + else: + major = major - 3 + l[0] = str(major) + release = string.join(l,'.') + if release < '6': + system = 'Solaris' + else: + # XXX Whatever the new SunOS marketing name is... + system = 'Solaris' + + elif system == 'IRIX64': + # IRIX reports IRIX64 on platforms with 64-bit support; yet it + # is really a version and not a different platform, since 32-bit + # apps are also supported.. + system = 'IRIX' + if version: + version = version + ' (64bit)' + else: + version = '64bit' + + elif system in ('win32','win16'): + # In case one of the other tricks + system = 'Windows' + + return system,release,version + +### Various internal helpers + +def _platform(*args): + + """ Helper to format the platform string in a filename + compatible format e.g. "system-version-machine". + """ + # Format the platform string + platform = string.join( + map(string.strip, + filter(len,args)), + '-') + + # Cleanup some possible filename obstacles... + replace = string.replace + platform = replace(platform,' ','_') + platform = replace(platform,'/','-') + platform = replace(platform,'\\','-') + platform = replace(platform,':','-') + platform = replace(platform,';','-') + platform = replace(platform,'"','-') + platform = replace(platform,'(','-') + platform = replace(platform,')','-') + + # No need to report 'unknown' information... + platform = replace(platform,'unknown','') + + # Fold '--'s and remove trailing '-' + while 1: + cleaned = replace(platform,'--','-') + if cleaned == platform: + break + platform = cleaned + while platform[-1] == '-': + platform = platform[:-1] + + return platform + +def _node(default=''): + + """ Helper to determine the node name of this machine. + """ + try: + import socket + except ImportError: + # No sockets... + return default + try: + return socket.gethostname() + except socket.error: + # Still not working... + return default + +# os.path.abspath is new in Python 1.5.2: +if not hasattr(os.path,'abspath'): + + def _abspath(path, + + isabs=os.path.isabs,join=os.path.join,getcwd=os.getcwd, + normpath=os.path.normpath): + + if not isabs(path): + path = join(getcwd(), path) + return normpath(path) + +else: + + _abspath = os.path.abspath + +def _follow_symlinks(filepath): + + """ In case filepath is a symlink, follow it until a + real file is reached. + """ + filepath = _abspath(filepath) + while os.path.islink(filepath): + filepath = os.path.normpath( + os.path.join(filepath,os.readlink(filepath))) + return filepath + +def _syscmd_uname(option,default=''): + + """ Interface to the system's uname command. + """ + if sys.platform in ('dos','win32','win16','os2'): + # XXX Others too ? + return default + try: + f = os.popen('uname %s 2> /dev/null' % option) + except (AttributeError,os.error): + return default + output = string.strip(f.read()) + rc = f.close() + if not output or rc: + return default + else: + return output + +def _syscmd_file(target,default=''): + + """ Interface to the system's file command. + + The function uses the -b option of the file command to have it + ommit the filename in its output and if possible the -L option + to have the command follow symlinks. It returns default in + case the command should fail. + + """ + target = _follow_symlinks(target) + try: + f = os.popen('file %s 2> /dev/null' % target) + except (AttributeError,os.error): + return default + output = string.strip(f.read()) + rc = f.close() + if not output or rc: + return default + else: + return output + +### Information about the used architecture + +# Default values for architecture; non-empty strings override the +# defaults given as parameters +_default_architecture = { + 'win32': ('','WindowsPE'), + 'win16': ('','Windows'), + 'dos': ('','MSDOS'), +} + +_architecture_split = re.compile(r'[\s,]').split + +def architecture(executable=sys.executable,bits='',linkage=''): + + """ Queries the given executable (defaults to the Python interpreter + binary) for various architecture information. + + Returns a tuple (bits,linkage) which contains information about + the bit architecture and the linkage format used for the + executable. Both values are returned as strings. + + Values that cannot be determined are returned as given by the + parameter presets. If bits is given as '', the sizeof(pointer) + (or sizeof(long) on Python version < 1.5.2) is used as + indicator for the supported pointer size. + + The function relies on the system's "file" command to do the + actual work. This is available on most if not all Unix + platforms. On some non-Unix platforms where the "file" command + does not exist and the executable is set to the Python interpreter + binary defaults from _default_architecture are used. + + """ + # Use the sizeof(pointer) as default number of bits if nothing + # else is given as default. + if not bits: + import struct + try: + size = struct.calcsize('P') + except struct.error: + # Older installations can only query longs + size = struct.calcsize('l') + bits = str(size*8) + 'bit' + + # Get data from the 'file' system command + output = _syscmd_file(executable,'') + + if not output and \ + executable == sys.executable: + # "file" command did not return anything; we'll try to provide + # some sensible defaults then... + if _default_architecture.has_key(sys.platform): + b,l = _default_architecture[sys.platform] + if b: + bits = b + if l: + linkage = l + return bits,linkage + + # Split the output into a list of strings omitting the filename + fileout = _architecture_split(output)[1:] + + if 'executable' not in fileout: + # Format not supported + return bits,linkage + + # Bits + if '32-bit' in fileout: + bits = '32bit' + elif 'N32' in fileout: + # On Irix only + bits = 'n32bit' + elif '64-bit' in fileout: + bits = '64bit' + + # Linkage + if 'ELF' in fileout: + linkage = 'ELF' + elif 'PE' in fileout: + # E.g. Windows uses this format + if 'Windows' in fileout: + linkage = 'WindowsPE' + else: + linkage = 'PE' + elif 'COFF' in fileout: + linkage = 'COFF' + elif 'MS-DOS' in fileout: + linkage = 'MSDOS' + else: + # XXX the A.OUT format also falls under this class... + pass + + return bits,linkage + +### Portable uname() interface + +_uname_cache = None + +def uname(): + + """ Fairly portable uname interface. Returns a tuple + of strings (system,node,release,version,machine,processor) + identifying the underlying platform. + + Note that unlike the os.uname function this also returns + possible processor information as an additional tuple entry. + + Entries which cannot be determined are set to ''. + + """ + global _uname_cache + + if _uname_cache is not None: + return _uname_cache + + # Get some infos from the builtin os.uname API... + try: + system,node,release,version,machine = os.uname() + + except AttributeError: + # Hmm, no uname... we'll have to poke around the system then. + system = sys.platform + release = '' + version = '' + node = _node() + machine = '' + processor = '' + use_syscmd_ver = 1 + + # Try win32_ver() on win32 platforms + if system == 'win32': + release,version,csd,ptype = win32_ver() + if release and version: + use_syscmd_ver = 0 + + # Try the 'ver' system command available on some + # platforms + if use_syscmd_ver: + system,release,version = _syscmd_ver(system) + # Normalize system to what win32_ver() normally returns + # (_syscmd_ver() tends to return the vendor name as well) + if system == 'Microsoft Windows': + system = 'Windows' + + # In case we still don't know anything useful, we'll try to + # help ourselves + if system in ('win32','win16'): + if not version: + if system == 'win32': + version = '32bit' + else: + version = '16bit' + system = 'Windows' + + elif system[:4] == 'java': + release,vendor,vminfo,osinfo = java_ver() + system = 'Java' + version = string.join(vminfo,', ') + if not version: + version = vendor + + elif os.name == 'mac': + release,(version,stage,nonrel),machine = mac_ver() + system = 'MacOS' + + else: + # System specific extensions + if system == 'OpenVMS': + # OpenVMS seems to have release and version mixed up + if not release or release == '0': + release = version + version = '' + # Get processor information + try: + import vms_lib + except ImportError: + pass + else: + csid, cpu_number = vms_lib.getsyi('SYI$_CPU',0) + if (cpu_number >= 128): + processor = 'Alpha' + else: + processor = 'VAX' + else: + # Get processor information from the uname system command + processor = _syscmd_uname('-p','') + + # 'unknown' is not really any useful as information; we'll convert + # it to '' which is more portable + if system == 'unknown': + system = '' + if node == 'unknown': + node = '' + if release == 'unknown': + release = '' + if version == 'unknown': + version = '' + if machine == 'unknown': + machine = '' + if processor == 'unknown': + processor = '' + _uname_cache = system,node,release,version,machine,processor + return _uname_cache + +### Direct interfaces to some of the uname() return values + +def system(): + + """ Returns the system/OS name, e.g. 'Linux', 'Windows' or 'Java'. + + An empty string is returned if the value cannot be determined. + + """ + return uname()[0] + +def node(): + + """ Returns the computer's network name (which may not be fully + qualified) + + An empty string is returned if the value cannot be determined. + + """ + return uname()[1] + +def release(): + + """ Returns the system's release, e.g. '2.2.0' or 'NT' + + An empty string is returned if the value cannot be determined. + + """ + return uname()[2] + +def version(): + + """ Returns the system's release version, e.g. '#3 on degas' + + An empty string is returned if the value cannot be determined. + + """ + return uname()[3] + +def machine(): + + """ Returns the machine type, e.g. 'i386' + + An empty string is returned if the value cannot be determined. + + """ + return uname()[4] + +def processor(): + + """ Returns the (true) processor name, e.g. 'amdk6' + + An empty string is returned if the value cannot be + determined. Note that many platforms do not provide this + information or simply return the same value as for machine(), + e.g. NetBSD does this. + + """ + return uname()[5] + +### Various APIs for extracting information from sys.version + +_sys_version_parser = re.compile(r'([\w.+]+)\s*' + '$#?([^,]+),\s*([\w ]+),\s*([\w :]+)$\s*' + '\[([^\]]+)\]?') +_sys_version_cache = None + +def _sys_version(): + + """ Returns a parsed version of Python's sys.version as tuple + (version, buildno, builddate, compiler) referring to the Python + version, build number, build date/time as string and the compiler + identification string. + + Note that unlike the Python sys.version, the returned value + for the Python version will always include the patchlevel (it + defaults to '.0'). + + """ + global _sys_version_cache + + if _sys_version_cache is not None: + return _sys_version_cache + version, buildno, builddate, buildtime, compiler = \ + _sys_version_parser.match(sys.version).groups() + builddate = builddate + ' ' + buildtime + l = string.split(version, '.') + if len(l) == 2: + l.append('0') + version = string.join(l, '.') + _sys_version_cache = (version, buildno, builddate, compiler) + return _sys_version_cache + +def python_version(): + + """ Returns the Python version as string 'major.minor.patchlevel' + + Note that unlike the Python sys.version, the returned value + will always include the patchlevel (it defaults to 0). + + """ + return _sys_version()[0] + +def python_version_tuple(): + + """ Returns the Python version as tuple (major, minor, patchlevel) + of strings. + + Note that unlike the Python sys.version, the returned value + will always include the patchlevel (it defaults to 0). + + """ + return string.split(_sys_version()[0], '.') + +def python_build(): + + """ Returns a tuple (buildno, builddate) stating the Python + build number and date as strings. + + """ + return _sys_version()[1:3] + +def python_compiler(): + + """ Returns a string identifying the compiler used for compiling + Python. + + """ + return _sys_version()[3] + +### The Opus Magnum of platform strings :-) + +_platform_cache = {} + +def platform(aliased=0, terse=0): + + """ Returns a single string identifying the underlying platform + with as much useful information as possible (but no more :). + + The output is intended to be human readable rather than + machine parseable. It may look different on different + platforms and this is intended. + + If "aliased" is true, the function will use aliases for + various platforms that report system names which differ from + their common names, e.g. SunOS will be reported as + Solaris. The system_alias() function is used to implement + this. + + Setting terse to true causes the function to return only the + absolute minimum information needed to identify the platform. + + """ + result = _platform_cache.get((aliased, terse), None) + if result is not None: + return result + + # Get uname information and then apply platform specific cosmetics + # to it... + system,node,release,version,machine,processor = uname() + if machine == processor: + processor = '' + if aliased: + system,release,version = system_alias(system,release,version) + + if system == 'Windows': + # MS platforms + rel,vers,csd,ptype = win32_ver(version) + if terse: + platform = _platform(system,release) + else: + platform = _platform(system,release,version,csd) + + elif system in ('Linux',): + # Linux based systems + distname,distversion,distid = dist('') + if distname and not terse: + platform = _platform(system,release,machine,processor, + 'with', + distname,distversion,distid) + else: + # If the distribution name is unknown check for libc vs. glibc + libcname,libcversion = libc_ver(sys.executable) + platform = _platform(system,release,machine,processor, + 'with', + libcname+libcversion) + elif system == 'Java': + # Java platforms + r,v,vminfo,(os_name,os_version,os_arch) = java_ver() + if terse: + platform = _platform(system,release,version) + else: + platform = _platform(system,release,version, + 'on', + os_name,os_version,os_arch) + + elif system == 'MacOS': + # MacOS platforms + if terse: + platform = _platform(system,release) + else: + platform = _platform(system,release,machine) + + else: + # Generic handler + if terse: + platform = _platform(system,release) + else: + bits,linkage = architecture(sys.executable) + platform = _platform(system,release,machine,processor,bits,linkage) + + _platform_cache[(aliased, terse)] = platform + return platform + +### Command line interface + +if __name__ == '__main__': + # Default is to print the aliased verbose platform string + terse = ('terse' in sys.argv or '--terse' in sys.argv) + aliased = (not 'nonaliased' in sys.argv and not '--nonaliased' in sys.argv) + print platform(aliased,terse) + sys.exit(0) Added: external/pybench-2.0/pybench.py ============================================================================== --- (empty file) +++ external/pybench-2.0/pybench.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,938 @@ +#!/usr/local/bin/python -O + +""" A Python Benchmark Suite + +""" +# +# Note: Please keep this module compatible to Python 1.5.2. +# +# Tests may include features in later Python versions, but these +# should then be embedded in try-except clauses in the configuration +# module Setup.py. +# + +# pybench Copyright +__copyright__ = """\ +Copyright (c), 1997-2006, Marc-Andre Lemburg (mal at lemburg.com) +Copyright (c), 2000-2006, eGenix.com Software GmbH (info at egenix.com) + + All Rights Reserved. + +Permission to use, copy, modify, and distribute this software and its +documentation for any purpose and without fee or royalty is hereby +granted, provided that the above copyright notice appear in all copies +and that both that copyright notice and this permission notice appear +in supporting documentation or portions thereof, including +modifications, that you make. + +THE AUTHOR MARC-ANDRE LEMBURG DISCLAIMS ALL WARRANTIES WITH REGARD TO +THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND +FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, +INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING +FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, +NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION +WITH THE USE OR PERFORMANCE OF THIS SOFTWARE ! +""" + +import sys, time, operator, string +from CommandLine import * + +try: + import cPickle + pickle = cPickle +except ImportError: + import pickle + +# Version number; version history: see README file ! +__version__ = '2.0' + +### Constants + +# Second fractions +MILLI_SECONDS = 1e3 +MICRO_SECONDS = 1e6 + +# Percent unit +PERCENT = 100 + +# Horizontal line length +LINE = 79 + +# Minimum test run-time +MIN_TEST_RUNTIME = 1e-3 + +# Number of calibration runs to use for calibrating the tests +CALIBRATION_RUNS = 20 + +# Number of calibration loops to run for each calibration run +CALIBRATION_LOOPS = 20 + +# Allow skipping calibration ? +ALLOW_SKIPPING_CALIBRATION = 1 + +# Timer types +TIMER_TIME_TIME = 'time.time' +TIMER_TIME_CLOCK = 'time.clock' +TIMER_SYSTIMES_PROCESSTIME = 'systimes.processtime' + +# Choose platform default timer +if sys.platform[:3] == 'win': + # On WinXP this has 2.5ms resolution + TIMER_PLATFORM_DEFAULT = TIMER_TIME_CLOCK +else: + # On Linux this has 1ms resolution + TIMER_PLATFORM_DEFAULT = TIMER_TIME_TIME + +# Print debug information ? +_debug = 0 + +### Helpers + +def get_timer(timertype): + + if timertype == TIMER_TIME_TIME: + return time.time + elif timertype == TIMER_TIME_CLOCK: + return time.clock + elif timertype == TIMER_SYSTIMES_PROCESSTIME: + import systimes + return systimes.processtime + else: + raise TypeError('unknown timer type: %s' % timertype) + +def get_machine_details(): + + import platform + if _debug: + print 'Getting machine details...' + buildno, builddate = platform.python_build() + python = platform.python_version() + if python > '2.0': + try: + unichr(100000) + except ValueError: + # UCS2 build (standard) + unicode = 'UCS2' + else: + # UCS4 build (most recent Linux distros) + unicode = 'UCS4' + else: + unicode = None + bits, linkage = platform.architecture() + return { + 'platform': platform.platform(), + 'processor': platform.processor(), + 'executable': sys.executable, + 'python': platform.python_version(), + 'compiler': platform.python_compiler(), + 'buildno': buildno, + 'builddate': builddate, + 'unicode': unicode, + 'bits': bits, + } + +def print_machine_details(d, indent=''): + + l = ['Machine Details:', + ' Platform ID: %s' % d.get('platform', 'n/a'), + ' Processor: %s' % d.get('processor', 'n/a'), + '', + 'Python:', + ' Executable: %s' % d.get('executable', 'n/a'), + ' Version: %s' % d.get('python', 'n/a'), + ' Compiler: %s' % d.get('compiler', 'n/a'), + ' Bits: %s' % d.get('bits', 'n/a'), + ' Build: %s (#%s)' % (d.get('builddate', 'n/a'), + d.get('buildno', 'n/a')), + ' Unicode: %s' % d.get('unicode', 'n/a'), + ] + print indent + string.join(l, '\n' + indent) + '\n' + +### Test baseclass + +class Test: + + """ All test must have this class as baseclass. It provides + the necessary interface to the benchmark machinery. + + The tests must set .rounds to a value high enough to let the + test run between 20-50 seconds. This is needed because + clock()-timing only gives rather inaccurate values (on Linux, + for example, it is accurate to a few hundreths of a + second). If you don't want to wait that long, use a warp + factor larger than 1. + + It is also important to set the .operations variable to a + value representing the number of "virtual operations" done per + call of .run(). + + If you change a test in some way, don't forget to increase + it's version number. + + """ + + ### Instance variables that each test should override + + # Version number of the test as float (x.yy); this is important + # for comparisons of benchmark runs - tests with unequal version + # number will not get compared. + version = 2.0 + + # The number of abstract operations done in each round of the + # test. An operation is the basic unit of what you want to + # measure. The benchmark will output the amount of run-time per + # operation. Note that in order to raise the measured timings + # significantly above noise level, it is often required to repeat + # sets of operations more than once per test round. The measured + # overhead per test round should be less than 1 second. + operations = 1 + + # Number of rounds to execute per test run. This should be + # adjusted to a figure that results in a test run-time of between + # 1-2 seconds. + rounds = 100000 + + ### Internal variables + + # Mark this class as implementing a test + is_a_test = 1 + + # Last timing: (real, run, overhead) + last_timing = (0.0, 0.0, 0.0) + + # Warp factor to use for this test + warp = 1 + + # Number of calibration runs to use + calibration_runs = CALIBRATION_RUNS + + # List of calibration timings + overhead_times = None + + # List of test run timings + times = [] + + # Timer used for the benchmark + timer = TIMER_PLATFORM_DEFAULT + + def __init__(self, warp=None, calibration_runs=None, timer=None): + + # Set parameters + if warp is not None: + self.rounds = int(self.rounds / warp) + if self.rounds == 0: + raise ValueError('warp factor set too high') + self.warp = warp + if calibration_runs is not None: + if (not ALLOW_SKIPPING_CALIBRATION and + calibration_runs < 1): + raise ValueError('at least one calibration run is required') + self.calibration_runs = calibration_runs + if timer is not None: + timer = timer + + # Init variables + self.times = [] + self.overhead_times = [] + + # We want these to be in the instance dict, so that pickle + # saves them + self.version = self.version + self.operations = self.operations + self.rounds = self.rounds + + def get_timer(self): + + """ Return the timer function to use for the test. + + """ + return get_timer(self.timer) + + def compatible(self, other): + + """ Return 1/0 depending on whether the test is compatible + with the other Test instance or not. + + """ + if self.version != other.version: + return 0 + if self.rounds != other.rounds: + return 0 + return 1 + + def calibrate_test(self): + + if self.calibration_runs == 0: + self.overhead_times = [0.0] + return + + calibrate = self.calibrate + timer = self.get_timer() + calibration_loops = range(CALIBRATION_LOOPS) + + # Time the calibration loop overhead + prep_times = [] + for i in range(self.calibration_runs): + t = timer() + for i in calibration_loops: + pass + t = timer() - t + prep_times.append(t) + min_prep_time = min(prep_times) + if _debug: + print + print 'Calib. prep time = %.6fms' % ( + min_prep_time * MILLI_SECONDS) + + # Time the calibration runs (doing CALIBRATION_LOOPS loops of + # .calibrate() method calls each) + for i in range(self.calibration_runs): + t = timer() + for i in calibration_loops: + calibrate() + t = timer() - t + self.overhead_times.append(t / CALIBRATION_LOOPS + - min_prep_time) + + # Check the measured times + min_overhead = min(self.overhead_times) + max_overhead = max(self.overhead_times) + if _debug: + print 'Calib. overhead time = %.6fms' % ( + min_overhead * MILLI_SECONDS) + if min_overhead < 0.0: + raise ValueError('calibration setup did not work') + if max_overhead - min_overhead > 0.1: + raise ValueError( + 'overhead calibration timing range too inaccurate: ' + '%r - %r' % (min_overhead, max_overhead)) + + def run(self): + + """ Run the test in two phases: first calibrate, then + do the actual test. Be careful to keep the calibration + timing low w/r to the test timing. + + """ + test = self.test + timer = self.get_timer() + + # Get calibration + min_overhead = min(self.overhead_times) + + # Test run + t = timer() + test() + t = timer() - t + if t < MIN_TEST_RUNTIME: + raise ValueError('warp factor too high: ' + 'test times are < 10ms') + eff_time = t - min_overhead + if eff_time < 0: + raise ValueError('wrong calibration') + self.last_timing = (eff_time, t, min_overhead) + self.times.append(eff_time) + + def calibrate(self): + + """ Calibrate the test. + + This method should execute everything that is needed to + setup and run the test - except for the actual operations + that you intend to measure. pybench uses this method to + measure the test implementation overhead. + + """ + return + + def test(self): + + """ Run the test. + + The test needs to run self.rounds executing + self.operations number of operations each. + + """ + return + + def stat(self): + + """ Return test run statistics as tuple: + + (minimum run time, + average run time, + total run time, + average time per operation, + minimum overhead time) + + """ + runs = len(self.times) + if runs == 0: + return 0.0, 0.0, 0.0, 0.0 + min_time = min(self.times) + total_time = reduce(operator.add, self.times, 0.0) + avg_time = total_time / float(runs) + operation_avg = total_time / float(runs + * self.rounds + * self.operations) + if self.overhead_times: + min_overhead = min(self.overhead_times) + else: + min_overhead = self.last_timing[2] + return min_time, avg_time, total_time, operation_avg, min_overhead + +### Load Setup + +# This has to be done after the definition of the Test class, since +# the Setup module will import subclasses using this class. + +import Setup + +### Benchmark base class + +class Benchmark: + + # Name of the benchmark + name = '' + + # Number of benchmark rounds to run + rounds = 1 + + # Warp factor use to run the tests + warp = 1 # Warp factor + + # Average benchmark round time + roundtime = 0 + + # Benchmark version number as float x.yy + version = 2.0 + + # Produce verbose output ? + verbose = 0 + + # Dictionary with the machine details + machine_details = None + + # Timer used for the benchmark + timer = TIMER_PLATFORM_DEFAULT + + def __init__(self, name, verbose=None, timer=None, warp=None, + calibration_runs=None): + + if name: + self.name = name + else: + self.name = '%04i-%02i-%02i %02i:%02i:%02i' % \ + (time.localtime(time.time())[:6]) + if verbose is not None: + self.verbose = verbose + if timer is not None: + self.timer = timer + if warp is not None: + self.warp = warp + if calibration_runs is not None: + self.calibration_runs = calibration_runs + + # Init vars + self.tests = {} + if _debug: + print 'Getting machine details...' + self.machine_details = get_machine_details() + + # Make .version an instance attribute to have it saved in the + # Benchmark pickle + self.version = self.version + + def get_timer(self): + + """ Return the timer function to use for the test. + + """ + return get_timer(self.timer) + + def compatible(self, other): + + """ Return 1/0 depending on whether the benchmark is + compatible with the other Benchmark instance or not. + + """ + if self.version != other.version: + return 0 + if (self.machine_details == other.machine_details and + self.timer != other.timer): + return 0 + if (self.calibration_runs == 0 and + other.calibration_runs != 0): + return 0 + if (self.calibration_runs != 0 and + other.calibration_runs == 0): + return 0 + return 1 + + def load_tests(self, setupmod, limitnames=None): + + # Add tests + if self.verbose: + print 'Searching for tests ...' + print '--------------------------------------' + for testclass in setupmod.__dict__.values(): + if not hasattr(testclass, 'is_a_test'): + continue + name = testclass.__name__ + if name == 'Test': + continue + if (limitnames is not None and + limitnames.search(name) is None): + continue + self.tests[name] = testclass( + warp=self.warp, + calibration_runs=self.calibration_runs, + timer=self.timer) + l = self.tests.keys() + l.sort() + if self.verbose: + for name in l: + print ' %s' % name + print '--------------------------------------' + print ' %i tests found' % len(l) + print + + def calibrate(self): + + print 'Calibrating tests. Please wait...' + if self.verbose: + print + print 'Test min max' + print '-' * LINE + tests = self.tests.items() + tests.sort() + for i in range(len(tests)): + name, test = tests[i] + test.calibrate_test() + if self.verbose: + print '%30s: %6.3fms %6.3fms' % \ + (name, + min(test.overhead_times) * MILLI_SECONDS, + max(test.overhead_times) * MILLI_SECONDS) + print + + def run(self): + + tests = self.tests.items() + tests.sort() + timer = self.get_timer() + print 'Running %i round(s) of the suite at warp factor %i:' % \ + (self.rounds, self.warp) + print + self.roundtimes = [] + for i in range(self.rounds): + if self.verbose: + print ' Round %-25i effective absolute overhead' % (i+1) + total_eff_time = 0.0 + for j in range(len(tests)): + name, test = tests[j] + if self.verbose: + print '%30s:' % name, + test.run() + (eff_time, abs_time, min_overhead) = test.last_timing + total_eff_time = total_eff_time + eff_time + if self.verbose: + print ' %5.0fms %5.0fms %7.3fms' % \ + (eff_time * MILLI_SECONDS, + abs_time * MILLI_SECONDS, + min_overhead * MILLI_SECONDS) + self.roundtimes.append(total_eff_time) + if self.verbose: + print (' ' + ' ------------------------------') + print (' ' + ' Totals: %6.0fms' % + (total_eff_time * MILLI_SECONDS)) + print + else: + print '* Round %i done in %.3f seconds.' % (i+1, + total_eff_time) + print + + def stat(self): + + """ Return benchmark run statistics as tuple: + + (minimum round time, + average round time, + maximum round time) + + XXX Currently not used, since the benchmark does test + statistics across all rounds. + + """ + runs = len(self.roundtimes) + if runs == 0: + return 0.0, 0.0 + min_time = min(self.roundtimes) + total_time = reduce(operator.add, self.roundtimes, 0.0) + avg_time = total_time / float(runs) + max_time = max(self.roundtimes) + return (min_time, avg_time, max_time) + + def print_header(self, title='Benchmark'): + + print '-' * LINE + print '%s: %s' % (title, self.name) + print '-' * LINE + print + print ' Rounds: %s' % self.rounds + print ' Warp: %s' % self.warp + print ' Timer: %s' % self.timer + print + if self.machine_details: + print_machine_details(self.machine_details, indent=' ') + print + + def print_benchmark(self, hidenoise=0, limitnames=None): + + print ('Test ' + ' minimum average operation overhead') + print '-' * LINE + tests = self.tests.items() + tests.sort() + total_min_time = 0.0 + total_avg_time = 0.0 + for name, test in tests: + if (limitnames is not None and + limitnames.search(name) is None): + continue + (min_time, + avg_time, + total_time, + op_avg, + min_overhead) = test.stat() + total_min_time = total_min_time + min_time + total_avg_time = total_avg_time + avg_time + print '%30s: %5.0fms %5.0fms %6.2fus %7.3fms' % \ + (name, + min_time * MILLI_SECONDS, + avg_time * MILLI_SECONDS, + op_avg * MICRO_SECONDS, + min_overhead *MILLI_SECONDS) + print '-' * LINE + print ('Totals: ' + ' %6.0fms %6.0fms' % + (total_min_time * MILLI_SECONDS, + total_avg_time * MILLI_SECONDS, + )) + print + + def print_comparison(self, compare_to, hidenoise=0, limitnames=None): + + # Check benchmark versions + if compare_to.version != self.version: + print ('* Benchmark versions differ: ' + 'cannot compare this benchmark to "%s" !' % + compare_to.name) + print + self.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) + return + + # Print header + compare_to.print_header('Comparing with') + print ('Test ' + ' minimum run-time average run-time') + print (' ' + ' this other diff this other diff') + print '-' * LINE + + # Print test comparisons + tests = self.tests.items() + tests.sort() + total_min_time = other_total_min_time = 0.0 + total_avg_time = other_total_avg_time = 0.0 + benchmarks_compatible = self.compatible(compare_to) + tests_compatible = 1 + for name, test in tests: + if (limitnames is not None and + limitnames.search(name) is None): + continue + (min_time, + avg_time, + total_time, + op_avg, + min_overhead) = test.stat() + total_min_time = total_min_time + min_time + total_avg_time = total_avg_time + avg_time + try: + other = compare_to.tests[name] + except KeyError: + other = None + if other is None: + # Other benchmark doesn't include the given test + min_diff, avg_diff = 'n/a', 'n/a' + other_min_time = 0.0 + other_avg_time = 0.0 + tests_compatible = 0 + else: + (other_min_time, + other_avg_time, + other_total_time, + other_op_avg, + other_min_overhead) = other.stat() + other_total_min_time = other_total_min_time + other_min_time + other_total_avg_time = other_total_avg_time + other_avg_time + if (benchmarks_compatible and + test.compatible(other)): + # Both benchmark and tests are comparible + min_diff = ((min_time * self.warp) / + (other_min_time * other.warp) - 1.0) + avg_diff = ((avg_time * self.warp) / + (other_avg_time * other.warp) - 1.0) + if hidenoise and abs(min_diff) < 10.0: + min_diff = '' + else: + min_diff = '%+5.1f%%' % (min_diff * PERCENT) + if hidenoise and abs(avg_diff) < 10.0: + avg_diff = '' + else: + avg_diff = '%+5.1f%%' % (avg_diff * PERCENT) + else: + # Benchmark or tests are not comparible + min_diff, avg_diff = 'n/a', 'n/a' + tests_compatible = 0 + print '%30s: %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' % \ + (name, + min_time * MILLI_SECONDS, + other_min_time * MILLI_SECONDS * compare_to.warp / self.warp, + min_diff, + avg_time * MILLI_SECONDS, + other_avg_time * MILLI_SECONDS * compare_to.warp / self.warp, + avg_diff) + print '-' * LINE + + # Summarise test results + if not benchmarks_compatible or not tests_compatible: + min_diff, avg_diff = 'n/a', 'n/a' + else: + if other_total_min_time != 0.0: + min_diff = '%+5.1f%%' % ( + ((total_min_time * self.warp) / + (other_total_min_time * compare_to.warp) - 1.0) * PERCENT) + else: + min_diff = 'n/a' + if other_total_avg_time != 0.0: + avg_diff = '%+5.1f%%' % ( + ((total_avg_time * self.warp) / + (other_total_avg_time * compare_to.warp) - 1.0) * PERCENT) + else: + avg_diff = 'n/a' + print ('Totals: ' + ' %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' % + (total_min_time * MILLI_SECONDS, + (other_total_min_time * compare_to.warp/self.warp + * MILLI_SECONDS), + min_diff, + total_avg_time * MILLI_SECONDS, + (other_total_avg_time * compare_to.warp/self.warp + * MILLI_SECONDS), + avg_diff + )) + print + print '(this=%s, other=%s)' % (self.name, + compare_to.name) + print + +class PyBenchCmdline(Application): + + header = ("PYBENCH - a benchmark test suite for Python " + "interpreters/compilers.") + + version = __version__ + + debug = _debug + + options = [ArgumentOption('-n', + 'number of rounds', + Setup.Number_of_rounds), + ArgumentOption('-f', + 'save benchmark to file arg', + ''), + ArgumentOption('-c', + 'compare benchmark with the one in file arg', + ''), + ArgumentOption('-s', + 'show benchmark in file arg, then exit', + ''), + ArgumentOption('-w', + 'set warp factor to arg', + Setup.Warp_factor), + ArgumentOption('-t', + 'run only tests with names matching arg', + ''), + ArgumentOption('-C', + 'set the number of calibration runs to arg', + CALIBRATION_RUNS), + SwitchOption('-d', + 'hide noise in comparisons', + 0), + SwitchOption('-v', + 'verbose output (not recommended)', + 0), + SwitchOption('--with-gc', + 'enable garbage collection', + 0), + SwitchOption('--with-syscheck', + 'use default sys check interval', + 0), + ArgumentOption('--timer', + 'use given timer', + TIMER_PLATFORM_DEFAULT), + ] + + about = """\ +The normal operation is to run the suite and display the +results. Use -f to save them for later reuse or comparisons. + +Available timers: + + time.time + time.clock + systimes.processtime + +Examples: + +python2.1 pybench.py -f p21.pybench +python2.5 pybench.py -f p25.pybench +python pybench.py -s p25.pybench -c p21.pybench +""" + copyright = __copyright__ + + def main(self): + + rounds = self.values['-n'] + reportfile = self.values['-f'] + show_bench = self.values['-s'] + compare_to = self.values['-c'] + hidenoise = self.values['-d'] + warp = int(self.values['-w']) + withgc = self.values['--with-gc'] + limitnames = self.values['-t'] + if limitnames: + if _debug: + print '* limiting test names to one with substring "%s"' % \ + limitnames + limitnames = re.compile(limitnames, re.I) + else: + limitnames = None + verbose = self.verbose + withsyscheck = self.values['--with-syscheck'] + calibration_runs = self.values['-C'] + timer = self.values['--timer'] + + print '-' * LINE + print 'PYBENCH %s' % __version__ + print '-' * LINE + print '* using Python %s' % (string.split(sys.version)[0]) + + # Switch off garbage collection + if not withgc: + try: + import gc + except ImportError: + print '* Python version doesn\'t support garbage collection' + else: + gc.disable() + print '* disabled garbage collection' + + # "Disable" sys check interval + if not withsyscheck: + # Too bad the check interval uses an int instead of a long... + value = 2147483647 + sys.setcheckinterval(value) + print '* system check interval set to maximum: %s' % value + + if timer == TIMER_SYSTIMES_PROCESSTIME: + import systimes + print '* using timer: systimes.processtime (%s)' % \ + systimes.SYSTIMES_IMPLEMENTATION + else: + print '* using timer: %s' % timer + + print + + if compare_to: + try: + f = open(compare_to,'rb') + bench = pickle.load(f) + bench.name = compare_to + f.close() + compare_to = bench + except IOError, reason: + print '* Error opening/reading file %s: %s' % ( + repr(compare_to), + reason) + compare_to = None + + if show_bench: + try: + f = open(show_bench,'rb') + bench = pickle.load(f) + bench.name = show_bench + f.close() + bench.print_header() + if compare_to: + bench.print_comparison(compare_to, + hidenoise=hidenoise, + limitnames=limitnames) + else: + bench.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) + except IOError: + print '* Error opening/reading file %s: %s' % ( + repr(show_bench), + reason) + print + return + + if reportfile: + print 'Creating benchmark: %s (rounds=%i, warp=%i)' % \ + (reportfile, rounds, warp) + print + + # Create benchmark object + bench = Benchmark(reportfile, + verbose=verbose, + timer=timer, + warp=warp, + calibration_runs=calibration_runs) + bench.rounds = rounds + bench.load_tests(Setup, limitnames=limitnames) + try: + bench.calibrate() + bench.run() + except KeyboardInterrupt: + print + print '*** KeyboardInterrupt -- Aborting' + print + return + bench.print_header() + if compare_to: + bench.print_comparison(compare_to, + hidenoise=hidenoise, + limitnames=limitnames) + else: + bench.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) + + # Ring bell + sys.stderr.write('\007') + + if reportfile: + try: + f = open(reportfile,'wb') + bench.name = reportfile + pickle.dump(bench,f) + f.close() + except IOError: + print '* Error opening/writing reportfile' + +if __name__ == '__main__': + PyBenchCmdline() Added: external/pybench-2.0/systimes.py ============================================================================== --- (empty file) +++ external/pybench-2.0/systimes.py Tue Jun 13 18:45:48 2006 @@ -0,0 +1,211 @@ +#!/usr/bin/env python + +""" systimes() user and system timer implementations for use by + pybench. + + This module implements various different strategies for measuring + performance timings. It tries to choose the best available method + based on the platforma and available tools. + + On Windows, it is recommended to have the Mark Hammond win32 + package installed. Alternatively, the Thomas Heller ctypes + packages can also be used. + + On Unix systems, the standard resource module provides the highest + resolution timings. Unfortunately, it is not available on all Unix + platforms. + + If no supported timing methods based on process time can be found, + the module reverts to the highest resolution wall-clock timer + instead. The system time part will then always be 0.0. + + The module exports one public API: + + def systimes(): + + Return the current timer values for measuring user and system + time as tuple of seconds (user_time, system_time). + + Copyright (c) 2006, Marc-Andre Lemburg (mal at egenix.com). See the + documentation for further information on copyrights, or contact + the author. All Rights Reserved. + +""" +import time, sys, struct + +# +# Note: Please keep this module compatible to Python 1.5.2. +# +# TODOs: +# +# * Add ctypes wrapper for new clock_gettime() real-time POSIX APIs; +# these will then provide nano-second resolution where available. +# +# * Add a function that returns the resolution of systimes() +# values, ie. systimesres(). +# + +### Choose an implementation + +SYSTIMES_IMPLEMENTATION = None +USE_CTYPES_GETPROCESSTIMES = 'cytpes GetProcessTimes() wrapper' +USE_WIN32PROCESS_GETPROCESSTIMES = 'win32process.GetProcessTimes()' +USE_RESOURCE_GETRUSAGE = 'resource.getrusage()' +USE_PROCESS_TIME_CLOCK = 'time.clock() (process time)' +USE_WALL_TIME_CLOCK = 'time.clock() (wall-clock)' +USE_WALL_TIME_TIME = 'time.time() (wall-clock)' + +if sys.platform[:3] == 'win': + # Windows platform + try: + import win32process + except ImportError: + try: + import ctypes + except ImportError: + # Use the wall-clock implementation time.clock(), since this + # is the highest resolution clock available on Windows + SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_CLOCK + else: + SYSTIMES_IMPLEMENTATION = USE_CTYPES_GETPROCESSTIMES + else: + SYSTIMES_IMPLEMENTATION = USE_WIN32PROCESS_GETPROCESSTIMES +else: + # All other platforms + try: + import resource + except ImportError: + pass + else: + SYSTIMES_IMPLEMENTATION = USE_RESOURCE_GETRUSAGE + +# Fall-back solution +if SYSTIMES_IMPLEMENTATION is None: + # Check whether we can use time.clock() as approximation + # for systimes() + start = time.clock() + time.sleep(0.1) + stop = time.clock() + if stop - start < 0.001: + # Looks like time.clock() is usable (and measures process + # time) + SYSTIMES_IMPLEMENTATION = USE_PROCESS_TIME_CLOCK + else: + # Use wall-clock implementation time.time() since this provides + # the highest resolution clock on most systems + SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_TIME + +### Implementations + +def getrusage_systimes(): + return resource.getrusage(resource.RUSAGE_SELF)[:2] + +def process_time_clock_systimes(): + return (time.clock(), 0.0) + +def wall_clock_clock_systimes(): + return (time.clock(), 0.0) + +def wall_clock_time_systimes(): + return (time.time(), 0.0) + +# Number of clock ticks per second for the values returned +# by GetProcessTimes() on Windows. +# +# Note: Ticks returned by GetProcessTimes() are 100ns intervals on +# Windows XP. However, the process times are only updated with every +# clock tick and the frequency of these is somewhat lower: depending +# on the OS version between 10ms and 15ms. Even worse, the process +# time seems to be allocated to process currently running when the +# clock interrupt arrives, ie. it is possible that the current time +# slice gets accounted to a different process. + +WIN32_PROCESS_TIMES_TICKS_PER_SECOND = 1e7 + +def win32process_getprocesstimes_systimes(): + d = win32process.GetProcessTimes(win32process.GetCurrentProcess()) + return (d['UserTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, + d['KernelTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) + +def ctypes_getprocesstimes_systimes(): + creationtime = ctypes.c_ulonglong() + exittime = ctypes.c_ulonglong() + kerneltime = ctypes.c_ulonglong() + usertime = ctypes.c_ulonglong() + rc = ctypes.windll.kernel32.GetProcessTimes( + ctypes.windll.kernel32.GetCurrentProcess(), + ctypes.byref(creationtime), + ctypes.byref(exittime), + ctypes.byref(kerneltime), + ctypes.byref(usertime)) + if not rc: + raise TypeError('GetProcessTimes() returned an error') + return (usertime.value / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, + kerneltime.value / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) + +# Select the default for the systimes() function + +if SYSTIMES_IMPLEMENTATION is USE_RESOURCE_GETRUSAGE: + systimes = getrusage_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_PROCESS_TIME_CLOCK: + systimes = process_time_clock_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_CLOCK: + systimes = wall_clock_clock_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_TIME: + systimes = wall_clock_time_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_WIN32PROCESS_GETPROCESSTIMES: + systimes = win32process_getprocesstimes_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_CTYPES_GETPROCESSTIMES: + systimes = ctypes_getprocesstimes_systimes + +else: + raise TypeError('no suitable systimes() implementation found') + +def processtime(): + + """ Return the total time spent on the process. + + This is the sum of user and system time as returned by + systimes(). + + """ + user, system = systimes() + return user + system + +### Testing + +def some_workload(): + x = 0L + for i in xrange(10000000L): + x = x + 1L + +def test_workload(): + print 'Testing systimes() under load conditions' + t0 = systimes() + some_workload() + t1 = systimes() + print 'before:', t0 + print 'after:', t1 + print 'differences:', (t1[0] - t0[0], t1[1] - t0[1]) + print + +def test_idle(): + print 'Testing systimes() under idle conditions' + t0 = systimes() + time.sleep(1) + t1 = systimes() + print 'before:', t0 + print 'after:', t1 + print 'differences:', (t1[0] - t0[0], t1[1] - t0[1]) + print + +if __name__ == '__main__': + print 'Using %s as timer' % SYSTIMES_IMPLEMENTATION + print + test_workload() + test_idle() From python-checkins at python.org Tue Jun 13 19:04:27 2006 From: python-checkins at python.org (marc-andre.lemburg) Date: Tue, 13 Jun 2006 19:04:27 +0200 (CEST) Subject: [Python-checkins] r46923 - python/trunk/Tools/pybench/pybench-2.0 Message-ID: <20060613170427.1C1CC1E4016@bag.python.org> Author: marc-andre.lemburg Date: Tue Jun 13 19:04:26 2006 New Revision: 46923 Added: python/trunk/Tools/pybench/pybench-2.0/ - copied from r46922, external/pybench-2.0/ Log: Update pybench to version 2.0. From python-checkins at python.org Tue Jun 13 19:07:14 2006 From: python-checkins at python.org (marc-andre.lemburg) Date: Tue, 13 Jun 2006 19:07:14 +0200 (CEST) Subject: [Python-checkins] r46924 - python/trunk/Tools/pybench/pybench-2.0 Message-ID: <20060613170714.ABB8A1E4005@bag.python.org> Author: marc-andre.lemburg Date: Tue Jun 13 19:07:14 2006 New Revision: 46924 Removed: python/trunk/Tools/pybench/pybench-2.0/ Log: Revert wrong svn copy. From python-checkins at python.org Tue Jun 13 19:14:37 2006 From: python-checkins at python.org (andrew.macintyre) Date: Tue, 13 Jun 2006 19:14:37 +0200 (CEST) Subject: [Python-checkins] r46925 - in python/trunk/Lib: dummy_thread.py test/test_thread.py Message-ID: <20060613171437.628591E4005@bag.python.org> Author: andrew.macintyre Date: Tue Jun 13 19:14:36 2006 New Revision: 46925 Modified: python/trunk/Lib/dummy_thread.py python/trunk/Lib/test/test_thread.py Log: fix exception usage Modified: python/trunk/Lib/dummy_thread.py ============================================================================== --- python/trunk/Lib/dummy_thread.py (original) +++ python/trunk/Lib/dummy_thread.py Tue Jun 13 19:14:36 2006 @@ -79,8 +79,7 @@ def stack_size(size=None): """Dummy implementation of thread.stack_size().""" if size is not None: - msg = "setting thread stack size not supported on this platform" - warnings.warn(msg, RuntimeWarning) + raise error("setting thread stack size not supported") return 0 class LockType(object): Modified: python/trunk/Lib/test/test_thread.py ============================================================================== --- python/trunk/Lib/test/test_thread.py (original) +++ python/trunk/Lib/test/test_thread.py Tue Jun 13 19:14:36 2006 @@ -133,7 +133,7 @@ thread.stack_size(4096) except ValueError: print 'caught expected ValueError setting stack_size(4096)' - except thread.ThreadError: + except thread.error: tss_supported = 0 print 'platform does not support changing thread stack size' From python-checkins at python.org Tue Jun 13 19:17:28 2006 From: python-checkins at python.org (andrew.macintyre) Date: Tue, 13 Jun 2006 19:17:28 +0200 (CEST) Subject: [Python-checkins] r46926 - in python/branches/aimacintyre-sf1454481/Lib: dummy_thread.py test/output/test_thread test/test_thread.py test/test_threading.py Message-ID: <20060613171728.3B09D1E4012@bag.python.org> Author: andrew.macintyre Date: Tue Jun 13 19:17:26 2006 New Revision: 46926 Modified: python/branches/aimacintyre-sf1454481/Lib/dummy_thread.py python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py Log: - fix exception usage (already in trunk) - bump small stack tests up to 256 to try and avoid tripping one of the Linux buildbots Modified: python/branches/aimacintyre-sf1454481/Lib/dummy_thread.py ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/dummy_thread.py (original) +++ python/branches/aimacintyre-sf1454481/Lib/dummy_thread.py Tue Jun 13 19:17:26 2006 @@ -79,8 +79,7 @@ def stack_size(size=None): """Dummy implementation of thread.stack_size().""" if size is not None: - msg = "setting thread stack size not supported on this platform" - warnings.warn(msg, RuntimeWarning) + raise error("setting thread stack size not supported") return 0 class LockType(object): Modified: python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/output/test_thread Tue Jun 13 19:17:26 2006 @@ -7,10 +7,10 @@ *** Changing thread stack size *** caught expected ValueError setting stack_size(4096) -successfully set stack_size(32768) +successfully set stack_size(262144) successfully set stack_size(1048576) successfully set stack_size(0) -trying stack_size = 32768 +trying stack_size = 262144 waiting for all tasks to complete all tasks done trying stack_size = 1048576 Modified: python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/test_thread.py Tue Jun 13 19:17:26 2006 @@ -133,20 +133,20 @@ thread.stack_size(4096) except ValueError: print 'caught expected ValueError setting stack_size(4096)' - except thread.ThreadError: + except thread.error: tss_supported = 0 print 'platform does not support changing thread stack size' if tss_supported: failed = lambda s, e: s != e fail_msg = "stack_size(%d) failed - should succeed" - for tss in (32768, 0x100000, 0): + for tss in (262144, 0x100000, 0): thread.stack_size(tss) if failed(thread.stack_size(), tss): raise ValueError, fail_msg % tss print 'successfully set stack_size(%d)' % tss - for tss in (32768, 0x100000): + for tss in (262144, 0x100000): print 'trying stack_size = %d' % tss next_ident = 0 for i in range(numtasks): Modified: python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py ============================================================================== --- python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py (original) +++ python/branches/aimacintyre-sf1454481/Lib/test/test_threading.py Tue Jun 13 19:17:26 2006 @@ -85,11 +85,11 @@ print 'all tasks done' self.assertEqual(numrunning.get(), 0) - # run with a minimum thread stack size (32kB) + # run with a minimum thread stack size (256kB) def test_various_ops_small_stack(self): if verbose: - print 'with 32kB thread stack size...' - threading.stack_size(0x8000) + print 'with 256kB thread stack size...' + threading.stack_size(0x20000) self.test_various_ops() threading.stack_size(0) From buildbot at python.org Tue Jun 13 19:31:05 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 17:31:05 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060613173105.D92601E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/174 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 20:37:07 2006 From: python-checkins at python.org (tim.peters) Date: Tue, 13 Jun 2006 20:37:07 +0200 (CEST) Subject: [Python-checkins] r46927 - python/trunk/Lib/test/test_thread.py Message-ID: <20060613183707.EBFC51E4005@bag.python.org> Author: tim.peters Date: Tue Jun 13 20:37:07 2006 New Revision: 46927 Modified: python/trunk/Lib/test/test_thread.py Log: Whitespace normalization. Modified: python/trunk/Lib/test/test_thread.py ============================================================================== --- python/trunk/Lib/test/test_thread.py (original) +++ python/trunk/Lib/test/test_thread.py Tue Jun 13 20:37:07 2006 @@ -121,7 +121,7 @@ if thread.stack_size() != 0: raise ValueError, "initial stack_size not 0" -thread.stack_size(0) +thread.stack_size(0) if thread.stack_size() != 0: raise ValueError, "stack_size not reset to default" From buildbot at python.org Tue Jun 13 20:44:41 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 18:44:41 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable 2.4 Message-ID: <20060613184441.3E6521E4005@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%25202.4/builds/98 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'Andrew MacIntyre': test fixes for SF1454481 Build Source Stamp: [branch branches/aimacintyre-sf1454481] 46926 Blamelist: Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 20:56:57 2006 From: python-checkins at python.org (marc-andre.lemburg) Date: Tue, 13 Jun 2006 20:56:57 +0200 (CEST) Subject: [Python-checkins] r46928 - python/trunk/Tools/pybench/Arithmetic.py python/trunk/Tools/pybench/Calls.py python/trunk/Tools/pybench/CommandLine.py python/trunk/Tools/pybench/Constructs.py python/trunk/Tools/pybench/Dict.py python/trunk/Tools/pybench/Exceptions.py python/trunk/Tools/pybench/Imports.py python/trunk/Tools/pybench/Instances.py python/trunk/Tools/pybench/Lists.py python/trunk/Tools/pybench/Lookups.py python/trunk/Tools/pybench/NewInstances.py python/trunk/Tools/pybench/Numbers.py python/trunk/Tools/pybench/README python/trunk/Tools/pybench/Setup.py python/trunk/Tools/pybench/Strings.py python/trunk/Tools/pybench/Tuples.py python/trunk/Tools/pybench/Unicode.py python/trunk/Tools/pybench/clockres.py python/trunk/Tools/pybench/pybench.py python/trunk/Tools/pybench/systimes.py Message-ID: <20060613185657.6341D1E4005@bag.python.org> Author: marc-andre.lemburg Date: Tue Jun 13 20:56:56 2006 New Revision: 46928 Added: python/trunk/Tools/pybench/clockres.py (contents, props changed) Modified: python/trunk/Tools/pybench/Arithmetic.py python/trunk/Tools/pybench/Calls.py python/trunk/Tools/pybench/CommandLine.py python/trunk/Tools/pybench/Constructs.py python/trunk/Tools/pybench/Dict.py python/trunk/Tools/pybench/Exceptions.py python/trunk/Tools/pybench/Imports.py python/trunk/Tools/pybench/Instances.py python/trunk/Tools/pybench/Lists.py python/trunk/Tools/pybench/Lookups.py python/trunk/Tools/pybench/NewInstances.py (contents, props changed) python/trunk/Tools/pybench/Numbers.py python/trunk/Tools/pybench/README python/trunk/Tools/pybench/Setup.py python/trunk/Tools/pybench/Strings.py python/trunk/Tools/pybench/Tuples.py python/trunk/Tools/pybench/Unicode.py python/trunk/Tools/pybench/pybench.py python/trunk/Tools/pybench/systimes.py Log: Updated to pybench 2.0. See svn.python.org/external/pybench-2.0 for the original import of that version. Note that platform.py was not copied over from pybench-2.0 since it is already part of Python 2.5. Modified: python/trunk/Tools/pybench/Arithmetic.py ============================================================================== --- python/trunk/Tools/pybench/Arithmetic.py (original) +++ python/trunk/Tools/pybench/Arithmetic.py Tue Jun 13 20:56:56 2006 @@ -2,7 +2,7 @@ class SimpleIntegerArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) rounds = 120000 @@ -157,9 +157,9 @@ class SimpleFloatArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) - rounds = 100000 + rounds = 120000 def test(self): @@ -312,7 +312,7 @@ class SimpleIntFloatArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) rounds = 120000 @@ -468,9 +468,9 @@ class SimpleLongArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) - rounds = 30000 + rounds = 60000 def test(self): @@ -623,9 +623,9 @@ class SimpleComplexArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) - rounds = 40000 + rounds = 80000 def test(self): Modified: python/trunk/Tools/pybench/Calls.py ============================================================================== --- python/trunk/Tools/pybench/Calls.py (original) +++ python/trunk/Tools/pybench/Calls.py Tue Jun 13 20:56:56 2006 @@ -2,7 +2,7 @@ class PythonFunctionCalls(Test): - version = 0.3 + version = 2.0 operations = 5*(1+4+4+2) rounds = 60000 @@ -111,9 +111,9 @@ class BuiltinFunctionCalls(Test): - version = 0.4 + version = 2.0 operations = 5*(2+5+5+5) - rounds = 30000 + rounds = 60000 def test(self): @@ -232,9 +232,9 @@ class PythonMethodCalls(Test): - version = 0.3 + version = 2.0 operations = 5*(6 + 5 + 4) - rounds = 20000 + rounds = 30000 def test(self): @@ -374,9 +374,9 @@ class Recursion(Test): - version = 0.3 + version = 2.0 operations = 5 - rounds = 50000 + rounds = 100000 def test(self): @@ -407,3 +407,98 @@ for i in xrange(self.rounds): pass + + +### Test to make Fredrik happy... + +if __name__ == '__main__': + import timeit + if 0: + timeit.TestClass = PythonFunctionCalls + timeit.main(['-s', 'test = TestClass(); test.rounds = 1000', + 'test.test()']) + else: + setup = """\ +global f,f1,g,h + +# define functions +def f(): + pass + +def f1(x): + pass + +def g(a,b,c): + return a,b,c + +def h(a,b,c,d=1,e=2,f=3): + return d,e,f + +i = 1 +""" + test = """\ +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) +""" + + timeit.main(['-s', setup, + test]) + + Modified: python/trunk/Tools/pybench/CommandLine.py ============================================================================== --- python/trunk/Tools/pybench/CommandLine.py (original) +++ python/trunk/Tools/pybench/CommandLine.py Tue Jun 13 20:56:56 2006 @@ -358,7 +358,7 @@ except self.InternalError: print - print '* Internal Error' + print '* Internal Error (use --debug to display the traceback)' if self.debug: print traceback.print_exc(20, sys.stdout) Modified: python/trunk/Tools/pybench/Constructs.py ============================================================================== --- python/trunk/Tools/pybench/Constructs.py (original) +++ python/trunk/Tools/pybench/Constructs.py Tue Jun 13 20:56:56 2006 @@ -2,7 +2,7 @@ class IfThenElse(Test): - version = 0.31 + version = 2.0 operations = 30*3 # hard to say... rounds = 150000 @@ -469,9 +469,9 @@ class NestedForLoops(Test): - version = 0.3 + version = 2.0 operations = 1000*10*5 - rounds = 150 + rounds = 300 def test(self): @@ -494,9 +494,9 @@ class ForLoops(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 8000 + rounds = 10000 def test(self): Modified: python/trunk/Tools/pybench/Dict.py ============================================================================== --- python/trunk/Tools/pybench/Dict.py (original) +++ python/trunk/Tools/pybench/Dict.py Tue Jun 13 20:56:56 2006 @@ -2,9 +2,9 @@ class DictCreation(Test): - version = 0.3 + version = 2.0 operations = 5*(5 + 5) - rounds = 60000 + rounds = 80000 def test(self): @@ -77,7 +77,7 @@ class DictWithStringKeys(Test): - version = 0.1 + version = 2.0 operations = 5*(6 + 6) rounds = 200000 @@ -166,9 +166,9 @@ class DictWithFloatKeys(Test): - version = 0.1 + version = 2.0 operations = 5*(6 + 6) - rounds = 200000 + rounds = 150000 def test(self): @@ -255,7 +255,7 @@ class DictWithIntegerKeys(Test): - version = 0.1 + version = 2.0 operations = 5*(6 + 6) rounds = 200000 @@ -344,13 +344,14 @@ class SimpleDictManipulation(Test): - version = 0.3 + version = 2.0 operations = 5*(6 + 6 + 6 + 6) - rounds = 50000 + rounds = 100000 def test(self): d = {} + has_key = d.has_key for i in xrange(self.rounds): @@ -368,12 +369,12 @@ x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -396,12 +397,12 @@ x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -424,12 +425,12 @@ x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -452,12 +453,12 @@ x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -480,12 +481,12 @@ x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -497,6 +498,7 @@ def calibrate(self): d = {} + has_key = d.has_key for i in xrange(self.rounds): pass Modified: python/trunk/Tools/pybench/Exceptions.py ============================================================================== --- python/trunk/Tools/pybench/Exceptions.py (original) +++ python/trunk/Tools/pybench/Exceptions.py Tue Jun 13 20:56:56 2006 @@ -2,9 +2,9 @@ class TryRaiseExcept(Test): - version = 0.1 - operations = 2 + 3 - rounds = 60000 + version = 2.0 + operations = 2 + 3 + 3 + rounds = 80000 def test(self): @@ -31,6 +31,18 @@ raise error,"something" except: pass + try: + raise error("something") + except: + pass + try: + raise error("something") + except: + pass + try: + raise error("something") + except: + pass def calibrate(self): @@ -42,9 +54,9 @@ class TryExcept(Test): - version = 0.1 + version = 2.0 operations = 15 * 10 - rounds = 200000 + rounds = 150000 def test(self): @@ -677,3 +689,11 @@ for i in xrange(self.rounds): pass + +### Test to make Fredrik happy... + +if __name__ == '__main__': + import timeit + timeit.TestClass = TryRaiseExcept + timeit.main(['-s', 'test = TestClass(); test.rounds = 1000', + 'test.test()']) Modified: python/trunk/Tools/pybench/Imports.py ============================================================================== --- python/trunk/Tools/pybench/Imports.py (original) +++ python/trunk/Tools/pybench/Imports.py Tue Jun 13 20:56:56 2006 @@ -6,9 +6,9 @@ class SecondImport(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 20000 + rounds = 40000 def test(self): @@ -51,9 +51,9 @@ class SecondPackageImport(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 20000 + rounds = 40000 def test(self): @@ -95,9 +95,9 @@ class SecondSubmoduleImport(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 20000 + rounds = 40000 def test(self): Modified: python/trunk/Tools/pybench/Instances.py ============================================================================== --- python/trunk/Tools/pybench/Instances.py (original) +++ python/trunk/Tools/pybench/Instances.py Tue Jun 13 20:56:56 2006 @@ -2,9 +2,9 @@ class CreateInstances(Test): - version = 0.2 + version = 2.0 operations = 3 + 7 + 4 - rounds = 60000 + rounds = 80000 def test(self): Modified: python/trunk/Tools/pybench/Lists.py ============================================================================== --- python/trunk/Tools/pybench/Lists.py (original) +++ python/trunk/Tools/pybench/Lists.py Tue Jun 13 20:56:56 2006 @@ -2,22 +2,23 @@ class SimpleListManipulation(Test): - version = 0.3 + version = 2.0 operations = 5* (6 + 6 + 6) - rounds = 60000 + rounds = 130000 def test(self): l = [] + append = l.append for i in xrange(self.rounds): - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -33,12 +34,12 @@ x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -54,12 +55,12 @@ x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -75,12 +76,12 @@ x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -96,12 +97,12 @@ x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -124,15 +125,16 @@ def calibrate(self): l = [] + append = l.append for i in xrange(self.rounds): pass class ListSlicing(Test): - version = 0.4 + version = 2.0 operations = 25*(3+1+2+1) - rounds = 400 + rounds = 800 def test(self): @@ -141,7 +143,7 @@ for i in xrange(self.rounds): - l = range(100) + l = n[:] for j in r: @@ -159,17 +161,14 @@ r = range(25) for i in xrange(self.rounds): - - l = range(100) - for j in r: pass class SmallLists(Test): - version = 0.3 + version = 2.0 operations = 5*(1+ 6 + 6 + 3 + 1) - rounds = 60000 + rounds = 80000 def test(self): @@ -177,12 +176,13 @@ l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -199,12 +199,13 @@ l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -221,12 +222,13 @@ l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -243,12 +245,13 @@ l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -265,12 +268,13 @@ l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -288,4 +292,4 @@ def calibrate(self): for i in xrange(self.rounds): - l = [] + pass Modified: python/trunk/Tools/pybench/Lookups.py ============================================================================== --- python/trunk/Tools/pybench/Lookups.py (original) +++ python/trunk/Tools/pybench/Lookups.py Tue Jun 13 20:56:56 2006 @@ -2,7 +2,7 @@ class SpecialClassAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -183,7 +183,7 @@ class NormalClassAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -369,7 +369,7 @@ class SpecialInstanceAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -557,7 +557,7 @@ class NormalInstanceAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -745,7 +745,7 @@ class BuiltinMethodLookup(Test): - version = 0.3 + version = 2.0 operations = 5*(3*5 + 3*5) rounds = 70000 Modified: python/trunk/Tools/pybench/NewInstances.py ============================================================================== --- python/trunk/Tools/pybench/NewInstances.py (original) +++ python/trunk/Tools/pybench/NewInstances.py Tue Jun 13 20:56:56 2006 @@ -1,8 +1,17 @@ from pybench import Test +# Check for new-style class support: +try: + class c(object): + pass +except NameError: + raise ImportError + +### + class CreateNewInstances(Test): - version = 0.1 + version = 2.0 operations = 3 + 7 + 4 rounds = 60000 Modified: python/trunk/Tools/pybench/Numbers.py ============================================================================== --- python/trunk/Tools/pybench/Numbers.py (original) +++ python/trunk/Tools/pybench/Numbers.py Tue Jun 13 20:56:56 2006 @@ -2,7 +2,7 @@ class CompareIntegers(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 rounds = 120000 @@ -198,9 +198,9 @@ class CompareFloats(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 - rounds = 60000 + rounds = 80000 def test(self): @@ -394,7 +394,7 @@ class CompareFloatsIntegers(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 rounds = 60000 @@ -590,9 +590,9 @@ class CompareLongs(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 - rounds = 60000 + rounds = 70000 def test(self): Modified: python/trunk/Tools/pybench/README ============================================================================== --- python/trunk/Tools/pybench/README (original) +++ python/trunk/Tools/pybench/README Tue Jun 13 20:56:56 2006 @@ -28,12 +28,37 @@ Micro-Manual ------------ -Run 'pybench.py -h' to see the help screen. -Run 'pybench.py' to just let the benchmark suite do it's thing and -'pybench.py -f ' to have it store the results in a file too. +Run 'pybench.py -h' to see the help screen. Run 'pybench.py' to run +the benchmark suite using default settings and 'pybench.py -f ' +to have it store the results in a file too. + +It is usually a good idea to run pybench.py multiple times to see +whether the environment, timers and benchmark run-times are suitable +for doing benchmark tests. + +You can use the comparison feature of pybench.py ('pybench.py -c +') to check how well the system behaves in comparison to a +reference run. + +If the differences are well below 10% for each test, then you have a +system that is good for doing benchmark testings. Of you get random +differences of more than 10% or significant differences between the +values for minimum and average time, then you likely have some +background processes running which cause the readings to become +inconsistent. Examples include: web-browsers, email clients, RSS +readers, music players, backup programs, etc. + +If you are only interested in a few tests of the whole suite, you can +use the filtering option, e.g. 'pybench.py -t string' will only +run/show the tests that have 'string' in their name. This is the current output of pybench.py --help: +""" +------------------------------------------------------------------------ +PYBENCH - a benchmark test suite for Python interpreters/compilers. +------------------------------------------------------------------------ + Synopsis: pybench.py [option] files... @@ -42,14 +67,14 @@ -f arg save benchmark to file arg () -c arg compare benchmark with the one in file arg () -s arg show benchmark in file arg, then exit () - -S show statistics of benchmarks (0) - -w arg set warp factor to arg (20) - -d hide noise in compares (0) - --no-gc disable garbage collection (0) - --no-syscheck "disable" sys check interval (set to sys.maxint) (0) - -t arg tests containing substring () - -C arg number of calibration runs (20) - -v generate verbose output + -w arg set warp factor to arg (10) + -t arg run only tests with names matching arg () + -C arg set the number of calibration runs to arg (20) + -d hide noise in comparisons (0) + -v verbose output (not recommended) (0) + --with-gc enable garbage collection (0) + --with-syscheck use default sys check interval (0) + --timer arg use given timer (time.time) -h show this help text --help show this help text --debug enable debugging @@ -57,17 +82,23 @@ --examples show examples of usage Version: - 1.3 + 2.0 The normal operation is to run the suite and display the -results. Use -f to save them for later reuse or comparisms. +results. Use -f to save them for later reuse or comparisons. -Examples: +Available timers: -python1.5 pybench.py -w 100 -f p15 -python1.4 pybench.py -w 100 -f p14 -python pybench.py -s p15 -c p14 + time.time + time.clock + systimes.processtime +Examples: + +python2.1 pybench.py -f p21.pybench +python2.5 pybench.py -f p25.pybench +python pybench.py -s p25.pybench -c p21.pybench +""" License ------- @@ -78,184 +109,103 @@ Sample output ------------- -PYBENCH 1.3 - -Machine Details: - Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64 - Executable: /home/lemburg/projects/Python/Installation/bin/python - Python: 2.5a1.0 - Compiler: GCC 3.3.4 (pre 3.3.5 20040809) - Build: Apr 9 2006 01:50:57 (#trunk) - -Searching for tests... - BuiltinFunctionCalls - BuiltinMethodLookup - CompareFloats - CompareFloatsIntegers - CompareIntegers - CompareInternedStrings - CompareLongs - CompareStrings - CompareUnicode - ConcatStrings - ConcatUnicode - CreateInstances - CreateStringsWithConcat - CreateUnicodeWithConcat - DictCreation - DictWithFloatKeys - DictWithIntegerKeys - DictWithStringKeys - ForLoops - IfThenElse - ListSlicing - NestedForLoops - NormalClassAttribute - NormalInstanceAttribute - PythonFunctionCalls - PythonMethodCalls - Recursion - SecondImport - SecondPackageImport - SecondSubmoduleImport - SimpleComplexArithmetic - SimpleDictManipulation - SimpleFloatArithmetic - SimpleIntFloatArithmetic - SimpleIntegerArithmetic - SimpleListManipulation - SimpleLongArithmetic - SmallLists - SmallTuples - SpecialClassAttribute - SpecialInstanceAttribute - StringMappings - StringPredicates - StringSlicing - TryExcept - TryRaiseExcept - TupleSlicing - UnicodeMappings - UnicodePredicates - UnicodeProperties - UnicodeSlicing - -Running 10 round(s) of the suite: - +""" +------------------------------------------------------------------------------- +PYBENCH 2.0 +------------------------------------------------------------------------------- +* using Python 2.4.2 +* disabled garbage collection +* system check interval set to maximum: 2147483647 +* using timer: time.time + +Calibrating tests. Please wait... + +Running 10 round(s) of the suite at warp factor 10: + +* Round 1 done in 6.388 seconds. +* Round 2 done in 6.485 seconds. +* Round 3 done in 6.786 seconds. ... +* Round 10 done in 6.546 seconds. - Round 10 real abs overhead - BuiltinFunctionCalls: 0.030r 0.030a 0.000o - BuiltinMethodLookup: 0.059r 0.060a 0.001o - CompareFloats: 0.050r 0.050a 0.000o - CompareFloatsIntegers: 0.050r 0.050a 0.000o - CompareIntegers: 0.070r 0.070a 0.000o - CompareInternedStrings: 0.039r 0.040a 0.001o - CompareLongs: 0.050r 0.050a 0.000o - CompareStrings: 0.060r 0.060a 0.000o - CompareUnicode: 0.060r 0.060a 0.000o - ConcatStrings: 0.040r 0.040a 0.000o - ConcatUnicode: 0.050r 0.050a 0.000o - CreateInstances: 0.050r 0.050a 0.000o - CreateStringsWithConcat: 0.029r 0.030a 0.001o - CreateUnicodeWithConcat: 0.060r 0.060a 0.000o - DictCreation: 0.040r 0.040a 0.000o - DictWithFloatKeys: 0.089r 0.090a 0.000o - DictWithIntegerKeys: 0.059r 0.060a 0.001o - DictWithStringKeys: 0.070r 0.070a 0.001o - ForLoops: 0.050r 0.050a 0.000o - IfThenElse: 0.070r 0.070a 0.000o - ListSlicing: 0.030r 0.030a 0.000o - NestedForLoops: 0.030r 0.030a 0.000o - NormalClassAttribute: 0.060r 0.060a 0.000o - NormalInstanceAttribute: 0.060r 0.060a 0.000o - PythonFunctionCalls: 0.060r 0.060a 0.000o - PythonMethodCalls: 0.050r 0.050a 0.000o - Recursion: 0.050r 0.050a 0.000o - SecondImport: 0.030r 0.030a 0.000o - SecondPackageImport: 0.030r 0.030a 0.000o - SecondSubmoduleImport: 0.040r 0.040a 0.000o - SimpleComplexArithmetic: 0.030r 0.030a 0.000o - SimpleDictManipulation: 0.040r 0.040a 0.000o - SimpleFloatArithmetic: 0.050r 0.050a 0.001o - SimpleIntFloatArithmetic: 0.060r 0.060a 0.000o - SimpleIntegerArithmetic: 0.060r 0.060a 0.000o - SimpleListManipulation: 0.030r 0.030a 0.000o - SimpleLongArithmetic: 0.030r 0.030a 0.000o - SmallLists: 0.050r 0.050a 0.000o - SmallTuples: 0.050r 0.050a 0.000o - SpecialClassAttribute: 0.060r 0.060a 0.000o - SpecialInstanceAttribute: 0.079r 0.080a 0.001o - StringMappings: 0.060r 0.060a 0.000o - StringPredicates: 0.049r 0.050a 0.001o - StringSlicing: 0.039r 0.040a 0.000o - TryExcept: 0.079r 0.080a 0.001o - TryRaiseExcept: 0.059r 0.060a 0.001o - TupleSlicing: 0.050r 0.050a 0.000o - UnicodeMappings: 0.070r 0.070a 0.001o - UnicodePredicates: 0.059r 0.060a 0.001o - UnicodeProperties: 0.059r 0.060a 0.001o - UnicodeSlicing: 0.050r 0.050a 0.000o - ---------------------- - Average round time: 2.937 seconds - - -Tests: per run per oper. overhead ------------------------------------------------------------------------- - BuiltinFunctionCalls: 29.85 ms 0.23 us 0.00 ms - BuiltinMethodLookup: 66.85 ms 0.13 us 0.50 ms - CompareFloats: 43.00 ms 0.10 us 0.00 ms - CompareFloatsIntegers: 51.80 ms 0.12 us 0.00 ms - CompareIntegers: 70.70 ms 0.08 us 0.50 ms - CompareInternedStrings: 41.40 ms 0.08 us 0.50 ms - CompareLongs: 47.90 ms 0.11 us 0.00 ms - CompareStrings: 58.50 ms 0.12 us 0.50 ms - CompareUnicode: 56.55 ms 0.15 us 0.50 ms - ConcatStrings: 44.75 ms 0.30 us 0.00 ms - ConcatUnicode: 54.55 ms 0.36 us 0.50 ms - CreateInstances: 50.95 ms 1.21 us 0.00 ms - CreateStringsWithConcat: 28.85 ms 0.14 us 0.50 ms - CreateUnicodeWithConcat: 53.75 ms 0.27 us 0.00 ms - DictCreation: 41.90 ms 0.28 us 0.00 ms - DictWithFloatKeys: 88.50 ms 0.15 us 0.50 ms - DictWithIntegerKeys: 62.55 ms 0.10 us 0.50 ms - DictWithStringKeys: 60.50 ms 0.10 us 0.50 ms - ForLoops: 46.90 ms 4.69 us 0.00 ms - IfThenElse: 60.55 ms 0.09 us 0.00 ms - ListSlicing: 29.90 ms 8.54 us 0.00 ms - NestedForLoops: 33.95 ms 0.10 us 0.00 ms - NormalClassAttribute: 62.75 ms 0.10 us 0.50 ms - NormalInstanceAttribute: 61.80 ms 0.10 us 0.50 ms - PythonFunctionCalls: 60.00 ms 0.36 us 0.00 ms - PythonMethodCalls: 50.00 ms 0.67 us 0.00 ms - Recursion: 46.85 ms 3.75 us 0.00 ms - SecondImport: 35.00 ms 1.40 us 0.00 ms - SecondPackageImport: 32.00 ms 1.28 us 0.00 ms - SecondSubmoduleImport: 38.00 ms 1.52 us 0.00 ms - SimpleComplexArithmetic: 26.85 ms 0.12 us 0.00 ms - SimpleDictManipulation: 40.85 ms 0.14 us 0.00 ms - SimpleFloatArithmetic: 48.70 ms 0.09 us 0.50 ms - SimpleIntFloatArithmetic: 57.70 ms 0.09 us 0.00 ms - SimpleIntegerArithmetic: 58.75 ms 0.09 us 0.50 ms - SimpleListManipulation: 34.80 ms 0.13 us 0.00 ms - SimpleLongArithmetic: 30.95 ms 0.19 us 0.50 ms - SmallLists: 47.60 ms 0.19 us 0.00 ms - SmallTuples: 48.80 ms 0.20 us 0.50 ms - SpecialClassAttribute: 61.70 ms 0.10 us 0.00 ms - SpecialInstanceAttribute: 76.70 ms 0.13 us 0.50 ms - StringMappings: 58.70 ms 0.47 us 0.00 ms - StringPredicates: 50.00 ms 0.18 us 1.00 ms - StringSlicing: 39.65 ms 0.23 us 0.50 ms - TryExcept: 84.45 ms 0.06 us 0.50 ms - TryRaiseExcept: 61.75 ms 4.12 us 0.50 ms - TupleSlicing: 48.95 ms 0.47 us 0.00 ms - UnicodeMappings: 71.50 ms 3.97 us 0.50 ms - UnicodePredicates: 52.75 ms 0.23 us 1.00 ms - UnicodeProperties: 61.90 ms 0.31 us 1.00 ms - UnicodeSlicing: 53.75 ms 0.31 us 0.50 ms ------------------------------------------------------------------------- - Average round time: 2937.00 ms - +------------------------------------------------------------------------------- +Benchmark: 2006-06-12 12:09:25 +------------------------------------------------------------------------------- + + Rounds: 10 + Warp: 10 + Timer: time.time + + Machine Details: + Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64 + Processor: x86_64 + + Python: + Executable: /usr/local/bin/python + Version: 2.4.2 + Compiler: GCC 3.3.4 (pre 3.3.5 20040809) + Bits: 64bit + Build: Oct 1 2005 15:24:35 (#1) + Unicode: UCS2 + + +Test minimum average operation overhead +------------------------------------------------------------------------------- + BuiltinFunctionCalls: 126ms 145ms 0.28us 0.274ms + BuiltinMethodLookup: 124ms 130ms 0.12us 0.316ms + CompareFloats: 109ms 110ms 0.09us 0.361ms + CompareFloatsIntegers: 100ms 104ms 0.12us 0.271ms + CompareIntegers: 137ms 138ms 0.08us 0.542ms + CompareInternedStrings: 124ms 127ms 0.08us 1.367ms + CompareLongs: 100ms 104ms 0.10us 0.316ms + CompareStrings: 111ms 115ms 0.12us 0.929ms + CompareUnicode: 108ms 128ms 0.17us 0.693ms + ConcatStrings: 142ms 155ms 0.31us 0.562ms + ConcatUnicode: 119ms 127ms 0.42us 0.384ms + CreateInstances: 123ms 128ms 1.14us 0.367ms + CreateNewInstances: 121ms 126ms 1.49us 0.335ms + CreateStringsWithConcat: 130ms 135ms 0.14us 0.916ms + CreateUnicodeWithConcat: 130ms 135ms 0.34us 0.361ms + DictCreation: 108ms 109ms 0.27us 0.361ms + DictWithFloatKeys: 149ms 153ms 0.17us 0.678ms + DictWithIntegerKeys: 124ms 126ms 0.11us 0.915ms + DictWithStringKeys: 114ms 117ms 0.10us 0.905ms + ForLoops: 110ms 111ms 4.46us 0.063ms + IfThenElse: 118ms 119ms 0.09us 0.685ms + ListSlicing: 116ms 120ms 8.59us 0.103ms + NestedForLoops: 125ms 137ms 0.09us 0.019ms + NormalClassAttribute: 124ms 136ms 0.11us 0.457ms + NormalInstanceAttribute: 110ms 117ms 0.10us 0.454ms + PythonFunctionCalls: 107ms 113ms 0.34us 0.271ms + PythonMethodCalls: 140ms 149ms 0.66us 0.141ms + Recursion: 156ms 166ms 3.32us 0.452ms + SecondImport: 112ms 118ms 1.18us 0.180ms + SecondPackageImport: 118ms 127ms 1.27us 0.180ms + SecondSubmoduleImport: 140ms 151ms 1.51us 0.180ms + SimpleComplexArithmetic: 128ms 139ms 0.16us 0.361ms + SimpleDictManipulation: 134ms 136ms 0.11us 0.452ms + SimpleFloatArithmetic: 110ms 113ms 0.09us 0.571ms + SimpleIntFloatArithmetic: 106ms 111ms 0.08us 0.548ms + SimpleIntegerArithmetic: 106ms 109ms 0.08us 0.544ms + SimpleListManipulation: 103ms 113ms 0.10us 0.587ms + SimpleLongArithmetic: 112ms 118ms 0.18us 0.271ms + SmallLists: 105ms 116ms 0.17us 0.366ms + SmallTuples: 108ms 128ms 0.24us 0.406ms + SpecialClassAttribute: 119ms 136ms 0.11us 0.453ms + SpecialInstanceAttribute: 143ms 155ms 0.13us 0.454ms + StringMappings: 115ms 121ms 0.48us 0.405ms + StringPredicates: 120ms 129ms 0.18us 2.064ms + StringSlicing: 111ms 127ms 0.23us 0.781ms + TryExcept: 125ms 126ms 0.06us 0.681ms + TryRaiseExcept: 133ms 137ms 2.14us 0.361ms + TupleSlicing: 117ms 120ms 0.46us 0.066ms + UnicodeMappings: 156ms 160ms 4.44us 0.429ms + UnicodePredicates: 117ms 121ms 0.22us 2.487ms + UnicodeProperties: 115ms 153ms 0.38us 2.070ms + UnicodeSlicing: 126ms 129ms 0.26us 0.689ms +------------------------------------------------------------------------------- +Totals: 6283ms 6673ms +""" ________________________________________________________________________ Writing New Tests @@ -293,7 +243,7 @@ # Number of rounds to execute per test run. This should be # adjusted to a figure that results in a test run-time of between - # 20-50 seconds. + # 1-2 seconds (at warp 1). rounds = 100000 def test(self): @@ -377,6 +327,41 @@ variable should be updated. Therefafter, comparisons with previous versions of the test will list as "n/a" to reflect the change. + +Version History +--------------- + + 2.0: rewrote parts of pybench which resulted in more repeatable + timings: + - made timer a parameter + - changed the platform default timer to use high-resolution + timers rather than process timers (which have a much lower + resolution) + - added option to select timer + - added process time timer (using systimes.py) + - changed to use min() as timing estimator (average + is still taken as well to provide an idea of the difference) + - garbage collection is turned off per default + - sys check interval is set to the highest possible value + - calibration is now a separate step and done using + a different strategy that allows measuring the test + overhead more accurately + - modified the tests to each give a run-time of between + 100-200ms using warp 10 + - changed default warp factor to 10 (from 20) + - compared results with timeit.py and confirmed measurements + - bumped all test versions to 2.0 + - updated platform.py to the latest version + - changed the output format a bit to make it look + nicer + - refactored the APIs somewhat + 1.3+: Steve Holden added the NewInstances test and the filtering + option during the NeedForSpeed sprint; this also triggered a long + discussion on how to improve benchmark timing and finally + resulted in the release of 2.0 + 1.3: initial checkin into the Python SVN repository + + Have fun, -- Marc-Andre Lemburg Modified: python/trunk/Tools/pybench/Setup.py ============================================================================== --- python/trunk/Tools/pybench/Setup.py (original) +++ python/trunk/Tools/pybench/Setup.py Tue Jun 13 20:56:56 2006 @@ -14,7 +14,7 @@ # Defaults Number_of_rounds = 10 -Warp_factor = 20 +Warp_factor = 10 # Import tests from Arithmetic import * @@ -24,8 +24,8 @@ from Instances import * try: from NewInstances import * -except: - print "Cannot test new-style objects" +except ImportError: + pass from Lists import * from Tuples import * from Dict import * Modified: python/trunk/Tools/pybench/Strings.py ============================================================================== --- python/trunk/Tools/pybench/Strings.py (original) +++ python/trunk/Tools/pybench/Strings.py Tue Jun 13 20:56:56 2006 @@ -3,9 +3,9 @@ class ConcatStrings(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 - rounds = 60000 + rounds = 100000 def test(self): @@ -85,7 +85,7 @@ class CompareStrings(Test): - version = 0.2 + version = 2.0 operations = 10 * 5 rounds = 200000 @@ -167,9 +167,9 @@ class CompareInternedStrings(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 - rounds = 200000 + rounds = 300000 def test(self): @@ -249,9 +249,9 @@ class CreateStringsWithConcat(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 - rounds = 80000 + rounds = 200000 def test(self): @@ -324,9 +324,9 @@ class StringSlicing(Test): - version = 0.1 + version = 2.0 operations = 5 * 7 - rounds = 100000 + rounds = 160000 def test(self): @@ -387,7 +387,7 @@ class StringMappings(Test): - version = 0.1 + version = 2.0 operations = 3 * (5 + 4 + 2 + 1) rounds = 70000 @@ -460,9 +460,9 @@ class StringPredicates(Test): - version = 0.1 + version = 2.0 operations = 10 * 7 - rounds = 80000 + rounds = 100000 def test(self): Modified: python/trunk/Tools/pybench/Tuples.py ============================================================================== --- python/trunk/Tools/pybench/Tuples.py (original) +++ python/trunk/Tools/pybench/Tuples.py Tue Jun 13 20:56:56 2006 @@ -2,18 +2,17 @@ class TupleSlicing(Test): - version = 0.31 + version = 2.0 operations = 3 * 25 * 10 * 7 - rounds = 400 + rounds = 500 def test(self): r = range(25) + t = tuple(range(100)) for i in xrange(self.rounds): - t = tuple(range(100)) - for j in r: m = t[50:] @@ -259,20 +258,17 @@ def calibrate(self): r = range(25) + t = tuple(range(100)) for i in xrange(self.rounds): - - t = tuple(range(100)) - for j in r: - pass class SmallTuples(Test): - version = 0.3 + version = 2.0 operations = 5*(1 + 3 + 6 + 2) - rounds = 80000 + rounds = 90000 def test(self): Modified: python/trunk/Tools/pybench/Unicode.py ============================================================================== --- python/trunk/Tools/pybench/Unicode.py (original) +++ python/trunk/Tools/pybench/Unicode.py Tue Jun 13 20:56:56 2006 @@ -8,7 +8,7 @@ class ConcatUnicode(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 rounds = 60000 @@ -90,7 +90,7 @@ class CompareUnicode(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 rounds = 150000 @@ -172,7 +172,7 @@ class CreateUnicodeWithConcat(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 rounds = 80000 @@ -247,9 +247,9 @@ class UnicodeSlicing(Test): - version = 0.1 + version = 2.0 operations = 5 * 7 - rounds = 100000 + rounds = 140000 def test(self): @@ -308,7 +308,7 @@ class UnicodeMappings(Test): - version = 0.1 + version = 2.0 operations = 3 * (5 + 4 + 2 + 1) rounds = 10000 @@ -381,9 +381,9 @@ class UnicodePredicates(Test): - version = 0.1 + version = 2.0 operations = 5 * 9 - rounds = 100000 + rounds = 120000 def test(self): @@ -458,7 +458,7 @@ else: class UnicodeProperties(Test): - version = 0.1 + version = 2.0 operations = 5 * 8 rounds = 100000 Added: python/trunk/Tools/pybench/clockres.py ============================================================================== --- (empty file) +++ python/trunk/Tools/pybench/clockres.py Tue Jun 13 20:56:56 2006 @@ -0,0 +1,44 @@ +#!/usr/bin/env python + +""" clockres - calculates the resolution in seconds of a given timer. + + Copyright (c) 2006, Marc-Andre Lemburg (mal at egenix.com). See the + documentation for further information on copyrights, or contact + the author. All Rights Reserved. + +""" +import time + +TEST_TIME = 1.0 + +def clockres(timer): + d = {} + wallclock = time.time + start = wallclock() + stop = wallclock() + TEST_TIME + spin_loops = range(1000) + while 1: + now = wallclock() + if now >= stop: + break + for i in spin_loops: + d[timer()] = 1 + values = d.keys() + values.sort() + min_diff = TEST_TIME + for i in range(len(values) - 1): + diff = values[i+1] - values[i] + if diff < min_diff: + min_diff = diff + return min_diff + +if __name__ == '__main__': + print 'Clock resolution of various timer implementations:' + print 'time.clock: %10.3fus' % (clockres(time.clock) * 1e6) + print 'time.time: %10.3fus' % (clockres(time.time) * 1e6) + try: + import systimes + print 'systimes.processtime: %10.3fus' % (clockres(systimes.processtime) * 1e6) + except ImportError: + pass + Modified: python/trunk/Tools/pybench/pybench.py ============================================================================== --- python/trunk/Tools/pybench/pybench.py (original) +++ python/trunk/Tools/pybench/pybench.py Tue Jun 13 20:56:56 2006 @@ -34,20 +34,7 @@ WITH THE USE OR PERFORMANCE OF THIS SOFTWARE ! """ -# Version number -__version__ = '1.3' - -# -# NOTE: Use xrange for all test loops unless you want to face -# a 20MB process ! -# -# All tests should have rounds set to values so that a run() -# takes between 20-50 seconds. This is to get fairly good -# clock() values. You can use option -w to speedup the tests -# by a fixed integer factor (the "warp factor"). -# - -import sys,time,operator +import sys, time, operator, string from CommandLine import * try: @@ -56,6 +43,111 @@ except ImportError: import pickle +# Version number; version history: see README file ! +__version__ = '2.0' + +### Constants + +# Second fractions +MILLI_SECONDS = 1e3 +MICRO_SECONDS = 1e6 + +# Percent unit +PERCENT = 100 + +# Horizontal line length +LINE = 79 + +# Minimum test run-time +MIN_TEST_RUNTIME = 1e-3 + +# Number of calibration runs to use for calibrating the tests +CALIBRATION_RUNS = 20 + +# Number of calibration loops to run for each calibration run +CALIBRATION_LOOPS = 20 + +# Allow skipping calibration ? +ALLOW_SKIPPING_CALIBRATION = 1 + +# Timer types +TIMER_TIME_TIME = 'time.time' +TIMER_TIME_CLOCK = 'time.clock' +TIMER_SYSTIMES_PROCESSTIME = 'systimes.processtime' + +# Choose platform default timer +if sys.platform[:3] == 'win': + # On WinXP this has 2.5ms resolution + TIMER_PLATFORM_DEFAULT = TIMER_TIME_CLOCK +else: + # On Linux this has 1ms resolution + TIMER_PLATFORM_DEFAULT = TIMER_TIME_TIME + +# Print debug information ? +_debug = 0 + +### Helpers + +def get_timer(timertype): + + if timertype == TIMER_TIME_TIME: + return time.time + elif timertype == TIMER_TIME_CLOCK: + return time.clock + elif timertype == TIMER_SYSTIMES_PROCESSTIME: + import systimes + return systimes.processtime + else: + raise TypeError('unknown timer type: %s' % timertype) + +def get_machine_details(): + + import platform + if _debug: + print 'Getting machine details...' + buildno, builddate = platform.python_build() + python = platform.python_version() + if python > '2.0': + try: + unichr(100000) + except ValueError: + # UCS2 build (standard) + unicode = 'UCS2' + else: + # UCS4 build (most recent Linux distros) + unicode = 'UCS4' + else: + unicode = None + bits, linkage = platform.architecture() + return { + 'platform': platform.platform(), + 'processor': platform.processor(), + 'executable': sys.executable, + 'python': platform.python_version(), + 'compiler': platform.python_compiler(), + 'buildno': buildno, + 'builddate': builddate, + 'unicode': unicode, + 'bits': bits, + } + +def print_machine_details(d, indent=''): + + l = ['Machine Details:', + ' Platform ID: %s' % d.get('platform', 'n/a'), + ' Processor: %s' % d.get('processor', 'n/a'), + '', + 'Python:', + ' Executable: %s' % d.get('executable', 'n/a'), + ' Version: %s' % d.get('python', 'n/a'), + ' Compiler: %s' % d.get('compiler', 'n/a'), + ' Bits: %s' % d.get('bits', 'n/a'), + ' Build: %s (#%s)' % (d.get('builddate', 'n/a'), + d.get('buildno', 'n/a')), + ' Unicode: %s' % d.get('unicode', 'n/a'), + ] + print indent + string.join(l, '\n' + indent) + '\n' + ### Test baseclass class Test: @@ -84,7 +176,7 @@ # Version number of the test as float (x.yy); this is important # for comparisons of benchmark runs - tests with unequal version # number will not get compared. - version = 1.0 + version = 2.0 # The number of abstract operations done in each round of the # test. An operation is the basic unit of what you want to @@ -97,36 +189,125 @@ # Number of rounds to execute per test run. This should be # adjusted to a figure that results in a test run-time of between - # 20-50 seconds. - rounds = 10000 + # 1-2 seconds. + rounds = 100000 ### Internal variables # Mark this class as implementing a test is_a_test = 1 - # Misc. internal variables - last_timing = (0,0,0) # last timing (real,run,calibration) - warp = 1 # warp factor this test uses - cruns = 20 # number of calibration runs - overhead = None # list of calibration timings + # Last timing: (real, run, overhead) + last_timing = (0.0, 0.0, 0.0) + + # Warp factor to use for this test + warp = 1 + + # Number of calibration runs to use + calibration_runs = CALIBRATION_RUNS + + # List of calibration timings + overhead_times = None + + # List of test run timings + times = [] - def __init__(self,warp=1): + # Timer used for the benchmark + timer = TIMER_PLATFORM_DEFAULT - if warp > 1: - self.rounds = self.rounds / warp + def __init__(self, warp=None, calibration_runs=None, timer=None): + + # Set parameters + if warp is not None: + self.rounds = int(self.rounds / warp) if self.rounds == 0: - self.rounds = 1 + raise ValueError('warp factor set too high') self.warp = warp + if calibration_runs is not None: + if (not ALLOW_SKIPPING_CALIBRATION and + calibration_runs < 1): + raise ValueError('at least one calibration run is required') + self.calibration_runs = calibration_runs + if timer is not None: + timer = timer + + # Init variables self.times = [] - self.overhead = [] + self.overhead_times = [] + # We want these to be in the instance dict, so that pickle # saves them self.version = self.version self.operations = self.operations self.rounds = self.rounds - def run(self, cruns): + def get_timer(self): + + """ Return the timer function to use for the test. + + """ + return get_timer(self.timer) + + def compatible(self, other): + + """ Return 1/0 depending on whether the test is compatible + with the other Test instance or not. + + """ + if self.version != other.version: + return 0 + if self.rounds != other.rounds: + return 0 + return 1 + + def calibrate_test(self): + + if self.calibration_runs == 0: + self.overhead_times = [0.0] + return + + calibrate = self.calibrate + timer = self.get_timer() + calibration_loops = range(CALIBRATION_LOOPS) + + # Time the calibration loop overhead + prep_times = [] + for i in range(self.calibration_runs): + t = timer() + for i in calibration_loops: + pass + t = timer() - t + prep_times.append(t) + min_prep_time = min(prep_times) + if _debug: + print + print 'Calib. prep time = %.6fms' % ( + min_prep_time * MILLI_SECONDS) + + # Time the calibration runs (doing CALIBRATION_LOOPS loops of + # .calibrate() method calls each) + for i in range(self.calibration_runs): + t = timer() + for i in calibration_loops: + calibrate() + t = timer() - t + self.overhead_times.append(t / CALIBRATION_LOOPS + - min_prep_time) + + # Check the measured times + min_overhead = min(self.overhead_times) + max_overhead = max(self.overhead_times) + if _debug: + print 'Calib. overhead time = %.6fms' % ( + min_overhead * MILLI_SECONDS) + if min_overhead < 0.0: + raise ValueError('calibration setup did not work') + if max_overhead - min_overhead > 0.1: + raise ValueError( + 'overhead calibration timing range too inaccurate: ' + '%r - %r' % (min_overhead, max_overhead)) + + def run(self): """ Run the test in two phases: first calibrate, then do the actual test. Be careful to keep the calibration @@ -134,27 +315,23 @@ """ test = self.test - calibrate = self.calibrate - clock = time.clock - # first calibrate - t = clock() - calibrate() - offset = clock() - t - if cruns: - for i in range(cruns-1): - t = clock() - calibrate() - t = clock() - t - if t < offset: - offset = t - # now the real thing - t = clock() + timer = self.get_timer() + + # Get calibration + min_overhead = min(self.overhead_times) + + # Test run + t = timer() test() - t = clock() - t - if t < 0.01: - sys.exit("Lower warp required: test times < 10 ms are unreliable") - self.last_timing = (t-offset,t,offset) - self.times.append(t-offset) + t = timer() - t + if t < MIN_TEST_RUNTIME: + raise ValueError('warp factor too high: ' + 'test times are < 10ms') + eff_time = t - min_overhead + if eff_time < 0: + raise ValueError('wrong calibration') + self.last_timing = (eff_time, t, min_overhead) + self.times.append(eff_time) def calibrate(self): @@ -176,33 +353,33 @@ self.operations number of operations each. """ - # do some tests return def stat(self): - """ Returns four values: - minimum round time - average time per round - average time per operation - average overhead time + """ Return test run statistics as tuple: + + (minimum run time, + average run time, + total run time, + average time per operation, + minimum overhead time) - XXX Should this take warp factors into account? """ runs = len(self.times) if runs == 0: - return 0,0 - mintime = min(self.times) - totaltime = reduce(operator.add,self.times,0.0) - avg = totaltime / float(runs) - op_avg = totaltime / float(runs * self.rounds * self.operations) - if self.overhead: - totaloverhead = reduce(operator.add,self.overhead,0.0) - ov_avg = totaloverhead / float(runs) + return 0.0, 0.0, 0.0, 0.0 + min_time = min(self.times) + total_time = reduce(operator.add, self.times, 0.0) + avg_time = total_time / float(runs) + operation_avg = total_time / float(runs + * self.rounds + * self.operations) + if self.overhead_times: + min_overhead = min(self.overhead_times) else: - # use self.last_timing - not too accurate - ov_avg = self.last_timing[2] - return mintime, avg, op_avg, ov_avg + min_overhead = self.last_timing[2] + return min_time, avg_time, total_time, operation_avg, min_overhead ### Load Setup @@ -215,153 +392,353 @@ class Benchmark: - name = '?' # Name of the benchmark - rounds = 1 # Number of rounds to run + # Name of the benchmark + name = '' + + # Number of benchmark rounds to run + rounds = 1 + + # Warp factor use to run the tests warp = 1 # Warp factor - roundtime = 0 # Average round time - version = None # Benchmark version number (see __init__) - # as float x.yy - def __init__(self): + # Average benchmark round time + roundtime = 0 - self.tests = {} - self.version = 0.31 + # Benchmark version number as float x.yy + version = 2.0 - def load_tests(self, setupmod, warp=1, limitnames="", verbose=0): + # Produce verbose output ? + verbose = 0 - self.warp = warp - if limitnames: - limitnames = re.compile(limitnames, re.I) + # Dictionary with the machine details + machine_details = None + + # Timer used for the benchmark + timer = TIMER_PLATFORM_DEFAULT + + def __init__(self, name, verbose=None, timer=None, warp=None, + calibration_runs=None): + + if name: + self.name = name else: - limitnames = None - tests = self.tests - if verbose: - print 'Searching for tests ...', - setupmod.__dict__.values() - for c in setupmod.__dict__.values(): - if not hasattr(c,'is_a_test'): + self.name = '%04i-%02i-%02i %02i:%02i:%02i' % \ + (time.localtime(time.time())[:6]) + if verbose is not None: + self.verbose = verbose + if timer is not None: + self.timer = timer + if warp is not None: + self.warp = warp + if calibration_runs is not None: + self.calibration_runs = calibration_runs + + # Init vars + self.tests = {} + if _debug: + print 'Getting machine details...' + self.machine_details = get_machine_details() + + # Make .version an instance attribute to have it saved in the + # Benchmark pickle + self.version = self.version + + def get_timer(self): + + """ Return the timer function to use for the test. + + """ + return get_timer(self.timer) + + def compatible(self, other): + + """ Return 1/0 depending on whether the benchmark is + compatible with the other Benchmark instance or not. + + """ + if self.version != other.version: + return 0 + if (self.machine_details == other.machine_details and + self.timer != other.timer): + return 0 + if (self.calibration_runs == 0 and + other.calibration_runs != 0): + return 0 + if (self.calibration_runs != 0 and + other.calibration_runs == 0): + return 0 + return 1 + + def load_tests(self, setupmod, limitnames=None): + + # Add tests + if self.verbose: + print 'Searching for tests ...' + print '--------------------------------------' + for testclass in setupmod.__dict__.values(): + if not hasattr(testclass, 'is_a_test'): continue - name = c.__name__ + name = testclass.__name__ if name == 'Test': continue - if limitnames is not None and limitnames.search(name) is None: + if (limitnames is not None and + limitnames.search(name) is None): continue - tests[name] = c(warp) - l = tests.keys() + self.tests[name] = testclass( + warp=self.warp, + calibration_runs=self.calibration_runs, + timer=self.timer) + l = self.tests.keys() l.sort() - if verbose: + if self.verbose: + for name in l: + print ' %s' % name + print '--------------------------------------' + print ' %i tests found' % len(l) print - for t in l: - print ' ', t - print len(l), "tests found" + + def calibrate(self): + + print 'Calibrating tests. Please wait...' + if self.verbose: + print + print 'Test min max' + print '-' * LINE + tests = self.tests.items() + tests.sort() + for i in range(len(tests)): + name, test = tests[i] + test.calibrate_test() + if self.verbose: + print '%30s: %6.3fms %6.3fms' % \ + (name, + min(test.overhead_times) * MILLI_SECONDS, + max(test.overhead_times) * MILLI_SECONDS) print - def run(self, verbose, cruns): + def run(self): tests = self.tests.items() tests.sort() - clock = time.clock - print 'Running %i round(s) of the suite at warp factor %i:' % (self.rounds, self.warp) + timer = self.get_timer() + print 'Running %i round(s) of the suite at warp factor %i:' % \ + (self.rounds, self.warp) print - roundtime = clock() + self.roundtimes = [] for i in range(self.rounds): - roundstarttime = clock() - if verbose: - print ' Round %-25i real abs overhead' % (i+1) + if self.verbose: + print ' Round %-25i effective absolute overhead' % (i+1) + total_eff_time = 0.0 for j in range(len(tests)): - name, t = tests[j] - if verbose: + name, test = tests[j] + if self.verbose: print '%30s:' % name, - t.run(cruns) - if verbose: - print ' %.3fr %.3fa %.3fo' % t.last_timing - if verbose: - print ' ----------------------' - print ' Average round time: %.3f seconds' % \ - ((clock() - roundtime)/(i+1)) + test.run() + (eff_time, abs_time, min_overhead) = test.last_timing + total_eff_time = total_eff_time + eff_time + if self.verbose: + print ' %5.0fms %5.0fms %7.3fms' % \ + (eff_time * MILLI_SECONDS, + abs_time * MILLI_SECONDS, + min_overhead * MILLI_SECONDS) + self.roundtimes.append(total_eff_time) + if self.verbose: + print (' ' + ' ------------------------------') + print (' ' + ' Totals: %6.0fms' % + (total_eff_time * MILLI_SECONDS)) print else: - print '%d done in %.3f seconds' % (i+1, (clock() - roundstarttime)) - self.roundtime = (clock() - roundtime) / self.rounds + print '* Round %i done in %.3f seconds.' % (i+1, + total_eff_time) print - def print_stat(self, compare_to=None, hidenoise=0): + def stat(self): + + """ Return benchmark run statistics as tuple: - if not compare_to: - print '%-30s min run avg run per oprn overhead' % 'Tests:' - print '-'*77 - tests = self.tests.items() - tests.sort() - totalmintime = 0 - for name,t in tests: - mintime,avg,op_avg,ov_avg = t.stat() - totalmintime += mintime - print '%30s: %9.2f ms %9.2f ms %6.2f us %6.2f' % \ - (name,mintime*1000.0,avg*1000.0,op_avg*1000000.0,ov_avg*1000.0) - print '-'*77 - print '%30s: %9.2f ms' % \ - ('Notional minimum round time', totalmintime * 1000.0) + (minimum round time, + average round time, + maximum round time) - else: - print 'Comparing with: %s (rounds=%i, warp=%i)' % \ - (compare_to.name,compare_to.rounds,compare_to.warp) - print '%-30s min run cmp run avg run diff' % \ - 'Tests:' - print '-'*77 - tests = self.tests.items() - tests.sort() - compatible = 1 - totalmintime = other_totalmintime = 0 - for name, t in tests: - mintime, avg, op_avg, ov_avg = t.stat() - totalmintime += mintime - try: - other = compare_to.tests[name] - except KeyError: - other = None - if other and other.version == t.version and \ - other.operations == t.operations: - mintime1, avg1, op_avg1, ov_avg1 = other.stat() - other_totalmintime += mintime1 - diff = ((mintime*self.warp)/(mintime1*other.warp) - 1.0)*100.0 - if hidenoise and abs(qop_avg) < 10: - diff = '' + XXX Currently not used, since the benchmark does test + statistics across all rounds. + + """ + runs = len(self.roundtimes) + if runs == 0: + return 0.0, 0.0 + min_time = min(self.roundtimes) + total_time = reduce(operator.add, self.roundtimes, 0.0) + avg_time = total_time / float(runs) + max_time = max(self.roundtimes) + return (min_time, avg_time, max_time) + + def print_header(self, title='Benchmark'): + + print '-' * LINE + print '%s: %s' % (title, self.name) + print '-' * LINE + print + print ' Rounds: %s' % self.rounds + print ' Warp: %s' % self.warp + print ' Timer: %s' % self.timer + print + if self.machine_details: + print_machine_details(self.machine_details, indent=' ') + print + + def print_benchmark(self, hidenoise=0, limitnames=None): + + print ('Test ' + ' minimum average operation overhead') + print '-' * LINE + tests = self.tests.items() + tests.sort() + total_min_time = 0.0 + total_avg_time = 0.0 + for name, test in tests: + if (limitnames is not None and + limitnames.search(name) is None): + continue + (min_time, + avg_time, + total_time, + op_avg, + min_overhead) = test.stat() + total_min_time = total_min_time + min_time + total_avg_time = total_avg_time + avg_time + print '%30s: %5.0fms %5.0fms %6.2fus %7.3fms' % \ + (name, + min_time * MILLI_SECONDS, + avg_time * MILLI_SECONDS, + op_avg * MICRO_SECONDS, + min_overhead *MILLI_SECONDS) + print '-' * LINE + print ('Totals: ' + ' %6.0fms %6.0fms' % + (total_min_time * MILLI_SECONDS, + total_avg_time * MILLI_SECONDS, + )) + print + + def print_comparison(self, compare_to, hidenoise=0, limitnames=None): + + # Check benchmark versions + if compare_to.version != self.version: + print ('* Benchmark versions differ: ' + 'cannot compare this benchmark to "%s" !' % + compare_to.name) + print + self.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) + return + + # Print header + compare_to.print_header('Comparing with') + print ('Test ' + ' minimum run-time average run-time') + print (' ' + ' this other diff this other diff') + print '-' * LINE + + # Print test comparisons + tests = self.tests.items() + tests.sort() + total_min_time = other_total_min_time = 0.0 + total_avg_time = other_total_avg_time = 0.0 + benchmarks_compatible = self.compatible(compare_to) + tests_compatible = 1 + for name, test in tests: + if (limitnames is not None and + limitnames.search(name) is None): + continue + (min_time, + avg_time, + total_time, + op_avg, + min_overhead) = test.stat() + total_min_time = total_min_time + min_time + total_avg_time = total_avg_time + avg_time + try: + other = compare_to.tests[name] + except KeyError: + other = None + if other is None: + # Other benchmark doesn't include the given test + min_diff, avg_diff = 'n/a', 'n/a' + other_min_time = 0.0 + other_avg_time = 0.0 + tests_compatible = 0 + else: + (other_min_time, + other_avg_time, + other_total_time, + other_op_avg, + other_min_overhead) = other.stat() + other_total_min_time = other_total_min_time + other_min_time + other_total_avg_time = other_total_avg_time + other_avg_time + if (benchmarks_compatible and + test.compatible(other)): + # Both benchmark and tests are comparible + min_diff = ((min_time * self.warp) / + (other_min_time * other.warp) - 1.0) + avg_diff = ((avg_time * self.warp) / + (other_avg_time * other.warp) - 1.0) + if hidenoise and abs(min_diff) < 10.0: + min_diff = '' + else: + min_diff = '%+5.1f%%' % (min_diff * PERCENT) + if hidenoise and abs(avg_diff) < 10.0: + avg_diff = '' else: - diff = '%+7.2f%%' % diff + avg_diff = '%+5.1f%%' % (avg_diff * PERCENT) else: - qavg, diff = 'n/a', 'n/a' - compatible = 0 - print '%30s: %8.2f ms %8.2f ms %8.2f ms %8s' % \ - (name,mintime*1000.0,mintime1*1000.0 * compare_to.warp/self.warp, avg*1000.0,diff) - print '-'*77 - # - # Summarise test results - # - if compatible and compare_to.roundtime > 0 and \ - compare_to.version == self.version: - print '%30s: %8.2f ms %8.2f ms %+7.2f%%' % \ - ('Notional minimum round time', totalmintime * 1000.0, - other_totalmintime * 1000.0 * compare_to.warp/self.warp, - ((totalmintime*self.warp)/ - (other_totalmintime*compare_to.warp)-1.0)*100.0) + # Benchmark or tests are not comparible + min_diff, avg_diff = 'n/a', 'n/a' + tests_compatible = 0 + print '%30s: %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' % \ + (name, + min_time * MILLI_SECONDS, + other_min_time * MILLI_SECONDS * compare_to.warp / self.warp, + min_diff, + avg_time * MILLI_SECONDS, + other_avg_time * MILLI_SECONDS * compare_to.warp / self.warp, + avg_diff) + print '-' * LINE + + # Summarise test results + if not benchmarks_compatible or not tests_compatible: + min_diff, avg_diff = 'n/a', 'n/a' + else: + if other_total_min_time != 0.0: + min_diff = '%+5.1f%%' % ( + ((total_min_time * self.warp) / + (other_total_min_time * compare_to.warp) - 1.0) * PERCENT) + else: + min_diff = 'n/a' + if other_total_avg_time != 0.0: + avg_diff = '%+5.1f%%' % ( + ((total_avg_time * self.warp) / + (other_total_avg_time * compare_to.warp) - 1.0) * PERCENT) else: - print '%30s: %9.2f ms n/a' % \ - ('Notional minimum round time', totalmintime * 1000.0) + avg_diff = 'n/a' + print ('Totals: ' + ' %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' % + (total_min_time * MILLI_SECONDS, + (other_total_min_time * compare_to.warp/self.warp + * MILLI_SECONDS), + min_diff, + total_avg_time * MILLI_SECONDS, + (other_total_avg_time * compare_to.warp/self.warp + * MILLI_SECONDS), + avg_diff + )) + print + print '(this=%s, other=%s)' % (self.name, + compare_to.name) print - -def print_machine(): - - import platform - print 'Machine Details:' - print ' Platform ID: %s' % platform.platform() - print ' Executable: %s' % sys.executable - # There's a bug in Python 2.2b1+... - if sys.version[:6] == '2.2b1+': - return - print ' Python: %s' % platform.python_version() - print ' Compiler: %s' % platform.python_compiler() - buildno, builddate = platform.python_build() - print ' Build: %s (#%s)' % (builddate, buildno) class PyBenchCmdline(Application): @@ -370,50 +747,64 @@ version = __version__ - options = [ArgumentOption('-n','number of rounds',Setup.Number_of_rounds), - ArgumentOption('-f','save benchmark to file arg',''), - ArgumentOption('-c','compare benchmark with the one in file arg',''), - ArgumentOption('-s','show benchmark in file arg, then exit',''), - SwitchOption('-S','show statistics of benchmarks',0), - ArgumentOption('-w','set warp factor to arg',Setup.Warp_factor), - SwitchOption('-d','hide noise in compares', 0), - SwitchOption('-v','verbose output (not recommended)', 0), - SwitchOption('--no-gc','disable garbage collection', 0), - SwitchOption('--no-syscheck', - '"disable" sys check interval (set to sys.maxint)', 0), - ArgumentOption('-t', 'tests containing substring', ''), - ArgumentOption('-C', 'number of calibration runs', 20) + debug = _debug + + options = [ArgumentOption('-n', + 'number of rounds', + Setup.Number_of_rounds), + ArgumentOption('-f', + 'save benchmark to file arg', + ''), + ArgumentOption('-c', + 'compare benchmark with the one in file arg', + ''), + ArgumentOption('-s', + 'show benchmark in file arg, then exit', + ''), + ArgumentOption('-w', + 'set warp factor to arg', + Setup.Warp_factor), + ArgumentOption('-t', + 'run only tests with names matching arg', + ''), + ArgumentOption('-C', + 'set the number of calibration runs to arg', + CALIBRATION_RUNS), + SwitchOption('-d', + 'hide noise in comparisons', + 0), + SwitchOption('-v', + 'verbose output (not recommended)', + 0), + SwitchOption('--with-gc', + 'enable garbage collection', + 0), + SwitchOption('--with-syscheck', + 'use default sys check interval', + 0), + ArgumentOption('--timer', + 'use given timer', + TIMER_PLATFORM_DEFAULT), ] about = """\ The normal operation is to run the suite and display the -results. Use -f to save them for later reuse or comparisms. +results. Use -f to save them for later reuse or comparisons. + +Available timers: + + time.time + time.clock + systimes.processtime Examples: -python1.5 pybench.py -w 100 -f p15 -python1.4 pybench.py -w 100 -f p14 -python pybench.py -s p15 -c p14 +python2.1 pybench.py -f p21.pybench +python2.5 pybench.py -f p25.pybench +python pybench.py -s p25.pybench -c p21.pybench """ copyright = __copyright__ - def handle_S(self, value): - - """ Display one line stats for each benchmark file given on the - command line. - - """ - for benchmark in self.files: - try: - f = open(benchmark, 'rb') - bench = pickle.load(f) - f.close() - except IOError: - print '* Error opening/reading file %s' % repr(benchmark) - else: - print '%s,%-.2f,ms' % (benchmark, bench.roundtime*1000.0) - return 0 - def main(self): rounds = self.values['-n'] @@ -421,37 +812,51 @@ show_bench = self.values['-s'] compare_to = self.values['-c'] hidenoise = self.values['-d'] - warp = self.values['-w'] - nogc = self.values['--no-gc'] + warp = int(self.values['-w']) + withgc = self.values['--with-gc'] limitnames = self.values['-t'] + if limitnames: + if _debug: + print '* limiting test names to one with substring "%s"' % \ + limitnames + limitnames = re.compile(limitnames, re.I) + else: + limitnames = None verbose = self.verbose - nosyscheck = self.values['--no-syscheck'] - cruns = self.values['-C'] - print "CRUNS:", cruns + withsyscheck = self.values['--with-syscheck'] + calibration_runs = self.values['-C'] + timer = self.values['--timer'] + + print '-' * LINE + print 'PYBENCH %s' % __version__ + print '-' * LINE + print '* using Python %s' % (string.split(sys.version)[0]) - print 'PYBENCH',__version__ - - # Switch off GC - if nogc: + # Switch off garbage collection + if not withgc: try: import gc except ImportError: - nogc = 0 + print '* Python version doesn\'t support garbage collection' else: - if self.values['--no-gc']: - gc.disable() - print 'NO GC' - - # maximise sys check interval - if nosyscheck: - sys.setcheckinterval(sys.maxint) - print 'CHECKINTERVAL =', sys.maxint + gc.disable() + print '* disabled garbage collection' - print + # "Disable" sys check interval + if not withsyscheck: + # Too bad the check interval uses an int instead of a long... + value = 2147483647 + sys.setcheckinterval(value) + print '* system check interval set to maximum: %s' % value + + if timer == TIMER_SYSTIMES_PROCESSTIME: + import systimes + print '* using timer: systimes.processtime (%s)' % \ + systimes.SYSTIMES_IMPLEMENTATION + else: + print '* using timer: %s' % timer - if not compare_to: - print_machine() - print + print if compare_to: try: @@ -460,8 +865,10 @@ bench.name = compare_to f.close() compare_to = bench - except IOError: - print '* Error opening/reading file',compare_to + except IOError, reason: + print '* Error opening/reading file %s: %s' % ( + repr(compare_to), + reason) compare_to = None if show_bench: @@ -470,37 +877,52 @@ bench = pickle.load(f) bench.name = show_bench f.close() - print 'Benchmark: %s (rounds=%i, warp=%i)' % \ - (bench.name,bench.rounds,bench.warp) - print - bench.print_stat(compare_to, hidenoise) + bench.print_header() + if compare_to: + bench.print_comparison(compare_to, + hidenoise=hidenoise, + limitnames=limitnames) + else: + bench.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) except IOError: - print '* Error opening/reading file',show_bench + print '* Error opening/reading file %s: %s' % ( + repr(show_bench), + reason) print return if reportfile: - if nogc: - print 'Benchmark: %s (rounds=%i, warp=%i, no GC)' % \ - (reportfile,rounds,warp) - else: - print 'Benchmark: %s (rounds=%i, warp=%i)' % \ - (reportfile,rounds,warp) + print 'Creating benchmark: %s (rounds=%i, warp=%i)' % \ + (reportfile, rounds, warp) print # Create benchmark object - bench = Benchmark() + bench = Benchmark(reportfile, + verbose=verbose, + timer=timer, + warp=warp, + calibration_runs=calibration_runs) bench.rounds = rounds - bench.load_tests(Setup, warp, limitnames, verbose) + bench.load_tests(Setup, limitnames=limitnames) try: - bench.run(verbose, cruns) + bench.calibrate() + bench.run() except KeyboardInterrupt: print print '*** KeyboardInterrupt -- Aborting' print return - bench.print_stat(compare_to) - # ring bell + bench.print_header() + if compare_to: + bench.print_comparison(compare_to, + hidenoise=hidenoise, + limitnames=limitnames) + else: + bench.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) + + # Ring bell sys.stderr.write('\007') if reportfile: Modified: python/trunk/Tools/pybench/systimes.py ============================================================================== --- python/trunk/Tools/pybench/systimes.py (original) +++ python/trunk/Tools/pybench/systimes.py Tue Jun 13 20:56:56 2006 @@ -16,7 +16,7 @@ platforms. If no supported timing methods based on process time can be found, - the module reverts to the highest resolution wall-time timer + the module reverts to the highest resolution wall-clock timer instead. The system time part will then always be 0.0. The module exports one public API: @@ -52,8 +52,8 @@ USE_WIN32PROCESS_GETPROCESSTIMES = 'win32process.GetProcessTimes()' USE_RESOURCE_GETRUSAGE = 'resource.getrusage()' USE_PROCESS_TIME_CLOCK = 'time.clock() (process time)' -USE_WALL_TIME_CLOCK = 'time.clock() (wall-time)' -USE_WALL_TIME_TIME = 'time.time() (wall-time)' +USE_WALL_TIME_CLOCK = 'time.clock() (wall-clock)' +USE_WALL_TIME_TIME = 'time.time() (wall-clock)' if sys.platform[:3] == 'win': # Windows platform @@ -63,7 +63,7 @@ try: import ctypes except ImportError: - # Use the wall-time implementation time.clock(), since this + # Use the wall-clock implementation time.clock(), since this # is the highest resolution clock available on Windows SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_CLOCK else: @@ -91,7 +91,7 @@ # time) SYSTIMES_IMPLEMENTATION = USE_PROCESS_TIME_CLOCK else: - # Use wall-time implementation time.time() since this provides + # Use wall-clock implementation time.time() since this provides # the highest resolution clock on most systems SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_TIME @@ -103,24 +103,27 @@ def process_time_clock_systimes(): return (time.clock(), 0.0) -def wall_time_clock_systimes(): +def wall_clock_clock_systimes(): return (time.clock(), 0.0) -def wall_time_time_systimes(): +def wall_clock_time_systimes(): return (time.time(), 0.0) # Number of clock ticks per second for the values returned # by GetProcessTimes() on Windows. # -# Note: Ticks returned by GetProcessTimes() are micro-seconds on -# Windows XP (though the docs say 100ns intervals) -WIN32_PROCESS_TIMES_TICKS_PER_SECOND = 10e6 +# Note: Ticks returned by GetProcessTimes() are 100ns intervals on +# Windows XP. However, the process times are only updated with every +# clock tick and the frequency of these is somewhat lower: depending +# on the OS version between 10ms and 15ms. Even worse, the process +# time seems to be allocated to process currently running when the +# clock interrupt arrives, ie. it is possible that the current time +# slice gets accounted to a different process. + +WIN32_PROCESS_TIMES_TICKS_PER_SECOND = 1e7 def win32process_getprocesstimes_systimes(): d = win32process.GetProcessTimes(win32process.GetCurrentProcess()) - # Note: I'm not sure whether KernelTime on Windows is the same as - # system time on Unix - I've yet to see a non-zero value for - # KernelTime on Windows. return (d['UserTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, d['KernelTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) @@ -149,10 +152,10 @@ systimes = process_time_clock_systimes elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_CLOCK: - systimes = wall_time_clock_systimes + systimes = wall_clock_clock_systimes elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_TIME: - systimes = wall_time_time_systimes + systimes = wall_clock_time_systimes elif SYSTIMES_IMPLEMENTATION is USE_WIN32PROCESS_GETPROCESSTIMES: systimes = win32process_getprocesstimes_systimes @@ -163,6 +166,17 @@ else: raise TypeError('no suitable systimes() implementation found') +def processtime(): + + """ Return the total time spent on the process. + + This is the sum of user and system time as returned by + systimes(). + + """ + user, system = systimes() + return user + system + ### Testing def some_workload(): From python-checkins at python.org Tue Jun 13 21:02:45 2006 From: python-checkins at python.org (andrew.macintyre) Date: Tue, 13 Jun 2006 21:02:45 +0200 (CEST) Subject: [Python-checkins] r46929 - in python/trunk/Lib/test: output/test_thread test_thread.py test_threading.py Message-ID: <20060613190245.0A6C31E4005@bag.python.org> Author: andrew.macintyre Date: Tue Jun 13 21:02:35 2006 New Revision: 46929 Modified: python/trunk/Lib/test/output/test_thread python/trunk/Lib/test/test_thread.py python/trunk/Lib/test/test_threading.py Log: Increase the small thread stack size to get the test to pass reliably on the one buildbot that insists on more than 32kB of thread stack. Modified: python/trunk/Lib/test/output/test_thread ============================================================================== --- python/trunk/Lib/test/output/test_thread (original) +++ python/trunk/Lib/test/output/test_thread Tue Jun 13 21:02:35 2006 @@ -7,10 +7,10 @@ *** Changing thread stack size *** caught expected ValueError setting stack_size(4096) -successfully set stack_size(32768) +successfully set stack_size(262144) successfully set stack_size(1048576) successfully set stack_size(0) -trying stack_size = 32768 +trying stack_size = 262144 waiting for all tasks to complete all tasks done trying stack_size = 1048576 Modified: python/trunk/Lib/test/test_thread.py ============================================================================== --- python/trunk/Lib/test/test_thread.py (original) +++ python/trunk/Lib/test/test_thread.py Tue Jun 13 21:02:35 2006 @@ -140,13 +140,13 @@ if tss_supported: failed = lambda s, e: s != e fail_msg = "stack_size(%d) failed - should succeed" - for tss in (32768, 0x100000, 0): + for tss in (262144, 0x100000, 0): thread.stack_size(tss) if failed(thread.stack_size(), tss): raise ValueError, fail_msg % tss print 'successfully set stack_size(%d)' % tss - for tss in (32768, 0x100000): + for tss in (262144, 0x100000): print 'trying stack_size = %d' % tss next_ident = 0 for i in range(numtasks): Modified: python/trunk/Lib/test/test_threading.py ============================================================================== --- python/trunk/Lib/test/test_threading.py (original) +++ python/trunk/Lib/test/test_threading.py Tue Jun 13 21:02:35 2006 @@ -85,11 +85,11 @@ print 'all tasks done' self.assertEqual(numrunning.get(), 0) - # run with a minimum thread stack size (32kB) + # run with a small(ish) thread stack size (256kB) def test_various_ops_small_stack(self): if verbose: - print 'with 32kB thread stack size...' - threading.stack_size(0x8000) + print 'with 256kB thread stack size...' + threading.stack_size(262144) self.test_various_ops() threading.stack_size(0) From python-checkins at python.org Tue Jun 13 21:20:07 2006 From: python-checkins at python.org (marc-andre.lemburg) Date: Tue, 13 Jun 2006 21:20:07 +0200 (CEST) Subject: [Python-checkins] r46930 - python/trunk/Tools/pybench/Calls.py python/trunk/Tools/pybench/clockres.py python/trunk/Tools/pybench/pybench.py Message-ID: <20060613192007.B8BD91E4005@bag.python.org> Author: marc-andre.lemburg Date: Tue Jun 13 21:20:07 2006 New Revision: 46930 Modified: python/trunk/Tools/pybench/Calls.py python/trunk/Tools/pybench/clockres.py python/trunk/Tools/pybench/pybench.py Log: Whitespace normalization. Modified: python/trunk/Tools/pybench/Calls.py ============================================================================== --- python/trunk/Tools/pybench/Calls.py (original) +++ python/trunk/Tools/pybench/Calls.py Tue Jun 13 21:20:07 2006 @@ -500,5 +500,3 @@ timeit.main(['-s', setup, test]) - - Modified: python/trunk/Tools/pybench/clockres.py ============================================================================== --- python/trunk/Tools/pybench/clockres.py (original) +++ python/trunk/Tools/pybench/clockres.py Tue Jun 13 21:20:07 2006 @@ -20,7 +20,7 @@ while 1: now = wallclock() if now >= stop: - break + break for i in spin_loops: d[timer()] = 1 values = d.keys() @@ -33,7 +33,7 @@ return min_diff if __name__ == '__main__': - print 'Clock resolution of various timer implementations:' + print 'Clock resolution of various timer implementations:' print 'time.clock: %10.3fus' % (clockres(time.clock) * 1e6) print 'time.time: %10.3fus' % (clockres(time.time) * 1e6) try: @@ -41,4 +41,3 @@ print 'systimes.processtime: %10.3fus' % (clockres(systimes.processtime) * 1e6) except ImportError: pass - Modified: python/trunk/Tools/pybench/pybench.py ============================================================================== --- python/trunk/Tools/pybench/pybench.py (original) +++ python/trunk/Tools/pybench/pybench.py Tue Jun 13 21:20:07 2006 @@ -418,7 +418,7 @@ def __init__(self, name, verbose=None, timer=None, warp=None, calibration_runs=None): - + if name: self.name = name else: @@ -438,7 +438,7 @@ if _debug: print 'Getting machine details...' self.machine_details = get_machine_details() - + # Make .version an instance attribute to have it saved in the # Benchmark pickle self.version = self.version From tim.peters at gmail.com Tue Jun 13 21:20:37 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 13 Jun 2006 15:20:37 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (80) In-Reply-To: <2mwtblo1bg.fsf@starship.python.net> References: <20060613035818.GA15597@python.psfb.org> <2mwtblo1bg.fsf@starship.python.net> Message-ID: <1f7befae0606131220m4855823aq5af3bae31b1e2030@mail.gmail.com> >> test_builtin leaked [8, 8, 8] references >> test_exceptions leaked [40, 40, 40] references >> ... [Michael Hudson, about 9 hours ago] > So who broke refcounting then? This is pretty new and must be > somewhere fairly core, hopefully someone remembers changing something > there? I don't remember any specific checkins though. Huh. I don't see leaks now (I was going to dig into this), and don't recall any recent checkins that said they were aiming at fixing this either. Even the massive test_decimal leaked [10679, 9236, 9985] references has gone away. From nnorwitz at gmail.com Tue Jun 13 21:52:47 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 13 Jun 2006 12:52:47 -0700 Subject: [Python-checkins] Python Regression Test Failures refleak (80) In-Reply-To: References: <20060613035818.GA15597@python.psfb.org> <2mwtblo1bg.fsf@starship.python.net> Message-ID: On 6/13/06, Georg Brandl wrote: > Michael Hudson wrote: > > Neal Norwitz writes: > > > >> test_builtin leaked [8, 8, 8] references > >> test_exceptions leaked [40, 40, 40] references > > > > So who broke refcounting then? This is pretty new and must be > > somewhere fairly core, hopefully someone remembers changing something > > there? I don't remember any specific checkins though. > > FWIW, it would perhaps be a good idea for the refleak mail > to include something like sys.subversion. The data is available in the docs.python.org/dev/results directory. If you want to copy the results into the mail it shouldn't be too hard. It's all in Misc/build.sh. The problem in this case was I was doing some testing in the same dir and had outstanding changes that screwed up ref counting. There are always a few outstanding changes. I can think of 4 of the top of my head: build.sh - don't ignore any leaky test regrtest.py - ignore test_tcl test_socket and test_threadedtempfile - sleeps to avoid the ref count problems. this gives threads time to shutdown and seems to be working. the test_socket needs to be improved, i will probably check in the tempfile fix. it's kinda ugly. i had sent a patch a while ago. it's the same thing with the bug fixed (forgot to increment count). n From buildbot at python.org Tue Jun 13 22:06:52 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 20:06:52 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060613200652.A8F241E4005@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/756 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 22:18:44 2006 From: python-checkins at python.org (thomas.heller) Date: Tue, 13 Jun 2006 22:18:44 +0200 (CEST) Subject: [Python-checkins] r46931 - python/trunk/Doc/lib/libctypes.tex Message-ID: <20060613201844.5EDEC1E4005@bag.python.org> Author: thomas.heller Date: Tue Jun 13 22:18:43 2006 New Revision: 46931 Modified: python/trunk/Doc/lib/libctypes.tex Log: More docs for ctypes. Modified: python/trunk/Doc/lib/libctypes.tex ============================================================================== --- python/trunk/Doc/lib/libctypes.tex (original) +++ python/trunk/Doc/lib/libctypes.tex Tue Jun 13 22:18:43 2006 @@ -66,14 +66,7 @@ >>> \end{verbatim} - -XXX Add section for Mac OS X. - - -\subsubsection{Finding shared libraries\label{ctypes-finding-shared-libraries}} - -XXX Add description of ctypes.util.find{\_}library (once I really -understand it enough to describe it). +% XXX Add section for Mac OS X. \subsubsection{Accessing functions from loaded dlls\label{ctypes-accessing-functions-from-loaded-dlls}} @@ -1413,62 +1406,199 @@ \subsection{ctypes reference\label{ctypes-ctypes-reference}} -\subsubsection{Loading shared libraries\label{ctypes-loading-shared-libraries}} +\subsubsection{Finding shared libraries\label{ctypes-finding-shared-libraries}} -\begin{classdesc}{LibraryLoader}{dlltype} -Class which loads shared libraries. -\end{classdesc} +When programming in a compiled language, shared libraries are accessed +when compiling/linking a program, and when the program is run. -\begin{methoddesc}{LoadLibrary}{name, mode=RTLD_LOCAL, handle=None} -Load a shared library. -\end{methoddesc} +The purpose of the \code{find{\_}library} function is to locate a library in +a way similar to what the compiler does (on platforms with several +versions of a shared library the most recent should be loaded), while +the ctypes library loaders act like when a program is run, and call +the runtime loader directly. + +The \code{ctypes.util} module provides a function which can help to +determine the library to load. + +\begin{datadescni}{find_library(name)} +Try to find a library and return a pathname. \var{name} is the +library name without any prefix like \var{lib}, suffix like \code{.so}, +\code{.dylib} or version number (this is the form used for the posix +linker option \programopt{-l}). If no library can be found, returns +\code{None}. +\end{datadescni} + +The exact functionality is system dependend. + +On Linux, \code{find{\_}library} tries to run external programs +(/sbin/ldconfig, gcc, and objdump) to find the library file. It +returns the filename of the library file. Here are sone examples: +\begin{verbatim} +>>> from ctypes.util import find_library +>>> find_library("m") +'libm.so.6' +>>> find_library("c") +'libc.so.6' +>>> find_library("bz2") +'libbz2.so.1.0' +>>> +\end{verbatim} + +On OS X, \code{find{\_}library} tries several predefined naming schemes and +paths to locate the library, and returns a full pathname if successfull: +\begin{verbatim} +>>> from ctypes.util import find_library +>>> find_library("c") +'/usr/lib/libc.dylib' +>>> find_library("m") +'/usr/lib/libm.dylib' +>>> find_library("bz2") +'/usr/lib/libbz2.dylib' +>>> find_library("AGL") +'/System/Library/Frameworks/AGL.framework/AGL' +>>> +\end{verbatim} + +On Windows, \code{find{\_}library} searches along the system search path, +and returns the full pathname, but since there is no predefined naming +scheme a call like \code{find{\_}library("c")} will fail and return +\code{None}. + +If wrapping a shared library with \code{ctypes}, it \emph{may} be better to +determine the shared library name at development type, and hardcode +that into the wrapper module instead of using \code{find{\_}library} to +locate the library at runtime. + + +\subsubsection{Loading shared libraries\label{ctypes-loading-shared-libraries}} + +There are several ways to loaded shared libraries into the Python +process. One way is to instantiate one of the following classes: \begin{classdesc}{CDLL}{name, mode=RTLD_LOCAL, handle=None} -XXX +Instances of this class represent loaded shared libraries. +Functions in these libraries use the standard C calling +convention, and are assumed to return \code{int}. \end{classdesc} -\begin{datadescni}{cdll} -XXX -\end{datadescni} +\begin{classdesc}{OleDLL}{name, mode=RTLD_LOCAL, handle=None} +Windows only: Instances of this class represent loaded shared +libraries, functions in these libraries use the \code{stdcall} +calling convention, and are assumed to return the windows specific +\class{HRESULT} code. \class{HRESULT} values contain information +specifying whether the function call failed or succeeded, together +with additional error code. If the return value signals a +failure, an \class{WindowsError} is automatically raised. +\end{classdesc} -\begin{funcdesc}{OleDLL}{name, mode=RTLD_LOCAL, handle=None} -XXX -\end{funcdesc} +\begin{classdesc}{WinDLL}{name, mode=RTLD_LOCAL, handle=None} +Windows only: Instances of this class represent loaded shared +libraries, functions in these libraries use the \code{stdcall} +calling convention, and are assumed to return \code{int} by default. + +On Windows CE only the standard calling convention is used, for +convenience the \class{WinDLL} and \class{OleDLL} use the standard calling +convention on this platform. +\end{classdesc} -\begin{datadescni}{oledll} -XXX -\end{datadescni} +The Python GIL is released before calling any function exported by +these libraries, and reaquired afterwards. -\begin{classdesc*}{py_object} -XXX -\end{classdesc*} +\begin{classdesc}{PyDLL}{name, mode=RTLD_LOCAL, handle=None} +Instances of this class behave like \class{CDLL} instances, except +that the Python GIL is \emph{not} released during the function call, +and after the function execution the Python error flag is checked. +If the error flag is set, a Python exception is raised. -\begin{funcdesc}{PyDLL}{name, mode=RTLD_LOCAL, handle=None} -XXX -\end{funcdesc} +Thus, this is only useful to call Python C api functions directly. +\end{classdesc} -\begin{datadescni}{pydll} -XXX -\end{datadescni} +All these classes can be instantiated by calling them with at least +one argument, the pathname of the shared library. If you have an +existing handle to an already loaded shard library, it can be passed +as the \code{handle} named parameter, otherwise the underlying platforms +\code{dlopen} or \method{LoadLibrary} function is used to load the library +into the process, and to get a handle to it. + +The \var{mode} parameter can be used to specify how the library is +loaded. For details, consult the \code{dlopen(3)} manpage, on Windows, +\var{mode} is ignored. \begin{datadescni}{RTLD_GLOBAL} -XXX +Flag to use as \var{mode} parameter. On platforms where this flag +is not available, it is defined as the integer zero. \end{datadescni} \begin{datadescni}{RTLD_LOCAL} -XXX +Flag to use as \var{mode} parameter. On platforms where this is not +available, it is the same as \var{RTLD{\_}GLOBAL}. \end{datadescni} -\begin{funcdesc}{WinDLL}{name, mode=RTLD_LOCAL, handle=None} -XXX -\end{funcdesc} +Instances of these classes have no public methods, however +\method{{\_}{\_}getattr{\_}{\_}} and \method{{\_}{\_}getitem{\_}{\_}} have special behaviour: functions +exported by the shared library can be accessed as attributes of by +index. Please note that both \method{{\_}{\_}getattr{\_}{\_}} and \method{{\_}{\_}getitem{\_}{\_}} +cache their result, so calling them repeatedly returns the same object +each time. + +The following public attributes are available, their name starts with +an underscore to not clash with exported function names: + +\begin{datadescni}{_handle: memberdesc} +The system handle used to access the library. +\end{datadescni} + +\begin{datadescni}{_name: memberdesc} +The name of the library passed in the contructor. +\end{datadescni} + +Shared libraries can also be loaded by using one of the prefabricated +objects, which are instances of the \class{LibraryLoader} class, either by +calling the \method{LoadLibrary} method, or by retrieving the library as +attribute of the loader instance. + +\begin{classdesc}{LibraryLoader}{dlltype} +Class which loads shared libraries. \code{dlltype} should be one +of the \class{CDLL}, \class{PyDLL}, \class{WinDLL}, or \class{OleDLL} types. + +\method{{\_}{\_}getattr{\_}{\_}} has special behaviour: It allows to load a shared +library by accessing it as attribute of a library loader +instance. The result is cached, so repeated attribute accesses +return the same library each time. +\end{classdesc} + +\begin{methoddesc}{LoadLibrary}{name, mode=RTLD_LOCAL, handle=None} +Load a shared library into the process and return it. This method +always creates a new instance of the library. All three +parameters are passed to the constructor of the library object. +\end{methoddesc} + +These prefabricated library loaders are available: + +\begin{datadescni}{cdll} +Loads \class{CDLL} instances. +\end{datadescni} \begin{datadescni}{windll} -XXX +Windows only: Loads \class{WinDLL} instances. \end{datadescni} -\begin{datadescni}{pythonapi()} -XXX +\begin{datadescni}{oledll} +Windows only: Loads \class{OleDLL} instances. +\end{datadescni} + +\begin{datadescni}{pydll} +Loads \class{PyDLL} instances. +\end{datadescni} + +For accessing the C Python api directly, a ready-to-use Python shared +library object is available: + +\begin{datadescni}{pythonapi} +An instance of \class{PyDLL} that exposes Python C api functions as +attributes. Note that all these functions are assumed to return +integers, which is of course not always the truth, so you have to +assign the correct \member{restype} attribute. \end{datadescni} @@ -1647,7 +1777,7 @@ on encoding/decoding errors. Examples of possible values are \code{"strict"}, \code{"replace"}, or \code{"ignore"}. -set{\_}conversion{\_}mode returns a 2-tuple containing the previous +\code{set{\_}conversion{\_}mode} returns a 2-tuple containing the previous conversion rules. On windows, the initial conversion rules are \code{('mbcs', 'ignore')}, on other systems \code{('ascii', 'strict')}. \end{funcdesc} @@ -1917,6 +2047,10 @@ or error information for a function or method call. \end{classdesc*} +\begin{classdesc*}{py_object} +Represents the C \code{PyObject *} datatype. +\end{classdesc*} + \subsubsection{Structured data types\label{ctypes-structured-data-types}} From buildbot at python.org Tue Jun 13 22:43:50 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 20:43:50 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060613204350.D75BC1E4005@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/657 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre,marc-andre.lemburg,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 23:34:25 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 13 Jun 2006 23:34:25 +0200 (CEST) Subject: [Python-checkins] r46932 - in python/trunk/Tools/pybench: package Message-ID: <20060613213425.835041E4004@bag.python.org> Author: brett.cannon Date: Tue Jun 13 23:34:24 2006 New Revision: 46932 Modified: python/trunk/Tools/pybench/ (props changed) python/trunk/Tools/pybench/package/ (props changed) Log: Ignore .pyc and .pyo files in Pybench. From buildbot at python.org Tue Jun 13 23:45:54 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 21:45:54 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20060613214554.8B4A71E4022@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/176 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.macintyre,marc-andre.lemburg,tim.peters BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From python-checkins at python.org Tue Jun 13 23:46:44 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 13 Jun 2006 23:46:44 +0200 (CEST) Subject: [Python-checkins] r46933 - in python/trunk: Lib/test/crashers/coerce.py Misc/NEWS Objects/classobject.c Message-ID: <20060613214644.A98161E401A@bag.python.org> Author: brett.cannon Date: Tue Jun 13 23:46:41 2006 New Revision: 46933 Removed: python/trunk/Lib/test/crashers/coerce.py Modified: python/trunk/Misc/NEWS python/trunk/Objects/classobject.c Log: If a classic class defined a __coerce__() method that just returned its two arguments in reverse, the interpreter would infinitely recourse trying to get a coercion that worked. So put in a recursion check after a coercion is made and the next call to attempt to use the coerced values. Fixes bug #992017 and closes crashers/coerce.py . Deleted: /python/trunk/Lib/test/crashers/coerce.py ============================================================================== --- /python/trunk/Lib/test/crashers/coerce.py Tue Jun 13 23:46:41 2006 +++ (empty file) @@ -1,9 +0,0 @@ - -# http://python.org/sf/992017 - -class foo: - def __coerce__(self, other): - return other, self - -if __name__ == '__main__': - foo()+1 # segfault: infinite recursion in C Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Tue Jun 13 23:46:41 2006 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Bug #992017: A classic class that defined a __coerce__() method that returned + its arguments swapped would infinitely recurse and segfault the interpreter. + - Fix the socket tests so they can be run concurrently. - Removed 5 integers from C frame objects (PyFrameObject). Modified: python/trunk/Objects/classobject.c ============================================================================== --- python/trunk/Objects/classobject.c (original) +++ python/trunk/Objects/classobject.c Tue Jun 13 23:46:41 2006 @@ -1368,10 +1368,13 @@ * argument */ result = generic_binary_op(v1, w, opname); } else { + if (Py_EnterRecursiveCall(" after coercion")) + return NULL; if (swapped) result = (thisfunc)(w, v1); else result = (thisfunc)(v1, w); + Py_LeaveRecursiveCall(); } Py_DECREF(coerced); return result; From python-checkins at python.org Tue Jun 13 23:50:29 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 13 Jun 2006 23:50:29 +0200 (CEST) Subject: [Python-checkins] r46934 - in python/branches/release24-maint: Misc/NEWS Objects/classobject.c Message-ID: <20060613215029.261681E4005@bag.python.org> Author: brett.cannon Date: Tue Jun 13 23:50:24 2006 New Revision: 46934 Modified: python/branches/release24-maint/Misc/NEWS python/branches/release24-maint/Objects/classobject.c Log: Classic class that defined ``def __coerce__(self, other): return other, self`` would infinitely recourse and segfault the interpreter. Now a recursion check occurs after a coercion. Backport of fix for bug #992017. Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Tue Jun 13 23:50:24 2006 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Bug #992017: A classic class that defined a __coerce__() method that returned + its arguments swapped would infinitely recurse and segfault the interpreter. + - Bug #532646: The object set to the __call__ attribute has its own __call__ attribute checked; this continues until the attribute can no longer be found or segfaulting. Recursion limit is now followed. Modified: python/branches/release24-maint/Objects/classobject.c ============================================================================== --- python/branches/release24-maint/Objects/classobject.c (original) +++ python/branches/release24-maint/Objects/classobject.c Tue Jun 13 23:50:24 2006 @@ -1428,10 +1428,13 @@ * argument */ result = generic_binary_op(v1, w, opname); } else { + if (Py_EnterRecursiveCall(" after coercion")) + return NULL; if (swapped) result = (thisfunc)(w, v1); else result = (thisfunc)(v1, w); + Py_LeaveRecursiveCall(); } Py_DECREF(coerced); return result; From g.brandl at gmx.net Tue Jun 13 23:52:54 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 13 Jun 2006 23:52:54 +0200 Subject: [Python-checkins] r46933 - in python/trunk: Lib/test/crashers/coerce.py Misc/NEWS Objects/classobject.c In-Reply-To: <20060613214644.A98161E401A@bag.python.org> References: <20060613214644.A98161E401A@bag.python.org> Message-ID: brett.cannon wrote: > Author: brett.cannon > Date: Tue Jun 13 23:46:41 2006 > New Revision: 46933 > > Removed: > python/trunk/Lib/test/crashers/coerce.py > Modified: > python/trunk/Misc/NEWS > python/trunk/Objects/classobject.c > Log: > If a classic class defined a __coerce__() method that just returned its two > arguments in reverse, the interpreter would infinitely recourse trying to get a > coercion that worked. So put in a recursion check after a coercion is made and > the next call to attempt to use the coerced values. > > Fixes bug #992017 and closes crashers/coerce.py . Shouldn't the previously crashing test be added to some appropriate test file? Georg From buildbot at python.org Tue Jun 13 23:55:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 21:55:37 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060613215537.ED12A1E400F@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/758 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 13 23:55:40 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 21:55:40 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060613215540.6617E1E400C@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/355 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: marc-andre.lemburg,thomas.heller Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From brett at python.org Wed Jun 14 00:19:20 2006 From: brett at python.org (Brett Cannon) Date: Tue, 13 Jun 2006 15:19:20 -0700 Subject: [Python-checkins] r46933 - in python/trunk: Lib/test/crashers/coerce.py Misc/NEWS Objects/classobject.c In-Reply-To: References: <20060613214644.A98161E401A@bag.python.org> Message-ID: Yep. I will do that now. -Brett On 6/13/06, Georg Brandl wrote: > > brett.cannon wrote: > > Author: brett.cannon > > Date: Tue Jun 13 23:46:41 2006 > > New Revision: 46933 > > > > Removed: > > python/trunk/Lib/test/crashers/coerce.py > > Modified: > > python/trunk/Misc/NEWS > > python/trunk/Objects/classobject.c > > Log: > > If a classic class defined a __coerce__() method that just returned its > two > > arguments in reverse, the interpreter would infinitely recourse trying > to get a > > coercion that worked. So put in a recursion check after a coercion is > made and > > the next call to attempt to use the coerced values. > > > > Fixes bug #992017 and closes crashers/coerce.py . > > Shouldn't the previously crashing test be added to some appropriate test > file? > > Georg > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060613/5102cc97/attachment.htm From python-checkins at python.org Wed Jun 14 00:22:11 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Wed, 14 Jun 2006 00:22:11 +0200 (CEST) Subject: [Python-checkins] r46935 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060613222211.39C1E1E4005@bag.python.org> Author: mateusz.rukowicz Date: Wed Jun 14 00:22:03 2006 New Revision: 46935 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Fixed one sigsegv and one refleak, there is no refleaks now. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Wed Jun 14 00:22:03 2006 @@ -1635,6 +1635,7 @@ long i, minsize; decimalobject *longer, *ans; contextobject *ctx2; + PyObject *flags; if (ISSPECIAL(self) || ISSPECIAL(other)) { decimalobject *nan; @@ -1676,13 +1677,15 @@ } for (i = 0; i < minsize; i++) { - if (self->digits[i] != other->digits[i]) + if (_limb_get_digit(self->limbs, self->ob_size, i) != /* SLOW */ + _limb_get_digit(other->limbs, other->ob_size, i)) goto next; } /* Okay, now the remaining digits of the longer one must consist of only digits. */ for (i = minsize; i < longer->ob_size; i++) - if (longer->digits[i] != 0) + /* SLOW */ + if (_limb_get_digit(longer->limbs, longer->ob_size, i) != 0) goto next; /* All digits match, so they're equal. */ return 0; @@ -1699,12 +1702,13 @@ if (!ctx2) return 0; /* error */ ctx2->rounding = ROUND_UP; /* away from 0 */ - if (context_ignore_all_flags(ctx2) == NULL) + if ((flags = context_ignore_all_flags(ctx2)) == NULL) return 0; /* error */ - Py_XDECREF(ctx2); ans = _do_decimal_subtract(self, other, ctx2); + Py_DECREF(ctx2); + Py_DECREF(flags); if (!ans) return 0; if (!decimal_nonzero(ans)) @@ -3421,7 +3425,8 @@ static PyObject * decimal_power(PyObject *self, PyObject *other, PyObject *modulo) { -/* decimalobject *res; +#if 0 + decimalobject *res; contextobject *ctx = getcontext(); if (!ctx) return NULL; @@ -3443,8 +3448,10 @@ ctx); Py_DECREF(other); Py_DECREF(modulo); - return (PyObject *)res;*/ + return (PyObject *)res; +#else Py_RETURN_NONE; +#endif } @@ -4828,7 +4835,8 @@ static PyObject * context_power(contextobject *self, PyObject *args) { -/* PyObject *a, *b, *c; +#if 0 + PyObject *a, *b, *c; decimalobject *dec_a = NULL, *dec_b = NULL, *dec_c = NULL, *res; if (!PyArg_ParseTuple(args, "OO|O:power", &a, &b, &c)) return NULL; @@ -4855,8 +4863,10 @@ Py_DECREF(dec_a); Py_DECREF(dec_b); Py_DECREF(dec_c); - return (PyObject *)res;*/ + return (PyObject *)res; +#else Py_RETURN_NONE; +#endif } From python-checkins at python.org Wed Jun 14 00:26:17 2006 From: python-checkins at python.org (brett.cannon) Date: Wed, 14 Jun 2006 00:26:17 +0200 (CEST) Subject: [Python-checkins] r46937 - python/trunk/Lib/test/test_coercion.py Message-ID: <20060613222617.306701E4004@bag.python.org> Author: brett.cannon Date: Wed Jun 14 00:26:13 2006 New Revision: 46937 Modified: python/trunk/Lib/test/test_coercion.py Log: Missed test for rev. 46933; infinite recursion from __coerce__() returning its arguments reversed. Modified: python/trunk/Lib/test/test_coercion.py ============================================================================== --- python/trunk/Lib/test/test_coercion.py (original) +++ python/trunk/Lib/test/test_coercion.py Wed Jun 14 00:26:13 2006 @@ -2,7 +2,7 @@ import sys import warnings import unittest -from test.test_support import run_unittest +from test.test_support import run_unittest, TestFailed # Fake a number that implements numeric methods through __coerce__ class CoerceNumber: @@ -318,6 +318,24 @@ return 0 self.assertEquals(cmp(ClassicWackyComparer(), evil_coercer), 0) + def test_infinite_rec_classic_classes(self): + # if __coerce__() returns its arguments reversed it causes an infinite + # recursion for classic classes. + class Tester: + def __coerce__(self, other): + return other, self + + exc = TestFailed("__coerce__() returning its arguments reverse " + "should raise RuntimeError") + try: + Tester() + 1 + except (RuntimeError, TypeError): + return + except: + raise exc + else: + raise exc + def test_main(): warnings.filterwarnings("ignore", r'complex divmod, // and % are deprecated', From buildbot at python.org Wed Jun 14 00:48:31 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 22:48:31 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060613224831.D70791E4004@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1154 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon,gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 00:49:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 22:49:36 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060613224936.F2F771E4004@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1058 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon,gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 14 00:53:50 2006 From: python-checkins at python.org (gerhard.haering) Date: Wed, 14 Jun 2006 00:53:50 +0200 (CEST) Subject: [Python-checkins] r46938 - python/trunk/Doc/lib/libsqlite3.tex Message-ID: <20060613225350.5B85B1E4004@bag.python.org> Author: gerhard.haering Date: Wed Jun 14 00:53:48 2006 New Revision: 46938 Modified: python/trunk/Doc/lib/libsqlite3.tex Log: Updated documentation for pysqlite 2.3.0 API. Modified: python/trunk/Doc/lib/libsqlite3.tex ============================================================================== --- python/trunk/Doc/lib/libsqlite3.tex (original) +++ python/trunk/Doc/lib/libsqlite3.tex Wed Jun 14 00:53:48 2006 @@ -195,6 +195,14 @@ \verbatiminput{sqlite3/complete_statement.py} \end{funcdesc} +\begin{funcdesc}{}enable_callback_tracebacks{flag} +By default you will not get any tracebacks in user-defined functions, +aggregates, converters, authorizer callbacks etc. If you want to debug them, +you can call this function with \var{flag} as True. Afterwards, you will get +tracebacks from callbacks on \code{sys.stderr}. Use \constant{False} to disable +the feature again. +\end{funcdesc} + \subsection{Connection Objects \label{sqlite3-Connection-Objects}} A \class{Connection} instance has the following attributes and methods: @@ -237,8 +245,7 @@ called as SQL function. The function can return any of the types supported by SQLite: unicode, str, -int, long, float, buffer and None. Exceptions in the function are ignored and -they are handled as if the function returned None. +int, long, float, buffer and None. Example: @@ -254,7 +261,7 @@ will return the final result of the aggregate. The \code{finalize} method can return any of the types supported by SQLite: -unicode, str, int, long, float, buffer and None. Any exceptions are ignored. +unicode, str, int, long, float, buffer and None. Example: @@ -283,6 +290,34 @@ \end{verbatim} \end{methoddesc} +\begin{methoddesc}{interrupt}{} + +You can call this method from a different thread to abort any queries that +might be executing on the connection. The query will then abort and the caller +will get an exception. +\end{methoddesc} + +\begin{methoddesc}{set_authorizer}{authorizer_callback} + +This routine registers a callback. The callback is invoked for each attempt to +access a column of a table in the database. The callback should return +\constant{SQLITE_OK} if access is allowed, \constant{SQLITE_DENY} if the entire +SQL statement should be aborted with an error and \constant{SQLITE_IGNORE} if +the column should be treated as a NULL value. These constants are available in +the \module{sqlite3} module. + +The first argument to the callback signifies what kind of operation is to be +authorized. The second and third argument will be arguments or \constant{None} +depending on the first argument. The 4th argument is the name of the database +("main", "temp", etc.) if applicable. The 5th argument is the name of the +inner-most trigger or view that is responsible for the access attempt or +\constant{None} if this access attempt is directly from input SQL code. + +Please consult the SQLite documentation about the possible values for the first +argument and the meaning of the second and third argument depending on the +first one. All necessary constants are available in the \module{sqlite3} +module. +\end{methoddesc} \begin{memberdesc}{row_factory} You can change this attribute to a callable that accepts the cursor and From buildbot at python.org Wed Jun 14 01:03:58 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 23:03:58 +0000 Subject: [Python-checkins] buildbot warnings in sparc solaris10 gcc 2.4 Message-ID: <20060613230359.78B8E1E4004@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%25202.4/builds/167 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 01:10:56 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 23:10:56 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060613231057.180FF1E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/59 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 01:24:45 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 23:24:45 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060613232445.50EAA1E4004@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/659 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 01:37:16 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 23:37:16 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060613233716.EC7D81E400C@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/177 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 01:47:13 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 23:47:13 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060613234713.5BE851E4004@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/1008 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon,gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 01:59:00 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 13 Jun 2006 23:59:00 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060613235900.AD98E1E4004@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/710 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon,gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 02:48:41 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 00:48:41 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060614004841.90B2A1E4004@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/639 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon,gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From tim.peters at gmail.com Wed Jun 14 03:37:26 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 13 Jun 2006 21:37:26 -0400 Subject: [Python-checkins] r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sqlite/module.c Modules/_sqlite/module.h In-Reply-To: <20060613222453.4561E1E4004@bag.python.org> References: <20060613222453.4561E1E4004@bag.python.org> Message-ID: <1f7befae0606131837x2a4c71bdo588544a759968542@mail.gmail.com> > Author: gerhard.haering > Date: Wed Jun 14 00:24:47 2006 > New Revision: 46936 > > Modified: > python/trunk/Lib/sqlite3/test/regression.py > python/trunk/Lib/sqlite3/test/types.py > python/trunk/Lib/sqlite3/test/userfunctions.py > python/trunk/Modules/_sqlite/connection.c > python/trunk/Modules/_sqlite/cursor.c > python/trunk/Modules/_sqlite/module.c > python/trunk/Modules/_sqlite/module.h > Log: > Merged changes from external pysqlite 2.3.0 release. Documentation updates will > follow in a few hours at the latest. Then we should be ready for beta1. Please note that many (all?) non-Windows buildbots are failing test_sqlite now, and this one is even pumping out complaints from the system malloc()/free(): http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/1009/step-test/0 That's not really what we want to see in beta1 :-) From python-checkins at python.org Wed Jun 14 06:09:27 2006 From: python-checkins at python.org (tim.peters) Date: Wed, 14 Jun 2006 06:09:27 +0200 (CEST) Subject: [Python-checkins] r46939 - in python/trunk: Doc/lib/libdifflib.tex Lib/difflib.py Misc/NEWS Message-ID: <20060614040927.5D3281E4004@bag.python.org> Author: tim.peters Date: Wed Jun 14 06:09:25 2006 New Revision: 46939 Modified: python/trunk/Doc/lib/libdifflib.tex python/trunk/Lib/difflib.py python/trunk/Misc/NEWS Log: SequenceMatcher.get_matching_blocks(): This now guarantees that adjacent triples in the result list describe non-adjacent matching blocks. That's _nice_ to have, and Guido said he wanted it. Not a bugfix candidate: Guido or not ;-), this changes visible endcase semantics (note that some tests had to change), and nothing about this was documented before. Since it was working as designed, and behavior was consistent with the docs, it wasn't "a bug". Modified: python/trunk/Doc/lib/libdifflib.tex ============================================================================== --- python/trunk/Doc/lib/libdifflib.tex (original) +++ python/trunk/Doc/lib/libdifflib.tex Wed Jun 14 06:09:25 2006 @@ -419,6 +419,16 @@ len(\var{b}), 0)}. It is the only triple with \code{\var{n} == 0}. % Explain why a dummy is used! + If + \code{(\var{i}, \var{j}, \var{n})} and + \code{(\var{i'}, \var{j'}, \var{n'})} are adjacent triples in the list, + and the second is not the last triple in the list, then + \code{\var{i}+\var{n} != \var{i'}} or + \code{\var{j}+\var{n} != \var{j'}}; in other words, adjacent triples + always describe non-adjacent equal blocks. + \versionchanged[The guarantee that adjacent triples always describe + non-adjacent blocks was implemented]{2.5} + \begin{verbatim} >>> s = SequenceMatcher(None, "abxcd", "abcd") >>> s.get_matching_blocks() Modified: python/trunk/Lib/difflib.py ============================================================================== --- python/trunk/Lib/difflib.py (original) +++ python/trunk/Lib/difflib.py Wed Jun 14 06:09:25 2006 @@ -86,8 +86,7 @@ >>> for block in s.get_matching_blocks(): ... print "a[%d] and b[%d] match for %d elements" % block a[0] and b[0] match for 8 elements - a[8] and b[17] match for 6 elements - a[14] and b[23] match for 15 elements + a[8] and b[17] match for 21 elements a[29] and b[38] match for 0 elements Note that the last tuple returned by .get_matching_blocks() is always a @@ -101,8 +100,7 @@ ... print "%6s a[%d:%d] b[%d:%d]" % opcode equal a[0:8] b[0:8] insert a[8:8] b[8:17] - equal a[8:14] b[17:23] - equal a[14:29] b[23:38] + equal a[8:29] b[17:38] See the Differ class for a fancy human-friendly file differencer, which uses SequenceMatcher both to compare sequences of lines, and to compare @@ -461,7 +459,11 @@ Each triple is of the form (i, j, n), and means that a[i:i+n] == b[j:j+n]. The triples are monotonically increasing in - i and in j. + i and in j. New in Python 2.5, it's also guaranteed that if + (i, j, n) and (i', j', n') are adjacent triples in the list, and + the second is not the last triple in the list, then i+n != i' or + j+n != j'. IOW, adjacent triples never describe adjacent equal + blocks. The last triple is a dummy, (len(a), len(b), 0), and is the only triple with n==0. @@ -474,7 +476,6 @@ if self.matching_blocks is not None: return self.matching_blocks la, lb = len(self.a), len(self.b) - self.matching_blocks = matching_blocks = [] # This is most naturally expressed as a recursive algorithm, but # at least one user bumped into extreme use cases that exceeded @@ -483,6 +484,7 @@ # results to `matching_blocks` in a loop; the matches are sorted # at the end. queue = [(0, la, 0, lb)] + matching_blocks = [] while queue: alo, ahi, blo, bhi = queue.pop() i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi) @@ -496,8 +498,32 @@ if i+k < ahi and j+k < bhi: queue.append((i+k, ahi, j+k, bhi)) matching_blocks.sort() - matching_blocks.append( (la, lb, 0) ) - return matching_blocks + + # It's possible that we have adjacent equal blocks in the + # matching_blocks list now. Starting with 2.5, this code was added + # to collapse them. + i1 = j1 = k1 = 0 + non_adjacent = [] + for i2, j2, k2 in matching_blocks: + # Is this block adjacent to i1, j1, k1? + if i1 + k1 == i2 and j1 + k1 == j2: + # Yes, so collapse them -- this just increases the length of + # the first block by the length of the second, and the first + # block so lengthebd remains the block to compare against. + k1 += k2 + else: + # Not adjacent. Remember the first block (k1==0 means it's + # the dummy we started with), and make the second block the + # new block to compare against. + if k1: + non_adjacent.append((i1, j1, k1)) + i1, j1, k1 = i2, j2, k2 + if k1: + non_adjacent.append((i1, j1, k1)) + + non_adjacent.append( (la, lb, 0) ) + self.matching_blocks = non_adjacent + return self.matching_blocks def get_opcodes(self): """Return list of 5-tuples describing how to turn a into b. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 14 06:09:25 2006 @@ -156,6 +156,12 @@ Library ------- +- ``difflib``'s ``SequenceMatcher.get_matching_blocks()`` was changed to + guarantee that adjacent triples in the return list always describe + non-adjacent blocks. Previously, a pair of matching blocks could end + up being described by multiple adjacent triples that formed a partition + of the matching pair. + - Bug #1498146: fix optparse to handle Unicode strings in option help, description, and epilog. From python-checkins at python.org Wed Jun 14 06:13:03 2006 From: python-checkins at python.org (tim.peters) Date: Wed, 14 Jun 2006 06:13:03 +0200 (CEST) Subject: [Python-checkins] r46940 - python/trunk/Lib/difflib.py Message-ID: <20060614041303.462AB1E4004@bag.python.org> Author: tim.peters Date: Wed Jun 14 06:13:00 2006 New Revision: 46940 Modified: python/trunk/Lib/difflib.py Log: Repaired typo in new comment. Modified: python/trunk/Lib/difflib.py ============================================================================== --- python/trunk/Lib/difflib.py (original) +++ python/trunk/Lib/difflib.py Wed Jun 14 06:13:00 2006 @@ -509,7 +509,7 @@ if i1 + k1 == i2 and j1 + k1 == j2: # Yes, so collapse them -- this just increases the length of # the first block by the length of the second, and the first - # block so lengthebd remains the block to compare against. + # block so lengthened remains the block to compare against. k1 += k2 else: # Not adjacent. Remember the first block (k1==0 means it's From python-checkins at python.org Wed Jun 14 06:15:28 2006 From: python-checkins at python.org (tim.peters) Date: Wed, 14 Jun 2006 06:15:28 +0200 (CEST) Subject: [Python-checkins] r46941 - python/trunk/Lib/sqlite3/test/userfunctions.py Message-ID: <20060614041528.535601E4004@bag.python.org> Author: tim.peters Date: Wed Jun 14 06:15:27 2006 New Revision: 46941 Modified: python/trunk/Lib/sqlite3/test/userfunctions.py Log: Whitespace normalization. Modified: python/trunk/Lib/sqlite3/test/userfunctions.py ============================================================================== --- python/trunk/Lib/sqlite3/test/userfunctions.py (original) +++ python/trunk/Lib/sqlite3/test/userfunctions.py Wed Jun 14 06:15:27 2006 @@ -403,7 +403,7 @@ def suite(): function_suite = unittest.makeSuite(FunctionTests, "Check") aggregate_suite = unittest.makeSuite(AggregateTests, "Check") - authorizer_suite = unittest.makeSuite(AuthorizerTests, "Check") + authorizer_suite = unittest.makeSuite(AuthorizerTests, "Check") return unittest.TestSuite((function_suite, aggregate_suite, authorizer_suite)) def test(): From buildbot at python.org Wed Jun 14 06:18:40 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 04:18:40 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060614041840.6CD361E4004@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/406 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon,gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 14 06:25:04 2006 From: python-checkins at python.org (fred.drake) Date: Wed, 14 Jun 2006 06:25:04 +0200 (CEST) Subject: [Python-checkins] r46942 - python/trunk/Lib/test/test_sgmllib.py Message-ID: <20060614042504.B51B31E4004@bag.python.org> Author: fred.drake Date: Wed Jun 14 06:25:02 2006 New Revision: 46942 Modified: python/trunk/Lib/test/test_sgmllib.py Log: - make some disabled tests run what they intend when enabled - remove some over-zealous triple-quoting Modified: python/trunk/Lib/test/test_sgmllib.py ============================================================================== --- python/trunk/Lib/test/test_sgmllib.py (original) +++ python/trunk/Lib/test/test_sgmllib.py Wed Jun 14 06:25:02 2006 @@ -301,8 +301,8 @@ # that needs to be carefully considered before changing it. def _test_starttag_end_boundary(self): - self.check_events("""""", [("starttag", "a", [("b", "<")])]) - self.check_events("""""", [("starttag", "a", [("b", ">")])]) + self.check_events("", [("starttag", "a", [("b", "<")])]) + self.check_events("", [("starttag", "a", [("b", ">")])]) def _test_buffer_artefacts(self): output = [("starttag", "a", [("b", "<")])] @@ -322,17 +322,17 @@ self.check_events([""], output) output = [("comment", "abc")] - self._run_check(["", ""], output) - self._run_check(["<", "!--abc-->"], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check(["", ""], output) + self.check_events(["", ""], output) + self.check_events(["<", "!--abc-->"], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events(["", ""], output) def _test_starttag_junk_chars(self): self.check_parse_error("<") From python-checkins at python.org Wed Jun 14 07:04:48 2006 From: python-checkins at python.org (fred.drake) Date: Wed, 14 Jun 2006 07:04:48 +0200 (CEST) Subject: [Python-checkins] r46943 - python/trunk/Lib/test/test_sgmllib.py Message-ID: <20060614050448.52F911E4004@bag.python.org> Author: fred.drake Date: Wed Jun 14 07:04:47 2006 New Revision: 46943 Modified: python/trunk/Lib/test/test_sgmllib.py Log: add tests for two cases that are handled correctly in the current code, but that SF patch 1504676 as written mis-handles Modified: python/trunk/Lib/test/test_sgmllib.py ============================================================================== --- python/trunk/Lib/test/test_sgmllib.py (original) +++ python/trunk/Lib/test/test_sgmllib.py Wed Jun 14 07:04:47 2006 @@ -218,7 +218,9 @@ """Substitution of entities and charrefs in attribute values""" # SF bug #1452246 self.check_events("""""", + f="&xxx;" g=' !' h='Ǵ' + i='x?a=b&c=d;' + j='*' k='*'>""", [("starttag", "a", [("b", "<"), ("c", "<>"), ("d", "<->"), @@ -226,7 +228,10 @@ ("f", "&xxx;"), ("g", " !"), ("h", "Ǵ"), - ("i", "x?a=b&c=d;"), ])]) + ("i", "x?a=b&c=d;"), + ("j", "*"), + ("k", "*"), + ])]) def test_attr_funky_names(self): self.check_events("""""", [ From python-checkins at python.org Wed Jun 14 07:15:51 2006 From: python-checkins at python.org (fred.drake) Date: Wed, 14 Jun 2006 07:15:51 +0200 (CEST) Subject: [Python-checkins] r46944 - python/trunk/Lib/sgmllib.py Message-ID: <20060614051551.746F91E4004@bag.python.org> Author: fred.drake Date: Wed Jun 14 07:15:51 2006 New Revision: 46944 Modified: python/trunk/Lib/sgmllib.py Log: explain an XXX in more detail Modified: python/trunk/Lib/sgmllib.py ============================================================================== --- python/trunk/Lib/sgmllib.py (original) +++ python/trunk/Lib/sgmllib.py Wed Jun 14 07:15:51 2006 @@ -246,6 +246,9 @@ self.__starttag_text = rawdata[start_pos:match.end(1) + 1] return k # XXX The following should skip matching quotes (' or ") + # As a shortcut way to exit, this isn't so bad, but shouldn't + # be used to locate the actual end of the start tag since the + # < or > characters may be embedded in an attribute value. match = endbracket.search(rawdata, i+1) if not match: return -1 From python-checkins at python.org Wed Jun 14 07:21:06 2006 From: python-checkins at python.org (martin.v.loewis) Date: Wed, 14 Jun 2006 07:21:06 +0200 (CEST) Subject: [Python-checkins] r46945 - in python/trunk: Doc/api/concrete.tex Include/unicodeobject.h Lib/encodings/mbcs.py Misc/NEWS Modules/_codecsmodule.c Objects/unicodeobject.c Message-ID: <20060614052106.D38171E4018@bag.python.org> Author: martin.v.loewis Date: Wed Jun 14 07:21:04 2006 New Revision: 46945 Modified: python/trunk/Doc/api/concrete.tex python/trunk/Include/unicodeobject.h python/trunk/Lib/encodings/mbcs.py python/trunk/Misc/NEWS python/trunk/Modules/_codecsmodule.c python/trunk/Objects/unicodeobject.c Log: Patch #1455898: Incremental mode for "mbcs" codec. Modified: python/trunk/Doc/api/concrete.tex ============================================================================== --- python/trunk/Doc/api/concrete.tex (original) +++ python/trunk/Doc/api/concrete.tex Wed Jun 14 07:21:04 2006 @@ -1431,6 +1431,18 @@ raised by the codec. \end{cfuncdesc} +\begin{cfuncdesc}{PyObject*}{PyUnicode_DecodeMBCSStateful}{const char *s, + int size, + const char *errors, + int *consumed} + If \var{consumed} is \NULL{}, behave like + \cfunction{PyUnicode_DecodeMBCS()}. If \var{consumed} is not \NULL{}, + \cfunction{PyUnicode_DecodeMBCSStateful()} will not decode trailing lead + byte and the number of bytes that have been decoded will be stored in + \var{consumed}. + \versionadded{2.5} +\end{cfuncdesc} + \begin{cfuncdesc}{PyObject*}{PyUnicode_EncodeMBCS}{const Py_UNICODE *s, Py_ssize_t size, const char *errors} Modified: python/trunk/Include/unicodeobject.h ============================================================================== --- python/trunk/Include/unicodeobject.h (original) +++ python/trunk/Include/unicodeobject.h Wed Jun 14 07:21:04 2006 @@ -938,6 +938,13 @@ const char *errors /* error handling */ ); +PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCSStateful( + const char *string, /* MBCS encoded string */ + Py_ssize_t length, /* size of string */ + const char *errors, /* error handling */ + Py_ssize_t *consumed /* bytes consumed */ + ); + PyAPI_FUNC(PyObject*) PyUnicode_AsMBCSString( PyObject *unicode /* Unicode object */ ); Modified: python/trunk/Lib/encodings/mbcs.py ============================================================================== --- python/trunk/Lib/encodings/mbcs.py (original) +++ python/trunk/Lib/encodings/mbcs.py Wed Jun 14 07:21:04 2006 @@ -22,9 +22,10 @@ def encode(self, input, final=False): return codecs.mbcs_encode(input,self.errors)[0] -class IncrementalDecoder(codecs.IncrementalDecoder): - def decode(self, input, final=False): - return codecs.mbcs_decode(input,self.errors)[0] +class IncrementalDecoder(codecs.BufferedIncrementalDecoder): + def _buffer_decode(self, input, errors, final): + return codecs.mbcs_decode(input,self.errors,final) + class StreamWriter(Codec,codecs.StreamWriter): pass Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 14 07:21:04 2006 @@ -156,6 +156,9 @@ Library ------- +- Patch #1455898: The MBCS codec now supports the incremental mode for + double-byte encodings. + - ``difflib``'s ``SequenceMatcher.get_matching_blocks()`` was changed to guarantee that adjacent triples in the return list always describe non-adjacent blocks. Previously, a pair of matching blocks could end Modified: python/trunk/Modules/_codecsmodule.c ============================================================================== --- python/trunk/Modules/_codecsmodule.c (original) +++ python/trunk/Modules/_codecsmodule.c Wed Jun 14 07:21:04 2006 @@ -479,15 +479,20 @@ PyObject *args) { const char *data; - Py_ssize_t size; + Py_ssize_t size, consumed; const char *errors = NULL; + int final = 1; + PyObject *decoded; - if (!PyArg_ParseTuple(args, "t#|z:mbcs_decode", - &data, &size, &errors)) + if (!PyArg_ParseTuple(args, "t#|zi:mbcs_decode", + &data, &size, &errors, &final)) return NULL; - return codec_tuple(PyUnicode_DecodeMBCS(data, size, errors), - size); + decoded = PyUnicode_DecodeMBCSStateful( + data, size, errors, final ? NULL : &consumed); + if (!decoded) + return NULL; + return codec_tuple(decoded, final ? size : consumed); } #endif /* MS_WINDOWS */ Modified: python/trunk/Objects/unicodeobject.c ============================================================================== --- python/trunk/Objects/unicodeobject.c (original) +++ python/trunk/Objects/unicodeobject.c Wed Jun 14 07:21:04 2006 @@ -2820,65 +2820,199 @@ /* --- MBCS codecs for Windows -------------------------------------------- */ -PyObject *PyUnicode_DecodeMBCS(const char *s, - Py_ssize_t size, - const char *errors) +#if SIZEOF_INT < SIZEOF_SSIZE_T +#define NEED_RETRY +#endif + +/* XXX This code is limited to "true" double-byte encodings, as + a) it assumes an incomplete character consists of a single byte, and + b) IsDBCSLeadByte (probably) does not work for non-DBCS multi-byte + encodings, see IsDBCSLeadByteEx documentation. */ + +static int is_dbcs_lead_byte(const char *s, int offset) +{ + const char *curr = s + offset; + + if (IsDBCSLeadByte(*curr)) { + const char *prev = CharPrev(s, curr); + return (prev == curr) || !IsDBCSLeadByte(*prev) || (curr - prev == 2); + } + return 0; +} + +/* + * Decode MBCS string into unicode object. If 'final' is set, converts + * trailing lead-byte too. Returns consumed size if succeed, -1 otherwise. + */ +static int decode_mbcs(PyUnicodeObject **v, + const char *s, /* MBCS string */ + int size, /* sizeof MBCS string */ + int final) { - PyUnicodeObject *v; Py_UNICODE *p; - DWORD usize; + Py_ssize_t n = 0; + int usize = 0; + + assert(size >= 0); + + /* Skip trailing lead-byte unless 'final' is set */ + if (!final && size >= 1 && is_dbcs_lead_byte(s, size - 1)) + --size; /* First get the size of the result */ - assert(size < INT_MAX); - usize = MultiByteToWideChar(CP_ACP, 0, s, (int)size, NULL, 0); - if (size > 0 && usize==0) - return PyErr_SetFromWindowsErrWithFilename(0, NULL); + if (size > 0) { + usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0); + if (usize == 0) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } - v = _PyUnicode_New(usize); - if (v == NULL) - return NULL; - if (usize == 0) - return (PyObject *)v; - p = PyUnicode_AS_UNICODE(v); - if (0 == MultiByteToWideChar(CP_ACP, 0, s, (int)size, p, usize)) { - Py_DECREF(v); - return PyErr_SetFromWindowsErrWithFilename(0, NULL); + if (*v == NULL) { + /* Create unicode object */ + *v = _PyUnicode_New(usize); + if (*v == NULL) + return -1; + } + else { + /* Extend unicode object */ + n = PyUnicode_GET_SIZE(*v); + if (_PyUnicode_Resize(v, n + usize) < 0) + return -1; + } + + /* Do the conversion */ + if (size > 0) { + p = PyUnicode_AS_UNICODE(*v) + n; + if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } + + return size; +} + +PyObject *PyUnicode_DecodeMBCSStateful(const char *s, + Py_ssize_t size, + const char *errors, + Py_ssize_t *consumed) +{ + PyUnicodeObject *v = NULL; + int done; + + if (consumed) + *consumed = 0; + +#ifdef NEED_RETRY + retry: + if (size > INT_MAX) + done = decode_mbcs(&v, s, INT_MAX, 0); + else +#endif + done = decode_mbcs(&v, s, (int)size, !consumed); + + if (done < 0) { + Py_XDECREF(v); + return NULL; } + if (consumed) + *consumed += done; + +#ifdef NEED_RETRY + if (size > INT_MAX) { + s += done; + size -= done; + goto retry; + } +#endif + return (PyObject *)v; } -PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p, +PyObject *PyUnicode_DecodeMBCS(const char *s, Py_ssize_t size, const char *errors) { - PyObject *repr; - char *s; - DWORD mbcssize; - - /* If there are no characters, bail now! */ - if (size==0) - return PyString_FromString(""); + return PyUnicode_DecodeMBCSStateful(s, size, errors, NULL); +} + +/* + * Convert unicode into string object (MBCS). + * Returns 0 if succeed, -1 otherwise. + */ +static int encode_mbcs(PyObject **repr, + const Py_UNICODE *p, /* unicode */ + int size) /* size of unicode */ +{ + int mbcssize = 0; + Py_ssize_t n = 0; + + assert(size >= 0); /* First get the size of the result */ - assert(size 0) { + mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL); + if (mbcssize == 0) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } - repr = PyString_FromStringAndSize(NULL, mbcssize); - if (repr == NULL) - return NULL; - if (mbcssize == 0) - return repr; + if (*repr == NULL) { + /* Create string object */ + *repr = PyString_FromStringAndSize(NULL, mbcssize); + if (*repr == NULL) + return -1; + } + else { + /* Extend string object */ + n = PyString_Size(*repr); + if (_PyString_Resize(repr, n + mbcssize) < 0) + return -1; + } /* Do the conversion */ - s = PyString_AS_STRING(repr); - assert(size < INT_MAX); - if (0 == WideCharToMultiByte(CP_ACP, 0, p, (int)size, s, mbcssize, NULL, NULL)) { - Py_DECREF(repr); - return PyErr_SetFromWindowsErrWithFilename(0, NULL); + if (size > 0) { + char *s = PyString_AS_STRING(*repr) + n; + if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } + + return 0; +} + +PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p, + Py_ssize_t size, + const char *errors) +{ + PyObject *repr = NULL; + int ret; + +#ifdef NEED_RETRY + retry: + if (size > INT_MAX) + ret = encode_mbcs(&repr, p, INT_MAX); + else +#endif + ret = encode_mbcs(&repr, p, (int)size); + + if (ret < 0) { + Py_XDECREF(repr); + return NULL; } + +#ifdef NEED_RETRY + if (size > INT_MAX) { + p += INT_MAX; + size -= INT_MAX; + goto retry; + } +#endif + return repr; } @@ -2893,6 +3027,8 @@ NULL); } +#undef NEED_RETRY + #endif /* MS_WINDOWS */ /* --- Character Mapping Codec -------------------------------------------- */ From buildbot at python.org Wed Jun 14 07:36:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 05:36:17 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060614053617.7012F1E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/358 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Wed Jun 14 07:51:57 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 05:51:57 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060614055157.F1D771E400F@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/179 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 14 08:08:31 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 08:08:31 +0200 (CEST) Subject: [Python-checkins] r46946 - in python/trunk: Lib/shelve.py Misc/NEWS Message-ID: <20060614060831.ED3F21E4010@bag.python.org> Author: georg.brandl Date: Wed Jun 14 08:08:31 2006 New Revision: 46946 Modified: python/trunk/Lib/shelve.py python/trunk/Misc/NEWS Log: Bug #1339007: Shelf objects now don't raise an exception in their __del__ method when initialization failed. Modified: python/trunk/Lib/shelve.py ============================================================================== --- python/trunk/Lib/shelve.py (original) +++ python/trunk/Lib/shelve.py Wed Jun 14 08:08:31 2006 @@ -139,6 +139,9 @@ self.dict = 0 def __del__(self): + if not hasattr(self, 'writeback'): + # __init__ didn't succeed, so don't bother closing + return self.close() def sync(self): Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 14 08:08:31 2006 @@ -156,6 +156,9 @@ Library ------- +- Bug #1339007: Shelf objects now don't raise an exception in their + __del__ method when initialization failed. + - Patch #1455898: The MBCS codec now supports the incremental mode for double-byte encodings. From python-checkins at python.org Wed Jun 14 08:08:34 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 08:08:34 +0200 (CEST) Subject: [Python-checkins] r46947 - in python/branches/release24-maint: Lib/shelve.py Misc/NEWS Message-ID: <20060614060834.E38451E4016@bag.python.org> Author: georg.brandl Date: Wed Jun 14 08:08:33 2006 New Revision: 46947 Modified: python/branches/release24-maint/Lib/shelve.py python/branches/release24-maint/Misc/NEWS Log: Bug #1339007: Shelf objects now don't raise an exception in their __del__ method when initialization failed. (backport from rev. 46946) Modified: python/branches/release24-maint/Lib/shelve.py ============================================================================== --- python/branches/release24-maint/Lib/shelve.py (original) +++ python/branches/release24-maint/Lib/shelve.py Wed Jun 14 08:08:33 2006 @@ -145,6 +145,9 @@ self.dict = 0 def __del__(self): + if not hasattr(self, 'writeback'): + # __init__ didn't succeed, so don't bother closing + return self.close() def sync(self): Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Wed Jun 14 08:08:33 2006 @@ -53,6 +53,9 @@ Library ------- +- Bug #1339007: Shelf objects now don't raise an exception in their + __del__ method when initialization failed. + - Bug #1501223: Possible overflow in _PySys_Init() when reading the code page under Windows. From python-checkins at python.org Wed Jun 14 08:18:16 2006 From: python-checkins at python.org (thomas.heller) Date: Wed, 14 Jun 2006 08:18:16 +0200 (CEST) Subject: [Python-checkins] r46948 - python/trunk/Lib/ctypes/__init__.py Message-ID: <20060614061816.176A11E4004@bag.python.org> Author: thomas.heller Date: Wed Jun 14 08:18:15 2006 New Revision: 46948 Modified: python/trunk/Lib/ctypes/__init__.py Log: Fix docstring. Modified: python/trunk/Lib/ctypes/__init__.py ============================================================================== --- python/trunk/Lib/ctypes/__init__.py (original) +++ python/trunk/Lib/ctypes/__init__.py Wed Jun 14 08:18:15 2006 @@ -69,7 +69,7 @@ restype: the result type argtypes: a sequence specifying the argument types - The function prototype can be called in three ways to create a + The function prototype can be called in different ways to create a callable object: prototype(integer address) -> foreign function From python-checkins at python.org Wed Jun 14 08:29:08 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 08:29:08 +0200 (CEST) Subject: [Python-checkins] r46949 - python/trunk/Doc/ref/ref5.tex Message-ID: <20060614062908.AC8291E4004@bag.python.org> Author: georg.brandl Date: Wed Jun 14 08:29:07 2006 New Revision: 46949 Modified: python/trunk/Doc/ref/ref5.tex Log: Bug #1501122: mention __gt__ &co in description of comparison order. Modified: python/trunk/Doc/ref/ref5.tex ============================================================================== --- python/trunk/Doc/ref/ref5.tex (original) +++ python/trunk/Doc/ref/ref5.tex Wed Jun 14 08:29:07 2006 @@ -907,7 +907,10 @@ the values of two objects. The objects need not have the same type. If both are numbers, they are converted to a common type. Otherwise, objects of different types \emph{always} compare unequal, and are -ordered consistently but arbitrarily. +ordered consistently but arbitrarily. You can control comparison +behavior of objects of non-builtin types by defining a \code{__cmp__} +method or rich comparison methods like \code{__gt__}, described in +section~\ref{specialnames}. (This unusual definition of comparison was used to simplify the definition of operations like sorting and the \keyword{in} and @@ -952,7 +955,8 @@ a dictionary for emptiness by comparing it to \code{\{\}}.} \item -Most other types compare unequal unless they are the same object; +Most other objects of builtin types compare unequal unless they are +the same object; the choice whether one object is considered smaller or larger than another one is made arbitrarily but consistently within one execution of a program. From python-checkins at python.org Wed Jun 14 08:29:13 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 08:29:13 +0200 (CEST) Subject: [Python-checkins] r46950 - python/branches/release24-maint/Doc/ref/ref5.tex Message-ID: <20060614062913.A98261E4004@bag.python.org> Author: georg.brandl Date: Wed Jun 14 08:29:13 2006 New Revision: 46950 Modified: python/branches/release24-maint/Doc/ref/ref5.tex Log: Bug #1501122: mention __gt__ &co in description of comparison order. (backport from rev. 46949) Modified: python/branches/release24-maint/Doc/ref/ref5.tex ============================================================================== --- python/branches/release24-maint/Doc/ref/ref5.tex (original) +++ python/branches/release24-maint/Doc/ref/ref5.tex Wed Jun 14 08:29:13 2006 @@ -908,7 +908,10 @@ the values of two objects. The objects need not have the same type. If both are numbers, they are converted to a common type. Otherwise, objects of different types \emph{always} compare unequal, and are -ordered consistently but arbitrarily. +ordered consistently but arbitrarily. You can control comparison +behavior of objects of non-builtin types by defining a \code{__cmp__} +method or rich comparison methods like \code{__gt__}, described in +section~\ref{specialnames}. (This unusual definition of comparison was used to simplify the definition of operations like sorting and the \keyword{in} and @@ -953,7 +956,8 @@ a dictionary for emptiness by comparing it to \code{\{\}}.} \item -Most other types compare unequal unless they are the same object; +Most other objects of builtin types compare unequal unless they are +the same object; the choice whether one object is considered smaller or larger than another one is made arbitrarily but consistently within one execution of a program. From python-checkins at python.org Wed Jun 14 09:08:39 2006 From: python-checkins at python.org (thomas.heller) Date: Wed, 14 Jun 2006 09:08:39 +0200 (CEST) Subject: [Python-checkins] r46951 - python/trunk/Doc/lib/libctypes.tex Message-ID: <20060614070839.51F3F1E4004@bag.python.org> Author: thomas.heller Date: Wed Jun 14 09:08:38 2006 New Revision: 46951 Modified: python/trunk/Doc/lib/libctypes.tex Log: Write more docs. Modified: python/trunk/Doc/lib/libctypes.tex ============================================================================== --- python/trunk/Doc/lib/libctypes.tex (original) +++ python/trunk/Doc/lib/libctypes.tex Wed Jun 14 09:08:38 2006 @@ -1576,19 +1576,19 @@ These prefabricated library loaders are available: \begin{datadescni}{cdll} -Loads \class{CDLL} instances. +Creates \class{CDLL} instances. \end{datadescni} \begin{datadescni}{windll} -Windows only: Loads \class{WinDLL} instances. +Windows only: Creates \class{WinDLL} instances. \end{datadescni} \begin{datadescni}{oledll} -Windows only: Loads \class{OleDLL} instances. +Windows only: Creates \class{OleDLL} instances. \end{datadescni} \begin{datadescni}{pydll} -Loads \class{PyDLL} instances. +Creates \class{PyDLL} instances. \end{datadescni} For accessing the C Python api directly, a ready-to-use Python shared @@ -1598,49 +1598,149 @@ An instance of \class{PyDLL} that exposes Python C api functions as attributes. Note that all these functions are assumed to return integers, which is of course not always the truth, so you have to -assign the correct \member{restype} attribute. +assign the correct \member{restype} attribute to use these functions. \end{datadescni} \subsubsection{Foreign functions\label{ctypes-foreign-functions}} -The ultimate goal of \code{ctypes} is to call functions in shared -libraries, aka as foreign functions. Foreign function instances can -be created by retrieving them as attributes of loaded shared -libraries, or by instantiating a \emph{function prototype}. +As explained in the previous section, foreign functions can be +accessed as attributes of loaded shared libraries. The function +objects created in this way by default accept any number of arguments, +accept any ctypes data instances as arguments, and return the default +result type specified by the library loader. They are instances of a +private class: + +\begin{classdesc*}{_FuncPtr} +Base class for C callable foreign functions. +\end{classdesc*} + +This behaviour can be customized by assigning to special attributes of +the foreign function object. + +\begin{memberdesc}{restype} +Assign a ctypes type to specify the result type of the foreign +function. Use \code{None} for \code{void} a function not returning +anything. + +It is possible to assign a callable Python object that is not a +ctypes type, in this case the function is assumed to return an +integer, and the callable will be called with this integer, +allowing to do further processing or error checking. Using this +is deprecated, for more flexible postprocessing or error checking +use a ctypes data type as \member{restype} and assign a callable to the +\member{errcheck} attribute. +\end{memberdesc} -By default, functions got as attributes of loaded shared libraries -accept any arguments, and have a return type of \class{c{\_}int}. +\begin{memberdesc}{argtypes} +Assign a tuple of ctypes types to specify the argument types that +the function accepts. Functions using the \code{stdcall} calling +convention can only be called with the same number of arguments as +the length of this tuple; functions using the C calling convention +accept additional, unspecified arguments as well. + +When a foreign function is called, each actual argument is passed +to the \method{from{\_}param} class method of the items in the +\member{argtypes} tuple, this method allows to adapt the actual +argument to an object that the foreign function accepts. For +example, a \class{c{\_}char{\_}p} item in the \member{argtypes} tuple will +convert a unicode string passed as argument into an byte string +using ctypes conversion rules. +\end{memberdesc} -Function prototypes are created by factory functions. +\begin{memberdesc}{errcheck} +Assign a Python function or another callable to this attribute. +The callable will be called with three or more arguments: +\end{memberdesc} -They are created by calling one of the following factory functions: +\begin{funcdescni}{callable}{result, func, arguments, *others} +\code{result} is what the foreign function returns, as specified by the +\member{restype} attribute. + +\code{func} is the foreign function object itself, this allows to +reuse the same callable object to check or postprocess the results +of several functions. + +\code{arguments} is a tuple containing the parameters originally +passed to the function call, this allows to specialize the +behaviour on the arguments used. + +The object that this function returns will be returned from the +foreign function call, but it can also check the result value and +raise an exception if the foreign function call failed. + +\code{others} will usually be an empty tuple, it is only used in +combination with \var{paramflags} documented below. +\end{funcdescni} + +Instances of foreign functions are also C compatible data types; they +represent C function pointers. + +Foreign functions can also be created by instantiating function +prototypes. Function prototypes are similar to function prototypes in +C; they describe a function (return type, argument types, calling +convention) without defining an implementation. The factory +functions must be called with the desired result type and the argument +types of the function. \begin{funcdesc}{CFUNCTYPE}{restype, *argtypes} -This is a factory function that returns a function prototype. The -function prototype describes a function that has a result type of -\member{restype}, and accepts arguments as specified by -\member{argtypes}. The function prototype can be used to construct -several kinds of functions, depending on how the prototype is -called. - -The prototypes returned by \function{CFUNCTYPE} or \code{PYFUNCTYPE} create -functions that use the standard C calling convention, prototypes -returned from \function{WINFUNCTYPE} (on Windows) use the \code{{\_}{\_}stdcall} -calling convention. - -Functions created by calling the \function{CFUNCTYPE} and \function{WINFUNCTYPE} -prototypes release the Python GIL before entering the foreign -function, and acquire it back after leaving the function code. +The returned function prototype creates functions that use the +standard C calling convention. The function will release the GIL +during the call. \end{funcdesc} \begin{funcdesc}{WINFUNCTYPE}{restype, *argtypes} -TBD +Windows only: The returned function prototype creates functions +that use the \code{stdcall} calling convention, except on Windows CE +where \function{WINFUNCTYPE} is the same as \function{CFUNCTYPE}. The function +will release the GIL during the call. \end{funcdesc} \begin{funcdesc}{PYFUNCTYPE}{restype, *argtypes} -TBD -\end{funcdesc} +The returned function prototype creates functions that use the +Python calling convention. The function will \emph{not} release the +GIL during the call. +\end{funcdesc} + +Function prototypes created by the factory functions can be +instantiated in different ways, depending on the type and number of +the parameters in the call. + +\begin{funcdescni}{prototype}{address} +Returns a foreign function at the specified address. +\end{funcdescni} + +\begin{funcdescni}{prototype}{callable} +Create a C callable function (a callback function) from a Python +\code{callable}. +\end{funcdescni} + +\begin{funcdescni}{prototype}{}{name, library\optional{, paramflags}} +Returns a foreign function by looking up \var{name} in the shared +library \code{dll}. +\end{funcdescni} + +\begin{funcdescni}{prototype}{}{ordinal, library\optional{, paramflags}} +Returns a foreign function which is exported by the ordinal number +\code{ordinal} in the shared library \code{dll}. This mechanism only +exists on Windows. +\end{funcdescni} + +\begin{funcdescni}{prototype}{vtbl_index, name\optional{, paramflags\optional{, iid}}} +Returns a foreign function that will call a COM method. +\code{vtbl{\_}index} is the index into the virtual function table, a +small nonnegative integer. \var{name} is name of the COM method. +\var{iid} is an optional pointer to the interface identifier which +is used in extended error reporting. + +COM methods use a special calling convention: They require a +pointer to the COM interface as first argument, in addition to +those parameters that are specified in the \member{argtypes} tuple. +\end{funcdescni} + +XXX Document paramflags. + +XXX Where does the exception description belong? \begin{excdesc}{ArgumentError()} This exception is raised when a foreign function call cannot @@ -1826,11 +1926,18 @@ \begin{methoddesc}{from_address}{address} This method returns a ctypes type instance using the memory -specified by address. +specified by address which must be an integer. \end{methoddesc} \begin{methoddesc}{from_param}{obj} -This method adapts obj to a ctypes type. +This method adapts obj to a ctypes type. It is called with the +actual object used in a foreign function call, when the type is +present in the foreign functions \member{argtypes} tuple; it must +return an object that can be used as function call parameter. + +All ctypes data types have a default implementation of this +classmethod, normally it returns \code{obj} if that is an instance of +the type. Some types accept other objects as well. \end{methoddesc} \begin{methoddesc}{in_dll}{name, library} @@ -2051,6 +2158,10 @@ Represents the C \code{PyObject *} datatype. \end{classdesc*} +The \code{ctypes.wintypes} module provides quite some other Windows +specific data types, for example \code{HWND}, \code{WPARAM}, or \code{DWORD}. +Some useful structures like \code{MSG} or \code{RECT} are also defined. + \subsubsection{Structured data types\label{ctypes-structured-data-types}} From buildbot at python.org Wed Jun 14 09:14:08 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 07:14:08 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP 2.4 Message-ID: <20060614071408.6E1031E4004@bag.python.org> The Buildbot has detected a new failure of x86 XP 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%25202.4/builds/163 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: georg.brandl Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From nnorwitz at gmail.com Wed Jun 14 09:34:28 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 14 Jun 2006 00:34:28 -0700 Subject: [Python-checkins] r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sqlite/module.c Modules/_sqlite/module.h In-Reply-To: <1f7befae0606131837x2a4c71bdo588544a759968542@mail.gmail.com> References: <20060613222453.4561E1E4004@bag.python.org> <1f7befae0606131837x2a4c71bdo588544a759968542@mail.gmail.com> Message-ID: On 6/13/06, Tim Peters wrote: > > Author: gerhard.haering > > Date: Wed Jun 14 00:24:47 2006 > > New Revision: 46936 > > > > Modified: > > python/trunk/Lib/sqlite3/test/regression.py > > python/trunk/Lib/sqlite3/test/types.py > > python/trunk/Lib/sqlite3/test/userfunctions.py > > python/trunk/Modules/_sqlite/connection.c > > python/trunk/Modules/_sqlite/cursor.c > > python/trunk/Modules/_sqlite/module.c > > python/trunk/Modules/_sqlite/module.h > > Log: > > Merged changes from external pysqlite 2.3.0 release. Documentation updates will > > follow in a few hours at the latest. Then we should be ready for beta1. > > Please note that many (all?) non-Windows buildbots are failing > test_sqlite now, and this one is even pumping out complaints from the > system malloc()/free(): > > http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/1009/step-test/0 > > That's not really what we want to see in beta1 :-) Conditional jump or move depends on uninitialised value(s) at 0x62D5998: sqlite3VdbeMemRelease (vdbemem.c:196) by 0x62D5CDB: sqlite3VdbeMemSetStr (vdbemem.c:383) by 0x62D28FC: sqlite3_result_error (vdbeapi.c:97) by 0x618CBA3: _step_callback (connection.c:561) by 0x62CE083: sqlite3VdbeExec (vdbe.c:4393) by 0x62D2B0C: sqlite3_step (vdbeapi.c:211) by 0x6192DAD: _sqlite_step_with_busyhandler (util.c:33) by 0x618FD97: _query_execute (cursor.c:610) by 0x61904DD: cursor_execute (cursor.c:721) by 0x4F8FCD: PyCFunction_Call (methodobject.c:73) Conditional jump or move depends on uninitialised value(s) at 0x62D5998: sqlite3VdbeMemRelease (vdbemem.c:196) by 0x62D5CDB: sqlite3VdbeMemSetStr (vdbemem.c:383) by 0x62D28FC: sqlite3_result_error (vdbeapi.c:97) by 0x618CC74: _step_callback (connection.c:585) by 0x62CE083: sqlite3VdbeExec (vdbe.c:4393) by 0x62D2B0C: sqlite3_step (vdbeapi.c:211) by 0x6192DAD: _sqlite_step_with_busyhandler (util.c:33) by 0x618FD97: _query_execute (cursor.c:610) by 0x61904DD: cursor_execute (cursor.c:721) by 0x4F8FCD: PyCFunction_Call (methodobject.c:73) 53 bytes in 1 blocks are definitely lost in loss record 13 of 286 at 0x4A19B7E: malloc (vg_replace_malloc.c:149) by 0x62CC36A: sqlite3MallocRaw (util.c:276) by 0x62D582B: sqlite3VdbeMemMakeWriteable (vdbemem.c:99) by 0x62D5D6A: sqlite3VdbeMemSetStr (vdbemem.c:433) by 0x62D28FC: sqlite3_result_error (vdbeapi.c:97) by 0x618CC74: _step_callback (connection.c:585) by 0x62CE083: sqlite3VdbeExec (vdbe.c:4393) by 0x62D2B0C: sqlite3_step (vdbeapi.c:211) by 0x6192DAD: _sqlite_step_with_busyhandler (util.c:33) by 0x618FD97: _query_execute (cursor.c:610) by 0x61904DD: cursor_execute (cursor.c:721) by 0x4F8FCD: PyCFunction_Call (methodobject.c:73) 57 bytes in 1 blocks are definitely lost in loss record 14 of 286 at 0x4A19B7E: malloc (vg_replace_malloc.c:149) by 0x62CC36A: sqlite3MallocRaw (util.c:276) by 0x62D582B: sqlite3VdbeMemMakeWriteable (vdbemem.c:99) by 0x62D5D6A: sqlite3VdbeMemSetStr (vdbemem.c:433) by 0x62D28FC: sqlite3_result_error (vdbeapi.c:97) by 0x618CBA3: _step_callback (connection.c:561) by 0x62CE083: sqlite3VdbeExec (vdbe.c:4393) by 0x62D2B0C: sqlite3_step (vdbeapi.c:211) by 0x6192DAD: _sqlite_step_with_busyhandler (util.c:33) by 0x618FD97: _query_execute (cursor.c:610) by 0x61904DD: cursor_execute (cursor.c:721) by 0x4F8FCD: PyCFunction_Call (methodobject.c:73) From neal at metaslash.com Wed Jun 14 10:14:07 2006 From: neal at metaslash.com (Neal Norwitz) Date: Wed, 14 Jun 2006 04:14:07 -0400 Subject: [Python-checkins] Python Regression Test Failures basics (2) Message-ID: <20060614081407.GA18376@python.psfb.org> case $MAKEFLAGS in \ *-s*) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py -q build;; \ *) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py build;; \ esac running build running build_ext db.h: found (4, 1) in /usr/include db lib: using (4, 1) db-4.1 sqlite: found /usr/include/sqlite3.h /usr/include/sqlite3.h: version 3.2.1 INFO: Can't locate Tcl/Tk libs and/or headers running build_scripts [34100 refs] ./python -E -c 'import sys ; from distutils.util import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform [8686 refs] find ./Lib -name '*.py[co]' -print | xargs rm -f ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test test_sqlite failed -- errors occurred; run in verbose mode for details test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 286 tests OK. 1 test failed: test_sqlite 30 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [432978 refs] make: [test] Error 1 (ignored) ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test test_sqlite failed -- errors occurred; run in verbose mode for details test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 286 tests OK. 1 test failed: test_sqlite 30 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [432392 refs] make: *** [test] Error 1 From python-checkins at python.org Wed Jun 14 10:31:39 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 10:31:39 +0200 (CEST) Subject: [Python-checkins] r46952 - python/trunk/Doc/ref/ref3.tex Message-ID: <20060614083139.9F5FF1E4004@bag.python.org> Author: georg.brandl Date: Wed Jun 14 10:31:39 2006 New Revision: 46952 Modified: python/trunk/Doc/ref/ref3.tex Log: Bug #1153163: describe __add__ vs __radd__ behavior when adding objects of same type/of subclasses of the other. Modified: python/trunk/Doc/ref/ref3.tex ============================================================================== --- python/trunk/Doc/ref/ref3.tex (original) +++ python/trunk/Doc/ref/ref3.tex Wed Jun 14 10:31:39 2006 @@ -1918,13 +1918,28 @@ \function{pow()}\bifuncindex{pow}, \code{**}, \code{<<}, \code{>>}, \code{\&}, \code{\^}, \code{|}) with reflected (swapped) operands. These functions are only called if the left -operand does not support the corresponding operation. For instance, -to evaluate the expression \var{x}\code{-}\var{y}, where \var{y} is an -instance of a class that has an \method{__rsub__()} method, -\code{\var{y}.__rsub__(\var{x})} is called. Note that ternary +operand does not support the corresponding operation and the +operands are of different types.\footnote{ + For operands of the same type, it is assumed that if the + non-reflected method (such as \method{__add__()}) fails the + operation is not supported, which is why the reflected method + is not called.} +For instance, to evaluate the expression \var{x}\code{-}\var{y}, +where \var{y} is an instance of a class that has an +\method{__rsub__()} method, \code{\var{y}.__rsub__(\var{x})} +is called if \code{\var{x}.__sub__(\var{y})} returns +\var{NotImplemented}. + +Note that ternary \function{pow()}\bifuncindex{pow} will not try calling \method{__rpow__()} (the coercion rules would become too complicated). + +\note{If the right operand's type is a subclass of the left operand's + type and that subclass provides the reflected method for the + operation, this method will be called before the right operand's + non-reflected method. This behavior allows subclasses to + override their ancestors' operations.} \end{methoddesc} \begin{methoddesc}[numeric object]{__iadd__}{self, other} From python-checkins at python.org Wed Jun 14 10:31:42 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 10:31:42 +0200 (CEST) Subject: [Python-checkins] r46953 - python/branches/release24-maint/Doc/ref/ref3.tex Message-ID: <20060614083142.69B411E4004@bag.python.org> Author: georg.brandl Date: Wed Jun 14 10:31:41 2006 New Revision: 46953 Modified: python/branches/release24-maint/Doc/ref/ref3.tex Log: Bug #1153163: describe __add__ vs __radd__ behavior when adding objects of same type/of subclasses of the other. (backport from rev. 46952) Modified: python/branches/release24-maint/Doc/ref/ref3.tex ============================================================================== --- python/branches/release24-maint/Doc/ref/ref3.tex (original) +++ python/branches/release24-maint/Doc/ref/ref3.tex Wed Jun 14 10:31:41 2006 @@ -1873,13 +1873,28 @@ \function{pow()}\bifuncindex{pow}, \code{**}, \code{<}\code{<}, \code{>}\code{>}, \code{\&}, \code{\^}, \code{|}) with reflected (swapped) operands. These functions are only called if the left -operand does not support the corresponding operation. For instance, -to evaluate the expression \var{x}\code{-}\var{y}, where \var{y} is an -instance of a class that has an \method{__rsub__()} method, -\code{\var{y}.__rsub__(\var{x})} is called. Note that ternary +operand does not support the corresponding operation and the +operands are of different types.\footnote{ + For operands of the same type, it is assumed that if the + non-reflected method (such as \method{__add__()}) fails the + operation is not supported, which is why the reflected method + is not called.} +For instance, to evaluate the expression \var{x}\code{-}\var{y}, +where \var{y} is an instance of a class that has an +\method{__rsub__()} method, \code{\var{y}.__rsub__(\var{x})} +is called if \code{\var{x}.__sub__(\var{y})} returns +\var{NotImplemented}. + +Note that ternary \function{pow()}\bifuncindex{pow} will not try calling \method{__rpow__()} (the coercion rules would become too complicated). + +\note{If the right operand's type is a subclass of the left operand's + type and that subclass provides the reflected method for the + operation, this method will be called before the right operand's + non-reflected method. This behavior allows subclasses to + override their ancestors' operations.} \end{methoddesc} \begin{methoddesc}[numeric object]{__iadd__}{self, other} From python-checkins at python.org Wed Jun 14 10:42:12 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 10:42:12 +0200 (CEST) Subject: [Python-checkins] r46954 - python/trunk/Lib/mimetypes.py Message-ID: <20060614084212.D13481E4004@bag.python.org> Author: georg.brandl Date: Wed Jun 14 10:42:11 2006 New Revision: 46954 Modified: python/trunk/Lib/mimetypes.py Log: Bug #1202018: add some common mime.types locations. Modified: python/trunk/Lib/mimetypes.py ============================================================================== --- python/trunk/Lib/mimetypes.py (original) +++ python/trunk/Lib/mimetypes.py Wed Jun 14 10:42:11 2006 @@ -33,6 +33,10 @@ knownfiles = [ "/etc/mime.types", + "/etc/httpd/mime.types", # Mac OS X + "/etc/httpd/conf/mime.types", # Apache + "/etc/apache/mime.types", # Apache 1 + "/etc/apache2/mime.types", # Apache 2 "/usr/local/etc/httpd/conf/mime.types", "/usr/local/lib/netscape/mime.types", "/usr/local/etc/httpd/conf/mime.types", # Apache 1.2 From python-checkins at python.org Wed Jun 14 10:50:03 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 10:50:03 +0200 (CEST) Subject: [Python-checkins] r46955 - in python/trunk: Lib/SimpleHTTPServer.py Misc/NEWS Message-ID: <20060614085003.D5B611E400E@bag.python.org> Author: georg.brandl Date: Wed Jun 14 10:50:03 2006 New Revision: 46955 Modified: python/trunk/Lib/SimpleHTTPServer.py python/trunk/Misc/NEWS Log: Bug #1117556: SimpleHTTPServer now tries to find and use the system's mime.types file for determining MIME types. Modified: python/trunk/Lib/SimpleHTTPServer.py ============================================================================== --- python/trunk/Lib/SimpleHTTPServer.py (original) +++ python/trunk/Lib/SimpleHTTPServer.py Wed Jun 14 10:50:03 2006 @@ -191,7 +191,9 @@ return self.extensions_map[ext] else: return self.extensions_map[''] - + + if not mimetypes.inited: + mimetypes.init() # try to read system mime.types extensions_map = mimetypes.types_map.copy() extensions_map.update({ '': 'application/octet-stream', # Default Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 14 10:50:03 2006 @@ -156,6 +156,9 @@ Library ------- +- Bug #1117556: SimpleHTTPServer now tries to find and use the system's + mime.types file for determining MIME types. + - Bug #1339007: Shelf objects now don't raise an exception in their __del__ method when initialization failed. From python-checkins at python.org Wed Jun 14 10:50:06 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 10:50:06 +0200 (CEST) Subject: [Python-checkins] r46956 - in python/branches/release24-maint: Lib/SimpleHTTPServer.py Misc/NEWS Message-ID: <20060614085006.EE5961E400E@bag.python.org> Author: georg.brandl Date: Wed Jun 14 10:50:05 2006 New Revision: 46956 Modified: python/branches/release24-maint/Lib/SimpleHTTPServer.py python/branches/release24-maint/Misc/NEWS Log: Bug #1117556: SimpleHTTPServer now tries to find and use the system's mime.types file for determining MIME types. (backport from rev. 46955) Modified: python/branches/release24-maint/Lib/SimpleHTTPServer.py ============================================================================== --- python/branches/release24-maint/Lib/SimpleHTTPServer.py (original) +++ python/branches/release24-maint/Lib/SimpleHTTPServer.py Wed Jun 14 10:50:05 2006 @@ -185,7 +185,9 @@ return self.extensions_map[ext] else: return self.extensions_map[''] - + + if not mimetypes.inited: + mimetypes.init() # try to read system mime.types extensions_map = mimetypes.types_map.copy() extensions_map.update({ '': 'application/octet-stream', # Default Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Wed Jun 14 10:50:05 2006 @@ -53,6 +53,9 @@ Library ------- +- Bug #1117556: SimpleHTTPServer now tries to find and use the system's + mime.types file for determining MIME types. + - Bug #1339007: Shelf objects now don't raise an exception in their __del__ method when initialization failed. From buildbot at python.org Wed Jun 14 11:01:13 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 09:01:13 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian 2.4 Message-ID: <20060614090113.3673A1E4004@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%25202.4/builds/28 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 14 11:09:09 2006 From: python-checkins at python.org (thomas.heller) Date: Wed, 14 Jun 2006 11:09:09 +0200 (CEST) Subject: [Python-checkins] r46957 - python/trunk/Doc/lib/libctypes.tex Message-ID: <20060614090909.70C191E4004@bag.python.org> Author: thomas.heller Date: Wed Jun 14 11:09:08 2006 New Revision: 46957 Modified: python/trunk/Doc/lib/libctypes.tex Log: Document paramflags. Modified: python/trunk/Doc/lib/libctypes.tex ============================================================================== --- python/trunk/Doc/lib/libctypes.tex (original) +++ python/trunk/Doc/lib/libctypes.tex Wed Jun 14 11:09:08 2006 @@ -1615,6 +1615,9 @@ Base class for C callable foreign functions. \end{classdesc*} +Instances of foreign functions are also C compatible data types; they +represent C function pointers. + This behaviour can be customized by assigning to special attributes of the foreign function object. @@ -1653,7 +1656,7 @@ The callable will be called with three or more arguments: \end{memberdesc} -\begin{funcdescni}{callable}{result, func, arguments, *others} +\begin{funcdescni}{callable}{result, func, arguments} \code{result} is what the foreign function returns, as specified by the \member{restype} attribute. @@ -1668,13 +1671,15 @@ The object that this function returns will be returned from the foreign function call, but it can also check the result value and raise an exception if the foreign function call failed. - -\code{others} will usually be an empty tuple, it is only used in -combination with \var{paramflags} documented below. \end{funcdescni} -Instances of foreign functions are also C compatible data types; they -represent C function pointers. +\begin{excdesc}{ArgumentError()} +This exception is raised when a foreign function call cannot +convert one of the passed arguments. +\end{excdesc} + + +\subsubsection{Function prototypes\label{ctypes-function-prototypes}} Foreign functions can also be created by instantiating function prototypes. Function prototypes are similar to function prototypes in @@ -1715,15 +1720,12 @@ \code{callable}. \end{funcdescni} -\begin{funcdescni}{prototype}{}{name, library\optional{, paramflags}} -Returns a foreign function by looking up \var{name} in the shared -library \code{dll}. -\end{funcdescni} - -\begin{funcdescni}{prototype}{}{ordinal, library\optional{, paramflags}} -Returns a foreign function which is exported by the ordinal number -\code{ordinal} in the shared library \code{dll}. This mechanism only -exists on Windows. +\begin{funcdescni}{prototype}{func_spec\optional{, paramflags}} +Returns a foreign function exported by a shared library. +\code{func{\_}spec} must be a 2-tuple \code{(name{\_}or{\_}ordinal, library)}. +The first item is the name of the exported function as string, or +the ordinal of the exported function as small integer. The second +item is the shared library instance. \end{funcdescni} \begin{funcdescni}{prototype}{vtbl_index, name\optional{, paramflags\optional{, iid}}} @@ -1738,14 +1740,120 @@ those parameters that are specified in the \member{argtypes} tuple. \end{funcdescni} -XXX Document paramflags. +The optional \var{paramflags} parameter creates foreign function +wrappers with much more functionality than the features described +above. -XXX Where does the exception description belong? +\var{paramflags} must be a tuple of the same length as \member{argtypes}. -\begin{excdesc}{ArgumentError()} -This exception is raised when a foreign function call cannot -convert one of the passed arguments. -\end{excdesc} +Each item in this tuple contains further information about a +parameter, it must be a tuple containing 1, 2, or 3 items. + +The first item is an integer containing flags for the parameter. + +\begin{datadescni}{1} +Specifies an input parameter to the function. +\end{datadescni} + +\begin{datadescni}{2} +Output parameter. The foreign function fills in a value. +\end{datadescni} + +\begin{datadescni}{4} +Input parameter which defaults to the integer zero. +\end{datadescni} + +The optional second item is the parameter name as string. If this is +specified, the foreign function can be called with named parameters. + +The optional third item is the default value for this parameter. + +This example demonstrates how to wrap the Windows \code{MessageBoxA} +function so that it supports default parameters and named arguments. +The C declaration from the windows header file is this: +\begin{verbatim} +WINUSERAPI int WINAPI +MessageBoxA( + HWND hWnd , + LPCSTR lpText, + LPCSTR lpCaption, + UINT uType); +\end{verbatim} + +Here is the wrapping with \code{ctypes}: +\begin{quote} +\begin{verbatim}>>> from ctypes import c_int, WINFUNCTYPE, windll +>>> from ctypes.wintypes import HWND, LPCSTR, UINT +>>> prototype = WINFUNCTYPE(c_int, HWND, LPCSTR, LPCSTR, c_uint) +>>> paramflags = (1, "hwnd", 0), (1, "text", "Hi"), (1, "caption", None), (1, "flags", 0) +>>> MessageBox = prototype(("MessageBoxA", windll.user32), paramflags) +>>>\end{verbatim} +\end{quote} + +The MessageBox foreign function can now be called in these ways: +\begin{verbatim} +>>> MessageBox() +>>> MessageBox(text="Spam, spam, spam") +>>> MessageBox(flags=2, text="foo bar") +>>> +\end{verbatim} + +A second example demonstrates output parameters. The win32 +\code{GetWindowRect} function retrieves the dimensions of a specified +window by copying them into \code{RECT} structure that the caller has to +supply. Here is the C declaration: +\begin{verbatim} +WINUSERAPI BOOL WINAPI +GetWindowRect( + HWND hWnd, + LPRECT lpRect); +\end{verbatim} + +Here is the wrapping with \code{ctypes}: +\begin{quote} +\begin{verbatim}>>> from ctypes import POINTER, WINFUNCTYPE, windll +>>> from ctypes.wintypes import BOOL, HWND, RECT +>>> prototype = WINFUNCTYPE(BOOL, HWND, POINTER(RECT)) +>>> paramflags = (1, "hwnd"), (2, "lprect") +>>> GetWindowRect = prototype(("GetWindowRect", windll.user32), paramflags) +>>>\end{verbatim} +\end{quote} + +Functions with output parameters will automatically return the output +parameter value if there is a single one, or a tuple containing the +output parameter values when there are more than one, so the +GetWindowRect function now returns a RECT instance, when called. + +Output parameters can be combined with the \member{errcheck} protocol to do +further output processing and error checking. The win32 +\code{GetWindowRect} api function returns a \code{BOOL} to signal success or +failure, so this function could do the error checking, and raises an +exception when the api call failed: +\begin{verbatim} +>>> def errcheck(result, func, args): +... if not result: +... raise WinError() +... return args +>>> GetWindowRect.errcheck = errcheck +>>> +\end{verbatim} + +If the \member{errcheck} function returns the argument tuple it receives +unchanged, \code{ctypes} continues the normal processing it does on the +output parameters. If you want to return a tuple of window +coordinates instead of a \code{RECT} instance, you can retrieve the +fields in the function and return them instead, the normal processing +will no longer take place: +\begin{verbatim} +>>> def errcheck(result, func, args): +... if not result: +... raise WinError() +... rc = args[1] +... return rc.left, rc.top, rc.bottom, rc.right +>>> +>>> GetWindowRect.errcheck = errcheck +>>> +\end{verbatim} \subsubsection{Utility functions\label{ctypes-utility-functions}} From buildbot at python.org Wed Jun 14 11:14:09 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 09:14:09 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD 2.4 Message-ID: <20060614091410.087681E4004@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%25202.4/builds/119 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 14 11:20:12 2006 From: python-checkins at python.org (thomas.heller) Date: Wed, 14 Jun 2006 11:20:12 +0200 (CEST) Subject: [Python-checkins] r46958 - python/trunk/Lib/ctypes/wintypes.py Message-ID: <20060614092012.38DED1E4004@bag.python.org> Author: thomas.heller Date: Wed Jun 14 11:20:11 2006 New Revision: 46958 Modified: python/trunk/Lib/ctypes/wintypes.py Log: Add an __all__ list, since this module does 'from ctypes import *'. Modified: python/trunk/Lib/ctypes/wintypes.py ============================================================================== --- python/trunk/Lib/ctypes/wintypes.py (original) +++ python/trunk/Lib/ctypes/wintypes.py Wed Jun 14 11:20:11 2006 @@ -154,3 +154,19 @@ ("dwReserved1", DWORD), ("cFileName", c_wchar * MAX_PATH), ("cAlternameFileName", c_wchar * 14)] + +__all__ = ['ATOM', 'BOOL', 'BOOLEAN', 'BYTE', 'COLORREF', 'DOUBLE', + 'DWORD', 'FILETIME', 'HACCEL', 'HANDLE', 'HBITMAP', 'HBRUSH', + 'HCOLORSPACE', 'HDC', 'HDESK', 'HDWP', 'HENHMETAFILE', 'HFONT', + 'HGDIOBJ', 'HGLOBAL', 'HHOOK', 'HICON', 'HINSTANCE', 'HKEY', + 'HKL', 'HLOCAL', 'HMENU', 'HMETAFILE', 'HMODULE', 'HMONITOR', + 'HPALETTE', 'HPEN', 'HRGN', 'HRSRC', 'HSTR', 'HTASK', 'HWINSTA', + 'HWND', 'LANGID', 'LARGE_INTEGER', 'LCID', 'LCTYPE', 'LGRPID', + 'LONG', 'LPARAM', 'LPCOLESTR', 'LPCSTR', 'LPCWSTR', 'LPOLESTR', + 'LPSTR', 'LPWSTR', 'MAX_PATH', 'MSG', 'OLESTR', 'POINT', + 'POINTL', 'RECT', 'RECTL', 'RGB', 'SC_HANDLE', + 'SERVICE_STATUS_HANDLE', 'SIZE', 'SIZEL', 'SMALL_RECT', 'UINT', + 'ULARGE_INTEGER', 'ULONG', 'VARIANT_BOOL', 'WCHAR', + 'WIN32_FIND_DATAA', 'WIN32_FIND_DATAW', 'WORD', 'WPARAM', '_COORD', + '_FILETIME', '_LARGE_INTEGER', '_POINTL', '_RECTL', '_SMALL_RECT', + '_ULARGE_INTEGER', 'tagMSG', 'tagPOINT', 'tagRECT', 'tagSIZE'] From buildbot at python.org Wed Jun 14 11:47:26 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 09:47:26 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060614094726.83BAE1E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/61 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From gh at ghaering.de Wed Jun 14 12:04:39 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 14 Jun 2006 12:04:39 +0200 Subject: [Python-checkins] r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sqlite/module.c Modules/_sqlite/module.h In-Reply-To: <1f7befae0606131837x2a4c71bdo588544a759968542@mail.gmail.com> References: <20060613222453.4561E1E4004@bag.python.org> <1f7befae0606131837x2a4c71bdo588544a759968542@mail.gmail.com> Message-ID: <448FDF37.9020809@ghaering.de> Tim Peters wrote: >> Author: gerhard.haering >> Date: Wed Jun 14 00:24:47 2006 >> New Revision: 46936 >> >> Modified: >> python/trunk/Lib/sqlite3/test/regression.py >> python/trunk/Lib/sqlite3/test/types.py >> python/trunk/Lib/sqlite3/test/userfunctions.py >> python/trunk/Modules/_sqlite/connection.c >> python/trunk/Modules/_sqlite/cursor.c >> python/trunk/Modules/_sqlite/module.c >> python/trunk/Modules/_sqlite/module.h >> Log: >> Merged changes from external pysqlite 2.3.0 release. Documentation updates will >> follow in a few hours at the latest. Then we should be ready for beta1. > > Please note that many (all?) non-Windows buildbots are failing > test_sqlite now For some reason, they don't seem to have picked up the changed tests of the sqlite3 module. At least the error messages look exactly like the ones I had when I ran the current code against old tests. I verified again that the tests do run fine on Linux x86 in debug and releaes mode from a fresh checkout. >, and this one is even pumping out complaints from the > system malloc()/free(): > > http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/1009/step-test/0 Damn. I don't have any idea yet why this happens :-( > That's not really what we want to see in beta1 :-) Agreed. I really hope the problems can be solved. I'd hate it, but if all else fails, Anthony please roll back checkins 46936 and 46938 for beta1. -- Gerhard From buildbot at python.org Wed Jun 14 15:18:03 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 13:18:03 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060614131803.B74721E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/182 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl,thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 14 15:59:16 2006 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 14 Jun 2006 15:59:16 +0200 (CEST) Subject: [Python-checkins] r46959 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060614135916.789C71E4005@bag.python.org> Author: andrew.kuchling Date: Wed Jun 14 15:59:15 2006 New Revision: 46959 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Add item Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Wed Jun 14 15:59:15 2006 @@ -1373,6 +1373,12 @@ '%H:%M:%S %Y-%m-%d') \end{verbatim} +\item The \method{SequenceMatcher.get_matching_blocks()} method +in the \module{difflib} module now guarantees to return a minimal list +of blocks describing matching subsequences. Previously, the algorithm would +occasionally break a block of matching elements into two list entries. +(Enhancement by Tim Peters.) + \item The \module{doctest} module gained a \code{SKIP} option that keeps an example from being executed at all. This is intended for code snippets that are usage examples intended for the reader and From gh at ghaering.de Wed Jun 14 16:02:26 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 14 Jun 2006 16:02:26 +0200 Subject: [Python-checkins] sqlite3 test errors - was : Re: r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sqlite/module.c Modules/_sqlite/module.h In-Reply-To: <448FDF37.9020809@ghaering.de> References: <20060613222453.4561E1E4004@bag.python.org> <1f7befae0606131837x2a4c71bdo588544a759968542@mail.gmail.com> <448FDF37.9020809@ghaering.de> Message-ID: <449016F2.7050302@ghaering.de> Co-posting to python-dev in the hope of getting help of people verifying my suspicion ... Gerhard H?ring wrote: > [...] > For some reason, they don't seem to have picked up the changed tests of > the sqlite3 module. At least the error messages look exactly like the > ones I had when I ran the current code against old tests. That guess was wrong. The failed sqlite3 tests come from an old SQLite version being linked against. Until recently, SQLite was buggy and it was only fixed in http://www.sqlite.org/cvstrac/chngview?cn=2981 that callbacks can throw errors that are usefully returned to the original caller. The tests for the sqlite3 module currently assume a recent version SQLite (3.3.something). Otherwise some tests will fail. Still, it can be built against any SQLite 3 release. Can somebody please also verify if the malloc/free error message goes away (it really only happened on Mac, didn't it?) if you upgrade SQLite to the latest version on the build host? -- Gerhard From gh at ghaering.de Wed Jun 14 16:32:16 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 14 Jun 2006 16:32:16 +0200 Subject: [Python-checkins] sqlite3 test errors - was : Re: r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sqlite/module.c Modules/_sqlite/module.h In-Reply-To: <449016F2.7050302@ghaering.de> References: <20060613222453.4561E1E4004@bag.python.org> <1f7befae0606131837x2a4c71bdo588544a759968542@mail.gmail.com> <448FDF37.9020809@ghaering.de> <449016F2.7050302@ghaering.de> Message-ID: <44901DF0.6060404@ghaering.de> Gerhard H?ring wrote: > Co-posting to python-dev in the hope of getting help of people verifying > my suspicion ... > > Gerhard H?ring wrote: >> [...] >> For some reason, they don't seem to have picked up the changed tests of >> the sqlite3 module. At least the error messages look exactly like the >> ones I had when I ran the current code against old tests. > > That guess was wrong. The failed sqlite3 tests come from an old SQLite > version being linked against. Until recently, SQLite was buggy and it > was only fixed in > > http://www.sqlite.org/cvstrac/chngview?cn=2981 > > that callbacks can throw errors that are usefully returned to the > original caller. > > The tests for the sqlite3 module currently assume a recent version > SQLite (3.3.something). Otherwise some tests will fail. > > Still, it can be built against any SQLite 3 release. > > Can somebody please also verify if the malloc/free error message goes > away (it really only happened on Mac, didn't it?) if you upgrade SQLite > to the latest version on the build host? With SQLite 3.2.8, I also get segfaults on Linux x86 (Ubuntu dapper, gcc). I've provided a preliminary patch (cannot check in from this place) that I've attached. Maybe somebody wants to test it, otherwise I'll make a few other tests in the late evening and probably also selectively disable certain tests in the test suite if the SQLite version is too old to pass them. -- Gerhard -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: result_error.patch Url: http://mail.python.org/pipermail/python-checkins/attachments/20060614/978af359/attachment.pot From python-checkins at python.org Wed Jun 14 17:52:33 2006 From: python-checkins at python.org (matt.fleming) Date: Wed, 14 Jun 2006 17:52:33 +0200 (CEST) Subject: [Python-checkins] r46960 - in sandbox/trunk/pdb: mconnection.py test/tcptest.py Message-ID: <20060614155233.90E011E4005@bag.python.org> Author: matt.fleming Date: Wed Jun 14 17:52:33 2006 New Revision: 46960 Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/test/tcptest.py Log: Move import of socket module nearer to the classes that actually use it in mconnection.py. Also change the TCP connection unit test to use threading.Timer instead of os.fork. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Wed Jun 14 17:52:33 2006 @@ -1,7 +1,6 @@ """ This file contains all connections that a debugger can create. """ -import socket NotImplementedMessage = "This method must be overriden in a subclass" @@ -90,6 +89,8 @@ MClientConnectionSerial = MServerConnectionSerial ### This might go in a different file +import socket + class MServerConnectionTCP(MServerConnectionInterface): """This is an implementation of a server class that uses the TCP protocol as its means of communication. Modified: sandbox/trunk/pdb/test/tcptest.py ============================================================================== --- sandbox/trunk/pdb/test/tcptest.py (original) +++ sandbox/trunk/pdb/test/tcptest.py Wed Jun 14 17:52:33 2006 @@ -2,7 +2,7 @@ import os import sys import socket -import time +import threading import unittest __addr__ = 'localhost:8000' @@ -12,26 +12,20 @@ class TestTCPConnections(unittest.TestCase): def testClientConnectAndRead(self): - pid = os.fork() - if pid == 0: - # Child - self.server = MServerConnectionTCP() - self.server.connect(__addr__) - self.server.write("good") - self.server.disconnect() - else: - # Parent - self.client = MClientConnectionTCP() - for i in range(30): - try: - self.client.connect(__addr__) - break - except socket.error: - pass - time.sleep(5) - line = self.client.readline() - self.assertEqual("good", line, "Could not read from server") - self.client.disconnect() + self.server = MServerConnectionTCP() + self.client = MClientConnectionTCP() + # Create a new thread for the client and delay for 3 seconds + t = threading.Timer(3.0, self.client.connect, [__addr__]) + t.start() + # Start server, which waits for a connection. self.client will connect + # in a few seconds + self.server.connect(__addr__) + + self.server.write("good") + line = self.client.readline() + self.assertEqual("good", line, "Could not read from server") + self.server.disconnect() + self.client.disconnect() if __name__ == '__main__': unittest.main() From python-checkins at python.org Wed Jun 14 18:46:44 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 14 Jun 2006 18:46:44 +0200 (CEST) Subject: [Python-checkins] r46961 - python/trunk/Doc/api/concrete.tex Message-ID: <20060614164644.063191E4005@bag.python.org> Author: georg.brandl Date: Wed Jun 14 18:46:43 2006 New Revision: 46961 Modified: python/trunk/Doc/api/concrete.tex Log: Bug #805015: doc error in PyUnicode_FromEncodedObject. Modified: python/trunk/Doc/api/concrete.tex ============================================================================== --- python/trunk/Doc/api/concrete.tex (original) +++ python/trunk/Doc/api/concrete.tex Wed Jun 14 18:46:43 2006 @@ -1001,21 +1001,14 @@ const char *errors} Coerce an encoded object \var{obj} to an Unicode object and return a reference with incremented refcount. + + String and other char buffer compatible objects are decoded + according to the given encoding and using the error handling + defined by errors. Both can be \NULL{} to have the interface + use the default values (see the next section for details). - Coercion is done in the following way: - -\begin{enumerate} -\item Unicode objects are passed back as-is with incremented - refcount. \note{These cannot be decoded; passing a non-\NULL{} - value for encoding will result in a \exception{TypeError}.} - -\item String and other char buffer compatible objects are decoded - according to the given encoding and using the error handling - defined by errors. Both can be \NULL{} to have the interface - use the default values (see the next section for details). - -\item All other objects cause an exception. -\end{enumerate} + All other objects, including Unicode objects, cause a + \exception{TypeError} to be set. The API returns \NULL{} if there was an error. The caller is responsible for decref'ing the returned objects. From brett at python.org Wed Jun 14 19:04:45 2006 From: brett at python.org (Brett Cannon) Date: Wed, 14 Jun 2006 10:04:45 -0700 Subject: [Python-checkins] sqlite3 test errors - was : Re: r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sqlite/module.c Message-ID: On 6/14/06, Gerhard H?ring wrote: > > Co-posting to python-dev in the hope of getting help of people verifying > my suspicion ... > > Gerhard H?ring wrote: > > [...] > > For some reason, they don't seem to have picked up the changed tests of > > the sqlite3 module. At least the error messages look exactly like the > > ones I had when I ran the current code against old tests. > > That guess was wrong. The failed sqlite3 tests come from an old SQLite > version being linked against. Until recently, SQLite was buggy and it > was only fixed in > > http://www.sqlite.org/cvstrac/chngview?cn=2981 > > that callbacks can throw errors that are usefully returned to the > original caller. > > The tests for the sqlite3 module currently assume a recent version > SQLite (3.3.something). Otherwise some tests will fail. > > Still, it can be built against any SQLite 3 release. Perhaps this is the wrong thing to do. Either sqlite3 should not build or raise an exception if it is built against a buggy version. The other option is to not run those known failing tests or error out immediately with a message stating that this tests fails on known versions of sqlite. -Brett Can somebody please also verify if the malloc/free error message goes > away (it really only happened on Mac, didn't it?) if you upgrade SQLite > to the latest version on the build host? > > -- Gerhard > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060614/377d452f/attachment.htm From buildbot at python.org Wed Jun 14 19:26:38 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 17:26:38 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060614172638.F0C7B1E4008@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/363 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From neal at metaslash.com Wed Jun 14 22:13:37 2006 From: neal at metaslash.com (Neal Norwitz) Date: Wed, 14 Jun 2006 16:13:37 -0400 Subject: [Python-checkins] Python Regression Test Failures basics (2) Message-ID: <20060614201337.GA2191@python.psfb.org> case $MAKEFLAGS in \ *-s*) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py -q build;; \ *) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py build;; \ esac running build running build_ext db.h: found (4, 1) in /usr/include db lib: using (4, 1) db-4.1 sqlite: found /usr/include/sqlite3.h /usr/include/sqlite3.h: version 3.2.1 INFO: Can't locate Tcl/Tk libs and/or headers running build_scripts [34100 refs] ./python -E -c 'import sys ; from distutils.util import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform [8686 refs] find ./Lib -name '*.py[co]' -print | xargs rm -f ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test test_sqlite failed -- errors occurred; run in verbose mode for details test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 286 tests OK. 1 test failed: test_sqlite 30 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [432972 refs] make: [test] Error 1 (ignored) ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test test_sqlite failed -- errors occurred; run in verbose mode for details test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 286 tests OK. 1 test failed: test_sqlite 30 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [432373 refs] make: *** [test] Error 1 From neal at metaslash.com Wed Jun 14 23:22:38 2006 From: neal at metaslash.com (Neal Norwitz) Date: Wed, 14 Jun 2006 17:22:38 -0400 Subject: [Python-checkins] Python Regression Test Failures all (1) Message-ID: <20060614212238.GA7763@python.psfb.org> test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socketserver test_softspace test_sort test_sqlite test test_sqlite failed -- errors occurred; run in verbose mode for details test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllibnet test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 292 tests OK. 1 test failed: test_sqlite 21 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl test_gl test_imgfile test_ioctl test_macfs test_macostools test_nis test_pep277 test_plistlib test_scriptpackages test_startfile test_sunaudiodev test_tcl test_unicode_file test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [440391 refs] From tim.peters at gmail.com Thu Jun 15 00:04:15 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 14 Jun 2006 18:04:15 -0400 Subject: [Python-checkins] [Python-Dev] sqlite3 test errors - was : Re: r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sql Message-ID: <1f7befae0606141504p194987b0q668d4203cd039392@mail.gmail.com> [Gerhard H?ring] >> ... >> Until recently, SQLite was buggy and it was only fixed in >> >> http://www.sqlite.org/cvstrac/chngview?cn=2981 >> >> that callbacks can throw errors that are usefully returned to the >> original caller. >> >> The tests for the sqlite3 module currently assume a recent version >> SQLite (3.3.something). Otherwise some tests will fail. Sounds like that explains why none of the Windows buildbots failed (on Windows, Python currently uses the sqlite checked in at http://svn.python.org/projects/external/sqlite-source-3.3.4 ). I suppose some OSes think they're doing you a favor by forcing you to be the system SQLite admin ;-) From python-checkins at python.org Thu Jun 15 00:28:39 2006 From: python-checkins at python.org (gerhard.haering) Date: Thu, 15 Jun 2006 00:28:39 +0200 (CEST) Subject: [Python-checkins] r46962 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Message-ID: <20060614222839.7CCDE1E4005@bag.python.org> Author: gerhard.haering Date: Thu Jun 15 00:28:37 2006 New Revision: 46962 Modified: python/trunk/Lib/sqlite3/test/hooks.py python/trunk/Lib/sqlite3/test/userfunctions.py python/trunk/Modules/_sqlite/connection.c Log: - Added version checks in C code to make sure we don't trigger bugs in older SQLite versions. - Added version checks in test suite so that we don't execute tests that we know will fail with older (buggy) SQLite versions. Now, all tests should run against all SQLite versions from 3.0.8 until 3.3.6 (latest one now). The sqlite3 module can be built against all these SQLite versions and the sqlite3 module does its best to not trigger bugs in SQLite, but using SQLite 3.3.3 or later is recommended. Modified: python/trunk/Lib/sqlite3/test/hooks.py ============================================================================== --- python/trunk/Lib/sqlite3/test/hooks.py (original) +++ python/trunk/Lib/sqlite3/test/hooks.py Thu Jun 15 00:28:37 2006 @@ -48,6 +48,8 @@ pass def CheckCollationIsUsed(self): + if sqlite.version_info < (3, 2, 1): # old SQLite versions crash on this test + return def mycoll(x, y): # reverse order return -cmp(x, y) Modified: python/trunk/Lib/sqlite3/test/userfunctions.py ============================================================================== --- python/trunk/Lib/sqlite3/test/userfunctions.py (original) +++ python/trunk/Lib/sqlite3/test/userfunctions.py Thu Jun 15 00:28:37 2006 @@ -200,6 +200,8 @@ self.failUnlessEqual(val, buffer("blob")) def CheckFuncException(self): + if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions + return cur = self.con.cursor() try: cur.execute("select raiseexception()") @@ -283,6 +285,8 @@ self.failUnlessEqual(e.args[0], "AggrNoStep instance has no attribute 'step'") def CheckAggrNoFinalize(self): + if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions + return cur = self.con.cursor() try: cur.execute("select nofinalize(t) from test") @@ -292,6 +296,8 @@ self.failUnlessEqual(e.args[0], "user-defined aggregate's 'finalize' method raised error") def CheckAggrExceptionInInit(self): + if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions + return cur = self.con.cursor() try: cur.execute("select excInit(t) from test") @@ -301,6 +307,8 @@ self.failUnlessEqual(e.args[0], "user-defined aggregate's '__init__' method raised error") def CheckAggrExceptionInStep(self): + if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions + return cur = self.con.cursor() try: cur.execute("select excStep(t) from test") @@ -310,6 +318,8 @@ self.failUnlessEqual(e.args[0], "user-defined aggregate's 'step' method raised error") def CheckAggrExceptionInFinalize(self): + if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions + return cur = self.con.cursor() try: cur.execute("select excFinalize(t) from test") Modified: python/trunk/Modules/_sqlite/connection.c ============================================================================== --- python/trunk/Modules/_sqlite/connection.c (original) +++ python/trunk/Modules/_sqlite/connection.c Thu Jun 15 00:28:37 2006 @@ -34,6 +34,17 @@ static int connection_set_isolation_level(Connection* self, PyObject* isolation_level); + +void _sqlite3_result_error(sqlite3_context* ctx, const char* errmsg, int len) +{ + /* in older SQLite versions, calling sqlite3_result_error in callbacks + * triggers a bug in SQLite that leads either to irritating results or + * segfaults, depending on the SQLite version */ +#if SQLITE_VERSION_NUMBER >= 3003003 + sqlite3_result_error(ctx, errmsg, len); +#endif +} + int connection_init(Connection* self, PyObject* args, PyObject* kwargs) { static char *kwlist[] = {"database", "timeout", "detect_types", "isolation_level", "check_same_thread", "factory", "cached_statements", NULL, NULL}; @@ -526,7 +537,7 @@ } else { PyErr_Clear(); } - sqlite3_result_error(context, "user-defined function raised exception", -1); + _sqlite3_result_error(context, "user-defined function raised exception", -1); } PyGILState_Release(threadstate); @@ -558,7 +569,7 @@ } else { PyErr_Clear(); } - sqlite3_result_error(context, "user-defined aggregate's '__init__' method raised error", -1); + _sqlite3_result_error(context, "user-defined aggregate's '__init__' method raised error", -1); goto error; } } @@ -582,7 +593,7 @@ } else { PyErr_Clear(); } - sqlite3_result_error(context, "user-defined aggregate's 'step' method raised error", -1); + _sqlite3_result_error(context, "user-defined aggregate's 'step' method raised error", -1); } error: @@ -619,7 +630,7 @@ } else { PyErr_Clear(); } - sqlite3_result_error(context, "user-defined aggregate's 'finalize' method raised error", -1); + _sqlite3_result_error(context, "user-defined aggregate's 'finalize' method raised error", -1); } else { _set_result(context, function_result); } From gh at ghaering.de Thu Jun 15 00:37:21 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Thu, 15 Jun 2006 00:37:21 +0200 Subject: [Python-checkins] [Python-Dev] sqlite3 test errors - was : Re: r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sql In-Reply-To: <1f7befae0606141504p194987b0q668d4203cd039392@mail.gmail.com> References: <1f7befae0606141504p194987b0q668d4203cd039392@mail.gmail.com> Message-ID: <44908FA1.5080008@ghaering.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Peters wrote: > [Gerhard H?ring] >>> ... >>> Until recently, SQLite was buggy and it was only fixed in >>> >>> http://www.sqlite.org/cvstrac/chngview?cn=2981 >>> >>> that callbacks can throw errors that are usefully returned to the >>> original caller. >>> >>> The tests for the sqlite3 module currently assume a recent version >>> SQLite (3.3.something). Otherwise some tests will fail. > > Sounds like that explains why none of the Windows buildbots failed (on > Windows, Python currently uses the sqlite checked in at > > http://svn.python.org/projects/external/sqlite-source-3.3.4 > > ). I suppose some OSes think they're doing you a favor by forcing you > to be the system SQLite admin ;-) Yes, this issue made development of the SQLite module a bit more "interesting" for me. And I deserved no better for committing changes so late before beta1. Anyway, I verified my theory about the SQLite bugs (that's what they are), and added version checks to the C code and to the test suite, so now Everything Should Work (*crossing fingers*). If anything should still fail, I'll ruthlessly blame Anthony, he brought the idea up of supporting older SQLite versions in the first place :-P - -- Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEkI+hdIO4ozGCH14RAlW/AJ4uQVZrvWC7265+9wshxaBotyLolgCgstKd 5xU5DZm1EC/G9qNctPlMcGc= =gaaO -----END PGP SIGNATURE----- From python-checkins at python.org Thu Jun 15 00:38:13 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 15 Jun 2006 00:38:13 +0200 (CEST) Subject: [Python-checkins] r46963 - python/trunk/Lib/SimpleHTTPServer.py Message-ID: <20060614223813.7977D1E4005@bag.python.org> Author: tim.peters Date: Thu Jun 15 00:38:13 2006 New Revision: 46963 Modified: python/trunk/Lib/SimpleHTTPServer.py Log: Whitespace normalization. Modified: python/trunk/Lib/SimpleHTTPServer.py ============================================================================== --- python/trunk/Lib/SimpleHTTPServer.py (original) +++ python/trunk/Lib/SimpleHTTPServer.py Thu Jun 15 00:38:13 2006 @@ -191,7 +191,7 @@ return self.extensions_map[ext] else: return self.extensions_map[''] - + if not mimetypes.inited: mimetypes.init() # try to read system mime.types extensions_map = mimetypes.types_map.copy() From buildbot at python.org Thu Jun 15 00:52:06 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 22:52:06 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP-2 trunk Message-ID: <20060614225207.0F6A71E4005@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-2%2520trunk/builds/649 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gerhard.haering Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From anthony at interlink.com.au Thu Jun 15 01:04:00 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 15 Jun 2006 09:04:00 +1000 Subject: [Python-checkins] =?iso-8859-1?q?=5BPython-Dev=5D__sqlite3_test_e?= =?iso-8859-1?q?rrors_-_was_=3A_Re=3A_r46936_-_in_python/trunk=3A_Lib/sqli?= =?iso-8859-1?q?te3/test/regression=2Epy=09Lib/sqlite3/test/types=2Epy_Lib?= =?iso-8859-1?q?/sqlite3/test/userfunctions=2Epy=09Modules/=5Fsqlite/conne?= =?iso-8859-1?q?ction=2Ec_Modules/=5Fsqlite/cursor=2Ec_Modules/=5Fsql?= In-Reply-To: <44908FA1.5080008@ghaering.de> References: <1f7befae0606141504p194987b0q668d4203cd039392@mail.gmail.com> <44908FA1.5080008@ghaering.de> Message-ID: <200606150904.03221.anthony@interlink.com.au> Well, the just-released Ubuntu 06.06 LTS (Long Term Support) ships with sqlite 3.2.8. I'd suggest that whatever version ships with Python should _at_ _least_ work with this version. 06.06 is supposed to be supported for a couple of years, at least. Since this is the latest and greatest version of what's probably the most rapidly updating Linux (I don't include gentoo, obviously, because gentoo scares me (ObBarryBaiting: funroll-loops.org)), I don't think we can expect many people's platforms to have the absolute latest 3.4.4 or whatever. Alternately, we ship the sqlite3 code with Python. I'm deeply unthrilled with this idea, as it means more emergency security releases to fix sqlite3 security bugs, as well as bloating out the size of the release. In the meantime, I'd suggest the appropriate fix is to roll back to the previous version of the python-sqlite bindings on the trunk. Anthony From buildbot at python.org Thu Jun 15 01:59:13 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 14 Jun 2006 23:59:13 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060614235913.E9AE91E4014@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/668 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 02:42:35 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 00:42:35 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060615004235.AA07E1E4006@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/365 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From anthony at interlink.com.au Thu Jun 15 03:46:17 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 15 Jun 2006 11:46:17 +1000 Subject: [Python-checkins] r46962 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c In-Reply-To: <20060614222839.7CCDE1E4005@bag.python.org> References: <20060614222839.7CCDE1E4005@bag.python.org> Message-ID: <200606151146.19472.anthony@interlink.com.au> On Thursday 15 June 2006 08:28, gerhard.haering wrote: > - Added version checks in C code to make sure we don't trigger bugs > in older SQLite versions. > - Added version checks in test suite so that we don't execute tests > that we know will fail with older (buggy) SQLite versions. > > Now, all tests should run against all SQLite versions from 3.0.8 > until 3.3.6 (latest one now). The sqlite3 module can be built > against all these SQLite versions and the sqlite3 module does its > best to not trigger bugs in SQLite, but using SQLite 3.3.3 or later > is recommended. > +void _sqlite3_result_error(sqlite3_context* ctx, const char* > errmsg, int len) +{ > + /* in older SQLite versions, calling sqlite3_result_error in > callbacks + * triggers a bug in SQLite that leads either to > irritating results or + * segfaults, depending on the SQLite > version */ > +#if SQLITE_VERSION_NUMBER >= 3003003 > + sqlite3_result_error(ctx, errmsg, len); > +#endif Could this code at least set a generic error message, saying something like "Unknown SQLite error in callback (see documentation for explanation)" and have a note in the docs saying "SQLite older than 3.3.3 is buggy and can't give useful callback errors". Hm. Is the underlying sqlite3 version number even exported to Python? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From python-checkins at python.org Thu Jun 15 06:54:32 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 15 Jun 2006 06:54:32 +0200 (CEST) Subject: [Python-checkins] r46964 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Message-ID: <20060615045432.361041E4005@bag.python.org> Author: neal.norwitz Date: Thu Jun 15 06:54:29 2006 New Revision: 46964 Modified: python/trunk/Lib/sqlite3/test/hooks.py python/trunk/Lib/sqlite3/test/userfunctions.py python/trunk/Modules/_sqlite/connection.c Log: Speculative checkin (requires approval of Gerhard Haering) This backs out the test changes in 46962 which prevented crashes by not running the tests via a version check. All the version checks added in that rev were removed from the tests. Code was added to the error handler in connection.c that seems to work with older versions of sqlite including 3.1.3. Modified: python/trunk/Lib/sqlite3/test/hooks.py ============================================================================== --- python/trunk/Lib/sqlite3/test/hooks.py (original) +++ python/trunk/Lib/sqlite3/test/hooks.py Thu Jun 15 06:54:29 2006 @@ -48,8 +48,6 @@ pass def CheckCollationIsUsed(self): - if sqlite.version_info < (3, 2, 1): # old SQLite versions crash on this test - return def mycoll(x, y): # reverse order return -cmp(x, y) Modified: python/trunk/Lib/sqlite3/test/userfunctions.py ============================================================================== --- python/trunk/Lib/sqlite3/test/userfunctions.py (original) +++ python/trunk/Lib/sqlite3/test/userfunctions.py Thu Jun 15 06:54:29 2006 @@ -200,8 +200,6 @@ self.failUnlessEqual(val, buffer("blob")) def CheckFuncException(self): - if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions - return cur = self.con.cursor() try: cur.execute("select raiseexception()") @@ -285,8 +283,6 @@ self.failUnlessEqual(e.args[0], "AggrNoStep instance has no attribute 'step'") def CheckAggrNoFinalize(self): - if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions - return cur = self.con.cursor() try: cur.execute("select nofinalize(t) from test") @@ -296,8 +292,6 @@ self.failUnlessEqual(e.args[0], "user-defined aggregate's 'finalize' method raised error") def CheckAggrExceptionInInit(self): - if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions - return cur = self.con.cursor() try: cur.execute("select excInit(t) from test") @@ -307,8 +301,6 @@ self.failUnlessEqual(e.args[0], "user-defined aggregate's '__init__' method raised error") def CheckAggrExceptionInStep(self): - if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions - return cur = self.con.cursor() try: cur.execute("select excStep(t) from test") @@ -318,8 +310,6 @@ self.failUnlessEqual(e.args[0], "user-defined aggregate's 'step' method raised error") def CheckAggrExceptionInFinalize(self): - if sqlite.version_info < (3, 3, 3): # don't raise bug in earlier SQLite versions - return cur = self.con.cursor() try: cur.execute("select excFinalize(t) from test") Modified: python/trunk/Modules/_sqlite/connection.c ============================================================================== --- python/trunk/Modules/_sqlite/connection.c (original) +++ python/trunk/Modules/_sqlite/connection.c Thu Jun 15 06:54:29 2006 @@ -42,6 +42,8 @@ * segfaults, depending on the SQLite version */ #if SQLITE_VERSION_NUMBER >= 3003003 sqlite3_result_error(ctx, errmsg, len); +#else + PyErr_SetString(OperationalError, errmsg); #endif } From neal at metaslash.com Thu Jun 15 07:02:51 2006 From: neal at metaslash.com (Neal Norwitz) Date: Thu, 15 Jun 2006 01:02:51 -0400 Subject: [Python-checkins] Python Regression Test Failures all (1) Message-ID: <20060615050251.GA25516@python.psfb.org> test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socketserver test_softspace test_sort test_sqlite test test_sqlite failed -- errors occurred; run in verbose mode for details test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllibnet test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile test_zipimport test_zlib 292 tests OK. 1 test failed: test_sqlite 21 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl test_gl test_imgfile test_ioctl test_macfs test_macostools test_nis test_pep277 test_plistlib test_scriptpackages test_startfile test_sunaudiodev test_tcl test_unicode_file test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [440391 refs] From buildbot at python.org Thu Jun 15 07:50:38 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 05:50:38 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060615055038.8CF0D1E4015@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/187 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 15 07:55:55 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 15 Jun 2006 07:55:55 +0200 (CEST) Subject: [Python-checkins] r46965 - python/trunk/Lib/test/test_time.py Message-ID: <20060615055555.34A231E4005@bag.python.org> Author: neal.norwitz Date: Thu Jun 15 07:55:49 2006 New Revision: 46965 Modified: python/trunk/Lib/test/test_time.py Log: Try to narrow window of failure on slow/busy boxes (ppc64 buildbot) Modified: python/trunk/Lib/test/test_time.py ============================================================================== --- python/trunk/Lib/test/test_time.py (original) +++ python/trunk/Lib/test/test_time.py Thu Jun 15 07:55:49 2006 @@ -193,13 +193,17 @@ time.ctime(None) def test_gmtime_without_arg(self): - t0 = time.mktime(time.gmtime()) - t1 = time.mktime(time.gmtime(None)) + gt0 = time.gmtime() + gt1 = time.gmtime(None) + t0 = time.mktime(gt0) + t1 = time.mktime(gt1) self.assert_(0 <= (t1-t0) < 0.2) def test_localtime_without_arg(self): - t0 = time.mktime(time.localtime()) - t1 = time.mktime(time.localtime(None)) + lt0 = time.localtime() + lt1 = time.localtime(None) + t0 = time.mktime(lt0) + t1 = time.mktime(lt1) self.assert_(0 <= (t1-t0) < 0.2) def test_main(): From gh at ghaering.de Thu Jun 15 07:57:17 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Thu, 15 Jun 2006 07:57:17 +0200 Subject: [Python-checkins] r46962 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c In-Reply-To: <200606151146.19472.anthony@interlink.com.au> References: <20060614222839.7CCDE1E4005@bag.python.org> <200606151146.19472.anthony@interlink.com.au> Message-ID: <4490F6BD.7080007@ghaering.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Anthony Baxter wrote: > On Thursday 15 June 2006 08:28, gerhard.haering wrote: > >> - Added version checks in C code to make sure we don't trigger bugs >> in older SQLite versions. >> - Added version checks in test suite so that we don't execute tests >> that we know will fail with older (buggy) SQLite versions. >> >> Now, all tests should run against all SQLite versions from 3.0.8 >> until 3.3.6 (latest one now). The sqlite3 module can be built >> against all these SQLite versions and the sqlite3 module does its >> best to not trigger bugs in SQLite, but using SQLite 3.3.3 or later >> is recommended. > >> +void _sqlite3_result_error(sqlite3_context* ctx, const char* >> errmsg, int len) +{ >> + /* in older SQLite versions, calling sqlite3_result_error in >> callbacks + * triggers a bug in SQLite that leads either to >> irritating results or + * segfaults, depending on the SQLite >> version */ >> +#if SQLITE_VERSION_NUMBER >= 3003003 >> + sqlite3_result_error(ctx, errmsg, len); >> +#endif > > Could this code at least set a generic error message, saying something > like "Unknown SQLite error in callback (see documentation for > explanation)" and have a note in the docs saying "SQLite older than > 3.3.3 is buggy and can't give useful callback errors". That's not possible in a straightforward manner, because the natural way of doing this requires a "sqlite3_result_error" that's not buggy :-/ Otherwise I could probably go the long-workaround-way by storing a reference to the Connection object in the SQLite context and then setting a new variable char* callback_error on the Connection object in case. That requires a bit more testing than is possible before beta1 and I don't like the idea of putting in a whole lot of extra code for working around SQLite bugs. FWIW the old pysqlite code simply ignored any errors in callbacks in the first place. And this is what we now still do for the old, buggy SQLite versions. > Hm. Is the underlying sqlite3 version number even exported to Python? Yes, as sqlite_version and sqlite_version_info. - -- Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEkPa9dIO4ozGCH14RAmCzAKCFhf7e3jr9eF6MozmU1/z9jwXvZwCcCVDe nJ/1FGUkDDh0aSU0zyG+aO4= =S06c -----END PGP SIGNATURE----- From nnorwitz at gmail.com Thu Jun 15 07:59:20 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 14 Jun 2006 22:59:20 -0700 Subject: [Python-checkins] r46962 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c In-Reply-To: <4490F6BD.7080007@ghaering.de> References: <20060614222839.7CCDE1E4005@bag.python.org> <200606151146.19472.anthony@interlink.com.au> <4490F6BD.7080007@ghaering.de> Message-ID: Can you take a look at my checkin? It only added 2 lines and the tests pass after removing the version checks. I don't know if there's a good reason to *not* do the fix I did. n -- On 6/14/06, Gerhard H?ring wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Anthony Baxter wrote: > > On Thursday 15 June 2006 08:28, gerhard.haering wrote: > > > >> - Added version checks in C code to make sure we don't trigger bugs > >> in older SQLite versions. > >> - Added version checks in test suite so that we don't execute tests > >> that we know will fail with older (buggy) SQLite versions. > >> > >> Now, all tests should run against all SQLite versions from 3.0.8 > >> until 3.3.6 (latest one now). The sqlite3 module can be built > >> against all these SQLite versions and the sqlite3 module does its > >> best to not trigger bugs in SQLite, but using SQLite 3.3.3 or later > >> is recommended. > > > >> +void _sqlite3_result_error(sqlite3_context* ctx, const char* > >> errmsg, int len) +{ > >> + /* in older SQLite versions, calling sqlite3_result_error in > >> callbacks + * triggers a bug in SQLite that leads either to > >> irritating results or + * segfaults, depending on the SQLite > >> version */ > >> +#if SQLITE_VERSION_NUMBER >= 3003003 > >> + sqlite3_result_error(ctx, errmsg, len); > >> +#endif > > > > Could this code at least set a generic error message, saying something > > like "Unknown SQLite error in callback (see documentation for > > explanation)" and have a note in the docs saying "SQLite older than > > 3.3.3 is buggy and can't give useful callback errors". > > That's not possible in a straightforward manner, because the natural way of > doing this requires a "sqlite3_result_error" that's not buggy :-/ > > Otherwise I could probably go the long-workaround-way by storing a > reference to the Connection object in the SQLite context and then setting a > new variable char* callback_error on the Connection object in case. > > That requires a bit more testing than is possible before beta1 and I don't > like the idea of putting in a whole lot of extra code for working around > SQLite bugs. > > FWIW the old pysqlite code simply ignored any errors in callbacks in the > first place. And this is what we now still do for the old, buggy SQLite > versions. > > > Hm. Is the underlying sqlite3 version number even exported to Python? > > Yes, as sqlite_version and sqlite_version_info. > > - -- Gerhard > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2.2 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFEkPa9dIO4ozGCH14RAmCzAKCFhf7e3jr9eF6MozmU1/z9jwXvZwCcCVDe > nJ/1FGUkDDh0aSU0zyG+aO4= > =S06c > -----END PGP SIGNATURE----- > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From nnorwitz at gmail.com Thu Jun 15 08:00:20 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 14 Jun 2006 23:00:20 -0700 Subject: [Python-checkins] r46962 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c In-Reply-To: References: <20060614222839.7CCDE1E4005@bag.python.org> <200606151146.19472.anthony@interlink.com.au> <4490F6BD.7080007@ghaering.de> Message-ID: I forgot to mention that all the builbots (even one on 3.1.3 or is that my laptop) pass with the change I checked in. probably rev 26964. n -- On 6/14/06, Neal Norwitz wrote: > Can you take a look at my checkin? It only added 2 lines and the > tests pass after removing the version checks. I don't know if there's > a good reason to *not* do the fix I did. > > n > -- > > On 6/14/06, Gerhard H?ring wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Anthony Baxter wrote: > > > On Thursday 15 June 2006 08:28, gerhard.haering wrote: > > > > > >> - Added version checks in C code to make sure we don't trigger bugs > > >> in older SQLite versions. > > >> - Added version checks in test suite so that we don't execute tests > > >> that we know will fail with older (buggy) SQLite versions. > > >> > > >> Now, all tests should run against all SQLite versions from 3.0.8 > > >> until 3.3.6 (latest one now). The sqlite3 module can be built > > >> against all these SQLite versions and the sqlite3 module does its > > >> best to not trigger bugs in SQLite, but using SQLite 3.3.3 or later > > >> is recommended. > > > > > >> +void _sqlite3_result_error(sqlite3_context* ctx, const char* > > >> errmsg, int len) +{ > > >> + /* in older SQLite versions, calling sqlite3_result_error in > > >> callbacks + * triggers a bug in SQLite that leads either to > > >> irritating results or + * segfaults, depending on the SQLite > > >> version */ > > >> +#if SQLITE_VERSION_NUMBER >= 3003003 > > >> + sqlite3_result_error(ctx, errmsg, len); > > >> +#endif > > > > > > Could this code at least set a generic error message, saying something > > > like "Unknown SQLite error in callback (see documentation for > > > explanation)" and have a note in the docs saying "SQLite older than > > > 3.3.3 is buggy and can't give useful callback errors". > > > > That's not possible in a straightforward manner, because the natural way of > > doing this requires a "sqlite3_result_error" that's not buggy :-/ > > > > Otherwise I could probably go the long-workaround-way by storing a > > reference to the Connection object in the SQLite context and then setting a > > new variable char* callback_error on the Connection object in case. > > > > That requires a bit more testing than is possible before beta1 and I don't > > like the idea of putting in a whole lot of extra code for working around > > SQLite bugs. > > > > FWIW the old pysqlite code simply ignored any errors in callbacks in the > > first place. And this is what we now still do for the old, buggy SQLite > > versions. > > > > > Hm. Is the underlying sqlite3 version number even exported to Python? > > > > Yes, as sqlite_version and sqlite_version_info. > > > > - -- Gerhard > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG v1.4.2.2 (GNU/Linux) > > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > > > iD8DBQFEkPa9dIO4ozGCH14RAmCzAKCFhf7e3jr9eF6MozmU1/z9jwXvZwCcCVDe > > nJ/1FGUkDDh0aSU0zyG+aO4= > > =S06c > > -----END PGP SIGNATURE----- > > _______________________________________________ > > Python-checkins mailing list > > Python-checkins at python.org > > http://mail.python.org/mailman/listinfo/python-checkins > > > From gh at ghaering.de Thu Jun 15 08:02:00 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Thu, 15 Jun 2006 08:02:00 +0200 Subject: [Python-checkins] [Python-Dev] sqlite3 test errors - was : Re: r46936 - in python/trunk: Lib/sqlite3/test/regression.py Lib/sqlite3/test/types.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c Modules/_sqlite/cursor.c Modules/_sql In-Reply-To: <200606150904.03221.anthony@interlink.com.au> References: <1f7befae0606141504p194987b0q668d4203cd039392@mail.gmail.com> <44908FA1.5080008@ghaering.de> <200606150904.03221.anthony@interlink.com.au> Message-ID: <4490F7D8.20101@ghaering.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Anthony Baxter wrote: > Well, the just-released Ubuntu 06.06 LTS (Long Term Support) ships > with sqlite 3.2.8. I'd suggest that whatever version ships with > Python should _at_ _least_ work with this version. I have no problems continuing to support any halfway sane version, and that is 3.0.8 or later (AFAIR the first non-beta SQLite 3.x release anyway). It just requires a bit more testing. > [...] > Alternately, we ship the sqlite3 code with Python. I'm deeply > unthrilled with this idea, as it means more emergency security > releases to fix sqlite3 security bugs, as well as bloating out the > size of the release. No, that's not an option. > In the meantime, I'd suggest the appropriate fix is to roll back to > the previous version of the python-sqlite bindings on the trunk. Why? The previous version is really no better, except it ignores errors in callbacks for all SQLite versions, instead of doing something useful and aborting the query with a useful error message for those SQLite versions that support it. - -- Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD4DBQFEkPfYdIO4ozGCH14RAvQ+AJdxy8Iy0sfkSQVxShmGbq/HGKRzAKCPKMtG ZoEqmcNrgMX6k/7xzy0HKA== =OeDy -----END PGP SIGNATURE----- From buildbot at python.org Thu Jun 15 08:39:42 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 06:39:42 +0000 Subject: [Python-checkins] buildbot failure in amd64 gentoo trunk Message-ID: <20060615063942.B4F531E4005@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1075 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 08:43:25 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 06:43:25 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060615064325.E9D4A1E4005@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/773 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 15 08:45:08 2006 From: python-checkins at python.org (martin.v.loewis) Date: Thu, 15 Jun 2006 08:45:08 +0200 (CEST) Subject: [Python-checkins] r46966 - python/trunk/Lib/encodings/mbcs.py Message-ID: <20060615064508.D9D7C1E4005@bag.python.org> Author: martin.v.loewis Date: Thu Jun 15 08:45:05 2006 New Revision: 46966 Modified: python/trunk/Lib/encodings/mbcs.py Log: Make import/lookup of mbcs fail on non-Windows systems. Modified: python/trunk/Lib/encodings/mbcs.py ============================================================================== --- python/trunk/Lib/encodings/mbcs.py (original) +++ python/trunk/Lib/encodings/mbcs.py Thu Jun 15 08:45:05 2006 @@ -7,6 +7,10 @@ (c) Copyright CNRI, All Rights Reserved. NO WARRANTY. """ +# Import them explicitly to cause an ImportError +# on non-Windows systems +from codecs import mbcs_encode, mbcs_decode +# for IncrementalDecoder, IncrementalEncoder, ... import codecs ### Codec APIs @@ -15,16 +19,16 @@ # Note: Binding these as C functions will result in the class not # converting them to methods. This is intended. - encode = codecs.mbcs_encode - decode = codecs.mbcs_decode + encode = mbcs_encode + decode = mbcs_decode class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.mbcs_encode(input,self.errors)[0] + return mbcs_encode(input,self.errors)[0] class IncrementalDecoder(codecs.BufferedIncrementalDecoder): def _buffer_decode(self, input, errors, final): - return codecs.mbcs_decode(input,self.errors,final) + return mbcs_decode(input,self.errors,final) class StreamWriter(Codec,codecs.StreamWriter): pass From buildbot at python.org Thu Jun 15 09:44:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 07:44:17 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060615074417.30DA11E4005@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/671 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From gh at ghaering.de Thu Jun 15 10:05:35 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Thu, 15 Jun 2006 10:05:35 +0200 Subject: [Python-checkins] r46962 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c In-Reply-To: References: <20060614222839.7CCDE1E4005@bag.python.org> <200606151146.19472.anthony@interlink.com.au> <4490F6BD.7080007@ghaering.de> Message-ID: <449114CF.3020602@ghaering.de> Neal Norwitz wrote: > I forgot to mention that all the builbots (even one on 3.1.3 or is > that my laptop) pass with the change I checked in. probably rev > 26964. Ok, but I believe they passed all tests before your checkin, too. > On 6/14/06, Neal Norwitz wrote: >> Can you take a look at my checkin? It only added 2 lines and the >> tests pass after removing the version checks. I don't know if there's >> a good reason to *not* do the fix I did. Doing it this way is a possible approach that should work ok, and it's probably a good one, too. It is functionally different from the real thing (sqlite3_result_error) though, because the query in progress is not aborted. If you want to keep your checkin, please restore this change though: http://svn.python.org/view/python/trunk/Lib/sqlite3/test/hooks.py?rev=46964&r1=46962&r2=46964 it addresses a different problem that I haven't fully debugged yet. All I know is that without it, the test suite will crash for SQLite 3.0.8 and not for 3.2.1 and later. It's most likely another SQLite bug of some sort. -- Gerhard From python-checkins at python.org Thu Jun 15 10:14:23 2006 From: python-checkins at python.org (ronald.oussoren) Date: Thu, 15 Jun 2006 10:14:23 +0200 (CEST) Subject: [Python-checkins] r46967 - in python/trunk: Doc/lib/libzipfile.tex Lib/test/test_zipfile.py Lib/test/test_zipfile64.py Lib/zipfile.py Misc/NEWS Message-ID: <20060615081423.8126C1E4005@bag.python.org> Author: ronald.oussoren Date: Thu Jun 15 10:14:18 2006 New Revision: 46967 Added: python/trunk/Lib/test/test_zipfile64.py Modified: python/trunk/Doc/lib/libzipfile.tex python/trunk/Lib/test/test_zipfile.py python/trunk/Lib/zipfile.py python/trunk/Misc/NEWS Log: Patch #1446489 (zipfile: support for ZIP64) Modified: python/trunk/Doc/lib/libzipfile.tex ============================================================================== --- python/trunk/Doc/lib/libzipfile.tex (original) +++ python/trunk/Doc/lib/libzipfile.tex Thu Jun 15 10:14:18 2006 @@ -17,7 +17,8 @@ Note}. This module does not currently handle ZIP files which have appended -comments, or multi-disk ZIP files. +comments, or multi-disk ZIP files. It can handle ZIP files that use the +ZIP64 extensions (that is ZIP files that are more than 4 GByte in size). The available attributes of this module are: @@ -25,6 +26,11 @@ The error raised for bad ZIP files. \end{excdesc} +\begin{excdesc}{LargeZipFile} + The error raised when a ZIP file would require ZIP64 functionality but that + has not been enabled. +\end{excdesc} + \begin{classdesc*}{ZipFile} The class for reading and writing ZIP files. See ``\citetitle{ZipFile Objects}'' (section \ref{zipfile-objects}) for @@ -77,7 +83,7 @@ \subsection{ZipFile Objects \label{zipfile-objects}} -\begin{classdesc}{ZipFile}{file\optional{, mode\optional{, compression}}} +\begin{classdesc}{ZipFile}{file\optional{, mode\optional{, compression\optional{, allowZip64}}}} Open a ZIP file, where \var{file} can be either a path to a file (a string) or a file-like object. The \var{mode} parameter should be \code{'r'} to read an existing file, \code{'w'} to @@ -100,6 +106,12 @@ is specified but the \refmodule{zlib} module is not available, \exception{RuntimeError} is also raised. The default is \constant{ZIP_STORED}. + If \var{allowZip64} is \code{True} zipfile will create zipfiles that use + the ZIP64 extensions when the zipfile is larger than 2GBytes. If it is + false (the default) zipfile will raise an exception when the zipfile would + require ZIP64 extensions. ZIP64 extensions are disabled by default because + the default zip and unzip commands on Unix (the InfoZIP utilities) don't + support these extensions. \end{classdesc} \begin{methoddesc}{close}{} @@ -132,8 +144,8 @@ \end{methoddesc} \begin{methoddesc}{testzip}{} - Read all the files in the archive and check their CRC's. Return the - name of the first bad file, or else return \code{None}. + Read all the files in the archive and check their CRC's and file + headers. Return the name of the first bad file, or else return \code{None}. \end{methoddesc} \begin{methoddesc}{write}{filename\optional{, arcname\optional{, @@ -284,10 +296,6 @@ Byte offset to the file header. \end{memberdesc} -\begin{memberdesc}[ZipInfo]{file_offset} - Byte offset to the start of the file data. -\end{memberdesc} - \begin{memberdesc}[ZipInfo]{CRC} CRC-32 of the uncompressed file. \end{memberdesc} Modified: python/trunk/Lib/test/test_zipfile.py ============================================================================== --- python/trunk/Lib/test/test_zipfile.py (original) +++ python/trunk/Lib/test/test_zipfile.py Thu Jun 15 10:14:18 2006 @@ -4,7 +4,7 @@ except ImportError: zlib = None -import zipfile, os, unittest +import zipfile, os, unittest, sys, shutil from StringIO import StringIO from tempfile import TemporaryFile @@ -28,14 +28,70 @@ zipfp = zipfile.ZipFile(f, "w", compression) zipfp.write(TESTFN, "another"+os.extsep+"name") zipfp.write(TESTFN, TESTFN) + zipfp.writestr("strfile", self.data) zipfp.close() # Read the ZIP archive zipfp = zipfile.ZipFile(f, "r", compression) self.assertEqual(zipfp.read(TESTFN), self.data) self.assertEqual(zipfp.read("another"+os.extsep+"name"), self.data) + self.assertEqual(zipfp.read("strfile"), self.data) + + # Print the ZIP directory + fp = StringIO() + stdout = sys.stdout + try: + sys.stdout = fp + + zipfp.printdir() + finally: + sys.stdout = stdout + + directory = fp.getvalue() + lines = directory.splitlines() + self.assertEquals(len(lines), 4) # Number of files + header + + self.assert_('File Name' in lines[0]) + self.assert_('Modified' in lines[0]) + self.assert_('Size' in lines[0]) + + fn, date, time, size = lines[1].split() + self.assertEquals(fn, 'another.name') + # XXX: timestamp is not tested + self.assertEquals(size, str(len(self.data))) + + # Check the namelist + names = zipfp.namelist() + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + + # Check infolist + infos = zipfp.infolist() + names = [ i.filename for i in infos ] + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + for i in infos: + self.assertEquals(i.file_size, len(self.data)) + + # check getinfo + for nm in (TESTFN, "another"+os.extsep+"name", "strfile"): + info = zipfp.getinfo(nm) + self.assertEquals(info.filename, nm) + self.assertEquals(info.file_size, len(self.data)) + + # Check that testzip doesn't raise an exception + zipfp.testzip() + + zipfp.close() + + + def testStored(self): for f in (TESTFN2, TemporaryFile(), StringIO()): self.zipTest(f, zipfile.ZIP_STORED) @@ -59,6 +115,197 @@ os.remove(TESTFN) os.remove(TESTFN2) +class TestZip64InSmallFiles(unittest.TestCase): + # These tests test the ZIP64 functionality without using large files, + # see test_zipfile64 for proper tests. + + def setUp(self): + self._limit = zipfile.ZIP64_LIMIT + zipfile.ZIP64_LIMIT = 5 + + line_gen = ("Test of zipfile line %d." % i for i in range(0, 1000)) + self.data = '\n'.join(line_gen) + + # Make a source file with some lines + fp = open(TESTFN, "wb") + fp.write(self.data) + fp.close() + + def largeFileExceptionTest(self, f, compression): + zipfp = zipfile.ZipFile(f, "w", compression) + self.assertRaises(zipfile.LargeZipFile, + zipfp.write, TESTFN, "another"+os.extsep+"name") + zipfp.close() + + def largeFileExceptionTest2(self, f, compression): + zipfp = zipfile.ZipFile(f, "w", compression) + self.assertRaises(zipfile.LargeZipFile, + zipfp.writestr, "another"+os.extsep+"name", self.data) + zipfp.close() + + def testLargeFileException(self): + for f in (TESTFN2, TemporaryFile(), StringIO()): + self.largeFileExceptionTest(f, zipfile.ZIP_STORED) + self.largeFileExceptionTest2(f, zipfile.ZIP_STORED) + + def zipTest(self, f, compression): + # Create the ZIP archive + zipfp = zipfile.ZipFile(f, "w", compression, allowZip64=True) + zipfp.write(TESTFN, "another"+os.extsep+"name") + zipfp.write(TESTFN, TESTFN) + zipfp.writestr("strfile", self.data) + zipfp.close() + + # Read the ZIP archive + zipfp = zipfile.ZipFile(f, "r", compression) + self.assertEqual(zipfp.read(TESTFN), self.data) + self.assertEqual(zipfp.read("another"+os.extsep+"name"), self.data) + self.assertEqual(zipfp.read("strfile"), self.data) + + # Print the ZIP directory + fp = StringIO() + stdout = sys.stdout + try: + sys.stdout = fp + + zipfp.printdir() + finally: + sys.stdout = stdout + + directory = fp.getvalue() + lines = directory.splitlines() + self.assertEquals(len(lines), 4) # Number of files + header + + self.assert_('File Name' in lines[0]) + self.assert_('Modified' in lines[0]) + self.assert_('Size' in lines[0]) + + fn, date, time, size = lines[1].split() + self.assertEquals(fn, 'another.name') + # XXX: timestamp is not tested + self.assertEquals(size, str(len(self.data))) + + # Check the namelist + names = zipfp.namelist() + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + + # Check infolist + infos = zipfp.infolist() + names = [ i.filename for i in infos ] + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + for i in infos: + self.assertEquals(i.file_size, len(self.data)) + + # check getinfo + for nm in (TESTFN, "another"+os.extsep+"name", "strfile"): + info = zipfp.getinfo(nm) + self.assertEquals(info.filename, nm) + self.assertEquals(info.file_size, len(self.data)) + + # Check that testzip doesn't raise an exception + zipfp.testzip() + + + zipfp.close() + + def testStored(self): + for f in (TESTFN2, TemporaryFile(), StringIO()): + self.zipTest(f, zipfile.ZIP_STORED) + + + if zlib: + def testDeflated(self): + for f in (TESTFN2, TemporaryFile(), StringIO()): + self.zipTest(f, zipfile.ZIP_DEFLATED) + + def testAbsoluteArcnames(self): + zipfp = zipfile.ZipFile(TESTFN2, "w", zipfile.ZIP_STORED, allowZip64=True) + zipfp.write(TESTFN, "/absolute") + zipfp.close() + + zipfp = zipfile.ZipFile(TESTFN2, "r", zipfile.ZIP_STORED) + self.assertEqual(zipfp.namelist(), ["absolute"]) + zipfp.close() + + + def tearDown(self): + zipfile.ZIP64_LIMIT = self._limit + os.remove(TESTFN) + os.remove(TESTFN2) + +class PyZipFileTests(unittest.TestCase): + def testWritePyfile(self): + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + fn = __file__ + if fn.endswith('.pyc') or fn.endswith('.pyo'): + fn = fn[:-1] + + zipfp.writepy(fn) + + bn = os.path.basename(fn) + self.assert_(bn not in zipfp.namelist()) + self.assert_(bn + 'o' in zipfp.namelist() or bn + 'c' in zipfp.namelist()) + zipfp.close() + + + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + fn = __file__ + if fn.endswith('.pyc') or fn.endswith('.pyo'): + fn = fn[:-1] + + zipfp.writepy(fn, "testpackage") + + bn = "%s/%s"%("testpackage", os.path.basename(fn)) + self.assert_(bn not in zipfp.namelist()) + self.assert_(bn + 'o' in zipfp.namelist() or bn + 'c' in zipfp.namelist()) + zipfp.close() + + def testWritePythonPackage(self): + import email + packagedir = os.path.dirname(email.__file__) + + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + zipfp.writepy(packagedir) + + # Check for a couple of modules at different levels of the hieararchy + names = zipfp.namelist() + self.assert_('email/__init__.pyo' in names or 'email/__init__.pyc' in names) + self.assert_('email/mime/text.pyo' in names or 'email/mime/text.pyc' in names) + + def testWritePythonDirectory(self): + os.mkdir(TESTFN2) + try: + fp = open(os.path.join(TESTFN2, "mod1.py"), "w") + fp.write("print 42\n") + fp.close() + + fp = open(os.path.join(TESTFN2, "mod2.py"), "w") + fp.write("print 42 * 42\n") + fp.close() + + fp = open(os.path.join(TESTFN2, "mod2.txt"), "w") + fp.write("bla bla bla\n") + fp.close() + + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + zipfp.writepy(TESTFN2) + + names = zipfp.namelist() + self.assert_('mod1.pyc' in names or 'mod1.pyo' in names) + self.assert_('mod2.pyc' in names or 'mod2.pyo' in names) + self.assert_('mod2.txt' not in names) + + finally: + shutil.rmtree(TESTFN2) + + + class OtherTests(unittest.TestCase): def testCloseErroneousFile(self): # This test checks that the ZipFile constructor closes the file object @@ -103,7 +350,8 @@ self.assertRaises(RuntimeError, zipf.testzip) def test_main(): - run_unittest(TestsWithSourceFile, OtherTests) + run_unittest(TestsWithSourceFile, TestZip64InSmallFiles, OtherTests, PyZipFileTests) + #run_unittest(TestZip64InSmallFiles) if __name__ == "__main__": test_main() Added: python/trunk/Lib/test/test_zipfile64.py ============================================================================== --- (empty file) +++ python/trunk/Lib/test/test_zipfile64.py Thu Jun 15 10:14:18 2006 @@ -0,0 +1,67 @@ +# Tests of the full ZIP64 functionality of zipfile +# The test_support.requires call is the only reason for keeping this separate +# from test_zipfile +from test import test_support +test_support.requires( + 'largefile', + 'test requires loads of disk-space bytes and a long time to run' + ) + +# We can test part of the module without zlib. +try: + import zlib +except ImportError: + zlib = None + +import zipfile, os, unittest + +from StringIO import StringIO +from tempfile import TemporaryFile + +from test.test_support import TESTFN, run_unittest + +TESTFN2 = TESTFN + "2" + +class TestsWithSourceFile(unittest.TestCase): + def setUp(self): + line_gen = ("Test of zipfile line %d." % i for i in range(0, 1000000)) + self.data = '\n'.join(line_gen) + + # Make a source file with some lines + fp = open(TESTFN, "wb") + fp.write(self.data) + fp.close() + + def zipTest(self, f, compression): + # Create the ZIP archive + filecount = int(((1 << 32) / len(self.data)) * 1.5) + zipfp = zipfile.ZipFile(f, "w", compression, allowZip64=True) + + for num in range(filecount): + zipfp.writestr("testfn%d"%(num,), self.data) + zipfp.close() + + # Read the ZIP archive + zipfp = zipfile.ZipFile(f, "r", compression) + for num in range(filecount): + self.assertEqual(zipfp.read("testfn%d"%(num,)), self.data) + zipfp.close() + + def testStored(self): + for f in (TESTFN2, TemporaryFile()): + self.zipTest(f, zipfile.ZIP_STORED) + + if zlib: + def testDeflated(self): + for f in (TESTFN2, TemporaryFile()): + self.zipTest(f, zipfile.ZIP_DEFLATED) + + def tearDown(self): + os.remove(TESTFN) + os.remove(TESTFN2) + +def test_main(): + run_unittest(TestsWithSourceFile) + +if __name__ == "__main__": + test_main() Modified: python/trunk/Lib/zipfile.py ============================================================================== --- python/trunk/Lib/zipfile.py (original) +++ python/trunk/Lib/zipfile.py Thu Jun 15 10:14:18 2006 @@ -1,7 +1,8 @@ -"Read and write ZIP files." - +""" +Read and write ZIP files. +""" import struct, os, time, sys -import binascii +import binascii, cStringIO try: import zlib # We may need its compression method @@ -9,12 +10,22 @@ zlib = None __all__ = ["BadZipfile", "error", "ZIP_STORED", "ZIP_DEFLATED", "is_zipfile", - "ZipInfo", "ZipFile", "PyZipFile"] + "ZipInfo", "ZipFile", "PyZipFile", "LargeZipFile" ] class BadZipfile(Exception): pass + + +class LargeZipFile(Exception): + """ + Raised when writing a zipfile, the zipfile requires ZIP64 extensions + and those extensions are disabled. + """ + error = BadZipfile # The exception raised by this module +ZIP64_LIMIT= (1 << 31) - 1 + # constants for Zip file compression methods ZIP_STORED = 0 ZIP_DEFLATED = 8 @@ -27,6 +38,11 @@ stringCentralDir = "PK\001\002" # magic number for central directory structFileHeader = "<4s2B4HlLL2H" # 12 items, file header record, 30 bytes stringFileHeader = "PK\003\004" # magic number for file header +structEndArchive64Locator = "<4slql" # 4 items, locate Zip64 header, 20 bytes +stringEndArchive64Locator = "PK\x06\x07" # magic token for locator header +structEndArchive64 = "<4sqhhllqqqq" # 10 items, end of archive (Zip64), 56 bytes +stringEndArchive64 = "PK\x06\x06" # magic token for Zip64 header + # indexes of entries in the central directory structure _CD_SIGNATURE = 0 @@ -75,6 +91,40 @@ pass return False +def _EndRecData64(fpin, offset, endrec): + """ + Read the ZIP64 end-of-archive records and use that to update endrec + """ + locatorSize = struct.calcsize(structEndArchive64Locator) + fpin.seek(offset - locatorSize, 2) + data = fpin.read(locatorSize) + sig, diskno, reloff, disks = struct.unpack(structEndArchive64Locator, data) + if sig != stringEndArchive64Locator: + return endrec + + if diskno != 0 or disks != 1: + raise BadZipfile("zipfiles that span multiple disks are not supported") + + # Assume no 'zip64 extensible data' + endArchiveSize = struct.calcsize(structEndArchive64) + fpin.seek(offset - locatorSize - endArchiveSize, 2) + data = fpin.read(endArchiveSize) + sig, sz, create_version, read_version, disk_num, disk_dir, \ + dircount, dircount2, dirsize, diroffset = \ + struct.unpack(structEndArchive64, data) + if sig != stringEndArchive64: + return endrec + + # Update the original endrec using data from the ZIP64 record + endrec[1] = disk_num + endrec[2] = disk_dir + endrec[3] = dircount + endrec[4] = dircount2 + endrec[5] = dirsize + endrec[6] = diroffset + return endrec + + def _EndRecData(fpin): """Return data from the "End of Central Directory" record, or None. @@ -88,6 +138,8 @@ endrec = list(endrec) endrec.append("") # Append the archive comment endrec.append(filesize - 22) # Append the record start offset + if endrec[-4] == -1 or endrec[-4] == 0xffffffff: + return _EndRecData64(fpin, -22, endrec) return endrec # Search the last END_BLOCK bytes of the file for the record signature. # The comment is appended to the ZIP file and has a 16 bit length. @@ -106,25 +158,50 @@ # Append the archive comment and start offset endrec.append(comment) endrec.append(filesize - END_BLOCK + start) + if endrec[-4] == -1 or endrec[-4] == 0xffffffff: + return _EndRecData64(fpin, - END_BLOCK + start, endrec) return endrec return # Error, return None -class ZipInfo: +class ZipInfo (object): """Class with attributes describing each file in the ZIP archive.""" + __slots__ = ( + 'orig_filename', + 'filename', + 'date_time', + 'compress_type', + 'comment', + 'extra', + 'create_system', + 'create_version', + 'extract_version', + 'reserved', + 'flag_bits', + 'volume', + 'internal_attr', + 'external_attr', + 'header_offset', + 'CRC', + 'compress_size', + 'file_size', + ) + def __init__(self, filename="NoName", date_time=(1980,1,1,0,0,0)): self.orig_filename = filename # Original file name in archive -# Terminate the file name at the first null byte. Null bytes in file -# names are used as tricks by viruses in archives. + + # Terminate the file name at the first null byte. Null bytes in file + # names are used as tricks by viruses in archives. null_byte = filename.find(chr(0)) if null_byte >= 0: filename = filename[0:null_byte] -# This is used to ensure paths in generated ZIP files always use -# forward slashes as the directory separator, as required by the -# ZIP format specification. - if os.sep != "/": + # This is used to ensure paths in generated ZIP files always use + # forward slashes as the directory separator, as required by the + # ZIP format specification. + if os.sep != "/" and os.sep in filename: filename = filename.replace(os.sep, "/") + self.filename = filename # Normalized file name self.date_time = date_time # year, month, day, hour, min, sec # Standard values: @@ -145,7 +222,6 @@ self.external_attr = 0 # External file attributes # Other attributes are set by class ZipFile: # header_offset Byte offset to the file header - # file_offset Byte offset to the start of the file data # CRC CRC-32 of the uncompressed file # compress_size Size of the compressed file # file_size Size of the uncompressed file @@ -162,29 +238,85 @@ CRC = self.CRC compress_size = self.compress_size file_size = self.file_size + + extra = self.extra + + if file_size > ZIP64_LIMIT or compress_size > ZIP64_LIMIT: + # File is larger than what fits into a 4 byte integer, + # fall back to the ZIP64 extension + fmt = '= 24: + counts = unpack(' ZIP64_LIMIT: + x = endrec[9] - size_cd - 56 - 20 + else: + x = endrec[9] - size_cd # "concat" is zero, unless zip was concatenated to another file concat = x - offset_cd if self.debug > 2: @@ -258,6 +393,8 @@ # self.start_dir: Position of start of central directory self.start_dir = offset_cd + concat fp.seek(self.start_dir, 0) + data = fp.read(size_cd) + fp = cStringIO.StringIO(data) total = 0 while total < size_cd: centdir = fp.read(46) @@ -275,8 +412,7 @@ total = (total + centdir[_CD_FILENAME_LENGTH] + centdir[_CD_EXTRA_FIELD_LENGTH] + centdir[_CD_COMMENT_LENGTH]) - x.header_offset = centdir[_CD_LOCAL_HEADER_OFFSET] + concat - # file_offset must be computed below... + x.header_offset = centdir[_CD_LOCAL_HEADER_OFFSET] (x.create_version, x.create_system, x.extract_version, x.reserved, x.flag_bits, x.compress_type, t, d, x.CRC, x.compress_size, x.file_size) = centdir[1:12] @@ -284,28 +420,14 @@ # Convert date/time code to (year, month, day, hour, min, sec) x.date_time = ( (d>>9)+1980, (d>>5)&0xF, d&0x1F, t>>11, (t>>5)&0x3F, (t&0x1F) * 2 ) + + x._decodeExtra() + x.header_offset = x.header_offset + concat self.filelist.append(x) self.NameToInfo[x.filename] = x if self.debug > 2: print "total", total - for data in self.filelist: - fp.seek(data.header_offset, 0) - fheader = fp.read(30) - if fheader[0:4] != stringFileHeader: - raise BadZipfile, "Bad magic number for file header" - fheader = struct.unpack(structFileHeader, fheader) - # file_offset is computed here, since the extra field for - # the central directory and for the local file header - # refer to different fields, and they can have different - # lengths - data.file_offset = (data.header_offset + 30 - + fheader[_FH_FILENAME_LENGTH] - + fheader[_FH_EXTRA_FIELD_LENGTH]) - fname = fp.read(fheader[_FH_FILENAME_LENGTH]) - if fname != data.orig_filename: - raise RuntimeError, \ - 'File name in directory "%s" and header "%s" differ.' % ( - data.orig_filename, fname) + def namelist(self): """Return a list of file names in the archive.""" @@ -334,6 +456,7 @@ except BadZipfile: return zinfo.filename + def getinfo(self, name): """Return the instance of ZipInfo given 'name'.""" return self.NameToInfo[name] @@ -347,7 +470,24 @@ "Attempt to read ZIP archive that was already closed" zinfo = self.getinfo(name) filepos = self.fp.tell() - self.fp.seek(zinfo.file_offset, 0) + + self.fp.seek(zinfo.header_offset, 0) + + # Skip the file header: + fheader = self.fp.read(30) + if fheader[0:4] != stringFileHeader: + raise BadZipfile, "Bad magic number for file header" + + fheader = struct.unpack(structFileHeader, fheader) + fname = self.fp.read(fheader[_FH_FILENAME_LENGTH]) + if fheader[_FH_EXTRA_FIELD_LENGTH]: + self.fp.read(fheader[_FH_EXTRA_FIELD_LENGTH]) + + if fname != zinfo.orig_filename: + raise BadZipfile, \ + 'File name in directory "%s" and header "%s" differ.' % ( + zinfo.orig_filename, fname) + bytes = self.fp.read(zinfo.compress_size) self.fp.seek(filepos, 0) if zinfo.compress_type == ZIP_STORED: @@ -388,6 +528,12 @@ if zinfo.compress_type not in (ZIP_STORED, ZIP_DEFLATED): raise RuntimeError, \ "That compression method is not supported" + if zinfo.file_size > ZIP64_LIMIT: + if not self._allowZip64: + raise LargeZipFile("Filesize would require ZIP64 extensions") + if zinfo.header_offset > ZIP64_LIMIT: + if not self._allowZip64: + raise LargeZipFile("Zipfile size would require ZIP64 extensions") def write(self, filename, arcname=None, compress_type=None): """Put the bytes from filename into the archive under the name @@ -407,16 +553,19 @@ zinfo.compress_type = self.compression else: zinfo.compress_type = compress_type - self._writecheck(zinfo) - fp = open(filename, "rb") + + zinfo.file_size = st.st_size zinfo.flag_bits = 0x00 zinfo.header_offset = self.fp.tell() # Start of header bytes + + self._writecheck(zinfo) + self._didModify = True + fp = open(filename, "rb") # Must overwrite CRC and sizes with correct data later zinfo.CRC = CRC = 0 zinfo.compress_size = compress_size = 0 zinfo.file_size = file_size = 0 self.fp.write(zinfo.FileHeader()) - zinfo.file_offset = self.fp.tell() # Start of file bytes if zinfo.compress_type == ZIP_DEFLATED: cmpr = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -15) @@ -461,8 +610,10 @@ zinfo.compress_type = self.compression else: zinfo = zinfo_or_arcname - self._writecheck(zinfo) zinfo.file_size = len(bytes) # Uncompressed size + zinfo.header_offset = self.fp.tell() # Start of header bytes + self._writecheck(zinfo) + self._didModify = True zinfo.CRC = binascii.crc32(bytes) # CRC-32 checksum if zinfo.compress_type == ZIP_DEFLATED: co = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, @@ -473,8 +624,8 @@ zinfo.compress_size = zinfo.file_size zinfo.header_offset = self.fp.tell() # Start of header bytes self.fp.write(zinfo.FileHeader()) - zinfo.file_offset = self.fp.tell() # Start of file bytes self.fp.write(bytes) + self.fp.flush() if zinfo.flag_bits & 0x08: # Write CRC and file sizes after the file data self.fp.write(struct.pack(" ZIP64_LIMIT \ + or zinfo.compress_size > ZIP64_LIMIT: + extra.append(zinfo.file_size) + extra.append(zinfo.compress_size) + file_size = 0xffffffff #-1 + compress_size = 0xffffffff #-1 + else: + file_size = zinfo.file_size + compress_size = zinfo.compress_size + + if zinfo.header_offset > ZIP64_LIMIT: + extra.append(zinfo.header_offset) + header_offset = 0xffffffff #-1 + else: + header_offset = zinfo.header_offset + + extra_data = zinfo.extra + if extra: + # Append a ZIP64 field to the extra's + extra_data = struct.pack( + ' ZIP64_LIMIT: + # Need to write the ZIP64 end-of-archive records + zip64endrec = struct.pack( + structEndArchive64, stringEndArchive64, + 44, 45, 45, 0, 0, count, count, pos2 - pos1, pos1) + self.fp.write(zip64endrec) + + zip64locrec = struct.pack( + structEndArchive64Locator, + stringEndArchive64Locator, 0, pos2, 1) + self.fp.write(zip64locrec) + + pos3 = self.fp.tell() + endrec = struct.pack(structEndArchive, stringEndArchive, + 0, 0, count, count, pos2 - pos1, 0xffffffff, 0) # -1, 0) + self.fp.write(endrec) + + else: + endrec = struct.pack(structEndArchive, stringEndArchive, + 0, 0, count, count, pos2 - pos1, pos1, 0) + self.fp.write(endrec) self.fp.flush() if not self._filePassed: self.fp.close() @@ -619,3 +820,80 @@ if basename: archivename = "%s/%s" % (basename, archivename) return (fname, archivename) + + +def main(args = None): + import textwrap + USAGE=textwrap.dedent("""\ + Usage: + zipfile.py -l zipfile.zip # Show listing of a zipfile + zipfile.py -t zipfile.zip # Test if a zipfile is valid + zipfile.py -e zipfile.zip target # Extract zipfile into target dir + zipfile.py -c zipfile.zip src ... # Create zipfile from sources + """) + if args is None: + args = sys.argv[1:] + + if not args or args[0] not in ('-l', '-c', '-e', '-t'): + print USAGE + sys.exit(1) + + if args[0] == '-l': + if len(args) != 2: + print USAGE + sys.exit(1) + zf = ZipFile(args[1], 'r') + zf.printdir() + zf.close() + + elif args[0] == '-t': + if len(args) != 2: + print USAGE + sys.exit(1) + zf = ZipFile(args[1], 'r') + zf.testzip() + print "Done testing" + + elif args[0] == '-e': + if len(args) != 3: + print USAGE + sys.exit(1) + + zf = ZipFile(args[1], 'r') + out = args[2] + for path in zf.namelist(): + if path.startswith('./'): + tgt = os.path.join(out, path[2:]) + else: + tgt = os.path.join(out, path) + + tgtdir = os.path.dirname(tgt) + if not os.path.exists(tgtdir): + os.makedirs(tgtdir) + fp = open(tgt, 'wb') + fp.write(zf.read(path)) + fp.close() + zf.close() + + elif args[0] == '-c': + if len(args) < 3: + print USAGE + sys.exit(1) + + def addToZip(zf, path, zippath): + if os.path.isfile(path): + zf.write(path, zippath, ZIP_DEFLATED) + elif os.path.isdir(path): + for nm in os.listdir(path): + addToZip(zf, + os.path.join(path, nm), os.path.join(zippath, nm)) + # else: ignore + + zf = ZipFile(args[1], 'w', allowZip64=True) + for src in args[2:]: + addToZip(zf, src, os.path.basename(src)) + + zf.close() + +if __name__ == "__main__": + main() Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 15 10:14:18 2006 @@ -152,6 +152,7 @@ aborts the db transaction safely when a modifier callback fails. Fixes SF python patch/bug #1408584. +- Patch #1446489: add support for the ZIP64 extensions to zipfile. Library ------- From nnorwitz at gmail.com Thu Jun 15 10:14:53 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 15 Jun 2006 01:14:53 -0700 Subject: [Python-checkins] r46962 - in python/trunk: Lib/sqlite3/test/hooks.py Lib/sqlite3/test/userfunctions.py Modules/_sqlite/connection.c In-Reply-To: <449114CF.3020602@ghaering.de> References: <20060614222839.7CCDE1E4005@bag.python.org> <200606151146.19472.anthony@interlink.com.au> <4490F6BD.7080007@ghaering.de> <449114CF.3020602@ghaering.de> Message-ID: On 6/15/06, Gerhard H?ring wrote: > Neal Norwitz wrote: > > I forgot to mention that all the builbots (even one on 3.1.3 or is > > that my laptop) pass with the change I checked in. probably rev > > 26964. > > Ok, but I believe they passed all tests before your checkin, too. Right, the difference is that they passed before the checkin because the code wasn't exercised if the version was too early. They passed after the checkin with the version checks removed, so the code was executed. > > On 6/14/06, Neal Norwitz wrote: > >> Can you take a look at my checkin? It only added 2 lines and the > >> tests pass after removing the version checks. I don't know if there's > >> a good reason to *not* do the fix I did. > > Doing it this way is a possible approach that should work ok, and it's > probably a good one, too. It is functionally different from the real > thing (sqlite3_result_error) though, because the query in progress is > not aborted. > > If you want to keep your checkin, please restore this change though: > > http://svn.python.org/view/python/trunk/Lib/sqlite3/test/hooks.py?rev=46964&r1=46962&r2=46964 Sure, I'll re-revert that change. (or will it be a re-re-revert?) > it addresses a different problem that I haven't fully debugged yet. All > I know is that without it, the test suite will crash for SQLite 3.0.8 > and not for 3.2.1 and later. It's most likely another SQLite bug of some > sort. It didn't crash for 3.1.3 on Mac OS X 10.3. I didn't pay close attention and verify the test was run though. I couldn't find an earlier version. Please try to talk to Anthony so we can figure out how to proceed with 2.5 and if anything needs to be changed. Probably the biggest concern is that in order to support this properly, APIs will need to change. n From python-checkins at python.org Thu Jun 15 10:16:46 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 15 Jun 2006 10:16:46 +0200 (CEST) Subject: [Python-checkins] r46968 - python/trunk/Lib/sqlite3/test/hooks.py Message-ID: <20060615081646.D44B41E4005@bag.python.org> Author: neal.norwitz Date: Thu Jun 15 10:16:44 2006 New Revision: 46968 Modified: python/trunk/Lib/sqlite3/test/hooks.py Log: Re-revert this change. Install the version check and don't run the test until Gerhard has time to fully debug the issue. This affects versions before 3.2.1 (possibly only versions earlier than 3.1.3). Based on discussion on python-checkins. Modified: python/trunk/Lib/sqlite3/test/hooks.py ============================================================================== --- python/trunk/Lib/sqlite3/test/hooks.py (original) +++ python/trunk/Lib/sqlite3/test/hooks.py Thu Jun 15 10:16:44 2006 @@ -48,6 +48,8 @@ pass def CheckCollationIsUsed(self): + if sqlite.version_info < (3, 2, 1): # old SQLite versions crash on this test + return def mycoll(x, y): # reverse order return -cmp(x, y) From nnorwitz at gmail.com Thu Jun 15 10:27:57 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 15 Jun 2006 01:27:57 -0700 Subject: [Python-checkins] r46967 - in python/trunk: Doc/lib/libzipfile.tex Lib/test/test_zipfile.py Lib/test/test_zipfile64.py Lib/zipfile.py Misc/NEWS In-Reply-To: <20060615081423.8126C1E4005@bag.python.org> References: <20060615081423.8126C1E4005@bag.python.org> Message-ID: On 6/15/06, ronald.oussoren wrote: > Author: ronald.oussoren > Date: Thu Jun 15 10:14:18 2006 > New Revision: 46967 > > Added: > python/trunk/Lib/test/test_zipfile64.py > Modified: > python/trunk/Doc/lib/libzipfile.tex > python/trunk/Lib/test/test_zipfile.py > python/trunk/Lib/zipfile.py > python/trunk/Misc/NEWS > Log: > Patch #1446489 (zipfile: support for ZIP64) > > > Modified: python/trunk/Doc/lib/libzipfile.tex Please add versionadded and versionchanged tags for any API diffs. > Modified: python/trunk/Lib/test/test_zipfile.py > ============================================================================== > --- python/trunk/Lib/test/test_zipfile.py (original) > +++ python/trunk/Lib/test/test_zipfile.py Thu Jun 15 10:14:18 2006 > +class TestZip64InSmallFiles(unittest.TestCase): > + # These tests test the ZIP64 functionality without using large files, > + # see test_zipfile64 for proper tests. > + > + def setUp(self): > + self._limit = zipfile.ZIP64_LIMIT > + zipfile.ZIP64_LIMIT = 5 > + > + line_gen = ("Test of zipfile line %d." % i for i in range(0, 1000)) > + self.data = '\n'.join(line_gen) > + > + # Make a source file with some lines > + fp = open(TESTFN, "wb") > + fp.write(self.data) > + fp.close() > + def tearDown(self): > + zipfile.ZIP64_LIMIT = self._limit > + os.remove(TESTFN) > + os.remove(TESTFN2) Please remove the files in in setUp too. That way the tests won't fail if another test left the files around. Same for the new test file (test_zip64.py I think). > @@ -103,7 +350,8 @@ > self.assertRaises(RuntimeError, zipf.testzip) > > def test_main(): > - run_unittest(TestsWithSourceFile, OtherTests) > + run_unittest(TestsWithSourceFile, TestZip64InSmallFiles, OtherTests, PyZipFileTests) > + #run_unittest(TestZip64InSmallFiles) Why is Zip64 commented out? > Added: python/trunk/Lib/test/test_zipfile64.py > + def zipTest(self, f, compression): > + # Create the ZIP archive > + filecount = int(((1 << 32) / len(self.data)) * 1.5) > + zipfp = zipfile.ZipFile(f, "w", compression, allowZip64=True) Should use // instead of / ? I need to finish the review tomorrow. n From python-checkins at python.org Thu Jun 15 10:52:33 2006 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 15 Jun 2006 10:52:33 +0200 (CEST) Subject: [Python-checkins] r46969 - in python/trunk: Lib/bsddb/__init__.py Lib/bsddb/dbutils.py Misc/NEWS Message-ID: <20060615085233.CD4501E4005@bag.python.org> Author: gregory.p.smith Date: Thu Jun 15 10:52:32 2006 New Revision: 46969 Modified: python/trunk/Lib/bsddb/__init__.py python/trunk/Lib/bsddb/dbutils.py python/trunk/Misc/NEWS Log: - bsddb: multithreaded DB access using the simple bsddb module interface now works reliably. It has been updated to use automatic BerkeleyDB deadlock detection and the bsddb.dbutils.DeadlockWrap wrapper to retry database calls that would previously deadlock. [SF python bug #775414] Modified: python/trunk/Lib/bsddb/__init__.py ============================================================================== --- python/trunk/Lib/bsddb/__init__.py (original) +++ python/trunk/Lib/bsddb/__init__.py Thu Jun 15 10:52:32 2006 @@ -33,7 +33,10 @@ #---------------------------------------------------------------------- -"""Support for BerkeleyDB 3.2 through 4.2. +"""Support for BerkeleyDB 3.3 through 4.4 with a simple interface. + +For the full featured object oriented interface use the bsddb.db module +instead. It mirrors the Sleepycat BerkeleyDB C API. """ try: @@ -43,8 +46,10 @@ # python as bsddb._bsddb. import _pybsddb _bsddb = _pybsddb + from bsddb3.dbutils import DeadlockWrap as _DeadlockWrap else: import _bsddb + from bsddb.dbutils import DeadlockWrap as _DeadlockWrap except ImportError: # Remove ourselves from sys.modules import sys @@ -70,7 +75,7 @@ exec """ class _iter_mixin(UserDict.DictMixin): def _make_iter_cursor(self): - cur = self.db.cursor() + cur = _DeadlockWrap(self.db.cursor) key = id(cur) self._cursor_refs[key] = ref(cur, self._gen_cref_cleaner(key)) return cur @@ -90,19 +95,19 @@ # since we're only returning keys, we call the cursor # methods with flags=0, dlen=0, dofs=0 - key = cur.first(0,0,0)[0] + key = _DeadlockWrap(cur.first, 0,0,0)[0] yield key next = cur.next while 1: try: - key = next(0,0,0)[0] + key = _DeadlockWrap(next, 0,0,0)[0] yield key except _bsddb.DBCursorClosedError: cur = self._make_iter_cursor() # FIXME-20031101-greg: race condition. cursor could # be closed by another thread before this call. - cur.set(key,0,0,0) + _DeadlockWrap(cur.set, key,0,0,0) next = cur.next except _bsddb.DBNotFoundError: return @@ -119,21 +124,21 @@ # FIXME-20031102-greg: race condition. cursor could # be closed by another thread before this call. - kv = cur.first() + kv = _DeadlockWrap(cur.first) key = kv[0] yield kv next = cur.next while 1: try: - kv = next() + kv = _DeadlockWrap(next) key = kv[0] yield kv except _bsddb.DBCursorClosedError: cur = self._make_iter_cursor() # FIXME-20031101-greg: race condition. cursor could # be closed by another thread before this call. - cur.set(key,0,0,0) + _DeadlockWrap(cur.set, key,0,0,0) next = cur.next except _bsddb.DBNotFoundError: return @@ -177,9 +182,9 @@ def _checkCursor(self): if self.dbc is None: - self.dbc = self.db.cursor() + self.dbc = _DeadlockWrap(self.db.cursor) if self.saved_dbc_key is not None: - self.dbc.set(self.saved_dbc_key) + _DeadlockWrap(self.dbc.set, self.saved_dbc_key) self.saved_dbc_key = None # This method is needed for all non-cursor DB calls to avoid @@ -192,15 +197,15 @@ self.dbc = None if save: try: - self.saved_dbc_key = c.current(0,0,0)[0] + self.saved_dbc_key = _DeadlockWrap(c.current, 0,0,0)[0] except db.DBError: pass - c.close() + _DeadlockWrap(c.close) del c for cref in self._cursor_refs.values(): c = cref() if c is not None: - c.close() + _DeadlockWrap(c.close) def _checkOpen(self): if self.db is None: @@ -211,73 +216,77 @@ def __len__(self): self._checkOpen() - return len(self.db) + return _DeadlockWrap(lambda: len(self.db)) # len(self.db) def __getitem__(self, key): self._checkOpen() - return self.db[key] + return _DeadlockWrap(lambda: self.db[key]) # self.db[key] def __setitem__(self, key, value): self._checkOpen() self._closeCursors() - self.db[key] = value + def wrapF(): + self.db[key] = value + _DeadlockWrap(wrapF) # self.db[key] = value def __delitem__(self, key): self._checkOpen() self._closeCursors() - del self.db[key] + def wrapF(): + del self.db[key] + _DeadlockWrap(wrapF) # del self.db[key] def close(self): self._closeCursors(save=0) if self.dbc is not None: - self.dbc.close() + _DeadlockWrap(self.dbc.close) v = 0 if self.db is not None: - v = self.db.close() + v = _DeadlockWrap(self.db.close) self.dbc = None self.db = None return v def keys(self): self._checkOpen() - return self.db.keys() + return _DeadlockWrap(self.db.keys) def has_key(self, key): self._checkOpen() - return self.db.has_key(key) + return _DeadlockWrap(self.db.has_key, key) def set_location(self, key): self._checkOpen() self._checkCursor() - return self.dbc.set_range(key) + return _DeadlockWrap(self.dbc.set_range, key) def next(self): self._checkOpen() self._checkCursor() - rv = self.dbc.next() + rv = _DeadlockWrap(self.dbc.next) return rv def previous(self): self._checkOpen() self._checkCursor() - rv = self.dbc.prev() + rv = _DeadlockWrap(self.dbc.prev) return rv def first(self): self._checkOpen() self._checkCursor() - rv = self.dbc.first() + rv = _DeadlockWrap(self.dbc.first) return rv def last(self): self._checkOpen() self._checkCursor() - rv = self.dbc.last() + rv = _DeadlockWrap(self.dbc.last) return rv def sync(self): self._checkOpen() - return self.db.sync() + return _DeadlockWrap(self.db.sync) #---------------------------------------------------------------------- @@ -385,5 +394,4 @@ except ImportError: db.DB_THREAD = 0 - #---------------------------------------------------------------------- Modified: python/trunk/Lib/bsddb/dbutils.py ============================================================================== --- python/trunk/Lib/bsddb/dbutils.py (original) +++ python/trunk/Lib/bsddb/dbutils.py Thu Jun 15 10:52:32 2006 @@ -22,14 +22,14 @@ # # import the time.sleep function in a namespace safe way to allow -# "from bsddb.db import *" +# "from bsddb.dbutils import *" # from time import sleep as _sleep import db # always sleep at least N seconds between retrys -_deadlock_MinSleepTime = 1.0/64 +_deadlock_MinSleepTime = 1.0/128 # never sleep more than N seconds between retrys _deadlock_MaxSleepTime = 3.14159 @@ -57,7 +57,7 @@ max_retries = _kwargs.get('max_retries', -1) if _kwargs.has_key('max_retries'): del _kwargs['max_retries'] - while 1: + while True: try: return function(*_args, **_kwargs) except db.DBLockDeadlockError: Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 15 10:52:32 2006 @@ -152,8 +152,14 @@ aborts the db transaction safely when a modifier callback fails. Fixes SF python patch/bug #1408584. +- bsddb: multithreaded DB access using the simple bsddb module interface + now works reliably. It has been updated to use automatic BerkeleyDB + deadlock detection and the bsddb.dbutils.DeadlockWrap wrapper to retry + database calls that would previously deadlock. [SF python bug #775414] + - Patch #1446489: add support for the ZIP64 extensions to zipfile. + Library ------- From buildbot at python.org Thu Jun 15 10:58:46 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 08:58:46 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060615085847.11C701E4005@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1174 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 11:07:43 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 09:07:43 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060615090744.0C1431E4005@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/775 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 11:08:12 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 09:08:12 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060615090812.B64721E4005@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1077 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 11:17:32 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 09:17:32 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060615091732.3B05C1E4005@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/1025 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 11:19:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 09:19:36 +0000 Subject: [Python-checkins] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20060615091936.A2A961E4005@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%2520trunk/builds/1026 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 15 11:23:52 2006 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 15 Jun 2006 11:23:52 +0200 (CEST) Subject: [Python-checkins] r46970 - python/trunk/Doc/lib/libbsddb.tex Message-ID: <20060615092352.BC44C1E4005@bag.python.org> Author: gregory.p.smith Date: Thu Jun 15 11:23:52 2006 New Revision: 46970 Modified: python/trunk/Doc/lib/libbsddb.tex Log: minor documentation cleanup. mention the bsddb.db interface explicitly by name. Modified: python/trunk/Doc/lib/libbsddb.tex ============================================================================== --- python/trunk/Doc/lib/libbsddb.tex (original) +++ python/trunk/Doc/lib/libbsddb.tex Thu Jun 15 11:23:52 2006 @@ -13,23 +13,29 @@ dictionaries. Keys and values must be strings, however, so to use other objects as keys or to store other kinds of objects the user must serialize them somehow, typically using \function{marshal.dumps()} or -\function{pickle.dumps}. +\function{pickle.dumps()}. The \module{bsddb} module requires a Berkeley DB library version from 3.3 thru 4.4. \begin{seealso} - \seeurl{http://pybsddb.sourceforge.net/}{Website with documentation - for the new python Berkeley DB interface that closely mirrors the - sleepycat object oriented interface provided in Berkeley DB 3 and 4.} + \seeurl{http://pybsddb.sourceforge.net/}{The website with documentation + for the \module{bsddb.db} python Berkeley DB interface that closely mirrors + the Sleepycat object oriented interface provided in Berkeley DB 3 and 4.} \seeurl{http://www.sleepycat.com/}{Sleepycat Software produces the - modern Berkeley DB library.} + Berkeley DB library.} \end{seealso} +A more modern DB, DBEnv and DBSequence object interface is available in the +\module{bsddb.db} module which closely matches the Sleepycat Berkeley DB C API +documented at the above URLs. Additional features provided by the +\module{bsddb.db} API include fine tuning, transactions, logging, and +multiprocess concurrent database access. + The following is a description of the legacy \module{bsddb} interface -compatible with the old python bsddb module. For details about the more -modern Db and DbEnv object oriented interface see the above mentioned -pybsddb URL. +compatible with the old python bsddb module. Starting in Python 2.5 this +interface should be safe for multithreaded access. The \module{bsddb.db} +API is recommended for threading users as it provides better control. The \module{bsddb} module defines the following functions that create objects that access the appropriate type of Berkeley DB file. The From buildbot at python.org Thu Jun 15 11:34:41 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 09:34:41 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060615093441.B61E81E4005@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/726 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 11:48:22 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 09:48:22 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060615094822.527AB1E4005@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/652 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 11:56:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 09:56:55 +0000 Subject: [Python-checkins] buildbot warnings in MIPS Debian trunk Message-ID: <20060615095655.575601E4005@bag.python.org> The Buildbot has detected a new failure of MIPS Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/MIPS%2520Debian%2520trunk/builds/234 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis,neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 15 11:57:03 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 15 Jun 2006 11:57:03 +0200 (CEST) Subject: [Python-checkins] r46971 - python/trunk/Lib/test/test_zipfile64.py Message-ID: <20060615095703.A47FF1E4005@bag.python.org> Author: neal.norwitz Date: Thu Jun 15 11:57:03 2006 New Revision: 46971 Modified: python/trunk/Lib/test/test_zipfile64.py Log: Steal the trick from test_compiler to print out a slow msg. This will hopefully get the buildbots to pass. Not sure this test will be feasible or even work. But everything is red now, so it can't get much worse. Modified: python/trunk/Lib/test/test_zipfile64.py ============================================================================== --- python/trunk/Lib/test/test_zipfile64.py (original) +++ python/trunk/Lib/test/test_zipfile64.py Thu Jun 15 11:57:03 2006 @@ -14,6 +14,8 @@ zlib = None import zipfile, os, unittest +import time +import sys from StringIO import StringIO from tempfile import TemporaryFile @@ -22,6 +24,9 @@ TESTFN2 = TESTFN + "2" +# How much time in seconds can pass before we print a 'Still working' message. +_PRINT_WORKING_MSG_INTERVAL = 5 * 60 + class TestsWithSourceFile(unittest.TestCase): def setUp(self): line_gen = ("Test of zipfile line %d." % i for i in range(0, 1000000)) @@ -37,14 +42,27 @@ filecount = int(((1 << 32) / len(self.data)) * 1.5) zipfp = zipfile.ZipFile(f, "w", compression, allowZip64=True) + next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL for num in range(filecount): zipfp.writestr("testfn%d"%(num,), self.data) + # Print still working message since this test can be really slow + if next_time <= time.time(): + next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL + print >>sys.__stdout__, \ + ' zipTest still working, be patient...' + sys.__stdout__.flush() zipfp.close() # Read the ZIP archive zipfp = zipfile.ZipFile(f, "r", compression) for num in range(filecount): self.assertEqual(zipfp.read("testfn%d"%(num,)), self.data) + # Print still working message since this test can be really slow + if next_time <= time.time(): + next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL + print >>sys.__stdout__, \ + ' zipTest still working, be patient...' + sys.__stdout__.flush() zipfp.close() def testStored(self): From python-checkins at python.org Thu Jun 15 12:24:50 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 15 Jun 2006 12:24:50 +0200 (CEST) Subject: [Python-checkins] r46972 - python/trunk/Lib/test/test_zipfile64.py Message-ID: <20060615102450.0F73F1E4005@bag.python.org> Author: neal.norwitz Date: Thu Jun 15 12:24:49 2006 New Revision: 46972 Modified: python/trunk/Lib/test/test_zipfile64.py Log: Print some more info to get an idea of how much longer the test will last Modified: python/trunk/Lib/test/test_zipfile64.py ============================================================================== --- python/trunk/Lib/test/test_zipfile64.py (original) +++ python/trunk/Lib/test/test_zipfile64.py Thu Jun 15 12:24:49 2006 @@ -48,8 +48,9 @@ # Print still working message since this test can be really slow if next_time <= time.time(): next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL - print >>sys.__stdout__, \ - ' zipTest still working, be patient...' + print >>sys.__stdout__, ( + ' zipTest still writing %d of %d, be patient...' % + (num, filecount)) sys.__stdout__.flush() zipfp.close() @@ -60,8 +61,9 @@ # Print still working message since this test can be really slow if next_time <= time.time(): next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL - print >>sys.__stdout__, \ - ' zipTest still working, be patient...' + print >>sys.__stdout__, ( + ' zipTest still reading %d of %d, be patient...' % + (num, filecount)) sys.__stdout__.flush() zipfp.close() From buildbot at python.org Thu Jun 15 14:11:05 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 12:11:05 +0000 Subject: [Python-checkins] buildbot warnings in x86 W2k trunk Message-ID: <20060615121105.00F881E4005@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/1096 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 17:39:03 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 15:39:03 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060615153903.DC9A61E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/369 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gregory.p.smith,neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 15 19:30:52 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Thu, 15 Jun 2006 19:30:52 +0200 (CEST) Subject: [Python-checkins] r46973 - python/branches/hoxworth-stdlib_logging-soc Message-ID: <20060615173052.C67B21E401E@bag.python.org> Author: jackilyn.hoxworth Date: Thu Jun 15 19:30:52 2006 New Revision: 46973 Added: python/branches/hoxworth-stdlib_logging-soc/ Log: Created folder remotely From python-checkins at python.org Thu Jun 15 19:41:34 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Thu, 15 Jun 2006 19:41:34 +0200 (CEST) Subject: [Python-checkins] r46974 - python/branches/hoxworth-stdlib_logging-soc/Lib Message-ID: <20060615174134.3A6271E4008@bag.python.org> Author: jackilyn.hoxworth Date: Thu Jun 15 19:41:33 2006 New Revision: 46974 Added: python/branches/hoxworth-stdlib_logging-soc/Lib/ - copied from r46973, python/trunk/Lib/ Log: made a copy From python-checkins at python.org Thu Jun 15 19:49:51 2006 From: python-checkins at python.org (phillip.eby) Date: Thu, 15 Jun 2006 19:49:51 +0200 (CEST) Subject: [Python-checkins] r46975 - in sandbox/branches/setuptools-0.6: ez_setup.py release.sh setup.py setuptools/__init__.py version.dat Message-ID: <20060615174951.BF9181E4003@bag.python.org> Author: phillip.eby Date: Thu Jun 15 19:49:51 2006 New Revision: 46975 Modified: sandbox/branches/setuptools-0.6/ez_setup.py sandbox/branches/setuptools-0.6/release.sh sandbox/branches/setuptools-0.6/setup.py sandbox/branches/setuptools-0.6/setuptools/__init__.py sandbox/branches/setuptools-0.6/version.dat Log: Bump version # to 0.6b4 - although it might really be 0.6c1 Modified: sandbox/branches/setuptools-0.6/ez_setup.py ============================================================================== --- sandbox/branches/setuptools-0.6/ez_setup.py (original) +++ sandbox/branches/setuptools-0.6/ez_setup.py Thu Jun 15 19:49:51 2006 @@ -14,7 +14,7 @@ This file can also be run as a script to install or upgrade setuptools. """ import sys -DEFAULT_VERSION = "0.6b3" +DEFAULT_VERSION = "0.6b4" DEFAULT_URL = "http://cheeseshop.python.org/packages/%s/s/setuptools/" % sys.version[:3] md5_data = { Modified: sandbox/branches/setuptools-0.6/release.sh ============================================================================== --- sandbox/branches/setuptools-0.6/release.sh (original) +++ sandbox/branches/setuptools-0.6/release.sh Thu Jun 15 19:49:51 2006 @@ -7,7 +7,7 @@ # If your initials aren't PJE, don't run it. :) # -export VERSION="0.6b3" +export VERSION="0.6b4" wpython setup.py setopt -r -c egg_info -o tag_build && \ wpython setup.py setopt -r setopt -r -c egg_info -o tag_svn_revision && \ Modified: sandbox/branches/setuptools-0.6/setup.py ============================================================================== --- sandbox/branches/setuptools-0.6/setup.py (original) +++ sandbox/branches/setuptools-0.6/setup.py Thu Jun 15 19:49:51 2006 @@ -19,7 +19,7 @@ d = {}; execfile(convert_path('setuptools/command/__init__.py'), d) SETUP_COMMANDS = d['__all__'] -VERSION = "0.6b3" +VERSION = "0.6b4" from setuptools import setup, find_packages import sys scripts = [] Modified: sandbox/branches/setuptools-0.6/setuptools/__init__.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/__init__.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/__init__.py Thu Jun 15 19:49:51 2006 @@ -7,7 +7,7 @@ from distutils.util import convert_path import os.path -__version__ = '0.6b3' +__version__ = '0.6b4' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', 'find_packages' Modified: sandbox/branches/setuptools-0.6/version.dat ============================================================================== --- sandbox/branches/setuptools-0.6/version.dat (original) +++ sandbox/branches/setuptools-0.6/version.dat Thu Jun 15 19:49:51 2006 @@ -1,6 +1,6 @@ [setuptools] status = 'beta' major = 0 -build = 3 +build = 4 minor = 6 From python-checkins at python.org Thu Jun 15 19:56:37 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Thu, 15 Jun 2006 19:56:37 +0200 (CEST) Subject: [Python-checkins] r46976 - python/branches/hoxworth-stdlib_logging-soc/asyncore.py Message-ID: <20060615175637.8AE001E4014@bag.python.org> Author: jackilyn.hoxworth Date: Thu Jun 15 19:56:36 2006 New Revision: 46976 Added: python/branches/hoxworth-stdlib_logging-soc/asyncore.py Log: Added a file remotely Added: python/branches/hoxworth-stdlib_logging-soc/asyncore.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/asyncore.py Thu Jun 15 19:56:36 2006 @@ -0,0 +1,596 @@ +# -*- Mode: Python -*- +# Id: asyncore.py,v 2.51 2000/09/07 22:29:26 rushing Exp +# Author: Sam Rushing + +# ====================================================================== +# Copyright 1996 by Sam Rushing +# +# All Rights Reserved +# +# Permission to use, copy, modify, and distribute this software and +# its documentation for any purpose and without fee is hereby +# granted, provided that the above copyright notice appear in all +# copies and that both that copyright notice and this permission +# notice appear in supporting documentation, and that the name of Sam +# Rushing not be used in advertising or publicity pertaining to +# distribution of the software without specific, written prior +# permission. +# +# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, +# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN +# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR +# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS +# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, +# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN +# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +# ====================================================================== + +"""Basic infrastructure for asynchronous socket service clients and servers. + +There are only two ways to have a program on a single processor do "more +than one thing at a time". Multi-threaded programming is the simplest and +most popular way to do it, but there is another very different technique, +that lets you have nearly all the advantages of multi-threading, without +actually using multiple threads. it's really only practical if your program +is largely I/O bound. If your program is CPU bound, then pre-emptive +scheduled threads are probably what you really need. Network servers are +rarely CPU-bound, however. + +If your operating system supports the select() system call in its I/O +library (and nearly all do), then you can use it to juggle multiple +communication channels at once; doing other work while your I/O is taking +place in the "background." Although this strategy can seem strange and +complex, especially at first, it is in many ways easier to understand and +control than multi-threaded programming. The module documented here solves +many of the difficult problems for you, making the task of building +sophisticated high-performance network servers and clients a snap. +""" + +import select +import socket +import sys +import time + +import os +from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, \ + ENOTCONN, ESHUTDOWN, EINTR, EISCONN, errorcode + +try: + socket_map +except NameError: + socket_map = {} + +class ExitNow(Exception): + pass + +def read(obj): + try: + obj.handle_read_event() + except ExitNow: + raise + except: + obj.handle_error() + +def write(obj): + try: + obj.handle_write_event() + except ExitNow: + raise + except: + obj.handle_error() + +def _exception (obj): + try: + obj.handle_expt_event() + except ExitNow: + raise + except: + obj.handle_error() + +def readwrite(obj, flags): + try: + if flags & (select.POLLIN | select.POLLPRI): + obj.handle_read_event() + if flags & select.POLLOUT: + obj.handle_write_event() + if flags & (select.POLLERR | select.POLLHUP | select.POLLNVAL): + obj.handle_expt_event() + except ExitNow: + raise + except: + obj.handle_error() + +def poll(timeout=0.0, map=None): + if map is None: + map = socket_map + if map: + r = []; w = []; e = [] + for fd, obj in map.items(): + is_r = obj.readable() + is_w = obj.writable() + if is_r: + r.append(fd) + if is_w: + w.append(fd) + if is_r or is_w: + e.append(fd) + if [] == r == w == e: + time.sleep(timeout) + else: + try: + r, w, e = select.select(r, w, e, timeout) + except select.error, err: + if err[0] != EINTR: + raise + else: + return + + for fd in r: + obj = map.get(fd) + if obj is None: + continue + read(obj) + + for fd in w: + obj = map.get(fd) + if obj is None: + continue + write(obj) + + for fd in e: + obj = map.get(fd) + if obj is None: + continue + _exception(obj) + +def poll2(timeout=0.0, map=None): + # Use the poll() support added to the select module in Python 2.0 + if map is None: + map = socket_map + if timeout is not None: + # timeout is in milliseconds + timeout = int(timeout*1000) + pollster = select.poll() + if map: + for fd, obj in map.items(): + flags = 0 + if obj.readable(): + flags |= select.POLLIN | select.POLLPRI + if obj.writable(): + flags |= select.POLLOUT + if flags: + # Only check for exceptions if object was either readable + # or writable. + flags |= select.POLLERR | select.POLLHUP | select.POLLNVAL + pollster.register(fd, flags) + try: + r = pollster.poll(timeout) + except select.error, err: + if err[0] != EINTR: + raise + r = [] + for fd, flags in r: + obj = map.get(fd) + if obj is None: + continue + readwrite(obj, flags) + +poll3 = poll2 # Alias for backward compatibility + +def loop(timeout=30.0, use_poll=False, map=None, count=None): + if map is None: + map = socket_map + + if use_poll and hasattr(select, 'poll'): + poll_fun = poll2 + else: + poll_fun = poll + + if count is None: + while map: + poll_fun(timeout, map) + + else: + while map and count > 0: + poll_fun(timeout, map) + count = count - 1 + +class dispatcher: + + debug = False + connected = False + accepting = False + closing = False + addr = None + + def __init__(self, sock=None, map=None): + if map is None: + self._map = socket_map + else: + self._map = map + + if sock: + self.set_socket(sock, map) + # I think it should inherit this anyway + self.socket.setblocking(0) + self.connected = True + # XXX Does the constructor require that the socket passed + # be connected? + try: + self.addr = sock.getpeername() + except socket.error: + # The addr isn't crucial + pass + else: + self.socket = None + + def __repr__(self): + status = [self.__class__.__module__+"."+self.__class__.__name__] + if self.accepting and self.addr: + status.append('listening') + elif self.connected: + status.append('connected') + if self.addr is not None: + try: + status.append('%s:%d' % self.addr) + except TypeError: + status.append(repr(self.addr)) + return '<%s at %#x>' % (' '.join(status), id(self)) + + def add_channel(self, map=None): + #self.log_info('adding channel %s' % self) + if map is None: + map = self._map + map[self._fileno] = self + + def del_channel(self, map=None): + fd = self._fileno + if map is None: + map = self._map + if map.has_key(fd): + #self.log_info('closing channel %d:%s' % (fd, self)) + del map[fd] + self._fileno = None + + def create_socket(self, family, type): + self.family_and_type = family, type + self.socket = socket.socket(family, type) + self.socket.setblocking(0) + self._fileno = self.socket.fileno() + self.add_channel() + + def set_socket(self, sock, map=None): + self.socket = sock +## self.__dict__['socket'] = sock + self._fileno = sock.fileno() + self.add_channel(map) + + def set_reuse_addr(self): + # try to re-use a server port if possible + try: + self.socket.setsockopt( + socket.SOL_SOCKET, socket.SO_REUSEADDR, + self.socket.getsockopt(socket.SOL_SOCKET, + socket.SO_REUSEADDR) | 1 + ) + except socket.error: + pass + + # ================================================== + # predicates for select() + # these are used as filters for the lists of sockets + # to pass to select(). + # ================================================== + + def readable(self): + return True + + def writable(self): + return True + + # ================================================== + # socket object methods. + # ================================================== + + def listen(self, num): + self.accepting = True + if os.name == 'nt' and num > 5: + num = 1 + return self.socket.listen(num) + + def bind(self, addr): + self.addr = addr + return self.socket.bind(addr) + + def connect(self, address): + self.connected = False + err = self.socket.connect_ex(address) + # XXX Should interpret Winsock return values + if err in (EINPROGRESS, EALREADY, EWOULDBLOCK): + return + if err in (0, EISCONN): + self.addr = address + self.connected = True + self.handle_connect() + else: + raise socket.error, (err, errorcode[err]) + + def accept(self): + # XXX can return either an address pair or None + try: + conn, addr = self.socket.accept() + return conn, addr + except socket.error, why: + if why[0] == EWOULDBLOCK: + pass + else: + raise + + def send(self, data): + try: + result = self.socket.send(data) + return result + except socket.error, why: + if why[0] == EWOULDBLOCK: + return 0 + else: + raise + return 0 + + def recv(self, buffer_size): + try: + data = self.socket.recv(buffer_size) + if not data: + # a closed connection is indicated by signaling + # a read condition, and having recv() return 0. + self.handle_close() + return '' + else: + return data + except socket.error, why: + # winsock sometimes throws ENOTCONN + if why[0] in [ECONNRESET, ENOTCONN, ESHUTDOWN]: + self.handle_close() + return '' + else: + raise + + def close(self): + self.del_channel() + self.socket.close() + + # cheap inheritance, used to pass all other attribute + # references to the underlying socket object. + def __getattr__(self, attr): + return getattr(self.socket, attr) + + # log and log_info may be overridden to provide more sophisticated + # logging and warning methods. In general, log is for 'hit' logging + # and 'log_info' is for informational, warning and error logging. + # edited out for SoC additions + + #def log(self, message): + # sys.stderr.write('log: %s\n' % str(message)) + + #def log_info(self, message, type='info'): + # if __debug__ or type != 'info': + # print '%s: %s' % (type, message) + +# ================================================================================================= +# SoC start edit +# log and log_info are kept for backwards compatibility, particularly with subclasses which +# override them. +# +# log is for hits (like an apache access log) +# log_info is for instrumentation (like an apache error log), regardless of severity. +# informational, warning, and error messages all go here. +# ================================================================================================= + + logger = None + + def _logger(self, level, msg, *args, **kwargs): + if logger is None: + import logging + self.logger = logging.getLogger("py.module.__asyncore__") + else: + self.logger.log(level, msg, *args, **kwargs) + + hit_logger = None + + def log(self, message): + if hit_logger is None: + import logging + self.hit_logger = logging.getLogger("py.module.__asyncore__.dispatcher.hits").info + else: + self.hit_logger(message) + + message_logger=None + + def log_info(self, message, type='info'): + if message_logger is None: + import logging + else: + self.message_logger = logging.getLogger("py.module.__asyncore__.dispatcher.messages") + # Figure out the level somehow -- maybe do google searches on asyncore and + # medusa to see what values actually get used for type? + self.message_logger(level, message) + +# ================================================================================================= +# SoC end edit +# ================================================================================================= + + + def handle_read_event(self): + if self.accepting: + # for an accepting socket, getting a read implies + # that we are connected + if not self.connected: + self.connected = True + self.handle_accept() + elif not self.connected: + self.handle_connect() + self.connected = True + self.handle_read() + else: + self.handle_read() + + def handle_write_event(self): + # getting a write implies that we are connected + if not self.connected: + self.handle_connect() + self.connected = True + self.handle_write() + + def handle_expt_event(self): + self.handle_expt() + + def handle_error(self): + nil, t, v, tbinfo = compact_traceback() + + # sometimes a user repr method will crash. + try: + self_repr = repr(self) + except: + self_repr = '<__repr__(self) failed for object at %0x>' % id(self) + + self.log_info( + 'uncaptured python exception, closing channel %s (%s:%s %s)' % ( + self_repr, + t, + v, + tbinfo + ), + 'error' + ) + self.close() + + def handle_expt(self): + self.log_info('unhandled exception', 'warning') + + def handle_read(self): + self.log_info('unhandled read event', 'warning') + + def handle_write(self): + self.log_info('unhandled write event', 'warning') + + def handle_connect(self): + self.log_info('unhandled connect event', 'warning') + + def handle_accept(self): + self.log_info('unhandled accept event', 'warning') + + def handle_close(self): + self.log_info('unhandled close event', 'warning') + self.close() + +# --------------------------------------------------------------------------- +# adds simple buffered output capability, useful for simple clients. +# [for more sophisticated usage use asynchat.async_chat] +# --------------------------------------------------------------------------- + +class dispatcher_with_send(dispatcher): + + def __init__(self, sock=None, map=None): + dispatcher.__init__(self, sock, map) + self.out_buffer = '' + + def initiate_send(self): + num_sent = 0 + num_sent = dispatcher.send(self, self.out_buffer[:512]) + self.out_buffer = self.out_buffer[num_sent:] + + def handle_write(self): + self.initiate_send() + + def writable(self): + return (not self.connected) or len(self.out_buffer) + + def send(self, data): + if self.debug: + self.log_info('sending %s' % repr(data)) + self.out_buffer = self.out_buffer + data + self.initiate_send() + +# --------------------------------------------------------------------------- +# used for debugging. +# --------------------------------------------------------------------------- + +def compact_traceback(): + t, v, tb = sys.exc_info() + tbinfo = [] + assert tb # Must have a traceback + while tb: + tbinfo.append(( + tb.tb_frame.f_code.co_filename, + tb.tb_frame.f_code.co_name, + str(tb.tb_lineno) + )) + tb = tb.tb_next + + # just to be safe + del tb + + file, function, line = tbinfo[-1] + info = ' '.join(['[%s|%s|%s]' % x for x in tbinfo]) + return (file, function, line), t, v, info + +def close_all(map=None): + if map is None: + map = socket_map + for x in map.values(): + x.socket.close() + map.clear() + +# Asynchronous File I/O: +# +# After a little research (reading man pages on various unixen, and +# digging through the linux kernel), I've determined that select() +# isn't meant for doing asynchronous file i/o. +# Heartening, though - reading linux/mm/filemap.c shows that linux +# supports asynchronous read-ahead. So _MOST_ of the time, the data +# will be sitting in memory for us already when we go to read it. +# +# What other OS's (besides NT) support async file i/o? [VMS?] +# +# Regardless, this is useful for pipes, and stdin/stdout... + +if os.name == 'posix': + import fcntl + + class file_wrapper: + # here we override just enough to make a file + # look like a socket for the purposes of asyncore. + + def __init__(self, fd): + self.fd = fd + + def recv(self, *args): + return os.read(self.fd, *args) + + def send(self, *args): + return os.write(self.fd, *args) + + read = recv + write = send + + def close(self): + os.close(self.fd) + + def fileno(self): + return self.fd + + class file_dispatcher(dispatcher): + + def __init__(self, fd, map=None): + dispatcher.__init__(self, None, map) + self.connected = True + self.set_file(fd) + # set it to non-blocking mode + flags = fcntl.fcntl(fd, fcntl.F_GETFL, 0) + flags = flags | os.O_NONBLOCK + fcntl.fcntl(fd, fcntl.F_SETFL, flags) + + def set_file(self, fd): + self._fileno = fd + self.socket = file_wrapper(fd) + self.add_channel() From python-checkins at python.org Thu Jun 15 19:57:04 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Thu, 15 Jun 2006 19:57:04 +0200 (CEST) Subject: [Python-checkins] r46977 - python/branches/hoxworth-stdlib_logging-soc/gopherlib.py Message-ID: <20060615175704.283BE1E401A@bag.python.org> Author: jackilyn.hoxworth Date: Thu Jun 15 19:57:03 2006 New Revision: 46977 Added: python/branches/hoxworth-stdlib_logging-soc/gopherlib.py Log: Added a file remotely Added: python/branches/hoxworth-stdlib_logging-soc/gopherlib.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/gopherlib.py Thu Jun 15 19:57:03 2006 @@ -0,0 +1,212 @@ +"""Gopher protocol client interface.""" + +__all__ = ["send_selector","send_query"] + +import logging +_log = logging.getLogger('py.gopherlib') + +import warnings +warnings.warn("the gopherlib module is deprecated", DeprecationWarning, + stacklevel=2) + +# Default selector, host and port +DEF_SELECTOR = '1/' +DEF_HOST = 'gopher.micro.umn.edu' +DEF_PORT = 70 + +# Recognized file types +A_TEXT = '0' +A_MENU = '1' +A_CSO = '2' +A_ERROR = '3' +A_MACBINHEX = '4' +A_PCBINHEX = '5' +A_UUENCODED = '6' +A_INDEX = '7' +A_TELNET = '8' +A_BINARY = '9' +A_DUPLICATE = '+' +A_SOUND = 's' +A_EVENT = 'e' +A_CALENDAR = 'c' +A_HTML = 'h' +A_TN3270 = 'T' +A_MIME = 'M' +A_IMAGE = 'I' +A_WHOIS = 'w' +A_QUERY = 'q' +A_GIF = 'g' +A_HTML = 'h' # HTML file +A_WWW = 'w' # WWW address +A_PLUS_IMAGE = ':' +A_PLUS_MOVIE = ';' +A_PLUS_SOUND = '<' + + +_names = dir() +_type_to_name_map = {} +def type_to_name(gtype): + """Map all file types to strings; unknown types become TYPE='x'.""" + global _type_to_name_map + if _type_to_name_map=={}: + for name in _names: + if name[:2] == 'A_': + _type_to_name_map[eval(name)] = name[2:] + if gtype in _type_to_name_map: + return _type_to_name_map[gtype] + return 'TYPE=%r' % (gtype,) + +# Names for characters and strings +CRLF = '\r\n' +TAB = '\t' + +def send_selector(selector, host, port = 0): + """Send a selector to a given host and port, return a file with the reply.""" + import socket + if not port: + i = host.find(':') + if i >= 0: + host, port = host[:i], int(host[i+1:]) + if not port: + port = DEF_PORT + elif type(port) == type(''): + port = int(port) + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + s.connect((host, port)) + s.sendall(selector + CRLF) + s.shutdown(1) + return s.makefile('rb') + +def send_query(selector, query, host, port = 0): + """Send a selector and a query string.""" + return send_selector(selector + '\t' + query, host, port) + +def path_to_selector(path): + """Takes a path as returned by urlparse and returns the appropriate selector.""" + if path=="/": + return "/" + else: + return path[2:] # Cuts initial slash and data type identifier + +def path_to_datatype_name(path): + """Takes a path as returned by urlparse and maps it to a string. + See section 3.4 of RFC 1738 for details.""" + if path=="/": + # No way to tell, although "INDEX" is likely + return "TYPE='unknown'" + else: + return type_to_name(path[1]) + +# The following functions interpret the data returned by the gopher +# server according to the expected type, e.g. textfile or directory + +def get_directory(f): + """Get a directory in the form of a list of entries.""" + entries = [] + while 1: + line = f.readline() + if not line: + _.log.info ('Unexpected EOF from server)') + break + if line[-2:] == CRLF: + line = line[:-2] + elif line[-1:] in CRLF: + line = line[:-1] + if line == '.': + break + if not line: + _.log.info ('(Empty line from server)') + continue + gtype = line[0] + parts = line[1:].split(TAB) + if len(parts) < 4: + _.log.info ('(Bad line from server: %r)' % (line,)) + continue + if len(parts) > 4: + if parts[4:] != ['+']: + _.log.info ('(Extra info from server:'), + _.log.info (parts[4:], ')') + else: + parts.append('') + parts.insert(0, gtype) + entries.append(parts) + return entries + +def get_textfile(f): + """Get a text file as a list of lines, with trailing CRLF stripped.""" + lines = [] + get_alt_textfile(f, lines.append) + return lines + +def get_alt_textfile(f, func): + """Get a text file and pass each line to a function, with trailing CRLF stripped.""" + while 1: + line = f.readline() + if not line: + print '(Unexpected EOF from server)' + break + if line[-2:] == CRLF: + line = line[:-2] + elif line[-1:] in CRLF: + line = line[:-1] + if line == '.': + break + if line[:2] == '..': + line = line[1:] + func(line) + +def get_binary(f): + """Get a binary file as one solid data block.""" + data = f.read() + return data + +def get_alt_binary(f, func, blocksize): + """Get a binary file and pass each block to a function.""" + while 1: + data = f.read(blocksize) + if not data: + break + func(data) + +def test(): + """Trivial test program.""" + import sys + import getopt + opts, args = getopt.getopt(sys.argv[1:], '') + selector = DEF_SELECTOR + type = selector[0] + host = DEF_HOST + if args: + host = args[0] + args = args[1:] + if args: + type = args[0] + args = args[1:] + if len(type) > 1: + type, selector = type[0], type + else: + selector = '' + if args: + selector = args[0] + args = args[1:] + query = '' + if args: + query = args[0] + args = args[1:] + if type == A_INDEX: + f = send_query(selector, query, host) + else: + f = send_selector(selector, host) + if type == A_TEXT: + lines = get_textfile(f) + for item in lines: print item + elif type in (A_MENU, A_INDEX): + entries = get_directory(f) + for item in entries: print item + else: + data = get_binary(f) + print 'binary data:', len(data), 'bytes:', repr(data[:100])[:40] + +# Run the test when run as script +if __name__ == '__main__': + test() From python-checkins at python.org Thu Jun 15 19:57:25 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Thu, 15 Jun 2006 19:57:25 +0200 (CEST) Subject: [Python-checkins] r46978 - python/branches/hoxworth-stdlib_logging-soc/httplib.py Message-ID: <20060615175725.41FFB1E4019@bag.python.org> Author: jackilyn.hoxworth Date: Thu Jun 15 19:57:23 2006 New Revision: 46978 Added: python/branches/hoxworth-stdlib_logging-soc/httplib.py Log: Added a file remotely Added: python/branches/hoxworth-stdlib_logging-soc/httplib.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/httplib.py Thu Jun 15 19:57:23 2006 @@ -0,0 +1,1421 @@ +"""HTTP/1.1 client library + + + + +HTTPConnection go through a number of "states", which defines when a client +may legally make another request or fetch the response for a particular +request. This diagram details these state transitions: + + (null) + | + | HTTPConnection() + v + Idle + | + | putrequest() + v + Request-started + | + | ( putheader() )* endheaders() + v + Request-sent + | + | response = getresponse() + v + Unread-response [Response-headers-read] + |\____________________ + | | + | response.read() | putrequest() + v v + Idle Req-started-unread-response + ______/| + / | + response.read() | | ( putheader() )* endheaders() + v v + Request-started Req-sent-unread-response + | + | response.read() + v + Request-sent + +This diagram presents the following rules: + -- a second request may not be started until {response-headers-read} + -- a response [object] cannot be retrieved until {request-sent} + -- there is no differentiation between an unread response body and a + partially read response body + +Note: this enforcement is applied by the HTTPConnection class. The + HTTPResponse class does not enforce this state machine, which + implies sophisticated clients may accelerate the request/response + pipeline. Caution should be taken, though: accelerating the states + beyond the above pattern may imply knowledge of the server's + connection-close behavior for certain requests. For example, it + is impossible to tell whether the server will close the connection + UNTIL the response headers have been read; this means that further + requests cannot be placed into the pipeline until it is known that + the server will NOT be closing the connection. + +Logical State __state __response +------------- ------- ---------- +Idle _CS_IDLE None +Request-started _CS_REQ_STARTED None +Request-sent _CS_REQ_SENT None +Unread-response _CS_IDLE +Req-started-unread-response _CS_REQ_STARTED +Req-sent-unread-response _CS_REQ_SENT +""" + +import errno +import mimetools +import socket +from urlparse import urlsplit + +import logging +_log = logging.getLogger('py.httplib') + +try: + from cStringIO import StringIO +except ImportError: + from StringIO import StringIO + +__all__ = ["HTTP", "HTTPResponse", "HTTPConnection", "HTTPSConnection", + "HTTPException", "NotConnected", "UnknownProtocol", + "UnknownTransferEncoding", "UnimplementedFileMode", + "IncompleteRead", "InvalidURL", "ImproperConnectionState", + "CannotSendRequest", "CannotSendHeader", "ResponseNotReady", + "BadStatusLine", "error", "responses"] + +HTTP_PORT = 80 +HTTPS_PORT = 443 + +_UNKNOWN = 'UNKNOWN' + +# connection states +_CS_IDLE = 'Idle' +_CS_REQ_STARTED = 'Request-started' +_CS_REQ_SENT = 'Request-sent' + +# status codes +# informational +CONTINUE = 100 +SWITCHING_PROTOCOLS = 101 +PROCESSING = 102 + +# successful +OK = 200 +CREATED = 201 +ACCEPTED = 202 +NON_AUTHORITATIVE_INFORMATION = 203 +NO_CONTENT = 204 +RESET_CONTENT = 205 +PARTIAL_CONTENT = 206 +MULTI_STATUS = 207 +IM_USED = 226 + +# redirection +MULTIPLE_CHOICES = 300 +MOVED_PERMANENTLY = 301 +FOUND = 302 +SEE_OTHER = 303 +NOT_MODIFIED = 304 +USE_PROXY = 305 +TEMPORARY_REDIRECT = 307 + +# client error +BAD_REQUEST = 400 +UNAUTHORIZED = 401 +PAYMENT_REQUIRED = 402 +FORBIDDEN = 403 +NOT_FOUND = 404 +METHOD_NOT_ALLOWED = 405 +NOT_ACCEPTABLE = 406 +PROXY_AUTHENTICATION_REQUIRED = 407 +REQUEST_TIMEOUT = 408 +CONFLICT = 409 +GONE = 410 +LENGTH_REQUIRED = 411 +PRECONDITION_FAILED = 412 +REQUEST_ENTITY_TOO_LARGE = 413 +REQUEST_URI_TOO_LONG = 414 +UNSUPPORTED_MEDIA_TYPE = 415 +REQUESTED_RANGE_NOT_SATISFIABLE = 416 +EXPECTATION_FAILED = 417 +UNPROCESSABLE_ENTITY = 422 +LOCKED = 423 +FAILED_DEPENDENCY = 424 +UPGRADE_REQUIRED = 426 + +# server error +INTERNAL_SERVER_ERROR = 500 +NOT_IMPLEMENTED = 501 +BAD_GATEWAY = 502 +SERVICE_UNAVAILABLE = 503 +GATEWAY_TIMEOUT = 504 +HTTP_VERSION_NOT_SUPPORTED = 505 +INSUFFICIENT_STORAGE = 507 +NOT_EXTENDED = 510 + +# Mapping status codes to official W3C names +responses = { + 100: 'Continue', + 101: 'Switching Protocols', + + 200: 'OK', + 201: 'Created', + 202: 'Accepted', + 203: 'Non-Authoritative Information', + 204: 'No Content', + 205: 'Reset Content', + 206: 'Partial Content', + + 300: 'Multiple Choices', + 301: 'Moved Permanently', + 302: 'Found', + 303: 'See Other', + 304: 'Not Modified', + 305: 'Use Proxy', + 306: '(Unused)', + 307: 'Temporary Redirect', + + 400: 'Bad Request', + 401: 'Unauthorized', + 402: 'Payment Required', + 403: 'Forbidden', + 404: 'Not Found', + 405: 'Method Not Allowed', + 406: 'Not Acceptable', + 407: 'Proxy Authentication Required', + 408: 'Request Timeout', + 409: 'Conflict', + 410: 'Gone', + 411: 'Length Required', + 412: 'Precondition Failed', + 413: 'Request Entity Too Large', + 414: 'Request-URI Too Long', + 415: 'Unsupported Media Type', + 416: 'Requested Range Not Satisfiable', + 417: 'Expectation Failed', + + 500: 'Internal Server Error', + 501: 'Not Implemented', + 502: 'Bad Gateway', + 503: 'Service Unavailable', + 504: 'Gateway Timeout', + 505: 'HTTP Version Not Supported', +} + +# maximal amount of data to read at one time in _safe_read +MAXAMOUNT = 1048576 + +class HTTPMessage(mimetools.Message): + + def addheader(self, key, value): + """Add header for field key handling repeats.""" + prev = self.dict.get(key) + if prev is None: + self.dict[key] = value + else: + combined = ", ".join((prev, value)) + self.dict[key] = combined + + def addcontinue(self, key, more): + """Add more field data from a continuation line.""" + prev = self.dict[key] + self.dict[key] = prev + "\n " + more + + def readheaders(self): + """Read header lines. + + Read header lines up to the entirely blank line that terminates them. + The (normally blank) line that ends the headers is skipped, but not + included in the returned list. If a non-header line ends the headers, + (which is an error), an attempt is made to backspace over it; it is + never included in the returned list. + + The variable self.status is set to the empty string if all went well, + otherwise it is an error message. The variable self.headers is a + completely uninterpreted list of lines contained in the header (so + printing them will reproduce the header exactly as it appears in the + file). + + If multiple header fields with the same name occur, they are combined + according to the rules in RFC 2616 sec 4.2: + + Appending each subsequent field-value to the first, each separated + by a comma. The order in which header fields with the same field-name + are received is significant to the interpretation of the combined + field value. + """ + # XXX The implementation overrides the readheaders() method of + # rfc822.Message. The base class design isn't amenable to + # customized behavior here so the method here is a copy of the + # base class code with a few small changes. + + self.dict = {} + self.unixfrom = '' + self.headers = hlist = [] + self.status = '' + headerseen = "" + firstline = 1 + startofline = unread = tell = None + if hasattr(self.fp, 'unread'): + unread = self.fp.unread + elif self.seekable: + tell = self.fp.tell + while True: + if tell: + try: + startofline = tell() + except IOError: + startofline = tell = None + self.seekable = 0 + line = self.fp.readline() + if not line: + self.status = 'EOF in headers' + break + # Skip unix From name time lines + if firstline and line.startswith('From '): + self.unixfrom = self.unixfrom + line + continue + firstline = 0 + if headerseen and line[0] in ' \t': + # XXX Not sure if continuation lines are handled properly + # for http and/or for repeating headers + # It's a continuation line. + hlist.append(line) + self.addcontinue(headerseen, line.strip()) + continue + elif self.iscomment(line): + # It's a comment. Ignore it. + continue + elif self.islast(line): + # Note! No pushback here! The delimiter line gets eaten. + break + headerseen = self.isheader(line) + if headerseen: + # It's a legal header line, save it. + hlist.append(line) + self.addheader(headerseen, line[len(headerseen)+1:].strip()) + continue + else: + # It's not a header line; throw it back and stop here. + if not self.dict: + self.status = 'No headers' + else: + self.status = 'Non-header line where header expected' + # Try to undo the read. + if unread: + unread(line) + elif tell: + self.fp.seek(startofline) + else: + self.status = self.status + '; bad seek' + break + +class HTTPResponse: + # strict: If true, raise BadStatusLine if the status line can't be + # parsed as a valid HTTP/1.0 or 1.1 status line. By default it is + # false because it prevents clients from talking to HTTP/0.9 + # servers. Note that a response with a sufficiently corrupted + # status line will look like an HTTP/0.9 response. + + # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details. + + def __init__(self, sock, debuglevel=0, strict=0, method=None): + self.fp = sock.makefile('rb', 0) + self.debuglevel = debuglevel + self.strict = strict + self._method = method + + self.msg = None + + # from the Status-Line of the response + self.version = _UNKNOWN # HTTP-Version + self.status = _UNKNOWN # Status-Code + self.reason = _UNKNOWN # Reason-Phrase + + self.chunked = _UNKNOWN # is "chunked" being used? + self.chunk_left = _UNKNOWN # bytes left to read in current chunk + self.length = _UNKNOWN # number of bytes left in response + self.will_close = _UNKNOWN # conn will close at end of response + + def _read_status(self): + # Initialize with Simple-Response defaults + line = self.fp.readline() + if self.debuglevel > 0: + print "reply:", repr(line) + if not line: + # Presumably, the server closed the connection before + # sending a valid response. + raise BadStatusLine(line) + try: + [version, status, reason] = line.split(None, 2) + except ValueError: + try: + [version, status] = line.split(None, 1) + reason = "" + except ValueError: + # empty version will cause next test to fail and status + # will be treated as 0.9 response. + version = "" + if not version.startswith('HTTP/'): + if self.strict: + self.close() + raise BadStatusLine(line) + else: + # assume it's a Simple-Response from an 0.9 server + self.fp = LineAndFileWrapper(line, self.fp) + return "HTTP/0.9", 200, "" + + # The status code is a three-digit number + try: + status = int(status) + if status < 100 or status > 999: + raise BadStatusLine(line) + except ValueError: + raise BadStatusLine(line) + return version, status, reason + + def begin(self): + if self.msg is not None: + # we've already started reading the response + return + + # read until we get a non-100 response + while True: + version, status, reason = self._read_status() + if status != CONTINUE: + break + # skip the header from the 100 response + while True: + skip = self.fp.readline().strip() + if not skip: + break + if self.debuglevel > 0: + print "header:", skip + + self.status = status + self.reason = reason.strip() + if version == 'HTTP/1.0': + self.version = 10 + elif version.startswith('HTTP/1.'): + self.version = 11 # use HTTP/1.1 code for HTTP/1.x where x>=1 + elif version == 'HTTP/0.9': + self.version = 9 + else: + raise UnknownProtocol(version) + + if self.version == 9: + self.length = None + self.chunked = 0 + self.will_close = 1 + self.msg = HTTPMessage(StringIO()) + return + + self.msg = HTTPMessage(self.fp, 0) + if self.debuglevel > 0: + for hdr in self.msg.headers: + _log.info( "header:", hdr,) + + # don't let the msg keep an fp + self.msg.fp = None + + # are we using the chunked-style of transfer encoding? + tr_enc = self.msg.getheader('transfer-encoding') + if tr_enc and tr_enc.lower() == "chunked": + self.chunked = 1 + self.chunk_left = None + else: + self.chunked = 0 + + # will the connection close at the end of the response? + self.will_close = self._check_close() + + # do we have a Content-Length? + # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked" + length = self.msg.getheader('content-length') + if length and not self.chunked: + try: + self.length = int(length) + except ValueError: + self.length = None + else: + self.length = None + + # does the body have a fixed length? (of zero) + if (status == NO_CONTENT or status == NOT_MODIFIED or + 100 <= status < 200 or # 1xx codes + self._method == 'HEAD'): + self.length = 0 + + # if the connection remains open, and we aren't using chunked, and + # a content-length was not provided, then assume that the connection + # WILL close. + if not self.will_close and \ + not self.chunked and \ + self.length is None: + self.will_close = 1 + + def _check_close(self): + conn = self.msg.getheader('connection') + if self.version == 11: + # An HTTP/1.1 proxy is assumed to stay open unless + # explicitly closed. + conn = self.msg.getheader('connection') + if conn and "close" in conn.lower(): + return True + return False + + # Some HTTP/1.0 implementations have support for persistent + # connections, using rules different than HTTP/1.1. + + # For older HTTP, Keep-Alive indiciates persistent connection. + if self.msg.getheader('keep-alive'): + return False + + # At least Akamai returns a "Connection: Keep-Alive" header, + # which was supposed to be sent by the client. + if conn and "keep-alive" in conn.lower(): + return False + + # Proxy-Connection is a netscape hack. + pconn = self.msg.getheader('proxy-connection') + if pconn and "keep-alive" in pconn.lower(): + return False + + # otherwise, assume it will close + return True + + def close(self): + if self.fp: + self.fp.close() + self.fp = None + + def isclosed(self): + # NOTE: it is possible that we will not ever call self.close(). This + # case occurs when will_close is TRUE, length is None, and we + # read up to the last byte, but NOT past it. + # + # IMPLIES: if will_close is FALSE, then self.close() will ALWAYS be + # called, meaning self.isclosed() is meaningful. + return self.fp is None + + # XXX It would be nice to have readline and __iter__ for this, too. + + def read(self, amt=None): + if self.fp is None: + return '' + + if self.chunked: + return self._read_chunked(amt) + + if amt is None: + # unbounded read + if self.length is None: + s = self.fp.read() + else: + s = self._safe_read(self.length) + self.length = 0 + self.close() # we read everything + return s + + if self.length is not None: + if amt > self.length: + # clip the read to the "end of response" + amt = self.length + + # we do not use _safe_read() here because this may be a .will_close + # connection, and the user is reading more bytes than will be provided + # (for example, reading in 1k chunks) + s = self.fp.read(amt) + if self.length is not None: + self.length -= len(s) + + return s + + def _read_chunked(self, amt): + assert self.chunked != _UNKNOWN + chunk_left = self.chunk_left + value = '' + + # XXX This accumulates chunks by repeated string concatenation, + # which is not efficient as the number or size of chunks gets big. + while True: + if chunk_left is None: + line = self.fp.readline() + i = line.find(';') + if i >= 0: + line = line[:i] # strip chunk-extensions + chunk_left = int(line, 16) + if chunk_left == 0: + break + if amt is None: + value += self._safe_read(chunk_left) + elif amt < chunk_left: + value += self._safe_read(amt) + self.chunk_left = chunk_left - amt + return value + elif amt == chunk_left: + value += self._safe_read(amt) + self._safe_read(2) # toss the CRLF at the end of the chunk + self.chunk_left = None + return value + else: + value += self._safe_read(chunk_left) + amt -= chunk_left + + # we read the whole chunk, get another + self._safe_read(2) # toss the CRLF at the end of the chunk + chunk_left = None + + # read and discard trailer up to the CRLF terminator + ### note: we shouldn't have any trailers! + while True: + line = self.fp.readline() + if line == '\r\n': + break + + # we read everything; close the "file" + self.close() + + return value + + def _safe_read(self, amt): + """Read the number of bytes requested, compensating for partial reads. + + Normally, we have a blocking socket, but a read() can be interrupted + by a signal (resulting in a partial read). + + Note that we cannot distinguish between EOF and an interrupt when zero + bytes have been read. IncompleteRead() will be raised in this + situation. + + This function should be used when bytes "should" be present for + reading. If the bytes are truly not available (due to EOF), then the + IncompleteRead exception can be used to detect the problem. + """ + s = [] + while amt > 0: + chunk = self.fp.read(min(amt, MAXAMOUNT)) + if not chunk: + raise IncompleteRead(s) + s.append(chunk) + amt -= len(chunk) + return ''.join(s) + + def getheader(self, name, default=None): + if self.msg is None: + raise ResponseNotReady() + return self.msg.getheader(name, default) + + def getheaders(self): + """Return list of (header, value) tuples.""" + if self.msg is None: + raise ResponseNotReady() + return self.msg.items() + + +class HTTPConnection: + + _http_vsn = 11 + _http_vsn_str = 'HTTP/1.1' + + response_class = HTTPResponse + default_port = HTTP_PORT + auto_open = 1 + debuglevel = 0 + strict = 0 + + def __init__(self, host, port=None, strict=None): + self.sock = None + self._buffer = [] + self.__response = None + self.__state = _CS_IDLE + self._method = None + + self._set_hostport(host, port) + if strict is not None: + self.strict = strict + + def _set_hostport(self, host, port): + if port is None: + i = host.rfind(':') + j = host.rfind(']') # ipv6 addresses have [...] + if i > j: + try: + port = int(host[i+1:]) + except ValueError: + raise InvalidURL("nonnumeric port: '%s'" % host[i+1:]) + host = host[:i] + else: + port = self.default_port + if host and host[0] == '[' and host[-1] == ']': + host = host[1:-1] + self.host = host + self.port = port + + def set_debuglevel(self, level): + self.debuglevel = level + + def connect(self): + """Connect to the host and port specified in __init__.""" + msg = "getaddrinfo returns an empty list" + for res in socket.getaddrinfo(self.host, self.port, 0, + socket.SOCK_STREAM): + af, socktype, proto, canonname, sa = res + try: + self.sock = socket.socket(af, socktype, proto) + if self.debuglevel > 0: + print "connect: (%s, %s)" % (self.host, self.port) + self.sock.connect(sa) + except socket.error, msg: + if self.debuglevel > 0: + print 'connect fail:', (self.host, self.port) + if self.sock: + self.sock.close() + self.sock = None + continue + break + if not self.sock: + raise socket.error, msg + + def close(self): + """Close the connection to the HTTP server.""" + if self.sock: + self.sock.close() # close it manually... there may be other refs + self.sock = None + if self.__response: + self.__response.close() + self.__response = None + self.__state = _CS_IDLE + + def send(self, str): + """Send `str' to the server.""" + if self.sock is None: + if self.auto_open: + self.connect() + else: + raise NotConnected() + + # send the data to the server. if we get a broken pipe, then close + # the socket. we want to reconnect when somebody tries to send again. + # + # NOTE: we DO propagate the error, though, because we cannot simply + # ignore the error... the caller will know if they can retry. + if self.debuglevel > 0: + print "send:", repr(str) + try: + self.sock.sendall(str) + except socket.error, v: + if v[0] == 32: # Broken pipe + self.close() + raise + + def _output(self, s): + """Add a line of output to the current request buffer. + + Assumes that the line does *not* end with \\r\\n. + """ + self._buffer.append(s) + + def _send_output(self): + """Send the currently buffered request and clear the buffer. + + Appends an extra \\r\\n to the buffer. + """ + self._buffer.extend(("", "")) + msg = "\r\n".join(self._buffer) + del self._buffer[:] + self.send(msg) + + def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0): + """Send a request to the server. + + `method' specifies an HTTP request method, e.g. 'GET'. + `url' specifies the object being requested, e.g. '/index.html'. + `skip_host' if True does not add automatically a 'Host:' header + `skip_accept_encoding' if True does not add automatically an + 'Accept-Encoding:' header + """ + + # if a prior response has been completed, then forget about it. + if self.__response and self.__response.isclosed(): + self.__response = None + + + # in certain cases, we cannot issue another request on this connection. + # this occurs when: + # 1) we are in the process of sending a request. (_CS_REQ_STARTED) + # 2) a response to a previous request has signalled that it is going + # to close the connection upon completion. + # 3) the headers for the previous response have not been read, thus + # we cannot determine whether point (2) is true. (_CS_REQ_SENT) + # + # if there is no prior response, then we can request at will. + # + # if point (2) is true, then we will have passed the socket to the + # response (effectively meaning, "there is no prior response"), and + # will open a new one when a new request is made. + # + # Note: if a prior response exists, then we *can* start a new request. + # We are not allowed to begin fetching the response to this new + # request, however, until that prior response is complete. + # + if self.__state == _CS_IDLE: + self.__state = _CS_REQ_STARTED + else: + raise CannotSendRequest() + + # Save the method we use, we need it later in the response phase + self._method = method + if not url: + url = '/' + str = '%s %s %s' % (method, url, self._http_vsn_str) + + self._output(str) + + if self._http_vsn == 11: + # Issue some standard headers for better HTTP/1.1 compliance + + if not skip_host: + # this header is issued *only* for HTTP/1.1 + # connections. more specifically, this means it is + # only issued when the client uses the new + # HTTPConnection() class. backwards-compat clients + # will be using HTTP/1.0 and those clients may be + # issuing this header themselves. we should NOT issue + # it twice; some web servers (such as Apache) barf + # when they see two Host: headers + + # If we need a non-standard port,include it in the + # header. If the request is going through a proxy, + # but the host of the actual URL, not the host of the + # proxy. + + netloc = '' + if url.startswith('http'): + nil, netloc, nil, nil, nil = urlsplit(url) + + if netloc: + self.putheader('Host', netloc.encode("idna")) + elif self.port == HTTP_PORT: + self.putheader('Host', self.host.encode("idna")) + else: + self.putheader('Host', "%s:%s" % (self.host.encode("idna"), self.port)) + + # note: we are assuming that clients will not attempt to set these + # headers since *this* library must deal with the + # consequences. this also means that when the supporting + # libraries are updated to recognize other forms, then this + # code should be changed (removed or updated). + + # we only want a Content-Encoding of "identity" since we don't + # support encodings such as x-gzip or x-deflate. + if not skip_accept_encoding: + self.putheader('Accept-Encoding', 'identity') + + # we can accept "chunked" Transfer-Encodings, but no others + # NOTE: no TE header implies *only* "chunked" + #self.putheader('TE', 'chunked') + + # if TE is supplied in the header, then it must appear in a + # Connection header. + #self.putheader('Connection', 'TE') + + else: + # For HTTP/1.0, the server will assume "not chunked" + pass + + def putheader(self, header, value): + """Send a request header line to the server. + + For example: h.putheader('Accept', 'text/html') + """ + if self.__state != _CS_REQ_STARTED: + raise CannotSendHeader() + + str = '%s: %s' % (header, value) + self._output(str) + + def endheaders(self): + """Indicate that the last header line has been sent to the server.""" + + if self.__state == _CS_REQ_STARTED: + self.__state = _CS_REQ_SENT + else: + raise CannotSendHeader() + + self._send_output() + + def request(self, method, url, body=None, headers={}): + """Send a complete request to the server.""" + + try: + self._send_request(method, url, body, headers) + except socket.error, v: + # trap 'Broken pipe' if we're allowed to automatically reconnect + if v[0] != 32 or not self.auto_open: + raise + # try one more time + self._send_request(method, url, body, headers) + + def _send_request(self, method, url, body, headers): + # honour explicitly requested Host: and Accept-Encoding headers + header_names = dict.fromkeys([k.lower() for k in headers]) + skips = {} + if 'host' in header_names: + skips['skip_host'] = 1 + if 'accept-encoding' in header_names: + skips['skip_accept_encoding'] = 1 + + self.putrequest(method, url, **skips) + + if body and ('content-length' not in header_names): + self.putheader('Content-Length', str(len(body))) + for hdr, value in headers.iteritems(): + self.putheader(hdr, value) + self.endheaders() + + if body: + self.send(body) + + def getresponse(self): + "Get the response from the server." + + # if a prior response has been completed, then forget about it. + if self.__response and self.__response.isclosed(): + self.__response = None + + # + # if a prior response exists, then it must be completed (otherwise, we + # cannot read this response's header to determine the connection-close + # behavior) + # + # note: if a prior response existed, but was connection-close, then the + # socket and response were made independent of this HTTPConnection + # object since a new request requires that we open a whole new + # connection + # + # this means the prior response had one of two states: + # 1) will_close: this connection was reset and the prior socket and + # response operate independently + # 2) persistent: the response was retained and we await its + # isclosed() status to become true. + # + if self.__state != _CS_REQ_SENT or self.__response: + raise ResponseNotReady() + + if self.debuglevel > 0: + response = self.response_class(self.sock, self.debuglevel, + strict=self.strict, + method=self._method) + else: + response = self.response_class(self.sock, strict=self.strict, + method=self._method) + + response.begin() + assert response.will_close != _UNKNOWN + self.__state = _CS_IDLE + + if response.will_close: + # this effectively passes the connection to the response + self.close() + else: + # remember this, so we can tell when it is complete + self.__response = response + + return response + +# The next several classes are used to define FakeSocket,a socket-like +# interface to an SSL connection. + +# The primary complexity comes from faking a makefile() method. The +# standard socket makefile() implementation calls dup() on the socket +# file descriptor. As a consequence, clients can call close() on the +# parent socket and its makefile children in any order. The underlying +# socket isn't closed until they are all closed. + +# The implementation uses reference counting to keep the socket open +# until the last client calls close(). SharedSocket keeps track of +# the reference counting and SharedSocketClient provides an constructor +# and close() method that call incref() and decref() correctly. + +class SharedSocket: + + def __init__(self, sock): + self.sock = sock + self._refcnt = 0 + + def incref(self): + self._refcnt += 1 + + def decref(self): + self._refcnt -= 1 + assert self._refcnt >= 0 + if self._refcnt == 0: + self.sock.close() + + def __del__(self): + self.sock.close() + +class SharedSocketClient: + + def __init__(self, shared): + self._closed = 0 + self._shared = shared + self._shared.incref() + self._sock = shared.sock + + def close(self): + if not self._closed: + self._shared.decref() + self._closed = 1 + self._shared = None + +class SSLFile(SharedSocketClient): + """File-like object wrapping an SSL socket.""" + + BUFSIZE = 8192 + + def __init__(self, sock, ssl, bufsize=None): + SharedSocketClient.__init__(self, sock) + self._ssl = ssl + self._buf = '' + self._bufsize = bufsize or self.__class__.BUFSIZE + + def _read(self): + buf = '' + # put in a loop so that we retry on transient errors + while True: + try: + buf = self._ssl.read(self._bufsize) + except socket.sslerror, err: + if (err[0] == socket.SSL_ERROR_WANT_READ + or err[0] == socket.SSL_ERROR_WANT_WRITE): + continue + if (err[0] == socket.SSL_ERROR_ZERO_RETURN + or err[0] == socket.SSL_ERROR_EOF): + break + raise + except socket.error, err: + if err[0] == errno.EINTR: + continue + if err[0] == errno.EBADF: + # XXX socket was closed? + break + raise + else: + break + return buf + + def read(self, size=None): + L = [self._buf] + avail = len(self._buf) + while size is None or avail < size: + s = self._read() + if s == '': + break + L.append(s) + avail += len(s) + all = "".join(L) + if size is None: + self._buf = '' + return all + else: + self._buf = all[size:] + return all[:size] + + def readline(self): + L = [self._buf] + self._buf = '' + while 1: + i = L[-1].find("\n") + if i >= 0: + break + s = self._read() + if s == '': + break + L.append(s) + if i == -1: + # loop exited because there is no more data + return "".join(L) + else: + all = "".join(L) + # XXX could do enough bookkeeping not to do a 2nd search + i = all.find("\n") + 1 + line = all[:i] + self._buf = all[i:] + return line + + def readlines(self, sizehint=0): + total = 0 + list = [] + while True: + line = self.readline() + if not line: + break + list.append(line) + total += len(line) + if sizehint and total >= sizehint: + break + return list + + def fileno(self): + return self._sock.fileno() + + def __iter__(self): + return self + + def next(self): + line = self.readline() + if not line: + raise StopIteration + return line + +class FakeSocket(SharedSocketClient): + + class _closedsocket: + def __getattr__(self, name): + raise error(9, 'Bad file descriptor') + + def __init__(self, sock, ssl): + sock = SharedSocket(sock) + SharedSocketClient.__init__(self, sock) + self._ssl = ssl + + def close(self): + SharedSocketClient.close(self) + self._sock = self.__class__._closedsocket() + + def makefile(self, mode, bufsize=None): + if mode != 'r' and mode != 'rb': + raise UnimplementedFileMode() + return SSLFile(self._shared, self._ssl, bufsize) + + def send(self, stuff, flags = 0): + return self._ssl.write(stuff) + + sendall = send + + def recv(self, len = 1024, flags = 0): + return self._ssl.read(len) + + def __getattr__(self, attr): + return getattr(self._sock, attr) + + +class HTTPSConnection(HTTPConnection): + "This class allows communication via SSL." + + default_port = HTTPS_PORT + + def __init__(self, host, port=None, key_file=None, cert_file=None, + strict=None): + HTTPConnection.__init__(self, host, port, strict) + self.key_file = key_file + self.cert_file = cert_file + + def connect(self): + "Connect to a host on a given (SSL) port." + + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.connect((self.host, self.port)) + ssl = socket.ssl(sock, self.key_file, self.cert_file) + self.sock = FakeSocket(sock, ssl) + + +class HTTP: + "Compatibility class with httplib.py from 1.5." + + _http_vsn = 10 + _http_vsn_str = 'HTTP/1.0' + + debuglevel = 0 + + _connection_class = HTTPConnection + + def __init__(self, host='', port=None, strict=None): + "Provide a default host, since the superclass requires one." + + # some joker passed 0 explicitly, meaning default port + if port == 0: + port = None + + # Note that we may pass an empty string as the host; this will throw + # an error when we attempt to connect. Presumably, the client code + # will call connect before then, with a proper host. + self._setup(self._connection_class(host, port, strict)) + + def _setup(self, conn): + self._conn = conn + + # set up delegation to flesh out interface + self.send = conn.send + self.putrequest = conn.putrequest + self.endheaders = conn.endheaders + self.set_debuglevel = conn.set_debuglevel + + conn._http_vsn = self._http_vsn + conn._http_vsn_str = self._http_vsn_str + + self.file = None + + def connect(self, host=None, port=None): + "Accept arguments to set the host/port, since the superclass doesn't." + + if host is not None: + self._conn._set_hostport(host, port) + self._conn.connect() + + def getfile(self): + "Provide a getfile, since the superclass' does not use this concept." + return self.file + + def putheader(self, header, *values): + "The superclass allows only one value argument." + self._conn.putheader(header, '\r\n\t'.join(values)) + + def getreply(self): + """Compat definition since superclass does not define it. + + Returns a tuple consisting of: + - server status code (e.g. '200' if all goes well) + - server "reason" corresponding to status code + - any RFC822 headers in the response from the server + """ + try: + response = self._conn.getresponse() + except BadStatusLine, e: + ### hmm. if getresponse() ever closes the socket on a bad request, + ### then we are going to have problems with self.sock + + ### should we keep this behavior? do people use it? + # keep the socket open (as a file), and return it + self.file = self._conn.sock.makefile('rb', 0) + + # close our socket -- we want to restart after any protocol error + self.close() + + self.headers = None + return -1, e.line, None + + self.headers = response.msg + self.file = response.fp + return response.status, response.reason, response.msg + + def close(self): + self._conn.close() + + # note that self.file == response.fp, which gets closed by the + # superclass. just clear the object ref here. + ### hmm. messy. if status==-1, then self.file is owned by us. + ### well... we aren't explicitly closing, but losing this ref will + ### do it + self.file = None + +if hasattr(socket, 'ssl'): + class HTTPS(HTTP): + """Compatibility with 1.5 httplib interface + + Python 1.5.2 did not have an HTTPS class, but it defined an + interface for sending http requests that is also useful for + https. + """ + + _connection_class = HTTPSConnection + + def __init__(self, host='', port=None, key_file=None, cert_file=None, + strict=None): + # provide a default host, pass the X509 cert info + + # urf. compensate for bad input. + if port == 0: + port = None + self._setup(self._connection_class(host, port, key_file, + cert_file, strict)) + + # we never actually use these for anything, but we keep them + # here for compatibility with post-1.5.2 CVS. + self.key_file = key_file + self.cert_file = cert_file + + +class HTTPException(Exception): + # Subclasses that define an __init__ must call Exception.__init__ + # or define self.args. Otherwise, str() will fail. + pass + +class NotConnected(HTTPException): + pass + +class InvalidURL(HTTPException): + pass + +class UnknownProtocol(HTTPException): + def __init__(self, version): + self.args = version, + self.version = version + +class UnknownTransferEncoding(HTTPException): + pass + +class UnimplementedFileMode(HTTPException): + pass + +class IncompleteRead(HTTPException): + def __init__(self, partial): + self.args = partial, + self.partial = partial + +class ImproperConnectionState(HTTPException): + pass + +class CannotSendRequest(ImproperConnectionState): + pass + +class CannotSendHeader(ImproperConnectionState): + pass + +class ResponseNotReady(ImproperConnectionState): + pass + +class BadStatusLine(HTTPException): + def __init__(self, line): + self.args = line, + self.line = line + +# for backwards compatibility +error = HTTPException + +class LineAndFileWrapper: + """A limited file-like object for HTTP/0.9 responses.""" + + # The status-line parsing code calls readline(), which normally + # get the HTTP status line. For a 0.9 response, however, this is + # actually the first line of the body! Clients need to get a + # readable file object that contains that line. + + def __init__(self, line, file): + self._line = line + self._file = file + self._line_consumed = 0 + self._line_offset = 0 + self._line_left = len(line) + + def __getattr__(self, attr): + return getattr(self._file, attr) + + def _done(self): + # called when the last byte is read from the line. After the + # call, all read methods are delegated to the underlying file + # object. + self._line_consumed = 1 + self.read = self._file.read + self.readline = self._file.readline + self.readlines = self._file.readlines + + def read(self, amt=None): + if self._line_consumed: + return self._file.read(amt) + assert self._line_left + if amt is None or amt > self._line_left: + s = self._line[self._line_offset:] + self._done() + if amt is None: + return s + self._file.read() + else: + return s + self._file.read(amt - len(s)) + else: + assert amt <= self._line_left + i = self._line_offset + j = i + amt + s = self._line[i:j] + self._line_offset = j + self._line_left -= amt + if self._line_left == 0: + self._done() + return s + + def readline(self): + if self._line_consumed: + return self._file.readline() + assert self._line_left + s = self._line[self._line_offset:] + self._done() + return s + + def readlines(self, size=None): + if self._line_consumed: + return self._file.readlines(size) + assert self._line_left + L = [self._line[self._line_offset:]] + self._done() + if size is None: + return L + self._file.readlines() + else: + return L + self._file.readlines(size) + +def test(): + """Test this module. + + A hodge podge of tests collected here, because they have too many + external dependencies for the regular test suite. + """ + + import sys + import getopt + opts, args = getopt.getopt(sys.argv[1:], 'd') + dl = 0 + for o, a in opts: + if o == '-d': dl = dl + 1 + host = 'www.python.org' + selector = '/' + if args[0:]: host = args[0] + if args[1:]: selector = args[1] + h = HTTP() + h.set_debuglevel(dl) + h.connect(host) + h.putrequest('GET', selector) + h.endheaders() + status, reason, headers = h.getreply() + print 'status =', status + print 'reason =', reason + print "read", len(h.getfile().read()) + print + if headers: + for header in headers.headers: print header.strip() + print + + # minimal test that code to extract host from url works + class HTTP11(HTTP): + _http_vsn = 11 + _http_vsn_str = 'HTTP/1.1' + + h = HTTP11('www.python.org') + h.putrequest('GET', 'http://www.python.org/~jeremy/') + h.endheaders() + h.getreply() + h.close() + + if hasattr(socket, 'ssl'): + + for host, selector in (('sourceforge.net', '/projects/python'), + ): + print "https://%s%s" % (host, selector) + hs = HTTPS() + hs.set_debuglevel(dl) + hs.connect(host) + hs.putrequest('GET', selector) + hs.endheaders() + status, reason, headers = hs.getreply() + print 'status =', status + print 'reason =', reason + print "read", len(hs.getfile().read()) + print + if headers: + for header in headers.headers: print header.strip() + print + +if __name__ == '__main__': + test() From python-checkins at python.org Thu Jun 15 19:57:47 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Thu, 15 Jun 2006 19:57:47 +0200 (CEST) Subject: [Python-checkins] r46979 - python/branches/hoxworth-stdlib_logging-soc/ihooks.py Message-ID: <20060615175747.BFDB81E4005@bag.python.org> Author: jackilyn.hoxworth Date: Thu Jun 15 19:57:47 2006 New Revision: 46979 Added: python/branches/hoxworth-stdlib_logging-soc/ihooks.py Log: Added a file remotely Added: python/branches/hoxworth-stdlib_logging-soc/ihooks.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/ihooks.py Thu Jun 15 19:57:47 2006 @@ -0,0 +1,522 @@ +"""Import hook support. + +Consistent use of this module will make it possible to change the +different mechanisms involved in loading modules independently. + +While the built-in module imp exports interfaces to the built-in +module searching and loading algorithm, and it is possible to replace +the built-in function __import__ in order to change the semantics of +the import statement, until now it has been difficult to combine the +effect of different __import__ hacks, like loading modules from URLs +by rimport.py, or restricted execution by rexec.py. + +This module defines three new concepts: + +1) A "file system hooks" class provides an interface to a filesystem. + +One hooks class is defined (Hooks), which uses the interface provided +by standard modules os and os.path. It should be used as the base +class for other hooks classes. + +2) A "module loader" class provides an interface to search for a +module in a search path and to load it. It defines a method which +searches for a module in a single directory; by overriding this method +one can redefine the details of the search. If the directory is None, +built-in and frozen modules are searched instead. + +Two module loader class are defined, both implementing the search +strategy used by the built-in __import__ function: ModuleLoader uses +the imp module's find_module interface, while HookableModuleLoader +uses a file system hooks class to interact with the file system. Both +use the imp module's load_* interfaces to actually load the module. + +3) A "module importer" class provides an interface to import a +module, as well as interfaces to reload and unload a module. It also +provides interfaces to install and uninstall itself instead of the +default __import__ and reload (and unload) functions. + +One module importer class is defined (ModuleImporter), which uses a +module loader instance passed in (by default HookableModuleLoader is +instantiated). + +The classes defined here should be used as base classes for extended +functionality along those lines. + +If a module importer class supports dotted names, its import_module() +must return a different value depending on whether it is called on +behalf of a "from ... import ..." statement or not. (This is caused +by the way the __import__ hook is used by the Python interpreter.) It +would also do wise to install a different version of reload(). + +""" + + +import __builtin__ +import imp +import os +import sys +import logging +_log = logging.getLogger('py.ihooks') + +__all__ = ["BasicModuleLoader","Hooks","ModuleLoader","FancyModuleLoader", + "BasicModuleImporter","ModuleImporter","install","uninstall"] + +VERBOSE = 0 + + +from imp import C_EXTENSION, PY_SOURCE, PY_COMPILED +from imp import C_BUILTIN, PY_FROZEN, PKG_DIRECTORY +BUILTIN_MODULE = C_BUILTIN +FROZEN_MODULE = PY_FROZEN + + +class _Verbose: + + def __init__(self, verbose = VERBOSE): + self.verbose = verbose + + def get_verbose(self): + return self.verbose + + def set_verbose(self, verbose): + self.verbose = verbose + + # XXX The following is an experimental interface + + def note(self, *args): + if self.verbose: + self.message(*args) + + def message(self, format, *args): + if args: + _log.info(format%args) + else: + _log.info(format) + + +class BasicModuleLoader(_Verbose): + + """Basic module loader. + + This provides the same functionality as built-in import. It + doesn't deal with checking sys.modules -- all it provides is + find_module() and a load_module(), as well as find_module_in_dir() + which searches just one directory, and can be overridden by a + derived class to change the module search algorithm when the basic + dependency on sys.path is unchanged. + + The interface is a little more convenient than imp's: + find_module(name, [path]) returns None or 'stuff', and + load_module(name, stuff) loads the module. + + """ + + def find_module(self, name, path = None): + if path is None: + path = [None] + self.default_path() + for dir in path: + stuff = self.find_module_in_dir(name, dir) + if stuff: return stuff + return None + + def default_path(self): + return sys.path + + def find_module_in_dir(self, name, dir): + if dir is None: + return self.find_builtin_module(name) + else: + try: + return imp.find_module(name, [dir]) + except ImportError: + return None + + def find_builtin_module(self, name): + # XXX frozen packages? + if imp.is_builtin(name): + return None, '', ('', '', BUILTIN_MODULE) + if imp.is_frozen(name): + return None, '', ('', '', FROZEN_MODULE) + return None + + def load_module(self, name, stuff): + file, filename, info = stuff + try: + return imp.load_module(name, file, filename, info) + finally: + if file: file.close() + + +class Hooks(_Verbose): + + """Hooks into the filesystem and interpreter. + + By deriving a subclass you can redefine your filesystem interface, + e.g. to merge it with the URL space. + + This base class behaves just like the native filesystem. + + """ + + # imp interface + def get_suffixes(self): return imp.get_suffixes() + def new_module(self, name): return imp.new_module(name) + def is_builtin(self, name): return imp.is_builtin(name) + def init_builtin(self, name): return imp.init_builtin(name) + def is_frozen(self, name): return imp.is_frozen(name) + def init_frozen(self, name): return imp.init_frozen(name) + def get_frozen_object(self, name): return imp.get_frozen_object(name) + def load_source(self, name, filename, file=None): + return imp.load_source(name, filename, file) + def load_compiled(self, name, filename, file=None): + return imp.load_compiled(name, filename, file) + def load_dynamic(self, name, filename, file=None): + return imp.load_dynamic(name, filename, file) + def load_package(self, name, filename, file=None): + return imp.load_module(name, file, filename, ("", "", PKG_DIRECTORY)) + + def add_module(self, name): + d = self.modules_dict() + if name in d: return d[name] + d[name] = m = self.new_module(name) + return m + + # sys interface + def modules_dict(self): return sys.modules + def default_path(self): return sys.path + + def path_split(self, x): return os.path.split(x) + def path_join(self, x, y): return os.path.join(x, y) + def path_isabs(self, x): return os.path.isabs(x) + # etc. + + def path_exists(self, x): return os.path.exists(x) + def path_isdir(self, x): return os.path.isdir(x) + def path_isfile(self, x): return os.path.isfile(x) + def path_islink(self, x): return os.path.islink(x) + # etc. + + def openfile(self, *x): return open(*x) + openfile_error = IOError + def listdir(self, x): return os.listdir(x) + listdir_error = os.error + # etc. + + +class ModuleLoader(BasicModuleLoader): + + """Default module loader; uses file system hooks. + + By defining suitable hooks, you might be able to load modules from + other sources than the file system, e.g. from compressed or + encrypted files, tar files or (if you're brave!) URLs. + + """ + + def __init__(self, hooks = None, verbose = VERBOSE): + BasicModuleLoader.__init__(self, verbose) + self.hooks = hooks or Hooks(verbose) + + def default_path(self): + return self.hooks.default_path() + + def modules_dict(self): + return self.hooks.modules_dict() + + def get_hooks(self): + return self.hooks + + def set_hooks(self, hooks): + self.hooks = hooks + + def find_builtin_module(self, name): + # XXX frozen packages? + if self.hooks.is_builtin(name): + return None, '', ('', '', BUILTIN_MODULE) + if self.hooks.is_frozen(name): + return None, '', ('', '', FROZEN_MODULE) + return None + + def find_module_in_dir(self, name, dir, allow_packages=1): + if dir is None: + return self.find_builtin_module(name) + if allow_packages: + fullname = self.hooks.path_join(dir, name) + if self.hooks.path_isdir(fullname): + stuff = self.find_module_in_dir("__init__", fullname, 0) + if stuff: + file = stuff[0] + if file: file.close() + return None, fullname, ('', '', PKG_DIRECTORY) + for info in self.hooks.get_suffixes(): + suff, mode, type = info + fullname = self.hooks.path_join(dir, name+suff) + try: + fp = self.hooks.openfile(fullname, mode) + return fp, fullname, info + except self.hooks.openfile_error: + pass + return None + + def load_module(self, name, stuff): + file, filename, info = stuff + (suff, mode, type) = info + try: + if type == BUILTIN_MODULE: + return self.hooks.init_builtin(name) + if type == FROZEN_MODULE: + return self.hooks.init_frozen(name) + if type == C_EXTENSION: + m = self.hooks.load_dynamic(name, filename, file) + elif type == PY_SOURCE: + m = self.hooks.load_source(name, filename, file) + elif type == PY_COMPILED: + m = self.hooks.load_compiled(name, filename, file) + elif type == PKG_DIRECTORY: + m = self.hooks.load_package(name, filename, file) + else: + raise ImportError, "Unrecognized module type (%r) for %s" % \ + (type, name) + finally: + if file: file.close() + m.__file__ = filename + return m + + +class FancyModuleLoader(ModuleLoader): + + """Fancy module loader -- parses and execs the code itself.""" + + def load_module(self, name, stuff): + file, filename, (suff, mode, type) = stuff + realfilename = filename + path = None + + if type == PKG_DIRECTORY: + initstuff = self.find_module_in_dir("__init__", filename, 0) + if not initstuff: + raise ImportError, "No __init__ module in package %s" % name + initfile, initfilename, initinfo = initstuff + initsuff, initmode, inittype = initinfo + if inittype not in (PY_COMPILED, PY_SOURCE): + if initfile: initfile.close() + raise ImportError, \ + "Bad type (%r) for __init__ module in package %s" % ( + inittype, name) + path = [filename] + file = initfile + realfilename = initfilename + type = inittype + + if type == FROZEN_MODULE: + code = self.hooks.get_frozen_object(name) + elif type == PY_COMPILED: + import marshal + file.seek(8) + code = marshal.load(file) + elif type == PY_SOURCE: + data = file.read() + code = compile(data, realfilename, 'exec') + else: + return ModuleLoader.load_module(self, name, stuff) + + m = self.hooks.add_module(name) + if path: + m.__path__ = path + m.__file__ = filename + try: + exec code in m.__dict__ + except: + d = self.hooks.modules_dict() + if name in d: + del d[name] + raise + return m + + +class BasicModuleImporter(_Verbose): + + """Basic module importer; uses module loader. + + This provides basic import facilities but no package imports. + + """ + + def __init__(self, loader = None, verbose = VERBOSE): + _Verbose.__init__(self, verbose) + self.loader = loader or ModuleLoader(None, verbose) + self.modules = self.loader.modules_dict() + + def get_loader(self): + return self.loader + + def set_loader(self, loader): + self.loader = loader + + def get_hooks(self): + return self.loader.get_hooks() + + def set_hooks(self, hooks): + return self.loader.set_hooks(hooks) + + def import_module(self, name, globals={}, locals={}, fromlist=[]): + name = str(name) + if name in self.modules: + return self.modules[name] # Fast path + stuff = self.loader.find_module(name) + if not stuff: + raise ImportError, "No module named %s" % name + return self.loader.load_module(name, stuff) + + def reload(self, module, path = None): + name = str(module.__name__) + stuff = self.loader.find_module(name, path) + if not stuff: + raise ImportError, "Module %s not found for reload" % name + return self.loader.load_module(name, stuff) + + def unload(self, module): + del self.modules[str(module.__name__)] + # XXX Should this try to clear the module's namespace? + + def install(self): + self.save_import_module = __builtin__.__import__ + self.save_reload = __builtin__.reload + if not hasattr(__builtin__, 'unload'): + __builtin__.unload = None + self.save_unload = __builtin__.unload + __builtin__.__import__ = self.import_module + __builtin__.reload = self.reload + __builtin__.unload = self.unload + + def uninstall(self): + __builtin__.__import__ = self.save_import_module + __builtin__.reload = self.save_reload + __builtin__.unload = self.save_unload + if not __builtin__.unload: + del __builtin__.unload + + +class ModuleImporter(BasicModuleImporter): + + """A module importer that supports packages.""" + + def import_module(self, name, globals=None, locals=None, fromlist=None): + parent = self.determine_parent(globals) + q, tail = self.find_head_package(parent, str(name)) + m = self.load_tail(q, tail) + if not fromlist: + return q + if hasattr(m, "__path__"): + self.ensure_fromlist(m, fromlist) + return m + + def determine_parent(self, globals): + if not globals or not "__name__" in globals: + return None + pname = globals['__name__'] + if "__path__" in globals: + parent = self.modules[pname] + assert globals is parent.__dict__ + return parent + if '.' in pname: + i = pname.rfind('.') + pname = pname[:i] + parent = self.modules[pname] + assert parent.__name__ == pname + return parent + return None + + def find_head_package(self, parent, name): + if '.' in name: + i = name.find('.') + head = name[:i] + tail = name[i+1:] + else: + head = name + tail = "" + if parent: + qname = "%s.%s" % (parent.__name__, head) + else: + qname = head + q = self.import_it(head, qname, parent) + if q: return q, tail + if parent: + qname = head + parent = None + q = self.import_it(head, qname, parent) + if q: return q, tail + raise ImportError, "No module named " + qname + + def load_tail(self, q, tail): + m = q + while tail: + i = tail.find('.') + if i < 0: i = len(tail) + head, tail = tail[:i], tail[i+1:] + mname = "%s.%s" % (m.__name__, head) + m = self.import_it(head, mname, m) + if not m: + raise ImportError, "No module named " + mname + return m + + def ensure_fromlist(self, m, fromlist, recursive=0): + for sub in fromlist: + if sub == "*": + if not recursive: + try: + all = m.__all__ + except AttributeError: + pass + else: + self.ensure_fromlist(m, all, 1) + continue + if sub != "*" and not hasattr(m, sub): + subname = "%s.%s" % (m.__name__, sub) + submod = self.import_it(sub, subname, m) + if not submod: + raise ImportError, "No module named " + subname + + def import_it(self, partname, fqname, parent, force_load=0): + if not partname: + raise ValueError, "Empty module name" + if not force_load: + try: + return self.modules[fqname] + except KeyError: + pass + try: + path = parent and parent.__path__ + except AttributeError: + return None + partname = str(partname) + stuff = self.loader.find_module(partname, path) + if not stuff: + return None + fqname = str(fqname) + m = self.loader.load_module(fqname, stuff) + if parent: + setattr(parent, partname, m) + return m + + def reload(self, module): + name = str(module.__name__) + if '.' not in name: + return self.import_it(name, name, None, force_load=1) + i = name.rfind('.') + pname = name[:i] + parent = self.modules[pname] + return self.import_it(name[i+1:], name, parent, force_load=1) + + +default_importer = None +current_importer = None + +def install(importer = None): + global current_importer + current_importer = importer or default_importer or ModuleImporter() + current_importer.install() + +def uninstall(): + global current_importer + current_importer.uninstall() From python-checkins at python.org Thu Jun 15 19:58:14 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Thu, 15 Jun 2006 19:58:14 +0200 (CEST) Subject: [Python-checkins] r46980 - python/branches/hoxworth-stdlib_logging-soc/mhlib.py Message-ID: <20060615175814.BFDB51E4005@bag.python.org> Author: jackilyn.hoxworth Date: Thu Jun 15 19:58:13 2006 New Revision: 46980 Added: python/branches/hoxworth-stdlib_logging-soc/mhlib.py Log: Added a file remotely Added: python/branches/hoxworth-stdlib_logging-soc/mhlib.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/mhlib.py Thu Jun 15 19:58:13 2006 @@ -0,0 +1,1003 @@ +"""MH interface -- purely object-oriented (well, almost) + +Executive summary: + +import mhlib + +mh = mhlib.MH() # use default mailbox directory and profile +mh = mhlib.MH(mailbox) # override mailbox location (default from profile) +mh = mhlib.MH(mailbox, profile) # override mailbox and profile + +mh.error(format, ...) # print error message -- can be overridden +s = mh.getprofile(key) # profile entry (None if not set) +path = mh.getpath() # mailbox pathname +name = mh.getcontext() # name of current folder +mh.setcontext(name) # set name of current folder + +list = mh.listfolders() # names of top-level folders +list = mh.listallfolders() # names of all folders, including subfolders +list = mh.listsubfolders(name) # direct subfolders of given folder +list = mh.listallsubfolders(name) # all subfolders of given folder + +mh.makefolder(name) # create new folder +mh.deletefolder(name) # delete folder -- must have no subfolders + +f = mh.openfolder(name) # new open folder object + +f.error(format, ...) # same as mh.error(format, ...) +path = f.getfullname() # folder's full pathname +path = f.getsequencesfilename() # full pathname of folder's sequences file +path = f.getmessagefilename(n) # full pathname of message n in folder + +list = f.listmessages() # list of messages in folder (as numbers) +n = f.getcurrent() # get current message +f.setcurrent(n) # set current message +list = f.parsesequence(seq) # parse msgs syntax into list of messages +n = f.getlast() # get last message (0 if no messagse) +f.setlast(n) # set last message (internal use only) + +dict = f.getsequences() # dictionary of sequences in folder {name: list} +f.putsequences(dict) # write sequences back to folder + +f.createmessage(n, fp) # add message from file f as number n +f.removemessages(list) # remove messages in list from folder +f.refilemessages(list, tofolder) # move messages in list to other folder +f.movemessage(n, tofolder, ton) # move one message to a given destination +f.copymessage(n, tofolder, ton) # copy one message to a given destination + +m = f.openmessage(n) # new open message object (costs a file descriptor) +m is a derived class of mimetools.Message(rfc822.Message), with: +s = m.getheadertext() # text of message's headers +s = m.getheadertext(pred) # text of message's headers, filtered by pred +s = m.getbodytext() # text of message's body, decoded +s = m.getbodytext(0) # text of message's body, not decoded +""" + +# XXX To do, functionality: +# - annotate messages +# - send messages +# +# XXX To do, organization: +# - move IntSet to separate file +# - move most Message functionality to module mimetools + + +# Customizable defaults + +MH_PROFILE = '~/.mh_profile' +PATH = '~/Mail' +MH_SEQUENCES = '.mh_sequences' +FOLDER_PROTECT = 0700 + + +# Imported modules + +import os +import sys +import re +import mimetools +import multifile +import shutil +from bisect import bisect +import logging +_log = logging.getLogger('py.mhlib') + +__all__ = ["MH","Error","Folder","Message"] + +# Exported constants + +class Error(Exception): + pass + + +class MH: + """Class representing a particular collection of folders. + Optional constructor arguments are the pathname for the directory + containing the collection, and the MH profile to use. + If either is omitted or empty a default is used; the default + directory is taken from the MH profile if it is specified there.""" + + def __init__(self, path = None, profile = None): + """Constructor.""" + if profile is None: profile = MH_PROFILE + self.profile = os.path.expanduser(profile) + if path is None: path = self.getprofile('Path') + if not path: path = PATH + if not os.path.isabs(path) and path[0] != '~': + path = os.path.join('~', path) + path = os.path.expanduser(path) + if not os.path.isdir(path): raise Error, 'MH() path not found' + self.path = path + + def __repr__(self): + """String representation.""" + return 'MH(%r, %r)' % (self.path, self.profile) + + def error(self, msg, *args): + """Routine to print an error. May be overridden by a derived class.""" + _log.info('MH error: %s\n' % (msg % args)) + + def getprofile(self, key): + """Return a profile entry, None if not found.""" + return pickline(self.profile, key) + + def getpath(self): + """Return the path (the name of the collection's directory).""" + return self.path + + def getcontext(self): + """Return the name of the current folder.""" + context = pickline(os.path.join(self.getpath(), 'context'), + 'Current-Folder') + if not context: context = 'inbox' + return context + + def setcontext(self, context): + """Set the name of the current folder.""" + fn = os.path.join(self.getpath(), 'context') + f = open(fn, "w") + f.write("Current-Folder: %s\n" % context) + f.close() + + def listfolders(self): + """Return the names of the top-level folders.""" + folders = [] + path = self.getpath() + for name in os.listdir(path): + fullname = os.path.join(path, name) + if os.path.isdir(fullname): + folders.append(name) + folders.sort() + return folders + + def listsubfolders(self, name): + """Return the names of the subfolders in a given folder + (prefixed with the given folder name).""" + fullname = os.path.join(self.path, name) + # Get the link count so we can avoid listing folders + # that have no subfolders. + nlinks = os.stat(fullname).st_nlink + if nlinks <= 2: + return [] + subfolders = [] + subnames = os.listdir(fullname) + for subname in subnames: + fullsubname = os.path.join(fullname, subname) + if os.path.isdir(fullsubname): + name_subname = os.path.join(name, subname) + subfolders.append(name_subname) + # Stop looking for subfolders when + # we've seen them all + nlinks = nlinks - 1 + if nlinks <= 2: + break + subfolders.sort() + return subfolders + + def listallfolders(self): + """Return the names of all folders and subfolders, recursively.""" + return self.listallsubfolders('') + + def listallsubfolders(self, name): + """Return the names of subfolders in a given folder, recursively.""" + fullname = os.path.join(self.path, name) + # Get the link count so we can avoid listing folders + # that have no subfolders. + nlinks = os.stat(fullname).st_nlink + if nlinks <= 2: + return [] + subfolders = [] + subnames = os.listdir(fullname) + for subname in subnames: + if subname[0] == ',' or isnumeric(subname): continue + fullsubname = os.path.join(fullname, subname) + if os.path.isdir(fullsubname): + name_subname = os.path.join(name, subname) + subfolders.append(name_subname) + if not os.path.islink(fullsubname): + subsubfolders = self.listallsubfolders( + name_subname) + subfolders = subfolders + subsubfolders + # Stop looking for subfolders when + # we've seen them all + nlinks = nlinks - 1 + if nlinks <= 2: + break + subfolders.sort() + return subfolders + + def openfolder(self, name): + """Return a new Folder object for the named folder.""" + return Folder(self, name) + + def makefolder(self, name): + """Create a new folder (or raise os.error if it cannot be created).""" + protect = pickline(self.profile, 'Folder-Protect') + if protect and isnumeric(protect): + mode = int(protect, 8) + else: + mode = FOLDER_PROTECT + os.mkdir(os.path.join(self.getpath(), name), mode) + + def deletefolder(self, name): + """Delete a folder. This removes files in the folder but not + subdirectories. Raise os.error if deleting the folder itself fails.""" + fullname = os.path.join(self.getpath(), name) + for subname in os.listdir(fullname): + fullsubname = os.path.join(fullname, subname) + try: + os.unlink(fullsubname) + except os.error: + self.error('%s not deleted, continuing...' % + fullsubname) + os.rmdir(fullname) + + +numericprog = re.compile('^[1-9][0-9]*$') +def isnumeric(str): + return numericprog.match(str) is not None + +class Folder: + """Class representing a particular folder.""" + + def __init__(self, mh, name): + """Constructor.""" + self.mh = mh + self.name = name + if not os.path.isdir(self.getfullname()): + raise Error, 'no folder %s' % name + + def __repr__(self): + """String representation.""" + return 'Folder(%r, %r)' % (self.mh, self.name) + + def error(self, *args): + """Error message handler.""" + self.mh.error(*args) + + def getfullname(self): + """Return the full pathname of the folder.""" + return os.path.join(self.mh.path, self.name) + + def getsequencesfilename(self): + """Return the full pathname of the folder's sequences file.""" + return os.path.join(self.getfullname(), MH_SEQUENCES) + + def getmessagefilename(self, n): + """Return the full pathname of a message in the folder.""" + return os.path.join(self.getfullname(), str(n)) + + def listsubfolders(self): + """Return list of direct subfolders.""" + return self.mh.listsubfolders(self.name) + + def listallsubfolders(self): + """Return list of all subfolders.""" + return self.mh.listallsubfolders(self.name) + + def listmessages(self): + """Return the list of messages currently present in the folder. + As a side effect, set self.last to the last message (or 0).""" + messages = [] + match = numericprog.match + append = messages.append + for name in os.listdir(self.getfullname()): + if match(name): + append(name) + messages = map(int, messages) + messages.sort() + if messages: + self.last = messages[-1] + else: + self.last = 0 + return messages + + def getsequences(self): + """Return the set of sequences for the folder.""" + sequences = {} + fullname = self.getsequencesfilename() + try: + f = open(fullname, 'r') + except IOError: + return sequences + while 1: + line = f.readline() + if not line: break + fields = line.split(':') + if len(fields) != 2: + self.error('bad sequence in %s: %s' % + (fullname, line.strip())) + key = fields[0].strip() + value = IntSet(fields[1].strip(), ' ').tolist() + sequences[key] = value + return sequences + + def putsequences(self, sequences): + """Write the set of sequences back to the folder.""" + fullname = self.getsequencesfilename() + f = None + for key, seq in sequences.iteritems(): + s = IntSet('', ' ') + s.fromlist(seq) + if not f: f = open(fullname, 'w') + f.write('%s: %s\n' % (key, s.tostring())) + if not f: + try: + os.unlink(fullname) + except os.error: + pass + else: + f.close() + + def getcurrent(self): + """Return the current message. Raise Error when there is none.""" + seqs = self.getsequences() + try: + return max(seqs['cur']) + except (ValueError, KeyError): + raise Error, "no cur message" + + def setcurrent(self, n): + """Set the current message.""" + updateline(self.getsequencesfilename(), 'cur', str(n), 0) + + def parsesequence(self, seq): + """Parse an MH sequence specification into a message list. + Attempt to mimic mh-sequence(5) as close as possible. + Also attempt to mimic observed behavior regarding which + conditions cause which error messages.""" + # XXX Still not complete (see mh-format(5)). + # Missing are: + # - 'prev', 'next' as count + # - Sequence-Negation option + all = self.listmessages() + # Observed behavior: test for empty folder is done first + if not all: + raise Error, "no messages in %s" % self.name + # Common case first: all is frequently the default + if seq == 'all': + return all + # Test for X:Y before X-Y because 'seq:-n' matches both + i = seq.find(':') + if i >= 0: + head, dir, tail = seq[:i], '', seq[i+1:] + if tail[:1] in '-+': + dir, tail = tail[:1], tail[1:] + if not isnumeric(tail): + raise Error, "bad message list %s" % seq + try: + count = int(tail) + except (ValueError, OverflowError): + # Can't use sys.maxint because of i+count below + count = len(all) + try: + anchor = self._parseindex(head, all) + except Error, msg: + seqs = self.getsequences() + if not head in seqs: + if not msg: + msg = "bad message list %s" % seq + raise Error, msg, sys.exc_info()[2] + msgs = seqs[head] + if not msgs: + raise Error, "sequence %s empty" % head + if dir == '-': + return msgs[-count:] + else: + return msgs[:count] + else: + if not dir: + if head in ('prev', 'last'): + dir = '-' + if dir == '-': + i = bisect(all, anchor) + return all[max(0, i-count):i] + else: + i = bisect(all, anchor-1) + return all[i:i+count] + # Test for X-Y next + i = seq.find('-') + if i >= 0: + begin = self._parseindex(seq[:i], all) + end = self._parseindex(seq[i+1:], all) + i = bisect(all, begin-1) + j = bisect(all, end) + r = all[i:j] + if not r: + raise Error, "bad message list %s" % seq + return r + # Neither X:Y nor X-Y; must be a number or a (pseudo-)sequence + try: + n = self._parseindex(seq, all) + except Error, msg: + seqs = self.getsequences() + if not seq in seqs: + if not msg: + msg = "bad message list %s" % seq + raise Error, msg + return seqs[seq] + else: + if n not in all: + if isnumeric(seq): + raise Error, "message %d doesn't exist" % n + else: + raise Error, "no %s message" % seq + else: + return [n] + + def _parseindex(self, seq, all): + """Internal: parse a message number (or cur, first, etc.).""" + if isnumeric(seq): + try: + return int(seq) + except (OverflowError, ValueError): + return sys.maxint + if seq in ('cur', '.'): + return self.getcurrent() + if seq == 'first': + return all[0] + if seq == 'last': + return all[-1] + if seq == 'next': + n = self.getcurrent() + i = bisect(all, n) + try: + return all[i] + except IndexError: + raise Error, "no next message" + if seq == 'prev': + n = self.getcurrent() + i = bisect(all, n-1) + if i == 0: + raise Error, "no prev message" + try: + return all[i-1] + except IndexError: + raise Error, "no prev message" + raise Error, None + + def openmessage(self, n): + """Open a message -- returns a Message object.""" + return Message(self, n) + + def removemessages(self, list): + """Remove one or more messages -- may raise os.error.""" + errors = [] + deleted = [] + for n in list: + path = self.getmessagefilename(n) + commapath = self.getmessagefilename(',' + str(n)) + try: + os.unlink(commapath) + except os.error: + pass + try: + os.rename(path, commapath) + except os.error, msg: + errors.append(msg) + else: + deleted.append(n) + if deleted: + self.removefromallsequences(deleted) + if errors: + if len(errors) == 1: + raise os.error, errors[0] + else: + raise os.error, ('multiple errors:', errors) + + def refilemessages(self, list, tofolder, keepsequences=0): + """Refile one or more messages -- may raise os.error. + 'tofolder' is an open folder object.""" + errors = [] + refiled = {} + for n in list: + ton = tofolder.getlast() + 1 + path = self.getmessagefilename(n) + topath = tofolder.getmessagefilename(ton) + try: + os.rename(path, topath) + except os.error: + # Try copying + try: + shutil.copy2(path, topath) + os.unlink(path) + except (IOError, os.error), msg: + errors.append(msg) + try: + os.unlink(topath) + except os.error: + pass + continue + tofolder.setlast(ton) + refiled[n] = ton + if refiled: + if keepsequences: + tofolder._copysequences(self, refiled.items()) + self.removefromallsequences(refiled.keys()) + if errors: + if len(errors) == 1: + raise os.error, errors[0] + else: + raise os.error, ('multiple errors:', errors) + + def _copysequences(self, fromfolder, refileditems): + """Helper for refilemessages() to copy sequences.""" + fromsequences = fromfolder.getsequences() + tosequences = self.getsequences() + changed = 0 + for name, seq in fromsequences.items(): + try: + toseq = tosequences[name] + new = 0 + except KeyError: + toseq = [] + new = 1 + for fromn, ton in refileditems: + if fromn in seq: + toseq.append(ton) + changed = 1 + if new and toseq: + tosequences[name] = toseq + if changed: + self.putsequences(tosequences) + + def movemessage(self, n, tofolder, ton): + """Move one message over a specific destination message, + which may or may not already exist.""" + path = self.getmessagefilename(n) + # Open it to check that it exists + f = open(path) + f.close() + del f + topath = tofolder.getmessagefilename(ton) + backuptopath = tofolder.getmessagefilename(',%d' % ton) + try: + os.rename(topath, backuptopath) + except os.error: + pass + try: + os.rename(path, topath) + except os.error: + # Try copying + ok = 0 + try: + tofolder.setlast(None) + shutil.copy2(path, topath) + ok = 1 + finally: + if not ok: + try: + os.unlink(topath) + except os.error: + pass + os.unlink(path) + self.removefromallsequences([n]) + + def copymessage(self, n, tofolder, ton): + """Copy one message over a specific destination message, + which may or may not already exist.""" + path = self.getmessagefilename(n) + # Open it to check that it exists + f = open(path) + f.close() + del f + topath = tofolder.getmessagefilename(ton) + backuptopath = tofolder.getmessagefilename(',%d' % ton) + try: + os.rename(topath, backuptopath) + except os.error: + pass + ok = 0 + try: + tofolder.setlast(None) + shutil.copy2(path, topath) + ok = 1 + finally: + if not ok: + try: + os.unlink(topath) + except os.error: + pass + + def createmessage(self, n, txt): + """Create a message, with text from the open file txt.""" + path = self.getmessagefilename(n) + backuppath = self.getmessagefilename(',%d' % n) + try: + os.rename(path, backuppath) + except os.error: + pass + ok = 0 + BUFSIZE = 16*1024 + try: + f = open(path, "w") + while 1: + buf = txt.read(BUFSIZE) + if not buf: + break + f.write(buf) + f.close() + ok = 1 + finally: + if not ok: + try: + os.unlink(path) + except os.error: + pass + + def removefromallsequences(self, list): + """Remove one or more messages from all sequences (including last) + -- but not from 'cur'!!!""" + if hasattr(self, 'last') and self.last in list: + del self.last + sequences = self.getsequences() + changed = 0 + for name, seq in sequences.items(): + if name == 'cur': + continue + for n in list: + if n in seq: + seq.remove(n) + changed = 1 + if not seq: + del sequences[name] + if changed: + self.putsequences(sequences) + + def getlast(self): + """Return the last message number.""" + if not hasattr(self, 'last'): + self.listmessages() # Set self.last + return self.last + + def setlast(self, last): + """Set the last message number.""" + if last is None: + if hasattr(self, 'last'): + del self.last + else: + self.last = last + +class Message(mimetools.Message): + + def __init__(self, f, n, fp = None): + """Constructor.""" + self.folder = f + self.number = n + if fp is None: + path = f.getmessagefilename(n) + fp = open(path, 'r') + mimetools.Message.__init__(self, fp) + + def __repr__(self): + """String representation.""" + return 'Message(%s, %s)' % (repr(self.folder), self.number) + + def getheadertext(self, pred = None): + """Return the message's header text as a string. If an + argument is specified, it is used as a filter predicate to + decide which headers to return (its argument is the header + name converted to lower case).""" + if pred is None: + return ''.join(self.headers) + headers = [] + hit = 0 + for line in self.headers: + if not line[0].isspace(): + i = line.find(':') + if i > 0: + hit = pred(line[:i].lower()) + if hit: headers.append(line) + return ''.join(headers) + + def getbodytext(self, decode = 1): + """Return the message's body text as string. This undoes a + Content-Transfer-Encoding, but does not interpret other MIME + features (e.g. multipart messages). To suppress decoding, + pass 0 as an argument.""" + self.fp.seek(self.startofbody) + encoding = self.getencoding() + if not decode or encoding in ('', '7bit', '8bit', 'binary'): + return self.fp.read() + try: + from cStringIO import StringIO + except ImportError: + from StringIO import StringIO + output = StringIO() + mimetools.decode(self.fp, output, encoding) + return output.getvalue() + + def getbodyparts(self): + """Only for multipart messages: return the message's body as a + list of SubMessage objects. Each submessage object behaves + (almost) as a Message object.""" + if self.getmaintype() != 'multipart': + raise Error, 'Content-Type is not multipart/*' + bdry = self.getparam('boundary') + if not bdry: + raise Error, 'multipart/* without boundary param' + self.fp.seek(self.startofbody) + mf = multifile.MultiFile(self.fp) + mf.push(bdry) + parts = [] + while mf.next(): + n = "%s.%r" % (self.number, 1 + len(parts)) + part = SubMessage(self.folder, n, mf) + parts.append(part) + mf.pop() + return parts + + def getbody(self): + """Return body, either a string or a list of messages.""" + if self.getmaintype() == 'multipart': + return self.getbodyparts() + else: + return self.getbodytext() + + +class SubMessage(Message): + + def __init__(self, f, n, fp): + """Constructor.""" + Message.__init__(self, f, n, fp) + if self.getmaintype() == 'multipart': + self.body = Message.getbodyparts(self) + else: + self.body = Message.getbodytext(self) + self.bodyencoded = Message.getbodytext(self, decode=0) + # XXX If this is big, should remember file pointers + + def __repr__(self): + """String representation.""" + f, n, fp = self.folder, self.number, self.fp + return 'SubMessage(%s, %s, %s)' % (f, n, fp) + + def getbodytext(self, decode = 1): + if not decode: + return self.bodyencoded + if type(self.body) == type(''): + return self.body + + def getbodyparts(self): + if type(self.body) == type([]): + return self.body + + def getbody(self): + return self.body + + +class IntSet: + """Class implementing sets of integers. + + This is an efficient representation for sets consisting of several + continuous ranges, e.g. 1-100,200-400,402-1000 is represented + internally as a list of three pairs: [(1,100), (200,400), + (402,1000)]. The internal representation is always kept normalized. + + The constructor has up to three arguments: + - the string used to initialize the set (default ''), + - the separator between ranges (default ',') + - the separator between begin and end of a range (default '-') + The separators must be strings (not regexprs) and should be different. + + The tostring() function yields a string that can be passed to another + IntSet constructor; __repr__() is a valid IntSet constructor itself. + """ + + # XXX The default begin/end separator means that negative numbers are + # not supported very well. + # + # XXX There are currently no operations to remove set elements. + + def __init__(self, data = None, sep = ',', rng = '-'): + self.pairs = [] + self.sep = sep + self.rng = rng + if data: self.fromstring(data) + + def reset(self): + self.pairs = [] + + def __cmp__(self, other): + return cmp(self.pairs, other.pairs) + + def __hash__(self): + return hash(self.pairs) + + def __repr__(self): + return 'IntSet(%r, %r, %r)' % (self.tostring(), self.sep, self.rng) + + def normalize(self): + self.pairs.sort() + i = 1 + while i < len(self.pairs): + alo, ahi = self.pairs[i-1] + blo, bhi = self.pairs[i] + if ahi >= blo-1: + self.pairs[i-1:i+1] = [(alo, max(ahi, bhi))] + else: + i = i+1 + + def tostring(self): + s = '' + for lo, hi in self.pairs: + if lo == hi: t = repr(lo) + else: t = repr(lo) + self.rng + repr(hi) + if s: s = s + (self.sep + t) + else: s = t + return s + + def tolist(self): + l = [] + for lo, hi in self.pairs: + m = range(lo, hi+1) + l = l + m + return l + + def fromlist(self, list): + for i in list: + self.append(i) + + def clone(self): + new = IntSet() + new.pairs = self.pairs[:] + return new + + def min(self): + return self.pairs[0][0] + + def max(self): + return self.pairs[-1][-1] + + def contains(self, x): + for lo, hi in self.pairs: + if lo <= x <= hi: return True + return False + + def append(self, x): + for i in range(len(self.pairs)): + lo, hi = self.pairs[i] + if x < lo: # Need to insert before + if x+1 == lo: + self.pairs[i] = (x, hi) + else: + self.pairs.insert(i, (x, x)) + if i > 0 and x-1 == self.pairs[i-1][1]: + # Merge with previous + self.pairs[i-1:i+1] = [ + (self.pairs[i-1][0], + self.pairs[i][1]) + ] + return + if x <= hi: # Already in set + return + i = len(self.pairs) - 1 + if i >= 0: + lo, hi = self.pairs[i] + if x-1 == hi: + self.pairs[i] = lo, x + return + self.pairs.append((x, x)) + + def addpair(self, xlo, xhi): + if xlo > xhi: return + self.pairs.append((xlo, xhi)) + self.normalize() + + def fromstring(self, data): + new = [] + for part in data.split(self.sep): + list = [] + for subp in part.split(self.rng): + s = subp.strip() + list.append(int(s)) + if len(list) == 1: + new.append((list[0], list[0])) + elif len(list) == 2 and list[0] <= list[1]: + new.append((list[0], list[1])) + else: + raise ValueError, 'bad data passed to IntSet' + self.pairs = self.pairs + new + self.normalize() + + +# Subroutines to read/write entries in .mh_profile and .mh_sequences + +def pickline(file, key, casefold = 1): + try: + f = open(file, 'r') + except IOError: + return None + pat = re.escape(key) + ':' + prog = re.compile(pat, casefold and re.IGNORECASE) + while 1: + line = f.readline() + if not line: break + if prog.match(line): + text = line[len(key)+1:] + while 1: + line = f.readline() + if not line or not line[0].isspace(): + break + text = text + line + return text.strip() + return None + +def updateline(file, key, value, casefold = 1): + try: + f = open(file, 'r') + lines = f.readlines() + f.close() + except IOError: + lines = [] + pat = re.escape(key) + ':(.*)\n' + prog = re.compile(pat, casefold and re.IGNORECASE) + if value is None: + newline = None + else: + newline = '%s: %s\n' % (key, value) + for i in range(len(lines)): + line = lines[i] + if prog.match(line): + if newline is None: + del lines[i] + else: + lines[i] = newline + break + else: + if newline is not None: + lines.append(newline) + tempfile = file + "~" + f = open(tempfile, 'w') + for line in lines: + f.write(line) + f.close() + os.rename(tempfile, file) + + +# Test program + +def test(): + global mh, f + os.system('rm -rf $HOME/Mail/@test') + mh = MH() + def do(s): print s; print eval(s) + do('mh.listfolders()') + do('mh.listallfolders()') + testfolders = ['@test', '@test/test1', '@test/test2', + '@test/test1/test11', '@test/test1/test12', + '@test/test1/test11/test111'] + for t in testfolders: do('mh.makefolder(%r)' % (t,)) + do('mh.listsubfolders(\'@test\')') + do('mh.listallsubfolders(\'@test\')') + f = mh.openfolder('@test') + do('f.listsubfolders()') + do('f.listallsubfolders()') + do('f.getsequences()') + seqs = f.getsequences() + seqs['foo'] = IntSet('1-10 12-20', ' ').tolist() + print seqs + f.putsequences(seqs) + do('f.getsequences()') + for t in reversed(testfolders): do('mh.deletefolder(%r)' % (t,)) + do('mh.getcontext()') + context = mh.getcontext() + f = mh.openfolder(context) + do('f.getcurrent()') + for seq in ('first', 'last', 'cur', '.', 'prev', 'next', + 'first:3', 'last:3', 'cur:3', 'cur:-3', + 'prev:3', 'next:3', + '1:3', '1:-3', '100:3', '100:-3', '10000:3', '10000:-3', + 'all'): + try: + do('f.parsesequence(%r)' % (seq,)) + except Error, msg: + print "Error:", msg + stuff = os.popen("pick %r 2>/dev/null" % (seq,)).read() + list = map(int, stuff.split()) + print list, "<-- pick" + do('f.listmessages()') + + +if __name__ == '__main__': + test() From python-checkins at python.org Thu Jun 15 20:04:40 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 15 Jun 2006 20:04:40 +0200 (CEST) Subject: [Python-checkins] r46981 - python/trunk/Lib/test/test_zipfile64.py Message-ID: <20060615180440.B6EFC1E4005@bag.python.org> Author: tim.peters Date: Thu Jun 15 20:04:40 2006 New Revision: 46981 Modified: python/trunk/Lib/test/test_zipfile64.py Log: Try to reduce the extreme peak memory and disk-space use of this test. It probably still requires more disk space than most buildbots have, and in any case is still so intrusive that if we don't find another way to test this I'm taking my buildbot offline permanently ;-) Modified: python/trunk/Lib/test/test_zipfile64.py ============================================================================== --- python/trunk/Lib/test/test_zipfile64.py (original) +++ python/trunk/Lib/test/test_zipfile64.py Thu Jun 15 20:04:40 2006 @@ -3,7 +3,7 @@ # from test_zipfile from test import test_support test_support.requires( - 'largefile', + 'largefile', 'test requires loads of disk-space bytes and a long time to run' ) @@ -29,22 +29,28 @@ class TestsWithSourceFile(unittest.TestCase): def setUp(self): - line_gen = ("Test of zipfile line %d." % i for i in range(0, 1000000)) + # Create test data. + # xrange() is important here -- don't want to create immortal space + # for a million ints. + line_gen = ("Test of zipfile line %d." % i for i in xrange(1000000)) self.data = '\n'.join(line_gen) - # Make a source file with some lines + # And write it to a file. fp = open(TESTFN, "wb") fp.write(self.data) fp.close() def zipTest(self, f, compression): - # Create the ZIP archive - filecount = int(((1 << 32) / len(self.data)) * 1.5) + # Create the ZIP archive. zipfp = zipfile.ZipFile(f, "w", compression, allowZip64=True) + # It will contain enough copies of self.data to reach about 6GB of + # raw data to store. + filecount = 6*1024**2 // len(self.data) + next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL for num in range(filecount): - zipfp.writestr("testfn%d"%(num,), self.data) + zipfp.writestr("testfn%d" % num, self.data) # Print still working message since this test can be really slow if next_time <= time.time(): next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL @@ -57,7 +63,7 @@ # Read the ZIP archive zipfp = zipfile.ZipFile(f, "r", compression) for num in range(filecount): - self.assertEqual(zipfp.read("testfn%d"%(num,)), self.data) + self.assertEqual(zipfp.read("testfn%d" % num), self.data) # Print still working message since this test can be really slow if next_time <= time.time(): next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL @@ -68,17 +74,22 @@ zipfp.close() def testStored(self): - for f in (TESTFN2, TemporaryFile()): + # Try the temp file first. If we do TESTFN2 first, then it hogs + # gigabytes of disk space for the duration of the test. + for f in TemporaryFile(), TESTFN2: self.zipTest(f, zipfile.ZIP_STORED) if zlib: def testDeflated(self): - for f in (TESTFN2, TemporaryFile()): + # Try the temp file first. If we do TESTFN2 first, then it hogs + # gigabytes of disk space for the duration of the test. + for f in TemporaryFile(), TESTFN2: self.zipTest(f, zipfile.ZIP_DEFLATED) def tearDown(self): - os.remove(TESTFN) - os.remove(TESTFN2) + for fname in TESTFN, TESTFN2: + if os.path.exists(fname): + os.remove(fname) def test_main(): run_unittest(TestsWithSourceFile) From python-checkins at python.org Thu Jun 15 20:06:30 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 15 Jun 2006 20:06:30 +0200 (CEST) Subject: [Python-checkins] r46982 - in python/trunk/Lib: test/test_zipfile.py zipfile.py Message-ID: <20060615180630.6C7CC1E400C@bag.python.org> Author: tim.peters Date: Thu Jun 15 20:06:29 2006 New Revision: 46982 Modified: python/trunk/Lib/test/test_zipfile.py python/trunk/Lib/zipfile.py Log: Whitespace normalization. Modified: python/trunk/Lib/test/test_zipfile.py ============================================================================== --- python/trunk/Lib/test/test_zipfile.py (original) +++ python/trunk/Lib/test/test_zipfile.py Thu Jun 15 20:06:29 2006 @@ -46,7 +46,7 @@ zipfp.printdir() finally: sys.stdout = stdout - + directory = fp.getvalue() lines = directory.splitlines() self.assertEquals(len(lines), 4) # Number of files + header @@ -133,13 +133,13 @@ def largeFileExceptionTest(self, f, compression): zipfp = zipfile.ZipFile(f, "w", compression) - self.assertRaises(zipfile.LargeZipFile, + self.assertRaises(zipfile.LargeZipFile, zipfp.write, TESTFN, "another"+os.extsep+"name") zipfp.close() def largeFileExceptionTest2(self, f, compression): zipfp = zipfile.ZipFile(f, "w", compression) - self.assertRaises(zipfile.LargeZipFile, + self.assertRaises(zipfile.LargeZipFile, zipfp.writestr, "another"+os.extsep+"name", self.data) zipfp.close() @@ -171,7 +171,7 @@ zipfp.printdir() finally: sys.stdout = stdout - + directory = fp.getvalue() lines = directory.splitlines() self.assertEquals(len(lines), 4) # Number of files + header Modified: python/trunk/Lib/zipfile.py ============================================================================== --- python/trunk/Lib/zipfile.py (original) +++ python/trunk/Lib/zipfile.py Thu Jun 15 20:06:29 2006 @@ -17,7 +17,7 @@ class LargeZipFile(Exception): - """ + """ Raised when writing a zipfile, the zipfile requires ZIP64 extensions and those extensions are disabled. """ @@ -99,20 +99,20 @@ fpin.seek(offset - locatorSize, 2) data = fpin.read(locatorSize) sig, diskno, reloff, disks = struct.unpack(structEndArchive64Locator, data) - if sig != stringEndArchive64Locator: + if sig != stringEndArchive64Locator: return endrec if diskno != 0 or disks != 1: raise BadZipfile("zipfiles that span multiple disks are not supported") - # Assume no 'zip64 extensible data' + # Assume no 'zip64 extensible data' endArchiveSize = struct.calcsize(structEndArchive64) fpin.seek(offset - locatorSize - endArchiveSize, 2) data = fpin.read(endArchiveSize) sig, sz, create_version, read_version, disk_num, disk_dir, \ dircount, dircount2, dirsize, diroffset = \ struct.unpack(structEndArchive64, data) - if sig != stringEndArchive64: + if sig != stringEndArchive64: return endrec # Update the original endrec using data from the ZIP64 record @@ -294,7 +294,7 @@ idx+=1 extra = extra[ln+4:] - + class ZipFile: """ Class with methods to open, read, write, close, list zip files. @@ -642,7 +642,7 @@ records.""" if self.fp is None: return - + if self.mode in ("w", "a") and self._didModify: # write ending records count = 0 pos1 = self.fp.tell() @@ -674,7 +674,7 @@ extra_data = struct.pack( ' Author: tim.peters Date: Thu Jun 15 20:07:28 2006 New Revision: 46983 Modified: python/trunk/Lib/test/test_zipfile64.py (props changed) Log: Add missing svn:eol-style property to text files. From buildbot at python.org Thu Jun 15 20:28:44 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 18:28:44 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP-2 trunk Message-ID: <20060615182844.352031E4003@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-2%2520trunk/builds/652 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 15 20:38:19 2006 From: python-checkins at python.org (tim.peters) Date: Thu, 15 Jun 2006 20:38:19 +0200 (CEST) Subject: [Python-checkins] r46984 - python/trunk/Lib/test/test_zipfile64.py Message-ID: <20060615183819.C14A91E4005@bag.python.org> Author: tim.peters Date: Thu Jun 15 20:38:19 2006 New Revision: 46984 Modified: python/trunk/Lib/test/test_zipfile64.py Log: Oops -- I introduced an off-by-6436159488 error. Modified: python/trunk/Lib/test/test_zipfile64.py ============================================================================== --- python/trunk/Lib/test/test_zipfile64.py (original) +++ python/trunk/Lib/test/test_zipfile64.py Thu Jun 15 20:38:19 2006 @@ -46,7 +46,7 @@ # It will contain enough copies of self.data to reach about 6GB of # raw data to store. - filecount = 6*1024**2 // len(self.data) + filecount = 6*1024**3 // len(self.data) next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL for num in range(filecount): From buildbot at python.org Thu Jun 15 20:39:00 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 18:39:00 +0000 Subject: [Python-checkins] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20060615183901.2065B1E4005@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%2520trunk/builds/1030 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 20:39:19 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 18:39:19 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060615183919.D6BA51E4003@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/1029 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 20:56:16 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 18:56:16 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060615185616.D69091E4003@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/729 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 21:02:10 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 19:02:10 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060615190210.7036D1E4003@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/768 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 21:32:51 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 19:32:51 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060615193251.C5A1D1E4003@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/192 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Jun 15 21:33:24 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 15 Jun 2006 19:33:24 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060615193324.382971E4003@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/415 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gregory.p.smith,neal.norwitz,ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 15 22:16:16 2006 From: python-checkins at python.org (phillip.eby) Date: Thu, 15 Jun 2006 22:16:16 +0200 (CEST) Subject: [Python-checkins] r46985 - sandbox/trunk/setuptools/setuptools/command/easy_install.py Message-ID: <20060615201616.E90031E4013@bag.python.org> Author: phillip.eby Date: Thu Jun 15 22:16:16 2006 New Revision: 46985 Modified: sandbox/trunk/setuptools/setuptools/command/easy_install.py Log: Implement detection of non-Python scripts, as described in http://mail.python.org/pipermail/distutils-sig/2006-June/006359.html Modified: sandbox/trunk/setuptools/setuptools/command/easy_install.py ============================================================================== --- sandbox/trunk/setuptools/setuptools/command/easy_install.py (original) +++ sandbox/trunk/setuptools/setuptools/command/easy_install.py Thu Jun 15 22:16:16 2006 @@ -575,8 +575,9 @@ def install_script(self, dist, script_name, script_text, dev_path=None): """Generate a legacy script wrapper and install it""" spec = str(dist.as_requirement()) + is_script = is_python_script(script_text, script_name) - if dev_path: + if is_script and dev_path: script_text = get_script_header(script_text) + ( "# EASY-INSTALL-DEV-SCRIPT: %(spec)r,%(script_name)r\n" "__requires__ = %(spec)r\n" @@ -585,14 +586,13 @@ "__file__ = %(dev_path)r\n" "execfile(__file__)\n" ) % locals() - else: + elif is_script: script_text = get_script_header(script_text) + ( "# EASY-INSTALL-SCRIPT: %(spec)r,%(script_name)r\n" "__requires__ = %(spec)r\n" "import pkg_resources\n" "pkg_resources.run_script(%(spec)r, %(script_name)r)\n" ) % locals() - self.write_script(script_name, script_text) def write_script(self, script_name, contents, mode="t", blockers=()): @@ -1401,11 +1401,10 @@ def get_script_header(script_text, executable=sys_executable): """Create a #! line, getting options (if any) from script_text""" from distutils.command.build_scripts import first_line_re - first, rest = (script_text+'\n').split('\n',1) + first = (script_text+'\n').splitlines()[0] match = first_line_re.match(first) options = '' if match: - script_text = rest options = match.group(1) or '' if options: options = ' '+options @@ -1433,6 +1432,48 @@ return + +def is_python(text, filename=''): + "Is this string a valid Python script?" + try: + compile(text, filename, 'exec') + except SyntaxError: + return False + else: + return True + + +def is_python_script(script_text, filename): + """Is this text, as a whole, a Python script? (as opposed to shell/bat/etc. + """ + if script_text.startswith('#!'): + # It begins with a '#!' line, so check if 'python' is in it somewhere + from distutils.command.build_scripts import first_line_re + lines = script_text.splitlines() + + if first_line_re.match(lines[0]): + return True # It's got a python "#!" line, consider it Python + else: + return False # It's some other scripting language + + if filename.endswith('.py') or filename.endswith('.pyw'): + return True # extension says it's Python + + if is_python(script_text, filename): + return True # it's syntactically valid Python + + return False # Not any Python I can recognize + + + + + + + + + + + def get_script_args(dist, executable=sys_executable): """Yield write_script() argument tuples for a distribution's entrypoints""" spec = str(dist.as_requirement()) From python-checkins at python.org Thu Jun 15 22:18:28 2006 From: python-checkins at python.org (phillip.eby) Date: Thu, 15 Jun 2006 22:18:28 +0200 (CEST) Subject: [Python-checkins] r46986 - in sandbox/branches/setuptools-0.6: EasyInstall.txt setuptools/command/easy_install.py Message-ID: <20060615201828.AC10C1E4003@bag.python.org> Author: phillip.eby Date: Thu Jun 15 22:18:28 2006 New Revision: 46986 Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py Log: Implement detection of non-Python scripts, as described in http://mail.python.org/pipermail/distutils-sig/2006-June/006359.html (merge from trunk) Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt ============================================================================== --- sandbox/branches/setuptools-0.6/EasyInstall.txt (original) +++ sandbox/branches/setuptools-0.6/EasyInstall.txt Thu Jun 15 22:18:28 2006 @@ -1095,6 +1095,9 @@ Release Notes/Change History ============================ +0.6b4 + * Fix creating Python wrappers for non-Python scripts + 0.6b3 * Fix local ``--find-links`` eggs not being copied except with ``--always-copy``. Modified: sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/command/easy_install.py Thu Jun 15 22:18:28 2006 @@ -575,8 +575,9 @@ def install_script(self, dist, script_name, script_text, dev_path=None): """Generate a legacy script wrapper and install it""" spec = str(dist.as_requirement()) + is_script = is_python_script(script_text, script_name) - if dev_path: + if is_script and dev_path: script_text = get_script_header(script_text) + ( "# EASY-INSTALL-DEV-SCRIPT: %(spec)r,%(script_name)r\n" "__requires__ = %(spec)r\n" @@ -585,14 +586,13 @@ "__file__ = %(dev_path)r\n" "execfile(__file__)\n" ) % locals() - else: + elif is_script: script_text = get_script_header(script_text) + ( "# EASY-INSTALL-SCRIPT: %(spec)r,%(script_name)r\n" "__requires__ = %(spec)r\n" "import pkg_resources\n" "pkg_resources.run_script(%(spec)r, %(script_name)r)\n" ) % locals() - self.write_script(script_name, script_text) def write_script(self, script_name, contents, mode="t", blockers=()): @@ -1401,11 +1401,10 @@ def get_script_header(script_text, executable=sys_executable): """Create a #! line, getting options (if any) from script_text""" from distutils.command.build_scripts import first_line_re - first, rest = (script_text+'\n').split('\n',1) + first = (script_text+'\n').splitlines()[0] match = first_line_re.match(first) options = '' if match: - script_text = rest options = match.group(1) or '' if options: options = ' '+options @@ -1433,6 +1432,48 @@ return + +def is_python(text, filename=''): + "Is this string a valid Python script?" + try: + compile(text, filename, 'exec') + except SyntaxError: + return False + else: + return True + + +def is_python_script(script_text, filename): + """Is this text, as a whole, a Python script? (as opposed to shell/bat/etc. + """ + if script_text.startswith('#!'): + # It begins with a '#!' line, so check if 'python' is in it somewhere + from distutils.command.build_scripts import first_line_re + lines = script_text.splitlines() + + if first_line_re.match(lines[0]): + return True # It's got a python "#!" line, consider it Python + else: + return False # It's some other scripting language + + if filename.endswith('.py') or filename.endswith('.pyw'): + return True # extension says it's Python + + if is_python(script_text, filename): + return True # it's syntactically valid Python + + return False # Not any Python I can recognize + + + + + + + + + + + def get_script_args(dist, executable=sys_executable): """Yield write_script() argument tuples for a distribution's entrypoints""" spec = str(dist.as_requirement()) From neal at metaslash.com Thu Jun 15 23:36:29 2006 From: neal at metaslash.com (Neal Norwitz) Date: Thu, 15 Jun 2006 17:36:29 -0400 Subject: [Python-checkins] Python Regression Test Failures all (1) Message-ID: <20060615213629.GA15149@python.psfb.org> test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 Exception in thread writer 0: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/threading.py", line 440, in run self.__target(*self.__args, **self.__kwargs) File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 260, in writerThread self.assertEqual(data, self.makeData(key)) File "/home/neal/python/trunk/Lib/unittest.py", line 334, in failUnlessEqual (msg or '%r != %r' % (first, second)) AssertionError: None != '0004-0004-0004-0004-0004' Exception in thread writer 2: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/threading.py", line 440, in run self.__target(*self.__args, **self.__kwargs) File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 260, in writerThread self.assertEqual(data, self.makeData(key)) File "/home/neal/python/trunk/Lib/unittest.py", line 334, in failUnlessEqual (msg or '%r != %r' % (first, second)) AssertionError: None != '2002-2002-2002-2002-2002' Exception in thread writer 1: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 460, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/threading.py", line 440, in run self.__target(*self.__args, **self.__kwargs) File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 260, in writerThread self.assertEqual(data, self.makeData(key)) File "/home/neal/python/trunk/Lib/unittest.py", line 334, in failUnlessEqual (msg or '%r != %r' % (first, second)) AssertionError: None != '1007-1007-1007-1007-1007' test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socketserver test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllibnet test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test_wait4 test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile /home/neal/python/trunk/Lib/struct.py:63: DeprecationWarning: struct integer overflow masking is deprecated return o.pack(*args) test_zipfile64 zipTest still reading 113 of 223, be patient... zipTest still reading 71 of 223, be patient... Exception exceptions.IOError: (28, 'No space left on device') in > ignored test test_zipfile64 failed -- Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/test_zipfile64.py", line 80, in testStored self.zipTest(f, zipfile.ZIP_STORED) File "/home/neal/python/trunk/Lib/test/test_zipfile64.py", line 53, in zipTest zipfp.writestr("testfn%d" % num, self.data) File "/home/neal/python/trunk/Lib/zipfile.py", line 627, in writestr self.fp.write(bytes) IOError: [Errno 28] No space left on device test_zipimport test_zlib 293 tests OK. 1 test failed: test_zipfile64 21 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl test_gl test_imgfile test_ioctl test_macfs test_macostools test_nis test_pep277 test_plistlib test_scriptpackages test_startfile test_sunaudiodev test_tcl test_unicode_file test_winreg test_winsound 1 skip unexpected on linux2: test_ioctl [441794 refs] From tim.peters at gmail.com Thu Jun 15 23:56:34 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 15 Jun 2006 17:56:34 -0400 Subject: [Python-checkins] r46971 - python/trunk/Lib/test/test_zipfile64.py In-Reply-To: <20060615095703.A47FF1E4005@bag.python.org> References: <20060615095703.A47FF1E4005@bag.python.org> Message-ID: <1f7befae0606151456v370f31a5ibd110964a665a48b@mail.gmail.com> > Author: neal.norwitz > Date: Thu Jun 15 11:57:03 2006 > New Revision: 46971 > > Modified: > python/trunk/Lib/test/test_zipfile64.py > Log: > Steal the trick from test_compiler to print out a slow msg. > This will hopefully get the buildbots to pass. Not sure this > test will be feasible or even work. But everything is red now, > so it can't get much worse. This isn't going to work ;-) Most of the buildbots fail with "No space left on device", and that was after I rearranged the test to cut down the peak disk use. I'm not sure exactly what that is now, but it's at least 6442244863 + 28888889 = 6,471,133,752 bytes. It seems likely to me that some of the failing buildbots have that much free disk space, but not on the partition used by TemporaryFile() (multi-gigabyte temp-space partitions are rare in my Unixy experience). Anyway, fix that and then there's that this test alone takes 50% longer than all the rest of the -uall tests combined on my box. I don't have a solution in mind other than disabling this test, but for 2.5b1 that's a "good enough" solution for me. From nnorwitz at gmail.com Fri Jun 16 01:38:30 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 15 Jun 2006 16:38:30 -0700 Subject: [Python-checkins] r46971 - python/trunk/Lib/test/test_zipfile64.py In-Reply-To: <1f7befae0606151456v370f31a5ibd110964a665a48b@mail.gmail.com> References: <20060615095703.A47FF1E4005@bag.python.org> <1f7befae0606151456v370f31a5ibd110964a665a48b@mail.gmail.com> Message-ID: On 6/15/06, Tim Peters wrote: > > Author: neal.norwitz > > Date: Thu Jun 15 11:57:03 2006 > > New Revision: 46971 > > > > Modified: > > python/trunk/Lib/test/test_zipfile64.py > > Log: > > Steal the trick from test_compiler to print out a slow msg. > > This will hopefully get the buildbots to pass. Not sure this > > test will be feasible or even work. But everything is red now, > > so it can't get much worse. > > This isn't going to work ;-) > > Most of the buildbots fail with "No space left on device", and that > was after I rearranged the test to cut down the peak disk use. I'm > not sure exactly what that is now, but it's at least > > 6442244863 + 28888889 = 6,471,133,752 > > bytes. > > It seems likely to me that some of the failing buildbots have that > much free disk space, but not on the partition used by TemporaryFile() > (multi-gigabyte temp-space partitions are rare in my Unixy > experience). > > Anyway, fix that and then there's that this test alone takes 50% > longer than all the rest of the -uall tests combined on my box. > > I don't have a solution in mind other than disabling this test, but > for 2.5b1 that's a "good enough" solution for me. I've come to a similar conclusion. I thought about checking the disk available before trying to run the test and skipping the test in that case. But it doesn't address how long it takes to run this test. Maybe this should be like one of the bigmem tests? n From brett at python.org Fri Jun 16 01:47:55 2006 From: brett at python.org (Brett Cannon) Date: Thu, 15 Jun 2006 16:47:55 -0700 Subject: [Python-checkins] r46971 - python/trunk/Lib/test/test_zipfile64.py In-Reply-To: References: <20060615095703.A47FF1E4005@bag.python.org> <1f7befae0606151456v370f31a5ibd110964a665a48b@mail.gmail.com> Message-ID: On 6/15/06, Neal Norwitz wrote: > > On 6/15/06, Tim Peters wrote: > > > Author: neal.norwitz > > > Date: Thu Jun 15 11:57:03 2006 > > > New Revision: 46971 > > > > > > Modified: > > > python/trunk/Lib/test/test_zipfile64.py > > > Log: > > > Steal the trick from test_compiler to print out a slow msg. > > > This will hopefully get the buildbots to pass. Not sure this > > > test will be feasible or even work. But everything is red now, > > > so it can't get much worse. > > > > This isn't going to work ;-) > > > > Most of the buildbots fail with "No space left on device", and that > > was after I rearranged the test to cut down the peak disk use. I'm > > not sure exactly what that is now, but it's at least > > > > 6442244863 + 28888889 = 6,471,133,752 > > > > bytes. > > > > It seems likely to me that some of the failing buildbots have that > > much free disk space, but not on the partition used by TemporaryFile() > > (multi-gigabyte temp-space partitions are rare in my Unixy > > experience). > > > > Anyway, fix that and then there's that this test alone takes 50% > > longer than all the rest of the -uall tests combined on my box. > > > > I don't have a solution in mind other than disabling this test, but > > for 2.5b1 that's a "good enough" solution for me. > > I've come to a similar conclusion. I thought about checking the disk > available before trying to run the test and skipping the test in that > case. But it doesn't address how long it takes to run this test. > Maybe this should be like one of the bigmem tests? Sounds like it. I don't think it necessarily calls for its own -u option for regrtest with bigmem being available. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060615/511d2f53/attachment-0001.htm From python-checkins at python.org Fri Jun 16 02:42:16 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 16 Jun 2006 02:42:16 +0200 (CEST) Subject: [Python-checkins] r46987 - sandbox/trunk/Doc/func-example.py Message-ID: <20060616004216.554C91E4003@bag.python.org> Author: andrew.kuchling Date: Fri Jun 16 02:42:15 2006 New Revision: 46987 Added: sandbox/trunk/Doc/func-example.py (contents, props changed) Log: Add larger example for functional HOWTO; I'll dissect it in a section to be written Added: sandbox/trunk/Doc/func-example.py ============================================================================== --- (empty file) +++ sandbox/trunk/Doc/func-example.py Fri Jun 16 02:42:15 2006 @@ -0,0 +1,199 @@ +#!/Users/andrewk/source/p/python/python.exe + +# Example: indexer that records info about all the files in a directory tree. + +import os, sys +import itertools, functools +import cPickle + +# +# Management of the list of indexing functions. +# + +_indexers = {} + +def register(ext, func): + """Registers the function 'func' + + >>> is_indexable_filename('foo.jpg') + False + >>> register('jpg', None) + >>> is_indexable_filename('foo.jpg') + True + >>> _indexers.clear() + """ + _indexers['.' + ext] = func + +def is_indexable_filename (fn): + """Returns true if there's an indexer available for the given filename. + + >>> register('txt', None) + >>> is_indexable_filename('foo.txt') + True + >>> is_indexable_filename('foo.jpg') + False + """ + base, ext = os.path.splitext(fn) + return _indexers.has_key(ext) + +def is_ignorable_directory (dirname): + """Return true if the directory with the given name shouldn't be scanned. + + >>> is_ignorable_directory('.svn') + True + >>> is_ignorable_directory('text') + False + """ + return (dirname in ('.svn', 'CVS')) + +def remove_punctuation (word): + """Removes leading and trailing punctuation characters from a word. + May return the empty string. + + >>> remove_punctuation('test') + 'test' + >>> remove_punctuation('comma,') + 'comma' + >>> remove_punctuation('()') + '' + """ + word = word.strip(',.?!"\'()[]#*\\') + return word + +# +# Functions for indexing directories and files +# + +def index (*args): + """Index the directory trees rooted at the specified paths. + Can take any number of arguments. + Returns the index data structure. + """ + idx = load_index() + for path in args: + index_tree(idx, path) + save_index(idx) + return idx + +def index_tree (idx, path): + """Index the contents of the files in the directory tree rooted at 'path'. + """ + for dirpath, dirnames, filenames in os.walk(path): + # Remove ignorable directories + for d in list(dirnames): + if is_ignorable_directory(d): + dirnames.remove(d) + + # Discard uninteresting filenames + filenames = [fn for fn in filenames + if is_indexable_filename(fn)] + + # Index files + for fn in filenames: + full_path = os.path.join(dirpath, fn) + index_file(idx, full_path) + +def index_file (idx, path): + """Index the contents of a single file. It's assumed that + an indexing function will be found for the file's type. + """ + assert is_indexable_filename(path) + base, ext = os.path.splitext(path) + + indexer = _indexers[ext] + record_func = functools.partial(record, idx) + indexer(path, record_func) + + +# +# Index data structure +# +# The index is a big dictionary: +# { word => [list of (filename, line number) tuples] } +# + +def lookup (idx, word): + """Return an iterator over the files and lines containing the requested + word. + """ + for file, line in idx.get(word, []): + yield (file, line) + +def record (idx, word, path, line=None): + """Add an index entry for the given word, using the specified path + and line number. The line number can be None. + + >>> record({}, 'word', '/path', None) + {'word': {('/path', None): 1}} + >>> record({}, 'word', '/path', 42) + {'word': {('/path', 42): 1}} + """ + d = idx.setdefault(word, {}) + key = (path, line) + if key not in d: + d[key] = 1 + return idx + +def load_index (): + """Read index from disk. + """ + index_filename = '/tmp/index' + if os.path.exists(index_filename): + input = open(index_filename, 'rb') + idx = cPickle.load(input) + input.close() + else: + idx = {} + + return idx + +def save_index (idx): + """Write index to disk. + """ + output = open('/tmp/index', 'wb') + cPickle.dump(idx, output, -1) + output.close() + + import pprint + print len(idx), 'words in index' + #print idx + ##pprint.pprint(idx) + +# +# File analysis functions +# + +def text_inspector (input_file, record_func): + line_num = 1 + for line in open(input_file, 'r'): + for word in line.split(): + word = remove_punctuation(word.lower()) + if word != '': + record_func(word, input_file, line_num) + line_num += 1 + + +if __name__ == '__main__': + if '-t' in sys.argv[1:]: + import doctest + doctest.testmod() + raise SystemExit + + register('txt', text_inspector) + #register('jpg', jpg_inspector) + #register('gif', gif_inspector) + + idx = index(*sys.argv[1:]) + + # Look up a word + for filename, line in lookup(idx, 'the'): + print filename, line + + +# Exercises: +# * Matching lines are output in random order. Output them in sorted order. +# [5] (One-line change) +# * Use itertools.groupby() for better output, i.e. file.txt: 1 3 4 5 +# [10] +# * Remove file entries before adding new ones. [15] + From python-checkins at python.org Fri Jun 16 02:44:02 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 16 Jun 2006 02:44:02 +0200 (CEST) Subject: [Python-checkins] r46988 - sandbox/trunk/Doc/functional.rst Message-ID: <20060616004402.F33521E4003@bag.python.org> Author: andrew.kuchling Date: Fri Jun 16 02:44:02 2006 New Revision: 46988 Modified: sandbox/trunk/Doc/functional.rst Log: Mention generator expressions Minor typographical fix Add notes for reference section Modified: sandbox/trunk/Doc/functional.rst ============================================================================== --- sandbox/trunk/Doc/functional.rst (original) +++ sandbox/trunk/Doc/functional.rst Fri Jun 16 02:44:02 2006 @@ -338,8 +338,11 @@ Note that in all case the resulting ``stripped_list`` is a Python list containing the resulting lines, not an iterator. This means that list -comprehensions aren't very useful if you're working with iterators -that return an infinite or a very large stream of data. +comprehensions aren't very useful if you're working with iterators +that return an infinite or a very large stream of data. Later we'll +discuss generator expressions, a feature that provides similar +capabilities as list comprehensions but can be used with infinite +iterators. List comprehensions have the form:: @@ -396,6 +399,28 @@ [ (x,y) for x in seq1 for y in seq2] +Generator Expressions +----------------------- + +Generator expressions are written like list comprehensions, but are +surrounded by parentheses (\samp{()}) and not square brackets +(\samp{[]}). The result of a generator expression +is an iterator that returns the computed elements without +materializing a list containing them all. + +:: + + (line.strip() for line in line_list) => + 'line 1', 'line 2' + +Generator expressions always have to be written inside parentheses, as +in the above example. The parentheses signalling a function call also +count, so if you want to create an iterator that will be immediately +passed to a function you could write:: + + obj_total = sum(obj.count for obj in list_all_objects()) + + Generators ----------------------- @@ -691,7 +716,7 @@ :: enumerate(['subject', 'verb', 'object']) => - [(0, 'subject'), (1, 'verb'), (2, 'object')] + (0, 'subject'), (1, 'verb'), (2, 'object') ``enumerate()`` is often used when looping through a list and recording the indexes at which certain conditions are met:: @@ -1118,3 +1143,14 @@ sys.stdout.write(str(elem)) sys.stdout.write(', ') print elem[-1] + + +References +-------------------- + +SICP + +Relevant manual sections + +XXX + From python-checkins at python.org Fri Jun 16 02:49:49 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Fri, 16 Jun 2006 02:49:49 +0200 (CEST) Subject: [Python-checkins] r46989 - python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Message-ID: <20060616004949.5D4731E4003@bag.python.org> Author: jackilyn.hoxworth Date: Fri Jun 16 02:49:49 2006 New Revision: 46989 Added: python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Log: Added a file remotely, not correct/complete Added: python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Fri Jun 16 02:49:49 2006 @@ -0,0 +1,14 @@ +# !/usr/bin/env python + +""" +Test harness for the standard library logging module. +This does not work at all (I don't know what I'm doing). +""" + +import logging, asyncore + +logging.basicConfig() + +log = logging.getLogger("py.asyncore") +log.setLevel(logging.DEBUG) # The level is set to DEBUG so nothing will be ignored + From python-checkins at python.org Fri Jun 16 06:30:36 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 16 Jun 2006 06:30:36 +0200 (CEST) Subject: [Python-checkins] r46990 - python/trunk/Lib/test/test_zipfile64.py Message-ID: <20060616043036.819371E4003@bag.python.org> Author: neal.norwitz Date: Fri Jun 16 06:30:34 2006 New Revision: 46990 Modified: python/trunk/Lib/test/test_zipfile64.py Log: Disable this test until we can determine what to do about it Modified: python/trunk/Lib/test/test_zipfile64.py ============================================================================== --- python/trunk/Lib/test/test_zipfile64.py (original) +++ python/trunk/Lib/test/test_zipfile64.py Fri Jun 16 06:30:34 2006 @@ -2,8 +2,11 @@ # The test_support.requires call is the only reason for keeping this separate # from test_zipfile from test import test_support +# XXX(nnorwitz): disable this test by looking for extra largfile resource +# which doesn't exist. This test takes over 30 minutes to run in general +# and requires more disk space than most of the buildbots. test_support.requires( - 'largefile', + 'extralargefile', 'test requires loads of disk-space bytes and a long time to run' ) From python-checkins at python.org Fri Jun 16 06:31:07 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 16 Jun 2006 06:31:07 +0200 (CEST) Subject: [Python-checkins] r46991 - python/trunk/Lib/tempfile.py Message-ID: <20060616043107.5E8CE1E4003@bag.python.org> Author: neal.norwitz Date: Fri Jun 16 06:31:06 2006 New Revision: 46991 Modified: python/trunk/Lib/tempfile.py Log: Param name is dir, not directory. Update docstring. Backport candidate Modified: python/trunk/Lib/tempfile.py ============================================================================== --- python/trunk/Lib/tempfile.py (original) +++ python/trunk/Lib/tempfile.py Fri Jun 16 06:31:06 2006 @@ -446,7 +446,7 @@ prefix=template, dir=None): """Create and return a temporary file. Arguments: - 'prefix', 'suffix', 'directory' -- as for mkstemp. + 'prefix', 'suffix', 'dir' -- as for mkstemp. 'mode' -- the mode argument to os.fdopen (default "w+b"). 'bufsize' -- the buffer size argument to os.fdopen (default -1). The file is created as mkstemp() would do it. From python-checkins at python.org Fri Jun 16 06:31:28 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 16 Jun 2006 06:31:28 +0200 (CEST) Subject: [Python-checkins] r46992 - python/trunk/Lib/test/test_bigmem.py Message-ID: <20060616043128.8FF641E4003@bag.python.org> Author: neal.norwitz Date: Fri Jun 16 06:31:28 2006 New Revision: 46992 Modified: python/trunk/Lib/test/test_bigmem.py Log: Add missing period in comment. Modified: python/trunk/Lib/test/test_bigmem.py ============================================================================== --- python/trunk/Lib/test/test_bigmem.py (original) +++ python/trunk/Lib/test/test_bigmem.py Fri Jun 16 06:31:28 2006 @@ -28,7 +28,7 @@ # - While the bigmemtest decorator speaks of 'minsize', all tests will # actually be called with a much smaller number too, in the normal # test run (5Kb currently.) This is so the tests themselves get frequent -# testing Consequently, always make all large allocations based on the +# testing. Consequently, always make all large allocations based on the # passed-in 'size', and don't rely on the size being very large. Also, # memuse-per-size should remain sane (less than a few thousand); if your # test uses more, adjust 'size' upward, instead. From python-checkins at python.org Fri Jun 16 06:32:43 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 16 Jun 2006 06:32:43 +0200 (CEST) Subject: [Python-checkins] r46993 - python/trunk/Mac/Modules/ae/_AEmodule.c Message-ID: <20060616043243.C83BB1E4003@bag.python.org> Author: neal.norwitz Date: Fri Jun 16 06:32:43 2006 New Revision: 46993 Modified: python/trunk/Mac/Modules/ae/_AEmodule.c Log: Fix whitespace, there are memory leaks in this module. Modified: python/trunk/Mac/Modules/ae/_AEmodule.c ============================================================================== --- python/trunk/Mac/Modules/ae/_AEmodule.c (original) +++ python/trunk/Mac/Modules/ae/_AEmodule.c Fri Jun 16 06:32:43 2006 @@ -829,21 +829,19 @@ static PyObject *AEDesc_get_data(AEDescObject *self, void *closure) { - - PyObject *res; - Size size; - char *ptr; - OSErr err; - - size = AEGetDescDataSize(&self->ob_itself); - if ( (res = PyString_FromStringAndSize(NULL, size)) == NULL ) - return NULL; - if ( (ptr = PyString_AsString(res)) == NULL ) - return NULL; - if ( (err=AEGetDescData(&self->ob_itself, ptr, size)) < 0 ) - return PyMac_Error(err); - return res; - + PyObject *res; + Size size; + char *ptr; + OSErr err; + + size = AEGetDescDataSize(&self->ob_itself); + if ( (res = PyString_FromStringAndSize(NULL, size)) == NULL ) + return NULL; + if ( (ptr = PyString_AsString(res)) == NULL ) + return NULL; + if ( (err=AEGetDescData(&self->ob_itself, ptr, size)) < 0 ) + return PyMac_Error(err); + return res; } #define AEDesc_set_data NULL @@ -1431,14 +1429,11 @@ PyObject *m; PyObject *d; - - - upp_AEIdleProc = NewAEIdleUPP(AEIdleProc); - upp_GenericEventHandler = NewAEEventHandlerUPP(GenericEventHandler); - PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_New); - PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_NewBorrowed); - PyMac_INIT_TOOLBOX_OBJECT_CONVERT(AEDesc, AEDesc_Convert); - + upp_AEIdleProc = NewAEIdleUPP(AEIdleProc); + upp_GenericEventHandler = NewAEEventHandlerUPP(GenericEventHandler); + PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_New); + PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_NewBorrowed); + PyMac_INIT_TOOLBOX_OBJECT_CONVERT(AEDesc, AEDesc_Convert); m = Py_InitModule("_AE", AE_methods); d = PyModule_GetDict(m); From cynrod_7 at hotmail.com Fri Jun 16 08:49:26 2006 From: cynrod_7 at hotmail.com (cynthia rodriguez) Date: Fri, 16 Jun 2006 06:49:26 +0000 Subject: [Python-checkins] (no subject) Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060616/169f0c21/attachment.htm From python-checkins at python.org Fri Jun 16 22:01:59 2006 From: python-checkins at python.org (matt.fleming) Date: Fri, 16 Jun 2006 22:01:59 +0200 (CEST) Subject: [Python-checkins] r46994 - sandbox/trunk/pdb/test/tcptest.py Message-ID: <20060616200159.0B6691E4005@bag.python.org> Author: matt.fleming Date: Fri Jun 16 22:01:58 2006 New Revision: 46994 Modified: sandbox/trunk/pdb/test/tcptest.py Log: Stop using a Timer object and just repeatedly try to connect. Tests should pass faster now. Modified: sandbox/trunk/pdb/test/tcptest.py ============================================================================== --- sandbox/trunk/pdb/test/tcptest.py (original) +++ sandbox/trunk/pdb/test/tcptest.py Fri Jun 16 22:01:58 2006 @@ -1,7 +1,11 @@ #!/usr/bin/env python -import os + +# This unit test doesn't use any of the debugger code. It is meant solely +# to test the connection classes. + import sys import socket +import thread import threading import unittest @@ -10,20 +14,32 @@ sys.path.append("..") from mconnection import MServerConnectionTCP, MClientConnectionTCP +# Try to connect the client to addr either until we've tried MAXTRIES +# times or until it succeeds. +MAXTRIES = 100 +def repeatedConnect(client, addr): + for i in range(MAXTRIES): + try: + client.connect(addr) + # The _sock variable appears when there's a connection + if client._sock: break + except socket.error: + pass + class TestTCPConnections(unittest.TestCase): - def testClientConnectAndRead(self): + def setUp(self): self.server = MServerConnectionTCP() self.client = MClientConnectionTCP() - # Create a new thread for the client and delay for 3 seconds - t = threading.Timer(3.0, self.client.connect, [__addr__]) - t.start() - # Start server, which waits for a connection. self.client will connect - # in a few seconds + + def testClientConnectAndRead(self): + t_id = thread.start_new_thread(repeatedConnect, (self.client,__addr__)) self.server.connect(__addr__) self.server.write("good") line = self.client.readline() self.assertEqual("good", line, "Could not read from server") + + def tearDown(self): self.server.disconnect() self.client.disconnect() From python-checkins at python.org Sat Jun 17 01:45:07 2006 From: python-checkins at python.org (fred.drake) Date: Sat, 17 Jun 2006 01:45:07 +0200 (CEST) Subject: [Python-checkins] r46995 - in python/trunk: Doc/lib/libsgmllib.tex Lib/sgmllib.py Lib/test/test_sgmllib.py Misc/ACKS Message-ID: <20060616234507.78A451E4005@bag.python.org> Author: fred.drake Date: Sat Jun 17 01:45:06 2006 New Revision: 46995 Modified: python/trunk/Doc/lib/libsgmllib.tex python/trunk/Lib/sgmllib.py python/trunk/Lib/test/test_sgmllib.py python/trunk/Misc/ACKS Log: SF patch 1504676: Make sgmllib char and entity references pluggable (implementation/tests contributed by Sam Ruby) Modified: python/trunk/Doc/lib/libsgmllib.tex ============================================================================== --- python/trunk/Doc/lib/libsgmllib.tex (original) +++ python/trunk/Doc/lib/libsgmllib.tex Sat Jun 17 01:45:06 2006 @@ -132,27 +132,59 @@ \begin{methoddesc}{handle_charref}{ref} This method is called to process a character reference of the form -\samp{\&\#\var{ref};}. In the base implementation, \var{ref} must -be a decimal number in the -range 0-255. It translates the character to \ASCII{} and calls the -method \method{handle_data()} with the character as argument. If -\var{ref} is invalid or out of range, the method -\code{unknown_charref(\var{ref})} is called to handle the error. A -subclass must override this method to provide support for named -character entities. +\samp{\&\#\var{ref};}. The base implementation uses +\method{convert_charref()} to convert the reference to a string. If +that method returns a string, it is passed to \method{handle_data()}, +otherwise \method{unknown_charref(\var{ref})} is called to handle the +error. +\versionchanged[Use \method{convert_charref()} instead of hard-coding +the conversion]{2.5} +\end{methoddesc} + +\begin{methoddesc}{convert_charref}{ref} +Convert a character reference to a string, or \code{None}. \var{ref} +is the reference passed in as a string. In the base implementation, +\var{ref} must be a decimal number in the range 0-255. It converts +the code point found using the \method{convert_codepoint()} method. +If \var{ref} is invalid or out of range, this method returns +\code{None}. This method is called by the default +\method{handle_charref()} implementation and by the attribute value +parser. +\versionadded{2.5} +\end{methoddesc} + +\begin{methoddesc}{convert_codepoint}{codepoint} +Convert a codepoint to a \class{str} value. Encodings can be handled +here if appropriate, though the rest of \module{sgmllib} is oblivious +on this matter. +\versionadded{2.5} \end{methoddesc} \begin{methoddesc}{handle_entityref}{ref} This method is called to process a general entity reference of the form \samp{\&\var{ref};} where \var{ref} is an general entity -reference. It looks for \var{ref} in the instance (or class) -variable \member{entitydefs} which should be a mapping from entity -names to corresponding translations. If a translation is found, it +reference. It converts \var{ref} by passing it to +\method{convert_entityref()}. If a translation is returned, it calls the method \method{handle_data()} with the translation; otherwise, it calls the method \code{unknown_entityref(\var{ref})}. The default \member{entitydefs} defines translations for \code{\&}, \code{\&apos}, \code{\>}, \code{\<}, and \code{\"}. +\versionchanged[Use \method{convert_entityref()} instead of hard-coding +the conversion]{2.5} +\end{methoddesc} + +\begin{methoddesc}{convert_entityref}{ref} +Convert a named entity reference to a \class{str} value, or +\code{None}. The resulting value will not be parsed. \var{ref} will +be only the name of the entity. The default implementation looks for +\var{ref} in the instance (or class) variable \member{entitydefs} +which should be a mapping from entity names to corresponding +translations. If no translation is available for \var{ref}, this +method returns \code{None}. This method is called by the default +\method{handle_entityref()} implementation and by the attribute value +parser. +\versionadded{2.5} \end{methoddesc} \begin{methoddesc}{handle_comment}{comment} Modified: python/trunk/Lib/sgmllib.py ============================================================================== --- python/trunk/Lib/sgmllib.py (original) +++ python/trunk/Lib/sgmllib.py Sat Jun 17 01:45:06 2006 @@ -53,6 +53,10 @@ # self.handle_entityref() with the entity reference as argument. class SGMLParser(markupbase.ParserBase): + # Definition of entities -- derived classes may override + entity_or_charref = re.compile('&(?:' + '([a-zA-Z][-.a-zA-Z0-9]*)|#([0-9]+)' + ')(;?)') def __init__(self, verbose=0): """Initialize and reset this instance.""" @@ -277,32 +281,8 @@ attrvalue[:1] == '"' == attrvalue[-1:]): # strip quotes attrvalue = attrvalue[1:-1] - l = 0 - new_attrvalue = '' - while l < len(attrvalue): - av_match = entityref.match(attrvalue, l) - if (av_match and av_match.group(1) in self.entitydefs and - attrvalue[av_match.end(1)] == ';'): - # only substitute entityrefs ending in ';' since - # otherwise we may break - # which is very common - new_attrvalue += self.entitydefs[av_match.group(1)] - l = av_match.end(0) - continue - ch_match = charref.match(attrvalue, l) - if ch_match: - try: - char = chr(int(ch_match.group(1))) - new_attrvalue += char - l = ch_match.end(0) - continue - except ValueError: - # invalid character reference, don't substitute - pass - # all other cases - new_attrvalue += attrvalue[l] - l += 1 - attrvalue = new_attrvalue + attrvalue = self.entity_or_charref.sub( + self._convert_ref, attrvalue) attrs.append((attrname.lower(), attrvalue)) k = match.end(0) if rawdata[j] == '>': @@ -311,6 +291,17 @@ self.finish_starttag(tag, attrs) return j + # Internal -- convert entity or character reference + def _convert_ref(self, match): + if match.group(2): + return self.convert_charref(match.group(2)) or \ + '&#%s%s' % match.groups()[1:] + elif match.group(3): + return self.convert_entityref(match.group(1)) or \ + '&%s;' % match.group(1) + else: + return '&%s' % match.group(1) + # Internal -- parse endtag def parse_endtag(self, i): rawdata = self.rawdata @@ -394,35 +385,51 @@ print '*** Unbalanced ' print '*** Stack:', self.stack - def handle_charref(self, name): - """Handle character reference, no need to override.""" + def convert_charref(self, name): + """Convert character reference, may be overridden.""" try: n = int(name) except ValueError: - self.unknown_charref(name) return if not 0 <= n <= 255: - self.unknown_charref(name) return - self.handle_data(chr(n)) + return self.convert_codepoint(n) + + def convert_codepoint(self, codepoint): + return chr(codepoint) + + def handle_charref(self, name): + """Handle character reference, no need to override.""" + replacement = convert_charref(name) + if replacement is None: + self.unknown_charref(name) + else: + self.handle_data(convert_charref(name)) # Definition of entities -- derived classes may override entitydefs = \ {'lt': '<', 'gt': '>', 'amp': '&', 'quot': '"', 'apos': '\''} - def handle_entityref(self, name): - """Handle entity references. + def convert_entityref(self, name): + """Convert entity references. - There should be no need to override this method; it can be - tailored by setting up the self.entitydefs mapping appropriately. + As an alternative to overriding this method; one can tailor the + results by setting up the self.entitydefs mapping appropriately. """ table = self.entitydefs if name in table: - self.handle_data(table[name]) + return table[name] else: - self.unknown_entityref(name) return + def handle_entityref(self, name): + """Handle entity references, no need to override.""" + replacement = convert_entityref(name) + if replacement is None: + self.unknown_entityref(name) + else: + self.handle_data(convert_entityref(name)) + # Example -- handle data, should be overridden def handle_data(self, data): pass Modified: python/trunk/Lib/test/test_sgmllib.py ============================================================================== --- python/trunk/Lib/test/test_sgmllib.py (original) +++ python/trunk/Lib/test/test_sgmllib.py Sat Jun 17 01:45:06 2006 @@ -64,6 +64,23 @@ self.setliteral() +class HTMLEntityCollector(EventCollector): + import re, htmlentitydefs + entity_or_charref = re.compile('(?:&([a-zA-Z][-.a-zA-Z0-9]*)' + '|&#(x[0-9a-zA-Z]+|[0-9]+))(;?)') + + def convert_charref(self, name): + self.append(("charref", "convert", name)) + if name.startswith('x'): + return unichr(int(name[1:],16)) + else: + return unichr(int(name)) + + def convert_entityref(self, name): + self.append(("entityref", "convert", name)) + return unichr(self.htmlentitydefs.name2codepoint[name]) + + class SGMLParserTestCase(unittest.TestCase): collector = EventCollector @@ -233,6 +250,16 @@ ("k", "*"), ])]) + def test_convert_overrides(self): + self.collector = HTMLEntityCollector + self.check_events('foo', [ + ('entityref', 'convert', 'ldquo'), + ('charref', 'convert', 'x201d'), + ('starttag', 'a', [('title', u'\u201ctest\u201d')]), + ('data', 'foo'), + ('endtag', 'a'), + ]) + def test_attr_funky_names(self): self.check_events("""""", [ ("starttag", "a", [("a.b", "v"), ("c:d", "v"), ("e-f", "v")]), Modified: python/trunk/Misc/ACKS ============================================================================== --- python/trunk/Misc/ACKS (original) +++ python/trunk/Misc/ACKS Sat Jun 17 01:45:06 2006 @@ -528,6 +528,7 @@ Saskia van Rossum Donald Wallace Rouse II Liam Routt +Sam Ruby Paul Rubin Audun S. Runde Jeff Rush From buildbot at python.org Sat Jun 17 02:06:57 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:06:57 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP-2 trunk Message-ID: <20060617000657.83BBB1E4005@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-2%2520trunk/builds/655 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:07:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:07:37 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060617000737.F30A91E4005@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1181 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:08:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:08:14 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060617000814.A3A001E4005@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1085 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:08:27 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:08:27 +0000 Subject: [Python-checkins] buildbot warnings in x86 W2k trunk Message-ID: <20060617000827.EF1B61E4005@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/1101 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From tim.peters at gmail.com Sat Jun 17 02:11:12 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 16 Jun 2006 20:11:12 -0400 Subject: [Python-checkins] r46995 - in python/trunk: Doc/lib/libsgmllib.tex Lib/sgmllib.py Lib/test/test_sgmllib.py Misc/ACKS In-Reply-To: <20060616234507.78A451E4005@bag.python.org> References: <20060616234507.78A451E4005@bag.python.org> Message-ID: <1f7befae0606161711h33bd88a7l622c566dc81dadf8@mail.gmail.com> Fred, please note that all the buildbots are failing now. From the errors, looks like it should be shallow to you. > Author: fred.drake > Date: Sat Jun 17 01:45:06 2006 > New Revision: 46995 > > Modified: > python/trunk/Doc/lib/libsgmllib.tex > python/trunk/Lib/sgmllib.py > python/trunk/Lib/test/test_sgmllib.py > python/trunk/Misc/ACKS > Log: > SF patch 1504676: Make sgmllib char and entity references pluggable > (implementation/tests contributed by Sam Ruby) From buildbot at python.org Sat Jun 17 02:20:50 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:20:50 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060617002051.1866F1E4005@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/782 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:24:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:24:30 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060617002430.85FD81E4005@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/658 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:25:50 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:25:50 +0000 Subject: [Python-checkins] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20060617002550.5EDE21E4005@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%2520trunk/builds/1033 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:28:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:28:02 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060617002802.EF92B1E4005@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/1032 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:32:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:32:55 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060617003255.8984D1E4005@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/732 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 02:38:15 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 00:38:15 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060617003815.EEC621E4005@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/771 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From fdrake at acm.org Sat Jun 17 02:56:47 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 16 Jun 2006 20:56:47 -0400 Subject: [Python-checkins] r46995 - in python/trunk: Doc/lib/libsgmllib.tex Lib/sgmllib.py Lib/test/test_sgmllib.py Misc/ACKS In-Reply-To: <1f7befae0606161711h33bd88a7l622c566dc81dadf8@mail.gmail.com> References: <20060616234507.78A451E4005@bag.python.org> <1f7befae0606161711h33bd88a7l622c566dc81dadf8@mail.gmail.com> Message-ID: <200606162056.48594.fdrake@acm.org> On Friday 16 June 2006 20:11, Tim Peters wrote: > Fred, please note that all the buildbots are failing now. From the > errors, looks like it should be shallow to you. Ah, see it; sorry. I'm running the rest of the tests now; should be committed in a minute. -Fred -- Fred L. Drake, Jr. From python-checkins at python.org Sat Jun 17 03:07:55 2006 From: python-checkins at python.org (fred.drake) Date: Sat, 17 Jun 2006 03:07:55 +0200 (CEST) Subject: [Python-checkins] r46996 - python/trunk/Lib/sgmllib.py Message-ID: <20060617010755.1C3C91E4005@bag.python.org> Author: fred.drake Date: Sat Jun 17 03:07:54 2006 New Revision: 46996 Modified: python/trunk/Lib/sgmllib.py Log: fix change that broke the htmllib tests Modified: python/trunk/Lib/sgmllib.py ============================================================================== --- python/trunk/Lib/sgmllib.py (original) +++ python/trunk/Lib/sgmllib.py Sat Jun 17 03:07:54 2006 @@ -424,11 +424,11 @@ def handle_entityref(self, name): """Handle entity references, no need to override.""" - replacement = convert_entityref(name) + replacement = self.convert_entityref(name) if replacement is None: self.unknown_entityref(name) else: - self.handle_data(convert_entityref(name)) + self.handle_data(self.convert_entityref(name)) # Example -- handle data, should be overridden def handle_data(self, data): From fdrake at acm.org Sat Jun 17 03:08:38 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 16 Jun 2006 21:08:38 -0400 Subject: [Python-checkins] r46995 - in python/trunk: Doc/lib/libsgmllib.tex Lib/sgmllib.py Lib/test/test_sgmllib.py Misc/ACKS In-Reply-To: <200606162056.48594.fdrake@acm.org> References: <20060616234507.78A451E4005@bag.python.org> <1f7befae0606161711h33bd88a7l622c566dc81dadf8@mail.gmail.com> <200606162056.48594.fdrake@acm.org> Message-ID: <200606162108.38993.fdrake@acm.org> On Friday 16 June 2006 20:56, Fred L. Drake, Jr. wrote: > On Friday 16 June 2006 20:11, Tim Peters wrote: > > Fred, please note that all the buildbots are failing now. From the > > errors, looks like it should be shallow to you. > > Ah, see it; sorry. > > I'm running the rest of the tests now; should be committed in a minute. This is done. I'll be writing some more tests later tonight or this weekend, as time allows. -Fred -- Fred L. Drake, Jr. From buildbot at python.org Sat Jun 17 03:39:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 01:39:20 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060617013920.A2CD41E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/373 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 04:46:29 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 02:46:29 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060617024629.D38341E4005@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/418 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 04:46:58 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 02:46:58 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060617024658.930701E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/195 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 05:26:46 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 03:26:46 +0000 Subject: [Python-checkins] buildbot warnings in MIPS Debian trunk Message-ID: <20060617032646.B93811E4005@bag.python.org> The Buildbot has detected a new failure of MIPS Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/MIPS%2520Debian%2520trunk/builds/239 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From service at IRS.gov Fri Jun 16 23:47:49 2006 From: service at IRS.gov (Internal Revenue Service) Date: Sat, 17 Jun 2006 06:47:49 +0900 (JST) Subject: [Python-checkins] refund of $63.80 Message-ID: <20060616214749.CBEC3B6B30E@iserver.gleason.jp> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060617/678cf67a/attachment.html From python-checkins at python.org Sat Jun 17 07:31:51 2006 From: python-checkins at python.org (neal.norwitz) Date: Sat, 17 Jun 2006 07:31:51 +0200 (CEST) Subject: [Python-checkins] r46997 - peps/trunk/pep-0356.txt Message-ID: <20060617053151.423E51E4005@bag.python.org> Author: neal.norwitz Date: Sat Jun 17 07:31:50 2006 New Revision: 46997 Modified: peps/trunk/pep-0356.txt Log: Best guess at new schedule. We can't release if the buildbots are red. xml issues haven't really been resolved. However, they were dropped and are no longer considered a blocking issue for 2.5. new test_zipimport64 takes too many resources to run (disk space and time) Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Sat Jun 17 07:31:50 2006 @@ -34,10 +34,10 @@ alpha 1: April 5, 2006 [completed] alpha 2: April 27, 2006 [completed] - beta 1: June 14, 2006 [planned] - beta 2: July 6, 2006 [planned] - rc 1: July 27, 2006 [planned] - final: August 3, 2006 [planned] + beta 1: June 20, 2006 [planned] + beta 2: July 12, 2006 [planned] + rc 1: August 1, 2006 [planned] + final: August 8, 2006 [planned] Completed features for 2.5 @@ -166,9 +166,9 @@ (Owner: Jeremy Hylton) http://python.org/sf/1191458 - - xmlplus/xmlcore situation wrt ElementTree needs resolution - (Owners: Fred Drake/Martin von Loewis/Fredrik Lundh) - http://mail.python.org/pipermail/python-dev/2005-December/058752.html + - test_zipfile64 takes too long and too much disk space for + most of the buildbots. How should this be handled? + It is currently disabled. - should C modules listed in "Undocumented modules" be removed too? "timing" (listed as obsolete), "cl" (listed as possibly not up-to-date), From fredrik at pythonware.com Sat Jun 17 08:53:36 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 17 Jun 2006 08:53:36 +0200 Subject: [Python-checkins] r46996 - python/trunk/Lib/sgmllib.py In-Reply-To: <20060617010755.1C3C91E4005@bag.python.org> References: <20060617010755.1C3C91E4005@bag.python.org> Message-ID: fred.drake wrote: > Modified: python/trunk/Lib/sgmllib.py > ============================================================================== > --- python/trunk/Lib/sgmllib.py (original) > +++ python/trunk/Lib/sgmllib.py Sat Jun 17 03:07:54 2006 > @@ -424,11 +424,11 @@ > > def handle_entityref(self, name): > """Handle entity references, no need to override.""" > - replacement = convert_entityref(name) > + replacement = self.convert_entityref(name) > if replacement is None: > self.unknown_entityref(name) > else: > - self.handle_data(convert_entityref(name)) > + self.handle_data(self.convert_entityref(name)) why call convert_entityref *twice* for each entity ? shouldn't this be: replacement = convert_entityref(name) if replacement is None: self.unknown_entityref(name) else: self.handle_data(replacement) From python-checkins at python.org Sat Jun 17 11:15:16 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 17 Jun 2006 11:15:16 +0200 (CEST) Subject: [Python-checkins] r46998 - in python/trunk: Lib/lib-tk/Tkinter.py Misc/NEWS Message-ID: <20060617091516.4BB441E4005@bag.python.org> Author: martin.v.loewis Date: Sat Jun 17 11:15:14 2006 New Revision: 46998 Modified: python/trunk/Lib/lib-tk/Tkinter.py python/trunk/Misc/NEWS Log: Patch #763580: Add name and value arguments to Tkinter variable classes. Modified: python/trunk/Lib/lib-tk/Tkinter.py ============================================================================== --- python/trunk/Lib/lib-tk/Tkinter.py (original) +++ python/trunk/Lib/lib-tk/Tkinter.py Sat Jun 17 11:15:14 2006 @@ -168,18 +168,30 @@ Subclasses StringVar, IntVar, DoubleVar, BooleanVar are specializations that constrain the type of the value returned from get().""" _default = "" - def __init__(self, master=None): - """Construct a variable with an optional MASTER as master widget. - The variable is named PY_VAR_number in Tcl. + def __init__(self, master=None, value=None, name=None): + """Construct a variable + + MASTER can be given as master widget. + VALUE is an optional value (defaults to "") + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. """ global _varnum if not master: master = _default_root self._master = master self._tk = master.tk - self._name = 'PY_VAR' + repr(_varnum) - _varnum = _varnum + 1 - self.set(self._default) + if name: + self._name = name + else: + self._name = 'PY_VAR' + `_varnum` + _varnum += 1 + if value != None: + self.set(value) + elif not self._tk.call("info", "exists", self._name): + self.set(self._default) def __del__(self): """Unset the variable in Tcl.""" self._tk.globalunsetvar(self._name) @@ -217,15 +229,29 @@ """Return all trace callback information.""" return map(self._tk.split, self._tk.splitlist( self._tk.call("trace", "vinfo", self._name))) + def __eq__(self, other): + """Comparison for equality (==). + + Note: if the Variable's master matters to behavior + also compare self._master == other._master + """ + return self.__class__.__name__ == other.__class__.__name__ \ + and self._name == other._name class StringVar(Variable): """Value holder for strings variables.""" _default = "" - def __init__(self, master=None): + def __init__(self, master=None, value=None, name=None): """Construct a string variable. - MASTER can be given as master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to "") + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def get(self): """Return value of variable as string.""" @@ -237,11 +263,17 @@ class IntVar(Variable): """Value holder for integer variables.""" _default = 0 - def __init__(self, master=None): + def __init__(self, master=None, value=None, name=None): """Construct an integer variable. - MASTER can be given as master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to 0) + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def set(self, value): """Set the variable to value, converting booleans to integers.""" @@ -256,11 +288,17 @@ class DoubleVar(Variable): """Value holder for float variables.""" _default = 0.0 - def __init__(self, master=None): + def __init__(self, master=None, value=None, name=None): """Construct a float variable. - MASTER can be given as a master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to 0.0) + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def get(self): """Return the value of the variable as a float.""" @@ -268,12 +306,18 @@ class BooleanVar(Variable): """Value holder for boolean variables.""" - _default = "false" - def __init__(self, master=None): + _default = False + def __init__(self, master=None, value=None, name=None): """Construct a boolean variable. - MASTER can be given as a master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to False) + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def get(self): """Return the value of the variable as a bool.""" Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 17 11:15:14 2006 @@ -163,6 +163,9 @@ Library ------- +- Patch #763580: Add name and value arguments to Tkinter variable + classes. + - Bug #1117556: SimpleHTTPServer now tries to find and use the system's mime.types file for determining MIME types. From python-checkins at python.org Sat Jun 17 11:20:42 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 17 Jun 2006 11:20:42 +0200 (CEST) Subject: [Python-checkins] r46999 - in python/trunk: Lib/lib-tk/Tkinter.py Misc/ACKS Misc/NEWS Message-ID: <20060617092042.55D121E4005@bag.python.org> Author: martin.v.loewis Date: Sat Jun 17 11:20:41 2006 New Revision: 46999 Modified: python/trunk/Lib/lib-tk/Tkinter.py python/trunk/Misc/ACKS python/trunk/Misc/NEWS Log: Patch #1096231: Add default argument to wm_iconbitmap. Modified: python/trunk/Lib/lib-tk/Tkinter.py ============================================================================== --- python/trunk/Lib/lib-tk/Tkinter.py (original) +++ python/trunk/Lib/lib-tk/Tkinter.py Sat Jun 17 11:20:41 2006 @@ -1500,10 +1500,19 @@ the group leader of this widget if None is given.""" return self.tk.call('wm', 'group', self._w, pathName) group = wm_group - def wm_iconbitmap(self, bitmap=None): + def wm_iconbitmap(self, bitmap=None, default=None): """Set bitmap for the iconified widget to BITMAP. Return - the bitmap if None is given.""" - return self.tk.call('wm', 'iconbitmap', self._w, bitmap) + the bitmap if None is given. + + Under Windows, the DEFAULT parameter can be used to set the icon + for the widget and any descendents that don't have an icon set + explicitely. DEFAULT can be the relative path to a .ico file + (example: root.iconbitmap(default='myicon.ico') ). See Tk + documentation for more information.""" + if default: + return self.tk.call('wm', 'iconbitmap', self._w, '-default', default) + else: + return self.tk.call('wm', 'iconbitmap', self._w, bitmap) iconbitmap = wm_iconbitmap def wm_iconify(self): """Display widget as icon.""" Modified: python/trunk/Misc/ACKS ============================================================================== --- python/trunk/Misc/ACKS (original) +++ python/trunk/Misc/ACKS Sat Jun 17 11:20:41 2006 @@ -205,6 +205,7 @@ Frederik Fix Hern�n Mart�nez Foffani Doug Fort +John Fouhy Martin Franklin Robin Friedrich Ivan Frohne Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 17 11:20:41 2006 @@ -163,6 +163,8 @@ Library ------- +- Patch #1096231: Add ``default`` argument to Tkinter.Wm.wm_iconbitmap. + - Patch #763580: Add name and value arguments to Tkinter variable classes. From python-checkins at python.org Sat Jun 17 11:25:16 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 17 Jun 2006 11:25:16 +0200 (CEST) Subject: [Python-checkins] r47000 - in python/trunk: Lib/lib-tk/Tkinter.py Misc/NEWS Message-ID: <20060617092516.50C951E4005@bag.python.org> Author: martin.v.loewis Date: Sat Jun 17 11:25:15 2006 New Revision: 47000 Modified: python/trunk/Lib/lib-tk/Tkinter.py python/trunk/Misc/NEWS Log: Patch #1494750: Destroy master after deleting children. Modified: python/trunk/Lib/lib-tk/Tkinter.py ============================================================================== --- python/trunk/Lib/lib-tk/Tkinter.py (original) +++ python/trunk/Lib/lib-tk/Tkinter.py Sat Jun 17 11:25:15 2006 @@ -1933,9 +1933,9 @@ def destroy(self): """Destroy this and all descendants widgets.""" for c in self.children.values(): c.destroy() + self.tk.call('destroy', self._w) if self.master.children.has_key(self._name): del self.master.children[self._name] - self.tk.call('destroy', self._w) Misc.destroy(self) def _do(self, name, args=()): # XXX Obsolete -- better use self.tk.call directly! Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 17 11:25:15 2006 @@ -163,6 +163,9 @@ Library ------- +- Patch #1494750: Destroy master after deleting children in + Tkinter.BaseWidget. + - Patch #1096231: Add ``default`` argument to Tkinter.Wm.wm_iconbitmap. - Patch #763580: Add name and value arguments to Tkinter variable From buildbot at python.org Sat Jun 17 11:39:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 09:39:36 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060617093936.4FF021E4005@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1183 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 13:09:48 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 11:09:48 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060617110948.42CEE1E4005@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/681 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sat Jun 17 13:10:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 11:10:36 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060617111036.5EF561E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/375 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 17 17:50:50 2006 From: python-checkins at python.org (george.yoshida) Date: Sat, 17 Jun 2006 17:50:50 +0200 (CEST) Subject: [Python-checkins] r47001 - peps/trunk/pep-0291.txt Message-ID: <20060617155050.988571E4005@bag.python.org> Author: george.yoshida Date: Sat Jun 17 17:50:49 2006 New Revision: 47001 Modified: peps/trunk/pep-0291.txt Log: Replace tabs to spaces. Modified: peps/trunk/pep-0291.txt ============================================================================== --- peps/trunk/pep-0291.txt (original) +++ peps/trunk/pep-0291.txt Sat Jun 17 17:50:49 2006 @@ -59,10 +59,10 @@ 1.5.2 string methods, Unicode, list comprehensions, augmented assignment (eg, +=), zip(), import x as y, dict.setdefault(), print >> f, - calling f(*args, **kw), plus all features below + calling f(*args, **kw), plus all features below 2.0 nested scopes, rich comparisons, - function attributes, plus all features below + function attributes, plus all features below 2.1 use of object or new-style classes, iterators, using generators, nested scopes, or // @@ -70,12 +70,12 @@ plus all features below 2.2 bool, True, False, basestring, enumerate(), - {}.pop(), PendingDeprecationWarning, - Universal Newlines, plus all features below + {}.pop(), PendingDeprecationWarning, + Universal Newlines, plus all features below - 2.3 generator expressions, multi-line imports, - decorators, int/long unification, set/frozenset, - reversed(), sorted(), "".rsplit() + 2.3 generator expressions, multi-line imports, + decorators, int/long unification, set/frozenset, + reversed(), sorted(), "".rsplit() 2.4 ??? From python-checkins at python.org Sat Jun 17 17:58:49 2006 From: python-checkins at python.org (george.yoshida) Date: Sat, 17 Jun 2006 17:58:49 +0200 (CEST) Subject: [Python-checkins] r47002 - peps/trunk/pep-0291.txt Message-ID: <20060617155849.98F781E4006@bag.python.org> Author: george.yoshida Date: Sat Jun 17 17:58:47 2006 New Revision: 47002 Modified: peps/trunk/pep-0291.txt Log: Add uuid information. uuid.py states that: This module works with Python 2.3 or higher. Modified: peps/trunk/pep-0291.txt ============================================================================== --- peps/trunk/pep-0291.txt (original) +++ peps/trunk/pep-0291.txt Sat Jun 17 17:58:47 2006 @@ -98,6 +98,7 @@ pybench Marc-Andre Lemburg 1.5.2 [3] sre Fredrik Lundh 2.1 subprocess Peter Astrand 2.2 + uuid Ka-Ping Yee 2.3 wsgiref Phillip J. Eby 2.1 xml (PyXML) Martin v. Loewis 2.0 xmlrpclib Fredrik Lundh 2.1 From python-checkins at python.org Sat Jun 17 18:31:54 2006 From: python-checkins at python.org (george.yoshida) Date: Sat, 17 Jun 2006 18:31:54 +0200 (CEST) Subject: [Python-checkins] r47003 - python/trunk/Doc/ref/ref8.tex Message-ID: <20060617163154.19E0A1E4005@bag.python.org> Author: george.yoshida Date: Sat Jun 17 18:31:52 2006 New Revision: 47003 Modified: python/trunk/Doc/ref/ref8.tex Log: markup fix Modified: python/trunk/Doc/ref/ref8.tex ============================================================================== --- python/trunk/Doc/ref/ref8.tex (original) +++ python/trunk/Doc/ref/ref8.tex Sat Jun 17 18:31:52 2006 @@ -34,7 +34,7 @@ \index{interactive mode} \refbimodindex{__main__} -Under {\UNIX}, a complete program can be passed to the interpreter in +Under \UNIX{}, a complete program can be passed to the interpreter in three forms: with the \programopt{-c} \var{string} command line option, as a file passed as the first command line argument, or as standard input. If the file or standard input is a tty device, the interpreter enters From python-checkins at python.org Sat Jun 17 18:32:37 2006 From: python-checkins at python.org (george.yoshida) Date: Sat, 17 Jun 2006 18:32:37 +0200 (CEST) Subject: [Python-checkins] r47004 - python/branches/release24-maint/Doc/ref/ref8.tex Message-ID: <20060617163237.4C7241E4005@bag.python.org> Author: george.yoshida Date: Sat Jun 17 18:32:36 2006 New Revision: 47004 Modified: python/branches/release24-maint/Doc/ref/ref8.tex Log: markup fix Modified: python/branches/release24-maint/Doc/ref/ref8.tex ============================================================================== --- python/branches/release24-maint/Doc/ref/ref8.tex (original) +++ python/branches/release24-maint/Doc/ref/ref8.tex Sat Jun 17 18:32:36 2006 @@ -34,7 +34,7 @@ \index{interactive mode} \refbimodindex{__main__} -Under {\UNIX}, a complete program can be passed to the interpreter in +Under \UNIX{}, a complete program can be passed to the interpreter in three forms: with the \programopt{-c} \var{string} command line option, as a file passed as the first command line argument, or as standard input. If the file or standard input is a tty device, the interpreter enters From buildbot at python.org Sat Jun 17 18:37:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 16:37:37 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20060617163737.D38991E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/196 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From python-checkins at python.org Sat Jun 17 18:39:14 2006 From: python-checkins at python.org (george.yoshida) Date: Sat, 17 Jun 2006 18:39:14 +0200 (CEST) Subject: [Python-checkins] r47005 - python/trunk/Doc/lib/libzipfile.tex python/trunk/Doc/lib/libzipimport.tex Message-ID: <20060617163914.1DCE31E4005@bag.python.org> Author: george.yoshida Date: Sat Jun 17 18:39:13 2006 New Revision: 47005 Modified: python/trunk/Doc/lib/libzipfile.tex python/trunk/Doc/lib/libzipimport.tex Log: Update url. Old url returned status code:301 Moved permanently. Modified: python/trunk/Doc/lib/libzipfile.tex ============================================================================== --- python/trunk/Doc/lib/libzipfile.tex (original) +++ python/trunk/Doc/lib/libzipfile.tex Sat Jun 17 18:39:13 2006 @@ -13,8 +13,8 @@ This module provides tools to create, read, write, append, and list a ZIP file. Any advanced use of this module will require an understanding of the format, as defined in -\citetitle[http://www.pkware.com/appnote.html]{PKZIP Application -Note}. +\citetitle[http://www.pkware.com/business_and_developers/developer/appnote/] +{PKZIP Application Note}. This module does not currently handle ZIP files which have appended comments, or multi-disk ZIP files. It can handle ZIP files that use the @@ -71,9 +71,9 @@ \begin{seealso} - \seetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note}{Documentation on the ZIP file format by Phil - Katz, the creator of the format and algorithms used.} + \seetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note}{Documentation on the ZIP file format by + Phil Katz, the creator of the format and algorithms used.} \seetitle[http://www.info-zip.org/pub/infozip/]{Info-ZIP Home Page}{ Information about the Info-ZIP project's ZIP archive @@ -255,9 +255,9 @@ \begin{memberdesc}[ZipInfo]{extra} Expansion field data. The - \citetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note} contains some comments on the internal structure of the data - contained in this string. + \citetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note} contains some comments on the internal + structure of the data contained in this string. \end{memberdesc} \begin{memberdesc}[ZipInfo]{create_system} Modified: python/trunk/Doc/lib/libzipimport.tex ============================================================================== --- python/trunk/Doc/lib/libzipimport.tex (original) +++ python/trunk/Doc/lib/libzipimport.tex Sat Jun 17 18:39:13 2006 @@ -50,9 +50,9 @@ \begin{seealso} - \seetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note}{Documentation on the ZIP file format by Phil - Katz, the creator of the format and algorithms used.} + \seetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note}{Documentation on the ZIP file format by + Phil Katz, the creator of the format and algorithms used.} \seepep{0273}{Import Modules from Zip Archives}{Written by James C. Ahlstrom, who also provided an implementation. Python 2.3 From python-checkins at python.org Sat Jun 17 18:40:46 2006 From: python-checkins at python.org (george.yoshida) Date: Sat, 17 Jun 2006 18:40:46 +0200 (CEST) Subject: [Python-checkins] r47006 - python/branches/release24-maint/Doc/lib/libzipfile.tex python/branches/release24-maint/Doc/lib/libzipimport.tex Message-ID: <20060617164046.BEF381E400B@bag.python.org> Author: george.yoshida Date: Sat Jun 17 18:40:46 2006 New Revision: 47006 Modified: python/branches/release24-maint/Doc/lib/libzipfile.tex python/branches/release24-maint/Doc/lib/libzipimport.tex Log: Update url. Old url returned status code:301 Moved permanently. (Backport from r47005) Modified: python/branches/release24-maint/Doc/lib/libzipfile.tex ============================================================================== --- python/branches/release24-maint/Doc/lib/libzipfile.tex (original) +++ python/branches/release24-maint/Doc/lib/libzipfile.tex Sat Jun 17 18:40:46 2006 @@ -13,8 +13,8 @@ This module provides tools to create, read, write, append, and list a ZIP file. Any advanced use of this module will require an understanding of the format, as defined in -\citetitle[http://www.pkware.com/appnote.html]{PKZIP Application -Note}. +\citetitle[http://www.pkware.com/business_and_developers/developer/appnote/] +{PKZIP Application Note}. This module does not currently handle ZIP files which have appended comments, or multi-disk ZIP files. @@ -65,9 +65,9 @@ \begin{seealso} - \seetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note}{Documentation on the ZIP file format by Phil - Katz, the creator of the format and algorithms used.} + \seetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note}{Documentation on the ZIP file format by + Phil Katz, the creator of the format and algorithms used.} \seetitle[http://www.info-zip.org/pub/infozip/]{Info-ZIP Home Page}{ Information about the Info-ZIP project's ZIP archive @@ -242,9 +242,9 @@ \begin{memberdesc}[ZipInfo]{extra} Expansion field data. The - \citetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note} contains some comments on the internal structure of the data - contained in this string. + \citetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note} contains some comments on the internal + structure of the data contained in this string. \end{memberdesc} \begin{memberdesc}[ZipInfo]{create_system} Modified: python/branches/release24-maint/Doc/lib/libzipimport.tex ============================================================================== --- python/branches/release24-maint/Doc/lib/libzipimport.tex (original) +++ python/branches/release24-maint/Doc/lib/libzipimport.tex Sat Jun 17 18:40:46 2006 @@ -50,9 +50,9 @@ \begin{seealso} - \seetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note}{Documentation on the ZIP file format by Phil - Katz, the creator of the format and algorithms used.} + \seetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note}{Documentation on the ZIP file format by + Phil Katz, the creator of the format and algorithms used.} \seepep{0273}{Import Modules from Zip Archives}{Written by James C. Ahlstrom, who also provided an implementation. Python 2.3 From buildbot at python.org Sat Jun 17 19:52:32 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 17:52:32 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD 2.4 Message-ID: <20060617175232.DDDE01E4005@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%25202.4/builds/122 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: george.yoshida Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sat Jun 17 20:44:28 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 17 Jun 2006 20:44:28 +0200 (CEST) Subject: [Python-checkins] r47007 - in python/trunk: Lib/lib-tk/turtle.py Misc/NEWS Message-ID: <20060617184428.35A8C1E4005@bag.python.org> Author: martin.v.loewis Date: Sat Jun 17 20:44:27 2006 New Revision: 47007 Modified: python/trunk/Lib/lib-tk/turtle.py python/trunk/Misc/NEWS Log: Patch #812986: Update the canvas even if not tracing. Modified: python/trunk/Lib/lib-tk/turtle.py ============================================================================== --- python/trunk/Lib/lib-tk/turtle.py (original) +++ python/trunk/Lib/lib-tk/turtle.py Sat Jun 17 20:44:27 2006 @@ -634,6 +634,7 @@ def _draw_turtle(self, position=[]): if not self._tracing: + self._canvas.update() return if position == []: position = self._position Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 17 20:44:27 2006 @@ -163,6 +163,8 @@ Library ------- +- Patch #812986: Update turtle output even if not tracing. + - Patch #1494750: Destroy master after deleting children in Tkinter.BaseWidget. From g.brandl at gmx.net Sat Jun 17 20:56:23 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 17 Jun 2006 20:56:23 +0200 Subject: [Python-checkins] r47002 - peps/trunk/pep-0291.txt In-Reply-To: <20060617155849.98F781E4006@bag.python.org> References: <20060617155849.98F781E4006@bag.python.org> Message-ID: george.yoshida wrote: > Author: george.yoshida > Date: Sat Jun 17 17:58:47 2006 > New Revision: 47002 > > Modified: > peps/trunk/pep-0291.txt > Log: > Add uuid information. > > uuid.py states that: > This module works with Python 2.3 or higher. Did you ask Ping whether he wants to keep this compatibility? IIRC we wanted to reduce such dependencies. Georg From python-checkins at python.org Sat Jun 17 21:03:34 2006 From: python-checkins at python.org (martin.v.loewis) Date: Sat, 17 Jun 2006 21:03:34 +0200 (CEST) Subject: [Python-checkins] r47008 - in python/trunk: Lib/lib-tk/tkMessageBox.py Misc/NEWS Message-ID: <20060617190334.550251E4005@bag.python.org> Author: martin.v.loewis Date: Sat Jun 17 21:03:26 2006 New Revision: 47008 Modified: python/trunk/Lib/lib-tk/tkMessageBox.py python/trunk/Misc/NEWS Log: Patch #815924: Restore ability to pass type= and icon= Modified: python/trunk/Lib/lib-tk/tkMessageBox.py ============================================================================== --- python/trunk/Lib/lib-tk/tkMessageBox.py (original) +++ python/trunk/Lib/lib-tk/tkMessageBox.py Sat Jun 17 21:03:26 2006 @@ -63,9 +63,10 @@ # # convenience stuff -def _show(title=None, message=None, icon=None, type=None, **options): - if icon: options["icon"] = icon - if type: options["type"] = type +# Rename _icon and _type options to allow overriding them in options +def _show(title=None, message=None, _icon=None, _type=None, **options): + if _icon and "icon" not in options: options["icon"] = _icon + if _type and "type" not in options: options["type"] = _type if title: options["title"] = title if message: options["message"] = message res = Message(**options).show() Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Jun 17 21:03:26 2006 @@ -163,6 +163,9 @@ Library ------- +- Patch #815924: Restore ability to pass type= and icon= in tkMessageBox + functions. + - Patch #812986: Update turtle output even if not tracing. - Patch #1494750: Destroy master after deleting children in From martin at v.loewis.de Sat Jun 17 21:11:47 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 17 Jun 2006 21:11:47 +0200 Subject: [Python-checkins] r47002 - peps/trunk/pep-0291.txt In-Reply-To: References: <20060617155849.98F781E4006@bag.python.org> Message-ID: <449453F3.30900@v.loewis.de> Georg Brandl wrote: >> uuid.py states that: >> This module works with Python 2.3 or higher. > > Did you ask Ping whether he wants to keep this compatibility? > IIRC we wanted to reduce such dependencies. I was wondering about the same. If it is listed only because it works with 2.3, not because ongoing compatibility with 2.3 is required, it should be removed from the PEP. The comment could be removed from the file, or it could stay until it becomes false. Regards, Martin From buildbot at python.org Sat Jun 17 21:56:23 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 19:56:23 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060617195623.E793B1E400A@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/63 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: george.yoshida Build Had Warnings: warnings test sincerely, -The Buildbot From amk at amk.ca Sat Jun 17 23:07:16 2006 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 17 Jun 2006 17:07:16 -0400 Subject: [Python-checkins] r47003 - python/trunk/Doc/ref/ref8.tex In-Reply-To: <20060617163154.19E0A1E4005@bag.python.org> References: <20060617163154.19E0A1E4005@bag.python.org> Message-ID: <20060617210716.GA11407@Andrew-iBook2.local> On Sat, Jun 17, 2006 at 06:31:54PM +0200, george.yoshida wrote: > -Under {\UNIX}, a complete program can be passed to the interpreter in > +Under \UNIX{}, a complete program can be passed to the interpreter in What's the problem with the original markup? I don't actually see any difference -- in the original, \UNIX is in a group and in the new the group is empty -- and at least the HTML output looks correct (see http://docs.python.org/ref/programs.html). Actually, I suspect the {} is unnecessary in this case because in "\UNIX,", the comma would indicate the end of the macro. The group would only be needed if the macro was followed by a space, as in "The \UNIX{} operating system". --amk From python-checkins at python.org Sun Jun 18 00:37:46 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 18 Jun 2006 00:37:46 +0200 (CEST) Subject: [Python-checkins] r47009 - python/trunk/Lib/lib-tk/Tkinter.py Message-ID: <20060617223746.DE2F51E4005@bag.python.org> Author: neal.norwitz Date: Sun Jun 18 00:37:45 2006 New Revision: 47009 Modified: python/trunk/Lib/lib-tk/Tkinter.py Log: Fix typo in docstring Modified: python/trunk/Lib/lib-tk/Tkinter.py ============================================================================== --- python/trunk/Lib/lib-tk/Tkinter.py (original) +++ python/trunk/Lib/lib-tk/Tkinter.py Sun Jun 18 00:37:45 2006 @@ -1506,7 +1506,7 @@ Under Windows, the DEFAULT parameter can be used to set the icon for the widget and any descendents that don't have an icon set - explicitely. DEFAULT can be the relative path to a .ico file + explicitly. DEFAULT can be the relative path to a .ico file (example: root.iconbitmap(default='myicon.ico') ). See Tk documentation for more information.""" if default: From python-checkins at python.org Sun Jun 18 00:38:16 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 18 Jun 2006 00:38:16 +0200 (CEST) Subject: [Python-checkins] r47010 - python/trunk/Modules/_bsddb.c Message-ID: <20060617223816.47E051E4005@bag.python.org> Author: neal.norwitz Date: Sun Jun 18 00:38:15 2006 New Revision: 47010 Modified: python/trunk/Modules/_bsddb.c Log: Fix memory leak reported by valgrind while running test_subprocess Modified: python/trunk/Modules/_bsddb.c ============================================================================== --- python/trunk/Modules/_bsddb.c (original) +++ python/trunk/Modules/_bsddb.c Sun Jun 18 00:38:15 2006 @@ -2721,6 +2721,8 @@ */ if (size == 0 && (flags & DB_FAST_STAT)) { flags = 0; + if (!err) + free(sp); goto redo_stat_for_length; } From buildbot at python.org Sun Jun 18 01:51:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Sat, 17 Jun 2006 23:51:17 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060617235117.D6A7D1E4005@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/685 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From dynkin at gmail.com Sun Jun 18 04:40:22 2006 From: dynkin at gmail.com (George Yoshida) Date: Sun, 18 Jun 2006 11:40:22 +0900 Subject: [Python-checkins] r47002 - peps/trunk/pep-0291.txt In-Reply-To: <449453F3.30900@v.loewis.de> References: <20060617155849.98F781E4006@bag.python.org> <449453F3.30900@v.loewis.de> Message-ID: <2f188ee80606171940v635bcb52gd1d1b034f9dccf32@mail.gmail.com> On 6/18/06, "Martin v. L?wis" wrote: > Georg Brandl wrote: > >> uuid.py states that: > >> This module works with Python 2.3 or higher. > > > > Did you ask Ping whether he wants to keep this compatibility? > > IIRC we wanted to reduce such dependencies. > > I was wondering about the same. If it is listed only because > it works with 2.3, not because ongoing compatibility with 2.3 > is required, it should be removed from the PEP. The comment > could be removed from the file, or it could stay until it becomes > false. I looked at uuid.py and followed py-dev discussions carefully and ctypes are there. Apart from the Ka-Pin's initial plan, it's unlikely that current uuid is assumed to run on vanilla Python 2.3 or 2.4. I was fooled by that comment. I'll ask Ka-Ping and py-dev forks. Thanks for alerting me. -- george From python-checkins at python.org Sun Jun 18 04:57:36 2006 From: python-checkins at python.org (fred.drake) Date: Sun, 18 Jun 2006 04:57:36 +0200 (CEST) Subject: [Python-checkins] r47011 - python/trunk/Doc/ref/ref8.tex Message-ID: <20060618025736.42EA51E4005@bag.python.org> Author: fred.drake Date: Sun Jun 18 04:57:35 2006 New Revision: 47011 Modified: python/trunk/Doc/ref/ref8.tex Log: remove unnecessary markup Modified: python/trunk/Doc/ref/ref8.tex ============================================================================== --- python/trunk/Doc/ref/ref8.tex (original) +++ python/trunk/Doc/ref/ref8.tex Sun Jun 18 04:57:35 2006 @@ -34,7 +34,7 @@ \index{interactive mode} \refbimodindex{__main__} -Under \UNIX{}, a complete program can be passed to the interpreter in +Under \UNIX, a complete program can be passed to the interpreter in three forms: with the \programopt{-c} \var{string} command line option, as a file passed as the first command line argument, or as standard input. If the file or standard input is a tty device, the interpreter enters From python-checkins at python.org Sun Jun 18 04:58:20 2006 From: python-checkins at python.org (fred.drake) Date: Sun, 18 Jun 2006 04:58:20 +0200 (CEST) Subject: [Python-checkins] r47012 - python/branches/release24-maint/Doc/ref/ref8.tex Message-ID: <20060618025820.8FF971E4005@bag.python.org> Author: fred.drake Date: Sun Jun 18 04:58:20 2006 New Revision: 47012 Modified: python/branches/release24-maint/Doc/ref/ref8.tex Log: remove unnecessary markup Modified: python/branches/release24-maint/Doc/ref/ref8.tex ============================================================================== --- python/branches/release24-maint/Doc/ref/ref8.tex (original) +++ python/branches/release24-maint/Doc/ref/ref8.tex Sun Jun 18 04:58:20 2006 @@ -34,7 +34,7 @@ \index{interactive mode} \refbimodindex{__main__} -Under \UNIX{}, a complete program can be passed to the interpreter in +Under \UNIX, a complete program can be passed to the interpreter in three forms: with the \programopt{-c} \var{string} command line option, as a file passed as the first command line argument, or as standard input. If the file or standard input is a tty device, the interpreter enters From fdrake at acm.org Sun Jun 18 04:58:29 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Sat, 17 Jun 2006 22:58:29 -0400 Subject: [Python-checkins] r47003 - python/trunk/Doc/ref/ref8.tex In-Reply-To: <20060617210716.GA11407@Andrew-iBook2.local> References: <20060617163154.19E0A1E4005@bag.python.org> <20060617210716.GA11407@Andrew-iBook2.local> Message-ID: <200606172258.29930.fdrake@acm.org> On Saturday 17 June 2006 17:07, A.M. Kuchling wrote: > Actually, I suspect the {} is unnecessary in this case because in > "\UNIX,", the comma would indicate the end of the macro. The group > would only be needed if the macro was followed by a space, as in "The > \UNIX{} operating system". That's correct. No change was needed, and the best change would have been to simply remove the group. This is now done in Subversion. -Fred -- Fred L. Drake, Jr. From buildbot at python.org Sun Jun 18 05:03:31 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 18 Jun 2006 03:03:31 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060618030331.DBE781E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/380 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From dynkin at gmail.com Sun Jun 18 17:37:46 2006 From: dynkin at gmail.com (George Yoshida) Date: Mon, 19 Jun 2006 00:37:46 +0900 Subject: [Python-checkins] r47003 - python/trunk/Doc/ref/ref8.tex In-Reply-To: <20060617210716.GA11407@Andrew-iBook2.local> References: <20060617163154.19E0A1E4005@bag.python.org> <20060617210716.GA11407@Andrew-iBook2.local> Message-ID: <2f188ee80606180837n51d4769bw798504420232bee1@mail.gmail.com> On 6/18/06, A.M. Kuchling wrote: > What's the problem with the original markup? I don't actually see any > difference -- in the original, \UNIX is in a group and in the new the > group is empty -- and at least the HTML output looks correct (see > http://docs.python.org/ref/programs.html). My change was done to make \UNIX macro look like other parts of it. > Actually, I suspect the {} is unnecessary in this case because in > "\UNIX,", the comma would indicate the end of the macro. The group > would only be needed if the macro was followed by a space, as in "The > \UNIX{} operating system". I'm afraid I was mistaken about the meaning of curly braces. I thought \UNIX or \UNIX{} is a one-word and didn't know {} means grouping. Thanks for clarifying what they do. -- george From fdrake at acm.org Sun Jun 18 19:44:59 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Sun, 18 Jun 2006 13:44:59 -0400 Subject: [Python-checkins] r47003 - python/trunk/Doc/ref/ref8.tex In-Reply-To: <2f188ee80606180837n51d4769bw798504420232bee1@mail.gmail.com> References: <20060617163154.19E0A1E4005@bag.python.org> <20060617210716.GA11407@Andrew-iBook2.local> <2f188ee80606180837n51d4769bw798504420232bee1@mail.gmail.com> Message-ID: <200606181345.00037.fdrake@acm.org> On Sunday 18 June 2006 11:37, George Yoshida wrote: > My change was done to make \UNIX macro look like other parts of it. I think there are two ways \UNIX is commonly used in the doc sources: \UNIX{} -- used when followed by whitespace \UNIX -- used when followed by punctuation So you're right that {\UNIX} was an aberration. -Fred -- Fred L. Drake, Jr. From python-checkins at python.org Sun Jun 18 21:35:02 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 18 Jun 2006 21:35:02 +0200 (CEST) Subject: [Python-checkins] r47013 - in python/trunk: Lib/test/test_socket.py Lib/test/test_support.py Lib/test/test_threadedtempfile.py Misc/build.sh Message-ID: <20060618193502.772BF1E4002@bag.python.org> Author: neal.norwitz Date: Sun Jun 18 21:35:01 2006 New Revision: 47013 Modified: python/trunk/Lib/test/test_socket.py python/trunk/Lib/test/test_support.py python/trunk/Lib/test/test_threadedtempfile.py python/trunk/Misc/build.sh Log: Prevent spurious leaks when running regrtest.py -R. There may be more issues that crop up from time to time, but this change seems to have been pretty stable (no spurious warnings) for about a week. Other modules which use threads may require similar use of threading_setup/threading_cleanup from test_support. Modified: python/trunk/Lib/test/test_socket.py ============================================================================== --- python/trunk/Lib/test/test_socket.py (original) +++ python/trunk/Lib/test/test_socket.py Sun Jun 18 21:35:01 2006 @@ -906,7 +906,10 @@ tests.append(BasicSocketPairTest) if sys.platform == 'linux2': tests.append(TestLinuxAbstractNamespace) + + thread_info = test_support.threading_setup() test_support.run_unittest(*tests) + test_support.threading_cleanup(*thread_info) if __name__ == "__main__": test_main() Modified: python/trunk/Lib/test/test_support.py ============================================================================== --- python/trunk/Lib/test/test_support.py (original) +++ python/trunk/Lib/test/test_support.py Sun Jun 18 21:35:01 2006 @@ -453,3 +453,26 @@ if verbose: print 'doctest (%s) ... %d tests with zero failures' % (module.__name__, t) return f, t + +#======================================================================= +# Threading support to prevent reporting refleaks when running regrtest.py -R + +def threading_setup(): + import threading + return len(threading._active), len(threading._limbo) + +def threading_cleanup(num_active, num_limbo): + import threading + import time + + _MAX_COUNT = 10 + count = 0 + while len(threading._active) != num_active and count < _MAX_COUNT: + count += 1 + time.sleep(0.1) + + count = 0 + while len(threading._limbo) != num_limbo and count < _MAX_COUNT: + count += 1 + time.sleep(0.1) + Modified: python/trunk/Lib/test/test_threadedtempfile.py ============================================================================== --- python/trunk/Lib/test/test_threadedtempfile.py (original) +++ python/trunk/Lib/test/test_threadedtempfile.py Sun Jun 18 21:35:01 2006 @@ -22,7 +22,7 @@ import thread # If this fails, we can't test this module import threading -from test.test_support import TestFailed +from test.test_support import TestFailed, threading_setup, threading_cleanup import StringIO from traceback import print_exc import tempfile @@ -48,6 +48,7 @@ def test_main(): threads = [] + thread_info = threading_setup() print "Creating" for i in range(NUM_THREADS): @@ -72,6 +73,7 @@ if errors: raise TestFailed(msg) + threading_cleanup(*thread_info) if __name__ == "__main__": import sys, getopt Modified: python/trunk/Misc/build.sh ============================================================================== --- python/trunk/Misc/build.sh (original) +++ python/trunk/Misc/build.sh Sun Jun 18 21:35:01 2006 @@ -60,7 +60,7 @@ # Note: test_XXX (none currently) really leak, but are disabled # so we don't send spam. Any test which really leaks should only # be listed here if there are also test cases under Lib/test/leakers. -LEAKY_TESTS="test_(ctypes|filecmp|socket|threadedtempfile|threading|urllib2)" +LEAKY_TESTS="test_(XXX)" # Currently no tests should report spurious leaks. # Skip these tests altogether when looking for leaks. These tests # do not need to be stored above in LEAKY_TESTS too. From python-checkins at python.org Sun Jun 18 21:37:41 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 18 Jun 2006 21:37:41 +0200 (CEST) Subject: [Python-checkins] r47014 - python/trunk/Lib/test/fork_wait.py python/trunk/Lib/test/test_fork1.py python/trunk/Lib/test/test_wait3.py python/trunk/Lib/test/test_wait4.py Message-ID: <20060618193741.4B9E01E4002@bag.python.org> Author: neal.norwitz Date: Sun Jun 18 21:37:40 2006 New Revision: 47014 Modified: python/trunk/Lib/test/fork_wait.py python/trunk/Lib/test/test_fork1.py python/trunk/Lib/test/test_wait3.py python/trunk/Lib/test/test_wait4.py Log: The hppa ubuntu box sometimes hangs forever in these tests. My guess is that the wait is failing for some reason. Use WNOHANG, so we won't wait until the buildbot kills the test suite. I haven't been able to reproduce the failure, so I'm not sure if this will help or not. Hopefully, this change will cause the test to fail, rather than hang. That will be better since we will get the rest of the test results. It may also help us debug the real problem. Modified: python/trunk/Lib/test/fork_wait.py ============================================================================== --- python/trunk/Lib/test/fork_wait.py (original) +++ python/trunk/Lib/test/fork_wait.py Sun Jun 18 21:37:40 2006 @@ -34,7 +34,14 @@ pass def wait_impl(self, cpid): - spid, status = os.waitpid(cpid, 0) + for i in range(10): + # waitpid() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status = os.waitpid(cpid, os.WNOHANG) + if spid == cpid: + break + time.sleep(2 * SHORTSLEEP) + self.assertEquals(spid, cpid) self.assertEquals(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) Modified: python/trunk/Lib/test/test_fork1.py ============================================================================== --- python/trunk/Lib/test/test_fork1.py (original) +++ python/trunk/Lib/test/test_fork1.py Sun Jun 18 21:37:40 2006 @@ -2,6 +2,7 @@ """ import os +import time from test.fork_wait import ForkWait from test.test_support import TestSkipped, run_unittest @@ -12,7 +13,14 @@ class ForkTest(ForkWait): def wait_impl(self, cpid): - spid, status = os.waitpid(cpid, 0) + for i in range(10): + # waitpid() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status = os.waitpid(cpid, os.WNOHANG) + if spid == cpid: + break + time.sleep(1.0) + self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) Modified: python/trunk/Lib/test/test_wait3.py ============================================================================== --- python/trunk/Lib/test/test_wait3.py (original) +++ python/trunk/Lib/test/test_wait3.py Sun Jun 18 21:37:40 2006 @@ -2,6 +2,7 @@ """ import os +import time from test.fork_wait import ForkWait from test.test_support import TestSkipped, run_unittest @@ -17,10 +18,14 @@ class Wait3Test(ForkWait): def wait_impl(self, cpid): - while 1: - spid, status, rusage = os.wait3(0) + for i in range(10): + # wait3() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status, rusage = os.wait3(os.WNOHANG) if spid == cpid: break + time.sleep(1.0) + self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) self.assertTrue(rusage) Modified: python/trunk/Lib/test/test_wait4.py ============================================================================== --- python/trunk/Lib/test/test_wait4.py (original) +++ python/trunk/Lib/test/test_wait4.py Sun Jun 18 21:37:40 2006 @@ -2,6 +2,7 @@ """ import os +import time from test.fork_wait import ForkWait from test.test_support import TestSkipped, run_unittest @@ -17,7 +18,13 @@ class Wait4Test(ForkWait): def wait_impl(self, cpid): - spid, status, rusage = os.wait4(cpid, 0) + for i in range(10): + # wait4() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status, rusage = os.wait4(cpid, os.WNOHANG) + if spid == cpid: + break + time.sleep(1.0) self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) self.assertTrue(rusage) From nnorwitz at gmail.com Sun Jun 18 21:39:26 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 18 Jun 2006 12:39:26 -0700 Subject: [Python-checkins] r47013 - in python/trunk: Lib/test/test_socket.py Lib/test/test_support.py Lib/test/test_threadedtempfile.py Misc/build.sh In-Reply-To: <20060618193502.772BF1E4002@bag.python.org> References: <20060618193502.772BF1E4002@bag.python.org> Message-ID: I should note that 10 is arbitrary. This waits a total of a second. That may not be enough time. On 6/18/06, neal.norwitz wrote: > Author: neal.norwitz > Date: Sun Jun 18 21:35:01 2006 > New Revision: 47013 > > Modified: > python/trunk/Lib/test/test_socket.py > python/trunk/Lib/test/test_support.py > python/trunk/Lib/test/test_threadedtempfile.py > python/trunk/Misc/build.sh > Log: > Prevent spurious leaks when running regrtest.py -R. There may be more > issues that crop up from time to time, but this change seems to have been > pretty stable (no spurious warnings) for about a week. > > Other modules which use threads may require similar use of > threading_setup/threading_cleanup from test_support. > > > Modified: python/trunk/Lib/test/test_socket.py > ============================================================================== > --- python/trunk/Lib/test/test_socket.py (original) > +++ python/trunk/Lib/test/test_socket.py Sun Jun 18 21:35:01 2006 > @@ -906,7 +906,10 @@ > tests.append(BasicSocketPairTest) > if sys.platform == 'linux2': > tests.append(TestLinuxAbstractNamespace) > + > + thread_info = test_support.threading_setup() > test_support.run_unittest(*tests) > + test_support.threading_cleanup(*thread_info) > > if __name__ == "__main__": > test_main() > > Modified: python/trunk/Lib/test/test_support.py > ============================================================================== > --- python/trunk/Lib/test/test_support.py (original) > +++ python/trunk/Lib/test/test_support.py Sun Jun 18 21:35:01 2006 > @@ -453,3 +453,26 @@ > if verbose: > print 'doctest (%s) ... %d tests with zero failures' % (module.__name__, t) > return f, t > + > +#======================================================================= > +# Threading support to prevent reporting refleaks when running regrtest.py -R > + > +def threading_setup(): > + import threading > + return len(threading._active), len(threading._limbo) > + > +def threading_cleanup(num_active, num_limbo): > + import threading > + import time > + > + _MAX_COUNT = 10 > + count = 0 > + while len(threading._active) != num_active and count < _MAX_COUNT: > + count += 1 > + time.sleep(0.1) > + > + count = 0 > + while len(threading._limbo) != num_limbo and count < _MAX_COUNT: > + count += 1 > + time.sleep(0.1) > + > > Modified: python/trunk/Lib/test/test_threadedtempfile.py > ============================================================================== > --- python/trunk/Lib/test/test_threadedtempfile.py (original) > +++ python/trunk/Lib/test/test_threadedtempfile.py Sun Jun 18 21:35:01 2006 > @@ -22,7 +22,7 @@ > > import thread # If this fails, we can't test this module > import threading > -from test.test_support import TestFailed > +from test.test_support import TestFailed, threading_setup, threading_cleanup > import StringIO > from traceback import print_exc > import tempfile > @@ -48,6 +48,7 @@ > > def test_main(): > threads = [] > + thread_info = threading_setup() > > print "Creating" > for i in range(NUM_THREADS): > @@ -72,6 +73,7 @@ > if errors: > raise TestFailed(msg) > > + threading_cleanup(*thread_info) > > if __name__ == "__main__": > import sys, getopt > > Modified: python/trunk/Misc/build.sh > ============================================================================== > --- python/trunk/Misc/build.sh (original) > +++ python/trunk/Misc/build.sh Sun Jun 18 21:35:01 2006 > @@ -60,7 +60,7 @@ > # Note: test_XXX (none currently) really leak, but are disabled > # so we don't send spam. Any test which really leaks should only > # be listed here if there are also test cases under Lib/test/leakers. > -LEAKY_TESTS="test_(ctypes|filecmp|socket|threadedtempfile|threading|urllib2)" > +LEAKY_TESTS="test_(XXX)" # Currently no tests should report spurious leaks. > > # Skip these tests altogether when looking for leaks. These tests > # do not need to be stored above in LEAKY_TESTS too. > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From nnorwitz at gmail.com Sun Jun 18 21:40:08 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 18 Jun 2006 12:40:08 -0700 Subject: [Python-checkins] r47014 - python/trunk/Lib/test/fork_wait.py python/trunk/Lib/test/test_fork1.py python/trunk/Lib/test/test_wait3.py python/trunk/Lib/test/test_wait4.py In-Reply-To: <20060618193741.4B9E01E4002@bag.python.org> References: <20060618193741.4B9E01E4002@bag.python.org> Message-ID: 10 times with 1 second waits is arbitrary. These values may need to be adjusted. On 6/18/06, neal.norwitz wrote: > Author: neal.norwitz > Date: Sun Jun 18 21:37:40 2006 > New Revision: 47014 > > Modified: > python/trunk/Lib/test/fork_wait.py > python/trunk/Lib/test/test_fork1.py > python/trunk/Lib/test/test_wait3.py > python/trunk/Lib/test/test_wait4.py > Log: > The hppa ubuntu box sometimes hangs forever in these tests. My guess > is that the wait is failing for some reason. Use WNOHANG, so we won't > wait until the buildbot kills the test suite. > > I haven't been able to reproduce the failure, so I'm not sure if > this will help or not. Hopefully, this change will cause the test > to fail, rather than hang. That will be better since we will get > the rest of the test results. It may also help us debug the real problem. > > > Modified: python/trunk/Lib/test/fork_wait.py > ============================================================================== > --- python/trunk/Lib/test/fork_wait.py (original) > +++ python/trunk/Lib/test/fork_wait.py Sun Jun 18 21:37:40 2006 > @@ -34,7 +34,14 @@ > pass > > def wait_impl(self, cpid): > - spid, status = os.waitpid(cpid, 0) > + for i in range(10): > + # waitpid() shouldn't hang, but some of the buildbots seem to hang > + # in the forking tests. This is an attempt to fix the problem. > + spid, status = os.waitpid(cpid, os.WNOHANG) > + if spid == cpid: > + break > + time.sleep(2 * SHORTSLEEP) > + > self.assertEquals(spid, cpid) > self.assertEquals(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) > > > Modified: python/trunk/Lib/test/test_fork1.py > ============================================================================== > --- python/trunk/Lib/test/test_fork1.py (original) > +++ python/trunk/Lib/test/test_fork1.py Sun Jun 18 21:37:40 2006 > @@ -2,6 +2,7 @@ > """ > > import os > +import time > from test.fork_wait import ForkWait > from test.test_support import TestSkipped, run_unittest > > @@ -12,7 +13,14 @@ > > class ForkTest(ForkWait): > def wait_impl(self, cpid): > - spid, status = os.waitpid(cpid, 0) > + for i in range(10): > + # waitpid() shouldn't hang, but some of the buildbots seem to hang > + # in the forking tests. This is an attempt to fix the problem. > + spid, status = os.waitpid(cpid, os.WNOHANG) > + if spid == cpid: > + break > + time.sleep(1.0) > + > self.assertEqual(spid, cpid) > self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) > > > Modified: python/trunk/Lib/test/test_wait3.py > ============================================================================== > --- python/trunk/Lib/test/test_wait3.py (original) > +++ python/trunk/Lib/test/test_wait3.py Sun Jun 18 21:37:40 2006 > @@ -2,6 +2,7 @@ > """ > > import os > +import time > from test.fork_wait import ForkWait > from test.test_support import TestSkipped, run_unittest > > @@ -17,10 +18,14 @@ > > class Wait3Test(ForkWait): > def wait_impl(self, cpid): > - while 1: > - spid, status, rusage = os.wait3(0) > + for i in range(10): > + # wait3() shouldn't hang, but some of the buildbots seem to hang > + # in the forking tests. This is an attempt to fix the problem. > + spid, status, rusage = os.wait3(os.WNOHANG) > if spid == cpid: > break > + time.sleep(1.0) > + > self.assertEqual(spid, cpid) > self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) > self.assertTrue(rusage) > > Modified: python/trunk/Lib/test/test_wait4.py > ============================================================================== > --- python/trunk/Lib/test/test_wait4.py (original) > +++ python/trunk/Lib/test/test_wait4.py Sun Jun 18 21:37:40 2006 > @@ -2,6 +2,7 @@ > """ > > import os > +import time > from test.fork_wait import ForkWait > from test.test_support import TestSkipped, run_unittest > > @@ -17,7 +18,13 @@ > > class Wait4Test(ForkWait): > def wait_impl(self, cpid): > - spid, status, rusage = os.wait4(cpid, 0) > + for i in range(10): > + # wait4() shouldn't hang, but some of the buildbots seem to hang > + # in the forking tests. This is an attempt to fix the problem. > + spid, status, rusage = os.wait4(cpid, os.WNOHANG) > + if spid == cpid: > + break > + time.sleep(1.0) > self.assertEqual(spid, cpid) > self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) > self.assertTrue(rusage) > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From buildbot at python.org Sun Jun 18 22:00:15 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 18 Jun 2006 20:00:15 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060618200015.C21DD1E4005@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1191 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Sun Jun 18 22:05:39 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 18 Jun 2006 20:05:39 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060618200540.1E73A1E4005@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1095 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Sun Jun 18 22:10:25 2006 From: python-checkins at python.org (neal.norwitz) Date: Sun, 18 Jun 2006 22:10:25 +0200 (CEST) Subject: [Python-checkins] r47015 - python/trunk/Lib/test/fork_wait.py python/trunk/Lib/test/test_fork1.py python/trunk/Lib/test/test_wait3.py python/trunk/Lib/test/test_wait4.py Message-ID: <20060618201025.4B1961E4009@bag.python.org> Author: neal.norwitz Date: Sun Jun 18 22:10:24 2006 New Revision: 47015 Modified: python/trunk/Lib/test/fork_wait.py python/trunk/Lib/test/test_fork1.py python/trunk/Lib/test/test_wait3.py python/trunk/Lib/test/test_wait4.py Log: Revert 47014 until it is more robust Modified: python/trunk/Lib/test/fork_wait.py ============================================================================== --- python/trunk/Lib/test/fork_wait.py (original) +++ python/trunk/Lib/test/fork_wait.py Sun Jun 18 22:10:24 2006 @@ -34,14 +34,7 @@ pass def wait_impl(self, cpid): - for i in range(10): - # waitpid() shouldn't hang, but some of the buildbots seem to hang - # in the forking tests. This is an attempt to fix the problem. - spid, status = os.waitpid(cpid, os.WNOHANG) - if spid == cpid: - break - time.sleep(2 * SHORTSLEEP) - + spid, status = os.waitpid(cpid, 0) self.assertEquals(spid, cpid) self.assertEquals(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) Modified: python/trunk/Lib/test/test_fork1.py ============================================================================== --- python/trunk/Lib/test/test_fork1.py (original) +++ python/trunk/Lib/test/test_fork1.py Sun Jun 18 22:10:24 2006 @@ -2,7 +2,6 @@ """ import os -import time from test.fork_wait import ForkWait from test.test_support import TestSkipped, run_unittest @@ -13,14 +12,7 @@ class ForkTest(ForkWait): def wait_impl(self, cpid): - for i in range(10): - # waitpid() shouldn't hang, but some of the buildbots seem to hang - # in the forking tests. This is an attempt to fix the problem. - spid, status = os.waitpid(cpid, os.WNOHANG) - if spid == cpid: - break - time.sleep(1.0) - + spid, status = os.waitpid(cpid, 0) self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) Modified: python/trunk/Lib/test/test_wait3.py ============================================================================== --- python/trunk/Lib/test/test_wait3.py (original) +++ python/trunk/Lib/test/test_wait3.py Sun Jun 18 22:10:24 2006 @@ -2,7 +2,6 @@ """ import os -import time from test.fork_wait import ForkWait from test.test_support import TestSkipped, run_unittest @@ -18,14 +17,10 @@ class Wait3Test(ForkWait): def wait_impl(self, cpid): - for i in range(10): - # wait3() shouldn't hang, but some of the buildbots seem to hang - # in the forking tests. This is an attempt to fix the problem. - spid, status, rusage = os.wait3(os.WNOHANG) + while 1: + spid, status, rusage = os.wait3(0) if spid == cpid: break - time.sleep(1.0) - self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) self.assertTrue(rusage) Modified: python/trunk/Lib/test/test_wait4.py ============================================================================== --- python/trunk/Lib/test/test_wait4.py (original) +++ python/trunk/Lib/test/test_wait4.py Sun Jun 18 22:10:24 2006 @@ -2,7 +2,6 @@ """ import os -import time from test.fork_wait import ForkWait from test.test_support import TestSkipped, run_unittest @@ -18,13 +17,7 @@ class Wait4Test(ForkWait): def wait_impl(self, cpid): - for i in range(10): - # wait4() shouldn't hang, but some of the buildbots seem to hang - # in the forking tests. This is an attempt to fix the problem. - spid, status, rusage = os.wait4(cpid, os.WNOHANG) - if spid == cpid: - break - time.sleep(1.0) + spid, status, rusage = os.wait4(cpid, 0) self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) self.assertTrue(rusage) From neal at metaslash.com Sun Jun 18 22:13:26 2006 From: neal at metaslash.com (Neal Norwitz) Date: Sun, 18 Jun 2006 16:13:26 -0400 Subject: [Python-checkins] Python Regression Test Failures basics (2) Message-ID: <20060618201326.GA19258@python.psfb.org> case $MAKEFLAGS in \ *-s*) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py -q build;; \ *) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes' ./python -E ./setup.py build;; \ esac running build running build_ext db.h: found (4, 1) in /usr/include db lib: using (4, 1) db-4.1 sqlite: found /usr/include/sqlite3.h /usr/include/sqlite3.h: version 3.2.1 INFO: Can't locate Tcl/Tk libs and/or headers running build_scripts [34100 refs] ./python -E -c 'import sys ; from distutils.util import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform [8686 refs] find ./Lib -name '*.py[co]' -print | xargs rm -f ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test test_wait3 failed -- Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 75, in test_wait self.wait_impl(cpid) File "/home/neal/python/trunk/Lib/test/test_wait3.py", line 29, in wait_impl self.assertEqual(spid, cpid) AssertionError: 18001 != 18597 test_wait4 Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile /home/neal/python/trunk/Lib/struct.py:63: DeprecationWarning: struct integer overflow masking is deprecated return o.pack(*args) test_zipfile64 test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run test_zipimport test_zlib 286 tests OK. 1 test failed: test_wait3 31 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 1 skip unexpected on linux2: test_ioctl [434383 refs] make: [test] Error 1 (ignored) ./python -E -tt ./Lib/test/regrtest.py -l test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bsddb3 skipped -- Use of the `bsddb' resource not enabled test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_curses test_curses skipped -- Use of the `curses' resource not enabled test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_ossaudiodev test_ossaudiodev skipped -- Use of the `audio' resource not enabled test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socket_ssl skipped -- Use of the `network' resource not enabled test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_timeout skipped -- Use of the `network' resource not enabled test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllib2net skipped -- Use of the `network' resource not enabled test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test test_wait3 failed -- Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 75, in test_wait self.wait_impl(cpid) File "/home/neal/python/trunk/Lib/test/test_wait3.py", line 29, in wait_impl self.assertEqual(spid, cpid) AssertionError: 18725 != 19248 test_wait4 Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile /home/neal/python/trunk/Lib/struct.py:63: DeprecationWarning: struct integer overflow masking is deprecated return o.pack(*args) test_zipfile64 test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run test_zipimport test_zlib 286 tests OK. 1 test failed: test_wait3 31 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_curses test_gl test_imgfile test_ioctl test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_startfile test_sunaudiodev test_tcl test_timeout test_unicode_file test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 1 skip unexpected on linux2: test_ioctl [433791 refs] make: *** [test] Error 1 From python-checkins at python.org Sun Jun 18 23:27:04 2006 From: python-checkins at python.org (thomas.heller) Date: Sun, 18 Jun 2006 23:27:04 +0200 (CEST) Subject: [Python-checkins] r47016 - python/trunk/Doc/lib/libctypes.tex Message-ID: <20060618212704.E1E4A1E4002@bag.python.org> Author: thomas.heller Date: Sun Jun 18 23:27:04 2006 New Revision: 47016 Modified: python/trunk/Doc/lib/libctypes.tex Log: Fix typos. Fix doctest example. Mention in the tutorial that 'errcheck' is explained in the ref manual. Use better wording in some places. Remoce code examples that shouldn't be in the tutorial. Remove some XXX notices. Modified: python/trunk/Doc/lib/libctypes.tex ============================================================================== --- python/trunk/Doc/lib/libctypes.tex (original) +++ python/trunk/Doc/lib/libctypes.tex Sun Jun 18 23:27:04 2006 @@ -118,7 +118,7 @@ On Windows, some dlls export functions not by name but by ordinal. These functions can be accessed by indexing the dll object with the -odinal number: +ordinal number: \begin{verbatim} >>> cdll.kernel32[1] # doctest: +WINDOWS <_FuncPtr object at 0x...> @@ -142,8 +142,8 @@ This example calls both functions with a NULL pointer (\code{None} should be used as the NULL pointer): \begin{verbatim} ->>> print libc.time(None) -114... +>>> print libc.time(None) # doctest: +SKIP +1150640792 >>> print hex(windll.kernel32.GetModuleHandleA(None)) # doctest: +WINDOWS 0x1d000000 >>> @@ -571,13 +571,12 @@ >>> \end{verbatim} -XXX Mention the \member{errcheck} protocol... - You can also use a callable Python object (a function or a class for -example) as the \member{restype} attribute. It will be called with the -\code{integer} the C function returns, and the result of this call will -be used as the result of your function call. This is useful to check -for error return values and automatically raise an exception: +example) as the \member{restype} attribute, if the foreign function returns +an integer. The callable will be called with the \code{integer} the C +function returns, and the result of this call will be used as the +result of your function call. This is useful to check for error return +values and automatically raise an exception: \begin{verbatim} >>> GetModuleHandle = windll.kernel32.GetModuleHandleA # doctest: +WINDOWS >>> def ValidHandle(value): @@ -602,6 +601,10 @@ an exception. \code{WinError} takes an optional error code parameter, if no one is used, it calls \function{GetLastError()} to retrieve it. +Please note that a much more powerful error checking mechanism is +available through the \member{errcheck} attribute; see the reference manual +for details. + \subsubsection{Passing pointers (or: passing parameters by reference)\label{ctypes-passing-pointers}} @@ -876,22 +879,18 @@ \subsubsection{Type conversions\label{ctypes-type-conversions}} Usually, ctypes does strict type checking. This means, if you have -\code{POINTER(c{\_}int)} in the \member{argtypes} list of a function or in the -\member{{\_}fields{\_}} of a structure definition, only instances of exactly the -same type are accepted. There are some exceptions to this rule, where -ctypes accepts other objects. For example, you can pass compatible -array instances instead of pointer types. So, for \code{POINTER(c{\_}int)}, -ctypes accepts an array of c{\_}int values: +\code{POINTER(c{\_}int)} in the \member{argtypes} list of a function or as the +type of a member field in a structure definition, only instances of +exactly the same type are accepted. There are some exceptions to this +rule, where ctypes accepts other objects. For example, you can pass +compatible array instances instead of pointer types. So, for +\code{POINTER(c{\_}int)}, ctypes accepts an array of c{\_}int: \begin{verbatim} >>> class Bar(Structure): ... _fields_ = [("count", c_int), ("values", POINTER(c_int))] ... >>> bar = Bar() ->>> print bar._objects -None >>> bar.values = (c_int * 3)(1, 2, 3) ->>> print bar._objects -{'1': ({}, )} >>> bar.count = 3 >>> for i in range(bar.count): ... print bar.values[i] @@ -912,9 +911,9 @@ Sometimes you have instances of incompatible types. In \code{C}, you can cast one type into another type. \code{ctypes} provides a \code{cast} -function which can be used in the same way. The Bar structure defined -above accepts \code{POINTER(c{\_}int)} pointers or \class{c{\_}int} arrays for its -\code{values} field, but not instances of other types: +function which can be used in the same way. The \code{Bar} structure +defined above accepts \code{POINTER(c{\_}int)} pointers or \class{c{\_}int} arrays +for its \code{values} field, but not instances of other types: \begin{verbatim} >>> bar.values = (c_byte * 4)() Traceback (most recent call last): @@ -1161,7 +1160,10 @@ >>> \end{verbatim} -So, our array sorted now: +It is quite interesting to see that the Windows \function{qsort} function +needs more comparisons than the linux version! + +As we can easily check, our array sorted now: \begin{verbatim} >>> for i in ia: print i, ... @@ -1172,14 +1174,14 @@ \textbf{Important note for callback functions:} Make sure you keep references to CFUNCTYPE objects as long as they are -used from C code. ctypes doesn't, and if you don't, they may be +used from C code. \code{ctypes} doesn't, and if you don't, they may be garbage collected, crashing your program when a callback is made. \subsubsection{Accessing values exported from dlls\label{ctypes-accessing-values-exported-from-dlls}} Sometimes, a dll not only exports functions, it also exports -values. An example in the Python library itself is the +variables. An example in the Python library itself is the \code{Py{\_}OptimizeFlag}, an integer set to 0, 1, or 2, depending on the \programopt{-O} or \programopt{-OO} flag given on startup. @@ -1250,9 +1252,6 @@ (indicated by the negative size member) is not wellknown, it is only used for testing. Try it out with \code{import {\_}{\_}hello{\_}{\_}} for example. -XXX Describe how to access the \var{code} member fields, which contain -the byte code for the modules. - \subsubsection{Surprises\label{ctypes-surprises}} From neal at metaslash.com Sun Jun 18 23:33:56 2006 From: neal at metaslash.com (Neal Norwitz) Date: Sun, 18 Jun 2006 17:33:56 -0400 Subject: [Python-checkins] Python Regression Test Failures all (1) Message-ID: <20060618213356.GA15209@python.psfb.org> test_grammar test_opcodes test_operations test_builtin test_exceptions test_types test_MimeWriter test_StringIO test___all__ test___future__ test__locale test_aepack test_aepack skipped -- No module named aepack test_al test_al skipped -- No module named al test_anydbm test_applesingle test_applesingle skipped -- No module named macostools test_array test_ast test_asynchat test_atexit test_audioop test_augassign test_base64 test_bastion test_bigmem test_binascii test_binhex test_binop test_bisect test_bool test_bsddb test_bsddb185 test_bsddb185 skipped -- No module named bsddb185 test_bsddb3 test_bufio test_bz2 test_cProfile test_calendar test_call test_capi test_cd test_cd skipped -- No module named cd test_cfgparser test_cgi test_charmapcodec test_cl test_cl skipped -- No module named cl test_class test_cmath test_cmd_line test_code test_codeccallbacks test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_codecs test_codeop test_coding test_coercion test_colorsys test_commands test_compare test_compile test_compiler test_complex test_contains test_contextlib test_cookie test_cookielib test_copy test_copy_reg test_cpickle test_crypt test_csv test_ctypes test_datetime test_dbm test_decimal test_decorators test_defaultdict test_deque test_descr test_descrtut test_dict test_difflib test_dircache test_dis test_distutils test_dl test_doctest test_doctest2 test_dumbdbm test_dummy_thread test_dummy_threading test_email test_email_codecs test_email_renamed test_enumerate test_eof test_errno test_exception_variations test_extcall test_fcntl test_file test_filecmp test_fileinput test_float test_fnmatch test_fork1 test_format test_fpformat test_frozen test_funcattrs test_functools test_future test_gc test_gdbm test_generators test_genexps test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global test_grp test_gzip test_hash test_hashlib test_heapq test_hexoct test_hmac test_hotshot test_htmllib test_htmlparser test_httplib test_imageop test_imaplib test_imgfile test_imgfile skipped -- No module named imgfile test_imp test_import test_importhooks test_index test_inspect test_ioctl test_ioctl skipped -- Unable to open /dev/tty test_isinstance test_iter test_iterlen test_itertools test_largefile test_list test_locale test_logging test_long test_long_future test_longexp test_macfs test_macfs skipped -- No module named macfs test_macostools test_macostools skipped -- No module named macostools test_macpath test_mailbox test_marshal test_math test_md5 test_mhlib test_mimetools test_mimetypes test_minidom test_mmap test_module test_multibytecodec test_multibytecodec_support test_multifile test_mutants test_netrc test_new test_nis test_nis skipped -- Local domain name not set test_normalization test_ntpath test_old_mailbox test_openpty test_operator test_optparse test_os test_parser test_peepholer test_pep247 test_pep263 test_pep277 test_pep277 skipped -- test works only on NT+ test_pep292 test_pep352 test_pickle test_pickletools test_pkg test_pkgimport test_platform test_plistlib test_plistlib skipped -- No module named plistlib test_poll test_popen [8691 refs] [8691 refs] [8691 refs] test_popen2 test_posix test_posixpath test_pow test_pprint test_profile test_profilehooks test_pty test_pwd test_pyclbr test_pyexpat test_queue test_quopri [9039 refs] [9039 refs] test_random test_re test_repr test_resource test_rfc822 test_rgbimg test_richcmp test_robotparser test_runpy test_sax test_scope test_scriptpackages test_scriptpackages skipped -- No module named aetools test_select test_set test_sets test_sgmllib test_sha test_shelve test_shlex test_shutil test_signal test_site test_slice test_socket test_socket_ssl test_socketserver test_softspace test_sort test_sqlite test_startfile test_startfile skipped -- cannot import name startfile test_str test_strftime test_string test_stringprep test_strop test_strptime test_struct test_structseq test_subprocess [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8902 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] [8686 refs] this bit of output is from a test of stdout in a different process ... [8686 refs] [8686 refs] [8902 refs] test_sunaudiodev test_sunaudiodev skipped -- No module named sunaudiodev test_sundry test_symtable test_syntax test_sys [8686 refs] [8686 refs] test_tarfile test_tcl test_tcl skipped -- No module named _tkinter test_tempfile [8686 refs] test_textwrap test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeout test_tokenize test_trace test_traceback test_transformer test_tuple test_ucn test_unary test_unicode test_unicode_file test_unicode_file skipped -- No Unicode filesystem semantics on this platform. test_unicodedata test_unittest test_univnewlines test_unpack test_urllib test_urllib2 test_urllib2net test_urllibnet test_urlparse test_userdict test_userlist test_userstring test_uu test_uuid test_wait3 test test_wait3 failed -- Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 75, in test_wait self.wait_impl(cpid) File "/home/neal/python/trunk/Lib/test/test_wait3.py", line 29, in wait_impl self.assertEqual(spid, cpid) AssertionError: 4595 != 15039 test_wait4 Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' Unhandled exception in thread started by > Traceback (most recent call last): File "/home/neal/python/trunk/Lib/test/fork_wait.py", line 30, in f self.alive[id] = os.getpid() AttributeError: 'NoneType' object has no attribute 'getpid' test_warnings test_wave test_weakref test_whichdb test_winreg test_winreg skipped -- No module named _winreg test_winsound test_winsound skipped -- No module named winsound test_with test_wsgiref test_xdrlib test_xml_etree test_xml_etree_c test_xmllib test_xmlrpc test_xpickle test_xrange test_zipfile /home/neal/python/trunk/Lib/struct.py:63: DeprecationWarning: struct integer overflow masking is deprecated return o.pack(*args) test_zipfile64 test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run test_zipimport test_zlib 292 tests OK. 1 test failed: test_wait3 22 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl test_gl test_imgfile test_ioctl test_macfs test_macostools test_nis test_pep277 test_plistlib test_scriptpackages test_startfile test_sunaudiodev test_tcl test_unicode_file test_winreg test_winsound test_zipfile64 1 skip unexpected on linux2: test_ioctl [441791 refs] From python-checkins at python.org Mon Jun 19 00:17:30 2006 From: python-checkins at python.org (georg.brandl) Date: Mon, 19 Jun 2006 00:17:30 +0200 (CEST) Subject: [Python-checkins] r47017 - in python/trunk: Lib/test/test_genexps.py Objects/abstract.c Objects/object.c Objects/typeobject.c Message-ID: <20060618221730.7A7621E4002@bag.python.org> Author: georg.brandl Date: Mon Jun 19 00:17:29 2006 New Revision: 47017 Modified: python/trunk/Lib/test/test_genexps.py python/trunk/Objects/abstract.c python/trunk/Objects/object.c python/trunk/Objects/typeobject.c Log: Patch #1507676: improve exception messages in abstract.c, object.c and typeobject.c. Modified: python/trunk/Lib/test/test_genexps.py ============================================================================== --- python/trunk/Lib/test/test_genexps.py (original) +++ python/trunk/Lib/test/test_genexps.py Mon Jun 19 00:17:29 2006 @@ -109,7 +109,7 @@ Traceback (most recent call last): File "", line 1, in -toplevel- (i for i in 6) - TypeError: iteration over non-sequence + TypeError: 'int' object is not iterable Verify late binding for the outermost if-expression Modified: python/trunk/Objects/abstract.c ============================================================================== --- python/trunk/Objects/abstract.c (original) +++ python/trunk/Objects/abstract.c Mon Jun 19 00:17:29 2006 @@ -14,9 +14,9 @@ /* Shorthands to return certain errors */ static PyObject * -type_error(const char *msg) +type_error(const char *msg, PyObject *obj) { - PyErr_SetString(PyExc_TypeError, msg); + PyErr_Format(PyExc_TypeError, msg, obj->ob_type->tp_name); return NULL; } @@ -130,10 +130,11 @@ return PySequence_GetItem(o, key_value); } else if (o->ob_type->tp_as_sequence->sq_item) - return type_error("sequence index must be integer"); + return type_error("sequence index must " + "be integer, not '%.200s'", key); } - return type_error("unsubscriptable object"); + return type_error("'%.200s' object is unsubscriptable", o); } int @@ -158,12 +159,13 @@ return PySequence_SetItem(o, key_value, value); } else if (o->ob_type->tp_as_sequence->sq_ass_item) { - type_error("sequence index must be integer"); + type_error("sequence index must be " + "integer, not '%.200s'", key); return -1; } } - type_error("object does not support item assignment"); + type_error("'%.200s' object does not support item assignment", o); return -1; } @@ -189,12 +191,13 @@ return PySequence_DelItem(o, key_value); } else if (o->ob_type->tp_as_sequence->sq_ass_item) { - type_error("sequence index must be integer"); + type_error("sequence index must be " + "integer, not '%.200s'", key); return -1; } } - type_error("object does not support item deletion"); + type_error("'%.200s' object does not support item deletion", o); return -1; } @@ -435,7 +438,8 @@ binop_type_error(PyObject *v, PyObject *w, const char *op_name) { PyErr_Format(PyExc_TypeError, - "unsupported operand type(s) for %s: '%s' and '%s'", + "unsupported operand type(s) for %.100s: " + "'%.100s' and '%.100s'", op_name, v->ob_type->tp_name, w->ob_type->tp_name); @@ -601,14 +605,14 @@ PyErr_Format( PyExc_TypeError, "unsupported operand type(s) for ** or pow(): " - "'%s' and '%s'", + "'%.100s' and '%.100s'", v->ob_type->tp_name, w->ob_type->tp_name); else PyErr_Format( PyExc_TypeError, "unsupported operand type(s) for pow(): " - "'%s', '%s', '%s'", + "'%.100s', '%.100s', '%.100s'", v->ob_type->tp_name, w->ob_type->tp_name, z->ob_type->tp_name); @@ -656,8 +660,8 @@ return NULL; } else { - return type_error( - "can't multiply sequence by non-int"); + return type_error("can't multiply sequence by " + "non-int of type '%.200s'", n); } return (*repeatfunc)(seq, count); } @@ -870,7 +874,7 @@ if (m && m->nb_negative) return (*m->nb_negative)(o); - return type_error("bad operand type for unary -"); + return type_error("bad operand type for unary -: '%.200s'", o); } PyObject * @@ -884,7 +888,7 @@ if (m && m->nb_positive) return (*m->nb_positive)(o); - return type_error("bad operand type for unary +"); + return type_error("bad operand type for unary +: '%.200s'", o); } PyObject * @@ -898,7 +902,7 @@ if (m && m->nb_invert) return (*m->nb_invert)(o); - return type_error("bad operand type for unary ~"); + return type_error("bad operand type for unary ~: '%.200s'", o); } PyObject * @@ -912,7 +916,7 @@ if (m && m->nb_absolute) return m->nb_absolute(o); - return type_error("bad operand type for abs()"); + return type_error("bad operand type for abs(): '%.200s'", o); } /* Add a check for embedded NULL-bytes in the argument. */ @@ -992,7 +996,8 @@ if (!PyObject_AsCharBuffer(o, &buffer, &buffer_len)) return int_from_string((char*)buffer, buffer_len); - return type_error("int() argument must be a string or a number"); + return type_error("int() argument must be a string or a " + "number, not '%.200s'", o); } /* Add a check for embedded NULL-bytes in the argument. */ @@ -1054,7 +1059,8 @@ if (!PyObject_AsCharBuffer(o, &buffer, &buffer_len)) return long_from_string(buffer, buffer_len); - return type_error("long() argument must be a string or a number"); + return type_error("long() argument must be a string or a " + "number, not '%.200s'", o); } PyObject * @@ -1108,7 +1114,7 @@ if (m && m->sq_length) return m->sq_length(s); - type_error("len() of unsized object"); + type_error("non-sequence object of type '%.200s' has no len()", s); return -1; } @@ -1141,7 +1147,7 @@ return result; Py_DECREF(result); } - return type_error("object can't be concatenated"); + return type_error("'%.200s' object can't be concatenated", s); } PyObject * @@ -1170,7 +1176,7 @@ return result; Py_DECREF(result); } - return type_error("object can't be repeated"); + return type_error("'%.200s' object can't be repeated", o); } PyObject * @@ -1194,7 +1200,7 @@ return result; Py_DECREF(result); } - return type_error("object can't be concatenated"); + return type_error("'%.200s' object can't be concatenated", s); } PyObject * @@ -1223,7 +1229,7 @@ return result; Py_DECREF(result); } - return type_error("object can't be repeated"); + return type_error("'%.200s' object can't be repeated", o); } PyObject * @@ -1247,7 +1253,7 @@ return m->sq_item(s, i); } - return type_error("unindexable object"); + return type_error("'%.200s' object is unindexable", s); } PyObject * @@ -1282,7 +1288,7 @@ return res; } - return type_error("unsliceable object"); + return type_error("'%.200s' object is unsliceable", s); } int @@ -1308,7 +1314,7 @@ return m->sq_ass_item(s, i, o); } - type_error("object does not support item assignment"); + type_error("'%.200s' object does not support item assignment", s); return -1; } @@ -1335,7 +1341,7 @@ return m->sq_ass_item(s, i, (PyObject *)NULL); } - type_error("object doesn't support item deletion"); + type_error("'%.200s' object doesn't support item deletion", s); return -1; } @@ -1374,7 +1380,7 @@ return res; } - type_error("object doesn't support slice assignment"); + type_error("'%.200s' object doesn't support slice assignment", s); return -1; } @@ -1403,7 +1409,7 @@ } return m->sq_ass_slice(s, i1, i2, (PyObject *)NULL); } - type_error("object doesn't support slice deletion"); + type_error("'%.200s' object doesn't support slice deletion", s); return -1; } @@ -1534,7 +1540,7 @@ it = PyObject_GetIter(v); if (it == NULL) { if (PyErr_ExceptionMatches(PyExc_TypeError)) - return type_error(m); + PyErr_SetString(PyExc_TypeError, m); return NULL; } @@ -1564,7 +1570,7 @@ it = PyObject_GetIter(seq); if (it == NULL) { - type_error("iterable argument required"); + type_error("argument of type '%.200s' is not iterable", seq); return -1; } @@ -1699,7 +1705,7 @@ if (m && m->mp_length) return m->mp_length(o); - type_error("len() of unsized object"); + type_error("non-mapping object of type '%.200s' has no len()", o); return -1; } @@ -1807,7 +1813,7 @@ "NULL result without error in PyObject_Call"); return result; } - PyErr_Format(PyExc_TypeError, "'%s' object is not callable", + PyErr_Format(PyExc_TypeError, "'%.200s' object is not callable", func->ob_type->tp_name); return NULL; } @@ -1896,7 +1902,7 @@ } if (!PyCallable_Check(func)) { - type_error("call of non-callable attribute"); + type_error("attribute of type '%.200s' is not callable", func); goto exit; } @@ -1935,7 +1941,7 @@ } if (!PyCallable_Check(func)) { - type_error("call of non-callable attribute"); + type_error("attribute of type '%.200s' is not callable", func); goto exit; } @@ -2287,9 +2293,7 @@ if (f == NULL) { if (PySequence_Check(o)) return PySeqIter_New(o); - PyErr_SetString(PyExc_TypeError, - "iteration over non-sequence"); - return NULL; + return type_error("'%.200s' object is not iterable", o); } else { PyObject *res = (*f)(o); Modified: python/trunk/Objects/object.c ============================================================================== --- python/trunk/Objects/object.c (original) +++ python/trunk/Objects/object.c Mon Jun 19 00:17:29 2006 @@ -1068,7 +1068,8 @@ return _Py_HashPointer(v); /* Use address as hash value */ } /* If there's a cmp but no hash defined, the object can't be hashed */ - PyErr_SetString(PyExc_TypeError, "unhashable type"); + PyErr_Format(PyExc_TypeError, "unhashable type: '%.200s'", + v->ob_type->tp_name); return -1; } @@ -1133,8 +1134,9 @@ else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return NULL; } } @@ -1179,8 +1181,9 @@ else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return -1; } } @@ -1277,8 +1280,9 @@ else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return NULL; } } @@ -1399,8 +1403,9 @@ else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return -1; } } @@ -1450,7 +1455,7 @@ if (descr == NULL) { PyErr_Format(PyExc_AttributeError, - "'%.50s' object has no attribute '%.400s'", + "'%.100s' object has no attribute '%.200s'", tp->tp_name, PyString_AS_STRING(name)); goto done; } @@ -1773,8 +1778,9 @@ assert(result); if (!PyList_Check(result)) { - PyErr_SetString(PyExc_TypeError, - "Expected keys() to be a list."); + PyErr_Format(PyExc_TypeError, + "Expected keys() to be a list, not '%.200s'", + result->ob_type->tp_name); goto error; } if (PyList_Sort(result) != 0) Modified: python/trunk/Objects/typeobject.c ============================================================================== --- python/trunk/Objects/typeobject.c (original) +++ python/trunk/Objects/typeobject.c Mon Jun 19 00:17:29 2006 @@ -1470,8 +1470,9 @@ return -1; } if (value != NULL && !PyDict_Check(value)) { - PyErr_SetString(PyExc_TypeError, - "__dict__ must be set to a dictionary"); + PyErr_Format(PyExc_TypeError, + "__dict__ must be set to a dictionary, " + "not a '%.200s'", value->ob_type->tp_name); return -1; } dict = *dictptr; @@ -1534,8 +1535,9 @@ Py_ssize_t i, n; if (!PyString_Check(s)) { - PyErr_SetString(PyExc_TypeError, - "__slots__ must be strings"); + PyErr_Format(PyExc_TypeError, + "__slots__ items must be strings, not '%.200s'", + s->ob_type->tp_name); return 0; } p = (unsigned char *) PyString_AS_STRING(s); @@ -2575,8 +2577,9 @@ args = PyObject_CallObject(getnewargs, NULL); Py_DECREF(getnewargs); if (args != NULL && !PyTuple_Check(args)) { - PyErr_SetString(PyExc_TypeError, - "__getnewargs__ should return a tuple"); + PyErr_Format(PyExc_TypeError, + "__getnewargs__ should return a tuple, " + "not '%.200s'", args->ob_type->tp_name); goto end; } } @@ -4352,8 +4355,9 @@ result = temp->ob_type->tp_as_number->nb_index(temp); } else { - PyErr_SetString(PyExc_TypeError, - "__index__ must return an int or a long"); + PyErr_Format(PyExc_TypeError, + "__index__ must return an int or a long, " + "not '%.200s'", temp->ob_type->tp_name); result = -1; } Py_DECREF(temp); @@ -4564,8 +4568,9 @@ func = lookup_method(self, "__cmp__", &cmp_str); } if (func != NULL) { + PyErr_Format(PyExc_TypeError, "unhashable type: '%.200s'", + self->ob_type->tp_name); Py_DECREF(func); - PyErr_SetString(PyExc_TypeError, "unhashable type"); return -1; } PyErr_Clear(); @@ -4742,8 +4747,9 @@ PyErr_Clear(); func = lookup_method(self, "__getitem__", &getitem_str); if (func == NULL) { - PyErr_SetString(PyExc_TypeError, - "iteration over non-sequence"); + PyErr_Format(PyExc_TypeError, + "'%.200s' object is not iterable", + self->ob_type->tp_name); return NULL; } Py_DECREF(func); @@ -4816,8 +4822,9 @@ if (res == NULL) return -1; if (res != Py_None) { - PyErr_SetString(PyExc_TypeError, - "__init__() should return None"); + PyErr_Format(PyExc_TypeError, + "__init__() should return None, not '%.200s'", + res->ob_type->tp_name); Py_DECREF(res); return -1; } From buildbot at python.org Mon Jun 19 00:31:29 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 18 Jun 2006 22:31:29 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060618223129.8CAC01E4005@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/426 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 19 02:31:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 19 Jun 2006 00:31:02 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060619003102.388271E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/384 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl,thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From ncoghlan at gmail.com Mon Jun 19 06:41:51 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 19 Jun 2006 14:41:51 +1000 Subject: [Python-checkins] r47017 - in python/trunk: Lib/test/test_genexps.py Objects/abstract.c Objects/object.c Objects/typeobject.c In-Reply-To: <20060618221730.7A7621E4002@bag.python.org> References: <20060618221730.7A7621E4002@bag.python.org> Message-ID: <44962B0F.2050700@gmail.com> georg.brandl wrote: > Author: georg.brandl > Date: Mon Jun 19 00:17:29 2006 > New Revision: 47017 > > Modified: > python/trunk/Lib/test/test_genexps.py > python/trunk/Objects/abstract.c > python/trunk/Objects/object.c > python/trunk/Objects/typeobject.c > Log: > Patch #1507676: improve exception messages in abstract.c, object.c and typeobject.c. Very nice! (it probably deserves a NEWS entry too) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python-checkins at python.org Mon Jun 19 07:40:46 2006 From: python-checkins at python.org (neal.norwitz) Date: Mon, 19 Jun 2006 07:40:46 +0200 (CEST) Subject: [Python-checkins] r47018 - in python/trunk: Include/setobject.h Objects/setobject.c Message-ID: <20060619054046.01E0D1E4002@bag.python.org> Author: neal.norwitz Date: Mon Jun 19 07:40:44 2006 New Revision: 47018 Modified: python/trunk/Include/setobject.h python/trunk/Objects/setobject.c Log: Use Py_ssize_t Modified: python/trunk/Include/setobject.h ============================================================================== --- python/trunk/Include/setobject.h (original) +++ python/trunk/Include/setobject.h Mon Jun 19 07:40:44 2006 @@ -35,14 +35,14 @@ struct _setobject { PyObject_HEAD - int fill; /* # Active + # Dummy */ - int used; /* # Active */ + Py_ssize_t fill; /* # Active + # Dummy */ + Py_ssize_t used; /* # Active */ /* The table contains mask + 1 slots, and that's a power of 2. * We store the mask instead of the size because the mask is more * frequently needed. */ - int mask; + Py_ssize_t mask; /* table points to smalltable for small tables, else to * additional malloc'ed memory. table is never NULL! This rule Modified: python/trunk/Objects/setobject.c ============================================================================== --- python/trunk/Objects/setobject.c (original) +++ python/trunk/Objects/setobject.c Mon Jun 19 07:40:44 2006 @@ -62,7 +62,7 @@ register Py_ssize_t i; register size_t perturb; register setentry *freeslot; - register unsigned int mask = so->mask; + register size_t mask = so->mask; setentry *table = so->table; register setentry *entry; register int cmp; @@ -140,7 +140,7 @@ register Py_ssize_t i; register size_t perturb; register setentry *freeslot; - register unsigned int mask = so->mask; + register size_t mask = so->mask; setentry *table = so->table; register setentry *entry; @@ -221,11 +221,11 @@ actually be smaller than the old one. */ static int -set_table_resize(PySetObject *so, int minused) +set_table_resize(PySetObject *so, Py_ssize_t minused) { - int newsize; + Py_ssize_t newsize; setentry *oldtable, *newtable, *entry; - int i; + Py_ssize_t i; int is_oldtable_malloced; setentry small_copy[PySet_MINSIZE]; @@ -314,7 +314,7 @@ static int set_add_entry(register PySetObject *so, setentry *entry) { - register int n_used; + register Py_ssize_t n_used; assert(so->fill <= so->mask); /* at least one empty slot */ n_used = so->used; @@ -330,7 +330,7 @@ set_add_key(register PySetObject *so, PyObject *key) { register long hash; - register int n_used; + register Py_ssize_t n_used; if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { @@ -403,10 +403,10 @@ { setentry *entry, *table; int table_is_malloced; - int fill; + Py_ssize_t fill; setentry small_copy[PySet_MINSIZE]; #ifdef Py_DEBUG - int i, n; + Py_ssize_t i, n; assert (PyAnySet_Check(so)); n = so->mask + 1; @@ -465,7 +465,7 @@ /* * Iterate over a set table. Use like so: * - * int pos; + * Py_ssize_t pos; * setentry *entry; * pos = 0; # important! pos should not otherwise be changed by you * while (set_next(yourset, &pos, &entry)) { @@ -479,7 +479,7 @@ set_next(PySetObject *so, Py_ssize_t *pos_ptr, setentry **entry_ptr) { Py_ssize_t i; - int mask; + Py_ssize_t mask; register setentry *table; assert (PyAnySet_Check(so)); @@ -501,7 +501,7 @@ set_dealloc(PySetObject *so) { register setentry *entry; - int fill = so->fill; + Py_ssize_t fill = so->fill; PyObject_GC_UnTrack(so); Py_TRASHCAN_SAFE_BEGIN(so) if (so->weakreflist != NULL) @@ -570,7 +570,7 @@ set_merge(PySetObject *so, PyObject *otherset) { PySetObject *other; - register int i; + register Py_ssize_t i; register setentry *entry; assert (PyAnySet_Check(so)); @@ -637,7 +637,7 @@ static PyObject * set_pop(PySetObject *so) { - register int i = 0; + register Py_ssize_t i = 0; register setentry *entry; PyObject *key; @@ -655,7 +655,7 @@ */ entry = &so->table[0]; if (entry->key == NULL || entry->key == dummy) { - i = (int)entry->hash; + i = entry->hash; /* The hash field may be a real hash value, or it may be a * legit search finger, or it may be a once-legit search * finger that's out of bounds now because it wrapped around @@ -730,9 +730,9 @@ typedef struct { PyObject_HEAD PySetObject *si_set; /* Set to NULL when iterator is exhausted */ - int si_used; - int si_pos; - long len; + Py_ssize_t si_used; + Py_ssize_t si_pos; + Py_ssize_t len; } setiterobject; static void @@ -745,7 +745,7 @@ static PyObject * setiter_len(setiterobject *si) { - long len = 0; + Py_ssize_t len = 0; if (si->si_set != NULL && si->si_used == si->si_set->used) len = si->len; return PyInt_FromLong(len); @@ -761,7 +761,7 @@ static PyObject *setiter_iternext(setiterobject *si) { PyObject *key; - register int i, mask; + register Py_ssize_t i, mask; register setentry *entry; PySetObject *so = si->si_set; @@ -1007,7 +1007,7 @@ static void set_swap_bodies(PySetObject *a, PySetObject *b) { - int t; + Py_ssize_t t; setentry *u; setentry *(*f)(PySetObject *so, PyObject *key, long hash); setentry tab[PySet_MINSIZE]; @@ -2064,7 +2064,7 @@ static PyObject * test_c_api(PySetObject *so) { - int count; + Py_ssize_t count; char *s; Py_ssize_t i; PyObject *elem, *dup, *t, *f, *dup2; From python-checkins at python.org Mon Jun 19 08:35:55 2006 From: python-checkins at python.org (georg.brandl) Date: Mon, 19 Jun 2006 08:35:55 +0200 (CEST) Subject: [Python-checkins] r47019 - python/trunk/Misc/NEWS Message-ID: <20060619063555.5E3B31E4002@bag.python.org> Author: georg.brandl Date: Mon Jun 19 08:35:54 2006 New Revision: 47019 Modified: python/trunk/Misc/NEWS Log: Add news entry about error msg improvement. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Mon Jun 19 08:35:54 2006 @@ -12,6 +12,10 @@ Core and builtins ----------------- +- Patch #1507676: Error messages returned by invalid abstract object operations + (such as iterating over an integer) have been improved and now include the + type of the offending object to help with debugging. + - Bug #992017: A classic class that defined a __coerce__() method that returned its arguments swapped would infinitely recurse and segfault the interpreter. From python-checkins at python.org Mon Jun 19 09:07:49 2006 From: python-checkins at python.org (thomas.heller) Date: Mon, 19 Jun 2006 09:07:49 +0200 (CEST) Subject: [Python-checkins] r47020 - python/trunk/Lib/ctypes/test/test_loading.py Message-ID: <20060619070749.B93B81E4002@bag.python.org> Author: thomas.heller Date: Mon Jun 19 09:07:49 2006 New Revision: 47020 Modified: python/trunk/Lib/ctypes/test/test_loading.py Log: Try to repair the failing test on the OpenBSD buildbot. Trial and error... Modified: python/trunk/Lib/ctypes/test/test_loading.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_loading.py (original) +++ python/trunk/Lib/ctypes/test/test_loading.py Mon Jun 19 09:07:49 2006 @@ -16,7 +16,7 @@ else: for line in os.popen("ldd %s" % sys.executable): if "libc.so" in line: - if sys.platform == "openbsd3": + if sys.platform.startswith("openbsd3"): libc_name = line.split()[4] else: libc_name = line.split()[2] From python-checkins at python.org Mon Jun 19 09:45:17 2006 From: python-checkins at python.org (tim.peters) Date: Mon, 19 Jun 2006 09:45:17 +0200 (CEST) Subject: [Python-checkins] r47021 - in python/trunk/Lib: lib-tk/Tkinter.py test/test_support.py Message-ID: <20060619074517.410F91E4002@bag.python.org> Author: tim.peters Date: Mon Jun 19 09:45:16 2006 New Revision: 47021 Modified: python/trunk/Lib/lib-tk/Tkinter.py python/trunk/Lib/test/test_support.py Log: Whitespace normalization. Modified: python/trunk/Lib/lib-tk/Tkinter.py ============================================================================== --- python/trunk/Lib/lib-tk/Tkinter.py (original) +++ python/trunk/Lib/lib-tk/Tkinter.py Mon Jun 19 09:45:16 2006 @@ -170,11 +170,11 @@ _default = "" def __init__(self, master=None, value=None, name=None): """Construct a variable - + MASTER can be given as master widget. VALUE is an optional value (defaults to "") NAME is an optional Tcl name (defaults to PY_VARnum). - + If NAME matches an existing variable and VALUE is omitted then the existing value is retained. """ @@ -231,7 +231,7 @@ self._tk.call("trace", "vinfo", self._name))) def __eq__(self, other): """Comparison for equality (==). - + Note: if the Variable's master matters to behavior also compare self._master == other._master """ @@ -247,7 +247,7 @@ MASTER can be given as master widget. VALUE is an optional value (defaults to "") NAME is an optional Tcl name (defaults to PY_VARnum). - + If NAME matches an existing variable and VALUE is omitted then the existing value is retained. """ @@ -269,7 +269,7 @@ MASTER can be given as master widget. VALUE is an optional value (defaults to 0) NAME is an optional Tcl name (defaults to PY_VARnum). - + If NAME matches an existing variable and VALUE is omitted then the existing value is retained. """ @@ -294,7 +294,7 @@ MASTER can be given as master widget. VALUE is an optional value (defaults to 0.0) NAME is an optional Tcl name (defaults to PY_VARnum). - + If NAME matches an existing variable and VALUE is omitted then the existing value is retained. """ @@ -313,7 +313,7 @@ MASTER can be given as master widget. VALUE is an optional value (defaults to False) NAME is an optional Tcl name (defaults to PY_VARnum). - + If NAME matches an existing variable and VALUE is omitted then the existing value is retained. """ @@ -1504,7 +1504,7 @@ """Set bitmap for the iconified widget to BITMAP. Return the bitmap if None is given. - Under Windows, the DEFAULT parameter can be used to set the icon + Under Windows, the DEFAULT parameter can be used to set the icon for the widget and any descendents that don't have an icon set explicitly. DEFAULT can be the relative path to a .ico file (example: root.iconbitmap(default='myicon.ico') ). See Tk Modified: python/trunk/Lib/test/test_support.py ============================================================================== --- python/trunk/Lib/test/test_support.py (original) +++ python/trunk/Lib/test/test_support.py Mon Jun 19 09:45:16 2006 @@ -475,4 +475,3 @@ while len(threading._limbo) != num_limbo and count < _MAX_COUNT: count += 1 time.sleep(0.1) - From anthony at interlink.com.au Mon Jun 19 10:04:09 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 19 Jun 2006 18:04:09 +1000 Subject: [Python-checkins] TRUNK FREEZE IMMINENT FOR 2.5 BETA 1 - 00:00 UTC, 20-JUNE-2006 Message-ID: <200606191804.13248.anthony@interlink.com.au> The trunk will be FROZEN for 2.5b1 from 00:00UTC on Tuesday, 20th of June. That's about 16 hours from now. Please don't checkin while the trunk is frozen, unless you're one of the release team (me, Martin, Fred, Ronald). I'll send another note once we're done with the release. Please note that once this release is done, the trunk is in FEATURE FREEZE. No new features should be checked in without prior approval - checkins that violate this will quite probably get backed out. Once the beta is out, I expect that we'll get quite a bit more anal about any checkins that break the buildbots. Please, please make sure you run the test suite before checking in - and if you're at all concerned that your checkin might have strange platform dependencies, check the buildbot status page (http://www.python.org/dev/buildbot/trunk/) after your checkin to make sure it didn't break anything. The plan at the moment is to branch the trunk for release25-maint when the first release candidate for 2.5 final is cut. This is currently scheduled for August 1st - about 6 weeks away. Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From python-checkins at python.org Mon Jun 19 10:07:53 2006 From: python-checkins at python.org (walter.doerwald) Date: Mon, 19 Jun 2006 10:07:53 +0200 (CEST) Subject: [Python-checkins] r47022 - in python/trunk: Lib/test/test_curses.py Misc/NEWS Modules/_cursesmodule.c configure configure.in pyconfig.h.in Message-ID: <20060619080753.31C271E4002@bag.python.org> Author: walter.doerwald Date: Mon Jun 19 10:07:50 2006 New Revision: 47022 Modified: python/trunk/Lib/test/test_curses.py python/trunk/Misc/NEWS python/trunk/Modules/_cursesmodule.c python/trunk/configure python/trunk/configure.in python/trunk/pyconfig.h.in Log: Patch #1506645: add Python wrappers for the curses functions is_term_resized, resize_term and resizeterm. This uses three separate configure checks (one for each function). Modified: python/trunk/Lib/test/test_curses.py ============================================================================== --- python/trunk/Lib/test/test_curses.py (original) +++ python/trunk/Lib/test/test_curses.py Mon Jun 19 10:07:50 2006 @@ -212,6 +212,13 @@ m = curses.getmouse() curses.ungetmouse(*m) + if hasattr(curses, 'is_term_resized'): + curses.is_term_resized(*stdscr.getmaxyx()) + if hasattr(curses, 'resizeterm'): + curses.resizeterm(*stdscr.getmaxyx()) + if hasattr(curses, 'resize_term'): + curses.resize_term(*stdscr.getmaxyx()) + def unit_tests(): from curses import ascii for ch, expected in [('a', 'a'), ('A', 'A'), Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Mon Jun 19 10:07:50 2006 @@ -163,6 +163,8 @@ - Patch #1446489: add support for the ZIP64 extensions to zipfile. +- Patch #1506645: add Python wrappers for the curses functions + is_term_resized, resize_term and resizeterm. Library ------- Modified: python/trunk/Modules/_cursesmodule.c ============================================================================== --- python/trunk/Modules/_cursesmodule.c (original) +++ python/trunk/Modules/_cursesmodule.c Mon Jun 19 10:07:50 2006 @@ -44,7 +44,7 @@ mcprint mvaddchnstr mvaddchstr mvchgat mvcur mvinchnstr mvinchstr mvinnstr mmvwaddchnstr mvwaddchstr mvwchgat mvwgetnstr mvwinchnstr mvwinchstr mvwinnstr newterm - resizeterm restartterm ripoffline scr_dump + restartterm ripoffline scr_dump scr_init scr_restore scr_set scrl set_curterm set_term setterm tgetent tgetflag tgetnum tgetstr tgoto timeout tputs vidattr vidputs waddchnstr waddchstr wchgat @@ -1950,6 +1950,29 @@ return PyCursesCheckERR(intrflush(NULL,ch), "intrflush"); } +#ifdef HAVE_CURSES_IS_TERM_RESIZED +static PyObject * +PyCurses_Is_Term_Resized(PyObject *self, PyObject *args) +{ + int lines; + int columns; + int result; + + PyCursesInitialised + + if (!PyArg_ParseTuple(args,"ii:is_term_resized", &lines, &columns)) + return NULL; + result = is_term_resized(lines, columns); + if (result == TRUE) { + Py_INCREF(Py_True); + return Py_True; + } else { + Py_INCREF(Py_False); + return Py_False; + } +} +#endif /* HAVE_CURSES_IS_TERM_RESIZED */ + #if !defined(__NetBSD__) static PyObject * PyCurses_KeyName(PyObject *self, PyObject *args) @@ -2170,6 +2193,39 @@ } } +#ifdef HAVE_CURSES_RESIZETERM +static PyObject * +PyCurses_ResizeTerm(PyObject *self, PyObject *args) +{ + int lines; + int columns; + + PyCursesInitialised + + if (!PyArg_ParseTuple(args,"ii:resizeterm", &lines, &columns)) + return NULL; + + return PyCursesCheckERR(resizeterm(lines, columns), "resizeterm"); +} + +#endif + +#ifdef HAVE_CURSES_RESIZE_TERM +static PyObject * +PyCurses_Resize_Term(PyObject *self, PyObject *args) +{ + int lines; + int columns; + + PyCursesInitialised + + if (!PyArg_ParseTuple(args,"ii:resize_term", &lines, &columns)) + return NULL; + + return PyCursesCheckERR(resize_term(lines, columns), "resize_term"); +} +#endif /* HAVE_CURSES_RESIZE_TERM */ + static PyObject * PyCurses_setsyx(PyObject *self, PyObject *args) { @@ -2414,6 +2470,9 @@ {"initscr", (PyCFunction)PyCurses_InitScr, METH_NOARGS}, {"intrflush", (PyCFunction)PyCurses_IntrFlush, METH_VARARGS}, {"isendwin", (PyCFunction)PyCurses_isendwin, METH_NOARGS}, +#ifdef HAVE_CURSES_IS_TERM_RESIZED + {"is_term_resized", (PyCFunction)PyCurses_Is_Term_Resized, METH_VARARGS}, +#endif #if !defined(__NetBSD__) {"keyname", (PyCFunction)PyCurses_KeyName, METH_VARARGS}, #endif @@ -2441,6 +2500,12 @@ {"reset_prog_mode", (PyCFunction)PyCurses_reset_prog_mode, METH_NOARGS}, {"reset_shell_mode", (PyCFunction)PyCurses_reset_shell_mode, METH_NOARGS}, {"resetty", (PyCFunction)PyCurses_resetty, METH_NOARGS}, +#ifdef HAVE_CURSES_RESIZETERM + {"resizeterm", (PyCFunction)PyCurses_ResizeTerm, METH_VARARGS}, +#endif +#ifdef HAVE_CURSES_RESIZE_TERM + {"resize_term", (PyCFunction)PyCurses_Resize_Term, METH_VARARGS}, +#endif {"savetty", (PyCFunction)PyCurses_savetty, METH_NOARGS}, {"setsyx", (PyCFunction)PyCurses_setsyx, METH_VARARGS}, {"setupterm", (PyCFunction)PyCurses_setupterm, Modified: python/trunk/configure ============================================================================== --- python/trunk/configure (original) +++ python/trunk/configure Mon Jun 19 10:07:50 2006 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 46879 . +# From configure.in Revision: . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59 for python 2.5. # @@ -722,13 +722,13 @@ /^X$\/$.*/{ s//\1/; q; } s/.*/./; q'` srcdir=$ac_confdir - if test ! -r $srcdir/$ac_unique_file; then + if test ! -r "$srcdir/$ac_unique_file"; then srcdir=.. fi else ac_srcdir_defaulted=no fi -if test ! -r $srcdir/$ac_unique_file; then +if test ! -r "$srcdir/$ac_unique_file"; then if test "$ac_srcdir_defaulted" = yes; then { echo "$as_me: error: cannot find sources ($ac_unique_file) in $ac_confdir or .." >&2 { (exit 1); exit 1; }; } @@ -737,7 +737,7 @@ { (exit 1); exit 1; }; } fi fi -(cd $srcdir && test -r ./$ac_unique_file) 2>/dev/null || +(cd $srcdir && test -r "./$ac_unique_file") 2>/dev/null || { echo "$as_me: error: sources are in $srcdir, but \`cd $srcdir' does not work" >&2 { (exit 1); exit 1; }; } srcdir=`echo "$srcdir" | sed 's%$[^\\/]$[\\/]*$%\1%'` @@ -21854,6 +21854,174 @@ fi +echo "$as_me:$LINENO: checking for is_term_resized" >&5 +echo $ECHO_N "checking for is_term_resized... $ECHO_C" >&6 +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include +int +main () +{ +void *x=is_term_resized + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_CURSES_IS_TERM_RESIZED 1 +_ACEOF + + echo "$as_me:$LINENO: result: yes" >&5 +echo "${ECHO_T}yes" >&6 +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +echo "$as_me:$LINENO: result: no" >&5 +echo "${ECHO_T}no" >&6 + +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext + +echo "$as_me:$LINENO: checking for resize_term" >&5 +echo $ECHO_N "checking for resize_term... $ECHO_C" >&6 +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include +int +main () +{ +void *x=resize_term + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_CURSES_RESIZE_TERM 1 +_ACEOF + + echo "$as_me:$LINENO: result: yes" >&5 +echo "${ECHO_T}yes" >&6 +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +echo "$as_me:$LINENO: result: no" >&5 +echo "${ECHO_T}no" >&6 + +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext + +echo "$as_me:$LINENO: checking for resizeterm" >&5 +echo $ECHO_N "checking for resizeterm... $ECHO_C" >&6 +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include +int +main () +{ +void *x=resizeterm + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_CURSES_RESIZETERM 1 +_ACEOF + + echo "$as_me:$LINENO: result: yes" >&5 +echo "${ECHO_T}yes" >&6 +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +echo "$as_me:$LINENO: result: no" >&5 +echo "${ECHO_T}no" >&6 + +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext + echo "$as_me:$LINENO: checking for /dev/ptmx" >&5 echo $ECHO_N "checking for /dev/ptmx... $ECHO_C" >&6 Modified: python/trunk/configure.in ============================================================================== --- python/trunk/configure.in (original) +++ python/trunk/configure.in Mon Jun 19 10:07:50 2006 @@ -3295,6 +3295,27 @@ [Define if WINDOW in curses.h offers a field _flags.]) fi +AC_MSG_CHECKING(for is_term_resized) +AC_TRY_COMPILE([#include ], void *x=is_term_resized, + AC_DEFINE(HAVE_CURSES_IS_TERM_RESIZED, 1, Define if you have the 'is_term_resized' function.) + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no) +) + +AC_MSG_CHECKING(for resizeterm) +AC_TRY_COMPILE([#include ], void *x=resizeterm, + AC_DEFINE(HAVE_CURSES_RESIZETERM, 1, Define if you have the 'resizeterm' function.) + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no) +) + +AC_MSG_CHECKING(for resize_term) +AC_TRY_COMPILE([#include ], void *x=resize_term, + AC_DEFINE(HAVE_CURSES_RESIZE_TERM, 1, Define if you have the 'resize_term' function.) + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no) +) + AC_MSG_CHECKING(for /dev/ptmx) if test -e /dev/ptmx Modified: python/trunk/pyconfig.h.in ============================================================================== --- python/trunk/pyconfig.h.in (original) +++ python/trunk/pyconfig.h.in Mon Jun 19 10:07:50 2006 @@ -398,6 +398,15 @@ /* Define to 1 if you have the `realpath' function. */ #undef HAVE_REALPATH +/* Define to 1 if you have the `is_term_resized' function. */ +#undef HAVE_CURSES_IS_TERM_RESIZED + +/* Define to 1 if you have the `resize_term' function. */ +#undef HAVE_CURSES_RESIZE_TERM + +/* Define to 1 if you have the `resizeterm' function. */ +#undef HAVE_CURSES_RESIZETERM + /* Define if you have readline 2.1 */ #undef HAVE_RL_CALLBACK From python-checkins at python.org Mon Jun 19 10:14:09 2006 From: python-checkins at python.org (walter.doerwald) Date: Mon, 19 Jun 2006 10:14:09 +0200 (CEST) Subject: [Python-checkins] r47023 - python/trunk/configure.in Message-ID: <20060619081409.C38801E4002@bag.python.org> Author: walter.doerwald Date: Mon Jun 19 10:14:09 2006 New Revision: 47023 Modified: python/trunk/configure.in Log: Make check order match in configure and configure.in. Modified: python/trunk/configure.in ============================================================================== --- python/trunk/configure.in (original) +++ python/trunk/configure.in Mon Jun 19 10:14:09 2006 @@ -3302,16 +3302,16 @@ AC_MSG_RESULT(no) ) -AC_MSG_CHECKING(for resizeterm) -AC_TRY_COMPILE([#include ], void *x=resizeterm, - AC_DEFINE(HAVE_CURSES_RESIZETERM, 1, Define if you have the 'resizeterm' function.) +AC_MSG_CHECKING(for resize_term) +AC_TRY_COMPILE([#include ], void *x=resize_term, + AC_DEFINE(HAVE_CURSES_RESIZE_TERM, 1, Define if you have the 'resize_term' function.) AC_MSG_RESULT(yes), AC_MSG_RESULT(no) ) -AC_MSG_CHECKING(for resize_term) -AC_TRY_COMPILE([#include ], void *x=resize_term, - AC_DEFINE(HAVE_CURSES_RESIZE_TERM, 1, Define if you have the 'resize_term' function.) +AC_MSG_CHECKING(for resizeterm) +AC_TRY_COMPILE([#include ], void *x=resizeterm, + AC_DEFINE(HAVE_CURSES_RESIZETERM, 1, Define if you have the 'resizeterm' function.) AC_MSG_RESULT(yes), AC_MSG_RESULT(no) ) From python-checkins at python.org Mon Jun 19 10:14:28 2006 From: python-checkins at python.org (tim.peters) Date: Mon, 19 Jun 2006 10:14:28 +0200 (CEST) Subject: [Python-checkins] r47024 - python/trunk/Lib/test/test_threaded_import.py Message-ID: <20060619081428.B1A741E4002@bag.python.org> Author: tim.peters Date: Mon Jun 19 10:14:28 2006 New Revision: 47024 Modified: python/trunk/Lib/test/test_threaded_import.py Log: Repair KeyError when running test_threaded_import under -R, as reported by Neal on python-dev. Modified: python/trunk/Lib/test/test_threaded_import.py ============================================================================== --- python/trunk/Lib/test/test_threaded_import.py (original) +++ python/trunk/Lib/test/test_threaded_import.py Mon Jun 19 10:14:28 2006 @@ -30,11 +30,10 @@ if verbose: print "testing import hangers ...", - from test import threaded_import_hangers - + import test.threaded_import_hangers try: - if threaded_import_hangers.errors: - raise TestFailed(threaded_import_hangers.errors) + if test.threaded_import_hangers.errors: + raise TestFailed(test.threaded_import_hangers.errors) elif verbose: print "OK." finally: From python-checkins at python.org Mon Jun 19 10:32:46 2006 From: python-checkins at python.org (thomas.heller) Date: Mon, 19 Jun 2006 10:32:46 +0200 (CEST) Subject: [Python-checkins] r47025 - python/trunk/Lib/ctypes/test/test_loading.py Message-ID: <20060619083246.BFA771E4002@bag.python.org> Author: thomas.heller Date: Mon Jun 19 10:32:46 2006 New Revision: 47025 Modified: python/trunk/Lib/ctypes/test/test_loading.py Log: Next try to fix the OpenBSD buildbot tests: Use ctypes.util.find_library to locate the C runtime library on platforms where is returns useful results. Modified: python/trunk/Lib/ctypes/test/test_loading.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_loading.py (original) +++ python/trunk/Lib/ctypes/test/test_loading.py Mon Jun 19 10:32:46 2006 @@ -9,18 +9,10 @@ libc_name = "msvcrt" elif os.name == "ce": libc_name = "coredll" -elif sys.platform == "darwin": - libc_name = "libc.dylib" elif sys.platform == "cygwin": libc_name = "cygwin1.dll" else: - for line in os.popen("ldd %s" % sys.executable): - if "libc.so" in line: - if sys.platform.startswith("openbsd3"): - libc_name = line.split()[4] - else: - libc_name = line.split()[2] - break + libc_name = find_library("c") if is_resource_enabled("printing"): print "libc_name is", libc_name From python-checkins at python.org Mon Jun 19 11:09:44 2006 From: python-checkins at python.org (tim.peters) Date: Mon, 19 Jun 2006 11:09:44 +0200 (CEST) Subject: [Python-checkins] r47026 - python/trunk/Lib/test/test_optparse.py Message-ID: <20060619090944.C4D971E4002@bag.python.org> Author: tim.peters Date: Mon Jun 19 11:09:44 2006 New Revision: 47026 Modified: python/trunk/Lib/test/test_optparse.py Log: TestHelp.make_parser(): This was making a permanent change to os.environ (setting envar COLUMNS), which at least caused test_float_default() to fail if the tests were run more than once. This repairs the test_optparse -R failures Neal reported on python-dev. It also explains some seemingly bizarre test_optparse failures we saw a couple weeks ago on the buildbots, when test_optparse failed due to test_file failing to clean up after itself, and then test_optparse failed in an entirely different way when regrtest's -w option ran test_optparse a second time. It's now obvious that make_parser() permanently changing os.environ was responsible for the second half of that. Modified: python/trunk/Lib/test/test_optparse.py ============================================================================== --- python/trunk/Lib/test/test_optparse.py (original) +++ python/trunk/Lib/test/test_optparse.py Mon Jun 19 11:09:44 2006 @@ -1460,8 +1460,19 @@ make_option("--foo", action="append", type="string", dest='foo', help="store FOO in the foo list for later fooing"), ] + # The parser constructor looks at the COLUMNS envar. We need to + # restore the original value after the parser is constructed, else + # that's a permanent change possibly affecting other tests, and + # definitely affecting these tests when they're run multiple times. + orig_columns = os.environ.get('COLUMNS') os.environ['COLUMNS'] = str(columns) - return InterceptingOptionParser(option_list=options) + try: + return InterceptingOptionParser(option_list=options) + finally: + if orig_columns is None: + del os.environ['COLUMNS'] + else: + os.environ['COLUMNS'] = orig_columns def assertHelpEquals(self, expected_output): if type(expected_output) is types.UnicodeType: From python-checkins at python.org Mon Jun 19 14:04:15 2006 From: python-checkins at python.org (anthony.baxter) Date: Mon, 19 Jun 2006 14:04:15 +0200 (CEST) Subject: [Python-checkins] r47027 - in python/trunk: Doc/commontex/boilerplate.tex Include/patchlevel.h Lib/idlelib/NEWS.txt Lib/idlelib/idlever.py Misc/NEWS Message-ID: <20060619120415.EF14D1E4002@bag.python.org> Author: anthony.baxter Date: Mon Jun 19 14:04:15 2006 New Revision: 47027 Modified: python/trunk/Doc/commontex/boilerplate.tex python/trunk/Include/patchlevel.h python/trunk/Lib/idlelib/NEWS.txt python/trunk/Lib/idlelib/idlever.py python/trunk/Misc/NEWS Log: Preparing for 2.5b1. Modified: python/trunk/Doc/commontex/boilerplate.tex ============================================================================== --- python/trunk/Doc/commontex/boilerplate.tex (original) +++ python/trunk/Doc/commontex/boilerplate.tex Mon Jun 19 14:04:15 2006 @@ -5,5 +5,5 @@ Email: \email{docs at python.org} } -\date{\today} % XXX update before final release! +\date{20 June, 2006} % XXX update before final release! \input{patchlevel} % include Python version information Modified: python/trunk/Include/patchlevel.h ============================================================================== --- python/trunk/Include/patchlevel.h (original) +++ python/trunk/Include/patchlevel.h Mon Jun 19 14:04:15 2006 @@ -22,11 +22,11 @@ #define PY_MAJOR_VERSION 2 #define PY_MINOR_VERSION 5 #define PY_MICRO_VERSION 0 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_ALPHA -#define PY_RELEASE_SERIAL 2 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_BETA +#define PY_RELEASE_SERIAL 1 /* Version as a string */ -#define PY_VERSION "2.5a2" +#define PY_VERSION "2.5b1" /* Subversion Revision number of this file (not of the repository) */ #define PY_PATCHLEVEL_REVISION "$Revision$" Modified: python/trunk/Lib/idlelib/NEWS.txt ============================================================================== --- python/trunk/Lib/idlelib/NEWS.txt (original) +++ python/trunk/Lib/idlelib/NEWS.txt Mon Jun 19 14:04:15 2006 @@ -1,3 +1,8 @@ +What's New in IDLE 1.2b1? +========================= + +*Release date: 20-JUN-2006* + What's New in IDLE 1.2a2? ========================= Modified: python/trunk/Lib/idlelib/idlever.py ============================================================================== --- python/trunk/Lib/idlelib/idlever.py (original) +++ python/trunk/Lib/idlelib/idlever.py Mon Jun 19 14:04:15 2006 @@ -1 +1 @@ -IDLE_VERSION = "1.2a2" +IDLE_VERSION = "1.2b1" Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Mon Jun 19 14:04:15 2006 @@ -7,7 +7,7 @@ What's New in Python 2.5 beta 1? ================================= -*Release date: XX-JUN-2006* +*Release date: 20-JUN-2006* Core and builtins ----------------- From python-checkins at python.org Mon Jun 19 14:08:25 2006 From: python-checkins at python.org (anthony.baxter) Date: Mon, 19 Jun 2006 14:08:25 +0200 (CEST) Subject: [Python-checkins] r47028 - sandbox/trunk/welease/README.txt sandbox/trunk/welease/sign-and-check.sh sandbox/trunk/welease/welease.py Message-ID: <20060619120825.6F0431E4002@bag.python.org> Author: anthony.baxter Date: Mon Jun 19 14:08:25 2006 New Revision: 47028 Added: sandbox/trunk/welease/sign-and-check.sh (contents, props changed) Modified: sandbox/trunk/welease/README.txt sandbox/trunk/welease/welease.py Log: updates to the welease tool Modified: sandbox/trunk/welease/README.txt ============================================================================== --- sandbox/trunk/welease/README.txt (original) +++ sandbox/trunk/welease/README.txt Mon Jun 19 14:08:25 2006 @@ -1,13 +1,14 @@ This is a GUI for automation of Python releases. It's still very early days -for it yet. See the comments at the top of welease.py for more. +for it yet. See the comments at the top of welease.py for more information. Don't even think about running this on something other than a linux box, I doubt it will work worth a damn (It needs GNU tar, and a sane operating system) Requirements: - python-gtk (unsure which version - something vaguely recent should be ok) + python-gtk 2.4 or later twisted 2.1 or later GNU tar (with bzip2 support) + gnome-gpg (to be replaced with pygpgme) -- Anthony Added: sandbox/trunk/welease/sign-and-check.sh ============================================================================== --- (empty file) +++ sandbox/trunk/welease/sign-and-check.sh Mon Jun 19 14:08:25 2006 @@ -0,0 +1,14 @@ +#!/bin/sh + +for filename in $* +do + rm -f $filename.asc + gnome-gpg --sign --detach --armor $filename + if [ $? -ne 0 ]; then + echo "signing $filename failed!" + fi + gnome-gpg --verify-files $filename.asc + if [ $? -ne 0 ]; then + echo "signature for $filename failed!" + fi +done Modified: sandbox/trunk/welease/welease.py ============================================================================== --- sandbox/trunk/welease/welease.py (original) +++ sandbox/trunk/welease/welease.py Mon Jun 19 14:08:25 2006 @@ -41,7 +41,7 @@ # Check the revision in the checkout is the same as repository HEAD (fatal) # Check Lib/idlelib/idlever.py (fatal) # Check Misc/NEWS and Lib/idlelib/NEWS.txt (fatal) -# Check product_codes in Tools/msi/msi.py (fatal) +# Check product_codes in Tools/msi/uuids.py (fatal) # Check PCbuild/BUILDno.txt and PCbuild/pythoncore.vcproj [XXX] [2.4 only] # Check Doc/commontex/boilerplate.tex [XXX] # Check various release dates match [XXX] @@ -57,7 +57,11 @@ gtk2reactor.install() from twisted.internet import reactor, protocol, defer -import os, sys, re +import os, sys, re, time, calendar +import gettext +import compiler, compiler.ast +import imp + cfg = None @@ -74,7 +78,6 @@ warnDefer = None def __init__(self): - import gettext domain = gettext.textdomain() gtk.glade.textdomain(domain) self.xml = gtk.glade.XML("welease.glade", None, gettext.textdomain()) @@ -478,7 +481,6 @@ def parseNewsFile(filename): - import time vers = releasedate = err = None for line in open(filename): line = line.strip() @@ -505,10 +507,7 @@ # There has _got_ to be a better way to handle this! # Best would be if msi.py was at least importable on non-Windows. # Or I can parse the file by hand - blech. -import compiler, compiler.ast - def extractDictFromAST(ast, dictname): - import compiler, compiler.ast products = None for item in ast.getChildren(): if isinstance(item, compiler.ast.Assign): @@ -536,7 +535,6 @@ from uuids import product_codes unimportModules('uuids') else: - import compiler filename = os.path.join(dirname, 'msi.py') ast = compiler.parseFile(filename).getChildren()[1] product_codes = extractDictFromAST(ast, 'product_codes') @@ -550,12 +548,11 @@ code = maj + '150' # final, serial 0 else: code = maj + levels[serial[0]] + serial[1:] - print code, product_codes + #print code, product_codes return product_codes.get(code) def main(): global cfg - import imp cfgFileName = os.path.expanduser('~/.weleaserc') win = WeleaseWindow() From buildbot at python.org Mon Jun 19 14:27:42 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 19 Jun 2006 12:27:42 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060619122742.8312E1E4002@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/429 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl,thomas.heller,tim.peters,walter.doerwald Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 19 14:55:29 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 19 Jun 2006 12:55:29 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060619125529.9D94F1E4002@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/695 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 19 15:11:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 19 Jun 2006 13:11:28 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060619131128.9DC9F1E4002@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/673 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 19 19:31:16 2006 From: python-checkins at python.org (fred.drake) Date: Mon, 19 Jun 2006 19:31:16 +0200 (CEST) Subject: [Python-checkins] r47029 - python/trunk/Doc/Makefile Message-ID: <20060619173116.88C221E4002@bag.python.org> Author: fred.drake Date: Mon Jun 19 19:31:16 2006 New Revision: 47029 Modified: python/trunk/Doc/Makefile Log: remove non-working document formats from edist Modified: python/trunk/Doc/Makefile ============================================================================== --- python/trunk/Doc/Makefile (original) +++ python/trunk/Doc/Makefile Mon Jun 19 19:31:16 2006 @@ -691,7 +691,7 @@ # The small amount of additional work is a small price to pay for not # having to remember which order to do it in. ;) paperdist: distpdf distps pkglist -edist: disthtml distinfo zipisilo pkglist +edist: disthtml pkglist # The pkglist.html file is used as part of the download.html page on # python.org; it is not used as intermediate input here or as part of From python-checkins at python.org Mon Jun 19 23:17:36 2006 From: python-checkins at python.org (gerhard.haering) Date: Mon, 19 Jun 2006 23:17:36 +0200 (CEST) Subject: [Python-checkins] r47030 - python/trunk/Modules/_sqlite/connection.c python/trunk/Modules/_sqlite/module.h Message-ID: <20060619211736.6AB5F1E4002@bag.python.org> Author: gerhard.haering Date: Mon Jun 19 23:17:35 2006 New Revision: 47030 Modified: python/trunk/Modules/_sqlite/connection.c python/trunk/Modules/_sqlite/module.h Log: Fixed a memory leak that was introduced with incorrect usage of the Python weak reference API in pysqlite 2.2.1. Bumbed pysqlite version number to upcoming pysqlite 2.3.1 release. Modified: python/trunk/Modules/_sqlite/connection.c ============================================================================== --- python/trunk/Modules/_sqlite/connection.c (original) +++ python/trunk/Modules/_sqlite/connection.c Mon Jun 19 23:17:35 2006 @@ -664,7 +664,7 @@ for (i = 0; i < PyList_Size(self->statements); i++) { weakref = PyList_GetItem(self->statements, i); - if (weakref != Py_None) { + if (PyWeakref_GetObject(weakref) != Py_None) { if (PyList_Append(new_list, weakref) != 0) { Py_DECREF(new_list); return; Modified: python/trunk/Modules/_sqlite/module.h ============================================================================== --- python/trunk/Modules/_sqlite/module.h (original) +++ python/trunk/Modules/_sqlite/module.h Mon Jun 19 23:17:35 2006 @@ -25,7 +25,7 @@ #define PYSQLITE_MODULE_H #include "Python.h" -#define PYSQLITE_VERSION "2.3.0" +#define PYSQLITE_VERSION "2.3.1" extern PyObject* Error; extern PyObject* Warning; From python-checkins at python.org Mon Jun 19 23:21:32 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Mon, 19 Jun 2006 23:21:32 +0200 (CEST) Subject: [Python-checkins] r47031 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060619212132.E91FC1E400F@bag.python.org> Author: mateusz.rukowicz Date: Mon Jun 19 23:21:32 2006 New Revision: 47031 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Integer division and remainder implemented. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Mon Jun 19 23:21:32 2006 @@ -455,9 +455,12 @@ * at most as much elements as second + 1, it calculates prec *significant* * limbs, first must not be 0 */ /* XXX it's naive dividing, very slow */ +/* min_new_pos tells, when we should stop dividing, useful for integer division + * make it > flimbs - 2, and it will have no impact*/ + static long _limb_divide(long *first, long flimbs, long *second, long slimbs, - long *out, long prec, long *rest) + long *out, long prec, long *rest, long min_new_pos) { long rlimbs = 1; /* significant limbs of rest */ long is_significant = 0; /* tells whether non_zero limb has already showed up */ @@ -496,6 +499,22 @@ rest[0] = 0; rlimbs = _limb_size(rest, slimbs + 1); + if (min_new_pos == new_pos) + { + long last_written = out_pos + 1; + long i; + long rem_limbs = flimbs > slimbs + 1 ? flimbs : slimbs + 1; + /* XXX SLOW */ + for (i = new_pos - 1; i>=0; i--) { + _limb_move_left(rest, rem_limbs, 1); + rest[0] = first[i]; + } + for (i = 0; i + last_written < prec; i++) + out[i] = out[i+last_written]; + for (i = prec - last_written; i < prec ;i++) + out[i] = 0; + return new_pos; + } new_pos --; } @@ -667,6 +686,7 @@ static decimalobject *_do_decimal_subtract(decimalobject *, decimalobject *, contextobject *); static contextobject *context_shallow_copy(contextobject *); static PyObject *context_ignore_all_flags(contextobject *); +static PyObject *context_regard_flags(contextobject *, PyObject*); static decimalobject *_do_decimal_absolute(decimalobject *, contextobject *, int); /* Exception handlers *********************************************************/ @@ -2473,6 +2493,7 @@ int self_is_zero; int other_is_zero; int shouldround; + decimalobject *op1, *op2; PyObject *ans; if (ISSPECIAL(self) || ISSPECIAL(other)) { @@ -2570,6 +2591,8 @@ if (!ans1) return NULL; + + ans1->limbs[0] = 0; ans2 = _decimal_get_copy(self); if (!ans2) { @@ -2622,6 +2645,7 @@ Py_DECREF(ans2); return NULL; } + ans1->limbs[0] = 0; ans = Py_BuildValue("(OO)", ans1, ans2); Py_DECREF(ans1); @@ -2681,7 +2705,7 @@ return NULL; } - cmp = _do_real_decimal_compare(self, other, ctx); + cmp = _do_real_decimal_compare(abs1, abs2, ctx); Py_DECREF(abs1); Py_DECREF(abs2); @@ -2740,22 +2764,42 @@ } } + op1 = self; + op2 = other; + + /* if divmod, then op1->exp - op2->exp must be divisible by LOG */ + + if (divmod) { + long expdiff = op1->exp - op2->exp; + long tmp_exp = op1->exp; + /* we will do in loop, because of some issues with x % a, when x < 0 */ + while (expdiff % LOG) { + tmp_exp --; + expdiff = tmp_exp - op2->exp; + } + + op1 = _decimal_rescale(self, tmp_exp, ctx, -1, 0); + } - if (divmod == 0) { + if (1) { long prec_needed; /* how many significant digits we need */ long significant_limbs; /* how many significant limbs we need to achieve prec */ decimalobject *result; + decimalobject *remainder_ret; long *remainder; - long exp = self->exp - other->exp; + long exp = op1->exp - op2->exp; long expdiff; long rlimbs; long old_size; long adj1, adj2, adjusted = 0; long i; long max_size; + long min_expdiff; /* used when divmod */ + long remainder_limbs = op2->limb_count+1 > op1->limb_count ? + op2->limb_count + 1 : op1->limb_count; - if (!shouldround) + if (!shouldround && !divmod) Py_RETURN_NONE; /* TODO */ prec_needed = ctx->prec+1; @@ -2765,8 +2809,8 @@ significant_limbs = (prec_needed + LOG -1) / LOG; significant_limbs += ((prec_needed + LOG -1) % LOG) != 0; - remainder = (long*) PyMem_MALLOC((other->limb_count+1) * sizeof(long)); - memset(remainder, 0, sizeof(long) * (other->limb_count+1)); + remainder = (long*) PyMem_MALLOC((remainder_limbs) * sizeof(long)); + memset(remainder, 0, sizeof(long) * (remainder_limbs)); if (!remainder) { PyErr_NoMemory(); return NULL; @@ -2777,7 +2821,8 @@ * limb 1, to make rounding working properly */ result = _NEW_decimalobj(significant_limbs * LOG + LOG, sign, 0); for (i = 0; i < result->limb_count; i++) - result->limbs[0] = 0; + result->limbs[i] = 0; + result->ob_size -= LOG; result->limb_count --; if (!result) { @@ -2785,21 +2830,59 @@ return NULL; } - expdiff = _limb_divide(self->limbs, self->limb_count, - other->limbs, other->limb_count, result->limbs, - significant_limbs, remainder); + /* we want exp + expdiff * LOG + LOG >= 0 so: + * expdiff * LOG >= -exp - LOG + * expdiff >= (-exp - LOG)/ LOG */ + if (divmod) { + assert(!(exp % LOG)); + min_expdiff = (-exp - LOG)/ LOG; + + } + else + min_expdiff = op1->limb_count; + + expdiff = _limb_divide(op1->limbs, op1->limb_count, + op2->limbs, op2->limb_count, result->limbs, + significant_limbs, remainder, min_expdiff); result->limbs[significant_limbs] = 0; exp += expdiff * LOG + LOG; - rlimbs = _limb_size(remainder, other->limb_count + 1); - /* non-zero */ + rlimbs = _limb_size(remainder, remainder_limbs); + /* remainder non-zero */ if (!(rlimbs == 1 && remainder[0] == 0)) { - exp -= LOG; - result->limb_count ++; - result->ob_size += LOG; - _limb_move_left(result->limbs, result->limb_count, 1); - result->limbs[0] = 1; + /* we have not enough precision to do exact integer division */ + if (divmod && exp < 0) { + Py_DECREF(op1); + Py_DECREF(result); + PyMem_FREE(remainder); + return handle_DivisionImpossible(self->ob_type, ctx, NULL); + } + + if (!divmod) { + exp -= LOG; + result->limb_count ++; + result->ob_size += LOG; + _limb_move_left(result->limbs, result->limb_count, 1); + result->limbs[0] = 1; + } + } + + if (divmod) { + remainder_ret = _NEW_decimalobj(rlimbs * LOG, self->sign, 0); + if (!remainder_ret) { + Py_DECREF(op1); + Py_DECREF(result); + PyMem_FREE(remainder); + return NULL; + } + for (i = 0; i< rlimbs; i++) + remainder_ret->limbs[i] = remainder[i]; + if (expdiff <= 0) + remainder_ret->exp = op1->exp + expdiff * LOG; + else + remainder_ret->exp = op1->exp; + remainder_ret->ob_size = _limb_size_s(remainder_ret->limbs, remainder_ret->ob_size); } old_size = result->ob_size; @@ -2808,7 +2891,7 @@ result->exp = exp; - max_size = self->ob_size > other->ob_size ? self->ob_size : other->ob_size; + max_size = op1->ob_size > op2->ob_size ? op1->ob_size : op2->ob_size; /* adjusted is computed just like in the python code */ adjusted = self->ob_size - other->ob_size + 1; @@ -2821,28 +2904,99 @@ break; } } - assert(self->ob_size == _limb_size_s(self->limbs, self->ob_size)); - assert(other->ob_size == _limb_size_s(other->limbs, other->ob_size)); + assert(op1->ob_size == _limb_size_s(op1->limbs, op1->ob_size)); + assert(op2->ob_size == _limb_size_s(op2->limbs, op2->ob_size)); /* to be compatible with python implementation, result int must * have more than adjusted digits =] */ while (result->ob_size > adjusted && _limb_get_digit(result->limbs, result->ob_size, result->ob_size -1) == 0) { + /* when int dividing, exp should be 0 */ + if (result->exp >= 0 && divmod) + break; _limb_cut_one_digit(result->limbs, result->ob_size); result->ob_size --; result->exp ++; } + result->limb_count = (result->ob_size + LOG -1)/LOG; + + if (result->ob_size + result->exp > ctx->prec && shouldround && divmod) { + Py_DECREF(remainder_ret); + Py_DECREF(result); + Py_DECREF(op1); + PyMem_FREE(remainder); + + return handle_DivisionImpossible(self->ob_type, ctx, NULL); + } + /* TODO this *need* clean up */ + if (divmod) { + /* we need to rescale, to be compatible with python implementation */ + PyObject *flags; + contextobject *ctx2; + long remainder_exp = self->exp < other->exp ? self->exp : other->exp; + decimalobject *rescaled_rem = 0; + decimalobject *rescaled = _decimal_rescale(result, 0, ctx, -1, 0); + if (!rescaled) { + Py_DECREF(result); + Py_DECREF(remainder_ret); + Py_DECREF(op1); + return NULL; + } + ctx2 = context_copy(ctx); + if (!ctx2) { + Py_DECREF(result); + Py_DECREF(remainder_ret); + Py_DECREF(op1); + } + flags = context_ignore_all_flags(ctx2); + if (!flags) { + Py_DECREF(result); + Py_DECREF(remainder_ret); + Py_DECREF(op1); + return NULL; + } + rescaled_rem = _decimal_rescale(remainder_ret, remainder_exp, ctx2, -1, 0); + Py_DECREF(ctx2); + Py_DECREF(flags); + Py_DECREF(remainder_ret); + if (!rescaled_rem) { + Py_DECREF(rescaled); + Py_DECREF(result); + Py_DECREF(op1); + return NULL; + } + + if (shouldround) { + decimalobject *fixed = _decimal_fix(rescaled_rem, ctx); + if (!fixed) { + Py_DECREF(rescaled); + Py_DECREF(result); + Py_DECREF(op1); + Py_DECREF(rescaled_rem); + return NULL; + } + Py_DECREF(rescaled_rem); + rescaled_rem = fixed; + } + + ans = Py_BuildValue("(OO)", rescaled, rescaled_rem); + + Py_DECREF(rescaled); + Py_DECREF(result); + Py_DECREF(rescaled_rem); + Py_DECREF(op1); + } - if (shouldround) { + if (shouldround && !divmod) { decimalobject *fixed; fixed = _decimal_fix(result, ctx); Py_DECREF(result); ans = (PyObject*) fixed; } - else { + else if (!divmod){ ans = (PyObject*) result; } PyMem_FREE(remainder); @@ -3378,8 +3532,18 @@ _do_decimal_floor_div(decimalobject *self, decimalobject *other, contextobject *ctx) { - /* XXX */ - Py_RETURN_NONE; + decimalobject *ret; + PyObject *seq = _do_decimal__divide(self, other, 2, ctx); + if (!seq) + return NULL; + + if (PySequence_Check(seq)) { + ret = PySequence_GetItem(seq, 0); + Py_DECREF(seq); + } + else + ret = seq; + return ret; } DECIMAL_SPECIAL_2FUNC(decimal_floor_div) @@ -3398,8 +3562,28 @@ _do_decimal_remainder(decimalobject *self, decimalobject *other, contextobject *ctx) { - /* XXX */ - Py_RETURN_NONE; + decimalobject *ret; + if (ISSPECIAL(self) || ISSPECIAL(other)) { + decimalobject *nan; + if (_check_nans(self, other, ctx, &nan)) + return nan; + } + + if (decimal_nonzero(self) && !decimal_nonzero(other)) { + return handle_InvalidOperation(self->ob_type, ctx, "x % 0", NULL); + } + PyObject *seq = _do_decimal__divide(self, other, 3, ctx); + + if (!seq) + return NULL; + + if (PySequence_Check(seq)) { + ret = PySequence_GetItem(seq, 1); + Py_DECREF(seq); + } + else + ret = seq; + return ret; } DECIMAL_SPECIAL_2FUNC(decimal_remainder) @@ -3408,8 +3592,7 @@ _do_decimal_divmod(decimalobject *self, decimalobject *other, contextobject *ctx) { - /* XXX */ - Py_RETURN_NONE; + return _do_decimal__divide(self, other, 1, ctx); } DECIMAL_SPECIAL_2FUNC(decimal_divmod) From buildbot at python.org Mon Jun 19 23:24:04 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 19 Jun 2006 21:24:04 +0000 Subject: [Python-checkins] buildbot failure in x86 XP-2 trunk Message-ID: <20060619212404.631821E4004@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-2%2520trunk/builds/676 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gerhard.haering BUILD FAILED: failed compile sincerely, -The Buildbot From buildbot at python.org Tue Jun 20 00:25:42 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 19 Jun 2006 22:25:42 +0000 Subject: [Python-checkins] buildbot warnings in x86 Ubuntu dapper (icc) trunk Message-ID: <20060619222542.BC95C1E4002@bag.python.org> The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528icc%2529%2520trunk/builds/675 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gerhard.haering Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 20 00:49:37 2006 From: python-checkins at python.org (ka-ping.yee) Date: Tue, 20 Jun 2006 00:49:37 +0200 (CEST) Subject: [Python-checkins] r47032 - python/trunk/Lib/uuid.py Message-ID: <20060619224937.371221E4002@bag.python.org> Author: ka-ping.yee Date: Tue Jun 20 00:49:36 2006 New Revision: 47032 Modified: python/trunk/Lib/uuid.py Log: Remove Python 2.3 compatibility comment. Modified: python/trunk/Lib/uuid.py ============================================================================== --- python/trunk/Lib/uuid.py (original) +++ python/trunk/Lib/uuid.py Tue Jun 20 00:49:36 2006 @@ -42,8 +42,7 @@ # make a UUID from a 16-byte string >>> uuid.UUID(bytes=x.bytes) UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') - -This module works with Python 2.3 or higher.""" +""" __author__ = 'Ka-Ping Yee ' __date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') From python-checkins at python.org Tue Jun 20 01:21:27 2006 From: python-checkins at python.org (trent.mick) Date: Tue, 20 Jun 2006 01:21:27 +0200 (CEST) Subject: [Python-checkins] r47033 - in python/trunk: Include/pyexpat.h Modules/_elementtree.c Modules/expat/Makefile.in Modules/expat/amigaconfig.h Modules/expat/expat.h Modules/expat/expat_external.h Modules/expat/xmlparse.c Modules/expat/xmlrole.c Modules/expat/xmltok.c Modules/expat/xmltok.h Modules/expat/xmltok_impl.c Modules/expat/xmltok_ns.c Message-ID: <20060619232127.7B4021E400A@bag.python.org> Author: trent.mick Date: Tue Jun 20 01:21:25 2006 New Revision: 47033 Added: python/trunk/Modules/expat/amigaconfig.h (contents, props changed) Removed: python/trunk/Modules/expat/Makefile.in Modified: python/trunk/Include/pyexpat.h python/trunk/Modules/_elementtree.c python/trunk/Modules/expat/expat.h python/trunk/Modules/expat/expat_external.h python/trunk/Modules/expat/xmlparse.c python/trunk/Modules/expat/xmlrole.c python/trunk/Modules/expat/xmltok.c python/trunk/Modules/expat/xmltok.h python/trunk/Modules/expat/xmltok_impl.c python/trunk/Modules/expat/xmltok_ns.c Log: Upgrade pyexpat to expat 2.0.0 (http://python.org/sf/1462338). Modified: python/trunk/Include/pyexpat.h ============================================================================== --- python/trunk/Include/pyexpat.h (original) +++ python/trunk/Include/pyexpat.h Tue Jun 20 01:21:25 2006 @@ -16,8 +16,8 @@ the end, if needed */ const XML_LChar * (*ErrorString)(enum XML_Error code); enum XML_Error (*GetErrorCode)(XML_Parser parser); - int (*GetErrorColumnNumber)(XML_Parser parser); - int (*GetErrorLineNumber)(XML_Parser parser); + XML_Size (*GetErrorColumnNumber)(XML_Parser parser); + XML_Size (*GetErrorLineNumber)(XML_Parser parser); enum XML_Status (*Parse)( XML_Parser parser, const char *s, int len, int isFinal); XML_Parser (*ParserCreate_MM)( Modified: python/trunk/Modules/_elementtree.c ============================================================================== --- python/trunk/Modules/_elementtree.c (original) +++ python/trunk/Modules/_elementtree.c Tue Jun 20 01:21:25 2006 @@ -1989,7 +1989,7 @@ Py_XDECREF(res); } else { PyErr_Format( - PyExc_SyntaxError, "undefined entity &%s;: line %d, column %d", + PyExc_SyntaxError, "undefined entity &%s;: line %ld, column %ld", PyString_AS_STRING(key), EXPAT(GetErrorLineNumber)(self->parser), EXPAT(GetErrorColumnNumber)(self->parser) @@ -2350,7 +2350,7 @@ if (!ok) { PyErr_Format( - PyExc_SyntaxError, "%s: line %d, column %d", + PyExc_SyntaxError, "%s: line %ld, column %ld", EXPAT(ErrorString)(EXPAT(GetErrorCode)(self->parser)), EXPAT(GetErrorLineNumber)(self->parser), EXPAT(GetErrorColumnNumber)(self->parser) Deleted: /python/trunk/Modules/expat/Makefile.in ============================================================================== --- /python/trunk/Modules/expat/Makefile.in Tue Jun 20 01:21:25 2006 +++ (empty file) @@ -1,158 +0,0 @@ -################################################################ -# Process this file with top-level configure script to produce Makefile -# -# Copyright 2000 Clark Cooper -# -# This file is part of EXPAT. -# -# EXPAT is free software; you can redistribute it and/or modify it -# under the terms of the License (based on the MIT/X license) contained -# in the file COPYING that comes with this distribution. -# -# EXPAT IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, -# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. -# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY -# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, -# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE -# SOFTWARE OR THE USE OR OTHER DEALINGS IN EXPAT. -# - - -SHELL = @SHELL@ - -srcdir = @srcdir@ -top_srcdir = @top_srcdir@ -VPATH = @srcdir@ -prefix = @prefix@ -exec_prefix = @exec_prefix@ - -bindir = @bindir@ -sbindir = @sbindir@ -libexecdir = @libexecdir@ -datadir = @datadir@ -sysconfdir = @sysconfdir@ -sharedstatedir = @sharedstatedir@ -localstatedir = @localstatedir@ -libdir = @libdir@ -infodir = @infodir@ -mandir = @mandir@ -includedir = @includedir@ -oldincludedir = /usr/include - -subdir = lib - -top_builddir = .. - -INSTALL = @INSTALL@ -INSTALL_PROGRAM = @INSTALL_PROGRAM@ -INSTALL_SCRIPT = @INSTALL_SCRIPT@ -INSTALL_DATA = @INSTALL_DATA@ - -host_alias = @host_alias@ -host_triplet = @host@ -AS = @AS@ -CC = @CC@ -DLLTOOL = @DLLTOOL@ -LIBTOOL = @LIBTOOL@ -LN_S = @LN_S@ -OBJDUMP = @OBJDUMP@ -PACKAGE = @PACKAGE@ -RANLIB = @RANLIB@ -VERSION = @VERSION@ - -LIBRARY = libexpat.la -SOURCES = xmlparse.c xmltok.c xmlrole.c -OBJECTS = $(SOURCES:.c=.o) -LTOBJECTS = $(SOURCES:.c=.lo) - -TEMPLATES = xmltok_impl.c xmltok_ns.c -APIHEADER = expat.h -HEADERS = ascii.h iasciitab.h utf8tab.h xmltok.h asciitab.h latin1tab.h \ - nametab.h xmldef.h xmlrole.h xmltok_impl.h - -mkinstalldirs = $(SHELL) $(top_srcdir)/conftools/mkinstalldirs -CONFIG_HEADER = ../config.h -CONFIG_CLEAN_FILES = - -INCLUDES = -I$(srcdir) -I. -I.. -DEFS = @DEFS@ -DPACKAGE='"$(PACKAGE)"' -DVERSION='"$(PACKAGE)_$(VERSION)"' - -CPPFLAGS = @CPPFLAGS@ -LDFLAGS = @LDFLAGS@ -LIBS = @LIBS@ -CFLAGS = @CFLAGS@ - -LIBREVISION = @LIBREVISION@ -LIBCURRENT = @LIBCURRENT@ -LIBAGE = @LIBAGE@ - -COMPILE = $(CC) $(DEFS) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -LTCOMPILE = $(LIBTOOL) --mode=compile $(CC) $(DEFS) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -CCLD = $(CC) -LINK = $(LIBTOOL) --mode=link $(CCLD) -version-info $(LIBCURRENT):$(LIBREVISION):$(LIBAGE) $(CFLAGS) $(LDFLAGS) -o $@ -DIST_COMMON = Makefile.in - - -DISTFILES = $(DIST_COMMON) $(SOURCES) $(TEMPLATES) $(APIHEADER) $(HEADERS) - -TAR = gtar -GZIP_ENV = --best - -all: $(LIBRARY) - -.SUFFIXES: .c .lo .o -.PHONY: all clean distclean maintainer-clean - -.c.o: - $(COMPILE) -c $< - -.c.lo: - $(LTCOMPILE) -c $< - -Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status - cd $(top_builddir) \ - && CONFIG_FILES=$(subdir)/$@ CONFIG_HEADERS= $(SHELL) ./config.status - -$(top_builddir)/config.status: $(top_builddir)/configure - cd $(top_builddir) && $(MAKE) config.status - -$(top_builddir)/config.h: $(top_builddir)/config.h.in - cd $(top_builddir) && $(MAKE) config.h - -clean: - rm -f $(LIBRARY) *.o *.lo *~ - rm -rf .libs _libs - -distclean: clean - rm -f Makefile - -maintainer-clean: distclean - -check: $(SUBDIRS) - @echo - @echo This package does not yet have a regression test. - @echo - -install: $(LIBRARY) $(APIHEADER) - $(mkinstalldirs) $(libdir) $(includedir) - $(LIBTOOL) --mode=install $(INSTALL) $(LIBRARY) $(libdir)/$(LIBRARY) - $(INSTALL_DATA) $(APIHEADER) $(includedir) - -uninstall: - $(LIBTOOL) --mode=uninstall rm -f $(libdir)/$(LIBRARY); - rm -f $(libdir)/$(APIHEADER) - -$(LIBRARY): $(LTOBJECTS) - $(LINK) -rpath $(libdir) $(LDFLAGS) $(LTOBJECTS) - -xmlparse.o \ -xmlparse.lo: xmlparse.c expat.h xmlrole.h xmltok.h $(top_builddir)/config.h - -xmlrole.o \ -xmlrole.lo: xmlrole.c ascii.h xmlrole.h $(top_builddir)/config.h - -xmltok.o \ -xmltok.lo: xmltok.c xmltok_impl.c xmltok_ns.c \ - ascii.h asciitab.h iasciitab.h latin1tab.h nametab.h utf8tab.h \ - xmltok.h xmltok_impl.h $(top_builddir)/config.h Added: python/trunk/Modules/expat/amigaconfig.h ============================================================================== --- (empty file) +++ python/trunk/Modules/expat/amigaconfig.h Tue Jun 20 01:21:25 2006 @@ -0,0 +1,96 @@ +#ifndef AMIGACONFIG_H +#define AMIGACONFIG_H + +/* 1234 = LIL_ENDIAN, 4321 = BIGENDIAN */ +#define BYTEORDER 4321 + +/* Define to 1 if you have the `bcopy' function. */ +#define HAVE_BCOPY 1 + +/* Define to 1 if you have the header file. */ +#undef HAVE_CHECK_H + +/* Define to 1 if you have the header file. */ +#undef HAVE_DLFCN_H + +/* Define to 1 if you have the header file. */ +#define HAVE_FCNTL_H 1 + +/* Define to 1 if you have the `getpagesize' function. */ +#undef HAVE_GETPAGESIZE + +/* Define to 1 if you have the header file. */ +#define HAVE_INTTYPES_H 1 + +/* Define to 1 if you have the `memmove' function. */ +#define HAVE_MEMMOVE 1 + +/* Define to 1 if you have the header file. */ +#undef HAVE_MEMORY_H + +/* Define to 1 if you have a working `mmap' system call. */ +#undef HAVE_MMAP + +/* Define to 1 if you have the header file. */ +#define HAVE_STDINT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STDLIB_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRINGS_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRING_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_UNISTD_H 1 + +/* Define to the address where bug reports for this package should be sent. */ +#define PACKAGE_BUGREPORT "expat-bugs at mail.libexpat.org" + +/* Define to the full name of this package. */ +#define PACKAGE_NAME "expat" + +/* Define to the full name and version of this package. */ +#define PACKAGE_STRING "expat 1.95.8" + +/* Define to the one symbol short name of this package. */ +#undef PACKAGE_TARNAME + +/* Define to the version of this package. */ +#define PACKAGE_VERSION "1.95.8" + +/* Define to 1 if you have the ANSI C header files. */ +#define STDC_HEADERS 1 + +/* whether byteorder is bigendian */ +#define WORDS_BIGENDIAN + +/* Define to specify how much context to retain around the current parse + point. */ +#define XML_CONTEXT_BYTES 1024 + +/* Define to make parameter entity parsing functionality available. */ +#define XML_DTD + +/* Define to make XML Namespaces functionality available. */ +#define XML_NS + +/* Define to empty if `const' does not conform to ANSI C. */ +#undef const + +/* Define to `long' if does not define. */ +#undef off_t + +/* Define to `unsigned' if does not define. */ +#undef size_t + + +#endif /* AMIGACONFIG_H */ Modified: python/trunk/Modules/expat/expat.h ============================================================================== --- python/trunk/Modules/expat/expat.h (original) +++ python/trunk/Modules/expat/expat.h Tue Jun 20 01:21:25 2006 @@ -2,8 +2,8 @@ See the file COPYING for copying permission. */ -#ifndef XmlParse_INCLUDED -#define XmlParse_INCLUDED 1 +#ifndef Expat_INCLUDED +#define Expat_INCLUDED 1 #ifdef __VMS /* 0 1 2 3 0 1 2 3 @@ -17,6 +17,10 @@ #include #include "expat_external.h" +#ifdef __cplusplus +extern "C" { +#endif + struct XML_ParserStruct; typedef struct XML_ParserStruct *XML_Parser; @@ -87,7 +91,11 @@ XML_ERROR_NOT_SUSPENDED, XML_ERROR_ABORTED, XML_ERROR_FINISHED, - XML_ERROR_SUSPEND_PE + XML_ERROR_SUSPEND_PE, + /* Added in 2.0. */ + XML_ERROR_RESERVED_PREFIX_XML, + XML_ERROR_RESERVED_PREFIX_XMLNS, + XML_ERROR_RESERVED_NAMESPACE_URI }; enum XML_Content_Type { @@ -205,8 +213,8 @@ URI, the namespace separator character, and the local part of the name. If the namespace separator is '\0' then the namespace URI and the local part will be concatenated without any separator. - When a namespace is not declared, the name and prefix will be - passed through without expansion. + It is a programming error to use the separator '\0' with namespace + triplets (see XML_SetReturnNSTriplet). */ XMLPARSEAPI(XML_Parser) XML_ParserCreateNS(const XML_Char *encoding, XML_Char namespaceSeparator); @@ -897,9 +905,9 @@ was detected; otherwise the location is the location of the last parse event, as described above. */ -XMLPARSEAPI(int) XML_GetCurrentLineNumber(XML_Parser parser); -XMLPARSEAPI(int) XML_GetCurrentColumnNumber(XML_Parser parser); -XMLPARSEAPI(long) XML_GetCurrentByteIndex(XML_Parser parser); +XMLPARSEAPI(XML_Size) XML_GetCurrentLineNumber(XML_Parser parser); +XMLPARSEAPI(XML_Size) XML_GetCurrentColumnNumber(XML_Parser parser); +XMLPARSEAPI(XML_Index) XML_GetCurrentByteIndex(XML_Parser parser); /* Return the number of bytes in the current event. Returns 0 if the event is in an internal entity. @@ -974,7 +982,8 @@ XML_FEATURE_CONTEXT_BYTES, XML_FEATURE_MIN_SIZE, XML_FEATURE_SIZEOF_XML_CHAR, - XML_FEATURE_SIZEOF_XML_LCHAR + XML_FEATURE_SIZEOF_XML_LCHAR, + XML_FEATURE_NS /* Additional features must be added to the end of this enum. */ }; @@ -993,12 +1002,12 @@ releases. Micro is bumped with each release, and set to 0 with each change to major or minor version. */ -#define XML_MAJOR_VERSION 1 -#define XML_MINOR_VERSION 95 -#define XML_MICRO_VERSION 8 +#define XML_MAJOR_VERSION 2 +#define XML_MINOR_VERSION 0 +#define XML_MICRO_VERSION 0 #ifdef __cplusplus } #endif -#endif /* not XmlParse_INCLUDED */ +#endif /* not Expat_INCLUDED */ Modified: python/trunk/Modules/expat/expat_external.h ============================================================================== --- python/trunk/Modules/expat/expat_external.h (original) +++ python/trunk/Modules/expat/expat_external.h Tue Jun 20 01:21:25 2006 @@ -2,6 +2,9 @@ See the file COPYING for copying permission. */ +#ifndef Expat_External_INCLUDED +#define Expat_External_INCLUDED 1 + /* External API definitions */ #if defined(_MSC_EXTENSIONS) && !defined(__BEOS__) && !defined(__CYGWIN__) @@ -62,6 +65,7 @@ #endif #endif /* not defined XML_STATIC */ + /* If we didn't define it above, define it away: */ #ifndef XMLIMPORT #define XMLIMPORT @@ -90,3 +94,22 @@ typedef char XML_Char; typedef char XML_LChar; #endif /* XML_UNICODE */ + +#ifdef XML_LARGE_SIZE /* Use large integers for file/stream positions. */ +#if defined(XML_USE_MSC_EXTENSIONS) && _MSC_VER < 1400 +typedef __int64 XML_Index; +typedef unsigned __int64 XML_Size; +#else +typedef long long XML_Index; +typedef unsigned long long XML_Size; +#endif +#else +typedef long XML_Index; +typedef unsigned long XML_Size; +#endif /* XML_LARGE_SIZE */ + +#ifdef __cplusplus +} +#endif + +#endif /* not Expat_External_INCLUDED */ Modified: python/trunk/Modules/expat/xmlparse.c ============================================================================== --- python/trunk/Modules/expat/xmlparse.c (original) +++ python/trunk/Modules/expat/xmlparse.c Tue Jun 20 01:21:25 2006 @@ -8,6 +8,8 @@ #include "winconfig.h" #elif defined(MACOS_CLASSIC) #include "macconfig.h" +#elif defined(__amigaos4__) +#include "amigaconfig.h" #elif defined(HAVE_EXPAT_CONFIG_H) #include #endif /* ndef COMPILED_FROM_DSP */ @@ -456,7 +458,7 @@ char *m_bufferEnd; /* allocated end of buffer */ const char *m_bufferLim; - long m_parseEndByteIndex; + XML_Index m_parseEndByteIndex; const char *m_parseEndPtr; XML_Char *m_dataBuf; XML_Char *m_dataBufEnd; @@ -640,8 +642,8 @@ #define groupSize (parser->m_groupSize) #define namespaceSeparator (parser->m_namespaceSeparator) #define parentParser (parser->m_parentParser) -#define parsing (parser->m_parsingStatus.parsing) -#define finalBuffer (parser->m_parsingStatus.finalBuffer) +#define ps_parsing (parser->m_parsingStatus.parsing) +#define ps_finalBuffer (parser->m_parsingStatus.finalBuffer) #ifdef XML_DTD #define isParamEntity (parser->m_isParamEntity) #define useForeignDTD (parser->m_useForeignDTD) @@ -852,7 +854,7 @@ unknownEncodingRelease = NULL; unknownEncodingData = NULL; parentParser = NULL; - parsing = XML_INITIALIZED; + ps_parsing = XML_INITIALIZED; #ifdef XML_DTD isParamEntity = XML_FALSE; useForeignDTD = XML_FALSE; @@ -915,7 +917,7 @@ XXX There's no way for the caller to determine which of the XXX possible error cases caused the XML_STATUS_ERROR return. */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return XML_STATUS_ERROR; if (encodingName == NULL) protocolEncodingName = NULL; @@ -1143,7 +1145,7 @@ { #ifdef XML_DTD /* block after XML_Parse()/XML_ParseBuffer() has been called */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING; useForeignDTD = useDTD; return XML_ERROR_NONE; @@ -1156,7 +1158,7 @@ XML_SetReturnNSTriplet(XML_Parser parser, int do_nst) { /* block after XML_Parse()/XML_ParseBuffer() has been called */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return; ns_triplets = do_nst ? XML_TRUE : XML_FALSE; } @@ -1408,7 +1410,7 @@ enum XML_ParamEntityParsing peParsing) { /* block after XML_Parse()/XML_ParseBuffer() has been called */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return 0; #ifdef XML_DTD paramEntityParsing = peParsing; @@ -1421,7 +1423,7 @@ enum XML_Status XMLCALL XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: errorCode = XML_ERROR_SUSPENDED; return XML_STATUS_ERROR; @@ -1429,11 +1431,11 @@ errorCode = XML_ERROR_FINISHED; return XML_STATUS_ERROR; default: - parsing = XML_PARSING; + ps_parsing = XML_PARSING; } if (len == 0) { - finalBuffer = (XML_Bool)isFinal; + ps_finalBuffer = (XML_Bool)isFinal; if (!isFinal) return XML_STATUS_OK; positionPtr = bufferPtr; @@ -1441,19 +1443,19 @@ /* If data are left over from last buffer, and we now know that these data are the final chunk of input, then we have to check them again - to detect errors based on this information. + to detect errors based on that fact. */ errorCode = processor(parser, bufferPtr, parseEndPtr, &bufferPtr); if (errorCode == XML_ERROR_NONE) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: XmlUpdatePosition(encoding, positionPtr, bufferPtr, &position); positionPtr = bufferPtr; return XML_STATUS_SUSPENDED; case XML_INITIALIZED: case XML_PARSING: - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; /* fall through */ default: return XML_STATUS_OK; @@ -1470,7 +1472,7 @@ enum XML_Error result; parseEndByteIndex += len; positionPtr = s; - finalBuffer = (XML_Bool)isFinal; + ps_finalBuffer = (XML_Bool)isFinal; errorCode = processor(parser, s, parseEndPtr = s + len, &end); @@ -1480,7 +1482,7 @@ return XML_STATUS_ERROR; } else { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: result = XML_STATUS_SUSPENDED; break; @@ -1488,14 +1490,13 @@ case XML_PARSING: result = XML_STATUS_OK; if (isFinal) { - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; return result; } } } XmlUpdatePosition(encoding, positionPtr, end, &position); - positionPtr = end; nLeftOver = s + len - end; if (nLeftOver) { if (buffer == NULL || nLeftOver > bufferLim - buffer) { @@ -1518,9 +1519,13 @@ bufferLim = buffer + len * 2; } memcpy(buffer, end, nLeftOver); - bufferPtr = buffer; - bufferEnd = buffer + nLeftOver; } + bufferPtr = buffer; + bufferEnd = buffer + nLeftOver; + positionPtr = bufferPtr; + parseEndPtr = bufferEnd; + eventPtr = bufferPtr; + eventEndPtr = bufferPtr; return result; } #endif /* not defined XML_CONTEXT_BYTES */ @@ -1541,7 +1546,7 @@ const char *start; enum XML_Status result = XML_STATUS_OK; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: errorCode = XML_ERROR_SUSPENDED; return XML_STATUS_ERROR; @@ -1549,7 +1554,7 @@ errorCode = XML_ERROR_FINISHED; return XML_STATUS_ERROR; default: - parsing = XML_PARSING; + ps_parsing = XML_PARSING; } start = bufferPtr; @@ -1557,7 +1562,7 @@ bufferEnd += len; parseEndPtr = bufferEnd; parseEndByteIndex += len; - finalBuffer = (XML_Bool)isFinal; + ps_finalBuffer = (XML_Bool)isFinal; errorCode = processor(parser, start, parseEndPtr, &bufferPtr); @@ -1567,14 +1572,14 @@ return XML_STATUS_ERROR; } else { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: result = XML_STATUS_SUSPENDED; break; case XML_INITIALIZED: case XML_PARSING: if (isFinal) { - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; return result; } default: ; /* should not happen */ @@ -1589,7 +1594,7 @@ void * XMLCALL XML_GetBuffer(XML_Parser parser, int len) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: errorCode = XML_ERROR_SUSPENDED; return NULL; @@ -1601,9 +1606,9 @@ if (len > bufferLim - bufferEnd) { /* FIXME avoid integer overflow */ - int neededSize = len + (bufferEnd - bufferPtr); + int neededSize = len + (int)(bufferEnd - bufferPtr); #ifdef XML_CONTEXT_BYTES - int keep = bufferPtr - buffer; + int keep = (int)(bufferPtr - buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; @@ -1612,7 +1617,7 @@ if (neededSize <= bufferLim - buffer) { #ifdef XML_CONTEXT_BYTES if (keep < bufferPtr - buffer) { - int offset = (bufferPtr - buffer) - keep; + int offset = (int)(bufferPtr - buffer) - keep; memmove(buffer, &buffer[offset], bufferEnd - bufferPtr + keep); bufferEnd -= offset; bufferPtr -= offset; @@ -1625,7 +1630,7 @@ } else { char *newBuf; - int bufferSize = bufferLim - bufferPtr; + int bufferSize = (int)(bufferLim - bufferPtr); if (bufferSize == 0) bufferSize = INIT_BUFFER_SIZE; do { @@ -1639,7 +1644,7 @@ bufferLim = newBuf + bufferSize; #ifdef XML_CONTEXT_BYTES if (bufferPtr) { - int keep = bufferPtr - buffer; + int keep = (int)(bufferPtr - buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; memcpy(newBuf, &bufferPtr[-keep], bufferEnd - bufferPtr + keep); @@ -1668,13 +1673,13 @@ enum XML_Status XMLCALL XML_StopParser(XML_Parser parser, XML_Bool resumable) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: if (resumable) { errorCode = XML_ERROR_SUSPENDED; return XML_STATUS_ERROR; } - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; break; case XML_FINISHED: errorCode = XML_ERROR_FINISHED; @@ -1687,10 +1692,10 @@ return XML_STATUS_ERROR; } #endif - parsing = XML_SUSPENDED; + ps_parsing = XML_SUSPENDED; } else - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; } return XML_STATUS_OK; } @@ -1700,11 +1705,11 @@ { enum XML_Status result = XML_STATUS_OK; - if (parsing != XML_SUSPENDED) { + if (ps_parsing != XML_SUSPENDED) { errorCode = XML_ERROR_NOT_SUSPENDED; return XML_STATUS_ERROR; } - parsing = XML_PARSING; + ps_parsing = XML_PARSING; errorCode = processor(parser, bufferPtr, parseEndPtr, &bufferPtr); @@ -1714,14 +1719,14 @@ return XML_STATUS_ERROR; } else { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: result = XML_STATUS_SUSPENDED; break; case XML_INITIALIZED: case XML_PARSING: - if (finalBuffer) { - parsing = XML_FINISHED; + if (ps_finalBuffer) { + ps_parsing = XML_FINISHED; return result; } default: ; @@ -1746,7 +1751,7 @@ return errorCode; } -long XMLCALL +XML_Index XMLCALL XML_GetCurrentByteIndex(XML_Parser parser) { if (eventPtr) @@ -1758,7 +1763,7 @@ XML_GetCurrentByteCount(XML_Parser parser) { if (eventEndPtr && eventPtr) - return eventEndPtr - eventPtr; + return (int)(eventEndPtr - eventPtr); return 0; } @@ -1767,15 +1772,15 @@ { #ifdef XML_CONTEXT_BYTES if (eventPtr && buffer) { - *offset = eventPtr - buffer; - *size = bufferEnd - buffer; + *offset = (int)(eventPtr - buffer); + *size = (int)(bufferEnd - buffer); return buffer; } #endif /* defined XML_CONTEXT_BYTES */ return (char *) 0; } -int XMLCALL +XML_Size XMLCALL XML_GetCurrentLineNumber(XML_Parser parser) { if (eventPtr && eventPtr >= positionPtr) { @@ -1785,7 +1790,7 @@ return position.lineNumber + 1; } -int XMLCALL +XML_Size XMLCALL XML_GetCurrentColumnNumber(XML_Parser parser) { if (eventPtr && eventPtr >= positionPtr) { @@ -1836,7 +1841,7 @@ const XML_LChar * XMLCALL XML_ErrorString(enum XML_Error code) { - static const XML_LChar *message[] = { + static const XML_LChar* const message[] = { 0, XML_L("out of memory"), XML_L("syntax error"), @@ -1854,7 +1859,7 @@ XML_L("reference to invalid character number"), XML_L("reference to binary entity"), XML_L("reference to external entity in attribute"), - XML_L("xml declaration not at start of external entity"), + XML_L("XML or text declaration not at start of entity"), XML_L("unknown encoding"), XML_L("encoding specified in XML declaration is incorrect"), XML_L("unclosed CDATA section"), @@ -1874,7 +1879,10 @@ XML_L("parser not suspended"), XML_L("parsing aborted"), XML_L("parsing finished"), - XML_L("cannot suspend in external parameter entity") + XML_L("cannot suspend in external parameter entity"), + XML_L("reserved prefix (xml) must not be undeclared or bound to another namespace name"), + XML_L("reserved prefix (xmlns) must not be declared or undeclared"), + XML_L("prefix must not be bound to one of the reserved namespace names") }; if (code > 0 && code < sizeof(message)/sizeof(message[0])) return message[code]; @@ -1916,9 +1924,11 @@ const XML_Feature * XMLCALL XML_GetFeatureList(void) { - static XML_Feature features[] = { - {XML_FEATURE_SIZEOF_XML_CHAR, XML_L("sizeof(XML_Char)"), 0}, - {XML_FEATURE_SIZEOF_XML_LCHAR, XML_L("sizeof(XML_LChar)"), 0}, + static const XML_Feature features[] = { + {XML_FEATURE_SIZEOF_XML_CHAR, XML_L("sizeof(XML_Char)"), + sizeof(XML_Char)}, + {XML_FEATURE_SIZEOF_XML_LCHAR, XML_L("sizeof(XML_LChar)"), + sizeof(XML_LChar)}, #ifdef XML_UNICODE {XML_FEATURE_UNICODE, XML_L("XML_UNICODE"), 0}, #endif @@ -1935,11 +1945,12 @@ #ifdef XML_MIN_SIZE {XML_FEATURE_MIN_SIZE, XML_L("XML_MIN_SIZE"), 0}, #endif +#ifdef XML_NS + {XML_FEATURE_NS, XML_L("XML_NS"), 0}, +#endif {XML_FEATURE_END, NULL, 0} }; - features[0].value = sizeof(XML_Char); - features[1].value = sizeof(XML_LChar); return features; } @@ -2000,7 +2011,7 @@ const char **endPtr) { enum XML_Error result = doContent(parser, 0, encoding, start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result == XML_ERROR_NONE) { if (!storeRawNames(parser)) return XML_ERROR_NO_MEMORY; @@ -2036,21 +2047,21 @@ doContent (by detecting XML_TOK_NONE) without processing any xml text declaration - causing the error XML_ERROR_MISPLACED_XML_PI in doContent. */ - if (next == end && !finalBuffer) { + if (next == end && !ps_finalBuffer) { *endPtr = next; return XML_ERROR_NONE; } start = next; break; case XML_TOK_PARTIAL: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } eventPtr = start; return XML_ERROR_UNCLOSED_TOKEN; case XML_TOK_PARTIAL_CHAR: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } @@ -2080,7 +2091,7 @@ result = processXmlDecl(parser, 1, start, next); if (result != XML_ERROR_NONE) return result; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *endPtr = next; return XML_ERROR_NONE; @@ -2092,13 +2103,13 @@ } break; case XML_TOK_PARTIAL: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } return XML_ERROR_UNCLOSED_TOKEN; case XML_TOK_PARTIAL_CHAR: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } @@ -2116,7 +2127,7 @@ const char **endPtr) { enum XML_Error result = doContent(parser, 1, encoding, start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result == XML_ERROR_NONE) { if (!storeRawNames(parser)) return XML_ERROR_NO_MEMORY; @@ -2315,12 +2326,12 @@ XmlConvert(enc, &fromPtr, rawNameEnd, (ICHAR **)&toPtr, (ICHAR *)tag->bufEnd - 1); - convLen = toPtr - (XML_Char *)tag->buf; + convLen = (int)(toPtr - (XML_Char *)tag->buf); if (fromPtr == rawNameEnd) { tag->name.strLen = convLen; break; } - bufSize = (tag->bufEnd - tag->buf) << 1; + bufSize = (int)(tag->bufEnd - tag->buf) << 1; { char *temp = (char *)REALLOC(tag->buf, bufSize); if (temp == NULL) @@ -2508,12 +2519,12 @@ ICHAR *dataPtr = (ICHAR *)dataBuf; XmlConvert(enc, &s, end, &dataPtr, (ICHAR *)dataBufEnd); characterDataHandler(handlerArg, dataBuf, - dataPtr - (ICHAR *)dataBuf); + (int)(dataPtr - (ICHAR *)dataBuf)); } else characterDataHandler(handlerArg, (XML_Char *)s, - (XML_Char *)end - (XML_Char *)s); + (int)((XML_Char *)end - (XML_Char *)s)); } else if (defaultHandler) reportDefault(parser, enc, s, end); @@ -2538,7 +2549,7 @@ XmlConvert(enc, &s, next, &dataPtr, (ICHAR *)dataBufEnd); *eventEndPP = s; characterDataHandler(handlerArg, dataBuf, - dataPtr - (ICHAR *)dataBuf); + (int)(dataPtr - (ICHAR *)dataBuf)); if (s == next) break; *eventPP = s; @@ -2547,7 +2558,7 @@ else characterDataHandler(handlerArg, (XML_Char *)s, - (XML_Char *)next - (XML_Char *)s); + (int)((XML_Char *)next - (XML_Char *)s)); } else if (defaultHandler) reportDefault(parser, enc, s, next); @@ -2566,7 +2577,7 @@ break; } *eventPP = s = next; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -2822,7 +2833,7 @@ } if (!step) step = PROBE_STEP(uriHash, mask, nsAttsPower); - j < step ? ( j += nsAttsSize - step) : (j -= step); + j < step ? (j += nsAttsSize - step) : (j -= step); } } @@ -2845,8 +2856,10 @@ nsAtts[j].hash = uriHash; nsAtts[j].uriName = s; - if (!--nPrefixes) + if (!--nPrefixes) { + i += 2; break; + } } else /* not prefixed */ ((XML_Char *)s)[-1] = 0; /* clear flag */ @@ -2879,14 +2892,14 @@ prefixLen = 0; if (ns_triplets && binding->prefix->name) { for (; binding->prefix->name[prefixLen++];) - ; + ; /* prefixLen includes null terminator */ } tagNamePtr->localPart = localPart; tagNamePtr->uriLen = binding->uriLen; tagNamePtr->prefix = binding->prefix->name; tagNamePtr->prefixLen = prefixLen; for (i = 0; localPart[i++];) - ; + ; /* i includes null terminator */ n = i + binding->uriLen + prefixLen; if (n > binding->uriAlloc) { TAG *p; @@ -2901,12 +2914,13 @@ FREE(binding->uri); binding->uri = uri; } + /* if namespaceSeparator != '\0' then uri includes it already */ uri = binding->uri + binding->uriLen; memcpy(uri, localPart, i * sizeof(XML_Char)); + /* we always have a namespace separator between localPart and prefix */ if (prefixLen) { - uri = uri + (i - 1); - if (namespaceSeparator) - *uri = namespaceSeparator; + uri += i - 1; + *uri = namespaceSeparator; /* replace null terminator */ memcpy(uri + 1, binding->prefix->name, prefixLen * sizeof(XML_Char)); } tagNamePtr->str = binding->uri; @@ -2920,6 +2934,26 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, const XML_Char *uri, BINDING **bindingsPtr) { + static const XML_Char xmlNamespace[] = { + 'h', 't', 't', 'p', ':', '/', '/', + 'w', 'w', 'w', '.', 'w', '3', '.', 'o', 'r', 'g', '/', + 'X', 'M', 'L', '/', '1', '9', '9', '8', '/', + 'n', 'a', 'm', 'e', 's', 'p', 'a', 'c', 'e', '\0' + }; + static const int xmlLen = + (int)sizeof(xmlNamespace)/sizeof(XML_Char) - 1; + static const XML_Char xmlnsNamespace[] = { + 'h', 't', 't', 'p', ':', '/', '/', + 'w', 'w', 'w', '.', 'w', '3', '.', 'o', 'r', 'g', '/', + '2', '0', '0', '0', '/', 'x', 'm', 'l', 'n', 's', '/', '\0' + }; + static const int xmlnsLen = + (int)sizeof(xmlnsNamespace)/sizeof(XML_Char) - 1; + + XML_Bool mustBeXML = XML_FALSE; + XML_Bool isXML = XML_TRUE; + XML_Bool isXMLNS = XML_TRUE; + BINDING *b; int len; @@ -2927,8 +2961,39 @@ if (*uri == XML_T('\0') && prefix->name) return XML_ERROR_UNDECLARING_PREFIX; - for (len = 0; uri[len]; len++) - ; + if (prefix->name + && prefix->name[0] == XML_T('x') + && prefix->name[1] == XML_T('m') + && prefix->name[2] == XML_T('l')) { + + /* Not allowed to bind xmlns */ + if (prefix->name[3] == XML_T('n') + && prefix->name[4] == XML_T('s') + && prefix->name[5] == XML_T('\0')) + return XML_ERROR_RESERVED_PREFIX_XMLNS; + + if (prefix->name[3] == XML_T('\0')) + mustBeXML = XML_TRUE; + } + + for (len = 0; uri[len]; len++) { + if (isXML && (len > xmlLen || uri[len] != xmlNamespace[len])) + isXML = XML_FALSE; + + if (!mustBeXML && isXMLNS + && (len > xmlnsLen || uri[len] != xmlnsNamespace[len])) + isXMLNS = XML_FALSE; + } + isXML = isXML && len == xmlLen; + isXMLNS = isXMLNS && len == xmlnsLen; + + if (mustBeXML != isXML) + return mustBeXML ? XML_ERROR_RESERVED_PREFIX_XML + : XML_ERROR_RESERVED_NAMESPACE_URI; + + if (isXMLNS) + return XML_ERROR_RESERVED_NAMESPACE_URI; + if (namespaceSeparator) len++; if (freeBindingList) { @@ -2985,7 +3050,7 @@ const char **endPtr) { enum XML_Error result = doCdataSection(parser, encoding, &start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result != XML_ERROR_NONE) return result; if (start) { @@ -3044,7 +3109,7 @@ reportDefault(parser, enc, s, next); *startPtr = next; *nextPtr = next; - if (parsing == XML_FINISHED) + if (ps_parsing == XML_FINISHED) return XML_ERROR_ABORTED; else return XML_ERROR_NONE; @@ -3064,7 +3129,7 @@ XmlConvert(enc, &s, next, &dataPtr, (ICHAR *)dataBufEnd); *eventEndPP = next; characterDataHandler(handlerArg, dataBuf, - dataPtr - (ICHAR *)dataBuf); + (int)(dataPtr - (ICHAR *)dataBuf)); if (s == next) break; *eventPP = s; @@ -3073,7 +3138,7 @@ else characterDataHandler(handlerArg, (XML_Char *)s, - (XML_Char *)next - (XML_Char *)s); + (int)((XML_Char *)next - (XML_Char *)s)); } else if (defaultHandler) reportDefault(parser, enc, s, next); @@ -3100,7 +3165,7 @@ } *eventPP = s = next; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -3124,7 +3189,7 @@ const char **endPtr) { enum XML_Error result = doIgnoreSection(parser, encoding, &start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result != XML_ERROR_NONE) return result; if (start) { @@ -3169,7 +3234,7 @@ reportDefault(parser, enc, s, next); *startPtr = next; *nextPtr = next; - if (parsing == XML_FINISHED) + if (ps_parsing == XML_FINISHED) return XML_ERROR_ABORTED; else return XML_ERROR_NONE; @@ -3409,7 +3474,7 @@ tok = XmlPrologTok(encoding, start, end, &next); eventEndPtr = next; if (tok <= 0) { - if (!finalBuffer && tok != XML_TOK_INVALID) { + if (!ps_finalBuffer && tok != XML_TOK_INVALID) { *nextPtr = s; return XML_ERROR_NONE; } @@ -3432,7 +3497,7 @@ result = processXmlDecl(parser, 0, start, next); if (result != XML_ERROR_NONE) return result; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -3452,7 +3517,7 @@ then, when this routine is entered the next time, XmlPrologTok will return XML_TOK_INVALID, since the BOM is still in the buffer */ - else if (tok == XML_TOK_BOM && next == end && !finalBuffer) { + else if (tok == XML_TOK_BOM && next == end && !ps_finalBuffer) { *nextPtr = next; return XML_ERROR_NONE; } @@ -3472,7 +3537,7 @@ tok = XmlPrologTok(encoding, s, end, &next); if (tok <= 0) { - if (!finalBuffer && tok != XML_TOK_INVALID) { + if (!ps_finalBuffer && tok != XML_TOK_INVALID) { *nextPtr = s; return XML_ERROR_NONE; } @@ -3499,7 +3564,7 @@ processor = prologProcessor; return doProlog(parser, encoding, s, end, tok, next, - nextPtr, (XML_Bool)!finalBuffer); + nextPtr, (XML_Bool)!ps_finalBuffer); } static enum XML_Error PTRCALL @@ -3516,7 +3581,7 @@ for (;;) { tok = XmlPrologTok(enc, start, end, &next); if (tok <= 0) { - if (!finalBuffer && tok != XML_TOK_INVALID) { + if (!ps_finalBuffer && tok != XML_TOK_INVALID) { *nextPtr = s; return XML_ERROR_NONE; } @@ -3549,7 +3614,7 @@ const char *next = s; int tok = XmlPrologTok(encoding, s, end, &next); return doProlog(parser, encoding, s, end, tok, next, - nextPtr, (XML_Bool)!finalBuffer); + nextPtr, (XML_Bool)!ps_finalBuffer); } static enum XML_Error @@ -3738,7 +3803,8 @@ */ #ifdef XML_DTD if (doctypeSysid || useForeignDTD) { - dtd->hasParamEntityRefs = XML_TRUE; /* when docTypeSysid == NULL */ + XML_Bool hadParamEntityRefs = dtd->hasParamEntityRefs; + dtd->hasParamEntityRefs = XML_TRUE; if (paramEntityParsing && externalEntityRefHandler) { ENTITY *entity = (ENTITY *)lookup(&dtd->paramEntities, externalSubsetName, @@ -3754,11 +3820,17 @@ entity->systemId, entity->publicId)) return XML_ERROR_EXTERNAL_ENTITY_HANDLING; - if (dtd->paramEntityRead && - !dtd->standalone && - notStandaloneHandler && - !notStandaloneHandler(handlerArg)) - return XML_ERROR_NOT_STANDALONE; + if (dtd->paramEntityRead) { + if (!dtd->standalone && + notStandaloneHandler && + !notStandaloneHandler(handlerArg)) + return XML_ERROR_NOT_STANDALONE; + } + /* if we didn't read the foreign DTD then this means that there + is no external subset and we must reset dtd->hasParamEntityRefs + */ + else if (!doctypeSysid) + dtd->hasParamEntityRefs = hadParamEntityRefs; /* end of DTD - no need to update dtd->keepProcessing */ } useForeignDTD = XML_FALSE; @@ -3775,6 +3847,7 @@ last chance to read the foreign DTD */ if (useForeignDTD) { + XML_Bool hadParamEntityRefs = dtd->hasParamEntityRefs; dtd->hasParamEntityRefs = XML_TRUE; if (paramEntityParsing && externalEntityRefHandler) { ENTITY *entity = (ENTITY *)lookup(&dtd->paramEntities, @@ -3790,11 +3863,17 @@ entity->systemId, entity->publicId)) return XML_ERROR_EXTERNAL_ENTITY_HANDLING; - if (dtd->paramEntityRead && - !dtd->standalone && - notStandaloneHandler && - !notStandaloneHandler(handlerArg)) - return XML_ERROR_NOT_STANDALONE; + if (dtd->paramEntityRead) { + if (!dtd->standalone && + notStandaloneHandler && + !notStandaloneHandler(handlerArg)) + return XML_ERROR_NOT_STANDALONE; + } + /* if we didn't read the foreign DTD then this means that there + is no external subset and we must reset dtd->hasParamEntityRefs + */ + else + dtd->hasParamEntityRefs = hadParamEntityRefs; /* end of DTD - no need to update dtd->keepProcessing */ } } @@ -3935,7 +4014,7 @@ next - enc->minBytesPerChar); if (declEntity) { declEntity->textPtr = poolStart(&dtd->entityValuePool); - declEntity->textLen = poolLength(&dtd->entityValuePool); + declEntity->textLen = (int)(poolLength(&dtd->entityValuePool)); poolFinish(&dtd->entityValuePool); if (entityDeclHandler) { *eventEndPP = s; @@ -4496,7 +4575,7 @@ if (handleDefault && defaultHandler) reportDefault(parser, enc, s, next); - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -4527,7 +4606,7 @@ case -XML_TOK_PROLOG_S: if (defaultHandler) { reportDefault(parser, encoding, s, next); - if (parsing == XML_FINISHED) + if (ps_parsing == XML_FINISHED) return XML_ERROR_ABORTED; } *nextPtr = next; @@ -4551,13 +4630,13 @@ eventPtr = next; return XML_ERROR_INVALID_TOKEN; case XML_TOK_PARTIAL: - if (!finalBuffer) { + if (!ps_finalBuffer) { *nextPtr = s; return XML_ERROR_NONE; } return XML_ERROR_UNCLOSED_TOKEN; case XML_TOK_PARTIAL_CHAR: - if (!finalBuffer) { + if (!ps_finalBuffer) { *nextPtr = s; return XML_ERROR_NONE; } @@ -4566,7 +4645,7 @@ return XML_ERROR_JUNK_AFTER_DOC_ELEMENT; } eventPtr = s = next; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -4619,8 +4698,8 @@ textEnd, &next, XML_FALSE); if (result == XML_ERROR_NONE) { - if (textEnd != next && parsing == XML_SUSPENDED) { - entity->processed = next - textStart; + if (textEnd != next && ps_parsing == XML_SUSPENDED) { + entity->processed = (int)(next - textStart); processor = internalEntityProcessor; } else { @@ -4665,8 +4744,8 @@ if (result != XML_ERROR_NONE) return result; - else if (textEnd != next && parsing == XML_SUSPENDED) { - entity->processed = next - (char *)entity->textPtr; + else if (textEnd != next && ps_parsing == XML_SUSPENDED) { + entity->processed = (int)(next - (char *)entity->textPtr); return result; } else { @@ -4683,7 +4762,7 @@ processor = prologProcessor; tok = XmlPrologTok(encoding, s, end, &next); return doProlog(parser, encoding, s, end, tok, next, nextPtr, - (XML_Bool)!finalBuffer); + (XML_Bool)!ps_finalBuffer); } else #endif /* XML_DTD */ @@ -4691,7 +4770,7 @@ processor = contentProcessor; /* see externalEntityContentProcessor vs contentProcessor */ return doContent(parser, parentParser ? 1 : 0, encoding, s, end, - nextPtr, (XML_Bool)!finalBuffer); + nextPtr, (XML_Bool)!ps_finalBuffer); } } @@ -4800,9 +4879,8 @@ return XML_ERROR_NO_MEMORY; entity = (ENTITY *)lookup(&dtd->generalEntities, name, 0); poolDiscard(&temp2Pool); - /* first, determine if a check for an existing declaration is needed; - if yes, check that the entity exists, and that it is internal, - otherwise call the default handler (if called from content) + /* First, determine if a check for an existing declaration is needed; + if yes, check that the entity exists, and that it is internal. */ if (pool == &dtd->pool) /* are we called from prolog? */ checkEntityDecl = @@ -4821,13 +4899,16 @@ return XML_ERROR_ENTITY_DECLARED_IN_PE; } else if (!entity) { - /* cannot report skipped entity here - see comments on - skippedEntityHandler + /* Cannot report skipped entity here - see comments on + skippedEntityHandler. if (skippedEntityHandler) skippedEntityHandler(handlerArg, name, 0); */ + /* Cannot call the default handler because this would be + out of sync with the call to the startElementHandler. if ((pool == &tempPool) && defaultHandler) reportDefault(parser, enc, ptr, next); + */ break; } if (entity->open) { @@ -5127,12 +5208,12 @@ ICHAR *dataPtr = (ICHAR *)dataBuf; XmlConvert(enc, &s, end, &dataPtr, (ICHAR *)dataBufEnd); *eventEndPP = s; - defaultHandler(handlerArg, dataBuf, dataPtr - (ICHAR *)dataBuf); + defaultHandler(handlerArg, dataBuf, (int)(dataPtr - (ICHAR *)dataBuf)); *eventPP = s; } while (s != end); } else - defaultHandler(handlerArg, (XML_Char *)s, (XML_Char *)end - (XML_Char *)s); + defaultHandler(handlerArg, (XML_Char *)s, (int)((XML_Char *)end - (XML_Char *)s)); } @@ -5286,7 +5367,7 @@ if (!poolAppendChar(&tempPool, XML_T('='))) return NULL; len = dtd->defaultPrefix.binding->uriLen; - if (namespaceSeparator != XML_T('\0')) + if (namespaceSeparator) len--; for (i = 0; i < len; i++) if (!poolAppendChar(&tempPool, dtd->defaultPrefix.binding->uri[i])) @@ -5312,7 +5393,7 @@ if (!poolAppendChar(&tempPool, XML_T('='))) return NULL; len = prefix->binding->uriLen; - if (namespaceSeparator != XML_T('\0')) + if (namespaceSeparator) len--; for (i = 0; i < len; i++) if (!poolAppendChar(&tempPool, prefix->binding->uri[i])) @@ -6014,7 +6095,7 @@ } } if (pool->blocks && pool->start == pool->blocks->s) { - int blockSize = (pool->end - pool->start)*2; + int blockSize = (int)(pool->end - pool->start)*2; pool->blocks = (BLOCK *) pool->mem->realloc_fcn(pool->blocks, (offsetof(BLOCK, s) @@ -6028,7 +6109,7 @@ } else { BLOCK *tem; - int blockSize = pool->end - pool->start; + int blockSize = (int)(pool->end - pool->start); if (blockSize < INIT_BLOCK_SIZE) blockSize = INIT_BLOCK_SIZE; else Modified: python/trunk/Modules/expat/xmlrole.c ============================================================================== --- python/trunk/Modules/expat/xmlrole.c (original) +++ python/trunk/Modules/expat/xmlrole.c Tue Jun 20 01:21:25 2006 @@ -6,6 +6,8 @@ #include "winconfig.h" #elif defined(MACOS_CLASSIC) #include "macconfig.h" +#elif defined(__amigaos4__) +#include "amigaconfig.h" #else #ifdef HAVE_EXPAT_CONFIG_H #include @@ -793,7 +795,7 @@ return XML_ROLE_ATTLIST_NONE; case XML_TOK_NAME: { - static const char *types[] = { + static const char * const types[] = { KW_CDATA, KW_ID, KW_IDREF, Modified: python/trunk/Modules/expat/xmltok.c ============================================================================== --- python/trunk/Modules/expat/xmltok.c (original) +++ python/trunk/Modules/expat/xmltok.c Tue Jun 20 01:21:25 2006 @@ -6,6 +6,8 @@ #include "winconfig.h" #elif defined(MACOS_CLASSIC) #include "macconfig.h" +#elif defined(__amigaos4__) +#include "amigaconfig.h" #else #ifdef HAVE_EXPAT_CONFIG_H #include @@ -1451,7 +1453,7 @@ static int FASTCALL getEncodingIndex(const char *name) { - static const char *encodingNames[] = { + static const char * const encodingNames[] = { KW_ISO_8859_1, KW_US_ASCII, KW_UTF_8, @@ -1484,7 +1486,7 @@ static int -initScan(const ENCODING **encodingTable, +initScan(const ENCODING * const *encodingTable, const INIT_ENCODING *enc, int state, const char *ptr, Modified: python/trunk/Modules/expat/xmltok.h ============================================================================== --- python/trunk/Modules/expat/xmltok.h (original) +++ python/trunk/Modules/expat/xmltok.h Tue Jun 20 01:21:25 2006 @@ -111,8 +111,8 @@ typedef struct position { /* first line and first column are 0 not 1 */ - unsigned long lineNumber; - unsigned long columnNumber; + XML_Size lineNumber; + XML_Size columnNumber; } POSITION; typedef struct { Modified: python/trunk/Modules/expat/xmltok_impl.c ============================================================================== --- python/trunk/Modules/expat/xmltok_impl.c (original) +++ python/trunk/Modules/expat/xmltok_impl.c Tue Jun 20 01:21:25 2006 @@ -1714,7 +1714,7 @@ ptr += MINBPC(enc); break; default: - return ptr - start; + return (int)(ptr - start); } } } @@ -1750,7 +1750,7 @@ LEAD_CASE(2) LEAD_CASE(3) LEAD_CASE(4) #undef LEAD_CASE case BT_LF: - pos->columnNumber = (unsigned)-1; + pos->columnNumber = (XML_Size)-1; pos->lineNumber++; ptr += MINBPC(enc); break; @@ -1759,7 +1759,7 @@ ptr += MINBPC(enc); if (ptr != end && BYTE_TYPE(enc, ptr) == BT_LF) ptr += MINBPC(enc); - pos->columnNumber = (unsigned)-1; + pos->columnNumber = (XML_Size)-1; break; default: ptr += MINBPC(enc); Modified: python/trunk/Modules/expat/xmltok_ns.c ============================================================================== --- python/trunk/Modules/expat/xmltok_ns.c (original) +++ python/trunk/Modules/expat/xmltok_ns.c Tue Jun 20 01:21:25 2006 @@ -19,7 +19,7 @@ #endif } -static const ENCODING *NS(encodings)[] = { +static const ENCODING * const NS(encodings)[] = { &ns(latin1_encoding).enc, &ns(ascii_encoding).enc, &ns(utf8_encoding).enc, From python-checkins at python.org Tue Jun 20 01:57:42 2006 From: python-checkins at python.org (trent.mick) Date: Tue, 20 Jun 2006 01:57:42 +0200 (CEST) Subject: [Python-checkins] r47034 - python/trunk/Modules/expat/expat_external.h python/trunk/Modules/expat/pyexpatns.h Message-ID: <20060619235742.76E531E4002@bag.python.org> Author: trent.mick Date: Tue Jun 20 01:57:41 2006 New Revision: 47034 Added: python/trunk/Modules/expat/pyexpatns.h (contents, props changed) Modified: python/trunk/Modules/expat/expat_external.h Log: [ 1295808 ] expat symbols should be namespaced in pyexpat (http://python.org/sf/1295808) Modified: python/trunk/Modules/expat/expat_external.h ============================================================================== --- python/trunk/Modules/expat/expat_external.h (original) +++ python/trunk/Modules/expat/expat_external.h Tue Jun 20 01:57:41 2006 @@ -7,6 +7,10 @@ /* External API definitions */ +/* Namespace external symbols to allow multiple libexpat version to + co-exist. */ +#include "pyexpatns.h" + #if defined(_MSC_EXTENSIONS) && !defined(__BEOS__) && !defined(__CYGWIN__) #define XML_USE_MSC_EXTENSIONS 1 #endif Added: python/trunk/Modules/expat/pyexpatns.h ============================================================================== --- (empty file) +++ python/trunk/Modules/expat/pyexpatns.h Tue Jun 20 01:57:41 2006 @@ -0,0 +1,124 @@ +/* Copyright (c) 2005-2006 ActiveState Software Inc. + * + * Namespace all expat exported symbols to avoid dynamic loading symbol + * collisions when embedding Python. + * + * The Problem: + * - you embed Python in some app + * - the app dynamically loads libexpat of version X + * - the embedded Python imports pyexpat (which was built against + * libexpat version X+n) + * --> pyexpat gets the expat symbols from the already loaded and *older* + * libexpat: crash (Specifically the crash we observed was in + * getting an old XML_ErrorString (from xmlparse.c) and then calling + * it with newer values in the XML_Error enum: + * + * // pyexpat.c, line 1970 + * ... + * // Added in Expat 1.95.7. + * MYCONST(XML_ERROR_UNBOUND_PREFIX); + * ... + * + * + * The Solution: + * Prefix all a exported symbols with "PyExpat_". This is similar to + * what Mozilla does for some common libs: + * http://lxr.mozilla.org/seamonkey/source/modules/libimg/png/mozpngconf.h#115 + * + * The list of relevant exported symbols can be had with this command: + * + nm pyexpat.so \ + | grep -v " [a-zBUA] " \ + | grep -v "_fini\|_init\|initpyexpat" + * + * If any of those symbols are NOT prefixed with "PyExpat_" then + * a #define should be added for it here. + */ + +#ifndef PYEXPATNS_H +#define PYEXPATNS_H + +#define XML_DefaultCurrent PyExpat_XML_DefaultCurrent +#define XML_ErrorString PyExpat_XML_ErrorString +#define XML_ExpatVersion PyExpat_XML_ExpatVersion +#define XML_ExpatVersionInfo PyExpat_XML_ExpatVersionInfo +#define XML_ExternalEntityParserCreate PyExpat_XML_ExternalEntityParserCreate +#define XML_FreeContentModel PyExpat_XML_FreeContentModel +#define XML_GetBase PyExpat_XML_GetBase +#define XML_GetBuffer PyExpat_XML_GetBuffer +#define XML_GetCurrentByteCount PyExpat_XML_GetCurrentByteCount +#define XML_GetCurrentByteIndex PyExpat_XML_GetCurrentByteIndex +#define XML_GetCurrentColumnNumber PyExpat_XML_GetCurrentColumnNumber +#define XML_GetCurrentLineNumber PyExpat_XML_GetCurrentLineNumber +#define XML_GetErrorCode PyExpat_XML_GetErrorCode +#define XML_GetFeatureList PyExpat_XML_GetFeatureList +#define XML_GetIdAttributeIndex PyExpat_XML_GetIdAttributeIndex +#define XML_GetInputContext PyExpat_XML_GetInputContext +#define XML_GetParsingStatus PyExpat_XML_GetParsingStatus +#define XML_GetSpecifiedAttributeCount PyExpat_XML_GetSpecifiedAttributeCount +#define XmlGetUtf16InternalEncoding PyExpat_XmlGetUtf16InternalEncoding +#define XmlGetUtf16InternalEncodingNS PyExpat_XmlGetUtf16InternalEncodingNS +#define XmlGetUtf8InternalEncoding PyExpat_XmlGetUtf8InternalEncoding +#define XmlGetUtf8InternalEncodingNS PyExpat_XmlGetUtf8InternalEncodingNS +#define XmlInitEncoding PyExpat_XmlInitEncoding +#define XmlInitEncodingNS PyExpat_XmlInitEncodingNS +#define XmlInitUnknownEncoding PyExpat_XmlInitUnknownEncoding +#define XmlInitUnknownEncodingNS PyExpat_XmlInitUnknownEncodingNS +#define XML_MemFree PyExpat_XML_MemFree +#define XML_MemMalloc PyExpat_XML_MemMalloc +#define XML_MemRealloc PyExpat_XML_MemRealloc +#define XML_Parse PyExpat_XML_Parse +#define XML_ParseBuffer PyExpat_XML_ParseBuffer +#define XML_ParserCreate PyExpat_XML_ParserCreate +#define XML_ParserCreate_MM PyExpat_XML_ParserCreate_MM +#define XML_ParserCreateNS PyExpat_XML_ParserCreateNS +#define XML_ParserFree PyExpat_XML_ParserFree +#define XML_ParserReset PyExpat_XML_ParserReset +#define XmlParseXmlDecl PyExpat_XmlParseXmlDecl +#define XmlParseXmlDeclNS PyExpat_XmlParseXmlDeclNS +#define XmlPrologStateInit PyExpat_XmlPrologStateInit +#define XmlPrologStateInitExternalEntity PyExpat_XmlPrologStateInitExternalEntity +#define XML_ResumeParser PyExpat_XML_ResumeParser +#define XML_SetAttlistDeclHandler PyExpat_XML_SetAttlistDeclHandler +#define XML_SetBase PyExpat_XML_SetBase +#define XML_SetCdataSectionHandler PyExpat_XML_SetCdataSectionHandler +#define XML_SetCharacterDataHandler PyExpat_XML_SetCharacterDataHandler +#define XML_SetCommentHandler PyExpat_XML_SetCommentHandler +#define XML_SetDefaultHandler PyExpat_XML_SetDefaultHandler +#define XML_SetDefaultHandlerExpand PyExpat_XML_SetDefaultHandlerExpand +#define XML_SetDoctypeDeclHandler PyExpat_XML_SetDoctypeDeclHandler +#define XML_SetElementDeclHandler PyExpat_XML_SetElementDeclHandler +#define XML_SetElementHandler PyExpat_XML_SetElementHandler +#define XML_SetEncoding PyExpat_XML_SetEncoding +#define XML_SetEndCdataSectionHandler PyExpat_XML_SetEndCdataSectionHandler +#define XML_SetEndDoctypeDeclHandler PyExpat_XML_SetEndDoctypeDeclHandler +#define XML_SetEndElementHandler PyExpat_XML_SetEndElementHandler +#define XML_SetEndNamespaceDeclHandler PyExpat_XML_SetEndNamespaceDeclHandler +#define XML_SetEntityDeclHandler PyExpat_XML_SetEntityDeclHandler +#define XML_SetExternalEntityRefHandler PyExpat_XML_SetExternalEntityRefHandler +#define XML_SetExternalEntityRefHandlerArg PyExpat_XML_SetExternalEntityRefHandlerArg +#define XML_SetNamespaceDeclHandler PyExpat_XML_SetNamespaceDeclHandler +#define XML_SetNotationDeclHandler PyExpat_XML_SetNotationDeclHandler +#define XML_SetNotStandaloneHandler PyExpat_XML_SetNotStandaloneHandler +#define XML_SetParamEntityParsing PyExpat_XML_SetParamEntityParsing +#define XML_SetProcessingInstructionHandler PyExpat_XML_SetProcessingInstructionHandler +#define XML_SetReturnNSTriplet PyExpat_XML_SetReturnNSTriplet +#define XML_SetSkippedEntityHandler PyExpat_XML_SetSkippedEntityHandler +#define XML_SetStartCdataSectionHandler PyExpat_XML_SetStartCdataSectionHandler +#define XML_SetStartDoctypeDeclHandler PyExpat_XML_SetStartDoctypeDeclHandler +#define XML_SetStartElementHandler PyExpat_XML_SetStartElementHandler +#define XML_SetStartNamespaceDeclHandler PyExpat_XML_SetStartNamespaceDeclHandler +#define XML_SetUnknownEncodingHandler PyExpat_XML_SetUnknownEncodingHandler +#define XML_SetUnparsedEntityDeclHandler PyExpat_XML_SetUnparsedEntityDeclHandler +#define XML_SetUserData PyExpat_XML_SetUserData +#define XML_SetXmlDeclHandler PyExpat_XML_SetXmlDeclHandler +#define XmlSizeOfUnknownEncoding PyExpat_XmlSizeOfUnknownEncoding +#define XML_StopParser PyExpat_XML_StopParser +#define XML_UseForeignDTD PyExpat_XML_UseForeignDTD +#define XML_UseParserAsHandlerArg PyExpat_XML_UseParserAsHandlerArg +#define XmlUtf16Encode PyExpat_XmlUtf16Encode +#define XmlUtf8Encode PyExpat_XmlUtf8Encode + + +#endif /* !PYEXPATNS_H */ + From buildbot at python.org Tue Jun 20 02:36:34 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 00:36:34 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060620003634.A46371E4002@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/808 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: trent.mick Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 20 02:51:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 00:51:20 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060620005120.3822D1E4007@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/210 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: trent.mick Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 20 03:16:52 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 01:16:52 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060620011652.CFA5F1E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/391 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ka-ping.yee Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 20 04:14:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 02:14:36 +0000 Subject: [Python-checkins] buildbot failure in x86 XP-2 2.4 Message-ID: <20060620021436.83B6B1E4002@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-2%25202.4/builds/139 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'tim': just making sure this still works Build Source Stamp: [branch release24-maint] HEAD Blamelist: BUILD FAILED: failed svn sincerely, -The Buildbot From python-checkins at python.org Tue Jun 20 04:50:19 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 20 Jun 2006 04:50:19 +0200 (CEST) Subject: [Python-checkins] r47035 - peps/trunk/pep-0360.txt Message-ID: <20060620025019.802D11E400B@bag.python.org> Author: brett.cannon Date: Tue Jun 20 04:50:18 2006 New Revision: 47035 Modified: peps/trunk/pep-0360.txt Log: Remove pysqlite; main development now in Python's repository. Modified: peps/trunk/pep-0360.txt ============================================================================== --- peps/trunk/pep-0360.txt (original) +++ peps/trunk/pep-0360.txt Tue Jun 20 04:50:18 2006 @@ -90,23 +90,6 @@ * 1.4 (2.3) -pysqlite --------- - -:Web site: - http://www.sqlite.org/ -:Standard library name: - sqlite3 -:Contact person: - Gerhard H?ring -:Synchronisation history: - * 2.2.2 (2.5) - -Bugs should be reported to the pysqlite bug -tracker [#pysqlite-tracker]_ as well as any patches that are not -deemed critical. - - wsgiref ------- :Web site: @@ -128,9 +111,6 @@ .. [#python-tracker] Python tracker (http://sourceforge.net/tracker/?group_id=5470) -.. [#pysqlite-tracker] pysqlite tracker - (http://pysqlite.org/) - .. [#web-sig] Web-SIG mailing list (http://mail.python.org/mailman/listinfo/web-sig) From python-checkins at python.org Tue Jun 20 04:53:57 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 20 Jun 2006 04:53:57 +0200 (CEST) Subject: [Python-checkins] r47036 - peps/trunk/pep-0360.txt Message-ID: <20060620025357.028601E4002@bag.python.org> Author: brett.cannon Date: Tue Jun 20 04:53:56 2006 New Revision: 47036 Modified: peps/trunk/pep-0360.txt Log: Update expat to 2.0 . Modified: peps/trunk/pep-0360.txt ============================================================================== --- peps/trunk/pep-0360.txt (original) +++ peps/trunk/pep-0360.txt Tue Jun 20 04:53:56 2006 @@ -73,6 +73,7 @@ :Synchronisation history: * 1.95.8 (2.4) * 1.95.7 (2.3) + * 2.0 (2.5) Optik From python-checkins at python.org Tue Jun 20 04:59:36 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 20 Jun 2006 04:59:36 +0200 (CEST) Subject: [Python-checkins] r47037 - peps/trunk/pep-0360.txt Message-ID: <20060620025936.D02291E4002@bag.python.org> Author: brett.cannon Date: Tue Jun 20 04:59:36 2006 New Revision: 47037 Modified: peps/trunk/pep-0360.txt Log: Mention how bug fixes will be committed directly to the repository without worrying about any special notes for the projects. Modified: peps/trunk/pep-0360.txt ============================================================================== --- peps/trunk/pep-0360.txt (original) +++ peps/trunk/pep-0360.txt Tue Jun 20 04:59:36 2006 @@ -26,7 +26,13 @@ This PEP is meant to record details of packages in the stdlib that are maintained outside of Python's repository. Specifically, it is meant to keep track of any specific maintenance needs for each package. It -also is meant to allow people to know which version of a package is +should be mentioned that changes needed in order to fix bugs and keep +the code running on all of Python's supported platforms will be done +directly in Python's repository without worrying about going through +the contact developer. This is so that Python itself is not held up +by a single bug and allows the whole process to scale as needed. + +It also is meant to allow people to know which version of a package is released with which version of Python. From python-checkins at python.org Tue Jun 20 05:45:09 2006 From: python-checkins at python.org (anthony.baxter) Date: Tue, 20 Jun 2006 05:45:09 +0200 (CEST) Subject: [Python-checkins] r47038 - python/tags/r25b1 Message-ID: <20060620034509.663C21E4002@bag.python.org> Author: anthony.baxter Date: Tue Jun 20 05:45:08 2006 New Revision: 47038 Added: python/tags/r25b1/ - copied from r47037, python/trunk/ Log: Tagging for release of Python 2.5b1 From neal at metaslash.com Tue Jun 20 11:08:10 2006 From: neal at metaslash.com (Neal Norwitz) Date: Tue, 20 Jun 2006 05:08:10 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060620090810.GA32737@python.psfb.org> test_socket leaked [0, 0, 205] references From python-checkins at python.org Tue Jun 20 13:52:17 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 20 Jun 2006 13:52:17 +0200 (CEST) Subject: [Python-checkins] r47039 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060620115217.1C7921E4002@bag.python.org> Author: andrew.kuchling Date: Tue Jun 20 13:52:16 2006 New Revision: 47039 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Uncomment wsgiref section Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 20 13:52:16 2006 @@ -2074,10 +2074,9 @@ %====================================================================== -%\subsection{The wsgiref package\label{module-wsgiref}} +\subsection{The wsgiref package\label{module-wsgiref}} % XXX should this be in a PEP 333 section instead? -\begin{comment} The Web Server Gateway Interface (WSGI) v1.0 defines a standard interface between web servers and Python web applications and is @@ -2086,10 +2085,7 @@ The package includes a basic HTTP server that will run a WSGI application; this server is useful for debugging but isn't intended for -production use. - -% XXX structure of WSGI applications? -% XXX provide an example using Django or some other framework? +production use. Setting up a server takes only a few lines of code: \begin{verbatim} from wsgiref import simple_server @@ -2102,17 +2098,19 @@ httpd.serve_forever() \end{verbatim} +% XXX discuss structure of WSGI applications? +% XXX provide an example using Django or some other framework? \begin{seealso} +\seeurl{http://www.wsgi.org}{A central web site for WSGI-related resources.} + \seepep{333}{Python Web Server Gateway Interface v1.0}{PEP written by Phillip J. Eby.} \end{seealso} -\end{comment} - % ====================================================================== \section{Build and C API Changes\label{build-api}} From python-checkins at python.org Tue Jun 20 14:15:09 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 20 Jun 2006 14:15:09 +0200 (CEST) Subject: [Python-checkins] r47040 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060620121509.9E8A41E4002@bag.python.org> Author: andrew.kuchling Date: Tue Jun 20 14:15:09 2006 New Revision: 47040 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Add four library items Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 20 14:15:09 2006 @@ -1384,6 +1384,12 @@ code snippets that are usage examples intended for the reader and aren't actually test cases. +An \var{encoding} parameter was added to the \function{testfile()} +function and the \class{DocFileSuite} class to specify the file's +encoding. This makes it easier to use non-ASCII characters in +tests contained within a docstring. (Contributed by Bjorn Tillenius.) +% Patch 1080727 + \item The \module{fileinput} module was made more flexible. Unicode filenames are now supported, and a \var{mode} parameter that defaults to \code{"r"} was added to the @@ -1540,6 +1546,9 @@ performing many different operations and reducing the result to a single number as \file{pystone.py} does. +\item The \module{pyexpat} module now uses version 2.0 of the Expat parser. +(Contributed by Trent Mick.) + \item The old \module{regex} and \module{regsub} modules, which have been deprecated ever since Python 2.0, have finally been deleted. Other deleted modules: \module{statcache}, \module{tzparse}, @@ -1682,6 +1691,14 @@ (Contributed by Ka-Ping Yee.) +\item The \module{weakref} module's \class{WeakKeyDictionary} and +\class{WeakValueDictionary} types gained new methods for iterating +over the weak references contained in the dictionary. +\method{iterkeyrefs()} and \method{keyrefs()} methods were +added to \class{WeakKeyDictionary}, and +\method{itervaluerefs()} and \method{valuerefs()} were added to +\class{WeakValueDictionary}. (Contributed by Fred L.~Drake, Jr.) + \item The \module{webbrowser} module received a number of enhancements. It's now usable as a script with \code{python -m webbrowser}, taking a @@ -1705,6 +1722,12 @@ (Contributed by Skip Montanaro.) % Patch 1120353 +\item The \module{zipfile} module now supports the ZIP64 version of the +format, meaning that a .zip archive can now be larger than 4 GiB and +can contain individual files larger than 4 GiB. (Contributed by +Ronald Oussoren.) +% Patch 1446489 + \item The \module{zlib} module's \class{Compress} and \class{Decompress} objects now support a \method{copy()} method that makes a copy of the object's internal state and returns a new From python-checkins at python.org Tue Jun 20 14:19:54 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 20 Jun 2006 14:19:54 +0200 (CEST) Subject: [Python-checkins] r47041 - python/trunk/Doc/whatsnew/whatsnew20.tex python/trunk/Doc/whatsnew/whatsnew23.tex python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060620121954.E60091E4002@bag.python.org> Author: andrew.kuchling Date: Tue Jun 20 14:19:54 2006 New Revision: 47041 Modified: python/trunk/Doc/whatsnew/whatsnew20.tex python/trunk/Doc/whatsnew/whatsnew23.tex python/trunk/Doc/whatsnew/whatsnew25.tex Log: Terminology and typography fixes Modified: python/trunk/Doc/whatsnew/whatsnew20.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew20.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew20.tex Tue Jun 20 14:19:54 2006 @@ -777,7 +777,7 @@ Some work has been done to make integers and long integers a bit more interchangeable. In 1.5.2, large-file support was added for Solaris, -to allow reading files larger than 2Gb; this made the \method{tell()} +to allow reading files larger than 2~GiB; this made the \method{tell()} method of file objects return a long integer instead of a regular integer. Some code would subtract two file offsets and attempt to use the result to multiply a sequence or slice a string, but this raised a Modified: python/trunk/Doc/whatsnew/whatsnew23.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew23.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew23.tex Tue Jun 20 14:19:54 2006 @@ -1479,7 +1479,7 @@ ('amk', 500) \end{verbatim} -\item The \module{gzip} module can now handle files exceeding 2~Gb. +\item The \module{gzip} module can now handle files exceeding 2~GiB. \item The new \module{heapq} module contains an implementation of a heap queue algorithm. A heap is an array-like data structure that Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 20 14:19:54 2006 @@ -960,7 +960,7 @@ already more bytes than a 32-bit address space can contain. It's possible to address that much memory on a 64-bit platform, -however. The pointers for a list that size would only require 16GiB +however. The pointers for a list that size would only require 16~GiB of space, so it's not unreasonable that Python programmers might construct lists that large. Therefore, the Python interpreter had to be changed to use some type other than \ctype{int}, and this will be a @@ -1723,8 +1723,8 @@ % Patch 1120353 \item The \module{zipfile} module now supports the ZIP64 version of the -format, meaning that a .zip archive can now be larger than 4 GiB and -can contain individual files larger than 4 GiB. (Contributed by +format, meaning that a .zip archive can now be larger than 4~GiB and +can contain individual files larger than 4~GiB. (Contributed by Ronald Oussoren.) % Patch 1446489 From buildbot at python.org Tue Jun 20 14:37:08 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 12:37:08 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060620123708.E20291E4004@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/701 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 20 15:05:13 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 20 Jun 2006 15:05:13 +0200 (CEST) Subject: [Python-checkins] r47042 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060620130513.455281E4004@bag.python.org> Author: andrew.kuchling Date: Tue Jun 20 15:05:12 2006 New Revision: 47042 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Add introductory paragraphs summarizing the release; minor edits Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 20 15:05:12 2006 @@ -15,18 +15,41 @@ \maketitle \tableofcontents -This article explains the new features in Python 2.5. No release date -for Python 2.5 has been set; it will probably be released in the -autumn of 2006. \pep{356} describes the planned release schedule. +This article explains the new features in Python 2.5. The final +release of Python 2.5 is scheduled for August 2006; +\pep{356} describes the planned release schedule. Comments, suggestions, and error reports are welcome; please e-mail them to the author or open a bug in the Python bug tracker. % XXX Compare with previous release in 2 - 3 sentences here. - -This article doesn't attempt to provide a complete specification of -the new features, but instead provides a convenient overview. For -full details, you should refer to the documentation for Python 2.5. +The changes in Python 2.5 are an interesting mix of language and library +changes. The library changes +will be more important to Python's user community, I think, +because several widely-useful packages were added to the standard library; +the additions include +ElementTree for XML processing (section~\ref{module-etree}), +the SQLite database module (section~\ref{module-sqlite}), +and the \module{ctypes} module for calling C functions (\section~\ref{module-ctypes}). + +The language changes are of middling significance. Some pleasant new +features were added, but most of them aren't features that you'll use +every day. Conditional expressions were finally added to the language +using a novel syntax; see section~\ref{pep-308}. The new +'\keyword{with}' statement will make writing cleanup code easier +(section~\ref{pep-343}). Values can now be passed into generators +(section~\ref{pep-342}). Imports are now visible as either absolute +or relative (section~\ref{pep-328}). Some corner cases of exception +handling are handled better (section~\ref{pep-341}). All these +improvements are worthwhile, but they're improvements to one specific +language feature or another; none of them are broad modifications to +Python's semantics. + + +This article doesn't attempt to be a complete specification of the new +features, but instead is a brief introduction to each new feature. +For full details, you should refer to the documentation for Python +2.5. % XXX add hyperlink when the documentation becomes available online. If you want to understand the complete implementation and design rationale, refer to the PEP for a particular new feature. @@ -36,10 +59,10 @@ \section{PEP 308: Conditional Expressions\label{pep-308}} For a long time, people have been requesting a way to write -conditional expressions, expressions that return value A or value B -depending on whether a Boolean value is true or false. A conditional -expression lets you write a single assignment statement that has the -same effect as the following: +conditional expressions, which are expressions that return value A or +value B depending on whether a Boolean value is true or false. A +conditional expression lets you write a single assignment statement +that has the same effect as the following: \begin{verbatim} if condition: From python-checkins at python.org Tue Jun 20 15:11:29 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 20 Jun 2006 15:11:29 +0200 (CEST) Subject: [Python-checkins] r47043 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060620131129.A117E1E4013@bag.python.org> Author: andrew.kuchling Date: Tue Jun 20 15:11:29 2006 New Revision: 47043 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Minor edits and rearrangements; markup fix Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 20 15:11:29 2006 @@ -2,7 +2,6 @@ \usepackage{distutils} % $Id$ -% wsgiref section % Fix XXX comments % Count up the patches and bugs @@ -19,18 +18,13 @@ release of Python 2.5 is scheduled for August 2006; \pep{356} describes the planned release schedule. -Comments, suggestions, and error reports are welcome; please e-mail them -to the author or open a bug in the Python bug tracker. - -% XXX Compare with previous release in 2 - 3 sentences here. -The changes in Python 2.5 are an interesting mix of language and library -changes. The library changes -will be more important to Python's user community, I think, -because several widely-useful packages were added to the standard library; -the additions include -ElementTree for XML processing (section~\ref{module-etree}), -the SQLite database module (section~\ref{module-sqlite}), -and the \module{ctypes} module for calling C functions (\section~\ref{module-ctypes}). +The changes in Python 2.5 are an interesting mix of language and +library improvements. The library enhancements will be more important +to Python's user community, I think, because several widely-useful +packages were added. New modules include ElementTree for XML +processing (section~\ref{module-etree}), the SQLite database module +(section~\ref{module-sqlite}), and the \module{ctypes} module for +calling C functions (section~\ref{module-ctypes}). The language changes are of middling significance. Some pleasant new features were added, but most of them aren't features that you'll use @@ -45,15 +39,17 @@ language feature or another; none of them are broad modifications to Python's semantics. - -This article doesn't attempt to be a complete specification of the new -features, but instead is a brief introduction to each new feature. -For full details, you should refer to the documentation for Python -2.5. +This article doesn't try to be a complete specification of the new +features; instead changes are briefly introduced using helpful +examples. For full details, you should always refer to the +documentation for Python 2.5. % XXX add hyperlink when the documentation becomes available online. If you want to understand the complete implementation and design rationale, refer to the PEP for a particular new feature. +Comments, suggestions, and error reports for this document are +welcome; please e-mail them to the author or open a bug in the Python +bug tracker. %====================================================================== \section{PEP 308: Conditional Expressions\label{pep-308}} From python-checkins at python.org Tue Jun 20 15:20:30 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 20 Jun 2006 15:20:30 +0200 (CEST) Subject: [Python-checkins] r47044 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060620132030.BB6CA1E400A@bag.python.org> Author: andrew.kuchling Date: Tue Jun 20 15:20:30 2006 New Revision: 47044 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: [Bug #1504456] Mention xml -> xmlcore change Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Tue Jun 20 15:20:30 2006 @@ -1733,6 +1733,11 @@ Brandl.) % Patch #754022 +\item Python's standard library no longer includes +a package named \module{xml}; the library's XML-related package +has been renamed to \module{xmlcore}. This means +it will always be possible to import the standard library's +XML support whether or not the PyXML package is installed. \item The \module{xmlrpclib} module now supports returning \class{datetime} objects for the XML-RPC date type. Supply @@ -2348,6 +2353,10 @@ \member{rpc_paths} to \code{None} or an empty tuple disables this path checking. +\item Library: the \module{xml} package has been renamed to \module{xmlcore}. +The PyXML package will therefore be \module{xml}, and the Python +distribution's code will always be accessible as \module{xmlcore}. + \item C API: Many functions now use \ctype{Py_ssize_t} instead of \ctype{int} to allow processing more data on 64-bit machines. Extension code may need to make the same change to avoid From buildbot at python.org Tue Jun 20 16:47:46 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 14:47:46 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060620144746.E7C5F1E4004@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/434 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 20 16:57:59 2006 From: python-checkins at python.org (matt.fleming) Date: Tue, 20 Jun 2006 16:57:59 +0200 (CEST) Subject: [Python-checkins] r47045 - in sandbox/trunk/pdb: README.txt mconnection.py mpdb.py test/Makefile test/mpdbtest.py Message-ID: <20060620145759.49AC01E4004@bag.python.org> Author: matt.fleming Date: Tue Jun 20 16:57:58 2006 New Revision: 47045 Added: sandbox/trunk/pdb/test/Makefile sandbox/trunk/pdb/test/mpdbtest.py Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Log: Added a unit test for the pdbserver and target commands, and a Makefile to make it easier to run the tests. Use mpdb.msg() for all output in mpdb.py now. Added some items to the TODO list. Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Tue Jun 20 16:57:58 2006 @@ -9,7 +9,7 @@ aims to correct this wish. -=[TODO]=- -* Write more unit tests +* Write more unit tests, test the pdbserver and target commands. * sort out the namespace corruption - """ c:\soc\pdb\test.py(3)x() @@ -17,5 +17,7 @@ (MPdb)p i *** NameError: """ - +* Write a signal handler that scripts can import from mpdb that, when + the signal is received, start remote debugging. +* info target,threads command needs to be written. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Tue Jun 20 16:57:58 2006 @@ -157,7 +157,6 @@ def readline(self, bufsize=2048): line = self._sock.recv(bufsize) return line - def flush(self): pass Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Tue Jun 20 16:57:58 2006 @@ -50,7 +50,6 @@ """ This method rebinds the debugger's input to the object specified by 'new_input'. """ - self.stdin.flush() self.use_rawinput = False self.stdin = new_input @@ -116,7 +115,11 @@ target serial -- Use a remote computer via a serial line target tcp -- Use a remote computer via a socket connection """ - target, addr = args.split(' ') + try: + target, addr = args.split(' ') + except ValueError: + self.msg('Invalid arguments') + return if self.target == 'remote': self.msg('Already connected to a remote machine.') return @@ -184,7 +187,11 @@ `pdbserver ConnectionClass comm scriptfile [args ...]' """ - target, comm, scriptfile_and_args = args.split(' ') + try: + target, comm, scriptfile_and_args = args.split(' ') + except ValueError: + self.msg('Invalid arguments') + return if self.target == 'remote': self.msg('Already connected remotely') return @@ -208,6 +215,8 @@ self._rebind_output(self.connection) self._rebind_input(self.connection) +# This is a mess. It's only here so that I can test other features of the +# debugger whilst I'm writing them. It will be removed at some point. def main(options): opts = options[0] args = options[1] @@ -222,21 +231,21 @@ mpdb._runscript(mainpyfile) if mpdb._user_requested_quit: break - print "The program finished and will be restarted" + self.msg("The program finished and will be restarted") except SystemExit: # In most cases SystemExit does not warrant a post-mortem session. - print "The program exited via sys.exit(). " + \ - "Exit status:",sys.exc_info()[1] + mpdb.msg("The program exited via sys.exit(). " + \ + "Exit status:",sys.exc_info()[1]) except: - print traceback.format_exc() - print "Uncaught exception. Entering post mortem debugging" - print "Running 'cont' or 'step' will restart the program" + mpdb.msg(traceback.format_exc()) + mpdb.msg("Uncaught exception. Entering post mortem debugging") + mpdb.msg("Running 'cont' or 'step' will restart the program") t = sys.exc_info()[2] while t.tb_next is not None: t = t.tb_next mpdb.interaction(t.tb_frame,t) - print "Post mortem debugger finished. The " + \ - mainpyfile + " will be restarted" + mpdb.msg("Post mortem debugger finished. The " + \ + mainpyfile + " will be restarted") # Utility functions Added: sandbox/trunk/pdb/test/Makefile ============================================================================== --- (empty file) +++ sandbox/trunk/pdb/test/Makefile Tue Jun 20 16:57:58 2006 @@ -0,0 +1,17 @@ +# Makefile for mpdb's unit tests. +# +# To run all tests type: +# +# make test +# +# or 'make target' to run a specific test, such as 'make tcptest' + +PY = python2.5 + +test: mpdbtest tcptest + +mpdbtest: + $(PY) mpdbtest.py + +tcptest: + $(PY) tcptest.py Added: sandbox/trunk/pdb/test/mpdbtest.py ============================================================================== --- (empty file) +++ sandbox/trunk/pdb/test/mpdbtest.py Tue Jun 20 16:57:58 2006 @@ -0,0 +1,43 @@ +#!/usr/bin/env python + +import sys +import socket +import thread +import threading +import unittest + +__addr__ = 'localhost:8000' +script = "" + +sys.path.append("..") +from mpdb import MPdb +from mconnection import MServerConnectionTCP, MClientConnectionTCP + +def doTargetConnect(): + global g_client + while True: + try: + g_client.do_target('tcp '+__addr__) + if CONNECTED: + break + except error: + pass + g_client.onecmd('help') + g_client.onecmd('quit') + +class TestRemoteDebugging(unittest.TestCase): + def testPdbserver(self): + global g_server, g_client, CONNECTED + g_server = MPdb() + g_client = MPdb() + + self.server_tid = thread.start_new_thread(doTargetConnect, ()) + g_server.do_pdbserver('tcp '+__addr__+' '+script) + CONNECTED = True + + # XXX mpdb needs a bottom frame before it exits + g_server.botframe = None + g_server.cmdloop() + +if __name__ == '__main__': + unittest.main() From python-checkins at python.org Tue Jun 20 18:04:34 2006 From: python-checkins at python.org (george.yoshida) Date: Tue, 20 Jun 2006 18:04:34 +0200 (CEST) Subject: [Python-checkins] r47046 - peps/trunk/pep-0291.txt Message-ID: <20060620160434.112451E4004@bag.python.org> Author: george.yoshida Date: Tue Jun 20 18:04:33 2006 New Revision: 47046 Modified: peps/trunk/pep-0291.txt Log: Undo the previous change. The compatibility comment was not meant to be a requirement for future changes and the comment itself was removed from the code. Modified: peps/trunk/pep-0291.txt ============================================================================== --- peps/trunk/pep-0291.txt (original) +++ peps/trunk/pep-0291.txt Tue Jun 20 18:04:33 2006 @@ -98,7 +98,6 @@ pybench Marc-Andre Lemburg 1.5.2 [3] sre Fredrik Lundh 2.1 subprocess Peter Astrand 2.2 - uuid Ka-Ping Yee 2.3 wsgiref Phillip J. Eby 2.1 xml (PyXML) Martin v. Loewis 2.0 xmlrpclib Fredrik Lundh 2.1 From python-checkins at python.org Tue Jun 20 19:30:27 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 20 Jun 2006 19:30:27 +0200 (CEST) Subject: [Python-checkins] r47047 - python/trunk/Lib/test/test_socket_ssl.py Message-ID: <20060620173027.778201E4004@bag.python.org> Author: brett.cannon Date: Tue Jun 20 19:30:26 2006 New Revision: 47047 Modified: python/trunk/Lib/test/test_socket_ssl.py Log: Raise TestSkipped when the test socket connection is refused. Modified: python/trunk/Lib/test/test_socket_ssl.py ============================================================================== --- python/trunk/Lib/test/test_socket_ssl.py (original) +++ python/trunk/Lib/test/test_socket_ssl.py Tue Jun 20 19:30:26 2006 @@ -3,6 +3,7 @@ import sys from test import test_support import socket +import errno # Optionally test SSL support. This requires the 'network' resource as given # on the regrtest command line. @@ -54,6 +55,12 @@ for. If this message is seen often, test_timeout should be changed to use a more reliable address.""" % (ADDR,) return + except socket.err, exc: # In case connection is refused. + if (isinstance(exc.message, tuple) and + exc.message[0] == errno.ECONNREFUSED): + raise test_support.TestSkipped("test socket connection refused") + else: + raise exc ss = socket.ssl(s) # Read part of return welcome banner twice. From martin at v.loewis.de Tue Jun 20 19:35:55 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 Jun 2006 19:35:55 +0200 Subject: [Python-checkins] r47047 - python/trunk/Lib/test/test_socket_ssl.py In-Reply-To: <20060620173027.778201E4004@bag.python.org> References: <20060620173027.778201E4004@bag.python.org> Message-ID: <449831FB.9080602@v.loewis.de> brett.cannon wrote: > Author: brett.cannon > Date: Tue Jun 20 19:30:26 2006 > New Revision: 47047 > > Modified: > python/trunk/Lib/test/test_socket_ssl.py > Log: > Raise TestSkipped when the test socket connection is refused. Is the trunk already unfrozen? Regards, Martin From nnorwitz at gmail.com Tue Jun 20 19:50:36 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 20 Jun 2006 10:50:36 -0700 Subject: [Python-checkins] r47047 - python/trunk/Lib/test/test_socket_ssl.py In-Reply-To: <20060620173027.778201E4004@bag.python.org> References: <20060620173027.778201E4004@bag.python.org> Message-ID: I thought you could only use TestSkipped for whole files. What is done in other places is print a msg that the test is begin skipped. (Not to mention Martin's point. :-) n -- On 6/20/06, brett.cannon wrote: > Author: brett.cannon > Date: Tue Jun 20 19:30:26 2006 > New Revision: 47047 > > Modified: > python/trunk/Lib/test/test_socket_ssl.py > Log: > Raise TestSkipped when the test socket connection is refused. > > > Modified: python/trunk/Lib/test/test_socket_ssl.py > ============================================================================== > --- python/trunk/Lib/test/test_socket_ssl.py (original) > +++ python/trunk/Lib/test/test_socket_ssl.py Tue Jun 20 19:30:26 2006 > @@ -3,6 +3,7 @@ > import sys > from test import test_support > import socket > +import errno > > # Optionally test SSL support. This requires the 'network' resource as given > # on the regrtest command line. > @@ -54,6 +55,12 @@ > for. If this message is seen often, test_timeout should be changed to > use a more reliable address.""" % (ADDR,) > return > + except socket.err, exc: # In case connection is refused. > + if (isinstance(exc.message, tuple) and > + exc.message[0] == errno.ECONNREFUSED): > + raise test_support.TestSkipped("test socket connection refused") > + else: > + raise exc > > ss = socket.ssl(s) > # Read part of return welcome banner twice. > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From python-checkins at python.org Tue Jun 20 20:23:43 2006 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 20 Jun 2006 20:23:43 +0200 (CEST) Subject: [Python-checkins] r47048 - sandbox/trunk/Doc/functional.rst Message-ID: <20060620182343.7AF3D1E4013@bag.python.org> Author: andrew.kuchling Date: Tue Jun 20 20:23:43 2006 New Revision: 47048 Modified: sandbox/trunk/Doc/functional.rst Log: Tweak example Modified: sandbox/trunk/Doc/functional.rst ============================================================================== --- sandbox/trunk/Doc/functional.rst (original) +++ sandbox/trunk/Doc/functional.rst Tue Jun 20 20:23:43 2006 @@ -830,10 +830,15 @@ Personally I try to avoid lambdas, favoring short nested ``def`` statements like this:: - def combine_freq (a, b): - return 0, a[1] + b[1] + def output_total_freq (items): + """Takes a list of (element, frequency count) tuples + and returns the total number of occurrences. + """ - print reduce(combine_freq, items)[1] + def combine (a, b): + return 0, a[1] + b[1] + + return reduce(combine_freq, items)[1] You might disagree that this style is better. From python-checkins at python.org Tue Jun 20 21:20:18 2006 From: python-checkins at python.org (brett.cannon) Date: Tue, 20 Jun 2006 21:20:18 +0200 (CEST) Subject: [Python-checkins] r47049 - python/trunk/Lib/test/test_socket_ssl.py Message-ID: <20060620192018.76A7D1E4009@bag.python.org> Author: brett.cannon Date: Tue Jun 20 21:20:17 2006 New Revision: 47049 Modified: python/trunk/Lib/test/test_socket_ssl.py Log: Fix typo of exception name. Modified: python/trunk/Lib/test/test_socket_ssl.py ============================================================================== --- python/trunk/Lib/test/test_socket_ssl.py (original) +++ python/trunk/Lib/test/test_socket_ssl.py Tue Jun 20 21:20:17 2006 @@ -55,7 +55,7 @@ for. If this message is seen often, test_timeout should be changed to use a more reliable address.""" % (ADDR,) return - except socket.err, exc: # In case connection is refused. + except socket.error, exc: # In case connection is refused. if (isinstance(exc.message, tuple) and exc.message[0] == errno.ECONNREFUSED): raise test_support.TestSkipped("test socket connection refused") From brett at python.org Tue Jun 20 21:31:19 2006 From: brett at python.org (Brett Cannon) Date: Tue, 20 Jun 2006 12:31:19 -0700 Subject: [Python-checkins] r47047 - python/trunk/Lib/test/test_socket_ssl.py In-Reply-To: References: <20060620173027.778201E4004@bag.python.org> Message-ID: On 6/20/06, Neal Norwitz wrote: > > I thought you could only use TestSkipped for whole files. What is > done in other places is print a msg that the test is begin skipped. I went off the docs that just say it is for "a test"; doesn't specify individual test or test file. It can obviously be changed (and this time I will wait for an all-clear on the code freeze). (Not to mention Martin's point. :-) Oops. Noticed all of Andrew's checkins for What's New and it just slipped my mind. Really sorry about that, Anthony. I can back out the change if you want me to. -Brett n > -- > > On 6/20/06, brett.cannon wrote: > > Author: brett.cannon > > Date: Tue Jun 20 19:30:26 2006 > > New Revision: 47047 > > > > Modified: > > python/trunk/Lib/test/test_socket_ssl.py > > Log: > > Raise TestSkipped when the test socket connection is refused. > > > > > > Modified: python/trunk/Lib/test/test_socket_ssl.py > > > ============================================================================== > > --- python/trunk/Lib/test/test_socket_ssl.py (original) > > +++ python/trunk/Lib/test/test_socket_ssl.py Tue Jun 20 19:30:26 2006 > > @@ -3,6 +3,7 @@ > > import sys > > from test import test_support > > import socket > > +import errno > > > > # Optionally test SSL support. This requires the 'network' resource as > given > > # on the regrtest command line. > > @@ -54,6 +55,12 @@ > > for. If this message is seen often, test_timeout should be changed > to > > use a more reliable address.""" % (ADDR,) > > return > > + except socket.err, exc: # In case connection is refused. > > + if (isinstance(exc.message, tuple) and > > + exc.message[0] == errno.ECONNREFUSED): > > + raise test_support.TestSkipped("test socket connection > refused") > > + else: > > + raise exc > > > > ss = socket.ssl(s) > > # Read part of return welcome banner twice. > > _______________________________________________ > > Python-checkins mailing list > > Python-checkins at python.org > > http://mail.python.org/mailman/listinfo/python-checkins > > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20060620/86a55058/attachment.html From buildbot at python.org Tue Jun 20 21:58:53 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 19:58:53 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20060620195853.D1D941E4004@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/706 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 20 22:55:01 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 20 Jun 2006 20:55:01 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060620205501.DF2331E4004@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/436 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 20 23:19:24 2006 From: python-checkins at python.org (phillip.eby) Date: Tue, 20 Jun 2006 23:19:24 +0200 (CEST) Subject: [Python-checkins] r47050 - sandbox/trunk/setuptools/setuptools/package_index.py Message-ID: <20060620211924.BD6761E400E@bag.python.org> Author: phillip.eby Date: Tue Jun 20 23:19:24 2006 New Revision: 47050 Modified: sandbox/trunk/setuptools/setuptools/package_index.py Log: Fix ftp:// directory listing URLs from causing a crash when used in the URL or Download URL slot on PyPI. Modified: sandbox/trunk/setuptools/setuptools/package_index.py ============================================================================== --- sandbox/trunk/setuptools/setuptools/package_index.py (original) +++ sandbox/trunk/setuptools/setuptools/package_index.py Tue Jun 20 23:19:24 2006 @@ -162,7 +162,7 @@ self.fetched_urls[url] = self.fetched_urls[f.url] = True - if 'html' not in f.headers['content-type'].lower(): + if 'html' not in f.headers.get('content-type', '').lower(): f.close() # not html, we can't process it return From python-checkins at python.org Tue Jun 20 23:21:09 2006 From: python-checkins at python.org (phillip.eby) Date: Tue, 20 Jun 2006 23:21:09 +0200 (CEST) Subject: [Python-checkins] r47051 - in sandbox/branches/setuptools-0.6: EasyInstall.txt setuptools/package_index.py Message-ID: <20060620212109.2F63F1E400B@bag.python.org> Author: phillip.eby Date: Tue Jun 20 23:21:08 2006 New Revision: 47051 Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt sandbox/branches/setuptools-0.6/setuptools/package_index.py Log: Fix ``ftp://`` directory listing URLs from causing a crash when used in the "Home page" or "Download URL" slots on PyPI. (merged from the trunk) Modified: sandbox/branches/setuptools-0.6/EasyInstall.txt ============================================================================== --- sandbox/branches/setuptools-0.6/EasyInstall.txt (original) +++ sandbox/branches/setuptools-0.6/EasyInstall.txt Tue Jun 20 23:21:08 2006 @@ -1098,6 +1098,9 @@ 0.6b4 * Fix creating Python wrappers for non-Python scripts + * Fix ``ftp://`` directory listing URLs from causing a crash when used in the + "Home page" or "Download URL" slots on PyPI. + 0.6b3 * Fix local ``--find-links`` eggs not being copied except with ``--always-copy``. Modified: sandbox/branches/setuptools-0.6/setuptools/package_index.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/package_index.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/package_index.py Tue Jun 20 23:21:08 2006 @@ -162,7 +162,7 @@ self.fetched_urls[url] = self.fetched_urls[f.url] = True - if 'html' not in f.headers['content-type'].lower(): + if 'html' not in f.headers.get('content-type', '').lower(): f.close() # not html, we can't process it return From python-checkins at python.org Wed Jun 21 15:14:22 2006 From: python-checkins at python.org (matt.fleming) Date: Wed, 21 Jun 2006 15:14:22 +0200 (CEST) Subject: [Python-checkins] r47052 - in sandbox/trunk/pdb: mconnection.py mpdb.py test/Makefile Message-ID: <20060621131422.EE5F11E4005@bag.python.org> Author: matt.fleming Date: Wed Jun 21 15:14:22 2006 New Revision: 47052 Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/test/Makefile Log: Stop mixing tabs and spaces. Also allow 'make all' to make all tests (sticking with Makefile convention). Incorporate changes in pydb into our code so that we can start using 'info target' command. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Wed Jun 21 15:14:22 2006 @@ -80,7 +80,7 @@ self.input.close() def readline(self, bufsize=2048): - line = self.input.recv(bufsize) + line = self.input.readline(bufsize) return line def write(self, msg): Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Wed Jun 21 15:14:22 2006 @@ -45,18 +45,19 @@ self.prompt = '(MPdb)' self.target = 'local' # local connections by default self.connection = None + print self._info_cmds def _rebind_input(self, new_input): - """ This method rebinds the debugger's input to the object specified - by 'new_input'. - """ + """ This method rebinds the debugger's input to the object specified + by 'new_input'. + """ self.use_rawinput = False - self.stdin = new_input + self.stdin = new_input def _rebind_output(self, new_output): - """ This method rebinds the debugger's output to the object specified - by 'new_output'. - """ + """ This method rebinds the debugger's output to the object specified + by 'new_output'. + """ self.stdout.flush() self.stdout = new_output @@ -69,7 +70,7 @@ # The output from the command that we've just sent to the server # is returned along with the prompt of that server. So we keep reading # until we find our prompt. - while "(MPdb)" not in ret: + while self.prompt not in ret: ret += self.connection.readline() self.msg_nocr(ret) return @@ -88,9 +89,21 @@ out = self.stdout print >> out, msg, + def do_info(self, arg): + """Extends pydb do_info() to give info about the Mpdb extensions.""" + if not arg: + pydb.Pdb.do_info(self, arg) + return + + args = arg.split() + if 'target'.startswith(args[0]): + self.msg("target is %s" % self.target) + else: + pydb.Pdb.do_info(self, arg) + # Debugger commands def do_attach(self, addr): - """ Attach to a process or file outside of Pdb. + """ Attach to a process or file outside of Pdb. This command attaches to another target, of the same type as your last "target" command. The command may take as argument a process id or a device file. For a process id, you must have permission to send the @@ -102,18 +115,18 @@ to specify the program, and to load its symbol table. """ def do_target(self, args): - """ Connect to a target machine or process. + """ Connect to a target machine or process. The first argument is the type or protocol of the target machine -(which can be the name of a class that is avaible either in the current -working directory or in Python's PYTHONPATH environtment variable). +(which can be the name of a class that is available either in the current +working directory or in Python's PYTHONPATH environment variable). Remaining arguments are interpreted by the target protocol. For more information on the arguments for a particular protocol, type `help target ' followed by the protocol name. List of target subcommands: -target serial -- Use a remote computer via a serial line -target tcp -- Use a remote computer via a socket connection +target serial device-name -- Use a remote computer via a serial line +target tcp hostname:port -- Use a remote computer via a socket connection """ try: target, addr = args.split(' ') @@ -128,11 +141,14 @@ if self.connection: self.connection.disconnect() try: from mconnection import MClientConnectionTCP + # Matt - Where are the connection parameters? self.connection = MClientConnectionTCP() + except ImportError: self.msg('Could not import MClientConnectionTCP') return elif target == 'serial': + # Matt - Where are the connection parameters? if self.connection: self.connection.disconnect() try: from mconnection import MClientConnectionSerial @@ -154,7 +170,13 @@ self.msg('Unknown target type') return self.connection = eval(target+'()') - self.connection.connect(addr) + try: + self.connection.connect(addr) + except socket.error: + # Matt: I couldn't figure out what the right + # exception name was to use that getts the error message. + self.msg("Failed to connect to %s" % addr) + return # This interpreter no longer interprets commands but sends # them straight across this object's connection to a server. self.prompt = "" # Get our prompt from the server now Modified: sandbox/trunk/pdb/test/Makefile ============================================================================== --- sandbox/trunk/pdb/test/Makefile (original) +++ sandbox/trunk/pdb/test/Makefile Wed Jun 21 15:14:22 2006 @@ -6,12 +6,15 @@ # # or 'make target' to run a specific test, such as 'make tcptest' -PY = python2.5 +PY = python2.4 + +.PHONY: all test mpdbtest tcptest +all: test test: mpdbtest tcptest mpdbtest: - $(PY) mpdbtest.py + @$(PY) mpdbtest.py tcptest: - $(PY) tcptest.py + @$(PY) tcptest.py From anthony at interlink.com.au Wed Jun 21 18:38:48 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 22 Jun 2006 02:38:48 +1000 Subject: [Python-checkins] TRUNK is UNFROZEN, but in FEATURE FREEZE Message-ID: <200606220238.52248.anthony@interlink.com.au> 2.5b1 is out, so I'm declaring the SVN trunk unfrozen. Note, though, that as we're now post-beta, we're in FEATURE FREEZE. Really. This means you. :-) No new features should be checked in without prior approval - checkins that violate this will quite probably get backed out. I expect that we will also now be quite a bit more anal about any checkins that break the buildbots. Please, please make sure you run the test suite before checking in - and if you're at all concerned that your checkin might have strange platform dependencies, check the buildbot status page (http://www.python.org/dev/buildbot/trunk/) after your checkin to make sure it didn't break anything. Similarly, if you're fixing a bug, if at all possible write a test and check that in as well. The buildbots and a focus on testing should mean that 2.5 ends up being one of the most solid Python releases so far. Please help us achieve this goal. The feature freeze on the trunk will continue until we branch for release candidate 1 of 2.5 - sometime in the second half of July, probably. If you really have the need to do new work on the trunk before then, please work on a branch. Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From python-checkins at python.org Wed Jun 21 18:58:00 2006 From: python-checkins at python.org (brett.cannon) Date: Wed, 21 Jun 2006 18:58:00 +0200 (CEST) Subject: [Python-checkins] r47053 - python/trunk/Lib/test/test_socket_ssl.py Message-ID: <20060621165800.0DF461E4007@bag.python.org> Author: brett.cannon Date: Wed Jun 21 18:57:57 2006 New Revision: 47053 Modified: python/trunk/Lib/test/test_socket_ssl.py Log: At the C level, tuple arguments are passed in directly to the exception constructor, meaning it is treated as *args, not as a single argument. This means using the 'message' attribute won't work (until Py3K comes around), and so one must grab from 'arg' to get the error number. Modified: python/trunk/Lib/test/test_socket_ssl.py ============================================================================== --- python/trunk/Lib/test/test_socket_ssl.py (original) +++ python/trunk/Lib/test/test_socket_ssl.py Wed Jun 21 18:57:57 2006 @@ -56,11 +56,11 @@ use a more reliable address.""" % (ADDR,) return except socket.error, exc: # In case connection is refused. - if (isinstance(exc.message, tuple) and - exc.message[0] == errno.ECONNREFUSED): - raise test_support.TestSkipped("test socket connection refused") + if exc.args[0] == errno.ECONNREFUSED: + print "Connection refused when connecting to", ADDR + return else: - raise exc + raise ss = socket.ssl(s) # Read part of return welcome banner twice. From python-checkins at python.org Wed Jun 21 19:10:18 2006 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 21 Jun 2006 19:10:18 +0200 (CEST) Subject: [Python-checkins] r47054 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060621171018.E61F71E4012@bag.python.org> Author: andrew.kuchling Date: Wed Jun 21 19:10:18 2006 New Revision: 47054 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Link to LibRef module documentation Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Wed Jun 21 19:10:18 2006 @@ -1842,6 +1842,9 @@ \seeurl{http://starship.python.net/crew/theller/ctypes/} {The ctypes web page, with a tutorial, reference, and FAQ.} +\seeurl{../lib/module-ctypes.html}{The documentation +for the \module{ctypes} module.} + \end{seealso} @@ -1953,7 +1956,6 @@ \seeurl{http://effbot.org/zone/element-index.htm} {Official documentation for ElementTree.} - \end{seealso} @@ -2004,6 +2006,13 @@ return the digest value as a binary string or a string of hex digits, and \method{copy()} returns a new hashing object with the same digest state. +\begin{seealso} + +\seeurl{../lib/module-hashlib.html}{The documentation +for the \module{hashlib} module.} + +\end{seealso} + %====================================================================== \subsection{The sqlite3 package\label{module-sqlite}} @@ -2114,6 +2123,9 @@ {The SQLite web page; the documentation describes the syntax and the available data types for the supported SQL dialect.} +\seeurl{../lib/module-sqlite3.html}{The documentation +for the \module{sqlite3} module.} + \seepep{249}{Database API Specification 2.0}{PEP written by Marc-Andr\'e Lemburg.} From python-checkins at python.org Wed Jun 21 19:17:10 2006 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 21 Jun 2006 19:17:10 +0200 (CEST) Subject: [Python-checkins] r47055 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060621171710.DDB9F1E4008@bag.python.org> Author: andrew.kuchling Date: Wed Jun 21 19:17:10 2006 New Revision: 47055 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Note some of Barry's work Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Wed Jun 21 19:17:10 2006 @@ -1409,6 +1409,10 @@ tests contained within a docstring. (Contributed by Bjorn Tillenius.) % Patch 1080727 +\item The \module{email} package has been updated to version 4.0. +% XXX need to provide some more detail here +(Contributed by Barry Warsaw.) + \item The \module{fileinput} module was made more flexible. Unicode filenames are now supported, and a \var{mode} parameter that defaults to \code{"r"} was added to the @@ -1427,6 +1431,7 @@ collection sweep will be made. The existing \function{gc.collect()} function now takes an optional \var{generation} argument of 0, 1, or 2 to specify which generation to collect. +(Contributed by Barry Warsaw.) \item The \function{nsmallest()} and \function{nlargest()} functions in the \module{heapq} module @@ -2392,6 +2397,7 @@ The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this article: Phillip J. Eby, Kent Johnson, Martin von~L\"owis, Fredrik Lundh, -Gustavo Niemeyer, James Pryor, Mike Rovner, Scott Weikart, Thomas Wouters. +Gustavo Niemeyer, James Pryor, Mike Rovner, Scott Weikart, Barry Warsaw, +Thomas Wouters. \end{document} From python-checkins at python.org Wed Jun 21 19:17:28 2006 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 21 Jun 2006 19:17:28 +0200 (CEST) Subject: [Python-checkins] r47056 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060621171728.C02A61E4008@bag.python.org> Author: andrew.kuchling Date: Wed Jun 21 19:17:28 2006 New Revision: 47056 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Bump version Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Wed Jun 21 19:17:28 2006 @@ -6,7 +6,7 @@ % Count up the patches and bugs \title{What's New in Python 2.5} -\release{0.2} +\release{0.3} \author{A.M. Kuchling} \authoraddress{\email{amk at amk.ca}} From buildbot at python.org Wed Jun 21 19:26:15 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 21 Jun 2006 17:26:15 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP trunk Message-ID: <20060621172616.19E8E1E4007@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/1085 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Wed Jun 21 19:45:17 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 21 Jun 2006 19:45:17 +0200 (CEST) Subject: [Python-checkins] r47057 - in python/trunk: Lib/compiler/transformer.py Misc/NEWS Message-ID: <20060621174517.E9F9C1E4009@bag.python.org> Author: georg.brandl Date: Wed Jun 21 19:45:17 2006 New Revision: 47057 Modified: python/trunk/Lib/compiler/transformer.py python/trunk/Misc/NEWS Log: fix [ 1509132 ] compiler module builds incorrect AST for TryExceptFinally Modified: python/trunk/Lib/compiler/transformer.py ============================================================================== --- python/trunk/Lib/compiler/transformer.py (original) +++ python/trunk/Lib/compiler/transformer.py Wed Jun 21 19:45:17 2006 @@ -536,12 +536,7 @@ lineno=nodelist[0][2]) def try_stmt(self, nodelist): - # 'try' ':' suite (except_clause ':' suite)+ ['else' ':' suite] - # | 'try' ':' suite 'finally' ':' suite - if nodelist[3][0] != symbol.except_clause: - return self.com_try_finally(nodelist) - - return self.com_try_except(nodelist) + return self.com_try_except_finally(nodelist) def with_stmt(self, nodelist): return self.com_with(nodelist) @@ -917,18 +912,21 @@ bases.append(self.com_node(node[i])) return bases - def com_try_finally(self, nodelist): - # try_fin_stmt: "try" ":" suite "finally" ":" suite - return TryFinally(self.com_node(nodelist[2]), - self.com_node(nodelist[5]), - lineno=nodelist[0][2]) - - def com_try_except(self, nodelist): - # try_except: 'try' ':' suite (except_clause ':' suite)* ['else' suite] + def com_try_except_finally(self, nodelist): + # ('try' ':' suite + # ((except_clause ':' suite)+ ['else' ':' suite] ['finally' ':' suite] + # | 'finally' ':' suite)) + + if nodelist[3][0] == token.NAME: + # first clause is a finally clause: only try-finally + return TryFinally(self.com_node(nodelist[2]), + self.com_node(nodelist[5]), + lineno=nodelist[0][2]) + #tryexcept: [TryNode, [except_clauses], elseNode)] - stmt = self.com_node(nodelist[2]) clauses = [] elseNode = None + finallyNode = None for i in range(3, len(nodelist), 3): node = nodelist[i] if node[0] == symbol.except_clause: @@ -944,9 +942,16 @@ clauses.append((expr1, expr2, self.com_node(nodelist[i+2]))) if node[0] == token.NAME: - elseNode = self.com_node(nodelist[i+2]) - return TryExcept(self.com_node(nodelist[2]), clauses, elseNode, - lineno=nodelist[0][2]) + if node[1] == 'else': + elseNode = self.com_node(nodelist[i+2]) + elif node[1] == 'finally': + finallyNode = self.com_node(nodelist[i+2]) + try_except = TryExcept(self.com_node(nodelist[2]), clauses, elseNode, + lineno=nodelist[0][2]) + if finallyNode: + return TryFinally(try_except, finallyNode, lineno=nodelist[0][2]) + else: + return try_except def com_with(self, nodelist): # with_stmt: 'with' expr [with_var] ':' suite Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 21 19:45:17 2006 @@ -4,8 +4,25 @@ (editors: check NEWS.help for information about editing NEWS using ReST.) +What's New in Python 2.5 beta 2? +================================ + +*Release date: XX-JUL-2006* + +Core and builtins +----------------- + + +Library +------- + +- The compiler module now correctly compiles the new try-except-finally + statement (bug #1509132). + + + What's New in Python 2.5 beta 1? -================================= +================================ *Release date: 20-JUN-2006* From python-checkins at python.org Wed Jun 21 19:52:37 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 21 Jun 2006 19:52:37 +0200 (CEST) Subject: [Python-checkins] r47058 - python/trunk/Lib/test/test_fcntl.py Message-ID: <20060621175237.491061E4005@bag.python.org> Author: georg.brandl Date: Wed Jun 21 19:52:36 2006 New Revision: 47058 Modified: python/trunk/Lib/test/test_fcntl.py Log: Make test_fcntl aware of netbsd3. Modified: python/trunk/Lib/test/test_fcntl.py ============================================================================== --- python/trunk/Lib/test/test_fcntl.py (original) +++ python/trunk/Lib/test/test_fcntl.py Wed Jun 21 19:52:36 2006 @@ -20,9 +20,10 @@ if sys.platform.startswith('atheos'): start_len = "qq" -if sys.platform in ('netbsd1', 'netbsd2', 'Darwin1.2', 'darwin', - 'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5', 'freebsd6', - 'freebsd7', +if sys.platform in ('netbsd1', 'netbsd2', 'netbsd3', + 'Darwin1.2', 'darwin', + 'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5', + 'freebsd6', 'freebsd7', 'bsdos2', 'bsdos3', 'bsdos4', 'openbsd', 'openbsd2', 'openbsd3'): if struct.calcsize('l') == 8: From python-checkins at python.org Wed Jun 21 19:53:18 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 21 Jun 2006 19:53:18 +0200 (CEST) Subject: [Python-checkins] r47059 - python/trunk/Lib/test/regrtest.py Message-ID: <20060621175318.2E7851E4005@bag.python.org> Author: georg.brandl Date: Wed Jun 21 19:53:17 2006 New Revision: 47059 Modified: python/trunk/Lib/test/regrtest.py Log: Patch #1509001: expected skips for netbsd3. Modified: python/trunk/Lib/test/regrtest.py ============================================================================== --- python/trunk/Lib/test/regrtest.py (original) +++ python/trunk/Lib/test/regrtest.py Wed Jun 21 19:53:17 2006 @@ -1274,6 +1274,37 @@ test_winreg test_winsound """, + 'netbsd3': + """ + test_aepack + test_al + test_applesingle + test_bsddb + test_bsddb185 + test_bsddb3 + test_cd + test_cl + test_ctypes + test_curses + test_dl + test_gdbm + test_gl + test_imgfile + test_linuxaudiodev + test_locale + test_macfs + test_macostools + test_nis + test_ossaudiodev + test_pep277 + test_sqlite + test_startfile + test_sunaudiodev + test_tcl + test_unicode_file + test_winreg + test_winsound + """, } _expectations['freebsd5'] = _expectations['freebsd4'] _expectations['freebsd6'] = _expectations['freebsd4'] From nnorwitz at gmail.com Wed Jun 21 19:53:43 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 21 Jun 2006 10:53:43 -0700 Subject: [Python-checkins] r47057 - in python/trunk: Lib/compiler/transformer.py Misc/NEWS In-Reply-To: <20060621174517.E9F9C1E4009@bag.python.org> References: <20060621174517.E9F9C1E4009@bag.python.org> Message-ID: Can we get a test for this? On 6/21/06, georg.brandl wrote: > Author: georg.brandl > Date: Wed Jun 21 19:45:17 2006 > New Revision: 47057 > > Modified: > python/trunk/Lib/compiler/transformer.py > python/trunk/Misc/NEWS > Log: > fix [ 1509132 ] compiler module builds incorrect AST for TryExceptFinally > > > > Modified: python/trunk/Lib/compiler/transformer.py > ============================================================================== > --- python/trunk/Lib/compiler/transformer.py (original) > +++ python/trunk/Lib/compiler/transformer.py Wed Jun 21 19:45:17 2006 > @@ -536,12 +536,7 @@ > lineno=nodelist[0][2]) > > def try_stmt(self, nodelist): > - # 'try' ':' suite (except_clause ':' suite)+ ['else' ':' suite] > - # | 'try' ':' suite 'finally' ':' suite > - if nodelist[3][0] != symbol.except_clause: > - return self.com_try_finally(nodelist) > - > - return self.com_try_except(nodelist) > + return self.com_try_except_finally(nodelist) > > def with_stmt(self, nodelist): > return self.com_with(nodelist) > @@ -917,18 +912,21 @@ > bases.append(self.com_node(node[i])) > return bases > > - def com_try_finally(self, nodelist): > - # try_fin_stmt: "try" ":" suite "finally" ":" suite > - return TryFinally(self.com_node(nodelist[2]), > - self.com_node(nodelist[5]), > - lineno=nodelist[0][2]) > - > - def com_try_except(self, nodelist): > - # try_except: 'try' ':' suite (except_clause ':' suite)* ['else' suite] > + def com_try_except_finally(self, nodelist): > + # ('try' ':' suite > + # ((except_clause ':' suite)+ ['else' ':' suite] ['finally' ':' suite] > + # | 'finally' ':' suite)) > + > + if nodelist[3][0] == token.NAME: > + # first clause is a finally clause: only try-finally > + return TryFinally(self.com_node(nodelist[2]), > + self.com_node(nodelist[5]), > + lineno=nodelist[0][2]) > + > #tryexcept: [TryNode, [except_clauses], elseNode)] > - stmt = self.com_node(nodelist[2]) > clauses = [] > elseNode = None > + finallyNode = None > for i in range(3, len(nodelist), 3): > node = nodelist[i] > if node[0] == symbol.except_clause: > @@ -944,9 +942,16 @@ > clauses.append((expr1, expr2, self.com_node(nodelist[i+2]))) > > if node[0] == token.NAME: > - elseNode = self.com_node(nodelist[i+2]) > - return TryExcept(self.com_node(nodelist[2]), clauses, elseNode, > - lineno=nodelist[0][2]) > + if node[1] == 'else': > + elseNode = self.com_node(nodelist[i+2]) > + elif node[1] == 'finally': > + finallyNode = self.com_node(nodelist[i+2]) > + try_except = TryExcept(self.com_node(nodelist[2]), clauses, elseNode, > + lineno=nodelist[0][2]) > + if finallyNode: > + return TryFinally(try_except, finallyNode, lineno=nodelist[0][2]) > + else: > + return try_except > > def com_with(self, nodelist): > # with_stmt: 'with' expr [with_var] ':' suite > > Modified: python/trunk/Misc/NEWS > ============================================================================== > --- python/trunk/Misc/NEWS (original) > +++ python/trunk/Misc/NEWS Wed Jun 21 19:45:17 2006 > @@ -4,8 +4,25 @@ > > (editors: check NEWS.help for information about editing NEWS using ReST.) > > +What's New in Python 2.5 beta 2? > +================================ > + > +*Release date: XX-JUL-2006* > + > +Core and builtins > +----------------- > + > + > +Library > +------- > + > +- The compiler module now correctly compiles the new try-except-finally > + statement (bug #1509132). > + > + > + > What's New in Python 2.5 beta 1? > -================================= > +================================ > > *Release date: 20-JUN-2006* > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From g.brandl at gmx.net Wed Jun 21 20:08:29 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Jun 2006 20:08:29 +0200 Subject: [Python-checkins] r47057 - in python/trunk: Lib/compiler/transformer.py Misc/NEWS In-Reply-To: References: <20060621174517.E9F9C1E4009@bag.python.org> Message-ID: Neal Norwitz wrote: > Can we get a test for this? Yes. (What we actually should do is to compile the library and test suite with compiler and run it) Georg From buildbot at python.org Wed Jun 21 21:59:47 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 21 Jun 2006 19:59:47 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060621195947.539321E4005@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/438 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: brett.cannon Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 21 22:55:04 2006 From: python-checkins at python.org (gerhard.haering) Date: Wed, 21 Jun 2006 22:55:04 +0200 (CEST) Subject: [Python-checkins] r47060 - python/trunk/Lib/sqlite3/test/userfunctions.py Message-ID: <20060621205504.D12ED1E4005@bag.python.org> Author: gerhard.haering Date: Wed Jun 21 22:55:04 2006 New Revision: 47060 Modified: python/trunk/Lib/sqlite3/test/userfunctions.py Log: Removed call to enable_callback_tracebacks that slipped in by accident. Modified: python/trunk/Lib/sqlite3/test/userfunctions.py ============================================================================== --- python/trunk/Lib/sqlite3/test/userfunctions.py (original) +++ python/trunk/Lib/sqlite3/test/userfunctions.py Wed Jun 21 22:55:04 2006 @@ -365,7 +365,6 @@ class AuthorizerTests(unittest.TestCase): def setUp(self): - sqlite.enable_callback_tracebacks(1) self.con = sqlite.connect(":memory:") self.con.executescript(""" create table t1 (c1, c2); From buildbot at python.org Wed Jun 21 23:35:27 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 21 Jun 2006 21:35:27 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable trunk Message-ID: <20060621213527.8899B1E4005@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%2520trunk/builds/767 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gerhard.haering Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Wed Jun 21 23:58:51 2006 From: python-checkins at python.org (armin.rigo) Date: Wed, 21 Jun 2006 23:58:51 +0200 (CEST) Subject: [Python-checkins] r47061 - in python/trunk: Lib/test/test_exceptions.py Objects/abstract.c Objects/typeobject.c Message-ID: <20060621215851.7976E1E4007@bag.python.org> Author: armin.rigo Date: Wed Jun 21 23:58:50 2006 New Revision: 47061 Modified: python/trunk/Lib/test/test_exceptions.py python/trunk/Objects/abstract.c python/trunk/Objects/typeobject.c Log: Fix for an obscure bug introduced by revs 46806 and 46808, with a test. The problem of checking too eagerly for recursive calls is the following: if a RuntimeError is caused by recursion, and if code needs to normalize it immediately (as in the 2nd test), then PyErr_NormalizeException() needs a call to the RuntimeError class to instantiate it, and this hits the recursion limit again... causing PyErr_NormalizeException() to never finish. Moved this particular recursion check to slot_tp_call(), which is not involved in instantiating built-in exceptions. Backport candidate. Modified: python/trunk/Lib/test/test_exceptions.py ============================================================================== --- python/trunk/Lib/test/test_exceptions.py (original) +++ python/trunk/Lib/test/test_exceptions.py Wed Jun 21 23:58:50 2006 @@ -305,6 +305,18 @@ x = DerivedException(fancy_arg=42) self.assertEquals(x.fancy_arg, 42) + def testInfiniteRecursion(self): + def f(): + return f() + self.assertRaises(RuntimeError, f) + + def g(): + try: + return g() + except ValueError: + return -1 + self.assertRaises(RuntimeError, g) + def test_main(): run_unittest(ExceptionTests) Modified: python/trunk/Objects/abstract.c ============================================================================== --- python/trunk/Objects/abstract.c (original) +++ python/trunk/Objects/abstract.c Wed Jun 21 23:58:50 2006 @@ -1796,17 +1796,7 @@ ternaryfunc call; if ((call = func->ob_type->tp_call) != NULL) { - PyObject *result = NULL; - /* slot_tp_call() will be called and ends up calling - PyObject_Call() if the object returned for __call__ has - __call__ itself defined upon it. This can be an infinite - recursion if you set __call__ in a class to an instance of - it. */ - if (Py_EnterRecursiveCall(" in __call__")) { - return NULL; - } - result = (*call)(func, arg, kw); - Py_LeaveRecursiveCall(); + PyObject *result = (*call)(func, arg, kw); if (result == NULL && !PyErr_Occurred()) PyErr_SetString( PyExc_SystemError, Modified: python/trunk/Objects/typeobject.c ============================================================================== --- python/trunk/Objects/typeobject.c (original) +++ python/trunk/Objects/typeobject.c Wed Jun 21 23:58:50 2006 @@ -4590,7 +4590,16 @@ if (meth == NULL) return NULL; + + /* PyObject_Call() will end up calling slot_tp_call() again if + the object returned for __call__ has __call__ itself defined + upon it. This can be an infinite recursion if you set + __call__ in a class to an instance of it. */ + if (Py_EnterRecursiveCall(" in __call__")) + return NULL; res = PyObject_Call(meth, args, kwds); + Py_LeaveRecursiveCall(); + Py_DECREF(meth); return res; } From python-checkins at python.org Thu Jun 22 00:11:17 2006 From: python-checkins at python.org (armin.rigo) Date: Thu, 22 Jun 2006 00:11:17 +0200 (CEST) Subject: [Python-checkins] r47062 - in python/branches/release24-maint: Lib/test/test_exceptions.py Objects/abstract.c Objects/typeobject.c Message-ID: <20060621221117.E1DCD1E4007@bag.python.org> Author: armin.rigo Date: Thu Jun 22 00:11:16 2006 New Revision: 47062 Modified: python/branches/release24-maint/Lib/test/test_exceptions.py python/branches/release24-maint/Objects/abstract.c python/branches/release24-maint/Objects/typeobject.c Log: Backport of r47061. Modified: python/branches/release24-maint/Lib/test/test_exceptions.py ============================================================================== --- python/branches/release24-maint/Lib/test/test_exceptions.py (original) +++ python/branches/release24-maint/Lib/test/test_exceptions.py Thu Jun 22 00:11:16 2006 @@ -209,3 +209,18 @@ test_capi2() unlink(TESTFN) + +def test_infinite_recursion(): + def g(): + try: + return g() + except ValueError: + return -1 + try: + g() + except RuntimeError: + pass + else: + print "Expected exception" + +test_infinite_recursion() Modified: python/branches/release24-maint/Objects/abstract.c ============================================================================== --- python/branches/release24-maint/Objects/abstract.c (original) +++ python/branches/release24-maint/Objects/abstract.c Thu Jun 22 00:11:16 2006 @@ -1792,11 +1792,7 @@ ternaryfunc call; if ((call = func->ob_type->tp_call) != NULL) { - PyObject *result = NULL; - if (Py_EnterRecursiveCall(" in __call__")) - return NULL; - result = (*call)(func, arg, kw); - Py_LeaveRecursiveCall(); + PyObject *result = (*call)(func, arg, kw); if (result == NULL && !PyErr_Occurred()) PyErr_SetString( PyExc_SystemError, Modified: python/branches/release24-maint/Objects/typeobject.c ============================================================================== --- python/branches/release24-maint/Objects/typeobject.c (original) +++ python/branches/release24-maint/Objects/typeobject.c Thu Jun 22 00:11:16 2006 @@ -4530,7 +4530,16 @@ if (meth == NULL) return NULL; + + /* PyObject_Call() will end up calling slot_tp_call() again if + the object returned for __call__ has __call__ itself defined + upon it. This can be an infinite recursion if you set + __call__ in a class to an instance of it. */ + if (Py_EnterRecursiveCall(" in __call__")) + return NULL; res = PyObject_Call(meth, args, kwds); + Py_LeaveRecursiveCall(); + Py_DECREF(meth); return res; } From buildbot at python.org Thu Jun 22 00:16:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 21 Jun 2006 22:16:28 +0000 Subject: [Python-checkins] buildbot failure in x86 cygwin trunk Message-ID: <20060621221628.73FAD1E4007@bag.python.org> The Buildbot has detected a new failure of x86 cygwin trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520cygwin%2520trunk/builds/870 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From buildbot at python.org Thu Jun 22 00:16:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 21 Jun 2006 22:16:28 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060621221628.758701E4013@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/399 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: gerhard.haering Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Thu Jun 22 02:09:12 2006 From: python-checkins at python.org (matt.fleming) Date: Thu, 22 Jun 2006 02:09:12 +0200 (CEST) Subject: [Python-checkins] r47063 - sandbox/trunk/pdb/test/Makefile sandbox/trunk/pdb/test/mpdbtest.py sandbox/trunk/pdb/test/tcptest.py sandbox/trunk/pdb/test/test_mconnection.py sandbox/trunk/pdb/test/test_mpdb.py Message-ID: <20060622000912.C9E011E4007@bag.python.org> Author: matt.fleming Date: Thu Jun 22 02:09:11 2006 New Revision: 47063 Added: sandbox/trunk/pdb/test/test_mconnection.py - copied, changed from r47045, sandbox/trunk/pdb/test/tcptest.py sandbox/trunk/pdb/test/test_mpdb.py - copied, changed from r47045, sandbox/trunk/pdb/test/mpdbtest.py Removed: sandbox/trunk/pdb/test/mpdbtest.py sandbox/trunk/pdb/test/tcptest.py Modified: sandbox/trunk/pdb/test/Makefile Log: Using test_* filename convention. Also added some more test cases for the mconnection file. Modified: sandbox/trunk/pdb/test/Makefile ============================================================================== --- sandbox/trunk/pdb/test/Makefile (original) +++ sandbox/trunk/pdb/test/Makefile Thu Jun 22 02:09:11 2006 @@ -6,15 +6,15 @@ # # or 'make target' to run a specific test, such as 'make tcptest' -PY = python2.4 +PY = python -.PHONY: all test mpdbtest tcptest +.PHONY: all test test_mpdb test_mconnection all: test -test: mpdbtest tcptest +test: test_mpdb test_mconnection -mpdbtest: - @$(PY) mpdbtest.py +test_mpdb: + @$(PY) test_mpdb.py -tcptest: - @$(PY) tcptest.py +test_mconnection: + @$(PY) test_mconnection.py Deleted: /sandbox/trunk/pdb/test/mpdbtest.py ============================================================================== --- /sandbox/trunk/pdb/test/mpdbtest.py Thu Jun 22 02:09:11 2006 +++ (empty file) @@ -1,43 +0,0 @@ -#!/usr/bin/env python - -import sys -import socket -import thread -import threading -import unittest - -__addr__ = 'localhost:8000' -script = "" - -sys.path.append("..") -from mpdb import MPdb -from mconnection import MServerConnectionTCP, MClientConnectionTCP - -def doTargetConnect(): - global g_client - while True: - try: - g_client.do_target('tcp '+__addr__) - if CONNECTED: - break - except error: - pass - g_client.onecmd('help') - g_client.onecmd('quit') - -class TestRemoteDebugging(unittest.TestCase): - def testPdbserver(self): - global g_server, g_client, CONNECTED - g_server = MPdb() - g_client = MPdb() - - self.server_tid = thread.start_new_thread(doTargetConnect, ()) - g_server.do_pdbserver('tcp '+__addr__+' '+script) - CONNECTED = True - - # XXX mpdb needs a bottom frame before it exits - g_server.botframe = None - g_server.cmdloop() - -if __name__ == '__main__': - unittest.main() Deleted: /sandbox/trunk/pdb/test/tcptest.py ============================================================================== --- /sandbox/trunk/pdb/test/tcptest.py Thu Jun 22 02:09:11 2006 +++ (empty file) @@ -1,47 +0,0 @@ -#!/usr/bin/env python - -# This unit test doesn't use any of the debugger code. It is meant solely -# to test the connection classes. - -import sys -import socket -import thread -import threading -import unittest - -__addr__ = 'localhost:8000' - -sys.path.append("..") -from mconnection import MServerConnectionTCP, MClientConnectionTCP - -# Try to connect the client to addr either until we've tried MAXTRIES -# times or until it succeeds. -MAXTRIES = 100 -def repeatedConnect(client, addr): - for i in range(MAXTRIES): - try: - client.connect(addr) - # The _sock variable appears when there's a connection - if client._sock: break - except socket.error: - pass - -class TestTCPConnections(unittest.TestCase): - def setUp(self): - self.server = MServerConnectionTCP() - self.client = MClientConnectionTCP() - - def testClientConnectAndRead(self): - t_id = thread.start_new_thread(repeatedConnect, (self.client,__addr__)) - self.server.connect(__addr__) - - self.server.write("good") - line = self.client.readline() - self.assertEqual("good", line, "Could not read from server") - - def tearDown(self): - self.server.disconnect() - self.client.disconnect() - -if __name__ == '__main__': - unittest.main() Copied: sandbox/trunk/pdb/test/test_mconnection.py (from r47045, sandbox/trunk/pdb/test/tcptest.py) ============================================================================== --- sandbox/trunk/pdb/test/tcptest.py (original) +++ sandbox/trunk/pdb/test/test_mconnection.py Thu Jun 22 02:09:11 2006 @@ -3,20 +3,27 @@ # This unit test doesn't use any of the debugger code. It is meant solely # to test the connection classes. +import os import sys import socket import thread -import threading import unittest +from test import test_support +from socket import gaierror + +# Global vars __addr__ = 'localhost:8000' +MAXTRIES = 100 +TESTFN = 'device' sys.path.append("..") -from mconnection import MServerConnectionTCP, MClientConnectionTCP +from mconnection import (MServerConnectionTCP, MClientConnectionTCP, + MServerConnectionSerial, MClientConnectionSerial, + ConnectionRefused) # Try to connect the client to addr either until we've tried MAXTRIES # times or until it succeeds. -MAXTRIES = 100 def repeatedConnect(client, addr): for i in range(MAXTRIES): try: @@ -30,18 +37,106 @@ def setUp(self): self.server = MServerConnectionTCP() self.client = MClientConnectionTCP() - + + def testClientConnectToServer(self): + """(tcp) Connect client to server. """ + thread.start_new_thread(repeatedConnect, (self.client, __addr__)) + self.server.connect(__addr__) + def testClientConnectAndRead(self): - t_id = thread.start_new_thread(repeatedConnect, (self.client,__addr__)) + """(tcp) Connect to server and read/write. """ + thread.start_new_thread(repeatedConnect, (self.client,__addr__)) self.server.connect(__addr__) self.server.write("good") line = self.client.readline() self.assertEqual("good", line, "Could not read from server") + self.client.write('success') + line = self.server.readline() + self.assertEqual('success\n', line, 'Could not read from client') + + def testDisconnectDisconnected(self): + """(tcp) Disconnect a disconnected session. """ + s = MServerConnectionTCP() + s.disconnect() + s.disconnect() + + def testReadline(self): + """(tcp) Make sure readline method works. """ + thread.start_new_thread(repeatedConnect, (self.client,__addr__)) + self.server.connect(__addr__) + + self.client.write('good') + line = self.server.readline() + self.assertEquals('good\n', line, 'Could not read first line.') + + def testConnectionRefused(self): + """(tcp) Test refused connection. """ + thread.start_new_thread(repeatedConnect, (self.client, __addr__)) + self.server.connect(__addr__) + + # Set up second server on same port + s = MServerConnectionTCP() + self.assertRaises(ConnectionRefused, s.connect, __addr__) + + def tearDown(self): + self.server.disconnect() + self.client.disconnect() + +class TestSerialConnections(unittest.TestCase): + """ This test just uses a file instead of a serial device, which + on *nix systems is just files anyway. + """ + def setUp(self): + self.server = MServerConnectionSerial() + self.client = MClientConnectionSerial() + fd = open(TESTFN, "wr+") + fd.close() + self.server.connect(TESTFN) + self.client.connect(TESTFN) + + def testClientToServerConnect(self): + """(serial) Connect client to server. """ + self.client.disconnect() + self.server.disconnect() + + def testClientWriteRead(self): + """(serial) Connect client to server and read/write. """ + self.client.write('success!') + line = self.server.readline() + self.assertEquals('success!\n', line, 'Could not read from client.') + + # Unfortunately the text file doesn't erase what we've written like a + # device of stream, so we have to close the the file and re-open it. + self.server.disconnect() + self.server.connect(TESTFN) + self.server.write('great!') + line = self.client.readline() + self.assertEquals('great!\n', line, 'Could not read from server.') + + def testDisconnectDisconnected(self): + """(serial) Disconnect a disconnected session. """ + self.server.disconnect() + + def testReadline(self): + """(serial) Make sure readline method works. """ + self.client.write('success!\nNext line.') + self.client.disconnect() + line = self.server.readline() + self.assertEquals('success!\n', line, 'Could not read first line') + line = self.server.readline() + self.assertEquals('Next line.\n', line, 'Could not read second line.') + line = self.server.readline() + self.assertEquals('', line, 'Could not read third line.') def tearDown(self): self.server.disconnect() self.client.disconnect() + os.remove(TESTFN) + +def test_main(): + test_support.run_unittest(TestTCPConnections, TestSerialConnections) + if __name__ == '__main__': - unittest.main() + test_main() Copied: sandbox/trunk/pdb/test/test_mpdb.py (from r47045, sandbox/trunk/pdb/test/mpdbtest.py) ============================================================================== --- sandbox/trunk/pdb/test/mpdbtest.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Thu Jun 22 02:09:11 2006 @@ -3,41 +3,81 @@ import sys import socket import thread -import threading import unittest +from test import test_support + +# Global vars __addr__ = 'localhost:8000' script = "" +g_server = None +g_client = None +CONNECTED = False +# Commands to execute on the server +cmds = ['info target', 'help', 'quit'] sys.path.append("..") from mpdb import MPdb from mconnection import MServerConnectionTCP, MClientConnectionTCP -def doTargetConnect(): +def doTargetConnect(cmds=None): global g_client while True: try: g_client.do_target('tcp '+__addr__) if CONNECTED: break - except error: + except socket.error: pass - g_client.onecmd('help') - g_client.onecmd('quit') + if cmds: + for c in cmds: + g_client.onecmd(c) +class MPdbTest(MPdb): + def __init__(self): + MPdb.__init__(self) + self.lines = [] + + def msg_nocr(self, msg): + self.lines.append(msg) + class TestRemoteDebugging(unittest.TestCase): + """ Test Case to make sure debugging remotely works properly. """ + def setUp(self): + self.server = MPdb() + global g_client + g_client = MPdbTest() + + def tearDown(self): + global CONNECTED + self.server.connection.disconnect() + CONNETED = False + def testPdbserver(self): - global g_server, g_client, CONNECTED - g_server = MPdb() - g_client = MPdb() + """ Test the pdbserver command. """ + global CONNECTED self.server_tid = thread.start_new_thread(doTargetConnect, ()) - g_server.do_pdbserver('tcp '+__addr__+' '+script) + self.server.do_pdbserver('tcp '+__addr__+' '+script) CONNECTED = True + def testCommandsOnServer(self): + """ Test all supported commands on the pdbserver. + """ + global CONNECTED, g_client + + self.server_tid = thread.start_new_thread(doTargetConnect, (cmds,)) + self.server.do_pdbserver('tcp '+__addr__+' '+script) + CONNECTED = True + # XXX mpdb needs a bottom frame before it exits - g_server.botframe = None - g_server.cmdloop() + self.server.botframe = None + self.server.cmdloop() + self.assertEquals('target is remote\n', g_client.lines[1]) + +def test_main(): + test_support.run_unittest(TestRemoteDebugging) + if __name__ == '__main__': - unittest.main() + test_main() From python-checkins at python.org Thu Jun 22 08:30:56 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 22 Jun 2006 08:30:56 +0200 (CEST) Subject: [Python-checkins] r47064 - in python/trunk: Makefile.pre.in Misc/NEWS Message-ID: <20060622063056.23DE51E4007@bag.python.org> Author: neal.norwitz Date: Thu Jun 22 08:30:50 2006 New Revision: 47064 Modified: python/trunk/Makefile.pre.in python/trunk/Misc/NEWS Log: Copy the wsgiref package during make install. Modified: python/trunk/Makefile.pre.in ============================================================================== --- python/trunk/Makefile.pre.in (original) +++ python/trunk/Makefile.pre.in Thu Jun 22 08:30:50 2006 @@ -708,7 +708,7 @@ encodings compiler hotshot \ email email/mime email/test email/test/data \ sqlite3 sqlite3/test \ - logging bsddb bsddb/test csv \ + logging bsddb bsddb/test csv wsgiref \ ctypes ctypes/test ctypes/macholib idlelib idlelib/Icons \ distutils distutils/command distutils/tests $(XMLLIBSUBDIRS) \ setuptools setuptools/command setuptools/tests setuptools.egg-info \ Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 22 08:30:50 2006 @@ -19,6 +19,8 @@ - The compiler module now correctly compiles the new try-except-finally statement (bug #1509132). +- The wsgiref package is now installed properly on Unix. + What's New in Python 2.5 beta 1? From nnorwitz at gmail.com Thu Jun 22 08:35:07 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 21 Jun 2006 23:35:07 -0700 Subject: [Python-checkins] Things to remember when adding *packages* to stdlib Message-ID: I believe this change is all that's necessary on the Unix side to install wsgiref. Can someone please update the Windows build files to ensure wsgiref is installed in b2? Don't forget to update the NEWS entry too. Also, all committers and reviewers, try to remember that when a package (meaning directory) is added to the stdlib, we need to update Makefile.pre.in and the Windows build files. (I forgot this time too.) Is there some documentation that should be updated as a reminder? Maybe someone could come up with a heuristic to add to Misc/build.sh which we could test in there. n -- On 6/21/06, neal.norwitz wrote: > Author: neal.norwitz > Date: Thu Jun 22 08:30:50 2006 > New Revision: 47064 > > Modified: > python/trunk/Makefile.pre.in > python/trunk/Misc/NEWS > Log: > Copy the wsgiref package during make install. > > > > Modified: python/trunk/Makefile.pre.in > ============================================================================== > --- python/trunk/Makefile.pre.in (original) > +++ python/trunk/Makefile.pre.in Thu Jun 22 08:30:50 2006 > @@ -708,7 +708,7 @@ > encodings compiler hotshot \ > email email/mime email/test email/test/data \ > sqlite3 sqlite3/test \ > - logging bsddb bsddb/test csv \ > + logging bsddb bsddb/test csv wsgiref \ > ctypes ctypes/test ctypes/macholib idlelib idlelib/Icons \ > distutils distutils/command distutils/tests $(XMLLIBSUBDIRS) \ > setuptools setuptools/command setuptools/tests setuptools.egg-info \ > > Modified: python/trunk/Misc/NEWS > ============================================================================== > --- python/trunk/Misc/NEWS (original) > +++ python/trunk/Misc/NEWS Thu Jun 22 08:30:50 2006 > @@ -19,6 +19,8 @@ > - The compiler module now correctly compiles the new try-except-finally > statement (bug #1509132). > > +- The wsgiref package is now installed properly on Unix. > + > > > What's New in Python 2.5 beta 1? > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From python-checkins at python.org Thu Jun 22 08:35:30 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 22 Jun 2006 08:35:30 +0200 (CEST) Subject: [Python-checkins] r47065 - python/trunk/Doc/commontex/boilerplate.tex Message-ID: <20060622063530.E6DC51E400E@bag.python.org> Author: neal.norwitz Date: Thu Jun 22 08:35:30 2006 New Revision: 47065 Modified: python/trunk/Doc/commontex/boilerplate.tex Log: Reset the doc date to today for the automatic doc builds Modified: python/trunk/Doc/commontex/boilerplate.tex ============================================================================== --- python/trunk/Doc/commontex/boilerplate.tex (original) +++ python/trunk/Doc/commontex/boilerplate.tex Thu Jun 22 08:35:30 2006 @@ -5,5 +5,5 @@ Email: \email{docs at python.org} } -\date{20 June, 2006} % XXX update before final release! +\date{\today} % XXX update before final release! \input{patchlevel} % include Python version information From buildbot at python.org Thu Jun 22 09:28:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 07:28:36 +0000 Subject: [Python-checkins] buildbot warnings in x86 cygwin trunk Message-ID: <20060622072836.71C3C1E4007@bag.python.org> The Buildbot has detected a new failure of x86 cygwin trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520cygwin%2520trunk/builds/871 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From neal at metaslash.com Thu Jun 22 11:08:26 2006 From: neal at metaslash.com (Neal Norwitz) Date: Thu, 22 Jun 2006 05:08:26 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (2) Message-ID: <20060622090826.GA7100@python.psfb.org> test_descr leaked [25, 25, 25] references test_socket leaked [214, -214, 0] references From python-checkins at python.org Thu Jun 22 14:54:44 2006 From: python-checkins at python.org (matt.fleming) Date: Thu, 22 Jun 2006 14:54:44 +0200 (CEST) Subject: [Python-checkins] r47066 - sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Message-ID: <20060622125444.EDB121E4007@bag.python.org> Author: matt.fleming Date: Thu Jun 22 14:54:44 2006 New Revision: 47066 Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Log: Added exceptions, no longer have the namespace corruption problem, imitate fileobject.close() method with disconnect(). Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Thu Jun 22 14:54:44 2006 @@ -10,13 +10,6 @@ -=[TODO]=- * Write more unit tests, test the pdbserver and target commands. -* sort out the namespace corruption - - """ - c:\soc\pdb\test.py(3)x() - -> print c[i+1], i - (MPdb)p i - *** NameError: - """ * Write a signal handler that scripts can import from mpdb that, when the signal is received, start remote debugging. * info target,threads command needs to be written. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Thu Jun 22 14:54:44 2006 @@ -4,6 +4,11 @@ NotImplementedMessage = "This method must be overriden in a subclass" +### Exceptions +class ConnectionRefused(Exception): pass +class DroppedConnection(Exception): pass +class ReadOnClose(Exception): pass + class MServerConnectionInterface(object): """ This is an abstract class that specifies the interface a server connection class must implement. If a target is given, we'll @@ -15,8 +20,8 @@ raise NotImplementedError, NotImplementedMessage def disconnect(self): - """ This method is called to disconnect connections.""" - raise NotImplementedError, NotImplementedMessage + """ This method is called to disconnect connections.""" + raise NotImplementedError, NotImplementedMessage def readline(self): """ This method reads a line of data of maximum length 'bufsize' @@ -71,20 +76,29 @@ (e.g. /dev/ttyS0, /dev/ttya, COM1, etc.). """ self._dev = device - self.output = open(self._dev, 'w') - self.input = open(self._dev, 'r') + try: + self.output = open(self._dev, 'w') + self.input = open(self._dev, 'r') + except IOError: + raise ConnectionRefused def disconnect(self): """ Close the serial device. """ + if self.output is None and self.input is None: + return self.output.close() self.input.close() + def readline(self, bufsize=2048): line = self.input.readline(bufsize) return line def write(self, msg): + if msg[-1] is not '\n': + msg += '\n' self.output.write(msg) + self.output.flush() MClientConnectionSerial = MServerConnectionSerial @@ -97,7 +111,7 @@ """ def __init__(self): self.listening = False - self.output = self.input = None + self._sock = self.output = self.input = None MServerConnectionInterface.__init__(self) def connect(self, addr): @@ -110,15 +124,22 @@ if not self.listening: self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - self._sock.bind((self.host, self.port)) + try: + self._sock.bind((self.host, self.port)) + except socket.error: + raise ConnectionRefused self._sock.listen(1) self.listening = True self.output, addr = self._sock.accept() self.input = self.output def disconnect(self): + # These two should either _both_ be None, or neither should be None + if self.output is None and self._sock is None: + return self.output.close() self._sock.close() + self._sock = None self.listening = False def readline(self, bufsize=2048): @@ -139,6 +160,7 @@ def __init__(self): """ Specify the address to connection to. """ MClientConnectionInterface.__init__(self) + self._sock = self.output = self.input = None def connect(self, addr): """Connect to the server. 'input' reads data from the @@ -163,4 +185,10 @@ def disconnect(self): """ Close the socket to the server. """ - self._sock.close() + # We shouldn't bail if we haven't been connected yet + if self._sock is None: + return + else: + self._sock.close() + + Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Thu Jun 22 14:54:44 2006 @@ -45,7 +45,6 @@ self.prompt = '(MPdb)' self.target = 'local' # local connections by default self.connection = None - print self._info_cmds def _rebind_input(self, new_input): """ This method rebinds the debugger's input to the object specified @@ -160,8 +159,8 @@ if '.' in target: if self.connection: self.connection.disconnect() # We dynamically load the class for the connection - base = cls[:cls.rfind('.')] - cls = cls[cls.rfind('.')+1:] + base = target[:target.rfind('.')] + cls = target[target.rfind('.')+1:] exec 'from ' + base + ' import ' + cls else: try: From python-checkins at python.org Thu Jun 22 15:10:24 2006 From: python-checkins at python.org (andrew.kuchling) Date: Thu, 22 Jun 2006 15:10:24 +0200 (CEST) Subject: [Python-checkins] r47067 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060622131024.5BB5A1E4007@bag.python.org> Author: andrew.kuchling Date: Thu Jun 22 15:10:23 2006 New Revision: 47067 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Mention how to suppress warnings Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Thu Jun 22 15:10:23 2006 @@ -1174,6 +1174,16 @@ an import would have picked up a directory as a package but no \file{__init__.py} was found. (Implemented by Thomas Wouters.) +To suppress these warnings, you can either supply +\command{-W'ignore:Not importing directory'} when running the Python +interpreter, or use the \module{warnings} module to suppress the +message: + +\begin{verbatim} +warnings.filterwarnings('ignore', 'Not importing directory', + ImportWarning) +\end{verbatim} + \item The list of base classes in a class definition can now be empty. As an example, this is now legal: @@ -2396,8 +2406,8 @@ The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this -article: Phillip J. Eby, Kent Johnson, Martin von~L\"owis, Fredrik Lundh, -Gustavo Niemeyer, James Pryor, Mike Rovner, Scott Weikart, Barry Warsaw, -Thomas Wouters. +article: Phillip J. Eby, "Ralf W. Grosse-Kunstleve, Kent Johnson, +Martin von~L\"owis, Fredrik Lundh, Gustavo Niemeyer, James Pryor, Mike +Rovner, Scott Weikart, Barry Warsaw, Thomas Wouters. \end{document} From buildbot at python.org Thu Jun 22 15:38:31 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 13:38:31 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD trunk Message-ID: <20060622133831.B1E9D1E4009@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%2520trunk/builds/916 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 22 15:55:42 2006 From: python-checkins at python.org (matt.fleming) Date: Thu, 22 Jun 2006 15:55:42 +0200 (CEST) Subject: [Python-checkins] r47068 - sandbox/trunk/pdb/mthread.py Message-ID: <20060622135542.D93421E4015@bag.python.org> Author: matt.fleming Date: Thu Jun 22 15:55:42 2006 New Revision: 47068 Added: sandbox/trunk/pdb/mthread.py Log: Adding mthread.py that will eventually turn into the thread debugging code. Added: sandbox/trunk/pdb/mthread.py ============================================================================== --- (empty file) +++ sandbox/trunk/pdb/mthread.py Thu Jun 22 15:55:42 2006 @@ -0,0 +1,97 @@ +""" This file contains all code for allowing the debugging of threads. """ + +import sys +import thread +import threading +import time +from mpdb import MPdb + +# Globals +STOP = False +t_dict = {} +t_current = None + +class MyThread(threading.Thread): + def run(self): + while True: + if STOP: break + x = 2 + +class MTracer(MPdb): + """ A class to trace a thread. """ + def __init__(self): + MPdb.__init__(self) + self.quitting = False + self.botframe = None + self.stopframe = None + self.returnframe = None + self.thread = threading.currentThread().getName() + + def trace_dispatch(self, frame, event, arg): + try: + if not t_dict[self.thread]: + return + except KeyError: + return + if self.quitting: + return # None + if event == 'line': + return self.dispatch_line(frame) + if event == 'call': + return self.dispatch_call(frame, arg) + if event == 'return': + return self.dispatch_return(frame, arg) + if event == 'exception': + return self.dispatch_exception(frame, arg) + if event == 'c_call': + return self.trace_dispatch + if event == 'c_exception': + return self.trace_dispatch + if event == 'c_return': + return self.trace_dispatch + print 'bdb.Bdb.dispatch: unknown debugging event:', repr(event) + return self.trace_dispatch + + + # Override Pdb methods + + def interaction(self, frame, traceback): + print "Thread[%s]: %s" % (self.thread, frame.f_code) + +def trace_dispatch(self, frame, somethinelse): + """ Create a new MTracer object for a thread and set that threads + tracing function to the MTracer objects trace method. Put this thread + into the global dict of threads that we can trace. + """ + global t_current + m = MTracer() + + # This is supposed to simulate the idea of the first thread becomes + # the current thread we're debugging. In mpdb the user will have to + # manually change the currently debugged thread with a command. + # This thread is not necessarily the first 'created' in a script, but + # the first to be 'run'. + if t_current is None: + global t_dict + t_current = m.thread + t_dict[m.thread] = True + sys.settrace(m.trace_dispatch) + +def threads(): + # Set the globla tracing function + threading.settrace(trace_dispatch) + + global STOP, t_dict + + t_list = [] + # Create 3 threads + for i in range(3): + t = MyThread() + t.start() + t_list.append(t) + + STOP = True + +if __name__ == '__main__': + threads() + From python-checkins at python.org Thu Jun 22 16:46:18 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 22 Jun 2006 16:46:18 +0200 (CEST) Subject: [Python-checkins] r47069 - python/trunk/Lib/compiler/transformer.py Message-ID: <20060622144618.589601E4007@bag.python.org> Author: georg.brandl Date: Thu Jun 22 16:46:17 2006 New Revision: 47069 Modified: python/trunk/Lib/compiler/transformer.py Log: Set lineno correctly on list, tuple and dict literals. Modified: python/trunk/Lib/compiler/transformer.py ============================================================================== --- python/trunk/Lib/compiler/transformer.py (original) +++ python/trunk/Lib/compiler/transformer.py Thu Jun 22 16:46:17 2006 @@ -727,17 +727,17 @@ def atom_lpar(self, nodelist): if nodelist[1][0] == token.RPAR: - return Tuple(()) + return Tuple((), lineno=nodelist[0][2]) return self.com_node(nodelist[1]) def atom_lsqb(self, nodelist): if nodelist[1][0] == token.RSQB: - return List(()) + return List((), lineno=nodelist[0][2]) return self.com_list_constructor(nodelist[1]) def atom_lbrace(self, nodelist): if nodelist[1][0] == token.RBRACE: - return Dict(()) + return Dict((), lineno=nodelist[0][2]) return self.com_dictmaker(nodelist[1]) def atom_backquote(self, nodelist): @@ -1141,7 +1141,7 @@ values = [] for i in range(1, len(nodelist), 2): values.append(self.com_node(nodelist[i])) - return List(values) + return List(values, lineno=values[0].lineno) if hasattr(symbol, 'gen_for'): def com_generator_expression(self, expr, node): @@ -1188,7 +1188,7 @@ for i in range(1, len(nodelist), 4): items.append((self.com_node(nodelist[i]), self.com_node(nodelist[i+2]))) - return Dict(items) + return Dict(items, lineno=items[0][0].lineno) def com_apply_trailer(self, primaryNode, nodelist): t = nodelist[1][0] From python-checkins at python.org Thu Jun 22 16:46:46 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 22 Jun 2006 16:46:46 +0200 (CEST) Subject: [Python-checkins] r47070 - python/trunk/Lib/test/test_compiler.py Message-ID: <20060622144646.D06811E4007@bag.python.org> Author: georg.brandl Date: Thu Jun 22 16:46:46 2006 New Revision: 47070 Modified: python/trunk/Lib/test/test_compiler.py Log: Test for correct compilation of try-except-finally stmt. Test for correct lineno on list, tuple, dict literals. Modified: python/trunk/Lib/test/test_compiler.py ============================================================================== --- python/trunk/Lib/test/test_compiler.py (original) +++ python/trunk/Lib/test/test_compiler.py Thu Jun 22 16:46:46 2006 @@ -56,6 +56,15 @@ def testYieldExpr(self): compiler.compile("def g(): yield\n\n", "", "exec") + def testTryExceptFinally(self): + # Test that except and finally clauses in one try stmt are recognized + c = compiler.compile("try:\n 1/0\nexcept:\n e = 1\nfinally:\n f = 1", + "", "exec") + dct = {} + exec c in dct + self.assertEquals(dct.get('e'), 1) + self.assertEquals(dct.get('f'), 1) + def testDefaultArgs(self): self.assertRaises(SyntaxError, compiler.parse, "def foo(a=1, b): pass") @@ -103,6 +112,12 @@ l = [(x, y) for x, y in zip(range(5), range(5,10))] l[0] l[3:4] +d = {'a': 2} +d = {} +t = () +t = (1, 2) +l = [] +l = [1, 2] if l: pass else: From buildbot at python.org Thu Jun 22 16:57:42 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 14:57:42 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060622145743.1646F1E4007@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/224 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Thu Jun 22 17:50:09 2006 From: python-checkins at python.org (fred.drake) Date: Thu, 22 Jun 2006 17:50:09 +0200 (CEST) Subject: [Python-checkins] r47071 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060622155009.6F44C1E4007@bag.python.org> Author: fred.drake Date: Thu Jun 22 17:50:08 2006 New Revision: 47071 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: fix markup nit Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Thu Jun 22 17:50:08 2006 @@ -1175,7 +1175,7 @@ \file{__init__.py} was found. (Implemented by Thomas Wouters.) To suppress these warnings, you can either supply -\command{-W'ignore:Not importing directory'} when running the Python +\code{\programopt{-W}'ignore:Not importing directory'} when running the Python interpreter, or use the \module{warnings} module to suppress the message: From python-checkins at python.org Thu Jun 22 18:49:15 2006 From: python-checkins at python.org (brett.cannon) Date: Thu, 22 Jun 2006 18:49:15 +0200 (CEST) Subject: [Python-checkins] r47072 - in python/trunk: Lib/test/test_warnings.py Lib/warnings.py Misc/NEWS Message-ID: <20060622164915.5EC921E4008@bag.python.org> Author: brett.cannon Date: Thu Jun 22 18:49:14 2006 New Revision: 47072 Modified: python/trunk/Lib/test/test_warnings.py python/trunk/Lib/warnings.py python/trunk/Misc/NEWS Log: 'warning's was improperly requiring that a command-line Warning category be both a subclass of Warning and a subclass of types.ClassType. The latter is no longer true thanks to new-style exceptions. Closes bug #1510580. Thanks to AMK for the test. Modified: python/trunk/Lib/test/test_warnings.py ============================================================================== --- python/trunk/Lib/test/test_warnings.py (original) +++ python/trunk/Lib/test/test_warnings.py Thu Jun 22 18:49:14 2006 @@ -81,6 +81,19 @@ self.assertEqual(msg.message, text) self.assertEqual(msg.category, 'UserWarning') + def test_options(self): + # Uses the private _setoption() function to test the parsing + # of command-line warning arguments + self.assertRaises(warnings._OptionError, + warnings._setoption, '1:2:3:4:5:6') + self.assertRaises(warnings._OptionError, + warnings._setoption, 'bogus::Warning') + self.assertRaises(warnings._OptionError, + warnings._setoption, 'ignore:2::4:-5') + warnings._setoption('error::Warning::0') + self.assertRaises(UserWarning, warnings.warn, 'convert to error') + + def test_main(verbose=None): # Obscure hack so that this test passes after reloads or repeated calls # to test_main (regrtest -R). Modified: python/trunk/Lib/warnings.py ============================================================================== --- python/trunk/Lib/warnings.py (original) +++ python/trunk/Lib/warnings.py Thu Jun 22 18:49:14 2006 @@ -254,8 +254,7 @@ cat = getattr(m, klass) except AttributeError: raise _OptionError("unknown warning category: %r" % (category,)) - if (not isinstance(cat, types.ClassType) or - not issubclass(cat, Warning)): + if not issubclass(cat, Warning): raise _OptionError("invalid warning category: %r" % (category,)) return cat Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 22 18:49:14 2006 @@ -16,6 +16,10 @@ Library ------- +- Bug #1510580: The 'warnings' module improperly required that a Warning + category be either a types.ClassType and a subclass of Warning. The proper + check is just that it is a subclass with Warning as the documentation states. + - The compiler module now correctly compiles the new try-except-finally statement (bug #1509132). From buildbot at python.org Thu Jun 22 20:07:24 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 18:07:24 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060622180724.9303C1E400E@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/403 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake,georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 22 20:33:54 2006 From: python-checkins at python.org (ronald.oussoren) Date: Thu, 22 Jun 2006 20:33:54 +0200 (CEST) Subject: [Python-checkins] r47073 - python/trunk/Mac/BuildScript/resources/Welcome.rtf Message-ID: <20060622183354.8B36F1E4007@bag.python.org> Author: ronald.oussoren Date: Thu Jun 22 20:33:54 2006 New Revision: 47073 Modified: python/trunk/Mac/BuildScript/resources/Welcome.rtf Log: MacOSX: Add a message to the first screen of the installer that tells users how to avoid updates to their shell profile. Modified: python/trunk/Mac/BuildScript/resources/Welcome.rtf ============================================================================== --- python/trunk/Mac/BuildScript/resources/Welcome.rtf (original) +++ python/trunk/Mac/BuildScript/resources/Welcome.rtf Thu Jun 22 20:33:54 2006 @@ -1,4 +1,4 @@ -{\rtf1\mac\ansicpg10000\cocoartf824\cocoasubrtf330 +{\rtf1\mac\ansicpg10000\cocoartf824\cocoasubrtf410 {\fonttbl\f0\fswiss\fcharset77 Helvetica;\f1\fswiss\fcharset77 Helvetica-Bold;} {\colortbl;\red255\green255\blue255;} \paperw11900\paperh16840\margl1440\margr1440\vieww9920\viewh10660\viewkind0 @@ -12,4 +12,7 @@ \ MacPython consists of the Python programming language interpreter, plus a set of programs to allow easy access to it for Mac users (an integrated development environment, an applet builder), plus a set of pre-built extension modules that open up specific Macintosh technologies to Python programs (Carbon, AppleScript, Quicktime, more).\ \ -See the ReadMe file for more information.} \ No newline at end of file +See the ReadMe file for more information.\ +\ +\ +This package will by default update your shell profile to ensure that this version of Python is on the search path of your shell. Please deselect the "Shell profile updater" package on the package customization screen if you want to avoid this modification. } \ No newline at end of file From buildbot at python.org Thu Jun 22 20:56:33 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 18:56:33 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060622185633.D93941E4007@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/443 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake,georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 22 21:02:19 2006 From: python-checkins at python.org (georg.brandl) Date: Thu, 22 Jun 2006 21:02:19 +0200 (CEST) Subject: [Python-checkins] r47074 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060622190219.49F021E4007@bag.python.org> Author: georg.brandl Date: Thu Jun 22 21:02:18 2006 New Revision: 47074 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Fix my name ;) Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Thu Jun 22 21:02:18 2006 @@ -1744,7 +1744,7 @@ additional feature, an \var{autoraise} parameter that signals whether to raise the open window when possible. A number of additional browsers were added to the supported list such as Firefox, Opera, -Konqueror, and elinks. (Contributed by Oleg Broytmann and George +Konqueror, and elinks. (Contributed by Oleg Broytmann and Georg Brandl.) % Patch #754022 From python-checkins at python.org Thu Jun 22 21:07:37 2006 From: python-checkins at python.org (thomas.heller) Date: Thu, 22 Jun 2006 21:07:37 +0200 (CEST) Subject: [Python-checkins] r47075 - python/trunk/Doc/lib/libctypes.tex Message-ID: <20060622190737.6415D1E4017@bag.python.org> Author: thomas.heller Date: Thu Jun 22 21:07:36 2006 New Revision: 47075 Modified: python/trunk/Doc/lib/libctypes.tex Log: Small fixes, mostly in the markup. Modified: python/trunk/Doc/lib/libctypes.tex ============================================================================== --- python/trunk/Doc/lib/libctypes.tex (original) +++ python/trunk/Doc/lib/libctypes.tex Thu Jun 22 21:07:36 2006 @@ -1543,13 +1543,13 @@ The following public attributes are available, their name starts with an underscore to not clash with exported function names: -\begin{datadescni}{_handle: memberdesc} +\begin{memberdesc}{_handle} The system handle used to access the library. -\end{datadescni} +\end{memberdesc} -\begin{datadescni}{_name: memberdesc} +\begin{memberdesc}{_name} The name of the library passed in the contructor. -\end{datadescni} +\end{memberdesc} Shared libraries can also be loaded by using one of the prefabricated objects, which are instances of the \class{LibraryLoader} class, either by @@ -1748,7 +1748,7 @@ Each item in this tuple contains further information about a parameter, it must be a tuple containing 1, 2, or 3 items. -The first item is an integer containing flags for the parameter. +The first item is an integer containing flags for the parameter: \begin{datadescni}{1} Specifies an input parameter to the function. @@ -2374,7 +2374,7 @@ \code{u} field is defined as anonymous field, it is now possible to access the members directly off the TYPEDESC instance. \code{td.lptdesc} and \code{td.u.lptdesc} are equivalent, but the former -is faster since it does not need to create a temporary \code{{\_}U} +is faster since it does not need to create a temporary union instance: \begin{verbatim} td = TYPEDESC() @@ -2386,7 +2386,7 @@ It is possible to defined sub-subclasses of structures, they inherit the fields of the base class. If the subclass definition has a -separate``{\_}fields{\_}`` variable, the fields specified in this are +separate \member{{\_}fields{\_}} variable, the fields specified in this are appended to the fields of the base class. From buildbot at python.org Thu Jun 22 21:16:51 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 19:16:51 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060622191651.A5C8E1E400B@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/1074 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 22 22:06:46 2006 From: python-checkins at python.org (peter.astrand) Date: Thu, 22 Jun 2006 22:06:46 +0200 (CEST) Subject: [Python-checkins] r47076 - python/trunk/Lib/test/test_subprocess.py Message-ID: <20060622200646.A4FC21E400B@bag.python.org> Author: peter.astrand Date: Thu Jun 22 22:06:46 2006 New Revision: 47076 Modified: python/trunk/Lib/test/test_subprocess.py Log: Make it possible to run test_subprocess.py on Python 2.2, which lacks test_support.is_resource_enabled. Modified: python/trunk/Lib/test/test_subprocess.py ============================================================================== --- python/trunk/Lib/test/test_subprocess.py (original) +++ python/trunk/Lib/test/test_subprocess.py Thu Jun 22 22:06:46 2006 @@ -384,7 +384,8 @@ def test_no_leaking(self): # Make sure we leak no resources - if test_support.is_resource_enabled("subprocess") and not mswindows: + if not hasattr(test_support, "is_resource_enabled") \ + or test_support.is_resource_enabled("subprocess") and not mswindows: max_handles = 1026 # too much for most UNIX systems else: max_handles = 65 From python-checkins at python.org Thu Jun 22 22:21:26 2006 From: python-checkins at python.org (peter.astrand) Date: Thu, 22 Jun 2006 22:21:26 +0200 (CEST) Subject: [Python-checkins] r47077 - python/trunk/Lib/popen2.py python/trunk/Lib/subprocess.py Message-ID: <20060622202126.7299F1E4008@bag.python.org> Author: peter.astrand Date: Thu Jun 22 22:21:26 2006 New Revision: 47077 Modified: python/trunk/Lib/popen2.py python/trunk/Lib/subprocess.py Log: Applied patch #1506758: Prevent MemoryErrors with large MAXFD. Modified: python/trunk/Lib/popen2.py ============================================================================== --- python/trunk/Lib/popen2.py (original) +++ python/trunk/Lib/popen2.py Thu Jun 22 22:21:26 2006 @@ -79,7 +79,7 @@ def _run_child(self, cmd): if isinstance(cmd, basestring): cmd = ['/bin/sh', '-c', cmd] - for i in range(3, MAXFD): + for i in xrange(3, MAXFD): try: os.close(i) except OSError: Modified: python/trunk/Lib/subprocess.py ============================================================================== --- python/trunk/Lib/subprocess.py (original) +++ python/trunk/Lib/subprocess.py Thu Jun 22 22:21:26 2006 @@ -941,7 +941,7 @@ def _close_fds(self, but): - for i in range(3, MAXFD): + for i in xrange(3, MAXFD): if i == but: continue try: From python-checkins at python.org Thu Jun 22 22:28:33 2006 From: python-checkins at python.org (peter.astrand) Date: Thu, 22 Jun 2006 22:28:33 +0200 (CEST) Subject: [Python-checkins] r47078 - python/branches/release24-maint/Lib/popen2.py python/branches/release24-maint/Lib/subprocess.py Message-ID: <20060622202833.7ACC21E4008@bag.python.org> Author: peter.astrand Date: Thu Jun 22 22:28:33 2006 New Revision: 47078 Modified: python/branches/release24-maint/Lib/popen2.py python/branches/release24-maint/Lib/subprocess.py Log: Applied patch #1506758: Prevent MemoryErrors with large MAXFD. Backport of 47077. Modified: python/branches/release24-maint/Lib/popen2.py ============================================================================== --- python/branches/release24-maint/Lib/popen2.py (original) +++ python/branches/release24-maint/Lib/popen2.py Thu Jun 22 22:28:33 2006 @@ -65,7 +65,7 @@ def _run_child(self, cmd): if isinstance(cmd, basestring): cmd = ['/bin/sh', '-c', cmd] - for i in range(3, MAXFD): + for i in xrange(3, MAXFD): try: os.close(i) except OSError: Modified: python/branches/release24-maint/Lib/subprocess.py ============================================================================== --- python/branches/release24-maint/Lib/subprocess.py (original) +++ python/branches/release24-maint/Lib/subprocess.py Thu Jun 22 22:28:33 2006 @@ -867,7 +867,7 @@ def _close_fds(self, but): - for i in range(3, MAXFD): + for i in xrange(3, MAXFD): if i == but: continue try: From buildbot at python.org Thu Jun 22 22:33:43 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 20:33:43 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060622203344.03EC81E4008@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/405 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl,ronald.oussoren,thomas.heller Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Thu Jun 22 22:57:48 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 22 Jun 2006 20:57:48 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo 2.4 Message-ID: <20060622205748.D9CD11E4008@bag.python.org> The Buildbot has detected a new failure of x86 gentoo 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%25202.4/builds/181 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: peter.astrand Build Had Warnings: warnings test sincerely, -The Buildbot From neal at metaslash.com Thu Jun 22 23:27:13 2006 From: neal at metaslash.com (Neal Norwitz) Date: Thu, 22 Jun 2006 17:27:13 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060622212713.GA4107@python.psfb.org> test_descr leaked [25, 25, 25] references From python-checkins at python.org Fri Jun 23 05:32:45 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 23 Jun 2006 05:32:45 +0200 (CEST) Subject: [Python-checkins] r47079 - python/trunk/Objects/typeobject.c Message-ID: <20060623033245.351121E4008@bag.python.org> Author: neal.norwitz Date: Fri Jun 23 05:32:44 2006 New Revision: 47079 Modified: python/trunk/Objects/typeobject.c Log: Fix refleak Modified: python/trunk/Objects/typeobject.c ============================================================================== --- python/trunk/Objects/typeobject.c (original) +++ python/trunk/Objects/typeobject.c Fri Jun 23 05:32:44 2006 @@ -4595,8 +4595,10 @@ the object returned for __call__ has __call__ itself defined upon it. This can be an infinite recursion if you set __call__ in a class to an instance of it. */ - if (Py_EnterRecursiveCall(" in __call__")) + if (Py_EnterRecursiveCall(" in __call__")) { + Py_DECREF(meth); return NULL; + } res = PyObject_Call(meth, args, kwds); Py_LeaveRecursiveCall(); From buildbot at python.org Fri Jun 23 07:11:46 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 23 Jun 2006 05:11:46 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060623051146.E84811E4009@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/407 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 23 08:03:45 2006 From: python-checkins at python.org (fred.drake) Date: Fri, 23 Jun 2006 08:03:45 +0200 (CEST) Subject: [Python-checkins] r47080 - in python/trunk/Lib: sgmllib.py test/test_sgmllib.py Message-ID: <20060623060345.C3C3F1E4008@bag.python.org> Author: fred.drake Date: Fri Jun 23 08:03:45 2006 New Revision: 47080 Modified: python/trunk/Lib/sgmllib.py python/trunk/Lib/test/test_sgmllib.py Log: - SF bug #853506: IP6 address parsing in sgmllib ('[' and ']' were not accepted in unquoted attribute values) - cleaned up tests of character and entity reference decoding so the tests cover the documented relationships among handle_charref, handle_entityref, convert_charref, convert_codepoint, and convert_entityref, without bringing up Unicode issues that sgmllib cannot be involved in Modified: python/trunk/Lib/sgmllib.py ============================================================================== --- python/trunk/Lib/sgmllib.py (original) +++ python/trunk/Lib/sgmllib.py Fri Jun 23 08:03:45 2006 @@ -33,7 +33,7 @@ tagfind = re.compile('[a-zA-Z][-_.a-zA-Z0-9]*') attrfind = re.compile( r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' - r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$_#=~\'"@]*))?') + r'(\'[^\']*\'|"[^"]*"|[][\-a-zA-Z0-9./,:;+*%?!&$_#=~\'"@]*))?') class SGMLParseError(RuntimeError): @@ -400,11 +400,11 @@ def handle_charref(self, name): """Handle character reference, no need to override.""" - replacement = convert_charref(name) + replacement = self.convert_charref(name) if replacement is None: self.unknown_charref(name) else: - self.handle_data(convert_charref(name)) + self.handle_data(replacement) # Definition of entities -- derived classes may override entitydefs = \ Modified: python/trunk/Lib/test/test_sgmllib.py ============================================================================== --- python/trunk/Lib/test/test_sgmllib.py (original) +++ python/trunk/Lib/test/test_sgmllib.py Fri Jun 23 08:03:45 2006 @@ -1,4 +1,6 @@ +import htmlentitydefs import pprint +import re import sgmllib import unittest from test import test_support @@ -65,20 +67,34 @@ class HTMLEntityCollector(EventCollector): - import re, htmlentitydefs + entity_or_charref = re.compile('(?:&([a-zA-Z][-.a-zA-Z0-9]*)' '|&#(x[0-9a-zA-Z]+|[0-9]+))(;?)') def convert_charref(self, name): self.append(("charref", "convert", name)) - if name.startswith('x'): - return unichr(int(name[1:],16)) - else: - return unichr(int(name)) + if name[0] != "x": + return EventCollector.convert_charref(self, name) + + def convert_codepoint(self, codepoint): + self.append(("codepoint", "convert", codepoint)) + EventCollector.convert_codepoint(self, codepoint) def convert_entityref(self, name): self.append(("entityref", "convert", name)) - return unichr(self.htmlentitydefs.name2codepoint[name]) + return EventCollector.convert_entityref(self, name) + + # These to record that they were called, then pass the call along + # to the default implementation so that it's actions can be + # recorded. + + def handle_charref(self, data): + self.append(("charref", data)) + sgmllib.SGMLParser.handle_charref(self, data) + + def handle_entityref(self, data): + self.append(("entityref", data)) + sgmllib.SGMLParser.handle_entityref(self, data) class SGMLParserTestCase(unittest.TestCase): @@ -251,13 +267,23 @@ ])]) def test_convert_overrides(self): + # This checks that the character and entity reference + # conversion helpers are called at the documented times. No + # attempt is made to really change what the parser accepts. + # self.collector = HTMLEntityCollector - self.check_events('foo', [ + self.check_events(('foo' + '&foobar;*'), [ ('entityref', 'convert', 'ldquo'), ('charref', 'convert', 'x201d'), - ('starttag', 'a', [('title', u'\u201ctest\u201d')]), + ('starttag', 'a', [('title', '“test”')]), ('data', 'foo'), ('endtag', 'a'), + ('entityref', 'foobar'), + ('entityref', 'convert', 'foobar'), + ('charref', '42'), + ('charref', 'convert', '42'), + ('codepoint', 'convert', 42), ]) def test_attr_funky_names(self): @@ -265,6 +291,14 @@ ("starttag", "a", [("a.b", "v"), ("c:d", "v"), ("e-f", "v")]), ]) + def test_attr_value_ip6_url(self): + # http://www.python.org/sf/853506 + self.check_events(("" + ""), [ + ("starttag", "a", [("href", "http://[1080::8:800:200C:417A]/")]), + ("starttag", "a", [("href", "http://[1080::8:800:200C:417A]/")]), + ]) + def test_illegal_declarations(self): s = 'abcdef' self.check_events(s, [ From python-checkins at python.org Fri Jun 23 08:12:32 2006 From: python-checkins at python.org (fred.drake) Date: Fri, 23 Jun 2006 08:12:32 +0200 (CEST) Subject: [Python-checkins] r47081 - in python/branches/release24-maint/Lib: sgmllib.py test/test_sgmllib.py Message-ID: <20060623061232.089E71E400C@bag.python.org> Author: fred.drake Date: Fri Jun 23 08:12:31 2006 New Revision: 47081 Modified: python/branches/release24-maint/Lib/sgmllib.py python/branches/release24-maint/Lib/test/test_sgmllib.py Log: - SF bug #853506: IP6 address parsing in sgmllib ('[' and ']' were not accepted in unquoted attribute values) Modified: python/branches/release24-maint/Lib/sgmllib.py ============================================================================== --- python/branches/release24-maint/Lib/sgmllib.py (original) +++ python/branches/release24-maint/Lib/sgmllib.py Fri Jun 23 08:12:31 2006 @@ -33,7 +33,7 @@ tagfind = re.compile('[a-zA-Z][-_.a-zA-Z0-9]*') attrfind = re.compile( r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' - r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$_#=~\'"@]*))?') + r'(\'[^\']*\'|"[^"]*"|[][\-a-zA-Z0-9./,:;+*%?!&$_#=~\'"@]*))?') class SGMLParseError(RuntimeError): Modified: python/branches/release24-maint/Lib/test/test_sgmllib.py ============================================================================== --- python/branches/release24-maint/Lib/test/test_sgmllib.py (original) +++ python/branches/release24-maint/Lib/test/test_sgmllib.py Fri Jun 23 08:12:31 2006 @@ -219,6 +219,14 @@ ("starttag", "a", [("a.b", "v"), ("c:d", "v"), ("e-f", "v")]), ]) + def test_attr_value_ip6_url(self): + # http://www.python.org/sf/853506 + self.check_events(("" + ""), [ + ("starttag", "a", [("href", "http://[1080::8:800:200C:417A]/")]), + ("starttag", "a", [("href", "http://[1080::8:800:200C:417A]/")]), + ]) + def test_illegal_declarations(self): s = 'abcdef' self.check_events(s, [ From buildbot at python.org Fri Jun 23 08:17:42 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 23 Jun 2006 06:17:42 +0000 Subject: [Python-checkins] buildbot failure in amd64 gentoo trunk Message-ID: <20060623061742.76AD41E4008@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1135 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From buildbot at python.org Fri Jun 23 08:22:21 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 23 Jun 2006 06:22:21 +0000 Subject: [Python-checkins] buildbot failure in amd64 gentoo 2.4 Message-ID: <20060623062221.56F441E4008@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%25202.4/builds/180 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: fred.drake BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From python-checkins at python.org Fri Jun 23 12:59:58 2006 From: python-checkins at python.org (matt.fleming) Date: Fri, 23 Jun 2006 12:59:58 +0200 (CEST) Subject: [Python-checkins] r47082 - in sandbox/trunk/pdb: README.txt doc mconnection.py mpdb.py mthread.py test/test_mconnection.py test/test_mpdb.py Message-ID: <20060623105958.6EF271E4009@bag.python.org> Author: matt.fleming Date: Fri Jun 23 12:59:57 2006 New Revision: 47082 Added: sandbox/trunk/pdb/doc/ Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/mthread.py sandbox/trunk/pdb/test/test_mconnection.py sandbox/trunk/pdb/test/test_mpdb.py Log: Introduce protocol-ignorant exceptions for mconnection which makes it eaiser for mpdb to look for the same exceptions regardless of which protocol is being used for remote debugging. Add a doc directory for documentation which I'll start to fill in soon. Update the README.txt to reflect dicussion via e-mail my mentor, regarding what tasks are to be completed. Fix some test cases and documentat _why_ we need to have each connection class implement a flush method. Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Fri Jun 23 12:59:57 2006 @@ -12,5 +12,14 @@ * Write more unit tests, test the pdbserver and target commands. * Write a signal handler that scripts can import from mpdb that, when the signal is received, start remote debugging. -* info target,threads command needs to be written. +* info [target/threads] + set debug threads [on|off] + show debug threads, command needs to be written. +* Implement thread debugging, commands to switch between threads. +* Lightweight and heavyweight mechanism for setting up threads. +* Write a proper entry point to mpdb (a rewrite of the main method in mpdb.py). +* Extend mconnection FIFO's +* mconnection should use the exceptions that have been introduced and mpdb + should check for these exceptions being raised. +* Write documentation Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Fri Jun 23 12:59:57 2006 @@ -5,11 +5,11 @@ NotImplementedMessage = "This method must be overriden in a subclass" ### Exceptions -class ConnectionRefused(Exception): pass +class ConnectionFailed(Exception): pass class DroppedConnection(Exception): pass class ReadOnClose(Exception): pass -class MServerConnectionInterface(object): +class MConnectionServerInterface(object): """ This is an abstract class that specifies the interface a server connection class must implement. If a target is given, we'll set up a connection on that target @@ -35,7 +35,7 @@ """ raise NotImplementedError, NotImplementedMessage -class MClientConnectionInterface(object): +class MConnectionClientInterface(object): """ This is the interface that a client connection should implement. """ def connect(self, target): @@ -59,14 +59,14 @@ ### This might go in a different file -class MServerConnectionSerial(MServerConnectionInterface): +class MConnectionServerSerial(MConnectionServerInterface): """ This server connection class that allows a connection to a target via a serial line. """ def __init__(self): - MServerConnectionInterface.__init__(self) + MConnectionServerInterface.__init__(self) self.input = None self.output = None @@ -100,19 +100,19 @@ self.output.write(msg) self.output.flush() -MClientConnectionSerial = MServerConnectionSerial +MConnectionClientSerial = MConnectionServerSerial ### This might go in a different file import socket -class MServerConnectionTCP(MServerConnectionInterface): +class MConnectionServerTCP(MConnectionServerInterface): """This is an implementation of a server class that uses the TCP protocol as its means of communication. """ def __init__(self): self.listening = False self._sock = self.output = self.input = None - MServerConnectionInterface.__init__(self) + MConnectionServerInterface.__init__(self) def connect(self, addr): """Set to allow a connection from a client. 'addr' specifies @@ -126,8 +126,9 @@ self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) try: self._sock.bind((self.host, self.port)) - except socket.error: - raise ConnectionRefused + except socket.error, e: + # Use e[1] as a more detailed error message + raise ConnectionFailed, e[1] self._sock.listen(1) self.listening = True self.output, addr = self._sock.accept() @@ -147,19 +148,16 @@ if line[-1].isalpha(): line += '\n' return line - def flush(self): - pass - def write(self, msg): self.output.sendall(msg) -class MClientConnectionTCP(MClientConnectionInterface): +class MConnectionClientTCP(MConnectionClientInterface): """ A class that allows a connection to be made from a debugger to a server via TCP. """ def __init__(self): """ Specify the address to connection to. """ - MClientConnectionInterface.__init__(self) + MConnectionClientInterface.__init__(self) self._sock = self.output = self.input = None def connect(self, addr): @@ -180,9 +178,6 @@ line = self._sock.recv(bufsize) return line - def flush(self): - pass - def disconnect(self): """ Close the socket to the server. """ # We shouldn't bail if we haven't been connected yet Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Fri Jun 23 12:59:57 2006 @@ -19,6 +19,7 @@ import pydb import socket import sys +import time import traceback line_prefix = '\n-> ' @@ -45,6 +46,7 @@ self.prompt = '(MPdb)' self.target = 'local' # local connections by default self.connection = None + self.debug_thread = False def _rebind_input(self, new_input): """ This method rebinds the debugger's input to the object specified @@ -59,6 +61,13 @@ """ self.stdout.flush() self.stdout = new_output + if not hasattr(self.stdout, 'flush'): + # Add a dummy flush method because cmdloop() in cmd.py + # uses this code: + # self.stdout.write(self.prompt) + # self.stdout.flush() + # line = self.readline() + self.stdout.flush = lambda: None def remote_onecmd(self, line): """ All commands in 'line' are sent across this object's connection @@ -100,6 +109,36 @@ else: pydb.Pdb.do_info(self, arg) + def do_thread(self, arg): + """ Enable thread debugging. """ + # XXX Rocky, how are we subclassing info/set commands? This will do + # for now. + if arg == 'info': + if not self.debug_thread: + self.msg('Thread debugging is not on.') + return + try: + import threading + except ImportError: + self.msg('Thread debugging not available.') + self.msg(threading.enumerate()) + return + if arg == 'debug': + try: + import threading + import mthread + except ImportError: + self.msg('Thread debugging not available.') + # We do not continue with the main thread for as long as there + # is another thread running. (Will change later so you can choose + # between threads). + ev = threading.Event() + mthread.set_event(ev) + threading.settrace(mthread.trace_dispatch) + self.msg('Thread debugging on.') + self.debug_thread = True + return + # Debugger commands def do_attach(self, addr): """ Attach to a process or file outside of Pdb. @@ -139,21 +178,23 @@ # TODO: need to save state of current debug session if self.connection: self.connection.disconnect() try: - from mconnection import MClientConnectionTCP + from mconnection import (MConnectionClientTCP, + ConnectionFailed) # Matt - Where are the connection parameters? - self.connection = MClientConnectionTCP() + self.connection = MConnectionClientTCP() except ImportError: - self.msg('Could not import MClientConnectionTCP') + self.msg('Could not import MConnectionClientTCP') return elif target == 'serial': # Matt - Where are the connection parameters? if self.connection: self.connection.disconnect() try: - from mconnection import MClientConnectionSerial - self.connection = MClientConnectionSerial() + from mconnection import (MConnectionClientSerial, + ConnectionFailed) + self.connection = MConnectionClientSerial() except ImportError: - self.msg('Could not import MClientConnectionSerial') + self.msg('Could not import MConnectionClientSerial') return else: if '.' in target: @@ -161,7 +202,7 @@ # We dynamically load the class for the connection base = target[:target.rfind('.')] cls = target[target.rfind('.')+1:] - exec 'from ' + base + ' import ' + cls + exec 'from ' + base + ' import (' + cls + ', ConnectionFailed)' else: try: __import__(target) @@ -171,17 +212,15 @@ self.connection = eval(target+'()') try: self.connection.connect(addr) - except socket.error: - # Matt: I couldn't figure out what the right - # exception name was to use that getts the error message. - self.msg("Failed to connect to %s" % addr) + except ConnectionFailed, err: + self.msg("Failed to connect to %s: (%s)" % (addr, err)) return # This interpreter no longer interprets commands but sends # them straight across this object's connection to a server. self.prompt = "" # Get our prompt from the server now line = self.connection.readline() self.msg_nocr(line) - self.msg = self.connection.write + self._rebind_output(self.connection) self.onecmd = self.remote_onecmd self.target = 'remote' @@ -218,23 +257,29 @@ return if target == 'tcp': try: - from mconnection import MServerConnectionTCP - self.connection = MServerConnectionTCP() + from mconnection import (MConnectionServerTCP, + ConnectionFailed) + self.connection = MConnectionServerTCP() except ImportError: - self.msg('Could not load MServerConnectionTCP class') + self.msg('Could not load MConnectionServerTCP class') return else: if '.' in target: base = target[:target.rfind('.')] target = target[target.rfind('.')+1:] - exec 'from ' + base + ' import ' + target + exec 'from ' + base + ' import (' + target + \ + ', ConnectionFailed)' else: __import__(target) self.connection = eval(target+'()') - self.connection.connect(comm) + try: + self.connection.connect(comm) + except ConnectionFailed, err: + self.msg("Failed to connect to %s: (%s)" % (addr, err)) + return self.target = 'remote' - self._rebind_output(self.connection) self._rebind_input(self.connection) + self._rebind_output(self.connection) # This is a mess. It's only here so that I can test other features of the # debugger whilst I'm writing them. It will be removed at some point. @@ -252,7 +297,7 @@ mpdb._runscript(mainpyfile) if mpdb._user_requested_quit: break - self.msg("The program finished and will be restarted") + mpdb.msg("The program finished and will be restarted") except SystemExit: # In most cases SystemExit does not warrant a post-mortem session. mpdb.msg("The program exited via sys.exit(). " + \ Modified: sandbox/trunk/pdb/mthread.py ============================================================================== --- sandbox/trunk/pdb/mthread.py (original) +++ sandbox/trunk/pdb/mthread.py Fri Jun 23 12:59:57 2006 @@ -1,97 +1,61 @@ """ This file contains all code for allowing the debugging of threads. """ import sys -import thread import threading import time -from mpdb import MPdb -# Globals -STOP = False -t_dict = {} -t_current = None - -class MyThread(threading.Thread): - def run(self): - while True: - if STOP: break - x = 2 +# Globals that are private to this file +_threads = [] +_stop = False +g_event = None -class MTracer(MPdb): +class MTracer(object): """ A class to trace a thread. """ - def __init__(self): - MPdb.__init__(self) - self.quitting = False - self.botframe = None - self.stopframe = None - self.returnframe = None - self.thread = threading.currentThread().getName() + def __init__(self, event): + self.thread = threading.currentThread() def trace_dispatch(self, frame, event, arg): - try: - if not t_dict[self.thread]: - return - except KeyError: - return - if self.quitting: - return # None if event == 'line': - return self.dispatch_line(frame) + print '*** line' + return self.trace_dispatch if event == 'call': - return self.dispatch_call(frame, arg) + print '*** call' + return self.trace_dispatch if event == 'return': - return self.dispatch_return(frame, arg) + print '*** return' + return self.trace_dispatch if event == 'exception': - return self.dispatch_exception(frame, arg) + print '*** exception' + return self.trace_dispatch if event == 'c_call': + print '*** c_call' return self.trace_dispatch if event == 'c_exception': + print '*** c_exception' return self.trace_dispatch if event == 'c_return': + print '*** c_return' return self.trace_dispatch print 'bdb.Bdb.dispatch: unknown debugging event:', repr(event) return self.trace_dispatch +def trace_dispatch(frame, event, arg): + """ Create a new MTracer object for a thread and set that thread's + tracing function to the MTracer objects trace method. + """ + m = MTracer(g_event) + global _threads + _threads.append(threading.currentThread()) - # Override Pdb methods + sys.settrace(m.trace_dispatch) - def interaction(self, frame, traceback): - print "Thread[%s]: %s" % (self.thread, frame.f_code) +def set_event(e): + global g_event + g_event = e -def trace_dispatch(self, frame, somethinelse): - """ Create a new MTracer object for a thread and set that threads - tracing function to the MTracer objects trace method. Put this thread - into the global dict of threads that we can trace. - """ - global t_current - m = MTracer() - # This is supposed to simulate the idea of the first thread becomes - # the current thread we're debugging. In mpdb the user will have to - # manually change the currently debugged thread with a command. - # This thread is not necessarily the first 'created' in a script, but - # the first to be 'run'. - if t_current is None: - global t_dict - t_current = m.thread - t_dict[m.thread] = True - sys.settrace(m.trace_dispatch) -def threads(): - # Set the globla tracing function - threading.settrace(trace_dispatch) - - global STOP, t_dict - - t_list = [] - # Create 3 threads - for i in range(3): - t = MyThread() - t.start() - t_list.append(t) + - STOP = True + -if __name__ == '__main__': - threads() - Modified: sandbox/trunk/pdb/test/test_mconnection.py ============================================================================== --- sandbox/trunk/pdb/test/test_mconnection.py (original) +++ sandbox/trunk/pdb/test/test_mconnection.py Fri Jun 23 12:59:57 2006 @@ -18,9 +18,9 @@ TESTFN = 'device' sys.path.append("..") -from mconnection import (MServerConnectionTCP, MClientConnectionTCP, - MServerConnectionSerial, MClientConnectionSerial, - ConnectionRefused) +from mconnection import (MConnectionServerTCP, MConnectionClientTCP, + MConnectionServerSerial, MConnectionClientSerial, + ConnectionFailed) # Try to connect the client to addr either until we've tried MAXTRIES # times or until it succeeds. @@ -35,8 +35,8 @@ class TestTCPConnections(unittest.TestCase): def setUp(self): - self.server = MServerConnectionTCP() - self.client = MClientConnectionTCP() + self.server = MConnectionServerTCP() + self.client = MConnectionClientTCP() def testClientConnectToServer(self): """(tcp) Connect client to server. """ @@ -57,7 +57,7 @@ def testDisconnectDisconnected(self): """(tcp) Disconnect a disconnected session. """ - s = MServerConnectionTCP() + s = MConnectionServerTCP() s.disconnect() s.disconnect() @@ -70,14 +70,14 @@ line = self.server.readline() self.assertEquals('good\n', line, 'Could not read first line.') - def testConnectionRefused(self): - """(tcp) Test refused connection. """ + def testErrorAddressAlreadyInUse(self): + """(tcp) Test address already in use error. """ thread.start_new_thread(repeatedConnect, (self.client, __addr__)) self.server.connect(__addr__) # Set up second server on same port - s = MServerConnectionTCP() - self.assertRaises(ConnectionRefused, s.connect, __addr__) + s = MConnectionServerTCP() + self.assertRaises(ConnectionFailed, s.connect, __addr__) def tearDown(self): self.server.disconnect() @@ -88,8 +88,8 @@ on *nix systems is just files anyway. """ def setUp(self): - self.server = MServerConnectionSerial() - self.client = MClientConnectionSerial() + self.server = MConnectionServerSerial() + self.client = MConnectionClientSerial() fd = open(TESTFN, "wr+") fd.close() self.server.connect(TESTFN) Modified: sandbox/trunk/pdb/test/test_mpdb.py ============================================================================== --- sandbox/trunk/pdb/test/test_mpdb.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Fri Jun 23 12:59:57 2006 @@ -18,7 +18,6 @@ sys.path.append("..") from mpdb import MPdb -from mconnection import MServerConnectionTCP, MClientConnectionTCP def doTargetConnect(cmds=None): global g_client From python-checkins at python.org Fri Jun 23 13:18:42 2006 From: python-checkins at python.org (matt.fleming) Date: Fri, 23 Jun 2006 13:18:42 +0200 (CEST) Subject: [Python-checkins] r47083 - in sandbox/trunk/pdb: mconnection.py mpdb.py test/test_mconnection.py Message-ID: <20060623111842.272CB1E4009@bag.python.org> Author: matt.fleming Date: Fri Jun 23 13:18:41 2006 New Revision: 47083 Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/test/test_mconnection.py Log: There should be no distinction between serial servers and clients. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Fri Jun 23 13:18:41 2006 @@ -59,9 +59,11 @@ ### This might go in a different file -class MConnectionServerSerial(MConnectionServerInterface): +# Not, serial protocol does not require the distinction between server and +# client. +class MConnectionSerial(MConnectionServerInterface): - """ This server connection class that allows a connection to a + """ This connection class that allows a connection to a target via a serial line. """ @@ -79,8 +81,9 @@ try: self.output = open(self._dev, 'w') self.input = open(self._dev, 'r') - except IOError: - raise ConnectionRefused + except IOError,e: + # Use e[1] for more detail about why the connection failed + raise ConnectionFailed, e[1] def disconnect(self): """ Close the serial device. """ @@ -100,7 +103,6 @@ self.output.write(msg) self.output.flush() -MConnectionClientSerial = MConnectionServerSerial ### This might go in a different file import socket Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Fri Jun 23 13:18:41 2006 @@ -190,11 +190,11 @@ # Matt - Where are the connection parameters? if self.connection: self.connection.disconnect() try: - from mconnection import (MConnectionClientSerial, + from mconnection import (MConnectionSerial, ConnectionFailed) - self.connection = MConnectionClientSerial() + self.connection = MConnectionSerial() except ImportError: - self.msg('Could not import MConnectionClientSerial') + self.msg('Could not import MConnectionSerial') return else: if '.' in target: @@ -263,6 +263,14 @@ except ImportError: self.msg('Could not load MConnectionServerTCP class') return + elif target == 'serial': + try: + from mconnection import (MConnectionSerial, + ConnectionFailed) + self.connection = MConnectionSerial() + except ImportError: + self.msg('Could not load MConnectionSerial class') + return else: if '.' in target: base = target[:target.rfind('.')] @@ -275,7 +283,7 @@ try: self.connection.connect(comm) except ConnectionFailed, err: - self.msg("Failed to connect to %s: (%s)" % (addr, err)) + self.msg("Failed to connect to %s: (%s)" % (comm, err)) return self.target = 'remote' self._rebind_input(self.connection) Modified: sandbox/trunk/pdb/test/test_mconnection.py ============================================================================== --- sandbox/trunk/pdb/test/test_mconnection.py (original) +++ sandbox/trunk/pdb/test/test_mconnection.py Fri Jun 23 13:18:41 2006 @@ -19,8 +19,7 @@ sys.path.append("..") from mconnection import (MConnectionServerTCP, MConnectionClientTCP, - MConnectionServerSerial, MConnectionClientSerial, - ConnectionFailed) + MConnectionSerial, ConnectionFailed) # Try to connect the client to addr either until we've tried MAXTRIES # times or until it succeeds. @@ -88,8 +87,8 @@ on *nix systems is just files anyway. """ def setUp(self): - self.server = MConnectionServerSerial() - self.client = MConnectionClientSerial() + self.server = MConnectionSerial() + self.client = MConnectionSerial() fd = open(TESTFN, "wr+") fd.close() self.server.connect(TESTFN) From python-checkins at python.org Fri Jun 23 21:06:28 2006 From: python-checkins at python.org (matt.fleming) Date: Fri, 23 Jun 2006 21:06:28 +0200 (CEST) Subject: [Python-checkins] r47084 - in sandbox/trunk/pdb: README.txt mconnection.py mpdb.py mthread.py test/test_mconnection.py test/test_mpdb.py Message-ID: <20060623190628.F1D431E4009@bag.python.org> Author: matt.fleming Date: Fri Jun 23 21:06:28 2006 New Revision: 47084 Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/mthread.py sandbox/trunk/pdb/test/test_mconnection.py sandbox/trunk/pdb/test/test_mpdb.py Log: Added test cases, got the beginnings of thread debugging working. Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Fri Jun 23 21:06:28 2006 @@ -6,20 +6,42 @@ -=[Abstract]=- This project is part of a Google Summer of Code 2006 project. Many people have stated that they would like to see improvements in pdb. This projects -aims to correct this wish. +aims to fulfill this wish. -=[TODO]=- -* Write more unit tests, test the pdbserver and target commands. +* Write more unit tests * Write a signal handler that scripts can import from mpdb that, when the signal is received, start remote debugging. * info [target/threads] set debug threads [on|off] show debug threads, command needs to be written. +* Decide on a way to execute commands for a specific thread. * Implement thread debugging, commands to switch between threads. -* Lightweight and heavyweight mechanism for setting up threads. -* Write a proper entry point to mpdb (a rewrite of the main method in mpdb.py). + - Because the 'main' thread may exit unexpectedly we need to keep + any other threads that we're debugging alive. This may mean that + we have to write some code when the main debugger exits to check + for any threads being debugged and not exit until all threads have + finished (or at least ask if they're sure they wanna leave). +* Lightweight and heavyweight mechanism for setting up threads + - The reason we want a lightweight mechanism is so that we can + place mpdb.set_trace() inside a script so that we can debug + any threads created with the threading module. It has to be + lighweight because the programmer might not want all the features + of an MPdb instance, if for example they only care about debugging + this one thread instance. + - We need a heavyweight mechanism to allow a programmer to inspect + and control all threads. +* Provide a proper top-level methods including, set_trace(), post_mortem(), + run(), remote_sighandler() (for allowing a signal to start + remote debugging) * Extend mconnection FIFO's * mconnection should use the exceptions that have been introduced and mpdb should check for these exceptions being raised. * Write documentation + - Debugger commands + - Debugger model/architecture: + - Debugging outside a process + - Debugging remotely + - Debugging threads + Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Fri Jun 23 21:06:28 2006 @@ -7,9 +7,10 @@ ### Exceptions class ConnectionFailed(Exception): pass class DroppedConnection(Exception): pass -class ReadOnClose(Exception): pass +class ReadError(Exception): pass +class WriteError(Exception): pass -class MConnectionServerInterface(object): +class MConnectionInterface(object): """ This is an abstract class that specifies the interface a server connection class must implement. If a target is given, we'll set up a connection on that target @@ -35,40 +36,17 @@ """ raise NotImplementedError, NotImplementedMessage -class MConnectionClientInterface(object): - """ This is the interface that a client connection should implement. - """ - def connect(self, target): - """ This method is called to connect to a target. """ - raise NotImplementedError, NotImplementedMessage - - def disconnect(self): - """ This method use to disconnect a target.""" - raise NotImplementedError, NotImplementedMessage - - def readline(self, bufsize): - """ This method reads a line of data of maxium length 'bufisze' - from a connected target. - """ - raise NotImplementedError, NotImplementedMessage - - def write(self, msg): - """ This method is called write 'msg' to the connected target. - """ - raise NotImplementedError, NotImplementedMessage - - ### This might go in a different file # Not, serial protocol does not require the distinction between server and # client. -class MConnectionSerial(MConnectionServerInterface): +class MConnectionSerial(MConnectionInterface): """ This connection class that allows a connection to a target via a serial line. """ def __init__(self): - MConnectionServerInterface.__init__(self) + MConnectionInterface.__init__(self) self.input = None self.output = None @@ -94,27 +72,33 @@ def readline(self, bufsize=2048): - line = self.input.readline(bufsize) + try: + line = self.input.readline(bufsize) + except IOError, e: + raise ReadError, e[1] return line def write(self, msg): if msg[-1] is not '\n': msg += '\n' - self.output.write(msg) - self.output.flush() + try: + self.output.write(msg) + self.output.flush() + except IOError, e: + raise WriteError, e[1] ### This might go in a different file import socket -class MConnectionServerTCP(MConnectionServerInterface): +class MConnectionServerTCP(MConnectionInterface): """This is an implementation of a server class that uses the TCP protocol as its means of communication. """ def __init__(self): self.listening = False self._sock = self.output = self.input = None - MConnectionServerInterface.__init__(self) + MConnectionInterface.__init__(self) def connect(self, addr): """Set to allow a connection from a client. 'addr' specifies @@ -137,8 +121,7 @@ self.input = self.output def disconnect(self): - # These two should either _both_ be None, or neither should be None - if self.output is None and self._sock is None: + if self.output is None or self._sock is None: return self.output.close() self._sock.close() @@ -146,20 +129,26 @@ self.listening = False def readline(self, bufsize=2048): - line = self.input.recv(bufsize) + try: + line = self.input.recv(bufsize) + except socket.error, e: + raise ReadError, e[1] if line[-1].isalpha(): line += '\n' return line def write(self, msg): - self.output.sendall(msg) + try: + self.output.sendall(msg) + except socket.error, e: + raise WriteError, e[1] -class MConnectionClientTCP(MConnectionClientInterface): +class MConnectionClientTCP(MConnectionInterface): """ A class that allows a connection to be made from a debugger to a server via TCP. """ def __init__(self): """ Specify the address to connection to. """ - MConnectionClientInterface.__init__(self) + MConnectionInterface.__init__(self) self._sock = self.output = self.input = None def connect(self, addr): @@ -171,13 +160,22 @@ self.host = h self.port = int(p) self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self._sock.connect((self.host, self.port)) + try: + self._sock.connect((self.host, self.port)) + except socket.error, e: + raise ConnectionFailed, e[1] def write(self, msg): - self._sock.sendall(msg) + try: + self._sock.sendall(msg) + except socket.error, e: + raise WriteError, e[1] def readline(self, bufsize=2048): - line = self._sock.recv(bufsize) + try: + line = self._sock.recv(bufsize) + except socket.error, e: + raise ReadError, e[1] return line def disconnect(self): Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Fri Jun 23 21:06:28 2006 @@ -20,8 +20,11 @@ import socket import sys import time +import threading import traceback +__all__ = ["MPdb", "pdbserver", "target"] + line_prefix = '\n-> ' class MPdb(pydb.Pdb): @@ -47,6 +50,9 @@ self.target = 'local' # local connections by default self.connection = None self.debug_thread = False + self.waiter = threading.Event() + self.tracers = [] + self.threads = [] def _rebind_input(self, new_input): """ This method rebinds the debugger's input to the object specified @@ -117,28 +123,39 @@ if not self.debug_thread: self.msg('Thread debugging is not on.') return - try: - import threading - except ImportError: - self.msg('Thread debugging not available.') - self.msg(threading.enumerate()) + # XXX We need to remove old threads once the script has finished + # and currently, we don't. + self.msg(self.threads) return if arg == 'debug': - try: - import threading - import mthread - except ImportError: - self.msg('Thread debugging not available.') # We do not continue with the main thread for as long as there # is another thread running. (Will change later so you can choose # between threads). - ev = threading.Event() - mthread.set_event(ev) - threading.settrace(mthread.trace_dispatch) + threading.settrace(self.thread_trace_dispatch) self.msg('Thread debugging on.') self.debug_thread = True return + def thread_trace_dispatch(self, frame, event, arg): + """ Create an MTracer object so trace the thread. """ + # This method is called when a thread is being created with the + # threading module. The _MainThread is no longer of primary concern, + # this new thread is. + try: + from mthread import MTracer + except ImportError: + self.msg('Could not import mthread.MTracer') + sys.settrace(None) + return # Thread not being traced + + self.threads.append(threading.currentThread()) + self.msg('New thread: %s' % self.threads[-1].getName()) + m = MTracer(self.stdout) + self.tracers.append(m) + sys.settrace(m.trace_dispatch) + + + # Debugger commands def do_attach(self, addr): """ Attach to a process or file outside of Pdb. @@ -289,16 +306,39 @@ self._rebind_input(self.connection) self._rebind_output(self.connection) -# This is a mess. It's only here so that I can test other features of the -# debugger whilst I'm writing them. It will be removed at some point. -def main(options): - opts = options[0] - args = options[1] - if args: - mainpyfile = args[0] - if not os.path.exists(mainpyfile): - print 'Error:', mainpyfile, 'does not exist' +def pdbserver(protocol, address, filename): + """ This method sets up a pdbserver debugger that allows debuggers + to connect to 'address' using 'protocol'. The argument 'filename' + is the name of the file that is being debugged. + """ + pass + + +def target(protocol, address): + """ Connect to a pdbserver at 'address' using 'protocol'. """ + pass + + +def main(): + """ Main entry point to this module. """ + opts, args = parse_opts() + if not opts.scriptname: + if not args[0]: + print 'Error: mpdb.py must be called with a script name!' + sys.exit(1) + else: + mainpyfile = args[0] + if not os.path.exists(mainpyfile): + print 'Error:', mainpyfile, 'does not exist' + sys.exit(1) + if opts.remote: + if not opts.protocol: + print 'Protocol must be specified for remote debugging' sys.exit(1) + if not opts.debugger: + pdbserver(opts.protocol, opts.address, mainpyfile) + else: + target(opts.protocol, opts.address) mpdb = MPdb() while 1: try: @@ -328,14 +368,18 @@ parser = OptionParser() parser.add_option("-s", "--script", dest="scriptname", help="The script to debug") - parser.add_option("-l", "--local-debugee", dest="local_debugee", + parser.add_option("-l", "--local-debugee", dest="local", action="store_true", help="This script is to be debugged locally, from " + \ "another process") - parser.add_option("-r", "--remote-debugee", dest="remote_debugee", + parser.add_option("-p", "--protocol", dest="protocol", + help="The protocol to use for remote communication") + parser.add_option("-r", "--remote-debugee", dest="remote", action="store_true", help="This script is to be debugged by a remote " + \ "debugger") + parser.add_option("-a", "--address", dest="address", + help="The protocol-specific address of this debugger") parser.add_option("-d", "--debugger", dest="debugger", action="store_true", help="Invoke the debugger.") @@ -344,6 +388,6 @@ return (options,args) if __name__ == '__main__': - main(parse_opts()) + main() Modified: sandbox/trunk/pdb/mthread.py ============================================================================== --- sandbox/trunk/pdb/mthread.py (original) +++ sandbox/trunk/pdb/mthread.py Fri Jun 23 21:06:28 2006 @@ -2,30 +2,29 @@ import sys import threading -import time - -# Globals that are private to this file -_threads = [] -_stop = False -g_event = None class MTracer(object): """ A class to trace a thread. """ - def __init__(self, event): + def __init__(self, stdout=None): self.thread = threading.currentThread() - + # No other thread can be debugged whilst this is set + # (including the MainThread) + if stdout is None: + stdout = sys.stdout + self.out = stdout + def trace_dispatch(self, frame, event, arg): if event == 'line': - print '*** line' + print >> self.out, self.thread.getName(),'*** line' return self.trace_dispatch if event == 'call': - print '*** call' + print >> self.out, self.thread.getName(), '*** call' return self.trace_dispatch if event == 'return': - print '*** return' + print >> self.out, self.thread.getName(), '*** return' return self.trace_dispatch if event == 'exception': - print '*** exception' + print >> self.out, '*** exception' return self.trace_dispatch if event == 'c_call': print '*** c_call' @@ -39,20 +38,8 @@ print 'bdb.Bdb.dispatch: unknown debugging event:', repr(event) return self.trace_dispatch -def trace_dispatch(frame, event, arg): - """ Create a new MTracer object for a thread and set that thread's - tracing function to the MTracer objects trace method. - """ - m = MTracer(g_event) - global _threads - _threads.append(threading.currentThread()) - - sys.settrace(m.trace_dispatch) -def set_event(e): - global g_event - g_event = e - + Modified: sandbox/trunk/pdb/test/test_mconnection.py ============================================================================== --- sandbox/trunk/pdb/test/test_mconnection.py (original) +++ sandbox/trunk/pdb/test/test_mconnection.py Fri Jun 23 21:06:28 2006 @@ -13,7 +13,7 @@ from socket import gaierror # Global vars -__addr__ = 'localhost:8000' +__addr__ = 'localhost:8002' MAXTRIES = 100 TESTFN = 'device' @@ -78,6 +78,15 @@ s = MConnectionServerTCP() self.assertRaises(ConnectionFailed, s.connect, __addr__) + def testInvalidServerAddress(self): + """(tcp) Connect to an invalid hostname. """ + addr = 'fff.209320909xcmnm2iu3-=0-0-z.,x.,091209:2990' + self.assertRaises(ConnectionFailed, self.server.connect, addr) + + def testConnectionRefused(self): + """(tcp) Test connection refused error. """ + self.assertRaises(ConnectionFailed, self.client.connect, __addr__) + def tearDown(self): self.server.disconnect() self.client.disconnect() Modified: sandbox/trunk/pdb/test/test_mpdb.py ============================================================================== --- sandbox/trunk/pdb/test/test_mpdb.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Fri Jun 23 21:06:28 2006 @@ -8,7 +8,7 @@ from test import test_support # Global vars -__addr__ = 'localhost:8000' +__addr__ = 'localhost:8002' script = "" g_server = None g_client = None From python-checkins at python.org Fri Jun 23 21:23:41 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 23 Jun 2006 21:23:41 +0200 (CEST) Subject: [Python-checkins] r47085 - python/trunk/Doc/howto/Makefile Message-ID: <20060623192341.604291E4009@bag.python.org> Author: andrew.kuchling Date: Fri Jun 23 21:23:40 2006 New Revision: 47085 Modified: python/trunk/Doc/howto/Makefile Log: Fit Makefile for the Python doc environment better; this is a step toward including the howtos in the build process. * Put LaTeX output in ../paper-/. * Put HTML output in ../html/ * Explain some of the Makefile variables * Remove some cruft dating to my environment (e.g. the 'web' target) This makefile isn't currently invoked by the documentation build process, so these changes won't destabilize anything. Modified: python/trunk/Doc/howto/Makefile ============================================================================== --- python/trunk/Doc/howto/Makefile (original) +++ python/trunk/Doc/howto/Makefile Fri Jun 23 21:23:40 2006 @@ -1,88 +1,84 @@ +# Makefile for the HOWTO directory +# LaTeX HOWTOs can be turned into HTML, PDF, PS, DVI or plain text output. +# reST HOWTOs can only be turned into HTML. -MKHOWTO=../tools/mkhowto -WEBDIR=. +# Variables to change + +# Paper size for non-HTML formats (letter or a4) +PAPER=letter + +# Arguments to rst2html.py, and location of the script RSTARGS = --input-encoding=utf-8 -VPATH=.:dvi:pdf:ps:txt +RST2HTML = rst2html.py -# List of HOWTOs that aren't to be processed +# List of HOWTOs that aren't to be processed. This should contain the +# base name of the HOWTO without any extension (e.g. 'advocacy', +# 'unicode'). +REMOVE_HOWTOS = -REMOVE_HOWTO = +MKHOWTO=../tools/mkhowto +WEBDIR=. +PAPERDIR=../paper-$(PAPER) +HTMLDIR=../html # Determine list of files to be built - -HOWTO=$(filter-out $(REMOVE_HOWTO),$(wildcard *.tex)) -RST_SOURCES = $(shell echo *.rst) -DVI =$(patsubst %.tex,%.dvi,$(HOWTO)) -PDF =$(patsubst %.tex,%.pdf,$(HOWTO)) -PS =$(patsubst %.tex,%.ps,$(HOWTO)) -TXT =$(patsubst %.tex,%.txt,$(HOWTO)) -HTML =$(patsubst %.tex,%,$(HOWTO)) +TEX_SOURCES = $(wildcard *.tex) +RST_SOURCES = $(wildcard *.rst) +TEX_NAMES = $(filter-out $(REMOVE_HOWTOS),$(patsubst %.tex,%,$(TEX_SOURCES))) + +PAPER_PATHS=$(addprefix $(PAPERDIR)/,$(TEX_NAMES)) +DVI =$(addsuffix .dvi,$(PAPER_PATHS)) +PDF =$(addsuffix .pdf,$(PAPER_PATHS)) +PS =$(addsuffix .ps,$(PAPER_PATHS)) + +ALL_HOWTO_NAMES = $(TEX_NAMES) $(patsubst %.rst,%,$(RST_SOURCES)) +HOWTO_NAMES = $(filter-out $(REMOVE_HOWTOS),$(ALL_HOWTO_NAMES)) +HTML = $(addprefix $(HTMLDIR)/,$(HOWTO_NAMES)) # Rules for building various formats -%.dvi : %.tex + +# reST to HTML +$(HTMLDIR)/%: %.rst + if [ ! -d $@ ] ; then mkdir $@ ; fi + $(RST2HTML) $(RSTARGS) $< >$@/index.html + +# LaTeX to various output formats +$(PAPERDIR)/%.dvi : %.tex $(MKHOWTO) --dvi $< - mv $@ dvi + mv $*.dvi $@ -%.pdf : %.tex +$(PAPERDIR)/%.pdf : %.tex $(MKHOWTO) --pdf $< - mv $@ pdf + mv $*.pdf $@ -%.ps : %.tex +$(PAPERDIR)/%.ps : %.tex $(MKHOWTO) --ps $< - mv $@ ps + mv $*.ps $@ -%.txt : %.tex +$(HTMLDIR)/% : %.tex + $(MKHOWTO) --html --iconserver="." --dir $@ $< + +# Rule that isn't actually used -- we no longer support the 'txt' target. +$(PAPERDIR)/%.txt : %.tex $(MKHOWTO) --text $< mv $@ txt -% : %.tex - $(MKHOWTO) --html --iconserver="." $< - tar -zcvf html/$*.tgz $* - #zip -r html/$*.zip $* - default: @echo "'all' -- build all files" - @echo "'dvi', 'pdf', 'ps', 'txt', 'html' -- build one format" - -all: $(HTML) - -.PHONY : dvi pdf ps txt html rst -dvi: $(DVI) + @echo "'dvi', 'pdf', 'ps', 'html' -- build one format" -pdf: $(PDF) -ps: $(PS) -txt: $(TXT) -html:$(HTML) - -# Rule to build collected tar files -dist: #all - for i in dvi pdf ps txt ; do \ - cd $$i ; \ - tar -zcf All.tgz *.$$i ;\ - cd .. ;\ - done - -# Rule to copy files to the Web tree on AMK's machine -web: dist - cp dvi/* $(WEBDIR)/dvi - cp ps/* $(WEBDIR)/ps - cp pdf/* $(WEBDIR)/pdf - cp txt/* $(WEBDIR)/txt - for dir in $(HTML) ; do cp -rp $$dir $(WEBDIR) ; done - for ltx in $(HOWTO) ; do cp -p $$ltx $(WEBDIR)/latex ; done +all: dvi pdf ps html -rst: unicode.html - -%.html: %.rst - rst2html $(RSTARGS) $< >$@ +.PHONY : dvi pdf ps html +dvi: $(DVI) +pdf: $(PDF) +ps: $(PS) +html: $(HTML) clean: - rm -f *~ *.log *.ind *.l2h *.aux *.toc *.how - rm -f *.dvi *.ps *.pdf *.bkm - rm -f unicode.html + rm -f *~ *.log *.ind *.l2h *.aux *.toc *.how *.bkm + rm -f *.dvi *.pdf *.ps clobber: - rm dvi/* ps/* pdf/* txt/* html/* - - - + rm -rf $(HTML) + rm -rf $(DVI) $(PDF) $(PS) From python-checkins at python.org Fri Jun 23 23:16:19 2006 From: python-checkins at python.org (hyeshik.chang) Date: Fri, 23 Jun 2006 23:16:19 +0200 (CEST) Subject: [Python-checkins] r47086 - in python/trunk: Misc/NEWS Python/codecs.c Message-ID: <20060623211619.1DA121E400D@bag.python.org> Author: hyeshik.chang Date: Fri Jun 23 23:16:18 2006 New Revision: 47086 Modified: python/trunk/Misc/NEWS python/trunk/Python/codecs.c Log: Bug #1511381: codec_getstreamcodec() in codec.c is corrected to omit a default "error" argument for NULL pointer. This allows the parser to take a codec from cjkcodecs again. (Reported by Taewook Kang and reviewed by Walter Doerwald) Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Fri Jun 23 23:16:18 2006 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Bug #1511381: codec_getstreamcodec() in codec.c is corrected to + omit a default "error" argument for NULL pointer. This allows + the parser to take a codec from cjkcodecs again. Library ------- Modified: python/trunk/Python/codecs.c ============================================================================== --- python/trunk/Python/codecs.c (original) +++ python/trunk/Python/codecs.c Fri Jun 23 23:16:18 2006 @@ -249,14 +249,17 @@ const char *errors, const int index) { - PyObject *codecs, *streamcodec; + PyObject *codecs, *streamcodec, *codeccls; codecs = _PyCodec_Lookup(encoding); if (codecs == NULL) return NULL; - streamcodec = PyEval_CallFunction( - PyTuple_GET_ITEM(codecs, index), "Os", stream, errors); + codeccls = PyTuple_GET_ITEM(codecs, index); + if (errors != NULL) + streamcodec = PyObject_CallFunction(codeccls, "Os", stream, errors); + else + streamcodec = PyObject_CallFunction(codeccls, "O", stream); Py_DECREF(codecs); return streamcodec; } From python-checkins at python.org Sat Jun 24 00:50:55 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Sat, 24 Jun 2006 00:50:55 +0200 (CEST) Subject: [Python-checkins] r47087 - sandbox/trunk/decimal-c/Makefile sandbox/trunk/decimal-c/README.txt sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060623225055.C902F1E400A@bag.python.org> Author: mateusz.rukowicz Date: Sat Jun 24 00:50:54 2006 New Revision: 47087 Modified: sandbox/trunk/decimal-c/Makefile sandbox/trunk/decimal-c/README.txt sandbox/trunk/decimal-c/_decimal.c Log: Powering works, there is only sqrt and remainderNear left unimplemented. Some update to README. Modified: sandbox/trunk/decimal-c/Makefile ============================================================================== --- sandbox/trunk/decimal-c/Makefile (original) +++ sandbox/trunk/decimal-c/Makefile Sat Jun 24 00:50:54 2006 @@ -1,7 +1,7 @@ # change PYTHON_25 to point to a 2.5 HEAD build # (a fairly recent HEAD is necessary) -PYTHON_25=../../python/python -PYTH_DIR=../../python +PYTHON_25=../../pyth/python/python +PYTH_DIR=../../pyth/python/ all: _decimal.so run module: @@ -18,10 +18,10 @@ gdb $(PYTHON_25) #_decimal.o: _decimal.c decimal.h -# gcc -pthread -fno-strict-aliasing -g -O1 -DNEDEBUG -Wall -Wstrict-prototypes -fPIC -I${PYTH_DIR}/Include -I${PYTH_DIR} -c _decimal.c -o _decimal.o +# gcc -pthread -fno-strict-aliasing -g -std=c89 -DNEDEBUG -Wall -Wstrict-prototypes -fPIC -I${PYTH_DIR}/Include -I${PYTH_DIR} -c _decimal.c -o _decimal.o #_decimal.so: _decimal.o # gcc -pthread -shared _decimal.o -g -o _decimal.so -_decimal.so: +_decimal.so: _decimal.c $(PYTHON_25) setup.py build cp build/lib*/_decimal.so . Modified: sandbox/trunk/decimal-c/README.txt ============================================================================== --- sandbox/trunk/decimal-c/README.txt (original) +++ sandbox/trunk/decimal-c/README.txt Sat Jun 24 00:50:54 2006 @@ -4,10 +4,9 @@ _decimal builds as a regular stand-alone python extension module decimal.py and test_decimal.py are copies from the 2.5 trunk -The plan is to change decimal.Decimal to inherit from _decimal.Decimal -and replace the pure-python methods one-by-one while keeping all the -tests passing. This should allow us to tackle the job in small pieces -and allow many people to tackle it at once. +At the beginning decimal was meant to inherit from _decimal, and wrap +not implemented functions. Now, when almost all functions work properly +it is not necessary. Edit the Makefile to point PYTHON_25 to point to a trunk build of python and then run 'make' or 'make all' to build the extension and run the Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Sat Jun 24 00:50:54 2006 @@ -688,6 +688,7 @@ static PyObject *context_ignore_all_flags(contextobject *); static PyObject *context_regard_flags(contextobject *, PyObject*); static decimalobject *_do_decimal_absolute(decimalobject *, contextobject *, int); +static PyObject *decimal_int(decimalobject*); /* Exception handlers *********************************************************/ @@ -2934,8 +2935,8 @@ /* TODO this *need* clean up */ if (divmod) { /* we need to rescale, to be compatible with python implementation */ - PyObject *flags; - contextobject *ctx2; + PyObject *flags = 0; + contextobject *ctx2 = 0; long remainder_exp = self->exp < other->exp ? self->exp : other->exp; decimalobject *rescaled_rem = 0; decimalobject *rescaled = _decimal_rescale(result, 0, ctx, -1, 0); @@ -2947,12 +2948,16 @@ } ctx2 = context_copy(ctx); if (!ctx2) { + Py_DECREF(rescaled); Py_DECREF(result); Py_DECREF(remainder_ret); Py_DECREF(op1); + return NULL; } flags = context_ignore_all_flags(ctx2); if (!flags) { + Py_DECREF(ctx2); + Py_DECREF(rescaled); Py_DECREF(result); Py_DECREF(remainder_ret); Py_DECREF(op1); @@ -3601,6 +3606,192 @@ _do_decimal_power(decimalobject *self, decimalobject *other, decimalobject *modulo, contextobject *ctx) { + decimalobject *ret=0, *val, *mul; + long n; + int sign; + long spot; + int mod = modulo != Py_None; + long firstprec = ctx->prec; + double a, b; + + if (ISINF(other) || ADJUSTED(other) > 8) { + return handle_InvalidOperation(self->ob_type, ctx, "x ** INF", NULL); + } + + if (ISSPECIAL(self) || ISSPECIAL(other)) { + decimalobject *nan; + if (_check_nans(self, other, ctx, &nan)) + return nan; + } + + if (!_decimal_isint(other)) { + return handle_InvalidOperation(self->ob_type, ctx, + "x ** (non-integer)", NULL); + } + + if (!decimal_nonzero(self) && !decimal_nonzero(other)) { + return handle_InvalidOperation(self->ob_type, ctx, "0 ** 0", NULL); + } + + if (!decimal_nonzero(other)) { + ret = _NEW_decimalobj(1, 0, 0); + ret->limbs[0] = 1; + return ret; + } + + { + PyObject *tmp = decimal_int(other); + if (!tmp) + return NULL; + + n = PyInt_AsLong(tmp); + Py_DECREF(tmp); + + if (PyErr_Occurred()) { + return NULL; + } + + } + sign = (self->sign&1) && n&1; + + + if (ISINF(self)) { + if (mod) { + return handle_InvalidOperation(self->ob_type, ctx, + "INF % x", NULL); + } + + ret = _NEW_decimalobj(1, sign, 0); + ret->limbs[0] = 0; + + if (n > 0) + ret->sign = sign ? SIGN_NEGINF : SIGN_POSINF; + + return ret; + } + + /* XXX temporary solution */ + a = ((double) self->exp + (double) self->ob_size - (double) 1) * (double) n; + b = ctx->Emax; + + if (!mod && n > 0 && /* (self->exp + self->ob_size -1) * n > ctx->Emax && */ + a > b && + decimal_nonzero(self)) { + + if (handle_Rounded(ctx, NULL)) { + return NULL; + } + + if (handle_Inexact(ctx, NULL)) { + return NULL; + } + + return handle_Overflow(self->ob_type, ctx, "Big power", sign); + } + + firstprec = ctx->prec; + /* XXX I am not really sure is this ok, in python implementation it is + * prec + 1 + len(str(n))*/ + ctx->prec += 1; + { + long t = n; + if(!t) + ctx->prec += 1; + else + while(t) + { + ctx->prec += 1; + t/=10; + } + } + + if (!mod && ctx->prec > PyDecimal_DefaultContext->Emax) { + ctx->prec = firstprec; + return handle_Overflow(self->ob_type, ctx, "Too much precision", sign); + } + mul = _decimal_get_copy(self); + val = _NEW_decimalobj(1,0,0); + val->limbs[0] = 1; + + if(!val || !mul) { + Py_XDECREF(mul); + Py_XDECREF(val); + return NULL; + } + + if (n < 0) { + decimalobject *tmp; + n *= -1; + tmp = _do_decimal__divide(val, mul, 0, ctx); + if (!tmp) { + Py_DECREF(mul); + Py_DECREF(val); + return NULL; + } + + Py_DECREF(mul); + mul = tmp; + } + + spot = 1; + while (spot <= n) { + spot <<= 1; + } + + spot >>= 1; + + while (spot) { + decimalobject *tmp; + + tmp = _do_decimal_multiply(val, val, ctx); + + if (!tmp) + goto err; + + Py_DECREF(val); + val = tmp; + + if (spot & n) { + tmp = _do_decimal_multiply(val, mul, ctx); + + if (!tmp) + goto err; + Py_DECREF(val); + val = tmp; + } + + if (mod) { + tmp = _do_decimal_remainder(val, modulo, ctx); + if (!tmp) + goto err; + Py_DECREF(val); + val = tmp; + } + spot >>= 1; + } + + ctx->prec = firstprec; + + if (ctx->rounding_dec == ALWAYS_ROUND) { + ret = _decimal_fix(val, ctx); + if (!ret) + goto err; + } + else { + ret = val; + Py_INCREF(val); + } + + Py_DECREF(val); + Py_DECREF(mul); + return ret; + +err: + ctx->prec = firstprec; + Py_DECREF(val); + Py_DECREF(mul); + Py_XDECREF(ret); + return NULL; /* XXX */ Py_RETURN_NONE; } @@ -3608,13 +3799,30 @@ static PyObject * decimal_power(PyObject *self, PyObject *other, PyObject *modulo) { -#if 0 decimalobject *res; + decimalobject *dec; contextobject *ctx = getcontext(); if (!ctx) return NULL; - other = _convert_to_decimal(self->ob_type, other, ctx, 0); - if (other == NULL || other == Py_NotImplemented) return other; +/* other = _convert_to_decimal(self->ob_type, other, ctx, 0); + if (other == NULL || other == Py_NotImplemented) return other; */ + + if (PyDecimal_Check(self)) { + other = _convert_to_decimal(self->ob_type, other, ctx, 0); + if (!other || other == Py_NotImplemented) + return other; + dec = other; + + } + + else { + assert(PyDecimal_Check(other)); + self = _convert_to_decimal(other->ob_type, self, ctx, 0); + if (!self || self == Py_NotImplemented) + return self; + dec = self; + } + if (modulo == Py_None) { Py_INCREF(modulo); @@ -3629,12 +3837,9 @@ (decimalobject *)other, (decimalobject *)modulo, ctx); - Py_DECREF(other); + Py_DECREF(dec); Py_DECREF(modulo); return (PyObject *)res; -#else - Py_RETURN_NONE; -#endif } @@ -3891,8 +4096,9 @@ return 1; /* Here comes the tricky part. We have to find out * whether the fractional part consists only of zeroes. */ - for (i = d->ob_size-1; i > d->ob_size + d->exp; --i) { - if (d->digits[i] > 0) + for (i = d->ob_size-1; i >= d->ob_size + d->exp; --i) { +/* if (d->digits[i] > 0)*/ + if(_limb_get_digit(d->limbs, d->ob_size, i) > 0) return 0; } return 1; @@ -5018,11 +5224,11 @@ static PyObject * context_power(contextobject *self, PyObject *args) { -#if 0 - PyObject *a, *b, *c; + PyObject *a, *b, *c = NULL; decimalobject *dec_a = NULL, *dec_b = NULL, *dec_c = NULL, *res; if (!PyArg_ParseTuple(args, "OO|O:power", &a, &b, &c)) return NULL; + dec_a = (decimalobject *)_convert_to_decimal( &PyDecimal_DecimalType, a, self, 1); if (dec_a == NULL) @@ -5035,21 +5241,25 @@ return NULL; } - dec_c = (decimalobject *)_convert_to_decimal( - &PyDecimal_DecimalType, c, self, 1); - if (dec_c == NULL) { - Py_DECREF(dec_a); - Py_DECREF(dec_b); - return NULL; + if (c) { + dec_c = (decimalobject *)_convert_to_decimal( + &PyDecimal_DecimalType, c, self, 1); + if (dec_c == NULL) { + dec_c = Py_None; + Py_INCREF(Py_None); + /* XXX is it ok ? */ + PyErr_Clear(); + } + } + else { + dec_c = Py_None; + Py_INCREF(Py_None); } res = _do_decimal_power(dec_a, dec_b, dec_c, self); Py_DECREF(dec_a); Py_DECREF(dec_b); Py_DECREF(dec_c); return (PyObject *)res; -#else - Py_RETURN_NONE; -#endif } From python-checkins at python.org Sat Jun 24 02:27:49 2006 From: python-checkins at python.org (sean.reifschneider) Date: Sat, 24 Jun 2006 02:27:49 +0200 (CEST) Subject: [Python-checkins] r47088 - sandbox/trunk/pybch/pybch.py Message-ID: <20060624002749.C78291E50CA@bag.python.org> Author: sean.reifschneider Date: Sat Jun 24 02:27:48 2006 New Revision: 47088 Modified: sandbox/trunk/pybch/pybch.py Log: Finally worked out the math, I think it's functional now. Need to run more tests and tests of different versions. Modified: sandbox/trunk/pybch/pybch.py ============================================================================== --- sandbox/trunk/pybch/pybch.py (original) +++ sandbox/trunk/pybch/pybch.py Sat Jun 24 02:27:48 2006 @@ -42,7 +42,7 @@ ######################## def shortifyTestName(s): #if s[:6] == 'Tests.': s = s[6:] - s = s.split('.')[-1] + s = '.'.join(s.split('.')[-2:]) return(s) @@ -75,33 +75,39 @@ / len(testSource[2])) compareAverage = (reduce(lambda x,y: x+y, testCompare[2][1], 0) / len(testCompare[2])) + sourceBest = min(testSource[2][1]) + compareBest = min(testCompare[2][1]) # calculate normalization normalizationFactor = float(testCompare[2][0]) / float(testSource[2][0]) - print normalizationFactor #@@@ - # calculate averages - sourceAverages = [] - for n in testSource[2][1]: - sourceAverages.append(n / sourceAverage * 100.0) - compareAverages = [] - for n in testCompare[2][1]: - compareAverages.append(n / compareAverage * 100.0) - - sourceAveragesStr = ' '.join(map(lambda x: '%5.1f%%' - % x, sourceAverages)) - compareAveragesStr = ' '.join(map(lambda x: '%5.1f%%' - % x, compareAverages)) - - difference = min(testCompare[2][1]) - min(testSource[2][1]) - overallDiff = overallDiff + difference - if difference > 0: - overallSlowdowns = overallSlowdowns + difference - if difference < 0: - overallSpeedups = overallSpeedups + difference + # compare + difference = (compareBest - + (normalizationFactor * sourceBest)) / compareBest + differenceStr = '%7.1f' % difference + if differenceStr.strip() == '-0.0': differenceStr = ' 0.0' + differenceShorter = float(differenceStr.strip()) + + # debugging + if verbose > 2: + print 'Source Laps: ', testSource[2][0] + print 'Compare Laps: ', testCompare[2][0] + print 'Source Best:', sourceBest + print 'Compare Best:', compareBest + print 'Source Average:', sourceAverage + print 'Compare Average:', compareAverage + print 'Normalization Factor:', normalizationFactor + print 'Difference:', difference + print 'Difference Str:', differenceStr + print 'Difference Shorter:', differenceShorter + + overallDiff = overallDiff + differenceShorter + if differenceShorter > 0: + overallSlowdowns = overallSlowdowns + differenceShorter + if differenceShorter < 0: + overallSpeedups = overallSpeedups + differenceShorter - print '%-30s %s' % ( testSource[0], sourceAveragesStr ) - print '%-30s %s -> %5.1f%%' % ( '', compareAveragesStr, difference ) + print '%-50s -> %s%%' % ( testSource[0], differenceStr ) print '=' * 78 print '%68s %5.1f%%' % ( 'Overall difference:', overallDiff ) @@ -131,11 +137,26 @@ metavar = 'TEST_REGEX') parser.add_option('-v', '--verbose', dest = 'verbose', action = 'count', default = 0, help = 'Increase verbosity level.') +parser.add_option('-l', '--list-tests', dest = 'listTests', action = 'count', + default = 0, help = 'List available tests.') options, args = parser.parse_args() +# list tests +if options.listTests: + import Tests + for moduleName in Tests.testModules: + exec('from Tests import %s' % moduleName) + module = eval(moduleName) + for testClass in module.__dict__.values(): + if (not hasattr(testClass, 'is_a_test') + or 'TestHelpers' in str(testClass)): + continue + print shortifyTestName(str(testClass)) + sys.exit(0) + if options.compareDestFileLoad: # load results from a file - testResults = pickle.load(open(options.compareDestFileLoad, 'r')) + testData = pickle.load(open(options.compareDestFileLoad, 'r')) else: # run tests locally import Tests From python-checkins at python.org Sat Jun 24 12:19:51 2006 From: python-checkins at python.org (sean.reifschneider) Date: Sat, 24 Jun 2006 12:19:51 +0200 (CEST) Subject: [Python-checkins] r47089 - sandbox/trunk/pybch/pybch.py Message-ID: <20060624101951.B8C161E400A@bag.python.org> Author: sean.reifschneider Date: Sat Jun 24 12:19:49 2006 New Revision: 47089 Modified: sandbox/trunk/pybch/pybch.py Log: More math changes. Now I'm seeing all kinds of skew in the results. Ugh. Modified: sandbox/trunk/pybch/pybch.py ============================================================================== --- sandbox/trunk/pybch/pybch.py (original) +++ sandbox/trunk/pybch/pybch.py Sat Jun 24 12:19:49 2006 @@ -46,15 +46,15 @@ return(s) -######################################################### -def compareResults(testResults, verbose, compareResults): +########################################################### +def compareResults(testResults, verbose, compareAgainst): # display comparison results print ('Comparing %(version)s (%(build)s)' - % compareResults['environment']) + % compareAgainst['environment']) print (' to %(version)s (%(build)s)' % testResults['environment']) print ('Comparing [%(environment)s] on %(host)s' - % compareResults['environment']) + % compareAgainst['environment']) print (' to [%(environment)s] on %(host)s' % testResults['environment']) print @@ -69,21 +69,18 @@ overallSpeedups = 0.0 overallSlowdowns = 0.0 for testSource in testList: - compareResults = compareResults['results'][testSource[1]] - testCompare = [ None, None, [compareResults[0], compareResults[1]] ] - sourceAverage = (reduce(lambda x,y: x+y, testSource[2][1], 0) - / len(testSource[2])) - compareAverage = (reduce(lambda x,y: x+y, testCompare[2][1], 0) - / len(testCompare[2])) + compareData = compareAgainst['results'][testSource[1]] + testCompare = [ None, None, [compareData[0], compareData[1]] ] sourceBest = min(testSource[2][1]) compareBest = min(testCompare[2][1]) # calculate normalization - normalizationFactor = float(testCompare[2][0]) / float(testSource[2][0]) + normalizationFactor = compareBest / sourceBest + sourceLapsNormalized = testSource[2][0] * normalizationFactor # compare - difference = (compareBest - - (normalizationFactor * sourceBest)) / compareBest + difference = 100.0 - ((sourceLapsNormalized / testCompare[2][0]) + * 100.0) differenceStr = '%7.1f' % difference if differenceStr.strip() == '-0.0': differenceStr = ' 0.0' differenceShorter = float(differenceStr.strip()) @@ -91,11 +88,10 @@ # debugging if verbose > 2: print 'Source Laps: ', testSource[2][0] + print 'Source Laps Normalized: ', sourceLapsNormalized print 'Compare Laps: ', testCompare[2][0] print 'Source Best:', sourceBest print 'Compare Best:', compareBest - print 'Source Average:', sourceAverage - print 'Compare Average:', compareAverage print 'Normalization Factor:', normalizationFactor print 'Difference:', difference print 'Difference Str:', differenceStr @@ -107,12 +103,12 @@ if differenceShorter < 0: overallSpeedups = overallSpeedups + differenceShorter - print '%-50s -> %s%%' % ( testSource[0], differenceStr ) + print '%-63s -> %s%%' % ( testSource[0], differenceStr ) print '=' * 78 print '%68s %5.1f%%' % ( 'Overall difference:', overallDiff ) - print '%68s %5.1f%%' % ( 'Overall speedups:', overallSpeedups ) - print '%68s %5.1f%%' % ( 'Overall slowdowns:', overallSlowdowns ) + print '%68s %5.1f%%' % ( 'Total speedups:', overallSpeedups ) + print '%68s %5.1f%%' % ( 'Total slowdowns:', overallSlowdowns ) ################################## From python-checkins at python.org Sat Jun 24 17:52:00 2006 From: python-checkins at python.org (matt.fleming) Date: Sat, 24 Jun 2006 17:52:00 +0200 (CEST) Subject: [Python-checkins] r47090 - in sandbox/trunk/pdb: README.txt mconnection.py mpdb.py test/test_mconnection.py test/test_mpdb.py Message-ID: <20060624155200.5B8491E400B@bag.python.org> Author: matt.fleming Date: Sat Jun 24 17:51:59 2006 New Revision: 47090 Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/test/test_mconnection.py sandbox/trunk/pdb/test/test_mpdb.py Log: Started to fix unit tests and introduce some more to test the top-level routines. Fixed and unclear statement in README.txt Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Sat Jun 24 17:51:59 2006 @@ -25,10 +25,9 @@ * Lightweight and heavyweight mechanism for setting up threads - The reason we want a lightweight mechanism is so that we can place mpdb.set_trace() inside a script so that we can debug - any threads created with the threading module. It has to be - lighweight because the programmer might not want all the features - of an MPdb instance, if for example they only care about debugging - this one thread instance. + the script. It has to be lighweight because the programmer + might not want all the features of an MPdb instance, if for example + they only care about debugging this one thread instance. - We need a heavyweight mechanism to allow a programmer to inspect and control all threads. * Provide a proper top-level methods including, set_trace(), post_mortem(), Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Sat Jun 24 17:51:59 2006 @@ -155,7 +155,6 @@ """Connect to the server. 'input' reads data from the server. 'output' writes data to the server. Specify the address of the server (e.g. host:2020). """ - h, p = addr.split(':') self.host = h self.port = int(p) Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Sat Jun 24 17:51:59 2006 @@ -53,11 +53,13 @@ self.waiter = threading.Event() self.tracers = [] self.threads = [] + self._info_cmds.append('target') def _rebind_input(self, new_input): """ This method rebinds the debugger's input to the object specified by 'new_input'. """ + self.stdin.flush() self.use_rawinput = False self.stdin = new_input @@ -84,7 +86,7 @@ # The output from the command that we've just sent to the server # is returned along with the prompt of that server. So we keep reading # until we find our prompt. - while self.prompt not in ret: + while self.local_prompt not in ret: ret += self.connection.readline() self.msg_nocr(ret) return @@ -110,31 +112,32 @@ return args = arg.split() - if 'target'.startswith(args[0]): + if 'target'.startswith(args[0]) and len(args[0]) > 2: self.msg("target is %s" % self.target) - else: - pydb.Pdb.do_info(self, arg) - - def do_thread(self, arg): - """ Enable thread debugging. """ - # XXX Rocky, how are we subclassing info/set commands? This will do - # for now. - if arg == 'info': + elif 'thread'.startswith(args[0]) and len(args[0])> 2: if not self.debug_thread: self.msg('Thread debugging is not on.') return - # XXX We need to remove old threads once the script has finished - # and currently, we don't. + # We need some way to remove old thread instances self.msg(self.threads) return - if arg == 'debug': - # We do not continue with the main thread for as long as there - # is another thread running. (Will change later so you can choose - # between threads). + else: + pydb.Pdb.do_info(self, arg) + + + def do_set(self, arg): + """ Extends pydb do_set() to allow setting thread debugging. """ + if not arg: + pydb.Pdb.do_set(self, arg) + return + + args = arg.split() + if 'thread'.startswith(args[0]): threading.settrace(self.thread_trace_dispatch) - self.msg('Thread debugging on.') + self.msg('Thread debugging on') self.debug_thread = True return + def thread_trace_dispatch(self, frame, event, arg): """ Create an MTracer object so trace the thread. """ @@ -197,14 +200,12 @@ try: from mconnection import (MConnectionClientTCP, ConnectionFailed) - # Matt - Where are the connection parameters? self.connection = MConnectionClientTCP() except ImportError: self.msg('Could not import MConnectionClientTCP') return elif target == 'serial': - # Matt - Where are the connection parameters? if self.connection: self.connection.disconnect() try: from mconnection import (MConnectionSerial, @@ -234,10 +235,14 @@ return # This interpreter no longer interprets commands but sends # them straight across this object's connection to a server. - self.prompt = "" # Get our prompt from the server now + # XXX: In the remote_onecmd method we use the local_prompt string + # to find where the end of the message from the server is. We + # really need a way to get the prompt from the server for checking + # in remote_onecmd, because it may be different to this client's. + self.local_prompt = self.prompt + self.prompt = "" line = self.connection.readline() self.msg_nocr(line) - self._rebind_output(self.connection) self.onecmd = self.remote_onecmd self.target = 'remote' @@ -306,39 +311,52 @@ self._rebind_input(self.connection) self._rebind_output(self.connection) -def pdbserver(protocol, address, filename): +def pdbserver(addr): """ This method sets up a pdbserver debugger that allows debuggers to connect to 'address' using 'protocol'. The argument 'filename' is the name of the file that is being debugged. """ - pass - - -def target(protocol, address): - """ Connect to a pdbserver at 'address' using 'protocol'. """ - pass - + m = MPdb() + position = addr.rfind(' ') + mainpyfile = addr[position+1:] + m.mainpyfile = mainpyfile + m.do_pdbserver(addr) + m._runscript(mainpyfile) + sys.exit() + +def target(addr): + """ Connect this debugger to a pdbserver at 'addr'. 'addr' is + a protocol-specific address. i.e. + tcp = 'tcp mydomainname.com:9876' + serial = '/dev/ttyC0' + """ + m = MPdb() + # Look Ma, no script! + m.do_target(addr) + while True: + try: + m.cmdloop() + except: + sys.exit() def main(): """ Main entry point to this module. """ opts, args = parse_opts() - if not opts.scriptname: - if not args[0]: - print 'Error: mpdb.py must be called with a script name!' - sys.exit(1) - else: - mainpyfile = args[0] - if not os.path.exists(mainpyfile): - print 'Error:', mainpyfile, 'does not exist' - sys.exit(1) - if opts.remote: - if not opts.protocol: - print 'Protocol must be specified for remote debugging' + if opts.target: + target(opts.target) + elif opts.pdbserver: + pdbserver(opts.pdbserver) + else: + if not opts.scriptname: + if not args: + print 'Error: mpdb.py must be called with a script name if ' \ + + '-p or -t switches are not specified.' + sys.exit(1) + else: + mainpyfile = args[0] + if not os.path.exists(mainpyfile): + print 'Error:', mainpyfile, 'does not exist' sys.exit(1) - if not opts.debugger: - pdbserver(opts.protocol, opts.address, mainpyfile) - else: - target(opts.protocol, opts.address) mpdb = MPdb() while 1: try: @@ -368,23 +386,14 @@ parser = OptionParser() parser.add_option("-s", "--script", dest="scriptname", help="The script to debug") - parser.add_option("-l", "--local-debugee", dest="local", - action="store_true", - help="This script is to be debugged locally, from " + \ - "another process") - parser.add_option("-p", "--protocol", dest="protocol", - help="The protocol to use for remote communication") - parser.add_option("-r", "--remote-debugee", dest="remote", - action="store_true", - help="This script is to be debugged by a remote " + \ - "debugger") - parser.add_option("-a", "--address", dest="address", - help="The protocol-specific address of this debugger") - parser.add_option("-d", "--debugger", dest="debugger", - action="store_true", - help="Invoke the debugger.") + parser.add_option("-t", "--target", dest="target", + help="Specify a target to connect to. The arguments" \ + + " should be of form, 'protocol address'.") + parser.add_option("-p", "--pdbserver", dest="pdbserver", + help="Start the debugger and execute the pdbserver " \ + + "command. The arguments should be of the form," \ + + " 'protocol address scriptname'.") (options, args) = parser.parse_args() - # We don't currently support any arguments return (options,args) if __name__ == '__main__': Modified: sandbox/trunk/pdb/test/test_mconnection.py ============================================================================== --- sandbox/trunk/pdb/test/test_mconnection.py (original) +++ sandbox/trunk/pdb/test/test_mconnection.py Sat Jun 24 17:51:59 2006 @@ -87,6 +87,12 @@ """(tcp) Test connection refused error. """ self.assertRaises(ConnectionFailed, self.client.connect, __addr__) + def testInvalidAddressPortPair(self): + """(tcp) Test invald hostname, port pair. """ + addr = 'localhost 8000' + # Rocky: Should this be a ValueError or some other sort of exception? + self.assertRaises(ValueError, self.server.connect, addr) + def tearDown(self): self.server.disconnect() self.client.disconnect() Modified: sandbox/trunk/pdb/test/test_mpdb.py ============================================================================== --- sandbox/trunk/pdb/test/test_mpdb.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Sat Jun 24 17:51:59 2006 @@ -10,28 +10,22 @@ # Global vars __addr__ = 'localhost:8002' script = "" -g_server = None -g_client = None -CONNECTED = False -# Commands to execute on the server -cmds = ['info target', 'help', 'quit'] +MAXTRIES = 100 sys.path.append("..") -from mpdb import MPdb +from mpdb import MPdb, pdbserver, target + +TESTFN = 'tester' + +def connect_to_target(client, address=None): + if address is None: + address = __addr__ + client.do_target('tcp '+address) + + while 'Failed' in client.lines[0]: + client.lines = [] + client.do_target('tcp '+address) -def doTargetConnect(cmds=None): - global g_client - while True: - try: - g_client.do_target('tcp '+__addr__) - if CONNECTED: - break - except socket.error: - pass - if cmds: - for c in cmds: - g_client.onecmd(c) - class MPdbTest(MPdb): def __init__(self): MPdb.__init__(self) @@ -44,36 +38,68 @@ """ Test Case to make sure debugging remotely works properly. """ def setUp(self): self.server = MPdb() - global g_client - g_client = MPdbTest() + self.client = MPdbTest() def tearDown(self): - global CONNECTED - self.server.connection.disconnect() - CONNETED = False - - def testPdbserver(self): - """ Test the pdbserver command. """ - global CONNECTED + if self.server.connection: + self.server.connection.disconnect() + import os + if TESTFN in os.listdir('.'): + os.unlink(TESTFN) + +# Whilst a lot of the tests below seem to duplicate tests from +# test_mconnection, we need to make sure that the methods that mpdb provides +# are not having side effects, and causing tests that should pass to fail +# and vice versa. - self.server_tid = thread.start_new_thread(doTargetConnect, ()) + def testPdbserver(self): + """ Test the pdbserver. """ + thread.start_new_thread(connect_to_target, (self.client,)) self.server.do_pdbserver('tcp '+__addr__+' '+script) - CONNECTED = True + self.assertEquals('remote', self.server.target) - def testCommandsOnServer(self): - """ Test all supported commands on the pdbserver. - """ - global CONNECTED, g_client - - self.server_tid = thread.start_new_thread(doTargetConnect, (cmds,)) - self.server.do_pdbserver('tcp '+__addr__+' '+script) - CONNECTED = True - - # XXX mpdb needs a bottom frame before it exits - self.server.botframe = None - self.server.cmdloop() + def testTarget(self): + """ Test the target command. """ + addr = 'tcp '+__addr__+' '+script + thread.start_new_thread(self.client.do_pdbserver, (addr,)) + + # There was a problem with earlier unit tests that they were + # working "by coincidence". This ensures that we don't "assume" + # the connection is ready until the target has changed from "local" + # to "remote". + connect_to_target(self.client, __addr__) + self.assertEquals('remote', self.client.target, 'Target is wrong.') + + + def testRebindOutput(self): + """ Test rebinding output. """ + f = open(TESTFN, 'w+') + self.server._rebind_output(f) + self.server.msg('some text') + f.close() + + f = open(TESTFN, 'r') + line = f.readline() + f.close() + self.assertEquals('some text\n', line, 'Could not rebind output') + + def testRebindInput(self): + """ Test rebinding input. """ + f = open(TESTFN, 'w+') + f.write('help') + f.close() + + f = open(TESTFN, 'r') + self.server._rebind_input(f) + line = self.server.stdin.readline() + + self.assertEquals(line, 'help', 'Could not rebind input.') + + def testTargetRoutine(self): + """ Test that the top-level target routine works properly. """ + invalid_address = 'tcp ::::::' + self.assertRaises(ValueError, target, invalid_address) - self.assertEquals('target is remote\n', g_client.lines[1]) def test_main(): test_support.run_unittest(TestRemoteDebugging) From martin at v.loewis.de Sun Jun 25 19:14:25 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 25 Jun 2006 19:14:25 +0200 Subject: [Python-checkins] Things to remember when adding *packages* to stdlib In-Reply-To: References: Message-ID: <449EC471.2000809@v.loewis.de> Neal Norwitz wrote: > I believe this change is all that's necessary on the Unix side to > install wsgiref. Can someone please update the Windows build files to > ensure wsgiref is installed in b2? Don't forget to update the NEWS > entry too. It's installed in b1 already. The msi generator picks up all .py files in Lib automatically, except for those that have been explicitly excluded (the plat-* ones). > Maybe someone could come up with a heuristic to add to Misc/build.sh > which we could test in there. I think "make install INSTALL=true|grep true" should print the names of all .py files in Lib, except for the ones in plat-*. Regards, Martin From python-checkins at python.org Sun Jun 25 22:44:17 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 25 Jun 2006 22:44:17 +0200 (CEST) Subject: [Python-checkins] r47091 - python/trunk/Lib/idlelib/Debugger.py Message-ID: <20060625204417.26EB71E400B@bag.python.org> Author: ronald.oussoren Date: Sun Jun 25 22:44:16 2006 New Revision: 47091 Modified: python/trunk/Lib/idlelib/Debugger.py Log: Workaround for bug #1512124 Without this patch IDLE will get unresponsive when you open the debugger window on OSX. This is both using the system Tcl/Tk on Tiger as the latest universal download from tk-components.sf.net. Modified: python/trunk/Lib/idlelib/Debugger.py ============================================================================== --- python/trunk/Lib/idlelib/Debugger.py (original) +++ python/trunk/Lib/idlelib/Debugger.py Sun Jun 25 22:44:16 2006 @@ -4,6 +4,7 @@ from Tkinter import * from WindowList import ListedToplevel from ScrolledList import ScrolledList +import macosxSupport class Idb(bdb.Bdb): @@ -322,7 +323,13 @@ class StackViewer(ScrolledList): def __init__(self, master, flist, gui): - ScrolledList.__init__(self, master, width=80) + if macosxSupport.runningAsOSXApp(): + # At least on with the stock AquaTk version on OSX 10.4 you'll + # get an shaking GUI that eventually kills IDLE if the width + # argument is specified. + ScrolledList.__init__(self, master) + else: + ScrolledList.__init__(self, master, width=80) self.flist = flist self.gui = gui self.stack = [] From buildbot at python.org Sun Jun 25 22:53:50 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 25 Jun 2006 20:53:50 +0000 Subject: [Python-checkins] buildbot failure in amd64 gentoo trunk Message-ID: <20060625205350.22DE01E400B@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1139 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Sun Jun 25 23:14:20 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 25 Jun 2006 23:14:20 +0200 (CEST) Subject: [Python-checkins] r47092 - in python/trunk/Mac/Demo: applescript.html calldll index.html Message-ID: <20060625211420.669B61E400B@bag.python.org> Author: ronald.oussoren Date: Sun Jun 25 23:14:19 2006 New Revision: 47092 Removed: python/trunk/Mac/Demo/calldll/ Modified: python/trunk/Mac/Demo/applescript.html python/trunk/Mac/Demo/index.html Log: Drop the calldll demo's for macos, calldll isn't present anymore, no need to keep the demo's around. Modified: python/trunk/Mac/Demo/applescript.html ============================================================================== --- python/trunk/Mac/Demo/applescript.html (original) +++ python/trunk/Mac/Demo/applescript.html Sun Jun 25 23:14:19 2006 @@ -4,6 +4,10 @@
Using the Open Scripting Architecture from Python

+
NOTE: this document describes the OSA support that is shipped with +the core python distribution. Most users are better of with the more +userfriendly appscript library. +
OSA support in Python is still not 100% complete, but there is already enough in place to allow you to do some nifty things with other programs from your python program.
@@ -355,4 +359,4 @@ - \ No newline at end of file + Modified: python/trunk/Mac/Demo/index.html ============================================================================== --- python/trunk/Mac/Demo/index.html (original) +++ python/trunk/Mac/Demo/index.html Sun Jun 25 23:14:19 2006 @@ -3,15 +3,16 @@
Macintosh Python crash course

-This set of documents provides an introduction to various aspects of +
This set of documents provides an introduction to various aspects of Python programming on the Mac. It is assumed that the reader is already familiar with Python and, to some extent, with MacOS Toolbox programming. Other readers may find something interesting here too, -your mileage may vary.
+your mileage may vary.
-There is a companion document Using Python on the Mac -which you should read before starting here: it explains the basics of using -python on the Macintosh.
+
As the previous paragraph reveals to the careful observer these examples +are dated, most of them were writting before OSX and haven't been updated +afterwards. They still show how to use the Carbon wrappers but aren't +necessarily the best way to use the Carbon API's in OSX.
Another set of Macintosh-savvy examples, more aimed at beginners, is maintained by Joseph Strout, at Python Tidbits in documentation section on the webserver.
-The W widget set by Just van Rossum, which is used by the Python IDE, does not -have complete documentation as of this writing, but Corran Webster has -documented most of it on his -Python Page.
+
The W widget set by Just van Rossum, does not have complete documentation as +of this writing, but Corran Webster has documented most of it on his +Python Page.
There are also some documentation links, as well as other MacPython-related pages, in the From python-checkins at python.org Sun Jun 25 23:15:58 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 25 Jun 2006 23:15:58 +0200 (CEST) Subject: [Python-checkins] r47093 - python/trunk/Mac/BuildScript/scripts/postflight.framework Message-ID: <20060625211558.0235C1E401D@bag.python.org> Author: ronald.oussoren Date: Sun Jun 25 23:15:58 2006 New Revision: 47093 Modified: python/trunk/Mac/BuildScript/scripts/postflight.framework Log: Use a path without a double slash to compile the .py files after installation (macosx, binary installer). This fixes bug #1508369 for python 2.5. Modified: python/trunk/Mac/BuildScript/scripts/postflight.framework ============================================================================== --- python/trunk/Mac/BuildScript/scripts/postflight.framework (original) +++ python/trunk/Mac/BuildScript/scripts/postflight.framework Sun Jun 25 23:15:58 2006 @@ -4,7 +4,7 @@ # PYVER="@PYVER@" -FWK="/Library/Frameworks/Python.framework/Versions/@PYVER@/" +FWK="/Library/Frameworks/Python.framework/Versions/@PYVER@" "${FWK}/bin/python" -Wi -tt \ "${FWK}/lib/python${PYVER}/compileall.py" \ From python-checkins at python.org Sun Jun 25 23:19:06 2006 From: python-checkins at python.org (ronald.oussoren) Date: Sun, 25 Jun 2006 23:19:06 +0200 (CEST) Subject: [Python-checkins] r47094 - python/trunk/Makefile.pre.in Message-ID: <20060625211906.E92D61E400B@bag.python.org> Author: ronald.oussoren Date: Sun Jun 25 23:19:06 2006 New Revision: 47094 Modified: python/trunk/Makefile.pre.in Log: Also install the .egg-info files in Lib. This will cause wsgiref.egg-info to be installed. Modified: python/trunk/Makefile.pre.in ============================================================================== --- python/trunk/Makefile.pre.in (original) +++ python/trunk/Makefile.pre.in Sun Jun 25 23:19:06 2006 @@ -733,7 +733,7 @@ else true; \ fi; \ done - @for i in $(srcdir)/Lib/*.py $(srcdir)/Lib/*.doc; \ + @for i in $(srcdir)/Lib/*.py $(srcdir)/Lib/*.doc $(srcdir)/Lib/*.egg-info ; \ do \ if test -x $$i; then \ $(INSTALL_SCRIPT) $$i $(DESTDIR)$(LIBDEST); \ From buildbot at python.org Mon Jun 26 00:41:15 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 25 Jun 2006 22:41:15 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060625224115.D5D171E400C@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/411 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 01:57:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Sun, 25 Jun 2006 23:57:30 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060625235730.A4D011E400C@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/236 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 26 04:19:07 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Mon, 26 Jun 2006 04:19:07 +0200 (CEST) Subject: [Python-checkins] r47095 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060626021907.B48001E400C@bag.python.org> Author: mateusz.rukowicz Date: Mon Jun 26 04:19:06 2006 New Revision: 47095 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Square rooting works, some minor bugs in rounding functions fixed, when limb_count != (ob_size + LOG -1)/LOG. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Mon Jun 26 04:19:06 2006 @@ -689,6 +689,11 @@ static PyObject *context_regard_flags(contextobject *, PyObject*); static decimalobject *_do_decimal_absolute(decimalobject *, contextobject *, int); static PyObject *decimal_int(decimalobject*); +static decimalobject *_do_decimal_add(decimalobject *, decimalobject *, contextobject *); +static PyObject *_do_decimal__divide(decimalobject *, decimalobject *, int, contextobject *); +static decimalobject *_decimal_fromliteral(PyTypeObject *, char *str, long, contextobject *); +static decimalobject *_do_decimal_multiply(decimalobject *, decimalobject *, contextobject *); +static decimalobject *_do_decimal_subtract(decimalobject *, decimalobject *, contextobject *); /* Exception handlers *********************************************************/ @@ -1014,6 +1019,7 @@ } if(_limb_get_digit(new->limbs, new->ob_size, 0) == 0) new->ob_size --; + new->limb_count = (new->ob_size + LOG - 1)/ LOG; return new; } @@ -1052,6 +1058,7 @@ if (new2->ob_size > prec) { _limb_cut_one_digit(new2->limbs,new2->ob_size); new2->ob_size--; + new2->limb_count = (new2->ob_size + LOG -1)/LOG; new2->exp++; } return new2; @@ -1077,6 +1084,7 @@ if (new->ob_size > prec) { _limb_cut_one_digit(new->limbs,new->ob_size); new->ob_size--; + new->limb_count = (new->ob_size + LOG - 1)/LOG; new->exp++; } return new; @@ -1277,6 +1285,7 @@ _limb_cut_one_digit(new->limbs, new->ob_size); /* VERY SLOW */ new->ob_size --; } + new->limb_count = (new->ob_size + LOG - 1)/LOG; new->exp += expdiff; if (handle_Rounded(ctx, NULL) != 0) { Py_DECREF(new); @@ -2070,10 +2079,356 @@ } +/* this code is rewritten from python version, this might change + * in future */ + static decimalobject * _do_decimal_sqrt(decimalobject *self, contextobject *ctx) { - Py_RETURN_NONE; + decimalobject *ret = 0; + decimalobject *ans = 0; + decimalobject *tmp = 0, *tmp2 = 0, *tmp3 = 0; + contextobject *ctx2 = 0; + decimalobject *half = 0; + PyObject *flags = 0; + long expadd; + long firstprec; + long i; + long Emax; + long Emin; + long maxp; + long rounding; + long prevexp; + + if (ISSPECIAL(self)) { + decimalobject *nan; + if (_check_nans(self, NULL, ctx, &nan)) + return nan; + + if (ISINF(self) && (self->sign&1) == 0) + return _decimal_get_copy(self); + } + + if (!decimal_nonzero(self)) { + ret = _NEW_decimalobj(1, 0, 0); + if (!ret) + return NULL; + + ret->limbs[0] = 0; + ret->exp = self->exp / 2; + if (self->exp < 0 && self->exp%2) + ret->exp --; + ret->sign = self->sign & 1; + + return ret; + } + + if (self->sign&1) { + return handle_InvalidOperation(self->ob_type, ctx, "sqrt(-x), x > 0", NULL); + } + + tmp = _NEW_decimalobj(self->ob_size + 1, self->sign, self->exp); + + if (!tmp) + return NULL; + + expadd = tmp->exp/2; + if (tmp->exp < 0 && tmp->exp % 2) + expadd --; + + if (tmp->exp &1) { + _limb_first_n_digits(self->limbs, self->ob_size, 0, tmp->limbs, tmp->ob_size); + } + else { + tmp->ob_size --; + tmp->limb_count = self->limb_count; + for(i = 0; i < tmp->limb_count ; i++) + tmp->limbs[i] = self->limbs[i]; + } + + tmp->exp = 0; + + ctx2 = context_copy(ctx); + + if (!ctx2) { + Py_DECREF(tmp); + return NULL; + } + + flags = context_ignore_all_flags(ctx2); + + if (!flags) { + Py_DECREF(tmp); + Py_DECREF(ctx2); + return NULL; + } + + ans = _NEW_decimalobj(3, 0, 0); + + if (!ans) { + Py_DECREF(tmp); + Py_DECREF(ctx2); + Py_DECREF(flags); + } + + tmp2 = _NEW_decimalobj(3, 0, 0); + + if (!tmp2) { + Py_DECREF(tmp); + Py_DECREF(ctx2); + Py_DECREF(flags); + Py_DECREF(ans); + } + + if ((ADJUSTED(tmp) & 1) == 0) { + ans->limbs[0] = 819; + ans->exp = ADJUSTED(tmp) - 2; + tmp2->limbs[0] = 259; + tmp2->exp = -2; + } + else { + ans->limbs[0] = 259; + ans->exp = tmp->exp + tmp->ob_size - 3; + tmp2->limbs[0] = 819; + tmp2->exp = -3; + } + + firstprec = ctx2->prec; + ctx2->prec = 3; + /* ans += tmp * tmp2 */ + + tmp3 = _do_decimal_multiply(tmp, tmp2, ctx2); + + if (!tmp3) + goto err; + + Py_DECREF(tmp2); + tmp2 = _do_decimal_add(ans, tmp3, ctx2); + Py_DECREF(tmp3); + tmp3 = 0; + + if (!tmp2) + goto err; + + Py_DECREF(ans); + ans = tmp2; + tmp2 = 0; + ans->exp -= 1 + ADJUSTED(tmp)/2; + if (1 + ADJUSTED(tmp) < 0 && (1 + ADJUSTED(tmp)) % 2) + ans->exp --; + + Emax = ctx2->Emax; + Emin = ctx2->Emin; + + ctx2->Emax = PyDecimal_DefaultContext->Emax; + ctx2->Emin = PyDecimal_DefaultContext->Emin; + + half = _decimal_fromliteral(self->ob_type, "0.5", 3, ctx2); + + if (!half) + goto err; + + maxp = firstprec + 2; + rounding = ctx2->rounding; + ctx2->rounding = ROUND_HALF_EVEN; + + while (1) { + ctx2->prec = 2 * ctx2->prec - 2 < maxp ? 2 * ctx2->prec - 2: maxp; + /* ans = half * (ans + tmp/ans) */ + tmp2 = _do_decimal__divide(tmp, ans, 0, ctx2); + if (!tmp2) + goto err; + /* ans = half * (ans + tmp2) */ + tmp3 = _do_decimal_add(ans, tmp2, ctx2); + if (!tmp3) + goto err; + + /* ans = half * tmp3 */ + Py_DECREF(ans); + ans = _do_decimal_multiply(half, tmp3, ctx2); + + if (!ans) + goto err; + + Py_DECREF(tmp2); + Py_DECREF(tmp3); + tmp2 = 0; + tmp3 = 0; + + if (ctx2->prec == maxp) + break; + } + + ctx2->prec = firstprec; + prevexp = ADJUSTED(ans); + tmp2 = _decimal_round(ans, -1, ctx2, -1); + if (!tmp2) + goto err; + Py_DECREF(ans); + ans = _NEW_decimalobj(tmp2->ob_size + 1, tmp2->sign, tmp2->exp); + if (!ans) + goto err; + + ctx2->prec = firstprec + 1; + if (prevexp != ADJUSTED(tmp2)) { + _limb_first_n_digits(tmp2->limbs, tmp2->ob_size, 0, ans->limbs, ans->ob_size); + ans->exp --; + } + else { + ans->limb_count = tmp2->limb_count; + ans->ob_size = tmp2->ob_size; + for (i = 0; i < ans->limb_count; i++) + ans->limbs[i] = tmp2->limbs[i]; + } + + Py_DECREF(tmp2); + tmp2 = 0; + + { + int cmp; + decimalobject *lower; + half->exp = ans->exp - 1; + half->limbs[0] = 5; + lower = _do_decimal_subtract(ans, half, ctx2); + if (!lower) + goto err; + + ctx2->rounding = ROUND_UP; + tmp2 = _do_decimal_multiply(lower, lower, ctx2); + Py_DECREF(lower); + lower = 0; + if (!tmp2) { + goto err; + } + + cmp = _do_real_decimal_compare(tmp2, tmp, ctx2); + if (PyErr_Occurred()) { + goto err; + } + + Py_DECREF(tmp2); + tmp2 = 0; + + if (cmp == 1) { + half->exp = ans->exp; + half->limbs[0] = 1; + tmp2 = _do_decimal_subtract(ans, half, ctx2); + if (!tmp2) + goto err; + Py_DECREF(ans); + ans = tmp2; + tmp2 = 0; + } + else { + decimalobject *upper; + half->exp = ans->exp-1; + half->limbs[0] = 5; + upper = _do_decimal_add(ans, half, ctx2); + if (!upper) + goto err; + ctx2->rounding = ROUND_DOWN; + + tmp2 = _do_decimal_multiply(upper, upper, ctx2); + + Py_DECREF(upper); + upper = 0; + + cmp = _do_real_decimal_compare(tmp2, tmp, ctx2); + if (PyErr_Occurred()) + goto err; + + Py_DECREF(tmp2); + tmp2 = 0; + + if (cmp == -1) { + half->exp = ans->exp; + half->limbs[0] = 1; + tmp2 = _do_decimal_add(ans, half, ctx2); + + if (!tmp2) + goto err; + Py_DECREF(ans); + ans = tmp2; + tmp2 = 0; + } + } + } + + ans->exp += expadd; + ctx2->rounding = rounding; + + tmp2 = _decimal_fix(ans, ctx2); + if (!tmp2) + goto err; + Py_DECREF(ans); + ans = tmp2; + tmp2 = 0; + + rounding = ctx2->rounding_dec; + ctx2->rounding_dec = NEVER_ROUND; + + { + int cmp; + tmp2 = _do_decimal_multiply(ans, ans, ctx2); + if (!tmp2) + goto err; + + cmp = _do_real_decimal_compare(tmp2, self, ctx2); + + if (PyErr_Occurred()) + goto err; + + Py_DECREF(tmp2); + tmp2 = 0; + + if (cmp != 0) { + if (handle_Rounded(ctx, NULL)) + goto err; + + if (handle_Inexact(ctx, NULL)) + goto err; + } + + else { + long exp = self->exp / 2; + if (self->exp < 0 && self->exp % 2) + exp --; + ctx2->prec += ans->exp - exp; + tmp2 = _decimal_rescale(ans, exp, ctx2, -1, 1); + + if (!tmp2) + goto err; + Py_DECREF(ans); + ans = tmp2; + tmp2 = 0; + ctx2->prec = firstprec; + } + } + + tmp2 = _decimal_fix(ans, ctx); + if (!tmp2) + goto err; + Py_DECREF(ans); + ans = tmp2; + tmp2 = 0; + + Py_DECREF(flags); + Py_DECREF(ctx2); + Py_DECREF(half); + Py_DECREF(tmp); + + return ans; + +err: + Py_XDECREF(tmp); + Py_XDECREF(tmp2); + Py_XDECREF(tmp3); + Py_XDECREF(ans); + Py_XDECREF(flags); + Py_XDECREF(ctx2); + Py_XDECREF(ret); + Py_XDECREF(half); + return NULL; } DECIMAL_UNARY_FUNC(sqrt) @@ -3510,6 +3865,7 @@ size = _limb_multiply(self->limbs, self->ob_size, other->limbs, other->ob_size, ans->limbs); ans->ob_size = size; + ans->limb_count = (size + LOG -1 )/LOG; done: From python-checkins at python.org Mon Jun 26 04:52:11 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Mon, 26 Jun 2006 04:52:11 +0200 (CEST) Subject: [Python-checkins] r47096 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060626025211.ABF081E400C@bag.python.org> Author: mateusz.rukowicz Date: Mon Jun 26 04:52:11 2006 New Revision: 47096 Modified: sandbox/trunk/decimal-c/_decimal.c Log: Added __cmp__, one refleak fixed. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Mon Jun 26 04:52:11 2006 @@ -1768,6 +1768,28 @@ } DECIMAL_BINARY_FUNC(compare) +static int +decimal_cmp(decimalobject *self, decimalobject *other) +{ + contextobject *ctx; + int res; + ctx = getcontext(); + if (!ctx) return NULL; + + other = (decimalobject *)_convert_to_decimal(self->ob_type, + other, ctx, 0); + + if (!other) return NULL; + if (other == Py_NotImplemented) return other; + + res = _do_real_decimal_compare(self, other, ctx); + Py_DECREF(other); + if (PyErr_Occurred()) + return NULL; + + return res; + +} static PyObject * decimal_richcompare(decimalobject *self, decimalobject *other, @@ -1781,6 +1803,9 @@ other = (decimalobject *)_convert_to_decimal(self->ob_type, (PyObject *)other, ctx, 0); + + if (!other) + return NULL; if ((PyObject*)other == Py_NotImplemented) return (PyObject*)other; res = _do_real_decimal_compare(self, other, ctx); @@ -2551,7 +2576,7 @@ static PyObject * decimal_as_tuple(decimalobject *self) { - PyObject *digits, *res, *d; + PyObject *digits, *res, *d, *inf; long i; digits = PyTuple_New(self->ob_size); @@ -2566,8 +2591,15 @@ if(!ISINF(self)) res = Py_BuildValue("(bOn)", self->sign % 2, digits, self->exp); - else - res = Py_BuildValue("(bOO)", self->sign % 2, digits, PyString_FromString("F")); + else { + inf = PyString_FromString("F"); + if (!inf) { + Py_DECREF(digits); + return NULL; + } + res = Py_BuildValue("(bOO)", self->sign % 2, digits, inf); + Py_DECREF(inf); + } Py_DECREF(digits); return res; } @@ -5204,7 +5236,7 @@ 0, /* tp_print */ 0, /* tp_getattr */ 0, /* tp_setattr */ - 0, /* tp_compare */ + decimal_cmp, /* tp_compare */ (reprfunc)decimal_repr, /* tp_repr */ &decimal_as_number, /* tp_as_number */ 0, /* tp_as_sequence */ From nnorwitz at gmail.com Mon Jun 26 05:54:29 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 25 Jun 2006 20:54:29 -0700 Subject: [Python-checkins] r47096 - sandbox/trunk/decimal-c/_decimal.c In-Reply-To: <20060626025211.ABF081E400C@bag.python.org> References: <20060626025211.ABF081E400C@bag.python.org> Message-ID: > Modified: sandbox/trunk/decimal-c/_decimal.c > ============================================================================== > --- sandbox/trunk/decimal-c/_decimal.c (original) > +++ sandbox/trunk/decimal-c/_decimal.c Mon Jun 26 04:52:11 2006 > @@ -1768,6 +1768,28 @@ > } > DECIMAL_BINARY_FUNC(compare) > > +static int > +decimal_cmp(decimalobject *self, decimalobject *other) > +{ ... > + res = _do_real_decimal_compare(self, other, ctx); > + Py_DECREF(other); > + if (PyErr_Occurred()) > + return NULL; > + > + return res; Won't res be NULL if an error occurred? If so, there's no reason for the if. n From neal at metaslash.com Mon Jun 26 11:06:52 2006 From: neal at metaslash.com (Neal Norwitz) Date: Mon, 26 Jun 2006 05:06:52 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060626090652.GA16565@python.psfb.org> test_socket leaked [0, 205, -205] references From python-checkins at python.org Mon Jun 26 14:40:02 2006 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 26 Jun 2006 14:40:02 +0200 (CEST) Subject: [Python-checkins] r47097 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060626124002.E933D1E4004@bag.python.org> Author: andrew.kuchling Date: Mon Jun 26 14:40:02 2006 New Revision: 47097 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: [Bug #1511998] Various comments from Nick Coghlan; thanks! Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Mon Jun 26 14:40:02 2006 @@ -817,7 +817,7 @@ decorator that are useful for writing objects for use with the '\keyword{with}' statement. -The decorator is called \function{contextfactory}, and lets you write +The decorator is called \function{contextmanager}, and lets you write a single generator function instead of defining a new class. The generator should yield exactly one value. The code up to the \keyword{yield} will be executed as the \method{__enter__()} method, and the value @@ -831,9 +831,9 @@ using this decorator as: \begin{verbatim} -from contextlib import contextfactory +from contextlib import contextmanager - at contextfactory + at contextmanager def db_transaction (connection): cursor = connection.cursor() try: @@ -1748,11 +1748,12 @@ Brandl.) % Patch #754022 -\item Python's standard library no longer includes -a package named \module{xml}; the library's XML-related package -has been renamed to \module{xmlcore}. This means -it will always be possible to import the standard library's -XML support whether or not the PyXML package is installed. +\item The standard library's XML-related package +has been renamed to \module{xmlcore}. The \module{xml} module will +now import either the \module{xmlcore} or PyXML version of subpackages +such as \module{xml.dom}. The renaming means it will always be +possible to import the standard library's XML support whether or not +the PyXML package is installed. \item The \module{xmlrpclib} module now supports returning \class{datetime} objects for the XML-RPC date type. Supply @@ -2168,7 +2169,7 @@ host = '' port = 8000 -httpd = make_server(host, port, wsgi_app) +httpd = simple_server.make_server(host, port, wsgi_app) httpd.serve_forever() \end{verbatim} @@ -2218,12 +2219,13 @@ for_loop = ast.body[1] \end{verbatim} -No documentation has been written for the AST code yet. To start -learning about it, read the definition of the various AST nodes in -\file{Parser/Python.asdl}. A Python script reads this file and -generates a set of C structure definitions in -\file{Include/Python-ast.h}. The \cfunction{PyParser_ASTFromString()} -and \cfunction{PyParser_ASTFromFile()}, defined in +No official documentation has been written for the AST code yet, but +\pep{339} discusses the design. To start learning about the code, read the +definition of the various AST nodes in \file{Parser/Python.asdl}. A +Python script reads this file and generates a set of C structure +definitions in \file{Include/Python-ast.h}. The +\cfunction{PyParser_ASTFromString()} and +\cfunction{PyParser_ASTFromFile()}, defined in \file{Include/pythonrun.h}, take Python source as input and return the root of an AST representing the contents. This AST can then be turned into a code object by \cfunction{PyAST_Compile()}. For more @@ -2406,8 +2408,8 @@ The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this -article: Phillip J. Eby, "Ralf W. Grosse-Kunstleve, Kent Johnson, -Martin von~L\"owis, Fredrik Lundh, Gustavo Niemeyer, James Pryor, Mike -Rovner, Scott Weikart, Barry Warsaw, Thomas Wouters. +article: Nick Coghlan, Phillip J. Eby, "Ralf W. Grosse-Kunstleve, Kent +Johnson, Martin von~L\"owis, Fredrik Lundh, Gustavo Niemeyer, James +Pryor, Mike Rovner, Scott Weikart, Barry Warsaw, Thomas Wouters. \end{document} From python-checkins at python.org Mon Jun 26 14:43:43 2006 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 26 Jun 2006 14:43:43 +0200 (CEST) Subject: [Python-checkins] r47098 - python/trunk/Doc/whatsnew/whatsnew25.tex Message-ID: <20060626124343.858971E4004@bag.python.org> Author: andrew.kuchling Date: Mon Jun 26 14:43:43 2006 New Revision: 47098 Modified: python/trunk/Doc/whatsnew/whatsnew25.tex Log: Describe workaround for PyRange_New()'s removal Modified: python/trunk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/trunk/Doc/whatsnew/whatsnew25.tex (original) +++ python/trunk/Doc/whatsnew/whatsnew25.tex Mon Jun 26 14:43:43 2006 @@ -2276,7 +2276,12 @@ \item The \cfunction{PyRange_New()} function was removed. It was never documented, never used in the core code, and had dangerously lax -error checking. +error checking. In the unlikely case that your extensions were using +it, you can replace it by something like the following: +\begin{verbatim} +range = PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", + start, stop, step); +\end{verbatim} \end{itemize} @@ -2408,7 +2413,7 @@ The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this -article: Nick Coghlan, Phillip J. Eby, "Ralf W. Grosse-Kunstleve, Kent +article: Nick Coghlan, Phillip J. Eby, Ralf W. Grosse-Kunstleve, Kent Johnson, Martin von~L\"owis, Fredrik Lundh, Gustavo Niemeyer, James Pryor, Mike Rovner, Scott Weikart, Barry Warsaw, Thomas Wouters. From python-checkins at python.org Mon Jun 26 15:08:24 2006 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 26 Jun 2006 15:08:24 +0200 (CEST) Subject: [Python-checkins] r47099 - python/trunk/Lib/mailbox.py Message-ID: <20060626130824.C10231E4004@bag.python.org> Author: andrew.kuchling Date: Mon Jun 26 15:08:24 2006 New Revision: 47099 Modified: python/trunk/Lib/mailbox.py Log: [Bug #1512163] Fix typo. This change will probably break tests on FreeBSD buildbots, but I'll check in a fix for that next. Modified: python/trunk/Lib/mailbox.py ============================================================================== --- python/trunk/Lib/mailbox.py (original) +++ python/trunk/Lib/mailbox.py Mon Jun 26 15:08:24 2006 @@ -15,7 +15,7 @@ import rfc822 import StringIO try: - import fnctl + import fcntl except ImportError: fcntl = None From buildbot at python.org Mon Jun 26 15:08:34 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 13:08:34 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20060626130834.BCFE01E4004@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1237 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 15:11:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 13:11:02 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD trunk Message-ID: <20060626131102.92B1B1E400C@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%2520trunk/builds/931 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 26 15:12:16 2006 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 26 Jun 2006 15:12:16 +0200 (CEST) Subject: [Python-checkins] r47100 - python/trunk/Lib/mailbox.py Message-ID: <20060626131216.AE8DE1E4004@bag.python.org> Author: andrew.kuchling Date: Mon Jun 26 15:12:16 2006 New Revision: 47100 Modified: python/trunk/Lib/mailbox.py Log: [Bug #1512163] Use one set of locking methods, lockf(); remove the flock() calls. On FreeBSD, the two methods lockf() and flock() end up using the same mechanism and the second one fails. A Linux man page claims that the two methods are orthogonal (so locks acquired one way don't interact with locks acquired the other way) but that clearly must be false. Modified: python/trunk/Lib/mailbox.py ============================================================================== --- python/trunk/Lib/mailbox.py (original) +++ python/trunk/Lib/mailbox.py Mon Jun 26 15:12:16 2006 @@ -1798,7 +1798,7 @@ def _lock_file(f, dotlock=True): - """Lock file f using lockf, flock, and dot locking.""" + """Lock file f using lockf and dot locking.""" dotlock_done = False try: if fcntl: @@ -1810,14 +1810,6 @@ f.name) else: raise - try: - fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB) - except IOError, e: - if e.errno == errno.EWOULDBLOCK: - raise ExternalClashError('flock: lock unavailable: %s' % - f.name) - else: - raise if dotlock: try: pre_lock = _create_temporary(f.name + '.lock') @@ -1845,16 +1837,14 @@ except: if fcntl: fcntl.lockf(f, fcntl.LOCK_UN) - fcntl.flock(f, fcntl.LOCK_UN) if dotlock_done: os.remove(f.name + '.lock') raise def _unlock_file(f): - """Unlock file f using lockf, flock, and dot locking.""" + """Unlock file f using lockf and dot locking.""" if fcntl: fcntl.lockf(f, fcntl.LOCK_UN) - fcntl.flock(f, fcntl.LOCK_UN) if os.path.exists(f.name + '.lock'): os.remove(f.name + '.lock') From python-checkins at python.org Mon Jun 26 15:23:11 2006 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 26 Jun 2006 15:23:11 +0200 (CEST) Subject: [Python-checkins] r47101 - python/trunk/Lib/test/test_mailbox.py Message-ID: <20060626132311.1D18C1E4004@bag.python.org> Author: andrew.kuchling Date: Mon Jun 26 15:23:10 2006 New Revision: 47101 Modified: python/trunk/Lib/test/test_mailbox.py Log: Add a test for a conflicting lock. On slow machines, maybe the time intervals (2 sec, 0.5 sec) will be too tight. I'll see how the buildbots like it. Modified: python/trunk/Lib/test/test_mailbox.py ============================================================================== --- python/trunk/Lib/test/test_mailbox.py (original) +++ python/trunk/Lib/test/test_mailbox.py Mon Jun 26 15:23:10 2006 @@ -720,6 +720,28 @@ self.assert_(contents == open(self._path, 'rb').read()) self._box = self._factory(self._path) + def test_lock_conflict(self): + # Fork off a subprocess that will lock the file for 2 seconds, + # unlock it, and then exit. + pid = os.fork() + if pid == 0: + # In the child, lock the mailbox. + self._box.lock() + time.sleep(2) + self._box.unlock() + os._exit(0) + + # In the parent, sleep a bit to give the child time to acquire + # the lock. + time.sleep(0.5) + self.assertRaises(mailbox.ExternalClashError, + self._box.lock) + + # Wait for child to exit. Locking should now succeed. + pid, status = os.wait() + self._box.lock() + self._box.unlock() + class TestMbox(_TestMboxMMDF): From python-checkins at python.org Mon Jun 26 15:29:42 2006 From: python-checkins at python.org (matt.fleming) Date: Mon, 26 Jun 2006 15:29:42 +0200 (CEST) Subject: [Python-checkins] r47102 - in sandbox/trunk/pdb: mconnection.py mpdb.py test/test_mconnection.py test/test_mpdb.py Message-ID: <20060626132942.1C3811E4004@bag.python.org> Author: matt.fleming Date: Mon Jun 26 15:29:41 2006 New Revision: 47102 Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/test/test_mconnection.py sandbox/trunk/pdb/test/test_mpdb.py Log: Raise ConnectionFailed for invalid address to target() and pdbserver(). Fix unit tests. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Mon Jun 26 15:29:41 2006 @@ -104,7 +104,10 @@ """Set to allow a connection from a client. 'addr' specifies the hostname and port combination of the server. """ - h,p = addr.split(':') + try: + h,p = addr.split(':') + except ValueError: + raise ConnectionFailed, 'Invalid address' self.host = h self.port = int(p) if not self.listening: @@ -155,7 +158,10 @@ """Connect to the server. 'input' reads data from the server. 'output' writes data to the server. Specify the address of the server (e.g. host:2020). """ - h, p = addr.split(':') + try: + h, p = addr.split(':') + except ValueError: + raise ConnectionFailed, 'Invalid address' self.host = h self.port = int(p) self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Mon Jun 26 15:29:41 2006 @@ -59,7 +59,6 @@ """ This method rebinds the debugger's input to the object specified by 'new_input'. """ - self.stdin.flush() self.use_rawinput = False self.stdin = new_input @@ -116,7 +115,7 @@ self.msg("target is %s" % self.target) elif 'thread'.startswith(args[0]) and len(args[0])> 2: if not self.debug_thread: - self.msg('Thread debugging is not on.') + self.errmsg('Thread debugging is not on.') return # We need some way to remove old thread instances self.msg(self.threads) @@ -147,7 +146,7 @@ try: from mthread import MTracer except ImportError: - self.msg('Could not import mthread.MTracer') + self.errmsg('Could not import mthread.MTracer') sys.settrace(None) return # Thread not being traced @@ -189,11 +188,11 @@ try: target, addr = args.split(' ') except ValueError: - self.msg('Invalid arguments') + self.errmsg('Invalid arguments') return if self.target == 'remote': - self.msg('Already connected to a remote machine.') - return + self.errmsg('Already connected to a remote machine.') + return if target == 'tcp': # TODO: need to save state of current debug session if self.connection: self.connection.disconnect() @@ -203,7 +202,7 @@ self.connection = MConnectionClientTCP() except ImportError: - self.msg('Could not import MConnectionClientTCP') + self.errmsg('Could not import MConnectionClientTCP') return elif target == 'serial': if self.connection: self.connection.disconnect() @@ -212,7 +211,7 @@ ConnectionFailed) self.connection = MConnectionSerial() except ImportError: - self.msg('Could not import MConnectionSerial') + self.errmsg('Could not import MConnectionSerial') return else: if '.' in target: @@ -220,18 +219,23 @@ # We dynamically load the class for the connection base = target[:target.rfind('.')] cls = target[target.rfind('.')+1:] - exec 'from ' + base + ' import (' + cls + ', ConnectionFailed)' + try: + exec 'from ' + base + ' import (' + cls + ', \ + ConnectionFailed)' + except ImportError: + self.errmsg('Unknown target type') + return else: try: __import__(target) except ImportError: - self.msg('Unknown target type') - return + self.errmsg('Unknown target type') + return self.connection = eval(target+'()') try: self.connection.connect(addr) except ConnectionFailed, err: - self.msg("Failed to connect to %s: (%s)" % (addr, err)) + self.errmsg("Failed to connect to %s: (%s)" % (addr, err)) return # This interpreter no longer interprets commands but sends # them straight across this object's connection to a server. @@ -245,6 +249,7 @@ self.msg_nocr(line) self.onecmd = self.remote_onecmd self.target = 'remote' + return def do_detach(self, args): """ Detach a process or file previously attached. @@ -270,12 +275,12 @@ """ try: - target, comm, scriptfile_and_args = args.split(' ') + target, comm = args.split(' ') except ValueError: - self.msg('Invalid arguments') + self.errmsg('Invalid arguments') return if self.target == 'remote': - self.msg('Already connected remotely') + self.errmsg('Already connected remotely') return if target == 'tcp': try: @@ -283,7 +288,7 @@ ConnectionFailed) self.connection = MConnectionServerTCP() except ImportError: - self.msg('Could not load MConnectionServerTCP class') + self.errmsg('Could not load MConnectionServerTCP class') return elif target == 'serial': try: @@ -291,21 +296,29 @@ ConnectionFailed) self.connection = MConnectionSerial() except ImportError: - self.msg('Could not load MConnectionSerial class') + self.errmsg('Could not load MConnectionSerial class') return else: if '.' in target: base = target[:target.rfind('.')] target = target[target.rfind('.')+1:] - exec 'from ' + base + ' import (' + target + \ + try: + exec 'from ' + base + ' import (' + target + \ ', ConnectionFailed)' + except ImportError: + self.errmsg('Unknown protocol') + return else: - __import__(target) + try: + __import__(target) + except ImportError: + self.errmsg('Unknown protocol') + return self.connection = eval(target+'()') try: self.connection.connect(comm) except ConnectionFailed, err: - self.msg("Failed to connect to %s: (%s)" % (comm, err)) + self.errmsg("Failed to connect to %s: (%s)" % (comm, err)) return self.target = 'remote' self._rebind_input(self.connection) Modified: sandbox/trunk/pdb/test/test_mconnection.py ============================================================================== --- sandbox/trunk/pdb/test/test_mconnection.py (original) +++ sandbox/trunk/pdb/test/test_mconnection.py Mon Jun 26 15:29:41 2006 @@ -29,7 +29,7 @@ client.connect(addr) # The _sock variable appears when there's a connection if client._sock: break - except socket.error: + except ConnectionFailed: pass class TestTCPConnections(unittest.TestCase): @@ -90,9 +90,8 @@ def testInvalidAddressPortPair(self): """(tcp) Test invald hostname, port pair. """ addr = 'localhost 8000' - # Rocky: Should this be a ValueError or some other sort of exception? - self.assertRaises(ValueError, self.server.connect, addr) - + self.assertRaises(ConnectionFailed, self.server.connect, addr) + def tearDown(self): self.server.disconnect() self.client.disconnect() Modified: sandbox/trunk/pdb/test/test_mpdb.py ============================================================================== --- sandbox/trunk/pdb/test/test_mpdb.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Mon Jun 26 15:29:41 2006 @@ -1,5 +1,6 @@ #!/usr/bin/env python +import os import sys import socket import thread @@ -9,23 +10,32 @@ # Global vars __addr__ = 'localhost:8002' -script = "" +script = 'thread_script.py' MAXTRIES = 100 sys.path.append("..") from mpdb import MPdb, pdbserver, target +from mconnection import (MConnectionClientTCP, MConnectionServerTCP, + ConnectionFailed) TESTFN = 'tester' +# This provides us with a fine-grain way of connecting to a server def connect_to_target(client, address=None): if address is None: address = __addr__ - client.do_target('tcp '+address) - - while 'Failed' in client.lines[0]: - client.lines = [] - client.do_target('tcp '+address) + client.connection = MConnectionClientTCP() + while True: + try: + client.connection.connect(address) + except ConnectionFailed, e: + # This is the error message we expect i.e. when the server thread + # hasn't started yet. Otherwise it's probably a _real_ error + if 'Connection refused' in e: pass + else: raise ConnectionFailed, e + break + class MPdbTest(MPdb): def __init__(self): MPdb.__init__(self) @@ -36,14 +46,7 @@ class TestRemoteDebugging(unittest.TestCase): """ Test Case to make sure debugging remotely works properly. """ - def setUp(self): - self.server = MPdb() - self.client = MPdbTest() - def tearDown(self): - if self.server.connection: - self.server.connection.disconnect() - import os if TESTFN in os.listdir('.'): os.unlink(TESTFN) @@ -54,25 +57,42 @@ def testPdbserver(self): """ Test the pdbserver. """ - thread.start_new_thread(connect_to_target, (self.client,)) - self.server.do_pdbserver('tcp '+__addr__+' '+script) - self.assertEquals('remote', self.server.target) + client = MPdbTest() + thread.start_new_thread(connect_to_target, (client,)) + + self.server1 = MPdb() + self.server1.do_pdbserver('tcp '+__addr__) + self.server1.connection.disconnect() + + self.server2 = MPdbTest() + self.server2.do_pdbserver('unknown_protocol unknownhost') + line = self.server2.lines[0] + self.assertEquals('*** Unknown protocol\n', line) + + def testTarget(self): """ Test the target command. """ - addr = 'tcp '+__addr__+' '+script - thread.start_new_thread(self.client.do_pdbserver, (addr,)) + server = MConnectionServerTCP() + thread.start_new_thread(server.connect, (__addr__,)) + + self.client1 = MPdbTest() + connect_to_target(self.client1) - # There was a problem with earlier unit tests that they were - # working "by coincidence". This ensures that we don't "assume" - # the connection is ready until the target has changed from "local" - # to "remote". - connect_to_target(self.client, __addr__) - self.assertEquals('remote', self.client.target, 'Target is wrong.') + self.client2 = MPdbTest() + self.client2.do_target('dlkdlksldkslkd') + line = self.client2.lines[0] + self.assertEquals('*** Invalid arguments\n', line) + + + self.client3 = MPdbTest() + + server.disconnect() def testRebindOutput(self): """ Test rebinding output. """ + self.server = MPdb() f = open(TESTFN, 'w+') self.server._rebind_output(f) self.server.msg('some text') @@ -83,8 +103,11 @@ f.close() self.assertEquals('some text\n', line, 'Could not rebind output') + def testRebindInput(self): """ Test rebinding input. """ + self.server = MPdb() + f = open(TESTFN, 'w+') f.write('help') f.close() @@ -97,8 +120,8 @@ def testTargetRoutine(self): """ Test that the top-level target routine works properly. """ - invalid_address = 'tcp ::::::' - self.assertRaises(ValueError, target, invalid_address) + #invalid_address = 'tcp ::::::' + pass def test_main(): From buildbot at python.org Mon Jun 26 15:53:24 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 13:53:24 +0000 Subject: [Python-checkins] buildbot warnings in amd64 gentoo trunk Message-ID: <20060626135324.756F91E4004@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1143 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 15:55:21 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 13:55:21 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060626135521.8950C1E4004@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/1084 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 16:00:24 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 14:00:24 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD trunk Message-ID: <20060626140024.D12BD1E4004@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%2520trunk/builds/933 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 16:22:13 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 14:22:13 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20060626142213.205061E4006@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/838 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 16:25:26 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 14:25:26 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP trunk Message-ID: <20060626142526.E044F1E4019@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/1109 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 26 16:33:24 2006 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 26 Jun 2006 16:33:24 +0200 (CEST) Subject: [Python-checkins] r47103 - python/trunk/Lib/test/test_mailbox.py Message-ID: <20060626143324.D00A91E4004@bag.python.org> Author: andrew.kuchling Date: Mon Jun 26 16:33:24 2006 New Revision: 47103 Modified: python/trunk/Lib/test/test_mailbox.py Log: Windows doesn't have os.fork(). I'll just disable this test for now Modified: python/trunk/Lib/test/test_mailbox.py ============================================================================== --- python/trunk/Lib/test/test_mailbox.py (original) +++ python/trunk/Lib/test/test_mailbox.py Mon Jun 26 16:33:24 2006 @@ -723,6 +723,8 @@ def test_lock_conflict(self): # Fork off a subprocess that will lock the file for 2 seconds, # unlock it, and then exit. + if not hasattr(os, 'fork'): + return pid = os.fork() if pid == 0: # In the child, lock the mailbox. From buildbot at python.org Mon Jun 26 16:46:04 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 14:46:04 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060626144604.6DA001E4004@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/823 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 16:49:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 14:49:28 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060626144928.3CE171E4004@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/239 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Mon Jun 26 16:51:21 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 14:51:21 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060626145121.70BA81E4006@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/413 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 26 18:41:14 2006 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 26 Jun 2006 18:41:14 +0200 (CEST) Subject: [Python-checkins] r47104 - peps/trunk/pep-3103.txt Message-ID: <20060626164114.1D9D51E4003@bag.python.org> Author: guido.van.rossum Date: Mon Jun 26 18:41:13 2006 New Revision: 47104 Added: peps/trunk/pep-3103.txt (contents, props changed) Log: Checkpoint submit so I can edit this elsewhere. Added: peps/trunk/pep-3103.txt ============================================================================== --- (empty file) +++ peps/trunk/pep-3103.txt Mon Jun 26 18:41:13 2006 @@ -0,0 +1,449 @@ +PEP: 3103 +Title: A Switch/Case Statement +Version: $Revision$ +Last-Modified: $Date$ +Author: guido at python.org (Guido van Rossum) +Status: Draft +Type: Standards Track +Python-Version: 3.0 +Content-Type: text/x-rst +Created: 25-Jun-2006 +Post-History: never + + +Abstract +======== + +Python-dev has recently seen a flurry of discussion on adding a switch +statement. In this PEP I'm trying to extract my own preferences from +the smorgasboard of proposals, discussing alternatives and explaining +my choices where I can. I'll also indicate how strongly I feel about +alternatives I discuss. + +This PEP should be seen as an alternative to PEP 275. My views are +somewhat different from that PEP's author, but I'm grateful for the +work done in that PEP. + + +Rationale +========= + +A common programming idiom is to consider an expression and do +different things depending on its value. This is usually done with a +chain of if/elif tests; I'll refer to this form as the "if/elif +chain". There are two main motivations to want to introduce new +syntax for this idiom: + +- It is repetitive: the variable and the test operator, usually '==' + or 'in', are repeated in each if/elif branch. + +- It is inefficient: when an expressaion matches the last test value + (or no test value at all) it is compared to each of the preceding + test values. + +Both of these complaints are relatively mild; there isn't a lot of +readability or performance to be gained by writing this differently. +Yet, some kind of switch statement is found in many languages and it +is not unreasonable to expect that its addition to Python will allow +us to write up certain code more cleanly and efficiently than before. + +There are forms of dispatch that are not suitable for the proposed +switch statement; for example, when the number of cases is not +statically known, or when it is desirable to place the code for +different cases in different classes or files. + + +Basic Syntax +============ + +I'm considering several variants of the syntax first proposed in PEP +275 here. There are lots of other possibilities, but I don't see that +they add anything. + +My current preference is alternative 2. + +I should not that all alternatives here have the "implicit break" +property: at the end of the suite for a particular case, the control +flow jumps to the end of the whole switch statement. There is no way +to pass control from one case to another. This in contrast to C, +where an explicit 'break' statement is required to prevent falling +through to the next case. + +In all alternatives, the else-suite is optional. It is more Pythonic +to use 'else' here rather than introducing a new reserved word, +'default', as in C. + +Semantics are discussed in the next top-level section. + +Alternative 1 +------------- + +This is the preferred form in PEP 275:: + + switch EXPR: + case EXPR: + SUITE + case EXPR: + SUITE + ... + else: + SUITE + +The main downside is that the suites where all the action is are +indented two levels deep. + +Alternative 2 +------------- + +This is Fredrik Lundh's preferred form; it differs by not indenting +the cases:: + + switch EXPR: + case EXPR: + SUITE + case EXPR: + SUITE + .... + else: + SUITE + +Alternative 3 +------------- + +This is the same as alternative 2 but leaves out the colon after the +switch:: + + switch EXPR + case EXPR: + SUITE + case EXPR: + SUITE + .... + else: + SUITE + +The hope of this alternative is that is will upset the auto-indent +logic of the average Python-aware text editor less. But it looks +strange to me. + +Alternative 4 +------------- + +This leaves out the 'case' keyword on the basis that it is redundant:: + + switch EXPR: + EXPR: + SUITE + EXPR: + SUITE + ... + else: + SUITE + +Unfortunately now we are forced to indent the case expressions, +because otherwise (at least in the absence of an 'else' keyword) the +parser would have a hard time distinguishing between an unindented +case expression (which continues the switch statement) or an unrelated +statement that starts like an expression (such as an assignment or a +procedure call). The parser is not smart enough to backtrack once it +sees the colon. This is my least favorite alternative. + + +Extended Syntax +=============== + +There is one additional concern that needs to be addressed +syntactically. Often two or more values need to be treated the same. +In C, this done by writing multiple case labels together without any +code between them. The "fall through" semantics then mean that these +are all handled by the same code. Since the Python switch will not +have fall-through semantics (which have yet to find a champion) we +need another solution. Here are some alternatives. + +Alternative A +------------- + +Use:: + + case EXPR: + +to match on a single expression; use:: + + case EXPR, EXPR, ...: + +to match on mulltiple expressions. The is interpreted so that if EXPR +is a parenthesized tuple or another expression whose value is a tuple, +the switch expression must equal that tuple, not one of its elements. +This means that we cannot use a variable to indicate multiple cases. +While this is also true in C's switch statement, it is a relatively +common occurrence in Python (see for example sre_compile.py). + +Alternative B +------------- + +Use:: + + case EXPR: + +to match on a single expression; use:: + + case in EXPR_LIST: + +to match on multiple expressions. If EXPR_LIST is a single +expression, the 'in' forces its interpretation as an iterable (or +something supporting __contains__, in a minority semantics +alternative). If it is multiple expressions, each of those is +considered for a match. + +Alternative C +------------- + +Use:: + + case EXPR: + +to match on a single expression; use:: + + case EXPR, EXPR, ...: + +to match on multiple expressions (as in alternative A); and use:: + + case *EXPR: + +to match on the elements of an expression whose value is an iterable. +The latter two cases can be combined, so that the true syntax is more +like this:: + + case [*]EXPR, [*]EXPR, ...: + +Note that the * notation is similar to the use of prefix * already in +use for variable-length parameter lists and for passing computed +argument lists, and often proposed for value-unpacking (e.g. "a, b, +*c = X" as an alternative to "(a, b), c = X[:2], X[2:]"). + +Alternative D +------------- + +This is a mixture of alternatives B and C; the syntax is like +alternative B but instead of the 'in' keyword it uses '*'. This is +more limited, but still allows the same flexibility. It uses:: + + case EXPR: + +to match on a single expression and:: + + case *EXPR: + +to match on the elements of an iterable. If one wants to specify +multiple matches in one case, one can write this:: + + case *(EXPR, EXPR, ...): + +or perhaps this (although it's a bit strange because the relative +priority of '*' and ',' is different than elsewhere):: + + case * EXPR, EXPR, ...: + +Discussion +---------- + +Alternatives B, C and D are motivated by the desire to specify +multiple cases with the same treatment using a variable representing a +set (usually a tuple) rather than spelling them out. The motivation +for this is usually that if one has several switches over the same set +of cases it's a shame to have to spell out all the alternatives each +time. An additional motivation is to be able to specify *ranges* to +be matched easily and efficiently, similar to Pascal's "1..1000:" +notation. At the same time we want to prevent the kind of mistake +that is common in exception handling (and which will be addressed in +Python 3000 by changing the syntax of the except clause): writing +"case 1, 2:" where "case (1, 2):" was meant, or vice versa. + +The case could be made that the need is insufficient for the added +complexity; C doesn't have a way to express ranges either, and it's +used a lot more than Pascal these days. Also, if a dispatch method +based on dict lookup is chosen as the semantics, large ranges could be +inefficient (consider range(1, sys.maxint)). + +All in all my preferences are (in descending preference) B, A, D', C +where D' is D without the third possibility. + + +Semantics +========= + +There are several issues to review before we can choose the right +semantics. + +If/Elif Chain vs. Dict-based Dispatch +------------------------------------- + +There are two main schools of thought about the switch statement's +semantics. School I wants to define the switch statement in term of +an equivalent if/elif chain. School II prefers to think of it as a +dispatch on a precomputed dictionary. + +The difference is mainly important when either the switch expression +or one of the case expressions is not hashable; school I wants this to +be handled as it would be by an if/elif chain (i.e. hashability of the +expressions involved doesn't matter) while school II is willing to say +that the switch expression and all the case expressions must be +hashable if a switch is to be used; otherwise the user should have +written an if/elif chain. + +There's also a difference of opinion regarding the treatment of +duplicate cases (i.e. two or more cases with the same match +expression). School I wants to treat this the same is an if/elif +chain would treat it (i.e. the first match wins and the code for the +second match is silently unreachable); school II generally wants this +to be an error at the time the switch is frozen. + +There's also a school III which states that the definition of a switch +statement should be in terms of an equivalent if/elif chain, with the +exception that all the expressions must be hashable. + +School I believes that the if/elif chain is the only reasonably, +surprise-free of defining switch semantics, and that optimizations as +suggested by PEP 275's Solution 1 are sufficient to make most common +uses fast. + +School II sees nothing but trouble in that approach: in an if/elif +chain, the test "x == y" might well be comparing two unhashable values +(e.g. two lists); even "x == 1" could be comparing a user-defined +class instance that is not hashable but happens to define equality to +integers. Worse, the hash function might have a bug or a side effect; +if we generate code that believes the hash, a buggy hash might +generate an incorrect match, and if we generate code that catches +errors in the hash to fall back on an if/elif chain, we might hide +genuine bugs. In addition, school II sees little value in allowing +cases involving unhashable values; after all if the user expects such +values, they can just as easily write an if/elif chain. School II +also doesn't believe that it's fair to allow dead code due to +overlappin cases to occur unflagged, when the dict-based dispatch +implementation makes it so easy to trap this. + +School III admits the problems with making hash() optional, but still +believes that the true semantics should be defined by an if/elif chain +even if the implementation should be allowed to use dict-based +dispatch as an optimization. This means that duplicate cases must be +resolved by always choosing the first case, making the second case +undiagnosed dead code. + +Personally, I'm in school II: I believe that the dict-based dispatch +is the one true implementation for switch statements and that we +should face the limitiations and benefits up front. + +When to Freeze the Dispatch Dict +-------------------------------- + +For the supporters of school II (dict-based dispatch), the next big +dividing issue is when to create the dict used for switching. I call +this "freezing the dict". + +The main problem that makes this interesting is the observation that +Python doesn't have named compile-time constants. What is +conceptually a constant, such as re.IGNORECASE, is a variable to the +compiler, and there's nothing to stop crooked code from modifying its +value. + +Option 1 +'''''''' + +The most limiting option is to freeze the dict in the compiler. This +would require that the case expressions are all literals or +compile-time expressions involving only literals and operators whose +semantics are known to the compiler, since with the current state of +Python's dynamic semantics and single-module compilation, there is no +hope for the compiler to know with sufficient certainty the values of +any variables occurring in such expressions. This is widely though +not universally considered too restrictive. + +Raymond Hettinger is the main advocate of this approach. He proposes +a syntax where only a single literal of certain types is allowed as +the case expression. It has the advantage of being unambiguous and +easy to implement. + +My may complaint about this is that by disallowing "named constants" +we force programmers to give up good habits. Named constants are +introduced in most languages to solve the problem of "magic numbers" +occurring in the source code. For example, sys.maxint is a lot more +readable than 2147483647. Raymond proposes to use string literals +instead of named "enums", observing that the string literal's content +can be the name that the constant would otherwise have. Thus, we +could write "case 'IGNORECASE':" instead of "case re.IGNORECASE:". +However, if there is a spelling error in the string literal, the case +will silently be ignored, and who knows when the bug is detected. If +there is a spelling error in a NAME, however, the error will be caught +as soon as it is evaluated. Also, sometimes the constants are +externally defined (e.g. when parsing an file format like JPEG) and we +can't easily choose appropriate string values. Using an explicit +mappping dict sounds like a poor hack. + +Option 2 +'''''''' + +The oldest proposal to deal with this is to freeze the dispatch dict +the first time the switch is executed. At this point we can assume +that all the named "constants" (constant in the programmer's mind, +though not to the compiler) used as case expressions are defined -- +otherwise an if/elif chain would have little chance of success either. +Assuming the switch will be executed many times, doing some extra work +the first time pays back quickly by very quick dispatch times later. + +A mostly theoretical objection to this option is that there is no +obvious object where the dispatch dict can be stored. It can't be +stored on the code object, which is supposed to be immutable; it can't +be stored on the function object, since many function objects may be +created for the same function (e.g. for nested functions). In +practice, I'm sure that something can be found; it could be stored in +a section of the code object that's not considered when comparing two +code objects or when pickling or marshalling a code object; or all +switches could be stored in a dict indexed by weak references to code +objects. + +Another objection is that the first-use rule allows obfuscated code +like this:: + + def foo(x, y): + switch x: + case y: + print 42 + +To the untrained eye (not familiar with Python) this code would be +equivalent to this:: + + def foo(x, y): + if x == y: + print 42 + +but that's not what it does (unless it is always called with the same +value as the second argument). This has been addressed by suggesting +that the case expressions should not be allowed to reference local +variables. But this is somewhat arbitrary. + +A final objection is that in a multi-threaded application, the +first-use rule requires intricate locking in order to guarantee the +correct semantics. (The first-use rule suggests a promise that side +effects of case expressions are incurred exactly once.) + +Option 3 +'''''''' + +TBD + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: From python-checkins at python.org Mon Jun 26 18:54:39 2006 From: python-checkins at python.org (georg.brandl) Date: Mon, 26 Jun 2006 18:54:39 +0200 (CEST) Subject: [Python-checkins] r47105 - peps/trunk/pep-3099.txt Message-ID: <20060626165439.CBED71E4003@bag.python.org> Author: georg.brandl Date: Mon Jun 26 18:54:39 2006 New Revision: 47105 Modified: peps/trunk/pep-3099.txt Log: Fix URL tpyo Modified: peps/trunk/pep-3099.txt ============================================================================== --- peps/trunk/pep-3099.txt (original) +++ peps/trunk/pep-3099.txt Mon Jun 26 18:54:39 2006 @@ -81,7 +81,7 @@ anything except string literals or comments. Thread: sets in P3K? - http://mail.python.org/pipermail/python-3000/2006-April/01474.html + http://mail.python.org/pipermail/python-3000/2006-April/001474.html * Slices and extended slices won't go away (even if the __getslice__ and __setslice__ APIs may be replaced) nor will they return views From python-checkins at python.org Mon Jun 26 19:00:35 2006 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 26 Jun 2006 19:00:35 +0200 (CEST) Subject: [Python-checkins] r47106 - python/trunk/Lib/test/test_mailbox.py Message-ID: <20060626170035.BC0781E4003@bag.python.org> Author: andrew.kuchling Date: Mon Jun 26 19:00:35 2006 New Revision: 47106 Modified: python/trunk/Lib/test/test_mailbox.py Log: Attempt to fix build failure on OS X and Debian alpha; the symptom is consistent with os.wait() returning immediately because some other subprocess had previously exited; the test suite then immediately tries to lock the mailbox and gets an error saying it's already locked. To fix this, do a waitpid() so the test suite only continues once the intended child process has exited. Modified: python/trunk/Lib/test/test_mailbox.py ============================================================================== --- python/trunk/Lib/test/test_mailbox.py (original) +++ python/trunk/Lib/test/test_mailbox.py Mon Jun 26 19:00:35 2006 @@ -740,7 +740,7 @@ self._box.lock) # Wait for child to exit. Locking should now succeed. - pid, status = os.wait() + exited_pid, status = os.waitpid(pid, 0) self._box.lock() self._box.unlock() From python-checkins at python.org Mon Jun 26 20:05:40 2006 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 26 Jun 2006 20:05:40 +0200 (CEST) Subject: [Python-checkins] r47107 - peps/trunk/Makefile peps/trunk/pep-0000.txt peps/trunk/pep-3103.txt Message-ID: <20060626180540.7F71D1E400C@bag.python.org> Author: guido.van.rossum Date: Mon Jun 26 20:05:39 2006 New Revision: 47107 Modified: peps/trunk/Makefile peps/trunk/pep-0000.txt peps/trunk/pep-3103.txt Log: Add PEP 3103: A Switch/Case Statement (GvR). Also fix a reference to cvs in the Makefile. Modified: peps/trunk/Makefile ============================================================================== --- peps/trunk/Makefile (original) +++ peps/trunk/Makefile Mon Jun 26 20:05:39 2006 @@ -23,4 +23,4 @@ -rm *.html update: - cvs update -P -d + svn update Modified: peps/trunk/pep-0000.txt ============================================================================== --- peps/trunk/pep-0000.txt (original) +++ peps/trunk/pep-0000.txt Mon Jun 26 20:05:39 2006 @@ -99,6 +99,7 @@ S 754 IEEE 754 Floating Point Special Values Warnes S 3101 Advanced String Formatting Talin S 3102 Keyword-Only Arguments Talin + S 3103 A Switch/Case Statement GvR Finished PEPs (done, implemented in Subversion) @@ -431,6 +432,7 @@ I 3100 Python 3.0 Plans Kuchling, Cannon S 3101 Advanced String Formatting Talin S 3102 Keyword-Only Arguments Talin + S 3103 A Switch/Case Statement GvR Key Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Mon Jun 26 20:05:39 2006 @@ -8,7 +8,7 @@ Python-Version: 3.0 Content-Type: text/x-rst Created: 25-Jun-2006 -Post-History: never +Post-History: 26-Jun-2006 Abstract @@ -24,6 +24,11 @@ somewhat different from that PEP's author, but I'm grateful for the work done in that PEP. +This PEP introduces canonical names for the many variants that have +been discussed for different aspects of the syntax and semantics, such +as "alternative 2", "school II", "Option 3" and so on. Hopefully +these names will help the discussion. + Rationale ========= @@ -216,10 +221,10 @@ case [*]EXPR, [*]EXPR, ...: -Note that the * notation is similar to the use of prefix * already in -use for variable-length parameter lists and for passing computed -argument lists, and often proposed for value-unpacking (e.g. "a, b, -*c = X" as an alternative to "(a, b), c = X[:2], X[2:]"). +The * notation is similar to the use of prefix * already in use for +variable-length parameter lists and for passing computed argument +lists, and often proposed for value-unpacking (e.g. "a, b, *c = X" as +an alternative to "(a, b), c = X[:2], X[2:]"). Alternative D ------------- @@ -390,16 +395,17 @@ Assuming the switch will be executed many times, doing some extra work the first time pays back quickly by very quick dispatch times later. -A mostly theoretical objection to this option is that there is no -obvious object where the dispatch dict can be stored. It can't be -stored on the code object, which is supposed to be immutable; it can't -be stored on the function object, since many function objects may be -created for the same function (e.g. for nested functions). In -practice, I'm sure that something can be found; it could be stored in -a section of the code object that's not considered when comparing two -code objects or when pickling or marshalling a code object; or all -switches could be stored in a dict indexed by weak references to code -objects. +An objection to this option is that there is no obvious object where +the dispatch dict can be stored. It can't be stored on the code +object, which is supposed to be immutable; it can't be stored on the +function object, since many function objects may be created for the +same function (e.g. for nested functions). In practice, I'm sure that +something can be found; it could be stored in a section of the code +object that's not considered when comparing two code objects or when +pickling or marshalling a code object; or all switches could be stored +in a dict indexed by weak references to code objects. The solution +should also be careful not to leak switch dicts between multiple +interpreters. Another objection is that the first-use rule allows obfuscated code like this:: @@ -419,17 +425,101 @@ but that's not what it does (unless it is always called with the same value as the second argument). This has been addressed by suggesting that the case expressions should not be allowed to reference local -variables. But this is somewhat arbitrary. +variables, but this is somewhat arbitrary. A final objection is that in a multi-threaded application, the first-use rule requires intricate locking in order to guarantee the correct semantics. (The first-use rule suggests a promise that side -effects of case expressions are incurred exactly once.) +effects of case expressions are incurred exactly once.) This may be +as tricky as the import lock has proved to be, since the lock has to +be held while all the case expressions are being evaluated. Option 3 '''''''' -TBD +A proposal that has been winning support (including mine) is to freeze +a switch's dict when the innermost function containing it is defined. +The switch dict is stored on the function object, just as parameter +defaults are, and in fact the case expressions are evaluated at the +same time and in the same scope as the parameter defaults (i.e. in the +scope containing the function definition). + +This option has the advantage of avoiding many of the finesses needed +to make option 2 work: there's no need for locking, no worry about +immutable code objects or multiple interpreters. It also provides a +clear explanation for why locals can't be referenced in case +expressions. + +This option works just as well for situations where one would +typically use a switch; case expressions involving imported or global +named constants work exactly the same way as in option 2, as long as +they are imported or defined before the function definition is +encountered. + +A downside however is that the dispatch dict for a switch inside a +nested function must be recomputed each time the nested function is +defined. For certain "functional" styles of programming this may make +switch unattractive in nested functions. (Unless all case expressions +are compile-time constants; then the compiler is of course free to +optimize away the swich freezing code and make the dispatch table part +of the code object.) + +Another downside is that under this option, there's no clear moment +when the dispatch dict is frozen for a switch that doesn't occur +inside a function. There are a few pragmatic choices for how to treat +a switch outside a function: + + (a) Disallow it. + (b) Translate it into an if/elif chain. + (c) Allow only compile-time constant expressions. + (d) Compute the dispatch dict each time the switch is reached. + (e) Like (b) but tests that all expressions evaluated are hashable. + +Of these, (a) seems too restrictive: it's uniformly worse than (c); +and (d) has poor performance for little or no benefits compared to +(b). It doesn't make sense to have a performance-critical inner loop +at the module level, as all local variable references are slow there; +hence (b) is my (weak) favorite. Perhaps I should favor (e), which +attempts to prevent atypical use of a switch; examples that work +interactively but not in a function are annoying. In the end I don't +think this issue is all that important (except it must be resolved +somehow) and am willing to leave it up to whoever ends up implementing +it. + +When a switch occurs in a class but not in a function, we can freeze +the dispatch dict at the same time the temporary function object +representing the class body is created. This means the case +expressions can reference module globals but not class variables. +Alternatively, if we choose (b) above, we could choose this +implementation inside a class definition as well. + +Option 4 +'''''''' + +There are a number of proposals to add a construct to the language +that makes the concept of a value pre-computed at function definition +time generally available, without tying it either to parameter default +values or case expressions. Some keywords proposed include 'const', +'static', 'only' or 'cached'. The associated syntax and semantics +vary. + +These proposals are out of scope for this PEP, except to suggest that +*if* such a proposal is accepted, there are two ways for the switch to +benefit: we could require case expressions to be either compile-time +constants or pre-computed values; or we could make pre-computed values +the default (and only) evaluation mode for case expressions. The +latter would be my preference, since I don't see a use for more +dynamic case expressions that isn't addressed adequately by writing an +explicit if/elif chain. + + +Conclusion +========== + +It is too early to decide. I'd like to see at least one completed +proposal for pre-computed values before deciding. In the mean time, +Python is fine without a switch statement, and perhaps those who claim +it would be a mistake to add one are right. Copyright From python-checkins at python.org Mon Jun 26 20:09:41 2006 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 26 Jun 2006 20:09:41 +0200 (CEST) Subject: [Python-checkins] r47108 - peps/trunk/pep-3103.txt Message-ID: <20060626180941.40D841E4003@bag.python.org> Author: guido.van.rossum Date: Mon Jun 26 20:09:41 2006 New Revision: 47108 Modified: peps/trunk/pep-3103.txt Log: Fix a reST markup warning. Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Mon Jun 26 20:09:41 2006 @@ -221,7 +221,7 @@ case [*]EXPR, [*]EXPR, ...: -The * notation is similar to the use of prefix * already in use for +The `*` notation is similar to the use of prefix `*` already in use for variable-length parameter lists and for passing computed argument lists, and often proposed for value-unpacking (e.g. "a, b, *c = X" as an alternative to "(a, b), c = X[:2], X[2:]"). From buildbot at python.org Mon Jun 26 20:22:08 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 26 Jun 2006 18:22:08 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060626182208.AFC301E4004@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/453 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Mon Jun 26 20:47:03 2006 From: python-checkins at python.org (david.goodger) Date: Mon, 26 Jun 2006 20:47:03 +0200 (CEST) Subject: [Python-checkins] r47109 - peps/trunk/pep-3103.txt Message-ID: <20060626184703.F0D231E4003@bag.python.org> Author: david.goodger Date: Mon Jun 26 20:47:03 2006 New Revision: 47109 Modified: peps/trunk/pep-3103.txt Log: typo fixes Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Mon Jun 26 20:47:03 2006 @@ -42,7 +42,7 @@ - It is repetitive: the variable and the test operator, usually '==' or 'in', are repeated in each if/elif branch. -- It is inefficient: when an expressaion matches the last test value +- It is inefficient: when an expression matches the last test value (or no test value at all) it is compared to each of the preceding test values. @@ -67,7 +67,7 @@ My current preference is alternative 2. -I should not that all alternatives here have the "implicit break" +I should note that all alternatives here have the "implicit break" property: at the end of the suite for a particular case, the control flow jumps to the end of the whole switch statement. There is no way to pass control from one case to another. This in contrast to C, @@ -324,7 +324,7 @@ cases involving unhashable values; after all if the user expects such values, they can just as easily write an if/elif chain. School II also doesn't believe that it's fair to allow dead code due to -overlappin cases to occur unflagged, when the dict-based dispatch +overlapping cases to occur unflagged, when the dict-based dispatch implementation makes it so easy to trap this. School III admits the problems with making hash() optional, but still @@ -368,7 +368,7 @@ the case expression. It has the advantage of being unambiguous and easy to implement. -My may complaint about this is that by disallowing "named constants" +My main complaint about this is that by disallowing "named constants" we force programmers to give up good habits. Named constants are introduced in most languages to solve the problem of "magic numbers" occurring in the source code. For example, sys.maxint is a lot more @@ -377,10 +377,10 @@ can be the name that the constant would otherwise have. Thus, we could write "case 'IGNORECASE':" instead of "case re.IGNORECASE:". However, if there is a spelling error in the string literal, the case -will silently be ignored, and who knows when the bug is detected. If +will silently be ignored, and who knows when the bug is detected. If there is a spelling error in a NAME, however, the error will be caught as soon as it is evaluated. Also, sometimes the constants are -externally defined (e.g. when parsing an file format like JPEG) and we +externally defined (e.g. when parsing a file format like JPEG) and we can't easily choose appropriate string values. Using an explicit mappping dict sounds like a poor hack. From python-checkins at python.org Mon Jun 26 22:48:08 2006 From: python-checkins at python.org (ka-ping.yee) Date: Mon, 26 Jun 2006 22:48:08 +0200 (CEST) Subject: [Python-checkins] r47110 - peps/trunk/pep-3103.txt Message-ID: <20060626204808.0FF571E400F@bag.python.org> Author: ka-ping.yee Date: Mon Jun 26 22:48:07 2006 New Revision: 47110 Modified: peps/trunk/pep-3103.txt Log: Add a note on what School I finds risky about School II's approach. Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Mon Jun 26 22:48:07 2006 @@ -307,13 +307,19 @@ statement should be in terms of an equivalent if/elif chain, with the exception that all the expressions must be hashable. -School I believes that the if/elif chain is the only reasonably, -surprise-free of defining switch semantics, and that optimizations as -suggested by PEP 275's Solution 1 are sufficient to make most common -uses fast. - -School II sees nothing but trouble in that approach: in an if/elif -chain, the test "x == y" might well be comparing two unhashable values +School I believes that the if/elif chain is the only reasonable, +surprise-free way of defining switch semantics, and that optimizations +as suggested by PEP 275's Solution 1 are sufficient to make most +common uses fast. School I sees trouble in the approach of +pre-freezing a dispatch dictionary because it places a new and unusual +burden on programmers to understand exactly what kinds of case values +are allowed to be frozen and when the case values will be frozen, or +they might be surprised by the switch statement's behavior. + +School II sees trouble in trying to achieve semantics that match +those of an if/elif chain while optimizing the switch statement into +a hash lookup in a dispatch dictionary. In an if/elif chain, the +test "x == y" might well be comparing two unhashable values (e.g. two lists); even "x == 1" could be comparing a user-defined class instance that is not hashable but happens to define equality to integers. Worse, the hash function might have a bug or a side effect; From neal at metaslash.com Mon Jun 26 23:07:54 2006 From: neal at metaslash.com (Neal Norwitz) Date: Mon, 26 Jun 2006 17:07:54 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060626210754.GA12085@python.psfb.org> test_socket leaked [0, 0, 205] references From python-checkins at python.org Tue Jun 27 00:08:11 2006 From: python-checkins at python.org (matt.fleming) Date: Tue, 27 Jun 2006 00:08:11 +0200 (CEST) Subject: [Python-checkins] r47111 - in sandbox/trunk/pdb: mconnection.py mpdb.py mthread.py test/test_mconnection.py test/test_mpdb.py test/thread_script.py Message-ID: <20060626220811.C58FF1E4003@bag.python.org> Author: matt.fleming Date: Tue Jun 27 00:08:11 2006 New Revision: 47111 Added: sandbox/trunk/pdb/test/thread_script.py Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/mthread.py sandbox/trunk/pdb/test/test_mconnection.py sandbox/trunk/pdb/test/test_mpdb.py Log: start implementing the thread command and of course, unit tests ;-) Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Tue Jun 27 00:08:11 2006 @@ -57,8 +57,8 @@ """ self._dev = device try: - self.output = open(self._dev, 'w') self.input = open(self._dev, 'r') + self.output = open(self._dev, 'w') except IOError,e: # Use e[1] for more detail about why the connection failed raise ConnectionFailed, e[1] Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Tue Jun 27 00:08:11 2006 @@ -53,6 +53,7 @@ self.waiter = threading.Event() self.tracers = [] self.threads = [] + self.current_thread = threading.currentThread() self._info_cmds.append('target') def _rebind_input(self, new_input): @@ -151,12 +152,10 @@ return # Thread not being traced self.threads.append(threading.currentThread()) - self.msg('New thread: %s' % self.threads[-1].getName()) - m = MTracer(self.stdout) + self.msg('New thread: %s' % self.threads[-1]) + m = MTracer(self.breaks, self.stdout) self.tracers.append(m) sys.settrace(m.trace_dispatch) - - # Debugger commands def do_attach(self, addr): @@ -232,7 +231,7 @@ self.errmsg('Unknown target type') return self.connection = eval(target+'()') - try: + try: self.connection.connect(addr) except ConnectionFailed, err: self.errmsg("Failed to connect to %s: (%s)" % (addr, err)) @@ -324,6 +323,39 @@ self._rebind_input(self.connection) self._rebind_output(self.connection) + def do_thread(self, arg): + """Use this command to switch between threads. +The new thread ID must be currently known. + +List of thread subcommands: + +thread apply -- Apply a command to a thread + +Type "help thread" followed by thread subcommand name for full documentation. +Command name abbreviations are allowed if unambiguous. +""" + args = arg.split() + if len(args) == 0: + self.msg('Current thread is %s' % self.current_thread) + return + if 'apply'.startswith(args[0]): + if len(args) < 2: + self.errmsg('Please specify a thread ID') + return + if len(args) < 3: + self.errmsg('Please specify a command following the thread' \ + + ' ID') + return + # These should always be in sync + if len(self.threads) == 0: + self.errmsg('No threads') + return + t = self.threads[int(args[1])-1] + t_tracer = self.tracers[int(args[1])-1] + func = args[2] + cmd = eval('t.' + func) + result = cmd(args[3:]) + def pdbserver(addr): """ This method sets up a pdbserver debugger that allows debuggers to connect to 'address' using 'protocol'. The argument 'filename' Modified: sandbox/trunk/pdb/mthread.py ============================================================================== --- sandbox/trunk/pdb/mthread.py (original) +++ sandbox/trunk/pdb/mthread.py Tue Jun 27 00:08:11 2006 @@ -1,18 +1,60 @@ """ This file contains all code for allowing the debugging of threads. """ +from bdb import Breakpoint +import os import sys import threading class MTracer(object): - """ A class to trace a thread. """ - def __init__(self, stdout=None): + """ A class to trace a thread. Breakpoints can be passed from + a main debugger to this debugger through the constructor + which is useful, for instance, if a breakpoint occurs inside + a thread's run() method. + """ + def __init__(self, breaks=None, stdout=None): self.thread = threading.currentThread() - # No other thread can be debugged whilst this is set - # (including the MainThread) if stdout is None: stdout = sys.stdout self.out = stdout + # Each tracer instance must keep track of its own breakpoints + if breaks is None: + breaks = {} + self.breaks = breaks + self.fncache = {} + + def canonic(self, filename): + if filename == "<" + filename[1:-1] + ">": + return filename + canonic = self.fncache.get(filename) + if not canonic: + canonic = os.path.abspath(filename) + canonic = os.path.normcase(canonic) + self.fncache[filename] = canonic + return canonic + + def info(self, arg): + args = arg.split() + if 'break'.startswith(args[0]): + return self.breaks + + def _get_break(self, filename, lineno): + """ Return the breakpoint at [filename:]lineno. """ + filename = self.canonic(filename) + return filename in self.breaks and \ + lineno in self.breaks[filename] + + def _get_breaks(self, filename, lineno): + """ Return all the breakpoints set in this thread. """ + filename = self.canonic(filename) + return filename in self.breaks and \ + lineno in self.breaks[filename] and \ + Breakpoint.bplist[filename, lineno] or [] + def _set_break(self, filename, lineno, temporary=0, cond = None, + funcname=None): + """ Set a breakpoint in this thread. """ + pass + def trace_dispatch(self, frame, event, arg): if event == 'line': print >> self.out, self.thread.getName(),'*** line' Modified: sandbox/trunk/pdb/test/test_mconnection.py ============================================================================== --- sandbox/trunk/pdb/test/test_mconnection.py (original) +++ sandbox/trunk/pdb/test/test_mconnection.py Tue Jun 27 00:08:11 2006 @@ -57,6 +57,7 @@ def testDisconnectDisconnected(self): """(tcp) Disconnect a disconnected session. """ s = MConnectionServerTCP() + s.disconnect() s.disconnect() @@ -142,6 +143,12 @@ line = self.server.readline() self.assertEquals('', line, 'Could not read third line.') + def testInvalidFilename(self): + """(serial) Connect to an invalid server. """ + client = MConnectionSerial() + self.assertRaises(ConnectionFailed, client.connect, + '/dev/pleasepleasepleasedontexit') + def tearDown(self): self.server.disconnect() self.client.disconnect() Modified: sandbox/trunk/pdb/test/test_mpdb.py ============================================================================== --- sandbox/trunk/pdb/test/test_mpdb.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Tue Jun 27 00:08:11 2006 @@ -40,6 +40,7 @@ def __init__(self): MPdb.__init__(self) self.lines = [] + self.botframe = None def msg_nocr(self, msg): self.lines.append(msg) @@ -70,7 +71,6 @@ line = self.server2.lines[0] self.assertEquals('*** Unknown protocol\n', line) - def testTarget(self): """ Test the target command. """ server = MConnectionServerTCP() @@ -80,13 +80,25 @@ connect_to_target(self.client1) self.client2 = MPdbTest() - self.client2.do_target('dlkdlksldkslkd') line = self.client2.lines[0] self.assertEquals('*** Invalid arguments\n', line) + # Hope there's nothing running on this port + self.client2.do_target('tcp localhost:9000') + line = self.client2.lines[1] + errmsg = '*** Failed to connect to localhost:9000: (Connection' \ + + ' refused)\n' + self.assertEquals(errmsg, line) + + addr = 'serial /dev/hopefullythisdevicenamedoesntexist' + self.client2.do_target(addr) + line = self.client2.lines[2] + errmsg = '*** Failed to connect to ' \ + + '/dev/hopefullythisdevicenamedoesntexist:' \ + + ' (No such file or directory)\n' - self.client3 = MPdbTest() + self.assertEquals(errmsg, line) server.disconnect() @@ -102,7 +114,6 @@ line = f.readline() f.close() self.assertEquals('some text\n', line, 'Could not rebind output') - def testRebindInput(self): """ Test rebinding input. """ @@ -116,13 +127,39 @@ self.server._rebind_input(f) line = self.server.stdin.readline() + f.close() self.assertEquals(line, 'help', 'Could not rebind input.') - def testTargetRoutine(self): - """ Test that the top-level target routine works properly. """ - #invalid_address = 'tcp ::::::' - pass + def testThread(self): + """ Test the thread command. """ + server = MConnectionServerTCP() + thread.start_new_thread(server.connect, (__addr__,)) + + self.client1 = MPdbTest() + connect_to_target(self.client1) + # Thread with no commands should return current thread + self.client1.onecmd('thread') + assert 'MainThread' in self.client1.lines[0] + + # 'thread apply' without thread ID should return an error message + self.client1.onecmd('thread apply') + line = self.client1.lines[1] + self.assertEquals('*** Please specify a thread ID\n', line) + + # Need a command to actually apply to a thread + self.client1.onecmd('thread apply 49843') + line = self.client1.lines[2] + errmsg = '*** Please specify a command following the thread ID\n' + self.assertEquals(errmsg, line) + + # We've still not started any threads + self.client1.onecmd('thread apply 2 info break') + line = self.client1.lines[3] + errmsg = '*** No threads\n' + self.assertEquals(errmsg, line) + + server.disconnect() def test_main(): test_support.run_unittest(TestRemoteDebugging) Added: sandbox/trunk/pdb/test/thread_script.py ============================================================================== --- (empty file) +++ sandbox/trunk/pdb/test/thread_script.py Tue Jun 27 00:08:11 2006 @@ -0,0 +1,21 @@ +#!/usr/bin/env python +""" This is a script that is being used to debug whilst trying to get +the thread debugging features working. +""" + +import threading + +class MyThread(threading.Thread): + def run(self): + for i in range(10): + print i + +def func(): + t = MyThread() + t.start() + t.join() + +if __name__ == '__main__': + func() + + From python-checkins at python.org Tue Jun 27 04:48:31 2006 From: python-checkins at python.org (mateusz.rukowicz) Date: Tue, 27 Jun 2006 04:48:31 +0200 (CEST) Subject: [Python-checkins] r47112 - sandbox/trunk/decimal-c/_decimal.c Message-ID: <20060627024831.1B0D51E4004@bag.python.org> Author: mateusz.rukowicz Date: Tue Jun 27 04:48:30 2006 New Revision: 47112 Modified: sandbox/trunk/decimal-c/_decimal.c Log: remainder_near implemented, some minor bugs fixed. Modified: sandbox/trunk/decimal-c/_decimal.c ============================================================================== --- sandbox/trunk/decimal-c/_decimal.c (original) +++ sandbox/trunk/decimal-c/_decimal.c Tue Jun 27 04:48:30 2006 @@ -694,6 +694,7 @@ static decimalobject *_decimal_fromliteral(PyTypeObject *, char *str, long, contextobject *); static decimalobject *_do_decimal_multiply(decimalobject *, decimalobject *, contextobject *); static decimalobject *_do_decimal_subtract(decimalobject *, decimalobject *, contextobject *); +static PyObject *context_ignore_flags(contextobject *self, PyObject *args); /* Exception handlers *********************************************************/ @@ -777,8 +778,9 @@ { HANDLE_ERROR(ctx, C_DIV_IMPOSSIBLE, expl, NULL); - Py_INCREF(PyDecimal_NaN); - return PyDecimal_NaN; +/* Py_INCREF(PyDecimal_NaN); */ + + return Py_BuildValue("(OO)", PyDecimal_NaN, PyDecimal_NaN); } static decimalobject * @@ -787,8 +789,15 @@ { HANDLE_ERROR(ctx, C_DIV_UNDEFINED, expl, NULL); - Py_INCREF(PyDecimal_NaN); - return PyDecimal_NaN; + if (!two) { + Py_INCREF(PyDecimal_NaN); + return PyDecimal_NaN; + } + + else { + return Py_BuildValue("(OO)", PyDecimal_NaN, PyDecimal_NaN); + } + } static int @@ -1399,6 +1408,7 @@ for (i = 0; i < tmp->ob_size-1; i++) tmp->digits[i] = tmp->digits[i+1]; /* TODO make last digit 0 */ tmp->ob_size--; + tmp->limb_count = (tmp->ob_size + LOG -1)/LOG; } tmp->exp = exp; @@ -1784,9 +1794,10 @@ res = _do_real_decimal_compare(self, other, ctx); Py_DECREF(other); - if (PyErr_Occurred()) + /* we will return 0 anyway */ +/* if (PyErr_Occurred()) return NULL; - +*/ return res; } @@ -2049,12 +2060,277 @@ } +/* it is rewritten from python implementation */ static decimalobject * _do_decimal_remainder_near(decimalobject *self, decimalobject *other, contextobject *ctx) { - /* XXX */ - Py_RETURN_NONE; + contextobject *ctx2; + decimalobject *side, *r; + decimalobject *comparison = 0; + int rounding_dec; + int decrease; + PyObject *ignored_flags; + PyObject *flags = 0; + int s1, s2; + if (ISSPECIAL(self) || ISSPECIAL(other)) { + decimalobject *nan; + if (_check_nans(self, other, ctx, &nan)) + return nan; + } + + if (decimal_nonzero(self) && !decimal_nonzero(other)) + return handle_InvalidOperation(self->ob_type, ctx, "x % 0", NULL); + + ctx2 = context_shallow_copy(ctx); + + if (!ctx2) + return NULL; + + ignored_flags = PyTuple_New(2); + + if (!ignored_flags) + goto err; + + Py_INCREF(errors[S_ROUNDED]); + Py_INCREF(errors[S_INEXACT]); + PyTuple_SET_ITEM(ignored_flags, 0, errors[S_ROUNDED]); + PyTuple_SET_ITEM(ignored_flags, 1, errors[S_INEXACT]); + + flags = context_ignore_flags(ctx2, ignored_flags); + Py_DECREF(ignored_flags); + + if (!flags) + goto err; + + { + PyObject *divmod = _do_decimal__divide(self, other, 1, ctx2); + if (!divmod) { + PyObject *r = context_regard_flags(ctx2, flags); + Py_XDECREF(r); + goto err; + } + + side = PySequence_GetItem(divmod, 0); + r = PySequence_GetItem(divmod, 1); + + Py_DECREF(divmod); + } + + if (GETNAN(r)) { + PyObject *ret; + ret = context_regard_flags(ctx2, flags); + if (!ret) + goto err; + + Py_DECREF(ret); + Py_DECREF(flags); + Py_DECREF(ctx2); + Py_DECREF(side); + + return r; + + } + + rounding_dec = ctx2->rounding_dec; + ctx2->rounding_dec = NEVER_ROUND; + + + { + decimalobject *two = _NEW_decimalobj(1, 0, 0); + if (!two) { + PyObject *r = context_regard_flags(ctx2, flags); + Py_XDECREF(r); + goto err; + } + + two->limbs[0] = 2; + two->sign = other->sign&1; + + comparison = _do_decimal__divide(other, two, 0, ctx2); + Py_DECREF(two); + + if (!comparison) + goto err; + } + + ctx2->rounding_dec = rounding_dec; + { + PyObject *ret; + ret = context_regard_flags(ctx2, flags); + if (!ret) + goto err; + Py_DECREF(ret); + } + s1 = r->sign; + s2 = comparison->sign; + /* we don't want to loose info about infinity */ + r->sign &= 254; + comparison->sign &= 254; + + { + int cmp = _do_real_decimal_compare(r, comparison, ctx2); + if (PyErr_Occurred()) + goto err; + + if (cmp == -1) { + PyObject *ret; + decimalobject *fixed; + r->sign = s1; + comparison->sign = s2; + + ret = _do_decimal__divide(self, other, 1, ctx2); + + if (!ret) + goto err; + Py_DECREF(ret); + fixed = _decimal_fix(r, ctx2); + Py_DECREF(r); + Py_DECREF(ctx2); + Py_DECREF(flags); + Py_DECREF(side); + Py_DECREF(comparison); + + return fixed; + } + } + + r->sign = s1; + comparison->sign = s2; + + rounding_dec = ctx2->rounding_dec; + ctx2->rounding_dec = NEVER_ROUND; + + { + PyObject *divmod = _do_decimal__divide(self, other, 1, ctx2); + + if (!divmod) + goto err; + + Py_DECREF(side); + Py_DECREF(r); + + side = PySequence_GetItem(divmod, 0); + r = PySequence_GetItem(divmod, 1); + Py_DECREF(divmod); + } + + ctx2->rounding_dec = rounding_dec; + + if (GETNAN(r)) { + Py_DECREF(ctx2); + Py_DECREF(flags); + Py_DECREF(side); + Py_DECREF(comparison); + return r; + } + + decrease = side->limbs[0] &1; + side->sign &= 254; + + s1 = r->sign; + s2 = comparison->sign; + r->sign &= 254; + comparison->sign &= 254; + + { + int cmp = _do_real_decimal_compare(r, comparison, ctx2); + + if (PyErr_Occurred()) + goto err; + + if (cmp == 1 || decrease && cmp == 0) { + r->sign = s1; + comparison->sign = s2; + ctx2->prec += 1; + + { + decimalobject *one; + decimalobject *tmp; + one = _NEW_decimalobj(1,0,0); + if (!one) + goto err; + + one->limbs[0] = 1; + tmp = _do_decimal_add(side, one, ctx2); + Py_DECREF(one); + + if (!tmp) + goto err; + + if (tmp->ob_size >= ctx2->prec) { + PyObject *ret; + PyObject *tup; + Py_DECREF(tmp); + ctx2->prec -= 1; + + tup = handle_DivisionImpossible(self->ob_type, ctx2, NULL); + + if (!tup) + goto err; + + ret = PySequence_GetItem(tup, 1); + + Py_DECREF(tup); + Py_DECREF(ctx2); + Py_XDECREF(flags); + Py_DECREF(side); + Py_DECREF(r); + Py_DECREF(comparison); + + return ret; + } + Py_DECREF(tmp); + + } + ctx2->prec -= 1; + + if (self->sign == other->sign) { + decimalobject *tmp = _do_decimal_subtract(r, other, ctx2); + if (!tmp) + goto err; + + Py_DECREF(r); + r = tmp; + tmp = 0; + } + else { + decimalobject *tmp = _do_decimal_add(r, other, ctx2); + if (!tmp) + goto err; + Py_DECREF(r); + r = tmp; + } + } + else { + r->sign = s1; + comparison->sign = s2; + } + } + + Py_DECREF(comparison); + Py_DECREF(side); + Py_DECREF(flags); + Py_DECREF(ctx2); + + { + decimalobject *fixed = _decimal_fix(r, ctx); + if (!fixed) { + Py_DECREF(r); + return NULL; + } + Py_DECREF(r); + return fixed; + } + +err: + Py_DECREF(ctx2); + Py_XDECREF(flags); + Py_XDECREF(side); + Py_XDECREF(r); + Py_XDECREF(comparison); + + return NULL; } DECIMAL_BINARY_FUNC(remainder_near) @@ -3187,10 +3463,13 @@ op2->limb_count + 1 : op1->limb_count; + /* if !shouldround, then we need log2(divider.int) + log10(op1.int) + * + some constant, to make sure, division is possible */ if (!shouldround && !divmod) - Py_RETURN_NONE; /* TODO */ - - prec_needed = ctx->prec+1; + prec_needed = op2->ob_size * 4 + op1->ob_size + 5; + /* we need only ctx->prec + 1 and remainder */ + else + prec_needed = ctx->prec+1; /* we need (prec_needed + LOG -1)/ LOG rounded up limbs, because * it may happen, that first limb is between 1 and 9, so we'll * get (significant_limbs-1) * LOG + 1 digits */ @@ -3271,6 +3550,7 @@ else remainder_ret->exp = op1->exp; remainder_ret->ob_size = _limb_size_s(remainder_ret->limbs, remainder_ret->ob_size); + remainder_ret->limb_count = (remainder_ret->ob_size + LOG -1)/LOG; } old_size = result->ob_size; @@ -5829,9 +6109,14 @@ return NULL; } /* regard_flags allows a list of flags as the first arg */ - flags = PyTuple_GET_ITEM(args, 0); - if (PyTuple_GET_SIZE(args) != 1 && !PySequence_Check(flags)) + flags = PySequence_GetItem(args, 0); +/* if (PyTuple_GET_SIZE(args) != 1 && !PySequence_Check(flags))*/ +/* flags = args; */ + if (!PyTuple_Check(flags) && !PyList_Check(flags)) + { + Py_DECREF(flags); flags = args; + } for (i = 0; i < PySequence_Size(flags); i++) { flag = PySequence_GetItem(flags, i); From python-checkins at python.org Tue Jun 27 06:06:46 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 27 Jun 2006 06:06:46 +0200 (CEST) Subject: [Python-checkins] r47113 - python/trunk/Misc/valgrind-python.supp Message-ID: <20060627040646.A7FD51E4004@bag.python.org> Author: neal.norwitz Date: Tue Jun 27 06:06:46 2006 New Revision: 47113 Modified: python/trunk/Misc/valgrind-python.supp Log: Ignore some more warnings in the dynamic linker on an older gentoo Modified: python/trunk/Misc/valgrind-python.supp ============================================================================== --- python/trunk/Misc/valgrind-python.supp (original) +++ python/trunk/Misc/valgrind-python.supp Tue Jun 27 06:06:46 2006 @@ -134,6 +134,15 @@ ### { + Generic gentoo ld problems + Memcheck:Cond + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so +} + +{ DBM problems, see test_dbm Memcheck:Param write(buf) From python-checkins at python.org Tue Jun 27 06:09:14 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 27 Jun 2006 06:09:14 +0200 (CEST) Subject: [Python-checkins] r47114 - python/trunk/Misc/build.sh Message-ID: <20060627040914.3BAEA1E4004@bag.python.org> Author: neal.norwitz Date: Tue Jun 27 06:09:13 2006 New Revision: 47114 Modified: python/trunk/Misc/build.sh Log: Instead of doing a make test, run the regression tests out of the installed copy. This will hopefully catch problems where directories are added under Lib/ but not to Makefile.pre.in. This breaks out the 2 runs of the test suite with and without -O which is also nicer. Modified: python/trunk/Misc/build.sh ============================================================================== --- python/trunk/Misc/build.sh (original) +++ python/trunk/Misc/build.sh Tue Jun 27 06:09:13 2006 @@ -54,6 +54,12 @@ INSTALL_DIR="/tmp/python-test/local" RSYNC_OPTS="-aC -e ssh" +# Always run the installed version of Python. +PYTHON=$INSTALL_DIR/bin/python + +# Python options and regression test program that should always be run. +REGRTEST_ARGS="-E -tt $INSTALL_DIR/lib/python2.5/test/regrtest.py" + REFLOG="build/reflog.txt.out" # These tests are not stable and falsely report leaks sometimes. # The entire leak report will be mailed if any test not in this list leaks. @@ -158,18 +164,24 @@ ## make and run basic tests F=make-test.out start=`current_time` - make test >& build/$F + $PYTHON $REGRTEST_ARGS >& build/$F NUM_FAILURES=`grep -ic " failed:" build/$F` update_status "Testing basics ($NUM_FAILURES failures)" "$F" $start - ## FIXME: should mail since -uall below should find same problems mail_on_failure "basics" build/$F + F=make-test-opt.out + start=`current_time` + $PYTHON -O $REGRTEST_ARGS >& build/$F + NUM_FAILURES=`grep -ic " failed:" build/$F` + update_status "Testing opt ($NUM_FAILURES failures)" "$F" $start + mail_on_failure "opt" build/$F + ## run the tests looking for leaks F=make-test-refleak.out start=`current_time` ## ensure that the reflog exists so the grep doesn't fail touch $REFLOG - ./python ./Lib/test/regrtest.py -R 4:3:$REFLOG -u network $LEAKY_SKIPS >& build/$F + $PYTHON $REGRTEST_ARGS -R 4:3:$REFLOG -u network $LEAKY_SKIPS >& build/$F NUM_FAILURES=`egrep -vc "$LEAKY_TESTS" $REFLOG` update_status "Testing refleaks ($NUM_FAILURES failures)" "$F" $start mail_on_failure "refleak" $REFLOG @@ -179,7 +191,7 @@ start=`current_time` ## skip curses when running from cron since there's no terminal ## skip sound since it's not setup on the PSF box (/dev/dsp) - ./python -E -tt ./Lib/test/regrtest.py -uall -x test_curses test_linuxaudiodev test_ossaudiodev >& build/$F + $PYTHON $REGRTEST_ARGS -uall -x test_curses test_linuxaudiodev test_ossaudiodev >& build/$F NUM_FAILURES=`grep -ic " failed:" build/$F` update_status "Testing all except curses and sound ($NUM_FAILURES failures)" "$F" $start mail_on_failure "all" build/$F @@ -189,7 +201,7 @@ ## make doc -cd Doc +cd $DIR/Doc F="make-doc.out" start=`current_time` make >& ../build/$F From python-checkins at python.org Tue Jun 27 06:12:59 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 27 Jun 2006 06:12:59 +0200 (CEST) Subject: [Python-checkins] r47115 - in python/trunk: Makefile.pre.in Misc/NEWS Message-ID: <20060627041259.869381E4004@bag.python.org> Author: neal.norwitz Date: Tue Jun 27 06:12:58 2006 New Revision: 47115 Modified: python/trunk/Makefile.pre.in python/trunk/Misc/NEWS Log: Fix SF bug #1513032, 'make install' failure on FreeBSD 5.3. No need to install lib-old, it's empty in 2.5. Modified: python/trunk/Makefile.pre.in ============================================================================== --- python/trunk/Makefile.pre.in (original) +++ python/trunk/Makefile.pre.in Tue Jun 27 06:12:58 2006 @@ -703,7 +703,7 @@ plat-mac/lib-scriptpackages/SystemEvents \ plat-mac/lib-scriptpackages/Terminal PLATMACPATH=:plat-mac:plat-mac/lib-scriptpackages -LIBSUBDIRS= lib-old lib-tk site-packages test test/output test/data \ +LIBSUBDIRS= lib-tk site-packages test test/output test/data \ test/decimaltestdata \ encodings compiler hotshot \ email email/mime email/test email/test/data \ Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Tue Jun 27 06:12:58 2006 @@ -28,6 +28,11 @@ - The wsgiref package is now installed properly on Unix. +Build +----- + +- Bug #1513032, 'make install' failed on FreeBSD 5.3 due to lib-old + trying to be installed even though it's empty. What's New in Python 2.5 beta 1? From python-checkins at python.org Tue Jun 27 06:23:08 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 27 Jun 2006 06:23:08 +0200 (CEST) Subject: [Python-checkins] r47116 - python/trunk/Misc/build.sh Message-ID: <20060627042308.A8C9A1E4004@bag.python.org> Author: neal.norwitz Date: Tue Jun 27 06:23:06 2006 New Revision: 47116 Modified: python/trunk/Misc/build.sh Log: Test unimportant change to verify buildbot does not try to build Modified: python/trunk/Misc/build.sh ============================================================================== --- python/trunk/Misc/build.sh (original) +++ python/trunk/Misc/build.sh Tue Jun 27 06:23:06 2006 @@ -220,4 +220,3 @@ rsync $RSYNC_OPTS html/* $REMOTE_SYSTEM:$REMOTE_DIR cd ../build rsync $RSYNC_OPTS index.html *.out $REMOTE_SYSTEM:$REMOTE_DIR/results/ - From python-checkins at python.org Tue Jun 27 06:26:33 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 27 Jun 2006 06:26:33 +0200 (CEST) Subject: [Python-checkins] r47117 - python/trunk/Misc/build.sh Message-ID: <20060627042633.A348C1E400C@bag.python.org> Author: neal.norwitz Date: Tue Jun 27 06:26:30 2006 New Revision: 47117 Modified: python/trunk/Misc/build.sh Log: Try again: test unimportant change to verify buildbot does not try to build Modified: python/trunk/Misc/build.sh ============================================================================== --- python/trunk/Misc/build.sh (original) +++ python/trunk/Misc/build.sh Tue Jun 27 06:26:30 2006 @@ -220,3 +220,4 @@ rsync $RSYNC_OPTS html/* $REMOTE_SYSTEM:$REMOTE_DIR cd ../build rsync $RSYNC_OPTS index.html *.out $REMOTE_SYSTEM:$REMOTE_DIR/results/ + From python-checkins at python.org Tue Jun 27 06:28:56 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 27 Jun 2006 06:28:56 +0200 (CEST) Subject: [Python-checkins] r47118 - in python/trunk: Makefile.pre.in Misc/NEWS Message-ID: <20060627042856.C01381E4004@bag.python.org> Author: neal.norwitz Date: Tue Jun 27 06:28:56 2006 New Revision: 47118 Modified: python/trunk/Makefile.pre.in python/trunk/Misc/NEWS Log: Verify buildbot picks up these changes (really needs testing after last change to Makefile.pre.in) Modified: python/trunk/Makefile.pre.in ============================================================================== --- python/trunk/Makefile.pre.in (original) +++ python/trunk/Makefile.pre.in Tue Jun 27 06:28:56 2006 @@ -701,7 +701,7 @@ plat-mac/lib-scriptpackages/Netscape \ plat-mac/lib-scriptpackages/StdSuites \ plat-mac/lib-scriptpackages/SystemEvents \ - plat-mac/lib-scriptpackages/Terminal + plat-mac/lib-scriptpackages/Terminal PLATMACPATH=:plat-mac:plat-mac/lib-scriptpackages LIBSUBDIRS= lib-tk site-packages test test/output test/data \ test/decimaltestdata \ Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Tue Jun 27 06:28:56 2006 @@ -31,7 +31,7 @@ Build ----- -- Bug #1513032, 'make install' failed on FreeBSD 5.3 due to lib-old +- Bug #1513032: 'make install' failed on FreeBSD 5.3 due to lib-old trying to be installed even though it's empty. From python-checkins at python.org Tue Jun 27 06:32:20 2006 From: python-checkins at python.org (neal.norwitz) Date: Tue, 27 Jun 2006 06:32:20 +0200 (CEST) Subject: [Python-checkins] r47119 - python/branches/release24-maint/Misc/build.sh Message-ID: <20060627043220.73DD41E4004@bag.python.org> Author: neal.norwitz Date: Tue Jun 27 06:32:20 2006 New Revision: 47119 Modified: python/branches/release24-maint/Misc/build.sh Log: Use a different install dir than 2.5. Otherwise running the 2.4 version at the same time as the 2.5 versions would break the 2.5 tests since the install dir was removed. Modified: python/branches/release24-maint/Misc/build.sh ============================================================================== --- python/branches/release24-maint/Misc/build.sh (original) +++ python/branches/release24-maint/Misc/build.sh Tue Jun 27 06:32:20 2006 @@ -51,7 +51,7 @@ REMOTE_SYSTEM="neal at dinsdale.python.org" REMOTE_DIR="/data/ftp.python.org/pub/docs.python.org/dev/2.4" RESULT_FILE="$DIR/build/index.html" -INSTALL_DIR="/tmp/python-test/local" +INSTALL_DIR="/tmp/python-test-2.4/local" RSYNC_OPTS="-aC -e ssh" REFLOG="build/reflog.txt.out" From nnorwitz at gmail.com Tue Jun 27 06:37:50 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 26 Jun 2006 21:37:50 -0700 Subject: [Python-checkins] Things to remember when adding *packages* to stdlib In-Reply-To: <449EC471.2000809@v.loewis.de> References: <449EC471.2000809@v.loewis.de> Message-ID: On 6/25/06, "Martin v. L?wis" wrote: > Neal Norwitz wrote: > > I believe this change is all that's necessary on the Unix side to > > install wsgiref. Can someone please update the Windows build files to > > ensure wsgiref is installed in b2? Don't forget to update the NEWS > > entry too. > > It's installed in b1 already. The msi generator picks up all .py files > in Lib automatically, except for those that have been explicitly > excluded (the plat-* ones). Ah cool. I was confusing it with extensions and mis-remembering. > > Maybe someone could come up with a heuristic to add to Misc/build.sh > > which we could test in there. > > I think "make install INSTALL=true|grep true" should print the names > of all .py files in Lib, except for the ones in plat-*. I modified the build.sh script to run the installed version. As long as some test references the new package, this should catch the problem. I sure hope we don't allow a new package without tests. :-) I also modified the buildbot config to ignore changes if all the files in a revision are under Demo, Doc, or Misc. This change (if I got it right and it works) should help reduce some builds that have little benefit. n From python-checkins at python.org Tue Jun 27 07:06:59 2006 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 27 Jun 2006 07:06:59 +0200 (CEST) Subject: [Python-checkins] r47120 - peps/trunk/pep-3103.txt Message-ID: <20060627050659.A63911E401D@bag.python.org> Author: guido.van.rossum Date: Tue Jun 27 07:06:58 2006 New Revision: 47120 Modified: peps/trunk/pep-3103.txt Log: Near-total rewrite of the section describing the different semantic schools, inspired by discussion on python-dev with Ka-Ping. Hopefully the different schools and their relative advantages and disadvantages are represented more correctly now. Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Tue Jun 27 07:06:58 2006 @@ -26,7 +26,7 @@ This PEP introduces canonical names for the many variants that have been discussed for different aspects of the syntax and semantics, such -as "alternative 2", "school II", "Option 3" and so on. Hopefully +as "alternative 1", "school II", "option 3" and so on. Hopefully these names will help the discussion. @@ -223,8 +223,8 @@ The `*` notation is similar to the use of prefix `*` already in use for variable-length parameter lists and for passing computed argument -lists, and often proposed for value-unpacking (e.g. "a, b, *c = X" as -an alternative to "(a, b), c = X[:2], X[2:]"). +lists, and often proposed for value-unpacking (e.g. ``a, b, *c = X`` as +an alternative to ``(a, b), c = X[:2], X[2:]``). Alternative D ------------- @@ -270,8 +270,8 @@ based on dict lookup is chosen as the semantics, large ranges could be inefficient (consider range(1, sys.maxint)). -All in all my preferences are (in descending preference) B, A, D', C -where D' is D without the third possibility. +All in all my preferences are (from most to least favorite) B, A, D', +C, where D' is D without the third possibility. Semantics @@ -283,66 +283,117 @@ If/Elif Chain vs. Dict-based Dispatch ------------------------------------- -There are two main schools of thought about the switch statement's -semantics. School I wants to define the switch statement in term of -an equivalent if/elif chain. School II prefers to think of it as a -dispatch on a precomputed dictionary. - -The difference is mainly important when either the switch expression -or one of the case expressions is not hashable; school I wants this to -be handled as it would be by an if/elif chain (i.e. hashability of the -expressions involved doesn't matter) while school II is willing to say -that the switch expression and all the case expressions must be -hashable if a switch is to be used; otherwise the user should have -written an if/elif chain. - -There's also a difference of opinion regarding the treatment of -duplicate cases (i.e. two or more cases with the same match -expression). School I wants to treat this the same is an if/elif -chain would treat it (i.e. the first match wins and the code for the -second match is silently unreachable); school II generally wants this -to be an error at the time the switch is frozen. - -There's also a school III which states that the definition of a switch -statement should be in terms of an equivalent if/elif chain, with the -exception that all the expressions must be hashable. - -School I believes that the if/elif chain is the only reasonable, -surprise-free way of defining switch semantics, and that optimizations -as suggested by PEP 275's Solution 1 are sufficient to make most -common uses fast. School I sees trouble in the approach of -pre-freezing a dispatch dictionary because it places a new and unusual -burden on programmers to understand exactly what kinds of case values -are allowed to be frozen and when the case values will be frozen, or -they might be surprised by the switch statement's behavior. - -School II sees trouble in trying to achieve semantics that match -those of an if/elif chain while optimizing the switch statement into -a hash lookup in a dispatch dictionary. In an if/elif chain, the -test "x == y" might well be comparing two unhashable values -(e.g. two lists); even "x == 1" could be comparing a user-defined -class instance that is not hashable but happens to define equality to -integers. Worse, the hash function might have a bug or a side effect; -if we generate code that believes the hash, a buggy hash might -generate an incorrect match, and if we generate code that catches -errors in the hash to fall back on an if/elif chain, we might hide -genuine bugs. In addition, school II sees little value in allowing -cases involving unhashable values; after all if the user expects such -values, they can just as easily write an if/elif chain. School II -also doesn't believe that it's fair to allow dead code due to -overlapping cases to occur unflagged, when the dict-based dispatch -implementation makes it so easy to trap this. - -School III admits the problems with making hash() optional, but still -believes that the true semantics should be defined by an if/elif chain -even if the implementation should be allowed to use dict-based -dispatch as an optimization. This means that duplicate cases must be -resolved by always choosing the first case, making the second case -undiagnosed dead code. +There are several main schools of thought about the switch statement's +semantics: + +- School I wants to define the switch statement in term of an + equivalent if/elif chain (possibly with some optimization thrown + in). + +- School II prefers to think of it as a dispatch on a precomputed + dict. There are different choices for when the precomputation + happens. + +- There's also school III, which agrees with School I that the + definition of a switch statement should be in terms of an equivalent + if/elif chain, but concedes to the optimization camp that all + expressions involved must be hashable. + +We need to further separate School I into School Ia and School Ib: + +- School Ia has a simple position: a switch statement is translated to + an equivalent if/elif chain, and that's that. It should not be + linked to optimization at all. That is also my main objection + against this school: without any hint of optimization, the switch + statement isn't attractive enough to warrant new syntax. + +- School Ib has a more complex position: it agrees with School II that + optimization is important, and is willing to concede the compiler + certain liberties to allow this. (For example, PEP 275 Solution 1.) + In particular, hash() of the switch and case expressions may or may + not be called (so it should be side-effect-free); and the case + expressions may not be evaluated each time as expected by the + if/elif chain behavior, so the case expressions should also be + side-effect free. My objection to this (elaborated below) is that + if either the hash() or the case expressions aren't + side-effect-free, optimized and unoptimized code may behave + differently. + +School II grew out of the realization that optimization of commonly +found cases isn't so easy, and that it's better to face this head on. +This will become clear below. + +The differences between School I (mostly School Ib) and School II are +threefold: + +- When optimizing using a dispatch dict, if either the switch + expression or the case expressions are unhashable (in which case + hash() raises an exception), School Ib requires catching the hash() + failure and falling back to an if/elif chain. School II simply lets + the exception happen. The problem with catching an exception in + hash() as required by School Ib, is that this may hide a genuine + bug. A possible way out is to only use a dispatch dict if all case + expressions are ints, strings or other built-ins with known good + hash behavior, and to only attempt to hash the switch expression if + it is also one of those types. Type objects should probably also be + supported here. This is the (only) problem that School III + addresses. + +- When optimizing using a dispatch dict, if the hash() function of any + expression involved returns an incorrect value, under school Ib, + optimized code will not behave the same as unoptimized code. This + is a well-known problem with optimization-related bugs, and waste + lots of developer time. Under School II, in this situation + incorrect results are produced at least consistently, which should + make debugging a bit easier. The way out proposed for the previous + bullet would also help here. + +- School Ib doesn't have a good optimization strategy if the case + expressions are named constants. The compiler cannot know their + values for sure, and it cannot know whether they are truly constant. + As a way out, it has been proposed to re-evaluate the expression + corresponding to the case once the dict has identified which case + should be taken, to verify that the value of the expression didn't + change. But strictly speaking, all the case expressions occurring + before that case would also have to be checked, in order to preserve + the true if/elif chain semantics, thereby completely killing the + optimization. Another proposed solution is to have callbacks + notifying the dispatch dict of changes in the value of variables or + attributes involved in the case expressions. But this is not likely + implementable in the general case, and would require many namespaces + to bear the burden of supporting such callbacks, which currently + don't exist at all. + +- Finally, there's a difference of opinion regarding the treatment of + duplicate cases (i.e. two or more cases with match expressions that + evaluates to the same value). School I wants to treat this the same + is an if/elif chain would treat it (i.e. the first match wins and + the code for the second match is silently unreachable); School II + wants this to be an error at the time the dispatch dict is frozen + (so dead code doesn't go undiagnosed). + +School I sees trouble in School II's approach of pre-freezing a +dispatch dict because it places a new and unusual burden on +programmers to understand exactly what kinds of case values are +allowed to be frozen and when the case values will be frozen, or they +might be surprised by the switch statement's behavior. + +School II doesn't believe that School Ia's unoptimized switch is worth +the effort, and it sees trouble in School Ib's proposal for +optimization, which can cause optimized and unoptimized code to behave +differently. + +In addition, school II sees little value in allowing cases involving +unhashable values; after all if the user expects such values, they can +just as easily write an if/elif chain. School II also doesn't believe +that it's right to allow dead code due to overlapping cases to occur +unflagged, when the dict-based dispatch implementation makes it so +easy to trap this. Personally, I'm in school II: I believe that the dict-based dispatch is the one true implementation for switch statements and that we -should face the limitiations and benefits up front. +should face the limitiations up front, so that we can reap maximal +benefits. When to Freeze the Dispatch Dict -------------------------------- From buildbot at python.org Tue Jun 27 07:47:51 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 27 Jun 2006 05:47:51 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060627054751.B7E081E4004@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/242 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 27 09:34:38 2006 From: python-checkins at python.org (vinay.sajip) Date: Tue, 27 Jun 2006 09:34:38 +0200 (CEST) Subject: [Python-checkins] r47121 - python/trunk/Lib/logging/handlers.py Message-ID: <20060627073438.2D1AD1E4019@bag.python.org> Author: vinay.sajip Date: Tue Jun 27 09:34:37 2006 New Revision: 47121 Modified: python/trunk/Lib/logging/handlers.py Log: Removed buggy exception handling in doRollover of rotating file handlers. Exceptions now propagate to caller. Modified: python/trunk/Lib/logging/handlers.py ============================================================================== --- python/trunk/Lib/logging/handlers.py (original) +++ python/trunk/Lib/logging/handlers.py Tue Jun 27 09:34:37 2006 @@ -128,12 +128,7 @@ dfn = self.baseFilename + ".1" if os.path.exists(dfn): os.remove(dfn) - try: - os.rename(self.baseFilename, dfn) - except (KeyboardInterrupt, SystemExit): - raise - except: - self.handleError(record) + os.rename(self.baseFilename, dfn) #print "%s -> %s" % (self.baseFilename, dfn) if self.encoding: self.stream = codecs.open(self.baseFilename, 'w', self.encoding) @@ -273,12 +268,7 @@ dfn = self.baseFilename + "." + time.strftime(self.suffix, timeTuple) if os.path.exists(dfn): os.remove(dfn) - try: - os.rename(self.baseFilename, dfn) - except (KeyboardInterrupt, SystemExit): - raise - except: - self.handleError(record) + os.rename(self.baseFilename, dfn) if self.backupCount > 0: # find the oldest log file and delete it s = glob.glob(self.baseFilename + ".20*") From python-checkins at python.org Tue Jun 27 09:37:46 2006 From: python-checkins at python.org (vinay.sajip) Date: Tue, 27 Jun 2006 09:37:46 +0200 (CEST) Subject: [Python-checkins] r47122 - python/branches/release24-maint/Lib/logging/handlers.py Message-ID: <20060627073746.AE28C1E4005@bag.python.org> Author: vinay.sajip Date: Tue Jun 27 09:37:45 2006 New Revision: 47122 Modified: python/branches/release24-maint/Lib/logging/handlers.py Log: Removed buggy exception handling in doRollover of rotating file handlers. Exceptions now propagate to caller. Modified: python/branches/release24-maint/Lib/logging/handlers.py ============================================================================== --- python/branches/release24-maint/Lib/logging/handlers.py (original) +++ python/branches/release24-maint/Lib/logging/handlers.py Tue Jun 27 09:37:45 2006 @@ -44,6 +44,8 @@ DEFAULT_SOAP_LOGGING_PORT = 9023 SYSLOG_UDP_PORT = 514 +_MIDNIGHT = 24 * 60 * 60 # number of seconds in a day + class BaseRotatingHandler(logging.FileHandler): """ Base class for handlers that rotate log files at a certain point. @@ -126,12 +128,7 @@ dfn = self.baseFilename + ".1" if os.path.exists(dfn): os.remove(dfn) - try: - os.rename(self.baseFilename, dfn) - except (KeyboardInterrupt, SystemExit): - raise - except: - self.handleError(record) + os.rename(self.baseFilename, dfn) #print "%s -> %s" % (self.baseFilename, dfn) if self.encoding: self.stream = codecs.open(self.baseFilename, 'w', self.encoding) @@ -217,12 +214,8 @@ currentMinute = t[4] currentSecond = t[5] # r is the number of seconds left between now and midnight - if (currentMinute == 0) and (currentSecond == 0): - r = (24 - currentHour) * 60 * 60 # number of hours in seconds - else: - r = (23 - currentHour) * 60 * 60 - r = r + (59 - currentMinute) * 60 # plus the number of minutes (in secs) - r = r + (60 - currentSecond) # plus the number of seconds + r = _MIDNIGHT - ((currentHour * 60 + currentMinute) * 60 + + currentSecond) self.rolloverAt = currentTime + r # If we are rolling over on a certain day, add in the number of days until # the next rollover, but offset by 1 since we just calculated the time @@ -275,12 +268,7 @@ dfn = self.baseFilename + "." + time.strftime(self.suffix, timeTuple) if os.path.exists(dfn): os.remove(dfn) - try: - os.rename(self.baseFilename, dfn) - except (KeyboardInterrupt, SystemExit): - raise - except: - self.handleError(record) + os.rename(self.baseFilename, dfn) if self.backupCount > 0: # find the oldest log file and delete it s = glob.glob(self.baseFilename + ".20*") From buildbot at python.org Tue Jun 27 10:17:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 27 Jun 2006 08:17:20 +0000 Subject: [Python-checkins] buildbot warnings in x86 OpenBSD 2.4 Message-ID: <20060627081720.50FD71E4014@bag.python.org> The Buildbot has detected a new failure of x86 OpenBSD 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%25202.4/builds/127 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: neal.norwitz,vinay.sajip Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 27 12:08:25 2006 From: python-checkins at python.org (ronald.oussoren) Date: Tue, 27 Jun 2006 12:08:25 +0200 (CEST) Subject: [Python-checkins] r47123 - python/trunk/Lib/distutils/sysconfig.py python/trunk/Lib/distutils/unixccompiler.py Message-ID: <20060627100825.96B111E4004@bag.python.org> Author: ronald.oussoren Date: Tue Jun 27 12:08:25 2006 New Revision: 47123 Modified: python/trunk/Lib/distutils/sysconfig.py python/trunk/Lib/distutils/unixccompiler.py Log: MacOSX: fix rather dumb buglet that made it impossible to create extensions on OSX 10.3 when using a binary distribution build on 10.4. Modified: python/trunk/Lib/distutils/sysconfig.py ============================================================================== --- python/trunk/Lib/distutils/sysconfig.py (original) +++ python/trunk/Lib/distutils/sysconfig.py Tue Jun 27 12:08:25 2006 @@ -512,7 +512,7 @@ for key in ('LDFLAGS', 'BASECFLAGS'): flags = _config_vars[key] flags = re.sub('-arch\s+\w+\s', ' ', flags) - flags = re.sub('-isysroot [^ \t]* ', ' ', flags) + flags = re.sub('-isysroot [^ \t]*', ' ', flags) _config_vars[key] = flags if args: Modified: python/trunk/Lib/distutils/unixccompiler.py ============================================================================== --- python/trunk/Lib/distutils/unixccompiler.py (original) +++ python/trunk/Lib/distutils/unixccompiler.py Tue Jun 27 12:08:25 2006 @@ -78,7 +78,7 @@ try: index = compiler_so.index('-isysroot') # Strip this argument and the next one: - del compiler_so[index:index+1] + del compiler_so[index:index+2] except ValueError: pass From python-checkins at python.org Tue Jun 27 12:52:50 2006 From: python-checkins at python.org (matt.fleming) Date: Tue, 27 Jun 2006 12:52:50 +0200 (CEST) Subject: [Python-checkins] r47124 - in sandbox/trunk/pdb: README.txt mpdb.py test/test_mpdb.py Message-ID: <20060627105250.7C6811E4004@bag.python.org> Author: matt.fleming Date: Tue Jun 27 12:52:49 2006 New Revision: 47124 Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/test/test_mpdb.py Log: Fix top-level pdbserver and target routines. Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Tue Jun 27 12:52:49 2006 @@ -42,5 +42,9 @@ - Debugging outside a process - Debugging remotely - Debugging threads +* Provide a command to distinguish between 'server' and 'client', because + at the moment 'info target' only tells the user whether they are local + or remote, and not whether they are the local/remote server or + local/remote client. Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Tue Jun 27 12:52:49 2006 @@ -46,6 +46,8 @@ sys.stdin and sys.stdout are used. """ pydb.Pdb.__init__(self, completekey, stdin, stdout) + self.orig_stdout = self.stdout + self.orig_stdin = self.stdin self.prompt = '(MPdb)' self.target = 'local' # local connections by default self.connection = None @@ -81,6 +83,18 @@ """ All commands in 'line' are sent across this object's connection instance variable. """ + # This is the simplest way I could think of to do this without + # breaking any of the inherited code from pydb/pdb. If we're a + # remote client, always call 'rquit' (remote quit) when connected to + # a pdbserver. This executes extra code to allow the client and server + # to quit cleanly. + if 'quit'.startswith(line): + line = 'rquit' + self.connection.write(line) + # Reset the onecmd method + self.onecmd = lambda x: pydb.Pdb.onecmd(self, x) + self.do_rquit(None) + return self.connection.write(line) ret = self.connection.readline() # The output from the command that we've just sent to the server @@ -245,6 +259,8 @@ self.local_prompt = self.prompt self.prompt = "" line = self.connection.readline() + while '(MPdb)' not in line: + line = self.connection.readline() self.msg_nocr(line) self.onecmd = self.remote_onecmd self.target = 'remote' @@ -315,6 +331,7 @@ return self.connection = eval(target+'()') try: + self.msg('Listening on: %s' % comm) self.connection.connect(comm) except ConnectionFailed, err: self.errmsg("Failed to connect to %s: (%s)" % (comm, err)) @@ -323,6 +340,24 @@ self._rebind_input(self.connection) self._rebind_output(self.connection) + def do_rquit(self, arg): + """ Quit a remote debugging session. The program being executed +is aborted. +""" + if self.target == 'local': + self.errmsg('Connected locally, cannot remotely quit') + return + self._rebind_output(self.orig_stdout) + self._rebind_input(self.orig_stdin) + if self.connection != None: + self.connection.disconnect() + if hasattr(self, 'local_prompt'): + self.prompt = self.local_prompt + self.msg('Exiting remote debugging...') + self.target = 'local' + self.do_quit(None) + + def do_thread(self, arg): """Use this command to switch between threads. The new thread ID must be currently known. @@ -354,19 +389,25 @@ t_tracer = self.tracers[int(args[1])-1] func = args[2] cmd = eval('t.' + func) - result = cmd(args[3:]) + try: + result = cmd(args[3:]) + except AttributeError: + self.errmsg('No such thread subcommand') + return -def pdbserver(addr): +def pdbserver(addr, args): """ This method sets up a pdbserver debugger that allows debuggers to connect to 'address' using 'protocol'. The argument 'filename' is the name of the file that is being debugged. """ m = MPdb() - position = addr.rfind(' ') - mainpyfile = addr[position+1:] + mainpyfile = args[0] m.mainpyfile = mainpyfile m.do_pdbserver(addr) - m._runscript(mainpyfile) + while True: + m._runscript(mainpyfile) + if m._user_requested_quit: + break sys.exit() def target(addr): @@ -375,6 +416,7 @@ tcp = 'tcp mydomainname.com:9876' serial = '/dev/ttyC0' """ + print addr m = MPdb() # Look Ma, no script! m.do_target(addr) @@ -390,7 +432,7 @@ if opts.target: target(opts.target) elif opts.pdbserver: - pdbserver(opts.pdbserver) + pdbserver(opts.pdbserver, args) else: if not opts.scriptname: if not args: Modified: sandbox/trunk/pdb/test/test_mpdb.py ============================================================================== --- sandbox/trunk/pdb/test/test_mpdb.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Tue Jun 27 12:52:49 2006 @@ -61,7 +61,7 @@ client = MPdbTest() thread.start_new_thread(connect_to_target, (client,)) - self.server1 = MPdb() + self.server1 = MPdbTest() self.server1.do_pdbserver('tcp '+__addr__) self.server1.connection.disconnect() From buildbot at python.org Tue Jun 27 12:59:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 27 Jun 2006 10:59:30 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20060627105930.3F0711E4004@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/828 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 27 13:52:50 2006 From: python-checkins at python.org (tim.peters) Date: Tue, 27 Jun 2006 13:52:50 +0200 (CEST) Subject: [Python-checkins] r47125 - in python/trunk/Lib: compiler/transformer.py test/test_mailbox.py test/test_warnings.py Message-ID: <20060627115250.B5FF61E4004@bag.python.org> Author: tim.peters Date: Tue Jun 27 13:52:49 2006 New Revision: 47125 Modified: python/trunk/Lib/compiler/transformer.py python/trunk/Lib/test/test_mailbox.py python/trunk/Lib/test/test_warnings.py Log: Whitespace normalization. Modified: python/trunk/Lib/compiler/transformer.py ============================================================================== --- python/trunk/Lib/compiler/transformer.py (original) +++ python/trunk/Lib/compiler/transformer.py Tue Jun 27 13:52:49 2006 @@ -922,7 +922,7 @@ return TryFinally(self.com_node(nodelist[2]), self.com_node(nodelist[5]), lineno=nodelist[0][2]) - + #tryexcept: [TryNode, [except_clauses], elseNode)] clauses = [] elseNode = None Modified: python/trunk/Lib/test/test_mailbox.py ============================================================================== --- python/trunk/Lib/test/test_mailbox.py (original) +++ python/trunk/Lib/test/test_mailbox.py Tue Jun 27 13:52:49 2006 @@ -738,12 +738,12 @@ time.sleep(0.5) self.assertRaises(mailbox.ExternalClashError, self._box.lock) - + # Wait for child to exit. Locking should now succeed. exited_pid, status = os.waitpid(pid, 0) self._box.lock() self._box.unlock() - + class TestMbox(_TestMboxMMDF): Modified: python/trunk/Lib/test/test_warnings.py ============================================================================== --- python/trunk/Lib/test/test_warnings.py (original) +++ python/trunk/Lib/test/test_warnings.py Tue Jun 27 13:52:49 2006 @@ -92,7 +92,7 @@ warnings._setoption, 'ignore:2::4:-5') warnings._setoption('error::Warning::0') self.assertRaises(UserWarning, warnings.warn, 'convert to error') - + def test_main(verbose=None): # Obscure hack so that this test passes after reloads or repeated calls From python-checkins at python.org Tue Jun 27 14:27:50 2006 From: python-checkins at python.org (nick.coghlan) Date: Tue, 27 Jun 2006 14:27:50 +0200 (CEST) Subject: [Python-checkins] r47126 - peps/trunk/pep-0328.txt Message-ID: <20060627122750.E60BE1E4004@bag.python.org> Author: nick.coghlan Date: Tue Jun 27 14:27:50 2006 New Revision: 47126 Modified: peps/trunk/pep-0328.txt Log: Add a note about the dependency between relative imports and the value in '__name__' Modified: peps/trunk/pep-0328.txt ============================================================================== --- peps/trunk/pep-0328.txt (original) +++ peps/trunk/pep-0328.txt Tue Jun 27 14:27:50 2006 @@ -267,6 +267,15 @@ is not usable in an expression. +Relative Imports and __name__ +=============================== + +Relative imports use a module's __name__ attribute to determine that +module's position in the package hierarchy. If the module's name does +not contain any package information (e.g. it is set to '__main__') +then relative imports are resolved as if the module were a top level +module, regardless of where the module is actually located on the file +system. References From python-checkins at python.org Tue Jun 27 14:30:11 2006 From: python-checkins at python.org (matt.fleming) Date: Tue, 27 Jun 2006 14:30:11 +0200 (CEST) Subject: [Python-checkins] r47127 - sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Message-ID: <20060627123011.39A891E4004@bag.python.org> Author: matt.fleming Date: Tue Jun 27 14:30:10 2006 New Revision: 47127 Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Log: Fix for sending numbers across a remote connection. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Tue Jun 27 14:30:10 2006 @@ -136,7 +136,7 @@ line = self.input.recv(bufsize) except socket.error, e: raise ReadError, e[1] - if line[-1].isalpha(): line += '\n' + if line[-1] != '\n': line += '\n' return line def write(self, msg): Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Tue Jun 27 14:30:10 2006 @@ -57,6 +57,7 @@ self.threads = [] self.current_thread = threading.currentThread() self._info_cmds.append('target') + self._info_cmds.append('thread') def _rebind_input(self, new_input): """ This method rebinds the debugger's input to the object specified @@ -151,7 +152,6 @@ self.msg('Thread debugging on') self.debug_thread = True return - def thread_trace_dispatch(self, frame, event, arg): """ Create an MTracer object so trace the thread. """ @@ -339,7 +339,7 @@ self.target = 'remote' self._rebind_input(self.connection) self._rebind_output(self.connection) - + def do_rquit(self, arg): """ Quit a remote debugging session. The program being executed is aborted. From python-checkins at python.org Tue Jun 27 14:53:53 2006 From: python-checkins at python.org (ronald.oussoren) Date: Tue, 27 Jun 2006 14:53:53 +0200 (CEST) Subject: [Python-checkins] r47128 - in python/trunk: Mac/BuildScript/build-installer.py setup.py Message-ID: <20060627125353.298681E4004@bag.python.org> Author: ronald.oussoren Date: Tue Jun 27 14:53:52 2006 New Revision: 47128 Modified: python/trunk/Mac/BuildScript/build-installer.py python/trunk/setup.py Log: Use staticly build copies of zlib and bzip2 to build the OSX installer, that way the resulting binaries have a better change of running on 10.3. This patch also updates the search logic for sleepycat db3/4, without this patch you cannot use a sleepycat build with a non-standard prefix; with this you can (at least on OSX) if you add the prefix to CPPFLAGS/LDFLAGS at configure-time. This change is needed to build the binary installer for OSX. Modified: python/trunk/Mac/BuildScript/build-installer.py ============================================================================== --- python/trunk/Mac/BuildScript/build-installer.py (original) +++ python/trunk/Mac/BuildScript/build-installer.py Tue Jun 27 14:53:52 2006 @@ -67,6 +67,8 @@ SDKPATH="/Developer/SDKs/MacOSX10.4u.sdk" #SDKPATH="/" +ARCHLIST=('i386', 'ppc',) + # Source directory (asume we're in Mac/BuildScript) SRCDIR=os.path.dirname( os.path.dirname( @@ -91,6 +93,26 @@ # batteries included python. LIBRARY_RECIPES=[ dict( + name="Bzip2 1.0.3", + url="http://www.bzip.org/1.0.3/bzip2-1.0.3.tar.gz", + configure=None, + install='make install PREFIX=%s/usr/local/ CFLAGS="-arch %s -isysroot %s"'%( + shellQuote(os.path.join(WORKDIR, 'libraries')), + ' -arch '.join(ARCHLIST), + SDKPATH, + ), + ), + dict( + name="ZLib 1.2.3", + url="http://www.gzip.org/zlib/zlib-1.2.3.tar.gz", + configure=None, + install='make install prefix=%s/usr/local/ CFLAGS="-arch %s -isysroot %s"'%( + shellQuote(os.path.join(WORKDIR, 'libraries')), + ' -arch '.join(ARCHLIST), + SDKPATH, + ), + ), + dict( # Note that GNU readline is GPL'd software name="GNU Readline 5.1.4", url="http://ftp.gnu.org/pub/gnu/readline/readline-5.1.tar.gz" , @@ -486,48 +508,49 @@ runCommand('patch -p%s < %s'%(recipe.get('patchlevel', 1), shellQuote(fn),)) - configure_args = [ - "--prefix=/usr/local", - "--enable-static", - "--disable-shared", - #"CPP=gcc -arch %s -E"%(' -arch '.join(archList,),), - ] - - if 'configure_pre' in recipe: - args = list(recipe['configure_pre']) - if '--disable-static' in args: - configure_args.remove('--enable-static') - if '--enable-shared' in args: - configure_args.remove('--disable-shared') - configure_args.extend(args) - - if recipe.get('useLDFlags', 1): - configure_args.extend([ - "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( - ' -arch '.join(archList), - shellQuote(SDKPATH)[1:-1], - shellQuote(basedir)[1:-1],), - "LDFLAGS=-syslibroot,%s -L%s/usr/local/lib -arch %s"%( - shellQuote(SDKPATH)[1:-1], - shellQuote(basedir)[1:-1], - ' -arch '.join(archList)), - ]) - else: - configure_args.extend([ - "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( - ' -arch '.join(archList), + if configure is not None: + configure_args = [ + "--prefix=/usr/local", + "--enable-static", + "--disable-shared", + #"CPP=gcc -arch %s -E"%(' -arch '.join(archList,),), + ] + + if 'configure_pre' in recipe: + args = list(recipe['configure_pre']) + if '--disable-static' in args: + configure_args.remove('--enable-static') + if '--enable-shared' in args: + configure_args.remove('--disable-shared') + configure_args.extend(args) + + if recipe.get('useLDFlags', 1): + configure_args.extend([ + "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( + ' -arch '.join(archList), + shellQuote(SDKPATH)[1:-1], + shellQuote(basedir)[1:-1],), + "LDFLAGS=-syslibroot,%s -L%s/usr/local/lib -arch %s"%( shellQuote(SDKPATH)[1:-1], - shellQuote(basedir)[1:-1],), - ]) + shellQuote(basedir)[1:-1], + ' -arch '.join(archList)), + ]) + else: + configure_args.extend([ + "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( + ' -arch '.join(archList), + shellQuote(SDKPATH)[1:-1], + shellQuote(basedir)[1:-1],), + ]) - if 'configure_post' in recipe: - configure_args = configure_args = list(recipe['configure_post']) + if 'configure_post' in recipe: + configure_args = configure_args = list(recipe['configure_post']) - configure_args.insert(0, configure) - configure_args = [ shellQuote(a) for a in configure_args ] + configure_args.insert(0, configure) + configure_args = [ shellQuote(a) for a in configure_args ] - print "Running configure for %s"%(name,) - runCommand(' '.join(configure_args) + ' 2>&1') + print "Running configure for %s"%(name,) + runCommand(' '.join(configure_args) + ' 2>&1') print "Running install for %s"%(name,) runCommand('{ ' + install + ' ;} 2>&1') @@ -550,7 +573,7 @@ os.makedirs(os.path.join(universal, 'usr', 'local', 'include')) for recipe in LIBRARY_RECIPES: - buildRecipe(recipe, universal, ('i386', 'ppc',)) + buildRecipe(recipe, universal, ARCHLIST) @@ -997,7 +1020,7 @@ buildPythonDocs() fn = os.path.join(WORKDIR, "_root", "Applications", "MacPython %s"%(getVersion(),), "Update Shell Profile.command") - shutil.copy("scripts/postflight.patch-profile", fn) + patchFile("scripts/postflight.patch-profile", fn) os.chmod(fn, 0755) folder = os.path.join(WORKDIR, "_root", "Applications", "MacPython %s"%( Modified: python/trunk/setup.py ============================================================================== --- python/trunk/setup.py (original) +++ python/trunk/setup.py Tue Jun 27 14:53:52 2006 @@ -638,6 +638,24 @@ db_inc_paths.append('/pkg/db-3.%d/include' % x) db_inc_paths.append('/opt/db-3.%d/include' % x) + # Add some common subdirectories for Sleepycat DB to the list, + # based on the standard include directories. This way DB3/4 gets + # picked up when it is installed in a non-standard prefix and + # the user has added that prefix into inc_dirs. + std_variants = [] + for dn in inc_dirs: + std_variants.append(os.path.join(dn, 'db3')) + std_variants.append(os.path.join(dn, 'db4')) + for x in (0,1,2,3,4): + std_variants.append(os.path.join(dn, "db4%d"%x)) + std_variants.append(os.path.join(dn, "db4.%d"%x)) + for x in (2,3): + std_variants.append(os.path.join(dn, "db3%d"%x)) + std_variants.append(os.path.join(dn, "db3.%d"%x)) + + db_inc_paths = std_variants + db_inc_paths + + db_ver_inc_map = {} class db_found(Exception): pass @@ -940,13 +958,23 @@ break if version >= version_req: if (self.compiler.find_library_file(lib_dirs, 'z')): + if sys.platform == "darwin": + zlib_extra_link_args = ('-Wl,-search_paths_first',) + else: + zlib_extra_link_args = () exts.append( Extension('zlib', ['zlibmodule.c'], - libraries = ['z']) ) + libraries = ['z'], + extra_link_args = zlib_extra_link_args)) # Gustavo Niemeyer's bz2 module. if (self.compiler.find_library_file(lib_dirs, 'bz2')): + if sys.platform == "darwin": + bz2_extra_link_args = ('-Wl,-search_paths_first',) + else: + bz2_extra_link_args = () exts.append( Extension('bz2', ['bz2module.c'], - libraries = ['bz2']) ) + libraries = ['bz2'], + extra_link_args = bz2_extra_link_args) ) # Interface to the Expat XML parser # From python-checkins at python.org Tue Jun 27 17:45:32 2006 From: python-checkins at python.org (ronald.oussoren) Date: Tue, 27 Jun 2006 17:45:32 +0200 (CEST) Subject: [Python-checkins] r47131 - python/trunk/Makefile.pre.in Message-ID: <20060627154532.A037B1E4021@bag.python.org> Author: ronald.oussoren Date: Tue Jun 27 17:45:32 2006 New Revision: 47131 Modified: python/trunk/Makefile.pre.in Log: macosx: Install a libpython2.5.a inside the framework as a symlink to the actual dylib at the root of the framework, that way tools that expect a unix-like install (python-config, but more importantly external products like mod_python) work correctly. Modified: python/trunk/Makefile.pre.in ============================================================================== --- python/trunk/Makefile.pre.in (original) +++ python/trunk/Makefile.pre.in Tue Jun 27 17:45:32 2006 @@ -931,7 +931,10 @@ $(INSTALL_SHARED) $(LDLIBRARY) $(DESTDIR)$(PYTHONFRAMEWORKPREFIX)/$(LDLIBRARY) # This installs Mac/Lib into the framework +# Install a number of symlinks to keep software that expects a normal unix +# install (which includes python-config) happy. frameworkinstallmaclib: + ln -s "../../../Python" "$(DESTDIR)$(prefix)/lib/python$(VERSION)/config/libpython$(VERSION).a" cd Mac && $(MAKE) installmacsubtree DESTDIR="$(DESTDIR)" # This installs the IDE, the Launcher and other apps into /Applications From nnorwitz at gmail.com Tue Jun 27 18:00:24 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 27 Jun 2006 09:00:24 -0700 Subject: [Python-checkins] r47121 - python/trunk/Lib/logging/handlers.py In-Reply-To: <20060627073438.2D1AD1E4019@bag.python.org> References: <20060627073438.2D1AD1E4019@bag.python.org> Message-ID: Can we get a NEWS entry? -- n On 6/27/06, vinay.sajip wrote: > Author: vinay.sajip > Date: Tue Jun 27 09:34:37 2006 > New Revision: 47121 > > Modified: > python/trunk/Lib/logging/handlers.py > Log: > Removed buggy exception handling in doRollover of rotating file handlers. Exceptions now propagate to caller. > > Modified: python/trunk/Lib/logging/handlers.py > ============================================================================== > --- python/trunk/Lib/logging/handlers.py (original) > +++ python/trunk/Lib/logging/handlers.py Tue Jun 27 09:34:37 2006 > @@ -128,12 +128,7 @@ > dfn = self.baseFilename + ".1" > if os.path.exists(dfn): > os.remove(dfn) > - try: > - os.rename(self.baseFilename, dfn) > - except (KeyboardInterrupt, SystemExit): > - raise > - except: > - self.handleError(record) > + os.rename(self.baseFilename, dfn) > #print "%s -> %s" % (self.baseFilename, dfn) > if self.encoding: > self.stream = codecs.open(self.baseFilename, 'w', self.encoding) > @@ -273,12 +268,7 @@ > dfn = self.baseFilename + "." + time.strftime(self.suffix, timeTuple) > if os.path.exists(dfn): > os.remove(dfn) > - try: > - os.rename(self.baseFilename, dfn) > - except (KeyboardInterrupt, SystemExit): > - raise > - except: > - self.handleError(record) > + os.rename(self.baseFilename, dfn) > if self.backupCount > 0: > # find the oldest log file and delete it > s = glob.glob(self.baseFilename + ".20*") > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From buildbot at python.org Tue Jun 27 18:30:21 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 27 Jun 2006 16:30:21 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060627163021.D98161E400C@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/247 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Tue Jun 27 19:30:51 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 27 Jun 2006 17:30:51 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060627173051.7DD091E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/421 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Tue Jun 27 20:01:47 2006 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 27 Jun 2006 20:01:47 +0200 (CEST) Subject: [Python-checkins] r47132 - peps/trunk/pep-3103.txt Message-ID: <20060627180147.9A23F1E4004@bag.python.org> Author: guido.van.rossum Date: Tue Jun 27 20:01:47 2006 New Revision: 47132 Modified: peps/trunk/pep-3103.txt Log: Standardize case of "school" -- there's no reason to capitalize this everywhere. Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Tue Jun 27 20:01:47 2006 @@ -294,12 +294,12 @@ dict. There are different choices for when the precomputation happens. -- There's also school III, which agrees with School I that the +- There's also school III, which agrees with school I that the definition of a switch statement should be in terms of an equivalent if/elif chain, but concedes to the optimization camp that all expressions involved must be hashable. -We need to further separate School I into School Ia and School Ib: +We need to further separate school I into school Ia and school Ib: - School Ia has a simple position: a switch statement is translated to an equivalent if/elif chain, and that's that. It should not be @@ -307,7 +307,7 @@ against this school: without any hint of optimization, the switch statement isn't attractive enough to warrant new syntax. -- School Ib has a more complex position: it agrees with School II that +- School Ib has a more complex position: it agrees with school II that optimization is important, and is willing to concede the compiler certain liberties to allow this. (For example, PEP 275 Solution 1.) In particular, hash() of the switch and case expressions may or may @@ -323,27 +323,27 @@ found cases isn't so easy, and that it's better to face this head on. This will become clear below. -The differences between School I (mostly School Ib) and School II are +The differences between school I (mostly school Ib) and school II are threefold: - When optimizing using a dispatch dict, if either the switch expression or the case expressions are unhashable (in which case - hash() raises an exception), School Ib requires catching the hash() + hash() raises an exception), school Ib requires catching the hash() failure and falling back to an if/elif chain. School II simply lets the exception happen. The problem with catching an exception in - hash() as required by School Ib, is that this may hide a genuine + hash() as required by school Ib, is that this may hide a genuine bug. A possible way out is to only use a dispatch dict if all case expressions are ints, strings or other built-ins with known good hash behavior, and to only attempt to hash the switch expression if it is also one of those types. Type objects should probably also be - supported here. This is the (only) problem that School III + supported here. This is the (only) problem that school III addresses. - When optimizing using a dispatch dict, if the hash() function of any expression involved returns an incorrect value, under school Ib, optimized code will not behave the same as unoptimized code. This is a well-known problem with optimization-related bugs, and waste - lots of developer time. Under School II, in this situation + lots of developer time. Under school II, in this situation incorrect results are produced at least consistently, which should make debugging a bit easier. The way out proposed for the previous bullet would also help here. @@ -368,18 +368,18 @@ duplicate cases (i.e. two or more cases with match expressions that evaluates to the same value). School I wants to treat this the same is an if/elif chain would treat it (i.e. the first match wins and - the code for the second match is silently unreachable); School II + the code for the second match is silently unreachable); school II wants this to be an error at the time the dispatch dict is frozen (so dead code doesn't go undiagnosed). -School I sees trouble in School II's approach of pre-freezing a +School I sees trouble in school II's approach of pre-freezing a dispatch dict because it places a new and unusual burden on programmers to understand exactly what kinds of case values are allowed to be frozen and when the case values will be frozen, or they might be surprised by the switch statement's behavior. -School II doesn't believe that School Ia's unoptimized switch is worth -the effort, and it sees trouble in School Ib's proposal for +School II doesn't believe that school Ia's unoptimized switch is worth +the effort, and it sees trouble in school Ib's proposal for optimization, which can cause optimized and unoptimized code to behave differently. From buildbot at python.org Tue Jun 27 20:06:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 27 Jun 2006 18:06:02 +0000 Subject: [Python-checkins] buildbot warnings in sparc Ubuntu dapper trunk Message-ID: <20060627180602.378111E4004@bag.python.org> The Buildbot has detected a new failure of sparc Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520Ubuntu%2520dapper%2520trunk/builds/458 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren,tim.peters Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Tue Jun 27 20:57:46 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Tue, 27 Jun 2006 20:57:46 +0200 (CEST) Subject: [Python-checkins] r47133 - python/branches/hoxworth-stdlib_logging-soc/Lib Message-ID: <20060627185746.D47691E4005@bag.python.org> Author: jackilyn.hoxworth Date: Tue Jun 27 20:57:46 2006 New Revision: 47133 Removed: python/branches/hoxworth-stdlib_logging-soc/Lib/ Log: Removed file/folder From python-checkins at python.org Tue Jun 27 21:02:40 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Tue, 27 Jun 2006 21:02:40 +0200 (CEST) Subject: [Python-checkins] r47134 - python/branches/hoxworth-stdlib_logging-soc/asyncore.py python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Message-ID: <20060627190240.3AB491E4004@bag.python.org> Author: jackilyn.hoxworth Date: Tue Jun 27 21:02:39 2006 New Revision: 47134 Removed: python/branches/hoxworth-stdlib_logging-soc/asyncore.py python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Log: Removed file/folder Deleted: /python/branches/hoxworth-stdlib_logging-soc/asyncore.py ============================================================================== --- /python/branches/hoxworth-stdlib_logging-soc/asyncore.py Tue Jun 27 21:02:39 2006 +++ (empty file) @@ -1,596 +0,0 @@ -# -*- Mode: Python -*- -# Id: asyncore.py,v 2.51 2000/09/07 22:29:26 rushing Exp -# Author: Sam Rushing - -# ====================================================================== -# Copyright 1996 by Sam Rushing -# -# All Rights Reserved -# -# Permission to use, copy, modify, and distribute this software and -# its documentation for any purpose and without fee is hereby -# granted, provided that the above copyright notice appear in all -# copies and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of Sam -# Rushing not be used in advertising or publicity pertaining to -# distribution of the software without specific, written prior -# permission. -# -# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, -# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN -# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR -# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS -# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, -# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN -# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. -# ====================================================================== - -"""Basic infrastructure for asynchronous socket service clients and servers. - -There are only two ways to have a program on a single processor do "more -than one thing at a time". Multi-threaded programming is the simplest and -most popular way to do it, but there is another very different technique, -that lets you have nearly all the advantages of multi-threading, without -actually using multiple threads. it's really only practical if your program -is largely I/O bound. If your program is CPU bound, then pre-emptive -scheduled threads are probably what you really need. Network servers are -rarely CPU-bound, however. - -If your operating system supports the select() system call in its I/O -library (and nearly all do), then you can use it to juggle multiple -communication channels at once; doing other work while your I/O is taking -place in the "background." Although this strategy can seem strange and -complex, especially at first, it is in many ways easier to understand and -control than multi-threaded programming. The module documented here solves -many of the difficult problems for you, making the task of building -sophisticated high-performance network servers and clients a snap. -""" - -import select -import socket -import sys -import time - -import os -from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, \ - ENOTCONN, ESHUTDOWN, EINTR, EISCONN, errorcode - -try: - socket_map -except NameError: - socket_map = {} - -class ExitNow(Exception): - pass - -def read(obj): - try: - obj.handle_read_event() - except ExitNow: - raise - except: - obj.handle_error() - -def write(obj): - try: - obj.handle_write_event() - except ExitNow: - raise - except: - obj.handle_error() - -def _exception (obj): - try: - obj.handle_expt_event() - except ExitNow: - raise - except: - obj.handle_error() - -def readwrite(obj, flags): - try: - if flags & (select.POLLIN | select.POLLPRI): - obj.handle_read_event() - if flags & select.POLLOUT: - obj.handle_write_event() - if flags & (select.POLLERR | select.POLLHUP | select.POLLNVAL): - obj.handle_expt_event() - except ExitNow: - raise - except: - obj.handle_error() - -def poll(timeout=0.0, map=None): - if map is None: - map = socket_map - if map: - r = []; w = []; e = [] - for fd, obj in map.items(): - is_r = obj.readable() - is_w = obj.writable() - if is_r: - r.append(fd) - if is_w: - w.append(fd) - if is_r or is_w: - e.append(fd) - if [] == r == w == e: - time.sleep(timeout) - else: - try: - r, w, e = select.select(r, w, e, timeout) - except select.error, err: - if err[0] != EINTR: - raise - else: - return - - for fd in r: - obj = map.get(fd) - if obj is None: - continue - read(obj) - - for fd in w: - obj = map.get(fd) - if obj is None: - continue - write(obj) - - for fd in e: - obj = map.get(fd) - if obj is None: - continue - _exception(obj) - -def poll2(timeout=0.0, map=None): - # Use the poll() support added to the select module in Python 2.0 - if map is None: - map = socket_map - if timeout is not None: - # timeout is in milliseconds - timeout = int(timeout*1000) - pollster = select.poll() - if map: - for fd, obj in map.items(): - flags = 0 - if obj.readable(): - flags |= select.POLLIN | select.POLLPRI - if obj.writable(): - flags |= select.POLLOUT - if flags: - # Only check for exceptions if object was either readable - # or writable. - flags |= select.POLLERR | select.POLLHUP | select.POLLNVAL - pollster.register(fd, flags) - try: - r = pollster.poll(timeout) - except select.error, err: - if err[0] != EINTR: - raise - r = [] - for fd, flags in r: - obj = map.get(fd) - if obj is None: - continue - readwrite(obj, flags) - -poll3 = poll2 # Alias for backward compatibility - -def loop(timeout=30.0, use_poll=False, map=None, count=None): - if map is None: - map = socket_map - - if use_poll and hasattr(select, 'poll'): - poll_fun = poll2 - else: - poll_fun = poll - - if count is None: - while map: - poll_fun(timeout, map) - - else: - while map and count > 0: - poll_fun(timeout, map) - count = count - 1 - -class dispatcher: - - debug = False - connected = False - accepting = False - closing = False - addr = None - - def __init__(self, sock=None, map=None): - if map is None: - self._map = socket_map - else: - self._map = map - - if sock: - self.set_socket(sock, map) - # I think it should inherit this anyway - self.socket.setblocking(0) - self.connected = True - # XXX Does the constructor require that the socket passed - # be connected? - try: - self.addr = sock.getpeername() - except socket.error: - # The addr isn't crucial - pass - else: - self.socket = None - - def __repr__(self): - status = [self.__class__.__module__+"."+self.__class__.__name__] - if self.accepting and self.addr: - status.append('listening') - elif self.connected: - status.append('connected') - if self.addr is not None: - try: - status.append('%s:%d' % self.addr) - except TypeError: - status.append(repr(self.addr)) - return '<%s at %#x>' % (' '.join(status), id(self)) - - def add_channel(self, map=None): - #self.log_info('adding channel %s' % self) - if map is None: - map = self._map - map[self._fileno] = self - - def del_channel(self, map=None): - fd = self._fileno - if map is None: - map = self._map - if map.has_key(fd): - #self.log_info('closing channel %d:%s' % (fd, self)) - del map[fd] - self._fileno = None - - def create_socket(self, family, type): - self.family_and_type = family, type - self.socket = socket.socket(family, type) - self.socket.setblocking(0) - self._fileno = self.socket.fileno() - self.add_channel() - - def set_socket(self, sock, map=None): - self.socket = sock -## self.__dict__['socket'] = sock - self._fileno = sock.fileno() - self.add_channel(map) - - def set_reuse_addr(self): - # try to re-use a server port if possible - try: - self.socket.setsockopt( - socket.SOL_SOCKET, socket.SO_REUSEADDR, - self.socket.getsockopt(socket.SOL_SOCKET, - socket.SO_REUSEADDR) | 1 - ) - except socket.error: - pass - - # ================================================== - # predicates for select() - # these are used as filters for the lists of sockets - # to pass to select(). - # ================================================== - - def readable(self): - return True - - def writable(self): - return True - - # ================================================== - # socket object methods. - # ================================================== - - def listen(self, num): - self.accepting = True - if os.name == 'nt' and num > 5: - num = 1 - return self.socket.listen(num) - - def bind(self, addr): - self.addr = addr - return self.socket.bind(addr) - - def connect(self, address): - self.connected = False - err = self.socket.connect_ex(address) - # XXX Should interpret Winsock return values - if err in (EINPROGRESS, EALREADY, EWOULDBLOCK): - return - if err in (0, EISCONN): - self.addr = address - self.connected = True - self.handle_connect() - else: - raise socket.error, (err, errorcode[err]) - - def accept(self): - # XXX can return either an address pair or None - try: - conn, addr = self.socket.accept() - return conn, addr - except socket.error, why: - if why[0] == EWOULDBLOCK: - pass - else: - raise - - def send(self, data): - try: - result = self.socket.send(data) - return result - except socket.error, why: - if why[0] == EWOULDBLOCK: - return 0 - else: - raise - return 0 - - def recv(self, buffer_size): - try: - data = self.socket.recv(buffer_size) - if not data: - # a closed connection is indicated by signaling - # a read condition, and having recv() return 0. - self.handle_close() - return '' - else: - return data - except socket.error, why: - # winsock sometimes throws ENOTCONN - if why[0] in [ECONNRESET, ENOTCONN, ESHUTDOWN]: - self.handle_close() - return '' - else: - raise - - def close(self): - self.del_channel() - self.socket.close() - - # cheap inheritance, used to pass all other attribute - # references to the underlying socket object. - def __getattr__(self, attr): - return getattr(self.socket, attr) - - # log and log_info may be overridden to provide more sophisticated - # logging and warning methods. In general, log is for 'hit' logging - # and 'log_info' is for informational, warning and error logging. - # edited out for SoC additions - - #def log(self, message): - # sys.stderr.write('log: %s\n' % str(message)) - - #def log_info(self, message, type='info'): - # if __debug__ or type != 'info': - # print '%s: %s' % (type, message) - -# ================================================================================================= -# SoC start edit -# log and log_info are kept for backwards compatibility, particularly with subclasses which -# override them. -# -# log is for hits (like an apache access log) -# log_info is for instrumentation (like an apache error log), regardless of severity. -# informational, warning, and error messages all go here. -# ================================================================================================= - - logger = None - - def _logger(self, level, msg, *args, **kwargs): - if logger is None: - import logging - self.logger = logging.getLogger("py.module.__asyncore__") - else: - self.logger.log(level, msg, *args, **kwargs) - - hit_logger = None - - def log(self, message): - if hit_logger is None: - import logging - self.hit_logger = logging.getLogger("py.module.__asyncore__.dispatcher.hits").info - else: - self.hit_logger(message) - - message_logger=None - - def log_info(self, message, type='info'): - if message_logger is None: - import logging - else: - self.message_logger = logging.getLogger("py.module.__asyncore__.dispatcher.messages") - # Figure out the level somehow -- maybe do google searches on asyncore and - # medusa to see what values actually get used for type? - self.message_logger(level, message) - -# ================================================================================================= -# SoC end edit -# ================================================================================================= - - - def handle_read_event(self): - if self.accepting: - # for an accepting socket, getting a read implies - # that we are connected - if not self.connected: - self.connected = True - self.handle_accept() - elif not self.connected: - self.handle_connect() - self.connected = True - self.handle_read() - else: - self.handle_read() - - def handle_write_event(self): - # getting a write implies that we are connected - if not self.connected: - self.handle_connect() - self.connected = True - self.handle_write() - - def handle_expt_event(self): - self.handle_expt() - - def handle_error(self): - nil, t, v, tbinfo = compact_traceback() - - # sometimes a user repr method will crash. - try: - self_repr = repr(self) - except: - self_repr = '<__repr__(self) failed for object at %0x>' % id(self) - - self.log_info( - 'uncaptured python exception, closing channel %s (%s:%s %s)' % ( - self_repr, - t, - v, - tbinfo - ), - 'error' - ) - self.close() - - def handle_expt(self): - self.log_info('unhandled exception', 'warning') - - def handle_read(self): - self.log_info('unhandled read event', 'warning') - - def handle_write(self): - self.log_info('unhandled write event', 'warning') - - def handle_connect(self): - self.log_info('unhandled connect event', 'warning') - - def handle_accept(self): - self.log_info('unhandled accept event', 'warning') - - def handle_close(self): - self.log_info('unhandled close event', 'warning') - self.close() - -# --------------------------------------------------------------------------- -# adds simple buffered output capability, useful for simple clients. -# [for more sophisticated usage use asynchat.async_chat] -# --------------------------------------------------------------------------- - -class dispatcher_with_send(dispatcher): - - def __init__(self, sock=None, map=None): - dispatcher.__init__(self, sock, map) - self.out_buffer = '' - - def initiate_send(self): - num_sent = 0 - num_sent = dispatcher.send(self, self.out_buffer[:512]) - self.out_buffer = self.out_buffer[num_sent:] - - def handle_write(self): - self.initiate_send() - - def writable(self): - return (not self.connected) or len(self.out_buffer) - - def send(self, data): - if self.debug: - self.log_info('sending %s' % repr(data)) - self.out_buffer = self.out_buffer + data - self.initiate_send() - -# --------------------------------------------------------------------------- -# used for debugging. -# --------------------------------------------------------------------------- - -def compact_traceback(): - t, v, tb = sys.exc_info() - tbinfo = [] - assert tb # Must have a traceback - while tb: - tbinfo.append(( - tb.tb_frame.f_code.co_filename, - tb.tb_frame.f_code.co_name, - str(tb.tb_lineno) - )) - tb = tb.tb_next - - # just to be safe - del tb - - file, function, line = tbinfo[-1] - info = ' '.join(['[%s|%s|%s]' % x for x in tbinfo]) - return (file, function, line), t, v, info - -def close_all(map=None): - if map is None: - map = socket_map - for x in map.values(): - x.socket.close() - map.clear() - -# Asynchronous File I/O: -# -# After a little research (reading man pages on various unixen, and -# digging through the linux kernel), I've determined that select() -# isn't meant for doing asynchronous file i/o. -# Heartening, though - reading linux/mm/filemap.c shows that linux -# supports asynchronous read-ahead. So _MOST_ of the time, the data -# will be sitting in memory for us already when we go to read it. -# -# What other OS's (besides NT) support async file i/o? [VMS?] -# -# Regardless, this is useful for pipes, and stdin/stdout... - -if os.name == 'posix': - import fcntl - - class file_wrapper: - # here we override just enough to make a file - # look like a socket for the purposes of asyncore. - - def __init__(self, fd): - self.fd = fd - - def recv(self, *args): - return os.read(self.fd, *args) - - def send(self, *args): - return os.write(self.fd, *args) - - read = recv - write = send - - def close(self): - os.close(self.fd) - - def fileno(self): - return self.fd - - class file_dispatcher(dispatcher): - - def __init__(self, fd, map=None): - dispatcher.__init__(self, None, map) - self.connected = True - self.set_file(fd) - # set it to non-blocking mode - flags = fcntl.fcntl(fd, fcntl.F_GETFL, 0) - flags = flags | os.O_NONBLOCK - fcntl.fcntl(fd, fcntl.F_SETFL, flags) - - def set_file(self, fd): - self._fileno = fd - self.socket = file_wrapper(fd) - self.add_channel() Deleted: /python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py ============================================================================== --- /python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Tue Jun 27 21:02:39 2006 +++ (empty file) @@ -1,14 +0,0 @@ -# !/usr/bin/env python - -""" -Test harness for the standard library logging module. -This does not work at all (I don't know what I'm doing). -""" - -import logging, asyncore - -logging.basicConfig() - -log = logging.getLogger("py.asyncore") -log.setLevel(logging.DEBUG) # The level is set to DEBUG so nothing will be ignored - From python-checkins at python.org Tue Jun 27 21:03:31 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Tue, 27 Jun 2006 21:03:31 +0200 (CEST) Subject: [Python-checkins] r47135 - python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Message-ID: <20060627190331.C60281E4004@bag.python.org> Author: jackilyn.hoxworth Date: Tue Jun 27 21:03:31 2006 New Revision: 47135 Added: python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Log: Added items remotely test_stdliblogging.py asyncore.py Added: python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/test_stdliblogging.py Tue Jun 27 21:03:31 2006 @@ -0,0 +1,33 @@ +# !/usr/bin/env python + +""" + +Test harness for the standard library logging module. + +""" + +import logging +import asyncore +from cStringIO import StringIO + +log=logging.getLogger("py.asyncore") +stringLog = StringIO() + +# define the handler and level +handler = logging.StreamHandler(stringLog) +log.setLevel(logging.INFO) + +# set a format for the output +formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s') +handler.setFormatter(formatter) + +# add the handler to the logger +log.addHandler(handler) + +asyncore.dispatcher().log("message") +print stringLog.getvalue() # For testing purposes + +if stringLog.getvalue() != "Error: It worked": + print "it worked" +else: + print "it didn't work" From python-checkins at python.org Tue Jun 27 21:03:44 2006 From: python-checkins at python.org (jackilyn.hoxworth) Date: Tue, 27 Jun 2006 21:03:44 +0200 (CEST) Subject: [Python-checkins] r47136 - python/branches/hoxworth-stdlib_logging-soc/asyncore.py Message-ID: <20060627190344.0B4161E4004@bag.python.org> Author: jackilyn.hoxworth Date: Tue Jun 27 21:03:43 2006 New Revision: 47136 Added: python/branches/hoxworth-stdlib_logging-soc/asyncore.py Log: Added items remotely test_stdliblogging.py asyncore.py Added: python/branches/hoxworth-stdlib_logging-soc/asyncore.py ============================================================================== --- (empty file) +++ python/branches/hoxworth-stdlib_logging-soc/asyncore.py Tue Jun 27 21:03:43 2006 @@ -0,0 +1,568 @@ +# -*- Mode: Python -*- +# Id: asyncore.py,v 2.51 2000/09/07 22:29:26 rushing Exp +# Author: Sam Rushing + +# ====================================================================== +# Copyright 1996 by Sam Rushing +# +# All Rights Reserved +# +# Permission to use, copy, modify, and distribute this software and +# its documentation for any purpose and without fee is hereby +# granted, provided that the above copyright notice appear in all +# copies and that both that copyright notice and this permission +# notice appear in supporting documentation, and that the name of Sam +# Rushing not be used in advertising or publicity pertaining to +# distribution of the software without specific, written prior +# permission. +# +# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, +# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN +# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR +# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS +# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, +# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN +# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +# ====================================================================== + +"""Basic infrastructure for asynchronous socket service clients and servers. + +There are only two ways to have a program on a single processor do "more +than one thing at a time". Multi-threaded programming is the simplest and +most popular way to do it, but there is another very different technique, +that lets you have nearly all the advantages of multi-threading, without +actually using multiple threads. it's really only practical if your program +is largely I/O bound. If your program is CPU bound, then pre-emptive +scheduled threads are probably what you really need. Network servers are +rarely CPU-bound, however. + +If your operating system supports the select() system call in its I/O +library (and nearly all do), then you can use it to juggle multiple +communication channels at once; doing other work while your I/O is taking +place in the "background." Although this strategy can seem strange and +complex, especially at first, it is in many ways easier to understand and +control than multi-threaded programming. The module documented here solves +many of the difficult problems for you, making the task of building +sophisticated high-performance network servers and clients a snap. +""" + +import select +import socket +import sys +import time + +import os +from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, \ + ENOTCONN, ESHUTDOWN, EINTR, EISCONN, errorcode + +try: + socket_map +except NameError: + socket_map = {} + +class ExitNow(Exception): + pass + +def read(obj): + try: + obj.handle_read_event() + except ExitNow: + raise + except: + obj.handle_error() + +def write(obj): + try: + obj.handle_write_event() + except ExitNow: + raise + except: + obj.handle_error() + +def _exception (obj): + try: + obj.handle_expt_event() + except ExitNow: + raise + except: + obj.handle_error() + +def readwrite(obj, flags): + try: + if flags & (select.POLLIN | select.POLLPRI): + obj.handle_read_event() + if flags & select.POLLOUT: + obj.handle_write_event() + if flags & (select.POLLERR | select.POLLHUP | select.POLLNVAL): + obj.handle_expt_event() + except ExitNow: + raise + except: + obj.handle_error() + +def poll(timeout=0.0, map=None): + if map is None: + map = socket_map + if map: + r = []; w = []; e = [] + for fd, obj in map.items(): + is_r = obj.readable() + is_w = obj.writable() + if is_r: + r.append(fd) + if is_w: + w.append(fd) + if is_r or is_w: + e.append(fd) + if [] == r == w == e: + time.sleep(timeout) + else: + try: + r, w, e = select.select(r, w, e, timeout) + except select.error, err: + if err[0] != EINTR: + raise + else: + return + + for fd in r: + obj = map.get(fd) + if obj is None: + continue + read(obj) + + for fd in w: + obj = map.get(fd) + if obj is None: + continue + write(obj) + + for fd in e: + obj = map.get(fd) + if obj is None: + continue + _exception(obj) + +def poll2(timeout=0.0, map=None): + # Use the poll() support added to the select module in Python 2.0 + if map is None: + map = socket_map + if timeout is not None: + # timeout is in milliseconds + timeout = int(timeout*1000) + pollster = select.poll() + if map: + for fd, obj in map.items(): + flags = 0 + if obj.readable(): + flags |= select.POLLIN | select.POLLPRI + if obj.writable(): + flags |= select.POLLOUT + if flags: + # Only check for exceptions if object was either readable + # or writable. + flags |= select.POLLERR | select.POLLHUP | select.POLLNVAL + pollster.register(fd, flags) + try: + r = pollster.poll(timeout) + except select.error, err: + if err[0] != EINTR: + raise + r = [] + for fd, flags in r: + obj = map.get(fd) + if obj is None: + continue + readwrite(obj, flags) + +poll3 = poll2 # Alias for backward compatibility + +def loop(timeout=30.0, use_poll=False, map=None, count=None): + if map is None: + map = socket_map + + if use_poll and hasattr(select, 'poll'): + poll_fun = poll2 + else: + poll_fun = poll + + if count is None: + while map: + poll_fun(timeout, map) + + else: + while map and count > 0: + poll_fun(timeout, map) + count = count - 1 + +class dispatcher: + + debug = False + connected = False + accepting = False + closing = False + addr = None + + def __init__(self, sock=None, map=None): + if map is None: + self._map = socket_map + else: + self._map = map + + if sock: + self.set_socket(sock, map) + # I think it should inherit this anyway + self.socket.setblocking(0) + self.connected = True + # XXX Does the constructor require that the socket passed + # be connected? + try: + self.addr = sock.getpeername() + except socket.error: + # The addr isn't crucial + pass + else: + self.socket = None + + def __repr__(self): + status = [self.__class__.__module__+"."+self.__class__.__name__] + if self.accepting and self.addr: + status.append('listening') + elif self.connected: + status.append('connected') + if self.addr is not None: + try: + status.append('%s:%d' % self.addr) + except TypeError: + status.append(repr(self.addr)) + return '<%s at %#x>' % (' '.join(status), id(self)) + + def add_channel(self, map=None): + #self.log_info('adding channel %s' % self) + if map is None: + map = self._map + map[self._fileno] = self + + def del_channel(self, map=None): + fd = self._fileno + if map is None: + map = self._map + if map.has_key(fd): + #self.log_info('closing channel %d:%s' % (fd, self)) + del map[fd] + self._fileno = None + + def create_socket(self, family, type): + self.family_and_type = family, type + self.socket = socket.socket(family, type) + self.socket.setblocking(0) + self._fileno = self.socket.fileno() + self.add_channel() + + def set_socket(self, sock, map=None): + self.socket = sock +## self.__dict__['socket'] = sock + self._fileno = sock.fileno() + self.add_channel(map) + + def set_reuse_addr(self): + # try to re-use a server port if possible + try: + self.socket.setsockopt( + socket.SOL_SOCKET, socket.SO_REUSEADDR, + self.socket.getsockopt(socket.SOL_SOCKET, + socket.SO_REUSEADDR) | 1 + ) + except socket.error: + pass + + # ================================================== + # predicates for select() + # these are used as filters for the lists of sockets + # to pass to select(). + # ================================================== + + def readable(self): + return True + + def writable(self): + return True + + # ================================================== + # socket object methods. + # ================================================== + + def listen(self, num): + self.accepting = True + if os.name == 'nt' and num > 5: + num = 1 + return self.socket.listen(num) + + def bind(self, addr): + self.addr = addr + return self.socket.bind(addr) + + def connect(self, address): + self.connected = False + err = self.socket.connect_ex(address) + # XXX Should interpret Winsock return values + if err in (EINPROGRESS, EALREADY, EWOULDBLOCK): + return + if err in (0, EISCONN): + self.addr = address + self.connected = True + self.handle_connect() + else: + raise socket.error, (err, errorcode[err]) + + def accept(self): + # XXX can return either an address pair or None + try: + conn, addr = self.socket.accept() + return conn, addr + except socket.error, why: + if why[0] == EWOULDBLOCK: + pass + else: + raise + + def send(self, data): + try: + result = self.socket.send(data) + return result + except socket.error, why: + if why[0] == EWOULDBLOCK: + return 0 + else: + raise + return 0 + + def recv(self, buffer_size): + try: + data = self.socket.recv(buffer_size) + if not data: + # a closed connection is indicated by signaling + # a read condition, and having recv() return 0. + self.handle_close() + return '' + else: + return data + except socket.error, why: + # winsock sometimes throws ENOTCONN + if why[0] in [ECONNRESET, ENOTCONN, ESHUTDOWN]: + self.handle_close() + return '' + else: + raise + + def close(self): + self.del_channel() + self.socket.close() + +# ================================================================================================= +# SoC start edit +# ================================================================================================= + + logger = None + + def _logger(self, level, msg, *args, **kwargs): + if self.logger is None: + import logging + self.logger = logging.getLogger("py.asyncore") + self.logger.log(level, msg, *args, **kwargs) + + hit_logger = None + + def log(self, message): + if self.hit_logger is None: + import logging + self.hit_logger = logging.getLogger("py.asyncore.dispatcher.hits").info + self.hit_logger(message) + + message_logger = None + + def log_info(self, message, type='info'): + if self.message_logger is None: + import logging + self.message_logger = logging.getLogger("py.asyncore.dispatcher.messages").info + self.message_logger(level, message) + +# ================================================================================================= +# SoC end edit +# ================================================================================================= + + + def handle_read_event(self): + if self.accepting: + # for an accepting socket, getting a read implies + # that we are connected + if not self.connected: + self.connected = True + self.handle_accept() + elif not self.connected: + self.handle_connect() + self.connected = True + self.handle_read() + else: + self.handle_read() + + def handle_write_event(self): + # getting a write implies that we are connected + if not self.connected: + self.handle_connect() + self.connected = True + self.handle_write() + + def handle_expt_event(self): + self.handle_expt() + + def handle_error(self): + nil, t, v, tbinfo = compact_traceback() + + # sometimes a user repr method will crash. + try: + self_repr = repr(self) + except: + self_repr = '<__repr__(self) failed for object at %0x>' % id(self) + + self.log_info( + 'uncaptured python exception, closing channel %s (%s:%s %s)' % ( + self_repr, + t, + v, + tbinfo + ), + 'error' + ) + self.close() + + def handle_expt(self): + self.log_info('unhandled exception', 'warning') + + def handle_read(self): + self.log_info('unhandled read event', 'warning') + + def handle_write(self): + self.log_info('unhandled write event', 'warning') + + def handle_connect(self): + self.log_info('unhandled connect event', 'warning') + + def handle_accept(self): + self.log_info('unhandled accept event', 'warning') + + def handle_close(self): + self.log_info('unhandled close event', 'warning') + self.close() + +# --------------------------------------------------------------------------- +# adds simple buffered output capability, useful for simple clients. +# [for more sophisticated usage use asynchat.async_chat] +# --------------------------------------------------------------------------- + +class dispatcher_with_send(dispatcher): + + def __init__(self, sock=None, map=None): + dispatcher.__init__(self, sock, map) + self.out_buffer = '' + + def initiate_send(self): + num_sent = 0 + num_sent = dispatcher.send(self, self.out_buffer[:512]) + self.out_buffer = self.out_buffer[num_sent:] + + def handle_write(self): + self.initiate_send() + + def writable(self): + return (not self.connected) or len(self.out_buffer) + + def send(self, data): + if self.debug: + self.log_info('sending %s' % repr(data)) + self.out_buffer = self.out_buffer + data + self.initiate_send() + +# --------------------------------------------------------------------------- +# used for debugging. +# --------------------------------------------------------------------------- + +def compact_traceback(): + t, v, tb = sys.exc_info() + tbinfo = [] + assert tb # Must have a traceback + while tb: + tbinfo.append(( + tb.tb_frame.f_code.co_filename, + tb.tb_frame.f_code.co_name, + str(tb.tb_lineno) + )) + tb = tb.tb_next + + # just to be safe + del tb + + file, function, line = tbinfo[-1] + info = ' '.join(['[%s|%s|%s]' % x for x in tbinfo]) + return (file, function, line), t, v, info + +def close_all(map=None): + if map is None: + map = socket_map + for x in map.values(): + x.socket.close() + map.clear() + +# Asynchronous File I/O: +# +# After a little research (reading man pages on various unixen, and +# digging through the linux kernel), I've determined that select() +# isn't meant for doing asynchronous file i/o. +# Heartening, though - reading linux/mm/filemap.c shows that linux +# supports asynchronous read-ahead. So _MOST_ of the time, the data +# will be sitting in memory for us already when we go to read it. +# +# What other OS's (besides NT) support async file i/o? [VMS?] +# +# Regardless, this is useful for pipes, and stdin/stdout... + +if os.name == 'posix': + import fcntl + + class file_wrapper: + # here we override just enough to make a file + # look like a socket for the purposes of asyncore. + + def __init__(self, fd): + self.fd = fd + + def recv(self, *args): + return os.read(self.fd, *args) + + def send(self, *args): + return os.write(self.fd, *args) + + read = recv + write = send + + def close(self): + os.close(self.fd) + + def fileno(self): + return self.fd + + class file_dispatcher(dispatcher): + + def __init__(self, fd, map=None): + dispatcher.__init__(self, None, map) + self.connected = True + self.set_file(fd) + # set it to non-blocking mode + flags = fcntl.fcntl(fd, fcntl.F_GETFL, 0) + flags = flags | os.O_NONBLOCK + fcntl.fcntl(fd, fcntl.F_SETFL, flags) + + def set_file(self, fd): + self._fileno = fd + self.socket = file_wrapper(fd) + self.add_channel() From python-checkins at python.org Wed Jun 28 07:03:23 2006 From: python-checkins at python.org (neal.norwitz) Date: Wed, 28 Jun 2006 07:03:23 +0200 (CEST) Subject: [Python-checkins] r47137 - python/trunk/Lib/mailbox.py Message-ID: <20060628050323.8AE881E4004@bag.python.org> Author: neal.norwitz Date: Wed Jun 28 07:03:22 2006 New Revision: 47137 Modified: python/trunk/Lib/mailbox.py Log: According to the man pages on Gentoo Linux and Tru64, EACCES or EAGAIN can be returned if fcntl (lockf) fails. This fixes the test failure on Tru64 by checking for either error rather than just EAGAIN. Modified: python/trunk/Lib/mailbox.py ============================================================================== --- python/trunk/Lib/mailbox.py (original) +++ python/trunk/Lib/mailbox.py Wed Jun 28 07:03:22 2006 @@ -1805,7 +1805,7 @@ try: fcntl.lockf(f, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError, e: - if e.errno == errno.EAGAIN: + if e.errno in (errno.EAGAIN, errno.EACCES): raise ExternalClashError('lockf: lock unavailable: %s' % f.name) else: From python-checkins at python.org Wed Jun 28 07:53:16 2006 From: python-checkins at python.org (neal.norwitz) Date: Wed, 28 Jun 2006 07:53:16 +0200 (CEST) Subject: [Python-checkins] r47138 - peps/trunk/pep-0356.txt Message-ID: <20060628055316.6719B1E401A@bag.python.org> Author: neal.norwitz Date: Wed Jun 28 07:53:15 2006 New Revision: 47138 Modified: peps/trunk/pep-0356.txt Log: Update with important bugs and some doco Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Wed Jun 28 07:53:15 2006 @@ -146,9 +146,16 @@ Open issues + - Bugs that need resolving before release: + http://python.org/sf/1504046 + http://python.org/sf/1512814 + http://python.org/sf/1512695 + http://python.org/sf/1501934 + - Missing documentation - * ctypes (Thomas Heller) * ElementTree/cElementTree (Fredrik Lundh) + Needs latex-ifcation, patch at http://python.org/sf/1504046 + * uuid (Ka-Ping Yee), patch at http://python.org/sf/1508766 - AST compiler problems (Owner: Jeremy Hylton) From python-checkins at python.org Wed Jun 28 08:28:32 2006 From: python-checkins at python.org (neal.norwitz) Date: Wed, 28 Jun 2006 08:28:32 +0200 (CEST) Subject: [Python-checkins] r47139 - in python/trunk: Misc/NEWS Modules/cPickle.c Message-ID: <20060628062832.1B3561E4004@bag.python.org> Author: neal.norwitz Date: Wed Jun 28 08:28:31 2006 New Revision: 47139 Modified: python/trunk/Misc/NEWS python/trunk/Modules/cPickle.c Log: Fix bug #1512695: cPickle.loads could crash if it was interrupted with a KeyboardInterrupt since PyTuple_Pack was passed a NULL. Will backport. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 28 08:28:31 2006 @@ -28,6 +28,12 @@ - The wsgiref package is now installed properly on Unix. +Extension Modules +----------------- + +- Bug #1512695: cPickle.loads could crash if it was interrupted with + a KeyboardInterrupt. + Build ----- Modified: python/trunk/Modules/cPickle.c ============================================================================== --- python/trunk/Modules/cPickle.c (original) +++ python/trunk/Modules/cPickle.c Wed Jun 28 08:28:31 2006 @@ -3628,10 +3628,14 @@ err: { - PyObject *tp, *v, *tb; + PyObject *tp, *v, *tb, *tmp_value; PyErr_Fetch(&tp, &v, &tb); - if ((r=PyTuple_Pack(3,v,cls,args))) { + tmp_value = v; + /* NULL occurs when there was a KeyboardInterrupt */ + if (tmp_value == NULL) + tmp_value = Py_None; + if ((r = PyTuple_Pack(3, tmp_value, cls, args))) { Py_XDECREF(v); v=r; } From python-checkins at python.org Wed Jun 28 08:38:26 2006 From: python-checkins at python.org (neal.norwitz) Date: Wed, 28 Jun 2006 08:38:26 +0200 (CEST) Subject: [Python-checkins] r47140 - in python/branches/release24-maint: Misc/NEWS Modules/cPickle.c Message-ID: <20060628063826.C91D51E4004@bag.python.org> Author: neal.norwitz Date: Wed Jun 28 08:38:25 2006 New Revision: 47140 Modified: python/branches/release24-maint/Misc/NEWS python/branches/release24-maint/Modules/cPickle.c Log: Backport 47139: Fix bug #1512695: cPickle.loads could crash if it was interrupted with a KeyboardInterrupt since PyTuple_Pack was passed a NULL. Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Wed Jun 28 08:38:25 2006 @@ -26,6 +26,9 @@ Extension Modules ----------------- +- Bug #1512695: cPickle.loads could crash if it was interrupted with + a KeyboardInterrupt. + - Change binascii.hexlify() to accept any read-only buffer and not just a char buffer. Modified: python/branches/release24-maint/Modules/cPickle.c ============================================================================== --- python/branches/release24-maint/Modules/cPickle.c (original) +++ python/branches/release24-maint/Modules/cPickle.c Wed Jun 28 08:38:25 2006 @@ -3633,10 +3633,14 @@ err: { - PyObject *tp, *v, *tb; + PyObject *tp, *v, *tb, *tmp_value; PyErr_Fetch(&tp, &v, &tb); - if ((r=PyTuple_Pack(3,v,cls,args))) { + tmp_value = v; + /* NULL occurs when there was a KeyboardInterrupt */ + if (tmp_value == NULL) + tmp_value = Py_None; + if ((r = PyTuple_Pack(3, tmp_value, cls, args))) { Py_XDECREF(v); v=r; } From buildbot at python.org Wed Jun 28 08:43:37 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 28 Jun 2006 06:43:37 +0000 Subject: [Python-checkins] buildbot failure in amd64 gentoo trunk Message-ID: <20060628064337.469FA1E4004@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/1153 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From python-checkins at python.org Wed Jun 28 08:47:51 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 28 Jun 2006 08:47:51 +0200 (CEST) Subject: [Python-checkins] r47141 - peps/trunk/pep-3100.txt Message-ID: <20060628064751.B8B3F1E4004@bag.python.org> Author: georg.brandl Date: Wed Jun 28 08:47:51 2006 New Revision: 47141 Modified: peps/trunk/pep-3100.txt Log: Add __main__() PEP. Modified: peps/trunk/pep-3100.txt ============================================================================== --- peps/trunk/pep-3100.txt (original) +++ peps/trunk/pep-3100.txt Wed Jun 28 08:47:51 2006 @@ -110,6 +110,7 @@ NB. {range(x)} means set([range(x)]), NOT set(range(x)). There's no literal for an empty set; use set() (or {1}&{2} :-). There's no frozenset literal; they are too rarely needed. +* Reconsider PEP 299 [30]_: special __main__() function in modules. To be removed: @@ -324,6 +325,9 @@ .. [29] python-3000 email ("bug in modulus?") http://mail.python.org/pipermail/python-3000/2006-May/001735.html +.. [30] PEP 299 ("Special __main__() function in modules") + http://www.python.org/dev/peps/pep-0299 + .. [#pep238] PEP 238 (Changing the Division Operator) http://www.python.org/dev/peps/pep-0238 From buildbot at python.org Wed Jun 28 09:45:54 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 28 Jun 2006 07:45:54 +0000 Subject: [Python-checkins] buildbot warnings in ia64 Debian unstable 2.4 Message-ID: <20060628074554.3C7301E4005@bag.python.org> The Buildbot has detected a new failure of ia64 Debian unstable 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Debian%2520unstable%25202.4/builds/111 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From neal at metaslash.com Wed Jun 28 11:07:26 2006 From: neal at metaslash.com (Neal Norwitz) Date: Wed, 28 Jun 2006 05:07:26 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060628090726.GA20021@python.psfb.org> test_socket leaked [0, 0, 214] references From buildbot at python.org Wed Jun 28 12:08:54 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 28 Jun 2006 10:08:54 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060628100854.25A401E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/423 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 28 12:41:48 2006 From: python-checkins at python.org (nick.coghlan) Date: Wed, 28 Jun 2006 12:41:48 +0200 (CEST) Subject: [Python-checkins] r47142 - in python/trunk: Doc/lib/librunpy.tex Lib/runpy.py Lib/test/test_runpy.py Message-ID: <20060628104148.362A01E4004@bag.python.org> Author: nick.coghlan Date: Wed Jun 28 12:41:47 2006 New Revision: 47142 Modified: python/trunk/Doc/lib/librunpy.tex python/trunk/Lib/runpy.py python/trunk/Lib/test/test_runpy.py Log: Make full module name available as __module_name__ even when __name__ is set to something else (like '__main__') Modified: python/trunk/Doc/lib/librunpy.tex ============================================================================== --- python/trunk/Doc/lib/librunpy.tex (original) +++ python/trunk/Doc/lib/librunpy.tex Wed Jun 28 12:41:47 2006 @@ -35,16 +35,19 @@ global variables below are defined in the supplied dictionary, those definitions are overridden by the \code{run_module} function. -The special global variables \code{__name__}, \code{__file__}, -\code{__loader__} and \code{__builtins__} are set in the globals -dictionary before the module code is executed. +The special global variables \code{__name__}, \code{__module_name__}, +\code{__file__}, \code{__loader__} and \code{__builtins__} are +set in the globals dictionary before the module code is executed. \code{__name__} is set to \var{run_name} if this optional argument is supplied, and the \var{mod_name} argument otherwise. +\code{__module_name__} is always set to \var{mod_name} (this allows +modules to use imports relative to their package name). + \code{__loader__} is set to the PEP 302 module loader used to retrieve -the code for the module (This loader may be a wrapper around the -standard import mechanism). +the code for the module (This will not be defined if the module was +found using the standard import mechanism). \code{__file__} is set to the name provided by the module loader. If the loader does not make filename information available, this @@ -55,10 +58,12 @@ If the argument \var{alter_sys} is supplied and evaluates to \code{True}, then \code{sys.argv[0]} is updated with the value of -\code{__file__} and \code{sys.modules[__name__]} is updated with a +\code{__file__} and \code{sys.modules[mod_name]} is updated with a temporary module object for the module being executed. Both -\code{sys.argv[0]} and \code{sys.modules[__name__]} are restored to -their original values before the function returns. +\code{sys.argv[0]} and \code{sys.modules[mod_name]} are restored to +their original values before the function returns. If \var{run_name} +differs from \var{mod_name} entries are made in \code{sys.modules} +for both names. Note that this manipulation of \module{sys} is not thread-safe. Other threads may see the partially initialised module, as well as the Modified: python/trunk/Lib/runpy.py ============================================================================== --- python/trunk/Lib/runpy.py (original) +++ python/trunk/Lib/runpy.py Wed Jun 28 12:41:47 2006 @@ -21,18 +21,19 @@ ] -def _run_code(code, run_globals, init_globals, +def _run_code(code, run_globals, init_globals, run_name, mod_name, mod_fname, mod_loader): """Helper for _run_module_code""" if init_globals is not None: run_globals.update(init_globals) - run_globals.update(__name__ = mod_name, + run_globals.update(__name__ = run_name, + __module_name__ = mod_name, __file__ = mod_fname, __loader__ = mod_loader) exec code in run_globals return run_globals -def _run_module_code(code, init_globals=None, +def _run_module_code(code, init_globals=None, run_name=None, mod_name=None, mod_fname=None, mod_loader=None, alter_sys=False): """Helper for run_module""" @@ -42,26 +43,33 @@ temp_module = imp.new_module(mod_name) mod_globals = temp_module.__dict__ saved_argv0 = sys.argv[0] - restore_module = mod_name in sys.modules - if restore_module: - saved_module = sys.modules[mod_name] + sentinel = object() + module_mod_name = sys.modules.get(mod_name, sentinel) + module_run_name = sys.modules.get(run_name, sentinel) sys.argv[0] = mod_fname sys.modules[mod_name] = temp_module + if run_name != mod_name: + sys.modules[run_name] = temp_module try: - _run_code(code, mod_globals, init_globals, + _run_code(code, mod_globals, init_globals, run_name, mod_name, mod_fname, mod_loader) finally: sys.argv[0] = saved_argv0 - if restore_module: - sys.modules[mod_name] = saved_module - else: - del sys.modules[mod_name] + if module_mod_name is not sentinel: + sys.modules[mod_name] = module_mod_name + else: + del sys.modules[mod_name] + if run_name != mod_name: + if module_run_name is not sentinel: + sys.modules[run_name] = module_run_name + else: + del sys.modules[run_name] # Copy the globals of the temporary module, as they # may be cleared when the temporary module goes away return mod_globals.copy() else: # Leave the sys module alone - return _run_code(code, {}, init_globals, + return _run_code(code, {}, init_globals, run_name, mod_name, mod_fname, mod_loader) @@ -92,7 +100,7 @@ if run_name is None: run_name = mod_name return _run_module_code(code, init_globals, run_name, - filename, loader, alter_sys) + mod_name, filename, loader, alter_sys) if __name__ == "__main__": Modified: python/trunk/Lib/test/test_runpy.py ============================================================================== --- python/trunk/Lib/test/test_runpy.py (original) +++ python/trunk/Lib/test/test_runpy.py Wed Jun 28 12:41:47 2006 @@ -23,6 +23,8 @@ "run_argv0 = sys.argv[0]\n" "if __name__ in sys.modules:\n" " run_name = sys.modules[__name__].__name__\n" + "if __module_name__ in sys.modules:\n" + " mod_name = sys.modules[__module_name__].__module_name__\n" "# Check nested operation\n" "import runpy\n" "nested = runpy._run_module_code('x=1\\n', mod_name='',\n" @@ -32,14 +34,16 @@ def test_run_module_code(self): initial = object() - name = "" + run_name = "" + mod_name = "" file = "Some other nonsense" loader = "Now you're just being silly" d1 = dict(initial=initial) saved_argv0 = sys.argv[0] d2 = _run_module_code(self.test_source, d1, - name, + run_name, + mod_name, file, loader, True) @@ -47,19 +51,23 @@ self.failUnless(d2["initial"] is initial) self.failUnless(d2["result"] == self.expected_result) self.failUnless(d2["nested"]["x"] == 1) - self.failUnless(d2["__name__"] is name) - self.failUnless(d2["run_name"] is name) + self.failUnless(d2["__name__"] is run_name) + self.failUnless(d2["run_name"] is run_name) + self.failUnless(d2["__module_name__"] is mod_name) + self.failUnless(d2["mod_name"] is mod_name) self.failUnless(d2["__file__"] is file) self.failUnless(d2["run_argv0"] is file) self.failUnless(d2["__loader__"] is loader) self.failUnless(sys.argv[0] is saved_argv0) - self.failUnless(name not in sys.modules) + self.failUnless(mod_name not in sys.modules) + self.failUnless(run_name not in sys.modules) def test_run_module_code_defaults(self): saved_argv0 = sys.argv[0] d = _run_module_code(self.test_source) self.failUnless(d["result"] == self.expected_result) self.failUnless(d["__name__"] is None) + self.failUnless(d["__module_name__"] is None) self.failUnless(d["__file__"] is None) self.failUnless(d["__loader__"] is None) self.failUnless(d["run_argv0"] is saved_argv0) From python-checkins at python.org Wed Jun 28 12:49:51 2006 From: python-checkins at python.org (armin.rigo) Date: Wed, 28 Jun 2006 12:49:51 +0200 (CEST) Subject: [Python-checkins] r47143 - python/trunk/Lib/test/crashers/bogus_code_obj.py python/trunk/Lib/test/crashers/gc_inspection.py Message-ID: <20060628104951.EBF351E4004@bag.python.org> Author: armin.rigo Date: Wed Jun 28 12:49:51 2006 New Revision: 47143 Added: python/trunk/Lib/test/crashers/bogus_code_obj.py (contents, props changed) python/trunk/Lib/test/crashers/gc_inspection.py (contents, props changed) Log: A couple of crashers of the "won't fix" kind. Added: python/trunk/Lib/test/crashers/bogus_code_obj.py ============================================================================== --- (empty file) +++ python/trunk/Lib/test/crashers/bogus_code_obj.py Wed Jun 28 12:49:51 2006 @@ -0,0 +1,9 @@ +""" +Broken bytecode objects can easily crash the interpreter. +""" + +import types + +co = types.CodeType(0, 0, 0, 0, '\x04\x71\x00\x00', (), + (), (), '', '', 1, '') +exec co Added: python/trunk/Lib/test/crashers/gc_inspection.py ============================================================================== --- (empty file) +++ python/trunk/Lib/test/crashers/gc_inspection.py Wed Jun 28 12:49:51 2006 @@ -0,0 +1,17 @@ +""" +gc.get_referrers() can be used to see objects before they are fully built. +""" + +import gc + + +def g(): + marker = object() + yield marker + # now the marker is in the tuple being constructed + [tup] = [x for x in gc.get_referrers(marker) if type(x) is tuple] + print tup + print tup[1] + + +tuple(g()) From python-checkins at python.org Wed Jun 28 13:08:51 2006 From: python-checkins at python.org (nick.coghlan) Date: Wed, 28 Jun 2006 13:08:51 +0200 (CEST) Subject: [Python-checkins] r47144 - peps/trunk/pep-0338.txt Message-ID: <20060628110851.A3C101E4004@bag.python.org> Author: nick.coghlan Date: Wed Jun 28 13:08:51 2006 New Revision: 47144 Modified: peps/trunk/pep-0338.txt Log: Add a note to PEP 338 about the use of relative imports Modified: peps/trunk/pep-0338.txt ============================================================================== --- peps/trunk/pep-0338.txt (original) +++ peps/trunk/pep-0338.txt Wed Jun 28 13:08:51 2006 @@ -180,6 +180,39 @@ and then invokes ``run_module(sys.argv[0], run_name="__main__", alter_sys=True)``. +Relative Imports +================ + +2.5b1 showed an annoying interaction between this PEP and PEP 328 - +explicit relative imports don't work from a main module, because +relative imports rely on ``__name__`` to determine the current module's +position in the package hierarchy. + +Accordingly, the operation of ``run_module()`` was modified so that +another special variable ``__module_name__`` was defined in the +namespace of the module being executed. This variable always holds +the true module name, even if ``__name__`` is set to something else +(like ``'__main__'``) + +Modules that don't rely on relative imports can be used from a +package as a main module without any changes. In order to both use +relative imports and also be usable as a main module, a module in a +package will currently need to use a structure similar to the +following:: + + # Docstring and any future statements + _is_main = False + if __name__ == "__main__": + _is_main = True + __name__ = __module_name__ + + # Support module section, including relative imports + + if _is_main: + # Main module section + +That said, Guido's recommended solution is to avoid using relative +imports in the first place. Resolved Issues ================ From buildbot at python.org Wed Jun 28 14:00:49 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 28 Jun 2006 12:00:49 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20060628120049.CD76C1E4004@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/1097 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Wed Jun 28 14:48:54 2006 From: python-checkins at python.org (matt.fleming) Date: Wed, 28 Jun 2006 14:48:54 +0200 (CEST) Subject: [Python-checkins] r47145 - in sandbox/trunk/pdb: README.txt mpdb.py mthread.py test/test_mpdb.py Message-ID: <20060628124854.6E7901E4005@bag.python.org> Author: matt.fleming Date: Wed Jun 28 14:48:52 2006 New Revision: 47145 Modified: sandbox/trunk/pdb/README.txt sandbox/trunk/pdb/mpdb.py sandbox/trunk/pdb/mthread.py sandbox/trunk/pdb/test/test_mpdb.py Log: More work in getting breakpoints working per-thread. Modified: sandbox/trunk/pdb/README.txt ============================================================================== --- sandbox/trunk/pdb/README.txt (original) +++ sandbox/trunk/pdb/README.txt Wed Jun 28 14:48:52 2006 @@ -46,5 +46,8 @@ at the moment 'info target' only tells the user whether they are local or remote, and not whether they are the local/remote server or local/remote client. - +* We can switch between threads (like in gdb) with 'thread ', but + this currently has _no_ effect. We should at some point be able to switch + threads and from them on not have to use the 'thread apply ' + syntax, but just '' which would be applied to the current thread. Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Wed Jun 28 14:48:52 2006 @@ -27,6 +27,10 @@ line_prefix = '\n-> ' +class Exit(Exception): + """ Causes a debugger to exit immediately. """ + pass + class MPdb(pydb.Pdb): """ This class extends the command set and functionality of the Python debugger and provides support for, @@ -50,12 +54,15 @@ self.orig_stdin = self.stdin self.prompt = '(MPdb)' self.target = 'local' # local connections by default + self.lastcmd = '' self.connection = None self.debug_thread = False - self.waiter = threading.Event() - self.tracers = [] - self.threads = [] - self.current_thread = threading.currentThread() + # We don't trace the MainThread, so the tracer for the main thread is + # None. + self.tracers = [None] + self.threads = [threading.currentThread()] + self.current_thread = self.threads[0] + self._info_cmds.append('target') self._info_cmds.append('thread') @@ -84,6 +91,9 @@ """ All commands in 'line' are sent across this object's connection instance variable. """ + if not line: + # Execute the previous command + line = self.lastcmd # This is the simplest way I could think of to do this without # breaking any of the inherited code from pydb/pdb. If we're a # remote client, always call 'rquit' (remote quit) when connected to @@ -104,6 +114,7 @@ while self.local_prompt not in ret: ret += self.connection.readline() self.msg_nocr(ret) + self.lastcmd = line return def msg_nocr(self, msg, out=None): @@ -134,11 +145,31 @@ self.errmsg('Thread debugging is not on.') return # We need some way to remove old thread instances - self.msg(self.threads) + for t in self.threads: + if t == self.current_thread: + self.msg('* %d %s' % (self.threads.index(t)+1, t)) + else: + self.msg(' %d %s' % (self.threads.index(t)+1, t)) return else: pydb.Pdb.do_info(self, arg) + def help_info(self, *args): + """Extends pydb help_info command. """ + self.subcommand_help('info', getattr(self,'help_info').__doc__, + self._info_cmds, self.info_helper, args[0]) + + def info_helper(self, cmd, label=False): + """Extends pydb info_helper() to give info about a single Mpdb + info extension.""" + if label: + self.msg_nocr("info %s --" % cmd) + if 'target'.startswith(cmd): + self.msg("Names of targets and files being debugged") + elif 'thread'.startswith(cmd): + self.msg('Information about active and inactive threads.') + else: + pydb.Pdb.info_helper(self, cmd) def do_set(self, arg): """ Extends pydb do_set() to allow setting thread debugging. """ @@ -167,7 +198,7 @@ self.threads.append(threading.currentThread()) self.msg('New thread: %s' % self.threads[-1]) - m = MTracer(self.breaks, self.stdout) + m = MTracer(self.breaks, self.filename, self.stdout) self.tracers.append(m) sys.settrace(m.trace_dispatch) @@ -345,7 +376,7 @@ is aborted. """ if self.target == 'local': - self.errmsg('Connected locally, cannot remotely quit') + self.errmsg('Connected locally; cannot remotely quit') return self._rebind_output(self.orig_stdout) self._rebind_input(self.orig_stdin) @@ -356,7 +387,7 @@ self.msg('Exiting remote debugging...') self.target = 'local' self.do_quit(None) - + raise Exit def do_thread(self, arg): """Use this command to switch between threads. @@ -369,30 +400,58 @@ Type "help thread" followed by thread subcommand name for full documentation. Command name abbreviations are allowed if unambiguous. """ + if not self.debug_thread: + self.errmsg('Thread debugging not on.') + return args = arg.split() if len(args) == 0: - self.msg('Current thread is %s' % self.current_thread) - return - if 'apply'.startswith(args[0]): - if len(args) < 2: - self.errmsg('Please specify a thread ID') - return - if len(args) < 3: - self.errmsg('Please specify a command following the thread' \ - + ' ID') - return - # These should always be in sync - if len(self.threads) == 0: - self.errmsg('No threads') - return - t = self.threads[int(args[1])-1] - t_tracer = self.tracers[int(args[1])-1] - func = args[2] - cmd = eval('t.' + func) + self.msg('Current thread is %d (%s)' % \ + (self.threads.index(self.current_thread)+1, + self.current_thread)) + return + if len(args) < 2: + if args[0].isdigit(): + # XXX Switch to a different thread, although this doesn't + # actually do anything yet. + t_num = int(args[0])-1 + if t_num > len(self.threads): + self.errmsg('Thread ID %d not known.' % t_num+1) + return + self.current_thread = self.threads[t_num] + self.msg('Switching to thread %d (%s)' % (t_num+1, \ + self.current_thread)) + return + self.errmsg('Please specify a Thread ID') + return + if len(args) < 3: + self.errmsg('Please specify a command following the thread' \ + + ' ID') + return + if len(self.threads) == 0: + self.errmsg('No threads') + return + if len(self.threads) < int(args[1]): + self.errmsg('Thread ID %d not known.' % int(args[1])) + return + # These should always be in sync + t = self.threads[int(args[1])-1] + t_tracer = self.tracers[int(args[1])-1] + func = args[2] + if len(args) > 2: + str_params = "" + for w in args[3:]: + str_params += w + str_params.rstrip() + eval('t_tracer.do_' + func + '(str_params)') + #except AttributeError: + # self.errmsg('No such thread subcommand') + # return + else: try: - result = cmd(args[3:]) + eval('t_tracer.do_'+func+'()') except AttributeError: - self.errmsg('No such thread subcommand') + self.errmsg('Undefined thread apply subcommand "%s".' \ + % args[0]) return def pdbserver(addr, args): @@ -405,10 +464,12 @@ m.mainpyfile = mainpyfile m.do_pdbserver(addr) while True: - m._runscript(mainpyfile) - if m._user_requested_quit: + try: + m._runscript(mainpyfile) + except pydb.gdb.Restart: + m.msg('Restarting') + except Exit: break - sys.exit() def target(addr): """ Connect this debugger to a pdbserver at 'addr'. 'addr' is @@ -431,8 +492,10 @@ opts, args = parse_opts() if opts.target: target(opts.target) + sys.exit() elif opts.pdbserver: pdbserver(opts.pdbserver, args) + sys.exit() else: if not opts.scriptname: if not args: @@ -455,6 +518,9 @@ # In most cases SystemExit does not warrant a post-mortem session. mpdb.msg("The program exited via sys.exit(). " + \ "Exit status:",sys.exc_info()[1]) + except Exit: + # This exception raised when we disconnect from a remote session + pass except: mpdb.msg(traceback.format_exc()) mpdb.msg("Uncaught exception. Entering post mortem debugging") Modified: sandbox/trunk/pdb/mthread.py ============================================================================== --- sandbox/trunk/pdb/mthread.py (original) +++ sandbox/trunk/pdb/mthread.py Wed Jun 28 14:48:52 2006 @@ -1,6 +1,8 @@ """ This file contains all code for allowing the debugging of threads. """ from bdb import Breakpoint +import linecache +import inspect import os import sys import threading @@ -11,16 +13,17 @@ which is useful, for instance, if a breakpoint occurs inside a thread's run() method. """ - def __init__(self, breaks=None, stdout=None): + def __init__(self, breaks={}, filename=None, stdout=None): self.thread = threading.currentThread() if stdout is None: stdout = sys.stdout self.out = stdout # Each tracer instance must keep track of its own breakpoints - if breaks is None: - breaks = {} self.breaks = breaks self.fncache = {} + self.filename = filename + self.lineno = 0 + self.curframe = None def canonic(self, filename): if filename == "<" + filename[1:-1] + ">": @@ -37,25 +40,215 @@ if 'break'.startswith(args[0]): return self.breaks - def _get_break(self, filename, lineno): + def get_break(self, filename, lineno): """ Return the breakpoint at [filename:]lineno. """ filename = self.canonic(filename) return filename in self.breaks and \ lineno in self.breaks[filename] - def _get_breaks(self, filename, lineno): + def get_breaks(self, filename, lineno): """ Return all the breakpoints set in this thread. """ filename = self.canonic(filename) return filename in self.breaks and \ lineno in self.breaks[filename] and \ Breakpoint.bplist[filename, lineno] or [] - def _set_break(self, filename, lineno, temporary=0, cond = None, + def set_break(self, filename, lineno, temporary=0, cond = None, funcname=None): """ Set a breakpoint in this thread. """ - pass + + # Derived classes and clients can call the following methods + # to manipulate breakpoints. These methods return an + # error message is something went wrong, None if all is well. + # Set_break prints out the breakpoint line and file:lineno. + # Call self.get_*break*() to see the breakpoints or better + # for bp in Breakpoint.bpbynumber: if bp: bp.bpprint(). + + filename = self.canonic(filename) + import linecache # Import as late as possible + line = linecache.getline(filename, lineno) + if not line: + return 'Line %s:%d does not exist' % (filename, + lineno) + if not filename in self.breaks: + self.breaks[filename] = [] + list = self.breaks[filename] + if not lineno in list: + list.append(lineno) + bp = Breakpoint(filename, lineno, temporary, cond, funcname) + + def do_break(self, arg=None): + """ thread-specific breakpoint information. """ + # XXX For now we don't support temporary breakpoints in threads + temporary = cond = False + if arg is None: + if self.lineno is None: + lineno = max(1, inspect.lineno(self.curframe)) + else: + lineno = self.lineno + 1 + filename = self.curframe.f_code.co_filename + else: + filename = None + lineno = None + comma = arg.find(',') + if comma > 0: + # parse stuff after comma: "condition" + cond = arg[comma+1:].lstrip() + arg = arg[:comma].rstrip() + (funcname, filename, lineno) = self.__parse_filepos(arg) + if lineno is None: return + + # FIXME This default setting doesn't match that used in + # do_clear. Perhaps one is non-optimial. + if not filename: + filename = self.defaultFile() + + # Check for reasonable breakpoint + line = self.checkline(filename, lineno) + if line: + # now set the break point + # Python 2.3.5 takes 5 args rather than 6. + # There is another way in configure to test for the version, + # but this works too. + try: + err = self.set_break(filename, line, temporary, cond, funcname) + except TypeError: + err = self.set_break(filename, line, temporary, cond) + + if err: print >> self.out, err + else: + bp = self.get_breaks(filename, line)[-1] + print >> self.out, "Breakpoint %d set in file %s, line %d." \ + % (bp.number, self.filename(bp.file), bp.line) + + def __parse_filepos(self, arg): + """__parse_filepos(self,arg)->(fn, filename, lineno) + + Parse arg as [filename:]lineno | function + Make sure it works for C:\foo\bar.py:12 + """ + colon = arg.rfind(':') + if colon >= 0: + filename = arg[:colon].rstrip() + f = self.lookupmodule(filename) + if not f: + print >> self.out, "%s not found from sys.path" % \ + self._saferepr(filename) + return (None, None, None) + else: + filename = f + arg = arg[colon+1:].lstrip() + try: + lineno = int(arg) + except ValueError, msg: + print >> self.out, 'Bad lineno: %s' % str(arg) + return (None, filename, None) + return (None, filename, lineno) + else: + # no colon; can be lineno or function + return self.__get_brkpt_lineno(arg) + + def __get_brkpt_lineno(self, arg): + """__get_brkpt_lineno(self,arg)->(filename, file, lineno) + + See if arg is a line number or a function name. Return what + we've found. None can be returned as a value in the triple.""" + funcname, filename = (None, None) + try: + # First try as an integer + lineno = int(arg) + filename = self.curframe.f_code.co_filename + except ValueError: + try: + func = eval(arg, self.curframe.f_globals, + self.curframe.f_locals) + except: + func = arg + try: + if hasattr(func, 'im_func'): + func = func.im_func + code = func.func_code + #use co_name to identify the bkpt (function names + #could be aliased, but co_name is invariant) + funcname = code.co_name + lineno = code.co_firstlineno + filename = code.co_filename + except: + # last thing to try + (ok, filename, ln) = self.lineinfo(arg) + if not ok: + print >> self.out, 'The specified object %s is not' % \ + str(repr(arg)), + print >> self.out, ' a function, or not found' \ + +' along sys.path or no line given.' + + return (None, None, None) + funcname = ok # ok contains a function name + lineno = int(ln) + return (funcname, filename, lineno) + + def lineinfo(self, identifier): + failed = (None, None, None) + # Input is identifier, may be in single quotes + idstring = identifier.split("'") + if len(idstring) == 1: + # not in single quotes + id = idstring[0].strip() + elif len(idstring) == 3: + # quoted + id = idstring[1].strip() + else: + return failed + if id == '': return failed + parts = id.split('.') + # Protection for derived debuggers + if parts[0] == 'self': + del parts[0] + if len(parts) == 0: + return failed + # Best first guess at file to look at + fname = self.defaultFile() + if len(parts) == 1: + item = parts[0] + else: + # More than one part. + # First is module, second is method/class + f = self.lookupmodule(parts[0]) + if f: + fname = f + item = parts[1] + answer = find_function(item, fname) + return answer or failed + + def defaultFile(self): + """Produce a reasonable default.""" + filename = self.curframe.f_code.co_filename + # Consider using is_exec_stmt(). I just don't understand + # the conditions under which the below test is true. + if filename == '' and self.mainpyfile: + filename = self.mainpyfile + return filename + + def checkline(self, filename, lineno): + """Check whether specified line seems to be executable. + + Return `lineno` if it is, 0 if not (e.g. a docstring, comment, blank + line or EOF). Warning: testing is not comprehensive. + """ + line = linecache.getline(filename, lineno) + if not line: + print >>self.out, 'End of file' + return 0 + line = line.strip() + # Don't allow setting breakpoint at a blank line + if (not line or (line[0] == '#') or + (line[:3] == '"""') or line[:3] == "'''"): + print >>self.out, '*** Blank or comment' + return 0 + return lineno def trace_dispatch(self, frame, event, arg): + self.curframe = frame if event == 'line': print >> self.out, self.thread.getName(),'*** line' return self.trace_dispatch @@ -79,12 +272,3 @@ return self.trace_dispatch print 'bdb.Bdb.dispatch: unknown debugging event:', repr(event) return self.trace_dispatch - - - - - - - - - Modified: sandbox/trunk/pdb/test/test_mpdb.py ============================================================================== --- sandbox/trunk/pdb/test/test_mpdb.py (original) +++ sandbox/trunk/pdb/test/test_mpdb.py Wed Jun 28 14:48:52 2006 @@ -100,8 +100,6 @@ self.assertEquals(errmsg, line) - server.disconnect() - def testRebindOutput(self): """ Test rebinding output. """ self.server = MPdb() @@ -138,28 +136,38 @@ self.client1 = MPdbTest() connect_to_target(self.client1) + + # Turn on thread debugging + self.client1.onecmd('set thread') + line = self.client1.lines[0] + self.assertEquals('Thread debugging on\n', line) + # Thread with no commands should return current thread self.client1.onecmd('thread') - assert 'MainThread' in self.client1.lines[0] + assert 'MainThread' in self.client1.lines[1] # 'thread apply' without thread ID should return an error message self.client1.onecmd('thread apply') - line = self.client1.lines[1] - self.assertEquals('*** Please specify a thread ID\n', line) + line = self.client1.lines[2] + errmsg = '*** Please specify a Thread ID\n' + self.assertEquals(errmsg, line) # Need a command to actually apply to a thread self.client1.onecmd('thread apply 49843') - line = self.client1.lines[2] + line = self.client1.lines[3] errmsg = '*** Please specify a command following the thread ID\n' self.assertEquals(errmsg, line) # We've still not started any threads self.client1.onecmd('thread apply 2 info break') - line = self.client1.lines[3] - errmsg = '*** No threads\n' + line = self.client1.lines[4] + errmsg = '*** Thread ID 2 not known.\n' self.assertEquals(errmsg, line) - - server.disconnect() + + self.client1.onecmd('thread') + line = self.client1.lines[5] + msg = 'Current thread is 1 (<_MainThread(MainThread, started)>)\n' + self.assertEquals(msg, line) def test_main(): test_support.run_unittest(TestRemoteDebugging) From python-checkins at python.org Wed Jun 28 16:25:21 2006 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 28 Jun 2006 16:25:21 +0200 (CEST) Subject: [Python-checkins] r47147 - python/trunk/Doc/lib/libuuid.tex Message-ID: <20060628142521.3AA6A1E4015@bag.python.org> Author: andrew.kuchling Date: Wed Jun 28 16:25:20 2006 New Revision: 47147 Added: python/trunk/Doc/lib/libuuid.tex Log: [Bug #1508766] Add docs for uuid module; docs written by George Yoshida, with minor rearrangements by me. Added: python/trunk/Doc/lib/libuuid.tex ============================================================================== --- (empty file) +++ python/trunk/Doc/lib/libuuid.tex Wed Jun 28 16:25:20 2006 @@ -0,0 +1,231 @@ +\section{\module{uuid} --- + UUID objects according to RFC 4122} +\declaremodule{builtin}{uuid} +\modulesynopsis{UUID objects (universally unique identifiers) according to RFC 4122} +\moduleauthor{Ka-Ping Yee}{ping at zesty.ca} +\sectionauthor{George Yoshida}{quiver at users.sourceforge.net} + +\versionadded{2.5} + +This module provides immutable \class{UUID} objects (the \class{UUID} class) +and the functions \function{uuid1()}, \function{uuid3()}, +\function{uuid4()}, \function{uuid5()} for generating version 1, 3, 4, +and 5 UUIDs as specified in \rfc{4122}. + +If all you want is a unique ID, you should probably call +\function{uuid1()} or \function{uuid4()}. Note that \function{uuid1()} +may compromise privacy since it creates a UUID containing the computer's +network address. \function{uuid4()} creates a random UUID. + +\begin{classdesc}{UUID}{\optional{hex\optional{, bytes\optional{, +fields\optional{, int\optional{, version}}}}}} + +%Instances of the UUID class represent UUIDs as specified in RFC 4122. +%UUID objects are immutable, hashable, and usable as dictionary keys. +%Converting a UUID to a string with str() yields something in the form +%'12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts +%four possible forms: a similar string of hexadecimal digits, or a +%string of 16 raw bytes as an argument named 'bytes', or a tuple of +%six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and +%48-bit values respectively) as an argument named 'fields', or a single +%128-bit integer as an argument named 'int'. + +Create a UUID from either a string of 32 hexadecimal digits, +a string of 16 bytes as the \var{bytes} argument, a tuple of six +integers (32-bit \var{time_low}, 16-bit \var{time_mid}, +16-bit \var{time_hi_version}, +8-bit \var{clock_seq_hi_variant}, 8-bit \var{clock_seq_low}, 48-bit \var{node}) +as the \var{fields} argument, or a single 128-bit integer as the \var{int} +argument. When a string of hex digits is given, curly braces, +hyphens, and a URN prefix are all optional. For example, these +expressions all yield the same UUID: + +\begin{verbatim} +UUID('{12345678-1234-5678-1234-567812345678}') +UUID('12345678123456781234567812345678') +UUID('urn:uuid:12345678-1234-5678-1234-567812345678') +UUID(bytes='\x12\x34\x56\x78'*4) +UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) +UUID(int=0x12345678123456781234567812345678) +\end{verbatim} + +Exactly one of \var{hex}, \var{bytes}, \var{fields}, or \var{int} must +be given. The \var{version} argument is optional; if given, the +resulting UUID will have its variant and version number set according to +RFC 4122, overriding bits in the given \var{hex}, \var{bytes}, +\var{fields}, or \var{int}. + +\end{classdesc} + +\class{UUID} instances have these read-only attributes: + +\begin{memberdesc}{bytes} +The UUID as a 16-byte string. +\end{memberdesc} + +\begin{memberdesc}{fields} +A tuple of the six integer fields of the UUID, which are also available +as six individual attributes and two derived attributes: + +\begin{tableii}{l|l}{member}{Field}{Meaning} + \lineii{time_low}{the first 32 bits of the UUID} + \lineii{time_mid}{the next 16 bits of the UUID} + \lineii{time_hi_version}{the next 16 bits of the UUID} + \lineii{clock_seq_hi_variant}{the next 8 bits of the UUID} + \lineii{clock_seq_low}{the next 8 bits of the UUID} + \lineii{node}{the last 48 bits of the UUID} + \lineii{time}{the 60-bit timestamp} + \lineii{clock_seq}{the 14-bit sequence number} +\end{tableii} + + +\end{memberdesc} + +\begin{memberdesc}{hex} +The UUID as a 32-character hexadecimal string. +\end{memberdesc} + +\begin{memberdesc}{int} +The UUID as a 128-bit integer. +\end{memberdesc} + +\begin{memberdesc}{urn} +The UUID as a URN as specified in RFC 4122. +\end{memberdesc} + +\begin{memberdesc}{variant} +The UUID variant, which determines the internal layout of the UUID. +This will be an integer equal to one of the constants +\constant{RESERVED_NCS}, +\constant{RFC_4122}, \constant{RESERVED_MICROSOFT}, or +\constant{RESERVED_FUTURE}). +\end{memberdesc} + +\begin{memberdesc}{version} +The UUID version number (1 through 5, meaningful only +when the variant is \constant{RFC_4122}). +\end{memberdesc} + +The \module{uuid} module defines the following functions + +\begin{funcdesc}{getnode}{} +Get the hardware address as a 48-bit integer. The first time this runs, +it may launch a separate program, which could be quite slow. If all +attempts to obtain the hardware address fail, we choose a random 48-bit +number with its eighth bit set to 1 as recommended in RFC 4122. +\end{funcdesc} +\index{getnode} + +\begin{funcdesc}{uuid1}{\optional{node\optional{, clock_seq}}} +Generate a UUID from a host ID, sequence number, and the current time. +If \var{node} is not given, \function{getnode()} is used to obtain the +hardware address. +If \var{clock_seq} is given, it is used as the sequence number; +otherwise a random 14-bit sequence number is chosen. +\end{funcdesc} +\index{uuid1} + +\begin{funcdesc}{uuid3}{namespace, name} +Generate a UUID based upon a MD5 hash of the \var{name} string value +drawn from a specified namespace. \var{namespace} +must be one of \constant{NAMESPACE_DNS}, +\constant{NAMESPACE_URL}, \constant{NAMESPACE_OID}, +or \constant{NAMESPACE_X500}. +\end{funcdesc} +\index{uuid3} + +\begin{funcdesc}{uuid4}{} +Generate a random UUID. +\end{funcdesc} +\index{uuid4} + +\begin{funcdesc}{uuid5}{namespace, name} +Generate a UUID based upon a SHA-1 hash of the \var{name} string value +drawn from a specified namespace. \var{namespace} +must be one of \constant{NAMESPACE_DNS}, +\constant{NAMESPACE_URL}, \constant{NAMESPACE_OID}, +or \constant{NAMESPACE_X500}. +\end{funcdesc} +\index{uuid5} + +The \module{uuid} module defines the following namespace constants +for use with \function{uuid3()} or \function{uuid5()}. + +\begin{datadesc}{NAMESPACE_DNS} +Fully-qualified domain name namespace UUID. +\end{datadesc} + +\begin{datadesc}{NAMESPACE_URL} +URL namespace UUID. +\end{datadesc} + +\begin{datadesc}{NAMESPACE_OID} +ISO OID namespace UUID. +\end{datadesc} + +\begin{datadesc}{NAMESPACE_X500} +X.500 DN namespace UUID. +\end{datadesc} + +The \module{uuid} module defines the following constants +for the possible values of the \member{variant} attribute: + +\begin{datadesc}{RESERVED_NCS} +Reserved for NCS compatibility. +\end{datadesc} + +\begin{datadesc}{RFC_4122} +Uses UUID layout specified in \rfc{4122}. +\end{datadesc} + +\begin{datadesc}{RESERVED_MICROSOFT} +Reserved for Microsoft backward compatibility. +\end{datadesc} + +\begin{datadesc}{RESERVED_FUTURE} +Reserved for future definition. +\end{datadesc} + + +\begin{seealso} + \seerfc{4122}{A Universally Unique IDentifier (UUID) URN Namespace}{ + This specifies a Uniform Resource Name namespace for UUIDs.} +\end{seealso} + +\subsection{Example \label{uuid-example}} + +Here is a typical usage: +\begin{verbatim} +>>> import uuid + +# make a UUID based on the host ID and current time +>>> uuid.uuid1() +UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + +# make a UUID using an MD5 hash of a namespace UUID and a name +>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') +UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + +# make a random UUID +>>> uuid.uuid4() +UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + +# make a UUID using a SHA-1 hash of a namespace UUID and a name +>>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') +UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + +# make a UUID from a string of hex digits (braces and hyphens ignored) +>>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') + +# convert a UUID to a string of hex digits in standard form +>>> str(x) +'00010203-0405-0607-0809-0a0b0c0d0e0f' + +# get the raw 16 bytes of the UUID +>>> x.bytes +'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + +# make a UUID from a 16-byte string +>>> uuid.UUID(bytes=x.bytes) +UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') +\end{verbatim} From python-checkins at python.org Wed Jun 28 16:27:21 2006 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 28 Jun 2006 16:27:21 +0200 (CEST) Subject: [Python-checkins] r47148 - python/trunk/Doc/lib/lib.tex Message-ID: <20060628142721.78AB71E4005@bag.python.org> Author: andrew.kuchling Date: Wed Jun 28 16:27:21 2006 New Revision: 47148 Modified: python/trunk/Doc/lib/lib.tex Log: [Bug #1508766] Add docs for uuid module; this puts the module in the 'Internet Protocols' section. Arguably this module could also have gone in the chapters on strings or encodings, maybe even the crypto chapter. Fred, please move if you see fit. Modified: python/trunk/Doc/lib/lib.tex ============================================================================== --- python/trunk/Doc/lib/lib.tex (original) +++ python/trunk/Doc/lib/lib.tex Wed Jun 28 16:27:21 2006 @@ -303,6 +303,7 @@ \input{libsmtplib} \input{libsmtpd} \input{libtelnetlib} +\input{libuuid} \input{liburlparse} \input{libsocksvr} \input{libbasehttp} From python-checkins at python.org Wed Jun 28 16:41:23 2006 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 28 Jun 2006 16:41:23 +0200 (CEST) Subject: [Python-checkins] r47149 - peps/trunk/pep-3103.txt Message-ID: <20060628144123.EDDB51E4005@bag.python.org> Author: guido.van.rossum Date: Wed Jun 28 16:41:23 2006 New Revision: 47149 Modified: peps/trunk/pep-3103.txt Log: Introducing school IIb -- duplicate cases resolved by case order, not errors. Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Wed Jun 28 16:41:23 2006 @@ -390,10 +390,38 @@ unflagged, when the dict-based dispatch implementation makes it so easy to trap this. +However, there are some use cases for overlapping/duplicate cases. +Suppose you're switching on some OS-specific constants (e.g. exported +by the os module or some module like that). You have a case for each. +But on some OS, two different constants have the same value (since on +that OS they are implemented the same way -- like O_TEXT and O_BINARY +on Unix). If duplicate cases are flagged as errors, your switch +wouldn't work at all on that OS. It would be much better if you could +arrange the cases so that one case has preference over another. + +There's also the (more likely) use case where you have a set of cases +to be treated the same, but one member of the set must be treated +differently. It would be convenient to put the exception in an +earlier case and be done with it. + +(Yes, it seems a shame not to be able to diagnose dead code due to +accidental case duplication. Maybe that's less important, and +pychecker can deal with it? After all we don't diagnose duplicate +method definitions either.) + +This suggests school IIb: like school II but redundant cases must be +resolved by choosing the first match. This is trivial to implement +when building the dispatch dict (skip keys already present). + +(An alternative would be to introduce new syntax to indicate "okay to +have overlapping cases" or "ok if this case is dead code" but I find +that overkill.) + Personally, I'm in school II: I believe that the dict-based dispatch is the one true implementation for switch statements and that we should face the limitiations up front, so that we can reap maximal -benefits. +benefits. I'm leaning towards school IIb -- duplicate cases should be +resolved by the ordering of the cases instead of flagged as errors. When to Freeze the Dispatch Dict -------------------------------- From anthony at interlink.com.au Wed Jun 28 17:20:04 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 29 Jun 2006 01:20:04 +1000 Subject: [Python-checkins] r47142 - in python/trunk: Doc/lib/librunpy.tex Lib/runpy.py Lib/test/test_runpy.py In-Reply-To: <20060628104148.362A01E4004@bag.python.org> References: <20060628104148.362A01E4004@bag.python.org> Message-ID: <200606290120.06338.anthony@interlink.com.au> On Wednesday 28 June 2006 20:41, nick.coghlan wrote: > Author: nick.coghlan > Date: Wed Jun 28 12:41:47 2006 > New Revision: 47142 > > Modified: > python/trunk/Doc/lib/librunpy.tex > python/trunk/Lib/runpy.py > python/trunk/Lib/test/test_runpy.py > Log: > Make full module name available as __module_name__ even when > __name__ is set to something else (like '__main__') Er. Um. Feature freeze? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From python-checkins at python.org Wed Jun 28 18:26:17 2006 From: python-checkins at python.org (trent.mick) Date: Wed, 28 Jun 2006 18:26:17 +0200 (CEST) Subject: [Python-checkins] r47150 - peps/trunk/pep-3103.txt Message-ID: <20060628162617.D594C1E4005@bag.python.org> Author: trent.mick Date: Wed Jun 28 18:26:17 2006 New Revision: 47150 Modified: peps/trunk/pep-3103.txt Log: I presume that this was the intent of this sentence. Modified: peps/trunk/pep-3103.txt ============================================================================== --- peps/trunk/pep-3103.txt (original) +++ peps/trunk/pep-3103.txt Wed Jun 28 18:26:17 2006 @@ -127,7 +127,7 @@ else: SUITE -The hope of this alternative is that is will upset the auto-indent +The hope of this alternative is that it will not upset the auto-indent logic of the average Python-aware text editor less. But it looks strange to me. From python-checkins at python.org Wed Jun 28 22:23:26 2006 From: python-checkins at python.org (georg.brandl) Date: Wed, 28 Jun 2006 22:23:26 +0200 (CEST) Subject: [Python-checkins] r47151 - in python/trunk: Lib/lib-tk/turtle.py Misc/NEWS Message-ID: <20060628202326.945531E4005@bag.python.org> Author: georg.brandl Date: Wed Jun 28 22:23:25 2006 New Revision: 47151 Modified: python/trunk/Lib/lib-tk/turtle.py python/trunk/Misc/NEWS Log: Fix end_fill(). Modified: python/trunk/Lib/lib-tk/turtle.py ============================================================================== --- python/trunk/Lib/lib-tk/turtle.py (original) +++ python/trunk/Lib/lib-tk/turtle.py Wed Jun 28 22:23:25 2006 @@ -721,7 +721,7 @@ def write(arg, move=0): _getpen().write(arg, move) def fill(flag): _getpen().fill(flag) def begin_fill(): _getpen().begin_fill() -def end_fill(): _getpen.end_fill() +def end_fill(): _getpen().end_fill() def circle(radius, extent=None): _getpen().circle(radius, extent) def goto(*args): _getpen().goto(*args) def heading(): return _getpen().heading() Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 28 22:23:25 2006 @@ -19,6 +19,8 @@ Library ------- +- Fix a bug in the turtle module's end_fill function. + - Bug #1510580: The 'warnings' module improperly required that a Warning category be either a types.ClassType and a subclass of Warning. The proper check is just that it is a subclass with Warning as the documentation states. From python-checkins at python.org Wed Jun 28 22:28:57 2006 From: python-checkins at python.org (trent.mick) Date: Wed, 28 Jun 2006 22:28:57 +0200 (CEST) Subject: [Python-checkins] r47152 - peps/trunk/pep-0356.txt Message-ID: <20060628202857.475B51E4005@bag.python.org> Author: trent.mick Date: Wed Jun 28 22:28:56 2006 New Revision: 47152 Modified: peps/trunk/pep-0356.txt Log: The expat 2.0 upgrade was done for 2.5b1 Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Wed Jun 28 22:28:56 2006 @@ -105,6 +105,9 @@ support naive introspection (e.g. having f.__name__ return the original function name). + - Upgrade pyexpat to use expat 2.0. + + Possible features for 2.5 Each feature below should implemented prior to beta1 or @@ -112,9 +115,6 @@ - Modules under consideration for inclusion: - - Upgrade pyexpat to use expat 2.0? - (Patch by Trent Mick, #1462338) - - Add new icons for MacOS and Unix with the new Python logo? (Owner: ???) MacOS: http://hcs.harvard.edu/~jrus/python/prettified-py-icons.png From python-checkins at python.org Wed Jun 28 22:30:41 2006 From: python-checkins at python.org (trent.mick) Date: Wed, 28 Jun 2006 22:30:41 +0200 (CEST) Subject: [Python-checkins] r47153 - python/trunk/Misc/NEWS Message-ID: <20060628203041.AC7D31E4008@bag.python.org> Author: trent.mick Date: Wed Jun 28 22:30:41 2006 New Revision: 47153 Modified: python/trunk/Misc/NEWS Log: Mention the expat upgrade and pyexpat fix I put in 2.5b1. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Jun 28 22:30:41 2006 @@ -141,6 +141,10 @@ Extension Modules ----------------- +- Bug #1295808: expat symbols should be namespaced in pyexpat + +- Patch #1462338: Upgrade pyexpat to expat 2.0.0 + - Change binascii.hexlify to accept a read-only buffer instead of only a char buffer and actually follow its documentation. From buildbot at python.org Thu Jun 29 01:50:41 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 28 Jun 2006 23:50:41 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060628235041.C94D91E4013@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/252 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,georg.brandl Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Thu Jun 29 02:51:54 2006 From: python-checkins at python.org (fred.drake) Date: Thu, 29 Jun 2006 02:51:54 +0200 (CEST) Subject: [Python-checkins] r47154 - in python/trunk/Lib: sgmllib.py test/test_sgmllib.py Message-ID: <20060629005154.1F1B71E4019@bag.python.org> Author: fred.drake Date: Thu Jun 29 02:51:53 2006 New Revision: 47154 Modified: python/trunk/Lib/sgmllib.py python/trunk/Lib/test/test_sgmllib.py Log: SF bug #1504333: sgmlib should allow angle brackets in quoted values (modified patch by Sam Ruby; changed to use separate REs for start and end tags to reduce matching cost for end tags; extended tests; updated to avoid breaking previous changes to support IPv6 addresses in unquoted attribute values) Modified: python/trunk/Lib/sgmllib.py ============================================================================== --- python/trunk/Lib/sgmllib.py (original) +++ python/trunk/Lib/sgmllib.py Thu Jun 29 02:51:53 2006 @@ -29,7 +29,12 @@ shorttagopen = re.compile('<[a-zA-Z][-.a-zA-Z0-9]*/') shorttag = re.compile('<([a-zA-Z][-.a-zA-Z0-9]*)/([^/]*)/') piclose = re.compile('>') -endbracket = re.compile('[<>]') +starttag = re.compile(r'<[a-zA-Z][-_.:a-zA-Z0-9]*\s*(' + r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' + r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$_#=~@]' + r'[][\-a-zA-Z0-9./,:;+*%?!&$_#=~\'"@]*(?=[\s>/<])))?' + r')*\s*/?\s*(?=[<>])') +endtag = re.compile(r'])') tagfind = re.compile('[a-zA-Z][-_.a-zA-Z0-9]*') attrfind = re.compile( r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' @@ -249,14 +254,10 @@ self.finish_shorttag(tag, data) self.__starttag_text = rawdata[start_pos:match.end(1) + 1] return k - # XXX The following should skip matching quotes (' or ") - # As a shortcut way to exit, this isn't so bad, but shouldn't - # be used to locate the actual end of the start tag since the - # < or > characters may be embedded in an attribute value. - match = endbracket.search(rawdata, i+1) + match = starttag.match(rawdata, i) if not match: return -1 - j = match.start(0) + j = match.end(0) # Now parse the data between i+1 and j into a tag and attrs attrs = [] if rawdata[i:i+2] == '<>': @@ -305,10 +306,10 @@ # Internal -- parse endtag def parse_endtag(self, i): rawdata = self.rawdata - match = endbracket.search(rawdata, i+1) + match = endtag.match(rawdata, i) if not match: return -1 - j = match.start(0) + j = match.end(0) tag = rawdata[i+2:j].strip().lower() if rawdata[j] == '>': j = j+1 Modified: python/trunk/Lib/test/test_sgmllib.py ============================================================================== --- python/trunk/Lib/test/test_sgmllib.py (original) +++ python/trunk/Lib/test/test_sgmllib.py Thu Jun 29 02:51:53 2006 @@ -286,6 +286,21 @@ ('codepoint', 'convert', 42), ]) + def test_attr_values_quoted_markup(self): + """Multi-line and markup in attribute values""" + self.check_events("""text""", + [("starttag", "a", [("title", "foo\n
bar")]), + ("data", "text"), + ("endtag", "a")]) + self.check_events("""text""", + [("starttag", "a", [("title", "less < than")]), + ("data", "text"), + ("endtag", "a")]) + self.check_events("""text""", + [("starttag", "a", [("title", "greater > than")]), + ("data", "text"), + ("endtag", "a")]) + def test_attr_funky_names(self): self.check_events("""""", [ ("starttag", "a", [("a.b", "v"), ("c:d", "v"), ("e-f", "v")]), From python-checkins at python.org Thu Jun 29 02:52:33 2006 From: python-checkins at python.org (fred.drake) Date: Thu, 29 Jun 2006 02:52:33 +0200 (CEST) Subject: [Python-checkins] r47155 - in python/branches/release24-maint/Lib: sgmllib.py test/test_sgmllib.py Message-ID: <20060629005233.1332F1E400E@bag.python.org> Author: fred.drake Date: Thu Jun 29 02:52:32 2006 New Revision: 47155 Modified: python/branches/release24-maint/Lib/sgmllib.py python/branches/release24-maint/Lib/test/test_sgmllib.py Log: SF bug #1504333: sgmlib should allow angle brackets in quoted values (modified patch by Sam Ruby; changed to use separate REs for start and end tags to reduce matching cost for end tags; extended tests; updated to avoid breaking previous changes to support IPv6 addresses in unquoted attribute values) Modified: python/branches/release24-maint/Lib/sgmllib.py ============================================================================== --- python/branches/release24-maint/Lib/sgmllib.py (original) +++ python/branches/release24-maint/Lib/sgmllib.py Thu Jun 29 02:52:32 2006 @@ -29,7 +29,12 @@ shorttagopen = re.compile('<[a-zA-Z][-.a-zA-Z0-9]*/') shorttag = re.compile('<([a-zA-Z][-.a-zA-Z0-9]*)/([^/]*)/') piclose = re.compile('>') -endbracket = re.compile('[<>]') +starttag = re.compile(r'<[a-zA-Z][-_.:a-zA-Z0-9]*\s*(' + r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' + r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$_#=~@]' + r'[][\-a-zA-Z0-9./,:;+*%?!&$_#=~\'"@]*(?=[\s>/<])))?' + r')*\s*/?\s*(?=[<>])') +endtag = re.compile(r'])') tagfind = re.compile('[a-zA-Z][-_.a-zA-Z0-9]*') attrfind = re.compile( r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' @@ -245,11 +250,10 @@ self.finish_shorttag(tag, data) self.__starttag_text = rawdata[start_pos:match.end(1) + 1] return k - # XXX The following should skip matching quotes (' or ") - match = endbracket.search(rawdata, i+1) + match = starttag.match(rawdata, i) if not match: return -1 - j = match.start(0) + j = match.end(0) # Now parse the data between i+1 and j into a tag and attrs attrs = [] if rawdata[i:i+2] == '<>': @@ -283,10 +287,10 @@ # Internal -- parse endtag def parse_endtag(self, i): rawdata = self.rawdata - match = endbracket.search(rawdata, i+1) + match = endtag.match(rawdata, i) if not match: return -1 - j = match.start(0) + j = match.end(0) tag = rawdata[i+2:j].strip().lower() if rawdata[j] == '>': j = j+1 Modified: python/branches/release24-maint/Lib/test/test_sgmllib.py ============================================================================== --- python/branches/release24-maint/Lib/test/test_sgmllib.py (original) +++ python/branches/release24-maint/Lib/test/test_sgmllib.py Thu Jun 29 02:52:32 2006 @@ -214,6 +214,21 @@ ("starttag", "e", [("a", "rgb(1,2,3)")]), ]) + def test_attr_values_quoted_markup(self): + """Multi-line and markup in attribute values""" + self.check_events("""text""", + [("starttag", "a", [("title", "foo\n
bar")]), + ("data", "text"), + ("endtag", "a")]) + self.check_events("""text""", + [("starttag", "a", [("title", "less < than")]), + ("data", "text"), + ("endtag", "a")]) + self.check_events("""text""", + [("starttag", "a", [("title", "greater > than")]), + ("data", "text"), + ("endtag", "a")]) + def test_attr_funky_names(self): self.check_events("""""", [ ("starttag", "a", [("a.b", "v"), ("c:d", "v"), ("e-f", "v")]), From buildbot at python.org Thu Jun 29 03:07:49 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 29 Jun 2006 01:07:49 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian 2.4 Message-ID: <20060629010749.ABD341E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%25202.4/builds/36 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: fred.drake BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From python-checkins at python.org Thu Jun 29 04:57:49 2006 From: python-checkins at python.org (fred.drake) Date: Thu, 29 Jun 2006 04:57:49 +0200 (CEST) Subject: [Python-checkins] r47156 - python/trunk/Misc/NEWS Message-ID: <20060629025749.7479B1E4005@bag.python.org> Author: fred.drake Date: Thu Jun 29 04:57:48 2006 New Revision: 47156 Modified: python/trunk/Misc/NEWS Log: document recent bugfixes in sgmllib Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 29 04:57:48 2006 @@ -19,6 +19,12 @@ Library ------- +- Bug #1504333: Make sgmllib support angle brackets in quoted + attribute values. + +- Bug #853506: Fix IPv6 address parsing in unquoted attributes in + sgmllib ('[' and ']' were not accepted). + - Fix a bug in the turtle module's end_fill function. - Bug #1510580: The 'warnings' module improperly required that a Warning From python-checkins at python.org Thu Jun 29 04:58:05 2006 From: python-checkins at python.org (fred.drake) Date: Thu, 29 Jun 2006 04:58:05 +0200 (CEST) Subject: [Python-checkins] r47157 - python/branches/release24-maint/Misc/NEWS Message-ID: <20060629025805.BAC071E4005@bag.python.org> Author: fred.drake Date: Thu Jun 29 04:58:04 2006 New Revision: 47157 Modified: python/branches/release24-maint/Misc/NEWS Log: document recent bugfixes in sgmllib Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Thu Jun 29 04:58:04 2006 @@ -56,6 +56,12 @@ Library ------- +- Bug #1504333: Make sgmllib support angle brackets in quoted attribute + values. + +- Bug #853506: Fix IPv6 address parsing in unquoted attributes in sgmllib + ('[' and ']' were not accepted). + - Bug #1117556: SimpleHTTPServer now tries to find and use the system's mime.types file for determining MIME types. From buildbot at python.org Thu Jun 29 06:03:42 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 29 Jun 2006 04:03:42 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian 2.4 Message-ID: <20060629040343.0E67D1E4012@bag.python.org> The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/71 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: fred.drake Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Thu Jun 29 06:10:11 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 29 Jun 2006 06:10:11 +0200 (CEST) Subject: [Python-checkins] r47158 - in python/trunk: Lib/test/test_bz2.py Lib/test/test_cmd_line.py Lib/test/test_commands.py Lib/test/test_fork1.py Lib/test/test_mailbox.py Lib/test/test_popen.py Lib/test/test_popen2.py Lib/test/test_select.py Lib/test/test_socketserver.py Lib/test/test_subprocess.py Lib/test/test_support.py Lib/test/test_wait3.py Lib/test/test_wait4.py Misc/NEWS Message-ID: <20060629041011.B41681E4005@bag.python.org> Author: neal.norwitz Date: Thu Jun 29 06:10:08 2006 New Revision: 47158 Modified: python/trunk/Lib/test/test_bz2.py python/trunk/Lib/test/test_cmd_line.py python/trunk/Lib/test/test_commands.py python/trunk/Lib/test/test_fork1.py python/trunk/Lib/test/test_mailbox.py python/trunk/Lib/test/test_popen.py python/trunk/Lib/test/test_popen2.py python/trunk/Lib/test/test_select.py python/trunk/Lib/test/test_socketserver.py python/trunk/Lib/test/test_subprocess.py python/trunk/Lib/test/test_support.py python/trunk/Lib/test/test_wait3.py python/trunk/Lib/test/test_wait4.py python/trunk/Misc/NEWS Log: Add new utility function, reap_children(), to test_support. This should be called at the end of each test that spawns children (perhaps it should be called from regrtest instead?). This will hopefully prevent some of the unexplained failures in the buildbots (hppa and alpha) during tests that spawn children. The problems were not reproducible. There were many zombies that remained at the end of several tests. In the worst case, this shouldn't cause any more problems, though it may not help either. Time will tell. Modified: python/trunk/Lib/test/test_bz2.py ============================================================================== --- python/trunk/Lib/test/test_bz2.py (original) +++ python/trunk/Lib/test/test_bz2.py Thu Jun 29 06:10:08 2006 @@ -352,6 +352,7 @@ BZ2DecompressorTest, FuncTest ) + test_support.reap_children() if __name__ == '__main__': test_main() Modified: python/trunk/Lib/test/test_cmd_line.py ============================================================================== --- python/trunk/Lib/test/test_cmd_line.py (original) +++ python/trunk/Lib/test/test_cmd_line.py Thu Jun 29 06:10:08 2006 @@ -87,6 +87,7 @@ def test_main(): test.test_support.run_unittest(CmdLineTest) + test.test_support.reap_children() if __name__ == "__main__": test_main() Modified: python/trunk/Lib/test/test_commands.py ============================================================================== --- python/trunk/Lib/test/test_commands.py (original) +++ python/trunk/Lib/test/test_commands.py Thu Jun 29 06:10:08 2006 @@ -5,7 +5,7 @@ import unittest import os, tempfile, re -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children from commands import * # The module says: @@ -58,6 +58,7 @@ def test_main(): run_unittest(CommandTests) + reap_children() if __name__ == "__main__": Modified: python/trunk/Lib/test/test_fork1.py ============================================================================== --- python/trunk/Lib/test/test_fork1.py (original) +++ python/trunk/Lib/test/test_fork1.py Thu Jun 29 06:10:08 2006 @@ -3,7 +3,7 @@ import os from test.fork_wait import ForkWait -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children try: os.fork @@ -18,6 +18,7 @@ def test_main(): run_unittest(ForkTest) + reap_children() if __name__ == "__main__": test_main() Modified: python/trunk/Lib/test/test_mailbox.py ============================================================================== --- python/trunk/Lib/test/test_mailbox.py (original) +++ python/trunk/Lib/test/test_mailbox.py Thu Jun 29 06:10:08 2006 @@ -1785,6 +1785,7 @@ TestMessageConversion, TestProxyFile, TestPartialFile, MaildirTestCase) test_support.run_unittest(*tests) + test_support.reap_children() if __name__ == '__main__': Modified: python/trunk/Lib/test/test_popen.py ============================================================================== --- python/trunk/Lib/test/test_popen.py (original) +++ python/trunk/Lib/test/test_popen.py Thu Jun 29 06:10:08 2006 @@ -6,7 +6,7 @@ import os import sys -from test.test_support import TestSkipped +from test.test_support import TestSkipped, reap_children from os import popen # Test that command-lines get down as we expect. @@ -35,5 +35,6 @@ def main(): print "Test popen:" _test_commandline() + reap_children() main() Modified: python/trunk/Lib/test/test_popen2.py ============================================================================== --- python/trunk/Lib/test/test_popen2.py (original) +++ python/trunk/Lib/test/test_popen2.py Thu Jun 29 06:10:08 2006 @@ -5,7 +5,7 @@ import os import sys -from test.test_support import TestSkipped +from test.test_support import TestSkipped, reap_children # popen2 contains its own testing routine # which is especially useful to see if open files @@ -75,3 +75,4 @@ main() _test() +reap_children() Modified: python/trunk/Lib/test/test_select.py ============================================================================== --- python/trunk/Lib/test/test_select.py (original) +++ python/trunk/Lib/test/test_select.py Thu Jun 29 06:10:08 2006 @@ -1,5 +1,5 @@ # Testing select module -from test.test_support import verbose +from test.test_support import verbose, reap_children import select import os @@ -65,5 +65,6 @@ continue print 'Unexpected return values from select():', rfd, wfd, xfd p.close() + reap_children() test() Modified: python/trunk/Lib/test/test_socketserver.py ============================================================================== --- python/trunk/Lib/test/test_socketserver.py (original) +++ python/trunk/Lib/test/test_socketserver.py Thu Jun 29 06:10:08 2006 @@ -1,7 +1,8 @@ # Test suite for SocketServer.py from test import test_support -from test.test_support import verbose, verify, TESTFN, TestSkipped +from test.test_support import (verbose, verify, TESTFN, TestSkipped, + reap_children) test_support.requires('network') from SocketServer import * @@ -199,6 +200,7 @@ testall() finally: cleanup() + reap_children() if __name__ == "__main__": test_main() Modified: python/trunk/Lib/test/test_subprocess.py ============================================================================== --- python/trunk/Lib/test/test_subprocess.py (original) +++ python/trunk/Lib/test/test_subprocess.py Thu Jun 29 06:10:08 2006 @@ -27,6 +27,16 @@ return re.sub(r"\[\d+ refs\]\r?\n?$", "", stderr) class ProcessTestCase(unittest.TestCase): + def setUp(self): + # Try to minimize the number of children we have so this test + # doesn't crash on some buildbots (Alphas in particular). + test_support.reap_children() + + def tearDown(self): + # Try to minimize the number of children we have so this test + # doesn't crash on some buildbots (Alphas in particular). + test_support.reap_children() + def mkstemp(self): """wrapper for mkstemp, calling mktemp if mkstemp is not available""" if hasattr(tempfile, "mkstemp"): @@ -600,6 +610,7 @@ def test_main(): test_support.run_unittest(ProcessTestCase) + test_support.reap_children() if __name__ == "__main__": test_main() Modified: python/trunk/Lib/test/test_support.py ============================================================================== --- python/trunk/Lib/test/test_support.py (original) +++ python/trunk/Lib/test/test_support.py Thu Jun 29 06:10:08 2006 @@ -475,3 +475,24 @@ while len(threading._limbo) != num_limbo and count < _MAX_COUNT: count += 1 time.sleep(0.1) + +def reap_children(): + """Use this function at the end of test_main() whenever sub-processes + are started. This will help ensure that no extra children (zombies) + stick around to hog resources and create problems when looking + for refleaks. + """ + + # Reap all our dead child processes so we don't leave zombies around. + # These hog resources and might be causing some of the buildbots to die. + import os + if hasattr(os, 'waitpid'): + any_process = -1 + while True: + try: + # This will raise an exception on Windows. That's ok. + pid, status = os.waitpid(any_process, os.WNOHANG) + if pid == 0: + break + except: + break Modified: python/trunk/Lib/test/test_wait3.py ============================================================================== --- python/trunk/Lib/test/test_wait3.py (original) +++ python/trunk/Lib/test/test_wait3.py Thu Jun 29 06:10:08 2006 @@ -3,7 +3,7 @@ import os from test.fork_wait import ForkWait -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children try: os.fork @@ -27,6 +27,7 @@ def test_main(): run_unittest(Wait3Test) + reap_children() if __name__ == "__main__": test_main() Modified: python/trunk/Lib/test/test_wait4.py ============================================================================== --- python/trunk/Lib/test/test_wait4.py (original) +++ python/trunk/Lib/test/test_wait4.py Thu Jun 29 06:10:08 2006 @@ -3,7 +3,7 @@ import os from test.fork_wait import ForkWait -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children try: os.fork @@ -24,6 +24,7 @@ def test_main(): run_unittest(Wait4Test) + reap_children() if __name__ == "__main__": test_main() Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 29 06:10:08 2006 @@ -48,6 +48,12 @@ - Bug #1513032: 'make install' failed on FreeBSD 5.3 due to lib-old trying to be installed even though it's empty. +Tests +----- + +- Call os.waitpid() at the end of tests that spawn child processes in order + to minimize resources (zombies). + What's New in Python 2.5 beta 1? ================================ From buildbot at python.org Thu Jun 29 06:48:54 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 29 Jun 2006 04:48:54 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060629044854.528A71E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/254 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake,neal.norwitz Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Thu Jun 29 07:48:15 2006 From: python-checkins at python.org (neal.norwitz) Date: Thu, 29 Jun 2006 07:48:15 +0200 (CEST) Subject: [Python-checkins] r47159 - python/trunk/Lib/test/test_socket_ssl.py Message-ID: <20060629054815.33EF81E4005@bag.python.org> Author: neal.norwitz Date: Thu Jun 29 07:48:14 2006 New Revision: 47159 Modified: python/trunk/Lib/test/test_socket_ssl.py Log: This should fix the buildbot failure on s/390 which can't connect to gmail.org. It makes the error message consistent and always sends to stderr. It would be much better for all the networking tests to hit only python.org. Modified: python/trunk/Lib/test/test_socket_ssl.py ============================================================================== --- python/trunk/Lib/test/test_socket_ssl.py (original) +++ python/trunk/Lib/test/test_socket_ssl.py Thu Jun 29 07:48:14 2006 @@ -34,6 +34,13 @@ def test_timeout(): test_support.requires('network') + def error_msg(extra_msg): + print >> sys.stderr, """\ + WARNING: an attempt to connect to %r %s, in + test_timeout. That may be legitimate, but is not the outcome we hoped + for. If this message is seen often, test_timeout should be changed to + use a more reliable address.""" % (ADDR, extra_msg) + if test_support.verbose: print "test_timeout ..." @@ -49,15 +56,11 @@ try: s.connect(ADDR) except socket.timeout: - print >> sys.stderr, """\ - WARNING: an attempt to connect to %r timed out, in - test_timeout. That may be legitimate, but is not the outcome we hoped - for. If this message is seen often, test_timeout should be changed to - use a more reliable address.""" % (ADDR,) + error_msg('timed out') return except socket.error, exc: # In case connection is refused. if exc.args[0] == errno.ECONNREFUSED: - print "Connection refused when connecting to", ADDR + error_msg('was refused') return else: raise From buildbot at python.org Thu Jun 29 08:20:38 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 29 Jun 2006 06:20:38 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060629062038.ADF201E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/428 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: fred.drake,neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From ncoghlan at gmail.com Thu Jun 29 14:42:49 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 Jun 2006 22:42:49 +1000 Subject: [Python-checkins] [Python-Dev] r47142 - in python/trunk: Doc/lib/librunpy.tex Lib/runpy.py Lib/test/test_runpy.py In-Reply-To: <200606290120.06338.anthony@interlink.com.au> References: <20060628104148.362A01E4004@bag.python.org> <200606290120.06338.anthony@interlink.com.au> Message-ID: <44A3CAC9.2020409@gmail.com> Anthony Baxter wrote: > On Wednesday 28 June 2006 20:41, nick.coghlan wrote: >> Author: nick.coghlan >> Date: Wed Jun 28 12:41:47 2006 >> New Revision: 47142 >> >> Modified: >> python/trunk/Doc/lib/librunpy.tex >> python/trunk/Lib/runpy.py >> python/trunk/Lib/test/test_runpy.py >> Log: >> Make full module name available as __module_name__ even when >> __name__ is set to something else (like '__main__') > > Er. Um. Feature freeze? Sorry about that - I was trying to deal with a conflict between PEP 328 and 338 (bug 1510172) and didn't even think about the fact that this counted as a new feature. See my response to your RFC about tightening up control of the trunk - I'd really like to make these two PEPs play nicely together before beta 2. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From buildbot at python.org Thu Jun 29 15:11:30 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 29 Jun 2006 13:11:30 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20060629131130.791131E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/255 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From python-checkins at python.org Thu Jun 29 19:47:02 2006 From: python-checkins at python.org (george.yoshida) Date: Thu, 29 Jun 2006 19:47:02 +0200 (CEST) Subject: [Python-checkins] r47160 - peps/trunk/pep-0356.txt Message-ID: <20060629174702.DB29A1E4006@bag.python.org> Author: george.yoshida Date: Thu Jun 29 19:47:01 2006 New Revision: 47160 Modified: peps/trunk/pep-0356.txt Log: amk checked in uuid doc. Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Thu Jun 29 19:47:01 2006 @@ -155,7 +155,6 @@ - Missing documentation * ElementTree/cElementTree (Fredrik Lundh) Needs latex-ifcation, patch at http://python.org/sf/1504046 - * uuid (Ka-Ping Yee), patch at http://python.org/sf/1508766 - AST compiler problems (Owner: Jeremy Hylton) From python-checkins at python.org Thu Jun 29 20:34:17 2006 From: python-checkins at python.org (thomas.heller) Date: Thu, 29 Jun 2006 20:34:17 +0200 (CEST) Subject: [Python-checkins] r47161 - in python/trunk: Misc/NEWS Modules/_ctypes/_ctypes.c Modules/_ctypes/callbacks.c Modules/_ctypes/callproc.c Message-ID: <20060629183417.D533E1E4005@bag.python.org> Author: thomas.heller Date: Thu Jun 29 20:34:15 2006 New Revision: 47161 Modified: python/trunk/Misc/NEWS python/trunk/Modules/_ctypes/_ctypes.c python/trunk/Modules/_ctypes/callbacks.c python/trunk/Modules/_ctypes/callproc.c Log: Protect the thread api calls in the _ctypes extension module within #ifdef WITH_THREADS/#endif blocks. Found by Sam Rushing. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 29 20:34:15 2006 @@ -19,6 +19,9 @@ Library ------- +- The '_ctypes' extension module now works when Python is configured + with the --without-threads option. + - Bug #1504333: Make sgmllib support angle brackets in quoted attribute values. Modified: python/trunk/Modules/_ctypes/_ctypes.c ============================================================================== --- python/trunk/Modules/_ctypes/_ctypes.c (original) +++ python/trunk/Modules/_ctypes/_ctypes.c Thu Jun 29 20:34:15 2006 @@ -4555,7 +4555,9 @@ ob_type is the metatype (the 'type'), defaults to PyType_Type, tp_base is the base type, defaults to 'object' aka PyBaseObject_Type. */ +#ifdef WITH_THREADS PyEval_InitThreads(); +#endif m = Py_InitModule3("_ctypes", module_methods, module_docs); if (!m) return; Modified: python/trunk/Modules/_ctypes/callbacks.c ============================================================================== --- python/trunk/Modules/_ctypes/callbacks.c (original) +++ python/trunk/Modules/_ctypes/callbacks.c Thu Jun 29 20:34:15 2006 @@ -127,7 +127,9 @@ PyObject *result; PyObject *arglist = NULL; int nArgs; +#ifdef WITH_THREADS PyGILState_STATE state = PyGILState_Ensure(); +#endif nArgs = PySequence_Length(converters); /* Hm. What to return in case of error? @@ -235,8 +237,9 @@ Py_XDECREF(result); Done: Py_XDECREF(arglist); - +#ifdef WITH_THREADS PyGILState_Release(state); +#endif } static void closure_fcn(ffi_cif *cif, @@ -397,12 +400,18 @@ LPVOID *ppv) { long result; +#ifdef WITH_THREADS PyGILState_STATE state; +#endif LoadPython(); +#ifdef WITH_THREADS state = PyGILState_Ensure(); +#endif result = Call_GetClassObject(rclsid, riid, ppv); +#ifdef WITH_THREADS PyGILState_Release(state); +#endif return result; } @@ -454,9 +463,13 @@ STDAPI DllCanUnloadNow(void) { long result; +#ifdef WITH_THREADS PyGILState_STATE state = PyGILState_Ensure(); +#endif result = Call_CanUnloadNow(); +#ifdef WITH_THREADS PyGILState_Release(state); +#endif return result; } Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Thu Jun 29 20:34:15 2006 @@ -617,7 +617,9 @@ void *resmem, int argcount) { +#ifdef WITH_THREADS PyThreadState *_save = NULL; /* For Py_BLOCK_THREADS and Py_UNBLOCK_THREADS */ +#endif ffi_cif cif; int cc; #ifdef MS_WIN32 @@ -649,8 +651,10 @@ return -1; } +#ifdef WITH_THREADS if ((flags & FUNCFLAG_PYTHONAPI) == 0) Py_UNBLOCK_THREADS +#endif #ifdef MS_WIN32 #ifndef DONT_USE_SEH __try { @@ -667,8 +671,10 @@ } #endif #endif +#ifdef WITH_THREADS if ((flags & FUNCFLAG_PYTHONAPI) == 0) Py_BLOCK_THREADS +#endif #ifdef MS_WIN32 #ifndef DONT_USE_SEH if (dwExceptionCode) { From python-checkins at python.org Thu Jun 29 20:58:48 2006 From: python-checkins at python.org (martin.v.loewis) Date: Thu, 29 Jun 2006 20:58:48 +0200 (CEST) Subject: [Python-checkins] r47162 - in python/trunk: Misc/NEWS PCbuild/readme.txt Message-ID: <20060629185848.4075C1E400F@bag.python.org> Author: martin.v.loewis Date: Thu Jun 29 20:58:44 2006 New Revision: 47162 Modified: python/trunk/Misc/NEWS python/trunk/PCbuild/readme.txt Log: Patch #1509163: MS Toolkit Compiler no longer available Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jun 29 20:58:44 2006 @@ -57,6 +57,11 @@ - Call os.waitpid() at the end of tests that spawn child processes in order to minimize resources (zombies). +Documentation +------------- + +- Patch #1509163: MS Toolkit Compiler no longer available. + What's New in Python 2.5 beta 1? ================================ Modified: python/trunk/PCbuild/readme.txt ============================================================================== --- python/trunk/PCbuild/readme.txt (original) +++ python/trunk/PCbuild/readme.txt Thu Jun 29 20:58:44 2006 @@ -280,12 +280,21 @@ Toolkit Compiler. This provides a way of building Python using freely available software. +Note that Microsoft have withdrawn the free MS Toolkit Compiler, so this can +no longer be considered a supported option. The instructions are still +correct, but you need to already have a copy of the compiler in order to use +them. Microsoft now supply Visual C++ 2005 Express Edition for free, but this +is NOT compatible with Visual C++ 7.1 (it uses a different C runtime), and so +cannot be used to build a version of Python compatible with the standard +python.org build. If you are interested in using Visual C++ 2005 Express +Edition, however, you should look at the PCBuild8 directory. + Requirements To build Python, the following tools are required: * The Visual C++ Toolkit Compiler - from http://msdn.microsoft.com/visualc/vctoolkit2003/ + no longer available for download - see above * A recent Platform SDK from http://www.microsoft.com/downloads/details.aspx?FamilyID=484269e2-3b89-47e3-8eb7-1f2be6d7123a * The .NET 1.1 SDK From python-checkins at python.org Thu Jun 29 21:20:12 2006 From: python-checkins at python.org (skip.montanaro) Date: Thu, 29 Jun 2006 21:20:12 +0200 (CEST) Subject: [Python-checkins] r47163 - python/trunk/Doc/lib/libstdtypes.tex Message-ID: <20060629192012.EE6171E4010@bag.python.org> Author: skip.montanaro Date: Thu Jun 29 21:20:09 2006 New Revision: 47163 Modified: python/trunk/Doc/lib/libstdtypes.tex Log: add string methods to index Modified: python/trunk/Doc/lib/libstdtypes.tex ============================================================================== --- python/trunk/Doc/lib/libstdtypes.tex (original) +++ python/trunk/Doc/lib/libstdtypes.tex Thu Jun 29 21:20:09 2006 @@ -567,6 +567,7 @@ \subsubsection{String Methods \label{string-methods}} +\indexii{string}{methods} These are the string methods which both 8-bit strings and Unicode objects support: From python-checkins at python.org Fri Jun 30 02:13:09 2006 From: python-checkins at python.org (vinay.sajip) Date: Fri, 30 Jun 2006 02:13:09 +0200 (CEST) Subject: [Python-checkins] r47164 - python/trunk/Lib/logging/config.py Message-ID: <20060630001309.708BA1E4005@bag.python.org> Author: vinay.sajip Date: Fri Jun 30 02:13:08 2006 New Revision: 47164 Modified: python/trunk/Lib/logging/config.py Log: Fixed bug in fileConfig() which failed to clear logging._handlerList Modified: python/trunk/Lib/logging/config.py ============================================================================== --- python/trunk/Lib/logging/config.py (original) +++ python/trunk/Lib/logging/config.py Fri Jun 30 02:13:08 2006 @@ -79,6 +79,7 @@ logging._acquireLock() try: logging._handlers.clear() + logging._handlerList = [] # Handlers add themselves to logging._handlers handlers = _install_handlers(cp, formatters) _install_loggers(cp, handlers) From nnorwitz at gmail.com Fri Jun 30 03:21:28 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 29 Jun 2006 18:21:28 -0700 Subject: [Python-checkins] r47164 - python/trunk/Lib/logging/config.py In-Reply-To: <20060630001309.708BA1E4005@bag.python.org> References: <20060630001309.708BA1E4005@bag.python.org> Message-ID: Can we get a test and Misc/NEWS entry? On 6/29/06, vinay.sajip wrote: > Author: vinay.sajip > Date: Fri Jun 30 02:13:08 2006 > New Revision: 47164 > > Modified: > python/trunk/Lib/logging/config.py > Log: > Fixed bug in fileConfig() which failed to clear logging._handlerList > > Modified: python/trunk/Lib/logging/config.py > ============================================================================== > --- python/trunk/Lib/logging/config.py (original) > +++ python/trunk/Lib/logging/config.py Fri Jun 30 02:13:08 2006 > @@ -79,6 +79,7 @@ > logging._acquireLock() > try: > logging._handlers.clear() > + logging._handlerList = [] > # Handlers add themselves to logging._handlers > handlers = _install_handlers(cp, formatters) > _install_loggers(cp, handlers) > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From buildbot at python.org Fri Jun 30 04:00:22 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 30 Jun 2006 02:00:22 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060630020023.3A91B1E4005@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/432 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: skip.montanaro,vinay.sajip Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 30 08:09:25 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 30 Jun 2006 08:09:25 +0200 (CEST) Subject: [Python-checkins] r47165 - peps/trunk/pep-0356.txt Message-ID: <20060630060925.8B0781E4005@bag.python.org> Author: neal.norwitz Date: Fri Jun 30 08:09:24 2006 New Revision: 47165 Modified: peps/trunk/pep-0356.txt Log: Add another bug that needs resolution (socket.close() not working) Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Fri Jun 30 08:09:24 2006 @@ -147,9 +147,10 @@ Open issues - Bugs that need resolving before release: - http://python.org/sf/1504046 + http://python.org/sf/1513223 http://python.org/sf/1512814 http://python.org/sf/1512695 + http://python.org/sf/1504046 http://python.org/sf/1501934 - Missing documentation From python-checkins at python.org Fri Jun 30 08:18:40 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 30 Jun 2006 08:18:40 +0200 (CEST) Subject: [Python-checkins] r47166 - in python/trunk: Lib/test/test_subprocess.py setup.py Message-ID: <20060630061840.A67A41E4005@bag.python.org> Author: tim.peters Date: Fri Jun 30 08:18:39 2006 New Revision: 47166 Modified: python/trunk/Lib/test/test_subprocess.py python/trunk/setup.py Log: Whitespace normalization. Modified: python/trunk/Lib/test/test_subprocess.py ============================================================================== --- python/trunk/Lib/test/test_subprocess.py (original) +++ python/trunk/Lib/test/test_subprocess.py Fri Jun 30 08:18:39 2006 @@ -28,13 +28,13 @@ class ProcessTestCase(unittest.TestCase): def setUp(self): - # Try to minimize the number of children we have so this test - # doesn't crash on some buildbots (Alphas in particular). + # Try to minimize the number of children we have so this test + # doesn't crash on some buildbots (Alphas in particular). test_support.reap_children() def tearDown(self): - # Try to minimize the number of children we have so this test - # doesn't crash on some buildbots (Alphas in particular). + # Try to minimize the number of children we have so this test + # doesn't crash on some buildbots (Alphas in particular). test_support.reap_children() def mkstemp(self): Modified: python/trunk/setup.py ============================================================================== --- python/trunk/setup.py (original) +++ python/trunk/setup.py Fri Jun 30 08:18:39 2006 @@ -653,7 +653,7 @@ std_variants.append(os.path.join(dn, "db3%d"%x)) std_variants.append(os.path.join(dn, "db3.%d"%x)) - db_inc_paths = std_variants + db_inc_paths + db_inc_paths = std_variants + db_inc_paths db_ver_inc_map = {} From python-checkins at python.org Fri Jun 30 08:30:08 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 30 Jun 2006 08:30:08 +0200 (CEST) Subject: [Python-checkins] r47167 - peps/trunk/pep-0291.txt Message-ID: <20060630063008.95FE91E4005@bag.python.org> Author: neal.norwitz Date: Fri Jun 30 08:30:07 2006 New Revision: 47167 Modified: peps/trunk/pep-0291.txt Log: Add a list of 2.4 language features, there are probably more I forgot Modified: peps/trunk/pep-0291.txt ============================================================================== --- peps/trunk/pep-0291.txt (original) +++ peps/trunk/pep-0291.txt Fri Jun 30 08:30:07 2006 @@ -72,12 +72,18 @@ 2.2 bool, True, False, basestring, enumerate(), {}.pop(), PendingDeprecationWarning, Universal Newlines, plus all features below + plus all features below 2.3 generator expressions, multi-line imports, decorators, int/long unification, set/frozenset, - reversed(), sorted(), "".rsplit() + reversed(), sorted(), "".rsplit(), + plus all features below - 2.4 ??? + 2.4 with statement, conditional expressions, + combined try/except/finally, relative imports, + yield expressions or generator.throw/send/close() + + 2.5 ??? Backward Compatible Packages, Modules, and Tools From python-checkins at python.org Fri Jun 30 08:38:34 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 30 Jun 2006 08:38:34 +0200 (CEST) Subject: [Python-checkins] r47168 - peps/trunk/pep-0000.txt peps/trunk/pep-0361.txt Message-ID: <20060630063834.165161E4005@bag.python.org> Author: neal.norwitz Date: Fri Jun 30 08:38:33 2006 New Revision: 47168 Added: peps/trunk/pep-0361.txt (contents, props changed) Modified: peps/trunk/pep-0000.txt Log: Skeleton 2.6 schedule andn plans Modified: peps/trunk/pep-0000.txt ============================================================================== --- peps/trunk/pep-0000.txt (original) +++ peps/trunk/pep-0000.txt Fri Jun 30 08:38:33 2006 @@ -66,6 +66,7 @@ I 339 Design of the CPython Compiler Cannon I 356 Python 2.5 Release Schedule Norwitz, et al I 360 Externally Maintained Packages Cannon + I 361 Python 2.6 Release Schedule Norwitz, et al I 3100 Python 3.0 Plans Kuchling, Cannon Accepted PEPs (accepted; may not be implemented yet) @@ -423,6 +424,7 @@ S 358 The "bytes" Object Schemenauer SW 359 The "make" Statement Bethard I 360 Externally Maintained Packages Cannon + I 361 Python 2.6 Release Schedule Norwitz, et al SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes P 3000 Python 3000 GvR Added: peps/trunk/pep-0361.txt ============================================================================== --- (empty file) +++ peps/trunk/pep-0361.txt Fri Jun 30 08:38:33 2006 @@ -0,0 +1,111 @@ +PEP: 361 +Title: Python 2.6 Release Schedule +Version: $Revision$ +Last-Modified: $Date$ +Author: Neal Norwitz +Status: Draft +Type: Informational +Created: 29-June-2006 +Python-Version: 2.6 +Post-History: + +Abstract + + This document describes the development and release schedule for + Python 2.6. The schedule primarily concerns itself with PEP-sized + items. Small features may be added up to and including the first + beta release. Bugs may be fixed until the final release. + + There will be at least two alpha releases, two beta releases, and + one release candidate. The release date is planned to be in XXX 2008. + + +Release Manager and Crew + + XXX volunteered to be Release Manager. + + XXX is building the Windows installers, + XXX is building the Mac installers, + XXX the doc packages and + XXX the RPMs. + + +Release Schedule + + Note that this schedule is completely tentative. The number of alphas, + betas and release candidates will be determined as the release process + unfolds. The minimal schedule is: + + June 2007: (re)confirm the crew and start deciding on schedule. + The initial 2.6 target is for April 2008. + + alpha 1: T - 16 weeks [planned] + alpha 2: T - 13 weeks [planned] + beta 1: T - 9 weeks [planned] + beta 2: T - 5 weeks [planned] + rc 1: T - 1 week [planned] + final: T [planned] + + +Completed features for 2.6 + + PEPs: + + None + + New modules in the standard library: + + None + + Other major features: + + None + + +Possible features for 2.6 + + New features *should* be implemented prior to alpha2, particularly + any C modifications or behavioral changes. New features *must* be + implemented prior to beta1 or will require Release Manager approval. + + The following PEPs are being worked on for possible inclusion in 2.6: + + - PEP 275: Switching on Multiple Values + - PEP 297: Support for System Upgrades + + Each non-trivial feature listed here that is not a PEP must be + discussed on python-dev. Other enhancements include: + + - distutils replacement (requires a PEP) + - turtle.py replacement or enhancements + + New modules in the standard library: + + - winerror + http://python.org/sf/1505257 + (Owner: MAL) + + +Deferred until 2.7 + + None + + +Open issues + + None + + +Copyright + + This document has been placed in the public domain. + + + +Local Variables: +mode: indented-text +indent-tabs-mode: nil +sentence-end-double-space: t +fill-column: 70 +coding: utf-8 +End: From python-checkins at python.org Fri Jun 30 09:24:05 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 30 Jun 2006 09:24:05 +0200 (CEST) Subject: [Python-checkins] r47169 - peps/trunk/pep-0356.txt Message-ID: <20060630072405.447DA1E4005@bag.python.org> Author: neal.norwitz Date: Fri Jun 30 09:24:04 2006 New Revision: 47169 Modified: peps/trunk/pep-0356.txt Log: Found a few more bugs Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Fri Jun 30 09:24:04 2006 @@ -147,11 +147,13 @@ Open issues - Bugs that need resolving before release: + http://python.org/sf/1513646 http://python.org/sf/1513223 http://python.org/sf/1512814 http://python.org/sf/1512695 http://python.org/sf/1504046 http://python.org/sf/1501934 + http://python.org/sf/1333982 - Missing documentation * ElementTree/cElementTree (Fredrik Lundh) From python-checkins at python.org Fri Jun 30 09:32:17 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 30 Jun 2006 09:32:17 +0200 (CEST) Subject: [Python-checkins] r47170 - python/trunk/Modules/_ctypes/callproc.c Message-ID: <20060630073217.B91141E4005@bag.python.org> Author: neal.norwitz Date: Fri Jun 30 09:32:16 2006 New Revision: 47170 Modified: python/trunk/Modules/_ctypes/callproc.c Log: Silence compiler warning Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Fri Jun 30 09:32:16 2006 @@ -82,6 +82,10 @@ #define DONT_USE_SEH #endif +#ifndef PY_FORMAT_SIZE_T +#define PY_FORMAT_SIZE_T "" +#endif + #ifdef MS_WIN32 PyObject *ComError; @@ -1486,7 +1490,8 @@ } if (size < dict->size) { PyErr_Format(PyExc_ValueError, - "minimum size is %d", dict->size); + "minimum size is %" PY_FORMAT_SIZE_T "d", + dict->size); return NULL; } if (obj->b_needsfree == 0) { From python-checkins at python.org Fri Jun 30 09:32:46 2006 From: python-checkins at python.org (neal.norwitz) Date: Fri, 30 Jun 2006 09:32:46 +0200 (CEST) Subject: [Python-checkins] r47171 - python/trunk/Objects/cellobject.c Message-ID: <20060630073246.E7BF51E4005@bag.python.org> Author: neal.norwitz Date: Fri Jun 30 09:32:46 2006 New Revision: 47171 Modified: python/trunk/Objects/cellobject.c Log: Another problem reported by Coverity. Backport candidate. Modified: python/trunk/Objects/cellobject.c ============================================================================== --- python/trunk/Objects/cellobject.c (original) +++ python/trunk/Objects/cellobject.c Fri Jun 30 09:32:46 2006 @@ -8,6 +8,8 @@ PyCellObject *op; op = (PyCellObject *)PyObject_GC_New(PyCellObject, &PyCell_Type); + if (op == NULL) + return NULL; op->ob_ref = obj; Py_XINCREF(obj); From tim.peters at gmail.com Fri Jun 30 10:20:18 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 30 Jun 2006 04:20:18 -0400 Subject: [Python-checkins] r47170 - python/trunk/Modules/_ctypes/callproc.c In-Reply-To: <20060630073217.B91141E4005@bag.python.org> References: <20060630073217.B91141E4005@bag.python.org> Message-ID: <1f7befae0606300120s29ed9c3fn9c4dff021d39cac3@mail.gmail.com> > Author: neal.norwitz > Date: Fri Jun 30 09:32:16 2006 > New Revision: 47170 > > Modified: > python/trunk/Modules/_ctypes/callproc.c > Log: > Silence compiler warning > > Modified: python/trunk/Modules/_ctypes/callproc.c > ============================================================================== > --- python/trunk/Modules/_ctypes/callproc.c (original) > +++ python/trunk/Modules/_ctypes/callproc.c Fri Jun 30 09:32:16 2006 > @@ -82,6 +82,10 @@ > #define DONT_USE_SEH > #endif > > +#ifndef PY_FORMAT_SIZE_T > +#define PY_FORMAT_SIZE_T "" > +#endif > + > #ifdef MS_WIN32 > PyObject *ComError; > > @@ -1486,7 +1490,8 @@ > } > if (size < dict->size) { > PyErr_Format(PyExc_ValueError, > - "minimum size is %d", dict->size); > + "minimum size is %" PY_FORMAT_SIZE_T "d", > + dict->size); Unfortunately, that doesn't make sense -- there's no guarantee that PyErr_Format() can understand PY_FORMAT_SIZE_T's expansion. PY_FORMAT_SIZE_T should only be used in calls to C's format functions (like printf or sprintf). PyErr_Format() in 2.5 makes sense of "%zd" itself, but before 2.5 does not, so "%zd" can't be used here either. #if PY_VERSION_HEX < 0x02050000 "minimum size is %d", dict->size); #else "minimum size is %zd", dict->size); #endif makes sense. That directly reflects that this module uses C int before 2.5, but uses Py_ssize_t starting with 2.5. From python-checkins at python.org Fri Jun 30 17:15:46 2006 From: python-checkins at python.org (matt.fleming) Date: Fri, 30 Jun 2006 17:15:46 +0200 (CEST) Subject: [Python-checkins] r47172 - sandbox/trunk/pdb/mpdb.py Message-ID: <20060630151546.7B2DB1E400A@bag.python.org> Author: matt.fleming Date: Fri Jun 30 17:15:43 2006 New Revision: 47172 Modified: sandbox/trunk/pdb/mpdb.py Log: Fix: Client connection contiuously in CLOSE_WAIT state. Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Fri Jun 30 17:15:43 2006 @@ -17,6 +17,7 @@ import os from optparse import OptionParser import pydb +from pydb.gdb import Restart import socket import sys import time @@ -108,11 +109,22 @@ return self.connection.write(line) ret = self.connection.readline() + if ret == '': + self.errmsg('Connection closed unexpectedly') + raise Exit # The output from the command that we've just sent to the server # is returned along with the prompt of that server. So we keep reading # until we find our prompt. + i = 1 while self.local_prompt not in ret: - ret += self.connection.readline() + if i == 100: + # We're probably _never_ going to get that data and that + # connection is probably dead. + self.errmsg('Connection died unexpectedly') + raise Exit + else: + ret += self.connection.readline() + i += 1 self.msg_nocr(ret) self.lastcmd = line return @@ -290,6 +302,9 @@ self.local_prompt = self.prompt self.prompt = "" line = self.connection.readline() + if line == '': + self.errmsg('Connection closed unexpectedly') + raise Exit while '(MPdb)' not in line: line = self.connection.readline() self.msg_nocr(line) @@ -454,19 +469,23 @@ % args[0]) return -def pdbserver(addr, args): +def pdbserver(addr): """ This method sets up a pdbserver debugger that allows debuggers - to connect to 'address' using 'protocol'. The argument 'filename' - is the name of the file that is being debugged. + to connect to 'addr', which a protocol-specific address, i.e. + tcp = 'tcp mydomainname.com:9876' + serial = 'serial /dev/ttyC0' """ m = MPdb() - mainpyfile = args[0] - m.mainpyfile = mainpyfile + m._sys_argv = ['python'] + for i in sys.argv: + m._sys_argv.append(i) + m._program_sys_argv = sys.argv[1:] + m.mainpyfile = m._program_sys_argv[1] m.do_pdbserver(addr) while True: try: - m._runscript(mainpyfile) - except pydb.gdb.Restart: + m._runscript(m.mainpyfile) + except Restart: m.msg('Restarting') except Exit: break @@ -475,9 +494,8 @@ """ Connect this debugger to a pdbserver at 'addr'. 'addr' is a protocol-specific address. i.e. tcp = 'tcp mydomainname.com:9876' - serial = '/dev/ttyC0' + serial = 'serial /dev/ttyC0' """ - print addr m = MPdb() # Look Ma, no script! m.do_target(addr) @@ -485,7 +503,7 @@ try: m.cmdloop() except: - sys.exit() + break def main(): """ Main entry point to this module. """ @@ -494,7 +512,7 @@ target(opts.target) sys.exit() elif opts.pdbserver: - pdbserver(opts.pdbserver, args) + pdbserver(opts.pdbserver) sys.exit() else: if not opts.scriptname: @@ -510,10 +528,19 @@ mpdb = MPdb() while 1: try: + mpdb._sys_argv = ['python'] + for i in sys.argv: + mpdb._sys_argv.append(i) + mpdb._program_sys_argv = mpdb._sys_argv[1:] mpdb._runscript(mainpyfile) if mpdb._user_requested_quit: break mpdb.msg("The program finished and will be restarted") + except Restart: + sys.argv = list(mpdb._program_sys_argv) + mpdb.msg('Restarting with %s with arguments:\n\t%s' + % (mpdb.filename(mainpyfile), + ' '.join(mpdb._program_sys_argv[1:]))) except SystemExit: # In most cases SystemExit does not warrant a post-mortem session. mpdb.msg("The program exited via sys.exit(). " + \ From python-checkins at python.org Fri Jun 30 18:55:41 2006 From: python-checkins at python.org (nick.coghlan) Date: Fri, 30 Jun 2006 18:55:41 +0200 (CEST) Subject: [Python-checkins] r47174 - peps/trunk/pep-0356.txt Message-ID: <20060630165541.7DD471E4007@bag.python.org> Author: nick.coghlan Date: Fri Jun 30 18:55:39 2006 New Revision: 47174 Modified: peps/trunk/pep-0356.txt Log: Include open issue for the PEP 328/338 relative import handling question Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Fri Jun 30 18:55:39 2006 @@ -155,6 +155,13 @@ http://python.org/sf/1501934 http://python.org/sf/1333982 + - Should relative imports from __main__ work when feasible? + Bug report: http://python.org/sf/1510172 + Rev 47142 updated -m switch to always set a __module_name__ attribute + Patch attached to bug report makes import.c aware of that attribute. + Needs a call from the release managers as to whether to apply the + second half of the fix, or else roll back the first half. + - Missing documentation * ElementTree/cElementTree (Fredrik Lundh) Needs latex-ifcation, patch at http://python.org/sf/1504046 From nnorwitz at gmail.com Fri Jun 30 19:04:18 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 30 Jun 2006 10:04:18 -0700 Subject: [Python-checkins] r47170 - python/trunk/Modules/_ctypes/callproc.c In-Reply-To: <1f7befae0606300120s29ed9c3fn9c4dff021d39cac3@mail.gmail.com> References: <20060630073217.B91141E4005@bag.python.org> <1f7befae0606300120s29ed9c3fn9c4dff021d39cac3@mail.gmail.com> Message-ID: Whoops, you're right. I'll try to remember to fix this when I get home tonight...unless someone beats me to it. :-) n -- On 6/30/06, Tim Peters wrote: > > Author: neal.norwitz > > Date: Fri Jun 30 09:32:16 2006 > > New Revision: 47170 > > > > Modified: > > python/trunk/Modules/_ctypes/callproc.c > > Log: > > Silence compiler warning > > > > Modified: python/trunk/Modules/_ctypes/callproc.c > > ============================================================================== > > --- python/trunk/Modules/_ctypes/callproc.c (original) > > +++ python/trunk/Modules/_ctypes/callproc.c Fri Jun 30 09:32:16 2006 > > @@ -82,6 +82,10 @@ > > #define DONT_USE_SEH > > #endif > > > > +#ifndef PY_FORMAT_SIZE_T > > +#define PY_FORMAT_SIZE_T "" > > +#endif > > + > > #ifdef MS_WIN32 > > PyObject *ComError; > > > > @@ -1486,7 +1490,8 @@ > > } > > if (size < dict->size) { > > PyErr_Format(PyExc_ValueError, > > - "minimum size is %d", dict->size); > > + "minimum size is %" PY_FORMAT_SIZE_T "d", > > + dict->size); > > Unfortunately, that doesn't make sense -- there's no guarantee that > PyErr_Format() can understand PY_FORMAT_SIZE_T's expansion. > PY_FORMAT_SIZE_T should only be used in calls to C's format functions > (like printf or sprintf). PyErr_Format() in 2.5 makes sense of "%zd" > itself, but before 2.5 does not, so "%zd" can't be used here either. > > #if PY_VERSION_HEX < 0x02050000 > "minimum size is %d", dict->size); > #else > "minimum size is %zd", dict->size); > #endif > > makes sense. That directly reflects that this module uses C int > before 2.5, but uses Py_ssize_t starting with 2.5. > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From theller at python.net Fri Jun 30 19:37:35 2006 From: theller at python.net (Thomas Heller) Date: Fri, 30 Jun 2006 19:37:35 +0200 Subject: [Python-checkins] r47170 - python/trunk/Modules/_ctypes/callproc.c In-Reply-To: References: <20060630073217.B91141E4005@bag.python.org> <1f7befae0606300120s29ed9c3fn9c4dff021d39cac3@mail.gmail.com> Message-ID: Neal Norwitz schrieb: > Whoops, you're right. I'll try to remember to fix this when I get home > tonight...unless someone beats me to it. :-) I'll do it. Thanks, Thomas > -- > > On 6/30/06, Tim Peters wrote: >> > Author: neal.norwitz >> > Date: Fri Jun 30 09:32:16 2006 >> > New Revision: 47170 >> > >> > Modified: >> > python/trunk/Modules/_ctypes/callproc.c >> > Log: >> > Silence compiler warning >> > >> > Modified: python/trunk/Modules/_ctypes/callproc.c >> > ============================================================================== >> > --- python/trunk/Modules/_ctypes/callproc.c (original) >> > +++ python/trunk/Modules/_ctypes/callproc.c Fri Jun 30 09:32:16 2006 >> > @@ -82,6 +82,10 @@ >> > #define DONT_USE_SEH >> > #endif >> > >> > +#ifndef PY_FORMAT_SIZE_T >> > +#define PY_FORMAT_SIZE_T "" >> > +#endif >> > + >> > #ifdef MS_WIN32 >> > PyObject *ComError; >> > >> > @@ -1486,7 +1490,8 @@ >> > } >> > if (size < dict->size) { >> > PyErr_Format(PyExc_ValueError, >> > - "minimum size is %d", dict->size); >> > + "minimum size is %" PY_FORMAT_SIZE_T "d", >> > + dict->size); >> >> Unfortunately, that doesn't make sense -- there's no guarantee that >> PyErr_Format() can understand PY_FORMAT_SIZE_T's expansion. >> PY_FORMAT_SIZE_T should only be used in calls to C's format functions >> (like printf or sprintf). PyErr_Format() in 2.5 makes sense of "%zd" >> itself, but before 2.5 does not, so "%zd" can't be used here either. >> >> #if PY_VERSION_HEX < 0x02050000 >> "minimum size is %d", dict->size); >> #else >> "minimum size is %zd", dict->size); >> #endif >> >> makes sense. That directly reflects that this module uses C int >> before 2.5, but uses Py_ssize_t starting with 2.5. >> _______________________________________________ >> Python-checkins mailing list >> Python-checkins at python.org >> http://mail.python.org/mailman/listinfo/python-checkins >> From python-checkins at python.org Fri Jun 30 19:44:55 2006 From: python-checkins at python.org (thomas.heller) Date: Fri, 30 Jun 2006 19:44:55 +0200 (CEST) Subject: [Python-checkins] r47175 - python/trunk/Modules/_ctypes/callproc.c Message-ID: <20060630174455.67F2A1E4002@bag.python.org> Author: thomas.heller Date: Fri Jun 30 19:44:54 2006 New Revision: 47175 Modified: python/trunk/Modules/_ctypes/callproc.c Log: Revert the use of PY_FORMAT_SIZE_T in PyErr_Format. Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Fri Jun 30 19:44:54 2006 @@ -1490,7 +1490,11 @@ } if (size < dict->size) { PyErr_Format(PyExc_ValueError, - "minimum size is %" PY_FORMAT_SIZE_T "d", +#if PY_VERSION_HEX < 0x02050000 + "minimum size is %d", +#else + "minimum size is %zd", +#endif dict->size); return NULL; } From theller at python.net Fri Jun 30 20:09:52 2006 From: theller at python.net (Thomas Heller) Date: Fri, 30 Jun 2006 20:09:52 +0200 Subject: [Python-checkins] r46894 - in python/trunk: Modules/timemodule.c Objects/exceptions.c Objects/fileobject.c In-Reply-To: <20060612154513.619FB1E4013@bag.python.org> References: <20060612154513.619FB1E4013@bag.python.org> Message-ID: kristjan.jonsson schrieb: > Author: kristjan.jonsson > Date: Mon Jun 12 17:45:12 2006 > New Revision: 46894 > > Modified: > python/trunk/Modules/timemodule.c > python/trunk/Objects/exceptions.c > python/trunk/Objects/fileobject.c > Log: > Fix the CRT argument error handling for VisualStudio .NET 2005. Install a CRT error handler and disable the assertion for debug builds. This causes CRT to set errno to EINVAL. > This update fixes crash cases in the test suite where the default CRT error handler would cause process exit. ... > Modified: python/trunk/Objects/exceptions.c > ============================================================================== > --- python/trunk/Objects/exceptions.c (original) > +++ python/trunk/Objects/exceptions.c Mon Jun 12 17:45:12 2006 > @@ -1967,6 +1967,29 @@ > if (PyDict_SetItemString(bdict, # TYPE, PyExc_ ## TYPE)) \ > Py_FatalError("Module dictionary insertion problem."); > > +#if defined _MSC_VER && _MSC_VER >= 1400 > +/* crt variable checking in VisualStudio .NET 2005 */ > +#include > + > +static int prevCrtReportMode; > +static _invalid_parameter_handler prevCrtHandler; > + > +/* Invalid parameter handler. Sets a ValueError exception */ > +static void > +InvalidParameterHandler( > + const wchar_t * expression, > + const wchar_t * function, > + const wchar_t * file, > + unsigned int line, > + uintptr_t pReserved) > +{ > + /* Do nothing, allow execution to continue. Usually this > + * means that the CRT will set errno to EINVAL > + */ > +} > +#endif > + > + > PyMODINIT_FUNC > _PyExc_Init(void) > { > @@ -2096,6 +2119,13 @@ > Py_FatalError("Cannot pre-allocate MemoryError instance\n"); > > Py_DECREF(bltinmod); > + > +#if defined _MSC_VER && _MSC_VER >= 1400 > + /* Set CRT argument error handler */ > + prevCrtHandler = _set_invalid_parameter_handler(InvalidParameterHandler); > + /* turn off assertions in debug mode */ > + prevCrtReportMode = _CrtSetReportMode(_CRT_ASSERT, 0); > +#endif > } > > void > @@ -2103,4 +2133,9 @@ > { > Py_XDECREF(PyExc_MemoryErrorInst); > PyExc_MemoryErrorInst = NULL; > +#if defined _MSC_VER && _MSC_VER >= 1400 > + /* reset CRT error handling */ > + _set_invalid_parameter_handler(prevCrtHandler); > + _CrtSetReportMode(_CRT_ASSERT, prevCrtReportMode); > +#endif > } ... These changes to Objects/exceptions.c break the build for Windows AMD64. Apparently the amd64 compiler from the Server 2003 SP1 SDK has _MSC_VER >= 1400, but does not know about this new error handling. The compiler identifies itself in this way: C:\Program Files\Microsoft Platform SDK>cl Microsoft (R) C/C++ Optimizing Compiler Version 14.00.40310.41 for AMD64 Copyright (C) Microsoft Corporation. All rights reserved. usage: cl [ option... ] filename... [ /link linkoption... ] C:\Program Files\Microsoft Platform SDK> Thomas From python-checkins at python.org Fri Jun 30 20:34:52 2006 From: python-checkins at python.org (tim.peters) Date: Fri, 30 Jun 2006 20:34:52 +0200 (CEST) Subject: [Python-checkins] r47176 - python/trunk/Modules/_ctypes/callproc.c Message-ID: <20060630183452.95D221E4002@bag.python.org> Author: tim.peters Date: Fri Jun 30 20:34:51 2006 New Revision: 47176 Modified: python/trunk/Modules/_ctypes/callproc.c Log: Remove now-unused fidding with PY_FORMAT_SIZE_T. Modified: python/trunk/Modules/_ctypes/callproc.c ============================================================================== --- python/trunk/Modules/_ctypes/callproc.c (original) +++ python/trunk/Modules/_ctypes/callproc.c Fri Jun 30 20:34:51 2006 @@ -82,10 +82,6 @@ #define DONT_USE_SEH #endif -#ifndef PY_FORMAT_SIZE_T -#define PY_FORMAT_SIZE_T "" -#endif - #ifdef MS_WIN32 PyObject *ComError; From tim.peters at gmail.com Fri Jun 30 20:37:51 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 30 Jun 2006 14:37:51 -0400 Subject: [Python-checkins] r46894 - in python/trunk: Modules/timemodule.c Objects/exceptions.c Objects/fileobject.c In-Reply-To: References: <20060612154513.619FB1E4013@bag.python.org> Message-ID: <1f7befae0606301137p714d43afhceffef125652a465@mail.gmail.com> Copying Kristj?n directly since he may not be subscribed to python-checkins. On 6/30/06, Thomas Heller wrote: > kristjan.jonsson schrieb: > > Author: kristjan.jonsson > > Date: Mon Jun 12 17:45:12 2006 > > New Revision: 46894 > > > > Modified: > > python/trunk/Modules/timemodule.c > > python/trunk/Objects/exceptions.c > > python/trunk/Objects/fileobject.c > > Log: > > Fix the CRT argument error handling for VisualStudio .NET 2005. Install a CRT error handler and disable the assertion for debug builds. This causes CRT to set errno to EINVAL. > > This update fixes crash cases in the test suite where the default CRT error handler would cause process exit. > ... > > Modified: python/trunk/Objects/exceptions.c > > ============================================================================== > > --- python/trunk/Objects/exceptions.c (original) > > +++ python/trunk/Objects/exceptions.c Mon Jun 12 17:45:12 2006 > > @@ -1967,6 +1967,29 @@ > > if (PyDict_SetItemString(bdict, # TYPE, PyExc_ ## TYPE)) \ > > Py_FatalError("Module dictionary insertion problem."); > > > > +#if defined _MSC_VER && _MSC_VER >= 1400 > > +/* crt variable checking in VisualStudio .NET 2005 */ > > +#include > > + > > +static int prevCrtReportMode; > > +static _invalid_parameter_handler prevCrtHandler; > > + > > +/* Invalid parameter handler. Sets a ValueError exception */ > > +static void > > +InvalidParameterHandler( > > + const wchar_t * expression, > > + const wchar_t * function, > > + const wchar_t * file, > > + unsigned int line, > > + uintptr_t pReserved) > > +{ > > + /* Do nothing, allow execution to continue. Usually this > > + * means that the CRT will set errno to EINVAL > > + */ > > +} > > +#endif > > + > > + > > PyMODINIT_FUNC > > _PyExc_Init(void) > > { > > @@ -2096,6 +2119,13 @@ > > Py_FatalError("Cannot pre-allocate MemoryError instance\n"); > > > > Py_DECREF(bltinmod); > > + > > +#if defined _MSC_VER && _MSC_VER >= 1400 > > + /* Set CRT argument error handler */ > > + prevCrtHandler = _set_invalid_parameter_handler(InvalidParameterHandler); > > + /* turn off assertions in debug mode */ > > + prevCrtReportMode = _CrtSetReportMode(_CRT_ASSERT, 0); > > +#endif > > } > > > > void > > @@ -2103,4 +2133,9 @@ > > { > > Py_XDECREF(PyExc_MemoryErrorInst); > > PyExc_MemoryErrorInst = NULL; > > +#if defined _MSC_VER && _MSC_VER >= 1400 > > + /* reset CRT error handling */ > > + _set_invalid_parameter_handler(prevCrtHandler); > > + _CrtSetReportMode(_CRT_ASSERT, prevCrtReportMode); > > +#endif > > } > ... > > These changes to Objects/exceptions.c break the build for Windows AMD64. Apparently the amd64 > compiler from the Server 2003 SP1 SDK has _MSC_VER >= 1400, but does not know about this new > error handling. > > The compiler identifies itself in this way: > > C:\Program Files\Microsoft Platform SDK>cl > Microsoft (R) C/C++ Optimizing Compiler Version 14.00.40310.41 for AMD64 > Copyright (C) Microsoft Corporation. All rights reserved. > > usage: cl [ option... ] filename... [ /link linkoption... ] > > C:\Program Files\Microsoft Platform SDK> > > > Thomas > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From python-checkins at python.org Fri Jun 30 20:47:56 2006 From: python-checkins at python.org (georg.brandl) Date: Fri, 30 Jun 2006 20:47:56 +0200 (CEST) Subject: [Python-checkins] r47177 - python/trunk/Doc/lib/libfuncs.tex Message-ID: <20060630184756.A79981E4002@bag.python.org> Author: georg.brandl Date: Fri Jun 30 20:47:56 2006 New Revision: 47177 Modified: python/trunk/Doc/lib/libfuncs.tex Log: Document decorator usage of property. Modified: python/trunk/Doc/lib/libfuncs.tex ============================================================================== --- python/trunk/Doc/lib/libfuncs.tex (original) +++ python/trunk/Doc/lib/libfuncs.tex Fri Jun 30 20:47:56 2006 @@ -789,7 +789,22 @@ If given, \var{doc} will be the docstring of the property attribute. Otherwise, the property will copy \var{fget}'s docstring (if it - exists). + exists). This makes it possible to create read-only properties + easily using \function{property} as a decorator: + +\begin{verbatim} +class Parrot(object): + def __init__(self): + self.__voltage = 100000 + + @property + def voltage(self): + """Get the current voltage.""" + return self.__voltage +\end{verbatim} + + turns the \method{voltage} method into a "getter" for a read-only attribute + with the same name. \versionadded{2.2} \versionchanged[Use \var{fget}'s docstring if no \var{doc} given]{2.5} From theller at python.net Fri Jun 30 20:58:04 2006 From: theller at python.net (Thomas Heller) Date: Fri, 30 Jun 2006 20:58:04 +0200 Subject: [Python-checkins] r46894 - in python/trunk: Modules/timemodule.c Objects/exceptions.c Objects/fileobject.c In-Reply-To: <1f7befae0606301137p714d43afhceffef125652a465@mail.gmail.com> References: <20060612154513.619FB1E4013@bag.python.org> <1f7befae0606301137p714d43afhceffef125652a465@mail.gmail.com> Message-ID: Tim Peters schrieb: > Copying Kristj?n directly since he may not be subscribed to python-checkins. [off-topic, sorry] I would have done this myself, but I cannot log into the starship to send or receive mail. Does anyone know what's up? Thomas From python-checkins at python.org Fri Jun 30 21:10:18 2006 From: python-checkins at python.org (martin.v.loewis) Date: Fri, 30 Jun 2006 21:10:18 +0200 (CEST) Subject: [Python-checkins] r47179 - peps/trunk/pep-0356.txt Message-ID: <20060630191018.6DF0B1E4002@bag.python.org> Author: martin.v.loewis Date: Fri Jun 30 21:10:18 2006 New Revision: 47179 Modified: peps/trunk/pep-0356.txt Log: Add the bugs that I consider critical before 2.5b2. Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Fri Jun 30 21:10:18 2006 @@ -154,6 +154,9 @@ http://python.org/sf/1504046 http://python.org/sf/1501934 http://python.org/sf/1333982 + http://python.org/sf/1513646 + http://python.org/sf/1508010 + http://python.org/sf/1475523 - Should relative imports from __main__ work when feasible? Bug report: http://python.org/sf/1510172 From python-checkins at python.org Fri Jun 30 21:21:25 2006 From: python-checkins at python.org (martin.v.loewis) Date: Fri, 30 Jun 2006 21:21:25 +0200 (CEST) Subject: [Python-checkins] r47180 - peps/trunk/pep-0356.txt Message-ID: <20060630192125.E559D1E4002@bag.python.org> Author: martin.v.loewis Date: Fri Jun 30 21:21:24 2006 New Revision: 47180 Modified: peps/trunk/pep-0356.txt Log: Added 1494314 per request from Jean-Paul Calderone. Modified: peps/trunk/pep-0356.txt ============================================================================== --- peps/trunk/pep-0356.txt (original) +++ peps/trunk/pep-0356.txt Fri Jun 30 21:21:24 2006 @@ -157,6 +157,7 @@ http://python.org/sf/1513646 http://python.org/sf/1508010 http://python.org/sf/1475523 + http://python.org/sf/1494314 - Should relative imports from __main__ work when feasible? Bug report: http://python.org/sf/1510172 From buildbot at python.org Fri Jun 30 21:23:02 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 30 Jun 2006 19:23:02 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20060630192302.AE0F91E4002@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/259 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From python-checkins at python.org Fri Jun 30 21:29:26 2006 From: python-checkins at python.org (fred.drake) Date: Fri, 30 Jun 2006 21:29:26 +0200 (CEST) Subject: [Python-checkins] r47181 - python/trunk/Doc/lib/libfuncs.tex Message-ID: <20060630192926.19CBF1E4002@bag.python.org> Author: fred.drake Date: Fri Jun 30 21:29:25 2006 New Revision: 47181 Modified: python/trunk/Doc/lib/libfuncs.tex Log: - consistency nit: always include "()" in \function and \method (*should* be done by the presentation, but that requires changes all over) - avoid spreading the __name meme Modified: python/trunk/Doc/lib/libfuncs.tex ============================================================================== --- python/trunk/Doc/lib/libfuncs.tex (original) +++ python/trunk/Doc/lib/libfuncs.tex Fri Jun 30 21:29:25 2006 @@ -781,30 +781,30 @@ \begin{verbatim} class C(object): def __init__(self): self.__x = None - def getx(self): return self.__x - def setx(self, value): self.__x = value - def delx(self): del self.__x + def getx(self): return self._x + def setx(self, value): self._x = value + def delx(self): del self._x x = property(getx, setx, delx, "I'm the 'x' property.") \end{verbatim} If given, \var{doc} will be the docstring of the property attribute. Otherwise, the property will copy \var{fget}'s docstring (if it exists). This makes it possible to create read-only properties - easily using \function{property} as a decorator: + easily using \function{property()} as a decorator: \begin{verbatim} class Parrot(object): def __init__(self): - self.__voltage = 100000 + self._voltage = 100000 @property def voltage(self): """Get the current voltage.""" - return self.__voltage + return self._voltage \end{verbatim} - turns the \method{voltage} method into a "getter" for a read-only attribute - with the same name. + turns the \method{voltage()} method into a ``getter'' for a read-only + attribute with the same name. \versionadded{2.2} \versionchanged[Use \var{fget}'s docstring if no \var{doc} given]{2.5} From buildbot at python.org Fri Jun 30 21:44:19 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 30 Jun 2006 19:44:19 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20060630194419.432531E4002@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/260 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From g.brandl at gmx.net Fri Jun 30 22:02:33 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 30 Jun 2006 22:02:33 +0200 Subject: [Python-checkins] r47181 - python/trunk/Doc/lib/libfuncs.tex In-Reply-To: <20060630192926.19CBF1E4002@bag.python.org> References: <20060630192926.19CBF1E4002@bag.python.org> Message-ID: fred.drake wrote: > Author: fred.drake > Date: Fri Jun 30 21:29:25 2006 > New Revision: 47181 > > Modified: > python/trunk/Doc/lib/libfuncs.tex > Log: > - consistency nit: always include "()" in \function and \method > (*should* be done by the presentation, but that requires changes all over) > - avoid spreading the __name meme > > > Modified: python/trunk/Doc/lib/libfuncs.tex > ============================================================================== > --- python/trunk/Doc/lib/libfuncs.tex (original) > +++ python/trunk/Doc/lib/libfuncs.tex Fri Jun 30 21:29:25 2006 > @@ -781,30 +781,30 @@ > \begin{verbatim} > class C(object): > def __init__(self): self.__x = None > - def getx(self): return self.__x > - def setx(self, value): self.__x = value > - def delx(self): del self.__x > + def getx(self): return self._x > + def setx(self, value): self._x = value > + def delx(self): del self._x > x = property(getx, setx, delx, "I'm the 'x' property.") > \end{verbatim} > > If given, \var{doc} will be the docstring of the property attribute. > Otherwise, the property will copy \var{fget}'s docstring (if it > exists). This makes it possible to create read-only properties > - easily using \function{property} as a decorator: > + easily using \function{property()} as a decorator: Note that I left out the parens here intentionally since they mustn't be used when writing the decorator, just to avoid possible mistakes. Georg From python-checkins at python.org Fri Jun 30 22:12:22 2006 From: python-checkins at python.org (matt.fleming) Date: Fri, 30 Jun 2006 22:12:22 +0200 (CEST) Subject: [Python-checkins] r47182 - sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Message-ID: <20060630201222.444A11E4002@bag.python.org> Author: matt.fleming Date: Fri Jun 30 22:12:21 2006 New Revision: 47182 Modified: sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py Log: Allow mpdb to use pydb's restart command, both locally and remotely. Modified: sandbox/trunk/pdb/mconnection.py ============================================================================== --- sandbox/trunk/pdb/mconnection.py (original) +++ sandbox/trunk/pdb/mconnection.py Fri Jun 30 22:12:21 2006 @@ -126,7 +126,7 @@ def disconnect(self): if self.output is None or self._sock is None: return - self.output.close() + self.output.shutdown(socket.SHUT_RDWR) self._sock.close() self._sock = None self.listening = False Modified: sandbox/trunk/pdb/mpdb.py ============================================================================== --- sandbox/trunk/pdb/mpdb.py (original) +++ sandbox/trunk/pdb/mpdb.py Fri Jun 30 22:12:21 2006 @@ -111,7 +111,8 @@ ret = self.connection.readline() if ret == '': self.errmsg('Connection closed unexpectedly') - raise Exit + self.onecmd = lambda x: pydb.Pdb.onecmd(self, x) + self.do_rquit(None) # The output from the command that we've just sent to the server # is returned along with the prompt of that server. So we keep reading # until we find our prompt. @@ -121,14 +122,34 @@ # We're probably _never_ going to get that data and that # connection is probably dead. self.errmsg('Connection died unexpectedly') - raise Exit + self.onecmd = lambda x: pydb.Pdb.onecmd(self, x) + self.do_rquit(None) else: ret += self.connection.readline() i += 1 + + # Some 'special' actions must be taken depending on the data returned + if 'restart_now' in ret: + self.connection.write('ACK:restart_now') + self.errmsg('Pdbserver restarting..') + # We've acknowledged a restart, which means that a new pdbserver + # process is started, so we have to connect all over again. + self._disconnect() + time.sleep(3.0) + self.do_target(self.target_addr) + return self.msg_nocr(ret) self.lastcmd = line return + def _disconnect(self): + """ Disconnect a connection. """ + self.connection.disconnect() + self.connection = None + self.target = 'local' + if hasattr(self, 'local_prompt'): + self.prompt = self.local_prompt + def msg_nocr(self, msg, out=None): """Common routine for reporting messages. Derived classes may want to override this to capture output. @@ -248,7 +269,7 @@ return if self.target == 'remote': self.errmsg('Already connected to a remote machine.') - return + return if target == 'tcp': # TODO: need to save state of current debug session if self.connection: self.connection.disconnect() @@ -301,6 +322,7 @@ # in remote_onecmd, because it may be different to this client's. self.local_prompt = self.prompt self.prompt = "" + self.target_addr = target + " " + addr line = self.connection.readline() if line == '': self.errmsg('Connection closed unexpectedly') @@ -382,6 +404,7 @@ except ConnectionFailed, err: self.errmsg("Failed to connect to %s: (%s)" % (comm, err)) return + self.pdbserver_addr = comm self.target = 'remote' self._rebind_input(self.connection) self._rebind_output(self.connection) @@ -395,15 +418,33 @@ return self._rebind_output(self.orig_stdout) self._rebind_input(self.orig_stdin) - if self.connection != None: - self.connection.disconnect() - if hasattr(self, 'local_prompt'): - self.prompt = self.local_prompt - self.msg('Exiting remote debugging...') + self._disconnect() self.target = 'local' self.do_quit(None) raise Exit + def do_restart(self, arg): + """ Extend pydb.do_restart to signal to any clients connected on + a debugger's connection that this debugger is going to be restarted. + All state is lost, and a new copy of the debugger is used. + """ + # We don't proceed with the restart until the action has been + # ACK'd by any connected clients + if self.connection != None: + self.msg('restart_now\n(MPdb)') + line = "" + while not 'ACK:restart_now' in line: + line = self.connection.readline() + try: + self.do_rquit(None) + except Exit: + pass + else: + self.msg("Re exec'ing\n\t%s" % self._sys_argv) + os.execvp(self._sys_argv[0], self._sys_argv) + + + def do_thread(self, arg): """Use this command to switch between threads. The new thread ID must be currently known. From martin at v.loewis.de Fri Jun 30 22:33:37 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 30 Jun 2006 22:33:37 +0200 Subject: [Python-checkins] r46894 - in python/trunk: Modules/timemodule.c Objects/exceptions.c Objects/fileobject.c In-Reply-To: References: <20060612154513.619FB1E4013@bag.python.org> Message-ID: <44A58AA1.2040100@v.loewis.de> Thomas Heller wrote: > These changes to Objects/exceptions.c break the build for Windows AMD64. Apparently the amd64 > compiler from the Server 2003 SP1 SDK has _MSC_VER >= 1400, but does not know about this new > error handling. I've a pending patch on my machine to fix that; no time yet to commit it. That patch got included into the 2.5b1 binaries that I created for AMD64. Regards, Martin From nnorwitz at gmail.com Fri Jun 30 22:45:30 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 30 Jun 2006 13:45:30 -0700 Subject: [Python-checkins] r47182 - sandbox/trunk/pdb/mconnection.py sandbox/trunk/pdb/mpdb.py In-Reply-To: <20060630201222.444A11E4002@bag.python.org> References: <20060630201222.444A11E4002@bag.python.org> Message-ID: Note: you probably shouldn't be calling shutdown(), but there is a bug in 2.5 where this is currently required. n -- On 6/30/06, matt.fleming wrote: > Author: matt.fleming > Date: Fri Jun 30 22:12:21 2006 > New Revision: 47182 > > Modified: > sandbox/trunk/pdb/mconnection.py > sandbox/trunk/pdb/mpdb.py > Log: > Allow mpdb to use pydb's restart command, both locally and remotely. > > > Modified: sandbox/trunk/pdb/mconnection.py > ============================================================================== > --- sandbox/trunk/pdb/mconnection.py (original) > +++ sandbox/trunk/pdb/mconnection.py Fri Jun 30 22:12:21 2006 > @@ -126,7 +126,7 @@ > def disconnect(self): > if self.output is None or self._sock is None: > return > - self.output.close() > + self.output.shutdown(socket.SHUT_RDWR) > self._sock.close() > self._sock = None > self.listening = False > > Modified: sandbox/trunk/pdb/mpdb.py > ============================================================================== > --- sandbox/trunk/pdb/mpdb.py (original) > +++ sandbox/trunk/pdb/mpdb.py Fri Jun 30 22:12:21 2006 > @@ -111,7 +111,8 @@ > ret = self.connection.readline() > if ret == '': > self.errmsg('Connection closed unexpectedly') > - raise Exit > + self.onecmd = lambda x: pydb.Pdb.onecmd(self, x) > + self.do_rquit(None) > # The output from the command that we've just sent to the server > # is returned along with the prompt of that server. So we keep reading > # until we find our prompt. > @@ -121,14 +122,34 @@ > # We're probably _never_ going to get that data and that > # connection is probably dead. > self.errmsg('Connection died unexpectedly') > - raise Exit > + self.onecmd = lambda x: pydb.Pdb.onecmd(self, x) > + self.do_rquit(None) > else: > ret += self.connection.readline() > i += 1 > + > + # Some 'special' actions must be taken depending on the data returned > + if 'restart_now' in ret: > + self.connection.write('ACK:restart_now') > + self.errmsg('Pdbserver restarting..') > + # We've acknowledged a restart, which means that a new pdbserver > + # process is started, so we have to connect all over again. > + self._disconnect() > + time.sleep(3.0) > + self.do_target(self.target_addr) > + return > self.msg_nocr(ret) > self.lastcmd = line > return > > + def _disconnect(self): > + """ Disconnect a connection. """ > + self.connection.disconnect() > + self.connection = None > + self.target = 'local' > + if hasattr(self, 'local_prompt'): > + self.prompt = self.local_prompt > + > def msg_nocr(self, msg, out=None): > """Common routine for reporting messages. Derived classes may want > to override this to capture output. > @@ -248,7 +269,7 @@ > return > if self.target == 'remote': > self.errmsg('Already connected to a remote machine.') > - return > + return > if target == 'tcp': > # TODO: need to save state of current debug session > if self.connection: self.connection.disconnect() > @@ -301,6 +322,7 @@ > # in remote_onecmd, because it may be different to this client's. > self.local_prompt = self.prompt > self.prompt = "" > + self.target_addr = target + " " + addr > line = self.connection.readline() > if line == '': > self.errmsg('Connection closed unexpectedly') > @@ -382,6 +404,7 @@ > except ConnectionFailed, err: > self.errmsg("Failed to connect to %s: (%s)" % (comm, err)) > return > + self.pdbserver_addr = comm > self.target = 'remote' > self._rebind_input(self.connection) > self._rebind_output(self.connection) > @@ -395,15 +418,33 @@ > return > self._rebind_output(self.orig_stdout) > self._rebind_input(self.orig_stdin) > - if self.connection != None: > - self.connection.disconnect() > - if hasattr(self, 'local_prompt'): > - self.prompt = self.local_prompt > - self.msg('Exiting remote debugging...') > + self._disconnect() > self.target = 'local' > self.do_quit(None) > raise Exit > > + def do_restart(self, arg): > + """ Extend pydb.do_restart to signal to any clients connected on > + a debugger's connection that this debugger is going to be restarted. > + All state is lost, and a new copy of the debugger is used. > + """ > + # We don't proceed with the restart until the action has been > + # ACK'd by any connected clients > + if self.connection != None: > + self.msg('restart_now\n(MPdb)') > + line = "" > + while not 'ACK:restart_now' in line: > + line = self.connection.readline() > + try: > + self.do_rquit(None) > + except Exit: > + pass > + else: > + self.msg("Re exec'ing\n\t%s" % self._sys_argv) > + os.execvp(self._sys_argv[0], self._sys_argv) > + > + > + > def do_thread(self, arg): > """Use this command to switch between threads. > The new thread ID must be currently known. > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From python-checkins at python.org Fri Jun 30 23:03:12 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 30 Jun 2006 23:03:12 +0200 (CEST) Subject: [Python-checkins] r47183 - sandbox/trunk/Doc/functional.rst Message-ID: <20060630210312.7BBD21E4002@bag.python.org> Author: andrew.kuchling Date: Fri Jun 30 23:03:11 2006 New Revision: 47183 Modified: sandbox/trunk/Doc/functional.rst Log: Revision pass: lots of small edits; request feedback; remove LaTeX markup :) ; add small references section Modified: sandbox/trunk/Doc/functional.rst ============================================================================== --- sandbox/trunk/Doc/functional.rst (original) +++ sandbox/trunk/Doc/functional.rst Fri Jun 30 23:03:11 2006 @@ -1,13 +1,14 @@ Functional Programming HOWTO ================================ -**Incomplete Draft** +(This is a first draft. Please send comments/error +reports/suggestions to amk at amk.ca.) -In this document, we'll take a tour of Python's features that are -well-suited to implementing programs in a functional style. After an -introduction to the concepts of functional programming, we'll look at -language features such as iterators and generators and relevant -library modules such as the ``itertools`` and ``functools`` modules. +In this document, we'll take a tour of Python's features suitable for +implementing programs in a functional style. After an introduction to +the concepts of functional programming, we'll look at language +features such as iterators and generators and relevant library modules +such as ``itertools`` and ``functools``. Introduction @@ -20,10 +21,11 @@ Programming languages support decomposing problems in several different ways: -* Most programming languages are **procedural**, meaning - that programs are lists of instructions that tell the computer what to - do with the program's input. - C, Pascal, and even Unix shells are procedural languages. +* Most programming languages are **procedural**: + programs are lists of instructions that tell the computer what to + do with the program's input. + C, Pascal, and even Unix shells are procedural languages. + * In **declarative** languages, you write a specification that describes the problem to be solved, and the language implementation figures out how to perform the computation efficiently. SQL is the declarative @@ -31,43 +33,54 @@ the data set you want to retrieve, and the SQL engine decides whether to scan tables or use indexes, which subclauses should be performed first, etc. -* **Object-oriented** programming is a style in which programs consist - of collections of objects. Objects have internal state and support various - methods that query or modify this internal state in some way. Smalltalk - and Java are object-oriented languages. C++ and Python are languages - that support object-oriented programming, but don't require it. + +* **Object-oriented** programs manipulate collections of objects. + Objects have internal state and support methods that query or modify + this internal state in some way. Smalltalk and Java are + object-oriented languages. C++ and Python are languages that + support object-oriented programming, but don't force the use + of object-oriented features. + * **Functional** programming decomposes a problem into a set of functions. Ideally, functions only take inputs and produce outputs, and don't have any internal state that affects the output produced for a given input. Well-known functional languages include the ML family (Standard ML, - OCaml, and other variants) and Haskell. + OCaml, and other variants) and Haskell. -The designers of some computer languages have chosen one style of -programming that they support best. This often makes it difficult to -write programs in the language that use a different style. Other -language designers try to produce multi-paradigm languages that -support several different styles. Lisp, C++, and Python are -multi-paradigm; you can write programs or libraries that are largely -procedural, object-oriented, or functional. In a large program, -different sections might be written in different styles; the GUI might -be object-oriented while the processing logic is procedural or -functional, for example. +The designers of some computer languages have chosen one approach to +programming that's emphasized. This often makes it difficult to +write programs that use a different approach. Other languages are +multi-paradigm languages that support several different approaches. Lisp, +C++, and Python are multi-paradigm; you can write programs or +libraries that are largely procedural, object-oriented, or functional +in all of these languages. In a large program, different sections +might be written using different approaches; the GUI might be object-oriented +while the processing logic is procedural or functional, for example. In a functional program, input flows through a set of functions. Each function operates on its input and produces some output. Functional -style avoids making functions have side effects that modify internal +style frowns upon functions with side effects that modify internal state or make other changes that aren't visible in the function's -return value. Avoiding side effects means not using data structures -that get updated as a program runs. Some languages are very strict -about this and don't even have assignment statements such as ``a=3`` -or ``c = a + b``, but it's difficult to avoid all side effects; some -Python code, such as a ``print`` statement or a ``time.sleep(1)``, -returns nothing and is only called for its side effects. +return value. Functions that have no side effects at all are +called **purely functional**. +Avoiding side effects means not using data structures +that get updated as a program runs; every function's output +must only depend on its input. + +Some languages are very strict about purity and don't even have +assignment statements such as ``a=3`` or ``c = a + b``, but it's +difficult to avoid all side effects. Printing to the screen or +writing to a disk file are side effects, for example. For example, in +Python a ``print`` statement or a ``time.sleep(1)`` both return no +useful value; they're only called for their side effects of sending +some text to the screen or pausing execution for a second. Python programs written in functional style usually won't go to the -extreme of avoiding all I/O or all assignments. The implementation of -a function will still use assignments to local variables, but won't -touch any global variables or have other side effects. +extreme of avoiding all I/O or all assignments; instead, they'll +provide a functional-appearing interface but will use non-functional +features internally. For example, the implementation of a function +will still use assignments to local variables, but won't modify global +variables or have other side effects. Functional programming can be considered the opposite of object-oriented programming. Objects are little capsules containing @@ -122,29 +135,30 @@ you use daily (the Python interpreter, your XML parser, your web browser) could be proven correct. Even if you wrote down or generated a proof, there would then be the question of verifying the proof; -maybe you only ***think*** you've proved that the program correct. +maybe there's an error in it, and you only ***think*** you've proved +that the program correct. Modularity '''''''''''''''''''''' A more practical benefit of functional programming is that it forces -you to break apart your problem into small pieces. It's easier to -specify and write a small function that does one thing than a large -function that performs a complicated transformation. Small functions -are easier to read and to check for errors. A program that's -structured into many small pieces is a **modular** program. +you to break apart your problem into small pieces. Programs are more +modular as a result. It's easier to specify and write a small +function that does one thing than a large function that performs a +complicated transformation. Small functions are also easier to read +and to check for errors. Ease of debugging and testing '''''''''''''''''''''''''''''''''' -It's also easier to test and to debug a functional-style program. +Testing and debugging a functional-style program is easier. -It's easier to debug because functions are generally small and clearly -specified. When a program doesn't work, each function is an interface -point where you can check that the data is correct. You can look at -the intermediate inputs and outputs to quickly isolate the function -that's responsible for a bug. +Debugging is simplified because functions are generally small and +clearly specified. When a program doesn't work, each function is an +interface point where you can check that the data are correct. You +can look at the intermediate inputs and outputs to quickly isolate the +function that's responsible for a bug. Testing is easier because each function is a potential subject for a unit test. Functions don't depend on system state that needs to be @@ -162,7 +176,7 @@ others will be useful in a wide variety of programs. For example, a function that takes a directory path and returns all the XML files in the directory, or a function that takes a filename and returns its -contents would be useful in many different situations. +contents, can be applied to many different situations. Over time you'll form a personal library of utilities. Often you'll assemble new programs by arranging existing functions in a new @@ -171,18 +185,17 @@ - Iterators ----------------------- -Iterators are an important foundation for writing functional-style -programs. +I'll start by looking at a Python language feature that's an important +foundation for writing functional-style programs: iterators. -An iterator is an object representing a stream of data that returns -the data one element at a time. A Python iterator must support a -method called ``next()`` that takes no arguments and always returns -the next element of the stream. If there are no more elements in the -stream, ``next()`` must raise the ``StopIteration`` exception. +An iterator is an object representing a stream of data; this object +returns the data one element at a time. A Python iterator must +support a method called ``next()`` that takes no arguments and always +returns the next element of the stream. If there are no more elements +in the stream, ``next()`` must raise the ``StopIteration`` exception. Iterators don't have to be finite, though; it's perfectly reasonable to write an iterator that produces an infinite stream of data. @@ -190,7 +203,8 @@ to return an iterator that will return the object's contents or elements, raising ``TypeError`` if the object doesn't support iteration. Several of Python's built-in data types support iteration, -the most common being lists and dictionaries. +the most common being lists and dictionaries. An object is called +an **iterable** object if you can get an iterator for it. You can experiment with the iteration interface manually:: @@ -221,8 +235,8 @@ for i in obj: print i -Iterators can be materialized as lists or tuples by using the ``list()`` or ``tuple()`` -constructor functions:: +Iterators can be materialized as lists or tuples by using the +``list()`` or ``tuple()`` constructor functions:: >>> L = [1,2,3] >>> iterator = iter(L) @@ -285,10 +299,14 @@ Dec 12 Oct 10 -This is just the default behaviour. If you want to iterate over keys, -values, or key/value pairs, you can explicitly call the -``iterkeys()``, ``itervalues()``, or ``iteritems()`` -methods to get an appropriate iterator. +Note that the order is essentially random, because it's based on the +hash ordering of the objects in the dictionary. + +Applying ``iter()`` to a dictionary always loops over the keys, but +dictionaries have methods that return other iterators. If you want to +iterate over keys, values, or key/value pairs, you can explicitly call +the ``iterkeys()``, ``itervalues()``, or ``iteritems()`` methods to +get an appropriate iterator. The ``dict()`` constructor can accept an iterator that returns a finite stream of ``(key, value)`` tuples:: @@ -317,10 +335,15 @@ List comprehensions ----------------------- +.. comment + + Maybe gencomps should be described and emphasized, and listcomps + be mentioned as an afterthought. + Two common operations on a stream are performing some operation for every element, or selecting a subset of elements that meet some condition. For example, given a list of strings, you might want to -pull out all the strings containing a given substring, or strip off +pull out all the strings containing a given substring or strip off trailing whitespace from each line. List comprehensions (or "listcomps") are a concise notation for such @@ -340,9 +363,8 @@ containing the resulting lines, not an iterator. This means that list comprehensions aren't very useful if you're working with iterators that return an infinite or a very large stream of data. Later we'll -discuss generator expressions, a feature that provides similar -capabilities as list comprehensions but can be used with infinite -iterators. +discuss generator expressions, which have the capabilities of list +comprehensions but can be used with infinite iterators. List comprehensions have the form:: @@ -403,8 +425,8 @@ ----------------------- Generator expressions are written like list comprehensions, but are -surrounded by parentheses (\samp{()}) and not square brackets -(\samp{[]}). The result of a generator expression +surrounded by parentheses ("()") and not square brackets +("[]"). The result of a generator expression is an iterator that returns the computed elements without materializing a list containing them all. @@ -520,7 +542,7 @@ -New generator features in Python 2.5 +Passing values into a generator '''''''''''''''''''''''''''''''''''''''''''''' In Python 2.4 and earlier, generators only produced output. Once a @@ -625,8 +647,7 @@ Built-in functions ---------------------------------------------- -Let's look in more detail at those built-in functions that are -relevant to iterators. +Let's look in more detail at built-in functions often used with iterators. Two Python's built-in functions, ``map()`` and ``filter()``, are somewhat obsolete; they duplicate the features of list comprehensions @@ -701,7 +722,7 @@ If you use ``operator.add`` with ``reduce()``, you'll add up all the elements of the iterable. This case is so common that there's a special -built-in for it:: +built-in called ``sum()`` to compute it:: reduce(operator.add, [1,2,3,4], 0) => 10 @@ -710,8 +731,20 @@ sum([]) => 0 -``enumerate(iter)`` counts off the elements in the iterable, return -pairs of the count and each element. +For many uses of ``reduce()``, though, it can be clearer to just write +the obvious ``for`` loop:: + + # Instead of: + product = reduce(operator.mul, [1,2,3], 1) + + # You can write: + product = 1 + for i in [1,2,3]: + product *= i + + +``enumerate(iter)`` counts off the elements in the iterable, returning +2-tuples containing the count and each element. :: @@ -724,7 +757,7 @@ f = open('data.txt', 'r') for i, line in enumerate(f): if line.strip() == '': - print 'Blank line at', line + print 'Blank line at line #%i' % i ``sorted(iterable, [cmp=None], [key=None], [reverse=False)`` collects all the elements of the iterable into a list, sorts @@ -745,12 +778,12 @@ [9878, 9828, 8442, 7953, 6431, 6213, 2207, 769] (For a more detailed discussion of sorting, see the Sorting mini-HOWTO -in the Python wiki at \url{http://wiki.python.org/moin/SortingHowto}.) +in the Python wiki at http://wiki.python.org/moin/SortingHowto.) The ``any(iter)`` and ``all(iter)`` built-ins look at the truth values of an iterable's contents. ``any()`` returns True if any element in the iterable is a true value, and ``all()`` -retturn True if all of the elements are true values:: +returns True if all of the elements are true values:: any([0,1,0]) => True @@ -770,16 +803,13 @@ ---------------------------------------------- When writing functional-style programs, you'll often need little -functions that act as predicates or that combine list elements in some -particular way. +functions that act as predicates or that combine elements in some way. -If there's a Python built-in that's suitable, or a module has a -function that does what you need, you don't need to define a new -function at all:: +If there's a Python built-in or a module function that's suitable, you +don't need to define a new function at all:: stripped_lines = [line.strip for line in lines] existing_files = filter(os.path.exists, file_list) - XXX pi-notation example If the function you need doesn't exist, you need to write it. One way to write small functions is to use the ``lambda`` statement. ``lambda`` @@ -804,7 +834,8 @@ def adder(x,y): return x + y -Which alternative is preferable? That's a style question. +Which alternative is preferable? That's a style question; my general +view is to avoid it. ``lambda`` is quite limited in the functions it can define. The result has to be computable as a single expression, which means you @@ -815,7 +846,22 @@ :: - freq = reduce(lambda a, b: (0, a[1] + b[1]), items)[1] + freq = reduce(lambda a, b: (0, a[1] + b[1]), items)[1] + +You can figure it out, but it takes time to disentangle the function +to figure out what's going on. Using a short nested +``def`` statements makes things a little bit better:: + + def combine (a, b): + return 0, a[1] + b[1] + + return reduce(combine_freq, items)[1] + +It would be best of all if I had simply used a ``for`` loop:: + + total = 0 + for a, b in items: + total += b Fredrik Lundh once suggested the following set of rules for refactoring uses of ``lambda``: @@ -827,28 +873,16 @@ 4) Convert the lambda to a def statement, using that name. 5) Remove the comment. -Personally I try to avoid lambdas, favoring short nested -``def`` statements like this:: - - def output_total_freq (items): - """Takes a list of (element, frequency count) tuples - and returns the total number of occurrences. - """ - - def combine (a, b): - return 0, a[1] + b[1] - - return reduce(combine_freq, items)[1] - -You might disagree that this style is better. +I really like these rules, but you're free todisagree that this style +is better. The itertools module ----------------------- The ``itertools`` module contains a number of commonly-used iterators -as well as functions for combining iterators. This section will -introduce the module's contents using small examples. +as well as functions for combining several iterators. This section +will introduce the module's contents by showing small examples. ``itertools.count(n)`` returns an infinite stream of integers, increasing by 1 each time. You can optionally supply the @@ -861,7 +895,7 @@ ``itertools.cycle(iter)`` saves a copy of the contents of a provided iterable and returns a new iterator that returns its elements from -first to last, and then repeats these elements infinitely. +first to last. The new iterator will repeat these elements infinitely. :: @@ -878,10 +912,10 @@ itertools.repeat('abc', 5) => abc, abc, abc, abc, abc -``itertools.chain(iterA, iterB, ...)`` takes an arbitrary number of -iterables as input, and returns all the elements of the first iterator, -then all the elements of the second, until all of the iterables -have been exhausted. +``itertools.chain(iterA, iterB, ...)`` takes an arbitrary number of +iterables as input, and returns all the elements of the first +iterator, then all the elements of the second, and so on, until all of +the iterables have been exhausted. :: @@ -895,25 +929,25 @@ ('a', 1), ('b', 2), ('c', 3) This iterator is intended to be used with iterables that are all of -the same length. If the iterables are of different lengths, -the resulting stream will be the same length as the shortest iterable. -Unfortunately, if you passed in any iterators as arguments, -an element may have been taken from the longer iterators -and discarded, meaning that you can't keep using them; if you do, -you risk skipping a discarded element. +the same length. If the iterables are of different lengths, the +resulting stream will be the same length as the shortest iterable. :: itertools.izip(['a', 'b'], (1, 2, 3)) => ('a', 1), ('b', 2) +You should avoid doing this, though, because an element may be taken +from the longer iterators and discarded. This means you can't go on +to use the iterators further because you risk skipping a discarded +element. ``itertools.islice(iter, [start], stop, [step])`` returns a stream that's a slice of the iterator. It can return the first ``stop`` elements. If you supply a starting index, you'll get ``stop-start`` elements, and if you supply a value for ``step` elements will be -skipped. Unlike Python's string and list slicing, you can't use -negative values for ``start``, ``stop``, or ``step``. +skipped accordingly. Unlike Python's string and list slicing, you +can't use negative values for ``start``, ``stop``, or ``step``. :: @@ -943,7 +977,7 @@ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... -Two functions are for calling other functions on the contents of an +Two functions are used for calling other functions on the contents of an iterable. ``itertools.imap(f, iterA, iterB, ...)`` returns @@ -953,13 +987,13 @@ itertools.imap(operator.add, [5, 6, 5], [1, 2, 3]) => 6, 8, 8 -(The ``operator`` module contains a set of functions +The ``operator`` module contains a set of functions corresponding to Python's operators. Some examples are ``operator.add(a, b)`` (adds two values), ``operator.ne(a, b)`` (same as ``a!=b``), and ``operator.attrgetter('id')`` (returns a callable that -fetches the ``"id"`` attribute), +fetches the ``"id"`` attribute). ``itertools.starmap(func, iter)`` assumes that the iterable will return a stream of tuples, and calls ``f()`` using these tuples as the @@ -1017,10 +1051,11 @@ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ... -The last function, ``itertools.groupby(iter, key_func=None)``, is the -most complicated. ``key_func(elem)`` is a function that can compute a -key value for each element returned by the iterable. If you don't -supply a key function, the key is simply each element itself. +The last function I'll discuss, ``itertools.groupby(iter, +key_func=None)``, is the most complicated. ``key_func(elem)`` is a +function that can compute a key value for each element returned by the +iterable. If you don't supply a key function, the key is simply each +element itself. ``groupby()`` collects all the consecutive elements from the underlying iterable that have the same key value, and returns a stream @@ -1068,7 +1103,7 @@ For programs written in a functional style, you'll sometimes want to construct variants of existing functions that have some of the parameters filled in. Consider a Python function ``f(a, b, c)``; you -could create a new function ``g(b, c)`` that was equivalent to +may wish to create a new function ``g(b, c)`` that was equivalent to ``f(1, b, c)``. This is called "partial function application". The constructor for ``partial`` takes the arguments ``(function, arg1, @@ -1089,15 +1124,6 @@ server_log('Unable to open socket') -Topics to place ------------------------------ - -XXX os.walk() - -XXX Need a large example. - -======= - Acknowledgements ------------------------------ @@ -1108,6 +1134,19 @@ .. comment + Topics to place + ----------------------------- + + XXX os.walk() + + XXX Need a large example. + + But will an example add much? I'll post a first draft and see + what the comments say. + +.. comment + + Original outline: Introduction Idea of FP Programs built out of functions @@ -1141,6 +1180,9 @@ .. comment + Handy little function for printing part of an iterator -- used + while writing this document. + import itertools def print_iter(it): slice = itertools.islice(it, 10) @@ -1153,9 +1195,30 @@ References -------------------- -SICP +General +''''''''''''''' + +**Structure and Interpretation of Computer Programs**, by +Harold Abelson and Gerald Jay Sussman with Julie Sussman. + +Full text at http://mitpress.mit.edu/sicp/. + +A classic textbook of computer science. Chapters 2 and 3 discuss the +use of sequences and streams to organize the data flow inside a +program. The book uses Scheme for its examples, but many of the +design approaches described in these chapters are applicable to +functional-style Python code. + +http://en.wikipedia.org/wiki/Functional_programming: +General Wikipedia entry describing functional programming. + + +Python documentation +''''''''''''''''''''''''''' -Relevant manual sections +http://docs.python.org/lib/module-itertools.html: +Documentation ``for the itertools`` module. -XXX +http://docs.python.org/lib/module-operator.html: +Documentation ``for the operator`` module. From buildbot at python.org Fri Jun 30 23:07:34 2006 From: buildbot at python.org (buildbot at python.org) Date: Fri, 30 Jun 2006 21:07:34 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20060630210734.38F901E4002@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/436 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From python-checkins at python.org Fri Jun 30 23:16:06 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 30 Jun 2006 23:16:06 +0200 (CEST) Subject: [Python-checkins] r47184 - sandbox/trunk/Doc/functional.rst Message-ID: <20060630211606.815451E4002@bag.python.org> Author: andrew.kuchling Date: Fri Jun 30 23:16:06 2006 New Revision: 47184 Modified: sandbox/trunk/Doc/functional.rst Log: Add version; note reservation about URL; add history Modified: sandbox/trunk/Doc/functional.rst ============================================================================== --- sandbox/trunk/Doc/functional.rst (original) +++ sandbox/trunk/Doc/functional.rst Fri Jun 30 23:16:06 2006 @@ -1,8 +1,12 @@ Functional Programming HOWTO ================================ +**Version 0.1** + (This is a first draft. Please send comments/error -reports/suggestions to amk at amk.ca.) +reports/suggestions to amk at amk.ca. This URL is probably not going to +be the final location of the document, so be careful about linking to +it -- you may want to add a disclaimer.) In this document, we'll take a tour of Python's features suitable for implementing programs in a functional style. After an introduction to @@ -1124,13 +1128,14 @@ server_log('Unable to open socket') -Acknowledgements ------------------------------- +Revision History and Acknowledgements +------------------------------------------------ The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this article: Raymond Hettinger, Jim Jewett. +Version 0.1: posted June 30 2006. .. comment From python-checkins at python.org Fri Jun 30 23:41:12 2006 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 30 Jun 2006 23:41:12 +0200 (CEST) Subject: [Python-checkins] r47185 - sandbox/trunk/Doc/functional.rst Message-ID: <20060630214112.4387E1E4002@bag.python.org> Author: andrew.kuchling Date: Fri Jun 30 23:41:11 2006 New Revision: 47185 Modified: sandbox/trunk/Doc/functional.rst Log: Typo fix Modified: sandbox/trunk/Doc/functional.rst ============================================================================== --- sandbox/trunk/Doc/functional.rst (original) +++ sandbox/trunk/Doc/functional.rst Fri Jun 30 23:41:11 2006 @@ -363,7 +363,7 @@ stripped_list = [line.strip() for line in line_list if line != ""] -Note that in all case the resulting ``stripped_list`` is a Python list +Note that in all cases the resulting ``stripped_list`` is a Python list containing the resulting lines, not an iterator. This means that list comprehensions aren't very useful if you're working with iterators that return an infinite or a very large stream of data. Later we'll From neal at metaslash.com Fri Jun 30 23:07:46 2006 From: neal at metaslash.com (Neal Norwitz) Date: Fri, 30 Jun 2006 17:07:46 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20060630210746.GA21379@python.psfb.org> test_socket leaked [212, -212, 0] references