From tim.peters at gmail.com Sun Aug 1 01:32:04 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 1 01:32:13 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200407281903.i6SJ3ea06175@guido.python.org> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> Message-ID: <1f7befae0407311632442e488b@mail.gmail.com> [Guido, patches PyImport_ExecCodeModuleEx to remove a non-pre-existing module from sys.modules if executing the module code raises an exception] It fixes at least one spurious -r test failure on Windows. Before: $ regrtest.py -uall test_sundry test___all__ test_sundry test___all__ test test___all__ failed -- Traceback (most recent call last): File "C:\Code\python\lib\test\test___all__.py", line 150, in test_all self.check_all("rlcompleter") File "C:\Code\python\lib\test\test___all__.py", line 40, in check_all "%s has no __all__ attribute" % modname) File "C:\Code\python\lib\test\test_support.py", line 208, in verify raise TestFailed(reason) TestFailed: rlcompleter has no __all__ attribute After: $ regrtest.py -uall test_sundry test___all__ test_sundry test___all__ All 2 tests OK. It does not fix at least one other one, because a second attempt to import curses still "works" on Windows: >>> import curses Traceback (most recent call last): File "", line 1, in ? File "C:\Code\python\lib\curses\__init__.py", line 15, in ? from _curses import * ImportError: No module named _curses >>> import curses >>> curses >>> So whatever this patch does , it breaks down somehow for non-importable packages. I guess that any path thru import.c calling PyImport_AddModule() before calling PyImport_ExecCodeModuleEx() will suffer similarly (because the former convinces the latter then that the module "is pre-existing", even if it really wasn't, and the latter won't remove it from sys.modules then despite that running its code fails). It breaks this Zope3 doctest twice, for an obvious reason: def test_bad_import(): """ >>> c = config.ConfigurationContext() >>> c.resolve('zope.configuration.tests.victim.x') Traceback (most recent call last): ... ConfigurationError: Couldn't import zope.configuration.tests.victim,""" \ """ No module named bad_to_the_bone Cleanup: >>> del sys.modules['zope.configuration.tests.victim'] >>> del sys.modules['zope.configuration.tests.bad'] """ That is, this test's cleanup currently requires that insane modules get left behind. It also breaks this Zope3 test, which uses a custom import hook: ERROR: test_reporting_exception_during_import (zope.importtool.tests.test_hook.HookTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\unittest.py", line 260, in run testMethod() File "C:\Code\ZopeX3\src\zope\importtool\tests\test_hook.py", line 198, in test_reporting_exception_during_import import error File "C:\Code\ZopeX3\src\zope\importtool\hook.py", line 72, in importhook v = self.previous(name, globals, locals, fromlist) KeyError: 'zope.importtool.tests.error' self.previous there appears to be bound to __builtin__.__import__. I don't understand this code, but expect it's shallow to someone who does. Overall, I'm +1, but think it needs more work to plug the "curses on Windows" kind of hole. From tismer at stackless.com Sun Aug 1 02:18:48 2004 From: tismer at stackless.com (Christian Tismer) Date: Sun Aug 1 02:16:25 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200407291820.i6TIKRi10773@guido.python.org> References: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz> <200407291820.i6TIKRi10773@guido.python.org> Message-ID: <410C36E8.7060102@stackless.com> Guido van Rossum wrote: >>It's going to be difficult to ensure that nothing ever gets a >>reference to a broken module, because of circular imports. Suppose A >>imports B, which imports A. If A's initialisation subsequently fails, >>then even if A is removed from sys.modules, B still contains a >>reference to the broken A. Well, circular imports first appeared to me like the reverse problem of circular references. The latter was only solvable by a garbage collector. Well, it's simpler after all. Circular imports, to be made consistent, would need something like a final "commit" after all imports succeeded. I think of a simple, generator-like "undoable assignment" to sys.modules, like ICON's "<-" operator. When a top-level import starts, the module gets into sys.modules, but this is also recorded as not finished. Further imports can happen, until this is all done. Then, the pending undos are cancelled, this is the "commit". If any uncaught exception occours, the latent undo is executed for all imports started in this context, and we end up with an exception and (hopefully) almost no side effects. Maybe this could be user-extensible, as on-failure-undo actions. > Yeah, but I don't mind; my patch is still a big improvement in other > cases and doesn't really make things any worse in the above case (B > has a reference to a broken A, just like without my patch). It is definately an improvement, much more at what people would expect, anyway. > Did anybody look at my patch? Shall I check it in? Yes I did. +1 -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tim.peters at gmail.com Sun Aug 1 07:50:57 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 1 07:51:01 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae0407311632442e488b@mail.gmail.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> <1f7befae0407311632442e488b@mail.gmail.com> Message-ID: <1f7befae04073122501f7e185f@mail.gmail.com> Attached patch builds on Guido's: - refactored - load_package() now also removes stuff from sys.modules if an error occurs and the stuff wasn't in sys.modules to begin with - patches test_pkgimport so it passes again (it was relying on junk remaining in sys.modules in one spot) - regrtest -uall passes on Windows with this patch (247 tests OK, 37 tests skipped). - oops -- fixes obsolute stuff in a docstring too (the LaTeX docs had already been corrected) - does not remove insane workarounds in .py modules (contortions trying to worm around that non-working modules were left behind by failing imports) "curses on Windows" no longer screws up. Before: $ regrtest.py -uall test_sundry test___all__ test_curses test_sundry test___all__ test test___all__ failed -- Traceback (most recent call last): File "C:\Code\python\lib\test\test___all__.py", line 150, in test_all self.check_all("rlcompleter") File "C:\Code\python\lib\test\test___all__.py", line 40, in check_all "%s has no __all__ attribute" % modname) File "C:\Code\python\lib\test\test_support.py", line 208, in verify raise TestFailed(reason) TestFailed: rlcompleter has no __all__ attribute test_curses test test_curses crashed -- exceptions.AttributeError: 'module' object has no attribute 'endwin' 1 test OK. 2 tests failed: test___all__ test_curses After: $ regrtest.py -uall test_sundry test___all__ test_curses test_sundry test___all__ test_curses test_curses skipped -- No module named _curses 2 tests OK. 1 test skipped: test_curses Those skips are all expected on win32. -------------- next part -------------- Index: Lib/test/test_pkgimport.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_pkgimport.py,v retrieving revision 1.7 diff -u -r1.7 test_pkgimport.py --- Lib/test/test_pkgimport.py 9 Aug 2002 16:37:36 -0000 1.7 +++ Lib/test/test_pkgimport.py 1 Aug 2004 05:30:10 -0000 @@ -66,12 +66,11 @@ try: __import__(self.module_name) except NameError: pass else: raise RuntimeError, 'Failed to induce NameError.' - module = __import__(self.module_name).foo # ...now change the module so that the NameError doesn't # happen self.rewrite_file('%s = 1' % var) - reload(module) + module = __import__(self.module_name).foo self.assertEqual(getattr(module, var), 1) Index: Python/import.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/import.c,v retrieving revision 2.233 diff -u -r2.233 import.c --- Python/import.c 27 Jun 2004 16:51:46 -0000 2.233 +++ Python/import.c 1 Aug 2004 05:30:11 -0000 @@ -557,6 +557,33 @@ return m; } +/* The same as PyImport_AddModule, except also sets created to 1 if a + * new module object is created, or to 0 if an existing module object + * is found. + */ +static PyObject * +_AddModuleEx(const char *name, int *created) +{ + PyObject *modules = PyImport_GetModuleDict(); + PyObject *m; + + *created = 0; + if ((m = PyDict_GetItemString(modules, name)) != NULL && + PyModule_Check(m)) + return m; + m = PyImport_AddModule((char *)name); + if (m != NULL) + *created = 1; + return m; +} + +/* Remove name from sys.modules. It must already be in sys.modules. */ +static void +_RemoveModule(const char *name) +{ + PyObject *modules = PyImport_GetModuleDict(); + PyDict_DelItemString(modules, name); /* if it fails, too bad */ +} /* Execute a code object in a module and return the module object WITH INCREMENTED REFERENCE COUNT */ @@ -572,8 +599,9 @@ { PyObject *modules = PyImport_GetModuleDict(); PyObject *m, *d, *v; + int created; - m = PyImport_AddModule(name); + m = _AddModuleEx(name, &created); if (m == NULL) return NULL; /* If the module is being reloaded, we get the old module back @@ -582,7 +610,7 @@ if (PyDict_GetItemString(d, "__builtins__") == NULL) { if (PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()) != 0) - return NULL; + goto error; } /* Remember the filename as the __file__ attribute */ v = NULL; @@ -601,7 +629,7 @@ v = PyEval_EvalCode((PyCodeObject *)co, d, d); if (v == NULL) - return NULL; + goto error; Py_DECREF(v); if ((m = PyDict_GetItemString(modules, name)) == NULL) { @@ -614,6 +642,11 @@ Py_INCREF(m); return m; + + error: + if (created) + _RemoveModule(name); + return NULL; } @@ -888,13 +921,16 @@ static PyObject * load_package(char *name, char *pathname) { - PyObject *m, *d, *file, *path; + PyObject *m, *d; + PyObject *file = NULL; + PyObject *path = NULL; int err; char buf[MAXPATHLEN+1]; FILE *fp = NULL; struct filedescr *fdp; + int created; - m = PyImport_AddModule(name); + m = _AddModuleEx(name, &created); if (m == NULL) return NULL; if (Py_VerboseFlag) @@ -903,19 +939,15 @@ d = PyModule_GetDict(m); file = PyString_FromString(pathname); if (file == NULL) - return NULL; + goto error; path = Py_BuildValue("[O]", file); - if (path == NULL) { - Py_DECREF(file); - return NULL; - } + if (path == NULL) + goto error; err = PyDict_SetItemString(d, "__file__", file); if (err == 0) err = PyDict_SetItemString(d, "__path__", path); - if (err != 0) { - m = NULL; - goto cleanup; - } + if (err != 0) + goto error; buf[0] = '\0'; fdp = find_module(name, "__init__", path, buf, sizeof(buf), &fp, NULL); if (fdp == NULL) { @@ -930,9 +962,15 @@ m = load_module(name, fp, buf, fdp->type, NULL); if (fp != NULL) fclose(fp); + goto cleanup; + + error: + m = NULL; cleanup: Py_XDECREF(path); Py_XDECREF(file); + if (created && m == NULL) + _RemoveModule(name); return m; } @@ -1272,7 +1310,7 @@ if (Py_VerboseFlag > 1) PySys_WriteStderr("# trying %s\n", buf); filemode = fdp->mode; - if (filemode[0] == 'U') + if (filemode[0] == 'U') filemode = "r" PY_STDIOTEXTMODE; fp = fopen(buf, filemode); if (fp != NULL) { @@ -2520,7 +2558,7 @@ { FILE *fp; if (fob == NULL) { - if (mode[0] == 'U') + if (mode[0] == 'U') mode = "r" PY_STDIOTEXTMODE; fp = fopen(pathname, mode); if (fp == NULL) @@ -2692,9 +2730,9 @@ The module name must include the full package name, if any."); PyDoc_STRVAR(doc_lock_held, -"lock_held() -> 0 or 1\n\ -Return 1 if the import lock is currently held.\n\ -On platforms without threads, return 0."); +"lock_held() -> boolean\n\ +Return True if the import lock is currently held, else False.\n\ +On platforms without threads, return False."); PyDoc_STRVAR(doc_acquire_lock, "acquire_lock() -> None\n\ From martin at v.loewis.de Sun Aug 1 12:47:59 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 1 12:47:59 2004 Subject: [Python-Dev] Changes to the installer builder Message-ID: <410CCA5F.3070900@v.loewis.de> I have added some documentation to msi.py, please let me know if something is still missing. I have also rearranged the versioning. msi.py now supports two "product lines": snapshots, and releases. Snapshots can be generated by anybody, whereas releases are only published by python.org. Each line has their own ProductVersion scheme: snapshots use major.minor.currentday; official releases use the version from the Python DLL. The relevant information is now extracted from patchlevel.h. Accordingly, there is no need to specify current_version anymore in config.py, and the "alpha" setting has been renamed to "snapshot". Regards, Martin From tim.peters at gmail.com Sun Aug 1 18:41:32 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 1 18:41:46 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae0407311632442e488b@mail.gmail.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> <1f7befae0407311632442e488b@mail.gmail.com> Message-ID: <1f7befae04080109413fe3a967@mail.gmail.com> [Tim] > ... > I guess that any path thru import.c calling PyImport_AddModule() before > calling PyImport_ExecCodeModuleEx() will suffer similarly (because the > former convinces the latter then that the module "is pre-existing", even if it > really wasn't, and the latter won't remove it from sys.modules then despite > that running its code fails). And I posted a patch later that "fixed this" in one case involving packages. I don't know much about import details, but I believe there are other cases -- just search for PyImport_AddModule. Some look harmless, but some don't: - PyImport_ImportFrozenModule. In case of a package, creates a module object first to set its __path__, before running the code. - zipimporter_load_module (in zipimport.c). Similarly, but also sets __loader__ before running the code, package or not. - Py_InitModule4 (in modsupport.c) leaves a damaged module behind if initialization fails. "A problem" is that PyImport_ExecCodeModuleEx seems to be the only real choke point, and is making a distinction between "old" and "new" modules that doesn't always match its callers' realities. This distinction came from here: [Guido] > What should it do if the module already existed (e.g. when used by reload())? > Strawman answer: leave it there -- the reload() semantics and common use > cases are best served by that. I understand the point there for reload(), but don't know of other use cases (let alone common ones ) for wanting to leave a module in sys.modules when its code fails. What are they? Maybe it would be better for PyImport_ExecCodeModuleEx to purge a broken module unconditionally, and change PyImport_ReloadModule to put it back? From guido at python.org Sun Aug 1 22:13:18 2004 From: guido at python.org (Guido van Rossum) Date: Sun Aug 1 22:13:26 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: Your message of "Sun, 01 Aug 2004 02:18:48 +0200." <410C36E8.7060102@stackless.com> References: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz> <200407291820.i6TIKRi10773@guido.python.org> <410C36E8.7060102@stackless.com> Message-ID: <200408012013.i71KDI726275@guido.python.org> > Circular imports, to be made consistent, would need > something like a final "commit" after all imports succeeded. Silly idea. Typically, circular imports aren't planned, they just happen, and (usually) just happen to work. --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer at stackless.com Sun Aug 1 23:03:32 2004 From: tismer at stackless.com (Christian Tismer) Date: Sun Aug 1 23:01:06 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408012013.i71KDI726275@guido.python.org> References: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz> <200407291820.i6TIKRi10773@guido.python.org> <410C36E8.7060102@stackless.com> <200408012013.i71KDI726275@guido.python.org> Message-ID: <410D5AA4.2060901@stackless.com> Guido van Rossum wrote: >>Circular imports, to be made consistent, would need >>something like a final "commit" after all imports succeeded. > > > Silly idea. Typically, circular imports aren't planned, they just > happen, and (usually) just happen to work. Yes, you name it, if you are talking about allowing circular imports without precautions and not expecting problems. Like circular references which aren't planned, they just happen, but they truely work fine after GC was introduced. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 want to see a silly guy's work? http://www.stackless.com/ From tim.peters at gmail.com Mon Aug 2 01:11:55 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 01:11:58 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae04080109413fe3a967@mail.gmail.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> <1f7befae0407311632442e488b@mail.gmail.com> <1f7befae04080109413fe3a967@mail.gmail.com> Message-ID: <1f7befae04080116116175c3a1@mail.gmail.com> [Tim] > ... > I understand the point there for reload(), but don't know of other use > cases (let alone common ones ) for wanting to leave a module in > sys.modules when its code fails. What are they? Maybe it would be > better for PyImport_ExecCodeModuleEx to purge a broken module > unconditionally, and change PyImport_ReloadModule to put it back? Attached is a patch that takes that approach. Code feels simpler/cleaner, since it makes a special case out of reload() instead of a special case out of everything except reload(). Still haven't thought of a case other than reload() that wants special treatment. The whole test suite still passes; it still fixes all the spurious test failures I know about on Windows when running the suite with -r. The internal _RemoveModule() is incidentally more careful in this version of the patch. The last patch's new _AddModuleEx() no longer exists. -------------- next part -------------- Index: fsrefs.py =================================================================== RCS file: /cvs-repository/ZODB3/Tools/Attic/fsrefs.py,v retrieving revision 1.7.4.3 diff -c -u -r1.7.4.3 fsrefs.py --- fsrefs.py 10 Jul 2004 02:42:11 -0000 1.7.4.3 +++ fsrefs.py 11 Jul 2004 01:45:23 -0000 @@ -20,12 +20,13 @@ fsrefs.py checks object sanity by trying to load the current revision of every object O in the database, and also verifies that every object -directly reachable from each such O exists in the database. Note that -application code implementing objects (or at least defining their classes) -must be available, an d on PYTHONPATH, for fsrefs to work, because objects -are actually loaded. On a ZEO server, all the application code typically -isn't present, so it may be difficult to run fsrefs usefully on a ZEO -server. +directly reachable from each such O exists in the database. + +It's hard to explain exactly what it does because it relies on undocumented +features in Python's cPickle module: many of the crucial steps of loading +an object are taken, but application objects aren't actually created. This +saves a lot of time, and allows fsrefs to be run even if the code +implementing the object classes isn't available. A read-only connection to the specified FileStorage is made, but it is not recommended to run fsrefs against a live FileStorage. Because a live @@ -36,10 +37,7 @@ fsrefs doesn't normally produce any output. If an object fails to load, the oid of the object is given in a message saying so, and if -v was specified then the traceback corresponding to the load failure is also displayed -(this is the only effect of the -v flag, and is usually very helpful; -v is -recommended for normal use). Note that, as mentioned above, a common -problem is to get a "failed to load" message simply because the module -containing the class of the object isn't on PYTHONPATH. +(this is the only effect of the -v flag). Two other kinds of errors are also detected, one strongly related to "failed to load", when an object O loads OK, and directly refers to a @@ -56,6 +54,10 @@ seen by fsrefs before P, an "O refers to the unloadable P" message will not be produced; a message saying that P can't be loaded will be produced when fsrefs later tries to load P, though. + +fsrefs also (indirectly) checks that the .index file is sane, because +fsrefs uses the index to get its idea of what constitutes "all the objects +in the database". Note these limitations: because fsrefs only looks at the current revision of objects, it does not attempt to load objects in versions, or non-current From tim.peters at gmail.com Mon Aug 2 01:14:39 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 01:14:42 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae04080116116175c3a1@mail.gmail.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> <1f7befae0407311632442e488b@mail.gmail.com> <1f7befae04080109413fe3a967@mail.gmail.com> <1f7befae04080116116175c3a1@mail.gmail.com> Message-ID: <1f7befae0408011614190382dc@mail.gmail.com> Oops, attached a wrong patch. Correct patch attached. -------------- next part -------------- Index: Lib/test/test_pkgimport.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_pkgimport.py,v retrieving revision 1.7 diff -u -r1.7 test_pkgimport.py --- Lib/test/test_pkgimport.py 9 Aug 2002 16:37:36 -0000 1.7 +++ Lib/test/test_pkgimport.py 1 Aug 2004 23:01:43 -0000 @@ -66,12 +66,11 @@ try: __import__(self.module_name) except NameError: pass else: raise RuntimeError, 'Failed to induce NameError.' - module = __import__(self.module_name).foo # ...now change the module so that the NameError doesn't # happen self.rewrite_file('%s = 1' % var) - reload(module) + module = __import__(self.module_name).foo self.assertEqual(getattr(module, var), 1) Index: Python/import.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/import.c,v retrieving revision 2.233 diff -u -r2.233 import.c --- Python/import.c 27 Jun 2004 16:51:46 -0000 2.233 +++ Python/import.c 1 Aug 2004 23:01:44 -0000 @@ -557,10 +557,25 @@ return m; } +/* Remove name from sys.modules, if it's there. */ +static void +_RemoveModule(const char *name) +{ + PyObject *modules = PyImport_GetModuleDict(); + if (PyDict_GetItemString(modules, name) == NULL) + return; + if (PyDict_DelItemString(modules, name) < 0) + Py_FatalError("import: deleting existing key in" + "sys.modules failed"); +} /* Execute a code object in a module and return the module object - WITH INCREMENTED REFERENCE COUNT */ - + * WITH INCREMENTED REFERENCE COUNT. If an error occurs, name is + * removed from sys.modules, to avoid leaving damaged module objects + * in sys.modules. The caller may wish to restore the original + * module object (if any) in this case; PyImport_ReloadModule is an + * example. + */ PyObject * PyImport_ExecCodeModule(char *name, PyObject *co) { @@ -582,7 +597,7 @@ if (PyDict_GetItemString(d, "__builtins__") == NULL) { if (PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()) != 0) - return NULL; + goto error; } /* Remember the filename as the __file__ attribute */ v = NULL; @@ -601,7 +616,7 @@ v = PyEval_EvalCode((PyCodeObject *)co, d, d); if (v == NULL) - return NULL; + goto error; Py_DECREF(v); if ((m = PyDict_GetItemString(modules, name)) == NULL) { @@ -614,6 +629,10 @@ Py_INCREF(m); return m; + + error: + _RemoveModule(name); + return NULL; } @@ -888,7 +907,9 @@ static PyObject * load_package(char *name, char *pathname) { - PyObject *m, *d, *file, *path; + PyObject *m, *d; + PyObject *file = NULL; + PyObject *path = NULL; int err; char buf[MAXPATHLEN+1]; FILE *fp = NULL; @@ -903,19 +924,15 @@ d = PyModule_GetDict(m); file = PyString_FromString(pathname); if (file == NULL) - return NULL; + goto error; path = Py_BuildValue("[O]", file); - if (path == NULL) { - Py_DECREF(file); - return NULL; - } + if (path == NULL) + goto error; err = PyDict_SetItemString(d, "__file__", file); if (err == 0) err = PyDict_SetItemString(d, "__path__", path); - if (err != 0) { - m = NULL; - goto cleanup; - } + if (err != 0) + goto error; buf[0] = '\0'; fdp = find_module(name, "__init__", path, buf, sizeof(buf), &fp, NULL); if (fdp == NULL) { @@ -930,6 +947,10 @@ m = load_module(name, fp, buf, fdp->type, NULL); if (fp != NULL) fclose(fp); + goto cleanup; + + error: + m = NULL; cleanup: Py_XDECREF(path); Py_XDECREF(file); @@ -1272,7 +1293,7 @@ if (Py_VerboseFlag > 1) PySys_WriteStderr("# trying %s\n", buf); filemode = fdp->mode; - if (filemode[0] == 'U') + if (filemode[0] == 'U') filemode = "r" PY_STDIOTEXTMODE; fp = fopen(buf, filemode); if (fp != NULL) { @@ -2234,6 +2255,7 @@ char buf[MAXPATHLEN+1]; struct filedescr *fdp; FILE *fp = NULL; + PyObject *newm; if (m == NULL || !PyModule_Check(m)) { PyErr_SetString(PyExc_TypeError, @@ -2275,10 +2297,18 @@ Py_XDECREF(path); if (fdp == NULL) return NULL; - m = load_module(name, fp, buf, fdp->type, NULL); + newm = load_module(name, fp, buf, fdp->type, NULL); if (fp) fclose(fp); - return m; + if (newm == NULL) { + /* load_module probably removed name from modules because of + * the error. Put back the original module object. We're + * going to return NULL in this case regardless of whether + * replacing name succeeds, so the return value is ignored. + */ + PyDict_SetItemString(modules, name, m); + } + return newm; } @@ -2520,7 +2550,7 @@ { FILE *fp; if (fob == NULL) { - if (mode[0] == 'U') + if (mode[0] == 'U') mode = "r" PY_STDIOTEXTMODE; fp = fopen(pathname, mode); if (fp == NULL) @@ -2692,9 +2722,9 @@ The module name must include the full package name, if any."); PyDoc_STRVAR(doc_lock_held, -"lock_held() -> 0 or 1\n\ -Return 1 if the import lock is currently held.\n\ -On platforms without threads, return 0."); +"lock_held() -> boolean\n\ +Return True if the import lock is currently held, else False.\n\ +On platforms without threads, return False."); PyDoc_STRVAR(doc_acquire_lock, "acquire_lock() -> None\n\ From tim.peters at gmail.com Mon Aug 2 01:37:30 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 01:37:36 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae0408011614190382dc@mail.gmail.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> <1f7befae0407311632442e488b@mail.gmail.com> <1f7befae04080109413fe3a967@mail.gmail.com> <1f7befae04080116116175c3a1@mail.gmail.com> <1f7befae0408011614190382dc@mail.gmail.com> Message-ID: <1f7befae0408011637458d824c@mail.gmail.com> And one more time. - regen'ed against current CVS import.c - removes some now-needless workarounds in .py files -------------- next part -------------- Index: Lib/pty.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/pty.py,v retrieving revision 1.16 diff -u -r1.16 pty.py --- Lib/pty.py 5 Jun 2004 16:27:16 -0000 1.16 +++ Lib/pty.py 1 Aug 2004 23:36:00 -0000 @@ -8,17 +8,6 @@ from select import select import os - -# Absurd: import termios and then delete it. This is to force an attempt -# to import pty to raise an ImportError on platforms that lack termios. -# Without this explicit import of termios here, some other module may -# import tty first, which in turn imports termios and dies with an -# ImportError then. But since tty *does* exist across platforms, that -# leaves a damaged module object for tty in sys.modules, and the import -# of tty here then appears to work despite that the tty imported is junk. -import termios -del termios - import tty __all__ = ["openpty","fork","spawn"] Index: Lib/test/test___all__.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test___all__.py,v retrieving revision 1.39 diff -u -r1.39 test___all__.py --- Lib/test/test___all__.py 1 Jul 2004 11:15:39 -0000 1.39 +++ Lib/test/test___all__.py 1 Aug 2004 23:36:00 -0000 @@ -21,20 +21,6 @@ except ImportError: # Silent fail here seems the best route since some modules # may not be available in all environments. - # Since an ImportError may leave a partial module object in - # sys.modules, get rid of that first. Here's what happens if - # you don't: importing pty fails on Windows because pty tries to - # import FCNTL, which doesn't exist. That raises an ImportError, - # caught here. It also leaves a partial pty module in sys.modules. - # So when test_pty is called later, the import of pty succeeds, - # but shouldn't. As a result, test_pty crashes with an - # AttributeError instead of an ImportError, and regrtest interprets - # the latter as a test failure (ImportError is treated as "test - # skipped" -- which is what test_pty should say on Windows). - try: - del sys.modules[modname] - except KeyError: - pass return verify(hasattr(sys.modules[modname], "__all__"), "%s has no __all__ attribute" % modname) Index: Lib/test/test_pkgimport.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_pkgimport.py,v retrieving revision 1.7 diff -u -r1.7 test_pkgimport.py --- Lib/test/test_pkgimport.py 9 Aug 2002 16:37:36 -0000 1.7 +++ Lib/test/test_pkgimport.py 1 Aug 2004 23:36:00 -0000 @@ -66,12 +66,11 @@ try: __import__(self.module_name) except NameError: pass else: raise RuntimeError, 'Failed to induce NameError.' - module = __import__(self.module_name).foo # ...now change the module so that the NameError doesn't # happen self.rewrite_file('%s = 1' % var) - reload(module) + module = __import__(self.module_name).foo self.assertEqual(getattr(module, var), 1) Index: Python/import.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/import.c,v retrieving revision 2.235 diff -u -r2.235 import.c --- Python/import.c 1 Aug 2004 23:26:05 -0000 2.235 +++ Python/import.c 1 Aug 2004 23:36:01 -0000 @@ -557,10 +557,25 @@ return m; } +/* Remove name from sys.modules, if it's there. */ +static void +_RemoveModule(const char *name) +{ + PyObject *modules = PyImport_GetModuleDict(); + if (PyDict_GetItemString(modules, name) == NULL) + return; + if (PyDict_DelItemString(modules, name) < 0) + Py_FatalError("import: deleting existing key in" + "sys.modules failed"); +} /* Execute a code object in a module and return the module object - WITH INCREMENTED REFERENCE COUNT */ - + * WITH INCREMENTED REFERENCE COUNT. If an error occurs, name is + * removed from sys.modules, to avoid leaving damaged module objects + * in sys.modules. The caller may wish to restore the original + * module object (if any) in this case; PyImport_ReloadModule is an + * example. + */ PyObject * PyImport_ExecCodeModule(char *name, PyObject *co) { @@ -582,7 +597,7 @@ if (PyDict_GetItemString(d, "__builtins__") == NULL) { if (PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()) != 0) - return NULL; + goto error; } /* Remember the filename as the __file__ attribute */ v = NULL; @@ -601,7 +616,7 @@ v = PyEval_EvalCode((PyCodeObject *)co, d, d); if (v == NULL) - return NULL; + goto error; Py_DECREF(v); if ((m = PyDict_GetItemString(modules, name)) == NULL) { @@ -614,6 +629,10 @@ Py_INCREF(m); return m; + + error: + _RemoveModule(name); + return NULL; } @@ -888,7 +907,9 @@ static PyObject * load_package(char *name, char *pathname) { - PyObject *m, *d, *file, *path; + PyObject *m, *d; + PyObject *file = NULL; + PyObject *path = NULL; int err; char buf[MAXPATHLEN+1]; FILE *fp = NULL; @@ -903,19 +924,15 @@ d = PyModule_GetDict(m); file = PyString_FromString(pathname); if (file == NULL) - return NULL; + goto error; path = Py_BuildValue("[O]", file); - if (path == NULL) { - Py_DECREF(file); - return NULL; - } + if (path == NULL) + goto error; err = PyDict_SetItemString(d, "__file__", file); if (err == 0) err = PyDict_SetItemString(d, "__path__", path); - if (err != 0) { - m = NULL; - goto cleanup; - } + if (err != 0) + goto error; buf[0] = '\0'; fdp = find_module(name, "__init__", path, buf, sizeof(buf), &fp, NULL); if (fdp == NULL) { @@ -930,6 +947,10 @@ m = load_module(name, fp, buf, fdp->type, NULL); if (fp != NULL) fclose(fp); + goto cleanup; + + error: + m = NULL; cleanup: Py_XDECREF(path); Py_XDECREF(file); @@ -2234,6 +2255,7 @@ char buf[MAXPATHLEN+1]; struct filedescr *fdp; FILE *fp = NULL; + PyObject *newm; if (m == NULL || !PyModule_Check(m)) { PyErr_SetString(PyExc_TypeError, @@ -2275,10 +2297,18 @@ Py_XDECREF(path); if (fdp == NULL) return NULL; - m = load_module(name, fp, buf, fdp->type, NULL); + newm = load_module(name, fp, buf, fdp->type, NULL); if (fp) fclose(fp); - return m; + if (newm == NULL) { + /* load_module probably removed name from modules because of + * the error. Put back the original module object. We're + * going to return NULL in this case regardless of whether + * replacing name succeeds, so the return value is ignored. + */ + PyDict_SetItemString(modules, name, m); + } + return newm; } From guido at python.org Mon Aug 2 01:52:18 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 2 01:52:24 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: Your message of "Sun, 01 Aug 2004 19:37:30 EDT." <1f7befae0408011637458d824c@mail.gmail.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> <1f7befae0407311632442e488b@mail.gmail.com> <1f7befae04080109413fe3a967@mail.gmail.com> <1f7befae04080116116175c3a1@mail.gmail.com> <1f7befae0408011614190382dc@mail.gmail.com> <1f7befae0408011637458d824c@mail.gmail.com> Message-ID: <200408012352.i71NqIm27124@guido.python.org> > And one more time. > > - regen'ed against current CVS import.c > - removes some now-needless workarounds in .py files Looks good. I say, check it in. Any thoughts on where this affects docs? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Mon Aug 2 03:17:49 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 03:17:53 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408012352.i71NqIm27124@guido.python.org> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com> <1f7befae04072719401df511c3@mail.gmail.com> <200407281841.i6SIfo205648@guido.python.org> <200407281903.i6SJ3ea06175@guido.python.org> <1f7befae0407311632442e488b@mail.gmail.com> <1f7befae04080109413fe3a967@mail.gmail.com> <1f7befae04080116116175c3a1@mail.gmail.com> <1f7befae0408011614190382dc@mail.gmail.com> <1f7befae0408011637458d824c@mail.gmail.com> <200408012352.i71NqIm27124@guido.python.org> Message-ID: <1f7befae04080118171be3f188@mail.gmail.com> [Guido] > Looks good. I say, check it in. I can't argue with that . > Any thoughts on where this affects docs? The only clear places I find are NEWS and C API's PyImport_ExecCodeModule docs. The latter doesn't mention sys.modules now, so I'll add stuff and note 2.4 changes. From greg at cosc.canterbury.ac.nz Mon Aug 2 03:37:42 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 2 03:37:49 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408012013.i71KDI726275@guido.python.org> Message-ID: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> > > Circular imports, to be made consistent, would need > > something like a final "commit" after all imports succeeded. > > Silly idea. Typically, circular imports aren't planned, they just > happen, and (usually) just happen to work. I thought Christian meant this was something that would be done automatically, not something the user would have to do explicitly. Maybe it could be as simple as saving a snapshot of sys.modules whenever importing of a module is begun, and if execution of its body doesn't complete, restoring the snapshot? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From pje at telecommunity.com Mon Aug 2 04:44:18 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 04:41:55 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> References: <200408012013.i71KDI726275@guido.python.org> Message-ID: <5.1.1.6.0.20040801224216.02a65c40@mail.telecommunity.com> At 01:37 PM 8/2/04 +1200, Greg Ewing wrote: > > > Circular imports, to be made consistent, would need > > > something like a final "commit" after all imports succeeded. > > > > Silly idea. Typically, circular imports aren't planned, they just > > happen, and (usually) just happen to work. > >I thought Christian meant this was something that would >be done automatically, not something the user would have >to do explicitly. > >Maybe it could be as simple as saving a snapshot of >sys.modules whenever importing of a module is begun, >and if execution of its body doesn't complete, restoring >the snapshot? That might be over-cautious. It's probably safe to keep modules whose body code executed without error. From pje at telecommunity.com Mon Aug 2 05:52:52 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 05:50:59 2004 Subject: [Python-Dev] Safe to change a thread's interpreter? Message-ID: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> Recently I've been researching implementation strategies for adding Java classloader-like capabilities to Python. I was pleasantly surprised to find out that CPython already supports multiple interpreters via the C API, where each "interpreter" includes fresh versions of 'sys', '__builtin__', etc. The C API doc for PyInterpreter_New(), however, says: """It is possible to insert objects created in one sub-interpreter into a namespace of another sub-interpreter; this should be done with great care to avoid sharing user-defined functions, methods, instances or classes between sub-interpreters, since import operations executed by such objects may affect the wrong (sub-)interpreter's dictionary of loaded modules. (XXX This is a hard-to-fix bug that will be addressed in a future release.)""" It seems to me that the bug described could be fixed (or at least worked around) by having __import__ temporarily change the 'interp' field of the current thread state to point to the interpreter that the __import__ function lives in. Then, at the end of the __import__, reset the 'interp' field back to its original value. (Of course, it would also have to fix up the linked lists of the interpreters' thread states during each swap, but that shouldn't be too difficult.) My question is: does this make sense, or am I completely out in left field here? The only thing I can think of that this would affect is the 'threading' module, in that trying to get the current thread from there (during such an import) might see a foreign interpreter's thread as its own. But, I'm hard-pressed to think of any damage that could possibly cause. Indeed, it seems to me that Python itself doesn't really care how many interpreters or thread states there are running around, and that it only has the linked lists to support "advanced debuggers". Even if it's undesirable to fix the problem this way in the Python core, would it be acceptable to do so in an extension module? What I have in mind is to create an extension module that wraps Py_InterpreterState/Py_ThreadState objects up in a subclassable extension type, designed to ensure the integrity of Python as a whole, while still allowing various import-related methods to be overridden in order to implement Java-style classloader hierarchies. So, you might do something like: from interpreter import Interpreter # Run 'somescript in its own interpreter. it = Interpreter() exit_code = it.run_main("somescript.py") # Release resources without waiting for GC it.close() My thought here also is that performing operations such as running code in a given Interpreter would also operate by swapping the thread state's 'interp' field. Thus, exceptions in the child interpreter would be seamlessly carried through to the parent interpreter. In order to implement the full Java classloader model, it would also be necessary to be able to force imports *not* to use the Interpreter that the code doing the import came from. (i.e. the equivalent of using 'java.lang.Thread.setContextClassLoader()'). This can also probably be implemented via a thread-local variable in the 'interpreter' module. So... must a thread state always reference the same interpreter object? If not, then I think I see a way to safely implement access to multiple interpreters from within Python itself. From martin at v.loewis.de Mon Aug 2 06:19:57 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 2 06:19:58 2004 Subject: [Python-Dev] Safe to change a thread's interpreter? In-Reply-To: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> References: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> Message-ID: <410DC0ED.5070201@v.loewis.de> Phillip J. Eby wrote: > Recently I've been researching implementation strategies for adding Java > classloader-like capabilities to Python. I was pleasantly surprised to > find out that CPython already supports multiple interpreters via the C > API, where each "interpreter" includes fresh versions of 'sys', > '__builtin__', etc. You should be aware that many of us consider the feature of multiple interpreters broken. For example, global variables in extension modules are shared across interpreters, and there is nothing that can be done about this, except for changing the entire C API. Regards, Martin From martin at v.loewis.de Mon Aug 2 06:41:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 2 06:41:44 2004 Subject: [Python-Dev] New developer Message-ID: <410DC608.2050403@v.loewis.de> I just granted Sean Reifschneider CVS write access, so he can directly edit the RPM spec files. Sean has been virtually the only author of that file for a long time. Regards, Martin From greg at cosc.canterbury.ac.nz Mon Aug 2 06:49:38 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 2 06:49:44 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <5.1.1.6.0.20040801224216.02a65c40@mail.telecommunity.com> Message-ID: <200408020449.i724nceH026667@cosc353.cosc.canterbury.ac.nz> > That might be over-cautious. It's probably safe to keep modules whose body > code executed without error. No, it's not. If A and B import each other, and A fails after importing B, then B ends up with a reference to a broken A even though it executed without error. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From pje at telecommunity.com Mon Aug 2 06:54:08 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 06:52:17 2004 Subject: [Python-Dev] Safe to change a thread's interpreter? In-Reply-To: <410DC0ED.5070201@v.loewis.de> References: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com> At 06:19 AM 8/2/04 +0200, Martin v. L?wis wrote: >Phillip J. Eby wrote: >>Recently I've been researching implementation strategies for adding Java >>classloader-like capabilities to Python. I was pleasantly surprised to >>find out that CPython already supports multiple interpreters via the C >>API, where each "interpreter" includes fresh versions of 'sys', >>'__builtin__', etc. > >You should be aware that many of us consider the feature of multiple >interpreters broken. For example, global variables in extension modules >are shared across interpreters, and there is nothing that can be done >about this, except for changing the entire C API. Yes, I saw that as a documented limitation. Are there undocumented limitations as well? Is the feature headed for deprecation? I guess I'm not understanding your implication(s), if any. From pje at telecommunity.com Mon Aug 2 06:57:31 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 06:55:40 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408020449.i724nceH026667@cosc353.cosc.canterbury.ac.nz> References: <5.1.1.6.0.20040801224216.02a65c40@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040802005457.027ef150@mail.telecommunity.com> At 04:49 PM 8/2/04 +1200, Greg Ewing wrote: > > That might be over-cautious. It's probably safe to keep modules whose > body > > code executed without error. > >No, it's not. If A and B import each other, and A fails after >importing B, then B ends up with a reference to a broken A >even though it executed without error. Argh. I can't believe I missed that. Now that you mention it, I think that point was already brought up just a few days ago here. Maybe even by you. :) Ah well, please pardon my sudden case of obliviousness. :) From anthony at interlink.com.au Mon Aug 2 08:32:03 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 2 08:32:16 2004 Subject: [Python-Dev] 2.4a2, and @decorators Message-ID: <410DDFE3.7010006@interlink.com.au> The @decorator patch has been landed on the trunk, after getting the go-ahead from the BDFL. I'll update PEP-0318 with the final syntax in the next day or so. We're aiming for 2.4a2 this Thursday - so can people please hold off on checking into the trunk from Wednesday evening onwards, please. At the moment, I'm guessing we're going to have a 2.4a3 in a couple of weeks. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Mon Aug 2 08:35:16 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 2 08:36:13 2004 Subject: [Python-Dev] Optik 1.5 in Python 2.4 In-Reply-To: <20040730223936.GA26696@cthulhu.gerg.ca> References: <20040730223936.GA26696@cthulhu.gerg.ca> Message-ID: <410DE0A4.70408@interlink.com.au> Greg Ward wrote: > So: I'd like to checkin Optik 1.5a1 as optparse.py right now, and worry > about the docs later. Hopefully before Python 2.4a2 goes out the door, > but no promises. Objections? If I don't hear any soon, I'll do the > deed tomorrow morning. +1 From martin at v.loewis.de Mon Aug 2 09:34:20 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 2 09:34:20 2004 Subject: [Python-Dev] Safe to change a thread's interpreter? In-Reply-To: <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com> References: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com> Message-ID: <410DEE7C.1060907@v.loewis.de> Phillip J. Eby wrote: > Yes, I saw that as a documented limitation. Are there undocumented > limitations as well? Might be. However, there is one more documented the limitation: The PEP 311 extensions only work for a single interpreter state, as the PEP explains. > Is the feature headed for deprecation? I guess > I'm not understanding your implication(s), if any. That is not surprising, as the implication is not at all obvious :-) Here it is: Because the feature is essentially ill-designed, I'm not going to comment on the actual question (or on any other actual question about that feature), except for pointing out that the feature is fundamentally flawed. This is just my own policy, though. Regards, Martin From tismer at stackless.com Mon Aug 2 11:01:26 2004 From: tismer at stackless.com (Christian Tismer) Date: Mon Aug 2 11:00:33 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> Message-ID: <410E02E6.9000205@stackless.com> Greg Ewing wrote: >>>Circular imports, to be made consistent, would need >>>something like a final "commit" after all imports succeeded. >> >>Silly idea. Typically, circular imports aren't planned, they just >>happen, and (usually) just happen to work. > > > I thought Christian meant this was something that would > be done automatically, not something the user would have > to do explicitly. > > Maybe it could be as simple as saving a snapshot of > sys.modules whenever importing of a module is begun, > and if execution of its body doesn't complete, restoring > the snapshot? This is exactly what I thought of. It seems to be correct, also with nested imports, since this is a stack-like structure of undoable things. Example __main__: import A #1 #6 A: try: import B #2 #5 except (ImportError, maybe others): # provide surrogate for B # other action that fails B: import A #3 import C #4 # do some stuff that fails #1: snapshot of sys.modules, A added [__main__, A] #2: snapshot of sys.modules, B added [__main__, A, B] #3: snapshot of sys.modules, A exists [__main__, A, B] #3: snapshot of sys.modules, C added [__main__, A, B, C] after failing imports: #5: restore [__main__, A] #5: restore [__main__] So one side effect is that completely valid imports in the context of a failing import are undone. I guess this is intended. A problem are side effects, of course. It would be nice to have a cleanup construct for completely or partially imported modules, to undo any side effects. How do we remove extension modules which have been loaded as a side effect? ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From walter at livinglogic.de Mon Aug 2 11:24:11 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Mon Aug 2 11:24:34 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410DDFE3.7010006@interlink.com.au> References: <410DDFE3.7010006@interlink.com.au> Message-ID: <410E083B.20109@livinglogic.de> Anthony Baxter wrote: > The @decorator patch has been landed on the trunk, after > getting the go-ahead from the BDFL. I'll update PEP-0318 > with the final syntax in the next day or so. > > We're aiming for 2.4a2 this Thursday - so can people please > hold off on checking into the trunk from Wednesday evening > onwards, please. > > At the moment, I'm guessing we're going to have a 2.4a3 in > a couple of weeks. I'm getting the following assertion from a debug build: Python 2.4a1+ (#1, Aug 2 2004, 11:21:35) [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-113)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> @staticmethod ... def x(): ... pass ... python: Python/ceval.c:868: PyEval_EvalFrame: Assertion `(stack_pointer - f->f_valuestack) <= f->f_stacksize' failed. Aborted Bye, Walter D?rwald From anthony at interlink.com.au Mon Aug 2 13:09:11 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 2 13:09:36 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <1091442793.2017.10.camel@localhost> References: <410DDFE3.7010006@interlink.com.au> <410E083B.20109@livinglogic.de> <1091442793.2017.10.camel@localhost> Message-ID: <410E20D7.2040900@interlink.com.au> Mark Russell wrote: > Sorry - my mistake for not running the test suite under the debug > build. I've attached a fix. Anthony, can you check this in as well > please? I've added "run the test suite under a debug build" to my list of build tasks for the release process thingy. Patch applied. Anthony From jjl at pobox.com Mon Aug 2 15:30:39 2004 From: jjl at pobox.com (John J Lee) Date: Mon Aug 2 15:27:38 2004 Subject: httplib patch for 2.4 (was: Re: [Python-Dev] 2.4a2 ...) In-Reply-To: <410DDFE3.7010006@interlink.com.au> References: <410DDFE3.7010006@interlink.com.au> Message-ID: On Mon, 2 Aug 2004, Anthony Baxter wrote: [...] > We're aiming for 2.4a2 this Thursday - so can people please > hold off on checking into the trunk from Wednesday evening > onwards, please. [...] Would somebody mind checking this in: http://www.python.org/sf/997626 It fixes a bug in httplib which badly breaks urllib2 now that urllib2 is using httplib.HTTPConnection instead of httplib.HTTP (POSTs don't work). It includes patch and a new test, and does not require doc updates, but nobody has reviewed it yet. John From jim at zope.com Mon Aug 2 16:01:28 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 16:05:02 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> References: <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <41065006.6020607@zope.com> <200407271445.i6REjEK24454@guido.python.org> <41067660.2070109@zope.com> <1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com> <20040727164913.GS7708@epoch.metaslash.com> <41068C2F.40102@zope.com> <200407272050.i6RKoEY25164@guido.python.org> <4107870C.1070404@zope.com> <5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com> <5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com> <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> Message-ID: <410E4938.80408@zope.com> Phillip J. Eby wrote: > At 08:28 AM 7/30/04 -0400, Jim Fulton wrote: > >> The problem with Python's current package system is that it's not >> possible, >> in general, to write packages that can be moved around within the package >> system, because relative imports aren't reobust or flexible enough. >> PEP 328 would fix this. PEP 328 would allow creation of packages that >> worked regardless of where there were places or that worked relative >> to a containing package regardless of where that was placed. You could >> then have different module spaces expressed by adding a new layer of >> top-level >> modules. You wouldn't need any new machinery beyond PEP 328. > > > Hm. The concept I'm proposing would allow code to be used in a module > space without needing to know about module spaces or be written to > co-operate in such a fashion. If you can make everybody conform to some > sort of coding standard, you wouldn't even need PEP 328 to accomplish > your goal. ;) I don't think that's true. If your package has sub-packages, there's no way for a subackage to import from the containing package without using an absolute import. Similarly, a package currentlly can't import from a sibling package without using an absolute import. With PEP 328, relative imports from parent or sibling (or cousin ...) packages will be possible. > > As a counterexample, consider seven Zope products that depend on two > different versions of PIL. Using PEP 328, you can't make this work > without either loading seven copies of PIL, or else requiring the > version number to be part of the package name. No, I don't think this is right. PEP 328 should allow you to create module spaces using container packages. For your example, we create two top-level packages, space1 and space 2. You put version x of PIL in space1. That package's absolute name is space1.PIL. You put version y of PIL in space2, creating space2.PIL. Now you put the products that depend on version x of PIL in space1. You put the products that depend on version y of PIL on space 2. The products mist use relative imports to import from PIL: from ..PIL import somePILmodule For this to work, PIL also has to use relative imports to import it's own modules. > You actually have it > somewhat better right now because you can at least fool the "this > package or absolute" mechanism into doing what you want. I assume you are fering to letting local modules hide global ones. This only works if the modules are included directly in the package. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From goodger at python.org Mon Aug 2 16:13:36 2004 From: goodger at python.org (David Goodger) Date: Mon Aug 2 16:13:41 2004 Subject: [Python-Dev] ConfigParser behavior change Message-ID: <410E4C10.3080009@python.org> Could I get some feedback re ? I'd like to resolve this before 2.4a2 if possible. In summary, RawConfigParser.set() doesn't allow non-string arguments for 'value' any more. This breaks Docutils code. I specifically chose RawConfigParser for Docutils internal settings support because it *did* allow non-string values. An earlier bug report (, "Support for non-string data in ConfigParser unclear/buggy") offered 4 options: (1) Cast the option to a string explicitly instead of assuming it is a string (2) Check that the option is a string (3) State in the documentation that the set() method only takes string values (4) State in the documentation that the raw parameter should be used when an option's value might not be of type string Options 2 & 3 were implemented in the "bug fix" (just a type check). I think that the behavior should not have been changed, rather that the documentation needed updating (i.e., choose option 4 instead). I volunteer to undo the change and update the documentation ASAP. The comment for bug 810843 says "This is documented behavior", but I couldn't find any such documentation pre-dating this change. Thanks! -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/426c69c5/signature.pgp From jim at zope.com Mon Aug 2 16:30:57 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 16:31:00 2004 Subject: [Python-Dev] Re: Can we limit the effects of module execution to sys.modules? (was Fix import errors to have data) In-Reply-To: <1f7befae04073021175bc48767@mail.gmail.com> References: <41065006.6020607@zope.com> <200407271445.i6REjEK24454@guido.python.org> <41067660.2070109@zope.com> <1f7befae040727085715ce2536@mail.gmail.com> <4107866C.7060609@zope.com> <1f7befae04073021175bc48767@mail.gmail.com> Message-ID: <410E5021.2020903@zope.com> Tim Peters wrote: > [Tim, wants to keep insane modules out of sys.modules] > > [Jim Fulton] > ... >>Do you think it's practical to limit the effects of module import to >>sys.modules, even by convention? > > > I'm sure you didn't intend that to be *so* extreme -- like surely a > module is allowed to initialize its own module-level variables. If I > read "effects" as "effects visible outside the module", then that's > what you said . Yup. :) >>If it is possible to limit the effects of import (even by convention), >>then I think it would be practical to roll-back changes to sys.modules. >>If it's not practical to limit the effects of module import then I think >>the problem is effectively unsolveable, short of making Python transactional. > > > There we don't agree -- I think it's already practical, based on that > virtually no Python application *intends* to catch errors from imports > other than ImportError, so that almost all "real bugs" in module > initialization are intended to stop execution. In turn, in the cases > where ImportErrors are intentionally caught now, they generally occur > in "import blocks" near the starts of all modules in the failing > import chain, and so none of the modules involved have yet *done* any > non-trivial initialization -- they're all still trying to import the > stuff they need to *start* doing the meat of their initialization. Except in cases where imports are places later in a module to make circular imports work. It would be nice to disallow circular imports, although I don't know if that's possible. (Some time, when I have time, I'd like to see if we can get rid of them in Zope 3. I'd like to have a tool to report circular imports, to keep them from creeping in, which they do so easily.) Having said that, you make a good point. If all modules did their imports before making any changes outside the modules, then it would be pretty safe to clean up the imports. > If > some modules happen to import successfully along the way, fine, they > should stay in sys.modules, and then importing them again later won't > run their initialization code again. This only works if all of the modules do all of their imports before doing other work. If there are circular imports, you could have: A defines class C A imports B B imports C from A A imports D and gets an import error If we clean up A and D, then B has a class from a module that doesn't exist. Hm. Suppose we have: A imports B B imports A A imports D and gets an import error We clean up A and D. What about the A that was imported into B? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From fdrake at acm.org Mon Aug 2 16:33:03 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon Aug 2 16:33:16 2004 Subject: [Python-Dev] ConfigParser behavior change In-Reply-To: <410E4C10.3080009@python.org> References: <410E4C10.3080009@python.org> Message-ID: <200408021033.03830.fdrake@acm.org> On Monday 02 August 2004 10:13 am, David Goodger wrote: > Could I get some feedback re ? > I'd like to resolve this before 2.4a2 if possible. David has (properly) been asking me to look into this, and i've managed not to have enough time. Sorry, David! > RawConfigParser.set() doesn't allow non-string arguments for 'value' > any more. This breaks Docutils code. I specifically chose > RawConfigParser for Docutils internal settings support because it > *did* allow non-string values. > > An earlier bug report (, "Support for > non-string data in ConfigParser unclear/buggy") offered 4 options: The ConfigParser documentation was certainly too vague, but the problem, as I see it, is that the module was never intended to store non-string values. I think allowing them is just asking for still more trouble from that module down the road. Sure, the module could be made happy by reverting the patch you cite, but happy-by-accident is a very fragile state to be in. > The comment for bug 810843 says "This is documented behavior", but I > couldn't find any such documentation pre-dating this change. This may have been a bug in my memory. -Fred -- Fred L. Drake, Jr. From pje at telecommunity.com Mon Aug 2 16:40:18 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 16:36:18 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410DDFE3.7010006@interlink.com.au> Message-ID: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> At 04:32 PM 8/2/04 +1000, Anthony Baxter wrote: >The @decorator patch has been landed on the trunk, after >getting the go-ahead from the BDFL. I'll update PEP-0318 >with the final syntax in the next day or so. @decorator won? When did that happen? From mwh at python.net Mon Aug 2 16:39:36 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 2 16:39:39 2004 Subject: [Python-Dev] refleak hunting season Message-ID: <2mpt69bo47.fsf@starship.python.net> Last time round I found a bunch of reference counting bugs in Python just *after* a major release; I'm trying to do it all before 2.4.0 this time. I've hacked regrtest again (patch attached). Here's my notes file, though as is the case with such things writing stuff down tends to cause thoughts which cause experiments which cause the notes to be out of date more or less immediately... test_codeccallbacks leaked [2, 2, 2, 2] references registries test_copy leaked [82, 82, 82, 82] references copy_reg.dispatch_table -- now taken care of in regrtest.py test_descr leaked [2, 2, 2, 2] references resurrection, still test_descrtut leaked [1, 1, 1, 1] references exec 'print x' in defaultdict() fixed, need to check in (bit subtle though) test_distutils leaked [2, 2, 2, 2] references no idea test_gc leaked [18, 18, 18, 18] references trashcan test_hotshot leaked [111, 111, 111, 111] references there's been a patch on SF for this for about a year! test_minidom leaked [1, 1, 1, 1] references testPickledDocument, don't know more test_mutants leaked [74, -1123, 1092, -521] references randomized test test_pkg leaked [3, 3, 3, 3] references something to do with execfile? or tempfile.mkstemp? the leak is on the dummy key from dictobject.c!! very odd. see bug #808596 for the explanation. test_pkgimport leaked [6, 6, 6, 6] references no idea similar to the above? (but certainly not the same) I think this might be caused by interning like the above. test_sax leaked [1899, 1899, 1899, 1899] references no idea -- scary numbers, though! test_threadedtempfile leaked [0, 1, -21, 9] references randomized test? test_threading_local leaked [9, 0, 7, 0] references not sure what's going on here; not sure it's a leak, but it's definitely odd test_unicode leaked [7, 7, 7, 7] references the codec registry test_urllib2 leaked [120, -80, -70, 120] references CacheFTPHandler test_xrange leaked [1, 1, 1, 1] references fixed in CVS I'd really appreciate someone taking a look at test_sax. The numbers for test_gc and test_descr are artifacts and if someone sees how to fix them, I'd be grateful. test_threading_local behaves very strangely. test_distutils I haven't even begun to look at. Generally, we're in pretty good shape! (Or our tests really suck, your call ;-). Cheers, mwh -------------- next part -------------- Index: regrtest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/regrtest.py,v retrieving revision 1.156 diff -c -r1.156 regrtest.py *** regrtest.py 26 Jul 2004 12:09:13 -0000 1.156 --- regrtest.py 2 Aug 2004 14:31:41 -0000 *************** *** 8,26 **** Command line options: ! -v: verbose -- run tests in verbose mode with output to stdout ! -q: quiet -- don't print anything except if a test fails ! -g: generate -- write the output file for a test instead of comparing it ! -x: exclude -- arguments are tests to *exclude* ! -s: single -- run only a single test (see below) ! -r: random -- randomize test execution order ! -f: fromfile -- read names of tests to run from a file (see below) ! -l: findleaks -- if GC is available detect tests that leak memory ! -u: use -- specify which special resource intensive tests to run ! -h: help -- print this text and exit ! -t: threshold -- call gc.set_threshold(N) ! -T: coverage -- turn on code coverage using the trace module ! -L: runleaks -- run the leaks(1) command just before exit If non-option arguments are present, they are names for tests to run, unless -x is given, in which case they are names for tests not to run. --- 8,27 ---- Command line options: ! -v: verbose -- run tests in verbose mode with output to stdout ! -q: quiet -- don't print anything except if a test fails ! -g: generate -- write the output file for a test instead of comparing it ! -x: exclude -- arguments are tests to *exclude* ! -s: single -- run only a single test (see below) ! -r: random -- randomize test execution order ! -f: fromfile -- read names of tests to run from a file (see below) ! -l: findleaks -- if GC is available detect tests that leak memory ! -u: use -- specify which special resource intensive tests to run ! -h: help -- print this text and exit ! -t: threshold -- call gc.set_threshold(N) ! -T: coverage -- turn on code coverage using the trace module ! -L: runleaks -- run the leaks(1) command just before exit ! -R: huntrleaks -- search for reference leaks (needs debug build, v. slow) If non-option arguments are present, they are names for tests to run, unless -x is given, in which case they are names for tests not to run. *************** *** 84,89 **** --- 85,91 ---- import getopt import random import warnings + import sre import cStringIO import traceback *************** *** 127,133 **** def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False, exclude=False, single=False, randomize=False, fromfile=None, ! findleaks=False, use_resources=None, trace=False, runleaks=False): """Execute a test suite. This also parses command-line options and modifies its behavior --- 129,136 ---- def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False, exclude=False, single=False, randomize=False, fromfile=None, ! findleaks=False, use_resources=None, trace=False, runleaks=False, ! huntrleaks=False): """Execute a test suite. This also parses command-line options and modifies its behavior *************** *** 152,162 **** test_support.record_original_stdout(sys.stdout) try: ! opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TL', ['help', 'verbose', 'quiet', 'generate', 'exclude', 'single', 'random', 'fromfile', 'findleaks', 'use=', 'threshold=', 'trace', ! 'runleaks' ]) except getopt.error, msg: usage(2, msg) --- 155,165 ---- test_support.record_original_stdout(sys.stdout) try: ! opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TLR', ['help', 'verbose', 'quiet', 'generate', 'exclude', 'single', 'random', 'fromfile', 'findleaks', 'use=', 'threshold=', 'trace', ! 'runleaks', 'huntrleaks' ]) except getopt.error, msg: usage(2, msg) *************** *** 191,196 **** --- 194,201 ---- gc.set_threshold(int(a)) elif o in ('-T', '--coverage'): trace = True + elif o in ('-R', '--huntrleaks'): + huntrleaks = True elif o in ('-u', '--use'): u = [x.lower() for x in a.split(',')] for r in u: *************** *** 288,294 **** tracer.runctx('runtest(test, generate, verbose, quiet, testdir)', globals=globals(), locals=vars()) else: ! ok = runtest(test, generate, verbose, quiet, testdir) if ok > 0: good.append(test) elif ok == 0: --- 293,299 ---- tracer.runctx('runtest(test, generate, verbose, quiet, testdir)', globals=globals(), locals=vars()) else: ! ok = runtest(test, generate, verbose, quiet, testdir, huntrleaks) if ok > 0: good.append(test) elif ok == 0: *************** *** 397,403 **** tests.sort() return stdtests + tests ! def runtest(test, generate, verbose, quiet, testdir=None): """Run a single test. test -- the name of the test generate -- if true, generate output, instead of running the test --- 402,408 ---- tests.sort() return stdtests + tests ! def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False): """Run a single test. test -- the name of the test generate -- if true, generate output, instead of running the test *************** *** 415,420 **** --- 420,427 ---- cfp = None else: cfp = cStringIO.StringIO() + if huntrleaks: + refrep = open("reflog.txt", "a") try: save_stdout = sys.stdout try: *************** *** 435,440 **** --- 442,478 ---- indirect_test = getattr(the_module, "test_main", None) if indirect_test is not None: indirect_test() + if huntrleaks: + import copy_reg + fs = warnings.filters[:] + ps = copy_reg.dispatch_table.copy() + import gc + def cleanup(): + import _strptime, urlparse, warnings + warnings.filters[:] = fs + gc.collect() + sre.purge() + _strptime._regex_cache.clear() + urlparse.clear_cache() + copy_reg.dispatch_table.clear() + copy_reg.dispatch_table.update(ps) + deltas = [] + if indirect_test: + for i in range(9): + rc = sys.gettotalrefcount() + indirect_test() + cleanup() + deltas.append(sys.gettotalrefcount() - rc - 2) + else: + for i in range(9): + rc = sys.gettotalrefcount() + reload(the_module) + cleanup() + deltas.append(sys.gettotalrefcount() - rc - 2) + if max(map(abs, deltas[-4:])) > 0: + print >>refrep, test, 'leaked', \ + deltas[-4:], 'references' + #refrep.flush() finally: sys.stdout = save_stdout except test_support.ResourceDenied, msg: *************** *** 486,492 **** fp.close() else: expected = test + "\n" ! if output == expected: return 1 print "test", test, "produced unexpected output:" sys.stdout.flush() --- 524,530 ---- fp.close() else: expected = test + "\n" ! if output == expected or huntrleaks: return 1 print "test", test, "produced unexpected output:" sys.stdout.flush() -------------- next part -------------- -- I never disputed the Perl hacking skill of the Slashdot creators. My objections are to the editors' taste, the site's ugly visual design, and the Slashdot community's raging stupidity. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq From pje at telecommunity.com Mon Aug 2 16:59:19 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 16:55:18 2004 Subject: [Python-Dev] Safe to change a thread's interpreter? In-Reply-To: <410DEE7C.1060907@v.loewis.de> References: <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com> <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com> <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040802104550.035e4110@mail.telecommunity.com> At 09:34 AM 8/2/04 +0200, Martin v. L?wis wrote: >Phillip J. Eby wrote: >>Yes, I saw that as a documented limitation. Are there undocumented >>limitations as well? > >Might be. However, there is one more documented the limitation: The PEP >311 extensions only work for a single interpreter state, as the PEP >explains. Maybe I'm misinterpreting it, but the "Design and Implementation" section sounds like the only issue would be that any automatically-allocated thread state would point to the primary interpreter. For my intended use, that's not actually a problem, since import operations would have to switch the interpreter to the "correct" one, anyway. From barry at python.org Mon Aug 2 17:01:32 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 2 17:01:29 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> References: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> Message-ID: <1091458891.9134.29.camel@localhost> On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote: > At 04:32 PM 8/2/04 +1000, Anthony Baxter wrote: > >The @decorator patch has been landed on the trunk, after > >getting the go-ahead from the BDFL. I'll update PEP-0318 > >with the final syntax in the next day or so. > > @decorator won? When did that happen? First I heard about it was when I saw Anthony's checkin messages. Maybe it was throwing that pie that inspired Guido -- an '@' does kind of look like a pie. I think Anthony's checkin message was accurate enough though -- it's the syntax everyone can hate equally. But I'm glad /something/ made it in! print>>-set-the-precedent-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/1dad5c36/attachment.pgp From skip at pobox.com Mon Aug 2 17:04:20 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 2 17:04:27 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <2mpt69bo47.fsf@starship.python.net> References: <2mpt69bo47.fsf@starship.python.net> Message-ID: <16654.22516.768093.314537@montanaro.dyndns.org> Michael> I've hacked regrtest again (patch attached). Why not just check your regrtest changes into cvs? Skip From mwh at python.net Mon Aug 2 17:11:36 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 2 17:11:38 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <16654.22516.768093.314537@montanaro.dyndns.org> (Skip Montanaro's message of "Mon, 2 Aug 2004 10:04:20 -0500") References: <2mpt69bo47.fsf@starship.python.net> <16654.22516.768093.314537@montanaro.dyndns.org> Message-ID: <2mllgxbmmv.fsf@starship.python.net> Skip Montanaro writes: > Michael> I've hacked regrtest again (patch attached). > > Why not just check your regrtest changes into cvs? Because I'm not quite happy with them; I hope to make the changes I want after the run I'm doing with -uall finishes (so, erm, Thursday or so :-). Also, there are entirely too many tests that fail when run with -R: test_builtin test_cgi test_decimal test_dircache test_doctest test_getopt test_logging test_multifile test_socket test_warnings test_zipimport (I meant to mention this in the first mail). I'll probably be filing bug reports on or trying to fix some of these -- tests that muck with the environment have a tendency to be a pain under any circumstances. Oh, and test_threaded_import still deadlocks. That's really annoying! Cheers, mwh -- MARVIN: What a depressingly stupid machine. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From anthony at interlink.com.au Mon Aug 2 17:14:50 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 2 17:15:09 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <1091458891.9134.29.camel@localhost> References: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> Message-ID: <410E5A6A.4020408@interlink.com.au> Barry Warsaw wrote: > On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote: >>@decorator won? When did that happen? > > > First I heard about it was when I saw Anthony's checkin messages. Maybe > it was throwing that pie that inspired Guido -- an '@' does kind of look > like a pie. I think Anthony's checkin message was accurate enough > though -- it's the syntax everyone can hate equally. But I'm glad > /something/ made it in! > > print>>-set-the-precedent-ly y'rs, > -Barry Hey, I was just going off Guido's decision (in email - looking back, it wasn't cc'd to python-dev, which probably explains the lack of 400 followups ) Channelling the BDFL, I think the idea was to put it in 2.4a2, and see how it works for people. If it turns out that it's really really really hated, we can try something else in a3. Guido can of course follow up to this (hint hint) and give his own answers. I've been playing with it for most of the last week, on and off, and I can say that I hate it less now. The @ makes it very obvious visually what's going on, which the other syntaxes didn't. But I like Barry's idea - I think it should be referred to at the 'pie decorator syntax'. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From walter at livinglogic.de Mon Aug 2 17:27:37 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Mon Aug 2 17:27:40 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <2mllgxbmmv.fsf@starship.python.net> References: <2mpt69bo47.fsf@starship.python.net> <16654.22516.768093.314537@montanaro.dyndns.org> <2mllgxbmmv.fsf@starship.python.net> Message-ID: <410E5D69.3060009@livinglogic.de> Michael Hudson wrote: > Skip Montanaro writes: > >> Michael> I've hacked regrtest again (patch attached). >> >>Why not just check your regrtest changes into cvs? > > Because I'm not quite happy with them; I hope to make the changes I > want after the run I'm doing with -uall finishes (so, erm, Thursday or > so :-). I would prefer it, if the refcounting was done for each test method not only for the complete test. This would help in finding the leak but unfortunately would required changes to unittest. Bye, Walter D?rwald From barry at python.org Mon Aug 2 17:33:01 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 2 17:32:58 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410E5A6A.4020408@interlink.com.au> References: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <410E5A6A.4020408@interlink.com.au> Message-ID: <1091460781.9150.37.camel@localhost> On Mon, 2004-08-02 at 11:14, Anthony Baxter wrote: > But I like Barry's idea - I think it should be referred to at the > 'pie decorator syntax'. :) I'm going to convert MM3 to use pie decorator syntax to play with this (in my CST). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/c3338b6a/attachment.pgp From mwh at python.net Mon Aug 2 17:39:27 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 2 17:39:28 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <410E5D69.3060009@livinglogic.de> ( =?iso-8859-1?q?Walter_D=F6rwald's_message_of?= "Mon, 02 Aug 2004 17:27:37 +0200") References: <2mpt69bo47.fsf@starship.python.net> <16654.22516.768093.314537@montanaro.dyndns.org> <2mllgxbmmv.fsf@starship.python.net> <410E5D69.3060009@livinglogic.de> Message-ID: <2mhdrlblcg.fsf@starship.python.net> Walter D?rwald writes: > Michael Hudson wrote: > >> Skip Montanaro writes: >> >>> Michael> I've hacked regrtest again (patch attached). >>> >>>Why not just check your regrtest changes into cvs? >> Because I'm not quite happy with them; I hope to make the changes I >> want after the run I'm doing with -uall finishes (so, erm, Thursday or >> so :-). > > I would prefer it, if the refcounting was done for each test method > not only for the complete test. This would help in finding the leak > but unfortunately would required changes to unittest. Certainly, this is a superior solution. However, it also has the problem that from regrtest you can't tell whether a given test is using unittest, doctest or whatever. It's probably possible to redesign things so this is all elegant and natural, but I actually wanted to get somewhere with this :-) Cheers, mwh -- I have a feeling that any simple problem can be made arbitrarily difficult by imposing a suitably heavy administrative process around the development. -- Joe Armstrong, comp.lang.functional From jim at zope.com Mon Aug 2 18:32:56 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 18:33:01 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists Message-ID: <410E6CB8.3070001@zope.com> I propose to add the following function to the imp module: exists(module_name) Test whether the named module can be found. This method should be used when you want to perform alternate actions depending on whether a module is available. Returns True of the module can be found and False otherwise. A True return value does not guarentee that the module will be importable. The module will not be imported as a result of calling exists, but parent modules may be. If a dotted module name is given, then parent modules will be imported if they haven't already been imported. If there are no objections, I'll try to get this in before Wednesday night. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From tim.peters at gmail.com Mon Aug 2 18:42:30 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 18:42:36 2004 Subject: [Python-Dev] Re: Can we limit the effects of module execution to sys.modules? (was Fix import errors to have data) In-Reply-To: <410E5021.2020903@zope.com> References: <41065006.6020607@zope.com> <200407271445.i6REjEK24454@guido.python.org> <41067660.2070109@zope.com> <1f7befae040727085715ce2536@mail.gmail.com> <4107866C.7060609@zope.com> <1f7befae04073021175bc48767@mail.gmail.com> <410E5021.2020903@zope.com> Message-ID: <1f7befae04080209424ee937a8@mail.gmail.com> [Tim Peters] >> ...I think it's already practical, based on that >> virtually no Python application *intends* to catch errors from imports >> other than ImportError, so that almost all "real bugs" in module >> initialization are intended to stop execution. In turn, in the cases >> where ImportErrors are intentionally caught now, they generally occur >> in "import blocks" near the starts of all modules in the failing >> import chain, and so none of the modules involved have yet *done* any >> non-trivial initialization -- they're all still trying to import the >> stuff they need to *start* doing the meat of their initialization. [Jim Fulton] > Except in cases where imports are places later in a module to make > circular imports work. That's being discussed in "the other" thread (the now-massive "Fix import errors to have data" thread). I never do circular imports intentionally, so my interest in that is unmeasurable -- on the low end . Note that I checked in changes last night so that a failing import in 2.4 *will* delete the failing importee from sys.modules. This breaks a couple Zope3 tests, BTW (explained in the other thread). > It would be nice to disallow circular imports, although I don't know if that's > possible. I doubt it, because it would break existing code "gratuitously". Maybe in Python 3. IMO the behavior of circular imports in CPython today is about half implementation accident. > (Some time, when I have time, I'd like to see if we can get rid of them in Zope 3. +1 > I'd like to have a tool to report circular imports, to keep them from creeping in, > which they do so easily.) Wasn't Fred working on such a tool? > Having said that, you make a good point. If all modules did their imports > before making any changes outside the modules, then it would be pretty safe > to clean up the imports. I think most code already does. >> If some modules happen to import successfully along the way, fine, they >> should stay in sys.modules, and then importing them again later won't >> run their initialization code again. > This only works if all of the modules do all of their imports before doing other > work. It works in many more cases than just that. do a ton of work import A do more work blow up A is almost certainly still fine in that case. I understand we can concoct pathological cases in which it isn't. > If there are circular imports, you could have: > > A defines class C > A imports B > B imports C from A > A imports D and gets an import error > > If we clean up A and D, then B has a class from a module that doesn't exist. Yes, and that's unpleasant. If C defines any methods, A.__dict__ will still be alive (because methods of C refer to it via their func_globals), but when the module object itself goes away its __dict__ values get set to None. So any method of C that refers to a module global is likely to suffer a mysterious None-related exception. It's not going to segfault, though. OTOH, to get into trouble, someone has to suppress the ImportError raised by trying to import A. You have to work pretty hard to get in trouble. > Hm. Suppose we have: > > A imports B > B imports A > A imports D and gets an import error > > We clean up A and D. What about the A that was imported into B? I believe that works the same as before the patch, except that 'A' and 'D' are no longer in sys.modules. The only "clean up" done by the patch is, in case of error, to remove the module name from sys.modules. That removes a reference to the module object; it doesn't do anything more than that directly. The module object in this case remains alive, and its __dict__ remains intact, because B holds a reference to the module object. If someone suppresses the ImportError from importing A, then, then the primary difference is that *another* attempt to import A will again raise ImportError now. Over in the other thread, people are talking about schemes to do better in circular import cases that suffer errors. My patch ignored those issues (beyond convincing myself segfaults wouldn't result), because Guido's patch I built on ignored them . That was enough to allow removing some atrocious code in the Python libraries trying to worm around damaged modules left behind in sys.modules, and appears so far to be enough to allow "regrtest.py -r" (randomize test order) to pass on Windows reliably for the first time in Python's history. So I think it's a real improvement, despite that it doesn't solve all import-error problems. From pje at telecommunity.com Mon Aug 2 18:55:07 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 18:51:07 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410E5A6A.4020408@interlink.com.au> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> Message-ID: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> At 01:14 AM 8/3/04 +1000, Anthony Baxter wrote: >Barry Warsaw wrote: >>On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote: >>>@decorator won? When did that happen? >> >>First I heard about it was when I saw Anthony's checkin messages. Maybe >>it was throwing that pie that inspired Guido -- an '@' does kind of look >>like a pie. I think Anthony's checkin message was accurate enough >>though -- it's the syntax everyone can hate equally. But I'm glad >>/something/ made it in! >>print>>-set-the-precedent-ly y'rs, >>-Barry > >Hey, I was just going off Guido's decision (in email - looking back, it >wasn't cc'd to python-dev, which probably explains the lack of 400 >followups ) > >Channelling the BDFL, I think the idea was to put it in 2.4a2, and see >how it works for people. If it turns out that it's really really really >hated, we can try something else in a3. Guido can of course follow up >to this (hint hint) and give his own answers. I would think the fact that the '[decorators]' syntax can be implemented in pure Python (no changes to the interpreter) for existing Python versions would give more weight to it. That is, if someone wants to implement a decorator that's forwards and backwards-compatible, that's possible with the list syntax, but not the @ syntax. From pje at telecommunity.com Mon Aug 2 18:57:51 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 18:53:50 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <410E4938.80408@zope.com> References: <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <41065006.6020607@zope.com> <200407271445.i6REjEK24454@guido.python.org> <41067660.2070109@zope.com> <1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com> <20040727164913.GS7708@epoch.metaslash.com> <41068C2F.40102@zope.com> <200407272050.i6RKoEY25164@guido.python.org> <4107870C.1070404@zope.com> <5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com> <5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com> <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com> At 10:01 AM 8/2/04 -0400, Jim Fulton wrote: >Phillip J. Eby wrote: >>At 08:28 AM 7/30/04 -0400, Jim Fulton wrote: >> >>>The problem with Python's current package system is that it's not possible, >>>in general, to write packages that can be moved around within the package >>>system, because relative imports aren't reobust or flexible enough. >>>PEP 328 would fix this. PEP 328 would allow creation of packages that >>>worked regardless of where there were places or that worked relative >>>to a containing package regardless of where that was placed. You could >>>then have different module spaces expressed by adding a new layer of >>>top-level >>>modules. You wouldn't need any new machinery beyond PEP 328. >> >>Hm. The concept I'm proposing would allow code to be used in a module >>space without needing to know about module spaces or be written to >>co-operate in such a fashion. If you can make everybody conform to some >>sort of coding standard, you wouldn't even need PEP 328 to accomplish >>your goal. ;) > >No, I don't think this is right. PEP 328 should allow you to create >module spaces using container packages. For your example, we create >two top-level packages, space1 and space 2. You put version x of PIL >in space1. That package's absolute name is space1.PIL. You put version y >of PIL in space2, creating space2.PIL. Now you put the products that >depend on version x of PIL in space1. You put the products that depend >on version y of PIL on space 2. The products mist use relative imports >to import from PIL: > > from ..PIL import somePILmodule > >For this to work, PIL also has to use relative imports to import it's own >modules. As I said, if you get to make PIL and everybody else rewrite their imports, you can do whatever you want to already. :) The point of my proposal is to make it possible *without* rewriting imports, in versions of Python from 2.2 up. From pje at telecommunity.com Mon Aug 2 19:01:02 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 18:57:00 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists In-Reply-To: <410E6CB8.3070001@zope.com> Message-ID: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com> At 12:32 PM 8/2/04 -0400, Jim Fulton wrote: >I propose to add the following function to the imp module: > > exists(module_name) > > Test whether the named module can be found. > > This method should be used when you want to perform > alternate actions depending on whether a module is available. > > Returns True of the module can be found and False otherwise. > A True return value does not guarentee that the module will be > importable. The module will not be imported as a result of > calling exists, but parent modules may be. > > If a dotted module name is given, then parent modules will be > imported if they haven't already been imported. > > >If there are no objections, I'll try to get this in before Wednesday night. Is this intended to work in the presence of import hooks? In particular, all the hooks offered by PEP 302, but also systems like 'iu', 'imputil', etc. Or, more specifically, will this be implemented in terms of the PEP 302 importer/loader protocols, or will those protocols need extending in order to allow such loaders (e.g. zipimport) to work with this new function? From pf_moore at yahoo.co.uk Mon Aug 2 19:06:09 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Mon Aug 2 19:05:56 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <410DDFE3.7010006@interlink.com.au> Message-ID: Anthony Baxter writes: > We're aiming for 2.4a2 this Thursday - so can people please > hold off on checking into the trunk from Wednesday evening > onwards, please. Can someone have a look at http://www.python.org/sf/870382 ? I'm not at all sure how to address Martin's comments, and I'd like to see this in Python 2.4, in some form or other. Thanks, Paul -- Ooh, how Gothic. Barring the milk. From pf_moore at yahoo.co.uk Mon Aug 2 19:09:57 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Mon Aug 2 19:11:04 2004 Subject: [Python-Dev] Re: Proposed Python api for testing whether a module exists References: <410E6CB8.3070001@zope.com> Message-ID: Jim Fulton writes: > I propose to add the following function to the imp module: > > exists(module_name) [...] > If there are no objections, I'll try to get this in before Wednesday night. Does this work in the presence of PEP 302 style import hooks? Specifically, the case where there's a zipfile on sys.path. Paul. -- A little inaccuracy sometimes saves tons of explanation -- Saki From bob at redivi.com Mon Aug 2 19:14:11 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 2 19:14:23 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> Message-ID: <5F9B9E10-E4A7-11D8-B47B-000A95686CD8@redivi.com> On Aug 2, 2004, at 12:55 PM, Phillip J. Eby wrote: > At 01:14 AM 8/3/04 +1000, Anthony Baxter wrote: >> Barry Warsaw wrote: >>> On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote: >>>> @decorator won? When did that happen? >>> >>> First I heard about it was when I saw Anthony's checkin messages. >>> Maybe >>> it was throwing that pie that inspired Guido -- an '@' does kind of >>> look >>> like a pie. I think Anthony's checkin message was accurate enough >>> though -- it's the syntax everyone can hate equally. But I'm glad >>> /something/ made it in! >>> print>>-set-the-precedent-ly y'rs, >>> -Barry >> >> Hey, I was just going off Guido's decision (in email - looking back, >> it >> wasn't cc'd to python-dev, which probably explains the lack of 400 >> followups ) >> >> Channelling the BDFL, I think the idea was to put it in 2.4a2, and see >> how it works for people. If it turns out that it's really really >> really >> hated, we can try something else in a3. Guido can of course follow up >> to this (hint hint) and give his own answers. > > I would think the fact that the '[decorators]' syntax can be > implemented in pure Python (no changes to the interpreter) for > existing Python versions would give more weight to it. That is, if > someone wants to implement a decorator that's forwards and > backwards-compatible, that's possible with the list syntax, but not > the @ syntax. ... but that also means you can still make the [decorators] syntax work in 2.4, if you want compatibility or don't like @syntax. -bob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3589 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/0b1718f6/smime.bin From tim.peters at gmail.com Mon Aug 2 19:15:20 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 19:15:23 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists In-Reply-To: <410E6CB8.3070001@zope.com> References: <410E6CB8.3070001@zope.com> Message-ID: <1f7befae0408021015624f0059@mail.gmail.com> [Jim Fulton] > I propose to add the following function to the imp module: > > exists(module_name) > > Test whether the named module can be found. +1. This can be much better than catching ImportError, for reasons previously given. site.py should be changed to use this to determine whether sitecustomize exists (as is, if sitecustomize exists but screws up by trying to import something that isn't on sys.path, site.py suppresses the ImportError raised *by* sitecustomize; this is bad). From amk at amk.ca Mon Aug 2 19:24:19 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 2 19:24:23 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: References: <410DDFE3.7010006@interlink.com.au> Message-ID: <20040802172419.GA9057@rogue.amk.ca> On Mon, Aug 02, 2004 at 06:06:09PM +0100, Paul Moore wrote: > Can someone have a look at http://www.python.org/sf/870382 ? I'm not > at all sure how to address Martin's comments, and I'd like to see this > in Python 2.4, in some form or other. If you'd like to encourage examination of a bug or patch at the next bug day, you can add them to the 'bugs for examination' section of http://www.python.org/moin/PythonBugDayStatus. No guarantees, of course -- if the bug can only be reviewed by one person, and that person isn't around for the bug day, nothing much can be done -- but at least it'll draw some attention to them. (Will send an announcement for the next bug day shortly...) --amk From pje at telecommunity.com Mon Aug 2 19:30:03 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 19:26:13 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5F9B9E10-E4A7-11D8-B47B-000A95686CD8@redivi.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> At 01:14 PM 8/2/04 -0400, Bob Ippolito wrote: >>I would think the fact that the '[decorators]' syntax can be implemented >>in pure Python (no changes to the interpreter) for existing Python >>versions would give more weight to it. That is, if someone wants to >>implement a decorator that's forwards and backwards-compatible, that's >>possible with the list syntax, but not the @ syntax. > >.. but that also means you can still make the [decorators] syntax work in >2.4, if you want compatibility or don't like @syntax. But then why not just make that the default syntax, so that no migration is necessary, and only one syntax has to be learned/explained to people? From jim at zope.com Mon Aug 2 19:32:12 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 19:32:16 2004 Subject: [Python-Dev] Re: Can we limit the effects of module execution to sys.modules? (was Fix import errors to have data) In-Reply-To: <1f7befae04080209424ee937a8@mail.gmail.com> References: <41065006.6020607@zope.com> <200407271445.i6REjEK24454@guido.python.org> <41067660.2070109@zope.com> <1f7befae040727085715ce2536@mail.gmail.com> <4107866C.7060609@zope.com> <1f7befae04073021175bc48767@mail.gmail.com> <410E5021.2020903@zope.com> <1f7befae04080209424ee937a8@mail.gmail.com> Message-ID: <410E7A9C.1060301@zope.com> Tim Peters wrote: > [Tim Peters] > ... >>I'd like to have a tool to report circular imports, to keep them from creeping in, >>which they do so easily.) > > > Wasn't Fred working on such a tool? He did, but I can't make sense of the output, so I can't use it. :) I'm hoping that soon Fred and I can spend some time to make sense of the output. ... Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jim at zope.com Mon Aug 2 19:37:10 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 19:37:15 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists In-Reply-To: <410E6F05.5040400@interlink.com.au> References: <410E6CB8.3070001@zope.com> <410E6F05.5040400@interlink.com.au> Message-ID: <410E7BC6.6040301@zope.com> Anthony Baxter wrote: > Jim Fulton wrote: > >> If a dotted module name is given, then parent modules will be >> imported if they haven't already been imported. > > > So you might end up spelling something like: > > if imp.exists('pkg') and imp.exists('pkg.subpkg') \ > and imp.exists('pkg.subpkg.module'): > > to avoid an importerror on the parent module? No. If pkg doesn't exist then exists will report False. Here's new wording: exists(module_name) Test whether the named module can be found. This method should be used when you want to perform alternate actions depending on whether a module is available. Returns True of the module can be found and False otherwise. A True return value does not guarentee that the module will be importable. The module will not be imported as a result of calling exists, but parent modules may be. If a dotted module name is given and parent modules exist, then the parent modules will be imported if they haven't already been imported. Thanks, Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jhylton at gmail.com Mon Aug 2 19:46:05 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Mon Aug 2 19:46:10 2004 Subject: httplib patch for 2.4 (was: Re: [Python-Dev] 2.4a2 ...) In-Reply-To: References: <410DDFE3.7010006@interlink.com.au> Message-ID: I'll take a look, probably tomorrow morning. Jeremy On Mon, 2 Aug 2004 14:30:39 +0100 (GMT Daylight Time), John J Lee wrote: > On Mon, 2 Aug 2004, Anthony Baxter wrote: > [...] > > We're aiming for 2.4a2 this Thursday - so can people please > > hold off on checking into the trunk from Wednesday evening > > onwards, please. > [...] > > Would somebody mind checking this in: > > http://www.python.org/sf/997626 > > It fixes a bug in httplib which badly breaks urllib2 now that urllib2 is > using httplib.HTTPConnection instead of httplib.HTTP (POSTs don't work). > > It includes patch and a new test, and does not require doc updates, but > nobody has reviewed it yet. From edloper at gradient.cis.upenn.edu Mon Aug 2 19:50:42 2004 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Mon Aug 2 19:49:41 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists In-Reply-To: <20040802164253.EBD151E4007@bag.python.org> References: <20040802164253.EBD151E4007@bag.python.org> Message-ID: <410E7EF2.9040707@gradient.cis.upenn.edu> > I propose to add the following function to the imp module: > > exists(module_name) > > Test whether the named module can be found. [...] > If a dotted module name is given, then parent modules will be > imported if they haven't already been imported. If a parent module doesn't exist, will it return False or raise an ImportError? -Edward From barry at python.org Mon Aug 2 20:01:23 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 2 20:01:19 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists In-Reply-To: <410E6CB8.3070001@zope.com> References: <410E6CB8.3070001@zope.com> Message-ID: <1091469682.18326.13.camel@localhost> On Mon, 2004-08-02 at 12:32, Jim Fulton wrote: > I propose to add the following function to the imp module: > > exists(module_name) > > Test whether the named module can be found. > > This method should be used when you want to perform > alternate actions depending on whether a module is available. > > Returns True of the module can be found and False otherwise. > A True return value does not guarentee that the module will be > importable. The module will not be imported as a result of > calling exists, but parent modules may be. > > If a dotted module name is given, then parent modules will be > imported if they haven't already been imported. +1, but shouldn't "parent module" really be spelled "parent package"? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/a6edcd8c/attachment.pgp From jim at zope.com Mon Aug 2 20:03:16 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 20:03:21 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists In-Reply-To: <410E7EF2.9040707@gradient.cis.upenn.edu> References: <20040802164253.EBD151E4007@bag.python.org> <410E7EF2.9040707@gradient.cis.upenn.edu> Message-ID: <410E81E4.7090702@zope.com> Edward Loper wrote: >> I propose to add the following function to the imp module: >> >> exists(module_name) >> >> Test whether the named module can be found. [...] >> If a dotted module name is given, then parent modules will be >> imported if they haven't already been imported. > > > If a parent module doesn't exist, will it return False or raise an > ImportError? Return false. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From python at rcn.com Mon Aug 2 08:20:18 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 2 20:21:53 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> Message-ID: <004101c47858$c8be8200$a2ae2c81@oemcomputer> Is anyone else seeing a stack overflow assertion error in test_pyclbr.py (WinMe, MSVC++6.0)? Raymond From guido at python.org Mon Aug 2 20:40:45 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 2 20:40:57 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: Your message of "Mon, 02 Aug 2004 13:37:42 +1200." <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408021840.i72Iejt28600@guido.python.org> > > > Circular imports, to be made consistent, would need > > > something like a final "commit" after all imports succeeded. > > > > Silly idea. Typically, circular imports aren't planned, they just > > happen, and (usually) just happen to work. > > I thought Christian meant this was something that would > be done automatically, not something the user would have > to do explicitly. OK, I misunderstood then. Apologies to Christian. > Maybe it could be as simple as saving a snapshot of > sys.modules whenever importing of a module is begun, > and if execution of its body doesn't complete, restoring > the snapshot? That sounds like overkill -- any modules that don't participate in the cycle but happen to be imported by one of the modules in the cycle would also be deleted. I still expect that the solution that Tim just checked in (delete any module that failed to complete execution) will solve most of the problems encountered in real life. I'm aware of the case you describe here: > No, it's not. If A and B import each other, and A fails after > importing B, then B ends up with a reference to a broken A > even though it executed without error. But I don't lose sleep over it; this would only be a problem if the main code attempts to import A but suppresses the import error *and* also uses B directly. This doesn't seem to be a likely scenario (modules that can be missed rarely participate in cycles). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 2 20:49:17 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 2 20:49:25 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Mon, 02 Aug 2004 16:32:03 +1000." <410DDFE3.7010006@interlink.com.au> References: <410DDFE3.7010006@interlink.com.au> Message-ID: <200408021849.i72InHa28632@guido.python.org> > The @decorator patch has been landed on the trunk, after > getting the go-ahead from the BDFL. I'll update PEP-0318 > with the final syntax in the next day or so. Thanks, Anthony! Especially thanks for thinking of updating the PEP. Jeremy reminded me last week that too often we're slacking on updating PEPs. The generator expressions PEP for example still doesn't reflect the current state of affairs. Maybe someone needs to put their foot down and require that PEPs are complete before features are checked in? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 2 20:50:12 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 2 20:50:21 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Mon, 02 Aug 2004 16:32:03 +1000." <410DDFE3.7010006@interlink.com.au> References: <410DDFE3.7010006@interlink.com.au> Message-ID: <200408021850.i72IoCg28650@guido.python.org> Did the relative import syntax that was approved ever get checked in? I thought it was on the list of things to land in 2.4? --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon Aug 2 20:53:29 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 2 20:53:30 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> Message-ID: <410E8DA9.5020304@v.loewis.de> Phillip J. Eby wrote: > But then why not just make that the default syntax, so that no migration > is necessary, and only one syntax has to be learned/explained to people? For a very simple reason: The time for discussion is now over. There is little point in going over all the same arguments again and again. For this alpha release, no amount of arguing will change what has been committed. You really have to organize a public outcry if you want the syntax changed before 2.4. Regards, Martin From martin at v.loewis.de Mon Aug 2 20:58:09 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 2 20:58:08 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: References: <410DDFE3.7010006@interlink.com.au> Message-ID: <410E8EC1.2070401@v.loewis.de> Paul Moore wrote: > Can someone have a look at http://www.python.org/sf/870382 ? I'm not > at all sure how to address Martin's comments, and I'd like to see this > in Python 2.4, in some form or other. It slipped through my attention because it is a bug report, not a patch. I will look at it again soon. That said, I should continue to require that anybody desiring incorporation of a patch should review 10 other patches first. Regards, Martin From guido at python.org Mon Aug 2 21:09:17 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 2 21:09:25 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Mon, 02 Aug 2004 13:30:03 EDT." <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> Message-ID: <200408021909.i72J9IZ28757@guido.python.org> > >>I would think the fact that the '[decorators]' syntax can be implemented > >>in pure Python (no changes to the interpreter) for existing Python > >>versions would give more weight to it. Can it? I must've missed that. It sure sounds like an incredible hack -- how to you prevent the default behavior that the list of decorators is thrown away by the interpreter? > >>That is, if someone wants to implement a decorator that's forwards > >>and backwards-compatible, that's possible with the list syntax, > >>but not the @ syntax. > > > >.. but that also means you can still make the [decorators] syntax > >work in 2.4, if you want compatibility or don't like @syntax. > > But then why not just make that the default syntax, so that no > migration is necessary, and only one syntax has to be > learned/explained to people? Because that syntax received significant boohs when I presented it at EuroPython. --Guido van Rossum (home page: http://www.python.org/~guido/) From jim at zope.com Mon Aug 2 21:16:33 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 21:16:35 2004 Subject: [Python-Dev] Proposed Python api for testing whether a module exists In-Reply-To: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com> References: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com> Message-ID: <410E9311.2090201@zope.com> Phillip J. Eby wrote: > At 12:32 PM 8/2/04 -0400, Jim Fulton wrote: > >> I propose to add the following function to the imp module: ... > Is this intended to work in the presence of import hooks? In > particular, all the hooks offered by PEP 302, but also systems like > 'iu', 'imputil', etc. > > Or, more specifically, will this be implemented in terms of the PEP 302 > importer/loader protocols, or will those protocols need extending in > order to allow such loaders (e.g. zipimport) to work with this new > function? Paul Moore wrote: ... > Does this work in the presence of PEP 302 style import hooks? > Specifically, the case where there's a zipfile on sys.path. Good question. I was planning to implement this in terms of imp.find_module, but that won't work in light of PEP 302. Dang. It's too bad that the imp.get_loader proposed in PEP 302 isn't implemented. Paul, do you have an implementation of what would have been imp.get_loader? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From amk at amk.ca Mon Aug 2 21:18:15 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 2 21:18:18 2004 Subject: [Python-Dev] Bug day coming up this Saturday Message-ID: <20040802191815.GA9528@rogue.amk.ca> The third Python Bug Day is coming up this Saturday, August 7 2004. Having more participation from people with CVS access would be very helpful. On the two previous bug days, there have been several non-developers contributing but only one or two people who can actually check in code or close bugs. You don't need to hang around all day; an hour or two is enough. This will be the last bug day before October. I won't be running a bug day in September because I'm getting married on September 19th and will need every spare weekend possible. If someone else wants to organize a September bug day, please do. (The tasks are pretty simple: updating the Wiki pages before the chosen date, and then staying on IRC for the entire day, to ensure that there's *someone* around to answer questions and commit things.) When? ===== This Saturday, August 7, 2004, from 9AM to 5PM EDT (1PM to 9PM GMT). You don't need to be around all day; feel free to stop by for a few hours and contribute. Where? ====== The #python-dev channel on irc.freenode.net. More information ================ For instructions and more information, see http://www.python.org/cgi-bin/moinmoin/PythonBugDay --amk From jim at zope.com Mon Aug 2 21:19:08 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 21:19:11 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410DDFE3.7010006@interlink.com.au> References: <410DDFE3.7010006@interlink.com.au> Message-ID: <410E93AC.9050807@zope.com> Anthony Baxter wrote: > The @decorator patch has been landed on the trunk, after > getting the go-ahead from the BDFL. I'll update PEP-0318 > with the final syntax in the next day or so. In the mean time, can anyone give a pointer to where this was proposed or discussed? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From tim.peters at gmail.com Mon Aug 2 21:31:32 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 21:31:35 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <004101c47858$c8be8200$a2ae2c81@oemcomputer> References: <004101c47858$c8be8200$a2ae2c81@oemcomputer> Message-ID: <1f7befae04080212312a1df319@mail.gmail.com> [Raymond Hettinger] > Is anyone else seeing a stack overflow assertion error in test_pyclbr.py > (WinMe, MSVC++6.0)? test_pyclbr passes on WinXP, VC7.1, release and debug builds, current CVS. From pf_moore at yahoo.co.uk Mon Aug 2 21:35:18 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Mon Aug 2 21:35:04 2004 Subject: [Python-Dev] Re: Proposed Python api for testing whether a module exists References: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com> <410E9311.2090201@zope.com> Message-ID: <1xipjpu1.fsf@yahoo.co.uk> Jim Fulton writes: > Phillip J. Eby wrote: >> At 12:32 PM 8/2/04 -0400, Jim Fulton wrote: >> >>> I propose to add the following function to the imp module: > > ... > >> Is this intended to work in the presence of import hooks? In >> particular, all the hooks offered by PEP 302, but also systems like >> 'iu', 'imputil', etc. >> Or, more specifically, will this be implemented in terms of the PEP >> 302 importer/loader protocols, or will those protocols need >> extending in order to allow such loaders (e.g. zipimport) to work >> with this new function? > > Paul Moore wrote: > ... > > Does this work in the presence of PEP 302 style import hooks? > > Specifically, the case where there's a zipfile on sys.path. > > Good question. I was planning to implement this in terms of > imp.find_module, but that won't work in light of PEP 302. Dang. > It's too bad that the imp.get_loader proposed in PEP 302 isn't > implemented. > > Paul, do you have an implementation of what would have been > imp.get_loader? Sorry, no. Just did the implementation, I mainly did some of the write-up, testing, and cheerleading. Paul. -- Ooh, how Gothic. Barring the milk. From pje at telecommunity.com Mon Aug 2 21:49:41 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 2 21:45:50 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408021909.i72J9IZ28757@guido.python.org> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> At 12:09 PM 8/2/04 -0700, Guido van Rossum wrote: > > >>I would think the fact that the '[decorators]' syntax can be implemented > > >>in pure Python (no changes to the interpreter) for existing Python > > >>versions would give more weight to it. > >Can it? I must've missed that. It sure sounds like an incredible >hack -- how to you prevent the default behavior that the list of >decorators is thrown away by the interpreter? By using sys.settrace (and a careful tracer implementation to avoid interfering with debuggers or other active tracers): def as(*decorators): if len(decorators)>1: decorators = list(decorators) decorators.reverse() def callback(frame,k,v): for d in decorators: v = d(v) frame.f_locals[k] = v add_assignment_advisor(callback) def add_assignment_advisor(callback,depth=2): """Invoke 'callback(frame,name,value)' on the next assignment in 'frame' The frame monitored is determined by the 'depth' argument, which gets passed to 'sys._getframe()'. Note that when 'callback' is invoked, the frame state will be as though the assignment hasn't happened yet, so any previous value of the assigned variable will be available in the frame's locals. (Unless there's no previous value, in which case there will be no such variable in the frame locals.) """ frame = sys._getframe(depth) oldtrace = [frame.f_trace] old_locals = frame.f_locals.copy() def tracer(frm,event,arg): if event=='call': # We don't want to trace into any calls if oldtrace[0]: # ...but give the previous tracer a chance to, if it wants return oldtrace[0](frm,event,arg) else: return None try: if frm is frame and event !='exception': # Aha, time to check for an assignment... for k,v in frm.f_locals.items(): if k not in old_locals: del frm.f_locals[k] break elif old_locals[k] is not v: frm.f_locals[k] = old_locals[k] break else: # No luck, keep tracing return tracer # Got it, fire the callback, then get the heck outta here... callback(frm,k,v) finally: # Give the previous tracer a chance to run before we return if oldtrace[0]: # And allow it to replace our idea of the "previous" tracer oldtrace[0] = oldtrace[0](frm,event,arg) # Unlink ourselves from the trace chain. frm.f_trace = oldtrace[0] sys.settrace(oldtrace[0]) return oldtrace[0] # Install the trace function frame.f_trace = tracer sys.settrace(tracer) class X(object): [as(classmethod)] def foo(cls): .... Of course, 'as' is a hack to let you use arbitrary callables as decorators, rather than having to use add_assignment_advisor directly. > > >>That is, if someone wants to implement a decorator that's forwards > > >>and backwards-compatible, that's possible with the list syntax, > > >>but not the @ syntax. > > > > > >.. but that also means you can still make the [decorators] syntax > > >work in 2.4, if you want compatibility or don't like @syntax. > > > > But then why not just make that the default syntax, so that no > > migration is necessary, and only one syntax has to be > > learned/explained to people? > >Because that syntax received significant boohs when I presented it at >EuroPython. And "@" *didn't*??? Ah well. C'est la vie. From tim.peters at gmail.com Mon Aug 2 21:45:27 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 2 21:46:04 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <004101c47858$c8be8200$a2ae2c81@oemcomputer> References: <004101c47858$c8be8200$a2ae2c81@oemcomputer> Message-ID: <1f7befae0408021245312d3c80@mail.gmail.com> [Raymond Hettinger] > Is anyone else seeing a stack overflow assertion error in test_pyclbr.py > (WinMe, MSVC++6.0)? BTW, which version of Python are you talking about? Current CVS can't compile under MSVC 6.0 anymore, because PC/pyconfig.h #defines HAVE_UINTPTR_T and HAVE_INTPTR_T now, which aren't correct for VC6, and then pyport.h goes nuts -- virtually nothing compiles anymore unless you use VC7.1. From jim at zope.com Mon Aug 2 21:55:10 2004 From: jim at zope.com (Jim Fulton) Date: Mon Aug 2 21:55:14 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> Message-ID: <410E9C1E.1010604@zope.com> Phillip J. Eby wrote: > At 12:09 PM 8/2/04 -0700, Guido van Rossum wrote: > ... >> Because that syntax received significant boohs when I presented it at >> EuroPython. > > > And "@" *didn't*??? Ah well. C'est la vie. I don't remember it being presented. Maybe I missed it, or forgot. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From skip at pobox.com Mon Aug 2 21:58:01 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 2 21:58:04 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> Message-ID: <16654.40137.30612.114797@montanaro.dyndns.org> >> Can it? I must've missed that. It sure sounds like an incredible >> hack -- how to you prevent the default behavior that the list of >> decorators is thrown away by the interpreter? Phillip> By using sys.settrace (and a careful tracer implementation to Phillip> avoid interfering with debuggers or other active tracers): I suspect Guido might class that as an "incredible hack". ;-) Skip From dave at boost-consulting.com Mon Aug 2 22:07:29 2004 From: dave at boost-consulting.com (David Abrahams) Date: Mon Aug 2 22:08:01 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <200408021909.i72J9IZ28757@guido.python.org> Message-ID: Guido van Rossum writes: >> >>I would think the fact that the '[decorators]' syntax can be implemented >> >>in pure Python (no changes to the interpreter) for existing Python >> >>versions would give more weight to it. > > Can it? I must've missed that. It sure sounds like an incredible > hack -- how to you prevent the default behavior that the list of > decorators is thrown away by the interpreter? See http://article.gmane.org/gmane.comp.python.devel/60542 I'm really surprised you missed it, and was likewise disappointed that it wasn't included in the weekly (?) summary. It seemed like a pretty significant development to me. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From foom at fuhm.net Mon Aug 2 23:13:22 2004 From: foom at fuhm.net (James Y Knight) Date: Mon Aug 2 23:13:23 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> Message-ID: On Aug 2, 2004, at 12:55 PM, Phillip J. Eby wrote: > I would think the fact that the '[decorators]' syntax can be > implemented in pure Python (no changes to the interpreter) for > existing Python versions would give more weight to it. That is, if > someone wants to implement a decorator that's forwards and > backwards-compatible, that's possible with the list syntax, but not > the @ syntax. First off, let me just say I'm happy *one* proposal was selected for inclusion. The @foo on the line before the function was not my favorite, but that's okay. It'll definitely work. However, I do think [decorator...] on the line before the function definition would be a far worse choice than @decorator. Decorators *are* special new syntax. At least @foo *looks* like new syntax. [decorators] looks like a expression creating a list, but instead is completely magical. Magical syntax seems very poor to me. About the only advantage it has is that you can implement it via a completely horrible hack in Python < 2.4. But, that is only a temporary situation. In a few years everyone will have Python >= 2.4 and we'd still be stuck with strange weird syntax that doesn't look like syntax. The one thing I do dislike about the new addition is the arbitrary (?) restriction on expression allowed in a decorator. From Grammar.. decorator: '@' dotted_name [ '(' [arglist] ')' ] That seems quite odd to me. Is there some reason it couldn't be: decorator: '@' test ? I don't see how that would cause a problem, but maybe I've missed something. Just testing out how this syntax looks, you can ignore the rest of this mail. :) @classmethod @arguments(string, int) @returns(list) def foo(s, i): return [s]*i or perhaps the same, but with a list allowed: @classmethod, arguments(string, int), returns(list) def foo(s, i): return [s]*i or original proposal: def foo(s, i) [classmethod, arguments(string, int), returns(list)]: return [s]*i Ah well, they all have their advantages. :) This one is workable. James From goodger at python.org Mon Aug 2 23:40:26 2004 From: goodger at python.org (David Goodger) Date: Mon Aug 2 23:40:48 2004 Subject: [Python-Dev] Re: ConfigParser behavior change In-Reply-To: <200408021033.03830.fdrake@acm.org> References: <410E4C10.3080009@python.org> <200408021033.03830.fdrake@acm.org> Message-ID: <410EB4CA.1030900@python.org> Skipped content of type multipart/mixed-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/5b55cd2f/signature.pgp From python at rcn.com Mon Aug 2 14:10:04 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 3 02:11:44 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <1f7befae0408021245312d3c80@mail.gmail.com> Message-ID: <006201c47889$a5554ca0$a2ae2c81@oemcomputer> > [Raymond Hettinger] > > Is anyone else seeing a stack overflow assertion error in test_pyclbr.py > > (WinMe, MSVC++6.0)? > > BTW, which version of Python are you talking about? Current CVS can't > compile under MSVC 6.0 anymore, because PC/pyconfig.h #defines > HAVE_UINTPTR_T and HAVE_INTPTR_T now, which aren't correct for VC6, > and then pyport.h goes nuts -- virtually nothing compiles anymore > unless you use VC7.1. I'm still with 6.0. Had to hack a couple lines of pyport.h and socketmodule.c which worked fine until yesterday. Too bad about 6.0. I had thought the plan was to support both 6.0 and 7.1. I'm not sure what we gained by dropping 6.0 right at the end :-( Raymond From tim.peters at gmail.com Tue Aug 3 02:18:57 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 3 02:18:59 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <006201c47889$a5554ca0$a2ae2c81@oemcomputer> References: <006201c47889$a5554ca0$a2ae2c81@oemcomputer> Message-ID: <1f7befae040802171858ab42d3@mail.gmail.com> [Raymond Hettinger] > Too bad about 6.0. I had thought the plan was to support both 6.0 and > 7.1. I'm not sure what we gained by dropping 6.0 right at the end :-( There's no plan to support MSVC 6.0. Martin routinely checks things in that break it, and that's fine. The only reason it *has* worked until recently is that people sometimes volunteer to patch up the 6.0 part of the tree. I gave some days to that at the start, because I didn't have a machine at the time that *could* run 7.1. Now I do, so I pay a lot less attention to 6.0 (and Win98SE) now. IOW, 6.0 never had, and still doesn't have, a champion (== someone willing to keep it working) for 2.4. I would have volunteered, but your report of a stack overflow in test_pyclbr scared me away . From guido at python.org Tue Aug 3 02:58:28 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 02:58:36 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Mon, 02 Aug 2004 15:49:41 EDT." <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> Message-ID: <200408030058.i730wSQ29267@guido.python.org> > >Can it? I must've missed that. It sure sounds like an incredible > >hack -- how to you prevent the default behavior that the list of > >decorators is thrown away by the interpreter? > > By using sys.settrace (and a careful tracer implementation to avoid > interfering with debuggers or other active tracers): Ah, yuck. Not an acceptable solution. And it doesn't let you write [classmethod] -- you have to wrap the 'classmethod' in something relatively ugly. But let's move on. > >Because that syntax received significant boohs when I presented it at > >EuroPython. > > And "@" *didn't*??? Ah well. C'est la vie. Indeed it didn't. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 03:05:15 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 03:05:22 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Mon, 02 Aug 2004 17:13:22 EDT." References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> Message-ID: <200408030105.i7315Fj29310@guido.python.org> > The one thing I do dislike about the new addition is the arbitrary (?) > restriction on expression allowed in a decorator. From Grammar.. > decorator: '@' dotted_name [ '(' [arglist] ')' ] > That seems quite odd to me. Is there some reason it couldn't be: > decorator: '@' test > ? The first patch on SF actually had '@' test, and I requested that it be changed. Most things that are 'test' but not 'dotted_name' optionally followed by an argument list don't make sense as decorators; if you really have a need to write @ foo or bar def f(): ... you can write deco = foo or bar @deco def f(): ... --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at cosc.canterbury.ac.nz Tue Aug 3 03:42:56 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 3 03:43:01 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <1091458891.9134.29.camel@localhost> Message-ID: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> > I think Anthony's checkin message was accurate enough though -- it's > the syntax everyone can hate equally. But I'm glad /something/ made > it in! Well, I'm not. It seems like a decision made in haste. Maybe I'll get used to it in a few decades, but right now I feel very disappointed. We were *so* close to agreeing on something nice, and then for some reason it all went pear-shaped. :-( Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From skip at pobox.com Tue Aug 3 04:01:34 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 3 04:01:36 2004 Subject: [Python-Dev] @decorators and PEP 318 In-Reply-To: <200408030105.i7315Fj29310@guido.python.org> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> Message-ID: <16654.61950.278345.835840@montanaro.dyndns.org> I'll be happy to update PEP 318 to conform to the pie decorator syntax. Can someone describe the application order? If I have @ decorator1 @ decorator2 def func(...): ... Which decorator is applied first to the function object, decorator1 or decorator2? Thx, Skip From greg at cosc.canterbury.ac.nz Tue Aug 3 04:01:35 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 3 04:01:46 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <410E02E6.9000205@stackless.com> Message-ID: <200408030201.i7321ZRo028011@cosc353.cosc.canterbury.ac.nz> > > Maybe it could be as simple as saving a snapshot of > > sys.modules whenever importing of a module is begun, > > and if execution of its body doesn't complete, restoring > > the snapshot? > > This is exactly what I thought of. > It seems to be correct, also with nested imports, > since this is a stack-like structure of undoable things. What happens in the presence of threads, though? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tismer at stackless.com Tue Aug 3 04:16:16 2004 From: tismer at stackless.com (Christian Tismer) Date: Tue Aug 3 04:13:50 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408021840.i72Iejt28600@guido.python.org> References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> <200408021840.i72Iejt28600@guido.python.org> Message-ID: <410EF570.3010901@stackless.com> Guido van Rossum wrote: >>>>Circular imports, to be made consistent, would need >>>>something like a final "commit" after all imports succeeded. >>> >>>Silly idea. Typically, circular imports aren't planned, they just >>>happen, and (usually) just happen to work. >> >>I thought Christian meant this was something that would >>be done automatically, not something the user would have >>to do explicitly. > > OK, I misunderstood then. Apologies to Christian. I love you for that one. >>Maybe it could be as simple as saving a snapshot of >>sys.modules whenever importing of a module is begun, >>and if execution of its body doesn't complete, restoring >>the snapshot? > > That sounds like overkill -- any modules that don't participate in the > cycle but happen to be imported by one of the modules in the cycle > would also be deleted. Yes, but that is ok, methinks. Any module that gets imported just on behalf of another module that *fails* to get imported, was just imported on behalf of that module, *not* by the user. And it is the user, finally, who has the say. He has to accept the extra imports of the successful imports, but by now means he is responsible of the side effects of imported code that he didn't write. If the user had imported that module before, he will still have it. If he had not, he also will not expect to have it now. I think it is just fine to remove what he didn't expect. The user says "I import that module now, and all what it needs". Nothing more, and nothing less. I know what I'm talking of, having paid for foreign kids for half of my life :-) > I still expect that the solution that Tim just checked in (delete any > module that failed to complete execution) will solve most of the > problems encountered in real life. I'm aware of the case you describe > here: Yes, but I think you are almost an atom's distance apart off of the final solution. Grab it! ciao -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Tue Aug 3 04:21:14 2004 From: tismer at stackless.com (Christian Tismer) Date: Tue Aug 3 04:19:03 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408030201.i7321ZRo028011@cosc353.cosc.canterbury.ac.nz> References: <200408030201.i7321ZRo028011@cosc353.cosc.canterbury.ac.nz> Message-ID: <410EF69A.6050306@stackless.com> Greg Ewing wrote: >>>Maybe it could be as simple as saving a snapshot of >>>sys.modules whenever importing of a module is begun, >>>and if execution of its body doesn't complete, restoring >>>the snapshot? >> >>This is exactly what I thought of. >>It seems to be correct, also with nested imports, >>since this is a stack-like structure of undoable things. > > > What happens in the presence of threads, though? To my memory, imports in the presense of threads *always* have been a nightmare. My advice always goues this way: If you must have threads, make sure to import everything through the main thread, before starting new ones. If you then still have problems, well, I have a different solution, which is also more efficient,... ... but this is another story. Off topic. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From anthony at interlink.com.au Tue Aug 3 04:26:45 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 3 04:28:02 2004 Subject: [Python-Dev] @decorators and PEP 318 In-Reply-To: <16654.61950.278345.835840@montanaro.dyndns.org> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <16654.61950.278345.835840@montanaro.dyndns.org> Message-ID: <410EF7E5.5000200@interlink.com.au> Skip Montanaro wrote: > I'll be happy to update PEP 318 to conform to the pie decorator syntax. Can > someone describe the application order? If I have > > @ decorator1 > @ decorator2 > def func(...): > ... > > Which decorator is applied first to the function object, decorator1 or > decorator2? func = decorator1(decorator2(func)) I hope to check in a revised 318 this afternoon. Anthony From anthony at interlink.com.au Tue Aug 3 04:30:19 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 3 04:30:47 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> Message-ID: <410EF8BB.7030605@interlink.com.au> Greg Ewing wrote: > Well, I'm not. It seems like a decision made in haste. > Maybe I'll get used to it in a few decades, but right > now I feel very disappointed. We were *so* close to > agreeing on something nice, and then for some reason > it all went pear-shaped. :-( PEP 318 is now nearly 14 months old. There's been a truly staggering number of posts on the subject - and every time it's raised, the same points come up. The only things to have happened recently were the new @ syntax, and PJE's settrace hack. If not now, when? This could drag on for another 14 months, otherwise - and I really don't want to hold up 2.4 final for that long. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Tue Aug 3 04:33:07 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 3 04:33:23 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408021850.i72IoCg28650@guido.python.org> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> Message-ID: <410EF963.40704@interlink.com.au> Guido van Rossum wrote: > Did the relative import syntax that was approved ever get checked in? > I thought it was on the list of things to land in 2.4? Nope. The 2.4 release schedule PEP now lists this as lacking someone to drive it. I won't/can't spend time on it. So, if one of the people who wants it could step forward, that'd be excellent. Even if it's just to do the multi-line import syntax of from foo import ( bar, baz, bar2, baz2, bar3, baz3 ) -- Anthony Baxter It's never too late to have a happy childhood. From skip at pobox.com Tue Aug 3 04:46:11 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 3 04:46:32 2004 Subject: [Python-Dev] @decorators and PEP 318 In-Reply-To: <410EF7E5.5000200@interlink.com.au> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <16654.61950.278345.835840@montanaro.dyndns.org> <410EF7E5.5000200@interlink.com.au> Message-ID: <16654.64627.453369.967660@montanaro.dyndns.org> Anthony> I hope to check in a revised 318 this afternoon. Cool. Let me know if you want any help/input/proofing/whatever... Skip From tim.peters at gmail.com Tue Aug 3 05:14:12 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 3 05:14:14 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <410EF570.3010901@stackless.com> References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> <200408021840.i72Iejt28600@guido.python.org> <410EF570.3010901@stackless.com> Message-ID: <1f7befae040802201470e3d4c@mail.gmail.com> [Christian Tismer] ... > Yes, but I think you are almost an atom's distance apart off of > the final solution. Grab it! If you mean this: >>> Maybe it could be as simple as saving a snapshot of >>> sys.modules whenever importing of a module is begun, >>> and if execution of its body doesn't complete, restoring >>> the snapshot? then no, that's a long way off in CPython. There's no choke point for when importing begins, or for when importing ends, and even __builtin__.__import__ is routinely replaced. Guido latched on to the only choke point there is: sooner or later, every import gimmick worthy of the name has to execute "the module's" code, whether it be direct import from Python, directly via C API calls, implicit package __init__.py imports, imports arranged via magical importers (like the .zip importer), etc. So that's what the patch targeted: there's one routine that executes a module's initialization code, all imports go thru it eventually, and that's the routine that now removes the module's name from sys.modules if the initialization code barfs. "The rest" of import logic is sprayed all over creation. To hook "begin import" and "end import" first requires that all importers be changed to have a formal notion of those events. The easiest start would be to use bytecodehacks to inject "call begin_import" and "call end_import" opcode sequences around the code generated for import statements <0.8 wink>. From pje at telecommunity.com Tue Aug 3 05:22:04 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 3 05:18:04 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae040802201470e3d4c@mail.gmail.com> References: <410EF570.3010901@stackless.com> <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> <200408021840.i72Iejt28600@guido.python.org> <410EF570.3010901@stackless.com> Message-ID: <5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com> At 11:14 PM 8/2/04 -0400, Tim Peters wrote: >[Christian Tismer] >.. > > Yes, but I think you are almost an atom's distance apart off of > > the final solution. Grab it! > >If you mean this: > > >>> Maybe it could be as simple as saving a snapshot of > >>> sys.modules whenever importing of a module is begun, > >>> and if execution of its body doesn't complete, restoring > >>> the snapshot? > >then no, that's a long way off in CPython. There's no choke point for >when importing begins, or for when importing ends, and even >__builtin__.__import__ is routinely replaced. Guido latched on to the >only choke point there is: sooner or later, every import gimmick >worthy of the name has to execute "the module's" code, whether it be >direct import from Python, directly via C API calls, implicit package >__init__.py imports, imports arranged via magical importers (like the >.zip importer), etc. So that's what the patch targeted: there's one >routine that executes a module's initialization code, all imports go >thru it eventually, and that's the routine that now removes the >module's name from sys.modules if the initialization code barfs. It's worse than that... 'ihooks' and 'imputil', for example, both do: exec code in module.__dict__ so they'd need to be changed to support this fix as well. (Not to mention any third-party import hooks...) From tim.peters at gmail.com Tue Aug 3 05:27:59 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 3 05:28:01 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com> References: <410EF570.3010901@stackless.com> <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> <200408021840.i72Iejt28600@guido.python.org> <410EF570.3010901@stackless.com> <5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com> Message-ID: <1f7befae0408022027465a8789@mail.gmail.com> [Phillip J. Eby] > It's worse than that... 'ihooks' and 'imputil', for example, both do: > > exec code in module.__dict__ > > so they'd need to be changed to support this fix as well. (Not to mention > any third-party import hooks...) Ouch. For that matter, they also avoid the "choke point" Sunday night's patch relied on. So there are actually no choke points for any part of the import process now. Maybe after each bytecode we could make the eval loop look to see whether sys.modules had been changed . From guido at python.org Tue Aug 3 05:51:38 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 05:51:47 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 13:42:56 +1200." <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408030351.i733pcO29560@guido.python.org> > Well, I'm not. It seems like a decision made in haste. Hardly. > Maybe I'll get used to it in a few decades, but right > now I feel very disappointed. We were *so* close to > agreeing on something nice, and then for some reason > it all went pear-shaped. :-( Not really. The contenders were: - [decorators] after the arguments. I really hate this, for good reasons which I won't repeat here. - [decorators] in front of the 'def'. For a while I really loved this, but several killer arguments were made against it, most having to do with the ambiguity, which confuses beginners. (A couple of people in the audience of my EuroPython keynote stood up and argued this with passion.) - Any number of variations on [decorators] before 'def', adding arbitrary mark-up to make it unambiguous. Including the 'as' keyword (which it isn't, by the way). All of which strike me as totally arbitrary. The 'as' keyword has a history of uses for renaming (e.g. in SQL and Python's import) but decorators are everything but renaming. - @decorators in front of 'def'. This is borrowed from Java (just like [decorators] was borrowed from C#) so minimizes invention. It isn't ambiguous, and as long as we're going to use arbitrary syntax we might as well use something that's familiar to at least one huge group of potential Python users. And yes, I'm not 100% comfortable with it either. But I know one thing: in this particular case, now is better than never. So I'm picking a solution everybody can hate equally. (What's wrong with pear-shaped, anyway? It's one of my favorite shapes. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Tue Aug 3 06:17:43 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 3 06:17:47 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408030105.i7315Fj29310@guido.python.org> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> Message-ID: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> On Aug 2, 2004, at 9:05 PM, Guido van Rossum wrote: > The first patch on SF actually had '@' test, and I requested that it > be changed. Most things that are 'test' but not 'dotted_name' > optionally followed by an argument list don't make sense as > decorators; if you really have a need to write > > @ foo or bar > def f(): ... > > you can write > > deco = foo or bar > @deco > def f(): ... An even better workaround is to write: def d(arg): return arg @d(foo or bar) def f(): ... However, it seems as if this restriction creates a new class of expression for no gain. It is true enough that *most* python expressions aren't useful to write after a @, but that's also true of a lot of other places an expression can be used (e.g. before a () of a function call. A list comprehension can never result in a callable object. An arithmetic operation usually won't.). The only real necessary restriction on the @ operator is that its argument be callable and take a single argument. Many expressions could return a callable object. Why not let them? Is it really worth having a special case just to SyntaxError expressions that sometimes won't result in an appropriate callable? Things someone might want to do, ordered roughly from most reasonable to least reasonable ;) @foo().bar() @foo or bar @mydecorators['foo'] @lambda f: foo(f) or bar(f) Why disallow these forms? It seems quite difficult, especially, to explain why the first one does not, or should not, work. James From neal at metaslash.com Tue Aug 3 06:20:18 2004 From: neal at metaslash.com (Neal Norwitz) Date: Tue Aug 3 06:20:21 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <2mpt69bo47.fsf@starship.python.net> References: <2mpt69bo47.fsf@starship.python.net> Message-ID: <20040803042017.GN7708@epoch.metaslash.com> On Mon, Aug 02, 2004 at 03:39:36PM +0100, Michael Hudson wrote: > > test_sax leaked [1899, 1899, 1899, 1899] references > no idea -- scary numbers, though! It looks like the worst (only?) leakers are: test_expat_incomplete test_expat_inpsource_location Both of these cause exceptions to be raised. I don't know the pyexpat code at all. But I think I fixed a few leaks with the attached patch. Some leaks were caught by the tests, but I doubt all were. The patch is a starting point if anyone else wants to give it a shot. If not, I'll add it to SF or review carefully and check in later. Neal -------------- next part -------------- Index: Modules/pyexpat.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/pyexpat.c,v retrieving revision 2.84 diff -u -w -r2.84 pyexpat.c --- Modules/pyexpat.c 12 Oct 2003 19:09:37 -0000 2.84 +++ Modules/pyexpat.c 3 Aug 2004 04:14:27 -0000 @@ -104,12 +104,12 @@ set_error_attr(PyObject *err, char *name, int value) { PyObject *v = PyInt_FromLong(value); + int rc = 0; - if (v != NULL && PyObject_SetAttrString(err, name, v) == -1) { - Py_DECREF(v); - return 0; - } - return 1; + if (v != NULL) + rc = PyObject_SetAttrString(err, name, v) != -1; + Py_XDECREF(v); + return rc; } /* Build and set an Expat exception, including positioning @@ -359,7 +359,7 @@ { PyThreadState *tstate = PyThreadState_GET(); PyFrameObject *f; - PyObject *res; + PyObject *res = NULL; if (c == NULL) return NULL; @@ -370,7 +370,7 @@ tstate->frame = f; #ifdef FIX_TRACE if (trace_frame(tstate, f, PyTrace_CALL, Py_None) < 0) { - return NULL; + goto cleanup; } #endif res = PyEval_CallObject(func, args); @@ -379,7 +379,7 @@ PyTraceBack_Here(f); #ifdef FIX_TRACE if (trace_frame_exc(tstate, f) < 0) { - return NULL; + goto cleanup; } } else { @@ -391,6 +391,7 @@ #else } #endif +cleanup: tstate->frame = f->f_back; Py_DECREF(f); return res; @@ -996,6 +997,7 @@ if (fp) { bytes_read = fread(buf, sizeof(char), BUF_SIZE, fp); if (bytes_read < 0) { + Py_XDECREF(readmethod); PyErr_SetFromErrno(PyExc_IOError); return NULL; } From eppstein at ics.uci.edu Tue Aug 3 06:28:52 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Tue Aug 3 06:28:56 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: In article <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>, James Y Knight wrote: > > @ foo or bar > > def f(): ... > > > > you can write > > > > deco = foo or bar > > @deco > > def f(): ... > > An even better workaround is to write: > def d(arg): return arg > > @d(foo or bar) > def f(): ... > > However, it seems as if this restriction creates a new class of > expression for no gain. It is true enough that *most* python > expressions aren't useful to write after a @, but that's also true of a > lot of other places an expression can be used (e.g. before a () of a > function call. A list comprehension can never result in a callable > object. An arithmetic operation usually won't.). As you say, there aren't a lot of types of Python expression that return callables. The main thing this restriction seems to prevent is @lambda, and I think preventing that is a good thing. The other possibility that comes to mind is @functable[index], but I'd have to see a use case before worrying too much that it's not allowed. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From exarkun at divmod.com Tue Aug 3 06:50:56 2004 From: exarkun at divmod.com (Jp Calderone) Date: Tue Aug 3 06:51:00 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <410F19B0.2090809@divmod.com> James Y Knight wrote: > > [snip] > > > An even better workaround is to write: > def d(arg): return arg > > @d(foo or bar) > def f(): ... > > However, it seems as if this restriction creates a new class of > expression for no gain. It is true enough that *most* python expressions > aren't useful to write after a @, but that's also true of a lot of other > places an expression can be used (e.g. before a () of a function call. A > list comprehension can never result in a callable object. An arithmetic > operation usually won't.). > > The only real necessary restriction on the @ operator is that its > argument be callable and take a single argument. Many expressions could > return a callable object. Why not let them? Is it really worth having a > special case just to SyntaxError expressions that sometimes won't result > in an appropriate callable? > > Things someone might want to do, ordered roughly from most reasonable to > least reasonable ;) > @foo().bar() > @foo or bar > @mydecorators['foo'] > @lambda f: foo(f) or bar(f) > > Why disallow these forms? It seems quite difficult, especially, to > explain why the first one does not, or should not, work. > > James > For what it's worth, I agree with James completely. Jp From python at rcn.com Mon Aug 2 18:54:35 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 3 06:56:12 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410F19B0.2090809@divmod.com> Message-ID: <002201c478b1$6461ebe0$eb23c797@oemcomputer> > > The only real necessary restriction on the @ operator is that its > > argument be callable and take a single argument. Many expressions could > > return a callable object. Why not let them? Is it really worth having a > > special case just to SyntaxError expressions that sometimes won't result > > in an appropriate callable? > > > > Things someone might want to do, ordered roughly from most reasonable to > > least reasonable ;) > > @foo().bar() > > @foo or bar > > @mydecorators['foo'] > > @lambda f: foo(f) or bar(f) > > > > Why disallow these forms? It seems quite difficult, especially, to > > explain why the first one does not, or should not, work. > > > > James > > > > For what it's worth, I agree with James completely. > > Jp I concur. Raymond From python at rcn.com Mon Aug 2 19:11:49 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 3 07:13:25 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects listobject.c, 2.218, 2.219 In-Reply-To: Message-ID: <002b01c478b3$cc9c7020$eb23c797@oemcomputer> > * complain about the unused name. > */ > ! return status, v; > } > > --- 863,867 ---- > * complain about the unused name. > */ > ! return status++, v; Does anyone else see this as perverse? I have to think hard about whether all the compilers are smart enough to optimize this away (probably, but maybe not). More importantly, it is obfuscated C and makes my stomach churn. This particular case introduces several lines for an assertion that is probably not worth the surrounding mess. Does anyone have a better way? Raymond From bob at redivi.com Tue Aug 3 07:28:00 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 3 07:28:02 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <002201c478b1$6461ebe0$eb23c797@oemcomputer> References: <002201c478b1$6461ebe0$eb23c797@oemcomputer> Message-ID: On Aug 2, 2004, at 12:54 PM, Raymond Hettinger wrote: >>> The only real necessary restriction on the @ operator is that its >>> argument be callable and take a single argument. Many expressions > could >>> return a callable object. Why not let them? Is it really worth > having a >>> special case just to SyntaxError expressions that sometimes won't > result >>> in an appropriate callable? >>> >>> Things someone might want to do, ordered roughly from most > reasonable to >>> least reasonable ;) >>> @foo().bar() >>> @foo or bar >>> @mydecorators['foo'] >>> @lambda f: foo(f) or bar(f) >>> >>> Why disallow these forms? It seems quite difficult, especially, to >>> explain why the first one does not, or should not, work. >>> >>> James >>> >> >> For what it's worth, I agree with James completely. >> >> Jp > > I concur. > > > Raymond Ditto. -bob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3589 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/9809f219/smime.bin From exarkun at divmod.com Tue Aug 3 07:36:19 2004 From: exarkun at divmod.com (Jp Calderone) Date: Tue Aug 3 07:36:25 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408030058.i730wSQ29267@guido.python.org> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> Message-ID: <410F2453.7000506@divmod.com> Guido van Rossum wrote: >>>Can it? I must've missed that. It sure sounds like an incredible >>>hack -- how to you prevent the default behavior that the list of >>>decorators is thrown away by the interpreter? >> >>By using sys.settrace (and a careful tracer implementation to avoid >>interfering with debuggers or other active tracers): > > > Ah, yuck. Not an acceptable solution. And it doesn't let you write > > [classmethod] > > -- you have to wrap the 'classmethod' in something relatively ugly. > Here's a brief test for a syntax-change-less implementation of this feature, not as complete as test_decorators, but a good start, I believe: def test(): try: from test import test_decorators except ImportError: test_decorators = None class DecoratorTest(object): __metaclass__ = DecoratableType def foo(self): print 'undecorated foo' decorate(staticmethod) def bar(x): print x decorate(classmethod) def baz(cls, y): print cls, y if test_decorators: counts = {} decorate(test_decorators.countcalls(counts), test_decorators.memoize) def quux(self): print 'quux called' o = DecoratorTest() o.foo() o.bar('static method on instance object') o.baz('class method on the instance') DecoratorTest.bar('static method on the class') DecoratorTest.baz('class method on the class') if test_decorators: print 'Calling quux once' o.quux() print 'Calling quux twice' o.quux() print 'Called quux', DecoratorTest.counts['quux'], 'times' And here's the implementation, without using settrace, and without requiring wrappers for individual decorators: import inspect MAGIC_NAME = '__internal_decorators_list__' class Decorator(object): def __init__(self, firstCallable, *callables): cf = inspect.currentframe() self.callables = [firstCallable] + list(callables) self.decoratesLine = cf.f_back.f_lineno + 1 cf.f_back.f_locals.setdefault(MAGIC_NAME, []).append(self) def __call__(self, f): i = iter(self.callables) f = i.next()(f) for c in i: f = c(f) return f decorate = Decorator class DecoratableType(type): def __new__(cls, name, bases, attrs): decorators = attrs.get(MAGIC_NAME, []) if decorators: del attrs[MAGIC_NAME] lines = {} for (k, v) in attrs.items(): try: source, lineno = inspect.getsourcelines(v) except: pass else: lines[lineno] = k for d in decorators: if d.decoratesLine in lines: k = lines[d.decoratesLine] attrs[k] = d(attrs[k]) return super(DecoratableType, cls).__new__( cls, name, bases, attrs) There are clear drawbacks to this approach. Metaclass required, no obvious ways to support free functions, and it depends _slightly_ hackishly on the inspect module. I think at least one of these problems can be solved (and line number handling can be made smarter to deal with intervening comments, whitespace, etc). What are the advantages? * It works with Python 2.2 and Python 2.3. * It requires no interpreter changes. It can be distributed with the standard library, as a cookbook recipe, or in the official documentation as a hint regarding more "advanced" decorator usage. * It introduces no new syntax and uses up no operator character. * It supports arbitrary expressions. * It's pure python. I realize there is little or no chance of '@decorator' being pulled from 2.4a2. I hope that something along the lines of the above will be considered, instead, for the next alpha, unless there is widespread community support for '@decorator', as opposed to the ridiculously faint support ("it's better than nothing") currently behind it. Jean-Paul Calderone From martin at v.loewis.de Tue Aug 3 08:00:57 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 3 08:00:57 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <006201c47889$a5554ca0$a2ae2c81@oemcomputer> References: <006201c47889$a5554ca0$a2ae2c81@oemcomputer> Message-ID: <410F2A19.1020209@v.loewis.de> Raymond Hettinger wrote: > I'm still with 6.0. Had to hack a couple lines of pyport.h and > socketmodule.c which worked fine until yesterday. > > Too bad about 6.0. I had thought the plan was to support both 6.0 and > 7.1. I'm not sure what we gained by dropping 6.0 right at the end :-( Someone please read the comment in PC/pyconfig.h: /* Atleast VC 7.1 has them. If some compiler does not provide them, #ifdef appropriately .*/ #define HAVE_UINTPTR_T 1 #define HAVE_INTPTR_T 1 I did not bother checking whether VC6 has uintptr_t - that does not mean that somebody else couldn't. Hacking a couple of lines is probably the wrong approach. Properly maintaining the VC6 port (i.e. sharing the well-designed changes to a couple of lines) would be a good idea. Regards, Martin From fdrake at acm.org Tue Aug 3 08:06:48 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue Aug 3 08:07:02 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <20040803042017.GN7708@epoch.metaslash.com> References: <2mpt69bo47.fsf@starship.python.net> <20040803042017.GN7708@epoch.metaslash.com> Message-ID: <200408030206.48728.fdrake@acm.org> On Tuesday 03 August 2004 12:20 am, Neal Norwitz wrote: > It looks like the worst (only?) leakers are: > > test_expat_incomplete > test_expat_inpsource_location > > Both of these cause exceptions to be raised. I don't know the pyexpat > code at all. But I think I fixed a few leaks with the attached patch. > Some leaks were caught by the tests, but I doubt all were. The patch > is a starting point if anyone else wants to give it a shot. If not, > I'll add it to SF or review carefully and check in later. Please post to SF and assign to me; I'll take a look sometime tomorrow. -Fred -- Fred L. Drake, Jr. From fperez528 at yahoo.com Tue Aug 3 08:15:16 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Tue Aug 3 08:15:17 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> Message-ID: Jp Calderone wrote: > Here's a brief test for a syntax-change-less implementation of this > feature, not as complete as test_decorators, but a good start, I believe: [...] > from 2.4a2. I hope that something along the lines of the above will be > considered, instead, for the next alpha, unless there is widespread > community support for '@decorator', as opposed to the ridiculously faint > support ("it's better than nothing") currently behind it. FWIW, I'd be very much +1 on something like this, which feels much more 'pythonic' and elegant to me than the @ syntax. I know I'm biased because the @ syntax will force me to change ipython and it will break current ipython for anyone wanting to use 2.4, but I'm honestly trying to assess it beyond ipython. And I still don't like it, while JP's solution seems nice and clean to me. Cheers, f From djc at object-craft.com.au Tue Aug 3 08:44:53 2004 From: djc at object-craft.com.au (Dave Cole) Date: Tue Aug 3 08:45:03 2004 Subject: [Python-Dev] Tuple/list assignment question Message-ID: <410F3465.9000005@object-craft.com.au> Is there any reason why something like this would not be a good idea? >>> a_list = [1, 2, 3, 4, 5] >>> a, b, *c = a_list You could then do things like this: >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] >>> for a, b *c in lol: ... - Dave -- http://www.object-craft.com.au From ncoghlan at iinet.net.au Tue Aug 3 09:02:53 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Aug 3 09:03:21 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <410F3465.9000005@object-craft.com.au> References: <410F3465.9000005@object-craft.com.au> Message-ID: <410F389D.6090609@iinet.net.au> Dave Cole wrote: > Is there any reason why something like this would not be a good idea? > > >>> a_list = [1, 2, 3, 4, 5] > >>> a, b, *c = a_list > > You could then do things like this: > > >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] > >>> for a, b *c in lol: > ... > > - Dave > As opposed to: >>> for a, b, c in ((x[0], x[1], x[2:]) for x in lol): print a, b, c With generator expressions around, I don't know that this case is common enough for special casing. . . Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268 From arigo at tunes.org Tue Aug 3 10:54:01 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Aug 3 10:58:44 2004 Subject: [Python-Dev] Optimized string concatenation Message-ID: <20040803085401.GA2798@vicky.ecs.soton.ac.uk> Hello, The SF patch http://www.python.org/sf/980695 about making repeated string concatenations efficient has been reviewed and is acceptable on technical grounds. This is about avoiding the quadratic behavior of s = '' for x in y: s += some_string(x) This leaves open the policy questions: * first, is that an implementation detail or a published feature? The question is important because the difference in performance is enormous -- we are not talking about 2x or even 10x faster but roughly Nx faster where N is the size of the input data set. * if it is a published feature, what about Jython? * The patch would encourage a coding style that gives program that essentially don't scale with Jython -- nor, for that matter, with 2.3 or older -- and worse, the programs would *appear* to work on Jython or 2.3 when tested with small or medium-sized data sets, but just appear to hang when run on larger data sets. Obviously, this is problem that has always been here, but if we fix it in 2.4 we can be sure that people will develop and test with 2.4, and less thoroughly on 2.3, and when they deploy on 2.3 platforms it will unexpectedly not scale. * discussed on SF too is whether we should remove the 'a=a+b' acceleration from the patch, keeping only 'a+=b'; see the SF tracker. This seems overkill, but should the acceleration be there but disabled by default? from __future__ import string_concatenate? A bient?t, Armin. From arigo at tunes.org Tue Aug 3 11:12:33 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Aug 3 11:17:17 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects listobject.c, 2.218, 2.219 In-Reply-To: <002b01c478b3$cc9c7020$eb23c797@oemcomputer> References: <002b01c478b3$cc9c7020$eb23c797@oemcomputer> Message-ID: <20040803091233.GB2798@vicky.ecs.soton.ac.uk> Hello Raymond, On Mon, Aug 02, 2004 at 01:11:49PM -0400, Raymond Hettinger wrote: > > * complain about the unused name. > > */ > > ! return status, v; > > } > > > > --- 863,867 ---- > > * complain about the unused name. > > */ > > ! return status++, v; > > Does anyone else see this as perverse? Yes! And obfuscated definitely. A general solution to this problem would be helpful... well in this case I can think about a specific solution: assert(!PyErr_Occurred()); Patch below. Armin *** listobject.c 3 Aug 2004 04:53:29 -0000 2.219 --- listobject.c 3 Aug 2004 09:15:45 -0000 *************** *** 829,835 **** { int i = -1; PyObject *v, *arg = NULL; - int status; if (!PyArg_UnpackTuple(args, "pop", 0, 1, &arg)) return NULL; --- 829,834 ---- *************** *** 852,868 **** } v = self->ob_item[i]; if (i == self->ob_size - 1) { ! status = list_resize(self, self->ob_size - 1); ! assert(status >= 0); return v; /* and v now owns the reference the list had */ } Py_INCREF(v); ! status = list_ass_slice(self, i, i+1, (PyObject *)NULL); ! assert(status >= 0); ! /* Use status, so that in a release build compilers don't ! * complain about the unused name. ! */ ! return status++, v; } /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */ --- 851,863 ---- } v = self->ob_item[i]; if (i == self->ob_size - 1) { ! list_resize(self, self->ob_size - 1); return v; /* and v now owns the reference the list had */ } Py_INCREF(v); ! list_ass_slice(self, i, i+1, (PyObject *)NULL); ! assert(!PyErr_Occurred()); ! return v; } /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */ From sjoerd at acm.org Tue Aug 3 11:48:24 2004 From: sjoerd at acm.org (Sjoerd Mullender) Date: Tue Aug 3 11:48:30 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects listobject.c, 2.218, 2.219 In-Reply-To: <20040803091233.GB2798@vicky.ecs.soton.ac.uk> References: <002b01c478b3$cc9c7020$eb23c797@oemcomputer> <20040803091233.GB2798@vicky.ecs.soton.ac.uk> Message-ID: <410F5F68.9030704@acm.org> Armin Rigo wrote: > Hello Raymond, > > On Mon, Aug 02, 2004 at 01:11:49PM -0400, Raymond Hettinger wrote: > >>> * complain about the unused name. >>> */ >>>! return status, v; >>> } >>> >>>--- 863,867 ---- >>> * complain about the unused name. >>> */ >>>! return status++, v; >> >>Does anyone else see this as perverse? > > > Yes! And obfuscated definitely. A general solution to this problem would be > helpful... well in this case I can think about a specific solution: This is indeed perverted. Another option is to add a statement (void) status; somewhere to the function and to remove the status++ from the return. This seems to shut up all compilers I've dealt with. > assert(!PyErr_Occurred()); > > Patch below. > > Armin > > *** listobject.c 3 Aug 2004 04:53:29 -0000 2.219 > --- listobject.c 3 Aug 2004 09:15:45 -0000 > *************** > *** 829,835 **** > { > int i = -1; > PyObject *v, *arg = NULL; > - int status; > > if (!PyArg_UnpackTuple(args, "pop", 0, 1, &arg)) > return NULL; > --- 829,834 ---- > *************** > *** 852,868 **** > } > v = self->ob_item[i]; > if (i == self->ob_size - 1) { > ! status = list_resize(self, self->ob_size - 1); > ! assert(status >= 0); > return v; /* and v now owns the reference the list had */ > } > Py_INCREF(v); > ! status = list_ass_slice(self, i, i+1, (PyObject *)NULL); > ! assert(status >= 0); > ! /* Use status, so that in a release build compilers don't > ! * complain about the unused name. > ! */ > ! return status++, v; > } > > /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */ > --- 851,863 ---- > } > v = self->ob_item[i]; > if (i == self->ob_size - 1) { > ! list_resize(self, self->ob_size - 1); > return v; /* and v now owns the reference the list had */ > } > Py_INCREF(v); > ! list_ass_slice(self, i, i+1, (PyObject *)NULL); > ! assert(!PyErr_Occurred()); > ! return v; > } > > /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */ > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/sjoerd%40acm.org -- Sjoerd Mullender -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 374 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/abc2c638/signature.pgp From mwh at python.net Tue Aug 3 12:03:54 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 3 12:03:55 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <20040803042017.GN7708@epoch.metaslash.com> (Neal Norwitz's message of "Tue, 3 Aug 2004 00:20:18 -0400") References: <2mpt69bo47.fsf@starship.python.net> <20040803042017.GN7708@epoch.metaslash.com> Message-ID: <2m4qnkbks5.fsf@starship.python.net> Neal Norwitz writes: > On Mon, Aug 02, 2004 at 03:39:36PM +0100, Michael Hudson wrote: >> >> test_sax leaked [1899, 1899, 1899, 1899] references >> no idea -- scary numbers, though! > > It looks like the worst (only?) leakers are: > > test_expat_incomplete > test_expat_inpsource_location Yes. The rest are all caused by the module level variable items containing it's old value... > Both of these cause exceptions to be raised. I don't know the pyexpat > code at all. But I think I fixed a few leaks with the attached patch. I certainly agree with your patch to set_error_attr (I had the same). Also, I think one needs to decref 'err' in set_error (PyErr_SetObject increfs its arguments). > Some leaks were caught by the tests, but I doubt all were. Probably not :-/ > The patch is a starting point if anyone else wants to give it a > shot. If not, I'll add it to SF or review carefully and check in > later. Well, it seems Fred wants a look. I'll attach my diffs to pyexpat.c to the report (there's another fairly obvious leak exposed by test_minidom). Cheers, mwh -- > I'm a little confused. That's because you're Australian! So all the blood flows to your head, away from the organ most normal guys think with. -- Mark Hammond & Tim Peters, comp.lang.python From mwh at python.net Tue Aug 3 12:33:13 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 3 12:33:16 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410E93AC.9050807@zope.com> (Jim Fulton's message of "Mon, 02 Aug 2004 15:19:08 -0400") References: <410DDFE3.7010006@interlink.com.au> <410E93AC.9050807@zope.com> Message-ID: <2mzn5ca4uu.fsf@starship.python.net> Jim Fulton writes: > Anthony Baxter wrote: >> The @decorator patch has been landed on the trunk, after >> getting the go-ahead from the BDFL. I'll update PEP-0318 >> with the final syntax in the next day or so. > > In the mean time, can anyone give a pointer to where this was proposed > or discussed? > AFAIK, in offline email (and occasionally on IRC, but that was mostly about implementation), presumably to avoid the thousands of emails cascade this subject inevitably seems to trigger. bike-shed-the-ultimate-ly y'rs mwh -- If a train station is a place where a train stops, what's a workstation? -- unknown (to me, at least) From mwh at python.net Tue Aug 3 12:36:31 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 3 12:36:33 2004 Subject: [Python-Dev] Bug day coming up this Saturday In-Reply-To: <20040802191815.GA9528@rogue.amk.ca> (A. M. Kuchling's message of "Mon, 2 Aug 2004 15:18:15 -0400") References: <20040802191815.GA9528@rogue.amk.ca> Message-ID: <2mvfg0a4pc.fsf@starship.python.net> "A.M. Kuchling" writes: > When? > ===== > > This Saturday, August 7, 2004, from 9AM to 5PM EDT (1PM to 9PM GMT). > You don't need to be around all day; feel free to stop by for a few hours > and contribute. I should be able to attend this one. Cheers, mwh -- (Of course SML does have its weaknesses, but by comparison, a discussion of C++'s strengths and flaws always sounds like an argument about whether one should face north or east when one is sacrificing one's goat to the rain god.) -- Thant Tessman From jim at zope.com Tue Aug 3 12:43:14 2004 From: jim at zope.com (Jim Fulton) Date: Tue Aug 3 12:43:57 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <410F6C42.8010001@zope.com> James Y Knight wrote: ... > Things someone might want to do, ordered roughly from most reasonable to > least reasonable ;) > @foo().bar() > @foo or bar > @mydecorators['foo'] > @lambda f: foo(f) or bar(f) > > Why disallow these forms? It seems quite difficult, especially, to > explain why the first one does not, or should not, work. +1 -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From mwh at python.net Tue Aug 3 13:36:26 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 3 13:36:28 2004 Subject: [Python-Dev] refleak hunting season In-Reply-To: <2mpt69bo47.fsf@starship.python.net> (Michael Hudson's message of "Mon, 02 Aug 2004 15:39:36 +0100") References: <2mpt69bo47.fsf@starship.python.net> Message-ID: <2mr7qoa1xh.fsf@starship.python.net> Michael Hudson writes: > Last time round I found a bunch of reference counting bugs in Python > just *after* a major release; I'm trying to do it all before 2.4.0 > this time. I think I've now accounted for all the apparent and real refleaks revealed by the test suite (not with -uall, will try that soon). I've checked my regrtest patches in. I've fixed a bunch of tests that failed when repeated and I've filed bugs against those I couldn't figure out. Hopefully we can keep on top of these things! Cheers, mwh -- Also, remember to put the galaxy back when you've finished, or an angry mob of astronomers will come round and kneecap you with a small telescope for littering. -- Simon Tatham, ucam.chat, from Owen Dunn's review of the year From marktrussell at btopenworld.com Tue Aug 3 13:51:14 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Tue Aug 3 13:51:17 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410F2453.7000506@divmod.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> Message-ID: <1091533874.3797.0.camel@localhost> On Tue, 2004-08-03 at 06:36, Jp Calderone wrote: > I realize there is little or no chance of '@decorator' being pulled > from 2.4a2. I hope that something along the lines of the above will be > considered, instead, for the next alpha, unless there is widespread > community support for '@decorator', as opposed to the ridiculously faint > support ("it's better than nothing") currently behind it. Personally I think it's not just better than nothing, it's better than any of the alternatives I've seen, including the do-nothing option. I added a comment to your blog post in defence of @decorator -- http://www.livejournal.com/users/jcalderone/3913.html?thread=2121#t2121 but lets not restart a syntax debate here. My guess is that the syntax will grow on people as they use it. If not there is the alternative of using one of the pure-python versions or just carrying on using the existing way of doing things. Mark From mcherm at mcherm.com Tue Aug 3 14:30:04 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 3 14:29:30 2004 Subject: [Python-Dev] Optimized string concatenation Message-ID: <1091536204.410f854c18727@mcherm.com> Armin writes: > The SF patch http://www.python.org/sf/980695 about making > repeated string > concatenations efficient has been reviewed and is acceptable > on technical grounds. [...] > This leaves open the policy questions: > > * first, is that an implementation detail or a published feature? IMHO, an implementation detail. Specifically, a published feature of CPython, but an implementation detail of the language Python. > * if it is a published feature, what about Jython? And I guess we should ask about Iron Python too. The answer is that they add this optimization if and only if the maintainers of those versions get around to it. Whether it is easily done given the underlying platform is another important consideration. But if we leave it OUT of the language spec then we avoid unnecessarily breaking other implementations. That being said, I'll note that both Jython and Iron Python already use unicode as the basic string type, which is a much bigger change (except to those who manage to use only 7-bit ASCII). > * The patch would encourage a coding style that gives program > that essentially > don't scale with Jython -- nor, for that matter, with 2.3 > or older Yes, but we ALREADY see lots of programs that use that coding style, even though every web page talking about Python optimization lists that as the #1 issue. Whether we like it or not, it seems that especially for novice programmers, the "s = s + x" idiom for accumulating a string is the "obvious" way to do it. Encouraging that may not be good, but going along rather than fighting it seems like a wise idea. > * discussed on SF too is whether we should remove the 'a=a+b' > acceleration > from the patch, keeping only 'a+=b'; see the SF tracker. Hmm... I couldn't think of any reason to limit the optimization to += until I actually went and read the comments in the SF tracker. What I took away from that discussion was that it's possible to optimize "a=a+b", but NOT possible to optimize "a=a+b+c". This is a subtle distinction that is harder to explain to people than simply saying "it only works with +=, not with +". From mcherm at mcherm.com Tue Aug 3 14:33:29 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 3 14:32:56 2004 Subject: [Python-Dev] Optimized string concatenation Message-ID: <1091536409.410f8619be6f0@mcherm.com> PLEASE IGNORE PREVIOUS MESSAGE. (I really ought to stop using a mailer which is one accidental click away from sending an unfinished email) Armin writes: > The SF patch http://www.python.org/sf/980695 about making > repeated string > concatenations efficient has been reviewed and is acceptable > on technical grounds. [...] > This leaves open the policy questions: > > * first, is that an implementation detail or a published feature? IMHO, an implementation detail. Specifically, a published feature of CPython, but an implementation detail of the language Python. > * if it is a published feature, what about Jython? And I guess we should ask about Iron Python too. The answer is that they add this optimization if and only if the maintainers of those versions get around to it. Whether it is easily done given the underlying platform is another important consideration. But if we leave it OUT of the language spec then we avoid unnecessarily breaking other implementations. That being said, I'll note that both Jython and Iron Python already use unicode as the basic string type, which is a much bigger change (except to those who manage to use only 7-bit ASCII). > * The patch would encourage a coding style that gives program > that essentially > don't scale with Jython -- nor, for that matter, with 2.3 > or older Yes, but we ALREADY see lots of programs that use that coding style, even though every web page talking about Python optimization lists that as the #1 issue. Whether we like it or not, it seems that especially for novice programmers, the "s = s + x" idiom for accumulating a string is the "obvious" way to do it. Encouraging that may not be good, but going along rather than fighting it seems like a wise idea. > * discussed on SF too is whether we should remove the 'a=a+b' > acceleration > from the patch, keeping only 'a+=b'; see the SF tracker. Hmm... I couldn't think of any reason to limit the optimization to += until I actually went and read the comments in the SF tracker. What I took away from that discussion was that it's possible to optimize "a=a+b", but NOT possible to optimize "a=a+b+c". This is a subtle distinction that is harder to explain to people than simply saying "it only works with +=, not with +". That's a fairly convincing point, so I guess I'm on the fence on this one. > This seems overkill, but should the acceleration be there but > disabled by > default? > > from __future__ import string_concatenate? Absolutely, unconditionally NOT. I'd rather just leave it out. -- Michael Chermside From jhylton at gmail.com Tue Aug 3 14:47:41 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Tue Aug 3 14:47:44 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410F2453.7000506@divmod.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> Message-ID: On Tue, 03 Aug 2004 01:36:19 -0400, Jp Calderone wrote: [JP provides a partial implementation of a variant decorator approach] > I realize there is little or no chance of '@decorator' being pulled > from 2.4a2. I hope that something along the lines of the above will be > considered, instead, for the next alpha, unless there is widespread > community support for '@decorator', as opposed to the ridiculously faint > support ("it's better than nothing") currently behind it. I'm not sure how to take your post. In particular, I don't see the implementation as the limiting factor in decorator design. We can probably implement any design that we want to. The question is actually about the preferred design. You seem to be suggesting a design where decorate() is used as a pseudo-keyword; assuming, for example, that you'd want it to work for functions as well as methods. I suspect it's futile to propose alternate designs at this point. Jeremy From jhylton at gmail.com Tue Aug 3 14:51:36 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Tue Aug 3 14:51:38 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects listobject.c, 2.218, 2.219 In-Reply-To: <410F5F68.9030704@acm.org> References: <002b01c478b3$cc9c7020$eb23c797@oemcomputer> <20040803091233.GB2798@vicky.ecs.soton.ac.uk> <410F5F68.9030704@acm.org> Message-ID: On Tue, 03 Aug 2004 11:48:24 +0200, Sjoerd Mullender wrote: > This is indeed perverted. Another option is to add a statement > (void) status; > somewhere to the function and to remove the status++ from the return. > This seems to shut up all compilers I've dealt with. I believe we use this idiom elsewhere in the Python codebase. If it's combined with a comment like /* avoid compile warning about unused variable */ it's quite clear. Jeremy From FBatista at uniFON.com.ar Tue Aug 3 15:08:53 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Aug 3 15:12:08 2004 Subject: [Python-Dev] Optimized string concatenation Message-ID: [Armin Rigo] #- s = '' #- for x in y: #- s += some_string(x) #- #- This leaves open the policy questions: #- #- * first, is that an implementation detail or a published feature? Depending on how much slower (if still) is now this idiom against... l = [some_string(x) for x in y] s = ''.join(l) ...maybe we should stop concatenating strings. One or two points of the FAQs should be reviewed, but only after a speed comparison. . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040803/357dd29e/attachment.html From pje at telecommunity.com Tue Aug 3 16:04:27 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 3 16:00:26 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410F2453.7000506@divmod.com> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> Message-ID: <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> At 01:36 AM 8/3/04 -0400, Jp Calderone wrote: > There are clear drawbacks to this approach. Metaclass required, no > obvious ways to support free functions, and it depends _slightly_ > hackishly on the inspect module. I think at least one of these > problems can be solved (and line number handling can be made smarter to > deal with intervening comments, whitespace, etc). But at least *two* of those drawbacks are critical, IMO: the metaclass requirement and the absence of free function support. > What are the advantages? > > * It works with Python 2.2 and Python 2.3. > > * It requires no interpreter changes. It can be distributed with the > standard library, as a cookbook recipe, or in the official documentation > as a hint regarding more "advanced" decorator usage. > > * It introduces no new syntax and uses up no operator character. > > * It supports arbitrary expressions. > > * It's pure python. If those are the only advantages, one might as well use the settrace-based hack, which doesn't have the problems that this one does. Note, by the way, that your 'decorate' function can be implemented by renaming my 'as' function to 'decorate' and dropping the 'decorators.reverse()' line. (i.e. 'as(x,y,z)' is the same as 'decorate(z,y,x)'). Although my examples wrapped such calls in brackets, there's no actual need for them. (Note also that the settrace hack works for class decorations as well as functions.) (FYI: 'as' and 'add_assignment_advisor' are available in the PyProtocols CVS head, in the 'protocols.advice' module.) > I realize there is little or no chance of '@decorator' being pulled > from 2.4a2. I hope that something along the lines of the above will be > considered, instead, for the next alpha, unless there is widespread > community support for '@decorator', as opposed to the ridiculously faint > support ("it's better than nothing") currently behind it. Actually, I'm not sure I agree that it's better than nothing. For one thing, I don't think that there's any ambiguity of interpretation when decorators are written as function calls, be it 'as(classmethod)' or 'decorate(spam)' -- and I doubt that the point was argued at EuroPython, given that Guido missed the thread about it. But... this is moot, since it clearly has been pronounced upon. That pretty much means folks who want decorators right away are left with using the hacks, and not having a syntax-compatible upgrade path. However, that's *still* better than where they were before, because none of the original syntax proposals were expected to be backward-compatible. In practical terms, all this means is that I'll just use the hacked syntax until I no longer need to support Python 2.3. From guido at python.org Tue Aug 3 16:04:38 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:04:42 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 01:36:19 EDT." <410F2453.7000506@divmod.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> Message-ID: <200408031404.i73E4cS30629@guido.python.org> > Here's a brief test for a syntax-change-less implementation of this > feature, not as complete as test_decorators, but a good start, I believe: [fast forward to syntax example] > decorate(staticmethod) > def bar(x): > print x > > decorate(classmethod) > def baz(cls, y): > print cls, y I'm speechless. If the ambiguous [classmethod] def foo(x): ... is rejected because it doesn't look like it does something to foo, how come there's suddenly a crop of solutions that have the same problem being proposed? What you write looks like a call to the function decorate(), followed by a function method definition. The "action-at-a-distance" that is presumed by the decorate() call is difficult to explain and a precedent for other worse hacks. Its only point in favor seems to be that it doesn't use '@'. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 16:13:30 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:13:40 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: Your message of "Tue, 03 Aug 2004 09:54:01 BST." <20040803085401.GA2798@vicky.ecs.soton.ac.uk> References: <20040803085401.GA2798@vicky.ecs.soton.ac.uk> Message-ID: <200408031413.i73EDUu30665@guido.python.org> > The SF patch http://www.python.org/sf/980695 about making repeated string > concatenations efficient has been reviewed and is acceptable on technical > grounds. This is about avoiding the quadratic behavior of > > s = '' > for x in y: > s += some_string(x) FWIW, I didn't approve it, Raymond did, IMO prematurely. :-( In the past similar schemes based on refcount were rejected because we couldn't be sure that there wasn't C code with an uncounted reference. Is that ruled out in this scheme? Also, I believe that even Java doesn't optimize this case in general -- only the much simpler case where you concatenate a number of strings in a single expression. > This leaves open the policy questions: > > * first, is that an implementation detail or a published feature? > The question is important because the difference in performance is > enormous -- we are not talking about 2x or even 10x faster but > roughly Nx faster where N is the size of the input data set. Like tail recursion, it will change the way people write code. I am going to make up a new rule here, and state that implementation features that affect not just the running speed but the O() rate for certain algorithms should be considered language features, because any implementation will be required to implement them in order to ensure code portability. > * if it is a published feature, what about Jython? > > * The patch would encourage a coding style that gives program that > essentially don't scale with Jython -- nor, for that matter, with > 2.3 or older -- and worse, the programs would *appear* to work on > Jython or 2.3 when tested with small or medium-sized data sets, > but just appear to hang when run on larger data sets. Obviously, > this is problem that has always been here, but if we fix it in 2.4 > we can be sure that people will develop and test with 2.4, and > less thoroughly on 2.3, and when they deploy on 2.3 platforms it > will unexpectedly not scale. > > * discussed on SF too is whether we should remove the 'a=a+b' > acceleration from the patch, keeping only 'a+=b'; see the SF > tracker. > > This seems overkill, but should the acceleration be there but > disabled by default? > > from __future__ import string_concatenate? Why do people want this so badly? Please leave well enough alone. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 16:18:15 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:18:20 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: Your message of "Tue, 03 Aug 2004 03:24:07 PDT." References: Message-ID: <200408031418.i73EIFR30699@guido.python.org> > Log Message: > Don't intern the filename of a file being compiled. > Hard to believe it ever helped anything, and it hurts finding reference > leaks. The intention was to introduce sharing of the filename object between code objects compiled from the same file (remember that every method is a separate code object). But I believe the sharing won't happen when the code is loaded from a bytecoded file instead, so it is indeed wasted efficiency. Good catch. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 16:22:25 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:22:47 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 11:33:13 BST." <2mzn5ca4uu.fsf@starship.python.net> References: <410DDFE3.7010006@interlink.com.au> <410E93AC.9050807@zope.com> <2mzn5ca4uu.fsf@starship.python.net> Message-ID: <200408031422.i73EMPX30737@guido.python.org> > Jim Fulton writes: > > In the mean time, can anyone give a pointer to where this was > > proposed or discussed? There are two slides in my EuroPython keynote (http://www.python.org/doc/essays/ppt/euro2004/euro2004.pdf) (pages 10 and 11) that mention it. It was discussion during and after this keynote that made me change my mind -- first I accepted that [deco] before def was unacceptable, then I found that @deco was not universally hated (Fredrik Lundh even liked it). --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Aug 3 04:24:03 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 3 16:25:41 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: <1091536409.410f8619be6f0@mcherm.com> Message-ID: <003101c47900$f23161e0$df30cb97@oemcomputer> > > * discussed on SF too is whether we should remove the 'a=a+b' > > acceleration > > from the patch, keeping only 'a+=b'; see the SF tracker. > Hmm... I couldn't think of any reason to limit the optimization > to += until I actually went and read the comments in the SF > tracker. What I took away from that discussion was that it's > possible to optimize "a=a+b", but NOT possible to optimize > "a=a+b+c". This is a subtle distinction that is harder to > explain to people than simply saying "it only works with +=, not > with +". > > That's a fairly convincing point, so I guess I'm on the fence > on this one. I'm not. Skipping a=a+b breaks symmetry with a+=b. More importantly, skipping a=a+b misses most of the potential benefits (see sre_parse.py for an example). PyBench, ParrotBench, and my other benchmarks all show gains when a=a+b is done inplace. The explanation is not hard. The CPython implementation can concatenate inplace two term expressions of the form a=a+b or a+=b. Expressions with more terms are not eligible for inplace concatenation. Raymond From martin at v.loewis.de Tue Aug 3 16:25:55 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 3 16:25:58 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: <200408031418.i73EIFR30699@guido.python.org> References: <200408031418.i73EIFR30699@guido.python.org> Message-ID: <410FA073.5070105@v.loewis.de> Guido van Rossum wrote: > The intention was to introduce sharing of the filename object between > code objects compiled from the same file (remember that every method > is a separate code object). > > But I believe the sharing won't happen when the code is loaded from a > bytecoded file instead, so it is indeed wasted efficiency. In 2.4, it would: strings that are interned when being marshalled will get interned on unmarshalling. Regards, Martin From guido at python.org Tue Aug 3 16:27:49 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:27:54 2004 Subject: [Python-Dev] PEP 324 (process module) In-Reply-To: Your message of "Tue, 03 Aug 2004 06:13:59 PDT." References: Message-ID: <200408031427.i73ERn330790@guido.python.org> I believe ActiveState's Trent Mick has released something along these lines. Does your proposed API match theirs? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 16:32:32 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:32:39 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 10:04:27 EDT." <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> Message-ID: <200408031432.i73EWWV30829@guido.python.org> > In practical terms, all this means is that I'll just use the hacked > syntax until I no longer need to support Python 2.3. I don't understand why you prefer your hack over the established way to do decorators pre-2.4, which is def foo(): ... foo = staticmethod(foo) This works across releases (including IronPython), doesn't require any magic, is documented, etc. So if your main constraint is that it be implementable pre-2.4, you already have a solution. Isn't that much better than spending effort on hacks based on sys.settrace (which are surely going to produce bafflement from users who aren't familiar with that implementation hack)? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 16:34:57 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:35:07 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: Your message of "Tue, 03 Aug 2004 16:25:55 +0200." <410FA073.5070105@v.loewis.de> References: <200408031418.i73EIFR30699@guido.python.org> <410FA073.5070105@v.loewis.de> Message-ID: <200408031434.i73EYvv30846@guido.python.org> > > The intention was to introduce sharing of the filename object between > > code objects compiled from the same file (remember that every method > > is a separate code object). > > > > But I believe the sharing won't happen when the code is loaded from a > > bytecoded file instead, so it is indeed wasted efficiency. > > In 2.4, it would: strings that are interned when being marshalled will > get interned on unmarshalling. Aha! So maybe we should reconsider whether mwh's removal of the filename interning in the compiler should be reverted. --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Aug 3 16:41:15 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 3 16:37:13 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031404.i73E4cS30629@guido.python.org> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> Message-ID: <5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com> At 07:04 AM 8/3/04 -0700, Guido van Rossum wrote: > > Here's a brief test for a syntax-change-less implementation of this > > feature, not as complete as test_decorators, but a good start, I believe: > >[fast forward to syntax example] > > > decorate(staticmethod) > > def bar(x): > > print x > > > > decorate(classmethod) > > def baz(cls, y): > > print cls, y > >I'm speechless. If the ambiguous > > [classmethod] > def foo(x): > ... > >is rejected because it doesn't look like it does something to foo, how >come there's suddenly a crop of solutions that have the same problem >being proposed? Because '[foo]' is absolutely a list with no side effects in today's Python. But '[foo()]' clearly *can* have some side effect, so if you're reading the code you have to look up 'foo' to understand what's happening there. This isn't at all sudden, btw: David Abrahams made the first 'decorate' proposal in June, and suggested the idea of using the debugger hook to implement it. I thought that one of the best parts of David Abrahams' idea was that requiring a function call made the [] syntax less ambiguous. > What you write looks like a call to the function >decorate(), followed by a function method definition. The >"action-at-a-distance" that is presumed by the decorate() call is >difficult to explain and a precedent for other worse hacks. I agree, which is why I wrap my decorators in [], even though it has no effect on the actual operation. It's pure "eye candy" sprinkled with syntactic sugar. :) > Its only >point in favor seems to be that it doesn't use '@'. Well, the other point is that it allows you to use C# syntax. Note that if it were spelled: [classmethod()] def foo(x): ... this would also resolve the ambiguity (IMO). From guido at python.org Tue Aug 3 16:40:05 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:40:19 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 00:17:43 EDT." <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <200408031440.i73Ee6V30875@guido.python.org> > The only real necessary restriction on the @ operator is that its > argument be callable and take a single argument. Many expressions > could return a callable object. Why not let them? Is it really worth > having a special case just to SyntaxError expressions that sometimes > won't result in an appropriate callable? > > Things someone might want to do, ordered roughly from most reasonable > to least reasonable ;) > @foo().bar() > @foo or bar > @mydecorators['foo'] > @lambda f: foo(f) or bar(f) > > Why disallow these forms? It seems quite difficult, especially, to > explain why the first one does not, or should not, work. I have a gut feeling about this one. I'm not sure where it comes from, but I have it. It may be that I want the compiler to be able to recognize certain decorators. So while it would be quite easy to change the syntax to @test in the future, I'd like to stick with the more restricted form unless a real use case is presented where allowing @test would increase readability. (@foo().bar() doesn't count because I don't expect you'll ever need that). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 16:50:00 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 16:50:06 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 10:41:15 EDT." <5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> <5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com> Message-ID: <200408031450.i73Eo0e30956@guido.python.org> > Because '[foo]' is absolutely a list with no side effects in today's > Python. But '[foo()]' clearly *can* have some side effect, so if > you're reading the code you have to look up 'foo' to understand > what's happening there. Too subtle for the folks who argued against [foo] on grounds of ambiguity I think. > This isn't at all sudden, btw: David Abrahams made the first > 'decorate' proposal in June, and suggested the idea of using the > debugger hook to implement it. I thought that one of the best parts > of David Abrahams' idea was that requiring a function call made the > [] syntax less ambiguous. And I think it is one of the worst. sys.settrace is an implementation feature not a language feature. > Well, the other point is that it allows you to use C# syntax. Note > that if it were spelled: > > [classmethod()] > def foo(x): > ... > > this would also resolve the ambiguity (IMO). Not IMO, that would just add more mystery (and would require yet another hack in classmethod to allow it to be called without args). --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Aug 3 04:48:31 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 3 16:50:08 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: <200408031413.i73EDUu30665@guido.python.org> Message-ID: <003201c47904$5cd353c0$df30cb97@oemcomputer> > > The SF patch http://www.python.org/sf/980695 about making repeated > string > > concatenations efficient has been reviewed and is acceptable on > technical > > grounds. This is about avoiding the quadratic behavior of > > > > s = '' > > for x in y: > > s += some_string(x) > > FWIW, I didn't approve it, Raymond did, IMO prematurely. :-( > > In the past similar schemes based on refcount were rejected because we > couldn't be sure that there wasn't C code with an uncounted > reference. Is that ruled out in this scheme? Yes it is. Unlike the refcount check proposed for PyList_Sort, Armin's patch only applies to the eval loop and cannot be called directly. I spent a *long* time reviewing and testing this patch. The code is clean. The concept works. It delivers significant, measurable benefits. The standard library itself has existing use cases. Every benchmark I tried showed results. The patch was not approved prematurely! On the off chance that Armin, Michael, and I missed a case, then including the patch in the second alpha is the best way to find it. > I am > going to make up a new rule here, and state that implementation > features that affect not just the running speed but the O() rate for > certain algorithms should be considered language features, because > any implementation will be required to implement them in order to > ensure code portability. Even list.sort() does not guarantee specific O() rates. Currently, the fastest way to write a merge(a,b) is sort(a+b). That relies on the current implementation recognizing that a and b are already sorted and doing a linear time concatenation. > Why do people want this so badly? Because the a=a+b form occurs everywhere, not just in newbie code. It is in the standard library, it shows up naturally in SAX, and it is often the most clear way to express an idea. Raymond From tim.peters at gmail.com Tue Aug 3 16:53:12 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 3 16:53:15 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects listobject.c, 2.218, 2.219 In-Reply-To: References: <002b01c478b3$cc9c7020$eb23c797@oemcomputer> <20040803091233.GB2798@vicky.ecs.soton.ac.uk> <410F5F68.9030704@acm.org> Message-ID: <1f7befae04080307536a8e3c5c@mail.gmail.com> [Sjoerd Mullender] > This is indeed perverted. Another option is to add a statement > (void) status; > somewhere to the function and to remove the status++ from the return. > This seems to shut up all compilers I've dealt with. If that's enough to shut compilers up, +1 from me. MSVC 7.1 doesn't whine even with no tricks. I do want to keep the two optimizations in list.pop(), which consist of not test+branch'ing the return values of two calls that have been proved safe. But things change over time, and in a debug build I do want to assert that they are in fact safe. I originally did this with a pile of #ifdef Py_DEBUG blocks, but (as the original checkin comment said) that was even uglier than the comma-expression hack. From barry at python.org Tue Aug 3 16:53:36 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 3 16:53:32 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031432.i73EWWV30829@guido.python.org> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> Message-ID: <1091544810.8416.83.camel@localhost> On Tue, 2004-08-03 at 10:32, Guido van Rossum wrote: > I don't understand why you prefer your hack over the established way > to do decorators pre-2.4, which is > > def foo(): ... > foo = staticmethod(foo) > > This works across releases (including IronPython), doesn't require any > magic, is documented, etc. So if your main constraint is that it be > implementable pre-2.4, you already have a solution. Isn't that much > better than spending effort on hacks based on sys.settrace (which are > surely going to produce bafflement from users who aren't familiar with > that implementation hack)? +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/72c063c2/attachment.pgp From marktrussell at btopenworld.com Tue Aug 3 13:06:54 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Tue Aug 3 16:56:37 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <1091531214.1502.20.camel@localhost> On Tue, 2004-08-03 at 05:17, James Y Knight wrote: > Things someone might want to do, ordered roughly from most reasonable > to least reasonable ;) > @foo().bar() > @foo or bar > @mydecorators['foo'] > @lambda f: foo(f) or bar(f) > > Why disallow these forms? It seems quite difficult, especially, to > explain why the first one does not, or should not, work. I don't think any of these seem that likely in practice, and if you do want them, it is not hard to say: # Hopefully some comment here explaining the reason why you're # doing this. And a more meaningful name than foo_or_bar! foo_or_bar = foo or bar @foo_or_bar def func(): pass I would say leave it like this for now. It is easy to relax the restriction later if there's a demand. Mark From pje at telecommunity.com Tue Aug 3 17:00:41 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 3 16:56:40 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031432.i73EWWV30829@guido.python.org> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com> At 07:32 AM 8/3/04 -0700, Guido van Rossum wrote: > > In practical terms, all this means is that I'll just use the hacked > > syntax until I no longer need to support Python 2.3. > >I don't understand why you prefer your hack over the established way >to do decorators pre-2.4, which is > > def foo(): ... > foo = staticmethod(foo) For the same reason that PEP 318 was written in the first place: to improve readability. >This works across releases (including IronPython), doesn't require any >magic, is documented, etc. True; and most of my existing decorator-using code works this way, and I don't plan on rushing out to change any of it any time soon. > Isn't that much >better than spending effort on hacks based on sys.settrace (which are >surely going to produce bafflement from users who aren't familiar with >that implementation hack)? Well, up until recently, I thought the [] syntax was still a shoo-in and that the settrace hack was just going to be a way to write backwards-compatible code. In the meantime, I've implemented a predicate-dispatch generic function system using the settrace hack for PyProtocols 1.0, and don't want to have to put off using it until 2.4. (FWIW, I still support 2.2.2, so "until 2.4" means maybe 1-2 years for me.) It looks like this: [when("isinstance(x,Foo) and y>23")] def do_something(x,y): # code for when x is a Foo and y>23 [when("isinstance(x,Bar) and y<99")] def do_something(x,y): # code for when x is a Bar and y<99 Rendered without any decorator syntax, it would have to be something like: def do_something_x_Foo_y_gt_23(x,y): # code for when x is a Foo and y>23 do_something.add("isinstance(x,Foo) and y>23", do_something_x_Foo_y_gt_23) def do_something_x_Bar_y_lt_99(x,y): # code for when x is a Bar and y>99 do_something.add("isinstance(x,Bar) and y>23", do_something_x_Bar_y_lt_99) ...which is a significant loss in readability. Now, it's true that this will just become '@when("whatever")' in 2.4, and I'm *fine* with that, just disappointed that the syntax won't be forward compatible. Indeed, what I'd rather focus on is making sure that the *semantics* will be forward compatible. Which I think they will be. What I'll probably do is make 'when()' and other decorators return a callable that deactivates the trace hack as a side effect of performing the decoration. That way, [when()] and @when() can coexist in a code base during the migration period. From jim at zope.com Tue Aug 3 16:58:53 2004 From: jim at zope.com (Jim Fulton) Date: Tue Aug 3 16:58:56 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031432.i73EWWV30829@guido.python.org> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> Message-ID: <410FA82D.8080502@zope.com> Guido van Rossum wrote: >>In practical terms, all this means is that I'll just use the hacked >>syntax until I no longer need to support Python 2.3. > > > I don't understand why you prefer your hack over the established way > to do decorators pre-2.4, which is > > def foo(): ... > foo = staticmethod(foo) > > This works across releases (including IronPython), doesn't require any > magic, is documented, etc. So if your main constraint is that it be > implementable pre-2.4, you already have a solution. Isn't that much > better than spending effort on hacks based on sys.settrace (which are > surely going to produce bafflement from users who aren't familiar with > that implementation hack)? I think this is a great point. It makes me wonder if: @staticmethod def foo(): ... if sufficiently better than: def foo(): ... foo = staticmethod(foo) to justify the language change. FWIW, It isn't to me. The new syntax is yet another rule that people have to know to understand Python code they read. That's OK if it produces enough value to justify the burden. I question whether that's the case here. Perhsps the difficulty in pickling an acceptable syntax should be taken as a warning sign that there's a problem with the feature. I wonder if perhaps the feature is too powerful. In particular, the need to support passing arguments to the descriptors seemes to add a lot of syntactic burden. I think that the most compeling common applications of decorators don't need to have decorator expressions. For example, for the common cases that this is trying to solve, I'd be happy to be able to say: def staticmethod foo(): .... and allow only names as decorators. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From arigo at tunes.org Tue Aug 3 16:58:56 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Aug 3 17:03:41 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: <003101c47900$f23161e0$df30cb97@oemcomputer> References: <1091536409.410f8619be6f0@mcherm.com> <003101c47900$f23161e0$df30cb97@oemcomputer> Message-ID: <20040803145856.GA7070@vicky.ecs.soton.ac.uk> Hello Raymond, On Mon, Aug 02, 2004 at 10:24:03PM -0400, Raymond Hettinger wrote: > The explanation is not hard. The CPython implementation can concatenate > inplace two term expressions of the form a=a+b or a+=b. Expressions > with more terms are not eligible for inplace concatenation. No: a+=b+c is eligible. That's my problem. Armin From guido at python.org Tue Aug 3 17:23:16 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 17:23:20 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 11:00:41 EDT." <5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com> Message-ID: <200408031523.i73FNGq31069@guido.python.org> > Now, it's true that this will just become '@when("whatever")' in > 2.4, and I'm *fine* with that, just disappointed that the syntax > won't be forward compatible. Implementability in 2.2 and 2.3 was never on the list of requirements for decorators. You're adding this constraint yourself. I know it was brought up before but I never agreed with that constraint (and the previous community favorite, [...]-after-args, wasn't compatible either). I presume your framework is already full of hacks, which is fine by me, but I don't want to use it as a deciding factor for the decorator syntax. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 17:24:46 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 17:24:52 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 10:58:53 EDT." <410FA82D.8080502@zope.com> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> <410FA82D.8080502@zope.com> Message-ID: <200408031524.i73FOkM31082@guido.python.org> > I think this is a great point. It makes me wonder if: > > @staticmethod > def foo(): > ... > > if sufficiently better than: > > def foo(): > ... > foo = staticmethod(foo) > > to justify the language change. FWIW, It isn't to me. Sigh. This discussion is going around in pointless circles; we're *months* past that point. You're wasting your time (and mine). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 3 17:27:26 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 17:27:31 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: Your message of "Mon, 02 Aug 2004 22:48:31 EDT." <003201c47904$5cd353c0$df30cb97@oemcomputer> References: <003201c47904$5cd353c0$df30cb97@oemcomputer> Message-ID: <200408031527.i73FRQb31100@guido.python.org> > I spent a *long* time reviewing and testing this patch. The code is > clean. The concept works. It delivers significant, measurable > benefits. The standard library itself has existing use cases. > Every benchmark I tried showed results. The patch was not approved > prematurely! Yes it was. You know I am interested in this kind of decision (otherwise you'd have just checked it in) so you should have left the honor to me. > > I am going to make up a new rule here, and state that > > implementation features that affect not just the running speed but > > the O() rate for certain algorithms should be considered language > > features, because any implementation will be required to implement > > them in order to ensure code portability. > > Even list.sort() does not guarantee specific O() rates. Currently, > the fastest way to write a merge(a,b) is sort(a+b). That relies on > the current implementation recognizing that a and b are already > sorted and doing a linear time concatenation. That's a much more obscure case. The string concatenation hack will affect a significant fraction of all code. > > Why do people want this so badly? > > Because the a=a+b form occurs everywhere, not just in newbie code. > It is in the standard library, it shows up naturally in SAX, and it > is often the most clear way to express an idea. And in 99% of those cases nobody cares about performance. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Aug 3 17:37:16 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 3 17:37:17 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: <200408031434.i73EYvv30846@guido.python.org> References: <200408031418.i73EIFR30699@guido.python.org> <410FA073.5070105@v.loewis.de> <200408031434.i73EYvv30846@guido.python.org> Message-ID: <410FB12C.4010105@v.loewis.de> Guido van Rossum wrote: > Aha! So maybe we should reconsider whether mwh's removal of the > filename interning in the compiler should be reverted. I just looked at the sizes of the four largest .pyc files wrt. to this change (ie. the one generated before 2.312, and the one generated after): before after pydoc 84711 91263 cookielib 57353 57353 pickletools 55862 57038 tarfile 54074 58974 So the patch does cause a size increase (and likely also a slowdown because of the extra memory allocations at startup). Whether this is relevant or not, I don't know. I think I would prefer if a different mechanism was found to account for the change in references during an import, e.g. by taking the number of interned strings before and after the import operation. Regards, Martin From dave at boost-consulting.com Tue Aug 3 17:44:51 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue Aug 3 17:45:21 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> <410FA82D.8080502@zope.com> Message-ID: Jim Fulton writes: > to justify the language change. FWIW, It isn't to me. The new > syntax is yet another rule that people have to know to understand > Python code they read. That's OK if it produces enough value to > justify the burden. I question whether that's the case here. > > Perhsps the difficulty in pickling an acceptable syntax should be > taken as a warning sign that there's a problem with the feature. I just want to be clear that I know implementing my proposal involves an ugly hack, and I'm not fond of it. Nonetheless, I think it's important. I made the proposal in response to exactly the same instinct that Jim is reflecting here. If people really need a prefix syntax for decorators it would be best to at least _start_ with an implementation that doesn't involve core language changes, because the whole area looks quite perilous from a language design point of view. I'm saying without any desire to change Guido's mind about '@', but just to explain the reasons that I brought up the idea in the first place. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From jim at zope.com Tue Aug 3 17:51:12 2004 From: jim at zope.com (Jim Fulton) Date: Tue Aug 3 17:51:15 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031524.i73FOkM31082@guido.python.org> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> <410FA82D.8080502@zope.com> <200408031524.i73FOkM31082@guido.python.org> Message-ID: <410FB470.7080709@zope.com> Guido van Rossum wrote: >>I think this is a great point. It makes me wonder if: >> >> @staticmethod >> def foo(): >> ... >> >>if sufficiently better than: >> >> def foo(): >> ... >> foo = staticmethod(foo) >> >>to justify the language change. FWIW, It isn't to me. > > > Sigh. This discussion is going around in pointless circles; we're > *months* past that point. You're wasting your time (and mine). Sorry. I think you'd be wise to consider the months without consensus as evidence of a problem. Don't bother wasting time with a reply. ;) Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From mwh at python.net Tue Aug 3 18:01:18 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 3 18:01:19 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: <410FB12C.4010105@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Tue, 03 Aug 2004 17:37:16 +0200") References: <200408031418.i73EIFR30699@guido.python.org> <410FA073.5070105@v.loewis.de> <200408031434.i73EYvv30846@guido.python.org> <410FB12C.4010105@v.loewis.de> Message-ID: <2misc09po1.fsf@starship.python.net> "Martin v. L?wis" writes: > Guido van Rossum wrote: >> Aha! So maybe we should reconsider whether mwh's removal of the >> filename interning in the compiler should be reverted. > > I just looked at the sizes of the four largest .pyc files wrt. to this > change (ie. the one generated before 2.312, and the one generated > after): > > before after > pydoc 84711 91263 > cookielib 57353 57353 > pickletools 55862 57038 > tarfile 54074 58974 > > So the patch does cause a size increase (and likely also a slowdown > because of the extra memory allocations at startup). Whether this is > relevant or not, I don't know. Me neither. Are you sure you regenerated cookielib? What would it cost to check if all strings could be stored via TYPE_STRINGREF? (Almost nothing in code terms...) > I think I would prefer if a different mechanism was found to account > for the change in references during an import, e.g. by taking the > number of interned strings before and after the import operation. Well, this isn't about import, it's more about execfile. I had entirely failed to twig that interning the filename might save space in the .pyc, so didn't see any downsides to this particular workaround. I'd be happy with any other (I mentioned not doing refcounting on dictobject.c's dummy; maybe I should actually see if that's possible :-). Cheers, mwh -- The gripping hand is really that there are morons everywhere, it's just that the Americon morons are funnier than average. -- Pim van Riezen, alt.sysadmin.recovery From pinard at iro.umontreal.ca Tue Aug 3 16:53:45 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Tue Aug 3 18:03:17 2004 Subject: [Python-Dev] Python in Unicode context Message-ID: <20040803145345.GA6250@titan.progiciels-bpi.ca> Hello, people. I'm switching from ISO-8859-1 to UTF-8 in my locale, knowing it may take a while before everything gets fully adapted. Of course, I am prepared to do whatever it means. On my side at least, the perception of a meaning is an evolving process. :-) So, my goal here is to share some of difficulties I see with the current setup of Python in Unicode context, under the hypothesis that Python should ideally be designed to alleviate the pain of migration. I hope this is not out of context on the Python development list. Converting a Python source file from ISO-8859-1 to UTF-8, back and forth at the charset level, is a snap within Vim, and I would like if it was (almost) a snap in the Python code as well. There is some amount of trickery that I could put in to achieve this, but too much trickery does not fit well in usual Python elegance. As Martin once put it, the ultimate goal is to convert data to Unicode as early as possible in a Python program, and back to the locale as late as possible. While it's very OK with me, we should not loose sight that people might adopt different approaches. One thing is that a Python module should have some way to know the encoding used in its source file, maybe some kind of `module.__coding__' next to `module.__file__', saving the coding effectively used while compilation was going on. When a Python module is compiled, per PEP 0263 as I understand it, strings are logically converted to UTF-8 before scanning, and produced str-strings (but not unicode-strings), converted back to the original file coding. When later, at runtime, the string has to be converted back to Unicode, it would help if the programmer did not have to hardwire the encoding in the program, and edit more than the `coding:' cookie at the beginning if s/he ever switches file charset. That same `module.__coding__' could also be used for other things, like for example, to decide at run-time whether codecs streawriters should be used or not. Another solution would of course be to edit all strings, or at least those containing non-ASCII characters, to prepend a `u' and turn them into Unicode strings. This is what I intend to do in practice. However, all this editing is cumbersome, especially until it is definitive. I wonder if some other cookie, next to the `coding:' cookie, could not be used to declare that all strings _in this module only_ should be interpreted as Unicode by default, but without the need of resorting to `u' prefix all over. That would be weaker than the `-U' switch on a Python call, but likely much more convenient as well. As a corollary, maybe that some `s' prefix could force `str' type in a Unicodized module. Another way of saying it would be that an unadorned string would have `s' or `u' implied, depending if the Unicode cookie is missing or given at the start of a module. I have the intuition, still unverified, but to be confirmed over time and maybe discussions, that the above would alleviate transition to Unicode, back and forth. P.S. - Should I say and confess, one thing I do not like much about Unicode is how proponents often perceive it, like a religion, and all the fanatism going with it. Unicode should be seen and implemented as a choice, more than a life commitment :-). Right now, my feeling is that Python asks a bit too much of a programmer, in terms of commitment, if we only consider the editing work required on sources to use it, or not. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From exarkun at divmod.com Tue Aug 3 18:06:12 2004 From: exarkun at divmod.com (Jp Calderone) Date: Tue Aug 3 18:06:16 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031404.i73E4cS30629@guido.python.org> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> <200408031404.i73E4cS30629@guido.python.org> Message-ID: <410FB7F4.2070309@divmod.com> Guido van Rossum wrote: > [snip] > > > I'm speechless. If the ambiguous > > [classmethod] > def foo(x): > ... > > is rejected because it doesn't look like it does something to foo, how > come there's suddenly a crop of solutions that have the same problem > being proposed? What you write looks like a call to the function > decorate(), followed by a function method definition. The > "action-at-a-distance" that is presumed by the decorate() call is > difficult to explain and a precedent for other worse hacks. Its only > point in favor seems to be that it doesn't use '@'. In my view, the strongest point in favor of a solution that involves calling functions rather than changing syntax is that the functions involved can be placed in the standard library rather than the interpreter. I believe a widely held view is that features can be supported by the stdlib do not merit language changes? Moreover, I have the impression that many people are clamoring for this feature, no matter how it ends up looking, because they simply *must have it*. Well, if they must have it, why wait for 2.4, when clearly they can have it right now? Jp From jhylton at gmail.com Tue Aug 3 18:09:40 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Tue Aug 3 18:09:44 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031524.i73FOkM31082@guido.python.org> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> <410FA82D.8080502@zope.com> <200408031524.i73FOkM31082@guido.python.org> Message-ID: On Tue, 03 Aug 2004 08:24:46 -0700, Guido van Rossum wrote: > Sigh. This discussion is going around in pointless circles; we're > *months* past that point. You're wasting your time (and mine). Is there a PEP that summarizes the current state of the decorators design? One of the goals of PEPs was to prevent design discussions from going in circles. Another goal was to keep core developers abreast of changes without having to read every message on python-dev. As you mentioned in an earlier email, I think the work on 2.4 would have been smoother if we had used the PEP process more effectively. We never had a decent PEP for the decorator syntax discussion. PEP 318 describes a particular design choice, but it's weak on rationale. There's no PEP that summarizes the various syntax options and the reason for the eventual choice. Facundo's PEP for decimal arithmetic (PEP 327) looks like an example of the right thing. I haven't followed the work closely, but it looks like the PEP describes the current state of the design and the rationale for it. The generator PEP is in better shape than it used to be. Jeremy From tim.peters at gmail.com Tue Aug 3 18:15:07 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 3 18:15:09 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410FA82D.8080502@zope.com> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> <410FA82D.8080502@zope.com> Message-ID: <1f7befae04080309153c7ac193@mail.gmail.com> I want to jump into this debate, but have stayed out of it so far and just don't have time for it. But I can't let this one pass! [Jim Fulton] > ... > Perhsps the difficulty in pickling an acceptable syntax should be taken as > a warning sign that there's a problem with the feature. There is, in fact, no such difficulty with any of these syntaxes: I've tried pickling all of them, under both pickle.py and cPickle, and they both pickle and unpickle fine. I think this has to do with that they're really all just strings . From guido at python.org Tue Aug 3 18:23:44 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 18:23:58 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: Your message of "Tue, 03 Aug 2004 17:37:16 +0200." <410FB12C.4010105@v.loewis.de> References: <200408031418.i73EIFR30699@guido.python.org> <410FA073.5070105@v.loewis.de> <200408031434.i73EYvv30846@guido.python.org> <410FB12C.4010105@v.loewis.de> Message-ID: <200408031623.i73GNiL31284@guido.python.org> > > Aha! So maybe we should reconsider whether mwh's removal of the > > filename interning in the compiler should be reverted. > > I just looked at the sizes of the four largest .pyc files wrt. to this > change (ie. the one generated before 2.312, and the one generated > after): > > before after > pydoc 84711 91263 > cookielib 57353 57353 > pickletools 55862 57038 > tarfile 54074 58974 Are you sure the cookielib numbers are correct? They show no difference, which would only make sense if there was only a single code object. > So the patch does cause a size increase (and likely also a slowdown > because of the extra memory allocations at startup). Whether this is > relevant or not, I don't know. Based on this data I'd like to see the change reverted; the motivation for the change was debuggability of leaks, not a real problem with the changed code. > I think I would prefer if a different mechanism was found to account > for the change in references during an import, e.g. by taking the > number of interned strings before and after the import operation. --Guido van Rossum (home page: http://www.python.org/~guido/) From astrand at lysator.liu.se Tue Aug 3 17:22:35 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Tue Aug 3 18:25:59 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: <200408031427.i73ERn330790@guido.python.org> References: <200408031427.i73ERn330790@guido.python.org> Message-ID: > I believe ActiveState's Trent Mick has released something along these > lines. Does your proposed API match theirs? No, not at all. There are some issues with this module, both regards the API and the implementation, which I'm not happy with. I tried to establish a discussion with Trent about this, but if I remember correctly, I never recieved a response. Here's some things which I'm not happy with: * There are three different "factory" classes: Process, ProcessOpen, ProcessProxy. I have only one, covering all cases. I think three different classes are confusing to users. Even I have trouble understanding the difference between them. * The code size is very large. The code is complex. * Trent's module always executes things through the shell, and deals with every ugly cornercase that comes from this. * The modules uses destructors, which I'm usually avoiding. So, the chances of getting our modules API-compatible are very small. /Peter ?strand From mwh at python.net Tue Aug 3 18:29:17 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 3 18:29:19 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_threadsignals.py, 1.2, 1.3 In-Reply-To: (fdrake@users.sourceforge.net's message of "Tue, 03 Aug 2004 09:14:16 -0700") References: Message-ID: <2mekmo9ode.fsf@starship.python.net> fdrake@users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Lib/test > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18587 > > Modified Files: > test_threadsignals.py > Log Message: > add missing import! Oops, sorry! Cheers, mwh -- But maybe I've just programmed in enough different languages to assume that they are, in fact, different. -- Tony J Ibbs explains why Python isn't Java on comp.lang.python From pje at telecommunity.com Tue Aug 3 18:34:16 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 3 18:30:19 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031523.i73FNGq31069@guido.python.org> References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040803113843.03190e60@mail.telecommunity.com> At 08:23 AM 8/3/04 -0700, Guido van Rossum wrote: > > Now, it's true that this will just become '@when("whatever")' in > > 2.4, and I'm *fine* with that, just disappointed that the syntax > > won't be forward compatible. > >Implementability in 2.2 and 2.3 was never on the list of requirements >for decorators. You're adding this constraint yourself. I know it >was brought up before but I never agreed with that constraint (and the >previous community favorite, [...]-after-args, wasn't compatible >either). All understood and agreed. I was only trying to answer the question that you asked; I was emphatically *not* trying to reopen the discussion. My "disappointment" here is on a par with being disappointed that a restaurant is out of my favorite dish: a trifling matter that doesn't deserve the time I've already spent on it, except that I hate being judged poorly for something that I didn't actually *do*. You seem to think that I'm saying "@" is a bad thing, or that "[]" is better for the language or community than "@". But I'm most emphatically *not* saying those things, and don't believe I ever have. >I presume your framework is already full of hacks, Well, I don't know if I'd call 4 hacks "full", especially since two of those are to implement decorators (class level and function level). The other two are to implement 'call-next-method' for generic functions, and to implement microthreads with generators. Out of nearly ninety thousand of lines of Python, I don't think that "full" is an accurate characterization here. > which is fine by >me, but I don't want to use it as a deciding factor for the decorator >syntax. I'm not asking you to, and I don't believe I ever have. I only replied to J.P. Calderone's suggestion of a less-functional hack (i.e. if you're going to hack, you might as well make it do the whole thing), and to *your* emails inquiring why I and other people thought a call-oriented syntax was better than bare "[]". With regard to the actual language change, I conceded the point when you said "let's move on" the other day. I understand your frustration with this issue, and I can also understand that you are probably feeling pretty second-guessed right now, but *I'm* not second-guessing you! So there's no call for the Spanish Inquisition here, and certainly not for personal insults because I answered questions that *you* asked. (The Python standard library actually has *six* modules or packages that use sys._getframe(), not including tests; shall we call it "full of hacks", too?) From guido at python.org Tue Aug 3 18:26:57 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 18:37:09 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 09:44:51 MDT." References: <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> <200408031432.i73EWWV30829@guido.python.org> <410FA82D.8080502@zope.com> Message-ID: <200408031626.i73GQvO31298@guido.python.org> > I made the proposal in response to exactly the same instinct that Jim > is reflecting here. If people really need a prefix syntax for > decorators it would be best to at least _start_ with an implementation > that doesn't involve core language changes, because the whole area > looks quite perilous from a language design point of view. We *had* an implementation without core language changes, but it was a postfix syntax. A prefix syntax should be designed with future usability in mind exclusively, unconstrained by "implementability without language changes". That's a much more stifling requirement than the requirement that existing code shouldn't be broken (which is stifling enough in itself but IMO unavoidable). --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer at stackless.com Tue Aug 3 18:51:12 2004 From: tismer at stackless.com (Christian Tismer) Date: Tue Aug 3 18:51:27 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae0408022027465a8789@mail.gmail.com> References: <410EF570.3010901@stackless.com> <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> <200408021840.i72Iejt28600@guido.python.org> <410EF570.3010901@stackless.com> <5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com> <1f7befae0408022027465a8789@mail.gmail.com> Message-ID: <410FC280.9060008@stackless.com> Tim Peters wrote: > [Phillip J. Eby] > >>It's worse than that... 'ihooks' and 'imputil', for example, both do: >> >> exec code in module.__dict__ >> >>so they'd need to be changed to support this fix as well. (Not to mention >>any third-party import hooks...) > > > Ouch. For that matter, they also avoid the "choke point" Sunday > night's patch relied on. So there are actually no choke points for > any part of the import process now. Maybe after each bytecode we > could make the eval loop look to see whether sys.modules had been > changed . :-( Life is so cruel... I agree it is a little harder, ahem :-) -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From martin at v.loewis.de Tue Aug 3 19:00:17 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 3 19:00:15 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: <200408031623.i73GNiL31284@guido.python.org> References: <200408031418.i73EIFR30699@guido.python.org> <410FA073.5070105@v.loewis.de> <200408031434.i73EYvv30846@guido.python.org> <410FB12C.4010105@v.loewis.de> <200408031623.i73GNiL31284@guido.python.org> Message-ID: <410FC4A1.7000409@v.loewis.de> Guido van Rossum wrote: > Are you sure the cookielib numbers are correct? They show no > difference, which would only make sense if there was only a single > code object. No. Turns out I did not have "old" bytecode for cookielib. Regards, Martin From guido at python.org Tue Aug 3 19:00:22 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 19:00:28 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: Your message of "Tue, 03 Aug 2004 12:06:12 EDT." <410FB7F4.2070309@divmod.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> <200408031404.i73E4cS30629@guido.python.org> <410FB7F4.2070309@divmod.com> Message-ID: <200408031700.i73H0MY31391@guido.python.org> > In my view, the strongest point in favor of a solution that involves > calling functions rather than changing syntax is that the functions > involved can be placed in the standard library rather than the > interpreter. > > I believe a widely held view is that features can be supported by > the stdlib do not merit language changes? No viewpoint ought to be held to the exclusion of all others. Using functions with "action-at-a-distance" through sys.settraceback may be okay for an obscure feature that can't be had any other way yet doesn't merit changes to the language, but that's not the situation for decorators. The widely held view here is that decorators need to be added as a syntactic feature to avoid the problems with the postfix notation used in 2.2 and 2.3. Decorators are slated to be an important new language feature and their design needs to be forward-looking, not constrained by what can be implemented in 2.3. One important feature of @deco (and of [deco] prefix as I proposed it) is that it is a compile-time feature. This has all sorts of advantages, since future compilers and tools will be able to know more about methods. Solutions based on sys.settrace fail badly here. > Moreover, I have the impression that many people are clamoring for > this feature, no matter how it ends up looking, because they simply > *must have it*. Well, if they must have it, why wait for 2.4, when > clearly they can have it right now? Sure, but that's not an argument against improving their situation in 2.4. If I had any hope that something better than @deco was possible but hasn't been discovered yet, I'd be holding off too. But I don't think that's the case. Every solution proposed has its own set of drawbacks, and the drawbacks aren't getting resolved by waiting. --Guido van Rossum (home page: http://www.python.org/~guido/) From mcherm at mcherm.com Tue Aug 3 19:07:45 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 3 19:07:09 2004 Subject: [Python-Dev] Re: PEP 324 (process module) Message-ID: <1091552865.410fc661b4abd@mcherm.com> Guido writes: > I believe ActiveState's Trent Mick has released something > along these lines. Does your proposed API match theirs? Peter replies: [...] > So, the chances of getting our modules API-compatible are very small. I'll note that my impression is that Peter's module is not aiming to be particularly innovative in terms of implementation, it's primary "feature" is having a decent API. And my impression is that it has been well received on c.l.py (for instance, I often see it recommended in response to newbie questions). He got a lot of input during the PEP process and tried to incorporate it. Last time I tried it there were still a few windows issues (docs say these have been resolved), but I thought the API was well-designed. And popen2 isn't. -- Michael Chermside From martin at v.loewis.de Tue Aug 3 19:12:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 3 19:12:43 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: <2misc09po1.fsf@starship.python.net> References: <200408031418.i73EIFR30699@guido.python.org> <410FA073.5070105@v.loewis.de> <200408031434.i73EYvv30846@guido.python.org> <410FB12C.4010105@v.loewis.de> <2misc09po1.fsf@starship.python.net> Message-ID: <410FC78C.4010609@v.loewis.de> Michael Hudson wrote: > Me neither. Are you sure you regenerated cookielib? Oops, no. Or, rather, yes - even the first number is already regenerated. > What would it cost to check if all strings could be stored via > TYPE_STRINGREF? (Almost nothing in code terms...) It would cause additional dictionary lookups on marshalling, which in turn would cause hash computation for all strings. Currently, as only interned strings are marshalled through TYPE_STRINGREF, we know that hash values are already computed for all of them, so we only pay a dict lookup per string (and perhaps an allocation if the string is new and the dict needs resizing). In addition, there is a slight semantical change, in that equal strings become shared on unmarshalling. I don't think this is relevant, though, since marshal never worried about shared strings in the first place. > I'd be happy with any other (I mentioned not doing > refcounting on dictobject.c's dummy; maybe I should actually see if > that's possible :-). Feel free to change the marshalling to try sharing all strings; it might be easier to revert the change to compile.c, though. Regards, Martin From mcherm at mcherm.com Tue Aug 3 19:22:42 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 3 19:22:06 2004 Subject: [Python-Dev] 2.4a2, and @decorators Message-ID: <1091553762.410fc9e2ef458@mcherm.com> James Knight writes: > However, it seems as if this restriction creates a new class of > expression for no gain. It is true enough that *most* python > expressions aren't useful to write after a @, but that's also > true of a > lot of other places an expression can be used (e.g. before a () of a > function call. A list comprehension can never result in a callable > object. An arithmetic operation usually won't.). > > The only real necessary restriction on the @ operator is that its > argument be callable and take a single argument. Many > expressions could > return a callable object. Why not let them? Is it really > worth having a > special case just to SyntaxError expressions that sometimes won't > result in an appropriate callable? > > Things someone might want to do, ordered roughly from most reasonable > to least reasonable ;) > @foo().bar() > @foo or bar > @mydecorators['foo'] > @lambda f: foo(f) or bar(f) > > Why disallow these forms? It seems quite difficult, especially, to > explain why the first one does not, or should not, work. Yeah... what he said. Seriously, a well-designed programming language has very strong orthogonality. A few simple concepts should suffice. "an expression" is an existing production which is well-known to users and well understood. I agree that using complex expressions in a decorator leads to difficult-to-read code, but I would STILL like it to be possible, just so I don't have to remember (or teach) decorators as a special case. -- Michael Chermside From exarkun at divmod.com Tue Aug 3 19:23:10 2004 From: exarkun at divmod.com (Jp Calderone) Date: Tue Aug 3 19:23:15 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031700.i73H0MY31391@guido.python.org> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> <200408031404.i73E4cS30629@guido.python.org> <410FB7F4.2070309@divmod.com> <200408031700.i73H0MY31391@guido.python.org> Message-ID: <410FC9FE.6060305@divmod.com> Guido van Rossum wrote: > > No viewpoint ought to be held to the exclusion of all others. Using > functions with "action-at-a-distance" through sys.settraceback may be > okay for an obscure feature that can't be had any other way yet > doesn't merit changes to the language, but that's not the situation > for decorators. The widely held view here is that decorators need to > be added as a syntactic feature to avoid the problems with the postfix > notation used in 2.2 and 2.3. Decorators are slated to be an > important new language feature and their design needs to be > forward-looking, not constrained by what can be implemented in 2.3. Just to be clear, the implementation I shared did not use sys.settrace(). It did rely on inspect.currentframe(), but so do a handful of other stdlib modules (as PJE pointed out). > > One important feature of @deco (and of [deco] prefix as I proposed it) > is that it is a compile-time feature. This has all sorts of > advantages, since future compilers and tools will be able to know more > about methods. Solutions based on sys.settrace fail badly here. > > >>Moreover, I have the impression that many people are clamoring for >>this feature, no matter how it ends up looking, because they simply >>*must have it*. Well, if they must have it, why wait for 2.4, when >>clearly they can have it right now? > > > Sure, but that's not an argument against improving their situation in > 2.4. If it is truly an improvement. Here we (and many others) disagree. I like Python(*) because it is simple and generally unsurprising. I think that '@deco' adds complexity, both for beginners and experienced programmers alike, so I do not see it as an improvement. I am content with "foo = deco(foo)" and various metaclass-based automatic decorations, and see no need for them to be moved into the core. I may be free to not use the '@deco' syntax in 2.4, but that will not prevent me from having to read it in the code of others, or from explaining it to other Python programmers. None of these are new points, so I don't expect a response to them. > > If I had any hope that something better than @deco was possible but > hasn't been discovered yet, I'd be holding off too. But I don't think > that's the case. Every solution proposed has its own set of > drawbacks, and the drawbacks aren't getting resolved by waiting. How many features that everyone hates can Python support? The Zen of Python might state that practicality beats purity, but if it continues to do so, time after time, at some point we're going to look up and wonder where that clean, readable, maintainable programming language we all loved has gone. Jp (*) And I really do like Python. A lot. Of the software I now write, at least 99.9% of it is in Python. The developers of this language _almost_ always show more good sense, intelligence, and skill than most other groups I can think of. You all deserve more praise than you're given. From martin at v.loewis.de Tue Aug 3 19:24:11 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 3 19:24:11 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <20040803145345.GA6250@titan.progiciels-bpi.ca> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> Message-ID: <410FCA3B.1020002@v.loewis.de> Fran?ois Pinard wrote: > One thing is that a Python module should have some way to know the > encoding used in its source file, maybe some kind of `module.__coding__' > next to `module.__file__', saving the coding effectively used while > compilation was going on. That would be possible to implement. Feel free to create a patch. > I wonder if some other cookie, next to the `coding:' > cookie, could not be used to declare that all strings _in this module > only_ should be interpreted as Unicode by default, but without the need > of resorting to `u' prefix all over. This could be a starting point of another syntax debate. For example, from __future__ import string_literals_are_unicode would be possible to implement. If PEP 244 would have been adapted, I would have proposed directive unicode_strings Other syntax forms would also be possible. Again, if you know a syntax which you like, propose a patch. Be prepared to also write a PEP defending that syntax. > P.S. - Should I say and confess, one thing I do not like much about > Unicode is how proponents often perceive it, like a religion, and all > the fanatism going with it. Unicode should be seen and implemented as > a choice, more than a life commitment :-). Right now, my feeling is that > Python asks a bit too much of a programmer, in terms of commitment, if > we only consider the editing work required on sources to use it, or not. Not sure what you are referring here to. You do have the choice of source encodings, and, in fact, "Unicode" is not a valid source encoding. "UTF-8" is, and from a Python point of view, there is absolutely no difference between that and, say, "ISO-8859-15" - you can choose whatever source encoding you like, and Python does not favour any of them (strictly speaking, it favour ASCII, then ISO-8859-1, then the rest). Choice of source encoding is different from the choice of string literals. You can use Unicode strings, or byte strings, or mix them. It really is your choice. Regards, Martin From DavidA at ActiveState.com Tue Aug 3 19:27:56 2004 From: DavidA at ActiveState.com (David Ascher) Date: Tue Aug 3 19:28:02 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: References: <200408031427.i73ERn330790@guido.python.org> Message-ID: <410FCB1C.3080009@ActiveState.com> Peter Astrand wrote: >>I believe ActiveState's Trent Mick has released something along these >>lines. Does your proposed API match theirs? [...] > So, the chances of getting our modules API-compatible are very small. I really don't want to get into a religious war about the relative benefits of the two modules (I don't know Peter's well enough to comment), but I think it would be nice for users if there weren't two modules with the same goal, different interfaces and the same _name_. I understand from Trent that Twisted has started to use his process.py, so having another process.py in the stdlib at some point would cause at least some users pain. --david From mal at egenix.com Tue Aug 3 19:35:59 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 3 19:36:17 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <410FCA3B.1020002@v.loewis.de> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> Message-ID: <410FCCFF.4050806@egenix.com> Martin v. L?wis wrote: > Fran?ois Pinard wrote: > >> One thing is that a Python module should have some way to know the >> encoding used in its source file, maybe some kind of `module.__coding__' >> next to `module.__file__', saving the coding effectively used while >> compilation was going on. > > That would be possible to implement. Feel free to create a patch. +1 >> I wonder if some other cookie, next to the `coding:' >> cookie, could not be used to declare that all strings _in this module >> only_ should be interpreted as Unicode by default, but without the need >> of resorting to `u' prefix all over. > > > This could be a starting point of another syntax debate. For example, > > from __future__ import string_literals_are_unicode > > would be possible to implement. If PEP 244 would have been adapted, I > would have proposed > > directive unicode_strings > > Other syntax forms would also be possible. Again, if you know a syntax > which you like, propose a patch. Be prepared to also write a PEP > defending that syntax. +1 Things that have been proposed earlier on, extended a bit: b'xxx' - return a buffer to hold binary data; same as buffer(s'abc') s'abc' - (forced) 8-bit string literal in source code encoding u'abc' - (forced) Unicode literal 'abc' - maps to s'abc' per default, can map to u'abc' based on the command line switch -U or a module switch -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From adamsz at gmail.com Tue Aug 3 19:46:49 2004 From: adamsz at gmail.com (Adam Souzis) Date: Tue Aug 3 19:46:55 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: <200408031527.i73FRQb31100@guido.python.org> References: <003201c47904$5cd353c0$df30cb97@oemcomputer> <200408031527.i73FRQb31100@guido.python.org> Message-ID: > > > Why do people want this so badly? As someone relatively new to python, it struck me as a language wart that i had to learn the form '"".join() as the proper way to do string concatenation. It violates the principles of OOWTI and its certainly not the obvious way to do string concatenation. This patch does not cover all the cases so we're still stuck with join(), but as long as it is not a documentated "feature" it will harmlessly improve the performance of countless lines of code where the coder is either untrained or too lazy to use join(). If its documented it'd just muddy the waters vis a vis join(), besides the issues with other Python implementation mentioned here. -- adam On Tue, 03 Aug 2004 08:27:26 -0700, Guido van Rossum wrote: > > I spent a *long* time reviewing and testing this patch. The code is > > clean. The concept works. It delivers significant, measurable > > benefits. The standard library itself has existing use cases. > > Every benchmark I tried showed results. The patch was not approved > > prematurely! > > Yes it was. You know I am interested in this kind of decision > (otherwise you'd have just checked it in) so you should have left the > honor to me. > > > > I am going to make up a new rule here, and state that > > > implementation features that affect not just the running speed but > > > the O() rate for certain algorithms should be considered language > > > features, because any implementation will be required to implement > > > them in order to ensure code portability. > > > > Even list.sort() does not guarantee specific O() rates. Currently, > > the fastest way to write a merge(a,b) is sort(a+b). That relies on > > the current implementation recognizing that a and b are already > > sorted and doing a linear time concatenation. > > That's a much more obscure case. The string concatenation hack will > affect a significant fraction of all code. > > > > Why do people want this so badly? > > > > Because the a=a+b form occurs everywhere, not just in newbie code. > > It is in the standard library, it shows up naturally in SAX, and it > > is often the most clear way to express an idea. > > And in 99% of those cases nobody cares about performance. > > > > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/adamsz%40gmail.com > From astrand at lysator.liu.se Tue Aug 3 20:01:23 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Tue Aug 3 20:01:34 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: <410FCB1C.3080009@ActiveState.com> References: <200408031427.i73ERn330790@guido.python.org> <410FCB1C.3080009@ActiveState.com> Message-ID: > > So, the chances of getting our modules API-compatible are very small. > > I really don't want to get into a religious war about the relative > benefits of the two modules (I don't know Peter's well enough to > comment), but I think it would be nice for users if there weren't two > modules with the same goal, different interfaces and the same _name_. I Yes, this could be a problem, but no-one has been able to come up with a solution: After 6 months, "process" and "popen" was the only suggested names. How about "subprocess"? /Peter ?strand From barry at python.org Tue Aug 3 20:01:42 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 3 20:01:39 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408030351.i733pcO29560@guido.python.org> References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> <200408030351.i733pcO29560@guido.python.org> Message-ID: <1091556102.7597.26.camel@localhost> On Mon, 2004-08-02 at 23:51, Guido van Rossum wrote: > And yes, I'm not 100% comfortable with it either. But I know one > thing: in this particular case, now is better than never. So I'm > picking a solution everybody can hate equally. > > (What's wrong with pear-shaped, anyway? It's one of my favorite > shapes. :-) So I conducted a little experiment, converting a bunch of modules that were using Python 2.3 compatible "post-processing" decorator syntax, to pie decorator syntax, so that I could get a more visceral reaction to the decision. For me, the short answer is: +1, and not just because "it's better than nothing". For longer functions, it's a clear win; the '@' stands out nicely, especially with the python-mode.el font-locking support I just added. :) It certainly exhibits all the benefits that motivated adding special syntax for decorators in the first place. For shorter functions, e.g. read-only accessors, my only (very minor) lament is that I can't compact the enter thing down to a single line. My fingers want to do something like: @property def get_bacon(self): return self._bacon but then I tell my fingers to do this instead: @property def get_bacon(self): return self._bacon Or, we'll compromise, my fingers and I, on: get_bacon = property(lambda self: self._bacon) So it's at least a wash there. (Really, pie decorators are better here too, IMO.) My only other minor lament has nothing to do with this specific syntax. I still wish there was a way to decorate multi-arg property()'s, but oh well. All in all, I'm quite happy with this syntax. Thanks Guido! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/d88a456e/attachment.pgp From pje at telecommunity.com Tue Aug 3 20:11:26 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 3 20:07:41 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <1091556102.7597.26.camel@localhost> References: <200408030351.i733pcO29560@guido.python.org> <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> <200408030351.i733pcO29560@guido.python.org> Message-ID: <5.1.1.6.0.20040803140853.034c46f0@mail.telecommunity.com> At 02:01 PM 8/3/04 -0400, Barry Warsaw wrote: >My only other minor lament has nothing to do with this specific syntax. >I still wish there was a way to decorate multi-arg property()'s, but oh >well. @property_getter def foo(self): ... @property_setter def foo(self,value): ... See Ed Loper's post at: http://mail.python.org/pipermail/python-dev/2004-April/043902.html and implementation at: http://www.cis.upenn.edu/~edloper/pydecorators.html From astrand at lysator.liu.se Tue Aug 3 20:09:29 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Tue Aug 3 20:11:06 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: <1091552865.410fc661b4abd@mcherm.com> References: <1091552865.410fc661b4abd@mcherm.com> Message-ID: > Last time I tried it there were still a few windows issues (docs say > these have been resolved), but I thought the API was well-designed. The details: The module works great on Windows currently, but requires the win32all extensions. The idea is to get rid of this dependency by writing a glue module, pretty much as _winreg. Work on such a _process module has been done on two fronts: * A friendly Python-developer has gotten quite far, but was experiencing some strange hangs. I haven't been able to look at this work yet. * I've started working on an implementation myself. I was using _winreg as some sort of template. The problem is: I've never written extension modules before :-) It was quite easy to export the numeric constants, but when I came to creating Python objects, I was lost. I'll guess I need to read some documentation. This is what remains: win32api: GetStdHandle GetCurrentProcess, DuplicateHandle win32pipe: CreatePipe win32process: CreateProcess, STARTUPINFO, GetExitCodeProcess win32event: WaitForSingleObject Here's the module so far: /* _process.c */ #include "windows.h" #include "Python.h" #include "structmember.h" #include "malloc.h" /* for alloca */ /* The win32api module reports the function name that failed, but this concept is not in the Python core. Hopefully it will one day, and in the meantime I dont want to lose this info... */ #define PyErr_SetFromWindowsErrWithFunction(rc, fnname) \ PyErr_SetFromWindowsErr(rc) /* Forward declares */ /* Doc strings */ // FIXME PyDoc_STRVAR(module_doc, "This module provides...\n"); static struct PyMethodDef process_methods[] = { NULL, }; static void insint(PyObject * d, char * name, long value) { PyObject *v = PyInt_FromLong(value); if (!v || PyDict_SetItemString(d, name, v)) PyErr_Clear(); Py_XDECREF(v); } #define ADD_INT(val) insint(d, #val, val) PyMODINIT_FUNC init_process(void) { PyObject *m, *d; m = Py_InitModule3("_process", process_methods, module_doc); d = PyModule_GetDict(m); Py_INCREF(PyExc_WindowsError); if (PyDict_SetItemString(d, "error", PyExc_WindowsError) != 0) return; /* Add the relevant constants */ ADD_INT(DUPLICATE_SAME_ACCESS); ADD_INT(STARTF_USESTDHANDLES); ADD_INT(STD_INPUT_HANDLE); ADD_INT(STD_OUTPUT_HANDLE); ADD_INT(STD_ERROR_HANDLE); ADD_INT(INFINITE); ADD_INT(WAIT_OBJECT_0); } (I will be away for a couple of days.) /Peter ?strand From barry at python.org Tue Aug 3 20:22:43 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 3 20:22:38 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <1091531214.1502.20.camel@localhost> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> <1091531214.1502.20.camel@localhost> Message-ID: <1091557363.7562.38.camel@localhost> On Tue, 2004-08-03 at 07:06, Mark Russell wrote: > I would say leave it like this for now. It is easy to relax the > restriction later if there's a demand. +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/4e411a24/attachment-0001.pgp From eppstein at ics.uci.edu Tue Aug 3 20:52:22 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Tue Aug 3 20:52:29 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> <1091531214.1502.20.camel@localhost> <1091557363.7562.38.camel@localhost> Message-ID: BTW, here's another use case I haven't seen suggested yet: def substitution_for(regexp): sub = re.compile(regexp).sub return lambda func: lambda text: sub(func,text) e.g. (example from some recent code of mine, redone as a decorator) @substitution_for('[a-z][A-Z]') def camelspace(m): """Insert space into camelcase regexp match.""" return m.group(0)[0]+' '+m.group(0)[1:] ... info.title = camelspace(info.basefilename) Instead my current code defines a separate variable for the re.compile().sub object, then later has to put this variable together correctly with the undecorated camelspace function to perform the substitution. Using decorators allows the sub and replacement function to become merged into a single object with a more intuitive API. (Also, my actual code uses more than one of these substitutions...) I don't generally prefer to use explicit lambdas but expanding them out with names makes the definition of substitution_for a lot longer without much readability gain in my eyes. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From foom at fuhm.net Tue Aug 3 21:08:43 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 3 21:08:48 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408031440.i73Ee6V30875@guido.python.org> References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> <200408031440.i73Ee6V30875@guido.python.org> Message-ID: <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net> On Aug 3, 2004, at 10:40 AM, Guido van Rossum wrote: > I have a gut feeling about this one. I'm not sure where it comes > from, but I have it. I guess I have the opposite gut feeling: special classes of expressions in the grammar are always bad. Michael Chermside's message said this better than I can. Heck, even 'except' can take an arbitrary expression as an argument! Now I'm damn sure I'd never write code that says except FooError and f() or MyErrors[5]: because that would be insane, but I'm glad python doesn't restrict the argument to some strange subset of expressions that someone decided were approved! Consistency of grammar is a key feature, even when it allows users to write insane programs. > It may be that I want the compiler to be able to > recognize certain decorators. If I understand what you're saying, it seems that you're thinking the @ form should be more a static declaration, not a dynamically evaluated expression, so that the compiler can reason about it without running code. However, that isn't the case even now, and if it was, it wouldn't fit in with the rest of the python language very well. Definitely an interesting idea, but I think it would require the @'s argument to be a compile-time macro, instead of a run-time expression. And python doesn't have compile-time macros. With @ taking a runtime expression, I don't see how the compiler could possibly do anything with its argument. Nothing can be guaranteed. As a postscript, I must say I am alarmed by some comments others have used to justify the current restricted state. In particular (paraphrased), "it's good because it keeps people from using lambda", and "it's good because it keeps people from doing '@1+1' by mistake. For the first: you hate lambdas so much? They are in python -- arbitrarily forbidding their use in certain expressions because you don't like them is pretty strange. For the second: You want to do type checking via syntax error?! *That* is insanity. And it doesn't even work well (of course). James From ncoghlan at iinet.net.au Tue Aug 3 21:19:37 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Aug 3 21:20:03 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: References: <003201c47904$5cd353c0$df30cb97@oemcomputer> <200408031527.i73FRQb31100@guido.python.org> Message-ID: <410FE549.1030407@iinet.net.au> Adam Souzis wrote: > As someone relatively new to python, it struck me as a language wart > that i had to learn the form '"".join() as the proper way to do string > concatenation. It violates the principles of OOWTI and its certainly > not the obvious way to do string concatenation. This patch does not > cover all the cases so we're still stuck with join(), but as long as > it is not a documentated "feature" it will harmlessly improve the > performance of countless lines of code where the coder is either > untrained or too lazy to use join(). If its documented it'd just > muddy the waters vis a vis join(), besides the issues with other > Python implementation mentioned here. If I understand correctly, you're suggesting that ''.join(strings) continue to be the recommended, portable, non-quadratic method for concatenating strings. And this patch be seen merely as a CPython optimisation for code which doesn't use the recommended string concatenation hack. . . Seems like a sensible way of looking at it to me. Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268 From pinard at iro.umontreal.ca Tue Aug 3 21:47:24 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Tue Aug 3 21:54:38 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <410FCCFF.4050806@egenix.com> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> Message-ID: <20040803194724.GB9031@titan.progiciels-bpi.ca> [M.-A. Lemburg] > Martin v. L?wis wrote: > Things that have been proposed earlier on, extended a bit: > b'xxx' - return a buffer to hold binary data; same as buffer(s'abc') > s'abc' - (forced) 8-bit string literal in source code encoding > u'abc' - (forced) Unicode literal I currently do not see the need of a fine distinction between `b' or `s' as a prefix. `s' and `u' are the first letter of the type (`str' or `unicode') and that makes them natural. > 'abc' - maps to s'abc' per default, can map to u'abc' based on the > command line switch -U or a module switch The idea would be, indeed, to create some kind of per-module switch. I'm less sure that `-U' is any useful in practice, as long as all of the library does not become "Unicode-aware", whatever that would imply... P.S. - Command line switch for command line switch :-), a switch for fully turning on the newer type system would be more productive than `-U', and put some pressure for refreshening the library in this area. Just curious, as I do not intend to volunteer in this area, is there something else than Exception in the Python internals that rely on old-style classes? -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From pinard at iro.umontreal.ca Tue Aug 3 21:35:16 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Tue Aug 3 21:54:39 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <410FCA3B.1020002@v.loewis.de> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> Message-ID: <20040803193516.GA9031@titan.progiciels-bpi.ca> [Martin von L?wis] > Fran?ois Pinard wrote: > > maybe some kind of `module.__coding__' next to `module.__file__', > >saving the coding effectively used while compilation was going on. > That would be possible to implement. Feel free to create a patch. I might try, and it would be my first Python patch. But please, please tell me if the idea is not welcome, as my free time is rather short and I already have a lot of things waiting for me! :-). > >I wonder if some other cookie, next to the `coding:' cookie, could > >not be used to declare that all strings _in this module only_ should > >be interpreted as Unicode by default, but without the need of > >resorting to `u' prefix all over. > [...] if you know a syntax which you like, propose a patch. Be > prepared to also write a PEP defending that syntax. Surely no particular syntax that I like enough for defending it. Anything reasonable would do as far as I am concerned, so I might propose a reasonable patch without involving myself into a crusade. Yet I may try to assemble and edit together the ideas of others, if it serves a purpose. > >Right now, my feeling is that Python asks a bit too much of a > >programmer, in terms of commitment, if we only consider the editing > >work required on sources to use it, or not. > Not sure what you are referring here to. There is currently a lot of effort involved in Python so Unicode strings and usual strings inter-operate correctly and automatically, also hiding as much as reasonable to the unwilling user whether if characters are large or narrow: s/he uses about the same code no matter what. The way Python does is rather lovely, in fact. :-) I'm going to transform a flurry of Latin-1 Python scripts to UTF-8, but not all of them, as I'm not going to impose Unicode in our team where it is not wanted. For French, and German and many others, we have been lucky enough for having one codepoint per character in Unicode, so we can hope that programs assuming that S[N] addresses the N'th (0-based) character of string S will work the same way irrelevant of if strings are narrow or wide. However, and I shall have the honesty to state it, this is *not* respectful of the general Unicode spirit: the Python implementation allows for independently addressable surrogate halves, combining zero-width diacritics, normal _and_ decomposed forms, directional marks, linguistic marks and various other such complexities. But in our case, where applications already work in Latin-1, abusing our Unicode luck, UTF-8 may _not_ be used as is, we ought to use Unicode or wide strings as well, for preserving S[N] addressability. So changing source encodings may be intimately tied to going Unicode whenever UTF-8 (or any other variable-length encoding) gets into the picture. > You do have the choice of source encodings, and, in fact, "Unicode" > is not a valid source encoding. "UTF-8" is [...] Guess that I know! :-) :-) > [...] from a Python point of view, there is absolutely no difference > between [UTF-8] and, say, "ISO-8859-15". Choice of source encoding > is different from the choice of string literals. You can use Unicode > strings, or byte strings, or mix them. It really is your choice. I hope that my explanation above helps at seeing that source encoding and choice of string literals are not as independent as one may think. A choice that I surely do _not_ have is to see bugs appear in programs merely because I changed the source encoding. Going from ISO 8859-1 to ISO 8859-15 for a Python source is probably fairly safe, because there is no need for switching the narrowness of strings. Going from ISO 8859-1 to UTF-8 is very unsafe, and editing all literal strings from narrow to wide, using `u' prefixes, becomes almost unavoidable. There ought to be a way to maintain a single Python source that would work dependably through re-encoding of the source, but not uselessly relying on wide strings when there is no need for them. That is, without marking all literal strings as being Unicode. Changing encoding from ISO 8859-1 to UTF-8 should not be a one-way, no-return ticket. Of course, it is very normal that sources may have to be adapted for the possibility of a Unicode context. There should be some good style and habits for writing re-encodable programs. So this exchange of thoughts. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From guido at python.org Tue Aug 3 22:14:58 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 3 22:15:05 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: Your message of "Tue, 03 Aug 2004 15:47:24 EDT." <20040803194724.GB9031@titan.progiciels-bpi.ca> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> Message-ID: <200408032014.i73KEwp31876@guido.python.org> > P.S. - Command line switch for command line switch :-), a switch for > fully turning on the newer type system would be more productive than > `-U', and put some pressure for refreshening the library in this area. > Just curious, as I do not intend to volunteer in this area, is there > something else than Exception in the Python internals that rely on > old-style classes? Probably not, but making Exception a new-style class won't be easy. (It will also break code that creates a class used as an exception that doesn't derive from Exception, but those should be shot. :-) Until Exception is a new-style class, such a switch wouldn't really work. --Guido van Rossum (home page: http://www.python.org/~guido/) From pf_moore at yahoo.co.uk Tue Aug 3 22:15:30 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Tue Aug 3 22:15:33 2004 Subject: [Python-Dev] Re: PEP 324 (process module) References: <1091552865.410fc661b4abd@mcherm.com> Message-ID: Michael Chermside writes: > Guido writes: >> I believe ActiveState's Trent Mick has released something >> along these lines. Does your proposed API match theirs? > > Peter replies: > [...] >> So, the chances of getting our modules API-compatible are very small. > > I'll note that my impression is that Peter's module is not aiming to > be particularly innovative in terms of implementation, it's primary > "feature" is having a decent API. And my impression is that it has > been well received on c.l.py (for instance, I often see it recommended > in response to newbie questions). He got a lot of input during the PEP > process and tried to incorporate it. Although Peter has been very receptive to feedback, he has some strong design goals (noted in the PEP) which make the resulting module feel too "low level" for me. I'd love to see a function process.filter(cmd, input=None) Execute cmd (if it's a string, use the shell, if it's a list, treat as an argv sequence) and pass it input as stdin (default, no stdin). Return the command's stdout. If an error occurs (either in the exec, or if the command returns a nonzero return code) raise an exception (and capture the stderr in the exception data). This function should deal with deadlock issues. I don't know of *any* process module that makes this "expect it to work" model any easier. And Peter's design constraint of avoiding the shell actively hurts here (although his interact() function is specifically there to avoid the deadlock issues that could otherwise occur with a naive attempt to implement this). It's possible that I could implement this function on top of Trent's module. Must try... Paul. -- "Bother," said the Borg, "We've assimilated Pooh." From eppstein at ics.uci.edu Tue Aug 3 22:22:11 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Tue Aug 3 22:22:16 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> <200408031440.i73Ee6V30875@guido.python.org> <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: In article <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net>, James Y Knight wrote: > As a postscript, I must say I am alarmed by some comments others have > used to justify the current restricted state. In particular > (paraphrased), "it's good because it keeps people from using lambda", > and "it's good because it keeps people from doing '@1+1' by mistake. > For the first: you hate lambdas so much? I think I was the one who posted the lambda comment. I also posted in a different message a use-case for decorators that had a line with two lambdas in it, so obviously I don't hate lambdas. I'm actually pretty neutral on restricted decorators vs @expression. But @lambda(func): body def func(...): ... is (I think) much more likely to be an abuse of language than the Pythonic way of writing something, so I don't see a lot of harm in preventing it. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From pf_moore at yahoo.co.uk Tue Aug 3 22:24:13 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Tue Aug 3 22:24:14 2004 Subject: [Python-Dev] PEP 309 (partial function application) Message-ID: Should this go into 2.4a2? It's marked as "accepted" in the PEP list, and an implementation exists as SF patch 941881. Paul -- The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair. -- Douglas Adams From jhylton at gmail.com Tue Aug 3 23:08:07 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Tue Aug 3 23:08:09 2004 Subject: [Python-Dev] PEP 309 (partial function application) In-Reply-To: References: Message-ID: On Tue, 03 Aug 2004 21:24:13 +0100, Paul Moore wrote: > Should this go into 2.4a2? It's marked as "accepted" in the PEP list, > and an implementation exists as SF patch 941881. Paul, Without comment on the merits of the PEP, it seems out of date to me. The PEP presents three different implementations of curry (one in Pyrex) without clarifying which one is the preferred one. It presents a summary at the end that would make more sense at the beginning. I would revise the PEP to: 1. Clearly describe what is being implemented for 2.4. Provide a specification for the behavior along with examples of use. Make sure you cover all the corner cases, e.g. can you provide a keyword arg without providing positional args. 2. Provide a rationale for why this belongs in the standard library. 3. Describe the @ syntax, but put it in a self-contained section and explain why it was not implemented. Jeremy From djc at object-craft.com.au Wed Aug 4 01:55:49 2004 From: djc at object-craft.com.au (Dave Cole) Date: Wed Aug 4 01:55:54 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <410F389D.6090609@iinet.net.au> References: <410F3465.9000005@object-craft.com.au> <410F389D.6090609@iinet.net.au> Message-ID: <41102605.2080807@object-craft.com.au> Nick Coghlan wrote: > Dave Cole wrote: > >> Is there any reason why something like this would not be a good idea? >> >> >>> a_list = [1, 2, 3, 4, 5] >> >>> a, b, *c = a_list >> >> You could then do things like this: >> >> >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] >> >>> for a, b *c in lol: >> ... >> >> - Dave >> > > As opposed to: > > >>> for a, b, c in ((x[0], x[1], x[2:]) for x in lol): print a, b, c Yes, as opposed to. > With generator expressions around, I don't know that this case is common > enough for special casing. . . This begs the question; do you prefer: >>> args = [4, 5, 6] >>> a_func(1, *args) or this: >>> args = [4, 5, 6] >>> apply(a_func, [1] + args) - Dave -- http://www.object-craft.com.au From tjreedy at udel.edu Wed Aug 4 02:25:19 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Aug 4 02:25:25 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> Message-ID: "Jp Calderone" wrote in message news:410FC9FE.6060305@divmod.com... > I like Python(*) because it is simple and generally unsurprising. Me too. > I think that '@deco' adds complexity, both for beginners and experienced > programmers alike, Decorators will add complexity, regardless of syntax. An advantage of the @ syntax is than it makes the complexiity separable and ignorable for someone learning/teaching basic Python, which includes list literals and function defs. To me, the [deco list] syntax is particularly bad in adding an advanced, esoteric, context-dependent, magic meaning to an hour 1 basic concept. > so I do not see it as an improvement. Apparently, the question of whether decos are an improvement has at least temporarily been decided in the affirmative. Terry J. Reedy From nas at arctrix.com Wed Aug 4 03:07:12 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Wed Aug 4 03:07:15 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: References: <410FC9FE.6060305@divmod.com> Message-ID: <20040804010712.GA10819@mems-exchange.org> On Tue, Aug 03, 2004 at 08:25:19PM -0400, Terry Reedy wrote: > Apparently, the question of whether decos are an improvement has > at least temporarily been decided in the affirmative. That's not apparent to me. It seems to me that most people just got sick of the endless debate and tuned out. The people who had urgent need for the feature were probably a bit more persistent. Personally, I don't see the cost vs. benefit of the feature to be compelling. It seems there is not a lot you can do with a decorator without resorting to ugly hacks (e.g. messing with bytecode). returning-to-tuned-out-mode-ly yr's Neil From greg at cosc.canterbury.ac.nz Wed Aug 4 03:14:32 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 4 03:14:38 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <1f7befae040802201470e3d4c@mail.gmail.com> Message-ID: <200408040114.i741EWcl029740@cosc353.cosc.canterbury.ac.nz> Tim Peters : > there's one routine that executes a module's initialization code, > all imports go thru it eventually, and that's the routine that now > removes the module's name from sys.modules if the initialization > code barfs. Couldn't that routine also save sys.modules before starting to execute the module's code? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From andrewm at object-craft.com.au Wed Aug 4 03:23:20 2004 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Wed Aug 4 03:23:19 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <20040804010712.GA10819@mems-exchange.org> References: <410FC9FE.6060305@divmod.com> <20040804010712.GA10819@mems-exchange.org> Message-ID: <20040804012320.B38523C7A0@coffee.object-craft.com.au> >That's not apparent to me. It seems to me that most people just got >sick of the endless debate and tuned out. It's also entirely possible that "most people" were happy with the way things were going and chose not to post... My personal opinion is that the pie syntax is acceptable, and the continuing debate is doing harm and providing no enlightenment. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From aahz at pythoncraft.com Wed Aug 4 03:33:59 2004 From: aahz at pythoncraft.com (Aahz) Date: Wed Aug 4 03:34:02 2004 Subject: [Python-Dev] 2.4a2 and relative import (PEP 328) In-Reply-To: <200408021850.i72IoCg28650@guido.python.org> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> Message-ID: <20040804013359.GA14122@panix.com> On Mon, Aug 02, 2004, Guido van Rossum wrote: > > Did the relative import syntax that was approved ever get checked in? > I thought it was on the list of things to land in 2.4? Sorry, I've been slacking on driving it. What it really needs right now is someone to implement it. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From tim.peters at gmail.com Wed Aug 4 04:03:44 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 4 04:03:46 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <200408040114.i741EWcl029740@cosc353.cosc.canterbury.ac.nz> References: <200408040114.i741EWcl029740@cosc353.cosc.canterbury.ac.nz> Message-ID: <1f7befae04080319031d9b1c94@mail.gmail.com> [Tim Peters] >> there's one routine that executes a module's initialization code, >> all imports go thru it eventually, Which isn't true. I'm hacking imputils and ihooks now. >> and that's the routine that now removes the module's name from >> sys.modules if the initialization code barfs. [Greg Ewing] > Couldn't that routine also save sys.modules before > starting to execute the module's code? With enough bother, sure. Go for it . From tim.peters at gmail.com Wed Aug 4 04:14:04 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 4 04:14:07 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <20040804012320.B38523C7A0@coffee.object-craft.com.au> References: <410FC9FE.6060305@divmod.com> <20040804010712.GA10819@mems-exchange.org> <20040804012320.B38523C7A0@coffee.object-craft.com.au> Message-ID: <1f7befae040803191421447be6@mail.gmail.com> [Andrew McNamara] > It's also entirely possible that "most people" were happy with the way > things were going and chose not to post... I was indeed "happy enough". > My personal opinion is that the pie syntax is acceptable, Ditto. It does simple things quite well, and it makes it obvious they're being done. Then again, I like "print >> fileobj" too <0.7 wink>. > and the continuing debate is doing harm We're in no danger of running out of pixels. > and providing no enlightenment. That's for sure. In the old days, Guido would Pronounce, and we'd all bite our tongues (although not necessarily each his own). The less time Guido can make for Python, the more important becomes graceful capitulation. From guido at python.org Wed Aug 4 04:24:59 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 4 04:25:09 2004 Subject: [Python-Dev] 2.4a2 and relative import (PEP 328) In-Reply-To: Your message of "Tue, 03 Aug 2004 21:33:59 EDT." <20040804013359.GA14122@panix.com> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> <20040804013359.GA14122@panix.com> Message-ID: <200408040224.i742Oxc32452@guido.python.org> > > Did the relative import syntax that was approved ever get checked in? > > I thought it was on the list of things to land in 2.4? > > Sorry, I've been slacking on driving it. What it really needs right now > is someone to implement it. It's not too late -- the decorator syntax quickly found someone who implemented the whole thing too. Hard to make a2, but a3 is still a possibility. --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at iinet.net.au Wed Aug 4 05:08:51 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 4 05:09:18 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> <200408031440.i73Ee6V30875@guido.python.org> <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <41105343.7020904@iinet.net.au> David Eppstein wrote: > I'm actually pretty neutral on restricted decorators vs @expression. But > @lambda(func): body > def func(...): ... > is (I think) much more likely to be an abuse of language than the > Pythonic way of writing something, so I don't see a lot of harm in > preventing it. And, as someone pointed out, *lifting* a restriction is a heck of a lot easier than imposing one later on (even after only 1 alpha). YouAintGonnaNeedIt and all that. . . Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268 From ncoghlan at iinet.net.au Wed Aug 4 05:21:56 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 4 05:22:18 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <41102605.2080807@object-craft.com.au> References: <410F3465.9000005@object-craft.com.au> <410F389D.6090609@iinet.net.au> <41102605.2080807@object-craft.com.au> Message-ID: <41105654.6080305@iinet.net.au> Dave Cole wrote: > This begs the question; do you prefer: > > >>> args = [4, 5, 6] > >>> a_func(1, *args) > > or this: > > >>> args = [4, 5, 6] > >>> apply(a_func, [1] + args) > > - Dave Oh, I certainly prefer the variable argument format, rather than having to use apply. The difference is that I've found that useful on several occasions, mainly because keyword arguments let you get around the 'only at the end' problem. As far as I know, the new syntax was to able to *completely* replace the functionality of 'apply' (I've certainly never needed to use it, no matter how complex the variable argument list games got). Whereas the list assignment proposal gives special synactic sugar to *one* form of slicing (x[0], ..., x[n], x[(n+1):]). As soon as the form of your slice changes, you're going to have to switch to a generator expression anyway. Hence the question about whether or not this special case was special enough to be given its own syntax, given that the comparable generator expression really isn't that complicated. *shrug* I'm not implacably opposed or anything - although I'd be interested in hearing from those around hear who teach Python as to whether they think it would be beneficial or not. Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268 From greg at cosc.canterbury.ac.nz Wed Aug 4 05:50:27 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 4 05:50:37 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <41105654.6080305@iinet.net.au> Message-ID: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz> Nick Coghlan : > Whereas the list assignment proposal gives special synactic sugar to > *one* form of slicing (x[0], ..., x[n], x[(n+1):]). It's a comparatively common one, though, I think. When doing things like parsing input, it's often natural to want to peel off the first few items of a list and leave the rest for processing later, based on what the initial items turn out to be. Having to explicitly slice off the part you want to unpack not only seems inefficient (constructing an intermediate list or tuple just to immediately unpack it and throw it away) it also tends to obfuscate what is going on. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From edloper at gradient.cis.upenn.edu Wed Aug 4 06:11:07 2004 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Wed Aug 4 06:10:21 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <20040804030933.210F61E4020@bag.python.org> References: <20040804030933.210F61E4020@bag.python.org> Message-ID: <411061DB.8010803@gradient.cis.upenn.edu> > It's also entirely possible that "most people" were happy with the way > things were going and chose not to post... > > My personal opinion is that the pie syntax is acceptable, and the > continuing debate is doing harm and providing no enlightenment. +1. (Thanks to the people who've devoted so much thought and energy to figuring out the best decorator syntax; I, for one, will be happy to have it available.) -Edward From trentm at ActiveState.com Wed Aug 4 06:31:21 2004 From: trentm at ActiveState.com (Trent Mick) Date: Wed Aug 4 06:31:28 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: ; from astrand@lysator.liu.se on Tue, Aug 03, 2004 at 05:22:35PM +0200 References: <200408031427.i73ERn330790@guido.python.org> Message-ID: <20040803213121.C9777@ActiveState.com> [Peter Astrand wrote] > > > I believe ActiveState's Trent Mick has released something along these > > lines. Does your proposed API match theirs? > > No, not at all. There are some issues with this module, both regards the > API and the implementation, which I'm not happy with. I tried to establish > a discussion with Trent about this, but if I remember correctly, I never > recieved a response. Yes, I am an ass. I am sorry about that Peter. I have had my blinders on for quite a while with work stuff. I have just started looking at your module so I can't be sure that we could find a happy common ground but I hope so. I realize that my process.py has some warts (including some you have pointed out below) and I am willing to work on those. > Here's some things which I'm not happy with: > > * There are three different "factory" classes: Process, ProcessOpen, > ProcessProxy. I have only one, covering all cases. I think three different > classes are confusing to users. Even I have trouble understanding the > difference between them. Part of the confusion, I think, is that my process.py's docstrings have lagged a little bit behind development and there is one glaring problem that ProcessOpen.__init__.__doc__ mistakenly has the description from ProcessProxy.__init__. Process: Create a process without doing without doing anything with stdin/stderr/stdout. Kind of like os.system in that regard. ProcessOpen: ...add stdin/stdout/stderr attributes (file-like objects). Kind of like os.popen3 in that regard. Separating the two allows Process' implementation to be simpler: no dup'ing (and inheritability handling, etc) of the std handlers is required. If general opinion is that this separation is not useful, then I am fine with merging the two. (Note that my vague memory is that this separation may be necessary to allow for new console handling with subsystem:windows apps on Windows. I can't remember exactly though.) ProcessProxy: ...a behemoth using a thread for each of stdin/stdout/stderr to allow the user to get an event-like IO interface. This is something that was required for Komodo (for my job) to allow interactive running (and stdin handling) of processes in a GUI terminal (i.e., if you know Komodo, to run commands in the "Command Output" tab). If this is deemed too controversial for the Python core then I think it would be possible to move this out to a separate module -- though I know that some people outside of Komodo have found this useful. > > * The code size is very large. The code is complex. > > * Trent's module always executes things through the shell, and deals with > every ugly cornercase that comes from this. Some of the large code size is _for_ execution through the shell. :) I think that execution via the shell should be a feature of a module like this (so that users can use some shell features) and I even think that it should be the default (so that, for example, Windows users don't have to learn what cmd.exe or command.com are to run "dir"). However, I absolutely agree that one should be able to run withOUT the shell (i.e. have an option for this). Other reasons for the size/complexity of my process.py over yours: - My process objects have a .kill() method -- which is actually quite a pain on Windows. - My module contains a work-around (113 lines) for a known bug in LinuxThreads where one cannot .wait() on a created process from a subthread of the thread that created the process. (1) This feature was a requirement for Komodo. Pulling it out to a separate module and just documenting the limitation (which really isn't a big deal for most common uses) is probably an option for me. In fact the workaround is probably not generally acceptible and should, at the very least, be made optional. (2) I haven't had a chance to check but I _think_ that recent Linuxs may have switched to the newer threading libraries out there that fix this. At least I remember reading about the new threading libraries (there was more that one competitor) a couple of years ago. - The ProcessProxy "behemoth" is responsible for about half of my process.py module. - My module includes some handling that for subsystem:windows vs. subsystem:console apps on Windows that I don't think yours does. (That isn't totally fair: I think yours has a few features that mine doesn't.) > * The modules uses destructors, which I'm usually avoiding. Is that evil? I have gotten mixed signals from the occassional lurking that I have done on the Python lists. Currently refcounts will ensure that my __del__'s get called. And I *do* have .close() methods for proper usage. Though that seem anathema to some Python programmers. > So, the chances of getting our modules API-compatible are very small. Presuming that I don't go dark again I hope that maybe we can reach some concensus. In the meantime, if you'd be willing to change your module's name to something other than process.py I think that would help discussions. (I don't know if that is a pain for you at this point. You mentioned "subprocess". Alternatively, how about "posixprocess"? Though, despite PEP 324's title, I don't know if that is completely accurate anymore.) Trent -- Trent Mick TrentM@ActiveState.com From djc at object-craft.com.au Wed Aug 4 06:36:56 2004 From: djc at object-craft.com.au (Dave Cole) Date: Wed Aug 4 06:37:01 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz> References: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz> Message-ID: <411067E8.3020002@object-craft.com.au> Greg Ewing wrote: > Nick Coghlan : > > >>Whereas the list assignment proposal gives special synactic sugar to >>*one* form of slicing (x[0], ..., x[n], x[(n+1):]). > > > It's a comparatively common one, though, I think. When doing things > like parsing input, it's often natural to want to peel off the first > few items of a list and leave the rest for processing later, based on > what the initial items turn out to be. > > Having to explicitly slice off the part you want to unpack not only > seems inefficient (constructing an intermediate list or tuple just to > immediately unpack it and throw it away) it also tends to obfuscate > what is going on. So I suppose you could avoid the unnecessary tuple construction by doing something like this: >>> a_list = [1, 2, 3, 4, 5] >>> a, b, * = a_list >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] >>> for a, b, * in lol: ... But that looks a bit bogus - but does avoid having to create the extra tuple. - Dave -- http://www.object-craft.com.au From martin at v.loewis.de Wed Aug 4 06:51:28 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 4 06:51:27 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz> References: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz> Message-ID: <41106B50.2000201@v.loewis.de> Greg Ewing wrote: > Having to explicitly slice off the part you want to unpack not only > seems inefficient (constructing an intermediate list or tuple just to > immediately unpack it and throw it away) it also tends to obfuscate > what is going on. Of course, the proposed syntax does not change this. Compare a,b,c = foo()[:3] to a,b,c,*_ = foo() In either case, the extra fields obfuscate things, and in either case, a second tuple is created. In the latter case, the tuple is even stored in a variable. Regards, Martin From kbk at shore.net Wed Aug 4 06:54:17 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Aug 4 06:54:22 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200408040454.i744sHIe016349@h006008a7bda6.ne.client2.attbi.com> Patch / Bug Summary ___________________ Patches : 276 open ( -3) / 2493 closed (+10) / 2769 total ( +7) Bugs : 766 open ( +4) / 4317 closed (+14) / 5083 total (+18) RFE : 146 open ( +1) / 130 closed ( +0) / 276 total ( +1) New / Reopened Patches ______________________ A BSD-style wait4 implementation (2004-07-29) http://python.org/sf/1000267 opened by chads posixpath._resolve_link typo (2004-07-31) CLOSED http://python.org/sf/1001109 opened by Ronald Oussoren socketmodule on OpenBSD/sparc64 (64bit machine) (2004-08-01) http://python.org/sf/1001610 opened by Aleksander Piotrowski Logger file objects (2004-08-02) http://python.org/sf/1001864 opened by Miki Tebeka O(1) xrange membership testing (2004-08-02) http://python.org/sf/1002085 opened by Mac-arena the Bored Zo Better extensibility for distutils commands (2004-08-02) CLOSED http://python.org/sf/1002241 opened by Fred L. Drake, Jr. Patches Closed ______________ posixpath._resolve_link typo (2004-07-31) http://python.org/sf/1001109 closed by ronaldoussoren Implementation for PEP 318 using java-style syntax (2004-06-25) http://python.org/sf/979728 closed by anthonybaxter Better extensibility for distutils commands (2004-08-02) http://python.org/sf/1002241 closed by fdrake HTTP basic authentication problem (2004-07-20) http://python.org/sf/994595 closed by loewis speed up md5: allow Encode and Decode to inline (2004-06-22) http://python.org/sf/977074 closed by loewis add option to NOT use ~/.netrc in nntplib.NNTP() (2003-10-14) http://python.org/sf/823072 closed by loewis askstring => grab fail (2004-05-02) http://python.org/sf/946153 closed by loewis 658254: accept None for time.ctime() and friends (2003-01-06) http://python.org/sf/663482 closed by fdrake Add support for sync and use "wish" options (2004-07-08) http://python.org/sf/986929 closed by loewis commands.py for Windows (2002-12-17) http://python.org/sf/655421 closed by zopezen New / Reopened Bugs ___________________ optparse error method doesn't print option info (2004-07-29) http://python.org/sf/1000439 opened by Ivan Nestlerode "make pdf" failure w/ 2.4 docs (2004-07-30) http://python.org/sf/1000841 opened by Mat Martineau Carbon.CF.CFMutableArray hangs interpreter (2004-07-30) http://python.org/sf/1000914 opened by Ronald Oussoren str.join([ str-subtype-instance ]) misbehaves (2004-07-31) http://python.org/sf/1001011 opened by Thomas Wouters setdefaulttimeout causes unnecessary timeouts on connect err (2004-07-31) CLOSED http://python.org/sf/1001018 opened by Mark Hammond wave.open() with unicode filename fails (2004-07-31) CLOSED http://python.org/sf/1001053 opened by John Popplewell incorrect reference to macro named foo (2004-07-31) CLOSED http://python.org/sf/1001088 opened by Clinton Roy hotspot profiler does not work correctly on P4 CPUs with HT (2004-07-31) http://python.org/sf/1001150 opened by Viktor Ferenczi glob unicode (2004-08-01) http://python.org/sf/1001604 opened by leve socketmodule does not build under cygwin (2004-08-02) http://python.org/sf/1001857 opened by Miki Tebeka IDLE hangs when inactive more than 2 hours (2004-08-02) http://python.org/sf/1001869 opened by Miki Tebeka os.path.realpath can't handle symlink loops (2004-04-05) http://python.org/sf/930024 reopened by loewis os.path.sameopenfile documentation wrong. (2004-08-02) http://python.org/sf/1002398 opened by Jeremy Fincher Wrong documentation of __radd__ etc. (2004-08-03) http://python.org/sf/1002453 opened by Hallvard B Furuseth MemoryError on AIX52 (2004-08-03) http://python.org/sf/1002465 opened by Datranet email message parser doesn't handle \r\n correctly (2004-08-03) http://python.org/sf/1002475 opened by Sjoerd Mullender test_decimal fails if repeated (2004-08-03) http://python.org/sf/1002530 opened by Michael Hudson test_logging fails if run twice (2004-08-03) http://python.org/sf/1002537 opened by Michael Hudson glob() unicode bug (2004-08-03) CLOSED http://python.org/sf/1002632 opened by leve asynchat does not accept 'long' terminator (2004-08-03) http://python.org/sf/1002763 opened by Jey Kottalam Bugs Closed ___________ "disjunct" should be "disjoint" (2004-07-25) http://python.org/sf/997533 closed by rhettinger Mingw needs to use the same MSVC runtime as Python (2004-01-04) http://python.org/sf/870382 closed by loewis Python 2.3.4, Solaris 7, socketmodule.c does not compile (2004-06-14) http://python.org/sf/972724 closed by brucedray setdefaulttimeout causes unnecessary timeouts on connect err (2004-07-30) http://python.org/sf/1001018 closed by tim_one wave.open() with unicode filename fails (2004-07-30) http://python.org/sf/1001053 closed by nnorwitz incorrect reference to macro named foo (2004-07-31) http://python.org/sf/1001088 closed by nnorwitz os.path.realpath can't handle symlink loops (2004-04-05) http://python.org/sf/930024 closed by akuchling leaking snippet (2003-09-18) http://python.org/sf/808596 closed by mwh refleaks in _hotshot.c (2003-09-18) http://python.org/sf/808756 closed by arigo math and cmath docs don't specify radians (2004-07-23) http://python.org/sf/996392 closed by tim_one bsddb has default flag of c, not r (2004-07-26) http://python.org/sf/998001 closed by akuchling glob() unicode bug (2004-08-03) http://python.org/sf/1002632 closed by lemburg boolobject.h documentation missing from Python/C API (2004-01-22) http://python.org/sf/882332 closed by fdrake Accept None for time.ctime() and friends (2002-12-24) http://python.org/sf/658254 closed by fdrake idle.py doesn't accept ( in some cases (2003-07-22) http://python.org/sf/775541 closed by kbk New / Reopened RFE __________________ Adding missing ISO 8859 codecs, especially Thai (2004-08-02) http://python.org/sf/1001895 opened by Peter Jacobi From djc at object-craft.com.au Wed Aug 4 06:57:29 2004 From: djc at object-craft.com.au (Dave Cole) Date: Wed Aug 4 06:57:34 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <41106B50.2000201@v.loewis.de> References: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz> <41106B50.2000201@v.loewis.de> Message-ID: <41106CB9.8000302@object-craft.com.au> Martin v. L?wis wrote: > Greg Ewing wrote: > >> Having to explicitly slice off the part you want to unpack not only >> seems inefficient (constructing an intermediate list or tuple just to >> immediately unpack it and throw it away) it also tends to obfuscate >> what is going on. > > > Of course, the proposed syntax does not change this. Compare > > a,b,c = foo()[:3] That is not really the use case for the feature. What I would really like is a nice way to avoid doing this: >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] >>> for tmp in lol: ... a, b = tmp[:2] ... c = tmp[2:] I think that the original is a nicer looking syntax, and is fairly consistent with the existing "varargs" handling in function signatures. >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] >>> for a, b, *c in lol: ... > to > > a,b,c,*_ = foo() > > In either case, the extra fields obfuscate things, and in either > case, a second tuple is created. In the latter case, the tuple > is even stored in a variable. -- http://www.object-craft.com.au From greg at cosc.canterbury.ac.nz Wed Aug 4 07:03:31 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 4 07:03:36 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <41106CB9.8000302@object-craft.com.au> Message-ID: <200408040503.i7453VQE030085@cosc353.cosc.canterbury.ac.nz> > That is not really the use case for the feature. What I would really > like is a nice way to avoid doing this: > > >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] > >>> for tmp in lol: > ... a, b = tmp[:2] > ... c = tmp[2:] Maybe the case of not caring about the rest could be addressed explicitly, e.g. a, b, c, ... = stuff Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tdelaney at avaya.com Wed Aug 4 07:07:42 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Wed Aug 4 07:07:48 2004 Subject: [Python-Dev] Tuple/list assignment question Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C000@au3010avexu1.global.avaya.com> Dave Cole wrote: > Is there any reason why something like this would not be a good idea? > > >>> a_list = [1, 2, 3, 4, 5] > >>> a, b, *c = a_list > > You could then do things like this: > > >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]] > >>> for a, b *c in lol: > ... This proposal has come up several times before (for the record, I like it). I believe there was a particular reason it's never gone forward, but I can't remember what it was. Unfortunately, it's not an easy thing to search for in the c.l.py archives :( I think it's about time this proposal got a PEP of its own ... something like "Enhanced Tuple Unpacking" (to match "Enhanced Argument Tuples" ... Tim Delaney From florian.proff.schulze at gmx.net Wed Aug 4 09:15:56 2004 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Wed Aug 4 09:12:03 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> <200408030351.i733pcO29560@guido.python.org> <1091556102.7597.26.camel@localhost> Message-ID: On Tue, 03 Aug 2004 14:01:42 -0400, Barry Warsaw wrote: > For longer functions, it's a clear win; the '@' stands out nicely, > especially with the python-mode.el font-locking support I just added. :) > It certainly exhibits all the benefits that motivated adding special > syntax for decorators in the first place. I think this is also a strong point for the '@' syntax. How should an editor know that a list before a method is a decoration and should be highlighted differently? The backwards compatibility to python < 2.4 doesn't count in my opinion, because other features like generators and list-comprehensions also can't be used in older pythons, and the old syntax with the decoration after the method body is still available, so this is no problem. +1 for the pie syntax from my little voice as a user of python Regards, Florian Schulze From martin at v.loewis.de Wed Aug 4 09:15:23 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 4 09:15:21 2004 Subject: [Python-Dev] Cygwin and sockets Message-ID: <41108D0B.4020003@v.loewis.de> We have a report (#1001857) that the socket module does not build on Cygwin. Can anybody confirm that? Regards, Martin From t-meyer at ihug.co.nz Wed Aug 4 09:42:30 2004 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Wed Aug 4 09:42:37 2004 Subject: [Python-Dev] Cygwin and sockets In-Reply-To: Message-ID: > We have a report (#1001857) that the socket module does not > build on Cygwin. Can anybody confirm that? By "anybody", did you mean anybody that you know & trust? <0.5 wink>. I can duplicate the process outlined in the bug report and get the same failure to compile (I've added a note to it). However, I don't know much more than that! (If I even use python in cygwin I typically use the distributed binary). =Tony Meyer From martin at v.loewis.de Wed Aug 4 11:22:02 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 4 11:22:01 2004 Subject: [Python-Dev] Cygwin and sockets In-Reply-To: References: Message-ID: <4110AABA.80407@v.loewis.de> Tony Meyer wrote: > By "anybody", did you mean anybody that you know & trust? <0.5 wink>. > > I can duplicate the process outlined in the bug report and get the same > failure to compile (I've added a note to it). This is good enough for the moment, although I would have preferred an answer like "I know, and I have a fix sitting on my harddisk" :-) Regards, Martin From mwh at python.net Wed Aug 4 12:29:48 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 4 12:29:50 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <410FC9FE.6060305@divmod.com> (Jp Calderone's message of "Tue, 03 Aug 2004 13:23:10 -0400") References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> <200408031404.i73E4cS30629@guido.python.org> <410FB7F4.2070309@divmod.com> <200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> Message-ID: <2macxb9owz.fsf@starship.python.net> Jp Calderone writes: > How many features that everyone hates can Python support? It's disingenuous to suggest that this or any other feature of Python is one "everyone hates". Hey, I am even quite happy with ''.join, another source of controversy that seems to have failed to completely halt the adoption of Python . Cheers, mwh -- If your telephone company installs a system in the woods with no one around to see them, do they still get it wrong? -- Robert Moir, alt.sysadmin.recovery From mwh at python.net Wed Aug 4 12:31:37 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 4 12:31:39 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net> (James Y. Knight's message of "Tue, 3 Aug 2004 15:08:43 -0400") References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> <200408031440.i73Ee6V30875@guido.python.org> <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <2m657z9oty.fsf@starship.python.net> James Y Knight writes: > Now I'm damn sure I'd never write code that says > except FooError and f() or MyErrors[5]: > because that would be insane, I think class C(random.choice([dict, list])): pass is my favourite example of this :-) Cheers, mwh -- That being done, all you have to do next is call free() slightly less often than malloc(). You may want to examine the Solaris system libraries for a particularly ambitious implementation of this technique. -- Eric O'Dell, comp.lang.dylan (& x-posts) From mwh at python.net Wed Aug 4 12:38:58 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 4 12:39:00 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <200408032014.i73KEwp31876@guido.python.org> (Guido van Rossum's message of "Tue, 03 Aug 2004 13:14:58 -0700") References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> Message-ID: <2m1xin9ohp.fsf@starship.python.net> Guido van Rossum writes: >> P.S. - Command line switch for command line switch :-), a switch for >> fully turning on the newer type system would be more productive than >> `-U', and put some pressure for refreshening the library in this area. >> Just curious, as I do not intend to volunteer in this area, is there >> something else than Exception in the Python internals that rely on >> old-style classes? > > Probably not, but making Exception a new-style class won't be easy. What makes you say that? I've just been remarking on comp.lang.python how having Exception be new-style in PyPy -- indeed, not having old-style classes and all -- has caused essentially no problems at all. Perhaps I'll work up a patch sometime and see what breaks. Cheers, mwh -- > Emacs is a fashion statement. No, Gnus is a fashion statement. Emacs is clothing. Everyone else is running around naked. -- Karl Kleinpaste & Jonadab the Unsightly One, gnu.emacs.gnus From mwh at python.net Wed Aug 4 12:53:57 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 4 12:53:58 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312 In-Reply-To: <410FC78C.4010609@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Tue, 03 Aug 2004 19:12:44 +0200") References: <200408031418.i73EIFR30699@guido.python.org> <410FA073.5070105@v.loewis.de> <200408031434.i73EYvv30846@guido.python.org> <410FB12C.4010105@v.loewis.de> <2misc09po1.fsf@starship.python.net> <410FC78C.4010609@v.loewis.de> Message-ID: <2mu0vj898a.fsf@starship.python.net> "Martin v. L?wis" writes: >> I'd be happy with any other (I mentioned not doing >> refcounting on dictobject.c's dummy; maybe I should actually see if >> that's possible :-). > > Feel free to change the marshalling to try sharing all strings; > it might be easier to revert the change to compile.c, though. I've reverted the change. I just I hope I remember this next time I run regrtest.py with -R... Cheers, mwh -- Clue: You've got the appropriate amount of hostility for the Monastery, however you are metaphorically getting out of the safari jeep and kicking the lions. -- coonec -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From jim at zope.com Wed Aug 4 15:46:55 2004 From: jim at zope.com (Jim Fulton) Date: Wed Aug 4 15:46:59 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> Message-ID: <4110E8CF.6070900@zope.com> Terry Reedy wrote: ... > Decorators will add complexity, regardless of syntax. Especially in their current over-powered form. If decorators didn't have to take arguments, then you could use a syntax like: def classmethod foo(cls, ...): ... This does add complexity, but I think it enhances understandability. I understand why some people want to be able to pass arguments to decorators, but frankly, I find many of the examples of doing so down right scary. > An advantage of the > @ syntax is than it makes the complexiity separable and ignorable for > someone learning/teaching basic Python, I don't agree that they are ignorable. People will see them in code and will *have* to understand what they mean. Given sme examples I've seen here, this will sometimes be a significant chalenge. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jim at zope.com Wed Aug 4 15:53:37 2004 From: jim at zope.com (Jim Fulton) Date: Wed Aug 4 15:53:40 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com> References: <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <41065006.6020607@zope.com> <200407271445.i6REjEK24454@guido.python.org> <41067660.2070109@zope.com> <1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com> <20040727164913.GS7708@epoch.metaslash.com> <41068C2F.40102@zope.com> <200407272050.i6RKoEY25164@guido.python.org> <4107870C.1070404@zope.com> <5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com> <5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com> <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com> Message-ID: <4110EA61.4010003@zope.com> Phillip J. Eby wrote: > At 10:01 AM 8/2/04 -0400, Jim Fulton wrote: > ... > The point of my > proposal is to make it possible *without* rewriting imports, in versions > of Python from 2.2 up. Your proposal would, at least, require canges to the pickle framework. I imagine it would affect other serialization and persistence frameworks as well. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From andrewm at object-craft.com.au Wed Aug 4 16:04:42 2004 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Wed Aug 4 16:04:37 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <4110E8CF.6070900@zope.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> Message-ID: <20040804140442.89F2B3C7A0@coffee.object-craft.com.au> > > An advantage of the >> @ syntax is than it makes the complexiity separable and ignorable for >> someone learning/teaching basic Python, > >I don't agree that they are ignorable. People will see them in code >and will *have* to understand what they mean. Given sme examples >I've seen here, this will sometimes be a significant chalenge. There are a number of "advanced" features in python that could be a barrier to understanding for new users, but which are rarely seen in practice - metaclasses being the classic example. I imagine these features are rarely seen because most people don't understand them, and those that do have the good taste not to use them gratuitously. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From astrand at lysator.liu.se Wed Aug 4 16:05:22 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Wed Aug 4 16:10:43 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: <20040803213121.C9777@ActiveState.com> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> Message-ID: > ProcessProxy: > ...a behemoth using a thread for each of stdin/stdout/stderr to > allow the user to get an event-like IO interface. Why not avoid threads on POSIX systems, and use select instead? My module does, although it does not provide an event-like IO interface. If you get rid of the threads, then you don't need the workaround code for Linux. > Some of the large code size is _for_ execution through the shell. :) I > think that execution via the shell should be a feature of a module like > this (so that users can use some shell features) and I even think that > it should be the default (so that, for example, Windows users don't have > to learn what cmd.exe or command.com are to run "dir"). However, I > absolutely agree that one should be able to run withOUT the shell (i.e. > have an option for this). You're right. My module should probably have an option for invoking through the shell, or at least document how to do it. I really don't want it as default, though. > Other reasons for the size/complexity of my process.py over yours: > - My process objects have a .kill() method -- which is actually quite a > pain on Windows. True. I guess my module would benefit from such a method as well. > - My module includes some handling that for subsystem:windows vs. > subsystem:console apps on Windows that I don't think yours does. Can you describe why this is needed/useful? > > * The modules uses destructors, which I'm usually avoiding. > > Is that evil? Destructors interferes with the GC. Someone else can probably fill in the details. > concensus. In the meantime, if you'd be willing to change your module's > name to something other than process.py I think that would help > discussions. (I don't know if that is a pain for you at this point. You > mentioned "subprocess". I can change, but I'd like more feedback before that. No-one has told me their opinion on the name "subprocess", for example, not even you :-) >Alternatively, how about "posixprocess"? Though, > despite PEP 324's title, I don't know if that is completely accurate > anymore.) Oh, I've forgotten to change the title. Yes, this is wrong, because the module certainly aims to work on non-POSIX systems as well (read: Windows). /Peter ?strand From guido at python.org Wed Aug 4 16:18:55 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 4 16:19:01 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: Your message of "Wed, 04 Aug 2004 17:03:31 +1200." <200408040503.i7453VQE030085@cosc353.cosc.canterbury.ac.nz> References: <200408040503.i7453VQE030085@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408041418.i74EIt001130@guido.python.org> For the record, let me just say that I'm -1 on adding this feature now. It's only a minor convenience while it's a fairly big change to the parser and code generator, but most of all, Python's growth needs to be reduced. If we were to add every "nice" feature that was invented (or present in other languages), Python would lose one of its main charms, which is that it is really a rather simple language that can be learned and put to production quickly. So while I appreciate the idea (which BTW has been proposed before) I think it's not worth adding now, and if you don't hear from me again on this topic it's because I haven't changed my mind... --Guido van Rossum (home page: http://www.python.org/~guido/) From jim at zope.com Wed Aug 4 16:31:57 2004 From: jim at zope.com (Jim Fulton) Date: Wed Aug 4 16:32:01 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <20040804140442.89F2B3C7A0@coffee.object-craft.com.au> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> <20040804140442.89F2B3C7A0@coffee.object-craft.com.au> Message-ID: <4110F35D.8020904@zope.com> Andrew McNamara wrote: >>>An advantage of the >>>@ syntax is than it makes the complexiity separable and ignorable for >>>someone learning/teaching basic Python, >> >>I don't agree that they are ignorable. People will see them in code >>and will *have* to understand what they mean. Given sme examples >>I've seen here, this will sometimes be a significant chalenge. > > > There are a number of "advanced" features in python that could be a > barrier to understanding for new users, but which are rarely seen in > practice - metaclasses being the classic example. > > I imagine these features are rarely seen because most people don't > understand them, and those that do have the good taste not to use them > gratuitously. This feature won't be rarely used. Class and static methods and properties are created fairly often, certainly far more often than meta classes. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From guido at python.org Wed Aug 4 16:34:38 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 4 16:34:45 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: Your message of "Wed, 04 Aug 2004 11:38:58 BST." <2m1xin9ohp.fsf@starship.python.net> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> <2m1xin9ohp.fsf@starship.python.net> Message-ID: <200408041434.i74EYdU01260@guido.python.org> > > Probably not, but making Exception a new-style class won't be easy. > > What makes you say that? I've just been remarking on > comp.lang.python how having Exception be new-style in PyPy -- > indeed, not having old-style classes and all -- has caused > essentially no problems at all. I believe that -- it's more that the existing infrastructure that creates the standard exception hierarchy isn't easily adapted. I also believe there's a conceptual problem with defining when something is an acceptable argument to 'raise' -- unless we insist that exceptions inherit from a designated base class, *every* object is acceptable, because if it isn't a class, it's an instance of a class, and raise allows either. I don't really think that "raise 42" ought to be acceptable, but I don't know how to prevent it without requiring a specific base class (excluding a whole slew of specific base classes seems wrong). Maybe we can accept old-style classes and instances, strings, and instances of Exception and its subclasses. But then we better be sure that we really want to insist on subclassing from Exception, because that's rather unpythonic. > Perhaps I'll work up a patch sometime and see what breaks. Please do! --Guido van Rossum (home page: http://www.python.org/~guido/) From bob at redivi.com Wed Aug 4 16:50:24 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Aug 4 16:50:30 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <4110E8CF.6070900@zope.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> Message-ID: <9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com> On Aug 4, 2004, at 9:46 AM, Jim Fulton wrote: > Terry Reedy wrote: > > ... > >> Decorators will add complexity, regardless of syntax. > > Especially in their current over-powered form. If decorators didn't > have to > take arguments, then you could use a syntax like: > > def classmethod foo(cls, ...): > ... > > This does add complexity, but I think it enhances understandability. > > I understand why some people want to be able to pass arguments to > decorators, > but frankly, I find many of the examples of doing so down right scary. All of the important use cases I've found and seen for decorators have arguments. PyObjC and ctypes (possibly Jython, IronPython) will use them to specify the input and output argument types for a function (integers, pointers, etc.). PJE has cooked up some interesting stuff using decorators to do generic functions. I use these sorts of things far more often than classmethod and staticmethod (or the like). I can also imagine using them for something like decorating functions with four character suite and event codes for responding to AppleEvents in Mac apps. I wouldn't really care for decorators at all if I couldn't use them to these sorts of things. Your proposal would make decorators nearly as bad as the current pre-2.4 situation. This is REALLY ugly: foo = decorator(....) @foo def fun(....): ... -bob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3589 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/ec2570d1/smime.bin From jim at zope.com Wed Aug 4 16:52:25 2004 From: jim at zope.com (Jim Fulton) Date: Wed Aug 4 16:52:29 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax Message-ID: <4110F829.2000609@zope.com> IMO, the most common uses of decorators will be to define properties, and class and static methods. IMO, these uses would be better served by a simpler syntax: def classmethod foo(cls, ...): ... This simplified syntax only allows names to specify decorators. It could allow multiple names, although I'm not sure it should, I find this *far* more readable and obvious than any of the other syntaxs I've seen propsed. Those applications that *need* decorator arguments could use the more complex pie-shaped notation. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From chrism at plope.com Wed Aug 4 16:57:33 2004 From: chrism at plope.com (Chris McDonough) Date: Wed Aug 4 16:57:37 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> Message-ID: <1091631453.26267.390.camel@athlon> On Wed, 2004-08-04 at 10:05, Peter Astrand wrote: > > ProcessProxy: > > ...a behemoth using a thread for each of stdin/stdout/stderr to > > allow the user to get an event-like IO interface. > > Why not avoid threads on POSIX systems, and use select instead? My module > does, although it does not provide an event-like IO interface. If you get > rid of the threads, then you don't need the workaround code for Linux. Doesn't select() effectively busy-wait on "real" files (pipes and file descriptors obtained via open(), as opposed to network sockets) on most (all?) UNIXen? At least this has been my finding under Linux. (See http://www.plope.com/Members/chrism/News_Item.2004-07-29.4755190458/talkback/1091579983/discussionitem_view for more info). - C From jim at zope.com Wed Aug 4 17:00:37 2004 From: jim at zope.com (Jim Fulton) Date: Wed Aug 4 17:00:40 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> <9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com> Message-ID: <4110FA15.2040808@zope.com> Bob Ippolito wrote: > ... > Your proposal would make decorators nearly as > bad as the current pre-2.4 situation. This is REALLY ugly: > > foo = decorator(....) > @foo > def fun(....): > ... I agree, that's really ugly. I wasn't proposing that. I proposed not allowing decorator arguments. Without decorator arguments, a much simpler syntax can be used, as in: def classmethod foo(...): ... Or, alternatively, I'm proposing allowing the above simpler syntax when arguments are not needed and allowing the pie syntax to handle more complex cases. The original motivation for decirators was to deal with things like properties and specialized methods. I'd like to see these cases handled well. I think these are going to be the most common cases and the cases that new Python programmers are most likely to come accross. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From fdrake at acm.org Wed Aug 4 17:00:34 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Aug 4 17:00:45 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: <4110F829.2000609@zope.com> References: <4110F829.2000609@zope.com> Message-ID: <200408041100.34439.fdrake@acm.org> On Wednesday 04 August 2004 10:52 am, Jim Fulton wrote: > IMO, the most common uses of decorators will be to define properties, > and class and static methods. IMO, these uses would be better served > by a simpler syntax: > > def classmethod foo(cls, ...): > ... This was rejected a long time ago because it complicated life for editor colorizing support and many similar tools. It especially complicates the creation of ad-hoc tools, and breaks ones that are already working. While pie-notation isn't my favorite, it's reasonable enough. The example @classmethod def makeAnother(cls): return cls("magic", 42) seems readable enough to me. -Fred -- Fred L. Drake, Jr. From barry at python.org Wed Aug 4 17:07:52 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 4 17:07:46 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <20040804010712.GA10819@mems-exchange.org> References: <410FC9FE.6060305@divmod.com> <20040804010712.GA10819@mems-exchange.org> Message-ID: <1091632072.8324.72.camel@localhost> On Tue, 2004-08-03 at 21:07, Neil Schemenauer wrote: > That's not apparent to me. It seems to me that most people just got > sick of the endless debate and tuned out. The people who had urgent > need for the feature were probably a bit more persistent. > Personally, I don't see the cost vs. benefit of the feature to be > compelling. It seems there is not a lot you can do with a decorator > without resorting to ugly hacks (e.g. messing with bytecode). I disagree. I've written a lot of code that uses pre-pie decorators, but it's all fairly localized. It isn't spread out in every module, but it's contained, e.g., within a database package (wrap mutators in transaction handling decorators). Those use my own custom decorator, but in almost all other examples (in my own code -- I know others have different use cases), the decorators are built-ins like property, staticmethod, etc. For me, the benefit is significant because while I would still continue to use this particular feature, I think the pie decorator syntax makes the code /more/ readable, or more readable in the right way and in the right place. In general, I predict most Python code will continue to be blissfully unadorned with decorators. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/2436bbc3/attachment-0001.pgp From barry at python.org Wed Aug 4 17:12:02 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 4 17:11:55 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <4110E8CF.6070900@zope.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com> <200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com> <200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> Message-ID: <1091632322.8323.81.camel@localhost> On Wed, 2004-08-04 at 09:46, Jim Fulton wrote: > I don't agree that they are ignorable. People will see them in code > and will *have* to understand what they mean. Given sme examples > I've seen here, this will sometimes be a significant chalenge. I think most of the cognitive challenge for decorators will be in understanding the side-effects of the decorator, not in understanding the syntax for wrapping a function in a decorator. That won't change by using some other (or no new) syntax -- the Python 2.3. spelling would be just as hairy. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/9776219e/attachment.pgp From guido at python.org Wed Aug 4 17:15:58 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 4 17:16:03 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: Your message of "Wed, 04 Aug 2004 10:52:25 EDT." <4110F829.2000609@zope.com> References: <4110F829.2000609@zope.com> Message-ID: <200408041515.i74FFxl01426@guido.python.org> Jim, you should've argued this months ago. Everything you bring up has already been debated endlessly, and I really don't feel like regurgitating the same answer again. Please stop. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Aug 4 17:20:48 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 4 17:20:53 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: Your message of "Wed, 04 Aug 2004 10:57:33 EDT." <1091631453.26267.390.camel@athlon> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> Message-ID: <200408041520.i74FKm001469@guido.python.org> > Doesn't select() effectively busy-wait on "real" files (pipes and file > descriptors obtained via open(), as opposed to network sockets) on most > (all?) UNIXen? At least this has been my finding under Linux. select() doesn't make sense for regular files, so tail -f can't use it. For Unix pipes, it works as advertised. On Windows, it *only* applies to sockets. --Guido van Rossum (home page: http://www.python.org/~guido/) From FBatista at uniFON.com.ar Wed Aug 4 17:17:38 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Wed Aug 4 17:21:27 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators Message-ID: Didn't followed closely this discussion, but my general feeling is that the development community position is not clear regarding this feature. Python is what it is now because of well thinked changes. I don't want, because of the hurry of 2.4, to include this particular feature without a hard approval of the community. Will be easiest to include it in 2.5 than to extract it. So, all that said, I'm +1 to take this out from 2.4. . Facundo #- -----Mensaje original----- #- De: Jim Fulton [mailto:jim@zope.com] #- Enviado el: Miercoles, 04 de Agosto de 2004 11:32 #- Para: Andrew McNamara #- CC: python-dev@python.org #- Asunto: Re: [Python-Dev] Re: 2.4a2, and @decorators #- #- #- Andrew McNamara wrote: #- >>>An advantage of the #- >>>@ syntax is than it makes the complexiity separable and #- ignorable for #- >>>someone learning/teaching basic Python, #- >> #- >>I don't agree that they are ignorable. People will see #- them in code #- >>and will *have* to understand what they mean. Given sme examples #- >>I've seen here, this will sometimes be a significant chalenge. #- > #- > #- > There are a number of "advanced" features in python that could be a #- > barrier to understanding for new users, but which are #- rarely seen in #- > practice - metaclasses being the classic example. #- > #- > I imagine these features are rarely seen because most people don't #- > understand them, and those that do have the good taste not #- to use them #- > gratuitously. #- #- This feature won't be rarely used. Class and static methods #- and properties are created fairly often, certainly far more often #- than meta classes. #- #- Jim #- #- -- #- Jim Fulton mailto:jim@zope.com Python Powered! #- CTO (540) 361-1714 http://www.python.org #- Zope Corporation http://www.zope.com http://www.zope.org #- _______________________________________________ #- Python-Dev mailing list #- Python-Dev@python.org #- http://mail.python.org/mailman/listinfo/python-dev #- Unsubscribe: #- http://mail.python.org/mailman/options/python-dev/fbatista%40 unifon.com.ar From guido at python.org Wed Aug 4 17:22:20 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 4 17:22:24 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: Your message of "Wed, 04 Aug 2004 11:07:52 EDT." <1091632072.8324.72.camel@localhost> References: <410FC9FE.6060305@divmod.com> <20040804010712.GA10819@mems-exchange.org> <1091632072.8324.72.camel@localhost> Message-ID: <200408041522.i74FMKW01491@guido.python.org> > In general, I predict most Python code will continue to be > blissfully unadorned with decorators. Right. Outside the test suite, there are very few uses of classmethod, staticmethod and property in the standard library. --Guido van Rossum (home page: http://www.python.org/~guido/) From bob at redivi.com Wed Aug 4 17:45:17 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Aug 4 17:45:26 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <4110FA15.2040808@zope.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> <9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com> <4110FA15.2040808@zope.com> Message-ID: <48F450D5-E62D-11D8-B64E-000A95686CD8@redivi.com> On Aug 4, 2004, at 11:00 AM, Jim Fulton wrote: > Bob Ippolito wrote: > ... > > > Your proposal would make decorators nearly as >> bad as the current pre-2.4 situation. This is REALLY ugly: >> foo = decorator(....) >> @foo >> def fun(....): >> ... > > I agree, that's really ugly. I wasn't proposing that. > > I proposed not allowing decorator arguments. Without decorator > arguments, > a much simpler syntax can be used, as in: > > def classmethod foo(...): > ... > > Or, alternatively, I'm proposing allowing the above simpler syntax > when arguments are not needed and allowing the pie syntax to handle > more complex cases. > > The original motivation for decirators was to deal with things like > properties and specialized methods. I'd like to see these cases > handled > well. I think these are going to be the most common cases and the > cases > that new Python programmers are most likely to come accross. Adding even more syntax sounds like a horrible idea (along with the fact that this particular syntax is hard to syntax highlight, will really confuse lots of existing tools, etc.). I also don't agree that those are the most common cases. I certainly haven't seen a whole lot of staticmethod/classmethod usage, and property is only useful for writing getters (with decorator syntax), so I don't think that will be used very often either. I think that decorators will be used about 98% for domain specific purposes with non-standard decorators and 2% will be uses of decorators that come in the standard library such as staticmethod, classmethod, and property (for writing getters). Anyway, it's probably too late to change anyone's mind. Decorators are in @as-is, and I don't think anything you can say now is going to change that. -bob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3589 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/a95fb78f/smime.bin From janssen at parc.com Wed Aug 4 17:47:32 2004 From: janssen at parc.com (Bill Janssen) Date: Wed Aug 4 17:48:03 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: Your message of "Wed, 04 Aug 2004 08:00:34 PDT." <200408041100.34439.fdrake@acm.org> Message-ID: <04Aug4.084736pdt."58612"@synergy1.parc.xerox.com> > > def classmethod foo(cls, ...): > > ... > > This was rejected a long time ago because it complicated life for editor > colorizing support Oh, Fred, please! "Complicated life"? (and, "Editor colorizing support"?!) Much as I love python-mode, it hardly seems a good reason to add a whole new syntactic branch to Python, particularly when editors are really good at identifying a small set of distinguished words in distinguished positions. By the way, if we're going down the road of compile-time markup, how about a real macro language :-? Bill From heikowu at ceosg.de Wed Aug 4 18:09:54 2004 From: heikowu at ceosg.de (Heiko Wundram) Date: Wed Aug 4 18:09:29 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: References: Message-ID: <200408041809.54164.heikowu@ceosg.de> Am Mittwoch, 4. August 2004 17:17 schrieb Batista, Facundo: > So, all that said, I'm +1 to take this out from 2.4. -1000 to take it out from 2.4... And +1 on Guido's intuition for choosing the @ syntax (it goes easily for me). I'd love to see something of the following form: class x: synchronized = threading.Synchronizer() @synchronized def y(self): When's threading.Synchronizer coming (just a threading.(R)Lock with an extra __call__ which prepares a method for synchronization with this lock)? I already have some patches which implement module/class/instance locking using just a simple RLock and the decorator syntax, and I'd gladly sign over the patches to the PSF. ;) Heiko. From pje at telecommunity.com Wed Aug 4 18:16:37 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 4 18:12:33 2004 Subject: [Python-Dev] Fix import errors to have data In-Reply-To: <4110EA61.4010003@zope.com> References: <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com> <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <41065006.6020607@zope.com> <200407271445.i6REjEK24454@guido.python.org> <41067660.2070109@zope.com> <1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com> <20040727164913.GS7708@epoch.metaslash.com> <41068C2F.40102@zope.com> <200407272050.i6RKoEY25164@guido.python.org> <4107870C.1070404@zope.com> <5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com> <5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com> <200407282213.i6SMDWD06878@guido.python.org> <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com> <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com> <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040804115739.02c88050@mail.telecommunity.com> At 09:53 AM 8/4/04 -0400, Jim Fulton wrote: >Phillip J. Eby wrote: >>At 10:01 AM 8/2/04 -0400, Jim Fulton wrote: > >.. > >>The point of my proposal is to make it possible *without* rewriting >>imports, in versions of Python from 2.2 up. > >Your proposal would, at least, require canges to the pickle framework. >I imagine it would affect other serialization and persistence frameworks >as well. Actually, in the updated interface I've been discussing off-list with a few people, all that would happen is that you'd need to be explicit about what interpreter you use to do the unpickling, if you don't want it to be the current one. E.g.: result = someInterpreter.call(somePickler.loads, data) In a framework like ZODB, however, I would assume that if supporting multiple interpreters were desirable, one would change the persistent object ID to include a reference to the {bundle|parcel|product} that originated the object. In any case, the implementation we're discussing (based on the C multiple interpreter API) should be completely transparent to end user code, to the extent that it should be possible to run two different versions of Zope side by side in the same process -- and that will probably make a nice "stress test" for the implementation when we're done with it. :) I hope that the end-product of our work will be an implementation of a Python-accessible API for interpreter management, and a PEP covering both that API and how to fix the most problematic aspects of the Python C API for multiple interpreters. (Specifically, additions to the API to allow extensions to manage selected static data as interpreter-specific rather than global, and for ensuring that imports stay interpreter-local when execution crosses interpreter boundaries.) I expect the fixes will be able to be used with versions of Python back to 2.2, which was the version that introduced the necessary base APIs for walking the interpreter and thread state chains. From pje at telecommunity.com Wed Aug 4 18:22:51 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 4 18:18:48 2004 Subject: [Python-Dev] Exception and new-style classes In-Reply-To: <200408041434.i74EYdU01260@guido.python.org> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> <2m1xin9ohp.fsf@starship.python.net> Message-ID: <5.1.1.6.0.20040804121826.02d0c110@mail.telecommunity.com> At 07:34 AM 8/4/04 -0700, Guido van Rossum wrote: > > > Probably not, but making Exception a new-style class won't be easy. > > > > What makes you say that? I've just been remarking on > > comp.lang.python how having Exception be new-style in PyPy -- > > indeed, not having old-style classes and all -- has caused > > essentially no problems at all. > >I believe that -- it's more that the existing infrastructure that >creates the standard exception hierarchy isn't easily adapted. > >I also believe there's a conceptual problem with defining when >something is an acceptable argument to 'raise' -- unless we insist >that exceptions inherit from a designated base class, *every* object >is acceptable, because if it isn't a class, it's an instance of a >class, and raise allows either. I don't really think that "raise 42" >ought to be acceptable, but I don't know how to prevent it without >requiring a specific base class (excluding a whole slew of specific >base classes seems wrong). > >Maybe we can accept old-style classes and instances, strings, and >instances of Exception and its subclasses. But then we better be sure >that we really want to insist on subclassing from Exception, because >that's rather unpythonic. I thought that was already the plan; I seem to recall dire warnings that it was going to be required someday. In any case, I had my eye on "fixing" this for next bug day (there's a SF bug # for it, that I don't recall at the moment). My plan was to allow any object that was an instance of Exception, even if it was new-style. In other words, new-style exceptions would have to include Exception in their base classes. Old-style exceptions wouldn't have to meet that requirement, for backward compatibility purposes. I assumed that the old-style (and string) compatibility would need to remain until 3.0. From astrand at lysator.liu.se Wed Aug 4 18:25:36 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Wed Aug 4 18:25:46 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: <1091631453.26267.390.camel@athlon> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> Message-ID: > > Why not avoid threads on POSIX systems, and use select instead? My module > > does, although it does not provide an event-like IO interface. If you get > > rid of the threads, then you don't need the workaround code for Linux. > > Doesn't select() effectively busy-wait on "real" files (pipes and file > descriptors obtained via open(), as opposed to network sockets) on most > (all?) UNIXen? At least this has been my finding under Linux. Yes, but a solution with threads will have problems as well, since read() and friends will return EOF rather than block when you reach the end. /Peter ?strand From chrism at plope.com Wed Aug 4 18:27:18 2004 From: chrism at plope.com (Chris McDonough) Date: Wed Aug 4 18:27:21 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: <200408041520.i74FKm001469@guido.python.org> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> <200408041520.i74FKm001469@guido.python.org> Message-ID: <1091636838.26267.406.camel@athlon> On Wed, 2004-08-04 at 11:20, Guido van Rossum wrote: > > Doesn't select() effectively busy-wait on "real" files (pipes and file > > descriptors obtained via open(), as opposed to network sockets) on most > > (all?) UNIXen? At least this has been my finding under Linux. > > select() doesn't make sense for regular files, so tail -f can't use > it. For Unix pipes, it works as advertised. On Windows, it *only* > applies to sockets. Aha. I think I understand now. Sorry for going off-topic, but this program (UNIX-only): import select import errno import fcntl import os def go(r, timeout=1): while 1: try: r, w, x = select.select(r, [], [], timeout) print r except select.error, err: if err[0] != errno.EINTR: raise r = w = x = [] p_in, p_out = os.pipe() for fd in p_in, p_out: flags = fcntl.fcntl(fd, fcntl.F_GETFL) fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NDELAY) os.close(p_in) go([p_out]) ... will busywait in select (p_out is always in the ready state; the select timeout is never reached). But if you comment out the "os.close(p_in)" line, it no longer busywaits (the select timeout is reached on every iteration). At least this is the behavior under Linux. This is a little unfortunate because the normal dance when communicating between parent and child in order to capture the output of a child process seems to be: 1) In the parent process, create a set of pipes that will represent stdin/stdout/stderr of the child. 2) fork 3) In the parent process, close the "unused" fds representing the ends of the pipes that the child will use as fds for stdin/stderr/stdout. 4) In the child process, dup these same fds to stdin, stdout, and stderr. ... so select seems to be unuseful here, unless step 3 isn't actually necessary. Still a bit confused and researching... - C From pje at telecommunity.com Wed Aug 4 18:36:09 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 4 18:32:05 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <200408041809.54164.heikowu@ceosg.de> References: Message-ID: <5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com> At 06:09 PM 8/4/04 +0200, Heiko Wundram wrote: >Am Mittwoch, 4. August 2004 17:17 schrieb Batista, Facundo: > > So, all that said, I'm +1 to take this out from 2.4. > >-1000 to take it out from 2.4... And +1 on Guido's intuition for choosing the >@ syntax (it goes easily for me). I'd love to see something of the following >form: > >class x: > synchronized = threading.Synchronizer() > > @synchronized > def y(self): > > >When's threading.Synchronizer coming (just a threading.(R)Lock with an extra >__call__ which prepares a method for synchronization with this lock)? I >already have some patches which implement module/class/instance locking using >just a simple RLock and the decorator syntax, and I'd gladly sign over the >patches to the PSF. ;) Note that your example, if I understand it correctly, creates a single lock for all instances of class 'x', rather than for individual instances of 'x'. This is not what I'd normally expect from a 'synchronized' method. From amk at amk.ca Wed Aug 4 18:35:45 2004 From: amk at amk.ca (A.M. Kuchling) Date: Wed Aug 4 18:35:55 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <200408041809.54164.heikowu@ceosg.de> References: <200408041809.54164.heikowu@ceosg.de> Message-ID: <20040804163545.GB21551@rogue.amk.ca> On Wed, Aug 04, 2004 at 06:09:54PM +0200, Heiko Wundram wrote: > When's threading.Synchronizer coming (just a threading.(R)Lock with an extra > __call__ which prepares a method for synchronization with this lock)? I > already have some patches which implement module/class/instance locking using There was a python-dev thread about useful decorators, starting here: http://mail.python.org/pipermail/python-dev/2004-April/043957.html I expect 2.4alpha3 will see the addition of a 'decorators' module. --amk From jacobs at theopalgroup.com Wed Aug 4 18:35:56 2004 From: jacobs at theopalgroup.com (Kevin Jacobs) Date: Wed Aug 4 18:35:59 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com> References: <5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com> Message-ID: <4111106C.4020300@theopalgroup.com> Phillip J. Eby wrote: > At 06:09 PM 8/4/04 +0200, Heiko Wundram wrote: > >> class x: >> synchronized = threading.Synchronizer() >> >> @synchronized >> def y(self): >> >> >> When's threading.Synchronizer coming (just a threading.(R)Lock with >> an extra >> __call__ which prepares a method for synchronization with this lock)? I >> already have some patches which implement module/class/instance >> locking using >> just a simple RLock and the decorator syntax, and I'd gladly sign >> over the >> patches to the PSF. ;) > > > Note that your example, if I understand it correctly, creates a single > lock for all instances of class 'x', rather than for individual > instances of 'x'. This is not what I'd normally expect from a > 'synchronized' method. Indeed it would, if implemented as described. However, one could use the same syntax and implement a Synchronizer that implemented locks at the instance level. Dirty tricks would be involved, but it is possible. (If I _ever_ get some free time, I will take a stab at doing it, too. Sounds like a good cookbook recipe) -Kevin From foom at fuhm.net Wed Aug 4 18:39:53 2004 From: foom at fuhm.net (James Y Knight) Date: Wed Aug 4 18:39:57 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: References: <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <200408030105.i7315Fj29310@guido.python.org> <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> <200408031440.i73Ee6V30875@guido.python.org> <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: On Aug 3, 2004, at 4:22 PM, David Eppstein wrote: > I think I was the one who posted the lambda comment. > I also posted in a different message a use-case for decorators that had > a line with two lambdas in it, so obviously I don't hate lambdas. Ah, I'm sorry then. I have noticed many people seem to be quite biased against lambdas here, so I just assumed. > I'm actually pretty neutral on restricted decorators vs @expression. > But > @lambda(func): body > def func(...): ... > is (I think) much more likely to be an abuse of language than the > Pythonic way of writing something, so I don't see a lot of harm in > preventing it. Oh, I agree, it's quite silly to write a piece of code like that, and if anyone wrote it in code I have to maintain I'd probably be quite irritated. So, if the conclusion to this is "well, we can always unrestrict it later, if someone comes up with a convincing use-case", then it'll likely never be changed, because I doubt that's gonna happen. The best I can do is foo().bar(), which I suspect at least one person will run into at some point and wonder about, then work around instead of presenting it to python-dev. My main argument is and has been simply: that's how everything else in python works, so this should be the same. Last message on this subject, I think everyone got the point by now and is either convinced or not. :) James From heikowu at ceosg.de Wed Aug 4 18:43:11 2004 From: heikowu at ceosg.de (Heiko Wundram) Date: Wed Aug 4 18:42:46 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <4111106C.4020300@theopalgroup.com> References: <5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com> <4111106C.4020300@theopalgroup.com> Message-ID: <200408041843.11994.heikowu@ceosg.de> Am Mittwoch, 4. August 2004 18:35 schrieb Kevin Jacobs: > > Note that your example, if I understand it correctly, creates a single > > lock for all instances of class 'x', rather than for individual > > instances of 'x'. This is not what I'd normally expect from a > > 'synchronized' method. > > Indeed it would, if implemented as described. However, one > could use the same syntax and implement a Synchronizer > that implemented locks at the instance level. Dirty tricks would > be involved, but it is possible. (If I _ever_ get some free time, > I will take a stab at doing it, too. Sounds like a good cookbook > recipe) It's pretty easy to implement a Synchronizer on instance level using the same semantics, as all method calls will receive the self parameter as the first parameter (except on builtin types, but you could also deal with this). I'll upload the patches I did to the sourceforge patch tracker once I get time to do so... Anyway, I think this should be part of the threading module (as it's about threads, not about decorators), and should also be part of the stdlib, as there's lots of things that can go wrong if you try to implement it yourself, and this is useful enough to warrant inclusion in the stdlib... Heiko. From chrism at plope.com Wed Aug 4 18:46:56 2004 From: chrism at plope.com (Chris McDonough) Date: Wed Aug 4 18:47:00 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> Message-ID: <1091638016.26267.418.camel@athlon> On Wed, 2004-08-04 at 12:25, Peter Astrand wrote: > > > Why not avoid threads on POSIX systems, and use select instead? My module > > > does, although it does not provide an event-like IO interface. If you get > > > rid of the threads, then you don't need the workaround code for Linux. > > > > Doesn't select() effectively busy-wait on "real" files (pipes and file > > descriptors obtained via open(), as opposed to network sockets) on most > > (all?) UNIXen? At least this has been my finding under Linux. > > Yes, but a solution with threads will have problems as well, since read() > and friends will return EOF rather than block when you reach the end. As I've found, if the end of the pipes in that represent the child's stderr/stdout fds are closed in the parent, a select() reading the "other" ends of these pipes will busywaut (at least on Linux). Other than than, I think this select() would be the absolute right thing on POSIX rather than using threads or polling. Is there a way around this problem or is it just a fact of life? - C From skip at pobox.com Wed Aug 4 19:26:27 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Aug 4 19:26:53 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: <04Aug4.084736pdt."58612"@synergy1.parc.xerox.com> References: <200408041100.34439.fdrake@acm.org> <04Aug4.084736pdt."58612"@synergy1.parc.xerox.com> Message-ID: <16657.7235.345770.620493@montanaro.dyndns.org> Fred> This was rejected a long time ago because it complicated life for Fred> editor colorizing support Bill> Much as I love python-mode, it hardly seems a good reason to add a Bill> whole new syntactic branch to Python, .. Though X/Emacs is popular within the Unix community, it's hardly the only editor that attempts to colorize Python code: http://www.python.org/cgi-bin/moinmoin/PythonEditors In fact, given the increased use of Python on Windows these days I'm skeptical it's the most widely used editor for Python code. (Not breaking etags/ctags is also a nice-to-have.) Skip From anthony at interlink.com.au Wed Aug 4 20:21:23 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Aug 4 20:21:58 2004 Subject: [Python-Dev] trunk frozen for 2.4a2 Message-ID: <41112923.8060505@interlink.com.au> Please hold off on any checkins to the trunk for the next 24 hours or so (I'll post a followup when we're done) for 2.4a2. (The usual exception for the release team of me, Fred and Martin applies). Once the second alpha is out the door, I plan to put together a list of items that are still pending for 2.4 - any that can't find someone to do the work will have to seriously be considered at risk. At the moment, the major piece of orphaned work is the relative imports and multi-line imports. The only other piece of major unfinished work that springs to mind is the $string interpolation, but there's a bunch of other smaller pieces. I updated the release PEP after the first alpha - it's probably useful for a few folks to scan through this and see if I missed anything. Ta, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From janssen at parc.com Wed Aug 4 20:26:36 2004 From: janssen at parc.com (Bill Janssen) Date: Wed Aug 4 20:28:17 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: Your message of "Wed, 04 Aug 2004 10:26:27 PDT." <16657.7235.345770.620493@montanaro.dyndns.org> Message-ID: <04Aug4.112642pdt."58612"@synergy1.parc.xerox.com> > In fact, given the increased use of Python on Windows these days I'm > skeptical it's the most widely used editor for Python code. Didn't mean to imply that it was, it's just what I use. By the way, I had firmly resolved to keep quiet about this decorators issue, but I lost my willpower there for a minute :0). Bill From trentm at ActiveState.com Wed Aug 4 20:30:56 2004 From: trentm at ActiveState.com (Trent Mick) Date: Wed Aug 4 20:31:06 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: ; from astrand@lysator.liu.se on Wed, Aug 04, 2004 at 04:05:22PM +0200 References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> Message-ID: <20040804113056.H3207@ActiveState.com> [Peter Astrand wrote] > > ProcessProxy: > > ...a behemoth using a thread for each of stdin/stdout/stderr to > > allow the user to get an event-like IO interface. > > Why not avoid threads on POSIX systems, and use select instead? My module > does, although it does not provide an event-like IO interface. Because that wouldn't be cross-platform... perhaps it would be possible though. I am not that experienced with select() -- I have generally eschewed it because I can't use it on Windows as well. > If you get rid of the threads, then you don't need the workaround code > for Linux. Slight misunderstanding there: the separate thread from which you cannot kill a subprocess on Linux is not one of these ProcessProxy threads. I.e. ignoring ProcessProxy the LinuxThreads-bug workaround is still necessary for Process and ProcessOpen for the user that starts a subprocess on their thread-A and wants to kill it on their thread-B. > You're right. My module should probably have an option for invoking > through the shell, or at least document how to do it. I really don't want > it as default, though. I have code to find the shell adequately so I don't see why we couldn't use it. As to making it the default or not: perhaps clear documentation could help avoid the expected "I can't run 'dir'!" newbie complaint, but I don't think it would get all of the them. The current popen*() execute via the shell. The lower-level guys, exec*() and spawn*(), do not. Still up for debate, I guess. :) > > - My module includes some handling that for subsystem:windows vs. > > subsystem:console apps on Windows that I don't think yours does. > > Can you describe why this is needed/useful? A subsystem:windows app (like Komodo, or Idle, or pythonw.exe, or any GUI app) doesn't have a console and hence doesn't have std handles. By default, executing any subsytem:console app (like 'ls' or 'echo' or 'python' or 'perl) from a subsystem:windows app will result in an AllocConsole call (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/allocconsole.asp) that will result in a console dialog popping up. You can see this on some Windows apps that haven't dealt with this issue. In particular, some installers out there will pop up an unprofessional console dialog when running some sub-processes. To avoid this one has to muck around with CreateProcess options. Or, kind of the reverse, if a seeming GUI app *does* have std handles (often redirected to log files, Komodo does this) and it wants to offer the ability to pop up a console (e.g. Komodo's feature to debug in a separate console) then one needs the ability to specify CREATE_NEW_CONSOLE. > I can change, but I'd like more feedback before that. No-one has told me > their opinion on the name "subprocess", for example, not even you :-) Sure, I don't mind "subprocess". I am hoping, though, that we can eventually merge our two modules and just call that one "process". Trent -- Trent Mick TrentM@ActiveState.com From tjreedy at udel.edu Wed Aug 4 20:40:08 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Aug 4 20:40:17 2004 Subject: [Python-Dev] Re: Re: 2.4a2, and @decorators References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> Message-ID: "Jim Fulton" wrote in message news:4110E8CF.6070900@zope.com... > Terry Reedy wrote: > > An advantage of the > > @ syntax is than it makes the complexiity separable and ignorable for > > someone learning/teaching basic Python, > > I don't agree that they are ignorable. Any advanced feature is ignorable is the context I specified -- learning/teaching basic Python. > People will see them in code and will *have* to understand what they mean. Any ignored feature becomes unignorable when one stumbles across it in 'wild' Python and want to really understand. And even with an understanding of the general feature, specific knowledge may also be required -- of the particular module or objects imported or in this case, the particular decorator. But there is currently *lots* of publicly available code without decorators to read, and more will be written. In addition, one will be able to use at least some modules that use them without knowing about them, just as with metaclasses, for instance. My particular point was separability. With @, one will still be able to learn everything about [] and def without hearing the word decorator, and without having to revise one's understanding upon encountering decorators. This was not true of the [deco] and deco within def syntax proposals. It is a minor point, but one that is a plus for me. Terry J. Reedy From jhylton at gmail.com Wed Aug 4 20:45:54 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Wed Aug 4 20:45:56 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <4110E8CF.6070900@zope.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> Message-ID: On Wed, 04 Aug 2004 09:46:55 -0400, Jim Fulton wrote: > Terry Reedy wrote: > > .... > > > Decorators will add complexity, regardless of syntax. > > Especially in their current over-powered form. If decorators didn't have to > take arguments, then you could use a syntax like: > > def classmethod foo(cls, ...): > ... > > This does add complexity, but I think it enhances understandability. > > I understand why some people want to be able to pass arguments to decorators, > but frankly, I find many of the examples of doing so down right scary. I wonder if we should have add a note to the Python style guide that forbids such things from the standard library <0.2 wink>. > > An advantage of the > > @ syntax is than it makes the complexiity separable and ignorable for > > someone learning/teaching basic Python, > > I don't agree that they are ignorable. People will see them in code > and will *have* to understand what they mean. Given sme examples > I've seen here, this will sometimes be a significant chalenge. It certainly depends on the use. I don't think @classmethod will be any more or less confusing than the current practice of calling foo = staticmethod(foo) after the def. I agree that more elaborate uses may be confusing; then again, I suspect that more elaborate code will be confusing regardless of how it uses decorators. Jeremy From astrand at lysator.liu.se Wed Aug 4 21:01:22 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Wed Aug 4 21:01:30 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: <1091636838.26267.406.camel@athlon> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> <200408041520.i74FKm001469@guido.python.org> <1091636838.26267.406.camel@athlon> Message-ID: > Sorry for going off-topic, but this program (UNIX-only): > ... will busywait in select (p_out is always in the ready state; the > select timeout is never reached). > > But if you comment out the "os.close(p_in)" line, it no longer busywaits > (the select timeout is reached on every iteration). At least this is > the behavior under Linux. This isn't strange. You are closing the (only) read-end of the pipe. When you do this, the pipe is broken. Consider this: >>> import os >>> r, w = os.pipe() >>> os.close(r) >>> os.write(w, "a") Traceback (most recent call last): File "", line 1, in ? OSError: [Errno 32] Broken pipe select() only indicates that something has happened on this filedescriptor. > This is a little unfortunate because the normal dance when communicating > between parent and child in order to capture the output of a child > process seems to be: > > 1) In the parent process, create a set of pipes that will represent > stdin/stdout/stderr of the child. > > 2) fork The problem with your example was that it didn't fork... So, there is no problem with using select() on pipes when communicating with a subprocess. It works great. Take a look at (my) process.py's communicate() method for some inspiration. /Peter ?strand From tim.peters at gmail.com Wed Aug 4 21:03:26 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 4 21:03:28 2004 Subject: [Python-Dev] No handlers could be found for logger "cookielib" Message-ID: <1f7befae0408041203475de3d9@mail.gmail.com> I first saw this last night, when running the -all tests on WinXP, CVS HEAD: ... test_univnewlines test_unpack test_urllib test_urllib2 No handlers could be found for logger "cookielib" HERE test_urllibnet ... It didn't act the same way in isolation. Anyone have a clue? The test still passes, it just spits out that unhelpful logger output -- sometimes. From astrand at lysator.liu.se Wed Aug 4 21:10:58 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Wed Aug 4 21:16:14 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: <20040804113056.H3207@ActiveState.com> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <20040804113056.H3207@ActiveState.com> Message-ID: > > Why not avoid threads on POSIX systems, and use select instead? My module > > does, although it does not provide an event-like IO interface. > > Because that wouldn't be cross-platform... perhaps it would be possible > though. I am not that experienced with select() -- I have generally > eschewed it because I can't use it on Windows as well. The best way, IMHO, is to use select() on POSIX systems and threads on Windows. This is what my module does. > > Can you describe why this is needed/useful? > > A subsystem:windows app (like Komodo, or Idle, or pythonw.exe, or any > GUI app) doesn't have a console and hence doesn't have std handles. By > default, executing any subsytem:console app (like 'ls' or 'echo' or > 'python' or 'perl) from a subsystem:windows app will result in an > AllocConsole call > (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/allocconsole.asp) > that will result in a console dialog popping up. You can see this on > some Windows apps that haven't dealt with this issue. In particular, > some installers out there will pop up an unprofessional console dialog > when running some sub-processes. To avoid this one has to muck around > with CreateProcess options. Or, kind of the reverse, if a seeming GUI > app *does* have std handles (often redirected to log files, Komodo does > this) and it wants to offer the ability to pop up a console (e.g. > Komodo's feature to debug in a separate console) then one needs the > ability to specify CREATE_NEW_CONSOLE. I see what you mean. My module actually supports passing all types of flags to CreateProcess, including CREATE_NEW_CONSOLE, so I would say that my module deals with these issues. Well, I don't have any special boolean flag or something like that for this specific case, but I see no reason to. > > I can change, but I'd like more feedback before that. No-one has told me > > their opinion on the name "subprocess", for example, not even you :-) > > Sure, I don't mind "subprocess". I am hoping, though, that we can > eventually merge our two modules and just call that one "process". I don't really see what we could gain from merging our modules. What we have now is two different modules with two different APIs, and applications which uses these. If we were to merge our modules, then the API of either your, mine or both modules would have to change, which means that the applications using these would not work with the merged module. I'm positive to cooperating with you, and perhaps borrow code and ideas from your module though (and you can of course borrow from mine). /Peter ?strand From python at rcn.com Wed Aug 4 09:16:54 2004 From: python at rcn.com (Raymond Hettinger) Date: Wed Aug 4 21:18:37 2004 Subject: [Python-Dev] trunk frozen for 2.4a2 In-Reply-To: <41112923.8060505@interlink.com.au> Message-ID: <004f01c479f3$06327380$e841fea9@oemcomputer> [Anthony Baxter] > I plan to put together > a list of items that are still pending for 2.4 - any that can't > find someone to do the work will have to seriously be considered > at risk. . . . The only other piece of > major unfinished work that springs to mind is the $string > interpolation, If Barry doesn't have time for this one, I'll work on it. Raymond From trentm at ActiveState.com Wed Aug 4 21:39:18 2004 From: trentm at ActiveState.com (Trent Mick) Date: Wed Aug 4 21:39:24 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: ; from astrand@lysator.liu.se on Wed, Aug 04, 2004 at 09:10:58PM +0200 References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <20040804113056.H3207@ActiveState.com> Message-ID: <20040804123918.C7039@ActiveState.com> [Peter Astrand wrote] > > > Why not avoid threads on POSIX systems, and use select instead? My module > > > does, although it does not provide an event-like IO interface. > > > > Because that wouldn't be cross-platform... perhaps it would be possible > > though. I am not that experienced with select() -- I have generally > > eschewed it because I can't use it on Windows as well. > > The best way, IMHO, is to use select() on POSIX systems and threads on > Windows. This is what my module does. I'm willing to check that out for ProcessProxy. However, as I said, I would be willing to remove ProcessProxy to a separate module (perhaps just used by Komodo -- which was the original motivation for me) if it is deemed not necessary/worthy for the core. > I don't really see what we could gain from merging our modules. One process control module to rule them all. Muwahahaha! Seriously, I'd like to see "one obvious way to do it" for process control in the core at some point (perhaps for Python 2.5) and my guess is that a something between our two can provide it. > What we have now is two different modules with two different APIs, and > applications which uses these. If we were to merge our modules, then > the API of either your, mine or both modules would have to change, which > means that the applications using these would not work with the merged > module. I can change Komodo. And I don't use Twisted so I can break them. (Muwahaha!) ... more seriously I think a good final API (even if it break backwards compat a little bit) is fine if the end result is a better module for the core -- and as long as there is a __version__ attribute or something that users can key on. Trent -- Trent Mick TrentM@ActiveState.com From chrism at plope.com Wed Aug 4 22:03:51 2004 From: chrism at plope.com (Chris McDonough) Date: Wed Aug 4 22:03:57 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> <200408041520.i74FKm001469@guido.python.org> <1091636838.26267.406.camel@athlon> Message-ID: <1091649831.26267.527.camel@athlon> On Wed, 2004-08-04 at 15:01, Peter Astrand wrote: > > > > But if you comment out the "os.close(p_in)" line, it no longer busywaits > > (the select timeout is reached on every iteration). At least this is > > the behavior under Linux. > > This isn't strange. You are closing the (only) read-end of the pipe. When > you do this, the pipe is broken. Consider this: > > >>> import os > >>> r, w = os.pipe() > >>> os.close(r) > >>> os.write(w, "a") > Traceback (most recent call last): > File "", line 1, in ? > OSError: [Errno 32] Broken pipe > > select() only indicates that something has happened on this > filedescriptor. Right. I understand that returning EOF on the reader side of the pipe is the inverse of the "broken pipe" behavior you demonstrate above. > > This is a little unfortunate because the normal dance when communicating > > between parent and child in order to capture the output of a child > > process seems to be: > > > > 1) In the parent process, create a set of pipes that will represent > > stdin/stdout/stderr of the child. > > > > 2) fork > > The problem with your example was that it didn't fork... I was all set to try to refute this, but after writing a minimal test program to do what I want to do, I find that you're right. That's good news! I'll need to revisit my workaroudns in the program that caused me to need to do this. Thanks for the schooling. > So, there is no problem with using select() on pipes when communicating > with a subprocess. It works great. Take a look at (my) process.py's > communicate() method for some inspiration. I've actually looked at it and it's quite nice, but it doesn't do one thing that I'd like to see as part of a process stdlib library. The use case I'm thinking of is one where a long-running program needs to monitor the output of many other potentially long-running processes, doing other things in the meantime. This kind of program tends to use select as a part of a mainloop where there might be other things going on (like handling network communications to/from sockets, updating a GUI, etc). Also, the output from child stderr and stdout potentially never end because the child process(es) may never end. In popen5, "communicate" is terminal. It calls select until there's no more data to get back and then unconditionally waits for the subprocess to finish, blocking the entire time. This isn't useful for the type of program I describe above. Of course, it wasn't meant to be, but having an API that could help with this sort of thing would be nice as well, although probably out of scope for PEP 234. - C From chrism at plope.com Wed Aug 4 22:15:50 2004 From: chrism at plope.com (Chris McDonough) Date: Wed Aug 4 22:15:54 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: <1091649831.26267.527.camel@athlon> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> <200408041520.i74FKm001469@guido.python.org> <1091636838.26267.406.camel@athlon> <1091649831.26267.527.camel@athlon> Message-ID: <1091650550.26267.538.camel@athlon> On Wed, 2004-08-04 at 16:03, Chris McDonough wrote: > I was all set to try to refute this, but after writing a minimal test > program to do what I want to do, I find that you're right. That's good > news! I'll need to revisit my workaroudns in the program that caused me > to need to do this. Thanks for the schooling. Ugh. I spoke a bit too soon.. The following program demonstrates that a particular usage of select (under Linux at least) always returns the output side of a pipe connected to a child process' stdout as "ready" after it gets any output from that child process, even if the child process has no further data to provide after it has provided a bit of data to the parent. This is what causes the "busywait" behavior I've experienced in the past (note that as this program runs, your CPU utilization will likely be near 100%). Or am I doing something silly? import select import errno import fcntl import os import stat import sys from fcntl import F_SETFL, F_GETFL def get_path(): """Return a list corresponding to $PATH, or a default.""" path = ["/bin", "/usr/bin", "/usr/local/bin"] if os.environ.has_key("PATH"): p = os.environ["PATH"] if p: path = p.split(os.pathsep) return path def get_execv_args(command): """Internal: turn a program name into a file name, using $PATH.""" commandargs = command.split() program = commandargs[0] if "/" in program: filename = program try: st = os.stat(filename) except os.error: return None, None else: path = get_path() for dir in path: filename = os.path.join(dir, program) try: st = os.stat(filename) except os.error: continue mode = st[stat.ST_MODE] if mode & 0111: break else: return None, None if not os.access(filename, os.X_OK): return None, None return filename, commandargs def spawn(command): """Start the subprocess.""" filename, argv = get_execv_args(command) if filename is None: raise RuntimeError, '%s is an invalid command' % command child_stdin, stdin = os.pipe() stdout, child_stdout = os.pipe() stderr, child_stderr = os.pipe() # open stderr, stdout in nonblocking mode so we can tail them # in the mainloop without blocking for fd in stdout, stderr: flags = fcntl.fcntl(fd, F_GETFL) fcntl.fcntl(fd, F_SETFL, flags | os.O_NDELAY) pid = os.fork() if pid != 0: # Parent os.close(child_stdin) os.close(child_stdout) os.close(child_stderr) return stdin, stdout, stderr else: # Child try: os.dup2(child_stdin, 0) os.dup2(child_stdout, 1) os.dup2(child_stderr, 2) for i in range(3, 256): try: os.close(i) except: pass os.execv(filename, argv) finally: os._exit(127) def go(out_fds): while 1: try: r, w, x = select.select(out_fds, [], [], 1) if not r: print "timed out" except select.error, err: if err[0] != errno.EINTR: raise for fd in r: sys.stdout.write(os.read(fd, 1024)) stdin, stderr, stdout = spawn('echo "foo"') go([stderr, stdout]) From bob at redivi.com Wed Aug 4 22:17:23 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Aug 4 22:17:34 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: <1091649831.26267.527.camel@athlon> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> <200408041520.i74FKm001469@guido.python.org> <1091636838.26267.406.camel@athlon> <1091649831.26267.527.camel@athlon> Message-ID: <4B77CAF8-E653-11D8-B9E0-000A95686CD8@redivi.com> On Aug 4, 2004, at 4:03 PM, Chris McDonough wrote: > On Wed, 2004-08-04 at 15:01, Peter Astrand wrote: >> So, there is no problem with using select() on pipes when >> communicating >> with a subprocess. It works great. Take a look at (my) process.py's >> communicate() method for some inspiration. > > I've actually looked at it and it's quite nice, but it doesn't do one > thing that I'd like to see as part of a process stdlib library. The > use > case I'm thinking of is one where a long-running program needs to > monitor the output of many other potentially long-running processes, > doing other things in the meantime. This kind of program tends to use > select as a part of a mainloop where there might be other things going > on (like handling network communications to/from sockets, updating a > GUI, etc). Also, the output from child stderr and stdout potentially > never end because the child process(es) may never end. Twisted handles this quite well already. The stdlib doesn't really do much to support this style of programming. -bob From barry at python.org Wed Aug 4 22:21:44 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 4 22:21:37 2004 Subject: [Python-Dev] trunk frozen for 2.4a2 In-Reply-To: <004f01c479f3$06327380$e841fea9@oemcomputer> References: <004f01c479f3$06327380$e841fea9@oemcomputer> Message-ID: <1091650904.8324.299.camel@localhost> On Wed, 2004-08-04 at 03:16, Raymond Hettinger wrote: > [Anthony Baxter] > > I plan to put together > > a list of items that are still pending for 2.4 - any that can't > > find someone to do the work will have to seriously be considered > > at risk. . . . The only other piece of > > major unfinished work that springs to mind is the $string > > interpolation, > > If Barry doesn't have time for this one, I'll work on it. I'm still planning to work on it and get it in for 2.4a3, and I know that Raymond has offered to help. Raymond, email me directly and we'll arrange a time to chat about it on irc. I have a couple of open issues that are waiting to be resolved before this PEP lands. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/e60e9354/attachment.pgp From nhodgson at bigpond.net.au Thu Aug 5 00:04:52 2004 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Thu Aug 5 00:04:57 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax References: <200408041100.34439.fdrake@acm.org> <04Aug4.084736pdt.58612@synergy1.parc.xerox.com> <16657.7235.345770.620493@montanaro.dyndns.org> Message-ID: <016101c47a6f$11959230$034b8890@neil> Skip Montanaro: > Though X/Emacs is popular within the Unix community, it's hardly the only > editor that attempts to colorize Python code: > > http://www.python.org/cgi-bin/moinmoin/PythonEditors > > In fact, given the increased use of Python on Windows these days I'm > skeptical it's the most widely used editor for Python code. Many of the other editors use the Scintilla editing component and rely on its built-in Python lexer. If no one else gets to it first, I'll update the lexer to recognize pie shaped decorators. Or whichever other syntax is accepted. Neil From nidoizo at yahoo.com Thu Aug 5 03:11:28 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu Aug 5 03:10:26 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <200408031450.i73Eo0e30956@guido.python.org> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> <5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com> <200408031450.i73Eo0e30956@guido.python.org> Message-ID: Just a question, when "as" become a keyword in Python 3, can we expect a new syntax using that keyword instead (or additionally)? There's multiple possible syntaxes using "as", even replacing simply @ by "as" would be an improvement IMHO, since that would be english-readable (as static method define foo), like most of Python. Guido van Rossum wrote: > Not IMO, that would just add more mystery (and would require yet > another hack in classmethod to allow it to be called without args). Regards, Nicolas From greg at cosc.canterbury.ac.nz Thu Aug 5 03:16:14 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 5 03:16:20 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C000@au3010avexu1.global.avaya.com> Message-ID: <200408050116.i751GEW7031460@cosc353.cosc.canterbury.ac.nz> > This proposal has come up several times before (for the record, I like > it). The fact that it keeps coming up suggests that it's a natural thing for people to want to do once they've seen the use of * in argument lists. I can't remember seeing any particular objection -- just a matter of inertia, I think (i.e. nice to have but not compelling enough to spur the BDFL to accept it). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Thu Aug 5 03:33:24 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 5 03:33:35 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <200408041434.i74EYdU01260@guido.python.org> Message-ID: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> > Maybe we can accept old-style classes and instances, strings, and > instances of Exception and its subclasses. Seems to me the point at which we start allowing new-style classes as exceptions should also be the point at which we drop the idea of string exceptions. Would that help? > I don't really think that "raise 42" > ought to be acceptable, but I don't know how to prevent it Maybe we need to think more deeply about *why* it shouldn't be acceptable. If we can figure out exactly what the criterion should be, maybe we can think of a reasonable way of testing for it. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Thu Aug 5 03:35:48 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 5 03:35:53 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: <4110F829.2000609@zope.com> Message-ID: <200408050135.i751ZmiH031501@cosc353.cosc.canterbury.ac.nz> > IMO, the most common uses of decorators will be to define properties, > and class and static methods. I thought we had decided that properties weren't one of the things addressed by the decorator mechanism? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From python at rcn.com Wed Aug 4 16:15:38 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 5 04:17:19 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <200408041418.i74EIt001130@guido.python.org> Message-ID: <000e01c47a2d$84f3be60$e841fea9@oemcomputer> > For the record, let me just say that I'm -1 on adding this feature > now. It's only a minor convenience while it's a fairly big change to > the parser and code generator ... That is a strong reason to deny the feature request. > ... but most of all, Python's growth needs > to be reduced. If we were to add every "nice" feature that was > invented (or present in other languages), Python would lose one of its > main charms, which is that it is really a rather simple language that > can be learned and put to production quickly. However, IMO this reason does not apply to the *arg feature request. Whenever a concept is already in a language, applying the same idea in other places does not make the language fatter or harder to learn. For instance, since generator expressions grow naturally from list comprehensions and regular generators, they are not likely to present a learning challenge. > ... I appreciate the idea (which BTW has been proposed before) IIRC, the conversation for the original proposal fizzled out without reaching a conclusion. So, resuming the discussion was probably a good idea. But now it has been officially zapped. Raymond From djc at object-craft.com.au Thu Aug 5 04:18:42 2004 From: djc at object-craft.com.au (Dave Cole) Date: Thu Aug 5 04:18:52 2004 Subject: [Python-Dev] Submitted socket.socketpair() patch to SF Message-ID: <41119902.8060305@object-craft.com.au> For the interested, I have just submitted patch #1003700 which implements the socketpair function in the socket module. The patch is relative to 2.4a1. There is a question in the patch summary: Outstanding question is whether the family, type, proto arguments should be kwargs as per socket() function, or positional as per fromfd() function. If kwargs, then it is arguable that the fromfd() arguments should be changed (in another patch). I noticed a couple of things in the documentation while implementing the patch: 1) The documentation for socket() does not make mention of the fact that all arguments are kwargs. 2) The documentation for fromfd() does not mention that it does a dup() on the supplied fd to create a socket object. - Dave -- http://www.object-craft.com.au From bac at OCF.Berkeley.EDU Thu Aug 5 04:19:17 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 5 04:19:15 2004 Subject: [Python-Dev] No handlers could be found for logger "cookielib" In-Reply-To: <1f7befae0408041203475de3d9@mail.gmail.com> References: <1f7befae0408041203475de3d9@mail.gmail.com> Message-ID: <41119925.1060001@ocf.berkeley.edu> Tim Peters wrote: > I first saw this last night, when running the -all tests on WinXP, CVS HEAD: > > ... > test_univnewlines > test_unpack > test_urllib > test_urllib2 > No handlers could be found for logger "cookielib" HERE > test_urllibnet > ... > > It didn't act the same way in isolation. Anyone have a clue? The > test still passes, it just spits out that unhelpful logger output -- > sometimes. See the thread at http://mail.python.org/pipermail/python-dev/2004-July/046396.html for an explanation of the message. Basically means no default handler is specified. -Brett From paul at prescod.net Thu Aug 5 04:32:30 2004 From: paul at prescod.net (Paul Prescod) Date: Thu Aug 5 04:37:46 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> Message-ID: <41119C3E.5060509@prescod.net> Greg Ewing wrote: >... >>I don't really think that "raise 42" >>ought to be acceptable, but I don't know how to prevent it > > > Maybe we need to think more deeply about *why* it shouldn't be > acceptable. If we can figure out exactly what the criterion should be, > maybe we can think of a reasonable way of testing for it. Exceptions naturally form a hierarchy. At the same time, inheritance of behaviour among exceptions is seldom necessary. Therefore, exceptions inherit from each other in order to build a classification system, not to share code. This is the opposite of the traditional reasons for classes inheriting from other classes in Python. This is why it seems "unpythonic" to require exceptions to be single-rooted. But having a proper classification system is exactly what is required to allow robust, modular code that catches the right exceptions under the right circumstances and responds in the right way. So it is pythonic after all. In a few senses the _current model_ is unpythonic. There is no catch-all root so you have to use a "bare" except to catch every exception type. This makes it hard to introspect on the caught object. But introspection is the MOST IMPORTANT THING when you are catching all exceptions (because you should be logging the exception or something). Paul Prescod From tim.peters at gmail.com Thu Aug 5 04:40:24 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 5 04:40:30 2004 Subject: [Python-Dev] No handlers could be found for logger "cookielib" In-Reply-To: <41119925.1060001@ocf.berkeley.edu> References: <1f7befae0408041203475de3d9@mail.gmail.com> <41119925.1060001@ocf.berkeley.edu> Message-ID: <1f7befae040804194050d68f20@mail.gmail.com> [Tim] >> I first saw this last night, when running the -all tests on WinXP, CVS HEAD: >> >> ... >> test_univnewlines >> test_unpack >> test_urllib >> test_urllib2 >> No handlers could be found for logger "cookielib" HERE >> test_urllibnet >> ... >> >> It didn't act the same way in isolation. Anyone have a clue? The >> test still passes, it just spits out that unhelpful logger output -- >> sometimes. [Brett] > See the thread at > http://mail.python.org/pipermail/python-dev/2004-July/046396.html for an > explanation of the message. Basically means no default handler is > specified. Ya, I know about that. The real questions are two others: 1. Why does it only appear *sometimes*, depending on how the test suite is run? It always appears when running -uall now. It never appears when running the surrounding tests in isolation. 2. Since it started happening very recently (I've run the full test suite often the last week), does it ring a bell with anyone who checked something in very recently? Somebody changed something relevant very recently. From ncoghlan at iinet.net.au Thu Aug 5 05:33:02 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Aug 5 05:33:23 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <200408050116.i751GEW7031460@cosc353.cosc.canterbury.ac.nz> References: <200408050116.i751GEW7031460@cosc353.cosc.canterbury.ac.nz> Message-ID: <4111AA6E.3090600@iinet.net.au> Greg Ewing wrote: > I can't remember seeing any particular objection -- just > a matter of inertia, I think (i.e. nice to have but not > compelling enough to spur the BDFL to accept it). This certainly summarises my reason for criticising it. It's application to a basic 'command parsing' scenario was a useful case I hadn't thought of, though: >>> commands = line.split() for line in commandFile.readlines() >>> for command, *args in commands do: func_table[command](*args) However, the slicing based alternative is fairly readable, too and if the BDFL says the interpreter changes needed to support the feature are too drastic, I'm willing to believe him :) Regards, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From andrewm at object-craft.com.au Thu Aug 5 05:36:33 2004 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu Aug 5 05:36:23 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: <4110F35D.8020904@zope.com> References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org> <410FC9FE.6060305@divmod.com> <4110E8CF.6070900@zope.com> <20040804140442.89F2B3C7A0@coffee.object-craft.com.au> <4110F35D.8020904@zope.com> Message-ID: <20040805033633.2D1BA3C221@coffee.object-craft.com.au> >> There are a number of "advanced" features in python that could be a >> barrier to understanding for new users, but which are rarely seen in >> practice - metaclasses being the classic example. >> >> I imagine these features are rarely seen because most people don't >> understand them, and those that do have the good taste not to use them >> gratuitously. > >This feature won't be rarely used. Class and static methods >and properties are created fairly often, certainly far more often >than meta classes. Sorry, what I meant was advanced uses of decorators will be rare. Certainly "staticmethod" and "classmethod" will be common. It's a personal opinion, but I find @staticmethod easier to parse visually than the current situation. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From guido at python.org Thu Aug 5 06:17:53 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 5 06:18:33 2004 Subject: [Python-Dev] Re: PEP 324 (process module) In-Reply-To: Your message of "Wed, 04 Aug 2004 12:46:56 EDT." <1091638016.26267.418.camel@athlon> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> <1091638016.26267.418.camel@athlon> Message-ID: <200408050417.i754Hr602375@guido.python.org> > As I've found, if the end of the pipes in that represent the child's > stderr/stdout fds are closed in the parent, a select() reading the > "other" ends of these pipes will busywaut (at least on Linux). Other > than than, I think this select() would be the absolute right thing on > POSIX rather than using threads or polling. Is there a way around this > problem or is it just a fact of life? The reading code needs to recognize the EOF and then conclude that it's not going to get anything else from that pipe. No different than sockets. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 5 06:36:04 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 5 06:36:24 2004 Subject: [Python-Dev] Re: 2.4a2, and @decorators In-Reply-To: Your message of "Wed, 04 Aug 2004 21:11:28 EDT." References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com> <1091458891.9134.29.camel@localhost> <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> <200408030058.i730wSQ29267@guido.python.org> <410F2453.7000506@divmod.com> <5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com> <200408031450.i73Eo0e30956@guido.python.org> Message-ID: <200408050436.i754a4k02477@guido.python.org> > Just a question, when "as" become a keyword in Python 3, can we expect a > new syntax using that keyword instead (or additionally)? There's > multiple possible syntaxes using "as", even replacing simply @ by "as" > would be an improvement IMHO, since that would be english-readable (as > static method define foo), like most of Python. When 'as' becomes a keyword, it is open for use in other syntax, but it won't be used for decorators (as long as I'm BDFL). 'as' suggests a renaming (like it does in import and in SQL) or perhaps a type (I've seen "as duck" in a new vaguely Pythonic language named "boo"), but that's not what decorators doo. --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at cosc.canterbury.ac.nz Thu Aug 5 07:50:37 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 5 07:50:43 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: <1091650550.26267.538.camel@athlon> Message-ID: <200408050550.i755ob1s031939@cosc353.cosc.canterbury.ac.nz> Chris McDonough : > The following program demonstrates that a particular usage of select > (under Linux at least) always returns the output side of a pipe > connected to a child process' stdout as "ready" after it gets any output > from that child process, even if the child process has no further data > to provide after it has provided a bit of data to the parent. > > Or am I doing something silly? > > for fd in r: > sys.stdout.write(os.read(fd, 1024)) You're not taking account of the possibility of EOF. When the child process finishes and closes its end of the pipe, the select will always report the pipe as ready for reading, because it is -- subsequent reads will return immediately with 0 bytes. You need to check whether the read returned 0 bytes, and take that as meaning that the child process has finished. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From bac at OCF.Berkeley.EDU Thu Aug 5 09:05:16 2004 From: bac at OCF.Berkeley.EDU (Brett Cannon) Date: Thu Aug 5 09:05:11 2004 Subject: [Python-Dev] python-dev Summary for 2004-07-16 through 2004-07-31 [draft] Message-ID: <4111DC2C.8070302@ocf.berkeley.edu> OK, so I procrastinated from doing more work on my thesis and did the next summary. Plan to send this out some time over the weekend. ------------------------------- ===================== Summary Announcements ===================== Python 2.4a2 should be coming out any day now, so keep an eye out for it and test the heck out of it. ======== Summary ======== -------------------------------------------------------------------------------------------------- Makin' bdist_wininst purty and nicer for them non-English speaking folks -------------------------------------------------------------------------------------------------- Thomas Heller asked if some better images for Distutil's bdist_wininst could be used. Some existing images were discussed, but no specific resolution was mentioned in the thread. Walter D?rwald brought up the point that if this was all being worked on he wouldn't mind having Unicode support for things such as author name and such. Thomas and Walter worked on it and that has gone in. Contributing threads: - `bdist_wininst `__ ------------------------------------------------------------------------- Assigning to None a no-no, but still okay for booleans ------------------------------------------------------------------------- Assigning to None is now a SyntaxError instead of a SyntaxWarning. Assignment to True and False, though, is still allowed and probably will not be restricted until Python 3.0 (which will probably have to wait until, if Guido's OSCON slides are to be believed, until he retires). The idea of restricting assignment, though, was brought up by Raymond Hettinger. Michael Hudson, though, pointed out it would be difficult and possibly require a special opcode to handle it. Contributing threads: - `Re: [Python-checkins] python/dist/src/Python compile.c, `__ - `None as constant. Still SyntaxWarning `__ ----------------------------------------------------------------- 'as' does not get to be treated with kiddie gloves ----------------------------------------------------------------- Philip Eby asked if 'as' was going to become a keyword. Guido said not now, but in Python 3.0 it will be. Contributing threads: - `"as" to be a keyword? `__ ------------------------------------------------------------- Hopefully you didn't love FCNTL like a child... -------------------------------------------------------------- ... since if you did that kid was a lazy punk that did nothing but cause trouble for Windows users. That's why it got kicked out of the house to never come back. Contributing threads: - `Kill FCNTL.py `__ ------------------------------------------------------------------------------------------------------------------------------------------ How to get Python to compile with Microsoft's free compiler that should just come with the OS standard ------------------------------------------------------------------------------------------------------------------------------------------ 1. Download the free .NET compiler 2. Download the Windows SDK (at http://www.microsoft.com/msdownload/platformsdk/sdkupdate/ with only IE) 3. Download Garth's tools to generate a Makefile from the .sln files (at http://mail.python.org/pipermail/python-dev/2004-February/042595.html ) 4. Compile 5. Realize you should be using an OS that doesn't make you go through this many hoops just to have a compiler for your platform Contributing threads: - `Non-Visual Studio builds on Windows XP? `__ --------------------------------------------------------------------------------------------------------------------------------- Tim Peters, whitespace Czar, will break your legs if you mess up the pristine whitespace in CVS --------------------------------------------------------------------------------------------------------------------------------- Tim Peters ran Tools/Scripts/reindent.py over the entire CVS tree and fixed it up. This means you had better not mess up and check in using anything but 4-space indents with no tabs! Contributing threads: - `Fun with whitespace `__ ----------------------------------------------------------- Thread safety == won't blow up in your face ----------------------------------------------------------- Paul Moore pointed out that the documentation for deques in the collections module stated they were thread-safe. It was clarified that the statement meant that internal state would not be corrupted if multiple threads accessed the same object; no guarantee that it isn't accessed in some order or anything. Basically the idea of thread-safety for C code is that it won't lead to the interpreter exploding, nothing more. And you don't even get that guarantee with Python code. Contributing threads: - `Thread safety of deques `__ ------------------------------------------------------------------------ LC_NUMERIC PEP gets a pristine, new PEP number ------------------------------------------------------------------------ Even though the code had already been put into the core, the PEP about LC_NUMERIC and being locale-independent never got a PEP number. Well, now it has one: `PEP 331`_. .. _PEP 331: http://www.python.org/peps/pep-0331.html Contributing threads: - `PEP 331: Locale-Independent Float/String Conversions `__ ----------------------------------------------------------------------- Edward Loper becomes one of us rowdy developers ----------------------------------------------------------------------- And he has already made his initial contribution by helping to rework doctest. Contributing threads: - `Would like to add Edward Loper as a Python developer `__ ------------------------------------------------------------------------------------------------------ Any misbehaving modules during initialization from import now get the boot ------------------------------------------------------------------------------------------------------ Jim Fulton wanted a better way to detect when an import failed thanks to another module being directly imported (e.g., module A imports B which raised TypeError from some initialization code from importing; importing would leave A and B in a shoddy state in sys.modules along with raising TypeError in the import from A instead of raising ImportError). While the latter still occurs, modules are not left in sys.modules in a broken state from exceptions being raised during initialization thanks to Guido and Tim Peters. There was a discussion on circular imports and how to handle those for proper rollback. Some suggestions were taking a snapshot of sys.modules and then restoring with that if something bad happens to putting in placeholder modules in sys.modules so as to not try to extra imports on modules in the process and to help track state. But this all gets sticky from side-effects that modules can do outside of themselves before they finish importing everyone else. If, for instance, module A, as a side-effect of importation, injected a custom version of len into module B. Now, if module A did this before doing all of its imports it could pull of the len injection but still fail from a bad import. That is not good. Basically the best solution is to not do that; there is a reason you should do all of your global imports as the first thing in a module. Contributing threads: - `Fix import errors to have data `__ ---------------------------------------------------------------------------------------------------------------------------- Stuff about Unicode, state, and how to handle when a stream terminated early; all CJK to me ---------------------------------------------------------------------------------------------------------------------------- Walter D?rwald noticed that codecs.StreamReader.read() would read a few more bytes when it discovered an error. That's bad since there might not be more bytes and continuing once an error has been found is just not right. So he wanted to fix that problem. Unfortunately he and MA Lemburg started to talk and I just couldn't follow everything about stateful and stateless decoders/encoders and the issues; I'm American so Unicode just doesn't fit in my brain well. So if you want to know what conclusions they reached you are going to read the thread on your own. Contributing threads: - `Decoding incomplete unicode `__ ---------------------------------------------------------------------- Use the docs to know what the public API is, people ---------------------------------------------------------------------- Fernando Perez got bit by having something removed from the 'keyword' module that was not part of the documented API. Turns out that running 'help' on the module listed the function in question and so he just went ahead an used it. That's not the right way to go about finding the public API of stdlib code. Always follow the documentation. If it is not listed there don't expect it to be there in the next version. While we try not to yank stuff out needlessly and will tweak things to be nice on occasion, we make no guarantees on stuff not listed in the API. The next best thing is what is listed in __all__ for a module. Since that is explicitly listed that usually can be considered part of the API. And lastly, anything starting with an underscore is off limits in terms of promises of keeping it around. Luckily Skip Montanaro applied a patch for pydoc to have it only list stuff in modules as specified by __all__, so it's a little safer out there. But the actual documentation is still king. Contributing threads: - `Rationale behind removing kwdict from keyword.py? `__ ------------------------------------------------------- Linking against static builds on Windows ------------------------------------------------------- Thomas Heller pointed out that one couldn't dynamically link against a statically built Python interpreter DLL. He and Martin v. L?wis discussed various ways to try to get this to work, but it doesn't appear they came up with one. Contributing threads: - `Static builds on Windows `__ From mal at egenix.com Thu Aug 5 09:45:28 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 5 09:45:32 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <41119C3E.5060509@prescod.net> References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> Message-ID: <4111E598.5020302@egenix.com> Paul Prescod wrote: > Greg Ewing wrote: > >> ... >> >>> I don't really think that "raise 42" >>> ought to be acceptable, but I don't know how to prevent it >> >> >> >> Maybe we need to think more deeply about *why* it shouldn't be >> acceptable. If we can figure out exactly what the criterion should be, >> maybe we can think of a reasonable way of testing for it. > > > Exceptions naturally form a hierarchy. At the same time, inheritance of > behaviour among exceptions is seldom necessary. Therefore, exceptions > inherit from each other in order to build a classification system, not > to share code. I wouldn't say that: exceptions can have error handlers, callbacks, inherited attributes, etc. etc. and you can put these to good use in your application. > This is the opposite of the traditional reasons for > classes inheriting from other classes in Python. This is why it seems > "unpythonic" to require exceptions to be single-rooted. I don't know what should be "unpythonic" about having a single root for exceptions. Would someone care to explain ? To me ... try: ... except Exception, errobj: # catches all exceptions pass ... is the most natural way of using that single root (and it already works great today). > But having a proper classification system is exactly what is required to > allow robust, modular code that catches the right exceptions under the > right circumstances and responds in the right way. So it is pythonic > after all. > > In a few senses the _current model_ is unpythonic. There is no catch-all > root so you have to use a "bare" except to catch every exception type. > This makes it hard to introspect on the caught object. But introspection > is the MOST IMPORTANT THING when you are catching all exceptions > (because you should be logging the exception or something). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 05 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Thu Aug 5 09:21:37 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 10:36:52 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <20040803193516.GA9031@titan.progiciels-bpi.ca> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <20040803193516.GA9031@titan.progiciels-bpi.ca> Message-ID: <4111E001.40601@v.loewis.de> Fran?ois Pinard wrote: > However, and I shall have the honesty to > state it, this is *not* respectful of the general Unicode spirit: the > Python implementation allows for independently addressable surrogate > halves This is only a problem if you have data which require surrogates (which I claim are rather uncommon at the moment), and you don't have a UCS-4 build of Python (in which surrogates don't exist). As more users demand convenient support for non-BMP characters, you'll find that more builds of Python become UCS-4. In fact, you might find that the build you are using already has sys.maxunicode > 65535. > combining zero-width diacritics Indeed. However, it is not clear to me how this problem could be addressed, and I'm not aware of any API (any language) that addresses it. Typically, people need things like this: - in a fixed-width terminal, what characters occupy what column. Notice that this involves East-Asian wide characters, where a single Unicode character (a "wide" character) occupies two columns. OTOH, with combining characters, a sequence of characters might be associated with a single column. Furthermore, some code points might not be associated with a column at all. - for a given font, how many points does a string occupy, horizontally and vertically. - where is the next word break I don't know what your application is, but I somewhat doubt it is as simple as "give me a thing describing the nth character, including combining diacritics". However, it is certainly possible to implement libraries on top of the existing code, and if there is a real need for that, somebody will contribute it. > normal _and_ decomposed forms, Terminology alert: the are multiple normal forms in Unicode, and some of them are decomposed (e.g. NFD, NFKD). I fail to see a problem with that. There are applications for all normal forms, and many applications don't need the overhead of normalization. It might be that the code for your languages becomes simpler when always assuming NFC, but this hardly holds for all languages, or all applications. > directional marks, linguistic marks and various other such complexities. Same comment as above: if this becomes a real problem, people will contribute code to deal with it. > But in our case, where applications already work in Latin-1, abusing our > Unicode luck, UTF-8 may _not_ be used as is, we ought to use Unicode or > wide strings as well, for preserving S[N] addressability. So changing > source encodings may be intimately tied to going Unicode whenever UTF-8 > (or any other variable-length encoding) gets into the picture. Yes. There is not much Python can do about this. UTF-8 is very nice for transfer of character data, but it does have most of the problems of a multi-byte encoding. I still prefer it over UTF-16 or UTF-32 for transfer, though. > I hope that my explanation above helps at seeing that source encoding > and choice of string literals are not as independent as one may think. It really depends on your processing needs. But yes, my advise still stands: convert to Unicode objects as early as possible in the processing. For source code involving non-ASCII characters, this means you really should use Unicode literals. Of course, my other advise also applies: if you have a program that deals with multiple languages, use only ASCII in the source, and use gettext for the messages. > There ought to be a way to maintain a single Python source that would > work dependably through re-encoding of the source, but not uselessly > relying on wide strings when there is no need for them. That is, > without marking all literal strings as being Unicode. Changing encoding > from ISO 8859-1 to UTF-8 should not be a one-way, no-return ticket. But it is not: as you say, you have to add u prefixes when going to UTF-8, yes. But then you can go back to Latin-1, with *no* change other than recoding, and changing the encoding declaration. The string literals can all stay as Unicode literals - the conversion to Latin-1 then really has *no* effect on the runtime semantics. > Of course, it is very normal that sources may have to be adapted for the > possibility of a Unicode context. There should be some good style and > habits for writing re-encodable programs. So this exchange of thoughts. If that is the goal, you really need Unicode literals - everything else *will* break under re-encoding. Regards, Martin From mal at egenix.com Thu Aug 5 11:00:22 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 5 11:00:26 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <20040803193516.GA9031@titan.progiciels-bpi.ca> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <20040803193516.GA9031@titan.progiciels-bpi.ca> Message-ID: <4111F726.8050905@egenix.com> [Source code encoding and string literals] > I hope that my explanation above helps at seeing that source encoding > and choice of string literals are not as independent as one may think. > A choice that I surely do _not_ have is to see bugs appear in programs > merely because I changed the source encoding. Going from ISO 8859-1 to > ISO 8859-15 for a Python source is probably fairly safe, because there > is no need for switching the narrowness of strings. Going from ISO > 8859-1 to UTF-8 is very unsafe, and editing all literal strings from > narrow to wide, using `u' prefixes, becomes almost unavoidable. Indeed. As always: explicit is better than implicit :-) The small "u" in front of the literal will tell all readers: this is Unicode text. We might introduce more ways to switch string literal interpretation depending on module or interpreter process scope. However, the small u is here to stay and it's available now, so why not use it ? Your concerns about programs breaking because of changes to the source encoding are valid, but not something that Python can address. You have the same problem with normal text documents: a spell checker might find wrong spellings of a word as a result of using a wrong encoding, but it is not fool proof and things get worse if you have multiple languages embedded in your program code. As general advice for writing i18n compliant programs, I can only suggest to keep programs written using a single source encoding and language that appeals to the programmer and place *all* string literals under gettext or similar tool control. I usually write programs in ASCII or Latin-1 and use English to write the string literals which then get mapped to user languages as necessary by means of gettext or custom translation logic. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 05 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Paul.Moore at atosorigin.com Thu Aug 5 11:18:25 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Thu Aug 5 11:18:25 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F47@UKDCX001.uk.int.atosorigin.com> From: Chris McDonough > I've actually looked at it and it's quite nice, but it doesn't > do one thing that I'd like to see as part of a process stdlib > library. The use case I'm thinking of is one where a long-running > program needs to monitor the output of many other potentially > long-running processes I have a couple of other use cases, as well. 1. Manage an interactive subprocess, sending it input and reading output in "chunks". This is an expect-like activity in some ways: I use it to wrap an alternative UI round an interactive command line process. I suspect Trent's ProcessProxy would help with this - must check. 2. Run a command with no input, and get its output back, like the shell backtick operator. Errors in the command should appear as exceptions. 3. Filter a string through an external command - the interact() method handles this, but again I'd like errors to appear as exceptions. To some extent, (2) and (3) are simple "helper" functions round the existing methods, but I believe such helpers would be useful. And the "convert subcommand error to exception" code is non-trivial, and hence useful to encapsulate in the library. This is getting pretty off-topic for python-dev. Is there any value in setting up a list/SIG for discussing the design of a stdlib process module? Paul. __________________________________________________________________________ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. __________________________________________________________________________ From vinay_sajip at yahoo.co.uk Thu Aug 5 11:44:37 2004 From: vinay_sajip at yahoo.co.uk (=?iso-8859-1?q?Vinay=20Sajip?=) Date: Thu Aug 5 11:44:39 2004 Subject: [Python-Dev] No handlers could be found for logger "cookielib" Message-ID: <20040805094437.32296.qmail@web25403.mail.ukl.yahoo.com> I just made a change to test_logging to clean up after itself, so that it removes all handlers that it creates. test_urllib2 seems to have handler creation commented out in NetworkTests.setUp(). A combination of the two may explain why the message shows up now. Vinay Sajip ___________________________________________________________ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com From mwh at python.net Thu Aug 5 13:15:18 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 13:15:19 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <200408032014.i73KEwp31876@guido.python.org> (Guido van Rossum's message of "Tue, 03 Aug 2004 13:14:58 -0700") References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> Message-ID: <2mllgt96pl.fsf@starship.python.net> Guido van Rossum writes: > (It will also break code that creates a class used as an exception > that doesn't derive from Exception, but those should be shot. :-) Would you like to guess how often that happens in the Python test suite? :-) Cheers, mwh -- ARTHUR: Yes. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying "Beware of the Leopard". -- The Hitch-Hikers Guide to the Galaxy, Episode 1 From mwh at python.net Thu Aug 5 13:16:38 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 13:16:40 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> (Greg Ewing's message of "Thu, 05 Aug 2004 13:33:24 +1200") References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> Message-ID: <2mhdrh96nd.fsf@starship.python.net> Greg Ewing writes: >> Maybe we can accept old-style classes and instances, strings, and >> instances of Exception and its subclasses. > > Seems to me the point at which we start allowing new-style classes as > exceptions should also be the point at which we drop the idea of > string exceptions. Would that help? It would probably make things a little simpler, but probably not in a major way. Cheers, mwh -- Or if you happen to be resigned to the size of your trouser snake and would rather not be reminded of it, training a shared classifier to reject penis-enlargement spam stops Barry from getting the help he so desperately needs. -- Tim Peters, c.l.python From mwh at python.net Thu Aug 5 13:18:18 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 13:18:19 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <4111E598.5020302@egenix.com> (M.'s message of "Thu, 05 Aug 2004 09:45:28 +0200") References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> Message-ID: <2md62596kl.fsf@starship.python.net> "M.-A. Lemburg" writes: > To me ... > > try: > ... > except Exception, errobj: > # catches all exceptions > pass > > ... is the most natural way of using that single root (and it already > works great today). Well, uh, it's not totally bullet proof: >>> class C: pass ... [24618 refs] >>> try: raise C ... except Exception, err: print err ... Traceback (most recent call last): File "", line 1, in ? __main__.C: <__main__.C instance at 0x403a7814> [24654 refs] >>> but I this really doesn't seem to happen in the wild. (I have a hacky patch which makes exceptions new-style which I'll post in a moment). Cheers, mwh -- Lisp nearing the age of 50 is the most modern language out there. GC, dynamic, reflective, the best OO model extant including GFs, procedural macros, and the only thing old-fashioned about it is that it is compiled and fast. -- Kenny Tilton, comp.lang.python From pyth at devel.trillke.net Thu Aug 5 13:29:28 2004 From: pyth at devel.trillke.net (Holger Krekel) Date: Thu Aug 5 13:29:31 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <200408032014.i73KEwp31876@guido.python.org> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> Message-ID: <20040805112928.GA5208@solar.trillke> Guido van Rossum wrote: > (It will also break code that creates a class used as an exception > that doesn't derive from Exception, but those should be shot. :-) Then i guess that searching down into a recursive structure and just raising an "i found it" result object up doesn't count as a use case in your book, right? It can avoid boilerplate code like return-if-not-None checks and I have used it for e.g. finding patterns in an AST-Tree. cheers, Holger From mal at egenix.com Thu Aug 5 13:48:23 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 5 13:48:26 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <2md62596kl.fsf@starship.python.net> References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> Message-ID: <41121E87.9010805@egenix.com> Michael Hudson wrote: > "M.-A. Lemburg" writes: > > >>To me ... >> >>try: >> ... >>except Exception, errobj: >> # catches all exceptions >> pass >> >>... is the most natural way of using that single root (and it already >>works great today). > > > Well, uh, it's not totally bullet proof: I meant that it works for the vast majority of all cases you see in practice. I haven't seen a non-Exception based exception in years. >>>>class C: pass > > ... > [24618 refs] > >>>>try: raise C > > ... except Exception, err: print err > ... > Traceback (most recent call last): > File "", line 1, in ? > __main__.C: <__main__.C instance at 0x403a7814> > [24654 refs] > > > but I this really doesn't seem to happen in the wild. > > (I have a hacky patch which makes exceptions new-style which I'll post > in a moment). > > Cheers, > mwh > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 05 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mwh at python.net Thu Aug 5 14:05:44 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 14:05:46 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <41121E87.9010805@egenix.com> (M.'s message of "Thu, 05 Aug 2004 13:48:23 +0200") References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> <41121E87.9010805@egenix.com> Message-ID: <2m8yct94dj.fsf@starship.python.net> "M.-A. Lemburg" writes: D> Michael Hudson wrote: >> "M.-A. Lemburg" writes: >> >>>To me ... >>> >>>try: >>> ... >>>except Exception, errobj: >>> # catches all exceptions >>> pass >>> >>>... is the most natural way of using that single root (and it already >>>works great today). >> Well, uh, it's not totally bullet proof: > > I meant that it works for the vast majority of all cases > you see in practice. OK, then we're on the same page. > I haven't seen a non-Exception based exception in years. I hadn't until I looked into test_opcodes last night! Cheers, mwh -- In that case I suggest that to get the correct image you look at the screen from inside the monitor whilst standing on your head. -- James Bonfield, http://www.ioccc.org/2000/rince.hint From mcherm at mcherm.com Thu Aug 5 15:04:09 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Thu Aug 5 15:03:13 2004 Subject: [Python-Dev] 2.4a2, and @decorators Message-ID: <1091711049.41123049900cb@mcherm.com> Michael Hudson writes: > > that would be insane, > > class C(random.choice([dict, list])): > pass > > is my favourite example of this :-) Where is THAT monstrocity used!? -- Michael Chermside From anthony at interlink.com.au Thu Aug 5 15:10:32 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Aug 5 15:10:58 2004 Subject: [Python-Dev] RELEASED Python 2.4, alpha 2 Message-ID: <411231C8.3020308@interlink.com.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I'm happy to announce the second alpha of Python 2.4. Python 2.4a2 is an alpha release. We'd greatly appreciate it if you could download it, kick the tires and let us know of any problems you find, but it is not suitable for production usage. ~ http://www.python.org/2.4 In this release we have new syntax for function decorators, a fix for failing imports so that they don't leave a broken module in sys.modules, a host of updated modules in the standard library (including optparse and doctest) and a large number of other bug fixes and improvements. See either the highlights, the What's New in Python 2.4, or the detailed NEWS file -- all available from the Python 2.4 webpage. There will probably be one more alpha in a few weeks before the first beta a few weeks after that. Please log any problems you have with this release in the SourceForge bug tracker (noting that you're using 2.4a2): ~ http://sourceforge.net/bugs/?group_id=5470 Enjoy the new release, Anthony Anthony Baxter anthony@python.org Python Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBEjHFDt3F8mpFyBYRAhgXAKCful63a7kAUcHnxFzCZSzq0bmZ7QCfZIqy t/PqaaLcdRL6IVUKPAWiytA= =pQNU -----END PGP SIGNATURE----- From mwh at python.net Thu Aug 5 15:27:57 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 15:27:58 2004 Subject: [Python-Dev] New-style exceptions In-Reply-To: <2md62596kl.fsf@starship.python.net> (Michael Hudson's message of "Thu, 05 Aug 2004 12:18:18 +0100") References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> Message-ID: <2mzn597m02.fsf_-_@starship.python.net> Michael Hudson writes: > (I have a hacky patch which makes exceptions new-style which I'll > post in a moment). Well, it turns out to be a bit big for attaching, so it's here: http://starship.python.net/crew/mwh/hacks/new-style-exceptions-hacking.diff This is very much a first cut; no attempt at subtlety. The procedure went roughly "Hack until it compiles, hack until it doesn't dump core immediately, hack until most tests pass." The good news: all tests but test_pickletools pass (and that's doomed; just look at it). The bad news: I've entirely broken raising old-style classes (well, not quite: >>> try: raise C ... except types.ClassType, c: print c ... __main__.C ) so I've had to make sure various classes used in the test suite inherit from exception. There was a bunch of shallow breakage -- f'ex str(old-style-class) is quite different from str(new-style-class), which broke various output comparison tests (try not to gag when you see how I worked around this) -- but not much that's deep. You can get a *different* kind of shallow breakage by essentially removing old-style classes (changing the default metatype to type), but then 'types.ClassType is type' and this from copy_reg.py: def pickle(ob_type, pickle_function, constructor_ob=None): if type(ob_type) is _ClassType: raise TypeError("copy_reg is not intended for use with classes") rather fails to do the right thing. I didn't pursue this one very far. Obviously, a better attempt would be to allow raising any old-style class or instance or any subtype of Exception or any instance of a subtype of Exception -- but that becomes tedious to spell. I suspect that it would be quite hard -- or at least prohibitively tedious -- to write code that worked with both old- and new-style exceptions, so I'm not sure a -N switch to turn them on would work. At least not without a small battery of helper functions that noone would bother to use. I guess this means making exceptions new-style might have to wait for a Python 3.0-ish flag day of some kind. Cheers, mwh -- Roll on a game of competetive offence-taking. -- Dan Sheppard, ucam.chat From mwh at python.net Thu Aug 5 15:30:20 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 15:30:22 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <1091711049.41123049900cb@mcherm.com> (Michael Chermside's message of "Thu, 5 Aug 2004 06:04:09 -0700") References: <1091711049.41123049900cb@mcherm.com> Message-ID: <2mvffx7lw3.fsf@starship.python.net> Michael Chermside writes: > Michael Hudson writes: >> > that would be insane, >> >> class C(random.choice([dict, list])): >> pass >> >> is my favourite example of this :-) > > Where is THAT monstrocity used!? Well, it's not, I hope :-) It might occasionally be useful for proving to people that Python really doesn't believe in declarations. Cheers, mwh -- Need to Know is usually an interesting UK digest of things that happened last week or might happen next week. [...] This week, nothing happened, and we don't care. -- NTK Now, 2000-12-29, http://www.ntk.net/ From tameyer at ihug.co.nz Thu Aug 5 03:56:49 2004 From: tameyer at ihug.co.nz (Tony Meyer) Date: Thu Aug 5 16:00:11 2004 Subject: [Python-Dev] RE: [spambayes-dev] bug in imap filter or in email package In-Reply-To: Message-ID: There has been a little bit of discussion on spambayes-dev about a bug with the 2.4a1 email package, where header lines that end with \r\n are not treated correctly (the value ends up with a \r at the end). A SF bug was opened for this: [ 1002475 ] email message parser doesn't handle \r\n correctly I've created a patch to fix this, and a couple of tests to add to test_email.py: [ 1003693 ] Fix for 1002475 (Feedparser not handling \r\n correctly) If someone would like to review this and check it in after 2.4a2 is all done that would be great. Maybe someone at the bug day? (I might come along to that, but it's the middle of the night, so probably not). Thanks! =Tony Meyer From chrism at plope.com Thu Aug 5 16:04:56 2004 From: chrism at plope.com (Chris McDonough) Date: Thu Aug 5 16:05:01 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: <200408050550.i755ob1s031939@cosc353.cosc.canterbury.ac.nz> References: <200408050550.i755ob1s031939@cosc353.cosc.canterbury.ac.nz> Message-ID: <1091714696.26267.623.camel@athlon> On Thu, 2004-08-05 at 01:50, Greg Ewing wrote: > Chris McDonough : > > > The following program demonstrates that a particular usage of select > > (under Linux at least) always returns the output side of a pipe > > connected to a child process' stdout as "ready" after it gets any output > > from that child process, even if the child process has no further data > > to provide after it has provided a bit of data to the parent. > > > > Or am I doing something silly? > > > > for fd in r: > > sys.stdout.write(os.read(fd, 1024)) > > You're not taking account of the possibility of EOF. When > the child process finishes and closes its end of the pipe, > the select will always report the pipe as ready for reading, > because it is -- subsequent reads will return immediately > with 0 bytes. > > You need to check whether the read returned 0 bytes, and > take that as meaning that the child process has finished. Of course! Thanks. Let us speak no more of this. ;-) - C From martin at v.loewis.de Thu Aug 5 16:19:33 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 16:19:30 2004 Subject: [Python-Dev] RELEASED Python 2.4, alpha 2 In-Reply-To: <411231C8.3020308@interlink.com.au> References: <411231C8.3020308@interlink.com.au> Message-ID: <411241F5.3080602@v.loewis.de> Anthony Baxter wrote: > Python 2.4a2 is an alpha release. We'd greatly appreciate it if you > could download it, kick the tires and let us know of any problems you > find, but it is not suitable for production usage. The Windows installer should support upgrading from a previous Python 2.4 installation. If you have previously installed 2.4a1, you may try this out; please report any problems you find. Regards, Martin From aahz at pythoncraft.com Thu Aug 5 17:22:03 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 5 17:22:07 2004 Subject: Except that! (was Re: [Python-Dev] Python in Unicode context) In-Reply-To: <41121E87.9010805@egenix.com> References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> <41121E87.9010805@egenix.com> Message-ID: <20040805152203.GB23495@panix.com> On Thu, Aug 05, 2004, M.-A. Lemburg wrote: > > I meant that it works for the vast majority of all cases you see in > practice. I haven't seen a non-Exception based exception in years. My current company still has lots of string exceptions. :-( I'm working on changing that. (Yes, we started with Python 1.4.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From aahz at pythoncraft.com Thu Aug 5 17:26:45 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 5 17:26:48 2004 Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in Unicode context) In-Reply-To: <20040805112928.GA5208@solar.trillke> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> <20040805112928.GA5208@solar.trillke> Message-ID: <20040805152645.GC23495@panix.com> On Thu, Aug 05, 2004, Holger Krekel wrote: > Guido van Rossum wrote: >> >> (It will also break code that creates a class used as an exception >> that doesn't derive from Exception, but those should be shot. :-) > > Then i guess that searching down into a recursive structure and just > raising an "i found it" result object up doesn't count as a use case in > your book, right? It can avoid boilerplate code like return-if-not-None > checks and I have used it for e.g. finding patterns in an AST-Tree. In cases where I've done this, I've always inherited from Exception or a subclass. Is there any reason not to? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From tim.peters at gmail.com Thu Aug 5 17:28:33 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 5 17:28:42 2004 Subject: [Python-Dev] New-style exceptions In-Reply-To: <2mzn597m02.fsf_-_@starship.python.net> References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> <2mzn597m02.fsf_-_@starship.python.net> Message-ID: <1f7befae04080508286f6bcae9@mail.gmail.com> [Michael Hudson] ... > Well, it turns out to be a bit big for attaching, so it's here: > > http://starship.python.net/crew/mwh/hacks/new-style-exceptions-hacking.diff > > This is very much a first cut; no attempt at subtlety. The procedure > went roughly "Hack until it compiles, hack until it doesn't dump core > immediately, hack until most tests pass." > > The good news: all tests but test_pickletools pass (and that's doomed; > just look at it). Eh? test_pickletools is a three-line test, which just runs the doctests in pickletools.py. The only exceptions mentioned in the latter are the builtin ValueError and OverflowError. What's the problem? From mwh at python.net Thu Aug 5 17:32:00 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 17:32:01 2004 Subject: [Python-Dev] New-style exceptions In-Reply-To: <1f7befae04080508286f6bcae9@mail.gmail.com> (Tim Peters's message of "Thu, 5 Aug 2004 11:28:33 -0400") References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> <2mzn597m02.fsf_-_@starship.python.net> <1f7befae04080508286f6bcae9@mail.gmail.com> Message-ID: <2mr7ql7g9b.fsf@starship.python.net> Tim Peters writes: > [Michael Hudson] > ... >> Well, it turns out to be a bit big for attaching, so it's here: >> >> http://starship.python.net/crew/mwh/hacks/new-style-exceptions-hacking.diff >> >> This is very much a first cut; no attempt at subtlety. The procedure >> went roughly "Hack until it compiles, hack until it doesn't dump core >> immediately, hack until most tests pass." >> >> The good news: all tests but test_pickletools pass (and that's doomed; >> just look at it). > > Eh? test_pickletools is a three-line test, which just runs the > doctests in pickletools.py. The only exceptions mentioned in the > latter are the builtin ValueError and OverflowError. What's the > problem? Sorry, was too obscure. One of the doctests pickles a couple instances of PicklingError and disassembles the pickle. That's not going to stay the same past an old-style/new-style transition. Cheers, mwh -- If a train station is a place where a train stops, what's a workstation? -- unknown (to me, at least) From pyth at devel.trillke.net Thu Aug 5 17:34:43 2004 From: pyth at devel.trillke.net (Holger Krekel) Date: Thu Aug 5 17:34:47 2004 Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in Unicode context) In-Reply-To: <20040805152645.GC23495@panix.com> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> <20040805112928.GA5208@solar.trillke> <20040805152645.GC23495@panix.com> Message-ID: <20040805153443.GB5208@solar.trillke> Aahz wrote: > On Thu, Aug 05, 2004, Holger Krekel wrote: > > Guido van Rossum wrote: > >> > >> (It will also break code that creates a class used as an exception > >> that doesn't derive from Exception, but those should be shot. :-) > > > > Then i guess that searching down into a recursive structure and just > > raising an "i found it" result object up doesn't count as a use case in > > your book, right? It can avoid boilerplate code like return-if-not-None > > checks and I have used it for e.g. finding patterns in an AST-Tree. > > In cases where I've done this, I've always inherited from Exception or a > subclass. Is there any reason not to? Sure, i can probably wrap the result object into some class which inherits from Exception. My point is that I like to regard try/except as a mechanism for "out-of-band" objects. Guidos "should be shot" seems to indicate he sees try/except only useful/applicable to exception-handling. Holger P.S.: thanks for changing the subject line, should have done that earlier. From fumanchu at amor.org Thu Aug 5 17:34:43 2004 From: fumanchu at amor.org (Robert Brewer) Date: Thu Aug 5 17:39:41 2004 Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python inUnicode context) Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DDF@exchange.hqamor.amorhq.net> Holger Krekel wrote: > Sure, i can probably wrap the result object into some class > which inherits from Exception. My point is that I like to > regard try/except as a mechanism for "out-of-band" objects. > Guidos "should be shot" seems to indicate he sees try/except > only useful/applicable to exception-handling. Putting on my "language lawyer" hat: "Exceptions are a means of breaking out of the normal flow of control of a code block in order to handle errors or other exceptional conditions. An exception is raised at the point where the error is detected; it may be handled by the surrounding code block or by any code block that directly or indirectly invoked the code block where the error occurred." If "exception" is only meant for errors, then don't use it for non-errors. But if "exception" is meant for "other exceptional conditions", then why not use it for them? Sounds to me like you just need to expand your use of the term "exception", and therefore, your use of Exception(). ;) Robert Brewer MIS Amor Ministries fumanchu@amor.org From aahz at pythoncraft.com Thu Aug 5 18:04:33 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 5 18:04:42 2004 Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in Unicode context) In-Reply-To: <20040805153443.GB5208@solar.trillke> References: <20040803145345.GA6250@titan.progiciels-bpi.ca> <410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> <20040803194724.GB9031@titan.progiciels-bpi.ca> <200408032014.i73KEwp31876@guido.python.org> <20040805112928.GA5208@solar.trillke> <20040805152645.GC23495@panix.com> <20040805153443.GB5208@solar.trillke> Message-ID: <20040805160433.GA9809@panix.com> On Thu, Aug 05, 2004, Holger Krekel wrote: > Aahz wrote: >> On Thu, Aug 05, 2004, Holger Krekel wrote: >>> Guido van Rossum wrote: >>>> >>>> (It will also break code that creates a class used as an exception >>>> that doesn't derive from Exception, but those should be shot. :-) >>> >>> Then i guess that searching down into a recursive structure and just >>> raising an "i found it" result object up doesn't count as a use case in >>> your book, right? It can avoid boilerplate code like return-if-not-None >>> checks and I have used it for e.g. finding patterns in an AST-Tree. >> >> In cases where I've done this, I've always inherited from Exception or a >> subclass. Is there any reason not to? > > Sure, i can probably wrap the result object into some class which > inherits from Exception. My point is that I like to regard try/except > as a mechanism for "out-of-band" objects. Guidos "should be shot" > seems to indicate he sees try/except only useful/applicable to > exception-handling. Nope, he meant exactly what he said: an exception that doesn't derive from Exception. After all, the iterator protocol relies on similar out-of-band exceptions, and the ``for`` loop has done the same with IndexError for years. Further discussion about Pythonic exception handling should probably get moved to comp.lang.python -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From foom at fuhm.net Thu Aug 5 18:10:34 2004 From: foom at fuhm.net (James Y Knight) Date: Thu Aug 5 18:10:39 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <410F3465.9000005@object-craft.com.au> References: <410F3465.9000005@object-craft.com.au> Message-ID: On Aug 3, 2004, at 2:44 AM, Dave Cole wrote: > >>> a_list = [1, 2, 3, 4, 5] > >>> a, b, *c = a_list While you're proposing expanding the domain of the * construct of function calling to other parts of python, I'd also like to suggest the following features, to make the ** construct also applicable to the rest of the language. To be read with an 70% wink. (I wouldn't be *un*happy if these were in python, but I am not seriously proposing these features for inclusion): Dict unpacking, parallels current sequence unpacking: >>> d = {'foo': 1, 'bar': 2} >>> {'foo': x, 'bar': y} = d >>> x, y (1, 2) Dict interpolation, similar to sequence interpolation and keyword interpolation in function calls: >>> d = {'foo': 1, 'bar': 2} >>> d2 = {'baz': 3, **d} >>> d2 {'baz': 3, 'foo': 1, 'bar': 2} Combining the two: >>> d = {'foo': 1, 'bar': 2, 'baz': 3} >>> {'foo': x, 'bar': y, **d2} = d >>> x, y, d2 (1, 2, {'baz': 3}) James From guido at python.org Thu Aug 5 18:36:36 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 5 18:36:41 2004 Subject: [Python-Dev] Call for defense of @decorators Message-ID: <200408051636.i75Gaak03654@guido.python.org> It seems a public outcry against @decorators has started. When I let Anthony check it in for 2.4a2, the plan was to see how it fares in a2 and a3, and possibly take it out in 2.4b1. If you don't want that to happen, you might want to say something in public in defense of @decorators. We also need help updating the PEP; it doesn't mention @decorators, and it doesn't even begin to mention the pro and con of many other suggestions (does it say why "def decorator func(...)" was rejected?). I don't have time for any of this; I can barely make time for a few emails per day. Perhaps the @advocates can elect or volunteer a PEP editor. I really don't care if everybody thinks it's ugly. I do care to find out about usability issues. For example, it may cause problems for Leo, but I don't know how bad that is. I also want to find out about superior syntax proposals (from __future__ import decorators might be acceptable). An argument about timing: IMO there's no point in putting it off until 2.5 -- the syntax options don't get prettier, backwards compatibility issues don't get easier by waiting, and as it is it will probably take a year from 2.4's release in December before it is the mainstream Python version. --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Aug 5 18:52:28 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 18:48:21 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> Message-ID: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> At 09:36 AM 8/5/04 -0700, Guido van Rossum wrote: >I also want to find out about >superior syntax proposals (from __future__ import decorators might be >acceptable). Does this mean that the C#-style syntax has a chance if it's got a __future__? :) Also, you might want to define "superior" in order to avoid re-opening the floodgates of syntax argument. With regard to the PEP, I thought there were two volunteers who mentioned an intent to work on it in the past week; if they are not still doing so, I'd be happy to at least add the issues with "def decorator functionname()" that I remember (visual confusion for decorators w/arguments, tool confusion for existing tools). From anthony at interlink.com.au Thu Aug 5 18:49:46 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Aug 5 18:50:12 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: <4112652A.90004@interlink.com.au> Guido van Rossum wrote: > We also need help updating the PEP; it doesn't mention @decorators, > and it doesn't even begin to mention the pro and con of many other > suggestions (does it say why "def decorator func(...)" was rejected?). > I don't have time for any of this; I can barely make time for a few > emails per day. Perhaps the @advocates can elect or volunteer a PEP > editor. I'm totally out of cycles to work on the PEP any more. I've sent the changes I made to Skip, who'd previously volunteered to help. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Thu Aug 5 18:59:37 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Aug 5 19:00:05 2004 Subject: [Python-Dev] trunk frozen for 2.4a2 In-Reply-To: <41112923.8060505@interlink.com.au> References: <41112923.8060505@interlink.com.au> Message-ID: <41126779.1080308@interlink.com.au> Anthony Baxter wrote: > Please hold off on any checkins to the trunk for the next 24 > hours or so (I'll post a followup when we're done) for 2.4a2. > (The usual exception for the release team of me, Fred and > Martin applies). The release is done. Please keep the trunk frozen for another 8 hours or so, in case I need to cut an emergency fix-up release. I'll send another email in the morning to open it up again. Thanks, Anthony From mwh at python.net Thu Aug 5 19:00:23 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 19:00:25 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> (Guido van Rossum's message of "Thu, 05 Aug 2004 09:36:36 -0700") References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: <2mn0197c60.fsf@starship.python.net> Guido van Rossum writes: > It seems a public outcry against @decorators has started. When I let > Anthony check it in for 2.4a2, the plan was to see how it fares in a2 > and a3, and possibly take it out in 2.4b1. If you don't want that to > happen, you might want to say something in public in defense of > @decorators. Like what? I don't want to see that happen. Do you want justifications, too? :-) I would beg of you to not give the idea that you or anyone else is going to be counting votes on this at some point. > An argument about timing: IMO there's no point in putting it off until > 2.5 -- the syntax options don't get prettier, backwards compatibility > issues don't get easier by waiting, and as it is it will probably take > a year from 2.4's release in December before it is the mainstream > Python version. +1 Cheers, mwh -- ARTHUR: The ravenours bugblatter beast of Traal ... is it safe? FORD: Oh yes, it's perfectly safe ... it's just us who are in trouble. -- The Hitch-Hikers Guide to the Galaxy, Episode 6 From guido at python.org Thu Aug 5 19:14:48 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 5 19:14:53 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: Your message of "Thu, 05 Aug 2004 12:52:28 EDT." <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> Message-ID: <200408051714.i75HEmE03796@guido.python.org> [Phillip] > Does this mean that the C#-style syntax has a chance if it's got a > __future__? :) I don't see how that would change the arguments against it. > Also, you might want to define "superior" in order to avoid > re-opening the floodgates of syntax argument. No, but I suggest that the proponents of syntax alternatives will have to agree amongst themselves on a single alternative that they can present to me. > With regard to the PEP, I thought there were two volunteers who > mentioned an intent to work on it in the past week; if they are not > still doing so, I'd be happy to at least add the issues with "def > decorator functionname()" that I remember (visual confusion for > decorators w/arguments, tool confusion for existing tools). Please do (or coordinate with Skip who seems to have attracted this volunteer position). [Michael] > Do you want justifications, too? :-) That's up to you. :-) > I would beg of you to not give the idea that you or anyone else is > going to be counting votes on this at some point. Python is not a democracy. I can't be swayed by votes, only by good arguments. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Thu Aug 5 19:39:48 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 5 19:39:52 2004 Subject: [Python-Dev] RELEASED Python 2.4, alpha 2 In-Reply-To: <411241F5.3080602@v.loewis.de> References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de> Message-ID: <1f7befae040805103911610582@mail.gmail.com> [Martin v. L?wis] > The Windows installer should support upgrading from a previous Python > 2.4 installation. If you have previously installed 2.4a1, you may try > this out; please report any problems you find. I tried that, and it seemed to work well, although it took a while. One glitch: I'm pretty sure I selected "install only for me" in the first screen. I wanted to go back to check, but it doesn't seem possible to go back to the first screen. Anyway, the Python DLL ended up in a system directory, and that's "the glitch". I expected/wanted it to end up in my Python directory. From niemeyer at conectiva.com Thu Aug 5 19:42:52 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 19:40:38 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: <20040805174252.GA27820@burma.localdomain> [...] > We also need help updating the PEP; it doesn't mention @decorators, > and it doesn't even begin to mention the pro and con of many other > suggestions (does it say why "def decorator func(...)" was rejected?). It doesn't seem to even mention it. I also belive this is a much better syntax, which doesn't introduce a new symbol, and reads more naturally. I'd really like to see the current implemented syntax replaced by this one, but I'm not sure what's the correct way to proceed from here. > I don't have time for any of this; I can barely make time for a few > emails per day. Perhaps the @advocates can elect or volunteer a PEP > editor. > > I really don't care if everybody thinks it's ugly. I do care to find You don't care if everybody thinks this is ugly!? That's bad. > out about usability issues. For example, it may cause problems for > Leo, but I don't know how bad that is. I also want to find out about > superior syntax proposals (from __future__ import decorators might be > acceptable). How can I help defining something superior? You said you don't want voting, and that this is not a democracy, and that you don't care if the current syntax is ugly. I'm worried about this issue. -- Gustavo Niemeyer http://niemeyer.net From niemeyer at conectiva.com Thu Aug 5 19:46:19 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 19:44:01 2004 Subject: [Python-Dev] Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: <4110F829.2000609@zope.com> References: <4110F829.2000609@zope.com> Message-ID: <20040805174619.GB27820@burma.localdomain> > IMO, the most common uses of decorators will be to define properties, > and class and static methods. IMO, these uses would be better served > by a simpler syntax: > > def classmethod foo(cls, ...): > ... > > This simplified syntax only allows names to specify decorators. It > could allow multiple names, although I'm not sure it should, > > I find this *far* more readable and obvious than any of the other syntaxs > I've seen propsed. Agreed. > Those applications that *need* decorator arguments could use the more > complex pie-shaped notation. I wouldn't care to define a decorator function to introduce arguments, and force every decorator function to take a single argument. -- Gustavo Niemeyer http://niemeyer.net From martin at v.loewis.de Thu Aug 5 19:56:16 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 19:56:13 2004 Subject: [Python-Dev] RELEASED Python 2.4, alpha 2 In-Reply-To: <1f7befae040805103911610582@mail.gmail.com> References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de> <1f7befae040805103911610582@mail.gmail.com> Message-ID: <411274C0.4070300@v.loewis.de> Tim Peters wrote: > I tried that, and it seemed to work well, although it took a while. Yes. I should probably speed it up by changing the UI sequence number of RemoveExistingApplications from 1510 to 1450. In case you wonder what this means: Action 1500 is InstallInitialize; anything from then on is under transaction control. This means that installer will roll back the uninstall in case the subsequent install fails. This is expensive, as it first moves all old files out of the place, then puts the new ones in, then deletes the old ones. At 1450, the files would be deleted immediately (I believe), at the cost of not being able to undo the deinstallation if the installation later fails (e.g. by user Cancel). > One glitch: I'm pretty sure I selected "install only for me" in the > first screen. I wanted to go back to check, but it doesn't seem > possible to go back to the first screen. Yes. I don't know what to do about this: the screen sets ALLUSERS=1, then performs FindRelatedProducts to find the old installation. If the old installation was per-machine, we only find it if we commit to making the new installation also per-machine. That decision cannot be taken back; the user would have to cancel and start over. > Anyway, the Python DLL ended > up in a system directory, and that's "the glitch". I expected/wanted > it to end up in my Python directory. Ok. I will see whether I can reproduce that. Regards, Martin From foom at fuhm.net Thu Aug 5 20:04:56 2004 From: foom at fuhm.net (James Y Knight) Date: Thu Aug 5 20:05:00 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: One thing I thought of recently that would be IMO a nice feature to have is something like this: >>> @public def f(...): pass @public y = 5 <<< which would essentially set __all__ = ['f', 'y'] for you. However, that isn't possible to do with the current function-modifying-only decorators. For this to be possible, the decorator would need to get as an argument the binding (perhaps in the form of the container and the name), as well as the object being assigned to that binding. (also the grammar would have to be updated but that's relatively easy I suspect) I don't have an argument or proposal here, just putting this out to think about. James PS: I like decorators, and the pie-shape is really growing on me. :) From tim.peters at gmail.com Thu Aug 5 20:23:08 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 5 20:23:11 2004 Subject: [Python-Dev] New-style exceptions In-Reply-To: <2mr7ql7g9b.fsf@starship.python.net> References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> <2mzn597m02.fsf_-_@starship.python.net> <1f7befae04080508286f6bcae9@mail.gmail.com> <2mr7ql7g9b.fsf@starship.python.net> Message-ID: <1f7befae0408051123581f5f43@mail.gmail.com> [Michael Hudson] > Sorry, was too obscure. One of the doctests pickles a couple > instances of PicklingError and disassembles the pickle. That's not > going to stay the same past an old-style/new-style transition. That's no problem. The real point of that part of the test is to exercise the different pickle protocols on an instance of a "foreign" (not defined in the same module) class. Any foreign class would do as well Creating an instance of pickle.PicklingError was just convenient; there was no intent to pick on an exception class. IOW, if this goes forward, it's no problem to change the test. From trentm at ActiveState.com Thu Aug 5 20:23:18 2004 From: trentm at ActiveState.com (Trent Mick) Date: Thu Aug 5 20:23:35 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805174252.GA27820@burma.localdomain>; from niemeyer@conectiva.com on Thu, Aug 05, 2004 at 02:42:52PM -0300 References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> Message-ID: <20040805112318.B13062@ActiveState.com> [Gustavo Niemeyer wrote] > How can I help defining something superior? You said you don't want > voting, and that this is not a democracy, and that you don't care > if the current syntax is ugly. I'm worried about this issue. You are being disingenuous, Gustavo. Guido said he doesn't want voting, **he wants reasoned arguments**. He cares more about **usability issues** than about the prettiness of the syntax. Trent -- Trent Mick TrentM@ActiveState.com From ronaldoussoren at mac.com Thu Aug 5 20:26:42 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu Aug 5 20:26:56 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: On 5-aug-04, at 18:36, Guido van Rossum wrote: > It seems a public outcry against @decorators has started. When I let > Anthony check it in for 2.4a2, the plan was to see how it fares in a2 > and a3, and possibly take it out in 2.4b1. If you don't want that to > happen, you might want to say something in public in defense of > @decorators. The outcry seems to be the same discussion as ever :-( I'm in favor of @decorators. It's easy to notice them when the are present and they are clearly special. The '[decorator] def ...' and 'decorate(decorator) def ...' are very magic, and are IMHO unfriendly to newbies (you must metion them in any introduction to python, because otherwise users would complety mis the signicance of the decorations). My particular use-case is PyObjC, which sometimes suffers from haveing veryLong_andSometimesAlsoVeryUgly_methodNames_. Having to repeat those three times when using a decorator is no fun. The only usage for decorators in PyObjC is for specifying Objective-C method signatures, which is sometimes required for the correct functioning of methods (I do try to deduce the correct method signature from methods in base classes and Objective-C protocols, but's that not always necessary). Some examples: class ModelObject (NSObject): @objc.accessor def messageLength(self): return "XIV" @objc.signature("v@:@i") def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, returnCode, contextInfo): pass The argument to objc.signature is fairly magic and I do try to abstract it away (such as with the introduction of objc.accessor), but that's not always possible. If decorators were not allowed to have arguments I'd have to introduce temporary functions that wouldn't help in readability. The generic example PJE is introducing in PEAK also seem a good usecase for decorators with arguments. Ronald From astrand at lysator.liu.se Thu Aug 5 20:30:03 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Thu Aug 5 20:30:13 2004 Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) In-Reply-To: <1091649831.26267.527.camel@athlon> References: <200408031427.i73ERn330790@guido.python.org> <20040803213121.C9777@ActiveState.com> <1091631453.26267.390.camel@athlon> <200408041520.i74FKm001469@guido.python.org> <1091636838.26267.406.camel@athlon> <1091649831.26267.527.camel@athlon> Message-ID: > In popen5, "communicate" is terminal. It calls select until there's no > more data to get back and then unconditionally waits for the subprocess > to finish, blocking the entire time. This isn't useful for the type of Yes, I agree. Other people has request this as well. But, even though this could be useful, I do not consider it a showstopper which needs to be solved before the module can be included in the stdlib. Windows support without need for win32all is a more major issue, I think. A non-terminal "communicate" can be added later. /Peter ?strand From mwh at python.net Thu Aug 5 20:30:27 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 5 20:30:29 2004 Subject: [Python-Dev] New-style exceptions In-Reply-To: <1f7befae0408051123581f5f43@mail.gmail.com> (Tim Peters's message of "Thu, 5 Aug 2004 14:23:08 -0400") References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> <41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net> <2mzn597m02.fsf_-_@starship.python.net> <1f7befae04080508286f6bcae9@mail.gmail.com> <2mr7ql7g9b.fsf@starship.python.net> <1f7befae0408051123581f5f43@mail.gmail.com> Message-ID: <2misbx77zw.fsf@starship.python.net> Tim Peters writes: > [Michael Hudson] >> Sorry, was too obscure. One of the doctests pickles a couple >> instances of PicklingError and disassembles the pickle. That's not >> going to stay the same past an old-style/new-style transition. > > That's no problem. The real point of that part of the test is to > exercise the different pickle protocols on an instance of a "foreign" > (not defined in the same module) class. Any foreign class would do as > well Creating an instance of pickle.PicklingError was just > convenient; there was no intent to pick on an exception class. IOW, > if this goes forward, it's no problem to change the test. Uhh, sure. It was just to tell people what to expect if they were so foolish as to download the patch and try it :-) I probably didn't choose my words very well. Cheers, mwh -- This is the fixed point problem again; since all some implementors do is implement the compiler and libraries for compiler writing, the language becomes good at writing compilers and not much else! -- Brian Rogoff, comp.lang.functional From niemeyer at conectiva.com Thu Aug 5 20:34:44 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 20:32:26 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805112318.B13062@ActiveState.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> Message-ID: <20040805183444.GA29796@burma.localdomain> > [Gustavo Niemeyer wrote] > > How can I help defining something superior? You said you don't want > > voting, and that this is not a democracy, and that you don't care > > if the current syntax is ugly. I'm worried about this issue. > > You are being disingenuous, Gustavo. Guido said he doesn't want voting, > **he wants reasoned arguments**. He cares more about **usability > issues** than about the prettiness of the syntax. I've read that in his mail. Do you have something to add, besides personal insults, or just happen to have a lot of free time? -- Gustavo Niemeyer http://niemeyer.net From pje at telecommunity.com Thu Aug 5 20:41:38 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 20:37:33 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051714.i75HEmE03796@guido.python.org> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> At 10:14 AM 8/5/04 -0700, Guido van Rossum wrote: >[Phillip] > > Does this mean that the C#-style syntax has a chance if it's got a > > __future__? :) > >I don't see how that would change the arguments against it. I thought the primary argument against it was that it changes the meaning of (potentially existing) Python code, and that you had rejected the "hard to learn" argument on the basis that people learn by pattern-matching rather than principle. I guess this is another reason to update the PEP... :) >No, but I suggest that the proponents of syntax alternatives will have >to agree amongst themselves on a single alternative that they can >present to me. I think that will pretty much guarantee that it's either @ or nothing: it appears that the two biggest (or at least most vocal) camps are: 1. people who want a "simpler" syntax that doesn't support arguments (who seem to mostly favor 'def classmethod foo()') 2. people who think that decorators without arguments are pointless, and don't agree amongst themselves on the proper syntax, but don't necessarily care that much as long as there *is* one. (But there may be a slight leaning towards either of the C#-inspired variants.) Personally, I think the C# list-before-function syntax is, on balance, the best of all the choices explored throughout the whole tangled history of PEP 318. It's visually attractive, doesn't *break* the parsing of existing tools (that may see '@' as a syntax error), and doesn't interfere with grepping for functions. The "learning" argument was done to death before you even invented the syntax: *any* decorator choice results in having to learn *something*. Of course, I suppose that nearly all of these can be turned into opposite arguments, for example that '@' causing breakage is a good thing, because it means you'll get an error sooner (e.g. if you run the program under an older Python). And round and round the argument goes. And every so often, somebody invents a new syntax, for example: foo = classmethod( def foo(cls): return 42 ) and then some people love it, and some hate it, and there we go back on the merry-go-round. Sigh. Just Pronounce on *something* and put us all out of your misery already. :) From skip at pobox.com Thu Aug 5 20:38:52 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 5 20:39:08 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051714.i75HEmE03796@guido.python.org> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <200408051714.i75HEmE03796@guido.python.org> Message-ID: <16658.32444.554073.223400@montanaro.dyndns.org> >> With regard to the PEP, I thought there were two volunteers who >> mentioned an intent to work on it in the past week; if they are not >> still doing so, I'd be happy to at least add the issues with "def >> decorator functionname()" that I remember (visual confusion for >> decorators w/arguments, tool confusion for existing tools). Guido> Please do (or coordinate with Skip who seems to have attracted Guido> this volunteer position). I volunteered simply because my name is currently on the PEP as one of its authors. One of the loudest cries on c.l.py came from someone (Edward Ream I think) complaining that the PEP as it currently stands doesn't really describe the changes that were just checked in. I can't really fault him for it. Had I (or associated with the PEP in the past) been alerted, I think that particular complaint could have been avoided. I do have Anthony's latest updates and will try to factor them into the PEP this evening. If others have editing suggestions, getting them to me earlier is better than later. Skip From eppstein at ics.uci.edu Thu Aug 5 20:39:10 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Thu Aug 5 20:39:15 2004 Subject: [Python-Dev] Re: Plea for simpler decorator syntax, in addition to pie-shaped syntax References: <4110F829.2000609@zope.com> <20040805174619.GB27820@burma.localdomain> Message-ID: In article <20040805174619.GB27820@burma.localdomain>, Gustavo Niemeyer wrote: > > IMO, the most common uses of decorators will be to define properties, > > and class and static methods. IMO, these uses would be better served > > by a simpler syntax: > > > > def classmethod foo(cls, ...): > > ... > > > > This simplified syntax only allows names to specify decorators. It > > could allow multiple names, although I'm not sure it should, > > > > I find this *far* more readable and obvious than any of the other syntaxs > > I've seen propsed. > > Agreed. Disagreed. It works fine when the decorator is as short as "classmethod" and the function signature is as short as "foo(cls, ...)". It breaks down when those are long enough that the whole thing doesn't fit on a single line, which I'm expecting will happen a reasonably large fraction of the time. I like pie. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From niemeyer at conectiva.com Thu Aug 5 20:48:45 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 20:46:30 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: <20040805184845.GB29796@burma.localdomain> Hello Ronald, > I'm in favor of @decorators. It's easy to notice them when the are > present and they are clearly special. The '[decorator] def ...' and Why are they special? Why should they be more important than any other part of the function definition? > 'decorate(decorator) def ...' are very magic, and are IMHO unfriendly > to newbies (you must metion them in any introduction to python, because > otherwise users would complety mis the signicance of the decorations). [...] > @objc.signature("v@:@i") > def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, > returnCode, contextInfo): > pass > > The argument to objc.signature is fairly magic and I do try to abstract > it away (such as with the introduction of objc.accessor), but that's > not always possible. If decorators were not allowed to have arguments > I'd have to introduce temporary functions that wouldn't help in > readability. Is special syntax in the language really required in this case, considering you're already doing something "fairly magic" anyways? What is objc.signature() doing? > The generic example PJE is introducing in PEAK also seem a good usecase > for decorators with arguments. Any pointers? I'd really like to "see the light" of complex decorators, as oposed to KISS, and use the current language features to implement that. -- Gustavo Niemeyer http://niemeyer.net From bob at redivi.com Thu Aug 5 20:59:04 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Aug 5 20:59:13 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805184845.GB29796@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> Message-ID: <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> On Aug 5, 2004, at 2:48 PM, Gustavo Niemeyer wrote: > Hello Ronald, > >> I'm in favor of @decorators. It's easy to notice them when the are >> present and they are clearly special. The '[decorator] def ...' and > > Why are they special? Why should they be more important than any other > part of the function definition? Because they take a function object as input and can do whatever they want with it and return something else. >> 'decorate(decorator) def ...' are very magic, and are IMHO unfriendly >> to newbies (you must metion them in any introduction to python, >> because >> otherwise users would complety mis the signicance of the decorations). > [...] >> @objc.signature("v@:@i") >> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, >> returnCode, contextInfo): >> pass >> >> The argument to objc.signature is fairly magic and I do try to >> abstract >> it away (such as with the introduction of objc.accessor), but that's >> not always possible. If decorators were not allowed to have arguments >> I'd have to introduce temporary functions that wouldn't help in >> readability. > > Is special syntax in the language really required in this case, > considering you're already doing something "fairly magic" anyways? The alternative would be (current syntax and current PyObjC) this: def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, returnCode, contextInfo): pass saveSheetDidDismiss_returnCode_contextInfo_ = objc.selector(saveSheetDidDismiss_returnCode_contextInfo_, signature='v@:@i') Yes, we pretty much do need special syntax (without using a hack like PJE's). Code like this is pretty commonplace in PyObjC projects. >> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, > What is objc.signature() doing? objc.signature wraps the function object with an objc.selector that specifies specific return and argument types. In this particular case, it declares that the selector saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an object and an integer as arguments. Without this, the selector can not be bridged correctly to the Objective C runtime and the program would crash. The ctypes package behaves similarly and would use decorators for the same thing. I imagine that other runtime/language bridges would also benefit from similar techniques (Jython, IronPython, Python.NET, JPython.. or whatever else). I can also imagine it being used for things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar manner. -bob From niemeyer at conectiva.com Thu Aug 5 21:05:49 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 21:03:33 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805183444.GA29796@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> Message-ID: <20040805190549.GC29796@burma.localdomain> > I've read that in his mail. Do you have something to add, > besides personal insults, or just happen to have a lot of > free time? Michael Hudson asked me to calm down, so here I am. I don't want to start a fight here, so I'm sorry if I took this too strongly. I just was a little bit disappointed to receive this kind of answer as the first comment to an email I sent as a way to try to collaborate. Unfortunately, everyone seems to be sick about this matter, and I'm still disappointed with the introduced syntax. I just wanted to be sure that such an important change won't be a reason to regret soon, and that being sick of something won't be a reason to introduce changes in the language without extensive debate (even if it takes longer than what is usually acceptable). Is there something to be done that could change the current decision? Voting? Collecting pros/cons and classifying them? If there's nothing to be done, I'll just give up. -- Gustavo Niemeyer http://niemeyer.net From ronaldoussoren at mac.com Thu Aug 5 21:07:18 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu Aug 5 21:07:53 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805184845.GB29796@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> Message-ID: On 5-aug-04, at 20:48, Gustavo Niemeyer wrote: > Hello Ronald, > >> I'm in favor of @decorators. It's easy to notice them when the are >> present and they are clearly special. The '[decorator] def ...' and > > Why are they special? Why should they be more important than any other > part of the function definition? > >> 'decorate(decorator) def ...' are very magic, and are IMHO unfriendly >> to newbies (you must metion them in any introduction to python, >> because >> otherwise users would complety mis the signicance of the decorations). > [...] >> @objc.signature("v@:@i") >> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, >> returnCode, contextInfo): >> pass >> >> The argument to objc.signature is fairly magic and I do try to >> abstract >> it away (such as with the introduction of objc.accessor), but that's >> not always possible. If decorators were not allowed to have arguments >> I'd have to introduce temporary functions that wouldn't help in >> readability. > > Is special syntax in the language really required in this case, > considering you're already doing something "fairly magic" anyways? > > What is objc.signature() doing? The argument is fairly magic, in that most people wouldn't know how the interpret it. The function itself is easy enough: it creates a custom method object. The meta-class for Objective-C classes extracts the method signature from that method object and uses it build the right method description on the Objective-C side of the bridge. A special syntax for descriptors is not stricly necessary, we are after all already using them with Python 2.2 and Python 2.3 code. It does make life a lot easier, the method names used in my example are not particularly long, I sometimes uses much longer names (because I have to [*]). Having 3 copies of such a name increases the likelyhood of errors, and makes it harder to spot that all 3 names are the same. To stretch a point: class syntax is also not necessary, instead of a class definition you can call type() with the right arguments. Does that mean we should do away with classes? > >> The generic example PJE is introducing in PEAK also seem a good >> usecase >> for decorators with arguments. > > Any pointers? > > I'd really like to "see the light" of complex decorators, as oposed > to KISS, and use the current language features to implement that. Whoops, I used the wrong word, I meant 'generic functions' instead of 'generic example'. He's doing something like this: @when("isinstance(db, OracleDB") def storeInDB(db, object): pass @when("isinstance(db, BerkelyDB") def storeInDB(db, object): pass The 'when' decorator converts the function into a generic function that uses the expressions to dispatch to the right function implementation at runtime. Ronald [*] "have too" because I haven't found a good way to automaticly convert Objective-C method names in sensible shorter names. Objective-C has segmented method names, like smalltalk. This looks a little like keyword arguments: @interface MyModel : NSObject {} -(int)sheetDidEnd:(NSSheet*)sheet returnCode:(int)code contextInfo:(void*)context; @end This looks like a method with 2 keyword arguments, but really isn't. The method name is 'sheetDid:returnCode:contextInfo:'. I convert that to a python identifier by replacing colons by underscores. That's an easy to remember rule, which means I don't have to replicate the vast amount of Cocoa documentation, but you do end up with some ungainly method names. The only way I see to fix this is to introduce segmented method in Python, but I don't think that would fly :-) :-) From martin at v.loewis.de Thu Aug 5 21:07:55 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 21:08:10 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805184845.GB29796@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> Message-ID: <4112858B.90406@v.loewis.de> Gustavo Niemeyer wrote: > Is special syntax in the language really required in this case, > considering you're already doing something "fairly magic" anyways? The syntax is not special for the specific application (Objective-C), but special for a class of applications (modifications of/annotations to the subsequent function). > What is objc.signature() doing? I'm only guessing here: it declares an Objective-C signature (i.e. a type) of the function. This is probably necessary for thunking. I can't see a way to do that in a comprehensible way without a decorator. It applies to the function, and it probably needs to wrap the function. So without a special syntax, it would have to go after the function. This is counter-intuitive, as the Objective-C signature has to correspond (I guess) with the Python signature, so it is desirable that they are closely together. It might be possible to put that annotation into the __doc__ string, but that would be abusing doc strings, and might be more error-prone. Regards, Martin From FBatista at uniFON.com.ar Thu Aug 5 21:05:37 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Thu Aug 5 21:09:50 2004 Subject: [Python-Dev] Call for defense of @decorators Message-ID: [Gustavo Niemeyer] #- Is there something to be done that could change the current #- decision? Voting? Collecting pros/cons and classifying them? Guido said it. All the people that doesn't like the actual state of things could choose together an alernative syntax and propose it to him. Then, I guess, he'll choose. Be aware that he wants good reasons, ;) Anyway, I think that administering a voting/brainstorm to choose another syntax is a good idea. Maybe in this list, maybe in another list, maybe on IRC, as you like. . Facundo From niemeyer at conectiva.com Thu Aug 5 21:17:10 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 21:14:56 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> Message-ID: <20040805191710.GD29796@burma.localdomain> Hi Bob, > >Why are they special? Why should they be more important than any other > >part of the function definition? > > Because they take a function object as input and can do whatever they > want with it and return something else. This seems extremely powerful. OTOH, perl is also powerful. > >>def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, > > >What is objc.signature() doing? > > objc.signature wraps the function object with an objc.selector that > specifies specific return and argument types. In this particular case, > it declares that the selector > saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an > object and an integer as arguments. Without this, the selector can not > be bridged correctly to the Objective C runtime and the program would > crash. Isn't metaclass usage helpful in this case? Or a perhaps a dictionary? __signature__ = {"funcname": "v@:@i"} or def funcname(...): ... funcname.signature = "v@:@i" and a postprocessor like: objc.register(classname) > The ctypes package behaves similarly and would use decorators for the > same thing. I imagine that other runtime/language bridges would also > benefit from similar techniques (Jython, IronPython, Python.NET, > JPython.. or whatever else). I can also imagine it being used for > things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar manner. Is this something good? I mean, having function wrappers all around the place? Wouldn't it be introducing unobvious code (magic?) in packages which are working fine as they are? Should I just throw away my t-shirt with the "zen of python" text? :-) -- Gustavo Niemeyer http://niemeyer.net From ixokai at gmail.com Thu Aug 5 21:24:14 2004 From: ixokai at gmail.com (IxokaI) Date: Thu Aug 5 21:24:20 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? Message-ID: So, I have been following Python-dev fairly closely ever since my company decided to rewrite everything in Python, and are now very interested in the future. Keeping track of the decorator debate has been a bit of a pain, and with the outcry against @decorators now and everyone throwing out the current proposals-- well, I thought maybe I could help, since I was keeping notes anyways. Someone said the PEP is out of date, and asked it to be updated with Pro's and Cons of the various ideas out there. So I added what I have to the Python Wiki; personally I think it'd be a good place for people to edit in their pro-con thoughts. I left out all the proposals on the PEP because they appear to all be rejected, although I don't remember why. I just got through devouring the hundred or so messages on @decorators made in the last week or so to find the various information I appended. http://www.python.org/moin/PythonDecorators I added "with", although I havn't seen it. "using" would work too. I personally do not like punctuation syntax-- I'd really like to see something english, even with the hated addition of a keyword. I understand Guido's disapproval for "as" so wanted to think of an alternate. --Stephen From pje at telecommunity.com Thu Aug 5 21:32:04 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 21:28:01 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805191710.GD29796@burma.localdomain> References: <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> Message-ID: <5.1.1.6.0.20040805152516.02a983c0@mail.telecommunity.com> At 04:17 PM 8/5/04 -0300, Gustavo Niemeyer wrote: > > The ctypes package behaves similarly and would use decorators for the > > same thing. I imagine that other runtime/language bridges would also > > benefit from similar techniques (Jython, IronPython, Python.NET, > > JPython.. or whatever else). I can also imagine it being used for > > things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar manner. > >Is this something good? I mean, having function wrappers all >around the place? Wouldn't it be introducing unobvious code (magic?) >in packages which are working fine as they are? > >Should I just throw away my t-shirt with the "zen of python" text? :-) This special syntax would be entirely unnecessary if Python function definitions (other than lambda) were expressions. Apart from that syntactic issue, decorators are nothing new, and people already use them. Is it better to bury the magic *after* the function body, or put it before? In every other programming language I know of where functions are first class objects, one applies decorators *before* the function body. The advantage of a specialized decorator syntax over simply wrapping decorator calls around function bodies, is that it's possible to reduce the nestedness of the expression, e.g.: @binding.Make(uponAssembly=True) @events.taskFactory def some_method(self): ... versus: binding.Make(events.taskFactory( def some_method(self): ... ), uponAssembly=True ) or today's: def some_method(self): ... some_method = binding.Make(events.TaskFactory(some_method),uponAssembly=True) Given that '...' can be arbitrarily long, and "flat is better than nested", this looks like a net win for this style of programming. Metaprogramming is an important part of Python, that has always been in Python and always should be. Given that, it is better to have a visible syntax that precedes the function body, and this is the fundamental basis for the existence of PEP 318. From ronaldoussoren at mac.com Thu Aug 5 21:28:13 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu Aug 5 21:28:34 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805191710.GD29796@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> Message-ID: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> On 5-aug-04, at 21:17, Gustavo Niemeyer wrote: > Hi Bob, > >>> Why are they special? Why should they be more important than any >>> other >>> part of the function definition? >> >> Because they take a function object as input and can do whatever they >> want with it and return something else. > > This seems extremely powerful. OTOH, perl is also powerful. They should IMHO be a part of the function definition because they are part of the specification of what the function will do. In the current situation decorators are not part of the function definition. > >>>> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, >> >>> What is objc.signature() doing? >> >> objc.signature wraps the function object with an objc.selector that >> specifies specific return and argument types. In this particular >> case, >> it declares that the selector >> saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an >> object and an integer as arguments. Without this, the selector can >> not >> be bridged correctly to the Objective C runtime and the program would >> crash. > > Isn't metaclass usage helpful in this case? > > Or a perhaps a dictionary? > > __signature__ = {"funcname": "v@:@i"} This moves the signature away from the function definition, meaning you have to take care to keep them synchronized. > > or > > def funcname(...): > ... > funcname.signature = "v@:@i" That should be workable for this specific example. It wouldn't work for the objc.accessor example. The objc.accessor function/decorator deduces the right kind of signature from the name and arguments of the function. > > and a postprocessor like: > > objc.register(classname) We already use a metaclass, which is working just fine ;) > >> The ctypes package behaves similarly and would use decorators for the >> same thing. I imagine that other runtime/language bridges would also >> benefit from similar techniques (Jython, IronPython, Python.NET, >> JPython.. or whatever else). I can also imagine it being used for >> things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar >> manner. > > Is this something good? I mean, having function wrappers all > around the place? Wouldn't it be introducing unobvious code (magic?) > in packages which are working fine as they are? Nobody is saying anything about modifying existing modules. Decorators might make it easier to write objects that implement a COM or AppleScript interface, that doesn't mean we have to convert existing modules into COM objects. > > Should I just throw away my t-shirt with the "zen of python" text? :-) Please don't, people need to be reminded of the zen sometimes :-) Ronald From bob at redivi.com Thu Aug 5 21:28:58 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Aug 5 21:29:08 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805191710.GD29796@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> Message-ID: On Aug 5, 2004, at 3:17 PM, Gustavo Niemeyer wrote: > Hi Bob, > >>> Why are they special? Why should they be more important than any >>> other >>> part of the function definition? >> >> Because they take a function object as input and can do whatever they >> want with it and return something else. > > This seems extremely powerful. OTOH, perl is also powerful. List comprehensions, generators, and generator expressions are also very powerful. Let's get rid of those too. What good is *args, **kwargs syntax if we can do it with apply? Why do we need a syntax to build dictionaries if we can just dict() and then set all of the keys one at a time like in most other languages? >>>> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, >> >>> What is objc.signature() doing? >> >> objc.signature wraps the function object with an objc.selector that >> specifies specific return and argument types. In this particular >> case, >> it declares that the selector >> saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an >> object and an integer as arguments. Without this, the selector can >> not >> be bridged correctly to the Objective C runtime and the program would >> crash. > > Isn't metaclass usage helpful in this case? > > Or a perhaps a dictionary? > > __signature__ = {"funcname": "v@:@i"} > > or > > def funcname(...): > ... > funcname.signature = "v@:@i" > > and a postprocessor like: > > objc.register(classname) The signature is tightly bound to the arguments that the function takes. The farther you move it away from the arguments the easier it is to make a difficult to diagnose mistake (in the case of PyObjC, a segfault hopefully either upon call or return, but quite possibly some time later). Metaclasses also do not always play very well with each other. Decorators are much simpler than metaclasses in concept and implementation. >> The ctypes package behaves similarly and would use decorators for the >> same thing. I imagine that other runtime/language bridges would also >> benefit from similar techniques (Jython, IronPython, Python.NET, >> JPython.. or whatever else). I can also imagine it being used for >> things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar >> manner. > > Is this something good? I mean, having function wrappers all > around the place? Wouldn't it be introducing unobvious code (magic?) > in packages which are working fine as they are? I think you're mistaken if you think that all of these sorts of things work just great as-is. It should be pretty obvious what a decorator with a sensible name does, just like when you subclass something or raise a particular kind of exception it should be pretty obvious what behavior that is going to have. I know what "raise SystemExit" does, just like I know what @objc.signature(...) or @classmethod would do. -bob From colanderman at gmail.com Thu Aug 5 21:31:40 2004 From: colanderman at gmail.com (Chris King) Date: Thu Aug 5 21:31:41 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> Message-ID: <875c7e07040805123136fe96de@mail.gmail.com> On Thu, 5 Aug 2004 21:07:18 +0200, Ronald Oussoren wrote: > > On 5-aug-04, at 20:48, Gustavo Niemeyer wrote: > > What is objc.signature() doing? > > The argument is fairly magic, in that most people wouldn't know how the > interpret it. The function itself is easy enough: it creates a custom > method object. The meta-class for Objective-C classes extracts the > method signature from that method object and uses it build the right > method description on the Objective-C side of the bridge. Isn't this more a use case for function attributes, rather than decorators? Decorators seem like overkill in this case. From guido at python.org Thu Aug 5 21:32:15 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 5 21:32:22 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: Your message of "Thu, 05 Aug 2004 14:41:38 EDT." <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> Message-ID: <200408051932.i75JWFj04174@guido.python.org> > At 10:14 AM 8/5/04 -0700, Guido van Rossum wrote: > >[Phillip] > > > Does this mean that the C#-style syntax has a chance if it's got a > > > __future__? :) > > > >I don't see how that would change the arguments against it. > > I thought the primary argument against it was that it changes the meaning > of (potentially existing) Python code, and that you had rejected the "hard > to learn" argument on the basis that people learn by pattern-matching > rather than principle. No, the reason I decided to drop that was was the ambiguity in people's heads. > I guess this is another reason to update the PEP... :) Indeed. > >No, but I suggest that the proponents of syntax alternatives will > >have to agree amongst themselves on a single alternative that they > >can present to me. > > I think that will pretty much guarantee that it's either @ or > nothing: it appears that the two biggest (or at least most vocal) > camps are: > > 1. people who want a "simpler" syntax that doesn't support arguments (who > seem to mostly favor 'def classmethod foo()') Tough beans. They should have a look at how decorators are used in C# and Java 1.5. > 2. people who think that decorators without arguments are pointless, and > don't agree amongst themselves on the proper syntax, but don't necessarily > care that much as long as there *is* one. (But there may be a slight > leaning towards either of the C#-inspired variants.) So they should defend @ because it's there. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 5 21:35:58 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 5 21:36:05 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: Your message of "Thu, 05 Aug 2004 16:05:37 -0300." References: Message-ID: <200408051935.i75JZw304222@guido.python.org> > Anyway, I think that administering a voting/brainstorm to choose another > syntax is a good idea. Maybe in this list, maybe in another list, maybe on > IRC, as you like. Ah yes, IRC, the most democratic medium. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From nidoizo at yahoo.com Thu Aug 5 21:39:56 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu Aug 5 21:40:02 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: References: Message-ID: IxokaI wrote: > I added "with", although I havn't seen it. "using" would work too. I > personally do not like punctuation syntax-- I'd really like to see > something english, even with the hated addition of a keyword. I > understand Guido's disapproval for "as" so wanted to think of an > alternate. Is there ambiguities with the following? (I don't want to propose them, but I would like to at least add them to Wiki, even to specify why they don't work or why we hate them): def decorator1(decoratorN(foo))(arg1, argN): pass def decorator1(decoratorN(foo(arg1, argN))): pass def(decorator1, decoratorN) foo(arg1, argN): pass And is the [decorator] list syntax before or after the "def"? Regards, Nicolas From tim.peters at gmail.com Thu Aug 5 21:40:05 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 5 21:40:08 2004 Subject: [Python-Dev] Re: Plea for simpler decorator syntax, in addition to pie-shaped syntax In-Reply-To: References: <4110F829.2000609@zope.com> <20040805174619.GB27820@burma.localdomain> Message-ID: <1f7befae04080512401906a0db@mail.gmail.com> [Gustavo Niemeyer] >>> IMO, the most common uses of decorators will be to define properties, >>> and class and static methods. IMO, these uses would be better served >>> by a simpler syntax: >>> >>> def classmethod foo(cls, ...): >>> ... I've seen this example several times today, and I have to say that every time I've seen it, my unstoppable gut reaction was "wait, why are they defining their own classmethod function here?!". I've had that problem since the first time this syntax vairant was suggested (loooooong ago), and it's not going away. Maybe it's 10+ years of "in Python, the name of the function comes after the 'def'" and I just can't adjust that to qualify "but is the *last* name after a 'def' preceding the first left paren following the 'def'"; or maybe it's because I've written God-only-knows how many tools that believe the same thing (the Emacs python-mode parser; the IDLE parser; any number of one-shot cheap-ass regexps). Whatever, I can't get used to it. So, sorry, but I like @classmethod def foo(cls, ...): unboundedly better than that. For that matter, I like it at least as well as any alternative to date, and better than most. I actively hate putting stuff between 'the def' and the function name. Then again, I'm old . From pje at telecommunity.com Thu Aug 5 21:45:35 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 21:41:28 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: Message-ID: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com> At 12:24 PM 8/5/04 -0700, IxokaI wrote: >I added "with", although I havn't seen it. Guido's reserving "with" for this purpose in some future Python: with x.y: .z = spam # set x.y.z = spam print .q.r # print x.y.q.r I updated the Wiki page, and added another syntax, the original "def function() [decorator]:" syntax. Interestingly, it looks like one of the options with the most pluses and fewest minuses of any syntax you've listed on that page; it's only ugly for long decorator definitions, and Guido said he didn't care if a syntax was ugly. :) From nas at arctrix.com Thu Aug 5 21:45:23 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Thu Aug 5 21:45:27 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> Message-ID: <20040805194523.GA20561@mems-exchange.org> On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote: > On 5-aug-04, at 21:17, Gustavo Niemeyer wrote: > > def funcname(...): > > ... > > funcname.signature = "v@:@i" > > That should be workable for this specific example. Even nicer if '_' is bound to the last function defined. class SomeClass(objc): def funcname(...): ... _.signature = "v@:@i" The objc metaclass could take the 'signature' function attribute and transform the function. > It wouldn't work for the objc.accessor example. The objc.accessor > function/decorator deduces the right kind of signature from the > name and arguments of the function. Can't the metaclass do that? Neil From ixokai at gmail.com Thu Aug 5 21:45:42 2004 From: ixokai at gmail.com (IxokaI) Date: Thu Aug 5 21:45:45 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: References: Message-ID: On Thu, 05 Aug 2004 15:39:56 -0400, Nicolas Fleury wrote: > IxokaI wrote: > > I added "with", although I havn't seen it. "using" would work too. I > > personally do not like punctuation syntax-- I'd really like to see > > something english, even with the hated addition of a keyword. I > > understand Guido's disapproval for "as" so wanted to think of an > > alternate. > > Is there ambiguities with the following? (I don't want to propose them, > but I would like to at least add them to Wiki, even to specify why they > don't work or why we hate them): > > def decorator1(decoratorN(foo))(arg1, argN): pass > def decorator1(decoratorN(foo(arg1, argN))): pass > def(decorator1, decoratorN) foo(arg1, argN): pass > Ack. I'd *never* use those syntaxes :) They are not ambigious, but just... complicated. The function name is too hidden, and really, that is in most cases the most important piece of information on that line. > And is the [decorator] list syntax before or after the "def"? Sorry, i was refeering to the C# like before. I'll fix. --Stephen From barry at python.org Thu Aug 5 21:47:28 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 5 21:47:21 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805194523.GA20561@mems-exchange.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <20040805194523.GA20561@mems-exchange.org> Message-ID: <1091735248.8540.81.camel@localhost> On Thu, 2004-08-05 at 15:45, Neil Schemenauer wrote: > Even nicer if '_' is bound to the last function defined. Except, that doesn't play nicely with i18n/gettext. ;/ -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/6971a015/attachment.pgp From edreamleo at charter.net Thu Aug 5 21:47:45 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Thu Aug 5 21:47:58 2004 Subject: [Python-Dev] About pep 318--Intro Message-ID: <002a01c47b25$144a5a40$6700a8c0@computer> I received the following from GvR in a private communication: > If good arguments against @ are presented it may be removed in 2.4b1 (that was always the plan but apparently not publicized). I am going to take this as an invitation to discuss @ syntax again. Given the controversy, a reexamination of the major issues seems not only reasonable, but desirable. Please feel free to send me links to relevant discussion, but given the apparent lack of consensus at present I would not like to be told to read the entire py-dev archives. > Please bring up the Leo issues in Python-dev. I'll be posting on separate threads soon. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From niemeyer at conectiva.com Thu Aug 5 21:52:01 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 21:49:44 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> Message-ID: <20040805195201.GE29796@burma.localdomain> > >This seems extremely powerful. OTOH, perl is also powerful. > > They should IMHO be a part of the function definition because they are > part of the specification of what the function will do. In the current > situation decorators are not part of the function definition. Ok, I understand that for objc case, extra magic is needed to make the system work. OTOH, is this kind of magic usual enough to justify adding extra syntax to the language? Won't it encourage usage of this facility abusively (as was just mentioned with SOAP, XML-RPC, Jython, IronPython, ...), making it inherently less readable? For example, someone mentioned the following URL: http://rafb.net/paste/results/bKBUem36.html Which provides: def magicprop(f): return property(*f()) class Foo(object): @magicprop def bar(): def get(self): ... def set(self): ... return get, set Notice that bar() is actually *called* inside the decorator. > >Or a perhaps a dictionary? > > > > __signature__ = {"funcname": "v@:@i"} > > This moves the signature away from the function definition, meaning you > have to take care to keep them synchronized. This might be spelled as: class C: signature = {} signature["f"] = "@v:@p" def f(self): pass > >or > > > > def funcname(...): > > ... > > funcname.signature = "v@:@i" > > That should be workable for this specific example. It wouldn't work for > the objc.accessor example. The objc.accessor function/decorator deduces > the right kind of signature from the name and arguments of the > function. So why does it need to be run together with the function definition? > >and a postprocessor like: > > > >objc.register(classname) > > We already use a metaclass, which is working just fine ;) Cool! Glad to know about another interesting metaclass usage. [...] > >>JPython.. or whatever else). I can also imagine it being used for > >>things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar > >>manner. > > > >Is this something good? I mean, having function wrappers all > >around the place? Wouldn't it be introducing unobvious code (magic?) > >in packages which are working fine as they are? > > Nobody is saying anything about modifying existing modules. Decorators > might make it easier to write objects that implement a COM or > AppleScript interface, that doesn't mean we have to convert existing > modules into COM objects. Yes, that's what I was talking about. Current modules already work fine, and I haven't seen usage cases that would benefit from the complex (or, more powerful) decorator syntax in a way that couldn't be easily spelled some other way. -- Gustavo Niemeyer http://niemeyer.net From pje at telecommunity.com Thu Aug 5 21:54:09 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 21:50:02 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <875c7e07040805123136fe96de@mail.gmail.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> Message-ID: <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com> At 03:31 PM 8/5/04 -0400, Chris King wrote: >On Thu, 5 Aug 2004 21:07:18 +0200, Ronald Oussoren > wrote: > > > > On 5-aug-04, at 20:48, Gustavo Niemeyer wrote: > > > What is objc.signature() doing? > > > > The argument is fairly magic, in that most people wouldn't know how the > > interpret it. The function itself is easy enough: it creates a custom > > method object. The meta-class for Objective-C classes extracts the > > method signature from that method object and uses it build the right > > method description on the Objective-C side of the bridge. > >Isn't this more a use case for function attributes, rather than >decorators? Decorators seem like overkill in this case. That argument has been done to death several times in the last year here. Function attributes aren't a replacement for decorators. From ronaldoussoren at mac.com Thu Aug 5 21:54:17 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu Aug 5 21:54:41 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805194523.GA20561@mems-exchange.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <20040805194523.GA20561@mems-exchange.org> Message-ID: <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com> On 5-aug-04, at 21:45, Neil Schemenauer wrote: > On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote: >> On 5-aug-04, at 21:17, Gustavo Niemeyer wrote: >>> def funcname(...): >>> ... >>> funcname.signature = "v@:@i" >> >> That should be workable for this specific example. > > Even nicer if '_' is bound to the last function defined. > > class SomeClass(objc): > def funcname(...): > ... > _.signature = "v@:@i" > > The objc metaclass could take the 'signature' function attribute and > transform the function. > >> It wouldn't work for the objc.accessor example. The objc.accessor >> function/decorator deduces the right kind of signature from the >> name and arguments of the function. > > Can't the metaclass do that? Not every method is a property accessor. The metaclass does calculate a default signature (all arguments are objects and the result is an object) when it doesn't have better information (such as the signature of a method with the same name in a superclass). Sadly enough that isn't good enough for property setters. Some parts of Cocoa require that a setter returns 'void'. I'm trying to push the automatic detection of the right signature as far as I can get to avoid having to use explicit annotations in user code, but sometimes you just cannot avoid adding annotations. Ronald -- X|support bv http://www.xsupport.nl/ T: +31 610271479 F: +31 204416173 From ronaldoussoren at mac.com Thu Aug 5 21:56:14 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu Aug 5 21:56:41 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com> Message-ID: <8190C1A5-E719-11D8-89DE-000D93AD379E@mac.com> On 5-aug-04, at 21:54, Phillip J. Eby wrote: > >> Isn't this more a use case for function attributes, rather than >> decorators? Decorators seem like overkill in this case. > > That argument has been done to death several times in the last year > here. Function attributes aren't a replacement for decorators. I couldn't agree more. Ronald From ronaldoussoren at mac.com Thu Aug 5 21:59:19 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu Aug 5 21:59:31 2004 Subject: [Python-Dev] About pep 318--Intro In-Reply-To: <002a01c47b25$144a5a40$6700a8c0@computer> References: <002a01c47b25$144a5a40$6700a8c0@computer> Message-ID: On 5-aug-04, at 21:47, Edward K. Ream wrote: > > Please feel free to send me links to relevant discussion, but given the > apparent lack of consensus at present I would not like to be told to > read > the entire py-dev archives. Only the hunderds of messages about decorator syntax would do. That would avoid rehashing the same points over and over and over again without adding new information. Ronald From pje at telecommunity.com Thu Aug 5 22:04:56 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 22:00:49 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805194523.GA20561@mems-exchange.org> References: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> Message-ID: <5.1.1.6.0.20040805155650.02a99c60@mail.telecommunity.com> At 03:45 PM 8/5/04 -0400, Neil Schemenauer wrote: >On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote: > > On 5-aug-04, at 21:17, Gustavo Niemeyer wrote: > > > def funcname(...): > > > ... > > > funcname.signature = "v@:@i" > > > > That should be workable for this specific example. > >Even nicer if '_' is bound to the last function defined. > > class SomeClass(objc): > def funcname(...): > ... > _.signature = "v@:@i" > >The objc metaclass could take the 'signature' function attribute and >transform the function. > > > It wouldn't work for the objc.accessor example. The objc.accessor > > function/decorator deduces the right kind of signature from the > > name and arguments of the function. > >Can't the metaclass do that? Arguing that decorators shouldn't exist isn't going to get much traction; the function attribute and metaclass objections have been argued to death numerous times previously. If you have to raise these arguments again, please make a Wiki page so they can get answered at *most* once more, and then we can copy the answers from the Wiki into the PEP. (This probably should have happened the first, second, or third times these arguments came around, and next time I will at least try to remember that the second time I answer the same argument I should darn well do so in either the PEP or the Wiki.) From nidoizo at yahoo.com Thu Aug 5 22:02:44 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu Aug 5 22:02:49 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: References: Message-ID: IxokaI wrote: > Ack. I'd *never* use those syntaxes :) They are not ambigious, but > just... complicated. The function name is too hidden, and really, that > is in most cases the most important piece of information on that line. Ok, now I understand. 1.If the syntax is "before def", it is outside the definition. 2.If the syntax is "just after def keyword", it hides the function name. 3.If the syntax is "after definition", it cause ugly line wraps. Maybe it's me, but I expect that me and beginners from C++ would use mostly use things like staticmethod and that makes me prefer #2 and #3 solutions. Regards, Nicolas From barry at python.org Thu Aug 5 22:03:16 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 5 22:03:07 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051932.i75JWFj04174@guido.python.org> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> <200408051932.i75JWFj04174@guido.python.org> Message-ID: <1091736196.8540.91.camel@localhost> On Thu, 2004-08-05 at 15:32, Guido van Rossum wrote: > > 2. people who think that decorators without arguments are pointless, and > > don't agree amongst themselves on the proper syntax, but don't necessarily > > care that much as long as there *is* one. (But there may be a slight > > leaning towards either of the C#-inspired variants.) > > So they should defend @ because it's there. I hate repeating myself, but I will anyway. :) I'm in camp 2, but now that pie decorators are in, and I've had a chance to convert my code to use them, I'm strongly +1 in favor of this syntax. It stands out nicely, and to me indicates a stronger affinity to the def that follows it than the C# syntax. I was never in favor of C# syntax, and I'm glad it wasn't chosen. I strongly disliked that it subtly changed the semantics of currently valid Python. I like that pie decorators code cannot run in older Pythons, because if it /could/ it certainly wouldn't work. 'scuze-me-while-i-eat-the-pie-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/af59ded4/attachment.pgp From guido at python.org Thu Aug 5 22:11:12 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 5 22:11:18 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: Your message of "Thu, 05 Aug 2004 16:04:56 EDT." <5.1.1.6.0.20040805155650.02a99c60@mail.telecommunity.com> References: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <5.1.1.6.0.20040805155650.02a99c60@mail.telecommunity.com> Message-ID: <200408052011.i75KBCc04451@guido.python.org> > If you have to raise these arguments again, please make a Wiki page > so they can get answered at *most* once more, and then we can copy > the answers from the Wiki into the PEP. (This probably should have > happened the first, second, or third times these arguments came > around, and next time I will at least try to remember that the > second time I answer the same argument I should darn well do so in > either the PEP or the Wiki.) Right. That's what a PEP is *supposed* to do. But we've been slacking on the PEP... --Guido van Rossum (home page: http://www.python.org/~guido/) From nas at arctrix.com Thu Aug 5 22:11:40 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Thu Aug 5 22:11:43 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <20040805194523.GA20561@mems-exchange.org> <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com> Message-ID: <20040805201139.GA20768@mems-exchange.org> On Thu, Aug 05, 2004 at 09:54:17PM +0200, Ronald Oussoren wrote: > Not every method is a property accessor. The metaclass does calculate a > default signature (all arguments are objects and the result is an > object) when it doesn't have better information (such as the signature > of a method with the same name in a superclass). Sorry I wasn't clearer. What I meant was if we have a nice syntax for setting function attributes then the metaclass to transform the functions just like a decorator could. Neil From nidoizo at yahoo.com Thu Aug 5 22:14:59 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu Aug 5 22:15:04 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: References: Message-ID: Nicolas Fleury wrote: > 1.If the syntax is "before def", it is outside the definition. > 2.If the syntax is "just after def keyword", it hides the function name. > 3.If the syntax is "after definition", it cause ugly line wraps. I just discovered a #4 on the Wiki: "at beginning of function body syntax". Best to me... From ronaldoussoren at mac.com Thu Aug 5 22:16:16 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu Aug 5 22:16:35 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805201139.GA20768@mems-exchange.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <20040805194523.GA20561@mems-exchange.org> <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com> <20040805201139.GA20768@mems-exchange.org> Message-ID: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com> On 5-aug-04, at 22:11, Neil Schemenauer wrote: > On Thu, Aug 05, 2004 at 09:54:17PM +0200, Ronald Oussoren wrote: >> Not every method is a property accessor. The metaclass does calculate >> a >> default signature (all arguments are objects and the result is an >> object) when it doesn't have better information (such as the signature >> of a method with the same name in a superclass). > > Sorry I wasn't clearer. What I meant was if we have a nice syntax > for setting function attributes then the metaclass to transform the > functions just like a decorator could. And how would that solve this: Ronald From niemeyer at conectiva.com Thu Aug 5 22:22:45 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 22:20:27 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <1091736196.8540.91.camel@localhost> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> <200408051932.i75JWFj04174@guido.python.org> <1091736196.8540.91.camel@localhost> Message-ID: <20040805202245.GH29796@burma.localdomain> > > So they should defend @ because it's there. > > I hate repeating myself, but I will anyway. :) Thanks. > I'm in camp 2, but now that pie decorators are in, and I've had a chance > to convert my code to use them, I'm strongly +1 in favor of this > syntax. It stands out nicely, and to me indicates a stronger affinity > to the def that follows it than the C# syntax. I'm glad to see people happy with the current implementation. > I was never in favor of C# syntax, and I'm glad it wasn't chosen. I > strongly disliked that it subtly changed the semantics of currently > valid Python. I like that pie decorators code cannot run in older > Pythons, because if it /could/ it certainly wouldn't work. I'm strongly against "I'm sick of it" and "I don't care [anymore]" implemented features. Seeing everyone complaining and no positive reactions to the current implementation certainly feels like something is wrong. -- Gustavo Niemeyer http://niemeyer.net From colanderman at gmail.com Thu Aug 5 22:21:53 2004 From: colanderman at gmail.com (Chris King) Date: Thu Aug 5 22:21:55 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <20040805194523.GA20561@mems-exchange.org> <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com> <20040805201139.GA20768@mems-exchange.org> <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com> Message-ID: <875c7e0704080513213cc4add0@mail.gmail.com> On Thu, 5 Aug 2004 22:16:16 +0200, Ronald Oussoren wrote: > And how would that solve this: > It wouldn't. Pattern matching would, though. But that's an argument for another day ;) (Read: I don't know which side of that argument I'm on) From pf_moore at yahoo.co.uk Thu Aug 5 22:23:28 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Thu Aug 5 22:23:24 2004 Subject: [Python-Dev] Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: Guido van Rossum writes: > It seems a public outcry against @decorators has started. When I let > Anthony check it in for 2.4a2, the plan was to see how it fares in a2 > and a3, and possibly take it out in 2.4b1. If you don't want that to > happen, you might want to say something in public in defense of > @decorators. OK then, they get +1 from me, FWIW. I don't intend to try to drown out the outcry - there have been far too many messages on this subject already. However, I will say this much: I believe that the functionality is worthwhile. Decorators add a new type of expressiveness which will take a while to show its full benefits. Until people get over the syntax issue, we won't see the real benefits coming through - and that won't be classmethod and the like, but more creative ideas that will take time to gel. Sure, we'll see some abuses, but my feeling is that there will be some genuinely useful decorators, which quite possibly would never have been identified without the language support directing people's thoughts to the idea of decorators. All the arguments are over syntax, but in my view, syntax is not the issue. Paul. -- Instant gratification takes too long -- Carrie Fisher From edreamleo at charter.net Thu Aug 5 22:32:59 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Thu Aug 5 22:33:08 2004 Subject: [Python-Dev] Questions about '@' in pep 318 Message-ID: <004a01c47b2b$65d19260$6700a8c0@computer> The "mandate" for this post is the following from GvR: > If good arguments against @ are presented it may be removed in 2.4b1 (that was always the plan but apparently not publicized). So I would like seriously to consider whether removing '@' might be a good idea. Note: my major concern with pep 318 has always been with syntax. I am not interested or qualified to discuss whether annotation in general is a good idea... My main questions: 1. Is '@' syntax required to get the job done cleanly? If the answer is clearly Yes, then consensus ought to be fairly easy, it seems to me. 2. If '@' is not required, why isn't it forbidden :-) There have been various statements to the effect that "almost every conceivable syntax alternative has been considered". I'd like to know if the following has been considered, and if so, what the objections to it were/are. Instead of: @accepts(int, (int,float)) Do: from __future__ import annotation as annotate # or a or whatever looks good. annotate.accepts(int, (int,float)) Clearly, with enough work the Python compiler could recognize this. Yes, it's a special case, but so what? This would avoid most problems with reserved words or keywords. And it would be in the spirit of letting modules encapsulate most major features. In other words, what ever happened to namespaces? 3. How "reliable" are the following design goals from pep 318? * be easy to type; programs that use it are expected to use it very frequently * not make it more difficult to scan through code quickly. It should still be easy to search for all definitions, a particular definition, or the arguments that a function accepts. - If visibility of code is a concern, use import annotation as XXXXX. - If brevity is wanted, use import annotation as a. I wonder why these design goals are even present. 4. Assuming that there are advantages to '@', why do they outweigh the disadvantages of changing the following paragraph of the Reference Manual? http://docs.python.org/ref/delimiters.html [quote] The following printing ASCII characters are not used in Python. Their occurrence outside string literals and comments is an unconditional error: @ $ ? [end quote] I regard any change to this paragraph as a major change to Python, in spite of its innocent appearance. Indeed, this paragraph is an excellent "line in the sand" for other tools. The pep contains the following design goal: * not needlessly complicate secondary support tools such as language-sensitive editors and other "toy parser tools out there. How is this not going to happen with the '@' syntax? All answers to these questions would be most appreciated. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From pje at telecommunity.com Thu Aug 5 22:48:09 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 22:44:04 2004 Subject: [Python-Dev] A usability argument for list-after-def Message-ID: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Guido has stated he'll accept one community syntax proposal as an alternative to "@". He's also ruled out his own previous proposal (list-before-def), and a variety of new-keyword alternatives involving such things as 'as' and 'with'. There doesn't appear to be a downside for decorator supporters to put forth another syntax, if they like it better than "@"; it seems the worst case is that we still end up with "@", if we end up with any syntax at all. Of the options on the Wiki page ( at http://www.python.org/moin/PythonDecorators ), this seems to leave only list-after-def, the previous community favorite and the first PEP 318 syntax to have been implemented. Guido's principal argument against list-after-def, if I recall correctly, was that it is ugly when multiple or lengthy decorators are involved. But, "ugly" isn't an argument any more, so that shouldn't rule out list-after-def. :) I only have one usability argument to favor list-after-def over "@": the former looks more like executable pseudocode than the latter. If I pretend that I do not know Python, that I am a programmer who has never written in Python and is reading a Python program found in a magazine or downloaded from somewhere, then if I look at: @events.taskFactory def monitorProcess(self,process): ... It is not immediately obvious that the previous line is part of the function definition; I am put off reading this code, because how do I know what "@" does? Brackets are normally used in English to indicate an annotation or "aside" of some sort; such as a commentary *about* the surrounding material. When I see this: def monitorProcess(self,process) [events.taskFactory]: ... My impression is that the decorator is somehow annotating or subscripting the function. In this specific example, I might wonder if this is how Python specifies a function return type. But if I see: def do_something(cls,data) [classmethod]: ... I am quickly alerted to this being some kind of annotation about the method itself, probably akin to public/private/static/friend etc. in other languages. And, if I look for the definition of 'events.taskFactory' or 'classmethod', I will quickly realize that this is an extensible annotation facility, as well. By contrast: @classmethod def do_something(cls,data): ... is also suggestive, but I personally find it far less conclusive, even with 'classmethod' as a semantic cue. Drawing on my experience of other languages and typical uses of "@", I think: * Is this some sort of array thing? (Perl) * Is this some sort of documentation tag? (Java... but then I think no, it's not in a comment, so that can't be it) * Is this some sort of escape character for literate programming? (e.g. FunnelWeb and other such tools) * Is it being used in place of the word "at"? (But what does it mean to be "at" a class method? It's on a different line; surely if it were related there would be some sort of syntactic indication like indentation or a colon or braces or a comma or *something* between it and the definition?) After some weighing of the evidence, I think I would most likely guess that this was some sort of documentation markup, rather than semantic markup. But my train of thought in reading the code has now been quite disrupted, and if I was reading this code casually I might be discouraged from continuing, unless I had some reason for wanting to learn Python besides just reading the code in question or hacking up a quick fix to something. Now, I don't know if my thought process here is at all typical or representative of anything, other than my own idiosyncratic self. But, it is the one nagging thing that I have against "@", and I suspect that it may be implicitly behind a lot of the general outcry against it. In other words, I think that "ugly" is just a pejorative term for the emotional reaction engendered by being forced to go through the above mental process for figuring it out. And, to a lesser extent, I think list-before-def required some of the same mental processing overhead, although I personally think it was somewhat less than that of "@". So there it is, my personal argument for list-after-def as better than "@", even if the list is over multiple lines: it's still much more clearly part of the function definition, and that IMO is the most important thing the syntax should convey, apart from listing the decorators themselves. From tjreedy at udel.edu Thu Aug 5 22:48:06 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Thu Aug 5 22:48:12 2004 Subject: [Python-Dev] Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org><20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> Message-ID: "Ronald Oussoren" >My particular use-case is PyObjC, which sometimes suffers from haveing >veryLong_andSometimesAlsoVeryUgly_methodNames_. Having to repeat those >three times when using a decorator is no fun. I did not get this at first (see below). I believe you mean more specifically "Having to repeat those three times when using the old no-special-deco-syntax post call is no fun." "Bob Ippolito" followed with > The alternative would be (current syntax and current PyObjC) this: > > def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, > returnCode, contextInfo): > pass > saveSheetDidDismiss_returnCode_contextInfo_ = > objc.selector(saveSheetDidDismiss_returnCode_contextInfo_, > signature='v@:@i') Aha! Now I get it (perhaps for the second time in an extended discussion). One argument for extended assignment was and is that not writing names, especially long names, twice makes for easier reading and fewer bugs. The argument for not writing names *three* times, is even stronger. If this rationale against the status quo (which many objectors on c.l.py favor) is not currently in the pep, I think it should be. Terry J. Reedy From edreamleo at charter.net Thu Aug 5 22:48:48 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Thu Aug 5 22:49:02 2004 Subject: [Python-Dev] The impact of '@' on Leo Message-ID: <005e01c47b2d$9c0dee80$6700a8c0@computer> The mandate for this post is the following from GvR: > Please bring up the Leo issues in Python-dev. The major concern I have with '@' syntax as it relates to Leo is the potential for ongoing and never-ending conflicts between Leo's directives and @x constructs in Python. Leo presently supports the following directives: @ (followed by one or more whitespace characters) @all, @asis, @c, @code, @color, @comment, @delims, @doc, @encoding, @end_raw, @file, @first, @header, @ignore, @killcolor, @language, @last, @lineending, @nocolor, @noheader, @noref, @nosent, @nowrap, @others, @pagewidth, @path, @quiet, @raw, @root, @root-code, @root-doc, @silent, @tabwidth, @terse, @thin, @unit, @verbose, @wrap Worse, Leo allows plugins to define their own directives. There is already a plugin that defines @wiki. Furthermore, @run, @test and @suite are also used in special contexts. Leo's users could probably live with a small, fixed set of @x constructs in Python. But if the Python programmer can create new @x identifiers then they have a big, big problem. What happens when another Python @x construct conflicts with one of these directives or some other directive? These directives act as a markup meta-language. Many of these directives are found interspersed in the code: a conflict with a construct in Python would have most unhappy consequences. To make matters worse, one of the _best_ choices I ever made in Leo was to remove all "escape" conventions regarding '@' (and other noweb constructs). This took a bit of courage, but it has turned out to be clearly the right choice. But if Python can now generate arbitrary @x strings Leo will have to do something really drastic. There are a number of markup languages that use '@' to from the underlying language (Python) to the markup language. I was always under the impression that the clear statement in the Reference Manual that at signs are invalid everywhere (except in comments and strings) was a clear signal of an intention to keep those symbols for other valid purposes. I do hope this won't change. Edward P.S. I recently wrote a pretty-printer for Python using the tokenize module. What will be the status of the '@' token? Is it still an "errortoken" ? Sure, this is a minor point, but it _will_ affect other tools. EKR -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From martin at v.loewis.de Thu Aug 5 22:49:21 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 22:49:26 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <8190C1A5-E719-11D8-89DE-000D93AD379E@mac.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com> <8190C1A5-E719-11D8-89DE-000D93AD379E@mac.com> Message-ID: <41129D51.6000300@v.loewis.de> Ronald Oussoren wrote: >> That argument has been done to death several times in the last year >> here. Function attributes aren't a replacement for decorators. > > > I couldn't agree more. While I agree with that statement, I think it was unhelpful. Chris did not want to know whether function attributes can, in general, replace decorators. Instead, he wanted to know whether the *specific* decorator could have been replaced with a function attribute. I don't really know the answer to that question, but my guess is: It could have. Regards, Martin From nas at arctrix.com Thu Aug 5 22:53:16 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Thu Aug 5 22:53:19 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <20040805194523.GA20561@mems-exchange.org> <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com> <20040805201139.GA20768@mems-exchange.org> <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com> Message-ID: <20040805205316.GA20997@mems-exchange.org> On Thu, Aug 05, 2004 at 10:16:16PM +0200, Ronald Oussoren wrote: > On 5-aug-04, at 22:11, Neil Schemenauer wrote: > >What I meant was if we have a nice syntax for setting function > >attributes then the metaclass to transform the functions just > >like a decorator could. > > And how would that solve this: > Is that supposed to be a good example how to use decorators? Anyhow, obviously a metaclass won't work for module level functions (unless you had a way of setting the metaclass of the module). Neil From martin at v.loewis.de Thu Aug 5 22:53:28 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 22:53:31 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <875c7e07040805123136fe96de@mail.gmail.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <875c7e07040805123136fe96de@mail.gmail.com> Message-ID: <41129E48.7050607@v.loewis.de> Chris King wrote: > Isn't this more a use case for function attributes, rather than > decorators? Decorators seem like overkill in this case. I don't fully understand the issues, but I believe that the declaration is more like a statement than a mere declaration (very similar to all other "declarations" in Python). In the specific case, I believe declaring the Objective-C signature also performs some kind of registration of the function with the Objective-C run-time. This cannot be done with function attributes; you would still need to invoke a do_register()/generate_wrappers() functions at some point. Even if this is a mere declaration, function attributes would harm readability: they would have to go after the function, whereas you want the Objective-C signature close to the Python signature. Regards, Martin From niemeyer at conectiva.com Thu Aug 5 22:55:59 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 22:54:02 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <004a01c47b2b$65d19260$6700a8c0@computer> References: <004a01c47b2b$65d19260$6700a8c0@computer> Message-ID: <20040805205559.GA31288@burma.localdomain> > The "mandate" for this post is the following from GvR: > > > If good arguments against @ are presented it may be removed in 2.4b1 > > (that was always the plan but apparently not publicized). > > So I would like seriously to consider whether removing '@' might be a > good idea. I'd be glad to find a good way to do so. The point is that people agree to disagree with most proposals (and I'm no exception here). > Note: my major concern with pep 318 has always been with syntax. I am > not interested or qualified to discuss whether annotation in general > is a good idea... I'm concerned about both, but there're people with good usage cases for it, and it has been decided that decorators are going in, so syntax is what is left to be discussed. > My main questions: > > 1. Is '@' syntax required to get the job done cleanly? > > If the answer is clearly Yes, then consensus ought to be fairly easy, > it seems to me. Unfortunately, it's not. > 2. If '@' is not required, why isn't it forbidden :-) Because it's not clearly "no". :-) [...] > Instead of: > > @accepts(int, (int,float)) > > Do: > > from __future__ import annotation as annotate # or a or whatever looks > good. > > annotate.accepts(int, (int,float)) I don't understand. How are you changing/annotating the function? [...] > - If visibility of code is a concern, use import annotation as XXXXX. > > - If brevity is wanted, use import annotation as a. > > I wonder why these design goals are even present. We're unable to agree into more obvious issues, I belive it'll be mostly impossible to agree in such highly subjective issues. > 4. Assuming that there are advantages to '@', why do they outweigh the > disadvantages of changing the following paragraph of the Reference > Manual? > > http://docs.python.org/ref/delimiters.html > > [quote] > The following printing ASCII characters are not used in Python. Their > occurrence outside string literals and comments is an unconditional error: > @ $ ? > [end quote] Good point. This paragraph should be changed, if it wasn't yet. > I regard any change to this paragraph as a major change to Python, in > spite of its innocent appearance. Indeed, this paragraph is an > excellent "line in the sand" for other tools. The pep contains the > following design goal: > > * not needlessly complicate secondary support tools such as > language-sensitive editors and other "toy parser tools out there. > > How is this not going to happen with the '@' syntax? Every tool which deals with Python's syntax must be adapted to the new syntax, no matter which new syntax is adopted. -- Gustavo Niemeyer http://niemeyer.net From martin at v.loewis.de Thu Aug 5 23:01:13 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:01:21 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805202245.GH29796@burma.localdomain> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> <200408051932.i75JWFj04174@guido.python.org> <1091736196.8540.91.camel@localhost> <20040805202245.GH29796@burma.localdomain> Message-ID: <4112A019.3080000@v.loewis.de> Gustavo Niemeyer wrote: > I'm strongly against "I'm sick of it" and "I don't care [anymore]" > implemented features. Seeing everyone complaining and no positive > reactions to the current implementation certainly feels like > something is wrong. Well, I am happy that some sort of syntax was implemented. I don't consider syntax enormously important, so it does not matter much to me which syntax is chosen. Regards, Martin From colanderman at gmail.com Thu Aug 5 23:01:38 2004 From: colanderman at gmail.com (Chris King) Date: Thu Aug 5 23:01:43 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <41129E48.7050607@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <875c7e07040805123136fe96de@mail.gmail.com> <41129E48.7050607@v.loewis.de> Message-ID: <875c7e07040805140174141409@mail.gmail.com> On Thu, 05 Aug 2004 22:53:28 +0200, "Martin v. L?wis" wrote: > Chris King wrote: > > Isn't this more a use case for function attributes, rather than > > decorators? Decorators seem like overkill in this case. > > In the specific case, I believe declaring the Objective-C > signature also performs some kind of registration of the > function with the Objective-C run-time. This cannot be done > with function attributes; you would still need to invoke > a do_register()/generate_wrappers() functions at some point. I held off posting that at first because of this (since I am not at all familiar with PyObjC), but then Ronald mentioned something about the registration and processing being done via a metaclass, rather than by the decorators themselves. > Even if this is a mere declaration, function attributes would > harm readability: they would have to go after the function, whereas > you want the Objective-C signature close to the Python signature. I should have been more specific -- I meant function attributes of this form (proposed and subsequently rejected somewhere else a while ago): def foo(a,b,c): .signature = 'v@:@i' pass I only bring this up because most (though I realize not all) use cases for decorators (at least in their current form) seem to involve some type of attribute or metadata. From tjreedy at udel.edu Thu Aug 5 23:03:17 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Thu Aug 5 23:03:24 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? References: Message-ID: "Nicolas Fleury" wrote in message news:ceu2ec$ugo$1@sea.gmane.org... > IxokaI wrote: > Is there ambiguities with the following? > > def decorator1(decoratorN(foo))(arg1, argN): pass > def decorator1(decoratorN(foo(arg1, argN))): pass > def(decorator1, decoratorN) foo(arg1, argN): pass If you name the decos decorator1, etc, not really ambigous, mearly hard to find the foo in the soup. But try def bar(baz(foo))(a1,aN): pass and it becomes easily ambigous. Anyway, by current Python syntax, decoN(foo) look like a function call on an *existing* object foo. The current situation is that name(arg) is always a func call except when name *immediately* follows def. Like Tim, I really don't want that to change. Terry J. Reedy From barry at python.org Thu Aug 5 23:06:51 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 5 23:06:44 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805202245.GH29796@burma.localdomain> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> <200408051932.i75JWFj04174@guido.python.org> <1091736196.8540.91.camel@localhost> <20040805202245.GH29796@burma.localdomain> Message-ID: <1091740011.8540.130.camel@localhost> On Thu, 2004-08-05 at 16:22, Gustavo Niemeyer wrote: > I'm strongly against "I'm sick of it" and "I don't care [anymore]" > implemented features. Seeing everyone complaining and no positive > reactions to the current implementation certainly feels like > something is wrong. Let me be clear. I want decorator support in Python 2.4. I would have accepted any of the syntaxes that made Guido's final cut, and I trust Guido to do the right thing. By choosing pie decorators, I think he has, and I will happily use them. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/74c8d9d5/attachment.pgp From martin at v.loewis.de Thu Aug 5 23:07:25 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:07:35 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805190549.GC29796@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> Message-ID: <4112A18D.5090801@v.loewis.de> Gustavo Niemeyer wrote: > Is there something to be done that could change the current > decision? Voting? Collecting pros/cons and classifying them? Depends on in what direction you want to make a change. It appears you want to avoid introducing any kind of syntax change. In that case, you should explain people how to spell classmethod and synchronized in a more convenient way, because that is what the stated motivation of PEP 318 is - you would have to explain why this motive is bad, irrelevant, or otherwise not a worthy goal. Or you could argue on a procedural basis: regardless of whether the feature is good or bad, the current implementation is unacceptable, as the PEP does not correspond with the implementation, the syntax is undocumented, the code has no test cases, and so on. I'm actually going to do that, because I do think the process is unacceptable, and should be either corrected or reversed (of course, this says nothing about the feature itself, or the code implementing it). Regards, Martin From pje at telecommunity.com Thu Aug 5 23:14:45 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 5 23:10:37 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <875c7e07040805140174141409@mail.gmail.com> References: <41129E48.7050607@v.loewis.de> <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <875c7e07040805123136fe96de@mail.gmail.com> <41129E48.7050607@v.loewis.de> Message-ID: <5.1.1.6.0.20040805171011.02a662e0@mail.telecommunity.com> At 05:01 PM 8/5/04 -0400, Chris King wrote: >I only bring this up because most (though I realize not all) use cases >for decorators (at least in their current form) seem to involve some >type of attribute or metadata. They may seem to, but that's only because most decorator proponents mostly refrain from using their current decorators in examples, for the sake of brevity. Decorators in PEAK, for example, create specialized descriptors for managing things such as asynchronous tasks, lazy attribute initialization, wrapping methods in transactions, automatic type conversions on assignment to attributes, etc. They're useful enough to motivate usage even without a decorator syntax, but they have little to do with "metadata" in the usual sense, since they *implement* what is specified by the "metadata", rather than simply holding the metadata for some class or other object to read and interpret. From bac at OCF.Berkeley.EDU Thu Aug 5 23:15:17 2004 From: bac at OCF.Berkeley.EDU (Brett Cannon) Date: Thu Aug 5 23:15:17 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? Message-ID: <4112A365.1000203@ocf.berkeley.edu> I am actually getting failures from test_timeout's testConnectTimeout since the Net connection I have at school lets me connect to Google in under the .001 connection timeout value. If I push it to .00000001 (a hundred-millionth of a second) it then fails consistently. Now the problem is that the second part of the test uses this and a fuzz value (currently at 2) to see if the test timed out within the proper amount of time. The comparison is basically if the amount of time it took to do the timed out failure is less than timeout + fuzz. So lowering this number could possibly affect the test, although at .001 seconds, I am doubting that will occur. But since these types of timing tests can be touchy I thought I would double-check. So if anyone thinks it is bad to lower the value to .00000001 then please let me know. Otherwise I will lower the value before the next alpha goes out. -Brett P.S.: the only other failures I have on OS X right now is test_curses (because I used -uall) and test__locale . Should we disable test__locale on OS X to shut it up since Martin seems to think the test isn't all that useful and won't work for OS X ever? From fdrake at acm.org Thu Aug 5 23:16:09 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu Aug 5 23:16:27 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <20040805205559.GA31288@burma.localdomain> References: <004a01c47b2b$65d19260$6700a8c0@computer> <20040805205559.GA31288@burma.localdomain> Message-ID: <200408051716.09292.fdrake@acm.org> On Thursday 05 August 2004 04:55 pm, Gustavo Niemeyer wrote: > Good point. This paragraph should be changed, if it wasn't yet. Now changed in CVS. > Every tool which deals with Python's syntax must be adapted > to the new syntax, no matter which new syntax is adopted. I expect many tools won't be affected by @decorations. Import checkers and function/method identifying tools probably won't care about the decorations. In the end, it very much depends on the assumptions of the tool and how it implements scanning and parsing. -Fred -- Fred L. Drake, Jr. From barry at python.org Thu Aug 5 23:19:53 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 5 23:19:44 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112A18D.5090801@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> Message-ID: <1091740792.8546.133.camel@localhost> On Thu, 2004-08-05 at 17:07, "Martin v. L?wis" wrote: > Or you could argue on a procedural basis: regardless of whether > the feature is good or bad, the current implementation is > unacceptable, as the PEP does not correspond with the > implementation, the syntax is undocumented, the code has no test > cases, and so on. I'm actually going to do that, because I do > think the process is unacceptable, and should be either corrected > or reversed (of course, this says nothing about the feature itself, > or the code implementing it). Martin makes a good point. Guido could threaten to remove the feature by beta 1 (and thus for 2.4 final) if the PEP is not brought up to date. not-that-i'm-volunteering-'cause-i'd-rather-see-it-done-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/e8230d4a/attachment.pgp From martin at v.loewis.de Thu Aug 5 23:20:57 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:21:00 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <004a01c47b2b$65d19260$6700a8c0@computer> References: <004a01c47b2b$65d19260$6700a8c0@computer> Message-ID: <4112A4B9.4010907@v.loewis.de> Edward K. Ream wrote: > 2. If '@' is not required, why isn't it forbidden :-) Because it is better than all alternatives. To see that, you would have to produce a list of all alternatives, and compare them one-by-one. > Instead of: > > @accepts(int, (int,float)) > > Do: > > from __future__ import annotation as annotate # or a or whatever looks good. > > annotate.accepts(int, (int,float)) That requires all annotations to be in the __future__ module. This is not acceptable, as there is a desire to provide user-defined annotations as well. > * be easy to type; programs that use it are expected to use it very > frequently > > * not make it more difficult to scan through code quickly. It should still > be easy to search for all definitions, a particular definition, or the > arguments that a function accepts. > > - If visibility of code is a concern, use import annotation as XXXXX. > > - If brevity is wanted, use import annotation as a. > > I wonder why these design goals are even present. The primary application for this is staticmethod, classmethod, and an imaginary synchronized declarator. They need to be easy to type because that is the intended improvement over the status quo, where you need to write foo = staticmethod(foo) after the function. They also should not be in the way of reading the function proper, hence the second requirement. > 4. Assuming that there are advantages to '@', why do they outweigh the > disadvantages of changing the following paragraph of the Reference Manual? > > http://docs.python.org/ref/delimiters.html > > [quote] > The following printing ASCII characters are not used in Python. Their > occurrence outside string literals and comments is an unconditional error: > @ $ ? > [end quote] I believe @ was chosen precisely for that reason - it is a character that is not yet used, so it cannot be confused with something else. > I regard any change to this paragraph as a major change to Python, in spite > of its innocent appearance. No doubt about that. This is a significant change. > * not needlessly complicate secondary support tools such as > language-sensitive editors and other "toy parser tools out there. > > How is this not going to happen with the '@' syntax? It's of course hard to tell how tools will react to that syntax, or with any other syntax change, for that matter. This specific is likely easy to implement, because it likely has only minimal impact on people's parsers: one can essentially treat a line starting with @ like a statement, e.g. by making @ a keyword. Tools that only look for parts of a Python code are likely interested in lines starting with a proper Python keyword (e.g. def, class, import) - those tools are not affected at all. Regards, Martin From bob at redivi.com Thu Aug 5 23:23:07 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Aug 5 23:23:21 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <1091740011.8540.130.camel@localhost> References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> <200408051932.i75JWFj04174@guido.python.org> <1091736196.8540.91.camel@localhost> <20040805202245.GH29796@burma.localdomain> <1091740011.8540.130.camel@localhost> Message-ID: On Aug 5, 2004, at 5:06 PM, Barry Warsaw wrote: > On Thu, 2004-08-05 at 16:22, Gustavo Niemeyer wrote: > >> I'm strongly against "I'm sick of it" and "I don't care [anymore]" >> implemented features. Seeing everyone complaining and no positive >> reactions to the current implementation certainly feels like >> something is wrong. > > Let me be clear. I want decorator support in Python 2.4. I would have > accepted any of the syntaxes that made Guido's final cut, and I trust > Guido to do the right thing. By choosing pie decorators, I think he > has, and I will happily use them. My sentiments exactly. -bob From martin at v.loewis.de Thu Aug 5 23:24:35 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:24:38 2004 Subject: [Python-Dev] The impact of '@' on Leo In-Reply-To: <005e01c47b2d$9c0dee80$6700a8c0@computer> References: <005e01c47b2d$9c0dee80$6700a8c0@computer> Message-ID: <4112A593.7080608@v.loewis.de> Edward K. Ream wrote: > The major concern I have with '@' syntax as it relates to Leo is the > potential for ongoing and never-ending conflicts between Leo's directives > and @x constructs in Python. Can you give an example of a file that would be difficult to understand under that change? If you allow @foo inside a Python file, it is not Python anymore, is it? Regards, Martin From jjl at pobox.com Thu Aug 5 23:25:59 2004 From: jjl at pobox.com (John J Lee) Date: Thu Aug 5 23:26:06 2004 Subject: [Python-Dev] No handlers could be found for logger "cookielib" In-Reply-To: <20040805094437.32296.qmail@web25403.mail.ukl.yahoo.com> References: <20040805094437.32296.qmail@web25403.mail.ukl.yahoo.com> Message-ID: On Thu, 5 Aug 2004, [iso-8859-1] Vinay Sajip wrote: > I just made a change to test_logging to clean up after > itself, so that it removes all handlers that it > creates. test_urllib2 seems to have handler creation > commented out in NetworkTests.setUp(). A combination > of the two may explain why the message shows up now. FWLIW: The handler creation in test_urllib2.py has been 'if 0'-ed out since that code was added to test_urllib2.py on the last bug day. It's there for debugging purposes, as the comment in that TestCase method points out. I just ran the entire test suite, so that Tim doesn't feel too lonely ;-) John From skip at pobox.com Thu Aug 5 23:29:21 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 5 23:29:37 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings Message-ID: <16658.42673.542930.749915@montanaro.dyndns.org> I really hate to disrupt this scintillating function decorator discussion... At work the goal for our C++ build environment is "no warnings with gcc -Wall". We are not there yet (and thankfully I am not a participant in that particular holy grail), but those involved in that effort encountered a warning the other day generated because _XOPEN_SOURCE is defined in pyconfig.h. Unfortunately GCC also defines it, so a "macro redefined" warning is emitted when compiling with GCC and including Python.h. I'm not about to suggest that the _XOPEN_SOURCE definition be removed from pyconfig.h.in. I trust that Martin v. L?wis and others involved have good reasons to leave it in. Perhaps adding #pragma GCC system_header to pyconfig.h.in would be acceptable though. This tells GCC that the file it appears in is a system header file and to not bitch about trivial stuff like macro redefinitions. This suggestion comes from one of the C++ experts here. I would never have figured it out myself. The scope of the pragma is only the file in which it appears, and it doesn't take effect for even that file until it's first encountered. This suggests that perhaps a slight restructuring of pyconfig.h.in would help to reduce its effect even more: #pragms GCC system_header (Pyconfig.h.in is almost there now.) Some exceptions to the above structure might be necessary if the definition of a macro in the first group is conditioned on the value of a macro from the second group. I don't believe this is the case, at least not at the moment. It's unlikely to ever be the case since any smarts for such tests more appropriately belong in the configure script (or would be hand-coded on non-configure-type platforms). I've adjusted my local copy of pyconfig.h.in and am building CVS HEAD and the 2.3 maintenance branch on both Solaris and MacOSX to see if any problems arise. I'd appreciate some feedback on whether or not this is deemed a reasonable idea though. Thx, Skip From martin at v.loewis.de Thu Aug 5 23:29:35 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:29:40 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <875c7e07040805140174141409@mail.gmail.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <875c7e07040805123136fe96de@mail.gmail.com> <41129E48.7050607@v.loewis.de> <875c7e07040805140174141409@mail.gmail.com> Message-ID: <4112A6BF.4040004@v.loewis.de> Chris King wrote: > I should have been more specific -- I meant function attributes of > this form (proposed and subsequently rejected somewhere else a while > ago): I see. So you would suggest to make a different extension *instead* of the currently proposed one. > def foo(a,b,c): > .signature = 'v@:@i' > pass > > I only bring this up because most (though I realize not all) use cases > for decorators (at least in their current form) seem to involve some > type of attribute or metadata. Yes. Remember that Python has no declarations, though - there is, in general, no separation between the definition of a property, and its "enforcement". For example, classmethod, staticmethod are not considered by the class, but declarations that have behaviour themselves. If the container (e.g. the class) would have to evaluate the attribute, it would need to know all attributes in advance - something which is not really possible. Many other languages have taken the path of making some of these attributes built-in (which is what you are essentially proposing) only to find out that this is insufficient, and needed to be extended by a more general mechanism later. Regards, Martin From nidoizo at yahoo.com Thu Aug 5 23:29:52 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu Aug 5 23:29:57 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: References: Message-ID: IxokaI wrote: > I added "with", although I havn't seen it. "using" would work too. I > personally do not like punctuation syntax-- I'd really like to see > something english, even with the hated addition of a keyword. I > understand Guido's disapproval for "as" so wanted to think of an > alternate. Other crazy ideas (in case it inspires anyone): accepts(int,int,def) returns(float,def) def bar(low,high): for def accepts(int,int) for def returns(float) def bar(low,high): equivalent to: @accepts(int,int) @returns(float) def bar(low,high): From martin at v.loewis.de Thu Aug 5 23:31:50 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:31:59 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <1091740792.8546.133.camel@localhost> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost> Message-ID: <4112A746.6010501@v.loewis.de> Barry Warsaw wrote: > Martin makes a good point. Guido could threaten to remove the feature > by beta 1 (and thus for 2.4 final) if the PEP is not brought up to date. > > not-that-i'm-volunteering-'cause-i'd-rather-see-it-done-ly y'rs, I'm atleast volunteering to take the code out if no proper implementation (including documentation and test cases) is delivered by some deadline. Regards, Martin From barry at python.org Thu Aug 5 23:32:41 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 5 23:32:32 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <4112A4B9.4010907@v.loewis.de> References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> Message-ID: <1091741561.8541.144.camel@localhost> On Thu, 2004-08-05 at 17:20, "Martin v. L?wis" wrote: > > How is this not going to happen with the '@' syntax? > > It's of course hard to tell how tools will react to that syntax, or > with any other syntax change, for that matter. This specific is likely > easy to implement, because it likely has only minimal impact on people's > parsers: one can essentially treat a line starting with @ like a > statement, e.g. by making @ a keyword. Not that it should have any impact on the decision in any way, but the regular expression python-mode.el uses to syntax color pie decorators is much simpler (to figure out and to read) than the previous decorator syntax support of list-before-colon. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/85adf199/attachment.pgp From martin at v.loewis.de Thu Aug 5 23:34:07 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:34:09 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: References: Message-ID: <4112A7CF.3030700@v.loewis.de> Nicolas Fleury wrote: > Other crazy ideas (in case it inspires anyone): > > accepts(int,int,def) > returns(float,def) > def bar(low,high): That doesn't work. If accepts and returns are callables (as they should be), then this already means something in current Python. So this would not be backwards compatible. > for def accepts(int,int) > for def returns(float) > def bar(low,high): This is probably very confusing to existing tools, which expect an identifier after the for, and a colon somewhere on the same line. Regards, Martin From edreamleo at charter.net Thu Aug 5 23:34:52 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Thu Aug 5 23:35:02 2004 Subject: [Python-Dev] Questions about '@' in pep 318 References: <004a01c47b2b$65d19260$6700a8c0@computer> <20040805205559.GA31288@burma.localdomain> Message-ID: <003601c47b34$0b4bf660$6700a8c0@computer> > > annotate.accepts(int, (int,float)) > > I don't understand. How are you changing/annotating the function? I'm glad you asked. My suggestion was that the compiler turn annotate.accepts(int, (int,float)) into exactly what it would have turned @accepts(int, (int,float)) into. I hope I am not being incredibly stupid, but I am thinking only of syntax here. So wherever the community would have decided to put: @accepts <> the new "syntax" would be: annotate.accepts <> Do you see? The annotate "module" probably doesn't exist at all. I think of it as a pseudo-module. That's why I wrote "from __future__ import annotation as annotate" in the original post. My thinking is this: apparently people like the @ syntax. Why not do some magic and compile module references to the annotation pseudo-module just as it is presently proposed to compile the @ syntax. That way we don't change the apparent syntax at all, we use the annotation namespace to denote what is happening, and we use annotate.x exactly as we would have used @x. Again, my thinking is that this would be completely independent of the other choices about where to use @. Wherever the consensus is, then use the annotation pseudo-module there. Have I made myself clear? Edward P.S. I wrote: from __future__ import annotation as annotate as a hint that one could write: from __future__ import annotation as a if brevity is important. Then we would write a.accepts instead of @accepts. EKR -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From aahz at pythoncraft.com Thu Aug 5 23:37:13 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 5 23:37:16 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: <20040805213713.GA28782@panix.com> On Thu, Aug 05, 2004, Phillip J. Eby wrote: > > So there it is, my personal argument for list-after-def as better than "@", > even if the list is over multiple lines: it's still much more clearly part > of the function definition, and that IMO is the most important thing the > syntax should convey, apart from listing the decorators themselves. +1 I've been staying out of this because I thought my preference had no chance to make it in (i.e. I don't care between "no decorators" and "@ decorators"), but if it's getting some revival, I want to make clear that I think putting decorators *after* ``def``, name, and parameter list but clearly *with* the function definition makes the most sense for readability -- in the simple cases. Who cares about the complex cases, because it's always going to be ugly? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From martin at v.loewis.de Thu Aug 5 23:40:56 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 5 23:41:00 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings In-Reply-To: <16658.42673.542930.749915@montanaro.dyndns.org> References: <16658.42673.542930.749915@montanaro.dyndns.org> Message-ID: <4112A968.2030401@v.loewis.de> Skip Montanaro wrote: > At work the goal for our C++ build environment is "no warnings with gcc > -Wall". We are not there yet (and thankfully I am not a participant in that > particular holy grail), but those involved in that effort encountered a > warning the other day generated because _XOPEN_SOURCE is defined in > pyconfig.h. Unfortunately GCC also defines it, so a "macro redefined" > warning is emitted when compiling with GCC and including Python.h. That is not true. GCC does not define that macro, on any platform. Some system header may define it, though. I believe this is a bug in the system - the macro *should* be defined in the application. What system are you referring to, and where does that define _XOPEN_SOURCE? Is that definition conditional? > Perhaps adding > > #pragma GCC system_header > > to pyconfig.h.in would be acceptable though. I'd prefer to solve the problem properly. If the system defines something, and we define it differently, this *is* a problem, and deserves a warning. We should find out why that is, and what can be done about. Perhaps we should not define _XOPEN_SOURCE on that platform until the vendor fixes the platform bug, or we should aim at defining it the same way the vendor does. Silencing errors is a risky thing. Regards, Martin From jjl at pobox.com Thu Aug 5 23:47:01 2004 From: jjl at pobox.com (John J Lee) Date: Thu Aug 5 23:47:10 2004 Subject: httplib patch for 2.4 (was: Re: [Python-Dev] 2.4a2 ...) In-Reply-To: References: <410DDFE3.7010006@interlink.com.au> Message-ID: On Mon, 2 Aug 2004, Jeremy Hylton wrote: > On Mon, 2 Aug 2004 14:30:39 +0100 (GMT Daylight Time), John J Lee > wrote: [...] > > Would somebody mind checking this in: > > > > http://www.python.org/sf/997626 > > > > It fixes a bug in httplib which badly breaks urllib2 now that urllib2 is > > using httplib.HTTPConnection instead of httplib.HTTP (POSTs don't work). > > > > It includes patch and a new test, and does not require doc updates, but > > nobody has reviewed it yet. > > I'll take a look, probably tomorrow morning. > > Jeremy [...] It don't think it's in especial need of reviewing by an expert. It just needs to get checked in! John From niemeyer at conectiva.com Thu Aug 5 23:53:31 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Aug 5 23:51:14 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112A18D.5090801@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> Message-ID: <20040805215331.GB31288@burma.localdomain> > Depends on in what direction you want to make a change. It > appears you want to avoid introducing any kind of syntax > change. In that case, you should explain people how to spell > classmethod and synchronized in a more convenient way, because > that is what the stated motivation of PEP 318 is - you would > have to explain why this motive is bad, irrelevant, or otherwise > not a worthy goal. > > Or you could argue on a procedural basis: regardless of whether > the feature is good or bad, the current implementation is > unacceptable, as the PEP does not correspond with the > implementation, the syntax is undocumented, the code has no test > cases, and so on. I'm actually going to do that, because I do > think the process is unacceptable, and should be either corrected > or reversed (of course, this says nothing about the feature itself, > or the code implementing it). Ok, I'll try to summarize the current status of the feature so that I (and others) can understand if there's something to be done: - Decorators are going in on 2.4. - There are two obvious usage cases for decorators: static and class methods; - There are more complex usage cases for decorators such as PyObjC which was already agreed to be something necessarily supported in the implementation; - People which want the powerful decorators don't care about the syntax, as far as the feature is implemented; - Adapting external tools should not be used as an argument; - We're clearly unable to get into a consensus about syntax; - All current syntax vs. problems to solve have been discussed extensively; - There are bizarre usage cases of the current decorator implementation, but this is something which is considered abusive and won't affect decisions since should be casual; - The @ character is used in at least two tools (Leo, IPython), and this is being considered as something bad, but not a show stopper; - The perlish argument is non-sense; I belive that either some different syntax which most people agree upon is raised, or we're done. -- Gustavo Niemeyer http://niemeyer.net From gregory.lielens at fft.be Thu Aug 5 23:52:26 2004 From: gregory.lielens at fft.be (Gregory Lielens) Date: Thu Aug 5 23:52:42 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: <001b01c47b36$8216f6d0$0f00000a@chikubi> > Of the options on the Wiki page ( at > http://www.python.org/moin/PythonDecorators ), this seems to > leave only > list-after-def, the previous community favorite and the first PEP 318 > syntax to have been implemented. > > Guido's principal argument against list-after-def, if I > recall correctly, > was that it is ugly when multiple or lengthy decorators are > involved. But, > "ugly" isn't an argument any more, so that shouldn't rule out > list-after-def. :) On a cosmetic point of view, I also prefer the list-after-def syntax, first time I saw it it was the one than seemed the easiest (the "def f() as [decorator]" variant was event clearer, but the argument that as is only for renaming has mitigaed that somewhat ;-) ) "Cosmetic" does not sound like a very convincing argument, but I feel that this whole discussion is purely on a cosmetic level anyway, it is just a way to express more clearly something that is already possible...and as feedback as been asked for... ;-) But does it means that the list-after-def behaves like all lists, so that these are possible and equivalent? def foo() [decorator1, decorator2(bar)]: ... def foo() [ decorator1, decorator2(bar)]: ... my_complex_decorator=[decorator1, decorator2(bar)] def foo() my_complex_decorator : ... If it is, I am +1 on list-after-def (and, because of the last example, +2 on a variant with a keyword or pseudo-keyword between the def and the list, like ("with" is only an example) : def foo() with [decorator1, decorator2(bar)]: ... my_complex_decorator=[decorator1, decorator2(bar)] def foo() with my_complex_decorator : ... def foo2() with my_complex_decorator : ... If not (i.e. the brackets are just a syntax hint), I am -1, I prefer the the pie syntax cause it would break consistency with other lists usage... For example, for j in [1,2]: ... for j in ["a","b", 1,2,3,4]: ... for j in mylist: ... are all possible, so it would be very surprising if the def syntax would not be similar, it would mean that the brackets after a def are not really a list and that would feel too much like special-casing to me... Best regards, Greg. From nidoizo at yahoo.com Thu Aug 5 23:55:00 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu Aug 5 23:55:08 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: <4112A7CF.3030700@v.loewis.de> References: <4112A7CF.3030700@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Nicolas Fleury wrote: >> accepts(int,int,def) >> returns(float,def) >> def bar(low,high): > > That doesn't work. If accepts and returns are callables > (as they should be), then this already means something > in current Python. So this would not be backwards > compatible. I'm sorry (I don't like that solution at all honestly), but what passing "def" to a callable means in current Python? It is not accepted by my interactive shell... Regards, Nicolas From ixokai at gmail.com Fri Aug 6 00:00:24 2004 From: ixokai at gmail.com (IxokaI) Date: Fri Aug 6 00:00:27 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <001b01c47b36$8216f6d0$0f00000a@chikubi> References: <001b01c47b36$8216f6d0$0f00000a@chikubi> Message-ID: For what its worth, this lurker until now is strongly +1 on list-after-def, regardless of how it turns out. Real list, fake list, pseudo-keyword seperator. I don't like @syntax because unless you're using whitespace between def's in the 'right' way its just.. not completely clear what belongs to what. class incdec: def increment (arg): return arg +1 increment = staticmethod(increment) @staticmethod def decrement(arg): return arg - 1 That looks like a mess. Python uses whitespace to group things, and @syntax is not in any way grouped or connected to the function.. No, its not ambigious, its just-- messy, unclear to a novice. However: class incdec: def increment(arg) [staticmethod]: return arg +1 def decrement(arg) using [staticmethod]: return arg -1 *clearly* shows that the decorator is decorating that specific function. You may not know just what its doing, but you know its directly related. I think that's very important. --Stephen On Thu, 5 Aug 2004 23:52:26 +0200, Gregory Lielens wrote: > > > Of the options on the Wiki page ( at > > http://www.python.org/moin/PythonDecorators ), this seems to > > leave only > > list-after-def, the previous community favorite and the first PEP 318 > > syntax to have been implemented. > > > > Guido's principal argument against list-after-def, if I > > recall correctly, > > was that it is ugly when multiple or lengthy decorators are > > involved. But, > > "ugly" isn't an argument any more, so that shouldn't rule out > > list-after-def. :) > > On a cosmetic point of view, I also prefer the list-after-def syntax, > first time I saw it it was the one than seemed the easiest (the "def f() > as [decorator]" variant was event clearer, but the argument that as is > only for renaming has mitigaed that somewhat ;-) ) > "Cosmetic" does not sound like a very convincing argument, but I feel > that this whole discussion is purely on a cosmetic level anyway, it is > just a way to express more clearly something that is already > possible...and as feedback as been asked for... ;-) > > But does it means that the list-after-def behaves like all lists, so > that these are possible and equivalent? > > def foo() [decorator1, decorator2(bar)]: > ... > > def foo() [ > decorator1, > decorator2(bar)]: > ... > > my_complex_decorator=[decorator1, decorator2(bar)] > def foo() my_complex_decorator : > ... > > If it is, I am +1 on list-after-def (and, because of the last example, > +2 on a variant with a keyword or pseudo-keyword between the def and the > list, like ("with" is only an example) : > > def foo() with [decorator1, decorator2(bar)]: > ... > my_complex_decorator=[decorator1, decorator2(bar)] > def foo() with my_complex_decorator : > ... > def foo2() with my_complex_decorator : > ... > > If not (i.e. the brackets are just a syntax hint), I am -1, I prefer the > the pie syntax cause it would break consistency with other lists > usage... > For example, > for j in [1,2]: > ... > for j in ["a","b", > 1,2,3,4]: > ... > for j in mylist: > ... > are all possible, so it would be very surprising if the def syntax would > not be similar, it would mean that the brackets after a def are not > really a list and that would feel too much like special-casing to me... > > Best regards, > > Greg. > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ixokai%40gmail.com > From niemeyer at conectiva.com Fri Aug 6 00:07:43 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 00:05:26 2004 Subject: [Python-Dev] Similar syntax Message-ID: <20040805220743.GC31288@burma.localdomain> > Ok, I'll try to summarize the current status of the feature > so that I (and others) can understand if there's something to > be done: > > - Decorators are going in on 2.4. [...] > - The @ character is used in at least two tools (Leo, IPython), > and this is being considered as something bad, but not a > show stopper; > > - The perlish argument is non-sense; > > I belive that either some different syntax which most > people agree upon is raised, or we're done. After writing this list, I thought about something which is close to the current implementation, but avoids @. If it was already suggested, please forgive me. At least I wasn't able to find it in the PEP/Wiki: def method(foo): pass and also def bar(low, high): ... Advantages over current syntax: - Avoids introducing a new meaningful character in the syntax; - Leaves Leo and IPython people happy; - ">" is already being used on "print >>", so special casing it wouldn't be something new; - Not acceptable in the current syntax; - Looks like a tag; -- Gustavo Niemeyer http://niemeyer.net From exarkun at divmod.com Fri Aug 6 00:07:25 2004 From: exarkun at divmod.com (Jp Calderone) Date: Fri Aug 6 00:07:30 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: <4112A7CF.3030700@v.loewis.de> References: <4112A7CF.3030700@v.loewis.de> Message-ID: <4112AF9D.1080202@divmod.com> Martin v. L=F6wis wrote: > Nicolas Fleury wrote: > = >> Other crazy ideas (in case it inspires anyone): >> >> accepts(int,int,def) >> returns(float,def) >> def bar(low,high): > = > = > That doesn't work. If accepts and returns are callables > (as they should be), then this already means something > in current Python. So this would not be backwards > compatible. Adding _any_ names to Python has this potential problem. I don't = think that's going to stop anyone from adding new modules to the = standard library or new builtins (2.4 introduces 3 new builtins). To say this isn't backwards compatible is true, but not in a sense = that strikes me as important. "accepts" and "returns" don't even need = to be builtins, they could be placed in a module with a handful of other = useful common decorators. Jp From edreamleo at charter.net Fri Aug 6 00:09:32 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 00:09:41 2004 Subject: [Python-Dev] The impact of '@' on Leo References: <005e01c47b2d$9c0dee80$6700a8c0@computer> <4112A593.7080608@v.loewis.de> Message-ID: <000701c47b38$e2e526b0$6700a8c0@computer> > > The major concern I have with '@' syntax as it relates to Leo is the > > potential for ongoing and never-ending conflicts between Leo's directives > > and @x constructs in Python. > > Can you give an example of a file that would be difficult to understand > under that change? If you allow @foo inside a Python file, it is not > Python anymore, is it? There are two files involved. The .leo file (an xml file) contains the "marked up" code. Leo turns this file into a visual outline containing zero or more "derived files" that could be written in any language. Here is the entire top-level node in the outline that represents the derived file leoApp.py: [Node starts] @first # -*- coding: utf-8 -*- @language python @tabwidth -4 import leoGlobals as g import os import sys class LeoApp: """A class representing the Leo application itself. Ivars of this class are Leo's global variables.""" @others [Node ends] - The [Node starts] and [Node ends] are _not_ part of the node, they delimit the node in this email. - The @first directive makes sure that the first line of leoApp.py is # -*- coding: utf-8 -*- - The @language python and @tabwidth directives tell Leo how to syntax color the text and how to handle tabs. - The @others directive tells Leo to insert all descendent nodes. In this case, each descendent node contains a single method of the LeoApp class. - There are other details to Leo's markup, but this gives you the general feel. Everything else is plain Python. When saving the .leo file, Leo produces leoApp.py following the rules of the markup. In particular, Leo writes all descendent nodes in place of the @others directive, indented the same as the @others directive. Most of these descendent nodes will contain few or no Leo directives: descendent nodes inherit most directives from their ancestors. The problem is that Leo doesn't really understand _any_ language, except to colorize it. Most Leo directives must start with @ in the leftmost column, but the @others and @all directives may contain leading whitespace. If @foo is both a valid Python construct and a valid Leo directive Leo will treat @foo as a Leo directive even if it is "supposed" to be Python. If @foo is not a valid Leo directive, Leo creates a "verbatim" sentinel line in the derived file to indicate that the next line isn't a directive, even though it sorta looks like one. Like this: #@verbatim @foo This is basically how Leo handle's languages with @ signs. There is no problem until @x in the language conflicts with a Leo directive, in which case Leo doesn't have any way of distinguishing which usage was intended. As I said in another post, Leo doesn't have escape conventions involving @ signs, and that has turned out well. There have been requests to allow strings other than "@" to delimit Leo directives, and that will probably happen, but in general Leo really would rather stick to "@" Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From pje at telecommunity.com Fri Aug 6 00:15:09 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 00:11:09 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <001b01c47b36$8216f6d0$0f00000a@chikubi> References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040805181155.021c9910@mail.telecommunity.com> At 11:52 PM 8/5/04 +0200, Gregory Lielens wrote: > > Of the options on the Wiki page ( at > > http://www.python.org/moin/PythonDecorators ), this seems to > > leave only > > list-after-def, the previous community favorite and the first PEP 318 > > syntax to have been implemented. > > > > Guido's principal argument against list-after-def, if I > > recall correctly, > > was that it is ugly when multiple or lengthy decorators are > > involved. But, > > "ugly" isn't an argument any more, so that shouldn't rule out > > list-after-def. :) > >On a cosmetic point of view, I also prefer the list-after-def syntax, >first time I saw it it was the one than seemed the easiest (the "def f() >as [decorator]" variant was event clearer, but the argument that as is >only for renaming has mitigaed that somewhat ;-) ) >"Cosmetic" does not sound like a very convincing argument, but I feel >that this whole discussion is purely on a cosmetic level anyway, it is >just a way to express more clearly something that is already >possible...and as feedback as been asked for... ;-) > >But does it means that the list-after-def behaves like all lists, so >that these are possible and equivalent? > >def foo() [decorator1, decorator2(bar)]: > ... > >def foo() [ > decorator1, > decorator2(bar)]: > ... > >my_complex_decorator=[decorator1, decorator2(bar)] >def foo() my_complex_decorator : > ... The third example isn't allowed, but the first two are. Remember that part of the point is that the [] are being used to visually signal annotation. The third syntax loses this point. Note that there is precedent for this kind of syntax restriction; a class' bases are a tuple, but that doesn't mean you can do this: yz = (Y,Z) class X yz: pass From edreamleo at charter.net Fri Aug 6 00:16:33 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 00:16:43 2004 Subject: [Python-Dev] Questions about '@' in pep 318 References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> Message-ID: <000f01c47b39$de3bcd70$6700a8c0@computer> > That requires all annotations to be in the __future__ module. This is not acceptable, as there is a desire to provide user-defined annotations as well. Oh dear. Does this mean that @x is a potential annotation for any identifier x? This would be very bad for Leo. I was using __future__ by way of explanation. I do hope namespaces could somehow denote annotations. My off-the-cuff suggestion was for pseudo-modules, so maybe the normal module rules could be sidestepped? Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From skip at pobox.com Fri Aug 6 00:19:16 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 00:19:33 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <20040805213713.GA28782@panix.com> References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> <20040805213713.GA28782@panix.com> Message-ID: <16658.45668.851011.950341@montanaro.dyndns.org> >> So there it is, my personal argument for list-after-def as better >> than "@" ... aahz> +1 What he said. +1. I had simply given up arguing the point because it seemed pointless and I was tired of arguing. I suppose there are more than a few people in this particular boat. Skip From edreamleo at charter.net Fri Aug 6 00:19:29 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 00:19:38 2004 Subject: [Python-Dev] Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> Message-ID: <002301c47b3a$469bc5f0$6700a8c0@computer> > Or you could argue on a procedural basis: regardless of whether > the feature is good or bad, the current implementation is > unacceptable, as the PEP does not correspond with the > implementation, the syntax is undocumented, the code has no test > cases, and so on. I'm actually going to do that, because I do > think the process is unacceptable, and should be either corrected > or reversed (of course, this says nothing about the feature itself, > or the code implementing it). Thank you, Martin. I think Python will be the better for holding itself to higher standards. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From eppstein at ics.uci.edu Fri Aug 6 00:22:54 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Fri Aug 6 00:23:00 2004 Subject: [Python-Dev] Re: Questions about '@' in pep 318 References: <004a01c47b2b$65d19260$6700a8c0@computer> <20040805205559.GA31288@burma.localdomain> <003601c47b34$0b4bf660$6700a8c0@computer> Message-ID: In article <003601c47b34$0b4bf660$6700a8c0@computer>, "Edward K. Ream" wrote: > I hope I am not being incredibly stupid, but I am thinking only of syntax > here. So wherever the community would have decided to put: > > @accepts <> > > the new "syntax" would be: > > annotate.accepts <> > > Do you see? The annotate "module" probably doesn't exist at all. I think of > it as a pseudo-module. That's why I wrote "from __future__ import > annotation as annotate" in the original post. > > My thinking is this: apparently people like the @ syntax. Why not do some > magic and compile module references to the annotation pseudo-module just as > it is presently proposed to compile the @ syntax. I'm not convinced this is possible in general (for all pythons, obviously it's possible by gross hacks in CPython) without making annotate into a keyword. And, I don't like the choice of the word, either, since I think decorators are more likely to be used to change the meaning of the definition and not just attach some meaningless annotation to it. Anyway, since decorators are quite different from normal control flow in Python I'd prefer to see them given a syntax that stands out, that makes it obvious that it's not just some slightly mysterious function call prior to the def. @syntax does that for me and yours doesn't. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From skip at pobox.com Fri Aug 6 00:24:27 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 00:24:42 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings In-Reply-To: <4112A968.2030401@v.loewis.de> References: <16658.42673.542930.749915@montanaro.dyndns.org> <4112A968.2030401@v.loewis.de> Message-ID: <16658.45979.672934.897388@montanaro.dyndns.org> >> At work the goal for our C++ build environment is "no warnings with >> gcc -Wall". We are not there yet (and thankfully I am not a >> participant in that particular holy grail), but those involved in >> that effort encountered a warning the other day generated because >> _XOPEN_SOURCE is defined in pyconfig.h. Unfortunately GCC also >> defines it, so a "macro redefined" warning is emitted when compiling >> with GCC and including Python.h. Martin> That is not true. GCC does not define that macro, on any Martin> platform. Martin> Some system header may define it, though. Well, yeah, that's what I meant. My apologies for the imprecise statement. Martin> I believe this is a bug in the system - the macro *should* be Martin> defined in the application. What system are you referring to, Martin> and where does that define _XOPEN_SOURCE? Is that definition Martin> conditional? The platform is Solaris/Intel (2.8 and/or 2.9). I'll have to get you more complete details tomorrow. The involved folks are gone for the day. Martin> Silencing errors is a risky thing. Agreed. I made them investigate the python-dev archives before contributing. They said, "Who is this Martin guy?" :-) wasn't-this-a-pleasant-diversion-from-pie-syntax?-ly y'rs, Skip From barry at python.org Fri Aug 6 00:25:17 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 00:25:09 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805215331.GB31288@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> Message-ID: <1091744717.8545.153.camel@localhost> On Thu, 2004-08-05 at 17:53, Gustavo Niemeyer wrote: > - There are two obvious usage cases for decorators: static and > class methods; > > - There are more complex usage cases for decorators such as > PyObjC which was already agreed to be something necessarily > supported in the implementation; I'm not sure these two points are accurate. There's definitely a large middle ground between staticmethod/classmethod and PyObjC's use cases. > - People which want the powerful decorators don't care about > the syntax, as far as the feature is implemented; I care deeply about the syntax, but I can accept any of the choices that have made Guido's final cut. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/8380bfdb/attachment.pgp From martin at v.loewis.de Fri Aug 6 00:26:02 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 00:26:08 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805215331.GB31288@burma.localdomain> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> Message-ID: <4112B3FA.3050106@v.loewis.de> Gustavo Niemeyer wrote: > - Decorators are going in on 2.4. Yes, that's the current status. Of course, it is difficult to predict, especially the future. Who knows what would happen if people find flaws in the proposed syntax, and no alternative is available? > - There are two obvious usage cases for decorators: static and > class methods; Correct. > - There are more complex usage cases for decorators such as > PyObjC which was already agreed to be something necessarily > supported in the implementation; Correct. > - People which want the powerful decorators don't care about > the syntax, as far as the feature is implemented; Some actually do care about the syntax as well. > - Adapting external tools should not be used as an argument; No, external tools need to be considered. Preferably in the concrete, not in the abstract. > - We're clearly unable to get into a consensus about syntax; Yes. I believe this is not specific to this change - people *always* argue about syntax. > - All current syntax vs. problems to solve have been discussed > extensively; Yes. > - There are bizarre usage cases of the current decorator > implementation, but this is something which is considered > abusive and won't affect decisions since should be casual; Yes. > - The @ character is used in at least two tools (Leo, IPython), > and this is being considered as something bad, but not a > show stopper; This is certainly bad. Whether it is a show-stopper, I don't know. I don't understand the Leo issue; I thought it would use the @ syntax only within Python comments, in which case there would be no conflicts. > - The perlish argument is non-sense; Yes. In a typical decorator, the only special characters on the line will be the @ and possibly () (perhaps also commas between arguments). This is completely unlike Perl which often has large fractions on special characters on a line. > I belive that either some different syntax which most > people agree upon is raised, or we're done. Perhaps. People really should use it to find potential flaws in the semantics of decorators, e.g. things you ought to be able to do with decorators but can't. Of course, test_decorators.py already gives a good idea what is possible, so it might be "good enough". Regards, Martin From s.percivall at chello.se Fri Aug 6 00:29:05 2004 From: s.percivall at chello.se (Simon Percivall) Date: Fri Aug 6 00:29:11 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: +1 on @decorator style for being clear, unambiguous and useful (and not bad to look at once you get used to it). clear: * it's more visible and less messy than any of the before-/after-/behind-def style decorations; * it stands out. this is a good thing. unambiguous: unlike the other viable prefix notation it makes 2.4 code special. useful: just having a special decoration syntax makes it useful. decorations can make the behaviour of a function or class seem more magic than otherwise, but at least the @decoration style makes it more obvious that something is going on. On 2004-08-05, at 18.36, Guido van Rossum wrote: > [...] you might want to say something in public in defense of > @decorators. From eppstein at ics.uci.edu Fri Aug 6 00:29:09 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Fri Aug 6 00:29:16 2004 Subject: [Python-Dev] Re: A usability argument for list-after-def References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: In article <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>, "Phillip J. Eby" wrote: > Guido has stated he'll accept one community syntax proposal as an > alternative to "@". He's also ruled out his own previous proposal > (list-before-def), and a variety of new-keyword alternatives involving such > things as 'as' and 'with'. > > There doesn't appear to be a downside for decorator supporters to put forth > another syntax, if they like it better than "@"; it seems the worst case is > that we still end up with "@", if we end up with any syntax at all. It's clear from previous discussions that there can be no single community preference. The downside to me is that all this brouhaha will prevent us from getting any syntax at all. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From eppstein at ics.uci.edu Fri Aug 6 00:24:41 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Fri Aug 6 00:30:24 2004 Subject: [Python-Dev] Re: The impact of '@' on Leo References: <005e01c47b2d$9c0dee80$6700a8c0@computer> Message-ID: In article <005e01c47b2d$9c0dee80$6700a8c0@computer>, "Edward K. Ream" wrote: > The major concern I have with '@' syntax as it relates to Leo is the > potential for ongoing and never-ending conflicts between Leo's directives > and @x constructs in Python. I don't know what Leo is, but the usual solution to this sort of problem is to introduce some kind of quoting convention, e.g. @@ gets passed through as a single @ to Python, any other @something gets treated as a Leo directive. Is there some reason that wouldn't work? -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From tim.peters at gmail.com Fri Aug 6 00:31:49 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 6 00:31:52 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <4112A365.1000203@ocf.berkeley.edu> References: <4112A365.1000203@ocf.berkeley.edu> Message-ID: <1f7befae040805153126848c1@mail.gmail.com> [Brett Cannon] > I am actually getting failures from test_timeout's testConnectTimeout It would have helped if you had been specific about the failures you're seeing, and whether you get them all the time, or just some of the time. > since the Net connection I have at school lets me connect to Google in > under the .001 connection timeout value. If I push it to .00000001 (a > hundred-millionth of a second) it then fails consistently. What fails? The test fails? The socket fails to connect? Note that .0000001 is the same as passing, e.g., 1e-200. select() only accepts arguments at microsecond granularity, so any positive non-zero timeout value < 1e-6 gets chopped to exactly 0.0. tv.tv_sec = (int)s->sock_timeout; tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6); The test really doesn't intend to pass a timeout of exactly 0 to select(), so we can't change this to less than 1e-6. > Now the problem is that the second part of the test uses this and a fuzz > value (currently at 2) to see if the test timed out within the proper > amount of time. The comparison is basically if the amount of time it > took to do the timed out failure is less than timeout + fuzz. So > lowering this number could possibly affect the test, although at .001 > seconds, I am doubting that will occur. But since these types of timing > tests can be touchy I thought I would double-check. Na, 2.0 is gigantic compared to .001 already. For the purposes of the test, 2 isn't really more gigantic compared to 1e-6. > So if anyone thinks it is bad to lower the value to .00000001 then > please let me know. Changing it to 1e-7 is out, for the reason explained earlier. I'd like to keep .001, because while the interface to select() *allows* specifying microsecond resolution, there's no guarantee that it can use that much precision. Most (all?) implementations should be able to deal with millisecond resolution, though. Perhaps we could pick on something other than www.google.com. Maybe www.python.org (everyone in America is far from that ). From martin at v.loewis.de Fri Aug 6 00:33:24 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 00:33:29 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <002301c47b3a$469bc5f0$6700a8c0@computer> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> Message-ID: <4112B5B4.9@v.loewis.de> Edward K. Ream wrote: > Thank you, Martin. I think Python will be the better for holding itself to > higher standards. I was somewhat too quick critizing the documentation, though: http://www.python.org/dev/doc/devel/ref/function.html does explain syntax and semantics, and there are testcases as well. So it's just the PEP that needs updating (a section in whatsnew also needs to be drafted). Then, of course, the text saying that "@" is not used needs to go. I wonder whether replacing "@" with "!" or "%" would do any good... In fact, any binary operator would work - Objective-C does use "+" and "-" at the beginning of a line... Regards, Martin From exarkun at divmod.com Fri Aug 6 00:33:40 2004 From: exarkun at divmod.com (Jp Calderone) Date: Fri Aug 6 00:33:45 2004 Subject: [Python-Dev] Re: A usability argument for list-after-def In-Reply-To: References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: <4112B5C4.9070804@divmod.com> David Eppstein wrote: > In article <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>, > "Phillip J. Eby" wrote: > > >>Guido has stated he'll accept one community syntax proposal as an >>alternative to "@". He's also ruled out his own previous proposal >>(list-before-def), and a variety of new-keyword alternatives involving such >>things as 'as' and 'with'. >> >>There doesn't appear to be a downside for decorator supporters to put forth >>another syntax, if they like it better than "@"; it seems the worst case is >>that we still end up with "@", if we end up with any syntax at all. > > > It's clear from previous discussions that there can be no single > community preference. The downside to me is that all this brouhaha will > prevent us from getting any syntax at all. > This would be a major downside, if the feature were unsupportable without syntax additions. Jp From martin at v.loewis.de Fri Aug 6 00:38:20 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 00:38:22 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <000f01c47b39$de3bcd70$6700a8c0@computer> References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> Message-ID: <4112B6DC.4080605@v.loewis.de> Edward K. Ream wrote: > Oh dear. Does this mean that @x is a potential annotation for any > identifier x? Yes. Take a look at the documentation, and the test case. > I was using __future__ by way of explanation. I do hope namespaces could > somehow denote annotations. My off-the-cuff suggestion was for > pseudo-modules, so maybe the normal module rules could be sidestepped? I don't see how this would be possible. The plan is that arbitrary callables can be used as decorations, as long as they take a single argument. Regards, Martin From bob at redivi.com Fri Aug 6 00:39:39 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 6 00:39:46 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112B5B4.9@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de> Message-ID: <55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com> On Aug 5, 2004, at 6:33 PM, Martin v. L?wis wrote: > Edward K. Ream wrote: >> Thank you, Martin. I think Python will be the better for holding >> itself to >> higher standards. > > I was somewhat too quick critizing the documentation, though: > > http://www.python.org/dev/doc/devel/ref/function.html > > does explain syntax and semantics, and there are testcases as well. So > it's just the PEP that needs updating (a section in whatsnew also > needs > to be drafted). Then, of course, the text saying that "@" is not used > needs to go. > > I wonder whether replacing "@" with "!" or "%" would do any good... > In fact, any binary operator would work - Objective-C does use "+" > and "-" at the beginning of a line... Except + and - would be valid Python syntax already ;) For reference, in Objective-C, + means class method and - means instance method. This is not decoration because they are in separate namespaces; +class methods are basically instance methods on an implicit metaclass. In Objective-C, + and - are roughly equivalent to Python's "def" keyword. Aesthetically I much prefer @ over ! or %. !looks !bad !next !to !text !because !it !doesn't !stand !out and %makes %me %think %of %comments %in %another %language. -bob From martin at v.loewis.de Fri Aug 6 00:41:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 00:41:50 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings In-Reply-To: <16658.45979.672934.897388@montanaro.dyndns.org> References: <16658.42673.542930.749915@montanaro.dyndns.org> <4112A968.2030401@v.loewis.de> <16658.45979.672934.897388@montanaro.dyndns.org> Message-ID: <4112B7A8.5090100@v.loewis.de> Skip Montanaro wrote: > The platform is Solaris/Intel (2.8 and/or 2.9). I'll have to get you more > complete details tomorrow. The involved folks are gone for the day. Ok. I'll get back to a Solaris machine next week, so I can investigate somewhat myself. Of course, the rule is, strictly speaking, "no warnings on the major platforms"; I'm uncertain whether Solaris still applies here. It is clearly not feasible to have Python compile with no warning no matter what the compiler or operating system is. The major platforms, of course, are Windows, Linux, OS X (i.e. the ones we provide binaries for). Regards, Martin From edreamleo at charter.net Fri Aug 6 00:43:42 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 00:43:51 2004 Subject: [Python-Dev] Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> Message-ID: <008901c47b3d$a89c17c0$6700a8c0@computer> > - The @ character is used in at least two tools (Leo, IPython), and this is being considered as something bad, but not a show stopper; I wonder how bad @ is, even for Leo. Is there any consensus or opinions about how many files will actually contain annotations? If there aren't many, people could change the delimiter for Leo directives from @ to @@ just for those files :-) Of course, Leo can't do this yet, but that's a nit... Edward P.S. Oh yeah, just to make sure: @@ won't be valid in annotations, will it? EKR -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From edreamleo at charter.net Fri Aug 6 00:46:27 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 00:46:35 2004 Subject: [Python-Dev] Similar syntax References: <20040805220743.GC31288@burma.localdomain> Message-ID: <000701c47b3e$0b14d540$6700a8c0@computer> > > def bar(low, high): ... > - Leaves Leo and IPython people happy; Yep. No problem with Leo. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From marktrussell at btopenworld.com Fri Aug 6 00:52:07 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Fri Aug 6 00:52:10 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112B5B4.9@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de> Message-ID: <1091746327.1419.18.camel@localhost> On Thu, 2004-08-05 at 23:33, "Martin v. L?wis" wrote: > it's just the PEP that needs updating (a section in whatsnew also needs > to be drafted). Then, of course, the text saying that "@" is not used > needs to go. I have no experience in working on PEPs, but I'm willing to have a go if nobody else has the time. > I wonder whether replacing "@" with "!" or "%" would do any good... > In fact, any binary operator would work - Objective-C does use "+" > and "-" at the beginning of a line... Good thought. Or even something like: |staticmethod| def foo(): pass I just tried converting test_decorators.py to this syntax, and it looks quite nice to my eye. Mark From bob at redivi.com Fri Aug 6 00:54:44 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 6 00:54:51 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <008901c47b3d$a89c17c0$6700a8c0@computer> References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <008901c47b3d$a89c17c0$6700a8c0@computer> Message-ID: <71A2DB28-E732-11D8-9D9A-000A95686CD8@redivi.com> On Aug 5, 2004, at 6:43 PM, Edward K. Ream wrote: >> - The @ character is used in at least two tools (Leo, IPython), > and this is being considered as something bad, but not a show > stopper; > > I wonder how bad @ is, even for Leo. Is there any consensus or > opinions > about how many files will actually contain annotations? If there > aren't > many, people could change the delimiter for Leo directives from @ to > @@ just > for those files :-) Of course, Leo can't do this yet, but that's a > nit... I think the idea is that Leo should escape the program code sort of like SQL escapes quotes. What you have right now just sounds fragile. @leo @@pythondecorator @leo @leo @@perlarray > P.S. Oh yeah, just to make sure: @@ won't be valid in annotations, > will > it? No. -bob From nhodgson at bigpond.net.au Fri Aug 6 00:58:16 2004 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Fri Aug 6 00:58:22 2004 Subject: [Python-Dev] Questions about '@' in pep 318 References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <1091741561.8541.144.camel@localhost> Message-ID: <028301c47b3f$b1b08bf0$034b8890@neil> Barry Warsaw: > Not that it should have any impact on the decision in any way, but > the regular expression python-mode.el uses to syntax color pie > decorators is much simpler (to figure out and to read) than the > previous decorator syntax support of list-before-colon. I've been looking at how complex decorators can get and wondered what you are actually highlighting as a decorator. Just the '@deco'? The line that starts with '@' or a whole expression. Without an updated PEP it is hard to work out what is possible but it seems that the @ can start a complex multiline expression: @NETSignature( \ Package("Engine.Ignition"), ("Control.Argument.Tenuous.Debate", invisible("Marshalling.Infrastructure.Transport"))) def Reignite(debate): pass Neil From tdelaney at avaya.com Fri Aug 6 00:58:53 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Fri Aug 6 00:59:03 2004 Subject: [Python-Dev] A usability argument for list-after-def Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C481@au3010avexu1.global.avaya.com> Aahz wrote: > I've been staying out of this because I thought my preference had no > chance to make it in (i.e. I don't care between "no decorators" and "@ > decorators"), but if it's getting some revival, I want to make clear > that I think putting decorators *after* ``def``, name, and parameter > list but clearly *with* the function definition makes the most sense > for readability -- in the simple cases. Who cares about the complex > cases, because it's always going to be ugly? +1 This has been my position all along. I understand the need for decorator syntax (and even defended pie-syntax on c.l.py for that reason) but I think that of all the alternatives, list after parameters, before colon is the best alternative presented. def func (params) [decorators]: pass To me this conveys the information I need clearly and in the order that I need it - name is most important, parameters second, and the "type" of function last. The vast majority of functions are between 1 and 3 parameters. Similarly, I doubt there will be many functions with more than a couple of decorators. Having everything on a single line in the majority of cases is IMO a big win - both for readability and understandability. Tim Delaney From martin at v.loewis.de Fri Aug 6 01:02:34 2004 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri Aug 6 01:02:37 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de> <55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com> Message-ID: <4112BC8A.5030604@v.loewis.de> > Except + and - would be valid Python syntax already ;) Ah, yes, the unary +. It's not clear to me why we need to have this - the unary - is certainly necessary. So we still have * and / :-) > Aesthetically I much prefer @ over ! or %. !looks !bad !next !to !text > !because !it !doesn't !stand !out and %makes %me %think %of %comments > %in %another %language. So we do need Unicode after all :-) ?classmethod def foo(): pass Regards, Martin From edreamleo at charter.net Fri Aug 6 01:06:12 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 01:06:23 2004 Subject: [Python-Dev] Questions about '@' in pep 318 References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> Message-ID: <000f01c47b40$cde05110$6700a8c0@computer> > > I was using __future__ by way of explanation. I do hope namespaces could > > somehow denote annotations. My off-the-cuff suggestion was for > > pseudo-modules, so maybe the normal module rules could be sidestepped? > > I don't see how this would be possible. The plan is that arbitrary > callables can be used as decorations, as long as they take a single > argument. Ok. Consider me dense. But I'm just wanting something that _looks_ like a module reference but isn't really. What it is really is a stand-in for '@'. Wouldn't this allow user-defined annotations, provided the compiler was in on the joke? In essence, what I am asking for is just-another-name-for-at-sign. So: just-another-name-for-at-sign.arbitrary-callable Or maybe I should hope for <...> :-) Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From martin at v.loewis.de Fri Aug 6 01:10:41 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 01:10:44 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: References: <4112A7CF.3030700@v.loewis.de> Message-ID: <4112BE71.90402@v.loewis.de> Nicolas Fleury wrote: >>> accepts(int,int,def) >>> returns(float,def) >>> def bar(low,high): >> >> >> That doesn't work. If accepts and returns are callables >> (as they should be), then this already means something >> in current Python. So this would not be backwards >> compatible. > > > I'm sorry (I don't like that solution at all honestly), but what passing > "def" to a callable means in current Python? It is not accepted by my > interactive shell... Right, I missed that point. In PEP 318, you can't define a decorator that takes a Python keyword as an argument (what would that mean, anyway), so you don't need to find syntax for that, either. If you meant to suggest that the last parameter indicates that this is not a function call but a decorator, then you just had your demonstration that this doesn't work: people are not going to recognize the keyword in that place, even if the compiler could be talked into accepting it. Regards, Martin From bob at redivi.com Fri Aug 6 01:13:14 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 6 01:13:20 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112BC8A.5030604@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de> <55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com> <4112BC8A.5030604@v.loewis.de> Message-ID: <07581E18-E735-11D8-9D9A-000A95686CD8@redivi.com> On Aug 5, 2004, at 7:02 PM, Martin v. L?wis wrote: >> Except + and - would be valid Python syntax already ;) > > Ah, yes, the unary +. It's not clear to me why we need to have this - > the unary - is certainly necessary. So we still have * and / :-) > >> Aesthetically I much prefer @ over ! or %. !looks !bad !next !to >> !text !because !it !doesn't !stand !out and %makes %me %think %of >> %comments %in %another %language. > > So we do need Unicode after all :-) > > ?classmethod > def foo(): > pass In AppleScript-ese that might be ?classmethod? def foo(): pass :) From martin at v.loewis.de Fri Aug 6 01:13:16 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 01:13:24 2004 Subject: [Python-Dev] Similar syntax In-Reply-To: <20040805220743.GC31288@burma.localdomain> References: <20040805220743.GC31288@burma.localdomain> Message-ID: <4112BF0C.9050309@v.loewis.de> Gustavo Niemeyer wrote: > After writing this list, I thought about something which is > close to the current implementation, but avoids @. If it was > already suggested, please forgive me. At least I wasn't able > to find it in the PEP/Wiki: > > > def method(foo): > pass This is discussed in the PEP: The '<...>' alternative presents parsing problems because '<' and '>' already parse as un-paired. They present a further parsing ambiguity because a right angle bracket might be a greater than symbol instead of a closer for the decorators. Regards, Martin From martin at v.loewis.de Fri Aug 6 01:15:42 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 01:15:45 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <000f01c47b40$cde05110$6700a8c0@computer> References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> <000f01c47b40$cde05110$6700a8c0@computer> Message-ID: <4112BF9E.40003@v.loewis.de> Edward K. Ream wrote: > Ok. Consider me dense. But I'm just wanting something that _looks_ like a > module reference but isn't really. What it is really is a stand-in for '@'. > Wouldn't this allow user-defined annotations, provided the compiler was in > on the joke? In essence, what I am asking for is > just-another-name-for-at-sign. In essence, you are proposing to reserve a new keyword. Solutions creating new keywords have been ruled out because they might break existing code. Regards, Martin From guido at python.org Fri Aug 6 01:24:59 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 01:25:06 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: Your message of "Fri, 06 Aug 2004 01:15:42 +0200." <4112BF9E.40003@v.loewis.de> References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> <000f01c47b40$cde05110$6700a8c0@computer> <4112BF9E.40003@v.loewis.de> Message-ID: <200408052324.i75NOxA05160@guido.python.org> > In essence, you are proposing to reserve a new keyword. Solutions > creating new keywords have been ruled out because they might break > existing code. Perhaps this could be addressed by requiring "from __future__ import decorators", for one release, just like was done for "yield". I expect that this would be acceptable to the ObjC folks, too. It wouldn't be my favorite, but I won't rule it out just because of the new keyword (and yes, this is a softening of my position on new keywords). --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Fri Aug 6 01:26:38 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 6 01:26:48 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <1f7befae040805153126848c1@mail.gmail.com> References: <4112A365.1000203@ocf.berkeley.edu> <1f7befae040805153126848c1@mail.gmail.com> Message-ID: <4112C22E.6080805@ocf.berkeley.edu> Tim Peters wrote: > [Brett Cannon] > >>I am actually getting failures from test_timeout's testConnectTimeout > > > It would have helped if you had been specific about the failures > you're seeing, and whether you get them all the time, or just some of > the time. > Sorry about that; almost filed a bug report and pasted it in there; just plain forgot to paste into here: ====================================================================== FAIL: testConnectTimeout (__main__.TimeoutTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "Lib/test/test_timeout.py", line 116, in testConnectTimeout self.addr_remote) AssertionError: error not raised ---------------------------------------------------------------------- And it always fails. > >>since the Net connection I have at school lets me connect to Google in >>under the .001 connection timeout value. If I push it to .00000001 (a >>hundred-millionth of a second) it then fails consistently. > > > What fails? The test fails? The socket fails to connect? > The test. No connection error is reported. > Note that .0000001 is the same as passing, e.g., 1e-200. I actually meant 1e-08, but apparently that is a moot point. > select() > only accepts arguments at microsecond granularity, so any positive > non-zero timeout value < 1e-6 gets chopped to exactly 0.0. > > tv.tv_sec = (int)s->sock_timeout; > tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6); > > The test really doesn't intend to pass a timeout of exactly 0 to > select(), so we can't change this to less than 1e-6. > That's weird because if I do the test with sock.settimeout(0.0) and then sock.connect(('www.google.com', 80)) I get:: Traceback (most recent call last): File "", line 1, in ? File "", line 1, in connect socket.error: (36, 'Operation now in progress') But if I do the exact same thing (new socket, etc.) after closing the previous one but with sock.settimeout(0.00000001) I get:: Traceback (most recent call last): File "", line 1, in ? File "", line 1, in connect socket.timeout: timed out which is what the test expects. Could this be a platform difference? Or is the conversion happening else where and thus not the equivalent of passing 0 to settimeout()? > >>Now the problem is that the second part of the test uses this and a fuzz >>value (currently at 2) to see if the test timed out within the proper >>amount of time. The comparison is basically if the amount of time it >>took to do the timed out failure is less than timeout + fuzz. So >>lowering this number could possibly affect the test, although at .001 >>seconds, I am doubting that will occur. But since these types of timing >>tests can be touchy I thought I would double-check. > > > Na, 2.0 is gigantic compared to .001 already. For the purposes of the > test, 2 isn't really more gigantic compared to 1e-6. > I figured as much. > >>So if anyone thinks it is bad to lower the value to .00000001 then >>please let me know. > > > Changing it to 1e-7 is out, for the reason explained earlier. I'd > like to keep .001, because while the interface to select() *allows* > specifying microsecond resolution, there's no guarantee that it can > use that much precision. Most (all?) implementations should be able > to deal with millisecond resolution, though. Perhaps we could pick on > something other than www.google.com. Maybe www.python.org (everyone > in America is far from that ). =) I like that idea of changing of the site we hit. Seems slightly rude to be eating Google's bandwidth with our regression suite when we have our own server we can pound (relatively speaking; obviously the test suite is not run by *that* many people). I will go ahead and do that. -Brett From fumanchu at amor.org Fri Aug 6 01:22:17 2004 From: fumanchu at amor.org (Robert Brewer) Date: Fri Aug 6 01:27:16 2004 Subject: [Python-Dev] A usability argument for list-after-def Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DE2@exchange.hqamor.amorhq.net> Skip wrote: > >> So there it is, my personal argument for > list-after-def as better > >> than "@" ... > > aahz> +1 > > What he said. > > +1. > > I had simply given up arguing the point because it seemed > pointless and I > was tired of arguing. I suppose there are more than a few > people in this > particular boat. ObAOLMeToo. +1 Good-thing-we're-NOT-voting-ly yours, Robert Brewer MIS Amor Ministries fumanchu@amor.org From janssen at parc.com Fri Aug 6 01:32:19 2004 From: janssen at parc.com (Bill Janssen) Date: Fri Aug 6 01:32:41 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: Your message of "Thu, 05 Aug 2004 11:26:42 PDT." Message-ID: <04Aug5.163225pdt."58612"@synergy1.parc.xerox.com> > class ModelObject (NSObject): > > @objc.accessor > def messageLength(self): > return "XIV" > > @objc.signature("v@:@i") > def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, > returnCode, contextInfo): > pass I'd prefer (and I don't remember what the issue about this was): def messageLength(self) as objc.accessor: return "XIV" def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, returnCode, contextInfo) as objc.signature("v@:@i"): pass or def foo () as staticmethod: I'm also happy with the def staticmethod foo(): proposal, but the point about breaking etags seems reasonable to me. Maybe when the PEP captures the dscussion we can all read it and come to a consensus, because we'll all understand what's wrong with the non-pie proposals. Bill From shalabh at cafepy.com Fri Aug 6 01:32:46 2004 From: shalabh at cafepy.com (Shalabh Chaturvedi) Date: Fri Aug 6 01:32:54 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <20040805232522.A1B8E1E400A@bag.python.org> References: <20040805232522.A1B8E1E400A@bag.python.org> Message-ID: <52559.162.119.64.111.1091748766.squirrel@162.119.64.111> I converted some code which makes good use decorators (an example follows): @paramnames(username='Username', password='Password') @webmethod(username=str, password=str) def login(self, username, password, return_to=None): request = current.request ... Here is some usability feedback: + Easy to type. I found myself typing the function name before going up one line and typing the decorators - @ decorator @ decorator . + One line is one decorator - this preserves clarity while writing and reading. + I look at the code and all decorators line up in a column just above the function - very easily browsable and hard to miss. No searching or scanning required. + Changing order or decorators (if it matters) is easy. The only downside I found was: - @ is a 'noisy' character and makes it look a little cluttered. Usually def has an empty line before it but with decorators there is some funky code sticking to it. Not a big deal, though. Also, this symptom might get alleviated once editors know how to color decorators nicely. An alternative character (as others have suggested) might be good too. If so, I found '|' particularly nice :) |paramnames(username='Username', password='Password') |webmethod(username=str, password=str) def login(self, username, password, return_to=None): request = current.request ... |webmethod() def logout(self, return_to=None): current.request.session.user = None return current.request.redirect(return_to or self._q_location) Appears accentuated, yet clean, specially for simple/single decorators. Gives an appearance that the decorators are 'connected' to the function. Conclusion: as a user I'm +1 on the @-decorator syntax, or any similar syntax using an alternative character. Cheers, Shalabh From fumanchu at amor.org Fri Aug 6 01:35:18 2004 From: fumanchu at amor.org (Robert Brewer) Date: Fri Aug 6 01:40:18 2004 Subject: [Python-Dev] Call for defense of @decorators Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DE3@exchange.hqamor.amorhq.net> Martin: > So we do need Unicode after all :-) > > ?classmethod > def foo(): > pass And after that, we can replace 'def' with \u0402 >:) ?classmethod ? foo(): pass Robert Brewer MIS Amor Ministries fumanchu@amor.org From tdelaney at avaya.com Fri Aug 6 01:48:29 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Fri Aug 6 01:48:45 2004 Subject: [Python-Dev] Re: Call for defense of @decorators Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C4B4@au3010avexu1.global.avaya.com> Shalabh Chaturvedi wrote: > |paramnames(username='Username', password='Password') > |webmethod(username=str, password=str) > def login(self, username, password, return_to=None): > request = current.request > ... > > |webmethod() > def logout(self, return_to=None): > current.request.session.user = None > return current.request.redirect(return_to or self._q_location) > > Appears accentuated, yet clean, specially for simple/single > decorators. > Gives an appearance that the decorators are 'connected' to the > function. This looks remarkably nice to me for a syntax before the definition. Astoundingly so. I still prefer list-before-colon, but the above would alleviate much of my distate for @ which is that it doesn't look as though the decorators are connected to the definition. Hmm - perhaps a further variation ... |paramnames(username='Username', password='Password') | |webmethod(username=str, password=str) | def login(self, username, password, return_to=None): request = current.request i.e. the decorators must be connected to the def, but you can use some whitespace to make them look better (for decorators with long parameter lists). Would be hell to parse though ... Tim Delaney From janssen at parc.com Fri Aug 6 01:48:23 2004 From: janssen at parc.com (Bill Janssen) Date: Fri Aug 6 01:48:49 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: Your message of "Thu, 05 Aug 2004 15:19:16 PDT." <16658.45668.851011.950341@montanaro.dyndns.org> Message-ID: <04Aug5.164830pdt."58612"@synergy1.parc.xerox.com> > > >> So there it is, my personal argument for list-after-def as better > >> than "@" ... > > aahz> +1 > > What he said. > > +1. > > I had simply given up arguing the point because it seemed pointless and I > was tired of arguing. I suppose there are more than a few people in this > particular boat. > > Skip Ditto. +1. Bill From barry at python.org Fri Aug 6 01:54:53 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 01:54:47 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <028301c47b3f$b1b08bf0$034b8890@neil> References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <1091741561.8541.144.camel@localhost> <028301c47b3f$b1b08bf0$034b8890@neil> Message-ID: <1091750093.8545.162.camel@localhost> Note: Reply-To set to python-mode@python.org On Thu, 2004-08-05 at 18:58, Neil Hodgson wrote: > I've been looking at how complex decorators can get and wondered what you > are actually highlighting as a decorator. Just the '@deco'? The line that > starts with '@' or a whole expression. It's damn simple: on a line where the first non-whitespace character (specifically, not space or tab) is an '@', then everything from there to the end of the line. It's more difficult to specify multi-line font-lock regular expressions in X/Emacs, and at least XEmacs has a hard time font-locking multiple lines on the fly, so I don't intend to figure that out. Patches are welcome if they work on both Emacs and XEmacs. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/4c5193f9/attachment.pgp From pje at telecommunity.com Fri Aug 6 02:14:37 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 02:10:34 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <04Aug5.163225pdt."58612"@synergy1.parc.xerox.com> References: Message-ID: <5.1.1.6.0.20040805201247.01edcd50@mail.telecommunity.com> At 04:32 PM 8/5/04 -0700, Bill Janssen wrote: >I'd prefer (and I don't remember what the issue about this was): > > def messageLength(self) as objc.accessor: > return "XIV" > > def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, >returnCode, contextInfo) as objc.signature("v@:@i"): > pass > >or > > def foo () as staticmethod: > >I'm also happy with the > > def staticmethod foo(): > >proposal, but the point about breaking etags seems reasonable to me. >Maybe when the PEP captures the dscussion we can all read it and come >to a consensus, because we'll all understand what's wrong with the >non-pie proposals. See http://www.python.org/moin/PythonDecorators for a current pros-and-cons tally on the syntaxes you referenced. From greg at cosc.canterbury.ac.nz Fri Aug 6 02:25:59 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 02:26:06 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <2mhdrh96nd.fsf@starship.python.net> Message-ID: <200408060025.i760Pxn9000712@cosc353.cosc.canterbury.ac.nz> > > Seems to me the point at which we start allowing new-style classes as > > exceptions should also be the point at which we drop the idea of > > string exceptions. Would that help? > > It would probably make things a little simpler, but probably not in a > major way. I was thinking it might avoid the need to enforce a common base class for exceptions, since it would remove the ambiguity of whether 'raise "spam"' is raising a string exception or an instance of class str. But if it's considered a good idea to enforce a common root anyway, I guess it doesn't make much difference. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From fperez528 at yahoo.com Fri Aug 6 02:30:20 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Fri Aug 6 02:30:27 2004 Subject: [Python-Dev] Re: Similar syntax References: <20040805220743.GC31288@burma.localdomain> Message-ID: Gustavo Niemeyer wrote: > > def method(foo): > pass > > and also > > > def bar(low, high): > ... > > Advantages over current syntax: > > - Avoids introducing a new meaningful character in the syntax; > > - Leaves Leo and IPython people happy; Yup (from the ipython side). Best, Fernando ps. I think I recall this havnig been discarded, but since it's not in the PEP, I don't really remember where. It may even be my memory playing tricks on me. From greg at cosc.canterbury.ac.nz Fri Aug 6 02:32:32 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 02:32:38 2004 Subject: [Python-Dev] Python in Unicode context In-Reply-To: <20040805112928.GA5208@solar.trillke> Message-ID: <200408060032.i760WWXw000745@cosc353.cosc.canterbury.ac.nz> > Then i guess that searching down into a recursive structure and just > raising an "i found it" result object up doesn't count as a use case in > your book, right? You can always wrap the object you want to return in a suitable subclass of Exception. I would prefer to write it that way anyway, otherwise I'd need some sort of "catch anything which *isn't* a subclass of Exception" statement, which is rather awkward to spell. If this is considered a common enough requirement, a special built-in Exception subclass could even be provided for it, to save people the bother of defining their own... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Fri Aug 6 02:35:05 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 02:35:13 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <2mvffx7lw3.fsf@starship.python.net> Message-ID: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz> > >> class C(random.choice([dict, list])): > >> pass > >> > >> is my favourite example of this :-) > > > > Where is THAT monstrocity used!? > > Well, it's not, I hope :-) Hmmm... maybe if you had some algorithm that was more efficient using either a list or a dict, depending on the input data, but in a way that makes it difficult to tell ahead of time which to use, and you want to improve the average behaviour... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From fperez528 at yahoo.com Fri Aug 6 02:39:41 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Fri Aug 6 02:39:48 2004 Subject: [Python-Dev] Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> Message-ID: Gustavo Niemeyer wrote: > - The @ character is used in at least two tools (Leo, IPython), > and this is being considered as something bad, but not a > show stopper; Just to record my comments here as the ipython author. I've held off on calling on the ipython lists for suggestions on what to replace @ with, waiting for the dust to settle on this matter. If @ ends up being accepted, ipython can obviously adapt. I'll replace it with alternative syntax, be it @@, %, or some other kind of special-casing trick. It would not make me happy, and it worries me that ipython explicitly uses (for good reasons, I like to think), the three characters explicitly avoided by python: @ -> ipython's 'magic' control system, which is fully user-extensible. ? -> ipython's object introspection system, with ?? showing additional info. $ -> in ipython's shell extensions, a syntax for quick shell output capture, either into strings ($astr=ls *.py) or lists ($$alist=ls *.py). I worry that soon I'll end up rewriting other parts of ipython as soon as ? and $ also become python characters. But I accept that fact, since nowhere did the docs ever _promise_ that these characters would remain off-limits forever. I knew there was a risk involved, and if that happens, I'll figure out a way to solve it. But I'd much rather not. Best, Fernando. From greg at cosc.canterbury.ac.nz Fri Aug 6 02:39:43 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 02:39:51 2004 Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in Unicode context) In-Reply-To: <20040805153443.GB5208@solar.trillke> Message-ID: <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz> > My point is that I like to regard try/except as a mechanism for > "out-of-band" objects. Guidos "should be shot" seems to indicate he > sees try/except only useful/applicable to exception-handling. If the root exception class were called something else, such as 'Raisable', would that make you feel better? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From art at wiktorsadowski.com Fri Aug 6 02:52:19 2004 From: art at wiktorsadowski.com (Wiktor Sadowski) Date: Fri Aug 6 02:52:05 2004 Subject: [Python-Dev] Decorator syntax proposal Message-ID: <04ac01c47b4f$a08c2b30$0201010a@kret> FWIW: [def][decorateMethod1(param)] [def][decorateMethod2(param)] def method(self,param): pass or all in one line [def][decorateFunc1(param)][decorateFunc2(param)] def func(param): pass [class][decorateClass(param)] class toBeDecorated: pass import hooks [import][modify(mod)] import foo Regards Wiktor Sadowski From tim.peters at gmail.com Fri Aug 6 03:22:28 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 6 03:22:38 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <4112C22E.6080805@ocf.berkeley.edu> References: <4112A365.1000203@ocf.berkeley.edu> <1f7befae040805153126848c1@mail.gmail.com> <4112C22E.6080805@ocf.berkeley.edu> Message-ID: <1f7befae04080518224f886206@mail.gmail.com> [Brett Cannon] > That's weird because if I do the test with sock.settimeout(0.0) ... That's very different. If you set the socket timeout to exactly 0, Python never calls select(). If you set it to anything > 0, Python does call select(), but may truncate the timeout to exactly 0 in order to call select. > and then sock.connect(('www.google.com', 80)) I get:: > > Traceback (most recent call last): > File "", line 1, in ? > File "", line 1, in connect > socket.error: (36, 'Operation now in progress') > > But if I do the exact same thing (new socket, etc.) after closing the > previous one but with sock.settimeout(0.00000001) I get:: > > Traceback (most recent call last): > File "", line 1, in ? > File "", line 1, in connect > socket.timeout: timed out It will all be obvious if you read sock_connect(). In particular, the message you got there is produced by if (timeout) { PyErr_SetString(socket_timeout, "timed out"); return NULL; } and you'll find that it's impossible for that to trigger if you set the socket timeout to exactly 0. I'd call that a bug, except it's not worth anyone's time to fix ... From anthony at interlink.com.au Fri Aug 6 02:12:15 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 03:36:54 2004 Subject: [Python-Dev] trunk is no longer frozen In-Reply-To: <41112923.8060505@interlink.com.au> References: <41112923.8060505@interlink.com.au> Message-ID: <4112CCDF.7030609@interlink.com.au> I'm done with the trunk. Please feel free to check in again. From anthony at interlink.com.au Fri Aug 6 03:21:32 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 03:38:33 2004 Subject: [Python-Dev] PEP 318, and the PEP process Message-ID: <4112DD1C.3060902@interlink.com.au> I'm extremely averse to making the Python development process any heavier than it has to be, but the complaint that PEP 318 should have been updated to the state of play before the checkin is a fair complaint. I'm not sure what the best approach here is - should we make a motherhood-and-apple-pie statement that, all things being equal, a PEP should be updated before the code it documents is checked in? Should we aim to have it done before the first release including the code? Before the first _final_ release that includes the code? My answers would be "before checkin, where possible; before first alpha/beta release, where really possible; before first final release, absolutely". Having said that, I don't think the lack of completed PEP is a reason to back out the @ syntax from CVS. If nothing else, it being present in a released alpha is giving us very real experience with the use of the feature. There's also the issue that there's a bunch of other existing PEPs describing features that aren't up to date at all. Which brings up another related point: Many PEPs contain "open issues" and the like, that are never ever backfilled. Largely this is a matter of round tuits - the PEP authors are generally drawn from a very small group of developers. How can we encourage other people to contribute to this? The obvious way (to me) is some form of Wiki annotation. I don't think we want to make the Wiki the primary copy of the document, but I think having the PEPs annotatable would be a win. -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Fri Aug 6 03:21:21 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 03:38:35 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: <4112DD11.1060006@interlink.com.au> Phillip J. Eby wrote: > Guido's principal argument against list-after-def, if I recall > correctly, was that it is ugly when multiple or lengthy decorators are > involved. But, "ugly" isn't an argument any more, so that shouldn't > rule out list-after-def. :) I think you're confusing "ugly" (the @ form) with "potentially hidden several lines down" (list-after-def). My beef with list-after-def is that it's easy to overlook. I suspect in many cases, the decorating of a function or method will fundamentally alter the behaviour of the function or method, so in that case, I'd prefer to make it as obvious as possible. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Fri Aug 6 03:21:54 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 03:38:46 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: <4112DD32.5080302@interlink.com.au> My take on the @decorator syntax is that after initially hating it, I've found it growing on me more and more. It's extremely clear and very readable. Most importantly, in my eyes, it's obviously grouped with the def that follows it. From being a -0 (when Guido first mentioned it) to a +0 (at the time of checkin) I'm now +1 on this form of decorator. (It's different in that way to "print >>", which I still hate ) A couple of other arguments that I've seen: "This is a backwards-incompatible change to Python's syntax." So are generator expressions, and they're way harder to spot in a piece of code. Python syntax changes over time, that's always been how it is. "The @ sign is used in tool X." This is disappointing, but it can't be the sole reason to not use @ - Python can't be constrained to not breaking any other existing third-party tools. Hopefully there's a solution to the Leo problem - perhaps doubling the @ sign for Leo? "The @ sign is used in Perl." And? -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Fri Aug 6 03:21:47 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 03:38:55 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112B5B4.9@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> Message-ID: <4112DD2B.5090403@interlink.com.au> Martin v. L?wis wrote: > does explain syntax and semantics, and there are testcases as well. So > it's just the PEP that needs updating (a section in whatsnew also needs > to be drafted). Andrew's already done this, a couple of days ago. As far as the rest of the documentation - yes, absolutely. But a change this deep can't be expected to find all of the doc places that need updating in a single go. :-/ -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Fri Aug 6 03:21:41 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 03:38:57 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112A18D.5090801@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de Message-ID: <4112DD25.2010105@interlink.com.au> Martin v. L?wis wrote: > Or you could argue on a procedural basis: regardless of whether > the feature is good or bad, the current implementation is > unacceptable, as the PEP does not correspond with the > implementation, the syntax is undocumented, the code has no test > cases, and so on. I'm actually going to do that, because I do > think the process is unacceptable, and should be either corrected > or reversed (of course, this says nothing about the feature itself, > or the code implementing it). Note that @decorators are hardly unique in not having an up-to-date PEP. Where it's different to the other cases is that they're rather controversial, and therefore it's more obvious that there's a problem. -- Anthony Baxter It's never too late to have a happy childhood. From dave at pythonapocrypha.com Fri Aug 6 03:59:30 2004 From: dave at pythonapocrypha.com (Dave Brueck) Date: Fri Aug 6 03:59:41 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112DD1C.3060902@interlink.com.au> References: <4112DD1C.3060902@interlink.com.au> Message-ID: <4112E602.4050009@pythonapocrypha.com> Anthony Baxter wrote: > Having said that, I don't think the lack of completed > PEP is a reason to back out the @ syntax from CVS. If > nothing else, it being present in a released alpha is > giving us very real experience with the use of the > feature. If I may, I'd like to chime in with an "I agree". My understanding is that the decorator change was checked in for the very reason you state - so that people can actually try it out instead of just talking about it. If that is the case, then having the PEP up to date before an alpha release would certainly be nice but doesn't seem to be a hard requirement. (FWIW, it seems that a lot of the uproar about the syntax change is that it is being interpreted as a permanent change to the language rather than just experimental functionality. I _think_ I read here on python-dev that it was semi-experimental, but if that's the case I'm not sure how well that message got out - e.g. the "What's New in Python 2.4a2" section of NEWS.html doesn't reflect this) From greg at cosc.canterbury.ac.nz Fri Aug 6 04:07:42 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 04:07:48 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com> Message-ID: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> "Phillip J. Eby" : > I updated the Wiki page, and added another syntax, the original "def > function() [decorator]:" syntax. Interestingly, it looks like one of the > options with the most pluses and fewest minuses of any syntax you've listed > on that page; it's only ugly for long decorator definitions, and Guido said > he didn't care if a syntax was ugly. :) Moreover, it seems to me that the main use cases proposed for long decorator expressions would be served just as well, or perhaps even better, by a convenient way of specifying function attributes, together with a suitable metaclass. def aMethodWithAnInt_andAString(self, x, y): @signature = "is" ... To me, that seems like a more logical place to put something like a signature for an external function interface. It's a detail of interest to the implementor of the function, but not to its user, so putting it above the "def" gives it too much prominence. Also, putting anything up there (especially something long) tends to interfere with one's ability to scan down a list of defs looking for a function name. I hope Guido will see these arguments as being rational ones about usability and not irrational ones about aesthetics. At the least, perhaps they could be included in the PEP for posterity. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From jhylton at gmail.com Fri Aug 6 04:09:07 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Aug 6 04:09:08 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <4112DD11.1060006@interlink.com.au> References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> <4112DD11.1060006@interlink.com.au> Message-ID: On Fri, 06 Aug 2004 11:21:21 +1000, Anthony Baxter wrote: > Phillip J. Eby wrote: > > Guido's principal argument against list-after-def, if I recall > > correctly, was that it is ugly when multiple or lengthy decorators are > > involved. But, "ugly" isn't an argument any more, so that shouldn't > > rule out list-after-def. :) > > I think you're confusing "ugly" (the @ form) with "potentially hidden > several lines down" (list-after-def). My beef with list-after-def is > that it's easy to overlook. I suspect in many cases, the decorating of > a function or method will fundamentally alter the behaviour of the > function or method, so in that case, I'd prefer to make it as obvious > as possible. Since this thread has become something of a poll anyway, I'll simply add: I agree. Jeremy From djc at object-craft.com.au Fri Aug 6 04:15:07 2004 From: djc at object-craft.com.au (Dave Cole) Date: Fri Aug 6 04:15:14 2004 Subject: [Python-Dev] trunk is no longer frozen In-Reply-To: <4112CCDF.7030609@interlink.com.au> References: <41112923.8060505@interlink.com.au> <4112CCDF.7030609@interlink.com.au> Message-ID: <4112E9AB.9060003@object-craft.com.au> Anthony Baxter wrote: > I'm done with the trunk. Please feel free to check in again. Although I have checkin permission on CVS I would appreciate it if someone could look at my socketpair() patch before I do check it in... Could people who care about the addition of socketpair() to the socket module please have a look at patch #1003700. The patch is relative to 2.4a1 but applies cleanly to 2.4a2 with slight offsets. - Dave -- http://www.object-craft.com.au From greg at cosc.canterbury.ac.nz Fri Aug 6 04:15:30 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 04:15:34 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com> Message-ID: <200408060215.i762FUeY000881@cosc353.cosc.canterbury.ac.nz> "Phillip J. Eby" : > That argument has been done to death several times in the last year > here. Function attributes aren't a replacement for decorators. Just to be clear, I wasn't arguing in my last post that decorators should be replaced by function attributes. I was questioning the assumption that "use cases exist for long decorators, therefore any syntax for decorators needs to accommodate them". In other words, a syntax for short decorators plus a syntax for long function arguments might be sufficient. There might even be a proof of sorts for this: arguments to the decorator can be substituted with attributes on the function about to be decorated, which the decorator extracts. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From python at rcn.com Thu Aug 5 16:15:49 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 6 04:17:39 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112DD1C.3060902@interlink.com.au> Message-ID: <00e001c47af6$b594fc40$e841fea9@oemcomputer> > I'm extremely averse to making the Python development > process any heavier than it has to be, but the complaint > that PEP 318 should have been updated to the state of > play before the checkin is a fair complaint. I'm not > sure what the best approach here is - should we make > a motherhood-and-apple-pie statement that, all things > being equal, a PEP should be updated before the code > it documents is checked in? Should we aim to have it > done before the first release including the code? Before > the first _final_ release that includes the code? > > My answers would be "before checkin, where possible; > before first alpha/beta release, where really possible; > before first final release, absolutely". That sounds reasonable. Also, it reflects an understanding that "where possible" sometimes means that: - the maintainers are temporarily out of cycles, - the discussion is on-going and not fully resolved, - the real docs are more important that the PEP, and - at some point, the implementation leads rather than lags the PEP. In the case decorators, it sounds like all of these apply. Also, it would seem that the primary interest in the pep is to frame on going arguments. IOW, this would not be the last update. With genexps, the information for the final pep update was not even available until the patch discussion and newsgroup discussion wound down. Brett can attest to the difficulty of summarizing a voluminous discussion that has not reached resolution. Interestingly, after the genexp update, no one seemed to care about or even read it. Everything they wanted to know had already been discussed ad nauseum or written it the docs. One other thought is that peps implemented in phases seem to get neglected. I don't think either Sets or Unifying Int/Long is current. Raymond From jhylton at gmail.com Fri Aug 6 04:18:53 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Aug 6 04:18:55 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112DD1C.3060902@interlink.com.au> References: <4112DD1C.3060902@interlink.com.au> Message-ID: On Fri, 06 Aug 2004 11:21:32 +1000, Anthony Baxter wrote: > I'm extremely averse to making the Python development > process any heavier than it has to be, but the complaint > that PEP 318 should have been updated to the state of > play before the checkin is a fair complaint. I'm not > sure what the best approach here is - should we make > a motherhood-and-apple-pie statement that, all things > being equal, a PEP should be updated before the code > it documents is checked in? Should we aim to have it > done before the first release including the code? Before > the first _final_ release that includes the code? My answer would be before checkin unless there is widespread consensus or there's only one obvious way to do it. Developer time is a scarce resource. Checking in code without having a decent PEP, particularly for major changes or new features, just wastes everyone's time. We spend hundreds of hours, collectively, repeating old debates and speculating about why something was done the way it was. Since no one (?) is doing Python development as a day job, it's particularly important that we make effective use of our spare time. A PEP makes the development process more heavyweight -- but that was its primary goal. We wanted to make sure the specification and rationale got written ahead of time. Then other developers would know what's happening, the spec would actually provide the new words for the language ref (do we have that for decorators or genexps?), and we'd be sure that the developer had thought through all the issues. Jeremy From anthony at interlink.com.au Fri Aug 6 04:18:55 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 04:19:10 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112E602.4050009@pythonapocrypha.com> References: <4112DD1C.3060902@interlink.com.au> <4112E602.4050009@pythonapocrypha.com> Message-ID: <4112EA8F.9030106@interlink.com.au> Dave Brueck wrote: > (FWIW, it seems that a lot of the uproar about the syntax change is that > it is being interpreted as a permanent change to the language rather > than just experimental functionality. I _think_ I read here on > python-dev that it was semi-experimental, but if that's the case I'm not > sure how well that message got out - e.g. the "What's New in Python > 2.4a2" section of NEWS.html doesn't reflect this) I thought about adding something to the release announcement about this, but decided against it, as I felt it wouldn't help the process, but would instead lead to even more complaints. My take on this was that the @syntax was put in 2.4a2, and was experimental only in that if there was a better solution to come up, we'd use that. That is, the default action was that it would stay in. Guido, or anyone else, can chime in and let me know if I've mis-stated this. -- Anthony Baxter It's never too late to have a happy childhood. From greg at cosc.canterbury.ac.nz Fri Aug 6 04:24:02 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 04:24:07 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com> Message-ID: <200408060224.i762O2Vc000926@cosc353.cosc.canterbury.ac.nz> Ronald Oussoren : > And how would that solve this: > FWIW, in decorator-plus-function-attributes style, this would be def do_something(x,y) [generic]: @when = "x in int and y in str" # body here will be executed when 'x' # is an integer and 'y' is a string Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From anthony at interlink.com.au Fri Aug 6 04:29:21 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 04:29:37 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112DD2B.5090403@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> Message-ID: <4112ED01.6010304@interlink.com.au> I wrote: > Martin v. L?wis wrote: > >> does explain syntax and semantics, and there are testcases as well. So >> it's just the PEP that needs updating (a section in whatsnew also needs >> to be drafted). > > > Andrew's already done this, a couple of days ago. I meant to include this link: http://www.python.org/dev/doc/devel/whatsnew/node5.html -- Anthony Baxter It's never too late to have a happy childhood. From greg at cosc.canterbury.ac.nz Fri Aug 6 04:35:08 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 04:35:13 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz> "Phillip J. Eby" : > Now, I don't know if my thought process here is at all typical or > representative of anything, other than my own idiosyncratic self. You're not alone. I agree with *everything* you said there. All of that stuff should go in the PEP, if it's not there already. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From niemeyer at conectiva.com Fri Aug 6 04:37:34 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 04:35:16 2004 Subject: [Python-Dev] Similar syntax In-Reply-To: <4112BF0C.9050309@v.loewis.de> References: <20040805220743.GC31288@burma.localdomain> <4112BF0C.9050309@v.loewis.de> Message-ID: <20040806023734.GA7230@burma.localdomain> > Gustavo Niemeyer wrote: > >After writing this list, I thought about something which is > >close to the current implementation, but avoids @. If it was > >already suggested, please forgive me. At least I wasn't able > >to find it in the PEP/Wiki: > > > > > > def method(foo): > > pass > > This is discussed in the PEP: > > The '<...>' alternative presents parsing problems because '<' and '>' > already parse as un-paired. They present a further parsing ambiguity > because a right angle bracket might be a greater than symbol instead of > a closer for the decorators. Gotcha! Thanks for pointing me this. -- Gustavo Niemeyer http://niemeyer.net From bob at redivi.com Fri Aug 6 04:38:25 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 6 04:38:35 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> Message-ID: On Aug 5, 2004, at 10:07 PM, Greg Ewing wrote: > "Phillip J. Eby" : > >> I updated the Wiki page, and added another syntax, the original "def >> function() [decorator]:" syntax. Interestingly, it looks like one of >> the >> options with the most pluses and fewest minuses of any syntax you've >> listed >> on that page; it's only ugly for long decorator definitions, and >> Guido said >> he didn't care if a syntax was ugly. :) > > Moreover, it seems to me that the main use cases proposed for long > decorator expressions would be served just as well, or perhaps even > better, by a convenient way of specifying function attributes, > together with a suitable metaclass. > > def aMethodWithAnInt_andAString(self, x, y): > @signature = "is" > ... > > To me, that seems like a more logical place to put something like a > signature for an external function interface. It's a detail of > interest to the implementor of the function, but not to its user, so > putting it above the "def" gives it too much prominence. > > Also, putting anything up there (especially something long) tends to > interfere with one's ability to scan down a list of defs looking for a > function name. > > I hope Guido will see these arguments as being rational ones about > usability and not irrational ones about aesthetics. At the least, > perhaps they could be included in the PEP for posterity. While using metaclasses makes a lot of things possible, I don't believe they are appropriate for this purpose. Decorating functions and methods have nothing to do with the class, why should the class have to change in order to support it? What about bare functions? Metaclasses can become difficult to use and understand when you're using several of them at once. If you look at it from another implementation standpoint, reasonable syntax for function attributes can be implemented rather easily with function decorators, but vice versa requires the use of metaclasses. I think implementing decorators is the more reasonable choice. Also consider how difficult it can be to integrate metaclasses into existing code, where decorators would not have this issue at all. -bob From greg at cosc.canterbury.ac.nz Fri Aug 6 04:41:53 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 04:41:57 2004 Subject: [Python-Dev] The impact of '@' on Leo In-Reply-To: <005e01c47b2d$9c0dee80$6700a8c0@computer> Message-ID: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz> "Edward K. Ream" : > I was always under the impression that the clear statement in the > Reference Manual that at signs are invalid everywhere (except in > comments and strings) was a clear signal of an intention to keep those > symbols for other valid purposes. I've only ever regarded that as a statement about the status quo, not any kind of promise about the future. But maybe it would be a good idea to explicitly reserve some character (perhaps only at the beginning of a line). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From adurdin at gmail.com Fri Aug 6 04:46:57 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Fri Aug 6 04:47:00 2004 Subject: [Python-Dev] Re: Call for defense of @decorators Message-ID: <59e9fd3a04080519462bf57899@mail.gmail.com> (This is a repeat of a message on python-list/c.l.p in the "Tweaking @decorator syntax" thread, but I thought it was applicable to post it here, based on how I understand Guido's message. However, this is not a defense but an attack on @decorators, and based not on whether it looks ugly or not but that it is unclear.) On 5 Aug 2004 16:04:00 -0700, Sandy Norton wrote in c.l.p, thread "Tweaking @decorator syntax": > Here are some more variations. > > Please copy/paste the code below into your favourite python editor and see > how each syntactical variation of the decorator code reads, and how quickly > you grok its structure and meaning. *snip examples* Of the examples above, I find the "Nested, 'decorate' keyword syntax" by far the most clear. It has the following features (in no particular order) which I find attractive: 1. The nesting makes it abundantly clear what the decoration applies to, namely the function definitions at the next indentation level. And the colon at the end of the "decorate" line is consistent with all other Python keywords which result in another indentation level (def, if, class, etc.) -- which is one reason why I dislike the "nested @decorator" and "nested |decorator|" options. The "Nested, minimal decorator syntax" has indentation, but lacks clarity: it appears merely to be one or a sequence of function calls followed inexplicably by a nested block. 2. The nesting makes it convenient to apply the same decoration to multiple functions, which I guess will be fairly common wherever decorators are used. 3. It doesn't involve adding any additional punctuation to the language. I strongly believe that the general lack of punctuation contributes greatly towards Python's readability. 4. The comma-separated list of decorators allows for more compactness without sacrificing legibility when multiple decorators are used. Whereas I quite dislike the @ syntax, for almost opposite reasons: 5. In appearance it looks like a special kind of statement; and it is not at all obvious to me that it is rather a kind of attribute or modifier of the following function definition. I don't know of any other construction in Python that performs a similar, unmarked (i.e. non-explicit) modification of the following statement. So to me the @syntax comes across as extremely un-Pythonic (by that I mean radically inconsistent with the conventions of the rest of the language). For this same reason, I dislike all the proposed syntaxes which involve placing something before the "def" without a subsequent indented block. 6. When using the same sets of decorators for multiple functions/methods in the same module/class, the @ syntax is inconvenient, as the decorators need to be repeated for each function. (Of course, the current (2.3) syntax to achieve the same result is similarly inconvenient; the @ syntax adds no benefit in this regard, but the "decorate" syntax does). 7. The @ syntax adds more punctuation to Python's syntax: a Bad Thing. Despite the risk of breaking compatibility with some older programs, I firmly believe that adding new keywords is a much better idea in general than adding new punctuation, because (a) keywords are more readable (especially to those unfamiliar with them), and (b) there is a very limited supply of punctuation marks on the keyboard, but a very large supply of words. 8. When using multiple decorators, each appears on a separate line. This not only confuses things with respect to point 5 above (e.g. does the first decorator somehow decorate the second, or do both apply to the function?), but also is IMO unnecessarily lengthy, as each line has the same basic purpose; In analogy, the latter is similar to the difference between x = 1 y = 2 z = 3 and x, y, z = 1, 2, 3 where a compact representation is possible and suitable where the three variables are relating to the same purpose within the program. -- Andrew Durdin From bob at redivi.com Fri Aug 6 04:47:25 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 6 04:47:33 2004 Subject: [Python-Dev] The impact of '@' on Leo In-Reply-To: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz> References: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz> Message-ID: On Aug 5, 2004, at 10:41 PM, Greg Ewing wrote: > "Edward K. Ream" : > >> I was always under the impression that the clear statement in the >> Reference Manual that at signs are invalid everywhere (except in >> comments and strings) was a clear signal of an intention to keep those >> symbols for other valid purposes. > > I've only ever regarded that as a statement about the status > quo, not any kind of promise about the future. > > But maybe it would be a good idea to explicitly reserve some > character (perhaps only at the beginning of a line). You mean like # ? :) -bob From skip at pobox.com Fri Aug 6 04:48:00 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 04:48:33 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112DD1C.3060902@interlink.com.au> References: <4112DD1C.3060902@interlink.com.au> Message-ID: <16658.61792.917900.348835@montanaro.dyndns.org> Anthony> Which brings up another related point: Many PEPs contain "open Anthony> issues" and the like, that are never ever backfilled. Largely Anthony> this is a matter of round tuits - the PEP authors are generally Anthony> drawn from a very small group of developers. How can we Anthony> encourage other people to contribute to this? The obvious way Anthony> (to me) is some form of Wiki annotation. I don't think we want Anthony> to make the Wiki the primary copy of the document, but I think Anthony> having the PEPs annotatable would be a win. For topics where a lot of user input is anticipated (or discovered) such as for PEP 318, perhaps the initial round of the PEP should be a (set of) Wiki page(s). As the concepts it contains firm up a more static document can be drawn up from that. Skip From mkc at mathdogs.com Fri Aug 6 04:46:51 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Fri Aug 6 04:50:41 2004 Subject: [Python-Dev] pre-PEP: Complete, Structured Regular Expression Group Matching Message-ID: If you've ever been frustrated that there wasn't an easy way to parse a string or file, even though you could easily match it with re.match, then this PEP may be for you. If what's being proposed seems a little ghastly, well, I'll just say that I've been wanting something like this for a long time and this is the best I could come up with. Your comments invited. Mike ############################################################################## PEP: XXX Title: Complete, Structured Regular Expression Group Matching Version: $Revision: 1.3 $ Last-Modified: $Date: 2004/08/02 05:18:31 $ Author: Mike Coleman Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 1-Aug-2004 Python-Version: 2.4 Post-History: 1-Aug-2004 Abstract ======== This proposal is to add a new method ``re.structmatch`` that fully captures matches for groups that match repeatedly. The method returns a structured match tree whose shape is determined by the pattern groups. Ultimately this will allow a string that can be described by a Python regular expressions (e.g., the contents of ``/etc/passwd`` or ``.ini`` files, or the output of ``diff``) to be parsed into the obvious parse tree with a single call to ``re.structmatch``. Motivation ========== A notable limitation of the ``re.match`` method is that it fails to capture all group match information for repeatedly matched groups. For example, in a call like this :: m0 = re.match(r'([A-Z]|[a-z])*', 'XxxxYzz') one would like to see that the group which matched four times matched the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``. In the current implementation, only the last match for each group (``'zz'`` in this case) is available, even though the other matches are calculated implicitly during the matching process. For simple cases, the missing strings might be easily recovered by other means, but for more complicated cases this is a significant annoyance. The simplest extension would be to collect all of the matches for each group in a list, so that in the call above for example :: m0.group(1) --> ['X', 'xxx', 'Y', 'zz'] (A mechanism like this is apparently to be added to Perl in version 6 [#PERL6]_, though that did not inspire this PEP.) This immediately suggests the question of what is to be done for nested repeats, like so :: m1 = re.match(r'("([A-Z]|[a-z])*"\s*)*', '"Xx" "yy" "ZzZ"') We could have :: m1.group(2) --> ['X', 'x', 'yy', 'Z', 'z', 'Z' ] but this flattened result would discard useful information about the structure of the match. A more useful result would be :: m1.group(2) --> [['X', 'x'], ['yy'], ['ZzZ']] This is definitely an improvement. Consider the following slightly more complicated case, though :: m1 = re.match(r'("([A-Z]|[a-z])*"(\s*))*', '"Xx" "yy" "ZzZ"') which would give :: m1.group(2) --> [['X', 'x'], ['yy'], ['Z', 'z', 'Z']] m1.group(3) --> [' ', ' ', ''] This is less useful than the putative conceptual structure of the match, which would be something like :: [[['X', 'x'], ' '], [['yy'], ' '], [['Z', 'z', 'Z'], '']] In this simple case, this structure could be recovered using the ``zip`` function, but in the general case this is a non-trivial transformation. This PEP proposes a mechanism to address the general case directly. Proposal ======== The new function ``re.structmatch`` has the same arguments as ``re.match`` and will match using exactly the same algorithm. Instead of returning a MatchObject upon success, ``re.structmatch`` will return a "tree" (or rather, a forest) in the form of a Python list. First we will define some terms. Groups in the input pattern are parenthesized subexpressions that "capture" strings when matched, and are of the form ``'(abc)'`` or ``'(?Pabc)'``. Leaf groups are groups that do not contain any subgroups. So, for example, the group ``'(abc)'`` is a leaf group and the group ``'(a(xyz)b)'`` is not a leaf group (but it does contain the leaf group ``'(xyz)'``). We shall call parenthesized expressions that are not groups "nongroups"; these include constructs like ``'(?:abc)'`` as well as lookahead and lookbehind assertions. The structure of the returned tree is determined from the grouping tree present in the pattern in the following manner: * Leaf groups not followed immediately by a repeat operator map to a single string:: re.structmatch(r'(...)', 'abcdef') --> ['abc'] re.structmatch(r'(...).(..)', 'abcdef') --> ['abc', 'ef'] * Leaf groups followed immediately by a repeat operator map to a list of strings:: re.structmatch(r'([^d])*', 'abcdef') --> [['a', 'b', 'c']] re.structmatch(r'([^d])*(.)*', 'abcdef') --> [['a', 'b', 'c'], ['d', 'e', 'f']] re.structmatch(r'(..)*', 'abcdef') --> [['ab', 'cd', 'ef']] re.structmatch(r'(.){2}', 'abcdef') --> [['a', 'b']] * Non-leaf groups not followed immediately by a repeat operator map to a list of the mappings of their subgroups:: re.structmatch(r'(...)', 'abcdef') --> ['abc'] re.structmatch(r'((...))', 'abcdef') --> [['abc']] re.structmatch(r'(((...)))', 'abcdef') --> [[['abc']]] re.structmatch(r'((...)())', 'abcdef') --> [['abc'], []] re.structmatch(r'(.(.)(.(.)).(.))', 'abcdef') --> ['a', ['b'], ['c', ['d']], 'e', ['f']] * Non-leaf groups followed immediately by a repeat operator map to a list of the mappings of their repeated matches:: re.structmatch(r'((.).(.))*', 'abcdef') --> [[['a', 'c'], ['d', 'f']]] re.structmatch(r'(([ab])*(x)*)*', 'baxbxx') --> [[['b', 'a'], ['x']], [['b'], ['x', 'x']]] * In the case of alternation, only the matched groups appear in the output:: re.structmatch(r'([^a])*|([^d])*', 'abcdef') --> [['a', 'b', 'c']] If it's important to know which alternative matched, named groups can be used. * Named groups map to a pair where the first member is the name and the second member is what the unnamed group would have mapped to:: re.structmatch(r'([^d])*(?P.)*', 'abcdef') --> [['a', 'b', 'c'], ('rest', ['d', 'e', 'f'])] * Nongroups do not affect the output structure. Compared to non-leaf groups, nongroups have the effect of "flattening" the output, like so:: re.structmatch(r'((.).(.))', 'abcdef') --> [['a', 'c']] re.structmatch(r'(.).(.)', 'abcdef') --> ['a', 'c'] re.structmatch(r'(?:(.).(.))', 'abcdef') --> ['a', 'c'] re.structmatch(r'((.).(.))*', 'abcdef') --> [[['a', 'c'], ['d', 'f']]] re.structmatch(r'(?:(.).(.))*', 'abcdef') --> [['a', 'c', 'd', 'f']] (Nongroups that do not contain leaf groups have no effect whatsoever on the output structure.) Additional Features ------------------- In many cases in which ``'re.structmatch'`` fails to match, the cause will be due to an unexpected error in the format of the string being matched. In order to assist the calling program in generating a more meaningful possible error message, ``'re.structmatch'`` will return the endpoint of the largest match against the searched string. So, for example :: re.structmatch('abcd', 'abxxx') --> 2 re.structmatch('abcde|z', 'abxxx') --> 2 re.structmatch('x*?y', 'xxx') --> 3 (This return value should be considered advisory rather than exact, as future improvements in the match algorithm may make it difficult to calculate the exact value.) Examples ======== Parsing ``/etc/group`` ---------------------- If ``/etc/group`` contains the following lines :: root:x:0: daemon:x:1: bin:x:2: sys:x:3: then it can be parsed as follows :: p = r'''(?xm) # VERBOSE, MULTILINE ( ( # one field, preceded by a ':' if # the field is not the line's first (?:^|:) ([^:\n]*) )* \n )* ''' s = open('/etc/group').read() tree = re.structmatch(p, s) which will give ``tree`` the following value:: [['root', 'x', '0', ''], ['daemon', 'x', '1', ''], ['bin', 'x', '2', ''], ['sys', 'x', '3', '']] Note that the above pattern is written to allow any ``/etc/group`` field to be empty. The pattern won't work correctly for versions of Python prior to 2.4 because of the possibility of repeating empty matches. This problem seems to have been fixed in Python 2.4. (A slightly more complicated pattern would work for pre-2.4 versions.) Parsing ``.ini`` files ---------------------- The Microsoft ``.ini`` file format is found in various contexts (there is one in the Python source distribution: ``Tools/freeze/extensions_win32.ini``). Given a file with contents :: [singers] good=Annie Lennox bad=William Shatner [comedians] good=Monty Python the file can be parsed with pattern :: r'''(?xm) # VERBOSE, MULTILINE \s* # leading whitespace ( # begin sections ^\[ ([^]]+) \] \s* # section header ( # begin params ^([^=]+) = # param name (.*) $ # param value \s* # intervening whitespace )* # end params )* # end sections \Z # assert that the entire # input was matched ''' to give :: [['singers', ['good', 'Annie Lennox'], ['bad', 'William Shatner']], ['comedians', ['good', 'Monty Python']]] The pattern given omits support for ``.ini`` file comments for the sake of simplicity, but this could be added. Details ======= The proposal states that ``re.structmatch`` will match using exactly the same algorithm as ``re.match``. This might be a little odd for a pattern like ``r'(x|y|z)*aaa\1'``, where the algorithm will require that the ``\1`` back-reference match (at most) one character. It's not obvious whether there is any better option, though, and there are advantages of implementation and simplicity for keeping the match algorithms of these two methods identical. (A similar argument applies to ``'(?P=NAME)'``.) Discussion ========== Part of the reason the proposed method would be so useful is that ``re.split`` currently does not split on empty matches. If it had this feature, one could split on lookahead and lookbehind assertions, which would significantly ease parsing of strings with a recursive regular structure (e.g., ``.ini`` files). Patch `#988761`_ will correct this ``re.split`` deficiency, if it is accepted. .. _#988761: https://sourceforge.net/tracker/?func=detail&aid=988761&group_id=5470&atid=305470 For particularly complex patterns, the current 99 group limit might actually become a practical problem. One could imagine a variation in which subtrees of named groups might be captured in dictionaries rather than lists, with the group names used as keys. Rejected Alternatives ===================== Several simpler alternatives are rejected in the `Motivation`_ section above. Although these alternatives would be better than nothing, they would not adequately satisfy the goal of this proposal, which is to allow the programmer to extract the structure of a string in an immediate manner. It would be possible to use tuples for some substructures in the return tree, rather than composing it strictly of lists. This alternative was rejected because it was thought useful to be able to modify the resulting tree in place, perhaps to add decorations, etc., and tuples would make this more difficult. References ========== .. [#PERL6] Multiple regex matches in Perl Apocalypse 5 (http://www.perl.com/pub/a/2002/06/04/apo5.html?page=22#rfc%20360:%20allow%20multiply%20matched%20groups%20in%20regexes%20to%20return%20a%20listref%20of%20all%20matches) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From niemeyer at conectiva.com Fri Aug 6 04:57:54 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 04:55:34 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <16658.45668.851011.950341@montanaro.dyndns.org> References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> <20040805213713.GA28782@panix.com> <16658.45668.851011.950341@montanaro.dyndns.org> Message-ID: <20040806025754.GB7230@burma.localdomain> > What he said. > > +1. +1 Given the discussed possibilities, I'm also more comfortable with list-after-def. > I had simply given up arguing the point because it seemed pointless > and I was tired of arguing. I suppose there are more than a few > people in this particular boat. That's exactly what I was afraid of. -- Gustavo Niemeyer http://niemeyer.net From skip at pobox.com Fri Aug 6 04:57:47 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 04:58:13 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112DD32.5080302@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <4112DD32.5080302@interlink.com.au> Message-ID: <16658.62379.437323.980365@montanaro.dyndns.org> Anthony> From being a -0 (when Guido first mentioned it) to a +0 (at the Anthony> time of checkin) I'm now +1 on this form of decorator. It's still not my first choice. As others have observed, most functions have no more than three parameters and it should be rare for a function to have more than two decorators, so in the common case the list-after-def works (decorators are prominent - on the def line), but don't obscure the beginning of the definition ("def", function name, parameter list). Anthony> (It's different in that way to "print >>", which I still hate Anthony> ) Odd, I've always liked it. I use it heavily for debugging. The ">>" is easy to grep for when I want to rip 'em out, and I can combine the convenience of the print statement with redirection to stderr. Anthony> "The @ sign is used in Perl." Anthony> And? I think the point there is that Perl has this history of using prefix punctuation ($, @, %) to modify the meaning of the identifier that follows, and more generally using punctuation in all sorts of non-mnemonic ways. To a certain degree @-decorators have that same feel. Skip From skip at pobox.com Fri Aug 6 05:09:12 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 05:09:28 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112E602.4050009@pythonapocrypha.com> References: <4112DD1C.3060902@interlink.com.au> <4112E602.4050009@pythonapocrypha.com> Message-ID: <16658.63064.942060.30357@montanaro.dyndns.org> >> Having said that, I don't think the lack of completed PEP is a reason >> to back out the @ syntax from CVS. If nothing else, it being present >> in a released alpha is giving us very real experience with the use of >> the feature. Dave> (FWIW, it seems that a lot of the uproar about the syntax change Dave> is that it is being interpreted as a permanent change to the Dave> language rather than just experimental functionality. I early June I suggested that two different candidate syntaxes be added for the alpha (Guido's previous before-the-def candidate and the list-after-def syntax) precisely so people could experiment with both variants and decide which one they liked better. Guido shot it down immediately: http://mail.python.org/pipermail/python-dev/2004-June/045192.html He was probably mostly averse to the two-at-once idea, but I also didn't sense any support for the idea of any experimental addition. I still think that finding a way to let users play with a couple syntax variants would be a reasonable way to sort things out. Skip From greg at cosc.canterbury.ac.nz Fri Aug 6 05:15:17 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 6 05:15:25 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C4B4@au3010avexu1.global.avaya.com> Message-ID: <200408060315.i763FHQV001012@cosc353.cosc.canterbury.ac.nz> "Delaney, Timothy C (Timothy)" : > the above would alleviate much of my distate for @ which is that it > doesn't look as though the decorators are connected to the definition. For something which looks more attached to the definition, how about staticmethod... def foo(): ### Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From pete at shinners.org Fri Aug 6 05:39:44 2004 From: pete at shinners.org (Pete Shinners) Date: Fri Aug 6 05:39:13 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> Message-ID: Guido van Rossum wrote: > It seems a public outcry against @decorators has started. When I let > Anthony check it in for 2.4a2, the plan was to see how it fares in a2 > and a3, and possibly take it out in 2.4b1. If you don't want that to > happen, you might want to say something in public in defense of > @decorators. All along I've not been too particular on the syntax for decorators. I see the need for tham, and still have sneaking suspicion this will end up changing the way people use Python in many contexts. Now that the @ syntax is out, I am feeling it is the best of the many available options. All the proposals has pros and cons, but the @ syntax seems to always win with more pros, especially after more experimentation. I look forward to taking advantage of the decorators very soon. I can't imagine there are any further syntax options remaining, although I wouldn't be opposed if the community could put something together. Anyways, this hardly turns out to be a rallying defense. But the need for functionality is there, and the many syntax proposals have been weighed. The better option has already been committed in my eyes. I'm just glad >> wasn't involved, heh. From tanzer at swing.co.at Fri Aug 6 05:43:14 2004 From: tanzer at swing.co.at (Christian Tanzer) Date: Fri Aug 6 05:43:29 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: Your message of "Thu, 05 Aug 2004 17:22:45 -0300." <20040805202245.GH29796@burma.localdomain> Message-ID: > > I'm in camp 2, but now that pie decorators are in, and I've had a chance > > to convert my code to use them, I'm strongly +1 in favor of this > > syntax. It stands out nicely, and to me indicates a stronger affinity > > to the def that follows it than the C# syntax. > > I'm glad to see people happy with the current implementation. > > > I was never in favor of C# syntax, and I'm glad it wasn't chosen. I > > strongly disliked that it subtly changed the semantics of currently > > valid Python. I like that pie decorators code cannot run in older > > Pythons, because if it /could/ it certainly wouldn't work. > > I'm strongly against "I'm sick of it" and "I don't care [anymore]" > implemented features. Seeing everyone complaining and no positive > reactions to the current implementation certainly feels like > something is wrong. Sigh. I'm just a lurker here and I don't really want to add to all the heat. All that said, I'm +1 on the pie syntax (and being ugly means that it makes decorations obvious). To all proponents of list markup: that would be very hard to look up in the documentation. `@` should be easy to find in the index. To Gustavo in particular: people are sick of hearing endless repeats of the same old tired arguments (I certainly am). -- Christian Tanzer http://www.c-tanzer.at/ From skip at pobox.com Fri Aug 6 05:54:31 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 05:54:42 2004 Subject: [Python-Dev] Re: PEP-0318 In-Reply-To: <41122FC2.1080608@interlink.com.au> References: <41122FC2.1080608@interlink.com.au> Message-ID: <16659.247.791367.302786@montanaro.dyndns.org> Anthony> ... here's what I managed [to edit in PEP 318] before a tidal Anthony> wave of crap flowed in... I checked in Anthony's changes to PEP 318 (sent in private email) with very few modifications. I simply didn't have enough time to do more this evening. I didn't even have the time to read back through the entire document. Anybody else who feels like taking a whack at it, feel free. I'll be out of Internet reach over the weekend, so it's unlikely I'll be able to do anything more with it until next week. Context diffs against v 1.15 would be best on the off-chance I get inputs from multiple people. A SourceForge patch would be the best place to stick such things so other people can see where your changes are. Note that I'm not really interested in thirty-leven new syntax proposals. What would be helpful are pointers to other documents and concise summaries of some of the discussions on c.l.py and python-dev. Incorporation of all or part of http://www.python.org/moin/PythonDecorators into the PEP would also be appreciated. Try to write objectively. A PEP isn't Usenet. Note how I'm being optimistic that multiple people will contribute. Lots of people have worn out their keyboards on this subject, so I know the energy is out there. It just needs to be focused a bit. Thx, Skip From fdrake at acm.org Fri Aug 6 05:56:29 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri Aug 6 05:56:41 2004 Subject: [Python-Dev] The impact of '@' on Leo In-Reply-To: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz> References: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408052356.29328.fdrake@acm.org> On Thursday 05 August 2004 10:41 pm, Greg Ewing wrote: > But maybe it would be a good idea to explicitly reserve some > character (perhaps only at the beginning of a line). Bob Ippolito responded: > You mean like # ? Yeah, like that. ;-) That's always accepted as the first non-blank character on a non-continuation line, and the only thing Python ever cares about after that are the encoding turd and a newline. Anything after the newline is, well, another line. I've really only managed to look at Leo much once, and Edward was sitting next to me at the time. Leo doesn't store Python code in a Leo document as any sort of embedded object that's labelled as Python code as far as I can tell (though it may in it's data model); it just more text, and can have @directives like any other text. I think this is inherently a risky approach to the data; there needs to be at least a way to say that some portion of the data is some other text format embedded into the Leo document. I'm sure Edward can find a way to make things work for Leo users, but I hope the .leo files have some sort of format version indicator to make upgrading relatively easy for Leo users. -Fred -- Fred L. Drake, Jr. From andrewm at object-craft.com.au Fri Aug 6 06:04:10 2004 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri Aug 6 06:04:13 2004 Subject: [Python-Dev] trunk is no longer frozen In-Reply-To: <4112E9AB.9060003@object-craft.com.au> References: <41112923.8060505@interlink.com.au> <4112CCDF.7030609@interlink.com.au> <4112E9AB.9060003@object-craft.com.au> Message-ID: <20040806040410.57C1E3C221@coffee.object-craft.com.au> >Although I have checkin permission on CVS I would appreciate it if >someone could look at my socketpair() patch before I do check it in... In the name of bringing this thread up out of the noise: we need a "go/no go" on applying this patch to the tree prior to a3 (the patch adds "socketpair" to the socket module. Can I ask if the gods can spend a moment reviewing it (it's well localised and includes test and documentation updates)? >Could people who care about the addition of socketpair() to the socket >module please have a look at patch #1003700. The patch is relative to >2.4a1 but applies cleanly to 2.4a2 with slight offsets. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From pje at telecommunity.com Fri Aug 6 06:12:22 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 06:08:22 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <200408060224.i762O2Vc000926@cosc353.cosc.canterbury.ac.nz> References: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com> Message-ID: <5.1.1.6.0.20040806001019.02ab6060@mail.telecommunity.com> At 02:24 PM 8/6/04 +1200, Greg Ewing wrote: >Ronald Oussoren : > > > And how would that solve this: > > > >FWIW, in decorator-plus-function-attributes style, this >would be > > def do_something(x,y) [generic]: > @when = "x in int and y in str" > # body here will be executed when 'x' > # is an integer and 'y' is a string Which reduces composability of decorators by introducting the possibility of attribute collisions, while simultaneously reducing clarity by separating decorators from their arguments. From niemeyer at conectiva.com Fri Aug 6 06:16:14 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 06:13:54 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: References: <20040805202245.GH29796@burma.localdomain> Message-ID: <20040806041614.GA7579@burma.localdomain> > Sigh. > > I'm just a lurker here and I don't really want to add to all the heat. There's no heat here. There's just a discussion about something polemic. > All that said, I'm +1 on the pie syntax (and being ugly means that it > makes decorations obvious). > > To all proponents of list markup: that would be very hard to look up > in the documentation. `@` should be easy to find in the index. > > To Gustavo in particular: people are sick of hearing endless repeats > of the same old tired arguments (I certainly am). I'm glad my tired arguments made you say your opinion about the current syntax. -- Gustavo Niemeyer http://niemeyer.net From nbastin at opnet.com Fri Aug 6 07:01:46 2004 From: nbastin at opnet.com (Nick Bastin) Date: Fri Aug 6 07:02:16 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings In-Reply-To: <4112B7A8.5090100@v.loewis.de> References: <16658.42673.542930.749915@montanaro.dyndns.org> <4112A968.2030401@v.loewis.de> <16658.45979.672934.897388@montanaro.dyndns.org> <4112B7A8.5090100@v.loewis.de> Message-ID: On Aug 5, 2004, at 6:41 PM, Martin v. L?wis wrote: > Skip Montanaro wrote: >> The platform is Solaris/Intel (2.8 and/or 2.9). I'll have to get you >> more >> complete details tomorrow. The involved folks are gone for the day. > > Ok. I'll get back to a Solaris machine next week, so I can investigate > somewhat myself. > > Of course, the rule is, strictly speaking, "no warnings on the major > platforms"; I'm uncertain whether Solaris still applies here. It is > clearly not feasible to have Python compile with no warning no matter > what the compiler or operating system is. The major platforms, of > course, are Windows, Linux, OS X (i.e. the ones we provide binaries > for). I've been trying to keep the solaris build warning-free for the last few months, although I've been using SunPro and not GCC. I would like to believe that Solaris is a 'major platform' (sparc, at least), but maybe I'm misguided.. :-) -- Nick From psoberoi at gmail.com Fri Aug 6 07:02:26 2004 From: psoberoi at gmail.com (Paramjit Oberoi) Date: Fri Aug 6 07:02:35 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> Message-ID: > > on that page; it's only ugly for long decorator definitions, and Guido said > > he didn't care if a syntax was ugly. :) > I hope Guido will see these arguments as being rational ones about > usability and not irrational ones about aesthetics. At the least, I'm worried by the thought expressed multiple times today that ugly is OK and aesthetics don't matter. I don't think aesthetics can be separated from usability, and FWIW, I would like to say that as far as I am concerned, Aesthetics are what make Python what it is. Yes, the development community is very nice, and so is the standard library, but the real thing distinguishes Python in my mind is that the programs are so incredibly nice to look at. It is the beauty of the programs that has kept me faithful and fanatically devoted to Python all these years. Python has gotten more complex over the years, but most of the additions---list/generator comprehensions, generators, and iterators in particular---have enhanced aesthetics of the language. I do reaize that there are some hard problems with decorator syntax, and that even a somewhat ugly choice will vastly improve some people's lives, but please don't say ugly is fine and aesthetics are unimportant. As Tim Peters said in import this, "Beautiful is better than ugly" and "Readability counts". -param PS1: This is just a general comment based on the atmosphere I have observed today, and not a specific criticism of anyone. Yes, I realize that when Greg said ,"...see these arguments as being rational ones about usability and not irrational ones about aesthetics" he meant that arguments about aesthetics are usually subjective. I apologise if I'm offending anyone or stating the obvious. PS2: I like the list-after-def syntax. From tim.peters at gmail.com Fri Aug 6 07:31:57 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 6 07:32:00 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> Message-ID: <1f7befae04080522314d818d8f@mail.gmail.com> [Paramjit Oberoi] > ... > As Tim Peters said in import this, "Beautiful is better than ugly" and > "Readability counts". Indeed. Luckily, since I was able to channel those words from the depths of Guido's bottomless mind, I can also channel what beauty and readability mean. ... > PS2: I like the list-after-def syntax. There's quite a peculiar split over that. I find the @ gimmick most readable of all, and especially because it forces one decoration per physical line. list-after-def looks like a train wreck piling up after the arglist half the time. But @ looks like a friendly little womb, with a happy little birth canal, out of which is born a single pure expression. What's more beautiful than the miracle of birth? Certainly not a train wreck. leave-the-miracle-of-death-for-lesser-languages-to-explore-ly y'rs - tim From ixokai at gmail.com Fri Aug 6 08:03:22 2004 From: ixokai at gmail.com (IxokaI) Date: Fri Aug 6 08:03:30 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <1f7befae04080522314d818d8f@mail.gmail.com> References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> <1f7befae04080522314d818d8f@mail.gmail.com> Message-ID: On Fri, 6 Aug 2004 01:31:57 -0400, Tim Peters wrote: > > PS2: I like the list-after-def syntax. > > There's quite a peculiar split over that. I find the @ gimmick most > readable of all, and especially because it forces one decoration per > physical line. list-after-def looks like a train wreck piling up > after the arglist half the time. But @ looks like a friendly little > womb, with a happy little birth canal, out of which is born a single > pure expression. What's more beautiful than the miracle of birth? > Certainly not a train wreck. Doesn't the fact that the @stuff is at the same level and has no connection to the def besides 'above' bother you in our universe of indentation-love? And the fact that the pie *is* "at", and why that it just doesn't "read"? '"define methy1 at staticmethod" is wrong, but "at staticmethod define methy2" is right. I like Python to be able to be read very easily. If we have to have it at the same level as def and above, we soo need a keyword..pweeze? class MyLittleToy: def methy1(self): print 'whee' @staticmethod def methy2(): print 'woo' @staticmethod def methy3(arg): for i in arg: print 'woohoo' ? That bothers me a looot... and is the primary reason I hate the pie. That and I really don't want anymore punctuation. --Stephen From martin at v.loewis.de Fri Aug 6 09:02:02 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 09:02:11 2004 Subject: [Python-Dev] trunk is no longer frozen In-Reply-To: <20040806040410.57C1E3C221@coffee.object-craft.com.au> References: <41112923.8060505@interlink.com.au> <4112CCDF.7030609@interlink.com.au> <4112E9AB.9060003@object-craft.com.au> <20040806040410.57C1E3C221@coffee.object-craft.com.au> Message-ID: <41132CEA.9000605@v.loewis.de> Andrew McNamara wrote: > In the name of bringing this thread up out of the noise: we need a > "go/no go" on applying this patch to the tree prior to a3 (the patch > adds "socketpair" to the socket module. Can I ask if the gods can > spend a moment reviewing it (it's well localised and includes test and > documentation updates)? Let me repeat my earlier response to this kind of request: If you want to "push" a patch into Python, the best way to get my attention is to review 10 patches, comment on them, and post the summary of the review to python-dev. If you do this often enough, you will become one of the gods yourself :-) That said, I think adding socketpair is a harmless extension. Regards, Martin From martin at v.loewis.de Fri Aug 6 09:26:34 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 09:26:43 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112DD1C.3060902@interlink.com.au> References: <4112DD1C.3060902@interlink.com.au> Message-ID: <411332AA.8000606@v.loewis.de> Anthony Baxter wrote: > I'm extremely averse to making the Python development > process any heavier than it has to be I think the original process (PEP1) goes like this: - PEP champion writes the PEP, starts discussion - PEP champion collects opinions, to avoid repeated discussion - BDFL pronounces - code gets written - code gets checked in So, in principle, the BDFL can only pronounce on what is in the PEP, however, that is not really important. What *is* really important is that there must be a PEP champion all the time. Once the PEP champion loses interest, or goes away, the process stops here and now - irrespective of any code that has been written, or pronouncements the BDFL has made. > Having said that, I don't think the lack of completed > PEP is a reason to back out the @ syntax from CVS. I do firmly think this is sufficient reason. It means there is no champion for the PEP, so there can't be a new feature. In essence, the champion is the one who "controls" the feature, and who is in charge of making it work well. He is the one to complain to, and he will make sure issues get resolved (not necessarily resolving them himself). I don't think the code needs to backed-out *now*; waiting for the next alpha would probably be enough. > There's also the issue that there's a bunch > of other existing PEPs describing features that aren't > up to date at all. This is the same problem. Nobody will complain if it works out all nicely, but we can forget about the PEP process if we are not willing to bow to the rules. If we don't like the rules, we should losen them, or perhaps drop the PEP process altogether. However, just ignoring it is not acceptable. > I don't > think we want to make the Wiki the primary copy of > the document, but I think having the PEPs annotatable > would be a win. The easiest way to achieve this is to embed a Wiki link in each PEP. Regards, Martin From martin at v.loewis.de Fri Aug 6 09:32:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 09:32:51 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112E602.4050009@pythonapocrypha.com> References: <4112DD1C.3060902@interlink.com.au> <4112E602.4050009@pythonapocrypha.com> Message-ID: <4113341E.4070502@v.loewis.de> Dave Brueck wrote: > (FWIW, it seems that a lot of the uproar about the syntax change is that > it is being interpreted as a permanent change to the language rather > than just experimental functionality. I _think_ I read here on > python-dev that it was semi-experimental, but if that's the case I'm not > sure how well that message got out - e.g. the "What's New in Python > 2.4a2" section of NEWS.html doesn't reflect this) Anthony's understanding is also mine: the feature is in the alpha so users can experiment with it, and accomodate with it. If the feature turns out to be completely unacceptable, it could be taken out. If users contribute something better, it can be replaced. However, users should not be under the impression that it magically changes or goes away without any action. Regards, Martin From gmccaughan at synaptics-uk.com Fri Aug 6 09:33:43 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Fri Aug 6 09:34:16 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz> References: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408060833.43982.gmccaughan@synaptics-uk.com> On Friday 2004-08-06 01:35, Greg Ewing wrote: > > >> class C(random.choice([dict, list])): > > >> pass > > >> > > >> is my favourite example of this :-) > > > > > > Where is THAT monstrocity used!? > > > > Well, it's not, I hope :-) > > Hmmm... maybe if you had some algorithm that was more efficient > using either a list or a dict, depending on the input data, but > in a way that makes it difficult to tell ahead of time which > to use, and you want to improve the average behaviour... ... then you'd do better to find which out of (dict,list) does better on average, and always use that one. Sorry. :-) -- g From lists at hlabs.spb.ru Fri Aug 6 13:42:39 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Fri Aug 6 09:37:00 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <4112A365.1000203@ocf.berkeley.edu> References: <4112A365.1000203@ocf.berkeley.edu> Message-ID: <41136EAF.7000908@hlabs.spb.ru> Brett Cannon wrote: > I am actually getting failures from test_timeout's testConnectTimeout > since the Net connection I have at school lets me connect to Google in > under the .001 connection timeout value. If I push it to .00000001 (a > hundred-millionth of a second) it then fails consistently. > > Now the problem is that the second part of the test uses this and a fuzz > value (currently at 2) to see if the test timed out within the proper > amount of time. The comparison is basically if the amount of time it > took to do the timed out failure is less than timeout + fuzz. So > lowering this number could possibly affect the test, although at .001 > seconds, I am doubting that will occur. But since these types of timing > tests can be touchy I thought I would double-check. > > So if anyone thinks it is bad to lower the value to .00000001 then > please let me know. Otherwise I will lower the value before the next > alpha goes out. > > -Brett > > P.S.: the only other failures I have on OS X right now is test_curses > (because I used -uall) and test__locale . Should we disable > test__locale on OS X to shut it up since Martin seems to think the test > isn't all that useful and won't work for OS X ever? Some time ago I submit patch for test_timeout.py: http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=728815 In my patch testConnectTimeout try connect to www.google.com:21 which seems like black hole. :) -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From martin at v.loewis.de Fri Aug 6 09:39:40 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 09:39:46 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <16658.63064.942060.30357@montanaro.dyndns.org> References: <4112DD1C.3060902@interlink.com.au> <4112E602.4050009@pythonapocrypha.com> <16658.63064.942060.30357@montanaro.dyndns.org> Message-ID: <411335BC.7010504@v.loewis.de> Skip Montanaro wrote: > I still think that finding a way to let users play with a couple syntax > variants would be a reasonable way to sort things out. I also disagree. People don't get the urgency of the need for feedback if they are left to guess which syntax will finally make it. If confronted with only a single choice, they focus on that syntax. If significant factual problems can be detected, the syntax can be discarded, and the reason for failure be recorded. In addition, a single approach gives focus for well-designed counter-proposals. If there are no flaws found and no "better" counter-proposals made, there is no reason to change the feature. Regards, Martin From pyth at devel.trillke.net Fri Aug 6 09:45:09 2004 From: pyth at devel.trillke.net (Holger Krekel) Date: Fri Aug 6 09:45:13 2004 Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in Unicode context) In-Reply-To: <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz> References: <20040805153443.GB5208@solar.trillke> <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz> Message-ID: <20040806074509.GD5208@solar.trillke> Greg Ewing wrote: > > My point is that I like to regard try/except as a mechanism for > > "out-of-band" objects. Guidos "should be shot" seems to indicate he > > sees try/except only useful/applicable to exception-handling. > > If the root exception class were called something else, > such as 'Raisable', would that make you feel better? Yes, I certainly wouldn't object. I guess this would mean 'Exception' would derive from Raisable because Exception itself should probably not go away. Hey, strings could inherit from Raisable, too! Just kidding :-) Then again, i think Python has a tradition of not requiring inheritance but just behaviour. And doesn't this whole issue only exist because "raise X" with X being a class and autoinstantiated is allowed? Well, anyway, let's not add too much to the current python-dev traffic with this issue. I think it has been brought up a couple of times already. Hey, i have an idea: why not create a python-syntax mailing list (or python-hell :-) ? cheers, Holger From mak at trisoft.com.pl Fri Aug 6 10:09:04 2004 From: mak at trisoft.com.pl (Grzegorz Makarewicz) Date: Fri Aug 6 10:09:06 2004 Subject: [Python-Dev] signed - unsigned - big troubles 2.4a2 Message-ID: <41133CA0.3060009@trisoft.com.pl> Many python programmas are relaying on computer int size - in prior versions bitnees is 32 long - now smoething happens and signed/unsigned value is not 32 bit long int but unsigned long int or even better unsigned uint64 consider that: a = ord(data[i]) .. data[i+4] - value is int or int64, signed or unsigned ? 0xf.. in prior versions up to 2.4.a2 int was used as signed value, now it is unsigned int64, so my question how to create signed int32 ? mak From anthony at interlink.com.au Fri Aug 6 10:11:12 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 10:11:26 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <411332AA.8000606@v.loewis.de> References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de> Message-ID: <41133D20.8040307@interlink.com.au> Martin v. L?wis wrote: > What *is* really important is that there must be a PEP > champion all the time. Once the PEP champion loses > interest, or goes away, the process stops here and now - > irrespective of any code that has been written, or > pronouncements the BDFL has made. This is a nice way to look at it. Unfortunately, we've not honoured this much. I'm happy to say that we should try and do this, but we have a significant issue with un-updated PEPs. >> Having said that, I don't think the lack of completed >> PEP is a reason to back out the @ syntax from CVS. > > > I do firmly think this is sufficient reason. It means > there is no champion for the PEP, so there can't be > a new feature. In essence, the champion is the one > who "controls" the feature, and who is in charge of > making it work well. He is the one to complain to, > and he will make sure issues get resolved (not necessarily > resolving them himself). A couple of people have commented that the int/long PEP, for instance, is not up-to-date. I don't think this means that the changes should be backed out. > I don't think the code needs to backed-out *now*; > waiting for the next alpha would probably be enough. Assuming the PEP can't be updated by then, sure. That then suggests (to me) that the process should be that the PEP should be updated to the state of the art before the next release that includes the new features or behavior (which was the first option in my original list). In many cases, the process goes more like: - PEP is written - code is checked in - issues occur, code is updated, but PEP is not updated to match it. In an extreme case, that's what's happened here - the basic goal of the PEP is still there, but the syntax is completely different, and a large chunk (class decorators) has been removed. I suspect part of the problem is the sheer amount of discussions on the topic, combined with decorator-fatigue on the part of the people who could update it. (In my case, it's mostly the former problem, combined with a lack of time). >> There's also the issue that there's a bunch >> of other existing PEPs describing features that aren't >> up to date at all. > > This is the same problem. Nobody will complain if it works > out all nicely, but we can forget about the PEP process > if we are not willing to bow to the rules. If we don't > like the rules, we should losen them, or perhaps drop > the PEP process altogether. However, just ignoring it is > not acceptable. Perhaps we need to collect a list of PEPs that are out-of-date, and push really hard to get them brought up to date. I'd like to see more use of the PEP status field in this way. -- Anthony Baxter It's never too late to have a happy childhood. From mak at trisoft.com.pl Fri Aug 6 10:33:49 2004 From: mak at trisoft.com.pl (Grzegorz Makarewicz) Date: Fri Aug 6 10:33:34 2004 Subject: [Python-Dev] signed - unsigned - big troubles 2.4a2 In-Reply-To: <41133CA0.3060009@trisoft.com.pl> References: <41133CA0.3060009@trisoft.com.pl> Message-ID: <4113426D.2030904@trisoft.com.pl> t = -2208988800.0 import time print time.localtime(t) another nasty situation > timestamp out of range for platform time_t mak From Paul.Moore at atosorigin.com Fri Aug 6 11:03:43 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Fri Aug 6 11:03:44 2004 Subject: [Python-Dev] Questions about '@' in pep 318 Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F4C@UKDCX001.uk.int.atosorigin.com> From: Edward K. Ream > The "mandate" for this post is the following from GvR: > >> If good arguments against @ are presented it may be removed in >> 2.4b1 (that was always the plan but apparently not publicized). > > So I would like seriously to consider whether removing '@' might > be a good idea. > > Note: my major concern with pep 318 has always been with syntax. > I am not interested or qualified to discuss whether annotation > in general is a good idea... One point here. To get the "@" syntax removed, you will (should) need to *either* convince everyone who dislikes the "@" syntax to back a single alternative syntax (experience shows that this is unlikely in the extreme) *or* argue that the whole decorator feature be removed. You can't do the latter without discussing semantics and "whether annotation in general is a good idea". Please do not mislead people into thinking that a discussion of syntax alone is of any value, *unless* there is a willingness on the part of those against the "@" syntax to compromise on a single alternative. Semantics is a different matter - I've not heard any strong arguments against the semantics of decorators. And process is a different matter again - but that's for a different posting (which I may well not make...) Paul. __________________________________________________________________________ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. __________________________________________________________________________ From marktrussell at btopenworld.com Fri Aug 6 11:22:15 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Fri Aug 6 11:22:21 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <59e9fd3a04080519462bf57899@mail.gmail.com> References: <59e9fd3a04080519462bf57899@mail.gmail.com> Message-ID: <1091784135.1770.4.camel@localhost> On Fri, 2004-08-06 at 03:46, Andrew Durdin wrote: > 8. When using multiple decorators, each appears on a separate line. No, you can put multiple decorators on a single line -- you just need a newline before the def. From test_decorators.py: class C(object): @funcattrs(abc=1) @staticmethod def foo(): return 42 Probably not good style (but neither is "if cond: stmt" I guess). Mark From marktrussell at btopenworld.com Fri Aug 6 11:22:17 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Fri Aug 6 11:22:23 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <59e9fd3a04080519462bf57899@mail.gmail.com> References: <59e9fd3a04080519462bf57899@mail.gmail.com> Message-ID: <1091784135.1770.6.camel@localhost> On Fri, 2004-08-06 at 03:46, Andrew Durdin wrote: > 8. When using multiple decorators, each appears on a separate line. No, you can put multiple decorators on a single line -- you just need a newline before the def. From test_decorators.py: class C(object): @funcattrs(abc=1) @staticmethod def foo(): return 42 Probably not good style (but neither is "if cond: stmt" I guess). Mark From martin at v.loewis.de Fri Aug 6 11:31:52 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 11:32:02 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <41133D20.8040307@interlink.com.au> References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de> <41133D20.8040307@interlink.com.au> Message-ID: <41135008.3070906@v.loewis.de> Anthony Baxter wrote: > This is a nice way to look at it. Unfortunately, we've > not honoured this much. I'm happy to say that we should > try and do this, but we have a significant issue with > un-updated PEPs. Yes. However, we can learn, and correct earlier errors. This doesn't all need to happen before 2.4, but we could start. For example, we could try listing issues with existing PEPs, and then ask the previous champions to deal with these issues. Responsiveness to such requests is an indication how responsible a contributor is. > A couple of people have commented that the int/long PEP, > for instance, is not up-to-date. I don't think this means > that the changes should be backed out. With Moshe Zadka and Guido van Rossum as champions. Looking at PEP 237, I fail to see why people say the PEP is not up-to-date. According to the PEP, we should be in stage B1, and AFAICT, we actually are in stage B1. The last change to the PEP is from December 2003. > In many cases, the process goes more like: > > - PEP is written > - code is checked in > - issues occur, code is updated, but PEP is not updated > to match it. This is IMO ok if the changes are "minor", where a minor change is one that doesn't deserve a PEP. Adding new functions or changing the precise conditions under which exceptions occur falls into this class. > In an extreme case, that's what's happened here - the basic > goal of the PEP is still there, but the syntax is completely > different, and a large chunk (class decorators) has been > removed. This is still not the process you described: no code had been checked in before. If the feature as described had been implemented before, then this change would have been a major change, and should not have been committed without a PEP of its own. > I suspect part of the problem is the sheer amount of > discussions on the topic, combined with decorator-fatigue on > the part of the people who could update it. (In my case, it's > mostly the former problem, combined with a lack of time). I'm not blaming anybody, and this is all understandable. We just have to find ways how to deal with the problem that nobody has time to really work on these things. My view is: it is sometimes better nothing gets done, rather than something being done poorly. Of course, this view only applies to new features - for bug fixes, it is better to do something that fixes the bug (like ripping the feature out) rather than doing nothing. > Perhaps we need to collect a list of PEPs that are out-of-date, > and push really hard to get them brought up to date. I'd like > to see more use of the PEP status field in this way. I don't think it is really that bad wrt. the current PEPs. Some of the issues I found are - PEP 263 states that Python 2.4 should give an error if no encoding declaration is specified. I propose to move this to 2.5. - PEP 322: we need to evaluate whether reversed is useless. - PEP 237: CSV is implemented, why is it still Draft, and open? - PEP 307: Likewise - PEP 331: Likewise (although I probably still owe a review) Regards, Martin From martin at v.loewis.de Fri Aug 6 11:37:26 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 11:37:31 2004 Subject: [Python-Dev] signed - unsigned - big troubles 2.4a2 In-Reply-To: <41133CA0.3060009@trisoft.com.pl> References: <41133CA0.3060009@trisoft.com.pl> Message-ID: <41135156.9050502@v.loewis.de> Grzegorz Makarewicz wrote: > Many python programmas are relaying on computer int size - in prior > versions bitnees is 32 long - now smoething happens and signed/unsigned > value is not 32 bit long int but unsigned long int or even better > unsigned uint64 > > consider that: > a = ord(data[i]) .. data[i+4] - value is int or int64, signed or > unsigned ? > > 0xf.. in prior versions up to 2.4.a2 int was used as signed value, now > it is unsigned int64, so my question how to create signed int32 ? No, it is not unsigned int64: Python 2.4a2 (#86, Aug 6 2004, 09:08:27) [GCC 3.3.4 (Debian 1:3.3.4-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff 28269553036454149273332760011886696253239742350009903329945699220681916415L >>> This number does not fit in 64 bits. Instead, Python integers are without bounds. See http://www.python.org/peps/pep-0237.html Regards, Martin From mal at egenix.com Fri Aug 6 11:37:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 6 11:37:53 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <41135008.3070906@v.loewis.de> References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de> <41133D20.8040307@interlink.com.au> <41135008.3070906@v.loewis.de> Message-ID: <4113516C.8050900@egenix.com> Martin v. L?wis wrote: > I don't think it is really that bad wrt. the current PEPs. > Some of the issues I found are > - PEP 263 states that Python 2.4 should give an error > if no encoding declaration is specified. I propose to > move this to 2.5. +1 People are just learning to use the source code encoding. Banging them on the head now with the SyntaxError would put them off, which is not really what the PEP intends to do. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mwh at python.net Fri Aug 6 12:16:10 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 12:16:16 2004 Subject: [Python-Dev] Re: Exceptional inheritance patterns In-Reply-To: <20040806074509.GD5208@solar.trillke> (Holger Krekel's message of "Fri, 6 Aug 2004 09:45:09 +0200") References: <20040805153443.GB5208@solar.trillke> <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz> <20040806074509.GD5208@solar.trillke> Message-ID: <2macx87es5.fsf@starship.python.net> Holger Krekel writes: > Greg Ewing wrote: >> > My point is that I like to regard try/except as a mechanism for >> > "out-of-band" objects. Guidos "should be shot" seems to indicate he >> > sees try/except only useful/applicable to exception-handling. >> >> If the root exception class were called something else, >> such as 'Raisable', would that make you feel better? > > Yes, I certainly wouldn't object. I guess this would mean 'Exception' > would derive from Raisable because Exception itself should probably not > go away. Hey, strings could inherit from Raisable, too! Just kidding :-) I would like an exception class that almost exceptions except KeyboardInterrupt, SystemExit and -- maybe -- RuntimeError and MemoryError inherited from. except ExceptionsButThoseNastyOnesIDontWantToCatch: pass ? > Then again, i think Python has a tradition of not requiring inheritance > but just behaviour. And doesn't this whole issue only exist because > "raise X" with X being a class and autoinstantiated is allowed? I would say that it's more because it's useful to organize exceptional conditions in a tree like hierarchy and inheritance is a usable way to do this. The fact that arranging things into tree like hierarchies *isn't* what inheritance is usually used for in Python (unlike many other languages) is what creates the dissonance, IMHO. > Well, anyway, let's not add too much to the current python-dev traffic > with this issue. I think it has been brought up a couple of times > already. Indeed, I think someone said what I just said above in this thread already :-) > Hey, i have an idea: why not create a python-syntax mailing list > (or python-hell :-) ? I think I've suggested that before :-) Today is the first day I remember where there are more new messages waiting for me in python-dev than comp.lang.python! Cheers, mwh -- "The future" has arrived but they forgot to update the docs. -- R. David Murray, 9 May 2000 From mcherm at mcherm.com Fri Aug 6 13:04:18 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Fri Aug 6 13:04:11 2004 Subject: [Python-Dev] Call for defense of @decorators Message-ID: <1091790258.411365b20c38a@mcherm.com> Gustavo Niemeyer writes: > Is there something to be done that could change the current > decision? Voting? Collecting pros/cons and classifying them? Yes, there is. And it's not voting . Write a simple "white paper" summarizing the major arguments and showing how they apply to the 3 or 4 top syntax proposals. Solicit input from c.l.py or python-dev after you have a first draft. After the 3rd draft is done, post it here and send it to Guido. Then sit back and quietly put up with the resulting decision whether you agree or not. At this point, I long ago said most everything I had to say and so I'm working on the "quietly put up with the results" part myself. -- Michael Chermside From mal at egenix.com Fri Aug 6 13:07:35 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 6 13:07:38 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <200408052324.i75NOxA05160@guido.python.org> References: <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> <000f01c47b40$cde05110$6700a8c0@computer> <4112BF9E.40003@v.loewis.de> <200408052324.i75NOxA05160@guido.python.org> Message-ID: <41136677.9090103@egenix.com> Guido van Rossum wrote: >>In essence, you are proposing to reserve a new keyword. Solutions >>creating new keywords have been ruled out because they might break >>existing code. > > > Perhaps this could be addressed by requiring "from __future__ import > decorators", for one release, just like was done for "yield". I > expect that this would be acceptable to the ObjC folks, too. It > wouldn't be my favorite, but I won't rule it out just because of the > new keyword (and yes, this is a softening of my position on new > keywords). At EuroPython 2004 I had the impression that folks were not really ready for a finalization on the feature. Your conclusion at the conference was to wait for another release before making a final decision which was well accepted by the audience. With all the fuzz around this issue, I think that was a wise decision and wonder why "something" now sneaked into 2.4a2. Not being a big fan of decorative art, I'm a firm -0 on whatever the final syntax may be, but just to throw in an idea from someone who has not participated in all the talk about decorator, here's some food for thought: """ 1. Instead of trying to overgeneralize, we only allow exactly *one* decorator per function definition. 2. Since decorators change the definition of a function, that one decorator call is placed directly after the "def" and before the function name. Examples: def staticcounted(func): return staticmethod(countcalls(func)) class MyClass: def classmethod test1(selfclass, x): """ Test method. """ print x def staticcounted test2(self, x): """ Test method. """ print x If someone needs more than one decorator or wants to do funky stuff, it's easy enough to define a combining generator for that particular need (see e.g. staticcounted). The only argument I heard at the conference against placing the decorator between the "def" and the function name was that of losing the "grep'ability" in this case. I don't consider that much of a loss, since it only applies to new code and cases where you actually make use of decorators. The simple nature of the syntax also guarantees that you can easily modify your grep expression to accomodate for possible decorators in front of the name (you'll only ever have one). """ If you don't like the idea, that's fine, but perhaps it spurs some new thoughts of how to make this whole decorator thing fit better the elegance of the language. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From edreamleo at charter.net Fri Aug 6 13:08:10 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 13:08:21 2004 Subject: [Python-Dev] Re: The impact of '@' on Leo References: <005e01c47b2d$9c0dee80$6700a8c0@computer> Message-ID: <003901c47ba5$a937e750$6700a8c0@computer> > I don't know what Leo is, but the usual solution to this sort of problem > is to introduce some kind of quoting convention, e.g. @@ gets passed > through as a single @ to Python, any other @something gets treated as a > Leo directive. Is there some reason that wouldn't work? Yes, in Leo's case there is a strong reason for not using an "escape" or "quoting" mechanism. As I said in my original post, not using an escape mechanism turns out to be the correct choice. The basic problem is that there are two (or is it three?) views of the data in the Leo world: the view presented to the user in the outline, the data in the .leo file, and the data in the derived files. Leo has a mechanism for automatically converting data in derived files into the one or two other forms. This is call "untangling". It turns out that it is extremely difficult to handle escapes properly when untangling. I know this sounds weird, and it's true nevertheless. If the @ syntax goes through, Leo will have to allow users to define their own lead-in string, and the format of derived files will have to change a bit to specify what the lead-in string is. It's not a huge deal, I think. It would surely be much easier than changing how Leo tangles or untangles code. BTW, I find the arguments for list-after-def to be persuasive, viz., that annotations should clearly be part of a function. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From richie at entrian.com Fri Aug 6 13:12:31 2004 From: richie at entrian.com (Richie Hindle) Date: Fri Aug 6 13:08:32 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: 20040805232522.A1B8E1E400A@bag.python.org Message-ID: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com> [Shalabh] > An alternative character (as others have suggested) > might be good too. If so, I found '|' particularly nice :) > > |paramnames(username='Username', password='Password') > |webmethod(username=str, password=str) > def login(self, username, password, return_to=None): > request = current.request > ... I much prefer this to @decorator. In addition to the advantages that Shalabh gives: + It doesn't break Leo, IPython, or any other tool that uses @ as a special character. + The association with pipes makes some sense: "take this thing and pass it through that thing to get a modified thing". Perhaps someone with write access to the Wiki page at http://www.python.org/moin/PythonDecorators could add this idea there? It would be a shame if it got lost in the high-volume traffic here on python-dev. Unless there's some overwhelming technical reason why reusing the '|' character isn't possible, of course. -- Richie Hindle richie@entrian.com From edreamleo at charter.net Fri Aug 6 13:29:35 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 13:29:43 2004 Subject: [Python-Dev] The impact of '@' on Leo References: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz> <200408052356.29328.fdrake@acm.org> Message-ID: <006b01c47ba8$a6bf8cf0$6700a8c0@computer> > I've really only managed to look at Leo much once, and Edward was sitting next > to me at the time. Hi Fred. Sitting next to you at the sprint was one of the most enlightening things I have done in a long time. I hope the next time we sprint together you'll be able to use Leo much more easily with cvs. I've spent that last six months on @thin, which should help a lot... > Leo doesn't store Python code in a Leo document as any > sort of embedded object that's labeled as Python code as far as I can tell > (though it may in it's data model); it just more text, and can have > @directives like any other text. I think this is inherently a risky approach > to the data; there needs to be at least a way to say that some portion of the > data is some other text format embedded into the Leo document. .leo files are XML files. The only thing that matters in that context is that the xml escapes are handled properly. What matters for the integrity of data in derived files is that uses don't mess with the so-called sentinel lines. These lines "carry" the outline structure. Destroy those lines, destroy something essential. Leo has a number of ways of recovering partially from such destruction, but the fact remains that uses insert, delete, fold or mutilate sentinel lines at their own risk. It's as simple as that. Sentinel lines start with #@ at present, and it's conceivable that they will always do so, even if @ gets used for something in Python. > I'm sure Edward can find a way to make things work for Leo users, but I hope > the .leo files have some sort of format version indicator to make upgrading > relatively easy for Leo users. Actually, both .leo files and derived files have various format indicators. However, the format of .leo files hasn't changed significantly enough to need a new number in years. Allowing different lead-in strings for Leo directives will have absolutely no effect on the format of .leo files. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From amk at amk.ca Fri Aug 6 13:34:26 2004 From: amk at amk.ca (A.M. Kuchling) Date: Fri Aug 6 13:34:41 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112B5B4.9@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de> Message-ID: <20040806113426.GA30824@rogue.amk.ca> On Fri, Aug 06, 2004 at 12:33:24AM +0200, "Martin v. L?wis" wrote: > it's just the PEP that needs updating (a section in whatsnew also needs > to be drafted). The current version of whatsnew already includes a section on decorators. That section, at http://www.python.org/dev/doc/devel/whatsnew/node5.html, may be useful as an overview of what's currently implemented in 2.4a2, given that the PEP isn't up-to-date. --amk From mwh at python.net Fri Aug 6 13:36:55 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 13:36:56 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112A746.6010501@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Thu, 05 Aug 2004 23:31:50 +0200") References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost> <4112A746.6010501@v.loewis.de> Message-ID: <2m657w7b1k.fsf@starship.python.net> "Martin v. L?wis" writes: > Barry Warsaw wrote: >> Martin makes a good point. Guido could threaten to remove the feature >> by beta 1 (and thus for 2.4 final) if the PEP is not brought up to date. >> not-that-i'm-volunteering-'cause-i'd-rather-see-it-done-ly y'rs, > > I'm atleast volunteering to take the code out if no proper > implementation (including documentation and test cases) is delivered by > some deadline. Uhh, there *are* test cases (test_decorators). I'm fairly sure there's at least ref manual documentation (in ref7.tex). Possibly there should be something elsewhere -- in the description of classmethod? Or are you talking in the abstract here? Cheers, mwh -- "The future" has arrived but they forgot to update the docs. -- R. David Murray, 9 May 2000 From mwh at python.net Fri Aug 6 13:40:23 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 13:40:25 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <4112A365.1000203@ocf.berkeley.edu> (Brett Cannon's message of "Thu, 05 Aug 2004 14:15:17 -0700") References: <4112A365.1000203@ocf.berkeley.edu> Message-ID: <2m1xik7avs.fsf@starship.python.net> Brett Cannon writes: > P.S.: the only other failures I have on OS X right now is test_curses > (because I used -uall) and test__locale . Should we disable > test__locale on OS X to shut it up since Martin seems to think the > test isn't all that useful and won't work for OS X ever? ONE of these days, I'll get around to doing something about test_curses... the test is being silly (or at least, that's what I remember from the last time I dug into this). I'm still seeing test_socket fail on OS X, though (Jaguar, testGetServByName). Cheers, mwh -- Ignoring the rules in the FAQ: 1" slice in spleen and prevention of immediate medical care. -- Mark C. Langston, asr From mwh at python.net Fri Aug 6 13:42:46 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 13:42:48 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <200408060833.43982.gmccaughan@synaptics-uk.com> (Gareth McCaughan's message of "Fri, 6 Aug 2004 08:33:43 +0100") References: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz> <200408060833.43982.gmccaughan@synaptics-uk.com> Message-ID: <2mwu0c5w7d.fsf@starship.python.net> Gareth McCaughan writes: > On Friday 2004-08-06 01:35, Greg Ewing wrote: >> > >> class C(random.choice([dict, list])): >> > >> pass >> > >> >> > >> is my favourite example of this :-) >> > > >> > > Where is THAT monstrocity used!? >> > >> > Well, it's not, I hope :-) >> >> Hmmm... maybe if you had some algorithm that was more efficient >> using either a list or a dict, depending on the input data, but >> in a way that makes it difficult to tell ahead of time which >> to use, and you want to improve the average behaviour... > > ... then you'd do better to find which out of (dict,list) > does better on average, and always use that one. Sorry. :-) By running the above code on a wide range of inputs? :-) Cheers, mwh -- To summarise the summary of the summary:- people are a problem. -- The Hitch-Hikers Guide to the Galaxy, Episode 12 From marktrussell at btopenworld.com Fri Aug 6 13:43:25 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Fri Aug 6 13:43:27 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <2m657w7b1k.fsf@starship.python.net> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost> <4112A746.6010501@v.loewis.de> <2m657w7b1k.fsf@starship.python.net> Message-ID: <1091792604.1770.8.camel@localhost> On Fri, 2004-08-06 at 12:36, Michael Hudson wrote: > Possibly > there should be something elsewhere -- in the description of > classmethod? That's already done - I updated the classmethod and staticmethod descriptions to use the new syntax, with a pointer to the ref manual desctiption. I think it is just the PEP that needs updating. Mark Russell From rnd at onego.ru Fri Aug 6 13:44:54 2004 From: rnd at onego.ru (Roman Suzi) Date: Fri Aug 6 13:45:06 2004 Subject: [Python-Dev] Questions about '@' in pep 318 Message-ID: Hello, I think @ before def are unacceptable because they aren't too Pythonic and thus break style and aestetics of Python. If we absolutely need those decorators, I propose to use existing construct which has some resemblance with other places of Python: def func(x, y, z) % decorator: ... and def func(x, y, z) % (decorator1, decorator2, ...): ... this has a meaning of "applying" decorators to the function (method). And it has minimal impact on readability. Sincerely yours, Roman A.Suzi -- - Petrozavodsk - Karelia - Russia - mailto:rnd@onego.ru - From martin at v.loewis.de Fri Aug 6 13:53:22 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 13:53:26 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <2m657w7b1k.fsf@starship.python.net> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost> <4112A746.6010501@v.loewis.de> <2m657w7b1k.fsf@starship.python.net> Message-ID: <41137132.8060506@v.loewis.de> Michael Hudson wrote: > Or are you talking in the abstract here? I was talking in a concrete, uninformed way. The tests should have been easy to find, although I had to grep to find the documentation in the language reference. Regards, Martin From martin at v.loewis.de Fri Aug 6 13:56:28 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 13:56:32 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: References: Message-ID: <411371EC.10403@v.loewis.de> Roman Suzi wrote: > I think @ before def are unacceptable because they aren't too Pythonic > and thus break style and aestetics of Python. This is not a convincing statement. Who is going to define what is Pythonic and what is not? My understanding of that term was that it can only refer to programs (e.g. a function could do something in a Pythonic way - namely, if that is how other Python practitioners would have written it). The language itself is by nature Pythonic - even if it would add curly braces. Of course, Pythonic version 2.4 differs from Pythonic version 1.4. Regards, Martin From gmccaughan at synaptics-uk.com Fri Aug 6 14:00:44 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Fri Aug 6 14:01:15 2004 Subject: [Python-Dev] 2.4a2, and @decorators In-Reply-To: <2mwu0c5w7d.fsf@starship.python.net> References: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz> <200408060833.43982.gmccaughan@synaptics-uk.com> <2mwu0c5w7d.fsf@starship.python.net> Message-ID: <200408061300.44475.gmccaughan@synaptics-uk.com> On Friday 2004-08-06 12:42, Michael Hudson wrote: >>>>>> class C(random.choice([dict, list])): >>>>>> pass >>>>>> >>>>>> is my favourite example of this :-) >>>>> >>>>> Where is THAT monstrocity used!? >>>> >>>> Well, it's not, I hope :-) >>> >>> Hmmm... maybe if you had some algorithm that was more efficient >>> using either a list or a dict, depending on the input data, but >>> in a way that makes it difficult to tell ahead of time which >>> to use, and you want to improve the average behaviour... >> >> ... then you'd do better to find which out of (dict,list) >> does better on average, and always use that one. Sorry. :-) > > By running the above code on a wide range of inputs? :-) No, you'd get more reliable results for a given expenditure of effort by running half as many input datasets with class C(dict) and class C(list). -- g From amk at amk.ca Fri Aug 6 14:04:05 2004 From: amk at amk.ca (A.M. Kuchling) Date: Fri Aug 6 14:04:24 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <2m1xik7avs.fsf@starship.python.net> References: <4112A365.1000203@ocf.berkeley.edu> <2m1xik7avs.fsf@starship.python.net> Message-ID: <20040806120405.GB30824@rogue.amk.ca> On Fri, Aug 06, 2004 at 12:40:23PM +0100, Michael Hudson wrote: > ONE of these days, I'll get around to doing something about > test_curses... the test is being silly (or at least, that's what I > remember from the last time I dug into this). What's test_curses doing wrong? --amk From rnd at onego.ru Fri Aug 6 14:13:30 2004 From: rnd at onego.ru (Roman Suzi) Date: Fri Aug 6 14:13:44 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <411371EC.10403@v.loewis.de> References: <411371EC.10403@v.loewis.de> Message-ID: By "pythonic" I mean "in accordance with other Python syntax". And I am not against '@', but rather against placing decorators before the def. IMHO, it is confusing, error-prone and out of style. Sincerely yours, Roman A.Suzi -- - Petrozavodsk - Karelia - Russia - mailto:rnd@onego.ru - On Fri, 6 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote: > Roman Suzi wrote: > > I think @ before def are unacceptable because they aren't too Pythonic > > and thus break style and aestetics of Python. > > This is not a convincing statement. Who is going to define what is > Pythonic and what is not? My understanding of that term was that it > can only refer to programs (e.g. a function could do something in > a Pythonic way - namely, if that is how other Python practitioners > would have written it). The language itself is by nature Pythonic - > even if it would add curly braces. Of course, Pythonic version 2.4 > differs from Pythonic version 1.4. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/rnd%40onego.ru > From della at linux.it Fri Aug 6 14:32:13 2004 From: della at linux.it (Matteo Dell'Amico) Date: Fri Aug 6 14:33:17 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <1091746327.1419.18.camel@localhost> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de> <1091746327.1419.18.camel@localhost> Message-ID: <1091795533.4501.6.camel@marvin.linux.it> Il ven, 2004-08-06 alle 00:52, Mark Russell ha scritto: > Good thought. Or even something like: > > |staticmethod| > def foo(): pass > > I just tried converting test_decorators.py to this syntax, and it looks > quite nice to my eye. I like it. Probably, I'd like even more: def foo(): pass I think these kind of proposal is extetically pleasing to my eye, has more or less all the pros of the list-before-def syntax, and wouldn't have its main drawback: the "magic" behaviour of evaluating a string, since is no python object. A drawback would be that no mainstream language uses this syntax, but I think the C# one is sufficiently similar. (and excuse me for proposing Just Another Syntax :-)) -- Ciao, Matteo From skip at pobox.com Fri Aug 6 14:45:42 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 14:46:04 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: References: <20040805202245.GH29796@burma.localdomain> Message-ID: <16659.32118.414524.314310@montanaro.dyndns.org> Christian> To all proponents of list markup: that would be very hard to Christian> look up in the documentation. `@` should be easy to find in Christian> the index. It would be easy to find if you search for "def" or "function"... This idea has been raised before and is a red herring in my opinion. Skip From della at linux.it Fri Aug 6 14:48:26 2004 From: della at linux.it (Matteo Dell'Amico) Date: Fri Aug 6 14:49:29 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <1091795533.4501.6.camel@marvin.linux.it> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de> <1091746327.1419.18.camel@localhost> <1091795533.4501.6.camel@marvin.linux.it> Message-ID: <1091796505.4501.8.camel@marvin.linux.it> Il ven, 2004-08-06 alle 14:32, Matteo Dell'Amico ha scritto: > Probably, I'd like even more: > > > def foo(): pass Please ignore me. I read the PEP, but I managed to forget it while reading python-dev. :-( -- Ciao, Matteo From john at neggie.net Fri Aug 6 14:54:45 2004 From: john at neggie.net (John Belmonte) Date: Fri Aug 6 14:54:54 2004 Subject: [Python-Dev] Would like to add Edward Loper as a Python developer In-Reply-To: <20040721211609.8FA333B8038@smtp.zope.com> References: <20040721211609.8FA333B8038@smtp.zope.com> Message-ID: <41137F95.4040606@neggie.net> Tim Peters wrote: > better support for writing "whole file" doctests (tutorial narratives > liberally illustrated with Python examples -- Zope3 is finding these very > helpful); and documenting all the additions that snuck into Python 2.3. I noticed that the "whole file doctests" item didn't make it to the DoctestIdeas page. I was looking for more details about that. I've found that for tutorial narratives, I really don't want to bother with expected output. For complex software, the output may not always be the same anyway. Also, it's not fun to manipulate input code that has prompts and is within a multi-line string. Syntax highlighting is not available, for one thing. I've made my own tutorial generator. Given this input script: ## Double hash comments treated as meta comments and not output to ## the tutorial. """ Triple-quote comments that start in the first column are treated as tutorial text. First we'll import some modules: """ import os """ Some code follows. In addition to hash comments, and triple-quote comments not starting in the first column are preserved. Note that code blocks such as functions must end with a blank line, just as is done in the interactive interpreter. """ def foo(): """foo description here""" print 'hello' # be friendly #my_path = 'foo/bar' my_path = os.path.join('foo', 'bar') my_path foo() the output is: Triple-quote comments that start in the first column are treated as tutorial text. First we'll import some modules: >>> import os Some code follows. In addition to hash comments, and triple-quote comments not starting in the first column are preserved. Note that code blocks such as functions must end with a blank line, just as is done in the interactive interpreter. >>> def foo(): ... """foo description here""" ... print 'hello' # be friendly ... >>> #my_path = 'foo/bar' >>> my_path = os.path.join('foo', 'bar') >>> my_path 'foo/bar' >>> foo() hello -John -- http://giftfile.org/ :: giftfile project From skip at pobox.com Fri Aug 6 14:55:52 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 14:56:30 2004 Subject: [Python-Dev] Re: Exceptional inheritance patterns In-Reply-To: <2macx87es5.fsf@starship.python.net> References: <20040805153443.GB5208@solar.trillke> <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz> <20040806074509.GD5208@solar.trillke> <2macx87es5.fsf@starship.python.net> Message-ID: <16659.32728.411134.439644@montanaro.dyndns.org> Michael> I would like an exception class that almost exceptions except Michael> KeyboardInterrupt, SystemExit and -- maybe -- RuntimeError and Michael> MemoryError inherited from. Michael> except ExceptionsButThoseNastyOnesIDontWantToCatch: Michael> pass Michael> ? I proposed a change to the exceptions hierarchy a few years ago that would allow this. It was obviously never implemented, but I no longer remember why. Skip From skip at pobox.com Fri Aug 6 15:00:58 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 15:01:16 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com> References: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com> Message-ID: <16659.33034.4995.438267@montanaro.dyndns.org> Richie> Perhaps someone with write access to the Wiki page at Richie> http://www.python.org/moin/PythonDecorators could add this idea Richie> there? Anyone can scribble on the Wiki. You just have to be logged in. Poke the UserPreferences link to sign up. If you had previously been registered it appears all login info was lost with the latest Moin upgrade on www.python.org. Skip From skip at pobox.com Fri Aug 6 15:18:33 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 15:18:49 2004 Subject: [Python-Dev] PEP 318 - writing to func_name attribute Message-ID: <16659.34089.645670.39263@montanaro.dyndns.org> One thing I added awhile back to PEP 318 as an open issue is that for ease of writing wrappers the func_name attribute of a function object should be writable. For example, in situations where the decorator returns a new function object (often, I would think) it would really be nice if that new function object had the same name as the undecorated function. Consider: def a(func): def _inner(*args, **kwds): return func(*args, **kwds) return _inner @a def func(*args, **kwds): print args print kwds print "func's name:", func.func_name I realize you can use new.function() to create a new function object, but that seems like a fair amount of extra complexity just to change the function's name. I'd prefer it if the decorator could be written as: def a(func): def _inner(*args, **kwds): return func(*args, **kwds) _inner.func_name = func.func_name return _inner That fails because func_name is readonly. Any chance this restriction can be loosened up? Skip From skip at pobox.com Fri Aug 6 15:21:02 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 15:21:20 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <1091792604.1770.8.camel@localhost> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost> <4112A746.6010501@v.loewis.de> <2m657w7b1k.fsf@starship.python.net> <1091792604.1770.8.camel@localhost> Message-ID: <16659.34238.443364.229226@montanaro.dyndns.org> Mark> I think it is just the PEP that needs updating. Not to pick on Mark, but everyone seems to think the PEP needs updating but nobody's stepping up to offer patches. Anthony and I are both about out of round tuits... Skip From mwh at python.net Fri Aug 6 15:39:25 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 15:39:26 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com> (Phillip J. Eby's message of "Thu, 05 Aug 2004 15:45:35 -0400") References: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com> Message-ID: <2msmb05qsy.fsf@starship.python.net> "Phillip J. Eby" writes: > At 12:24 PM 8/5/04 -0700, IxokaI wrote: > >>I added "with", although I havn't seen it. > > Guido's reserving "with" for this purpose in some future Python: > > with x.y: > .z = spam # set x.y.z = spam > print .q.r # print x.y.q.r Except that the only extant PEP involving with actually uses it for something else :-) I think talking about what Guido is or isn't doing is a bit ... wrong? Cheers, mwh -- 41. Some programming languages manage to absorb change, but withstand progress. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From barbidule at melshake.com Fri Aug 6 15:29:27 2004 From: barbidule at melshake.com (maxime B) Date: Fri Aug 6 15:40:10 2004 Subject: [Python-Dev] Re: Tuple/list assignment question References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C000@au3010avexu1.global.avaya.com> Message-ID: Delaney, Timothy C (Timothy avaya.com> writes: > I think it's about time this proposal got a PEP of its own ... something > like "Enhanced Tuple Unpacking" (to match "Enhanced Argument Tuples" ... Even the PEP would be rejected, I think it's a good idea to have the definitive answer of the BDFL. Maxime From tanzer at swing.co.at Fri Aug 6 15:42:50 2004 From: tanzer at swing.co.at (Christian Tanzer) Date: Fri Aug 6 15:43:45 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: Your message of "Fri, 06 Aug 2004 07:45:42 CDT." <16659.32118.414524.314310@montanaro.dyndns.org> Message-ID: > Christian> To all proponents of list markup: that would be very hard to > Christian> look up in the documentation. `@` should be easy to find in > Christian> the index. > > It would be easy to find if you search for "def" or "function"... This idea > has been raised before and is a red herring in my opinion. I know about looking at the documentation of `def`. But you are assuming here that the puzzled reader associates the (leading) list markup with the function definition. The trailing list markup (that I personally favored) hasn't been in contention since May? June? Anyway, the pie syntax offers more index entries than the list syntax. Tuning-out-for-good-now-ly yrs, -- Christian Tanzer http://www.c-tanzer.at/ From mwh at python.net Fri Aug 6 15:44:07 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 15:44:09 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <20040806120405.GB30824@rogue.amk.ca> (A. M. Kuchling's message of "Fri, 6 Aug 2004 08:04:05 -0400") References: <4112A365.1000203@ocf.berkeley.edu> <2m1xik7avs.fsf@starship.python.net> <20040806120405.GB30824@rogue.amk.ca> Message-ID: <2moelo5ql4.fsf@starship.python.net> "A.M. Kuchling" writes: > On Fri, Aug 06, 2004 at 12:40:23PM +0100, Michael Hudson wrote: >> ONE of these days, I'll get around to doing something about >> test_curses... the test is being silly (or at least, that's what I >> remember from the last time I dug into this). > > What's test_curses doing wrong? I think the problem is that it sometimes calls "curses.curs_set(1)" which doesn't work (returns ERR) when certain definitions (cuvis, I guess) aren't in the terminfo database, which gets translated into a Python exception. Cheers, mwh -- All parts should go together without forcing. You must remember that the parts you are reassembling were disassembled by you. Therefore, if you can't get them together again, there must be a reason. By all means, do not use a hammer. -- IBM maintenance manual, 1925 From richie at entrian.com Fri Aug 6 15:52:37 2004 From: richie at entrian.com (Richie Hindle) Date: Fri Aug 6 15:48:37 2004 Subject: [Python-Dev] Re: Call for defense of @decorators Message-ID: [Skip] > Anyone can scribble on the Wiki. You just have to be logged in. Poke the > UserPreferences link to sign up Thanks, Skip (and amk, who emailed me the same info). The page said it was immutable, and I didn't realise I could change that by logging in. I've added Shalabh's "|decorator" syntax to the wiki page at http://www.python.org/moin/PythonDecorators -- Richie Hindle richie@entrian.com From edreamleo at charter.net Fri Aug 6 15:50:08 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 15:50:17 2004 Subject: [Python-Dev] Questions about '@' in pep 318 References: <411371EC.10403@v.loewis.de> Message-ID: <007d01c47bbc$49a2a340$6700a8c0@computer> > > I think @ before def are unacceptable because they aren't too Pythonic > > and thus break style and aesthetics of Python. > > This is not a convincing statement. I agree. The argument that @ lines aren't clearly "attached" to functions seems much more convincing to me. Surely I am repeating somebody else's points, but I'm going to say them anyway: 1. There doesn't seem much debate that dropping @ from the list of symbols that can never appear in Python programs (outside of comments and strings) is, in fact, a major change to the language. 2. Maybe the @ syntax is somehow the best alternative, considered in isolation. Fair enough. But how much better? Enough to justify a major change to the language? What is so unreadable about def (...)[...]: ? 3. I agree with Phillip Eby that an @ line looks like some external document markup. This is, if fact, a common usage for the @ sign: Leo uses the @ sign for this reason. But to be fair, this is a very minor point. 4. No doubt Python programmers will grow used to any syntax whatever it is, including @ or def (...)[...]. Whatever slight discomfort people will feel with a new syntax will quickly fade. In short, a major change to Python (@) would seem to demand a truly convincing justification. IMO, the argument that @ is more readable than def (...)[...]: doesn't even come _close_ to such a justification. YMMV. And only GvR's mileage counts :-) Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From mwh at python.net Fri Aug 6 16:15:05 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 16:15:08 2004 Subject: [Python-Dev] OK to lower timeout for test_timeout's testConnectTimeout test? In-Reply-To: <2moelo5ql4.fsf@starship.python.net> (Michael Hudson's message of "Fri, 06 Aug 2004 14:44:07 +0100") References: <4112A365.1000203@ocf.berkeley.edu> <2m1xik7avs.fsf@starship.python.net> <20040806120405.GB30824@rogue.amk.ca> <2moelo5ql4.fsf@starship.python.net> Message-ID: <2mk6wc5p5i.fsf@starship.python.net> Michael Hudson writes: > "A.M. Kuchling" writes: > >> On Fri, Aug 06, 2004 at 12:40:23PM +0100, Michael Hudson wrote: >>> ONE of these days, I'll get around to doing something about >>> test_curses... the test is being silly (or at least, that's what I >>> remember from the last time I dug into this). >> >> What's test_curses doing wrong? > > I think the problem is that it sometimes calls "curses.curs_set(1)" > which doesn't work Argh; it *always* calls curses.curs_set(1) which *sometimes* doesn't work... > (returns ERR) when certain definitions (cuvis, I guess) aren't in > the terminfo database, which gets translated into a Python > exception. > -- About the use of language: it is impossible to sharpen a pencil with a blunt axe. It is equally vain to try to do it with ten blunt axes instead. -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time. From mwh at python.net Fri Aug 6 16:20:52 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 16:20:55 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <003601c47b34$0b4bf660$6700a8c0@computer> (Edward K. Ream's message of "Thu, 5 Aug 2004 16:34:52 -0500") References: <004a01c47b2b$65d19260$6700a8c0@computer> <20040805205559.GA31288@burma.localdomain> <003601c47b34$0b4bf660$6700a8c0@computer> Message-ID: <2mfz705ovv.fsf@starship.python.net> "Edward K. Ream" writes: >> > annotate.accepts(int, (int,float)) >> >> I don't understand. How are you changing/annotating the function? > > I'm glad you asked. My suggestion was that the compiler turn > > annotate.accepts(int, (int,float)) > > into exactly what it would have turned > > @accepts(int, (int,float)) > > into. I don't like this sort of idea: something that's done by the compiler should *look* like it is being done by the compiler, i.e. be new syntax. Cheers, mwh -- GAG: I think this is perfectly normal behaviour for a Vogon. ... VOGON: That is exactly what you always say. GAG: Well, I think that is probably perfectly normal behaviour for a psychiatrist. -- The Hitch-Hikers Guide to the Galaxy, Episode 9 From guido at python.org Fri Aug 6 16:22:55 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 16:23:01 2004 Subject: [Python-Dev] PEP 237 In-Reply-To: Your message of "Fri, 06 Aug 2004 18:11:12 +1000." <41133D20.8040307@interlink.com.au> References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de> <41133D20.8040307@interlink.com.au> Message-ID: <200408061422.i76EMu406927@guido.python.org> > A couple of people have commented that the int/long PEP, > for instance, is not up-to-date. I don't think this means > that the changes should be backed out. I've seen this particular PEP brought up a couple of times, but I don't see any significant inaccuracies in that PEP (except for some uses of "currently" referring to Python versions before 2.2). I've tried my best to keep the implementation sections up to date as our timeline and semantics changed. So please find another example. (And if the *only* reason to back out @decorators ever becomes the lack of a PEP champion, I'll champion it myself. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From edreamleo at charter.net Fri Aug 6 16:16:52 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 16:28:12 2004 Subject: [Python-Dev] PEP 318, and the PEP process References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de> Message-ID: <008b01c47bc0$0550dfa0$6700a8c0@computer> > I do firmly think this is sufficient reason. It means > there is no champion for the PEP, so there can't be > a new feature. In essence, the champion is the one > who "controls" the feature, and who is in charge of > making it work well. He is the one to complain to, > and he will make sure issues get resolved (not necessarily > resolving them himself). Python has matured into a language in which many people and companies have significant investments. Changes to Python matter. In this environment, making haste slowly makes more and more sense. In particular, not having pep's reflect reality is likely to waste a lot of time. It also just doesn't seem like proper procedure. This is a highly sensitive topic. I have the highest regard for Python's volunteers, and I have no intention of even suggesting what they should or shouldn't do. It is for GvR to pronounce, if he pleases. The present discussion of pep 318 shows the pitfalls in not treating Python as the supremely important language that it clearly is. I served, very briefly, on the ANSI C Standards Committee about 10 years ago. Clearly, Python needn't follow the extremely picky rules for public disclosure and public comment that are de rigueur in the Standards world. However, it might help to start shading just a bit in that direction, if only to head off the kind of public outcry we have had with 318. As I said on c.l.py, it does not seem fair to claim to be discussing a proposal publicly if it's pep does not reflect reality. The reason is simple: people like me will ignore discussion on python-dev if the pep does not seem to concern them. It's like announcing a meeting to discuss sewers, then using that (public) meeting to give all the county commissioners a raise. The open meeting law has been violated. Edward P.S. To respond to the original quote :-) My own opinion is the lack of a champion is more than sufficient reason to delay or kill a feature (or a pep). To repeat: changes to Python matter. If there isn't sufficient will to keep peps up-to-date, how can we be assured of the quality of the design, or of the code? Again, I have no right to demand that this criterion go into effect. EKR -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From mwh at python.net Fri Aug 6 16:36:11 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 16:36:13 2004 Subject: [Python-Dev] pre-PEP: Complete, Structured Regular Expression Group Matching In-Reply-To: (Mike Coleman's message of "05 Aug 2004 22:46:51 -0400") References: Message-ID: <2mbrho5o6c.fsf@starship.python.net> Mike Coleman writes: > If you've ever been frustrated that there wasn't an easy way to parse a string > or file, even though you could easily match it with re.match, then this PEP > may be for you. If what's being proposed seems a little ghastly, well, I'll > just say that I've been wanting something like this for a long time and this > is the best I could come up with. Your comments invited. Generally, it strikes me as mildly useful. An implementation would surely help :-) Presumably 'structmatch' would also become a metho on pattern objects? > Additional Features > ------------------- > > In many cases in which ``'re.structmatch'`` fails to match, the cause will > be due to an unexpected error in the format of the string being > matched. In order to assist the calling program in generating a more > meaningful possible error message, ``'re.structmatch'`` will return the > endpoint of the largest match against the searched string. So, for > example :: > > re.structmatch('abcd', 'abxxx') --> 2 > re.structmatch('abcde|z', 'abxxx') --> 2 > re.structmatch('x*?y', 'xxx') --> 3 > > (This return value should be considered advisory rather than exact, as > future improvements in the match algorithm may make it difficult to > calculate the exact value.) I don't really like this idea; it seems to be that it would be more helpful to raise an exception and attach this data to the exception. But that's a bit inconsisent with how match and search work. Cheers, mwh -- I have a cat, so I know that when she digs her very sharp claws into my chest or stomach it's really a sign of affection, but I don't see any reason for programming languages to show affection with pain. -- Erik Naggum, comp.lang.lisp From nhodgson at bigpond.net.au Fri Aug 6 16:44:58 2004 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Fri Aug 6 16:45:06 2004 Subject: [Python-Dev] Questions about '@' in pep 318 References: <411371EC.10403@v.loewis.de> <007d01c47bbc$49a2a340$6700a8c0@computer> Message-ID: <079701c47bc3$f2146ac0$034b8890@neil> Edward K. Ream: > 1. There doesn't seem much debate that dropping @ from the list of symbols > that can never appear in Python programs (outside of comments and strings) > is, in fact, a major change to the language. I do not believe using these symbols is a major change to the language. To me '@', '$' and '?' are currently unused symbols that may be used when there is a good reason to use them. In the past, there has been discussion about using '?' for inline if, '$' for string interpolation, and '@' for extended operators in IIRC matrix operations. Neil From edreamleo at charter.net Fri Aug 6 16:44:47 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 16:45:22 2004 Subject: [Python-Dev] Questions about '@' in pep 318 Message-ID: <016c01c47bc3$ec1f7420$6700a8c0@computer> > Please do not mislead people into thinking that a discussion of syntax alone is of any value, *unless* there is a willingness on the part of those against the "@" syntax to compromise on a single alternative. This statement seems a bit too strong, for the following reasons: 1. Adding @ impacts major external tools, negatively. 2. Adding @ is, in fact, a major change to the language. 3. The "willingness to compromise" is not really the driving force here: GvR is the only vote that matters. I have stated that Leo can probably live with the at sign. I have also stated elsewhere my reasons for not being particularly happy with it. For the record, I am +0 on the semantics. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From guido at python.org Fri Aug 6 16:45:51 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 16:45:57 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: Your message of "Fri, 06 Aug 2004 20:58:27 +1000." <41136453.1040908@interlink.com.au> References: <41136453.1040908@interlink.com.au> Message-ID: <200408061445.i76EjqT07026@guido.python.org> [Anthony] > I'm having trouble finding a single piece from you where you > outline the problems you have with the [decorators] after args, > before :. Could you possibly post a message to python-dev > stating your problems with it? It's the major one I've been > unable to find your objections to. And nobody else has expressed their objections to it either? :-) Just look at it when either the list of decorators or the argument list doesn't fit on one line: class C(object): def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42) [ staticmethod, funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") ]: """This method blah, blah. It supports the following arguments: - longArgumentOne -- a string giving ... - longArgumentTwo -- a number giving ... blah, blah. """ raise NotYetImplemented That's a total jumble of stuff ending with a smiley. (True story: I left out the colon when typing up this example and only noticed in proofing.) Problems with this form: - it hides crucial information (e.g. that it is a static method) after the signature, where it is easily missed - it's easy to miss the transition between a long argument list and a long decorator list - it's cumbersome to cut and paste a decorator list for reuse, because it starts and ends in the middle of a line Given that the whole point of adding decorator syntax is to move the decorator from the end ("foo = staticmethod(foo)" after a 100-line body) to the front, where it is more in-your-face, it should IMO be moved all the way to the front. (If I could design a language from scratch, I might want to move docstrings to before the 'def' as well; I've come to appreciate this in Java code. Fredrik Lundh's PythonDoc also uses comments prefixing the method definition; he also uses @ so he should be happy with this syntax, as he seemed to be when I mentioned it to him at EuroPython.) --Guido van Rossum (home page: http://www.python.org/~guido/) From jhylton at gmail.com Fri Aug 6 16:52:15 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Aug 6 16:52:17 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408061445.i76EjqT07026@guido.python.org> References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> Message-ID: On Fri, 06 Aug 2004 07:45:51 -0700, Guido van Rossum wrote: > [Anthony] > > I'm having trouble finding a single piece from you where you > > outline the problems you have with the [decorators] after args, > > before :. Could you possibly post a message to python-dev > > stating your problems with it? It's the major one I've been > > unable to find your objections to. > > And nobody else has expressed their objections to it either? :-) I thought it was obvious that this option was not acceptable. > (If I could design a language from scratch, I might want to move > docstrings to before the 'def' as well; I've come to appreciate this > in Java code. Fredrik Lundh's PythonDoc also uses comments prefixing > the method definition; he also uses @ so he should be happy with this > syntax, as he seemed to be when I mentioned it to him at EuroPython.) It's not at all clear that Fredrik likes it, but he doesn't participate in Python development any more so we may never know :-(. >From his blog: > ...so someone I've never heard of added a major syntactic feature to Python 2.4, > using a syntax that Guido doesn't really like. sorry, guys, but I liked Python better > when the core language was controlled by a single designer with a strong intuition. > the current design-by-arguing-until-nobody-cares-anymore-committee just don't > understand what made Python's design so damn good (as the "[new syntax] is no > worse than [existing operator] meaning [something when used with a given type]" > comment clearly shows). sigh. good thing nobody can force me to use decorators (or > update to 2.4, for that matter). time to finish my own interpreter, perhaps. or at least > start using a fresh one. Jeremy From guido at python.org Fri Aug 6 16:52:39 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 16:52:45 2004 Subject: [Python-Dev] Re: Exceptional inheritance patterns In-Reply-To: Your message of "Fri, 06 Aug 2004 07:55:52 CDT." <16659.32728.411134.439644@montanaro.dyndns.org> References: <20040805153443.GB5208@solar.trillke> <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz> <20040806074509.GD5208@solar.trillke> <2macx87es5.fsf@starship.python.net> <16659.32728.411134.439644@montanaro.dyndns.org> Message-ID: <200408061452.i76Eqd207077@guido.python.org> > Michael> I would like an exception class that almost exceptions except > Michael> KeyboardInterrupt, SystemExit and -- maybe -- RuntimeError and > Michael> MemoryError inherited from. > > Michael> except ExceptionsButThoseNastyOnesIDontWantToCatch: > Michael> pass > > Michael> ? > > I proposed a change to the exceptions hierarchy a few years ago that would > allow this. It was obviously never implemented, but I no longer remember > why. > > Skip Probably inertia and compatibility issues (2.2 must've been brand new when you proposed that). I've become swayed by Paul Prescod's recent argument that exceptions use subclassing for classification reasons and that there's nothing wrong with *requiring* them to subclass some common base class. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 6 16:53:51 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 16:53:55 2004 Subject: [Python-Dev] PEP 318 - writing to func_name attribute In-Reply-To: Your message of "Fri, 06 Aug 2004 08:18:33 CDT." <16659.34089.645670.39263@montanaro.dyndns.org> References: <16659.34089.645670.39263@montanaro.dyndns.org> Message-ID: <200408061453.i76Erpx07092@guido.python.org> > One thing I added awhile back to PEP 318 as an open issue is that for ease > of writing wrappers the func_name attribute of a function object should be > writable. For example, in situations where the decorator returns a new > function object (often, I would think) it would really be nice if that new > function object had the same name as the undecorated function. Consider: > > def a(func): > def _inner(*args, **kwds): > return func(*args, **kwds) > return _inner > > @a > def func(*args, **kwds): > print args > print kwds > > print "func's name:", func.func_name > > I realize you can use new.function() to create a new function object, but > that seems like a fair amount of extra complexity just to change the > function's name. I'd prefer it if the decorator could be written as: > > def a(func): > def _inner(*args, **kwds): > return func(*args, **kwds) > _inner.func_name = func.func_name > return _inner > > That fails because func_name is readonly. Any chance this restriction can > be loosened up? Good idea, independent from decorators; please add it to the PEP! --Guido van Rossum (home page: http://www.python.org/~guido/) From krunkle at gmail.com Fri Aug 6 16:54:50 2004 From: krunkle at gmail.com (Kendall Clark) Date: Fri Aug 6 16:56:25 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz> References: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz> Message-ID: On Fri, 06 Aug 2004 14:35:08 +1200, Greg Ewing wrote: > "Phillip J. Eby" : > > > Now, I don't know if my thought process here is at all typical or > > representative of anything, other than my own idiosyncratic self. > > You're not alone. I agree with *everything* you said there. +1 Philip's is the best argument I've heard so far. I favor list-after-def over @ for decorator syntax -- FWIW. List after def seems very unambiguously some kind of modifier of a function or method. Kendall Grant Clark Managing Editor, XML.com From krunkle at gmail.com Fri Aug 6 16:56:23 2004 From: krunkle at gmail.com (Kendall Clark) Date: Fri Aug 6 16:56:27 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz> References: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz> Message-ID: On Fri, 06 Aug 2004 14:35:08 +1200, Greg Ewing wrote: > "Phillip J. Eby" : > > > Now, I don't know if my thought process here is at all typical or > > representative of anything, other than my own idiosyncratic self. > > You're not alone. I agree with *everything* you said there. +1 Eby's is the best argument I've heard so far. I favor list-after-def over @ for decorator syntax -- FWIW. List after def seems very unambiguously some kind of modifier of a function or method. Kendall Grant Clark Managing Editor, XML.com From guido at python.org Fri Aug 6 16:56:53 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 16:56:59 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: Your message of "Fri, 06 Aug 2004 14:39:25 BST." <2msmb05qsy.fsf@starship.python.net> References: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com> <2msmb05qsy.fsf@starship.python.net> Message-ID: <200408061456.i76EurX07119@guido.python.org> > >>I added "with", although I havn't seen it. > > > > Guido's reserving "with" for this purpose in some future Python: > > > > with x.y: > > .z = spam # set x.y.z = spam > > print .q.r # print x.y.q.r > > Except that the only extant PEP involving with actually uses it for > something else :-) And I wish that PEP would propose a different name. (In fact, the fact that 'with' is slated for a different use should be added to it.) > I think talking about what Guido is or isn't doing is a bit > ... wrong? Yes if it's speculation (like what I would consider "pythonic"). In this case, I have repeatedly stated exactly what is quoted above as my preferred use for 'with' in Python 3.0. --Guido van Rossum (home page: http://www.python.org/~guido/) From krunkle at gmail.com Fri Aug 6 16:54:50 2004 From: krunkle at gmail.com (Kendall Clark) Date: Fri Aug 6 16:57:34 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz> References: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz> Message-ID: On Fri, 06 Aug 2004 14:35:08 +1200, Greg Ewing wrote: > "Phillip J. Eby" : > > > Now, I don't know if my thought process here is at all typical or > > representative of anything, other than my own idiosyncratic self. > > You're not alone. I agree with *everything* you said there. +1 Philip's is the best argument I've heard so far. I favor list-after-def over @ for decorator syntax -- FWIW. List after def seems very unambiguously some kind of modifier of a function or method. Kendall Grant Clark Managing Editor, XML.com From skip at pobox.com Fri Aug 6 16:56:08 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 16:58:37 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: References: <16659.32118.414524.314310@montanaro.dyndns.org> Message-ID: <16659.39944.597362.102214@montanaro.dyndns.org> >> It would be easy to find if you search for "def" or "function"... >> This idea has been raised before and is a red herring in my opinion. Christian> I know about looking at the documentation of `def`. But you Christian> are assuming here that the puzzled reader associates the Christian> (leading) list markup with the function definition. Ah, my apologies. I had list-after-def decorators in mind when I replied. Christian> The trailing list markup (that I personally favored) hasn't Christian> been in contention since May? June? Until a couple days ago I don't think anything regarding PEP 318 had been discussed since June on c.l.py or python-dev. Skip From pje at telecommunity.com Fri Aug 6 17:06:20 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 17:02:12 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com> References: <20040805232522.A1B8E1E400A@bag.python.org> Message-ID: <5.1.1.6.0.20040806110536.03583e40@mail.telecommunity.com> At 12:12 PM 8/6/04 +0100, Richie Hindle wrote: >Perhaps someone with write access to the Wiki page at >http://www.python.org/moin/PythonDecorators could add this idea there? FYI, all you need to do to get write access is "log in" by creating an account - Click on UserPreferences. From mwh at python.net Fri Aug 6 17:04:19 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 17:04:20 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <200408061456.i76EurX07119@guido.python.org> (Guido van Rossum's message of "Fri, 06 Aug 2004 07:56:53 -0700") References: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com> <2msmb05qsy.fsf@starship.python.net> <200408061456.i76EurX07119@guido.python.org> Message-ID: <2m7jsc5mvg.fsf@starship.python.net> Guido van Rossum writes: >> >>I added "with", although I havn't seen it. >> > >> > Guido's reserving "with" for this purpose in some future Python: >> > >> > with x.y: >> > .z = spam # set x.y.z = spam >> > print .q.r # print x.y.q.r >> >> Except that the only extant PEP involving with actually uses it for >> something else :-) > > And I wish that PEP would propose a different name. (In fact, the > fact that 'with' is slated for a different use should be added to it.) Noted. I'll do something about it eventually... >> I think talking about what Guido is or isn't doing is a bit >> ... wrong? > > Yes if it's speculation (like what I would consider "pythonic"). In > this case, I have repeatedly stated exactly what is quoted above as my > preferred use for 'with' in Python 3.0. Somehow I'd missed that. Cheers, mwh -- Its unmanageable complexity has spawned more fear-preventing tools than any other language, but the solution _should_ have been to create and use a language that does not overload the whole goddamn human brain with irrelevant details. -- Erik Naggum, comp.lang.lisp From pje at telecommunity.com Fri Aug 6 17:11:36 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 17:07:29 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <41136677.9090103@egenix.com> References: <200408052324.i75NOxA05160@guido.python.org> <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> <000f01c47b40$cde05110$6700a8c0@computer> <4112BF9E.40003@v.loewis.de> <200408052324.i75NOxA05160@guido.python.org> Message-ID: <5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com> At 01:07 PM 8/6/04 +0200, M.-A. Lemburg wrote: >1. Instead of trying to overgeneralize, we only allow exactly > *one* decorator per function definition. Not practical. Since many use cases for multiple decorators exist, restricting the syntax to one simply pushes the issue into functions to combine decorators, further decreasing readability. >2. Since decorators change the definition of a function, that > one decorator call is placed directly after the "def" and > before the function name. This makes it difficult to e.g. grep for 'def functionname', and will confuse the heck out of simplistic code scanners, not to mention Tim Peters. :) From guido at python.org Fri Aug 6 17:07:48 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 17:07:52 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: Your message of "Fri, 06 Aug 2004 10:52:15 EDT." References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> Message-ID: <200408061507.i76F7mI07201@guido.python.org> > It's not at all clear that Fredrik likes it, but he doesn't > participate in Python development any more so we may never know :-(. > From his blog: > > > ...so someone I've never heard of added a major syntactic feature > > to Python 2.4, using a syntax that Guido doesn't really > > like. sorry, guys, but I liked Python better when the core > > language was controlled by a single designer with a strong > > intuition. the current > > design-by-arguing-until-nobody-cares-anymore-committee just don't > > understand what made Python's design so damn good (as the "[new > > syntax] is no worse than [existing operator] meaning [something > > when used with a given type]" comment clearly shows). sigh. good > > thing nobody can force me to use decorators (or update to 2.4, for > > that matter). time to finish my own interpreter, perhaps. or at > > least start using a fresh one. > > Jeremy I saw that blog entry, and was surprised, because he really did seem to like it back then. It's clear that Fredrik (a) forgot that I showed it to him, and (b) misunderstood (or worse, deliberately misconstrued) how it came checked in. I'm disappointed. --Guido van Rossum (home page: http://www.python.org/~guido/) From edreamleo at charter.net Fri Aug 6 17:08:04 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 17:08:14 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> Message-ID: <017601c47bc7$2c84b720$6700a8c0@computer> [quote] Problems with this form: - it hides crucial information (e.g. that it is a static method) after the signature, where it is easily missed - it's easy to miss the transition between a long argument list and a long decorator list - it's cumbersome to cut and paste a decorator list for reuse, because it starts and ends in the middle of a line Given that the whole point of adding decorator syntax is to move the decorator from the end ("foo = staticmethod(foo)" after a 100-line body) to the front, where it is more in-your-face, it should IMO be moved all the way to the front. [end quote] Ah Jeeze :-) Are you really, truly saying that you want to introduce @ to help people format their declarations? This just seems like complete nonsense to me. How about: class C(object): def longMethodNameForEffect( longArgumentOne=None, longArgumentTwo=42)\ [ staticmethod, funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") ]: SNIP or even: class C(object): def longMethodNameForEffect( longArgumentOne=None, longArgumentTwo=42)\ \ # ***** DECORATION ***** [ staticmethod, funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") ]: SNIP Isn't "a total jumble of stuff ending with a smiley." a complete red herring? Look, there is a lot of information that has to be presented, whatever the means. How is _any_ syntax going to simplify: funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") ?? And are you seriously saying that problems cutting and pasting should drive this discussion? real-python-programmers-don't-need-help-formatting-ly yours, Edward P.S. In Leo one would render this: class C(object): def longMethodNameForEffect( longArgumentOne=None, longArgumentTwo=42) [ << decoration for longMethodNameForEffect >> ]: SNIP but I suppose that's not relevant :-) BTW, << decoration for longMethodNameForEffect >> would be syntax colored in red by default (with blue << and >>) so it would stand out. EKR -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From bkc at murkworks.com Fri Aug 6 17:22:33 2004 From: bkc at murkworks.com (Brad Clements) Date: Fri Aug 6 17:16:04 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408061445.i76EjqT07026@guido.python.org> References: Your message of "Fri, 06 Aug 2004 20:58:27 +1000." <41136453.1040908@interlink.com.au> Message-ID: <411369F9.4580.DDF2B6F1@coal.murkworks.com> On 6 Aug 2004 at 7:45, Guido van Rossum wrote: > Just look at it when either the list of decorators or the argument > list doesn't fit on one line: Could the decorator list be treated specially, like a docstring? class C(object): def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42): [ staticmethod, funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") ] """This method blah, blah. It supports the following arguments: - longArgumentOne -- a string giving ... - longArgumentTwo -- a number giving ... """ compare to: class C(object): @staticmethod @funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42): """This method blah, blah. It supports the following arguments: - longArgumentOne -- a string giving ... - longArgumentTwo -- a number giving ... """ -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements From pje at telecommunity.com Fri Aug 6 17:21:45 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 17:17:37 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <1091784135.1770.6.camel@localhost> References: <59e9fd3a04080519462bf57899@mail.gmail.com> <59e9fd3a04080519462bf57899@mail.gmail.com> Message-ID: <5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com> At 10:22 AM 8/6/04 +0100, Mark Russell wrote: >On Fri, 2004-08-06 at 03:46, Andrew Durdin wrote: > > 8. When using multiple decorators, each appears on a separate line. > >No, you can put multiple decorators on a single line -- you just need a >newline before the def. From test_decorators.py: > > class C(object): > @funcattrs(abc=1) @staticmethod > def foo(): return 42 > >Probably not good style (but neither is "if cond: stmt" I guess). *shudder* Can we change this? The positive benefits that the @ syntax *does* have, seem completely lost by doing this. From mwh at python.net Fri Aug 6 17:23:12 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 17:23:13 2004 Subject: [Python-Dev] PEP 318 - writing to func_name attribute In-Reply-To: <200408061453.i76Erpx07092@guido.python.org> (Guido van Rossum's message of "Fri, 06 Aug 2004 07:53:51 -0700") References: <16659.34089.645670.39263@montanaro.dyndns.org> <200408061453.i76Erpx07092@guido.python.org> Message-ID: <2m1xik5lzz.fsf@starship.python.net> Guido van Rossum writes: >> That fails because func_name is readonly. Any chance this restriction can >> be loosened up? > > Good idea, independent from decorators; please add it to the PEP! Quick question: would it be better to allow any object to be assigned to func_name and fix the (two, for core C code anway) places this matters or only allow strings to be assigned to func_name? I'm thinking probably the latter. Cheers, mwh -- Not only does the English Language borrow words from other languages, it sometimes chases them down dark alleys, hits them over the head, and goes through their pockets. -- Eddy Peters From mwh at python.net Fri Aug 6 17:28:26 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 17:28:28 2004 Subject: [Python-Dev] status of statistics.py? Message-ID: <2mwu0c476t.fsf@starship.python.net> I see statistics.py is still in nondist/sandbox/. Is there any deep reason for this? I see the file ends with XXX = """ Other possible functions include data groupers for binning, counting, and splitting into equivalence classes. """ but that hardly seems a blocker. I'd like to see the module in 2.4. Cheers, mwh -- Important data should not be entrusted to Pinstripe, as it may eat it and make loud belching noises. -- from the announcement of the beta of "Pinstripe" aka. Redhat 7.0 From mal at egenix.com Fri Aug 6 17:35:02 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 6 17:35:09 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com> References: <200408052324.i75NOxA05160@guido.python.org> <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> <000f01c47b40$cde05110$6700a8c0@computer> <4112BF9E.40003@v.loewis.de> <200408052324.i75NOxA05160@guido.python.org> <5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com> Message-ID: <4113A526.3090100@egenix.com> Phillip J. Eby wrote: > At 01:07 PM 8/6/04 +0200, M.-A. Lemburg wrote: > >> 1. Instead of trying to overgeneralize, we only allow exactly >> *one* decorator per function definition. > > > Not practical. Since many use cases for multiple decorators exist, > restricting the syntax to one simply pushes the issue into functions to > combine decorators, further decreasing readability. > > >> 2. Since decorators change the definition of a function, that >> one decorator call is placed directly after the "def" and >> before the function name. > > > This makes it difficult to e.g. grep for 'def functionname', and will > confuse the heck out of simplistic code scanners, not to mention Tim > Peters. :) Please read my entire post. Here's the grep pattern to use: "def( \w+)? funcname". -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From guido at python.org Fri Aug 6 17:37:57 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 17:38:02 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: Your message of "Fri, 06 Aug 2004 10:08:04 CDT." <017601c47bc7$2c84b720$6700a8c0@computer> References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <017601c47bc7$2c84b720$6700a8c0@computer> Message-ID: <200408061537.i76Fbv207317@guido.python.org> > Ah Jeeze :-) Are you really, truly saying that you want to introduce @ to > help people format their declarations? This just seems like complete > nonsense to me. Insulting words don't suit you, Edward. Yes, this whole thing is deeply about readability. It *started* with the observation that the post-function-body decorator "syntax" had usability / readability issues. A lot of Python is about that. --Guido van Rossum (home page: http://www.python.org/~guido/) From s.percivall at chello.se Fri Aug 6 17:38:21 2004 From: s.percivall at chello.se (Simon Percivall) Date: Fri Aug 6 17:38:23 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes Message-ID: On pep 318 (like everything else here). Is it actually supposed to work for classes? like the pep title says. It doesn't seem to work, instead it issues a SyntaxError on the class keyword. Also, the discussion is completely focused on decorating functions and methods. //Simon From barry at python.org Fri Aug 6 17:39:34 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 17:39:25 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> Message-ID: <1091806774.8414.63.camel@localhost> On Thu, 2004-08-05 at 20:39, Fernando Perez wrote: > If @ ends up being accepted, ipython can obviously adapt. I'll replace it with > alternative syntax, be it @@, %, or some other kind of special-casing trick. > > It would not make me happy, and it worries me that ipython explicitly uses (for > good reasons, I like to think), the three characters explicitly avoided by > python: So I spent a little time futzing with python-mode and some of my decorator code, to try some alternative leading characters. Of the ones that cannot be used in valid Python code today (i.e. no backward compatibility issues), I tried ':', '/', '=', '*', and '|'. : wasn't bad, and it had some nice symmetry with the function definition that follows. But those two little dots can be hard to read in some fonts and I'm worried that because colons are so special in Python, it might give existing tools some problems (python-mode had no problem with it though). / and | also weren't bad, although bar is probably preferred. Interestingly enough, in my font-lock settings, I italicize the decorator line, so these two characters looked pretty similar. Someone else pointing out that the bar kind of looks like it's anchoring the decorator to the function, and I got a similar sense of that. * was interesting because it evokes a feeling of a bulleted list, but ultimately it looked too jarring. = wasn't bad at all. It also had a bullet list feel to it, but without the discomfort of using stars. This was my favorite of the alternatives. In summary, I can definitely sympathize with the concern of tool authors who have settled on @ as a meta-character that Python will not use. I wonder how the Leo or ipython folks would feel if the only change to the current implementation of PEP 318 was to change the decorator introducing character. If we can increase the community's consensus by making this little change, I wouldn't oppose it. My preference then would be to use = with | and : as alternatives, in that order. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/014f8527/attachment.pgp From guido at python.org Fri Aug 6 17:39:55 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 17:40:00 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: Your message of "Fri, 06 Aug 2004 11:21:45 EDT." <5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com> References: <59e9fd3a04080519462bf57899@mail.gmail.com> <59e9fd3a04080519462bf57899@mail.gmail.com> <5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com> Message-ID: <200408061539.i76FdtG07333@guido.python.org> > >No, you can put multiple decorators on a single line -- you just need a > >newline before the def. From test_decorators.py: > > > > class C(object): > > @funcattrs(abc=1) @staticmethod > > def foo(): return 42 > > > >Probably not good style (but neither is "if cond: stmt" I guess). > > *shudder* > > Can we change this? The positive benefits that the @ syntax *does* > have, seem completely lost by doing this. That's a bit strong, but yes, it would be trivial to change. If that's all that's needed to save @deco, it's a lot easier than ripping the whole thing out... :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 6 17:40:31 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 17:40:36 2004 Subject: [Python-Dev] PEP 318 - writing to func_name attribute In-Reply-To: Your message of "Fri, 06 Aug 2004 16:23:12 BST." <2m1xik5lzz.fsf@starship.python.net> References: <16659.34089.645670.39263@montanaro.dyndns.org> <200408061453.i76Erpx07092@guido.python.org> <2m1xik5lzz.fsf@starship.python.net> Message-ID: <200408061540.i76FeVd07348@guido.python.org> > Quick question: would it be better to allow any object to be assigned > to func_name and fix the (two, for core C code anway) places this > matters or only allow strings to be assigned to func_name? > > I'm thinking probably the latter. Let's stick to strings. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri Aug 6 17:43:16 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 17:43:07 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <4112DD32.5080302@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <4112DD32.5080302@interlink.com.au> Message-ID: <1091806996.8413.66.camel@localhost> On Thu, 2004-08-05 at 21:21, Anthony Baxter wrote: > My take on the @decorator syntax is that after initially > hating it, I've found it growing on me more and more. > It's extremely clear and very readable. Most importantly, > in my eyes, it's obviously grouped with the def that > follows it. From being a -0 (when Guido first mentioned > it) to a +0 (at the time of checkin) I'm now +1 on this > form of decorator. I'm also +1 on using the format/position of the @decorators. It's vastly superior to list-before-colon or list-before-def. See my other post for alternative possible solutions to Leo/ipython problem. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/0c5906b4/attachment-0001.pgp From s.percivall at chello.se Fri Aug 6 17:43:13 2004 From: s.percivall at chello.se (Simon Percivall) Date: Fri Aug 6 17:43:16 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: References: Message-ID: <53CF6F70-E7BF-11D8-8E6E-0003934AD54A@chello.se> Sorry. I actually read some of the pep this time ... saw the Open Issues part. Still, is there any thinking about this? if I may ask. On 2004-08-06, at 17.38, Simon Percivall wrote: > On pep 318 (like everything else here). > > Is it actually supposed to work for classes? like the > pep title says. > > It doesn't seem to work, instead it issues a SyntaxError > on the class keyword. Also, the discussion is completely > focused on decorating functions and methods. > > //Simon From skip at pobox.com Fri Aug 6 17:48:16 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 17:48:46 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: References: Message-ID: <16659.43072.703540.287723@montanaro.dyndns.org> Simon> On pep 318 (like everything else here). Simon> Is it actually supposed to work for classes? That was the original proposal, way back when. The motivation for decorating classes seems much weaker than for functions. The current implementation doesn't decorate classes. The PEP title should probably be changed. Skip From barry at python.org Fri Aug 6 17:50:40 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 17:50:35 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <4112DD1C.3060902@interlink.com.au> References: <4112DD1C.3060902@interlink.com.au> Message-ID: <1091807440.8406.69.camel@localhost> On Thu, 2004-08-05 at 21:21, Anthony Baxter wrote: > There's also the issue that there's a bunch > of other existing PEPs describing features that aren't > up to date at all. Heh, yeah, let's rip out rich comparisons, since although the feature's been in Python since 2.1, the PEP was definitely out of date. 'Course, I just fixed it so go ahead and put them back in . -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/4b9a79ee/attachment.pgp From guido at python.org Fri Aug 6 17:50:44 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 17:50:49 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: Your message of "Fri, 06 Aug 2004 17:35:02 +0200." <4113A526.3090100@egenix.com> References: <200408052324.i75NOxA05160@guido.python.org> <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> <000f01c47b40$cde05110$6700a8c0@computer> <4112BF9E.40003@v.loewis.de> <200408052324.i75NOxA05160@guido.python.org> <5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com> <4113A526.3090100@egenix.com> Message-ID: <200408061550.i76Foig07409@guido.python.org> > Please read my entire post. Here's the grep pattern to > use: "def( \w+)? funcname". That doesn't work with grep on my box. 'nuff said. --Guido van Rossum (home page: http://www.python.org/~guido/) From jim at zope.com Fri Aug 6 17:54:53 2004 From: jim at zope.com (Jim Fulton) Date: Fri Aug 6 17:54:57 2004 Subject: [Python-Dev] I'd like to add an __iter__ method to unittest.TestSuite Message-ID: <4113A9CD.6030904@zope.com> TestSuite objects provide no public mechanism for iteration. We have a need to be able to iterate over them. Up to now, we've been using the _tests attribute, which is bad. If no one objects, I'll add an __iter__ method. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From guido at python.org Fri Aug 6 17:55:31 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 17:55:37 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: Your message of "Fri, 06 Aug 2004 17:38:21 +0200." References: Message-ID: <200408061555.i76FtV707448@guido.python.org> > On pep 318 (like everything else here). > > Is it actually supposed to work for classes? like the > pep title says. Like so many things in the PEP, it's out of date. > It doesn't seem to work, instead it issues a SyntaxError > on the class keyword. Also, the discussion is completely > focused on decorating functions and methods. Some people have argued that it should work for classes (Java and C# decorators work for any declaration). I've never been convinced, although I'm not dead set against it either. It would seem that the functionality isn't quite as useful there; you can get most of the same effects with metaclasses. I'm not aware of any idiom that takes a class and passes it through some kind of wrapper or transformation *after* the class declaration; if somebody is using this, it surely must be a very rare use indeed. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 6 17:57:16 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 17:57:23 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: Your message of "Fri, 06 Aug 2004 11:39:34 EDT." <1091806774.8414.63.camel@localhost> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> Message-ID: <200408061557.i76FvGo07481@guido.python.org> > So I spent a little time futzing with python-mode and some of my > decorator code, to try some alternative leading characters. Of the ones > that cannot be used in valid Python code today (i.e. no backward > compatibility issues), I tried ':', '/', '=', '*', and '|'. If the community can rally behind one of these, I think that would be acceptable. They all seem arbitrary, but so is the choice of '@'. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Fri Aug 6 17:58:19 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 6 17:58:21 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <200408061550.i76Foig07409@guido.python.org> References: <200408052324.i75NOxA05160@guido.python.org> <004a01c47b2b$65d19260$6700a8c0@computer> <4112A4B9.4010907@v.loewis.de> <000f01c47b39$de3bcd70$6700a8c0@computer> <4112B6DC.4080605@v.loewis.de> <000f01c47b40$cde05110$6700a8c0@computer> <4112BF9E.40003@v.loewis.de> <200408052324.i75NOxA05160@guido.python.org> <5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com> <4113A526.3090100@egenix.com> <200408061550.i76Foig07409@guido.python.org> Message-ID: <4113AA9B.5020608@egenix.com> Guido van Rossum wrote: >>Please read my entire post. Here's the grep pattern to >>use: "def( \w+)? funcname". > > That doesn't work with grep on my box. Dev-Python/Lib> egrep "def( \w+)? __init__" *.py Bastion.py: def __init__(self, get, name): Bastion.py: def __init__(self): ConfigParser.py: def __init__(self, msg=''): ConfigParser.py: def __init__(self, section): ... test-decorate.py:def staticmethod test(self): > 'nuff said. Indeed. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From edreamleo at charter.net Fri Aug 6 18:02:26 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Fri Aug 6 18:02:33 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <017601c47bc7$2c84b720$6700a8c0@computer> <200408061537.i76Fbv207317@guido.python.org> Message-ID: <002901c47bce$c51edc70$6700a8c0@computer> > Insulting words don't suit you, Edward. I apologize for the insult. None was intended. > Yes, this whole thing is deeply about readability. OK. I've said everything I can say on this topic, except this: I would like to commend you personally Guido, and everyone else on this list for their willingness to reopen what is no doubt was a painful discussion. Leo and I can live with whatever you decide. I hope you can convince most Python users that the choice is a good one. Bye for now. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From anthony at interlink.com.au Fri Aug 6 18:03:32 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 18:04:09 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <16659.43072.703540.287723@montanaro.dyndns.org> References: <16659.43072.703540.287723@montanaro.dyndns.org> Message-ID: <4113ABD4.9010809@interlink.com.au> Skip Montanaro wrote: > Simon> On pep 318 (like everything else here). > Simon> Is it actually supposed to work for classes? > > That was the original proposal, way back when. The motivation for > decorating classes seems much weaker than for functions. The current > implementation doesn't decorate classes. The PEP title should probably be > changed. And this is now done. There are probably still references to classes in inappropriate places in the doco. I will spend more time on it this weekend. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From tim.peters at gmail.com Fri Aug 6 18:06:49 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 6 18:06:53 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <1091806774.8414.63.camel@localhost> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> Message-ID: <1f7befae040806090644be9c04@mail.gmail.com> [Barry Warsaw] > So I spent a little time futzing with python-mode and some of my > decorator code, to try some alternative leading characters. Of the ones > that cannot be used in valid Python code today (i.e. no backward > compatibility issues), I tried ':', '/', '=', '*', and '|'. > > : wasn't bad, and it had some nice symmetry with the function definition > that follows. But those two little dots can be hard to read in some > fonts ... Indeed! I read that paragraph four times before I realized it started with a colon -- a fleck of dirt on the screen obscured it completely. > / and | also weren't bad, although bar is probably preferred. Ya, but then we preclude adding the unary-pipe operator. This is very important for numeric types that need to implement half the absolute value: >>> |3 1.5 >>> |-3 1.5 ... > Someone else pointing out that the bar kind of looks like it's anchoring the > decorator to the function, and I got a similar sense of that. OK, seriously, me too. > ... > In summary, I can definitely sympathize with the concern of tool authors > who have settled on @ as a meta-character that Python will not use. Yup. > I wonder how the Leo or ipython folks would feel if the only change to the > current implementation of PEP 318 was to change the decorator > introducing character. If we can increase the community's consensus by > making this little change, I wouldn't oppose it. Me neither. > My preference then would be to use = with | and : as alternatives, in > that order. Same here, except -1 on the colon. Although = would preclude introducing the unary-equals operator, which is very important for numeric types that need to implement the absolute value rotated by 90 degrees. From mwh at python.net Fri Aug 6 18:07:14 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 6 18:07:17 2004 Subject: [Python-Dev] PEP 318 - writing to func_name attribute In-Reply-To: <200408061540.i76FeVd07348@guido.python.org> (Guido van Rossum's message of "Fri, 06 Aug 2004 08:40:31 -0700") References: <16659.34089.645670.39263@montanaro.dyndns.org> <200408061453.i76Erpx07092@guido.python.org> <2m1xik5lzz.fsf@starship.python.net> <200408061540.i76FeVd07348@guido.python.org> Message-ID: <2mpt6445e5.fsf@starship.python.net> Guido van Rossum writes: >> Quick question: would it be better to allow any object to be assigned >> to func_name and fix the (two, for core C code anway) places this >> matters or only allow strings to be assigned to func_name? >> >> I'm thinking probably the latter. > > Let's stick to strings. Cool. http://www.python.org/sf/1004703 awaits review. Cheers, mwh -- ... Windows proponents tell you that it will solve things that your Unix system people keep telling you are hard. The Unix people are right: they are hard, and Windows does not solve them, ... -- Tim Bradshaw, comp.lang.lisp From anthony at interlink.com.au Fri Aug 6 18:09:08 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 18:11:15 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <16659.34238.443364.229226@montanaro.dyndns.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost> <4112A746.6010501@v.loewis.de> <2m657w7b1k.fsf@starship.python.net> <1091792604.1770.8.camel@localhost> <16659.34238.443364.229226@montanaro.dyndns.org> Message-ID: <4113AD24.9050205@interlink.com.au> Skip Montanaro wrote: > Mark> I think it is just the PEP that needs updating. > > Not to pick on Mark, but everyone seems to think the PEP needs updating but > nobody's stepping up to offer patches. Anthony and I are both about out of > round tuits... What Skip said. For a topic generating this much heat, volunteers have been thin on the ground. In case it's not clear - there is an _enormous_ amount of work that could be put into the PEP. Collecting posts from the 2+ years that this topic was active on python-dev is a mammoth task. It's also something that parallelises well. Please, if you care about this (and it's obvious from the volume of posts that lots of people do), spend a bit of time adding entries to the wiki. References to python-dev posts from the archives, in particular, are _very_ nice. I have a crapload of other things I'd much prefer to be working on, and help would be _much_ appreciated. Anthony From anthony at interlink.com.au Fri Aug 6 18:11:07 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 18:11:20 2004 Subject: [Python-Dev] PEP 318, and the PEP process In-Reply-To: <1091807440.8406.69.camel@localhost> References: <4112DD1C.3060902@interlink.com.au> <1091807440.8406.69.camel@localhost> Message-ID: <4113AD9B.3090003@interlink.com.au> Barry Warsaw wrote: > > Heh, yeah, let's rip out rich comparisons, since although the feature's > been in Python since 2.1, the PEP was definitely out of date. 'Course, > I just fixed it so go ahead and put them back in . Maybe rich comparisions should be prefixed with the $ sign, or the unicode glyph for euros, or whatever the local currency symbol is. i18n-ising-the-grammar-is-the-next-big-thing, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Fri Aug 6 18:13:57 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 18:14:08 2004 Subject: [Python-Dev] pie-thon redux Message-ID: <4113AE45.5080004@interlink.com.au> As a break from the decorator thread, NTK (www.ntk.net) had this writeup of the Pie-Thon. It amused me. >> HARD NEWS << Perl hullabaloos Last year, DAN SUGALSKI, lead developer of the forthcoming Perl6 virtual machine, Parrot, bet the Python developers that run Python faster on the fledgling VM - and the creator of Python could throw a pie at him at the next OSCON if he didn't. He didn't; the pie-ing was duly arranged. As everyone knows, while Perlites are chaotic/good trickster archetypes who love such events, Pythonistas are peaceful, have-their-glasses-on-a-little-string types, like hobbits or the Dutch. In the end, Guido van Rossum refused to throw the pie, and instead offered to share it as food with the Perl developers. Nothing, of course, could have been more guaranteed to throw Perlsters into violent rage. An extended period of acrimonious bargaining followed, in which the Perl crew grew more and more insistent that their own chief developer be humiliated, with many walking out of the session, muttering about "all foreplay and no sex". Later, the Perl faction took it upon themselves to pie Sugalski - much, we are sure, to the shock of the pacifistic Pythonese, who may well have planned that using their psychomathematics and indented whitespace necromancy all along. (Mind you, that didn't stop Guido finally joining in. Feel the punctuation rising in you, Guido!) http://www.sidhe.org/~dan/blog/archives/000372.html - Dan gets last laugh later in August -- Anthony Baxter It's never too late to have a happy childhood. From tim.peters at gmail.com Fri Aug 6 18:16:17 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 6 18:16:20 2004 Subject: [Python-Dev] I'd like to add an __iter__ method to unittest.TestSuite In-Reply-To: <4113A9CD.6030904@zope.com> References: <4113A9CD.6030904@zope.com> Message-ID: <1f7befae04080609165c13c748@mail.gmail.com> [Jim Fulton] > TestSuite objects provide no public mechanism for iteration. > We have a need to be able to iterate over them. Up to now, > we've been using the _tests attribute, which is bad. > > If no one objects, I'll add an __iter__ method. +1. The iteration protocol is endlessly useful, and this a natural use for it. From anthony at interlink.com.au Fri Aug 6 18:18:15 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 18:18:29 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <1f7befae040806090644be9c04@mail.gmail.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> Message-ID: <4113AF47.1010405@interlink.com.au> Tim Peters wrote: > [Barry Warsaw] >>... >>In summary, I can definitely sympathize with the concern of tool authors >>who have settled on @ as a meta-character that Python will not use. > > Yup. Having said that, it's quite possible that in the future some other feature will use the @ symbol - I think arrays have been mentioned in the same sense, although presumably not in the sense of @variable meaning "it's an array". So, we either: - explicitly list a symbol that Shall Not Be Used (I favour the unicode glyph for biohazard), or - change the language ref that talks about this to state that although Python currently doesn't use these symbols, this is not stopping a future version of Python from using them for some new feature. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From josh at janrain.com Fri Aug 6 18:20:20 2004 From: josh at janrain.com (Josh Hoyt) Date: Fri Aug 6 18:20:22 2004 Subject: [Python-Dev] Density of pie-decorator syntax Message-ID: <4113AFC4.5010004@janrain.com> One problem I have with the pie-decorator syntax is that it is very typographically dense. The @ character runs in with the following function name. If there are multiple lines of decorators, the problem is worsened. For clarity, the decorators will have no whitespace lines between them, creating a dense block. After experimenting, I have found that readability is increased if there is one space required after the @. Don't take my word for it: Current Python 2.4a2 syntax: @framework_stuff(lots, of, args) class Quux(object): @check_args(int, str) @counted @staticmethod def frobnicate(foo, bar): pass Proposed change requiring a space: @ framework_stuff(lots of args) class Quux(object): @ check_args(int, str) @ counted @ staticmethod def frobnicate(foo, bar): pass From jimjjewett at yahoo.com Fri Aug 6 18:31:47 2004 From: jimjjewett at yahoo.com (Jim J Jewett) Date: Fri Aug 6 18:31:50 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? Message-ID: <20040806163147.82284.qmail@web50710.mail.yahoo.com> Nicolas Fleury asked: > Is there ambiguities with the following? > def decorator1(decoratorN(foo(arg1, argN))): > pass Remember that there is already an obscure possibility for nested argument lists. def func (arg1, (arg2, arg3)): pass __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From guido at python.org Fri Aug 6 18:34:14 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 18:34:25 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: Your message of "Sat, 07 Aug 2004 02:18:15 +1000." <4113AF47.1010405@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> Message-ID: <200408061634.i76GYEZ07623@guido.python.org> > - change the language ref that talks about this to state that > although Python currently doesn't use these symbols, this is not > stopping a future version of Python from using them for some new > feature. Does that really have to be stated? Isn't this just giving in to the silly lawyers? A reference manual describes the status quo. How can it possibly begin to describe future directions? (Except when reserving certain classes of identifiers like C does.) --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony at interlink.com.au Fri Aug 6 18:37:13 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 18:37:24 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061634.i76GYEZ07623@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> Message-ID: <4113B3B9.6080801@interlink.com.au> Guido van Rossum wrote: >>- change the language ref that talks about this to state that >>although Python currently doesn't use these symbols, this is not >>stopping a future version of Python from using them for some new >>feature. > > > Does that really have to be stated? Isn't this just giving in to the > silly lawyers? A reference manual describes the status quo. How can > it possibly begin to describe future directions? (Except when > reserving certain classes of identifiers like C does.) Beats me. I'm just going from the comments that Edward(?) made that he thought the current wording meant that they'd never be used. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From michael.walter at gmail.com Fri Aug 6 18:40:52 2004 From: michael.walter at gmail.com (Michael Walter) Date: Fri Aug 6 18:40:56 2004 Subject: [Python-Dev] Density of pie-decorator syntax In-Reply-To: <4113AFC4.5010004@janrain.com> References: <4113AFC4.5010004@janrain.com> Message-ID: <877e9a1704080609407d680e45@mail.gmail.com> Please, no! :) Cheers, Michael On Fri, 06 Aug 2004 09:20:20 -0700, Josh Hoyt wrote: > One problem I have with the pie-decorator syntax is that it is very > typographically dense. The @ character runs in with the following > function name. If there are multiple lines of decorators, the problem is > worsened. For clarity, the decorators will have no whitespace lines > between them, creating a dense block. > > After experimenting, I have found that readability is increased if there > is one space required after the @. Don't take my word for it: > > Current Python 2.4a2 syntax: > > @framework_stuff(lots, of, args) > class Quux(object): > > @check_args(int, str) > @counted > @staticmethod > def frobnicate(foo, bar): > pass > > Proposed change requiring a space: > > @ framework_stuff(lots of args) > class Quux(object): > > @ check_args(int, str) > @ counted > @ staticmethod > def frobnicate(foo, bar): > pass > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com > From nidoizo at yahoo.com Fri Aug 6 18:50:27 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Fri Aug 6 18:50:34 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408061537.i76Fbv207317@guido.python.org> References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <017601c47bc7$2c84b720$6700a8c0@computer> <200408061537.i76Fbv207317@guido.python.org> Message-ID: Guido van Rossum wrote: > Yes, this whole thing is deeply about readability. It *started* with > the observation that the post-function-body decorator "syntax" had > usability / readability issues. A lot of Python is about that. I agree. In fact, I've been convinced by both sides that both @decorators and list-after-def solutions have major readability drawbacks. For example, this is more readable than both: decorate classmethod: def foo(arg1,arg2): ... But it is less usable when the decorator applies to a single function, unless a syntax is allowed to do it without indentation. And there's a new keyword. But adding a special character to avoid adding a new keyword? Wouldn't a "from __future__ import decorate" until Python 2.X or 3 be better? It seems like a good compromise to me... Regards, Nicolas From guido at python.org Fri Aug 6 18:55:12 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 18:55:29 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: Your message of "Sat, 07 Aug 2004 02:37:13 +1000." <4113B3B9.6080801@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> <4113B3B9.6080801@interlink.com.au> Message-ID: <200408061655.i76GtCJ07709@guido.python.org> > Beats me. I'm just going from the comments that Edward(?) made that > he thought the current wording meant that they'd never be used. He'd have a hard time proving that in court. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 6 18:58:59 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 18:59:05 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: Your message of "Fri, 06 Aug 2004 12:50:27 EDT." References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <017601c47bc7$2c84b720$6700a8c0@computer> <200408061537.i76Fbv207317@guido.python.org> Message-ID: <200408061658.i76Gwxf07727@guido.python.org> > decorate classmethod: > def foo(arg1,arg2): > ... Somebody should add to the PEP that solutions requiring an extra indent level for the function have a drawback. Having your minimal indent level be three deep is painful for those using limited-width windows; also, the inconsistent indentation between methods with and without decorators would be a readability problem (plus, adding/removing decorators would require reindenting the entire method body). --Guido van Rossum (home page: http://www.python.org/~guido/) From jlg at python.org Fri Aug 6 18:52:28 2004 From: jlg at python.org (Johannes Gijsbers) Date: Fri Aug 6 19:00:37 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> Message-ID: On Fri, 06 Aug 2004 11:39:34 -0400, Barry Warsaw wrote: > = wasn't bad at all. It also had a bullet list feel to it, but without > the discomfort of using stars. This was my favorite of the > alternatives. '=' has too srong an association with assignment for me, The first thing I wondered was where the '=' was assigning to. Then I realized that, in the context of decorators, it has to be the function/method that follows. So in the following example: =classmethod def foo(arg1,arg2): ... 'foo' '=' a classmethod. This does broaden the meaning of '=' from just equality of identity to belonging to a group, but that's mostly okay. However, the next example really bugs me: =accepts(int,int) =returns(float) def bar(low,high): ... 'bar' '=' in no sense an 'accepts' or a 'returns'. It doesn't seem right to use an assignment operator for decorating. Keeping Tim's comments about ':' in mind, my personal favorite is '|'. Johannes From niemeyer at conectiva.com Fri Aug 6 19:10:22 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 19:08:04 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408061658.i76Gwxf07727@guido.python.org> References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <017601c47bc7$2c84b720$6700a8c0@computer> <200408061537.i76Fbv207317@guido.python.org> <200408061658.i76Gwxf07727@guido.python.org> Message-ID: <20040806171022.GA3205@burma.localdomain> > > decorate classmethod: > > def foo(arg1,arg2): > > ... > > Somebody should add to the PEP that solutions requiring an extra > indent level for the function have a drawback. Having your minimal > indent level be three deep is painful for those using limited-width > windows; also, the inconsistent indentation between methods with and > without decorators would be a readability problem (plus, I still prefer this method over the current implementation, given the other pros/cons. Anyway, I'm adding these to the Wiki as negative points. > adding/removing decorators would require reindenting the entire method > body). I belive all of us are used to quickly reidenting Python code by now. :-) -- Gustavo Niemeyer http://niemeyer.net From shane.holloway at ieee.org Fri Aug 6 19:07:22 2004 From: shane.holloway at ieee.org (Shane Holloway (IEEE)) Date: Fri Aug 6 19:08:19 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <200408061555.i76FtV707448@guido.python.org> References: <200408061555.i76FtV707448@guido.python.org> Message-ID: <4113BACA.6040801@ieee.org> Guido van Rossum wrote: >>It doesn't seem to work, instead it issues a SyntaxError >>on the class keyword. Also, the discussion is completely >>focused on decorating functions and methods. > > > Some people have argued that it should work for classes (Java and C# > decorators work for any declaration). I've never been convinced, > although I'm not dead set against it either. > > It would seem that the functionality isn't quite as useful there; you > can get most of the same effects with metaclasses. I'm not aware of > any idiom that takes a class and passes it through some kind of > wrapper or transformation *after* the class declaration; if somebody > is using this, it surely must be a very rare use indeed. > > --Guido van Rossum (home page: http://www.python.org/~guido/) I have done a few things in the realm of Aspect Oriented programming, replacing __metaclass__ with a callable that delegates the "hard work" to type() and then plays with the resulting class rather like classmethod does. I remember trying the replace semantics after the end of the class, but preferred the in-the-face-"pay-attention" of using the metaclass hook. I think that class pie-decorators would serve the same purpose. I could see the protocols "implements" functionality expressed through class decorators, although what they have now works fine. On the positive side, decorators for classes will probably be significantly easier to understand and implement than metaclasses, though less powerful. So, my thoughts on class decorators are: 1) They have their purposes, and direct support will probably bring out more uses of them 2) Orthogonality is good -- the same syntax for methods and classes would be preferred, and understanding would transfer between the two forms 3) They are not needed right now -- lets just make sure they can work in the same way as method decorators if we want them later And for the record, I prefer the @-syntax. Readable, short, easily commentable, multi-line, and before the function declaration. I think it will be a nice addition. Though other characters would be fine, too. I'm really tired of typing my method names three times with the 2.3 syntax. ;) Thanks for your time, -Shane Holloway From martin at v.loewis.de Fri Aug 6 19:08:24 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 19:08:25 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <1091806774.8414.63.camel@localhost> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> Message-ID: <4113BB08.1080407@v.loewis.de> Barry Warsaw wrote: > In summary, I can definitely sympathize with the concern of tool authors > who have settled on @ as a meta-character that Python will not use. I > wonder how the Leo or ipython folks would feel if the only change to the > current implementation of PEP 318 was to change the decorator > introducing character. If we can increase the community's consensus by > making this little change, I wouldn't oppose it. My exact feelings. That it breaks existing tools *is* a relevant and objective objection, and if all it takes to fix it is to use a different character, this gets +1 from me. Regards, Martin From niemeyer at conectiva.com Fri Aug 6 19:11:54 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 19:09:35 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <411369F9.4580.DDF2B6F1@coal.murkworks.com> References: <41136453.1040908@interlink.com.au> <411369F9.4580.DDF2B6F1@coal.murkworks.com> Message-ID: <20040806171154.GB3205@burma.localdomain> > Could the decorator list be treated specially, like a docstring? > > class C(object): > > def longMethodNameForEffect(longArgumentOne=None, > longArgumentTwo=42): > [ > staticmethod, > funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", > status="experimental", author="BDFL") > ] > """This method blah, blah. There are compatibility problems, since this is valid code right now (same issues which left dec-before-def as a no-no). -- Gustavo Niemeyer http://niemeyer.net From tismer at stackless.com Fri Aug 6 19:15:04 2004 From: tismer at stackless.com (Christian Tismer) Date: Fri Aug 6 19:15:03 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061557.i76FvGo07481@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> Message-ID: <4113BC98.3020407@stackless.com> Guido van Rossum wrote: >>So I spent a little time futzing with python-mode and some of my >>decorator code, to try some alternative leading characters. Of the ones >>that cannot be used in valid Python code today (i.e. no backward >>compatibility issues), I tried ':', '/', '=', '*', and '|'. > > > If the community can rally behind one of these, I think that would be > acceptable. They all seem arbitrary, but so is the choice of '@'. :-) Not that I want to put more oil into this never ending story, but I think a few messages ago you mentioned that the primary purpose of all the decorator syntax effort is to get the foo = classmethod(foo) into the right place, not after but before the definition. Dunno if this was mentioned before, but why not simply do that? Move the statement right before the def and provide a way that this gets recognized correctly. No syntax change, just the convenience people asked for. Well, just an idea -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From barry at python.org Fri Aug 6 19:20:59 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 19:20:49 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> <1f7befae04080522314d818d8f@mail.gmail.com> Message-ID: <1091812859.8413.82.camel@localhost> On Fri, 2004-08-06 at 02:03, IxokaI wrote: > Doesn't the fact that the @stuff is at the same level and has no > connection to the def besides 'above' bother you in our universe of > indentation-love? It did for me...originally. In fact, early in the discussions I favored list-before-colon syntax. Now that I've had a chance to play with pie syntax and rewrite my code (many several of times) for all the various syntax options, I am definitely favoring decorator-before-def (e.g. pie decorators). My primary concerns with that were in using list notation which is legal code today (that's bad because decorators would subtly not do what's intended in older Pythons), and the visual attachment of decorators to the function they modify. The latter turns out not to be much of a problem in practice, IMO. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/4f6eb503/attachment-0001.pgp From tim.peters at gmail.com Fri Aug 6 19:21:42 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 6 19:21:44 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061634.i76GYEZ07623@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> Message-ID: <1f7befae0408061021600409fb@mail.gmail.com> [Guido] > A reference manual describes the status quo. How can > it possibly begin to describe future directions? (Except when > reserving certain classes of identifiers like C does.) If it's a designer's intent that, say, the '#' character will never be used by the language, then the reference manual can simply say that. In the absence of such explicit promises, it is indeed only describing the current state of the language, It's unreasonable to read the current Python ref as promising that @ (and ? and $) will never be used in any version of Python. It's understandable that people would bet on it, though, since that little set of unused characters hasn't changed in Python's history. If you want to say that Python will *never* use one (or more) of those characters, it's easy to add such a promise. If you don't want to make such a promise, then it may be clearer (for non-lawyers) to say that Python reserves @$?. From ixokai at gmail.com Fri Aug 6 19:23:19 2004 From: ixokai at gmail.com (IxokaI) Date: Fri Aug 6 19:23:24 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408061658.i76Gwxf07727@guido.python.org> References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <017601c47bc7$2c84b720$6700a8c0@computer> <200408061537.i76Fbv207317@guido.python.org> <200408061658.i76Gwxf07727@guido.python.org> Message-ID: On Fri, 06 Aug 2004 09:58:59 -0700, Guido van Rossum wrote: > > decorate classmethod: > > def foo(arg1,arg2): > > ... > Somebody should add to the PEP that solutions requiring an extra > indent level for the function have a drawback. Having your minimal > indent level be three deep is painful for those using limited-width > windows; also, the inconsistent indentation between methods with and > without decorators would be a readability problem (plus, > adding/removing decorators would require reindenting the entire method > body). Doh. That's true... What about: class foo: def bar(arg1, arg2): | staticmethod | interface(int, string) """ This is my doc string """ for i in range(arg1): print arg2 I've been convinced that the list-after-def is bad, but still loathe @ as painful, and really, really, really hate how the @decorators do not fit into our indented world of grouping stuff together well. I think a keyword like, say, "using" instead of | would be even better, but... :) class foo: def blah(self): return None | staticmethod | interface(int, string) def bar(arg1, arg2): """ This is my doc string """ for i in range(arg1): print arg2 In that example, only knowing "the bar goes down" clearly connects the decorators to the right method. And if 'blah' was one of those bad-styled one-line-functions... --Stephen From niemeyer at conectiva.com Fri Aug 6 19:28:48 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 19:26:29 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <1091790258.411365b20c38a@mcherm.com> References: <1091790258.411365b20c38a@mcherm.com> Message-ID: <20040806172848.GC3205@burma.localdomain> > > Is there something to be done that could change the current > > decision? Voting? Collecting pros/cons and classifying them? > > Yes, there is. And it's not voting . > > Write a simple "white paper" summarizing the major arguments > and showing how they apply to the 3 or 4 top syntax proposals. > Solicit input from c.l.py or python-dev after you have a first > draft. After the 3rd draft is done, post it here and send it > to Guido. Then sit back and quietly put up with the resulting > decision whether you agree or not. I just did that. I've gone through all the recent discussions updating the Wiki page with mentioned arguments and new syntax proposals. > At this point, I long ago said most everything I had to say > and so I'm working on the "quietly put up with the results" > part myself. I don't think there's something else I could collaborate either. -- Gustavo Niemeyer http://niemeyer.net From niemeyer at conectiva.com Fri Aug 6 19:34:04 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Fri Aug 6 19:31:48 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <16659.34238.443364.229226@montanaro.dyndns.org> References: <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost> <4112A746.6010501@v.loewis.de> <2m657w7b1k.fsf@starship.python.net> <1091792604.1770.8.camel@localhost> <16659.34238.443364.229226@montanaro.dyndns.org> Message-ID: <20040806173403.GD3205@burma.localdomain> > Mark> I think it is just the PEP that needs updating. > > Not to pick on Mark, but everyone seems to think the PEP needs > updating but nobody's stepping up to offer patches. Anthony and I are > both about out of round tuits... I've just updated the Wiki with all arguments and syntax proposals I could find in the recent discussions. Not sure if staying in the Wiki is ok or if it should be migrated to the PEP now. Adding information to the Wiki is a lot easier, so perhaps the PEP should be updated with information from there gradually, or after stabilization. -- Gustavo Niemeyer http://niemeyer.net From fperez528 at yahoo.com Fri Aug 6 19:40:47 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Fri Aug 6 19:40:55 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> Message-ID: Guido van Rossum wrote: >> - change the language ref that talks about this to state that >> although Python currently doesn't use these symbols, this is not >> stopping a future version of Python from using them for some new >> feature. > > Does that really have to be stated? Isn't this just giving in to the > silly lawyers? A reference manual describes the status quo. How can > it possibly begin to describe future directions? (Except when > reserving certain classes of identifiers like C does.) As I stated before, it was always perfectly clear to me that in using @$? for ipython, I was taking a risk. So I don't feel in any way 'lied to' by the documentation, since nowhere did it make any kind of promise for the future. It only stated the current status quo, and that's how I read it. I decided to make use of that status quo for my purposes, but being fully aware that I might have to adapt in the future. Best, f From psheer at icon.co.za Fri Aug 6 19:39:16 2004 From: psheer at icon.co.za (Paul Sheer) Date: Fri Aug 6 19:42:55 2004 Subject: [Python-Dev] Merging Python and Gtk... Message-ID: <20040806173916.GA2212@divinian> Now that Gtk is supported on Windows, X, and framebuffers, would it not be worthwhile to consider packaging Python and Gtk together? Python+Gtk is a killer combination, and I see the only argument against Python's mainstream adoption to be its lack of a standardized widget library as portable and eligant as Python itself. If Python and Gtk installed as one package, this requirement would be met. At the moment, getting Python to work with Gtk on an arbitrary Unix system requires compiling no less than 14 tar balls from source. There are also huge overlaps in functionality, like Python and Gtk image support (jpeg etc.) comments? -paul Paul Sheer . . . . . . . . . . . . . . . . . Tel . . +27 (0)21 6869634 Email . . . http://2038bug.com/email.gif . . . . . . . . . . . . . . . . http://www.icon.co.za/~psheer . . . . . . . . . http://rute.2038bug.com L I N U X . . . . . . . . . . . . . . . . The Choice of a GNU Generation From bac at OCF.Berkeley.EDU Fri Aug 6 19:50:01 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 6 19:50:02 2004 Subject: [Python-Dev] Merging Python and Gtk... In-Reply-To: <20040806173916.GA2212@divinian> References: <20040806173916.GA2212@divinian> Message-ID: <4113C4C9.2090904@ocf.berkeley.edu> Paul Sheer wrote: > Now that Gtk is supported on Windows, X, and framebuffers, > would it not be worthwhile to consider packaging Python and > Gtk together? > > Python+Gtk is a killer combination, and I see the only > argument against Python's mainstream adoption to be its lack > of a standardized widget library as portable and eligant as > Python itself. > > If Python and Gtk installed as one package, this requirement > would be met. At the moment, getting Python to work with > Gtk on an arbitrary Unix system requires compiling no less > than 14 tar balls from source. There are also huge overlaps in > functionality, like Python and Gtk image support (jpeg etc.) > > comments? > Isn't Gtk under the LGPL? That pretty much kills it right there if it is. Besides Guido has made the comment publicly (last PyCON) that if he were to choose a GUI toolkit to bundle with Python it would be wxPython. So there is already one vote going with another toolkit. And the 14 tarball thing ain't our fault. =) -Brett From fperez528 at yahoo.com Fri Aug 6 19:50:25 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Fri Aug 6 19:50:32 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> Message-ID: Tim Peters wrote: > [Barry Warsaw] >> In summary, I can definitely sympathize with the concern of tool authors >> who have settled on @ as a meta-character that Python will not use. > > Yup. > >> I wonder how the Leo or ipython folks would feel if the only change to the >> current implementation of PEP 318 was to change the decorator >> introducing character. If we can increase the community's consensus by >> making this little change, I wouldn't oppose it. > > Me neither. > >> My preference then would be to use = with | and : as alternatives, in >> that order. Well, here's from the ipython author... As I've said, I can adapt to @, but it will mean a bunch of silly work (lots of documentation changes to track), and significant disruption to a large existing user base. Note that ipython has made it into SuSe Professional, and it's available for OSX via Fink, Debian, Gentoo, and several other smaller Linux distributions, as well as WinXP/2k. I mention this only to point out the extent of the community which will feel this disruption, not in trying to force anyone's hand by a 'mob argument'. If the 'decorated line before def' approach wins in Guido's mind, and the choice of decorator character is the only remaining question, I'd have to say that using | (or one of the others) would definitely make my life easier. And that of all my users as well. The visual effect of | when initially proposed looked surprisingly nice to me, given its 'diagrammatic' appearance of drawing a graph between the decoration lines and the function definition. So FWIW, I'd say that I personally kind of like it. Though I tend to prefer the []-after-args-before-colon one even better, but I don't want to get into the whole debate. I'm trying to confine my comments to the effects of this change on ipython, which seems to be (with Leo) one of the worst-hit third-party tools by the '@' proposal. I'd like to say that I greatly appreciate the fact that the python core team is willing to keep my concerns in mind, even when it was my own choice to use @ without any promises from them. I think in the middle of a discussion which has sometimes strayed from the technical to the emotional, it's worth recognizing the quality of the people we have the privilege of working with. Best to all, f From mmclay at comcast.net Fri Aug 6 20:06:31 2004 From: mmclay at comcast.net (Michael McLay) Date: Fri Aug 6 20:05:15 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061557.i76FvGo07481@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> Message-ID: <200408061406.31244.mmclay@comcast.net> On Friday 06 August 2004 11:57 am, Guido van Rossum wrote: > > So I spent a little time futzing with python-mode and some of my > > decorator code, to try some alternative leading characters. Of the ones > > that cannot be used in valid Python code today (i.e. no backward > > compatibility issues), I tried ':', '/', '=', '*', and '|'. > > If the community can rally behind one of these, I think that would be > acceptable. They all seem arbitrary, but so is the choice of '@'. :-) Why was ';' not on Barry's list? It looks like it could be safely used as a leading character. >> def f(): ... ; bar = 3 File "", line 2 ; bar = 3 ^ SyntaxError: invalid syntax >>> ; descriptor=42 File "", line 1 ; descriptor=42 ^ SyntaxError: invalid syntax Using ';' instead of '@' would also make creating a list of decorators on one line consistent with other uses of ';' since ';' is already used to indicate multiple statement on one line. ;framework_stuff(lots, of, args) class Quux(object): ;check_args(int, str) ;counted ;staticmethod def frobnicate(foo, bar): pass Admittedly the ';' does suffer from not being as visible as '@', but at least the tail on the bottom dot makes it more readable than the ':' character. The other downside is the potential confusion it might cause to geriatric elisp programmers. From anthony at interlink.com.au Fri Aug 6 20:08:00 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 20:08:14 2004 Subject: [Python-Dev] PEP-0006, Bug Fix Releases Message-ID: <4113C900.6060902@interlink.com.au> This is an updated version of PEP-0006, the Bugfix Release PEP. Unless I hear some screams, I'll be checking this into CVS. It more accurately reflects the current state of the bugfix process. (This has been sitting in my todo list for a while) -------------- next part -------------- PEP: 6 Title: Bug Fix Releases Version: $Revision: 1.10 $ Author: aahz@pobox.com (Aahz), anthony@interlink.com.au (Anthony Baxter) Status: Active Type: Informational Created: 15-Mar-2001 Post-History: 15-Mar-2001 18-Apr-2001 Abstract Python has historically had only a single fork of development, with releases having the combined purpose of adding new features and delivering bug fixes (these kinds of releases will be referred to as "feature releases"). This PEP describes how to fork off maintenance, or bug fix, releases of old versions for the primary purpose of fixing bugs. This PEP is not, repeat NOT, a guarantee of the existence of bug fix releases; it only specifies a procedure to be followed if bug fix releases are desired by enough of the Python community willing to do the work. Motivation With the move to SourceForge, Python development has accelerated. There is a sentiment among part of the community that there was too much acceleration, and many people are uncomfortable with upgrading to new versions to get bug fixes when so many features have been added, sometimes late in the development cycle. One solution for this issue is to maintain the previous feature release, providing bug fixes until the next feature release. This should make Python more attractive for enterprise development, where Python may need to be installed on hundreds or thousands of machines. Prohibitions Bug fix releases are required to adhere to the following restrictions: 1. There must be zero syntax changes. All .pyc and .pyo files must work (no regeneration needed) with all patch releases forked off from a feature release. 2. There must be zero pickle changes. 3. There must be no incompatible C API changes. All extensions must continue to work without recompiling in all patch releases in the same fork as a feature release. Breaking any of these prohibitions requires a BDFL proclamation (and a prominent warning in the release notes). Not-Quite-Prohibitions Where possible, bug fix releases should also: 1. Have no new features. The purpose of a bug fix release is to fix bugs, not add the latest and greatest whizzo feature from the HEAD of the CVS root. 2. Be a painless upgrade. Users should feel confident that an upgrade from 2.x.y to 2.x.(y+1) will not break their running systems. This means that, unless it is necessary to fix a bug, the standard library should not change behaviour, or worse yet, APIs. Helping the Bug Fix Releases Happen Here's a few pointers on helping the bug fix release process along. 1. Backport bug fixes. If you fix a bug, and it seems appropriate, port it to the CVS branch for the current bug fix release. If you're unwilling or unable to backport it yourself, make a note in the commit message. 2. If you're not sure, ask. Ask the person managing the current bug fix releases if they think a particular fix is appropriate. 3. If there's a particular bug you'd particularly like fixed in a bug fix release, jump up and down and try to get it done. Do not wait until 48 hours before a bug fix release is due, and then start asking for bug fixes to be included. Version Numbers Starting with Python 2.0, all feature releases are required to have a version number of the form X.Y; patch releases will always be of the form X.Y.Z. The current feature release under development is referred to as release N; the just-released feature version is referred to as N-1. In CVS, the bug fix releases happen on a branch. For release 2.x, the branch is named 'release2x-maint'. For example, the branch for the 2.3 maintenance releases is release23-maint Procedure The process for managing patch releases is modeled in part on the cl Tsystem [1]. The Patch Czar is the counterpart to the BDFL for patch releases. However, the BDFL and designated appointees retain veto power over individual patches. As individual patches get contributed to the feature release fork, each patch contributor is requested to consider whether the patch is a bug fix suitable for inclusion in a patch release. If the patch is considered suitable, the patch contributor will mail the SourceForge patch (bug fix?) number to the maintainers' mailing list. In addition, anyone from the Python community is free to suggest patches for inclusion. Patches may be submitted specifically for patch releases; they should follow the guidelines in PEP 3 [2]. In general, though, it's probably better that a bug in a specific release also be fixed on the HEAD as well as the branch. The Patch Czar decides when there are a sufficient number of patches to warrant a release. The release gets packaged up, including a Windows installer, and made public. If any new bugs are found, they must be fixed immediately and a new patch release publicized (with an incremented version number). Bug fix releases are expected to occur at an interval of roughly six months. This is only a guideline, however - obviously, if a major bug is found, a bugfix release may be appropriate sooner. In general, only the N-1 release will be under active maintenance at any time. That is, during Python 2.4's development, Python 2.3 gets bugfix releases. Patch Czar History Anthony Baxter is the Patch Czar for 2.3.1 through 2.3.4. Barry Warsaw is the Patch Czar for 2.2.3. Guido van Rossum is the Patch Czar for 2.2.2. Michael Hudson is the Patch Czar for 2.2.1. Anthony Baxter is the Patch Czar for 2.1.2 and 2.1.3. Thomas Wouters is the Patch Czar for 2.1.1. Moshe Zadka is the Patch Czar for 2.0.1. History This PEP started life as a proposal on comp.lang.python. The original version suggested a single patch for the N-1 release to be released concurrently with the N release. The original version also argued for sticking with a strict bug fix policy. Following feedback from the BDFL and others, the draft PEP was written containing an expanded patch release cycle that permitted any previous feature release to obtain patches and also relaxed the strict bug fix requirement (mainly due to the example of PEP 235 [3], which could be argued as either a bug fix or a feature). Discussion then mostly moved to python-dev, where BDFL finally issued a proclamation basing the Python patch release process on Tcl's, which essentially returned to the original proposal in terms of being only the N-1 release and only bug fixes, but allowing multiple patch releases until release N is published. Anthony Baxter then took this PEP and revised it, based on lessons from the 2.3 release cycle. References [1] http://dev.scriptics.com:8080/cgi-bin/tct/tip/28.html [2] PEP 3, Guidelines for Handling Bug Reports, Hylton http://www.python.org/peps/pep-0003.html [3] PEP 235, Import on Case-Insensitive Platforms, Peters http://www.python.org/peps/pep-0235.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End: From anthony at interlink.com.au Fri Aug 6 20:15:31 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 20:15:50 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061406.31244.mmclay@comcast.net> References: <200408051636.i75Gaak03654@guido.python.org> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <200408061406.31244.mmclay@comcast.net> Message-ID: <4113CAC3.1010603@interlink.com.au> Michael McLay wrote: > On Friday 06 August 2004 11:57 am, Guido van Rossum wrote: > >>>So I spent a little time futzing with python-mode and some of my >>>decorator code, to try some alternative leading characters. Of the ones >>>that cannot be used in valid Python code today (i.e. no backward >>>compatibility issues), I tried ':', '/', '=', '*', and '|'. >> >>If the community can rally behind one of these, I think that would be >>acceptable. They all seem arbitrary, but so is the choice of '@'. :-) > > > Why was ';' not on Barry's list? It looks like it could be safely used as a > leading character. One objection is that there's a number of contexts where ; is the comment notation. Another is, to me, the ';' just looks misplaced. Maybe it was supposed to be on the end of the line before, but there's a line break in the wrong place? I also don't think ; is that much better than : - I think it violates one of the unwritten bits of Python Zen - Syntax Shall Not Look Like Grit On Tim's Monitor. -- Anthony Baxter It's never too late to have a happy childhood. From pje at telecommunity.com Fri Aug 6 20:21:07 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 20:17:00 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <200408061555.i76FtV707448@guido.python.org> References: Message-ID: <5.1.1.6.0.20040806141835.032fed90@mail.telecommunity.com> At 08:55 AM 8/6/04 -0700, Guido van Rossum wrote: >It would seem that the functionality isn't quite as useful there; you >can get most of the same effects with metaclasses. I'm not aware of >any idiom that takes a class and passes it through some kind of >wrapper or transformation *after* the class declaration; if somebody >is using this, it surely must be a very rare use indeed. Zope and PyProtocols use this approach for inline interface declarations, though it's implemented by _getframe and a temporary metaclass. E.g. Zope's: class Foo: implements(IFoo) works by effectively applying a decorator to the finished class. PyProtocols doesn't change the class in the process, but I think Zope might (e.g. to set __implements__). From pje at telecommunity.com Fri Aug 6 20:23:39 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 20:19:32 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061557.i76FvGo07481@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> Message-ID: <5.1.1.6.0.20040806142156.03300620@mail.telecommunity.com> At 08:57 AM 8/6/04 -0700, Guido van Rossum wrote: > > So I spent a little time futzing with python-mode and some of my > > decorator code, to try some alternative leading characters. Of the ones > > that cannot be used in valid Python code today (i.e. no backward > > compatibility issues), I tried ':', '/', '=', '*', and '|'. > >If the community can rally behind one of these, I think that would be >acceptable. They all seem arbitrary, but so is the choice of '@'. :-) How about '%'? This: %classmethod looks more like some kind of pragma or compiler directive to me than '@', which looks more like some kind of documentation. From martin at v.loewis.de Fri Aug 6 20:22:41 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 20:22:47 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061634.i76GYEZ07623@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> Message-ID: <4113CC71.4070104@v.loewis.de> Guido van Rossum wrote: > Does that really have to be stated? Isn't this just giving in to the > silly lawyers? A reference manual describes the status quo. How can > it possibly begin to describe future directions? However, the Python reference manual became more than a reference manual, as it also talks about what a Python implementation can and cannot do, e.g. for memory management. Providing future directions in the reference manual is a good thing to do if possible as it helps competing implementations and tool vendors. Clearly, people do make assumptions that a documented aspect of the language is not going to change, unless the documentation says that it is likely to change. Regards, Martin From pje at telecommunity.com Fri Aug 6 20:28:30 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 20:24:21 2004 Subject: [Python-Dev] pie-thon redux In-Reply-To: <4113AE45.5080004@interlink.com.au> Message-ID: <5.1.1.6.0.20040806142605.03303d90@mail.telecommunity.com> At 02:13 AM 8/7/04 +1000, Anthony Baxter wrote: >As a break from the decorator thread, NTK (www.ntk.net) >had this writeup of the Pie-Thon. It amused me. > >[snip] I'm not usually one for today's IM slang, but... OMG!!!! RTFLOL!!! :) >Feel the punctuation rising in you, Guido!) And so, the '@' sign, symbol of the pie, came into being... ;) From martin at v.loewis.de Fri Aug 6 20:25:28 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 20:25:30 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <4113BC98.3020407@stackless.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> Message-ID: <4113CD18.8080803@v.loewis.de> Christian Tismer wrote: > the primary purpose of all the decorator syntax effort > is to get the > > foo = classmethod(foo) > > into the right place, not after but before the definition. > > Dunno if this was mentioned before, but why not simply do that? > Move the statement right before the def and provide a way > that this gets recognized correctly. > > No syntax change, just the convenience people asked for. I don't think this can work. How precisely would you implement it? Python traditionally executes source code from top to bottom, except for loops. Regards, Martin From John.Marshall at ec.gc.ca Fri Aug 6 20:25:52 2004 From: John.Marshall at ec.gc.ca (John Marshall) Date: Fri Aug 6 20:26:04 2004 Subject: [Python-Dev] Re: elements of decorator syntax suggestions In-Reply-To: References: Message-ID: <1091816752.3439.33.camel@mango.cmc.ec.gc.ca> On Fri, 2004-08-06 at 17:19, Steven Bethard wrote: > I think one of the biggest reasons we're having such problems coming > to any agreement on decorator syntax is that each proposal makes a > number of syntax decisions, not just one. For decorators, I see the > following decisions that must be made: > > 1) Indicator > > 2) Location > > 3) List notation > > 4) Indentation I think you have done a good job identifying and isolating what the issues are regarding decorator options. I think another thing that should perhaps be made clearer (maybe for me only?) is to identify, in order of importance/signfigance, the relationship between decorators and the function: 1) is the function/method signature more important than the decorators (to the programmer)? 2) do the decorators belong or are affected by the function? 3) what other constructs in Python say/imply "look ahead for what I am about to affect/define"? I understand that designing programming languages is sometimes an art rather than a science (natural languages a chock full of exceptions to the rule). But if the priority of functions and decorators are established, then I think this would help. In response to my own questions above: 1) From my perspective, it seems that almost everybody would say that the function (class also) signature is more important than decorators. For example, how many people would say "I wonder where the classmethod function is?" Instead, people would say, "Where is my function xyz()". This seems to argue for the decorators following (somewhere) after the signature. Of course highlighting editors can help by marking off function names and parameters, but this argument only serves to confirm that the signature is the KEY thing programmers are interested in. Apart from aesthetics, which are subjective and could be debated endlessly, I cannot see how one could argue otherwise. 2) Perhaps I am not understanding fully. Can someone explain how decorators before the function logically belong outside of the function definition block if they are intended to affect the function? Are the decorators meant to affect something other than the function? It seems to me that in every other situation in Python (please correct me if I am wrong) blocks are the way we are able to identify what belongs to what (e.g., local variable to a function). The current decorator approach says to me, "I belong to the class" (if a class is involved), or "I belong to the module" (if no class is involved). 3) In a sense, this point is somewhat related to #2. Does Python support an implicit "I am doing something now, on something I haven't defined, but which is coming up next" anywhere else than the decorator before function construction? Even in classes, Python forces the use of self.xxx (explicit). Hope this contributes to the discussion. John From skip at pobox.com Fri Aug 6 20:34:58 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 20:35:21 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408061658.i76Gwxf07727@guido.python.org> References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <017601c47bc7$2c84b720$6700a8c0@computer> <200408061537.i76Fbv207317@guido.python.org> <200408061658.i76Gwxf07727@guido.python.org> Message-ID: <16659.53074.263268.946749@montanaro.dyndns.org> Guido> Somebody should add to the PEP that solutions requiring an extra Guido> indent level for the function have a drawback. Done. Skip From martin at v.loewis.de Fri Aug 6 20:38:16 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 20:38:19 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <007d01c47bbc$49a2a340$6700a8c0@computer> References: <411371EC.10403@v.loewis.de> <007d01c47bbc$49a2a340$6700a8c0@computer> Message-ID: <4113D018.3020809@v.loewis.de> Edward K. Ream wrote: > I agree. The argument that @ lines aren't clearly "attached" to functions > seems much more convincing to me. I can't see that. In Python, things at the same indentation level all belong together. Would you say that in try: foo() more_code() except expr: bar() the except is not attached to the try? You probably wouldn't, because you know they belong together. However, you equally have no problem seeing how while: foo() more_code() if expr: bar() the while and the if don't belong together. People can learn these things, and learning the @ thing seems very easy. > 2. Maybe the @ syntax is somehow the best alternative, considered in > isolation. Fair enough. But how much better? Enough to justify a major > change to the language? What is so unreadable about def (...)[...]: ? That parameters and decorators are essentially in a single sequence, just separated by )[. It is important that you can easily tell how many parameters a function has, and how it is decorated. Furthermore, this syntax *will* break more tools than the @decorator syntax. > In short, a major change to Python (@) would seem to demand a truly > convincing justification. IMO, the argument that @ is more readable than > def (...)[...]: doesn't even come _close_ to such a justification. Take into account also breakage of existing tools. Regards, Martin From amk at amk.ca Fri Aug 6 20:42:47 2004 From: amk at amk.ca (A.M. Kuchling) Date: Fri Aug 6 20:43:03 2004 Subject: [Python-Dev] Merging Python and Gtk... In-Reply-To: <20040806173916.GA2212@divinian> References: <20040806173916.GA2212@divinian> Message-ID: <20040806184247.GA32440@rogue.amk.ca> On Fri, Aug 06, 2004 at 07:39:16PM +0200, Paul Sheer wrote: > If Python and Gtk installed as one package, this requirement > would be met. At the moment, getting Python to work with This isn't really a core Python issue; distributing the GTk+ source with Python isn't going to happen because of the license, because the version would have to be kept in sync, etc. However, there's nothing stopping someone from building their own Windows installer that includes GTk+ as well as (or in place of) Tkinter. PEP 206 was started to describe a richer Python distribution, but its author no longer has time to work on the task. You could certainly pick up the PEP and resume working on it. --amk From marktrussell at btopenworld.com Fri Aug 6 20:50:07 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Fri Aug 6 20:50:11 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> Message-ID: <1091818207.7529.33.camel@localhost> On Fri, 2004-08-06 at 18:50, Fernando Perez wrote: > The visual effect of | when initially proposed looked surprisingly nice to me > given its 'diagrammatic' appearance of drawing a graph between the decoration > lines and the function definition. So FWIW, I'd say that I personally kind of > like it. I too like |. It is not quite as visible as @, but I don't think that's a serious problem, and not breaking ipython and other tools is a big win. I just tried implementing this change, and all the tests still pass (not a big surprise, but worth confirming I guess). Mark From marktrussell at btopenworld.com Fri Aug 6 20:50:09 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Fri Aug 6 20:50:15 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> Message-ID: <1091818207.7529.34.camel@localhost> On Fri, 2004-08-06 at 18:50, Fernando Perez wrote: > The visual effect of | when initially proposed looked surprisingly nice to me > given its 'diagrammatic' appearance of drawing a graph between the decoration > lines and the function definition. So FWIW, I'd say that I personally kind of > like it. I too like |. It is not quite as visible as @, but I don't think that's a serious problem, and not breaking ipython and other tools is a big win. I just tried implementing this change, and all the tests still pass (not a big surprise, but worth confirming I guess). Mark From python at rcn.com Fri Aug 6 08:51:29 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 6 20:53:18 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061406.31244.mmclay@comcast.net> Message-ID: <016d01c47b81$cd859d60$e841fea9@oemcomputer> > > > that cannot be used in valid Python code today (i.e. no backward > > > compatibility issues), I tried ':', '/', '=', '*', and '|'. Is '!' an option? It has a certain beauty and it looks like it points at the following line. Raymond From skip at pobox.com Fri Aug 6 20:58:22 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 20:58:45 2004 Subject: [Python-Dev] Questions about '@' in pep 318 In-Reply-To: <4113D018.3020809@v.loewis.de> References: <411371EC.10403@v.loewis.de> <007d01c47bbc$49a2a340$6700a8c0@computer> <4113D018.3020809@v.loewis.de> Message-ID: <16659.54478.967147.267963@montanaro.dyndns.org> Martin> In Python, things at the same indentation level all belong Martin> together. I think the point is that the collection of @-decorators and the immediately following function definition (which can be separated from each other by a lot of white space or comments, btw) effectively form a single function definition. I don't believe there are any other syntactic constructs in Python where the keyword introducing the construct ("def" in this case) doesn't occur at the beginning of the construct. Hmmm... Now that I think about it, might you "lose" an @ in a comment block since # and @ have more-or-less the same density? | might not suffer from that sort of confusion. # description of what decorator1 does # description of what decorator1 does # description of what decorator1 does @ decorator1 # description of what decorator2 does # description of what decorator2 does # description of what decorator2 does @ decorator2 # description of what func does # description of what func does # description of what func does def func(...): pass vs. # description of what decorator1 does # description of what decorator1 does # description of what decorator1 does | decorator1 # description of what decorator2 does # description of what decorator2 does # description of what decorator2 does | decorator2 # description of what func does # description of what func does # description of what func does def func(...): pass Eh, maybe not. Small point in any case, I'm sure. Skip From martin at v.loewis.de Fri Aug 6 20:59:06 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 20:59:09 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <016d01c47b81$cd859d60$e841fea9@oemcomputer> References: <016d01c47b81$cd859d60$e841fea9@oemcomputer> Message-ID: <4113D4FA.6090805@v.loewis.de> Raymond Hettinger wrote: > Is '!' an option? It has a certain beauty and it looks like it points > at the following line. Syntactically, yes. It reminds me of comments, though (e.g. in Xresource files). Regards, Martin From skip at pobox.com Fri Aug 6 21:01:14 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 21:01:31 2004 Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it? In-Reply-To: References: Message-ID: <16659.54650.788854.375861@montanaro.dyndns.org> >From Raymond's checkin: Programming Recommendations - Code should be written in a way that does not disadvantage other implementations of Python (PyPy, Jython, IronPython, Pyrex, Psyco, and such). For example, do not rely on CPython's efficient implementation of in-place string concatenation for statements in the form a+=b or a=a+b. Those statements run more slowly in Jython. In performance sensitive parts of the library, the ''.join() form should be used instead. This will assure that concatenation occurs in linear time across various implementations. I'm a bit confused. Why bother to speed up s+=t if we aren't supposed to use it? Skip From anthony at interlink.com.au Fri Aug 6 21:02:57 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 6 21:04:39 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <1091818207.7529.34.camel@localhost> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> Message-ID: <4113D5E1.6040006@interlink.com.au> Mark Russell wrote: > I too like |. It is not quite as visible as @, but I don't think that's > a serious problem, and not breaking ipython and other tools is a big > win. I just tried implementing this change, and all the tests still > pass (not a big surprise, but worth confirming I guess). To jump on the band-wagon - I can live with | as well. I was going to add it to the PEP rewrite, but I'll wait for a decision, to save myself the writing ;) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From pje at telecommunity.com Fri Aug 6 21:09:54 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 6 21:05:46 2004 Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it? In-Reply-To: <16659.54650.788854.375861@montanaro.dyndns.org> References: Message-ID: <5.1.1.6.0.20040806150836.02729ec0@mail.telecommunity.com> At 02:01 PM 8/6/04 -0500, Skip Montanaro wrote: >For example, do not rely on CPython's efficient implementation > of in-place string concatenation for statements in the form a+=b or > a=a+b. Those statements run more slowly in Jython. Is that really true? I thought the last time this came up, somebody said that common Java implementations actually had special case code to speed up string concatenation. From barry at python.org Fri Aug 6 21:09:52 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 21:09:44 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061539.i76FdtG07333@guido.python.org> References: <59e9fd3a04080519462bf57899@mail.gmail.com> <59e9fd3a04080519462bf57899@mail.gmail.com> <5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com> <200408061539.i76FdtG07333@guido.python.org> Message-ID: <1091819392.10217.3.camel@localhost> On Fri, 2004-08-06 at 11:39, Guido van Rossum wrote: > That's a bit strong, but yes, it would be trivial to change. If > that's all that's needed to save @deco, it's a lot easier than ripping > the whole thing out... :-) I haven't played with many multi-decorator decorators, but the fact that you can have more than one of them on a line without an intervening semi-colon doesn't seem right to me. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/58dafc0b/attachment.pgp From skip at pobox.com Fri Aug 6 21:11:35 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 21:11:57 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <016d01c47b81$cd859d60$e841fea9@oemcomputer> References: <200408061406.31244.mmclay@comcast.net> <016d01c47b81$cd859d60$e841fea9@oemcomputer> Message-ID: <16659.55271.674627.269077@montanaro.dyndns.org> Raymond> Is '!' an option? It has a certain beauty and it looks like it Raymond> points at the following line. \N{DOWNWARDS ARROW} anyone? Skip From python at rcn.com Fri Aug 6 09:32:21 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 6 21:34:09 2004 Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it? In-Reply-To: <16659.54650.788854.375861@montanaro.dyndns.org> Message-ID: <018101c47b87$82a884a0$e841fea9@oemcomputer> > I'm a bit confused. Why bother to speed up s+=t if we aren't supposed to > use it? In fact, it is does occur all over the place (esp. in newbie code) so it ought to run as fast as possible or least not be a total disaster. OTOH, library code gets run on various implementations of Python and it would be a mistake to change any library code from ''.join() and have it start performing badly on other implementations. That was essentially the only argument against the patch which is otherwise a pure win. On a side note, the += optimization only works if there is space (fragmentationwise, not total memory) to resize a string in-place. While this is usually true, ''.join() always runs in linear time. Also, ''.join() doesn't run through the eval-loop. IOW, notwithstanding cross-implementation performance, ''.join() is usually a better choice in performance sensitive code. The goal of the patch is not to change coding styles. Rather, the goal is to have it hurt less when you coded += anyways. Raymond From bob at redivi.com Fri Aug 6 21:34:01 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 6 21:34:13 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <4113D5E1.6040006@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> Message-ID: <91F184DE-E7DF-11D8-ADD9-000A95686CD8@redivi.com> On Aug 6, 2004, at 3:02 PM, Anthony Baxter wrote: > Mark Russell wrote: >> I too like |. It is not quite as visible as @, but I don't think >> that's >> a serious problem, and not breaking ipython and other tools is a big >> win. I just tried implementing this change, and all the tests still >> pass (not a big surprise, but worth confirming I guess). > > To jump on the band-wagon - I can live with | as well. I was going to > add it to the PEP rewrite, but I'll wait for a decision, to save myself > the writing ;) My only objection to | is that it looks near-identical to I in many fonts, such as the one I use for email, so it could potentially be confused with some sort of magical sys._getframe() using interface declaration rather than compile-type syntax. -bob From barry at python.org Fri Aug 6 21:39:53 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 21:39:43 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408061406.31244.mmclay@comcast.net> References: <200408051636.i75Gaak03654@guido.python.org> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <200408061406.31244.mmclay@comcast.net> Message-ID: <1091821193.10211.11.camel@localhost> On Fri, 2004-08-06 at 14:06, Michael McLay wrote: > Why was ';' not on Barry's list? It looks like it could be safely used as a > leading character. It wasn't because he didn't think of it . > Admittedly the ';' does suffer from not being as visible as '@', but at least > the tail on the bottom dot makes it more readable than the ':' character. > > The other downside is the potential confusion it might cause to geriatric > elisp programmers. I resemble that remark. :) But really, at the very least, semis suffer from readability problems similar to colons. bellying-up-to-the-bar-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/bd0df16a/attachment.pgp From tim.peters at gmail.com Fri Aug 6 21:41:19 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 6 21:41:22 2004 Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it? In-Reply-To: <5.1.1.6.0.20040806150836.02729ec0@mail.telecommunity.com> References: <5.1.1.6.0.20040806150836.02729ec0@mail.telecommunity.com> Message-ID: <1f7befae040806124128872bc5@mail.gmail.com> >> For example, do not rely on CPython's efficient implementation >> of in-place string concatenation for statements in the form a+=b or >> a=a+b. Those statements run more slowly in Jython. [Phillip J. Eby] > Is that really true? I thought the last time this came up, somebody said > that common Java implementations actually had special case code to speed up > string concatenation. Java != Jython. Java compilers know when they're dealing with strings, and the Java spec defines '+' on strings in terms of mutable-StringBuffer operations. Even so, the pragmatics of string concatenation in Java are complex, well illustrated by this excruciating "tech tip": http://java.sun.com/developer/JDCTechTips/2002/tt0305.html Jython is as blind as Python at compile-time about what the types are in "a=a+b". From skip at pobox.com Fri Aug 6 21:45:43 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 21:46:01 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings In-Reply-To: <4112A968.2030401@v.loewis.de> References: <16658.42673.542930.749915@montanaro.dyndns.org> <4112A968.2030401@v.loewis.de> Message-ID: <16659.57319.931941.158458@montanaro.dyndns.org> >> Unfortunately GCC also defines it, so a "macro redefined" warning is >> emitted when compiling with GCC and including Python.h. Martin> That is not true. GCC does not define that macro, on any Martin> platform. I was shown today that g++ defines that macro internally (I misspoke when I referred to gcc yesterday - I tend to lump them together and forget that C++ is a different language than C), at least 3.3.2 on Solaris/Intel does. Given this small source file: #include "Python.h" int main() { printf("%x\n", PyList_GetItem); return 0; } compilation with gcc (we're using 3.3.2, but I also tried with 3.4.0) produces no warnings, but compilation with g++ it produces this output: g++ -I/opt/lang/python/include/python2.3 -c example.c In file included from /opt/lang/python/include/python2.3/Python.h:8, from example.c:2: /opt/lang/python/include/python2.3/pyconfig.h:862:1: warning: "_XOPEN_SOURCE" redefined :73:1: warning: this is the location of the previous definition Sure enough, grepping through the gcc/g++ installation directory doesn't turn up an actual header file containing "#define _XOPEN_SOURCE". I did find a file (.../lib/gcc/i386-pc-solaris2.8/3.4.0/install-tools/ mkheaders.conf) that contains this line: FIXPROTO_DEFINES="-D_XOPEN_SOURCE" Martin> Some system header may define it, though. I believe this is a Martin> bug in the system - the macro *should* be defined in the Martin> application. What system are you referring to, and where does Martin> that define _XOPEN_SOURCE? Is that definition conditional? Does the fact that we're talking g++ and a built-in macro instead of gcc and a macro defined in a .h file suggest a different way to suppress this warning? A little more searching located the standards(5) man page on the system. In part it says: X/Open CAE To build or compile an application that conforms to one of the X/Open CAE specifications, use the following guidelines. Applications need not set the POSIX feature test macros if they require both CAE and POSIX functionality. ... SUSv2 The application must define _XOPEN_SOURCE=500. This suggests that Sun expects the application or its build tools to define _XOPEN_SOURCE, not the compiler or its header files. That suggests a bug in g++. Skip From skip at pobox.com Fri Aug 6 21:50:18 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 6 21:51:00 2004 Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it? In-Reply-To: <018101c47b87$82a884a0$e841fea9@oemcomputer> References: <16659.54650.788854.375861@montanaro.dyndns.org> <018101c47b87$82a884a0$e841fea9@oemcomputer> Message-ID: <16659.57594.110222.939272@montanaro.dyndns.org> >> I'm a bit confused. Why bother to speed up s+=t if we aren't >> supposed to use it? Raymond> In fact, it is does occur all over the place (esp. in newbie Raymond> code) so it ought to run as fast as possible or least not be a Raymond> total disaster. ... Thanks for the explanation. Viewing it in a larger context than just the checkin diff makes it a bit more clear that this doesn't refer to garden variety application code as well. Skip From martin at v.loewis.de Fri Aug 6 22:04:49 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 22:04:52 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings In-Reply-To: <16659.57319.931941.158458@montanaro.dyndns.org> References: <16658.42673.542930.749915@montanaro.dyndns.org> <4112A968.2030401@v.loewis.de> <16659.57319.931941.158458@montanaro.dyndns.org> Message-ID: <4113E461.20600@v.loewis.de> Skip Montanaro wrote: > I was shown today that g++ defines that macro internally (I misspoke when I > referred to gcc yesterday - I tend to lump them together and forget that C++ > is a different language than C), at least 3.3.2 on Solaris/Intel does. Ah, that one. I don't think it is a goal that Python generates no warnings if compiled with a C++ compiler (I don't think you actually *can* compile all of Python with a C++ compiler). > This suggests that Sun expects the application or its build tools to define > _XOPEN_SOURCE, not the compiler or its header files. That suggests a bug in > g++. Indeed, and I believe this is a known bug. g++ needs to define _XOPEN_SOURCE so that some of Sun's system headers declare a number of prototypes which are then referred-to in inline functions of libstdc++ header files; I think this is about wchar_t functions, thread functions, or C99 floating-point functions. It's not clear whether this is Sun's or g++' fault - g++ should probably fixinclude the system headers appropriately, but doesn't. This gives us a number of options: 1. On Solaris, define _XOPEN_SOURCE to the same value that g++ uses. This should then cause no redefinition, but might cause some functions go away. We would need an autoconf test to find out what the value of _XOPEN_SOURCE is that g++ uses. 2. As g++ is only used to compile main.cc, we could #undef _XOPEN_SOURCE there - either the one of g++, or our own one (appropriately commented, of course). 3. We could advise users to explicitly compile --without-cxx on Solaris when using g++. I believe it is not actually needed, but our autoconf test is not strong enough to find out, and errs on the side of building main.cc with a C++ compiler if one is available and the user hasn't requested that it won't. Regards, Martin From martin at v.loewis.de Fri Aug 6 22:07:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 22:07:50 2004 Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it? In-Reply-To: <16659.54650.788854.375861@montanaro.dyndns.org> References: <16659.54650.788854.375861@montanaro.dyndns.org> Message-ID: <4113E512.8050407@v.loewis.de> Skip Montanaro wrote: >>From Raymond's checkin: > > Programming Recommendations > > - Code should be written in a way that does not disadvantage other > implementations of Python (PyPy, Jython, IronPython, Pyrex, Psyco, and > such). For example, do not rely on CPython's efficient implementation > of in-place string concatenation for statements in the form a+=b or > a=a+b. Those statements run more slowly in Jython. In performance > sensitive parts of the library, the ''.join() form should be used > instead. This will assure that concatenation occurs in linear time > across various implementations. > > I'm a bit confused. Why bother to speed up s+=t if we aren't supposed to > use it? Because it helps users who aren't aware of the problem, and use the obvious approach. Developers of the standard libraries are supposed to know, and use the "better, but non-obvious" technique. Regards, Martin From barry at python.org Fri Aug 6 22:15:47 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 6 22:15:38 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <4113D5E1.6040006@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> Message-ID: <1091823347.10212.17.camel@localhost> On Fri, 2004-08-06 at 15:02, Anthony Baxter wrote: > To jump on the band-wagon - I can live with | as well. I was going to > add it to the PEP rewrite, but I'll wait for a decision, to save myself > the writing ;) Sure, me too (obviously). In a perfect world, I'd still prefer @, but if using | puts this issue to bed, +1 from me. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/762607dd/attachment.pgp From tim at zope.com Fri Aug 6 22:30:49 2004 From: tim at zope.com (Tim Peters) Date: Fri Aug 6 22:31:25 2004 Subject: [Python-Dev] Would like to add Edward Loper as a Python developer In-Reply-To: <41137F95.4040606@neggie.net> Message-ID: <20040806203049.AD9843B8038@smtp.zope.com> [John Belmonte] > I noticed that the "whole file doctests" item didn't make it to the > DoctestIdeas page. There's more on the wiki now, but no real details. Code for it exists in the doctest.py now on tim-doctest-branch. > I was looking for more details about that. Here's the class documentation. In context (but not in isolation as here), it's clear that it creates a unittest.TestSuite: def DocFileSuite(*paths, **kw): """Creates a suite of doctest files. One or more text file paths are given as strings. These should use "/" characters to separate path segments. Paths are relative to the directory of the calling module, or relative to the package passed as a keyword argument. A number of options may be provided as keyword arguments: package The name of a Python package. Text-file paths will be interpreted relative to the directory containing this package. The package may be supplied as a package object or as a dotted package name. setUp The name of a set-up function. This is called before running the tests in each file. tearDown The name of a tear-down function. This is called after running the tests in each file. globs A dictionary containing initial global variables for the tests. """ For concrete examples, put the Zope3 source code on a wall, and throw darts at it. > I've found that for tutorial narratives, I really don't want to bother > with expected output. Do you show any output at all? If not, this isn't for you. > For complex software, the output may not always be the same anyway. We want to focus on invariant (guaranteed) behavior in tests. I think the same is true of good docs. If you can't say anything about the output, you can't test it, and the documentation must be fuzzy too. > Also, it's not fun to manipulate input code that has prompts and is > within a multi-line string. The files DocFileSuite chews on are text files, not necessarily acceptable input to Python (in fact, they never are in practice). So the multi-line string concern doesn't exist in these. Mucking with prompts is indeed a price of entry. > Syntax highlighting is not available, for one thing. That depends on the editor, but I agree it's at best clumsy to set up. > I've made my own tutorial generator. Given this input script: > > ## Double hash comments treated as meta comments and not output to > ## the tutorial. > > """ > Triple-quote comments that start in the first column are treated as > tutorial text. > > First we'll import some modules: > """ > import os > > """ > Some code follows. In addition to hash comments, and triple-quote > comments not starting in the first column are preserved. Note that > code blocks such as functions must end with a blank line, just as > is done in the interactive interpreter. > """ > > def foo(): > """foo description here""" > print 'hello' # be friendly > > #my_path = 'foo/bar' > my_path = os.path.join('foo', 'bar') > my_path > foo() > > the output is: > > Triple-quote comments that start in the first column are treated as > tutorial text. > > First we'll import some modules: > > >>> import os > > Some code follows. In addition to hash comments, and triple-quote > comments not starting in the first column are preserved. Note that > code blocks such as functions must end with a blank line, just as > is done in the interactive interpreter. > > >>> def foo(): > ... """foo description here""" > ... print 'hello' # be friendly > ... > >>> #my_path = 'foo/bar' > >>> my_path = os.path.join('foo', 'bar') > >>> my_path > 'foo/bar' > >>> foo() > hello So if you put your output in a file, you could feed it exactly as-is to DocFileSuite(), and it would create a unittest for you that verified the output in your examples is in fact what it claims to be. From aahz at pythoncraft.com Fri Aug 6 22:40:36 2004 From: aahz at pythoncraft.com (Aahz) Date: Fri Aug 6 22:40:38 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> Message-ID: <20040806204035.GA18606@panix.com> On Fri, Aug 06, 2004, Jeremy Hylton wrote: > On Fri, 06 Aug 2004 07:45:51 -0700, Guido van Rossum wrote: >> [Anthony] >>> >>> I'm having trouble finding a single piece from you where you >>> outline the problems you have with the [decorators] after args, >>> before :. Could you possibly post a message to python-dev >>> stating your problems with it? It's the major one I've been >>> unable to find your objections to. >> >> And nobody else has expressed their objections to it either? :-) > > I thought it was obvious that this option was not acceptable. It's not obvious to me, but given Guido's concerns, what are the arguments against def foo [deco1(xyz), deco2] (arg1, arg2, *args, **kwargs): Since all functions MUST have a parameter list, everyone knows to keep scanning after the decorators. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From martin at v.loewis.de Fri Aug 6 22:51:53 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 6 22:51:55 2004 Subject: [Python-Dev] PyRun_ with file name Message-ID: <4113EF69.2020102@v.loewis.de> There is a constant issue of mixing CRTs on Windows. Patch 1003535 proposes to work around them by adding yet another PyRun_ function. I'm in favour, except that: - it should be called PyRun_Filename - it should have the maximum current parameter set, i.e. including CompilerFlags* I'm doubtful as to whether specifying an opening mode is necessary. What do you think? Martin From guido at python.org Fri Aug 6 23:16:22 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 6 23:16:28 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: Your message of "Fri, 06 Aug 2004 16:40:36 EDT." <20040806204035.GA18606@panix.com> References: <41136453.1040908@interlink.com.au> <200408061445.i76EjqT07026@guido.python.org> <20040806204035.GA18606@panix.com> Message-ID: <200408062116.i76LGMO08465@guido.python.org> > It's not obvious to me, but given Guido's concerns, what are the > arguments against > > def foo [deco1(xyz), deco2] (arg1, arg2, *args, **kwargs): Didn't you see my keynote at PyCon? I showed all 5 places where [deco] can be inserted in a function declaration (before the colon), because all 5 had been proposed, and 4 out of 5 had been rejected by then. > Since all functions MUST have a parameter list, everyone knows to keep > scanning after the decorators. Pretty much the same arguments apply as with the decorators after the argument list, plus some people like to grep for the function name followed by a left paren. --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Aug 6 11:17:55 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 6 23:19:47 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <1091823347.10212.17.camel@localhost> Message-ID: <019d01c47b96$429cce20$e841fea9@oemcomputer> > On Fri, 2004-08-06 at 15:02, Anthony Baxter wrote: > > > To jump on the band-wagon - I can live with | as well. I was going to > > add it to the PEP rewrite, but I'll wait for a decision, to save myself > > the writing ;) > > Sure, me too (obviously). In a perfect world, I'd still prefer @, but > if using | puts this issue to bed, +1 from me. FWIW, +1. Raymond From pyth at devel.trillke.net Sat Aug 7 00:05:42 2004 From: pyth at devel.trillke.net (Holger Krekel) Date: Sat Aug 7 00:05:45 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <019d01c47b96$429cce20$e841fea9@oemcomputer> References: <1091823347.10212.17.camel@localhost> <019d01c47b96$429cce20$e841fea9@oemcomputer> Message-ID: <20040806220542.GE5208@solar.trillke> Raymond Hettinger wrote: > > On Fri, 2004-08-06 at 15:02, Anthony Baxter wrote: > > > > > To jump on the band-wagon - I can live with | as well. I was going > to > > > add it to the PEP rewrite, but I'll wait for a decision, to save > myself > > > the writing ;) > > > > Sure, me too (obviously). In a perfect world, I'd still prefer @, but > > if using | puts this issue to bed, +1 from me. > > FWIW, +1. | FWIW def holger(): if anything: return +1 From jepler at unpythonic.net Sat Aug 7 00:09:26 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Sat Aug 7 00:09:53 2004 Subject: [Python-Dev] PyRun_ with file name In-Reply-To: <4113EF69.2020102@v.loewis.de> References: <4113EF69.2020102@v.loewis.de> Message-ID: <20040806220926.GA2225@unpythonic.net> [python.org/sf/1003535] Another alternative would be to complete the PyFile_* family of APIs (PyFile_Seek, etc), so that the C runtime routines would never be used. The "new" function in the PyRun_ family would take a PyFileObject*. Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/ce72a219/attachment.pgp From ncoghlan at iinet.net.au Sat Aug 7 00:55:51 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Aug 7 00:56:09 2004 Subject: [Python-Dev] python-dev Summary for 2004-07-16 through 2004-07-31 [draft] In-Reply-To: <4111DC2C.8070302@ocf.berkeley.edu> References: <4111DC2C.8070302@ocf.berkeley.edu> Message-ID: <41140C77.1040802@iinet.net.au> Brett Cannon wrote: > ------------------------------------------------------------------------------------------------------------------------------------------ > > How to get Python to compile with Microsoft's free compiler that should > just come with the OS standard > ------------------------------------------------------------------------------------------------------------------------------------------ > > 1. Download the free .NET compiler > 2. Download the Windows SDK (at > http://www.microsoft.com/msdownload/platformsdk/sdkupdate/ with only IE) > 3. Download Garth's tools to generate a Makefile from the .sln files (at > http://mail.python.org/pipermail/python-dev/2004-February/042595.html ) > 4. Compile > 5. Realize you should be using an OS that doesn't make you go through > this many hoops just to have a compiler for your platform > > Contributing threads: > - `Non-Visual Studio builds on Windows XP? > `__ Actually, step 4 is more complicated than that ;) Which was why I abandoned it at that point. . . and then discovered that my laptop's modem and built-in wireless don't work properly in Linux, unless I feel like paying for the Linuxant drivers, or getting coLinux to work (I'm partway through the latter with Fedora Core 3, since there won't be Linuxant drivers for Core 3 until it gets released). So. . . 4a. Get MINGW, since Garth's tool uses make, not nmake. Either that, or write the nmake makefile generation module for Garth's tool. Guess which one is likely to be less work ;) (especially since I already had mingw) 4b. Edit the .vcproj files and the .py files in Garth's tool to remove all references to 'odbc32.lib' and 'odbccp32.lib', since the free tools don't have those, and Python doesn't seem to actually need them (Visual Studio links to them by default). 4c. In gnu_make.py, line 49, change it to be: write(escape(makedef['all'])) (The version in the python-dev is missing the escape, which meant make couldn't match up the targets properly) 4d. Run Garth's tool to generate your makefiles for either debug or release. 4e. Set up a build environment with PATH, INCLUDE and LIB set correctly 4f. Run 'make -f pcbuild.gnu.mak python' 4g. Try to run the tests, and see the following: C:\DEVELO~1\projects\python\dist\src\PCbuild>python ..\PC\testpy.py 'import site' failed; use -v for traceback Traceback (most recent call last): File "..\PC\testpy.py", line 31, in ? import regrtest # Standard Python tester. File "C:\DEVELO~1\projects\python\dist\src\lib\test\regrtest.py", line 126, in ? from test import test_support File "C:\DEVELO~1\projects\python\dist\src\lib\test\test_support.py", line 154, in ? TESTFN_UNICODE_UNENCODEABLE.encode("Latin1") File "C:\development\projects\python\dist\src\lib\encodings\__init__.p y", line 30, in ? import codecs, exceptions, types, aliases File "C:\development\projects\python\dist\src\lib\codecs.py", line 76, in ? class Codec: File "C:\development\projects\python\dist\src\lib\codecs.py", line 76, in Codec class Codec: SystemError: no locals found when storing '__module__' Or, in debug: C:\DEVELO~1\projects\python\dist\src\PCbuild>python_d ..\PC\testpy.py 'import site' failed; use -v for traceback Traceback (most recent call last): File "..\PC\testpy.py", line 31, in ? import regrtest # Standard Python tester. File "C:\DEVELO~1\projects\python\dist\src\lib\test\regrtest.py", line 126, in ? from test import test_support File "C:\DEVELO~1\projects\python\dist\src\lib\test\test_support.py", line 154, in ? TESTFN_UNICODE_UNENCODEABLE.encode("Latin1") File "C:\development\projects\python\dist\src\lib\encodings\__init__.p y", line 30, in ? import codecs, exceptions, types, aliases File "C:\development\projects\python\dist\src\lib\codecs.py", line 76, in ? class Codec: File "C:\development\projects\python\dist\src\lib\codecs.py", line 76, in Codec class Codec: SystemError: no locals found when storing '__module__' [8229 refs] Still, even building is progress, right? Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From tismer at stackless.com Sat Aug 7 01:04:22 2004 From: tismer at stackless.com (Christian Tismer) Date: Sat Aug 7 01:01:53 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <4113CD18.8080803@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> Message-ID: <41140E76.2050601@stackless.com> Martin v. L?wis wrote: > Christian Tismer wrote: > >> the primary purpose of all the decorator syntax effort >> is to get the >> >> foo = classmethod(foo) >> >> into the right place, not after but before the definition. >> >> Dunno if this was mentioned before, but why not simply do that? >> Move the statement right before the def and provide a way >> that this gets recognized correctly. >> >> No syntax change, just the convenience people asked for. > > > I don't think this can work. How precisely would you implement > it? Python traditionally executes source code from top to bottom, > except for loops. Hey, this is an implementation detail, isn't it? :-) I can think of a couple of ways to implement it. One would be to defer execution of a statement with undefined names until the end of the whole code block in the case of a class definition and to resolve the names late. Cleaner would probably be to have something like a placeholder for the undefined function, assign to the name a special object and trap the def statement to resolve the object. (this comes to me since I'm just dealing with reference objects in PDF files, which are filled in later). No, I was really not concerned about how to do this, surely it won't work with plain sequential execution, but other proposals (which Guido considered once) like just putting [decorator] in front of things also needs some (simpler of course) special handling. At the risk of getting slogged, I also thought of myfunc := classmethod(myfunc) for a short time, with ":=" as a late binding assignment operator which defers its operation until myfunc is defined, but I think this is not a popular idea. :-) To save *some* typing (laziness was also a driver for this whole decorator nightmare), we could also remove the assignment and just have a function call like classmethod(myfunc) which reduces the re-typing to two. Anyway, compared to the expected effect, this whole discussion appears worse to me than the ternary expression thread last year, where one at least could learn something new. Fine that a?b:c and friends were put down. I really hate to see so much time wasted just for decorators. I could live with a special rule like my little proposal. I also agree that all the syntax pirouettes tried so far just add more ugliness to the language, just for nothing. Using a single char as a prefix might be a compromise after all. But must it be "@", such an eye-catching, ugly piece of flie-dirt? Looks like macro-alert, hey we get into special mode and decorate tokens in special contexts for special meaning. This is even worse than what Prothon tries with ? and !. If @, why not $, this might attract Perl users to Python. If we are getting perverse, why not simply go and add decorators using XML syntax? I'd-like-to-ask-green-peace-to-protect-my-language -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Sat Aug 7 01:20:53 2004 From: tismer at stackless.com (Christian Tismer) Date: Sat Aug 7 01:18:30 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <4113D5E1.6040006@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> Message-ID: <41141255.5010408@stackless.com> Anthony Baxter wrote: > Mark Russell wrote: > >> I too like |. It is not quite as visible as @, but I don't think that's >> a serious problem, and not breaking ipython and other tools is a big >> win. I just tried implementing this change, and all the tests still >> pass (not a big surprise, but worth confirming I guess). > > > To jump on the band-wagon - I can live with | as well. I was going to > add it to the PEP rewrite, but I'll wait for a decision, to save myself > the writing ;) Ok, I dislike special prefix chars at all, in a language that doesn't have this concept elsewhere (despite strings of course, but their prefixes are just regular chars), but the bar "|" appears a lot nicer than "@" to me. The bar is small and friendly, a small character for a small feature. With "@", I associate a powerful, magical Voodoo thing, something that I never expected to see in Python. Unless there is a real need, I'd say save "@" for a feature that really deserves such a powerful character. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From nhodgson at bigpond.net.au Sat Aug 7 01:27:02 2004 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Sat Aug 7 01:27:07 2004 Subject: [Python-Dev] 318: Annotation background Message-ID: <02e801c47c0c$e129eed0$e615a88d@neil> It appears from the discussion and PEP that there has not been a lot of experience here with the Java and C# analogues of decorators. In Java, especially server edition features like Entity Java Beans, there is a requirement for large volumes of supporting files for a system. Some of these are Java files defining a set of related classes and some are XML files defining how these can be used. For example, say I have two database tables, Master and Detail, defined as expected. These will need to be defined as Java classes, with fields corresponding to the database columns. Extra fields will be needed for transient attributes, some of which will be defined by the developer and others by the infrastructure such as dirty bits, transaction IDs, and remote connection handles. As well as the main class, there is a need for derived classes such as a remote interface that looks like the class, a value class that contains the fields (or a subset of the fields) that can be sensibly copied together to a remote system, and views of the data that turn relational artifacts like keys into a more object-oriented form, such as providing a pointer from a Detail to a Master or having the Detail records appear to be a list on the Master object. Maintaining all these files requires far too much work and you will always be coping with updates that forgot one file. Then along came XDoclet [1]. XDoclet uses doc comment tags to move the information about associated source and XML files into the main class file. For example, /** * @ejb.bean * type="CMP" * name="Master" */ public abstract class MasterEJB implements EntityBean { /** * @ejb.value-object * aggregate="example.vo.DetailValue" * aggregate-name="Details" * type="Collection" * @ejb.relation * name="Master-Detail" */ public abstract Collection getDetails(); public abstract void setDetails(Collection details); However, comments should be for documentation, not implementation code, and these 'comments' are now some of the most important and most difficult to maintain parts of the source. So there was a desire to move this into the main language where there could be more consistency and earlier checking for correctness. This has led to JSR 175 [2] which allows @ annotations to be applied directly to identifiers rather than indirectly through preceding comments. [1] http://xdoclet.sourceforge.net/xdoclet/index.html [2] http://www.jcp.org/aboutJava/communityprocess/review/jsr175/ Neil From tismer at stackless.com Sat Aug 7 01:29:39 2004 From: tismer at stackless.com (Christian Tismer) Date: Sat Aug 7 01:27:19 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <20040805194523.GA20561@mems-exchange.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com> <20040805191710.GD29796@burma.localdomain> <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> <20040805194523.GA20561@mems-exchange.org> Message-ID: <41141463.9050500@stackless.com> Neil Schemenauer wrote: > On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote: > >>On 5-aug-04, at 21:17, Gustavo Niemeyer wrote: >> >>> def funcname(...): >>> ... >>> funcname.signature = "v@:@i" >> >>That should be workable for this specific example. > > > Even nicer if '_' is bound to the last function defined. > > class SomeClass(objc): > def funcname(...): > ... > _.signature = "v@:@i" What if, by some special magic, the *next* function defined were bound to '_' ? This would give us the desired prefix behavior. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Sat Aug 7 01:38:30 2004 From: tismer at stackless.com (Christian Tismer) Date: Sat Aug 7 01:36:04 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> Message-ID: <41141676.6060602@stackless.com> Ronald Oussoren wrote: ... > Whoops, I used the wrong word, I meant 'generic functions' instead of > 'generic example'. He's doing something like this: > > @when("isinstance(db, OracleDB") > def storeInDB(db, object): > pass > > @when("isinstance(db, BerkelyDB") > def storeInDB(db, object): > pass Why is this needed? Can't we continue to use if isinstance(db, OracleDB): def storeInDB(db, object): pass if isinstance(db, BerkelyDB): def storeInDB(db, object): pass -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mkc at mathdogs.com Sat Aug 7 01:46:27 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sat Aug 7 01:46:32 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <2mbrho5o6c.fsf@starship.python.net> Message-ID: Michael Hudson writes: > Generally, it strikes me as mildly useful. An implementation would > surely help :-) If yours is the most positive review, there probably won't be one. :-) Seriously, though, I'd readily concede that there are different approaches to solving this problem, and I was hoping for some feedback and suggestions before I start whacking at the code. I also have considered the possibility that although this has been driving me nuts that few others will care--if this is true, it probably *shouldn't* go in the core libraries. > Presumably 'structmatch' would also become a metho on pattern objects? Yes. Good point. >> [mismatch index return] > I don't really like this idea; it seems to be that it would be more > helpful to raise an exception and attach this data to the exception. > But that's a bit inconsisent with how match and search work. Hmm. Yeah, if it weren't for the value of matching re.match semantics, it would make a lot more sense for this to be an exception. As it is, I could see it going either way. Or I suppose you could add a flag to determine which (but ugh). Thanks for your feedback! Mike From pje at telecommunity.com Sat Aug 7 01:58:46 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Aug 7 01:54:39 2004 Subject: [Python-Dev] Call for defense of @decorators In-Reply-To: <41141676.6060602@stackless.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805184845.GB29796@burma.localdomain> Message-ID: <5.1.1.6.0.20040806194231.027540b0@mail.telecommunity.com> At 01:38 AM 8/7/04 +0200, Christian Tismer wrote: >Ronald Oussoren wrote: > >>Whoops, I used the wrong word, I meant 'generic functions' instead of >>'generic example'. He's doing something like this: >>@when("isinstance(db, OracleDB") >>def storeInDB(db, object): >> pass >>@when("isinstance(db, BerkelyDB") >>def storeInDB(db, object): >> pass > >Why is this needed? Can't we continue to use > >if isinstance(db, OracleDB): > def storeInDB(db, object): > pass > >if isinstance(db, BerkelyDB): > def storeInDB(db, object): > pass No... the equivalent code generated is more like: def storeInDB(db,object): if isinstance(db,OracleDB): ... elif isinstance(db,BerkleyDB): ... with these important differences: * The behavior can be defined modularly, with different modules contributing possible execution paths, rather than requiring the function body to be defined in a single place. This makes functions "open" to extension, by adding more cases. For example, one might define a generic function to visit various document node types, and then developers who add new node types can write additional cases for the existing visitor function. * The dispatch algorithm supports multiple dispatch on arbitrary conditions on any parameters, even though the example Ronald gave only does single-dispatch on one parameter. 'and', 'or', and 'not' can be used to construct complex predicates. * An arbitrary number of tests may be applied to arbitrary expressions involving the parameters, but for each invocation of the function, each expression will be computed at most once and each test will be performed at most once, no matter how many 'when()' invocations comprise the total generic function body. Conditions which are "less discriminating" will be tested after conditions that are "more discriminating", in order to avoid computing expressions that may not have any effect on what implementation is finally selected. * The hierarchy of 'if' tests is automatically generated, based on logical implication relationships between the predicates, overlapping of ranges, etc. This removes a lot of tedious reasoning and coding that a human programmer would otherwise have to do to ensure a correct decision tree when the input is just a bunch of "business rules". The net effect is to have generic functions in Python, similar to those of Lisp or Dylan, but with the addition of arbitrary predicate evaluation, as in the research language Cecil. If you're curious about the basic concept and algorithm, see: http://citeseer.ist.psu.edu/chambers99efficient.html and the current Python implementation can be found in the CVS trunk version (1.0a0) of PyProtocols. It does not yet support constrained evaluation order, but it does extend the Chambers and Chen algorithm to support efficient range and equality comparisons (e.g. for numbers and strings), and has other extensions to support dispatching on class and interface using Python's particular MRO rules, which are a bit different from those of Dylan, Lisp, and Cecil. From fperez528 at yahoo.com Sat Aug 7 02:25:47 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Sat Aug 7 02:25:53 2004 Subject: [Python-Dev] IPython, @, and option E from the wiki Message-ID: Hi all, since the developers have been kind enough to take ipython's fate into account in the decorator debate, I figured I'd post an additional piece of info. Option E on the Wiki http://www.python.org/moin/PythonDecorators is: E. pie decorator at top of function body syntax def foo(arg1,arg2): @classmethod ... def bar(low,high): @accepts(int,int) @returns(float) ... I just want to mention a couple of things regarding this one, in case it becomes a contender in Guido's mind: 1. It would NOT cause ipython any problems whatsoever. Ipython only special-cases @foo at the beginning of a line, so this would be perfectly OK with ipython. I don't know if Leo would fare equally well with it. 2. I happen to like it quite a bit, since I already consider (because of the special role of docstrings) the top area of a function 'special'. So that area seems to me a reasonable choice for additional meta-information about the function's behavior. I agree decorators are more radical than docstrings, but at least there's a certain conceptual precedent. And it seems to leave the cleanness of the 'def' word which is a nice visible highlight of a function declaration, without causing the deep indentation issues of other proposals. 3. In this kind of setup, using | instead of @ would be ok as well, I guess: def bar(low,high): |accepts(int,int) |returns(float) ... Ok, nobody really wanted my opinion, so feel free to ignore #2/3 :) The main point was #1. Best, f From edloper at gradient.cis.upenn.edu Sat Aug 7 03:01:47 2004 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Sat Aug 7 03:00:50 2004 Subject: [Python-Dev] pre-PEP: Complete, Structured Regular Expression Group Matching Message-ID: <411429FB.6080603@gradient.cis.upenn.edu> Michael Hudson wrote: >> Generally, it strikes me as mildly useful. An implementation would >> surely help :-) Mike wrote: > If yours is the most positive review, there probably won't be one. :-) I wouldn't lose heart over the lack of response -- the current flood of decorator messages tends to drown out everything else. I like your idea in the abstract, but I have trouble following many of the examples in your pre-pep (and as a result, I'm not sure if I *really* understand what you're proposing). E.g., you have: | >>> m1 = re.match(r'("([A-Z]|[a-z])*"\s*)*', '"Xx" "yy" "ZzZ"') | >>> m1.group(2) | [['X', 'x'], ['yy'], ['ZzZ']] But it seems to me like the output should instead be: | [['X', 'x'], ['y', 'y'], ['Z', 'z', 'Z']] In particular, group two is "([A-Z]|[a-z])", which matches a single character; so it seems like the value of m1.group(2) should be a tree with leaves that are single characters. (Looking at it from another angle, I can't see any difference between "Xx" and "ZzZ" that would cause one to be decomposed and the other to be left as a string.) Similar comments apply for many of your other examples. So either: - Your examples are incorrect - My understanding of your proposed algorithm is incorrect If it's the former, then please fix the examples. If not, then perhaps we can talk about your algorithm some more off-list, so I can try to see where my misunderstanding is coming from. -Edward From python at rcn.com Fri Aug 6 16:30:10 2004 From: python at rcn.com (Raymond Hettinger) Date: Sat Aug 7 04:31:53 2004 Subject: [Python-Dev] pre-PEP: Complete, Structured Regular Expression Group Matching In-Reply-To: Message-ID: <000f01c47bc1$e1da4820$e841fea9@oemcomputer> > A notable limitation of the ``re.match`` method is that it fails to > capture all group match information for repeatedly matched groups. > For example, in a call like this :: > > m0 = re.match(r'([A-Z]|[a-z])*', 'XxxxYzz') > > one would like to see that the group which matched four times matched > the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``. I suspect there is a typo in the example and that the asterisk should precede the right paren: m0 = re.match(r'([A-Z]|[a-z]*)', 'XxxxYzz') Raymond From mkc at mathdogs.com Sat Aug 7 05:59:04 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sat Aug 7 05:59:09 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <000f01c47bc1$e1da4820$e841fea9@oemcomputer> Message-ID: "Raymond Hettinger" writes: > > A notable limitation of the ``re.match`` method is that it fails to > > capture all group match information for repeatedly matched groups. > > For example, in a call like this :: > > > > m0 = re.match(r'([A-Z]|[a-z])*', 'XxxxYzz') > > > > one would like to see that the group which matched four times matched > > the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``. > > I suspect there is a typo in the example and that the asterisk should > precede the right paren: > > m0 = re.match(r'([A-Z]|[a-z]*)', 'XxxxYzz') Oops, you are correct. In fact, I was thinking of m0 = re.match(r'([A-Z]*|[a-z]*)', 'XxxxYzz') when I wrote this, but since only single capitals are given in my example, it could be either. Fixed. Thanks! Mike From tjreedy at udel.edu Sat Aug 7 06:12:19 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Aug 7 06:12:24 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> Message-ID: "Guido van Rossum" wrote in message news:200408061634.i76GYEZ07623@guido.python.org... > > - change the language ref that talks about this to state that > > although Python currently doesn't use these symbols, this is not > > stopping a future version of Python from using them for some new > > feature. > > Does that really have to be stated? All that is needed is the addition of 'currently' in the sentence about illegality. Compare with "The Python compiler currently generates the following byte code instructions. " (LibRef 18.10.1) which has the same word to guard against the same false presumption of stability and consequent complaints. Terry J. Reedy From martin at v.loewis.de Sat Aug 7 06:19:08 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 06:19:12 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <41140E76.2050601@stackless.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> Message-ID: <4114583C.4070500@v.loewis.de> Christian Tismer wrote: > I can think of a couple of ways to implement it. One would be > to defer execution of a statement with undefined names until > the end of the whole code block in the case of a class > definition and to resolve the names late. That would be counter-intuitive. If I have foo = func1(foo) def foo(self): body bar = foo then your proposal is to execute foo=func1(foo) *after* bar=foo. I doubt most users can understand that easily. > Cleaner would probably be to have something like a placeholder > for the undefined function, assign to the name a special > object and trap the def statement to resolve the object. That would require a new keyword, right? Which one? > At the risk of getting slogged, I also thought of > > myfunc := classmethod(myfunc) > > for a short time, with ":=" as a late binding assignment > operator which defers its operation until myfunc is > defined, but I think this is not a popular idea. :-) This is also a syntax change, and one that is going to break existing tools: it reuses the colon, which many tools expect to introduce a block of some sort. > To save *some* typing (laziness was also a driver for this > whole decorator nightmare), we could also remove the assignment > and just have a function call like > > classmethod(myfunc) This breaks backward compatibility. This is already legal syntax, and should raise a NameError if myfunc is not defined (actually, this holds for the first proposal, too). Furthermore, it might be that myfunc *is* defined, so given classmethod(myfunc) def myfunc(): body1 if some_condition(): classmethod(myfunc) def myfunc(): body2 you couldn't make the second definition of myfunc a classmethod. > I could live > with a special rule like my little proposal. Your proposals are all by no means little. They are significant, counter-intuitive changes of the language. > But must it be "@", such an eye-catching, ugly piece > of flie-dirt? If you don't like the current proposal, try to find objective reasons, in addition to the subjective ones. Also, try to come up with counter-proposals that are well thought-out, instead of being created in a rush. In particular, if all you are concerned with is the specific choice of operator, propose a different one. Regards, Martin From martin at v.loewis.de Sat Aug 7 06:23:42 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 06:23:43 2004 Subject: [Python-Dev] python-dev Summary for 2004-07-16 through 2004-07-31 [draft] In-Reply-To: <41140C77.1040802@iinet.net.au> References: <4111DC2C.8070302@ocf.berkeley.edu> <41140C77.1040802@iinet.net.au> Message-ID: <4114594E.2030606@v.loewis.de> Nick Coghlan wrote: > 4b. Edit the .vcproj files and the .py files in Garth's tool to remove > all references to 'odbc32.lib' and 'odbccp32.lib', since the free tools > don't have those, and Python doesn't seem to actually need them (Visual > Studio links to them by default). > > 4c. In gnu_make.py, line 49, change it to be: > write(escape(makedef['all'])) > (The version in the python-dev is missing the escape, which meant > make couldn't match up the targets properly) Feel free to contribute patches. I think I already said that I see no problem with the Python script being distributed along with Python, provided its author(s) are willing to contribute it (it would be good if there was a single dedicated contact for maintaining it, though). For the odbc references, please contribute patches to take the references out of the projects. We really don't need odbc in Python. Regards, Martin From martin at v.loewis.de Sat Aug 7 06:28:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 06:28:48 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <41141255.5010408@stackless.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> Message-ID: <41145A7E.8010500@v.loewis.de> Christian Tismer wrote: > Ok, I dislike special prefix chars at all, in a language that > doesn't have this concept elsewhere (despite strings of course, > but their prefixes are just regular chars), Lists, tuples, and dictionaries are also introduced with non-letter characters. Furthermore, nested expressions use the same prefix char that lists use. # introduces comments. Regards, Martin From edloper at gradient.cis.upenn.edu Sat Aug 7 06:35:17 2004 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Sat Aug 7 06:34:14 2004 Subject: [Python-Dev] Decorators: vertical bar syntax Message-ID: <41145C05.1020501@gradient.cis.upenn.edu> The new "vertical bar" syntax was proposed recently, and has gotten almost unanimously positive feedback. In particular, of 17 responses to the syntax, only one has been negative (see table, below). I'm including both of the following variants: |classmethod |accepts(int, int) |returns(float) def foo(arg1, arg2): ... def foo(arg1, arg2): |classmethod |accepts(int, int) |returns(float) ... (I prefer the second, but would be happy with either.) The one negative comment was that "|" can look similar to "I" or "l", depending on the font; but it's only an issue for some fonts, and it should be largely mitigated by syntax-highlighting. Guido's response to the vertical bar syntax is: > If the community can rally behind one of these, I think that would > be acceptable. They all seem arbitrary, but so is the choice of > '@'. :-) So I wanted to see if anyone has anything *negative* to say about this syntax (or one of its variants). If possible, please respond by updating the wiki, rather than by email. This syntax is listed under H (pre-def) and E2 (post-def). You will need to sign in before you can edit the wiki. Wiki: (Feel free to update the wiki with positives, too, but I'm mainly trying to get a feel for whether anyone is strongly opposed to this syntax.) -Edward ====================================================================== The Vertical Bar Decorator Syntax ====================================================================== History: URL [1] First proposed (by Shalabh Chaturvedi) .../2004-August/047005.html Nested version proposed (by IxokaI) .../2004-August/047170.html All tests pass if @ changed to | .../2004-August/047189.html Responses (sorted): URL [1] +1 Shalabh Charturvedi .../2004-August/047005.html +1 Richie Hindle .../2004-August/047078.html +1 Barry Warsaw .../2004-August/047133.html also .../2004-August/047207.html +1 Tim Peters .../2004-August/047147.html +1 Johannes Gijsbers .../2004-August/047162.html +1 IxokaI .../2004-August/047170.html +1 Fernando Perez .../2004-August/047176.html also .../2004-August/047223.html +1 Mark Russell .../2004-August/047189.html +1 Anthony Baxter .../2004-August/047195.html +1 Raymond Hettinger .../2004-August/047212.html +1 Edward Loper (this email) +1? Holger Krekel .../2004-August/047213.html +0.5 Timothy Delaney .../2004-August/047007.html +0 GvR .../2004-August/047143.html +0? Skip Montanaro .../2004-August/047192.html +0 Christian Tismer .../2004-August/047217.html -1 Bob Ippolito .../2004-August/047200.html [1] All URLs are written in condensed form, 'cuz this email is too long already; to get a real URL, replace the "..." with "http://mail.python.org/pipermail/python-dev". From tjreedy at udel.edu Sat Aug 7 06:34:13 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Aug 7 06:34:20 2004 Subject: [Python-Dev] Re: PEP 0008 confusion - here it is, but don't use it? References: <16659.54650.788854.375861@montanaro.dyndns.org> <4113E512.8050407@v.loewis.de> Message-ID: "Martin v. Löwis" wrote in message news:4113E512.8050407@v.loewis.de... > > I'm a bit confused. Why bother to speed up s+=t if we aren't supposed to > > use it? > > Because it helps users who aren't aware of the problem, and use the > obvious approach. Developers of the standard libraries are supposed to > know, and use the "better, but non-obvious" technique. Also, users are freer to discard cross implementation portability than library. Dependence on ref counting to close files is another example. tjr From pythondev at bitfurnace.com Sat Aug 7 06:40:44 2004 From: pythondev at bitfurnace.com (damien morton) Date: Sat Aug 7 06:44:32 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <4114583C.4070500@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> Message-ID: <41145D4C.7040200@bitfurnace.com> Even though Im basically in favour of the C# style syntax, and mostly because I have been using C# a lot and find no problems recognising the syntax when I see it, I also find the "decorator on the line after the def" syntax appealing. Allow me to throw this syntax into the ring: def foo(arg1,arg2): \(classmethod) blah ... def bar(low,high): \(accepts(int,int), returns(float)) blah ... Allow me to also throw a possible variant def foo(arg1,arg2): \(classmethod) blah ... def bar(low,high): \(accepts(int,int), returns(float)) blah ... I strongly feel that any decorator syntax will need some kind of matched pair to delimit the decorator(s). Angle-brackets (single or double) seem very appealing def foo(arg1,arg2): blah ... def bar(low,high): blah ... From python-kbutler at sabaydi.com Sat Aug 7 06:55:35 2004 From: python-kbutler at sabaydi.com (Kevin J. Butler) Date: Sat Aug 7 06:55:42 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) Message-ID: <411460C7.4030708@sabaydi.com> Many people wrote: > Lots of options I haven't seen this mentioned, so I'm going to throw in my suggestion. The following makes sense to me, uses previously invalid syntax, and uses the '.' punctuation in a way similar to existing/proposed usage: def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42): .staticmethod .funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") """ asdfasdf """ raise NotYetImplemented If we want to allow one-liner decorators (questionable to me): def myMethod(self): .staticmethod raise NotYetImplemented Advantages/disadvantages of positioning at start of block: + Unambiguous "target" for decorators, matches Python's precedents for indentation and modify-what-came-before + Decorators won't get lost in long argument lists (they are indented differently in Guido's preferred formatting, and are separated by the smiley in the all-args-on-individual-lines formatting) + Separate from def syntax (to keep def the same avoiding breakage) 0 Perhaps decorators should be allowed before or after the docstring. If you have to choose, I'd choose making it before the docstring. + No extra cut-and-paste issues, no extra indentation level. + No smileys. (Does that get a :) or a ]: ?) Advantages/disadvantages of .decorators: + Will not be silently ignored (doesn't use currently legal syntax) + Simple for simple decorators, supports complex decorators as cleanly as possible + Less ugly (YMMV). Doesn't feel like executable line noise (to 1 out of 1 pythonistas polled...) + No new punctuation or keywords (avoids breaking Leo/IPython/existing code) + Really nice syntax for assigning attributes, if desired (Not sure I like this overloading, but it /looks/ good) def func(): .author = "Kevin Butler" pass +/0 Syntax obvious visually (Someone will complain that the leading period gets lost - that person should switch to a fixed-width font, as used when coding. <.5 wink>) Easy to highlight. 0 Although it is a punctuation-based syntax, it is compatible with existing/proposed '.' usage ('.' can mean subordination to a containing construct like 'with', and passing the implicit "func" argument is analogous to passing "self") 0 Compatible with some future "with ...:" syntax, as decorators must immediately follow a 'def ...:' (or possibly a 'class ...:'), so if there is a 'with ...:' what follows cannot be a decorator. - Minor extension to use of '.' - Some people may have too much dust on their monitors Misc items from the PEP applied to .decorators: + One decorator per line + Could use ".doc()" or ".doc = " as a docstring alternative ?+ Compile time (? as much as the @ syntax is?) + Works in interactive mode Public or private feedback appreciated, I will summarize private feedback to the list. kb From nbastin at opnet.com Sat Aug 7 06:56:34 2004 From: nbastin at opnet.com (Nick Bastin) Date: Sat Aug 7 06:58:33 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <27F1C2B9-E82E-11D8-BC7A-000D932927FE@opnet.com> On Aug 7, 2004, at 12:35 AM, Edward Loper wrote: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). > > I'm including both of the following variants: > > |classmethod > |accepts(int, int) > |returns(float) > def foo(arg1, arg2): > ... > > def foo(arg1, arg2): > |classmethod > |accepts(int, int) > |returns(float) I have to admit that at first look, the decorator-after-def syntax wasn't very appealing. However, although it's not how other languages generally do it (does anybody know of one?), it might be easier for new users to understand. i.e., I'm decorating 'this' thing, rather than the 'next' thing. Does anybody suggest how this syntax plays with docstrings? Are decorators after the doc string, or before, or doesn't it matter? My vote would be for after the doc string, but I thought I'd throw that out there. It's also entirely possible that no one else likes the after-def syntax, so maybe this isn't a bridge we'll have to cross.. :-) As for pipe versus @, I couldn't really care less, since my syntax highlighter will take care of it, but for new users I think '@' stands out much better. On the other hand, we might want to reserve '@' for some future use more aligned to its' usual meaning. Heck, I'm surprised that any rational discussion is happening at all at this point. Seriously, we've been discussing this for longer than I care to think about, and it's obvious that we're not going to get something that everyone agrees upon. If there were something that everyone could agree upon, we'd already have found it - it's not like a pile of new discussion is likely to change that, so if we stick with '@'-before-def, that's fine by me too. -- Nick From mkc at mathdogs.com Sat Aug 7 07:03:30 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sat Aug 7 07:03:35 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <411429FB.6080603@gradient.cis.upenn.edu> Message-ID: Edward Loper writes: > I wouldn't lose heart over the lack of response -- the current flood of > decorator messages tends to drown out everything else. No, not at all. I was hoping I might benefit from @-fatigue, though. :-) > Similar comments apply for many of your other examples. So either: > - Your examples are incorrect > - My understanding of your proposed algorithm is incorrect It's the former, unfortunately--sorry about that. I will post a corrected version shortly. Thanks for your feedback, Mike From tjreedy at udel.edu Sat Aug 7 07:05:55 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Aug 7 07:06:00 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: "Edward Loper" wrote in message news:41145C05.1020501@gradient.cis.upenn.edu... > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). Add another +1. Verticle rules have been used before to tie several lines together, either as continuation lines or as sub-lines. > |classmethod > |accepts(int, int) > |returns(float) > def foo(arg1, arg2): > ... > > def foo(arg1, arg2): > |classmethod > |accepts(int, int) > |returns(float) > ... > (I prefer the second, but would be happy with either.) Both logically and visually, I prefer the second also. By logically, I mean that it is more meaningful to me to read the decorators after reading that we are defining a function. My main objection is that I like the idea that the header is definition time info and the body is run time code. I consider the doc string in the body to be a wart, not a precedence to by copied, but maybe I should think of the body as starting after the last | or the closing ''' :-). > The one negative comment was that "|" can look similar to "I" or "l", > depending on the font; but it's only an issue for some fonts, and it > should be largely mitigated by syntax-highlighting. Is a space allowed after the prefic character, whatever it might be? With def foo(): | classmethod | accepts(int,int) | returns(float) the visual similarity to I doesn't matter much. Actual typos leading to test such as 'I returns' are guaranteed syntax errors, whereas a name such as Ireturns is not. The space even more clearly matches standard usage of a vertical rule and more clearly denotes the list as hanging off of 'def', so I would be tempted to require the space. Terry J. Reedy From mkc at mathdogs.com Sat Aug 7 07:08:08 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sat Aug 7 07:10:42 2004 Subject: [Python-Dev] pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: Message-ID: The first version contained several errors. Here is a new version with corrections. It also incorporates the feedback received so far. Mike ############################################################################## PEP: XXX Title: Complete, Structured Regular Expression Group Matching Version: $Revision: 1.4 $ Last-Modified: $Date: 2004/08/07 04:57:57 $ Author: Mike Coleman Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 1-Aug-2004 Python-Version: 2.4 Post-History: 1-Aug-2004 Abstract ======== This proposal is to add a new method ``re.structmatch`` that fully captures matches for groups that match repeatedly. The method returns a structured match tree whose shape is determined by the pattern groups. Ultimately this will allow a string that can be described by a Python regular expressions (e.g., the contents of ``/etc/passwd`` or ``.ini`` files, or the output of ``diff``) to be parsed into the obvious parse tree with a single call to ``re.structmatch``. Motivation ========== A notable limitation of the ``re.match`` method is that it fails to capture all group match information for repeatedly matched groups. For example, in a call like this :: m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz') one would like to see that the group which matched four times matched the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``. In the current implementation, only the last match for each group (``'zz'`` in this case) is available, even though the other matches are calculated implicitly during the matching process. For simple cases, the missing strings might be easily recovered by other means, but for more complicated cases this is a significant annoyance. The simplest extension would be to collect all of the matches for each group in a list, so that in the call above for example :: m0.group(1) --> ['X', 'xxx', 'Y', 'zz'] (A mechanism like this is apparently to be added to Perl in version 6 [#PERL6]_, though that did not inspire this PEP.) This immediately suggests the question of what is to be done for nested repeats, like so :: m1 = re.match(r'("([A-Z]+|[a-z]+)*"\s*)*', '"Xx" "yy" "ZzZ"') We could have :: m1.group(2) --> ['X', 'x', 'yy', 'Z', 'z', 'Z' ] but this flattened result would discard useful information about the structure of the match. A more useful result would be :: m1.group(2) --> [['X', 'x'], ['yy'], ['Z', 'z', 'Z']] This is definitely an improvement. Consider the following slightly more complicated case, though :: m1 = re.match(r'("([A-Z]+|[a-z]+)*"(\s*))*', '"Xx" "yy" "ZzZ"') which would give :: m1.group(2) --> [['X', 'x'], ['yy'], ['Z', 'z', 'Z']] m1.group(3) --> [' ', ' ', ''] This is less useful than the putative conceptual structure of the match, which would be something like :: [[['X', 'x'], ' '], [['yy'], ' '], [['Z', 'z', 'Z'], '']] In this simple case, this structure could be recovered using the ``zip`` function, but in the general case this is a non-trivial transformation. This PEP proposes a mechanism to address the general case directly. Proposal ======== The new function ``re.structmatch`` has the same arguments as ``re.match`` and will match using exactly the same algorithm. Instead of returning a MatchObject upon success, ``re.structmatch`` will return a "tree" (or rather, a forest) in the form of a Python list. First we will define some terms. Groups in the input pattern are parenthesized subexpressions that "capture" strings when matched, and are of the form ``'(abc)'`` or ``'(?Pabc)'``. Leaf groups are groups that do not contain any subgroups. So, for example, the group ``'(abc)'`` is a leaf group and the group ``'(a(xyz)b)'`` is not a leaf group (but it does contain the leaf group ``'(xyz)'``). We shall call parenthesized expressions that are not groups "nongroups"; these include constructs like ``'(?:abc)'`` as well as lookahead and lookbehind assertions. The structure of the returned tree is determined from the grouping tree present in the pattern in the following manner: * Leaf groups not followed immediately by a repeat operator map to a single string:: re.structmatch(r'(...)', 'abcdef') --> ['abc'] re.structmatch(r'(...).(..)', 'abcdef') --> ['abc', 'ef'] * Leaf groups followed immediately by a repeat operator map to a list of strings:: re.structmatch(r'([^d])*', 'abcdef') --> [['a', 'b', 'c']] re.structmatch(r'([^d])*(.)*', 'abcdef') --> [['a', 'b', 'c'], ['d', 'e', 'f']] re.structmatch(r'(..)*', 'abcdef') --> [['ab', 'cd', 'ef']] re.structmatch(r'(.){2}', 'abcdef') --> [['a', 'b']] * Non-leaf groups not followed immediately by a repeat operator map to a list of the mappings of their subgroups:: re.structmatch(r'(...)', 'abcdef') --> ['abc'] re.structmatch(r'((...))', 'abcdef') --> [['abc']] re.structmatch(r'(((...)))', 'abcdef') --> [[['abc']]] re.structmatch(r'((...)())', 'abcdef') --> [['abc'], []] re.structmatch(r'(.(.)(.(.)).(.))', 'abcdef') --> ['a', ['b'], ['c', ['d']], 'e', ['f']] * Non-leaf groups followed immediately by a repeat operator map to a list of the mappings of their repeated matches:: re.structmatch(r'((.).(.))*', 'abcdef') --> [[['a', 'c'], ['d', 'f']]] re.structmatch(r'(([ab])*(x)*)*', 'baxbxx') --> [[['b', 'a'], ['x']], [['b'], ['x', 'x']]] * In the case of alternation, only the matched groups appear in the output:: re.structmatch(r'([^a])+|([^d])+', 'abcdef') --> [['a', 'b', 'c']] If it's important to know which alternative matched, named groups can be used. * Named groups map to a pair where the first member is the name and the second member is what the unnamed group would have mapped to:: re.structmatch(r'([^d])*(?P.)*', 'abcdef') --> [['a', 'b', 'c'], ('rest', ['d', 'e', 'f'])] * Nongroups do not affect the output structure. Compared to non-leaf groups, nongroups have the effect of "flattening" the output, like so:: re.structmatch(r'((.).(.))', 'abcdef') --> [['a', 'c']] re.structmatch(r'(.).(.)', 'abcdef') --> ['a', 'c'] re.structmatch(r'(?:(.).(.))', 'abcdef') --> ['a', 'c'] re.structmatch(r'((.).(.))*', 'abcdef') --> [[['a', 'c'], ['d', 'f']]] re.structmatch(r'(?:(.).(.))*', 'abcdef') --> [['a', 'c', 'd', 'f']] (Nongroups that do not contain leaf groups have no effect whatsoever on the output structure.) Additional Features ------------------- By analogy with ``'re.match'``, regular expression objects will also have a ``'structmatch'`` method, with the same signature as the ``'match'`` method. In many cases in which ``'re.structmatch'`` fails to match, the cause will be due to an unexpected error in the format of the string being matched. In order to assist the calling program in generating a more meaningful possible error message, ``'re.structmatch'`` will return the endpoint of the largest match against the searched string. So, for example :: re.structmatch('abcd', 'abxxx') --> 2 re.structmatch('abcde|z', 'abxxx') --> 2 re.structmatch('z|abcde', 'abxxx') --> 2 re.structmatch('x*?y', 'xxx') --> 3 (This return value should be considered advisory rather than exact, as future improvements in the match algorithm may make it difficult to calculate the exact value.) An alternative suggested by Michael Hudson would be to throw an exception in this case. This makes more sense, but would have the disadvantage of diverging from the behavior of ``'re.match'``. Examples ======== Parsing ``/etc/group`` ---------------------- If ``/etc/group`` contains the following lines :: root:x:0: daemon:x:1: bin:x:2: sys:x:3: then it can be parsed as follows :: p = r'''(?xm) # VERBOSE, MULTILINE ( (?: # one field, preceded by a ':' if # the field is not the line's first (?:^|:) ([^:\n]*) )* \n )* \Z # assert that the entire # input was matched ''' s = open('/etc/group').read() tree = re.structmatch(p, s) which will give ``tree`` the following value:: [['root', 'x', '0', ''], ['daemon', 'x', '1', ''], ['bin', 'x', '2', ''], ['sys', 'x', '3', '']] Note that the above pattern is written to allow any ``/etc/group`` field to be empty. The pattern won't work correctly for versions of Python prior to 2.4 because of the possibility of repeating empty matches. This problem seems to have been fixed in Python 2.4. (A slightly more complicated pattern would work for pre-2.4 versions.) Parsing ``.ini`` files ---------------------- The Microsoft ``.ini`` file format is found in various contexts (there is one in the Python source distribution: ``Tools/freeze/extensions_win32.ini``). Given a file with contents :: [singers] good=Annie Lennox bad=William Shatner [comedians] good=Monty Python the file can be parsed with pattern :: r'''(?xm) # VERBOSE, MULTILINE \s* # leading whitespace ( # begin sections ^\[ ([^]]+) \] \s* # section header ( # begin params ^([^=]+) = # param name (.*) $ # param value \s* # intervening whitespace )* # end params )* # end sections \Z # assert that the entire # input was matched ''' to give :: [['singers', ['good', 'Annie Lennox'], ['bad', 'William Shatner']], ['comedians', ['good', 'Monty Python']]] The pattern given omits support for ``.ini`` file comments for the sake of simplicity, but this could be added. Details ======= The proposal states that ``re.structmatch`` will match using exactly the same algorithm as ``re.match``. This might be a little odd for a pattern like ``r'(x|y|z)*aaa\1'``, where the algorithm will require that the ``\1`` back-reference match (at most) one character. It's not obvious whether there is any better option, though, and there are advantages of implementation and simplicity for keeping the match algorithms of these two methods identical. (A similar argument applies to ``'(?P=NAME)'``.) Discussion ========== Part of the reason the proposed method would be so useful is that ``re.split`` currently does not split on empty matches. If it had this feature, one could split on lookahead and lookbehind assertions, which would significantly ease parsing of strings with a recursive regular structure (e.g., ``.ini`` files). Patch `#988761`_ will correct this ``re.split`` deficiency, if it is accepted. .. _#988761: https://sourceforge.net/tracker/?func=detail&aid=988761&group_id=5470&atid=305470 For particularly complex patterns, the current 99 group limit might actually become a practical problem. One could imagine a variation in which subtrees of named groups might be captured in dictionaries rather than lists, with the group names used as keys. Rejected Alternatives ===================== Several simpler alternatives are rejected in the `Motivation`_ section above. Although these alternatives would be better than nothing, they would not adequately satisfy the goal of this proposal, which is to allow the programmer to extract the structure of a string in an immediate manner. It would be possible to use tuples for some substructures in the return tree, rather than composing it strictly of lists. This alternative was rejected because it was thought useful to be able to modify the resulting tree in place, perhaps to add decorations, etc., and tuples would make this more difficult. References ========== .. [#PERL6] Multiple regex matches in Perl Apocalypse 5 (http://www.perl.com/pub/a/2002/06/04/apo5.html?page=22#rfc%20360:%20allow%20multiply%20matched%20groups%20in%20regexes%20to%20return%20a%20listref%20of%20all%20matches) Acknowledgements ================ Thanks to Raymond Hettinger, Michael Hudson and Edward Loper for feedback and improvements. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From bob at redivi.com Sat Aug 7 07:14:39 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat Aug 7 07:14:50 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: On Aug 7, 2004, at 12:35 AM, Edward Loper wrote: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). --- > The one negative comment was that "|" can look similar to "I" or "l", > depending on the font; but it's only an issue for some fonts, and it > should be largely mitigated by syntax-highlighting. > -1 Bob Ippolito .../2004-August/047200.html I'm actually -0, it doesn't really bother me, but I saw the potential issue immediately. When the | is viewed with common fonts in email, web pages, or maybe even print (if done semi-carelessly) it has the potential to be confusing. -bob From nhodgson at bigpond.net.au Sat Aug 7 08:10:54 2004 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Sat Aug 7 08:12:24 2004 Subject: [Python-Dev] Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <041501c47c45$54e3f630$e615a88d@neil> Edward Loper: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). > ... > So I wanted to see if anyone has anything *negative* to say about this > syntax (or one of its variants). One advantage of '@' is that, since it is currently not found in Python code, it is a very strong escape that can be inserted anywhere. For example, if there is later a need or desire to annotate function arguments individually as well as functions, or individual statements or expressions, then @ is a better choice. For example @RemoteCall def Ignite(engine @Remote, environment @Copy, thrust @Float): may be better at associating annotations more closely with their targets than @RemoteCall @Types(["p", "p", "f"]) @Copy("environment") def Ignite(engine, environment, thrust): Neil From nhodgson at bigpond.net.au Sat Aug 7 08:10:54 2004 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Sat Aug 7 08:13:05 2004 Subject: [Python-Dev] Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <041901c47c45$95c30470$e615a88d@neil> Edward Loper: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). > ... > So I wanted to see if anyone has anything *negative* to say about this > syntax (or one of its variants). One advantage of '@' is that, since it is currently not found in Python code, it is a very strong escape that can be inserted anywhere. For example, if there is later a need or desire to annotate function arguments individually as well as functions, or individual statements or expressions, then @ is a better choice. For example @RemoteCall def Ignite(engine @Remote, environment @Copy, thrust @Float): may be better at associating annotations more closely with their targets than @RemoteCall @Types(["p", "p", "f"]) @Copy("environment") def Ignite(engine, environment, thrust): Neil From tim.peters at gmail.com Sat Aug 7 08:27:20 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 08:27:29 2004 Subject: [Python-Dev] RELEASED Python 2.4, alpha 2 In-Reply-To: <411274C0.4070300@v.loewis.de> References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de> <1f7befae040805103911610582@mail.gmail.com> <411274C0.4070300@v.loewis.de> Message-ID: <1f7befae04080623276089908e@mail.gmail.com> [Tim, tries the "upgrade" install for 2.4a2, notes that it's pretty slow] [Martin v. L?wis] > Yes. I should probably speed it up by changing the UI sequence number > of RemoveExistingApplications from 1510 to 1450. In case you wonder > what this means: It's obvious: you would add 3000, then cyclically permute the first 3 digits right one position. Tell me something I don't know . > Action 1500 is InstallInitialize; anything from then > on is under transaction control. This means that installer will roll > back the uninstall in case the subsequent install fails. This is > expensive, as it first moves all old files out of the place, then puts > the new ones in, then deletes the old ones. At 1450, the files would > be deleted immediately (I believe), at the cost of not being able to > undo the deinstallation if the installation later fails (e.g. by > user Cancel). I wouldn't do this; it's good that as much of the process as possible is transactional. I doubt it took longer overall than what I normally do (run a full uninstall of the old version, then install the new version). >> One glitch: I'm pretty sure I selected "install only for me" in the >> first screen. I wanted to go back to check, but it doesn't seem >> possible to go back to the first screen. > Yes. I don't know what to do about this: the screen sets ALLUSERS=1, > then performs FindRelatedProducts to find the old installation. If the > old installation was per-machine, we only find it if we commit to making > the new installation also per-machine. Except that in this case, I'm pretty sure my 2.4a1 installation was also "only for me". I could be wrong about that. > That decision cannot be taken back; the user would have to cancel and start > over. Which is an option, so that's cool. >> Anyway, the Python DLL ended up in a system directory, and that's "the >> glitch". I expected/wanted it to end up in my Python directory. > Ok. I will see whether I can reproduce that. I believe I later saw a checkin that intended to fix this, and, if so, that's cool too. I hereby nominate you as MSI-builder For Life . From tim.peters at gmail.com Sat Aug 7 08:54:35 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 08:54:38 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz> <1f7befae04080522314d818d8f@mail.gmail.com> Message-ID: <1f7befae040806235454e01df9@mail.gmail.com> [ixokai@gmail.com] > Doesn't the fact that the @stuff is at the same level and has no > connection to the def besides 'above' bother you in our universe of > indentation-love? No, it doesn't. I routinely put comment blocks before my functions, "at the same level", and they're strongly connected to the function in reality if not by syntax rule. I *like* them at that level too, because they're usually about the external behavior of the function, not about its internals. Decorators are also more about externals than internals, so it feels right to me to see them at the same level as the function's comment block. Guido agrees, although he doesn't yet realize it , as evidenced by his earlier comment that docstrings probably would have been better placed before defs too. Comment blocks, docstrings, and decorators are all about what the function is like "from the outside". If Python supported design by contract natively, I'd also like to see method pre- and post-conditions at this level. > And the fact that the pie *is* "at", and why that it just doesn't > "read"? '"define methy1 at staticmethod" is wrong, but "at > staticmethod define methy2" is right. When I read # Here's a comment. I don't verbalize it as "splat here's a comment". I don't feel a need to verbalize "@" either. > I like Python to be able to be read very easily. I care about readability, but little about pronouncability. > If we have to have it at the same level as def and above, we soo need a > keyword..pweeze? Pick the keyword you'd like. Then whenever you see "@", pronounce that word instead . > class MyLittleToy: > def methy1(self): > print 'whee' > @staticmethod > def methy2(): > print 'woo' > @staticmethod > def methy3(arg): > for i in arg: > print 'woohoo' > > ? That bothers me a looot... and is the primary reason I hate the pie. > That and I really don't want anymore punctuation. That bothers me a lot too, but because the world really isn't running out of vertical whitespace: class MyLittleToy: def methy1(self): print 'whee' @staticmethod def methy2(): print 'woo' @staticmethod def methy3(arg): for i in arg: print 'woohoo' doesn't bother me. From mike.spam.filter at day8.com.au Sat Aug 7 09:00:01 2004 From: mike.spam.filter at day8.com.au (Mike Thompson) Date: Sat Aug 7 09:00:12 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: Terry Reedy wrote: > > Is a space allowed after the prefic character, whatever it might be? With > > def foo(): > | classmethod > | accepts(int,int) > | returns(float) > Thats it! I love it. From t-meyer at ihug.co.nz Sat Aug 7 09:14:29 2004 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Sat Aug 7 09:14:34 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: Message-ID: [Tim Peters explaining that you don't have to crunch everything up together] > That bothers me a lot too, but because the world really isn't running > out of vertical whitespace: > > class MyLittleToy: > > def methy1(self): > print 'whee' > > @staticmethod > def methy2(): > print 'woo' > > @staticmethod > def methy3(arg): > for i in arg: > print 'woohoo' If the syntax does end up along these lines (and I'm expressing no opinion either way), then maybe it would be worth explicitly telling people (i.e. in PEP 8) that they ought to put a blank line before decorators to make it clear? Yes, people should figure this themselves, but they probably won't <0.5 wink>, and it does make for ugly code. Same deal for "don't do '@this @and_this @on_the_same_line'" (unless that loses it's validity). (And if some other syntax 'wins', then a guide for using that couldn't hurt, too). =Tony Meyer p.s. Anyone else have a suspicion that Anthony checked in the @decorators to 2.4a2 as a sneaky way of getting more people than normal (all them riled up ones) to download the alpha and try it out? From s.percivall at chello.se Sat Aug 7 09:33:00 2004 From: s.percivall at chello.se (Simon Percivall) Date: Sat Aug 7 09:33:03 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <02B028A7-E844-11D8-8B30-0003934AD54A@chello.se> On 2004-08-07, at 06.35, Edward Loper wrote: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). -1 from me, so that's two. | just isn't a good sign. From s.percivall at chello.se Sat Aug 7 10:03:24 2004 From: s.percivall at chello.se (Simon Percivall) Date: Sat Aug 7 10:03:27 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <000401c47bec$588620a0$e841fea9@oemcomputer> References: <000401c47bec$588620a0$e841fea9@oemcomputer> Message-ID: <41F91659-E848-11D8-8B30-0003934AD54A@chello.se> On 2004-08-06, at 21.34, Raymond Hettinger wrote: >> | just isn't a good sign. > > How about the exclamation point? > > > Raymond > > If "@" is controversial enough to be a no-no, "!" gets +1 from me. From florian.proff.schulze at gmx.net Sat Aug 7 10:42:16 2004 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Sat Aug 7 10:37:44 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> Message-ID: On Fri, 06 Aug 2004 11:39:34 -0400, Barry Warsaw wrote: > So I spent a little time futzing with python-mode and some of my > decorator code, to try some alternative leading characters. Of the ones > that cannot be used in valid Python code today (i.e. no backward > compatibility issues), I tried ':', '/', '=', '*', and '|'. > > [snipped] > > My preference then would be to use = with | and : as alternatives, in > that order. Does it really need to be only one character? Would this make the parser to complicated? If not, I would propose '::' which stands out much more than ':' What about '~'? More: '|=' '|~' ':~' ':=' - This was brought up as late binding as well: name := staticmethod(name) def name(self): ... '&' - Used for too much else IMO. Now in context: ::accepts(int,int) ::returns(float) def bar(low,high): ... |=accepts(int,int) |=returns(float) def bar(low,high): ... |~accepts(int,int) |~returns(float) def bar(low,high): ... :~accepts(int,int) :~returns(float) def bar(low,high): ... :=accepts(int,int) :=returns(float) def bar(low,high): ... ~accepts(int,int) ~returns(float) def bar(low,high): ... ~~accepts(int,int) ~~returns(float) def bar(low,high): ... If all this is not possible, then I +1 for '|'. |accepts(int,int) |returns(float) def bar(low,high): ... Regards, Florian Schulze ps: Just for reference: @accepts(int,int) @returns(float) def bar(low,high): ... From martin at v.loewis.de Sat Aug 7 11:26:07 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 11:26:15 2004 Subject: [Python-Dev] MSI Installer issues In-Reply-To: <1f7befae04080623276089908e@mail.gmail.com> References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de> <1f7befae040805103911610582@mail.gmail.com> <411274C0.4070300@v.loewis.de> <1f7befae04080623276089908e@mail.gmail.com> Message-ID: <4114A02F.7060906@v.loewis.de> Tim Peters wrote: >>Yes. I don't know what to do about this: the screen sets ALLUSERS=1, >>then performs FindRelatedProducts to find the old installation. If the >>old installation was per-machine, we only find it if we commit to making >>the new installation also per-machine. > > > Except that in this case, I'm pretty sure my 2.4a1 installation was > also "only for me". I could be wrong about that. Indeed. Upgrading from a per-user installation is easy, as the whole process starts with the assumption that it is going to be per-user. Only per-machine were previously not upgradable. > I believe I later saw a checkin that intended to fix this, and, if so, > that's cool too. I hereby nominate you as MSI-builder For Life > . Yes, and thanks :-) Part of my recent talking goes along with the hope that others become interested enough in MSI as a technology to be able to contribute, though. While I have your attention: you have commented that the installer logo needs attribution, during the installation process. Is this still the case, and what is the text I should use? Any other attribution that needs to be made? A colleague has commented that logo is too low-res, is ugly, and does not feature reptiles. Should we look for an update to the logo? Regards, Martin From heikowu at ceosg.de Sat Aug 7 11:27:04 2004 From: heikowu at ceosg.de (Heiko Wundram) Date: Sat Aug 7 11:26:24 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <200408071127.04568.heikowu@ceosg.de> Am Samstag, 7. August 2004 06:35 schrieb Edward Loper: > I'm including both of the following variants: > |classmethod > |accepts(int, int) > |returns(float) > > def foo(arg1, arg2): > ... > > def foo(arg1, arg2): > |classmethod > |accepts(int, int) > |returns(float) > > ... > > (I prefer the second, but would be happy with either.) -0.5 for the first, because | hard to read (I just noticed that in my eMail client), and -1 for the second, because decorating a function (some of you suggested even _after_ the docstring) in the actual function takes away all there is to decorators: making it immediately clear that a function is being decorated. If we would have to specify decorators after the actual def and the doc-string, there'd be no need for decorators, as that's what we can do at the moment too. Heiko. From heikowu at ceosg.de Sat Aug 7 11:28:56 2004 From: heikowu at ceosg.de (Heiko Wundram) Date: Sat Aug 7 11:28:15 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) In-Reply-To: <411460C7.4030708@sabaydi.com> References: <411460C7.4030708@sabaydi.com> Message-ID: <200408071128.56011.heikowu@ceosg.de> Am Samstag, 7. August 2004 06:55 schrieb Kevin J. Butler: > The following makes sense to me, uses previously invalid syntax, and > uses the '.' punctuation in a way similar to existing/proposed usage: > > def longMethodNameForEffect(longArgumentOne=None, > longArgumentTwo=42): > .staticmethod > .funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", > status="experimental", author="BDFL") > """ > asdfasdf > """ > raise NotYetImplemented -1 Same objection from me as to the proposal of decorating with | in the function: it simply takes away the actual meaning from the decorator (which actually is to make it very clear to see that the function is special in some way). Heiko. From martin at v.loewis.de Sat Aug 7 11:31:29 2004 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 11:31:32 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> Message-ID: <4114A171.3050809@v.loewis.de> Florian Schulze wrote: > Does it really need to be only one character? Would this make the > parser to complicated? If not, I would propose '::' which stands out > much more than ':' Anything involving colon might break tools who believe that in Python, the colon opens a block. Of course, it might then also be that these tools recognize that there text after the colon (both after the first and after the second one), and assume that this uses the single-line version of suite. > What about '~'? It already is a unary operator: >>> def foo(): ... return 1 ... >>> ~foo() -2 Regards, Martin From martin at v.loewis.de Sat Aug 7 12:30:16 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 12:30:18 2004 Subject: [Python-Dev] PyRun_ with file name In-Reply-To: <20040806220926.GA2225@unpythonic.net> References: <4113EF69.2020102@v.loewis.de> <20040806220926.GA2225@unpythonic.net> Message-ID: <4114AF38.5040107@v.loewis.de> Jeff Epler wrote: > Another alternative would be to complete the PyFile_* family of APIs > (PyFile_Seek, etc), so that the C runtime routines would never be used. > The "new" function in the PyRun_ family would take a PyFileObject*. I'm not sure I like this alternative, though. It is more complex both for the caller and for the Python runtime. Regards, Martin From anthony at interlink.com.au Sat Aug 7 12:42:46 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat Aug 7 12:43:06 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <200408071127.04568.heikowu@ceosg.de> References: <41145C05.1020501@gradient.cis.upenn.edu> <200408071127.04568.heikowu@ceosg.de> Message-ID: <4114B226.4020405@interlink.com.au> Lest my previous post saying I could "live with" the pipe symbol be misinterpreted, I still prefer @ over |. My main reason for preferring the former is that it's far more obvious in a page of code (I tried both). The pipe tends to blend in to a disturbing degree. Making it a different colour in an editor also doesn't seem to help that much. I'm also concerned that we don't make a horrible precedent here - that a new language feature is changed solely because other tools that were playing outside the boundaries of the language will be impacted. This would be an utter pain in the arse going forward. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Sat Aug 7 12:43:50 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat Aug 7 12:44:10 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) In-Reply-To: <411460C7.4030708@sabaydi.com> References: <411460C7.4030708@sabaydi.com> Message-ID: <4114B266.3020306@interlink.com.au> Kevin J. Butler wrote: > Many people wrote: > > Lots of options > > I haven't seen this mentioned, so I'm going to throw in my suggestion. > > The following makes sense to me, uses previously invalid syntax, and > uses the '.' punctuation in a way similar to existing/proposed usage: -1. This violates my "syntax should not look like grit" rule. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From adurdin at gmail.com Sat Aug 7 14:06:36 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Sat Aug 7 14:06:38 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <4114A171.3050809@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <4114A171.3050809@v.loewis.de> Message-ID: <59e9fd3a04080705067633a790@mail.gmail.com> I don't like the pipe at all, for two reasons: One: as has already been mentioned, in many fonts it is easily confused with lowercase L: l | Two, and more importantly: the pipe is in the shifted position of the backslash key, which on different keyboards is in different locations, many of them awkward to reach while typing at speed. This problem is particularly severe on laptop keyboards. @ at least is consistently on the 2 key (except, of course, for all those European keyboards. I guess their differences are enough to knock down any argument about typing convenience :-) Andrew Durdin From adurdin at gmail.com Sat Aug 7 14:13:57 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Sat Aug 7 14:13:59 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <4114B226.4020405@interlink.com.au> References: <41145C05.1020501@gradient.cis.upenn.edu> <200408071127.04568.heikowu@ceosg.de> <4114B226.4020405@interlink.com.au> Message-ID: <59e9fd3a04080705132e5cf5d2@mail.gmail.com> Edward Loper wrote: >So I wanted to see if anyone has anything *negative* to say about this >syntax (or one of its variants). If possible, please respond by >updating the wiki, rather than by email. This syntax is listed under >H (pre-def) and E2 (post-def). My only problem with the pipe is that it's often awkward to type particularly on laptop keyboards where it gets squeezed into odd locations. I've added a comment to this effect on the wiki. From adurdin at gmail.com Sat Aug 7 14:36:59 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Sat Aug 7 14:37:02 2004 Subject: [Python-Dev] Decorators with arguments are curries! Message-ID: <59e9fd3a04080705361b6314be@mail.gmail.com> Having waded through all the decorator syntax discussions recently, and some of the historical ones, I haven't found this point mentioned before, so I'll bring it up now: Consider the following code: def foo(bar): return 1 a = foo a = foo(bar) The first assignment to a is binding a reference to a function; the second is calling the function. This is a very significant difference in python, and I'm concerned that all the current proposed decorator syntaxes[*] are liable to cause confusion on this point. For example: def foo_decorator(func): print "no params to this" return func def bar_decorator(func, param): print param return func @foo_decorator @bar_decorator("one param here") def decorated_func(): pass Here the first decorator statement is bare, while the second one includes parentheses and an argument; the first one looks like a function reference, while the second looks like a function call. I find this confusing; semantically, this appears to equivalent to (assuming expressions are allowed in decorator statements): # using curry as proposed in PEP 309 [**] @foo_decorator @curry(bar_decorator, "one param here") def decorated_func(): pass Most of my concern here is that this aspect of decorator syntax appears to be implicitly introducing a currying syntax in one special circumstance, which is then *not* transferable to currying functions in normal situations, as it would conflict with function calling. And if a currying syntax (or built-in) was introduced down the track, then these decorators would be inconsistent with them. Has anyone else noticed this issue? Andrew Durdin [*] Except those which don't allow for arguments [**] For a nice coincidence, PEP 309 suggested using '@' to mark curries From martin at v.loewis.de Sat Aug 7 14:52:04 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 14:52:17 2004 Subject: [Python-Dev] PEP-0006, Bug Fix Releases In-Reply-To: <4113C900.6060902@interlink.com.au> References: <4113C900.6060902@interlink.com.au> Message-ID: <4114D074.6080404@v.loewis.de> Anthony Baxter wrote: > 1. Backport bug fixes. If you fix a bug, and it seems appropriate, > port it to the CVS branch for the current bug fix release. If > you're unwilling or unable to backport it yourself, make a note > in the commit message. Would it be helpful to mandate a precise wording? > Starting with Python 2.0, all feature releases are required to have > a version number of the form X.Y; patch releases will always be of > the form X.Y.Z. Here is one instance of "patch release" left. > The process for managing patch releases is modeled in part on the cl > Tsystem [1]. This should have been left at "Tcl system". > [1] http://dev.scriptics.com:8080/cgi-bin/tct/tip/28.html This should be http://www.tcl.tk/cgi-bin/tct/tip/28.html at the moment, as Scriptics is gone. Regards, Martin From adurdin at gmail.com Sat Aug 7 14:59:02 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Sat Aug 7 14:59:08 2004 Subject: [Python-Dev] Re: Decorators with arguments are curries! In-Reply-To: <59e9fd3a04080705361b6314be@mail.gmail.com> References: <59e9fd3a04080705361b6314be@mail.gmail.com> Message-ID: <59e9fd3a040807055971518181@mail.gmail.com> On Sat, 7 Aug 2004 22:36:59 +1000, Andrew Durdin wrote: > # using curry as proposed in PEP 309 [**] > @foo_decorator > @curry(bar_decorator, "one param here") And that really should have been: # using rightcurry as mentioned in PEP 309 [**] @foo_decorator @rightcurry(bar_decorator, "one param here") But that's not a primary issue. The primary issues were: (1) That decorators with vs. without arguments look inconsistent, like function calls vs. function references, and (2) that consequentially, decorator syntax is implicitly introducing a non-viable currying syntax. From martin at v.loewis.de Sat Aug 7 15:00:13 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 15:00:19 2004 Subject: [Python-Dev] Decorators with arguments are curries! In-Reply-To: <59e9fd3a04080705361b6314be@mail.gmail.com> References: <59e9fd3a04080705361b6314be@mail.gmail.com> Message-ID: <4114D25D.7090207@v.loewis.de> Andrew Durdin wrote: > def bar_decorator(func, param): > print param > return func > > @foo_decorator > @bar_decorator("one param here") > def decorated_func(): > pass > > Here the first decorator statement is bare, while the second one > includes parentheses and an argument; the first one looks like a > function reference, while the second looks like a function call. Correct. And that is indeed the intended meaning. Did you try this out? It gives Traceback (most recent call last): File "b.py", line 9, in ? @foo_decorator TypeError: bar_decorator() takes exactly 2 arguments (1 given) (although, as you can see, the line number is off by one) See http://www.python.org/dev/doc/devel/ref/function.html on why this is so. > Most of my concern here is that this aspect of decorator syntax > appears to be implicitly introducing a currying syntax in one special > circumstance, which is then *not* transferable to currying functions > in normal situations, as it would conflict with function calling. And yet, the proposal does no such thing. Regards, Martin From adurdin at gmail.com Sat Aug 7 15:13:50 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Sat Aug 7 15:13:52 2004 Subject: [Python-Dev] Decorators with arguments are curries! In-Reply-To: <4114D25D.7090207@v.loewis.de> References: <59e9fd3a04080705361b6314be@mail.gmail.com> <4114D25D.7090207@v.loewis.de> Message-ID: <59e9fd3a04080706135e4f622b@mail.gmail.com> On Sat, 07 Aug 2004 15:00:13 +0200, "Martin v. L?wis" wrote: > Andrew Durdin wrote: > > > > Here the first decorator statement is bare, while the second one > > includes parentheses and an argument; the first one looks like a > > function reference, while the second looks like a function call. > > Correct. And that is indeed the intended meaning. Did you try this > out? It gives > > Traceback (most recent call last): > File "b.py", line 9, in ? > @foo_decorator > TypeError: bar_decorator() takes exactly 2 arguments (1 given) > > (although, as you can see, the line number is off by one) > > See > > http://www.python.org/dev/doc/devel/ref/function.html > > on why this is so. Ah! Quite so. The relevant quote would be: "The result [of the decorator expression] must be a callable, which is invoked with the function object as the only argument". I guess I didn't pay quite enough attention to the examples in PEP 318 to understand them properly :) From mwh at python.net Sat Aug 7 15:23:39 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 15:23:40 2004 Subject: [Python-Dev] PEP-0006, Bug Fix Releases In-Reply-To: <4113C900.6060902@interlink.com.au> (Anthony Baxter's message of "Sat, 07 Aug 2004 04:08:00 +1000") References: <4113C900.6060902@interlink.com.au> Message-ID: <2mllgr3wv8.fsf@starship.python.net> Anthony Baxter writes: > In addition, anyone from the Python community is free to suggest > patches for inclusion. Patches may be submitted specifically for > patch releases; they should follow the guidelines in PEP 3 [2]. > In general, though, it's probably better that a bug in a specific > release also be fixed on the HEAD as well as the branch. I thought the consensus policy was now stronger than this: bugfixes are tested on the trunk (unless the bug has gone away on the trunk as part of feature work or something). Submitting a patch that only applies to release2x-maint is not good form. > Bug fix releases are expected to occur at an interval of roughly > six months. This is only a guideline, however - obviously, if a > major bug is found, a bugfix release may be appropriate sooner. In > general, only the N-1 release will be under active maintenance at > any time. That is, during Python 2.4's development, Python 2.3 gets ^ only? > bugfix releases. Cheers, mwh -- But since your post didn't lay out your assumptions, your goals, or how you view language characteristics as fitting in with either, you're not a *natural* candidate for embracing Design by Contract <0.6 wink>. -- Tim Peters, giving Eiffel adoption advice From mwh at python.net Sat Aug 7 15:26:18 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 15:26:21 2004 Subject: [Python-Dev] pie-thon redux In-Reply-To: <5.1.1.6.0.20040806142605.03303d90@mail.telecommunity.com> (Phillip J. Eby's message of "Fri, 06 Aug 2004 14:28:30 -0400") References: <5.1.1.6.0.20040806142605.03303d90@mail.telecommunity.com> Message-ID: <2mhdrf3wqt.fsf@starship.python.net> "Phillip J. Eby" writes: > At 02:13 AM 8/7/04 +1000, Anthony Baxter wrote: >>As a break from the decorator thread, NTK (www.ntk.net) >>had this writeup of the Pie-Thon. It amused me. >> >>[snip] > > I'm not usually one for today's IM slang, but... > > OMG!!!! RTFLOL!!! :) Does this mean you're new to NTK? They also had the best write-up anywhere of the open sourcing of Principia/Zope: >> TRACKING << making good use of the things that we find Professional Web content systems companies, switching to Open Source - as recommended by their own *venture capitalist*? That's *sick*. ZOPE is a PERVERTED hybrid of the widely-regarded BOBO object-oriented CGI replacement, and the previously commercial PRINCIPIA Web application system. It's also got the Aqueduct relational database system thrown in. It's all free, source code is included, and even more horrifically, it's centered on the PYTHON language, that politically-correct, nicely-formatted, easy-to-read OPEN BLOODY SOURCE scripting language. ZOPE's creators will be releasing the source later today, so we haven't had a chance to play, but we hope it's bloody rubbish, otherwise we'll have nothing nasty to say at all. http://www.zope.org/ - nice clean well-written Website too. CURSE THEM! http://www.ntk.net/ Cheers, mwh -- -Dr. Olin Shivers, Ph.D., Cranberry-Melon School of Cucumber Science -- seen in comp.lang.scheme From python at rcn.com Sat Aug 7 03:31:19 2004 From: python at rcn.com (Raymond Hettinger) Date: Sat Aug 7 15:33:06 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) In-Reply-To: <4114B266.3020306@interlink.com.au> Message-ID: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer> > > I haven't seen this mentioned, so I'm going to throw in my suggestion. > > > > The following makes sense to me, uses previously invalid syntax, and > > uses the '.' punctuation in a way similar to existing/proposed usage: > > -1. This violates my "syntax should not look like grit" rule. Also, the dot prefix should probably be kept free in case a WITH statement is ever introduced. Raymond From mwh at python.net Sat Aug 7 15:56:07 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 15:56:09 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com> (Andrew Durdin's message of "Sat, 7 Aug 2004 22:06:36 +1000") References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <4114A171.3050809@v.loewis.de> <59e9fd3a04080705067633a790@mail.gmail.com> Message-ID: <2md6233vd4.fsf@starship.python.net> Andrew Durdin writes: > @ at least is consistently on the 2 key (except, of course, for all > those European keyboards. I guess their differences are enough to > knock down any argument about typing convenience :-) @ is shift-2 on my Mac and shift-' on my PC. But, hell, # is option-3 on my Mac and I live with that. Cheers, mwh -- I've even been known to get Marmite *near* my mouth -- but never actually in it yet. Vegamite is right out. UnicodeError: ASCII unpalatable error: vegamite found, ham expected -- Tim Peters, comp.lang.python From pf_moore at yahoo.co.uk Sat Aug 7 16:12:53 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Sat Aug 7 16:12:46 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <411429FB.6080603@gradient.cis.upenn.edu> Message-ID: Mike Coleman writes: > Edward Loper writes: >> I wouldn't lose heart over the lack of response -- the current >> flood of decorator messages tends to drown out everything else. > > No, not at all. I was hoping I might benefit from @-fatigue, though. > :-) I was interested in your message as a break from @decorators, but I couldn't muster the energy to think about the proposal because I've been reading too many @decorator messages. I guess you won one way and lost the other :-) Seriously, although I don't use the re module much (overreaction to an early exposure to Perl, I guess) the idea seems useful. I'm not entirely sure about the implementation, though - I dislike the name structmatch, as it doesn't say what's going on very well. At first I thought that having a separate method was overkill, but given that re.search and re.match both exist, I take that back. Maybe a better name would be re.parse? The function has a "feel" of parsing a string according to a pattern. The only other comment I have is that the semantics seem pretty complex - I think that in practice, they do more or less what you want them to, but the description is pretty obscure. And although I can see that the error return has some value, I suspect that it might actually complicate real use. A suggestion - would it be possible to implement re.structmatch as a pure Python prototype, to thrash out some of the usability questions? If the function appears valuable in such a form, arguing for incorporation into the re module would be a lot easier. Hope this helps, Paul. -- The only reason some people get lost in thought is because it's unfamiliar territory -- Paul Fix From mwh at python.net Sat Aug 7 16:16:09 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 16:16:11 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching In-Reply-To: (Mike Coleman's message of "06 Aug 2004 19:46:27 -0400") References: <2mbrho5o6c.fsf@starship.python.net> Message-ID: <2m7jsb3ufq.fsf@starship.python.net> Mike Coleman writes: > Michael Hudson writes: >> Generally, it strikes me as mildly useful. An implementation would >> surely help :-) > > If yours is the most positive review, there probably won't be one. :-) Well, I thought *someone* should say *something*, just to stop the proto-pep being lost. Have you sent it to the PEP editor yet? I think it's ready enough. The reason that I only said "mildly useful" was because I didn't think "OMG! I would so have liked to use this yesterday" or anything like that... > Seriously, though, I'd readily concede that there are different > approaches to solving this problem, and I was hoping for some > feedback and suggestions before I start whacking at the code. I > also have considered the possibility that although this has been > driving me nuts that few others will care--if this is true, it > probably *shouldn't* go in the core libraries. It depends a bit on whether it can be implemented as an addition in a non-painful way, or whether editing the sre code makes it very much easier. I don't know the answer, mind. >>> [mismatch index return] >> I don't really like this idea; it seems to be that it would be more >> helpful to raise an exception and attach this data to the exception. >> But that's a bit inconsisent with how match and search work. > > Hmm. Yeah, if it weren't for the value of matching re.match semantics, it > would make a lot more sense for this to be an exception. As it is, I could > see it going either way. I think you could make a case for raising an exception; the point of structmatch is to extract information, whereas at least sometimes you call re.match just to see if the data matches the pattern. > Or I suppose you could add a flag to determine which (but ugh). Err, let's not go there. Cheers, mwh -- Lisp nearing the age of 50 is the most modern language out there. GC, dynamic, reflective, the best OO model extant including GFs, procedural macros, and the only thing old-fashioned about it is that it is compiled and fast. -- Kenny Tilton, comp.lang.python From amk at amk.ca Sat Aug 7 16:23:27 2004 From: amk at amk.ca (A.M. Kuchling) Date: Sat Aug 7 16:23:48 2004 Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows Message-ID: <20040807142327.GA2529@rogue.amk.ca> A recent patch makes asyncore.py's .set_reuse_addr method work on Windows, which needs a different socket option: if sys.platform == 'win32': reuse_constant = socket.SO_EXCLUSIVEADDRUSE else: reuse_constant = socket.SO_REUSEADDR Question: should the condition be sys.platform.startswith('win') for the sake of win64, or other Windows variants? (I don't know which values sys.platform can have on Windows.) --amk From pf_moore at yahoo.co.uk Sat Aug 7 16:27:10 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Sat Aug 7 16:27:02 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <7jsb9g75.fsf@yahoo.co.uk> Mike Thompson writes: > Terry Reedy wrote: > >> Is a space allowed after the prefic character, whatever it might be? >> With >> def foo(): >> | classmethod >> | accepts(int,int) >> | returns(float) >> > > > Thats it! I love it. Note that there is no implementation of this currently available (whereas changing @ to | in the existing syntax is likely to be a trivial change). Proponents of this variant (or any other without an implementation available) will ultimately have to deliver code. I've added notes to the Wiki as to which variants have implementations, FWIW. Paul -- Instant gratification takes too long -- Carrie Fisher From martin at v.loewis.de Sat Aug 7 16:29:39 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 7 16:29:40 2004 Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows In-Reply-To: <20040807142327.GA2529@rogue.amk.ca> References: <20040807142327.GA2529@rogue.amk.ca> Message-ID: <4114E753.2050303@v.loewis.de> A.M. Kuchling wrote: > Question: should the condition be sys.platform.startswith('win') for > the sake of win64, or other Windows variants? (I don't know which > values sys.platform can have on Windows.) Win64 also uses "win32" as sys.platform. If you really want to find out this is not x86, you need to look at the compiler ID in sys.version. Regards, Martin From amk at amk.ca Sat Aug 7 16:51:42 2004 From: amk at amk.ca (A.M. Kuchling) Date: Sat Aug 7 16:52:01 2004 Subject: [Python-Dev] re.split on empty patterns Message-ID: <20040807145142.GB2529@rogue.amk.ca> The re.split() method ignores zero-length pattern matches. Patch #988761 adds an emptyok flag to split that causes zero-length matches to trigger a split. For example: >>> re.split(r'\b', 'this is a sentence')# does nothing; \b is always length 0 ['this is a sentence'] >>> re.split(r'\b', 'this is a sentence', emptyok=True) ['', 'this', ' ', 'is', ' ', 'a', ' ', 'sentence', ''] Without the patch, the various zero-length assertions are pretty useless; with it, they can serve a purpose with split(): >>> re.split(r'(?m)$', 'line1\nline2\n', emptyok=True) ['line1', '\nline2', '\n', ''] >>> # Split file into sections >>> re.split("(?m)(?=^[[])", """[section1] foo=bar [section2] coyote=wiley """, emptyok=True) ['', '[section1]\nfoo=bar\n\n', '[section2]\ncoyote=wiley\n'] Zero-length matches often result in a '' at the beginning or end, or between characters, but I think users can handle that. IMHO this feature is clearly useful, and would be happy to commit the patch as-is. Question: do we want to make this option the new default? Existing patterns that can produce zero-length matches would change their meanings: >>> re.split('x*', 'abxxxcdefxxx') ['ab', 'cdef', ''] >>> re.split('x*', 'abxxxcdefxxx', emptyok=True) ['', 'a', 'b', '', 'c', 'd', 'e', 'f', '', ''] (I think the result of the second match points up a bug in the patch; the empty strings in the middle seem wrong to me. Assume that gets fixed.) Anyway, we therefore can't just make this the default in 2.4. We could trigger a warning when emptyok is not supplied and a split pattern results in a zero-length match; users could supply emptyok=False to avoid the warning. Patterns that never have a zero-length match would never get the warning. 2.5 could then set emptyok to True. Note: raising the warning might cause a serious performance hit for patterns that get zero-length matches a lot, which would make 2.4 slower in certain cases. Thoughts? Does this need a PEP? --amk From amk at amk.ca Sat Aug 7 17:02:29 2004 From: amk at amk.ca (A.M. Kuchling) Date: Sat Aug 7 17:02:48 2004 Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows In-Reply-To: <4114E753.2050303@v.loewis.de> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> Message-ID: <20040807150229.GC2529@rogue.amk.ca> On Sat, Aug 07, 2004 at 04:29:39PM +0200, "Martin v. L?wis" wrote: > Win64 also uses "win32" as sys.platform. If you really want to find > out this is not x86, you need to look at the compiler ID in sys.version. OK; I just care about platforms where the alternative constant is necessary. It sounds to me like win32 is therefore the only relevant sys.platform string to check for. Thanks! (Hm... should it check for "cygwin", too? If any Cygwin maintainers have suggestions, please let me know.) --amk From amk at amk.ca Sat Aug 7 17:27:08 2004 From: amk at amk.ca (A.M. Kuchling) Date: Sat Aug 7 17:27:26 2004 Subject: [Python-Dev] Pickler()'s bin argument Message-ID: <20040807152708.GA2670@rogue.amk.ca> PEP 307 says this about Pickler: Passing 'bin' as a keyword argument is deprecated, and a PendingDeprecationWarning is issued in this case. You have to invoke the Python interpreter with -Wa or a variation on that to see PendingDeprecationWarning messages. In Python 2.4, the warning class may be upgraded to DeprecationWarning. Should I change it to become a DeprecationWarning in 2.4? --amk From barry at python.org Sat Aug 7 17:54:07 2004 From: barry at python.org (Barry Warsaw) Date: Sat Aug 7 17:54:11 2004 Subject: [Python-Dev] RE: [spambayes-dev] bug in imap filter or in email package In-Reply-To: References: Message-ID: <1091894046.1064.29.camel@anthem.wooz.org> On Wed, 2004-08-04 at 21:56, Tony Meyer wrote: > There has been a little bit of discussion on spambayes-dev about a bug with > the 2.4a1 email package, where header lines that end with \r\n are not > treated correctly (the value ends up with a \r at the end). > > A SF bug was opened for this: > > [ 1002475 ] email message parser doesn't handle \r\n correctly > > > I've created a patch to fix this, and a couple of tests to add to > test_email.py: > [ 1003693 ] Fix for 1002475 (Feedparser not handling \r\n correctly) > 305470> > > If someone would like to review this and check it in after 2.4a2 is all done > that would be great. Maybe someone at the bug day? (I might come along to > that, but it's the middle of the night, so probably not). Patch looks good to me. I'm checking them in (after a little stylistic hydrant-pissing :). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040807/be5d5348/attachment.pgp From pje at telecommunity.com Sat Aug 7 17:59:04 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Aug 7 17:55:00 2004 Subject: [Python-Dev] Decorators with arguments are curries! In-Reply-To: <59e9fd3a04080705361b6314be@mail.gmail.com> Message-ID: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com> At 10:36 PM 8/7/04 +1000, Andrew Durdin wrote: >The first assignment to a is binding a reference to a function; the >second is calling the function. This is a very significant difference >in python, and I'm concerned that all the current proposed decorator >syntaxes[*] are liable to cause confusion on this point. For example: > >def foo_decorator(func): > print "no params to this" > return func > >def bar_decorator(func, param): > print param > return func > >@foo_decorator >@bar_decorator("one param here") >def decorated_func(): > pass > >Here the first decorator statement is bare, while the second one >includes parentheses and an argument; the first one looks like a >function reference, while the second looks like a function call. Your example will fail, saying that bar_decorator is being called without enough arguments. Decorator syntax does *not* provide currying. You have to write something like this: def bar_decorator(param): def decorate(func): print param return func return decorate in order for your example to work. From tim.peters at gmail.com Sat Aug 7 17:56:09 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 17:56:11 2004 Subject: [Python-Dev] Pickler()'s bin argument In-Reply-To: <20040807152708.GA2670@rogue.amk.ca> References: <20040807152708.GA2670@rogue.amk.ca> Message-ID: <1f7befae04080708563a3735be@mail.gmail.com> [A.M. Kuchling] > PEP 307 says this about Pickler: > > Passing 'bin' as a keyword argument is deprecated, > and a PendingDeprecationWarning is issued in this case. You have > to invoke the Python interpreter with -Wa or a variation on that > to see PendingDeprecationWarning messages. In Python 2.4, the > warning class may be upgraded to DeprecationWarning. > > Should I change it to become a DeprecationWarning in 2.4? +1. 2.3's been current a long time, and AFAICT PendingDeprecationWarning isn't a lot more warning than a comment in the middle of cPickle.c. From guido at python.org Sat Aug 7 18:12:38 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 18:12:44 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: Your message of "Sat, 07 Aug 2004 00:12:19 EDT." References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> Message-ID: <200408071612.i77GCcK10255@guido.python.org> > > > - change the language ref that talks about this to state that > > > although Python currently doesn't use these symbols, this is not > > > stopping a future version of Python from using them for some new > > > feature. > > > > Does that really have to be stated? > > All that is needed is the addition of 'currently' in the sentence about > illegality. Compare with "The Python compiler currently generates the > following byte code instructions. " (LibRef 18.10.1) which has the same > word to guard against the same false presumption of stability and > consequent complaints. I still think it shouldn't be needed. Do we have to add 'currently' to every statement about the language? That doesn't make sense. The reference manual's title page already includes a version number. Shouldn't that be sufficient warning for those who want to interpret any part of the manual as a promise for all future? I really want to take a hard stance on this, because I believe the only reason this came up was that someone needed to find an argument against '@'. I don't think their argument would have a chance in court, so there's no reason to give in to them. Fight the trend to add silly disclaimers everywhere! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 7 18:17:42 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 18:17:47 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: Your message of "Sat, 07 Aug 2004 00:35:17 EDT." <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <200408071617.i77GHgs10281@guido.python.org> > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). I want to mention one argument for '@' that doesn't apply to '|': it will be familiar to Java programmers. > I'm including both of the following variants: > > |classmethod > |accepts(int, int) > |returns(float) > def foo(arg1, arg2): > ... > > def foo(arg1, arg2): > |classmethod > |accepts(int, int) > |returns(float) > ... > > (I prefer the second, but would be happy with either.) In the discussion on decorators months ago, solutions involving special syntax inside the block were ruled out early on. Basically, you shouldn't have to peek inside the block to find out important external properties of the function. (Yes, I know that docstrings violate this principle. For the record, it would be better if they were prefix too; and a prefix decorator syntax will let us fix this in the future but introducing a "doc" decorator.) Please add this to the Wiki for me. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 7 18:29:44 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 18:29:49 2004 Subject: [Python-Dev] Re: Decorators with arguments are curries! In-Reply-To: Your message of "Sat, 07 Aug 2004 22:59:02 +1000." <59e9fd3a040807055971518181@mail.gmail.com> References: <59e9fd3a04080705361b6314be@mail.gmail.com> <59e9fd3a040807055971518181@mail.gmail.com> Message-ID: <200408071629.i77GTiT10357@guido.python.org> > The primary issues were: (1) That decorators with vs. without > arguments look inconsistent, like function calls vs. function > references, and (2) that consequentially, decorator syntax is > implicitly introducing a non-viable currying syntax. I think you're mistaken. When using @foo, foo should be a function taking one argument; when using @foo(x,y,z), the *call* to foo(x,y,z) should return a function of one argument (or an equivalent callable object, of course). This is often done by defining an inner "helper" function, e.g.: def funcattrs(**kwds): def helper(func): func.__dict__.update(kwds) return helper @funcattr(counter=42, obsolete=True) def foobar(): pass --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 7 18:37:00 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 18:37:06 2004 Subject: [Python-Dev] Recommended way to tell platform In-Reply-To: Your message of "Sat, 07 Aug 2004 16:29:39 +0200." <4114E753.2050303@v.loewis.de> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> Message-ID: <200408071637.i77Gb0w10415@guido.python.org> > A.M. Kuchling wrote: > > Question: should the condition be sys.platform.startswith('win') for > > the sake of win64, or other Windows variants? (I don't know which > > values sys.platform can have on Windows.) > > Win64 also uses "win32" as sys.platform. If you really want to find > out this is not x86, you need to look at the compiler ID in sys.version. > > Regards, > Martin This does bring up an interesting question: what's the recommended way to distinguish a certain platform? There's os.name, sys.platform, the platform module, and I've also seen code testing for os.sep=='/'. I still prefer hasattr(, ) whenever applicable, e.g. preferring hasattr(os, 'fork') over os.name=='posix' (or os.name!='nt' :-), but sometimes that's not possible. What should be the preferred way? (It may be impossible to say because there are different use cases, but probably one of the most important cases is simply distinguishing Windows from the rest -- how should that be done?) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 7 18:38:34 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 18:38:40 2004 Subject: [Python-Dev] Pickler()'s bin argument In-Reply-To: Your message of "Sat, 07 Aug 2004 11:27:08 EDT." <20040807152708.GA2670@rogue.amk.ca> References: <20040807152708.GA2670@rogue.amk.ca> Message-ID: <200408071638.i77GcYm10437@guido.python.org> > PEP 307 says this about Pickler: > > Passing 'bin' as a keyword argument is deprecated, > and a PendingDeprecationWarning is issued in this case. You have > to invoke the Python interpreter with -Wa or a variation on that > to see PendingDeprecationWarning messages. In Python 2.4, the > warning class may be upgraded to DeprecationWarning. > > Should I change it to become a DeprecationWarning in 2.4? Sure. --Guido van Rossum (home page: http://www.python.org/~guido/) From colanderman at gmail.com Sat Aug 7 18:57:48 2004 From: colanderman at gmail.com (Chris King) Date: Sat Aug 7 18:57:51 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) In-Reply-To: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer> References: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer> Message-ID: <875c7e07040807095770541225@mail.gmail.com> On Fri, 6 Aug 2004 21:31:19 -0400, Raymond Hettinger wrote: > > > I haven't seen this mentioned, so I'm going to throw in my > suggestion. > > > > > > The following makes sense to me, uses previously invalid syntax, and > > > uses the '.' punctuation in a way similar to existing/proposed > usage: > > > > -1. This violates my "syntax should not look like grit" rule. > > Also, the dot prefix should probably be kept free in case a WITH > statement is ever introduced. Same here -- I'd rather see .foo at function level (i.e. not nested in a with block) provide true function attributes, not decorators. From colanderman at gmail.com Sat Aug 7 19:11:04 2004 From: colanderman at gmail.com (Chris King) Date: Sat Aug 7 19:11:06 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <875c7e070408071011495d1033@mail.gmail.com> On Sat, 07 Aug 2004 00:35:17 -0400, Edward Loper wrote: > |classmethod > |accepts(int, int) > |returns(float) > def foo(arg1, arg2): > ... Personally, I'd prefer @ over |: | looks too much like ASCII art (like someone might draw in a multiline comment in C); whereas @ at least looks like it's doing something. Though for me, @ brings to mind preprocessor directives, maybe that's for the best. > def foo(arg1, arg2): > |classmethod > |accepts(int, int) > |returns(float) > ... This does look prettier (especially the space version), but it looks even more like ASCII art. So for the record, assuming I'm -1 on @, I'm -1.5 on this ;) From bob at redivi.com Sat Aug 7 19:18:23 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat Aug 7 19:18:31 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <4114A171.3050809@v.loewis.de> <59e9fd3a04080705067633a790@mail.gmail.com> Message-ID: On Aug 7, 2004, at 8:06 AM, Andrew Durdin wrote: > I don't like the pipe at all, for two reasons: > > One: as has already been mentioned, in many fonts it is easily > confused with lowercase L: l | Actually when I initially brought up this concern, it was an uppercase "i" that I was talking about. Lowercase L of course also has this problem, but uppercase "i" is a relatively commonly used prefix. -bob From mkc at mathdogs.com Sat Aug 7 19:33:05 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sat Aug 7 19:33:10 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <2mbrho5o6c.fsf@starship.python.net> <2m7jsb3ufq.fsf@starship.python.net> Message-ID: Michael Hudson writes: > Well, I thought *someone* should say *something*, just to stop the > proto-pep being lost. Have you sent it to the PEP editor yet? I > think it's ready enough. I'm happy you did--I was just joking. Yes, I've sent it to the PEP guys, but no response yet. > The reason that I only said "mildly useful" was because I didn't think > "OMG! I would so have liked to use this yesterday" or anything like > that... Sure, I understand. I run into a lot of these cases at work, where the domain is bioinformatics (lots of pattern matching and parsing of ad hoc file formats). > It depends a bit on whether it can be implemented as an addition in a > non-painful way, or whether editing the sre code makes it very much > easier. I don't know the answer, mind. I'm thinking (hoping) that it could piggyback on the current re.match implementation in a fairly clean way. The algorithm is the same--we just have to "notice" a few more things while we're carrying it out and arrange to return those noticed things at the end. > I think you could make a case for raising an exception; the point of > structmatch is to extract information, whereas at least sometimes you > call re.match just to see if the data matches the pattern. Yeah, I'm leaning in this direction, too. Mike From tim.peters at gmail.com Sat Aug 7 19:33:44 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 19:33:47 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <200408071612.i77GCcK10255@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com> <4113AF47.1010405@interlink.com.au> <200408061634.i76GYEZ07623@guido.python.org> <200408071612.i77GCcK10255@guido.python.org> Message-ID: <1f7befae04080710335001c6b9@mail.gmail.com> [Guido] > I still think it shouldn't be needed. Do we have to add 'currently' > to every statement about the language? That doesn't make sense. The > reference manual's title page already includes a version number. > Shouldn't that be sufficient warning for those who want to interpret > any part of the manual as a promise for all future? Yes. > I really want to take a hard stance on this, because I believe the > only reason this came up was that someone needed to find an argument > against '@'. At least two reasonably popular Python tools use @ heavily now, and their authors didn't appear to give a rip about decorators one way or the other. The use of @ for any purpose in the core would have elicited similar concern. > I don't think their argument would have a chance in court, I believe they agree with that (partly because they both said so ). > so there's no reason to give in to them. Courts are adversarial. You don't want an adversarial relationship with Python users -- there are lots of things to consider besides what a court would say. > Fight the trend to add silly disclaimers everywhere! +1. OTOH, I'm also +1 on picking a character and promising (in the reference manual) that the language will never use it, to give authors of these kinds of tools a way to live peacefully with Python evolution. @ seems like a good choice for that. From mkc at mathdogs.com Sat Aug 7 19:46:10 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sat Aug 7 19:46:16 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <411429FB.6080603@gradient.cis.upenn.edu> Message-ID: Paul Moore writes: > I was interested in your message as a break from @decorators, but I > couldn't muster the energy to think about the proposal because I've > been reading too many @decorator messages. I guess you won one way and > lost the other :-) @:-) > Maybe a better name would be re.parse? The function has a "feel" of > parsing a string according to a pattern. This is a nice suggestion. The main objection I can think of is that we might be stealing that name from some future idea that would more deserve it. > The only other comment I have is that the semantics seem pretty > complex - I think that in practice, they do more or less what you want > them to, but the description is pretty obscure. Yeah, probably the doc will need to be carefully written, with good examples, to make this go down easily. I hope that for practical cases it'll be relatively straightforward. > And although I can see that the error return has some value, I suspect that > it might actually complicate real use. Someone else suggested throwing an exception instead, which I'm now leaning in favor of, too. > A suggestion - would it be possible to implement re.structmatch as a > pure Python prototype, to thrash out some of the usability questions? > If the function appears valuable in such a form, arguing for > incorporation into the re module would be a lot easier. I suspect that in this case a pure Python implementation might actually be a lot harder than implementing it as a C patch. This is because so much of the work is already done in the current re module; the C patch would probably not be that big. In order to do it in Python I'd end up reimplementing re.match in Python, which I wouldn't look forward to. It's because of this that I'm trying to get as much preliminary comment as I can. Cheers, Mike From guido at python.org Sat Aug 7 19:53:28 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 19:53:34 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) In-Reply-To: Your message of "Sat, 07 Aug 2004 12:57:48 EDT." <875c7e07040807095770541225@mail.gmail.com> References: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer> <875c7e07040807095770541225@mail.gmail.com> Message-ID: <200408071753.i77HrTL10619@guido.python.org> > Same here -- I'd rather see .foo at function level (i.e. not nested in > a with block) provide true function attributes, not decorators. Bleah on that idea. The body of the function should express what it does when it is called. Function attributes are for who calls it (roughly). --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Sat Aug 7 19:58:16 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 19:58:19 2004 Subject: [Python-Dev] re.split on empty patterns In-Reply-To: <20040807145142.GB2529@rogue.amk.ca> References: <20040807145142.GB2529@rogue.amk.ca> Message-ID: <1f7befae040807105878b47c61@mail.gmail.com> [A.M. Kuchling] wrote: > The re.split() method ignores zero-length pattern matches. Patch > #988761 adds an emptyok flag to split that causes zero-length matches > to trigger a split. ... > IMHO this feature is clearly useful, Yes it is! Or, more accurately, it can be, when it's intended to match an empty string. It's a bit fuzzy because regexps are so error-prone, and writing a regexp that matches an empty string by accident is easy. > and would be happy to commit the patch as-is. Haven't looked at the patch, though. > Question: do we want to make this option the new default? Existing > patterns that can produce zero-length matches would change their > meanings: > > >>> re.split('x*', 'abxxxcdefxxx') > ['ab', 'cdef', ''] > >>> re.split('x*', 'abxxxcdefxxx', emptyok=True) > ['', 'a', 'b', '', 'c', 'd', 'e', 'f', '', ''] > > (I think the result of the second match points up a bug in the patch; > the empty strings in the middle seem wrong to me. Assume that gets > fixed.) Agreed. > Anyway, we therefore can't just make this the default in 2.4. We > could trigger a warning when emptyok is not supplied and a split > pattern results in a zero-length match; users could supply > emptyok=False to avoid the warning. Patterns that never have a > zero-length match would never get the warning. 2.5 could then set > emptyok to True. > > Note: raising the warning might cause a serious performance hit for > patterns that get zero-length matches a lot, which would make 2.4 > slower in certain cases. If you don't intend to change the default, there's no problem. I like "no problem". This isn't so useful so often that it can't afford to wait for Python 3 to change. In the meantime, "emptyok" is an odd name since it's always "ok" to have an empty match. "split0=True" reads better to me, since the effect is to split on a 0-length match. split_on_empty_match would be too wordy. > Thoughts? Does this need a PEP? It will if an argument starts now . From s.percivall at chello.se Sat Aug 7 20:02:34 2004 From: s.percivall at chello.se (Simon Percivall) Date: Sat Aug 7 20:02:36 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <4113BACA.6040801@ieee.org> References: <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> Message-ID: I just wanted to say that I believe it should be allowed to decorate classes. There's not reason enough (reading the thread linked to by the PEP) to limit decorators this way. Like said several times in that thread, metaclasses are harder (to write and to understand) than decorators. Also, considering that you said you would have actually wanted docstrings above the definition of a function, wouldn't this apply to classes as well? Anyway, it's an artificial limit and it would be better to be able to test it out during the alpha. //Simon > Guido van Rossum wrote: > > > > Some people have argued that it should work for classes (Java and C# > > decorators work for any declaration). I've never been convinced, > > although I'm not dead set against it either. > > > > It would seem that the functionality isn't quite as useful there; you > > can get most of the same effects with metaclasses. I'm not aware of > > any idiom that takes a class and passes it through some kind of > > wrapper or transformation *after* the class declaration; if somebody > > is using this, it surely must be a very rare use indeed. > > From colanderman at gmail.com Sat Aug 7 20:07:35 2004 From: colanderman at gmail.com (Chris King) Date: Sat Aug 7 20:07:39 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) In-Reply-To: <200408071753.i77HrTL10619@guido.python.org> References: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer> <875c7e07040807095770541225@mail.gmail.com> <200408071753.i77HrTL10619@guido.python.org> Message-ID: <875c7e070408071107cfab038@mail.gmail.com> On Sat, 07 Aug 2004 10:53:28 -0700, Guido van Rossum wrote: > > Same here -- I'd rather see .foo at function level (i.e. not nested in > > a with block) provide true function attributes, not decorators. > > Bleah on that idea. The body of the function should express what it > does when it is called. Function attributes are for who calls it > (roughly). I'd still rather see .foo kept free for that purpose (in that context), in case we can convince you of the merits of function attributes sometime in the future ;) From bob at redivi.com Sat Aug 7 20:18:51 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat Aug 7 20:19:01 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: References: <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> Message-ID: <3BCE7F74-E89E-11D8-B7F3-000A95686CD8@redivi.com> On Aug 7, 2004, at 2:02 PM, Simon Percivall wrote: >> Guido van Rossum wrote: >> > >> > Some people have argued that it should work for classes (Java and C# >> > decorators work for any declaration). I've never been convinced, >> > although I'm not dead set against it either. >> > >> > It would seem that the functionality isn't quite as useful there; >> you >> > can get most of the same effects with metaclasses. I'm not aware of >> > any idiom that takes a class and passes it through some kind of >> > wrapper or transformation *after* the class declaration; if somebody >> > is using this, it surely must be a very rare use indeed. >> > > I just wanted to say that I believe it should be allowed to decorate > classes. There's not reason enough (reading the thread linked to by the > PEP) to limit decorators this way. Like said several times in that > thread, metaclasses are harder (to write and to understand) than > decorators. > > Also, considering that you said you would have actually wanted > docstrings above the definition of a function, wouldn't this apply > to classes as well? > > Anyway, it's an artificial limit and it would be better to be able > to test it out during the alpha. I've always been +1 for allowing them on classes too for the reasons that you state, and I would use them if they were there. However, since we do have metaclasses and I know how to use them it's not nearly as important to me as function decorators and I'd rather not get in another big thread about it. Note that you could make a docstring decorator that does just this.. def doc(docstring): def doc(fn): fn.__doc__ = docstring return fn return doc @doc("""foo does nothing and takes no arguments""") def foo(): pass It could of course work the same way for classes too. -bob From tim.peters at gmail.com Sat Aug 7 20:25:09 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 20:25:13 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <4114B226.4020405@interlink.com.au> References: <41145C05.1020501@gradient.cis.upenn.edu> <200408071127.04568.heikowu@ceosg.de> <4114B226.4020405@interlink.com.au> Message-ID: <1f7befae040807112543238881@mail.gmail.com> [Anthony Baxter] > Lest my previous post saying I could "live with" the > pipe symbol be misinterpreted, I still prefer @ over |. > My main reason for preferring the former is that it's > far more obvious in a page of code (I tried both). Hmm. Why would you *want* decorator annotations to be especially visible on a page of code? Indentation and blank lines still make class, method, and function boundaries clear, and that's the important "page level" structure. Decorators are details. > The pipe tends to blend in to a disturbing degree. Indeed, for me that's a reason to prefer | over @. BTW, while the PEP wasn't clear on this point last I looked, I hope it allows for whitespace after the | (or @, or =, or whatever it is). > Making it a different colour in an editor also doesn't seem to help > that much. Even better . > I'm also concerned that we don't make a horrible > precedent here - that a new language feature is > changed solely because other tools that were playing > outside the boundaries of the language will be > impacted. This would be an utter pain in the arse > going forward. Well, this particular language feature has changed every week for most of a year. If we're at the point where a change actually *helps* someone, I'm not gonna hold that against it. From tim.peters at gmail.com Sat Aug 7 20:35:25 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 20:35:27 2004 Subject: [Python-Dev] Recommended way to tell platform In-Reply-To: <200408071637.i77Gb0w10415@guido.python.org> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> <200408071637.i77Gb0w10415@guido.python.org> Message-ID: <1f7befae04080711351466da96@mail.gmail.com> [Guido] > This does bring up an interesting question: what's the recommended way > to distinguish a certain platform? There's os.name, sys.platform, the > platform module, and I've also seen code testing for os.sep=='/'. I ask myself that question every time I need to do it. Out of sheer momentum now, I almost always use if sys.platform in ('win32',): I dislike using os.name == 'nt', mostly because if we asked all Windows Python users which OS they use, only a tiny percentage would answer "NT". Since NT is a dying OS, 'nt' is an increasingly odd-looking thing to test agaist. OTOH, I have no idea when or why sys.platform == "win32" would give a different result than os.name == "nt". > I still prefer hasattr(, ) whenever applicable, > e.g. preferring hasattr(os, 'fork') over os.name=='posix' (or > os.name!='nt' :-), but sometimes that's not possible. Like whether to use time.time or time.clock. > What should be the preferred way? (It may be impossible to say > because there are different use cases, but probably one of the most > important cases is simply distinguishing Windows from the rest -- how > should that be done?) My way -- unless it's broken in some way I haven't bumped into yet. From rafaelemoranb at yahoo.es Sat Aug 7 20:36:10 2004 From: rafaelemoranb at yahoo.es (Rafael Ernesto Moran Bautista) Date: Sat Aug 7 20:36:12 2004 Subject: [Python-Dev] hi I am new in the list Message-ID: <20040807183610.77951.qmail@web41410.mail.yahoo.com> hi, my name is Rafael Morán and I am new in this list, I hope help in this an interesting project called Python. I think could help in these topics: Translations (my native language is spanish and know a little of english), Python programming, C/C++ programming. Att. Rafael Morán __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail From mwh at python.net Sat Aug 7 20:40:46 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 20:40:48 2004 Subject: [Python-Dev] Recommended way to tell platform In-Reply-To: <1f7befae04080711351466da96@mail.gmail.com> (Tim Peters's message of "Sat, 7 Aug 2004 14:35:25 -0400") References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> <200408071637.i77Gb0w10415@guido.python.org> <1f7befae04080711351466da96@mail.gmail.com> Message-ID: <2mvffu3i6p.fsf@starship.python.net> Tim Peters writes: > OTOH, I have no idea when or why sys.platform == "win32" would give a > different result than os.name == "nt". Probably never, but os.name == 'posix' covers a huge variety of sins, so which one you test affects the 'feel' of the code somewhat. Cheers, mwh -- We did requirements and task analysis, iterative design, and user testing. You'd almost think programming languages were an interface between people and computers. -- Steven Pemberton (one of the designers of Python's direct ancestor ABC) From tjreedy at udel.edu Sat Aug 7 20:45:32 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Aug 7 20:45:39 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? References: Message-ID: "Tony Meyer" wrote in message news:ECBA357DDED63B4995F5C1F5CBE5B1E86C36A8@its-xchg4.massey.ac.nz... >If the syntax does end up along these lines (and I'm expressing no opinion >either way), then maybe it would be worth explicitly telling people (i.e. in >PEP 8) that they ought to put a blank line before decorators to make it >clear? Does PEP 8 tell people that they should put a blank before comment lines introducing a function? >Same deal for "don't do '@this @and_this @on_the_same_line'" > (unless that loses it's validity). In PEP 8, maybe, but not the Ref Manual (unless grammatically prohibited). While uglier, this form does not stike me as being wholely different from the quasilist proposals that would spell the above [this, and_this, on_the_same_line]. Terry J. Reedy From pf_moore at yahoo.co.uk Sat Aug 7 21:13:37 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Sat Aug 7 21:13:26 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <411429FB.6080603@gradient.cis.upenn.edu> Message-ID: Mike Coleman writes: > I suspect that in this case a pure Python implementation might > actually be a lot harder than implementing it as a C patch. This is > because so much of the work is already done in the current re > module; the C patch would probably not be that big. In order to do > it in Python I'd end up reimplementing re.match in Python, which I > wouldn't look forward to. Fairy nuff. I was assuming (on no particular basis - you've obviously looked into this in a lot more detail than me) that you could implement it using re.match - but on reflection, I can see that this would probably not be true. > It's because of this that I'm trying to get as much preliminary > comment as I can. That's reasonable. As I said before, I have a tendency to avoid the re module too much (a perfect recent example was some code I wrote splitting a string on a ":" character, then laboriously trimming each part - I really should have done re.split on \s*:\s*). On that basis, your examples give me a mixed reaction - the results impress me with the ability to get exactly the right information out in a single parse, but the regular expressions needed look utterly unmaintainable. I suppose I'm saying that if I liked re's, I'd love this feature. As it is, it leaves me looking for the same feature but with a better pattern syntax. But your examples reinforce for me that "parse" is a reasonable name for the method. Paul. -- Home computers are being called upon to perform many new functions, including the consumption of homework formerly eaten by the dog -- Doug Larson From jhylton at gmail.com Sat Aug 7 21:29:19 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Sat Aug 7 21:29:25 2004 Subject: [Python-Dev] new failing test -- test_compiler Message-ID: I added a test for the compiler package, because I realized it would easily fall out of date if the language changed (say, decorators were added) and the compiler package wasn't updated. I added a test with a resource, so it would only fail if you run regrtest.py -u all. I think that's a little lame, but better than letting us get into beta without getting the compiler package fixed. At any rate, the test become somewhat more interesting than I expected because it fails in an unexpected way: Fatal Python error: deletion of interned string failed Aborted (core dumped) My best guess is that this is a stack overflow. In particular, gdb shows that the failure is 388 stack levels deep. The compiler package has very deep function calls, because it uses recursion to traverse abstract syntax trees. (I know, I know, we should just rewrite the compiler to use iterator instead of recursion .) Inside one of those deeply nested calls, it triggers GC which attempts to free a deeply nested tree that is now garbage. I think that's where it blows up. If anyone has suggestions for debugging, I'm all ears. Jeremy From mwh at python.net Sat Aug 7 21:53:00 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 21:53:01 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_builtin.py, 1.33, 1.34 In-Reply-To: (jhylton@users.sourceforge.net's message of "Sat, 07 Aug 2004 12:20:07 -0700") References: Message-ID: <2mr7qi3eub.fsf@starship.python.net> jhylton@users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Lib/test > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9225/Lib/test > > Modified Files: > test_builtin.py > Log Message: > Subclasses of string can no longer be interned. The semantics of > interning were not clear here -- a subclass could be mutable, for > example -- and had bugs. Explicitly interning a subclass of string > via intern() will raise a TypeError. Internal operations that attempt > to intern a string subclass will have no effect. > > Added a few tests to test_builtin that includes the old buggy code and > verifies that calls like PyObject_SetAttr() don't fail. Perhaps these > tests should have gone in test_string. Would have been nice if you'd run the test suite before checking this in... test_descr contains stuff like: s = madstring("x y") vereq(s, "x y") verify(intern(s).__class__ is str) verify(intern(s) is intern("x y")) vereq(intern(s), "x y") though if madstring was mad enough to redefine __hash__, maybe it wouldn't. Cheers, mwh -- > It might get my attention if you'd spin around in your chair, > spoke in tongues, and puked jets of green goblin goo. I can arrange for this. ;-) -- Barry Warsaw & Fred Drake From tim.peters at gmail.com Sat Aug 7 21:53:22 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 21:53:24 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: References: Message-ID: <1f7befae0408071253544d2c31@mail.gmail.com> [Jeremy Hylton] > I added a test for the compiler package, because I realized it would > easily fall out of date if the language changed (say, decorators were > added) and the compiler package wasn't updated. I added a test with a > resource, so it would only fail if you run regrtest.py -u all. I > think that's a little lame, but better than letting us get into beta > without getting the compiler package fixed. > > At any rate, the test become somewhat more interesting than I expected > because it fails in an unexpected way: > > Fatal Python error: deletion of interned string failed > Aborted (core dumped) > > My best guess is that this is a stack overflow. I assume you ran this on Linux. If so, and it is a stack overflow, then it's very peculiar that it dies in much the same way on Windows: """ $ regrtest.py -uall test_compiler test_compiler Fatal Python error: deletion of interned string failed This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. """ Anyone know who the support team for Python might be? > In particular, gdb shows that the failure is 388 stack levels deep. I've seen Zope gdb stack traces a lot deeper than that, so it's not necessarily indicative of a blown stack. > The compiler package has very deep function calls, because it uses > recursion to traverse abstract syntax trees. (I know, I know, we should just > rewrite the compiler to use iterator instead of recursion .) Inside one of > those deeply nested calls, it triggers GC which attempts to free a > deeply nested tree that is now garbage. I think that's where it blows > up. If anyone has suggestions for debugging, I'm all ears. Same thing on Windows. This is the string it's trying to delete: "abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" If that's the string it's trying to delete for you too, my suggestion is that you find out why . Seriously, it smells more like "a real bug" to me. op->ob_refcnt = 3; if (PyDict_DelItem(interned, op) != 0) Py_FatalError( "deletion of interned string failed"); sets the refcount to 3 first, and the refcount is 4 when the program dies. That means PyDict_DelItem took its if (ep->me_value == NULL) { PyErr_SetObject(PyExc_KeyError, key); return -1; } path. It's all consistent on Windows with a non-stack-overflowing genuine error. From tim.peters at gmail.com Sat Aug 7 21:59:43 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 21:59:47 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <1f7befae0408071253544d2c31@mail.gmail.com> References: <1f7befae0408071253544d2c31@mail.gmail.com> Message-ID: <1f7befae04080712597b9e7e0b@mail.gmail.com> > This is the string it's trying to delete: > > "abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" FYI, that string appears only in shlex.py (split across two source lines). Maybe that will help whittle it down. From tim.peters at gmail.com Sat Aug 7 22:13:19 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 22:13:24 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <1f7befae04080712597b9e7e0b@mail.gmail.com> References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> Message-ID: <1f7befae04080713137fc55b79@mail.gmail.com> BTW, think it's just coincidence that you also checked in a patch changing something-or-other having to do with string interning? I do . From mwh at python.net Sat Aug 7 22:14:53 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 22:14:56 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <1f7befae04080713137fc55b79@mail.gmail.com> (Tim Peters's message of "Sat, 7 Aug 2004 16:13:19 -0400") References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> <1f7befae04080713137fc55b79@mail.gmail.com> Message-ID: <2mn0163dtu.fsf@starship.python.net> Tim Peters writes: > BTW, think it's just coincidence that you also checked in a patch > changing something-or-other having to do with string interning? I do > . We found that looking for the cause of the test_compiler problems. It doesn't seem to be related. Cheers, mwh -- (Unfortunately, while you get Tom Baker saying "then we were attacked by monsters", he doesn't flash and make "neeeeooww-sploot" noises.) -- Gareth Marlow, ucam.chat, from Owen Dunn's review of the year From mwh at python.net Sat Aug 7 22:47:22 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 7 22:47:24 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <1f7befae04080712597b9e7e0b@mail.gmail.com> (Tim Peters's message of "Sat, 7 Aug 2004 15:59:43 -0400") References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> Message-ID: <2misbu3cbp.fsf@starship.python.net> Tim Peters writes: >> This is the string it's trying to delete: >> >> "abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" > > FYI, that string appears only in shlex.py (split across two source > lines). Maybe that will help whittle it down. Here's some code from Lib/compiler/transformer.py: def atom_string(self, nodelist): k = '' for node in nodelist: k += self.decode_literal(node[1]) n = Const(k) n.lineno = nodelist[0][2] return n what does this make you think of? Yes, it's the new += string optimizations; Python gets upset if you mutate interned strings... Armin is on the case... Cheers, mwh -- Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make the additional features appear necessary. -- Revised(5) Report on the Algorithmic Language Scheme From tim.peters at gmail.com Sat Aug 7 22:56:46 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 22:56:49 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <1f7befae04080712597b9e7e0b@mail.gmail.com> References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> Message-ID: <1f7befae04080713564dd7086e@mail.gmail.com> Weird. The very first time PyString_InternInPlace() sees "abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" it's *already* marked as SSTATE_INTERNED_MORTAL . So PyString_InternInPlace() just returns, without adding it to the interned dict. The error we get later is inevitable then. This happens while compile.c's PyCode_New is doing its "Intern selected string constants" loop. PyString_InternInPlace() is the only thing that marks a string as being in the SSTATE_INTERNED_MORTAL state, is it's A Mystery how this string got so marked. Unless it's a wild store. From amk at amk.ca Sat Aug 7 22:57:03 2004 From: amk at amk.ca (A.M. Kuchling) Date: Sat Aug 7 22:57:23 2004 Subject: [Python-Dev] Removing stuff from 2.4 Message-ID: <20040807205703.GA3501@rogue.amk.ca> Hunting for deprecated modules. * Deprecated in 2.1: TERMIOS.py, whrandom.py. * Deprecated in 2.2: mpz module, statcache module. There are also various 2.2-deprecated methods in the email package; Barry can decide if he wants to bother with them or not. * Deprecated in 2.3: xreadlines module, rotor module, rfc822 module, mimetools module, mimify module, MimeWriter module. Only rotor and xreadlines raise a DeprecationWarning in 2.3, so they're the only candidates for removal. The other modules, except for xreadlines, are marked as deprecated in PEP4; should we make them trigger DeprecationWarnings? My proposal: * Remove TERMIOS, mpz, statcache, xreadlines, rotor. * Make rfc822 raise PendingDeprecationWarning (so nothing will happen in 2.4 unless you have -Wa) * Make mimetools, mimify, Mimewrite raise DeprecationWarnings. (This is the part I'm least sure about; comments from people who know about these modules would be appreciated. Should they raise Pending instead?) I don't know what to do about whrandom, if anything; remove it? Leave it for 2.5? suggestions? --amk From guido at python.org Sat Aug 7 23:05:14 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 23:05:21 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: Your message of "Sat, 07 Aug 2004 20:02:34 +0200." References: <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> Message-ID: <200408072105.i77L5Ea10965@guido.python.org> > I just wanted to say that I believe it should be allowed to decorate > classes. There's not reason enough (reading the thread linked to by the > PEP) to limit decorators this way. Like said several times in that > thread, metaclasses are harder (to write and to understand) than > decorators. > > Also, considering that you said you would have actually wanted > docstrings above the definition of a function, wouldn't this apply > to classes as well? > > Anyway, it's an artificial limit and it would be better to be able > to test it out during the alpha. Are you volunteering to implement it? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 7 23:07:28 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 23:07:37 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: Your message of "Sat, 07 Aug 2004 14:25:09 EDT." <1f7befae040807112543238881@mail.gmail.com> References: <41145C05.1020501@gradient.cis.upenn.edu> <200408071127.04568.heikowu@ceosg.de> <4114B226.4020405@interlink.com.au> <1f7befae040807112543238881@mail.gmail.com> Message-ID: <200408072107.i77L7TM10984@guido.python.org> > Indeed, for me that's a reason to prefer | over @. BTW, while the PEP > wasn't clear on this point last I looked, I hope it allows for > whitespace after the | (or @, or =, or whatever it is). The @ is a token by itself so there's nothing to stop you to insert whitespace after it. The PEP should make this clear though. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 7 23:11:08 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 23:11:15 2004 Subject: [Python-Dev] Recommended way to tell platform In-Reply-To: Your message of "Sat, 07 Aug 2004 14:35:25 EDT." <1f7befae04080711351466da96@mail.gmail.com> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> <200408071637.i77Gb0w10415@guido.python.org> <1f7befae04080711351466da96@mail.gmail.com> Message-ID: <200408072111.i77LB8q10997@guido.python.org> > [Guido] > > This does bring up an interesting question: what's the recommended > > way to distinguish a certain platform? There's os.name, > > sys.platform, the platform module, and I've also seen code testing > > for os.sep=='/'. [Tim] > I ask myself that question every time I need to do it. Out of sheer > momentum now, I almost always use > > if sys.platform in ('win32',): Why the 'in singleton tuple'? Still holding out for that compiler optimization that promises to make tuples of constants code constants themselves? > I dislike using os.name == 'nt', mostly because if we asked all > Windows Python users which OS they use, only a tiny percentage would > answer "NT". Since NT is a dying OS, 'nt' is an increasingly > odd-looking thing to test agaist. And so is 'posix' when you're looking at Linux. :-) We can't change these values easily so they will look odder and odder as time progresses. > OTOH, I have no idea when or why sys.platform == "win32" would give > a different result than os.name == "nt". Maybe on cygwin? I really don't know. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 7 23:14:00 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 7 23:14:06 2004 Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there? In-Reply-To: Your message of "Sat, 07 Aug 2004 14:45:32 EDT." References: Message-ID: <200408072114.i77LE0b11018@guido.python.org> > >If the syntax does end up along these lines (and I'm expressing no > >opinion either way), then maybe it would be worth explicitly > >telling people (i.e. in PEP 8) that they ought to put a blank line > >before decorators to make it clear? > > Does PEP 8 tell people that they should put a blank before comment > lines introducing a function? No, but it tells them to *separate* methods by blank lines. So that's pretty clear guidance for where the put blank lines relative to the decorators. --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony at interlink.com.au Sat Aug 7 23:19:32 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat Aug 7 23:19:51 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040807205703.GA3501@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> Message-ID: <41154764.60809@interlink.com.au> A.M. Kuchling wrote: > * Make rfc822 raise PendingDeprecationWarning (so nothing will happen in 2.4 > unless you have -Wa) A few bits and pieces in the stdlib still use bits from rfc822. They all mostly look pretty shallow, except: > * Make mimetools, mimify, Mimewrite raise DeprecationWarnings. (This > is the part I'm least sure about; comments from people who know about > these modules would be appreciated. Should they raise Pending instead?) mimetools.Message subclasses rfc822.Message, and mimetools.Message is used extensively in the std lib. -- Anthony Baxter It's never too late to have a happy childhood. From barry at python.org Sat Aug 7 23:27:05 2004 From: barry at python.org (Barry Warsaw) Date: Sat Aug 7 23:27:10 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040807205703.GA3501@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> Message-ID: <1091914025.1064.48.camel@anthem.wooz.org> On Sat, 2004-08-07 at 16:57, A.M. Kuchling wrote: > Hunting for deprecated modules. > > * Deprecated in 2.1: TERMIOS.py, whrandom.py. > * Deprecated in 2.2: mpz module, statcache module. > > There are also various 2.2-deprecated methods in the email package; > Barry can decide if he wants to bother with them or not. I'll probably remove those. We're bumping the email package's own version number to 3. > My proposal: > * Remove TERMIOS, mpz, statcache, xreadlines, rotor. > * Make rfc822 raise PendingDeprecationWarning (so nothing will happen in 2.4 > unless you have -Wa) +1 > * Make mimetools, mimify, Mimewrite raise DeprecationWarnings. (This > is the part I'm least sure about; comments from people who know about > these modules would be appreciated. Should they raise Pending instead?) DeprecationWarnings on mimify and MIMEWriter. Not sure atm, what to do about mimetools, but it's clearly destined for death. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040807/9b9165b5/attachment.pgp From tim.peters at gmail.com Sat Aug 7 23:31:31 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 23:31:33 2004 Subject: [Python-Dev] Recommended way to tell platform In-Reply-To: <200408072111.i77LB8q10997@guido.python.org> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> <200408071637.i77Gb0w10415@guido.python.org> <1f7befae04080711351466da96@mail.gmail.com> <200408072111.i77LB8q10997@guido.python.org> Message-ID: <1f7befae04080714317191726b@mail.gmail.com> >> [Tim] almost always use >> >> if sys.platform in ('win32',): [Guido] > Why the 'in singleton tuple'? Just because it often turns out that someone else needs to add another platform later. Like if sys.platform in ('win32', 'os2emx', 'mac'): # There's no distinction among 'user', 'group' and 'world'; # replicate the 'user' bits. user = expected >> 6 expected = user * (1 + 8 + 64) > Still holding out for that compiler optimization that promises to make tuples of > constants code constants themselves? I take it you haven't checked compile.c lately . >> I dislike using os.name == 'nt', mostly because if we asked all >> Windows Python users which OS they use, only a tiny percentage would >> answer "NT". Since NT is a dying OS, 'nt' is an increasingly >> odd-looking thing to test agaist. > And so is 'posix' when you're looking at Linux. :-) > > We can't change these values easily so they will look odder and odder > as time progresses. Which is why I prefer using something with 'win32' in its name now. That's equally good for Windows 95, 98, 98SE, ME, NT, 2000, XP. Never mind that it also says 'win32' on Win64, since nobody uses the latter yet. >> OTOH, I have no idea when or why sys.platform == "win32" would give >> a different result than os.name == "nt". > Maybe on cygwin? I really don't know. Nope. sys.platform is 'cygwin' there, and os.name is 'posix'. So it's impossible to mistake Cygwin for a real Windows app . From python at rcn.com Sat Aug 7 11:32:48 2004 From: python at rcn.com (Raymond Hettinger) Date: Sat Aug 7 23:34:37 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040807205703.GA3501@rogue.amk.ca> Message-ID: <002401c47c61$813f55a0$e841fea9@oemcomputer> > I don't know what to do about whrandom, if anything; remove it? Leave > it for 2.5? suggestions? >From PEP4: "#This module has been documented as obsolete since Python 2.1, but listing in this PEP was neglected. The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that." Though the module is useless, I think we have to follow the PEP. Raymond From tim.peters at gmail.com Sat Aug 7 23:42:07 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 7 23:42:09 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <002401c47c61$813f55a0$e841fea9@oemcomputer> References: <002401c47c61$813f55a0$e841fea9@oemcomputer> Message-ID: <1f7befae040807144233fee779@mail.gmail.com> [Andrew] >> I don't know what to do about whrandom, if anything; remove it? Leave >> it for 2.5? suggestions? [Raymond] > From PEP4: "#This module has been documented as obsolete since > Python 2.1, but listing in this PEP was neglected. > The deprecation warning will be added to the module > one year after Python 2.3 is released, and the > module will be removed one year after that." > > Though the module is useless, I think we have to follow the PEP. whrandom should definitely get a DeprecationWarning for 2.4. 2.3 final was released in July of 2003, and it's already a year after. From amk at amk.ca Sat Aug 7 23:44:44 2004 From: amk at amk.ca (A.M. Kuchling) Date: Sat Aug 7 23:45:03 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <1f7befae040807144233fee779@mail.gmail.com> References: <002401c47c61$813f55a0$e841fea9@oemcomputer> <1f7befae040807144233fee779@mail.gmail.com> Message-ID: <20040807214444.GA3798@rogue.amk.ca> On Sat, Aug 07, 2004 at 05:42:07PM -0400, Tim Peters wrote: > whrandom should definitely get a DeprecationWarning for 2.4. 2.3 > final was released in July of 2003, and it's already a year after. Done. --amk From tjreedy at udel.edu Sat Aug 7 23:57:57 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Aug 7 23:58:06 2004 Subject: [Python-Dev] Re: Re: Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain><1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com><4113AF47.1010405@interlink.com.au><200408061634.i76GYEZ07623@guido.python.org> <200408071612.i77GCcK10255@guido.python.org> Message-ID: "Guido van Rossum" wrote in message news:200408071612.i77GCcK10255@guido.python.org... >Do we have to add 'currently' to every statement about the language? Of course not. I would draw the line with one more. > I really want to take a hard stance on this In that case, perhaps we should draw the line at one less and remove the current 'currently'. Then, no one like me could cite it as precedent;-). > I don't think their argument would have a chance in court. Depends on the lawyers. As their lawyer, I would argue something like the followihng: 1. Programming, like the law, requires careful writing. In addition to the C code, the Python Reference Manual is also carefully written. 2. While anything and everything could theoretically change in the next version, there are implicit and explicit promises that that will not happen. Also, some things are *much* more stable than others. 3. The Python manual uses 'forbidden' rarely. Its use is never necessary. Since the Ref Man is carefully written, its use must be intentional, to convey a nuance of meaning. The rational inference is that the nuance is that of considerably more permanence than for other things. Enough, I think. Terry J. Reedy From arigo at tunes.org Sun Aug 8 00:23:57 2004 From: arigo at tunes.org (Armin Rigo) Date: Sun Aug 8 00:29:03 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <2misbu3cbp.fsf@starship.python.net> References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> <2misbu3cbp.fsf@starship.python.net> Message-ID: <20040807222357.GA27723@vicky.ecs.soton.ac.uk> Hi, On Sat, Aug 07, 2004 at 09:47:22PM +0100, Michael Hudson wrote: > k = '' > for node in nodelist: > k += self.decode_literal(node[1]) > > what does this make you think of? Yes, it's the new += string > optimizations; Python gets upset if you mutate interned strings... > > Armin is on the case... Ahem, oups, yes, I am to blame for this memory corruption bug. In the += string optimization I forgot that the string's ob_refcnt field is not always "correct". String interning messes with its value. What occurred here is that we mutated a string which had only one reference left -- that is, one apart from the two that stringobject.c artificially removes to ob_refcnt when interning. This is now fixed by skipping the optimization if we got an interned string. While messing with ob_refcnt is always a bit fragile I am still convinced that the += optimization is safe again. A bient?t, Armin. From mkc at mathdogs.com Sun Aug 8 00:36:17 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sun Aug 8 00:36:21 2004 Subject: [Python-Dev] Re: re.split on empty patterns References: <20040807145142.GB2529@rogue.amk.ca> Message-ID: "A.M. Kuchling" writes: > >>> re.split('x*', 'abxxxcdefxxx') > ['ab', 'cdef', ''] > >>> re.split('x*', 'abxxxcdefxxx', emptyok=True) > ['', 'a', 'b', '', 'c', 'd', 'e', 'f', '', ''] > > (I think the result of the second match points up a bug in the patch; > the empty strings in the middle seem wrong to me. Assume that gets > fixed.) I believe it's correct in the sense that it matches the design I had in mind. Of course, this could be the wrong design choice. I think (some of) the alternatives can be stated this way: 1. Empty matches are not considered at all for splitting. (present behavior) 2. Empty matches are only considered when they are not adjacent to non-empty matches. (what you have in mind, I think) 3. Empty matches are always considered. (the patch behavior) (It's understood that matches that fall within other matches are not considered. For empty matches, an adjacent match doesn't count as "within".) I would say that alternatives 2 and 3 are better than 1 because they retain more information, and I would argue that alternative 3 is better than alternative 2 for the same reason. I don't have a killer example, but here is a somewhat abstract one that shows the difference between 2 and 3: # alternative 2: re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', 'bbczz'] re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz'] # alternative 3: re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', '', 'bbczz'] re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz'] In English, the third alternative allows you to notice that there were two adjacent matches, even if the second match was empty. With the second alternative, this is missed. Of course, because of the match algorithm, we cannot notice an empty match immediately followed by a non-empty match, so there's kind of an asymmetry here. With alternative 2 that asymmetry wouldn't be present, since we'd fail to notice empty matches on either side. Alternative 2 does have the advantage of matching the expectations of a naive user a little better. Alternative 3 is more powerful, but perhaps a little less obvious. (If you're reading this, what do you think?) > Anyway, we therefore can't just make this the default in 2.4. We > could trigger a warning when emptyok is not supplied and a split > pattern results in a zero-length match; users could supply > emptyok=False to avoid the warning. Patterns that never have a > zero-length match would never get the warning. 2.5 could then set > emptyok to True. I like this compromise. A variant would be to warn if the pattern *could* match empty, rather than warning when it *does* match empty. I'm not sure whether it would be easy to determine this, though. > Note: raising the warning might cause a serious performance hit for > patterns that get zero-length matches a lot, which would make 2.4 > slower in certain cases. The above variant would ameliorate this, though in theory no one should be using patterns that can empty match anyway. Cheers, Mike From mkc at mathdogs.com Sun Aug 8 00:44:06 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Sun Aug 8 00:44:10 2004 Subject: [Python-Dev] Re: re.split on empty patterns References: <20040807145142.GB2529@rogue.amk.ca> <1f7befae040807105878b47c61@mail.gmail.com> Message-ID: Tim Peters writes: > Yes it is! Or, more accurately, it can be, when it's intended to > match an empty string. It's a bit fuzzy because regexps are so > error-prone, and writing a regexp that matches an empty string by > accident is easy. I won't dispute that regexps are error-prone (having just proved it recently), but I would point out that any calls to the current re.split with a pattern that can match empty are probably kind of fuzzy anyway. Consider, for example, that re.search doesn't have an option that says "find matches but skip any empty matches". Along the same lines, I think re.findall doesn't really do what its doc claims. The doc says Empty matches are included in the result unless they touch the beginning of another match. but, for example >>> re.findall(r'(?=.)|b', 'abcabc') ['', '', '', '', '', ''] [24592 refs] >>> re.findall(r'b|(?=.)', 'abcabc') ['', 'b', '', '', 'b', ''] [24601 refs] with a recent python. It's not clear what "another match" would be here. The only way its beginning could be touched by an empty match would be if they both started in the same position. So presumably that other match could only be discovered by asking the RE engine to "find me the first match in this position that's not empty". It looks like the *actual* method being used, though, is more like the method used in my patch: for each position that's not interior to some previous match: find the first match at this position (if any) and add it to the list We could find the "longest" match instead of the "first" match, but that would be a lot more expensive in general, and not necessarily easier to understand. > In the meantime, "emptyok" is an odd > name since it's always "ok" to have an empty match. "split0=True" > reads better to me, since the effect is to split on a 0-length match. > split_on_empty_match would be too wordy. I was thinking "ok" in the sense of "okay to split on them" (empty matches). I don't have any strong preference on the name, though--I'd just like to get the feature in relatively soon (with semantics that everyone's happy with). ('split0' does look slightly '3133t', though. :-) Cheers, Mike From python at rcn.com Sat Aug 7 13:01:26 2004 From: python at rcn.com (Raymond Hettinger) Date: Sun Aug 8 01:03:15 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <20040807222357.GA27723@vicky.ecs.soton.ac.uk> Message-ID: <003b01c47c6d$e2fc94e0$e841fea9@oemcomputer> > the string's ob_refcnt field is not > always > "correct". String interning messes with its value. What occurred here is > that we mutated a string which had only one reference left -- that is, one > apart from the two that stringobject.c artificially removes to ob_refcnt > when > interning. Try adding a guard using s->ob_sstate. Raymond From ark-mlist at att.net Sun Aug 8 02:03:45 2004 From: ark-mlist at att.net (Andrew Koenig) Date: Sun Aug 8 02:03:51 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: Message-ID: <004501c47cdb$355743d0$6602a8c0@arkdesktop> > > def foo(): > > | classmethod > > | accepts(int,int) > > | returns(float) > > > > > Thats it! I love it. I like the following variation even better: def foo() | classmethod | accepts(int, int) | returns(float): Alternatively, def foo() | classmethod | accepts(int, int) | returns(float): Yes, I understand that the first of these would require allowing a newline after the | without ending the statement. If it were up to me, I would allow a newline to follow any operator, but if that's too radical, then allowing newlines between the def and the following : would be fine. From tim.peters at gmail.com Sun Aug 8 02:35:44 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 8 02:35:47 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <20040807222357.GA27723@vicky.ecs.soton.ac.uk> References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> <2misbu3cbp.fsf@starship.python.net> <20040807222357.GA27723@vicky.ecs.soton.ac.uk> Message-ID: <1f7befae04080717357814d502@mail.gmail.com> [Armin Rigo] > ... > This is now fixed by skipping the optimization if we got an interned string. > While messing with ob_refcnt is always a bit fragile I am still convinced that > the += optimization is safe again. Ya, this bug wasn't scary (an unlikely but clear-in-hindsight problem, specifically due to a subsystem lying about true refcounts -- not much of that goes on, thank God). Alas, test_compiler still fails (but in an ordinary "doesn't work" way, no longer in an internal-error way), so testing with -uall is still a bit broken. From fperez528 at yahoo.com Sun Aug 8 03:26:18 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Sun Aug 8 03:26:34 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> <200408071617.i77GHgs10281@guido.python.org> Message-ID: Guido van Rossum wrote: >> I'm including both of the following variants: [Let's call this option #1] >> |classmethod >> |accepts(int, int) >> |returns(float) >> def foo(arg1, arg2): >> ... [And this option #2] >> def foo(arg1, arg2): >> |classmethod >> |accepts(int, int) >> |returns(float) >> ... > In the discussion on decorators months ago, solutions involving > special syntax inside the block were ruled out early on. Basically, > you shouldn't have to peek inside the block to find out important > external properties of the function. (Yes, I know that docstrings > violate this principle. For the record, it would be better if they > were prefix too; and a prefix decorator syntax will let us fix this in > the future but introducing a "doc" decorator.) Allow me to try and make a reasoned argument against your point, and in favor of #2 (as labeled above), even for docstrings. The reason why I find #1 more difficult to mentally parse, is that as you read the code, you need to accumulate information in your head which is not yet tied to anything in particular (the def line hasn't arrived yet). In my brain, the process goes something like this (imagine this happening in code with multiple decorators, each with multiple lines of comments detailing its use): |classmethod ok, a class method is coming, but I don't know yet what it's called. Put this on the 'mental stack', let's see where it goes... |accepts(int, int) and it will take integer arguments; more for the stack... |returns(float) and return a float; add to the stack... def foo(arg1, arg2): Ah! It's function foo, with 2 args named arg1, arg2. Now I go back (mentally) and wrap together my knowledge about it, unwinding my stack, producing foo, a classmethod with arg1,arg2 as integers, and which returns a float. I find this unpleasant and inefficient from a cognitive viewpoint, because our brain is best at processing information when it has 'buckets' where to put it: if you read the def line first, the function name becomes the 'bucket' where all further information goes. That's why it's always harder to remember random bits of information than things which connect to other things you already know. For these reasons, I feel that option 2 is much more efficient. A similar outline of my mental process with option 2: >> def foo(arg1, arg2): Ok, a new function foo() goes in, it takes 2 args, arg1 and arg2. Make mental bucket labeled 'foo'. Anything that comes now, until we de-indent, goes into box 'foo'. No need to keep random bits of information up in the air until I know where they're supposed to go. >> |classmethod Store 'classmethod' in bucket 'foo'. >> |accepts(int, int) Store 'accepts(int, int)' in bucket 'foo'. >> |returns(float) Store 'returns(float)' in bucket 'foo'. Done. All information is stored in mental bucket 'foo', where it belongs. Bucket 'foo' was created when I saw the def line, nicely separated by indentation from all else, which is what my python mental processor is already well trained to do. Then, all the information that came afterwards (including possibly the docstring) fell nicely into that mental bucket. Option 1, which you seem to favor, reminds me of a problem I felt when studying German for the first time (Spanish is my native language). German has these funny 'split-verbs', like anrufen, which are used in a sentence split in two, and with the initial part of the verb only appearing at the very end of the sentence. A simple example (I don't recall the really painful ones anymore) is "Ich rufe dich an". You don't know that the verb is anrufen, until you see the 'an' at the end of the sentence. With long sentences, I remember feeling very frustrated, because I esentially had to buffer the whole thing in my head with very little structure, waiting for the last bit to come in. Then I needed to quickly reconstruct the meaning, now that I knew which verb was being used, by mentally reparsing the whole thing. It's been many years, but I distinctly remember this as being a significant hurdle in spoken conversation in German. I honestly feel adding this kind of process to python would be a significant reduction to the fluidity with which the language 'parses mentally'. And I like the fact that this syntax: def foo(...): @dec1 @dec2 "docstring" # code begins... sort of declares a 'header zone' for the function, which is visually well defined by indentation and string highlighting, but allows the 'def' part to be very easy to spot visually because it cuts the indentation cleanly. It feels like a quite natural convention for me to think that until I see the closing of the docstring, I'm in the 'header' or 'interface' zone of the function. I hope I've at least presented a clear argument in defense of this choice. Whether you agree or not, is up to you. I deliberatly didn't mention the @ vs | issue, since for this discussion it's completely irrelevant. Best, f From jason at tishler.net Sun Aug 8 04:32:19 2004 From: jason at tishler.net (Jason Tishler) Date: Sun Aug 8 04:25:41 2004 Subject: [Python-Dev] Cygwin and sockets In-Reply-To: <4110AABA.80407@v.loewis.de> References: <4110AABA.80407@v.loewis.de> Message-ID: <20040808023219.GE55644@tishler.net> Martin, On Wed, Aug 04, 2004 at 11:22:02AM +0200, "Martin v. L?wis" wrote: > Tony Meyer wrote: > >By "anybody", did you mean anybody that you know & trust? <0.5 wink>. > > > >I can duplicate the process outlined in the bug report and get the > >same failure to compile (I've added a note to it). > > This is good enough for the moment, although I would have preferred an > answer like "I know, and I have a fix sitting on my harddisk" :-) I've *just* returned from vacation. I will look into this problem after catching up and will report back when I can add something useful. Jason -- PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6 From raymond.hettinger at verizon.net Sat Aug 7 16:45:47 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun Aug 8 04:47:40 2004 Subject: [Python-Dev] decimal.Context.copy() shallow or deep? Message-ID: <004f01c47c8d$3a7cb1e0$e841fea9@oemcomputer> Currently, the copy() method for Context objects makes a shallow copy of the context. To get a deepcopy, you have to use copy.deepcopy(thecontext). Though, copies are usually shallow and deepcopies are deep, I'm now having misgivings about the copy method. I would think that a deep copy is almost invariably what you would want and expect, (i.e. the copy doesn't share traps and flags with the original). Do you guys think it should be left alone (shallow) or changed (made deep)? The wrinkle in all this is that internally the module makes many uses of copy and expects it to be shallow (or least, tests fail in droves when copy is made deep). So I would have to rename the current copy to _shallow_copy and replace the original with a deep version for the public API. Any thoughts? Raymond From tim.peters at gmail.com Sun Aug 8 04:53:40 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 8 04:53:43 2004 Subject: [Python-Dev] decimal.Context.copy() shallow or deep? In-Reply-To: <004f01c47c8d$3a7cb1e0$e841fea9@oemcomputer> References: <004f01c47c8d$3a7cb1e0$e841fea9@oemcomputer> Message-ID: <1f7befae040807195334227984@mail.gmail.com> [Raymond Hettinger] > Currently, the copy() method for Context objects makes a shallow copy of > the context. To get a deepcopy, you have to use > copy.deepcopy(thecontext). > > Though, copies are usually shallow and deepcopies are deep, I'm now > having misgivings about the copy method. I would think that a deep copy > is almost invariably what you would want and expect, (i.e. the copy > doesn't share traps and flags with the original). Hmm. In my head (which I'm sure doesn't match reality, else you wouldn't be saying this), sets of trap-enabled and it-happened sticky flags are just integers with meanings assigned to the bits, as they are in hardware and most APIs for controlling and querying FPU status. Then there's no difference between shallow and deep copying. > Do you guys think it should be left alone (shallow) or changed (made > deep)? A user-visible copy certainly should not share mutable state with the original. It's possible that really argues for a simpler internal representation of such state, though. > The wrinkle in all this is that internally the module makes many uses of > copy and expects it to be shallow (or least, tests fail in droves when > copy is made deep). So I would have to rename the current copy to > _shallow_copy and replace the original with a deep version for the > public API. > > Any thoughts? Whatever works . From python at rcn.com Sat Aug 7 17:24:03 2004 From: python at rcn.com (Raymond Hettinger) Date: Sun Aug 8 05:26:41 2004 Subject: [Python-Dev] decimal.Context.copy() shallow or deep? In-Reply-To: <1f7befae040807195334227984@mail.gmail.com> Message-ID: <000401c47c92$b38c43c0$1619c797@oemcomputer> > > Do you guys think it should be left alone (shallow) or changed (made > > deep)? > > A user-visible copy certainly should not share mutable state with the > original. I was afraid of that. What a PITA. > It's possible that really argues for a simpler internal > representation of such state, though. No doubt that would be the right thing to do (doubly so because that is how you would do it in C). Unfortunately, there a ton of code that has to change to get to an integer and bitmask approach. Also, you have to expose constants for masking. Then, you still have to deal with the underlying code liking the shallow copies. > > The wrinkle in all this is that internally the module makes many uses of > > copy and expects it to be shallow (or least, tests fail in droves when > > copy is made deep). So I would have to rename the current copy to > > _shallow_copy and replace the original with a deep version for the > > public API. > > > > Any thoughts? > > Whatever works . Done fast, right, and cheap -- pick any two. Raymond From martin at v.loewis.de Sun Aug 8 09:02:21 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 8 09:02:21 2004 Subject: [Python-Dev] Re: Recommended way to tell platform In-Reply-To: <200408071637.i77Gb0w10415@guido.python.org> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> <200408071637.i77Gb0w10415@guido.python.org> Message-ID: <4115CFFD.1000309@v.loewis.de> Guido van Rossum wrote: > I still prefer hasattr(, ) whenever applicable, > e.g. preferring hasattr(os, 'fork') over os.name=='posix' (or > os.name!='nt' :-), but sometimes that's not possible. > > What should be the preferred way? (It may be impossible to say > because there are different use cases, but probably one of the most > important cases is simply distinguishing Windows from the rest -- how > should that be done?) Precisely that: impossible to say. In most cases, you should not test for the platform, but just use the platform functionality, and fall back to something else if it isn't present. However, to reliably distinguish Windows from the rest, check whether os.platform is "win32". Regards, Martin From martin at v.loewis.de Sun Aug 8 09:22:32 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 8 09:22:32 2004 Subject: [Python-Dev] Recommended way to tell platform In-Reply-To: <1f7befae04080711351466da96@mail.gmail.com> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> <200408071637.i77Gb0w10415@guido.python.org> <1f7befae04080711351466da96@mail.gmail.com> Message-ID: <4115D4B8.90405@v.loewis.de> Tim Peters wrote: > I dislike using os.name == 'nt', mostly because if we asked all > Windows Python users which OS they use, only a tiny percentage would > answer "NT". A similar point holds for Linux, btw. os.name is "posix"; I doubt many Linux (or, Solaris) users would say that the name of their OS is "posix". It has the advantage over "nt" that "posix" is not dying. Regards, Martin From eppstein at ics.uci.edu Sun Aug 8 09:22:43 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Sun Aug 8 09:22:47 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <004501c47cdb$355743d0$6602a8c0@arkdesktop> Message-ID: In article <004501c47cdb$355743d0$6602a8c0@arkdesktop>, "Andrew Koenig" wrote: > I like the following variation even better: > > def foo() | > classmethod | > accepts(int, int) | > returns(float): > > Alternatively, > > def foo() | classmethod | accepts(int, int) | returns(float): > > Yes, I understand that the first of these would require allowing a newline > after the | without ending the statement. If it were up to me, I would > allow a newline to follow any operator, but if that's too radical, then > allowing newlines between the def and the following : would be fine. Did you omit the full wink? Or did you forget that | is already a binop? Strictly speaking I guess it's unambiguous if one doesn't allow arbitrary expressions as decorators, but... -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From martin at v.loewis.de Sun Aug 8 09:28:42 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 8 09:28:41 2004 Subject: [Python-Dev] hi I am new in the list In-Reply-To: <20040807183610.77951.qmail@web41410.mail.yahoo.com> References: <20040807183610.77951.qmail@web41410.mail.yahoo.com> Message-ID: <4115D62A.4040204@v.loewis.de> > I think could help in these topics: Translations (my > native language is spanish and know a little of > english), Python programming, C/C++ programming. Hi Rafael, Thanks for the offer. There is one area where we really could use some help: patches and bugs. Please go to sf.net/projects/python, and see whether you can review some patches, and whether you can offer fixes for some of the bugs. See http://www.python.org/dev/dev_intro.html for some procedural hints. Regards, Martin From anthony at interlink.com.au Sun Aug 8 09:59:22 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Sun Aug 8 09:59:48 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <1091914025.1064.48.camel@anthem.wooz.org> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> Message-ID: <4115DD5A.2070108@interlink.com.au> Barry Warsaw wrote: > I'll probably remove those. We're bumping the email package's own > version number to 3. The AddlistClass in email is marked as deprecated, but is deprecated in favour of a class in rfc822! >>* Make mimetools, mimify, Mimewrite raise DeprecationWarnings. (This >> is the part I'm least sure about; comments from people who know about >> these modules would be appreciated. Should they raise Pending instead?) > > > DeprecationWarnings on mimify and MIMEWriter. Not sure atm, what to do > about mimetools, but it's clearly destined for death. mimetools is used in a number of other modules - including urllib httplib cgi and urllib2. Worse, they use mimetools.Message, which is a subclass of rfc822.Message. urllib/urllib2's addinfourl.info() is documented as returning a mimetools.Message(). I mentioned this on IRC yesterday - I do not think we should put in a PendingDeprecationWarning (or a DeprecationWarning) for anything until we've stripped it's usage from the stdlib. If we can't be bothered to update our code to no longer use the code we want to dump, it hardly seems fair to expect our users to do so. I'd go so far as to say that this should be explicitly stated in the PEP. I also think that a "final" release should not issue PDWs from the test suite (with the obvious exception of code that is explicitly testing the PDWed feature. -- Anthony Baxter It's never too late to have a happy childhood. From scav at blueyonder.co.uk Sun Aug 8 12:04:25 2004 From: scav at blueyonder.co.uk (Peter Harris) Date: Sun Aug 8 11:50:29 2004 Subject: [Python-Dev] Re: Python-Dev Digest, Vol 13, Issue 77 In-Reply-To: <20040807132358.975AA1E4015@bag.python.org> References: <20040807132358.975AA1E4015@bag.python.org> Message-ID: <4115FAA9.7010707@blueyonder.co.uk> Andrew Durdin wrote: Message: 6 Date: Sat, 7 Aug 2004 22:36:59 +1000 From: Andrew Durdin Subject: [Python-Dev] Decorators with arguments are curries! To: python-dev@python.org Message-ID: <59e9fd3a04080705361b6314be@mail.gmail.com> Content-Type: text/plain; charset=US-ASCII Having waded through all the decorator syntax discussions recently, and some of the historical ones, I haven't found this point mentioned before, so I'll bring it up now: Consider the following code: def foo(bar): return 1 a = foo a = foo(bar) The first assignment to a is binding a reference to a function; the second is calling the function. This is a very significant difference in python, and I'm concerned that all the current proposed decorator syntaxes[*] are liable to cause confusion on this point. For example: def foo_decorator(func): print "no params to this" return func def bar_decorator(func, param): print param return func @foo_decorator @bar_decorator("one param here") def decorated_func(): pass Here the first decorator statement is bare, while the second one includes parentheses and an argument; the first one looks like a function reference, while the second looks like a function call. I find this confusing; semantically, this appears to equivalent to (assuming expressions are allowed in decorator statements): # using curry as proposed in PEP 309 [**] @foo_decorator @curry(bar_decorator, "one param here") def decorated_func(): pass Most of my concern here is that this aspect of decorator syntax appears to be implicitly introducing a currying syntax in one special circumstance, which is then *not* transferable to currying functions in normal situations, as it would conflict with function calling. And if a currying syntax (or built-in) was introduced down the track, then these decorators would be inconsistent with them. Has anyone else noticed this issue? Andrew Durdin [*] Except those which don't allow for arguments [**] For a nice coincidence, PEP 309 suggested using '@' to mark curries Hmm. I sort of see what you're on about, but please note: PEP309 no longer proposes (and didn't for long) using '@'. The callable class in PEP309 is called partial() not curry() because it's not technically a curried function in the narrow computer-science sense, and if we mis-named it the Haskell folks would point and laugh. You could use partial() on decorators as well as functions and classes, and though it would melt my brain, I'm sure someone out there would see a beautifully elegant use for it. PEP309 is accepted but not checked in; @ decorators are checked in but tentative. On topic: @ is OK but I prefer the recently suggested |. pie, pipe = (+0.5, +1) Peter Harris From rnd at onego.ru Sun Aug 8 12:09:25 2004 From: rnd at onego.ru (Roman Suzi) Date: Sun Aug 8 12:08:38 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <4114583C.4070500@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> Message-ID: On Sat, 7 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote: > >> But must it be "@", such an eye-catching, ugly piece >> of flie-dirt? > >If you don't like the current proposal, try to find objective >reasons, in addition to the subjective ones. Also, try to come >up with counter-proposals that are well thought-out, instead of >being created in a rush. In particular, if all you are concerned >with is the specific choice of operator, propose a different one. BTW, hat is wrong with my %-proposal: def foo(self) % decor(): body and def foo(self) % (decor(), decor1(args), ...): body - it seems to satisfy the needs. This form could also be added (for consistency): def foo(self): # old way body foo %= decor() or foo %= (decor(), decor1(args), ...) or foo = foo % (decor()) + and this doesnt require to add new symbol (@), + it is semantically consistant with % meaning as a formatting ("decorating") operation, although in case of def the part being decorated is on LHS, while in string formatting arguments are being "decorated" according to the format string. I want to hear the critics of this as I see this a better syntax to the decoration needs! >Regards, >Martin Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From rnd at onego.ru Sun Aug 8 12:16:28 2004 From: rnd at onego.ru (Roman Suzi) Date: Sun Aug 8 12:15:53 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: <004501c47cdb$355743d0$6602a8c0@arkdesktop> References: <004501c47cdb$355743d0$6602a8c0@arkdesktop> Message-ID: On Sat, 7 Aug 2004, Andrew Koenig wrote: >> > def foo(): >> > | classmethod >> > | accepts(int,int) >> > | returns(float) >> > >> >> >> Thats it! I love it. > >I like the following variation even better: > > def foo() | > classmethod | > accepts(int, int) | > returns(float): > >Alternatively, > > def foo() | classmethod | accepts(int, int) | returns(float): > >Yes, I understand that the first of these would require allowing a newline >after the | without ending the statement. If it were up to me, I would >allow a newline to follow any operator, but if that's too radical, then >allowing newlines between the def and the following : would be fine. I just anted to rewrite those "real-life-looking" examples with my proposed '%' syntax: def foo() % (classmethod, accepts(int,int), returns(float),): or formatted: def foo() % ( classmethod, accepts(int,int), returns(float),): Or it can be done the this way: def foo(): ... foo %= (classmethod, accepts(int,int), returns(float),) The operation could be called "decorative apply", if the LHS has __call__ attribute. Just a new form of % operation with special syntax for "def" operator. Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From martin at v.loewis.de Sun Aug 8 12:23:49 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 8 12:23:49 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> Message-ID: <4115FF35.4030608@v.loewis.de> Roman Suzi wrote: >>If you don't like the current proposal, try to find objective >>reasons, in addition to the subjective ones. Also, try to come >>up with counter-proposals that are well thought-out, instead of >>being created in a rush. In particular, if all you are concerned >>with is the specific choice of operator, propose a different one. > > > BTW, hat is wrong with my %-proposal: Wrt our discussion: it does not just replace the choice of character; it also replaces the position where the decorators are placed. Taken on its own, it is similar to C2 and C3 in http://www.python.org/moin/PythonDecorators and as such, it shares the same problems. Feel free to add it as C4 if you care. > I want to hear the critics of this as I see this a better syntax > to the decoration needs! Read the Wiki. Regards, Martin From marco at bubke.de Sun Aug 8 08:52:10 2004 From: marco at bubke.de (Marco Bubke) Date: Sun Aug 8 12:40:38 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: Edward Loper wrote: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). > > I'm including both of the following variants: > > |classmethod > |accepts(int, int) > |returns(float) > def foo(arg1, arg2): > ... > > def foo(arg1, arg2): > |classmethod > |accepts(int, int) > |returns(float) > ... I really like the second one maybe with '@'. For me a decorator is a property of the function like the attributes. I think this is function foo with two arguments, its a classmethod etc.. The function name and the arguments are much more important for me than the decorators. So they should be go after the doc string. Marco From rnd at onego.ru Sun Aug 8 13:07:11 2004 From: rnd at onego.ru (Roman Suzi) Date: Sun Aug 8 13:06:20 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <4115FF35.4030608@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> Message-ID: On Sun, 8 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote: >Roman Suzi wrote: >> >> BTW, hat is wrong with my %-proposal: > >Wrt our discussion: it does not just replace the choice of >character; it also replaces the position where the decorators >are placed. > >Taken on its own, it is similar to C2 and C3 in > >http://www.python.org/moin/PythonDecorators > >and as such, it shares the same problems. Feel free to add >it as C4 if you care. Yes, this is like C3 when '%' is used instead of "using" (pseudo)keyword. As for the problems, I do not agree that decorators visually mix with arguments: it is not a problem with +-s and *-s in an expression or other similar cases. Also: heavily decorated function/method probably tells that something is wrong with it's design. BTW, IMHO, "A" case has more drawbacks than "C" cases. I also like %-syntax being used in |-syntax style (E2). >> I want to hear the critics of this as I see this a better syntax >> to the decoration needs! > >Read the Wiki. Done. >Regards, >Martin > Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From gregory.lielens at fft.be Sun Aug 8 13:08:30 2004 From: gregory.lielens at fft.be (Gregory Lielens) Date: Sun Aug 8 13:08:55 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: Message-ID: <000001c47d38$0ce96740$1000000a@chikubi> > I just anted to rewrite those "real-life-looking" examples > with my proposed '%' syntax: > > def foo() % (classmethod, accepts(int,int), returns(float),): > > or formatted: > > def foo() % ( > classmethod, > accepts(int,int), > returns(float),): > > Or it can be done the this way: > > def foo(): > ... > > foo %= (classmethod, accepts(int,int), returns(float),) > > The operation could be called "decorative apply", if the LHS > has __call__ attribute. Just a new form of % operation with > special syntax for "def" operator. > > > Sincerely yours, Roman Suzi Yes, you should add this proposal in the wiki (http://www.python.org/cgi-bin/moinmoin/PythonDecorators) just after "C3. tuple-after-def syntax with a (pseudo-)keyword". Your idea is basically the same with a % instead of a pseudo-keyword... For me, +1 for these two (and your justification for using % is convincing for me, especially as it is difficult to agree on a best choice for the pseudo-keyword to use in C2, C3 so you are my current favorite ;-)) Having followed the thread, my own summary (that didn't change much these last days) is that all before-def variants are prefered because it is more aesthetically pleasing (I do not agree ;-) ) and it is more related to the "exterior" behavior of the function/method (i.e. seen from the caller perspective) than to it's internal behavior (i.e. seen from the implementer of the function, or when examining function algorithm). This last argument is nice and made me re-evaluate my gut feeling, but in the end it was not sufficient for changing my preference for the various C proposal (with C3 and yours beeing my favorites). Indeed, adding decorators (or any modifier/metadata) in front of a declaration is a valid way to proceed of course, but feel not in line with the rest of python syntax, and that is what bother me the most, it feel too much like an ad-hoc special casing...And anyway, function/method name and arguments are at least as important as decorator from a caller perspective, which for me mitigate a lot the argument that decorators should be in front: In a C++ like syntax, yes, sure, but with Python syntax it does not fit well in the general picture imho... Maybe I'll change my mind after a while, if before-def syntax is adopted, but for this it would help if this type of construct could be used not only for decorators (and with | instead of @ - pleaaase ;-) ), so that "special case" does not flash into my head each time I see it ;-) Regards, Greg. From gregory.lielens at fft.be Sun Aug 8 13:24:22 2004 From: gregory.lielens at fft.be (Gregory Lielens) Date: Sun Aug 8 13:24:30 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: Message-ID: <000101c47d3a$44400d00$1000000a@chikubi> Oups, saw you just added it, a C4 proposal :-) One remark though, I would far prefer if the implicit linebreak is not done, it is a special casing again, for little readability benefit imho... I appreciate consistency for syntax, as special cases often causes small syntax errors for me, that have to be corrected and remove part of the fun in programming: I really like to have a Python that reads like pseudo-code and have a nice coherent syntax easy to remember, sure it is maybe trivial compared to other things that makes a good programming langage but this is what stand out for me first time I saw a python script... Regards, Greg. From martin at v.loewis.de Sun Aug 8 13:39:39 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 8 13:39:38 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> Message-ID: <411610FB.9040201@v.loewis.de> Roman Suzi wrote: > As for the problems, I do not agree that decorators visually mix with > arguments: it is not a problem with +-s and *-s in an expression or other > similar cases. Also: heavily decorated function/method probably tells > that something is wrong with it's design. There is another drawback of course: it does not have an implementation. Regards, Martin From anthony at interlink.com.au Sun Aug 8 13:52:49 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Sun Aug 8 13:53:17 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> Message-ID: <41161411.2060804@interlink.com.au> Roman Suzi wrote: > On Sat, 7 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote: > >>>But must it be "@", such an eye-catching, ugly piece >>>of flie-dirt? >> >>If you don't like the current proposal, try to find objective >>reasons, in addition to the subjective ones. Also, try to come >>up with counter-proposals that are well thought-out, instead of >>being created in a rush. In particular, if all you are concerned >>with is the specific choice of operator, propose a different one. > > > BTW, hat is wrong with my %-proposal: > > def foo(self) % decor(): > body Guido has already ruled out the decorators-after-function-arguments form. See the last week's archives for python-dev - thread subject was "def fn (args) [dec,dec]:" -- Anthony Baxter It's never too late to have a happy childhood. From jim at zope.com Sun Aug 8 14:18:40 2004 From: jim at zope.com (Jim Fulton) Date: Sun Aug 8 14:18:44 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <41161A20.1080505@zope.com> Edward Loper wrote: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. OK, I'll try to fix that. ;) -1 But, to me, all of the variations on "special character + decorator" feel like a foreign language. It reminds me of languages like PHP that let you mix HTML and some scripting language. It looks like someone is trying to use 2 separate languages. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From rnd at onego.ru Sun Aug 8 15:03:14 2004 From: rnd at onego.ru (Roman Suzi) Date: Sun Aug 8 15:02:25 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <41161411.2060804@interlink.com.au> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <41161411.2060804@interlink.com.au> Message-ID: On Sun, 8 Aug 2004, Anthony Baxter wrote: >> BTW, hat is wrong with my %-proposal: >> >> def foo(self) % decor(): >> body > >Guido has already ruled out the decorators-after-function-arguments >form. See the last week's archives for python-dev - thread subject >was "def fn (args) [dec,dec]:" Hmmm... Maybe it's time to introduce better lambdas? ;) foo = decor( def foo(self): ... ... ) Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From rnd at onego.ru Sun Aug 8 15:11:37 2004 From: rnd at onego.ru (Roman Suzi) Date: Sun Aug 8 15:10:44 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <411610FB.9040201@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> <411610FB.9040201@v.loewis.de> Message-ID: On Sun, 8 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote: >Roman Suzi wrote: > >> As for the problems, I do not agree that decorators visually mix with >> arguments: it is not a problem with +-s and *-s in an expression or other >> similar cases. Also: heavily decorated function/method probably tells >> that something is wrong with it's design. > >There is another drawback of course: it does not have an implementation. Well, GvR convinced me decorators-before-def are needed, thus leaving my proposal void. However, I am not happy with those @s. >Regards, >Martin Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From jim at zope.com Sun Aug 8 15:53:46 2004 From: jim at zope.com (Jim Fulton) Date: Sun Aug 8 15:53:51 2004 Subject: [Python-Dev] Thoughts on validation, events, rules Message-ID: <4116306A.1000904@zope.com> I'm a bit unsatisfied with container constraints: http://dev.zope.org/Zope3/ContainmentConstraints While I think this was a step forward conceptually, it has turned out to be a bit unwieldy in practice: - Container constraints are difficult to express. I've written convenience API's to make the basic machinery of expressing constraints easier, but there is a problem that's not so easy to fix. Often you want to express a symmetric constraint between two interfaces. For example, you want to say that objects implementing I1 can only contain objects implementing I2 and that objects implementing I2 can only be contained in objects implementing I1. If the constraint is expressed in the interfaces themselves, you can only express it by either defining the interfaces in a single operation or by mutating the interfaces. I don't like the idea of mutating interfaces, but perhaps I just need to get over that. - Container constraints are cumbersome to check. - Container constraints force the contained objects to implement IContained. This is fine when the contained objects will actually access their containers. Often, however, we want to constrain an object's container for external reasons. For example, we might want to only allow indexes to be added to catalogs, even though indexes are passive and never invoke operations on their containers. In this case, the constraint is not part of the index's contract. I've been thinking of other ways to solve this problem. In the context of schema validation, we've been thinking of using subscription adapters: http://dev.zope.org/Zope3/NoMoreSchemaBinding http://mail.zope.org/pipermail/zope3-dev/2004-June/011315.html The advantage of subscription adapters is that they provide the ability to express constraints in a highly flexible and decentralized manner. Another advantage of using subscription adapters is that doing so takes advantage of a highly optimized mechanism and should be fairly efficient. It occurred to me today that subscription adapters provide a form of rule-based processing. Any number of rules (subscribers) can fire as the result of some event, where an event could take the form of a query, such as "validate adding this object to this container" or "validate setting this attribute to this value". It's interesting to note that validating adding an object to a container and validating assigning an attribute are both special cases of validating establishing a relationship between two objects. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jim at zope.com Sun Aug 8 15:58:32 2004 From: jim at zope.com (Jim Fulton) Date: Sun Aug 8 15:58:35 2004 Subject: [Python-Dev] Re: Thoughts on validation, events, rules In-Reply-To: <4116306A.1000904@zope.com> References: <4116306A.1000904@zope.com> Message-ID: <41163188.4080407@zope.com> Sorry, I sent this to the wrong list. I've been typing python-dev a lot lately. :) Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From adurdin at gmail.com Sun Aug 8 16:17:23 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Sun Aug 8 16:17:26 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> <411610FB.9040201@v.loewis.de> Message-ID: <59e9fd3a04080807177dd446af@mail.gmail.com> Having just gone through the wiki page again, I think the last example under A (quoted below) shows up very plainly the only really serious problem with the before-def decorator syntax: class TestDecorators(unittest.TestCase): ... def test_dotted(self): decorators = MiscDecorators() @decorators.author('Cleese') def foo(): return 42 self.assertEqual(foo(), 42) self.assertEqual(foo.author, 'Cleese') Suppose that you were not familiar enough with python to have heard of decorators: looking at this code, would you have any idea that the @ statement (potentially) fundamentally affects the behaviour of the foo() function? Is there anything except their proximity to suggest that they are interdependent? The only implication that they *might* be is the second assertEqual call, and that is by no means obvious. On the other hand, the after-def (but before-colon) variants clearly imply that the decorators are related to the function definition -- the reader might not understand their purpose, but they will not fail to connect them with the function. And here is a response to Guido's comments on the list-after-def syntax in another thread that were then incorporated into the wiki page. Guido> - it hides crucial information (e.g. that it is a static method) after the signature, where it is easily missed Not at all! The list of decorators would *be* a part of the signature. The decorators will be no more easily missed than the name of the last argument, which is arguably just as crucial. Guido> - it's easy to miss the transition between a long argument list and a long decorator list True. This is the main reason why having the list set off from the argument list with a keyword or symbol would be preferred; but suitable formatting would minimise the risk anyway -- see the example further below. Guido> - it's cumbersome to cut and paste a decorator list for reuse, because it starts and ends in the middle of a line This is (IMHO) a very weak argument. A decorator list with one or two decorators will probably be just as quick to type as to copy and paste with the before-def syntax, when the time to scroll between lines is taken into account (unless you're using a mouse, in which case highlighting within a line is not slow at all). Moreover, in more complicated decorator lists, the lists would likely be different (reducing the advantage of copy+paste); or if not, if you regularly use the same long decorator list, you're much better off defining your own decorator which applies those from the list, and then using that one repeatedly -- this will avoid errors and make maintenance easier. Guido> Given that the whole point of adding decorator syntax is to move the decorator from the end ("foo = staticmethod(foo)" after a 100-line body) to the front, where it is more in-your-face, it should IMO be moved all the way to the front. I disagree. Having the decorator at the end, after the function body, is obviously far from ideal, as it is extremely easy to miss; but the decorators are not the most important part of the function definition. The most important part (which should be most prominent, and thus first) is the "def" keyword introducing the function definition, and the function name. Whether the decorators are more or less important than the arguments (and thus which should come first) is less certain. The example of a long decorated function declaration that Guido gives was: def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42) [ staticmethod, funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL") ]: I don't think this is as clear as it could be (and an argument from the "ugliness" of syntax should be using the clearest possible formatting) -- breaking the line before the decorator list, and indenting the decorator list one more level (to set it off from the docstring/body) makes the whole much clearer, and prevents the decorator list from being confused with the arguments: def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42) \ [staticmethod, funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL")]: (Having the [ and ]: on their own lines at the same indentation level may be clearer yet) -- Andrew Durdin. From jhylton at gmail.com Sun Aug 8 16:26:33 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Sun Aug 8 16:26:35 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <1f7befae04080717357814d502@mail.gmail.com> References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> <2misbu3cbp.fsf@starship.python.net> <20040807222357.GA27723@vicky.ecs.soton.ac.uk> <1f7befae04080717357814d502@mail.gmail.com> Message-ID: On Sat, 7 Aug 2004 20:35:44 -0400, Tim Peters wrote: > [Armin Rigo] > > ... > > This is now fixed by skipping the optimization if we got an interned string. > > While messing with ob_refcnt is always a bit fragile I am still convinced that > > the += optimization is safe again. > > Ya, this bug wasn't scary (an unlikely but clear-in-hindsight problem, > specifically due to a subsystem lying about true refcounts -- not much > of that goes on, thank God). > > Alas, test_compiler still fails (but in an ordinary "doesn't work" > way, no longer in an internal-error way), so testing with -uall is > still a bit broken. I won't be able to fix it today, but I'll get on that asap. Jeremy From jason at tishler.net Sun Aug 8 16:54:42 2004 From: jason at tishler.net (Jason Tishler) Date: Sun Aug 8 16:47:51 2004 Subject: [Python-Dev] Cygwin and sockets In-Reply-To: <4110AABA.80407@v.loewis.de> References: <4110AABA.80407@v.loewis.de> Message-ID: <20040808145442.GF55644@tishler.net> Martin, On Wed, Aug 04, 2004 at 11:22:02AM +0200, "Martin v. L?wis" wrote: > Tony Meyer wrote: > >By "anybody", did you mean anybody that you know & trust? <0.5 wink>. > > > >I can duplicate the process outlined in the bug report and get the > >same failure to compile (I've added a note to it). The following: http://sf.net/tracker/?group_id=5470&atid=105470&func=detail&aid=728330 broke CVS for Cygwin too. > This is good enough for the moment, although I would have preferred an > answer like "I know, and I have a fix sitting on my harddisk" :-) I know, and I have a fix sitting on my harddisk. :,) See attach for the details. OK to apply? BTW, the socket module builds automatically for me. This may be because I have not upgraded to the latest Cygwin binutils due to the following issue: http://cygwin.com/ml/cygwin/2004-07/msg00997.html Jason -- PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6 -------------- next part -------------- Index: Modules/socketmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/socketmodule.c,v retrieving revision 1.298 diff -u -p -r1.298 socketmodule.c --- Modules/socketmodule.c 3 Aug 2004 08:52:45 -0000 1.298 +++ Modules/socketmodule.c 8 Aug 2004 14:29:33 -0000 @@ -285,6 +285,10 @@ int h_errno; /* not used */ #include "addrinfo.h" #endif +#if defined(__CYGWIN__) +#include "addrinfo.h" +#endif + #ifndef HAVE_INET_PTON int inet_pton(int af, const char *src, void *dst); const char *inet_ntop(int af, const void *src, char *dst, socklen_t size); From guido at python.org Sun Aug 8 17:25:19 2004 From: guido at python.org (Guido van Rossum) Date: Sun Aug 8 17:25:24 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: Your message of "Mon, 09 Aug 2004 00:17:23 +1000." <59e9fd3a04080807177dd446af@mail.gmail.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> <411610FB.9040201@v.loewis.de> <59e9fd3a04080807177dd446af@mail.gmail.com> Message-ID: <200408081525.i78FPJm20862@guido.python.org> > Having just gone through the wiki page again, I think the last example > under A (quoted below) shows up very plainly the only really serious > problem with the before-def decorator syntax: > > class TestDecorators(unittest.TestCase): > > ... > > def test_dotted(self): > decorators = MiscDecorators() > @decorators.author('Cleese') > def foo(): return 42 > self.assertEqual(foo(), 42) > self.assertEqual(foo.author, 'Cleese') > > Suppose that you were not familiar enough with python to have heard of > decorators: looking at this code, would you have any idea that the @ > statement (potentially) fundamentally affects the behaviour of the > foo() function? Is there anything except their proximity to suggest > that they are interdependent? The only implication that they *might* > be is the second assertEqual call, and that is by no means obvious. This an example of bad formatting. Tests often do that because their purpose is to test odd corners of functionality, not serve as sample code. Given that in typical example code there will always be a blank line separating the decorator from anything previous (and separating the body from what follows!) the connection between the decorator and the def is pretty clear. Also note that arguments based on partial knowledge of Python can be used as an argument against pretty much anything (just tweak the assumptions a bit), and hence have little value. --Guido van Rossum (home page: http://www.python.org/~guido/) From mark at prothon.org Sun Aug 8 00:55:53 2004 From: mark at prothon.org (Mark Hahn) Date: Sun Aug 8 18:20:39 2004 Subject: [Python-Dev] Decimal type question [Prothon] Message-ID: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> When I first announced Prothon, Tim Peters told me to feel free to post on Python lists because he considered Prothon a possible sandbox to learn from. So far I have interpreted this to mean c.l.p but I have been dissapointed in the quality of technical responses. Several people have suggested I post here instead. Let me know if you think I should go back to c.l.p. Prothon is considering using a floating Decimal type to fix the confusion around binary floating point mentioned in some of the Python pitfalls, warts, & gotchas. We are not considering total replacement of the binary Float because it would be a waste of the floating point hardware performance, and because the decimal implementation we are considering would be inferior in range. Our proposal it to add a new Decimal type "between" Int and Float. It would have less precision than the infinite precision of Int (Prothon uses Longs for Int), but more than Float. The Decimal would have less range than either Ints or Floats though (although 56 orders of magnitude is more range than is usually used in Ints). The Decimal type in .Net, which will be the VM for the first Prothon release, has a precision of 28 decimal digits and has a range of 7e-28 to 7e28. Decimal calculations that overflowed the range of 7e28 to 7e-28 (like 0.0000_0000_0000_0000_0000_0000_0000_7 / 2) would throw an exception instead of automatically converting to Float. Numeric constants containing only a decimal point would become a Decimal type: 1.354, 3452.7862. Any constant with the letter e would be a normal binary Float type: 1e-3, 1.23e62. Int divide "i/j" would always create a Decimal. Decimal would only support the basic ops: +, -, *, /, and simple funcs like min, max, and abs. The math module would automatically coerce Decimal arguments to Float for things like sin and log, just as it does with Int arguments. This would still solve the gotcha: 0.1 + 0.1 would equal 0.2 exactly. For that matter, any constant or expression (not including a divide) that didn't overflow/underflow the 28 digit precision would have an exact representation. Can anyone see any problem with this scheme? From ark-mlist at att.net Sun Aug 8 18:21:35 2004 From: ark-mlist at att.net (Andrew Koenig) Date: Sun Aug 8 18:21:26 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: Message-ID: <003101c47d63$c67b7330$6602a8c0@arkdesktop> > Did you omit the full wink? Or did you forget that | is already a binop? > Strictly speaking I guess it's unambiguous if one doesn't allow > arbitrary expressions as decorators, but... Neither, actually. It didn't occur to me that there might be an ambiguity any more than it occurs to me that f(3, 4) could be ambiguous in C because the comma might be an operator. I imagined the following rule: If a statement begins with "def", then every | outside parentheses delimits a decorator. If you insist on using an expression with | in it as a decorator, you have to parenthesize it. I can see now that even though this suggestion isn't really syntactically fuzzy, it might be considered too fussy. Still, I like it better than the prefix @ idea. I even realize why I don't like the @ syntax. It's not so much reserving @ as it is that it's a prefix syntax. It is ingrained in me that "def" starts a function definition, and now it doesn't any more. So, for example, if I use my editor to search for "def", I have to back up to see whether I've missed something before it. From pf_moore at yahoo.co.uk Sun Aug 8 18:35:16 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Sun Aug 8 18:35:05 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> <200408071617.i77GHgs10281@guido.python.org> Message-ID: Fernando Perez writes: >>> def foo(arg1, arg2): > > Ok, a new function foo() goes in, it takes 2 args, arg1 and arg2. > Make mental bucket labeled 'foo'. Anything that comes now, until we > de-indent, goes into box 'foo'. No need to keep random bits of > information up in the air until I know where they're supposed to go. Add a double-take here, because this is a method whose first argument isn't "self". We can't be sure it's not right yet, because of the staticmethod decorator, but store that for later thinking about... > >>> |classmethod > > Store 'classmethod' in bucket 'foo'. Oops. If we recall, foo doesn't have a "self" first argument, but neither does it have "cls". probably an error somewhere here, but we have to wait until the decorators are over to be sure. > >>> |accepts(int, int) > > Store 'accepts(int, int)' in bucket 'foo'. > >>> |returns(float) > > Store 'returns(float)' in bucket 'foo'. > > Done. All information is stored in mental bucket 'foo', where it > belongs. But we have a couple of inconsistencies to backtrack and resolve. (And the accepts/returns stuff is also inconsistent, but I'll stop picking holes in what is clearly just an example...) I'm not saying that the same issues mightn't exist with the decorator-before-def form, but I don't think the mental model for the decorator-after-def form is quite as clean as you'd like either... It's still subjective :-) Paul -- The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair. -- Douglas Adams From tim.peters at gmail.com Sun Aug 8 18:46:09 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 8 18:46:12 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> <2misbu3cbp.fsf@starship.python.net> <20040807222357.GA27723@vicky.ecs.soton.ac.uk> <1f7befae04080717357814d502@mail.gmail.com> Message-ID: <1f7befae0408080946767c0a1c@mail.gmail.com> [Tim] >> Alas, test_compiler still fails (but in an ordinary "doesn't work" >> way, no longer in an internal-error way), so testing with -uall is >> still a bit broken. [Jeremy] > I won't be able to fix it today, but I'll get on that asap. That's OK. I agree it's valuable to run this, so I fixed it. test_compiler passes on Windows now. But I have a gigabyte of RAM on this box to hold the stack . From tismer at stackless.com Sun Aug 8 18:48:56 2004 From: tismer at stackless.com (Christian Tismer) Date: Sun Aug 8 18:46:24 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <41145A7E.8010500@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41145A7E.8010500@v.loewis.de> Message-ID: <41165978.3010107@stackless.com> Martin v. L?wis wrote: > Christian Tismer wrote: > >> Ok, I dislike special prefix chars at all, in a language that >> doesn't have this concept elsewhere (despite strings of course, >> but their prefixes are just regular chars), > > > Lists, tuples, and dictionaries are also introduced with non-letter > characters. Furthermore, nested expressions use the same prefix > char that lists use. # introduces comments. I agree on the "#". Not so for lists, tuples and dicts. These are constructs in their own right, containing something, which is again made of normal Python objects. But what we here have is an escape symbol, which expresses that the token following it is special and has special semantics, and the reach of this special symbol then even extends over its own line, but modifies how the next def statement is understood. This is remarkably different from being usual. I should not have talked about prefix chars, but escape sequences, which fits much better. As somebody already mentioned (Jim Fulton I think) this feels like mixing two different languages into each other. all the best -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Sun Aug 8 19:04:35 2004 From: tismer at stackless.com (Christian Tismer) Date: Sun Aug 8 19:02:03 2004 Subject: [Python-Dev] capturing RETURN_VALUE Message-ID: <41165D23.4070202@stackless.com> Dear list, just by chance, I discovered a probably very unknown feature of the block stack machinery of Python's frames: -- It is possible to abandon a return statement. -- My question: Is this an artifact, or can I rely on this feature to persist for a while (while while >= 1 year) ? Example code: I have a code body which either computes and returns a value, or it raises an exception. Surrounding it, I have a while true loop, embracing a try..finally block. In case of an exception, it simply lets the exception pass. In case of a return value, it intercepts it and can do anything (in the example, it just decorates the value and returns). def return_capture(n, do_raise=False): _have_value = False # outer while loop, just there to get cancelled while True: try: # function body, returning a value or raising if do_raise: raise ValueError, "was told to raise" # signal valid value _have_value = True retval = n*n return finally: if _have_value: break # here we end after a captured return, and can do postprocessing retval = ("here the captured return value", retval) return retval Sample output: >>> return_capture(5) ('here the captured return value', 25) >>> return_capture(5, True) Traceback (most recent call last): File "", line 1, in ? File "", line 8, in return_capture ValueError: was told to raise >>> Again the question: Artifact, or a reliable feature? ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Sun Aug 8 20:30:30 2004 From: tismer at stackless.com (Christian Tismer) Date: Sun Aug 8 20:28:00 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <4114583C.4070500@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> Message-ID: <41167146.4070401@stackless.com> Martin v. L?wis wrote: > Christian Tismer wrote: [dropping things that I trashed while implementing] >> To save *some* typing (laziness was also a driver for this >> whole decorator nightmare), we could also remove the assignment >> and just have a function call like >> >> classmethod(myfunc) > > > This breaks backward compatibility. This is already legal > syntax, and should raise a NameError if myfunc is not > defined (actually, this holds for the first proposal, too). This is correct. I failed to understand that, initially, but when trying an implementation, I also found out that I need something to make this unambiguous. Here it is. What I added was yet another __stuff__ entry for new-style classes. It may also appear on module level, like the __metaclass__ feature works. So what I implemented was special treatment for decorators appearing this way: class MyClass(object): __decorators__ = classmethod, staticmethod, otherdeco classmethod(mymethod) def mymethod(self, *args, **kwds): ... The implementation intercepts all top-level occurances of objects mentioned in the __decorators__ sequence and treats the according statements as comments. They are all collected through the whole class definition, and put into an extra code object which is executed after the class code has finished. The generated post-code is of the form mymethod = classmethod(mymethod) and for nested cases mymethod = otherdeco(classmethod(mymethod)) ... > Your proposals are all by no means little. They are significant, > counter-intuitive changes of the language. What I depicted above doesn't seem so counter-intuitive to me. I don't claim it is *the* solution for the future, but it is a compromize which gives me the desired effect, without introducing new syntax, just a new __special_entry__. This makes obvious that this is a temporary feature which should be replaced by a better construct at some time, like __metaclass__ and __slots__ will probably, too. But it provides a solution without enforcing us to decide about new syntax right now, which appears to be too early for a group that can't find a common denominator, yet. > Also, try to come > up with counter-proposals that are well thought-out, instead of > being created in a rush. No rush, just incomplete. A side effect of this idea is that the decorator(funcname) items can appear wherever you like them in the class definition. I don't know if this is good. It might get used to advantage if we use decorators to define interface-like stuff, for instance. Then, a series of interface decorator "calls" might be put at the top of the class. My "implementation" is just a proof of concept. I didn't patch the code generator yet or hacked bytecode, but did a little source code transformation with he desired effect. This is enough to get a feeling for proof-of-concept. A drawback of this implementation is that the decorators are taken literally. It won't work if you rename them during the class definition. For my purposes it works fine, but I don't claim that it fulfils all other desired use cases. At least it is compatible, simple, complete, implemented in a way, and works for my applications. kind regards -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From edreamleo at charter.net Sun Aug 8 22:58:24 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Sun Aug 8 22:58:32 2004 Subject: [Python-Dev] Attribute strings? Message-ID: <001501c47d8a$72851660$6700a8c0@computer> Has anyone suggested the following? Create a new kind of string, _not_ a docstring, to hold annotations, etc. """@annotate whatever""" or even ""@" whatever""" Pros: Open ended possibilities, might quiet dissent, could be placed before function or inside it. Biggest drawback: something that looks like a string will affect generated code. thinking-inside-the-string'ly yours, Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From guido at python.org Sun Aug 8 23:04:11 2004 From: guido at python.org (Guido van Rossum) Date: Sun Aug 8 23:04:17 2004 Subject: [Python-Dev] Attribute strings? In-Reply-To: Your message of "Sun, 08 Aug 2004 15:58:24 CDT." <001501c47d8a$72851660$6700a8c0@computer> References: <001501c47d8a$72851660$6700a8c0@computer> Message-ID: <200408082104.i78L4Bb21243@guido.python.org> > Has anyone suggested the following? Not AFAIK. > Create a new kind of string, _not_ a docstring, to hold annotations, etc. > > """@annotate whatever""" or even ""@" whatever""" The first is a valid string literal now, and thus gets the same fate as [decorators] -- lots of people get very uncomfortable at the ambiguity of assigning a new meaning to something that was legal syntax before (even if it was meaningless). The second is not a valid string literal now -- doesn't that cause the same problems for Leo as @decorators? --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Sun Aug 8 23:31:14 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 8 23:31:19 2004 Subject: [Python-Dev] Tuple/list assignment question In-Reply-To: <000e01c47a2d$84f3be60$e841fea9@oemcomputer> References: <000e01c47a2d$84f3be60$e841fea9@oemcomputer> Message-ID: <41169BA2.9030108@ocf.berkeley.edu> Raymond Hettinger wrote: >[Guido] >>... I appreciate the idea (which BTW has been proposed before) > > > IIRC, the conversation for the original proposal fizzled out without > reaching a conclusion. So, resuming the discussion was probably a good > idea. But now it has been officially zapped. > > Actually, it was shot down by Guido last time as well: http://www.python.org/dev/summary/2002-11-16_2002-11-30.html#half-baked-proposal-and-in-assignments History has spoken! The power of the python-dev Summary and Google's 'site' functionality strikes again! =) -Brett From edreamleo at charter.net Sun Aug 8 23:20:21 2004 From: edreamleo at charter.net (Edward K. Ream) Date: Sun Aug 8 23:40:03 2004 Subject: [Python-Dev] re: Attribute Strings Message-ID: <000701c47d8d$8391aec0$6700a8c0@computer> > The second is not a valid string literal now -- That's why I suggested it. ""@" would be a new token, so the statement about '@' in the delims section would remain essentially unchanged. > doesn't that cause the same problems for Leo as @decorators? No. At most leading whitespace may precede Leo's directives. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From bac at OCF.Berkeley.EDU Sun Aug 8 23:46:37 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 8 23:46:42 2004 Subject: [Python-Dev] Optimized string concatenation In-Reply-To: <410FE549.1030407@iinet.net.au> References: <003201c47904$5cd353c0$df30cb97@oemcomputer> <200408031527.i73FRQb31100@guido.python.org> <410FE549.1030407@iinet.net.au> Message-ID: <41169F3D.7060508@ocf.berkeley.edu> Nick Coghlan wrote: > Adam Souzis wrote: > >> As someone relatively new to python, it struck me as a language wart >> that i had to learn the form '"".join() as the proper way to do string >> concatenation. It violates the principles of OOWTI and its certainly >> not the obvious way to do string concatenation. This patch does not >> cover all the cases so we're still stuck with join(), but as long as >> it is not a documentated "feature" it will harmlessly improve the >> performance of countless lines of code where the coder is either >> untrained or too lazy to use join(). If its documented it'd just >> muddy the waters vis a vis join(), besides the issues with other >> Python implementation mentioned here. > > > If I understand correctly, you're suggesting that ''.join(strings) > continue to be the recommended, portable, non-quadratic method for > concatenating strings. > I think the "portable" label is a little misleading. It isn't like using string concatenation now is non-portable, it's just slow. The other labels are correct, though. -Brett From nbastin at opnet.com Sun Aug 8 23:54:38 2004 From: nbastin at opnet.com (Nick Bastin) Date: Sun Aug 8 23:54:44 2004 Subject: [Python-Dev] re: Attribute Strings In-Reply-To: <000701c47d8d$8391aec0$6700a8c0@computer> References: <000701c47d8d$8391aec0$6700a8c0@computer> Message-ID: <8B2986D7-E985-11D8-A534-000D932927FE@opnet.com> On Aug 8, 2004, at 5:20 PM, Edward K. Ream wrote: >> The second is not a valid string literal now -- > > That's why I suggested it. ""@" would be a new token, so the statement > about '@' in the delims section would remain essentially unchanged. I don't know if anyone is actually seriously considering this, but I'm -1. We're venturing into an area where the syntax makes people really not want to use the feature (and ""@" probably doesn't please the people who don't like the aesthetics of '@', so all we're doing is fixing the problem for IPython and a few other corner people). My impression is that -before-def is going to be The Syntax(tm), but Guido is content to let be something other than '@'. If that's the case, then I'm not sure what all the alternate placement syntaxes are being proposed for. If that's not the case, then I'll start lobbying harder for -after-def (although not ""@")... ;-) -- Nick From guido at python.org Sun Aug 8 23:59:54 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 00:00:00 2004 Subject: [Python-Dev] Re: Attribute Strings In-Reply-To: Your message of "Sun, 08 Aug 2004 16:20:21 CDT." <000701c47d8d$8391aec0$6700a8c0@computer> References: <000701c47d8d$8391aec0$6700a8c0@computer> Message-ID: <200408082159.i78LxsO21394@guido.python.org> > > The second is not a valid string literal now -- > > That's why I suggested it. ""@" would be a new token, so the statement > about '@' in the delims section would remain essentially unchanged. Hm. You might as well propose @#$% then. :-) Anyway, the problem with anything inside "string literals" (real or perceived) is that then the parser has to recursively parse the expression in the string literal, because it has to generate code for it. I consider this too perverse to bother. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Mon Aug 9 01:09:41 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 01:09:44 2004 Subject: [Python-Dev] capturing RETURN_VALUE In-Reply-To: <41165D23.4070202@stackless.com> References: <41165D23.4070202@stackless.com> Message-ID: <1f7befae040808160924e4d8a0@mail.gmail.com> [Christian Tismer] > just by chance, I discovered a probably very unknown > feature of the block stack machinery of Python's frames: > > -- It is possible to abandon a return statement. -- [if the return tickles an enclosing finally block, and the finally block, e.g., does a different return, or breaks out of a loop, or raises an exception] > My question: Is this an artifact, or can I rely on this > feature to persist for a while (while while >= 1 year) ? Yes, it's reliable. The Java spec is clearer about it, but Python acts the same way. If Python allowed continue statements in finally blocks, you'd also find that a finally block could "abandon" a break statement (if the break tickled an enclosing finally block in the same loop, and the finally block did a continue). > I have a code body which either computes and returns a value, > or it raises an exception. Surrounding it, I have a while > true loop, embracing a try..finally block. In case of an > exception, it simply lets the exception pass. It did in your specific example, but not necessarily. In your specific example, the finally block didn't execute "break" if an exception was raised, it only did "break" if the finally block was entered because of a return. If it had done "break" in the exception case too, it would have "abandoned" the exception too: def return_capture(n, do_raise=False): _have_value = False retval = 666 while True: try: if do_raise: raise ValueError, "was told to raise" _have_value = True retval = n*n return finally: break retval = ("here the captured return value", retval) return retval >>> return_capture(12) ('here the captured return value', 144) >>> return_capture(12, True) ('here the captured return value', 666) >>> So there's really no difference between how returns and exceptions work here. From tismer at stackless.com Mon Aug 9 01:18:23 2004 From: tismer at stackless.com (Christian Tismer) Date: Mon Aug 9 01:15:51 2004 Subject: [Python-Dev] capturing RETURN_VALUE In-Reply-To: <1f7befae040808160924e4d8a0@mail.gmail.com> References: <41165D23.4070202@stackless.com> <1f7befae040808160924e4d8a0@mail.gmail.com> Message-ID: <4116B4BF.5030409@stackless.com> Tim Peters wrote: ... > It did in your specific example, but not necessarily. In your > specific example, the finally block didn't execute "break" if an > exception was raised, it only did "break" if the finally block was > entered because of a return. If it had done "break" in the exception > case too, it would have "abandoned" the exception too: Sure, that was by design. I'm trying to inline functions into each other, with smallest possible changes to the bytecodes of the inlined functions. Exceptions shall pass through, returns shall be captured and *not* cause the embracing function to return, but to continue with the "returned" value. thanks for this answer -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tim.peters at gmail.com Mon Aug 9 01:32:00 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 01:32:04 2004 Subject: [Python-Dev] capturing RETURN_VALUE In-Reply-To: <4116B4BF.5030409@stackless.com> References: <41165D23.4070202@stackless.com> <1f7befae040808160924e4d8a0@mail.gmail.com> <4116B4BF.5030409@stackless.com> Message-ID: <1f7befae04080816325f3fa2f2@mail.gmail.com> [Tim Peters] >> It did in your specific example, but not necessarily. In your >> specific example, the finally block didn't execute "break" if an >> exception was raised, it only did "break" if the finally block was >> entered because of a return. If it had done "break" in the exception >> case too, it would have "abandoned" the exception too: [Christian Tismer] > Sure, that was by design. > I'm trying to inline functions into each other, with > smallest possible changes to the bytecodes of the > inlined functions. Exceptions shall pass through, > returns shall be captured and *not* cause the embracing > function to return, but to continue with the "returned" > value. The point was that exceptions and returns act the same way: RETURN_VALUE isn't unique here, it's just another instance of leaving a try-block's suite, the same in this respect as an exception or a break statement. That's why it's reliable over time: it's a general mechanism at work, not an arbitrary hack specific to RETURN_VALUE. From greg at cosc.canterbury.ac.nz Mon Aug 9 01:58:12 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 01:58:26 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: Message-ID: <200408082358.i78NwCPf005658@cosc353.cosc.canterbury.ac.nz> > I realize that when Greg said ,"...see these arguments as being > rational ones about usability and not irrational ones about > aesthetics" he meant that arguments about aesthetics are usually > subjective. Partly that, but also that I don't think all of the arguments about @ vs. [...] are purely subjective. It seems a fairly objective fact to me that [...] carries more meaning in terms of pre-existing connotations. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Mon Aug 9 02:01:47 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 02:01:52 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: Message-ID: <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz> IxokaI : > '"define methy1 at staticmethod" is wrong, but "at > staticmethod define methy2" is right. Pardon? They *both* sound like utter nonsense to me. That's the other main objection I have to '@'. I just can't help pronouncing it in my head as "at". Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From kamikaze at kuoi.asui.uidaho.edu Mon Aug 9 02:25:56 2004 From: kamikaze at kuoi.asui.uidaho.edu (Mark 'Kamikaze' Hughes) Date: Mon Aug 9 02:26:01 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <20040809002556.GA75629@kuoi.asui.uidaho.edu> Sat, Aug 07, 2004 at 12:35:17AM -0400 in <41145C05.1020501@gradient.cis.upenn.edu>, Edward Loper spake: >The new "vertical bar" syntax was proposed recently, and has gotten >almost unanimously positive feedback. In particular, of 17 responses >to the syntax, only one has been negative (see table, below). >I'm including both of the following variants: > |classmethod > |accepts(int, int) > |returns(float) > def foo(arg1, arg2): > ... > def foo(arg1, arg2): > |classmethod > |accepts(int, int) > |returns(float) > ... >(I prefer the second, but would be happy with either.) >The one negative comment was that "|" can look similar to "I" or "l", >depending on the font; but it's only an issue for some fonts, and it >should be largely mitigated by syntax-highlighting. >So I wanted to see if anyone has anything *negative* to say about this >syntax (or one of its variants). If possible, please respond by >updating the wiki, rather than by email. This syntax is listed under >H (pre-def) and E2 (post-def). You will need to sign in before you >can edit the wiki. > Wiki: -1. I think vbar-before is an extremely hard syntax to read, and vbar-after is extremely misleading (but I don't like where docstrings are, either). Wiki updated. -- Mark Hughes "Doing the impossible makes us mighty." -Captain Malcolm Reynolds, Firefly From tim.peters at gmail.com Mon Aug 9 02:42:57 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 02:42:59 2004 Subject: [Python-Dev] PyRun_ with file name In-Reply-To: <4113EF69.2020102@v.loewis.de> References: <4113EF69.2020102@v.loewis.de> Message-ID: <1f7befae040808174233fb20db@mail.gmail.com> [Martin v. L?wis] > There is a constant issue of mixing CRTs on Windows. Patch 1003535 > proposes to work around them by adding yet another PyRun_ function. > I'm in favour, except that: > - it should be called PyRun_Filename > - it should have the maximum current parameter set, i.e. including > CompilerFlags* The patch appears to include a whole bunch of irrelevant changes too (which I've noted on the tracker). > I'm doubtful as to whether specifying an opening mode is necessary. Or useful <0.5 wink>. > What do you think? If you understand the problem, and believe it will fix it, +1 (I don't think I fully understand the problem, but it gets points from me for being a small change). From greg at cosc.canterbury.ac.nz Mon Aug 9 03:05:48 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 03:05:55 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408061445.i76EjqT07026@guido.python.org> Message-ID: <200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz> Guido: > Just look at it when either the list of decorators or the argument > list doesn't fit on one line: You don't *have* to format it quite that badly. def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42) \ [staticmethod, funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL")]: If you can arrange for the '[' to appear immediately before the first decorator, I think it makes it a lot easier to recognise the construct for what it is. I also think that's a rather untypical decorator argument you've used there, being full of spaces, brackets and other confounding characters. That sort of thing is going to look confusing in any context, whether function decorators are involved or not. Perhaps someone can post some real-life use cases written with this syntax, so we can see what it would *really* look like in typical use, as opposed to made-up worst-case examples? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Mon Aug 9 03:07:55 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 03:08:05 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: Message-ID: <200408090107.i7917tOl005740@cosc353.cosc.canterbury.ac.nz> Jeremy Hylton : > I thought it was obvious that this option was not acceptable. Trouble is, it's equally "obvious" to the opponents of "@" that *it's* not acceptable... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From kamikaze at kuoi.asui.uidaho.edu Mon Aug 9 03:11:00 2004 From: kamikaze at kuoi.asui.uidaho.edu (Mark 'Kamikaze' Hughes) Date: Mon Aug 9 03:11:03 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz> References: <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz> Message-ID: <20040809011100.GB75629@kuoi.asui.uidaho.edu> Mon, Aug 09, 2004 at 12:01:47PM +1200 in <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>, Greg Ewing spake: >IxokaI : >> '"define methy1 at staticmethod" is wrong, but "at >> staticmethod define methy2" is right. >Pardon? They *both* sound like utter nonsense to me. >That's the other main objection I have to '@'. I just >can't help pronouncing it in my head as "at". Try pronouncing it as "attribute" (n.) or "attribute" (v.). Either one makes sense in context. -- Mark Hughes "Doing the impossible makes us mighty." -Captain Malcolm Reynolds, Firefly From bob at redivi.com Mon Aug 9 03:25:20 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 9 03:25:30 2004 Subject: [Python-Dev] @decorators, the PEP and the "options" out there? In-Reply-To: <20040809011100.GB75629@kuoi.asui.uidaho.edu> References: <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz> <20040809011100.GB75629@kuoi.asui.uidaho.edu> Message-ID: On Aug 8, 2004, at 9:11 PM, Mark 'Kamikaze' Hughes wrote: > Mon, Aug 09, 2004 at 12:01:47PM +1200 in > <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>, > Greg Ewing spake: >> IxokaI : >>> '"define methy1 at staticmethod" is wrong, but "at >>> staticmethod define methy2" is right. >> Pardon? They *both* sound like utter nonsense to me. >> That's the other main objection I have to '@'. I just >> can't help pronouncing it in my head as "at". > > Try pronouncing it as "attribute" (n.) or "attribute" (v.). Either > one makes sense in context. Not really. It's actually a transformation, not an attribute. A more appropriate verb would be "make" or "is", I guess. @staticmethod def foo(...): pass -bob From janssen at parc.com Mon Aug 9 03:31:49 2004 From: janssen at parc.com (Bill Janssen) Date: Mon Aug 9 03:32:24 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: Your message of "Sat, 07 Aug 2004 09:17:42 PDT." <200408071617.i77GHgs10281@guido.python.org> Message-ID: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> > In the discussion on decorators months ago, solutions involving > special syntax inside the block were ruled out early on. Basically, > you shouldn't have to peek inside the block to find out important > external properties of the function. Guido, could you expand on this principle a bit? Just stated as it is, it sounds as pointless (to my untutored mind, at least :-), as the arguments about "@ is ugly, so it shouldn't be used". After all, isn't "what the function does" an important external property of the function, and don't you have to peek inside the block to determine that? For instance, how do you know whether a function is a generator unless you look for yield statements inside it? Bill From greg at cosc.canterbury.ac.nz Mon Aug 9 04:02:39 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 04:02:51 2004 Subject: [SPAM: 3.611] [Python-Dev] IPython, @, and option E from the wiki In-Reply-To: Message-ID: <200408090202.i7922diB005813@cosc353.cosc.canterbury.ac.nz> 3. In this kind of setup, using | instead of @ would be ok as well, I guess: Fernando Perez : > def bar(low,high): > |accepts(int,int) > |returns(float) How about def bar(low,high): ...accepts(int,int) ...returns(float) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Mon Aug 9 04:12:49 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 04:13:00 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: Message-ID: <200408090212.i792CnP9005824@cosc353.cosc.canterbury.ac.nz> Terry Reedy : > Is a space allowed after the prefic character, whatever it might be? With > > def foo(): > | classmethod > | accepts(int,int) > | returns(float) It certainly looks much better with the space. But I'm not sure if it should be required -- it would be a strange kind of rule, inconsistent with the lexical style of the rest of the language. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Mon Aug 9 04:16:45 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 04:16:50 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <041501c47c45$54e3f630$e615a88d@neil> Message-ID: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> > For example, if there is later a need or desire to annotate function > arguments individually as well as functions, or individual > statements or expressions, then @ is a better choice. For example > > @RemoteCall > def Ignite(engine @Remote, environment @Copy, thrust @Float): Nooooo..... If @decorator stays in, I fervently hope it won't be taken as a precedent for using @ for other things all over the place. Otherwise we really *will* end up with a language that resembles Perl. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Mon Aug 9 04:29:07 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 04:29:20 2004 Subject: [Python-Dev] Suggesting '.' decorators (PEP318) In-Reply-To: <200408071128.56011.heikowu@ceosg.de> Message-ID: <200408090229.i792T7QI005872@cosc353.cosc.canterbury.ac.nz> Heiko Wundram : > Same objection from me as to the proposal of decorating with | in > the function: it simply takes away the actual meaning from the > decorator (which actually is to make it very clear to see that the > function is special in some way). I think a problem we have here is that how important it is to know that a function is being decorated depends on the nature of the decorator. For example, it can be important for the user of a function to know that it's a classmethod or staticmethod. On the other hand, the Objective-C signature of a PyObjC method is something that's only of interest to the implementor of the method. So there isn't going to be a one-size-fits-all approach that will give all decorators exactly the right amount of prominence. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From jepler at unpythonic.net Mon Aug 9 04:29:34 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Mon Aug 9 04:30:07 2004 Subject: [Python-Dev] The date in the generated documentation Message-ID: <20040809022934.GB6582@unpythonic.net> This is probably the strangest question I've ever asked python-dev. For the Python bug day I updated my CVS, and later on typed "make" in the Doc directory. I'm now viewing the generated html/lib/lib.html file, and it's displayed in that weird font that Japanese web pages always use, and the top of the document says Release 2.4a2 ??16?8?7? which is, as nearly as I can tell, a (correct?) Japanese "Emperor Date". http://www.i18nguy.com/l10n/emperor-date.html gives a little explanation for us westerners, the first two characters match the Kanji he gives for the current era (beginning in 1989), and the Western calendar year is now 1989 + 16 - 1. Also, the strings "Context", "Index", and "About this document" are written in Japanese characters. This appears to affect the other documents (ref/ref.html, etc) generated as well. Nothing on my system should be particularly Japanese (my locale is en_US.UTF-8, for example). While I installed a lot of fonts with this system, I haven't done anything else that would lead me to expect this. The OS is Fedora Core 2 upgraded from FC1 if that helps ring anybody's bell. The paper-letter/*.pdf files that "make pdf" generates have the proper "August 8, 2004" date in them. Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040808/c5869781/attachment.pgp From greg at cosc.canterbury.ac.nz Mon Aug 9 04:31:00 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 9 04:31:09 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com> Message-ID: <200408090231.i792V0Y0005878@cosc353.cosc.canterbury.ac.nz> > Two, and more importantly: the pipe is in the shifted position of the > backslash key, which on different keyboards is in different locations, > many of them awkward to reach while typing at speed. But realistically, how many function decorators do you expect to be typing per second? :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From nhodgson at bigpond.net.au Mon Aug 9 04:37:05 2004 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Mon Aug 9 04:37:10 2004 Subject: [Python-Dev] Decorators: vertical bar syntax References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> Message-ID: <019301c47db9$c269ec80$a9498890@neil> Greg Ewing: > > @RemoteCall > > def Ignite(engine @Remote, environment @Copy, thrust @Float): > > Nooooo..... > > If @decorator stays in, I fervently hope it won't be taken as a > precedent for using @ for other things all over the place. Otherwise > we really *will* end up with a language that resembles Perl. A motivation for this was that one of the examples @accepts(int,int) @returns(float) def bar(low,high): which separates related information too much for me. If decorators need to decorate parameters, they are more likely to be understood correctly by a reader if the decorations are next to the parameters. Neil From jepler at unpythonic.net Mon Aug 9 04:43:36 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Mon Aug 9 04:44:05 2004 Subject: [Python-Dev] The date in the generated documentation In-Reply-To: <20040809022934.GB6582@unpythonic.net> References: <20040809022934.GB6582@unpythonic.net> Message-ID: <20040809024336.GC6582@unpythonic.net> I had to edit the lines in the file /usr/share/latex2html/l2hconf.pm which mentioned japanese (by default! I'm certain I never changed them) and change them back to english. The generated html documentation is now much improved. Sorry to bother everyone for what turned out to be a fedora bug / configuration problem. Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040808/3a7fb980/attachment.pgp From aahz at pythoncraft.com Mon Aug 9 05:35:54 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 9 05:35:56 2004 Subject: [Python-Dev] pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching In-Reply-To: References: Message-ID: <20040809033554.GA21340@panix.com> On Sat, Aug 07, 2004, Mike Coleman wrote: > > Motivation > ========== This section needs fleshing out to explain why re.structmatch works better for some purposes than forcing people to switch to a full-blown parser. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From aahz at pythoncraft.com Mon Aug 9 05:40:36 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 9 05:40:38 2004 Subject: [Python-Dev] re.split on empty patterns In-Reply-To: <20040807145142.GB2529@rogue.amk.ca> References: <20040807145142.GB2529@rogue.amk.ca> Message-ID: <20040809034036.GB21340@panix.com> On Sat, Aug 07, 2004, A.M. Kuchling wrote: > > Question: do we want to make this option the new default? Agreed with Tim: -1 until Python 3 -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From aahz at pythoncraft.com Mon Aug 9 05:51:38 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 9 05:51:46 2004 Subject: [Python-Dev] Decimal type question [Prothon] In-Reply-To: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: <20040809035138.GC21340@panix.com> On Sat, Aug 07, 2004, Mark Hahn wrote: > > Can anyone see any problem with this scheme? Yes: it doesn't follow the decimal floating point standard. Read PEP327 for an exhaustive discussion of the subject WRT Python (which is nowhere near as exhausting as reading the discussion in the python-dev archives). Side note: I haven't bothered checking recently, but since you were complaining about the quality of technical responses on c.l.py, I've been mostly ignoring you because of your habit of top-posting. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From tim.peters at gmail.com Mon Aug 9 06:01:17 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 06:01:21 2004 Subject: [Python-Dev] Re: MSI Installer issues In-Reply-To: <4114A02F.7060906@v.loewis.de> References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de> <1f7befae040805103911610582@mail.gmail.com> <411274C0.4070300@v.loewis.de> <1f7befae04080623276089908e@mail.gmail.com> <4114A02F.7060906@v.loewis.de> Message-ID: <1f7befae040808210137f08a9e@mail.gmail.com> [Martin v. L?wis] > ... > While I have your attention: you have commented that the installer logo > needs attribution, during the installation process. Is this still the > case, and what is the text I should use? Any other attribution that > needs to be made? Its creator (Erik van Blokland) asked for attribution, in return for doing the work. All I know is here: http://mail.python.org/pipermail/python-dev/2004-July/046317.html He seemed happy with the attribution quoted there. > A colleague has commented that logo is too low-res, is ugly, and does > not feature reptiles. Should we look for an update to the logo? I'm surprised this didn't trigger a flood of responses dwarfing the decorators thread! I'm out of any logo debate. "low-res" may be a feature, as some people have very low screen resolution settings and very large fonts. Etc, etc. If you want the quest to end, don't ask Guido to approve a new logo <0.9 wink>. From dima at trit.org Mon Aug 9 06:31:05 2004 From: dima at trit.org (Dima Dorfman) Date: Mon Aug 9 06:33:28 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: <410EF963.40704@interlink.com.au> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> <410EF963.40704@interlink.com.au> Message-ID: <20040809043104.GA82909@trit.org> Anthony Baxter wrote: > Guido van Rossum wrote: > > Did the relative import syntax that was approved ever get checked in? > > I thought it was on the list of things to land in 2.4? > > Nope. The 2.4 release schedule PEP now lists this as lacking someone > to drive it. I won't/can't spend time on it. So, if one of the people > who wants it could step forward, that'd be excellent. Even if it's > just to do the multi-line import syntax of > > from foo import ( bar, baz, bar2, > baz2, bar3, baz3 ) I took a stab at implementing this (the easy part of PEP 328--parens around import lists). Are the exact semantics of what's allowed documented somewhere? PEP 328 mostly talks about relative and absolute imports, and it doesn't specify the exact semantics of where parentheses should be allowed. My patch (attached) accepts import (os, sys) from sys import (stdin, stdout, stderr) import (os) from sys import (*) but rejects from (sys) import stdin import (os), (sys) import (os,) Should any of those be allowed? Anything that I missed? The patch is incomplete in other ways; the docs haven't been updated and neither have the parser module and compile package. If it's decided that it would be okay to include this separately from relative/absolute import support (they're different features, really), I'll complete the patch. Dima. -------------- next part -------------- Index: Python/compile.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/compile.c,v retrieving revision 2.314 diff -u -c -r2.314 compile.c *** Python/compile.c 6 Aug 2004 19:46:34 -0000 2.314 --- Python/compile.c 9 Aug 2004 04:01:38 -0000 *************** *** 3333,3374 **** static void com_import_stmt(struct compiling *c, node *n) { int i; REQ(n, import_stmt); ! /* 'import' dotted_name (',' dotted_name)* | ! 'from' dotted_name 'import' ('*' | NAME (',' NAME)*) */ ! if (STR(CHILD(n, 0))[0] == 'f') { PyObject *tup; - /* 'from' dotted_name 'import' ... */ REQ(CHILD(n, 1), dotted_name); ! ! if (TYPE(CHILD(n, 3)) == STAR) { tup = Py_BuildValue("(s)", "*"); ! } else { ! tup = PyTuple_New((NCH(n) - 2)/2); ! for (i = 3; i < NCH(n); i += 2) { ! PyTuple_SET_ITEM(tup, (i-3)/2, PyString_FromString(STR( ! CHILD(CHILD(n, i), 0)))); } } com_addoparg(c, LOAD_CONST, com_addconst(c, tup)); Py_DECREF(tup); com_push(c, 1); com_addopname(c, IMPORT_NAME, CHILD(n, 1)); ! if (TYPE(CHILD(n, 3)) == STAR) com_addbyte(c, IMPORT_STAR); else { ! for (i = 3; i < NCH(n); i += 2) ! com_from_import(c, CHILD(n, i)); com_addbyte(c, POP_TOP); } com_pop(c, 1); } else { ! /* 'import' ... */ ! for (i = 1; i < NCH(n); i += 2) { ! node *subn = CHILD(n, i); REQ(subn, dotted_as_name); com_addoparg(c, LOAD_CONST, com_addconst(c, Py_None)); com_push(c, 1); --- 3333,3379 ---- static void com_import_stmt(struct compiling *c, node *n) { + node *nn; int i; REQ(n, import_stmt); ! n = CHILD(n, 0); ! /* import_stmt: just_import | from_import */ ! if (TYPE(n) == from_import) { ! /* from_import: 'from' dotted_name 'import' ('*' | '(' '*' ')' ! | '(' import_as_names ')' | import_as_names) */ PyObject *tup; REQ(CHILD(n, 1), dotted_name); ! nn = CHILD(n, 3 + (TYPE(CHILD(n, 3)) == LPAR)); ! if (TYPE(nn) == STAR) tup = Py_BuildValue("(s)", "*"); ! else { ! REQ(nn, import_as_names); ! tup = PyTuple_New((NCH(nn) + 1) / 2); ! for (i = 0; i < NCH(nn); i += 2) { ! PyTuple_SET_ITEM(tup, i / 2, PyString_FromString(STR( ! CHILD(CHILD(nn, i), 0)))); } } com_addoparg(c, LOAD_CONST, com_addconst(c, tup)); Py_DECREF(tup); com_push(c, 1); com_addopname(c, IMPORT_NAME, CHILD(n, 1)); ! if (TYPE(nn) == STAR) com_addbyte(c, IMPORT_STAR); else { ! for (i = 0; i < NCH(nn); i += 2) ! com_from_import(c, CHILD(nn, i)); com_addbyte(c, POP_TOP); } com_pop(c, 1); } else { ! /* 'import' ('(' dotted_as_names ')' | dotted_as_names)*/ ! nn = CHILD(n, 1 + (TYPE(CHILD(n, 1)) == LPAR)); ! REQ(nn, dotted_as_names); ! for (i = 0; i < NCH(nn); i += 2) { ! node *subn = CHILD(nn, i); REQ(subn, dotted_as_name); com_addoparg(c, LOAD_CONST, com_addconst(c, Py_None)); com_push(c, 1); *************** *** 6131,6144 **** static void symtable_import(struct symtable *st, node *n) { int i; ! /* import_stmt: 'import' dotted_as_name (',' dotted_as_name)* ! | 'from' dotted_name 'import' ! ('*' | import_as_name (',' import_as_name)*) ! import_as_name: NAME [NAME NAME] ! */ ! if (STR(CHILD(n, 0))[0] == 'f') { /* from */ node *dotname = CHILD(n, 1); if (strcmp(STR(CHILD(dotname, 0)), "__future__") == 0) { /* check for bogus imports */ if (n->n_lineno >= st->st_future->ff_last_lineno) { --- 6136,6150 ---- static void symtable_import(struct symtable *st, node *n) { + node *nn; int i; ! /* import_stmt: just_import | from_import */ ! n = CHILD(n, 0); ! if (TYPE(n) == from_import) { ! /* from_import: 'from' dotted_name 'import' ('*' | '(' '*' ')' ! | '(' import_as_names ')' | import_as_names) */ node *dotname = CHILD(n, 1); + REQ(dotname, dotted_name); if (strcmp(STR(CHILD(dotname, 0)), "__future__") == 0) { /* check for bogus imports */ if (n->n_lineno >= st->st_future->ff_last_lineno) { *************** *** 6148,6154 **** return; } } ! if (TYPE(CHILD(n, 3)) == STAR) { if (st->st_cur->ste_type != TYPE_MODULE) { if (symtable_warn(st, "import * only allowed at module level") < 0) --- 6154,6161 ---- return; } } ! nn = CHILD(n, 3 + (TYPE(CHILD(n, 3)) == LPAR)); ! if (TYPE(nn) == STAR) { if (st->st_cur->ste_type != TYPE_MODULE) { if (symtable_warn(st, "import * only allowed at module level") < 0) *************** *** 6157,6164 **** st->st_cur->ste_optimized |= OPT_IMPORT_STAR; st->st_cur->ste_opt_lineno = n->n_lineno; } else { ! for (i = 3; i < NCH(n); i += 2) { ! node *c = CHILD(n, i); if (NCH(c) > 1) /* import as */ symtable_assign(st, CHILD(c, 2), DEF_IMPORT); --- 6164,6172 ---- st->st_cur->ste_optimized |= OPT_IMPORT_STAR; st->st_cur->ste_opt_lineno = n->n_lineno; } else { ! REQ(nn, import_as_names); ! for (i = 0; i < NCH(nn); i += 2) { ! node *c = CHILD(nn, i); if (NCH(c) > 1) /* import as */ symtable_assign(st, CHILD(c, 2), DEF_IMPORT); *************** *** 6167,6176 **** DEF_IMPORT); } } ! } else { ! for (i = 1; i < NCH(n); i += 2) { ! symtable_assign(st, CHILD(n, i), DEF_IMPORT); ! } } } --- 6175,6186 ---- DEF_IMPORT); } } ! } else { ! /* 'import' ('(' dotted_as_names ')' | dotted_as_names)*/ ! nn = CHILD(n, 1 + (TYPE(CHILD(n, 1)) == LPAR)); ! REQ(nn, dotted_as_names); ! for (i = 0; i < NCH(nn); i += 2) ! symtable_assign(st, CHILD(nn, i), DEF_IMPORT); } } Index: Python/future.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/future.c,v retrieving revision 2.13 diff -u -c -r2.13 future.c *** Python/future.c 11 Dec 2002 14:04:59 -0000 2.13 --- Python/future.c 9 Aug 2004 04:01:38 -0000 *************** *** 18,35 **** { int i; char *feature; ! node *ch; ! REQ(n, import_stmt); /* must by from __future__ import ... */ ! ! for (i = 3; i < NCH(n); i += 2) { ! ch = CHILD(n, i); ! if (TYPE(ch) == STAR) { ! PyErr_SetString(PyExc_SyntaxError, ! FUTURE_IMPORT_STAR); ! PyErr_SyntaxLocation(filename, ch->n_lineno); ! return -1; ! } REQ(ch, import_as_name); feature = STR(CHILD(ch, 0)); if (strcmp(feature, FUTURE_NESTED_SCOPES) == 0) { --- 18,35 ---- { int i; char *feature; ! node *ch, *nn; ! REQ(n, from_import); ! nn = CHILD(n, 3 + (TYPE(CHILD(n, 3)) == LPAR)); ! if (TYPE(nn) == STAR) { ! PyErr_SetString(PyExc_SyntaxError, FUTURE_IMPORT_STAR); ! PyErr_SyntaxLocation(filename, nn->n_lineno); ! return -1; ! } ! REQ(nn, import_as_names); ! for (i = 0; i < NCH(nn); i += 2) { ! ch = CHILD(nn, i); REQ(ch, import_as_name); feature = STR(CHILD(ch, 0)); if (strcmp(feature, FUTURE_NESTED_SCOPES) == 0) { *************** *** 188,194 **** case import_stmt: { node *name; ! if (STR(CHILD(n, 0))[0] != 'f') { /* from */ ff->ff_last_lineno = n->n_lineno; return 0; } --- 188,195 ---- case import_stmt: { node *name; ! n = CHILD(n, 0); ! if (TYPE(n) != from_import) { ff->ff_last_lineno = n->n_lineno; return 0; } Index: Grammar/Grammar =================================================================== RCS file: /cvsroot/python/python/dist/src/Grammar/Grammar,v retrieving revision 1.50 diff -u -c -r1.50 Grammar *** Grammar/Grammar 2 Aug 2004 06:09:53 -0000 1.50 --- Grammar/Grammar 9 Aug 2004 04:01:38 -0000 *************** *** 51,59 **** return_stmt: 'return' [testlist] yield_stmt: 'yield' testlist raise_stmt: 'raise' [test [',' test [',' test]]] ! import_stmt: 'import' dotted_as_name (',' dotted_as_name)* | 'from' dotted_name 'import' ('*' | import_as_name (',' import_as_name)*) import_as_name: NAME [NAME NAME] dotted_as_name: dotted_name [NAME NAME] dotted_name: NAME ('.' NAME)* global_stmt: 'global' NAME (',' NAME)* exec_stmt: 'exec' expr ['in' test [',' test]] --- 51,63 ---- return_stmt: 'return' [testlist] yield_stmt: 'yield' testlist raise_stmt: 'raise' [test [',' test [',' test]]] ! import_stmt: just_import | from_import ! just_import: 'import' ('(' dotted_as_names ')' | dotted_as_names) ! from_import: 'from' dotted_name 'import' ('*' | '(' '*' ')' | '(' import_as_names ')' | import_as_names) import_as_name: NAME [NAME NAME] dotted_as_name: dotted_name [NAME NAME] + import_as_names: import_as_name (',' import_as_name)* + dotted_as_names: dotted_as_name (',' dotted_as_name)* dotted_name: NAME ('.' NAME)* global_stmt: 'global' NAME (',' NAME)* exec_stmt: 'exec' expr ['in' test [',' test]] Index: Lib/test/test_compile.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_compile.py,v retrieving revision 1.22 diff -u -c -r1.22 test_compile.py *** Lib/test/test_compile.py 2 Aug 2004 08:30:07 -0000 1.22 --- Lib/test/test_compile.py 9 Aug 2004 04:01:39 -0000 *************** *** 211,216 **** --- 211,263 ---- self.assertRaises(SyntaxError, compile, stmt, 'tmp', 'single') self.assertRaises(SyntaxError, compile, stmt, 'tmp', 'exec') + def test_import(self): + # These should pass + exec ''' + from __future__ import (nested_scopes, generators) + import sys + import (os, + sys) + from sys import (stdin, stderr, + stdout) + from keyword import (*)''' + # These should fail + try: + exec 'from (sys) import stdin' + self.fail("parens not allowed around single dotted_name") + except SyntaxError: + pass + try: + exec 'import (os), (sys)' + self.fail("entire name list must be enclosed in parens") + except SyntaxError: + pass + try: + exec 'import ((os), (sys))' + self.fail("subparens not allowed") + except SyntaxError: + pass + try: + exec 'import sys)' + self.fail("mismatched parens") + except SyntaxError: + pass + try: + exec 'from sys import stdin)' + self.fail("mismatched parens") + except SyntaxError: + pass + try: + exec 'import (os,)' + self.fail("not a tuple") + except SyntaxError: + pass + try: + exec 'from __future__ import nested_scopes)' + self.fail("mismatched parens in __future__"); + except SyntaxError: + pass + def test_main(): test_support.run_unittest(TestSpecifics) Index: Lib/test/test_grammar.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_grammar.py,v retrieving revision 1.50 diff -u -c -r1.50 test_grammar.py *** Lib/test/test_grammar.py 19 May 2004 08:20:09 -0000 1.50 --- Lib/test/test_grammar.py 9 Aug 2004 04:01:42 -0000 *************** *** 417,428 **** try: raise KeyboardInterrupt except KeyboardInterrupt: pass ! print 'import_stmt' # 'import' NAME (',' NAME)* | 'from' NAME 'import' ('*' | NAME (',' NAME)*) import sys import time, sys from time import time from sys import * from sys import path, argv print 'global_stmt' # 'global' NAME (',' NAME)* def f(): --- 417,434 ---- try: raise KeyboardInterrupt except KeyboardInterrupt: pass ! print 'just_import' # 'import' ('(' dotted_as_names ')' | dotted_as_names) import sys + import (sys) import time, sys + import (time, sys) + print 'from_import' # 'from' dotted_name 'import' ('*' | '(' '*' ')' | '(' import_as_names ')' | import_as_names) from time import time + from time import (time) from sys import * + from sys import (*) from sys import path, argv + from sys import (path, argv) print 'global_stmt' # 'global' NAME (',' NAME)* def f(): Index: Lib/test/output/test_grammar =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/output/test_grammar,v retrieving revision 1.20 diff -u -c -r1.20 test_grammar *** Lib/test/output/test_grammar 21 May 2003 17:34:49 -0000 1.20 --- Lib/test/output/test_grammar 9 Aug 2004 04:01:42 -0000 *************** *** 35,41 **** testing continue and break in try/except in loop return_stmt raise_stmt ! import_stmt global_stmt exec_stmt assert_stmt --- 35,42 ---- testing continue and break in try/except in loop return_stmt raise_stmt ! just_import ! from_import global_stmt exec_stmt assert_stmt From stephen at xemacs.org Mon Aug 9 06:58:12 2004 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon Aug 9 06:58:19 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com> (Andrew Durdin's message of "Sat, 7 Aug 2004 22:06:36 +1000") References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <4114A171.3050809@v.loewis.de> <59e9fd3a04080705067633a790@mail.gmail.com> Message-ID: <87d620lxgb.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Andrew" == Andrew Durdin writes: Andrew> @ at least is consistently on the 2 key (except, of Andrew> course, for all those European keyboards. I guess their Andrew> differences are enough to knock down any argument about Andrew> typing convenience :-) The other exception is Japanese keyboards (unshifted to the immediate right of "P"). -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From mark at prothon.org Mon Aug 9 07:13:20 2004 From: mark at prothon.org (Mark Hahn) Date: Mon Aug 9 07:13:39 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> <20040809035138.GC21340@panix.com> Message-ID: <1qygyzxyh5qnx.1r2aymzvv6xlw$.dlg@40tude.net> On Sun, 8 Aug 2004 23:51:38 -0400, Aahz wrote: > On Sat, Aug 07, 2004, Mark Hahn wrote: > > > > Can anyone see any problem with this scheme? > > Yes: it doesn't follow the decimal floating point standard. > Read PEP327 for an exhaustive discussion of the subject WRT > Python (which is nowhere near as exhausting as reading the > discussion in the python-dev archives). I don't care too much about it being a standard but I do care about it sucking. Tim Peters filled me in on what's wrong with the MS type. I am more interested now in hearing about people's feelings on the idea of having the Decimal type "in-between" Int and Float. Assume a decent Decimal implementation is used. > Side note: I haven't bothered checking recently, but since > you were complaining about the quality of technical responses > on c.l.py, I've been mostly ignoring you because of your > habit of top-posting. FWIW, I learned how to post in the c.l.p style back in early April which which was four months ago. I'm sorry about my earlier postings. It was hard to overcome an emailing style I had used in the business world for 25 years. From adurdin at gmail.com Mon Aug 9 07:28:52 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Mon Aug 9 07:28:58 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <200408081525.i78FPJm20862@guido.python.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> <411610FB.9040201@v.loewis.de> <59e9fd3a04080807177dd446af@mail.gmail.com> <200408081525.i78FPJm20862@guido.python.org> Message-ID: <59e9fd3a04080822284be45e94@mail.gmail.com> On Sun, 08 Aug 2004 08:25:19 -0700, Guido van Rossum wrote: > > This an example of bad formatting. Tests often do that because their > purpose is to test odd corners of functionality, not serve as sample > code. Given that in typical example code there will always be a blank > line separating the decorator from anything previous (and separating > the body from what follows!) the connection between the decorator and > the def is pretty clear. In such a case, it would be clear that the author of the code considered the decorator and the def to be connected as regards to their function within the program; that the decorator can fundamentally alter the meaning of the def statement is not at all clear -- and it ought to be! By contrast, those proposed syntaxes which have the decorators in the def statement (somewhere between the def and the colon) do not suffer this opacity. > Also note that arguments based on partial knowledge of Python can be > used as an argument against pretty much anything (just tweak the > assumptions a bit), and hence have little value. In the general case, this is probably true; but this argument of mine is valid for any level of Python user familiar with versions up to 2.3, who has *only* not seen of or heard of the new decorator syntax. This is because the @decorator syntax has a hidden side-effect, when compared with normal Python statements. It is not at all consistent with *any* other valid Python syntax that I know. Consistency may not always be desirable; but I would have thought it generally agreed that introducing a severe inconsistency for the sake of only one feature is not a good design choice. If you believe that introducing 'new-style' statements/expressions that implicitly affect what follows them is a good idea *in general*, and is well-suited to many features other than decorator expressions, then by all means, introduce it first for decorators, and subsequently for these other features--but at least let us know *why* these 'new-style' statements/expressions are a good idea in general (not just for decorators). Because frankly, and with respect, I haven't seen any compelling arguments that introducing new statements/expressions with implicit, non-obvious effects is a good thing for Python. To me, such a thing reminds me of the horrible implicitly-set $_ and $? (and other such creatures) in Perl. Relevant quotes from the Zen of Python (not for argument but contemplation :-) "Explicit is better than implicit." "Special cases aren't special enough to break the rules." "Although practicality beats purity." "Now is better than never. Although never is often better than *right* now." I know you've said you hate the list-after-def syntax. I still hope that you can change you mind there, or at least reduce "hate" to "objective reasons for dislike" :-) The latter half of my previous post was an attempt at this by addressing your stated reasons for disliking it. Andrew From martin at v.loewis.de Mon Aug 9 08:14:22 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 9 08:14:22 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <41167146.4070401@stackless.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <41167146.4070401@stackless.com> Message-ID: <4117163E.7050203@v.loewis.de> Christian Tismer wrote: > What I added was yet another __stuff__ entry for new-style > classes. It may also appear on module level, like the > __metaclass__ feature works. While this "works", I still think it is a bad idea, for two reasons: 1. It only helps the parser, A human reader would have to scroll to the top of the class (or worse, to the top of the module) to find out. 2. For the author, it is redundant typing - the author *knows* this is a decorator, so having to type this again in full is redundant. 1 and 2 together make this a write-only syntax. 3. It means that the parser has to recognize __decorators__ This is not often the case; I believe the only other exception is __future__ (which I also dislike, despite its cuteness). [As any good list of 2 reasons, this list of course has 3] > But it provides a solution without enforcing us to decide > about new syntax right now, which appears to be too early > for a group that can't find a common denominator, yet. There is a solution already: put the function wrappers after the declaration. The goal of this change is specifically to come up with a syntax. If no syntax could be find, it would be best not to change anything. I'm not worried that the group cannot agree on syntax: people can *never* agree on syntax, not just in the case of Python. If you wait for agreement, you wait forever. > For my purposes it works fine, but I don't claim that it fulfils > all other desired use cases. At least it is compatible, simple, > complete, implemented in a way, and works for my applications. It's not any more compatbible than the 2.4a2 approach: - old code (which does not use the feature) continues to work correctly (*) - new code (which uses the feature) won't work on old implementations. (*) Actually, there is a slight incompatibility in your approach: Code that used to run and used to raise an exception would, under your change, now suddenly succeed without exception. Regards, Martin From lists at hlabs.spb.ru Mon Aug 9 12:20:32 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Mon Aug 9 08:14:41 2004 Subject: [Python-Dev] Density of pie-decorator syntax In-Reply-To: <4113AFC4.5010004@janrain.com> References: <4113AFC4.5010004@janrain.com> Message-ID: <41174FF0.8020103@hlabs.spb.ru> Josh Hoyt wrote: > One problem I have with the pie-decorator syntax is that it is very > typographically dense. The @ character runs in with the following > function name. If there are multiple lines of decorators, the problem is > worsened. For clarity, the decorators will have no whitespace lines > between them, creating a dense block. > > After experimenting, I have found that readability is increased if there > is one space required after the @. Don't take my word for it: > > Current Python 2.4a2 syntax: > > @framework_stuff(lots, of, args) > class Quux(object): > > @check_args(int, str) > @counted > @staticmethod > def frobnicate(foo, bar): > pass > > Proposed change requiring a space: > > @ framework_stuff(lots of args) > class Quux(object): > > @ check_args(int, str) > @ counted > @ staticmethod > def frobnicate(foo, bar): > pass "@ decorator" working just fine with Python 2.4.a2. Try this... :-) -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From martin at v.loewis.de Mon Aug 9 08:19:30 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 9 08:19:30 2004 Subject: [Python-Dev] Cygwin and sockets In-Reply-To: <20040808145442.GF55644@tishler.net> References: <4110AABA.80407@v.loewis.de> <20040808145442.GF55644@tishler.net> Message-ID: <41171772.2050602@v.loewis.de> Jason Tishler wrote: > See attach for the details. OK to apply? Yes, go ahead. I'd like to find a better way though to determine whether we need to include the emulated addrinfo.h > BTW, the socket module builds automatically for me. It might be that the bug reporter is mistaken in his claim that building _socked.pyd was skipped. More likely, it failed, but setup.py proceeded happily. Please close the bug report when you are done, indicating that it now works for you. Thanks for looking into this, Martin From martin at v.loewis.de Mon Aug 9 08:27:09 2004 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon Aug 9 08:27:09 2004 Subject: [Python-Dev] Decimal type question [Prothon] In-Reply-To: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: <4117193D.5030001@v.loewis.de> > Can anyone see any problem with this scheme? How does that interact with (binary) floating point numbers? I think the numerical people might dislike that approach, as all their algorithms, C interfaces, and so on are heavily tied to IEEE-754. Not being a NumPy user myself, I cannot say how significant that is, and what could be done about it. For example, would it be possible to automatically fall back to binary floating point if the result cannot be represented exactly (which would be most divide operations)? Would that help? Regards, Martin From sjoerd at acm.org Mon Aug 9 08:53:45 2004 From: sjoerd at acm.org (Sjoerd Mullender) Date: Mon Aug 9 08:53:58 2004 Subject: [Python-Dev] The date in the generated documentation In-Reply-To: <20040809024336.GC6582@unpythonic.net> References: <20040809022934.GB6582@unpythonic.net> <20040809024336.GC6582@unpythonic.net> Message-ID: <41171F79.4040601@acm.org> Jeff Epler wrote: > I had to edit the lines in the file /usr/share/latex2html/l2hconf.pm > which mentioned japanese (by default! I'm certain I never changed them) > and change them back to english. > > The generated html documentation is now much improved. > > Sorry to bother everyone for what turned out to be a fedora bug / > configuration problem. I see this also. I filed a bug with Fedora. -- Sjoerd Mullender -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 374 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/4c37abf8/signature.pgp From martin at v.loewis.de Mon Aug 9 09:08:50 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 9 09:08:50 2004 Subject: [Python-Dev] PyRun_ with file name In-Reply-To: <1f7befae040808174233fb20db@mail.gmail.com> References: <4113EF69.2020102@v.loewis.de> <1f7befae040808174233fb20db@mail.gmail.com> Message-ID: <41172302.6090001@v.loewis.de> Tim Peters wrote: > If you understand the problem, and believe it will fix it, +1 (I don't > think I fully understand the problem, but it gets points from me for > being a small change). To fully understand it, you need to spend three days in a locked room together with the VC debugger :-) The submitter claims that VC7 has changed the CRT ABI, I believe this is incorrect. Instead, the problem results from having to CRT copies simultaneously, even though they all have the very same code sice ages. Here is the full story: - there are two file structures, struct FILE and struct _FILEX, see internal.h. _FILEX is typedef struct { FILE f; CRITICAL_SECTION lock; } _FILEX; - For historical reasons, the lock couldn't be added to the existing FILE objects (stdin, stdout, etc) since compiled code *knew* how large struct FILE is, and where therefore _iob[2] (say) is. - To solve this, MS added a second array of CRITICAL_SECTION (see mlock.c), and a helper function _lock to lock one of these. - Now, _lock_file becomes (see _file.c) void __cdecl _lock_file (void *pf) { if((pf >= (void *)_iob) && (pf <= (void *)(&_iob[_IOB_ENTRIES-1]))) _lock( _STREAM_LOCKS + (int)((FILE *)pf - _iob) ); else EnterCriticalSection( &(((_FILEX *)pf)->lock) ); } So essentially, if it is a builtin FILE, find its lock elsewhere, otherwise, it is a _FILEX, and find its lock inside the FILE itself. - Suppose we have an embedded Python, and the container is compiled with VC7 (or 5, doesn't matter). The host application does FILE *f = fopen("some file", "r"); PyRun_SimpleFile(f); This allocates a builtin FILE (say, fileno 5) inside the VC7 CRT, and passes it to python23.dll. We do fread, and fread (from the VC6 CRT) does _lock_file. This checks whether this is a builtin FILE, which it isn't, so the VC6 CRT assumes it is a FILEX, and uses the CRTITICAL_SECTION. Of course, this is really the storage of fileno 6 of the VC7 CRT, which is not an initialized CRTICAL_SECTION. The end result is that the _lock_file operation crashes. As MSDN explains, you must not pass FILE* across CRTs. The proposed patch avoids the problem by passing file names instead. Regards, Martin From stephen at xemacs.org Mon Aug 9 09:19:31 2004 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon Aug 9 09:19:39 2004 Subject: [Python-Dev] pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching In-Reply-To: (Mike Coleman's message of "07 Aug 2004 01:08:08 -0400") References: Message-ID: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Mike" == Mike Coleman writes: Mike> Motivation Mike> ========== Mike> A notable limitation of the ``re.match`` method is that it Mike> fails to capture all group match information for repeatedly Mike> matched groups. For example, in a call like this :: Mike> m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz') Mike> one would like to see that the group which matched four Mike> times matched the strings ``'X'``, ``'xxx'``, ``'Y'`` and Mike> ``'zz'``. Sure, but regexp syntax is a horrible way to express that. This feature would be an attractive nuisance, IMHO. For example: Mike> Parsing ``/etc/group`` Mike> ---------------------- Mike> If ``/etc/group`` contains the following lines :: Mike> root:x:0: Mike> daemon:x:1: Mike> bin:x:2: Mike> sys:x:3: Mike> then it can be parsed as follows :: Mike> p = r'((?:(?:^|:)([^:\n]*))*\n)*\Z' This is a _easy_ one, but even it absolutely requires being written with (?xm) and lots of comments, don't you think? If you're going to be writing a multiline, verbose regular expression, why not write a grammar instead, which (assuming a modicum of library support) will be shorter and self-documenting? -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From lists at hlabs.spb.ru Mon Aug 9 13:52:54 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Mon Aug 9 09:47:02 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint Message-ID: <41176596.5060209@hlabs.spb.ru> Is there a plan for implementing a base class for int and long (like basestring for str and unicode): >>> issubclass(int, baseint) and issubclass(long, baseint) True ? -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From tismer at stackless.com Mon Aug 9 11:04:05 2004 From: tismer at stackless.com (Christian Tismer) Date: Mon Aug 9 11:04:02 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <4117163E.7050203@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <41167146.4070401@stackless.com> <4117163E.7050203@v.loewis.de> Message-ID: <41173E05.40003@stackless.com> Martin v. L?wis wrote: > Christian Tismer wrote: > >> What I added was yet another __stuff__ entry for new-style >> classes. It may also appear on module level, like the >> __metaclass__ feature works. > > > While this "works", I still think it is a bad idea, for two > reasons: Just as a remark: You are shifting topic, now. We were not discussign what's nice or good or not, it was about a correct proposal, which I supplied. I was not intending to discuss if this is a good approach. > 1. It only helps the parser, A human reader would have to > scroll to the top of the class (or worse, to the top of > the module) to find out. compare __slots__, compare __metaclass__, and I mentioned already that these are intermediate construct, too. > 2. For the author, it is redundant typing - the author *knows* > this is a decorator, so having to type this again in full > is redundant. > 1 and 2 together make this a write-only syntax. This argument is a little overdone. To repeat the function name just once isn't that bad. > 3. It means that the parser has to recognize __decorators__ > This is not often the case; I believe the only other > exception is __future__ (which I also dislike, despite > its cuteness). > [As any good list of 2 reasons, this list of course has 3] The parser doesn't have to recognize __decorators__. See the implementation of __metaclass__. This is a lookup of a name, like it happens already. But it is true, the parser then has to generate different code for the members of the __decorators__ sequence. >> But it provides a solution without enforcing us to decide >> about new syntax right now, which appears to be too early >> for a group that can't find a common denominator, yet. > > There is a solution already: put the function wrappers after > the declaration. The goal of this change is specifically to > come up with a syntax. If no syntax could be find, it would > be best not to change anything. (**) That's of course a very good argument. > I'm not worried that the group cannot agree on syntax: > people can *never* agree on syntax, not just in the case > of Python. If you wait for agreement, you wait forever. Ok. My real fear is that we get to a solution which will spoil Python's appearance in an unrevertable way, just because the relevant people are getting bored by discussion and want to stop it by supplying a premature solution. ... > It's not any more compatbible than the 2.4a2 approach: > - old code (which does not use the feature) continues to work > correctly (*) > - new code (which uses the feature) won't work on old > implementations. > (*) Actually, there is a slight incompatibility in your approach: > Code that used to run and used to raise an exception would, under > your change, now suddenly succeed without exception. I don't see it as a requirement that code that ran before, continues to run after a new construct is introduced. By adding __decorators__ to your class, you decide to use this feature, you also change its implementation and put the decorators where you like them. Relying on exceptions which happened before the change is nothing that I want to be concerned with. Changing from old-style to new-style classes is also an operation that will cause small semantic differences, but I never heared someone arguing against this. I don't think you need such far-fetched argumentation to kill this proposal. Stick with (**), it is convincing enough. :-) ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Mon Aug 9 11:16:52 2004 From: tismer at stackless.com (Christian Tismer) Date: Mon Aug 9 11:21:49 2004 Subject: [Python-Dev] capturing RETURN_VALUE In-Reply-To: <1f7befae04080816325f3fa2f2@mail.gmail.com> References: <41165D23.4070202@stackless.com> <1f7befae040808160924e4d8a0@mail.gmail.com> <4116B4BF.5030409@stackless.com> <1f7befae04080816325f3fa2f2@mail.gmail.com> Message-ID: <41174104.2060606@stackless.com> Tim Peters wrote: ... > The point was that exceptions and returns act the same way: > RETURN_VALUE isn't unique here, it's just another instance of leaving > a try-block's suite, the same in this respect as an exception or a > break statement. That's why it's reliable over time: it's a general > mechanism at work, not an arbitrary hack specific to RETURN_VALUE. Yes, that was special for me: That return is _not_ special. I should have known because I know the implementation so very well, but I always felt that return is something final. Maybe it would make sense to add a few sentences about this fact somewhere (where?) in the docs; It was quite remarkable for me. Actually, there is nothing final in Python: It is not possible to deduce from an inner code block whether it will be able to finish the enclosing function, unless you know that your code block consumes 20 block levels (and the latter is probably a much less reliable fact. :-) return-finally-break - ly y'rs -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From walter at livinglogic.de Mon Aug 9 12:38:58 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Mon Aug 9 12:39:04 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <41141255.5010408@stackless.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> Message-ID: <41175442.4010902@livinglogic.de> Christian Tismer wrote: > Anthony Baxter wrote: > >> Mark Russell wrote: >> >>> I too like |. It is not quite as visible as @, but I don't think that's >>> a serious problem, and not breaking ipython and other tools is a big >>> win. I just tried implementing this change, and all the tests still >>> pass (not a big surprise, but worth confirming I guess). >> >> To jump on the band-wagon - I can live with | as well. I was going to >> add it to the PEP rewrite, but I'll wait for a decision, to save myself >> the writing ;) > > Ok, I dislike special prefix chars at all, in a language that > doesn't have this concept elsewhere (despite strings of course, > but their prefixes are just regular chars), appart from * and ** for argument passing. I can't understand why we can't have a new keyword for decorators. If I remember correctly the introduction of yield didn't result in such a public outcry. We'd have to change our programs once if a variable names collides with the new keyword, but that's better than having to look at @s for the rest of our Python days. So how about: make classmethod def foo(cls, bar): ... > but the bar "|" appears a lot nicer than "@" to me. > The bar is small and friendly, a small character for a small > feature. With "@", I associate a powerful, magical Voodoo > thing, something that I never expected to see in Python. > Unless there is a real need, I'd say save "@" for a feature that > really deserves such a powerful character. Bye, Walter D?rwald From mwh at python.net Mon Aug 9 12:45:10 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 9 12:45:12 2004 Subject: [Python-Dev] new failing test -- test_compiler In-Reply-To: <1f7befae0408080946767c0a1c@mail.gmail.com> (Tim Peters's message of "Sun, 8 Aug 2004 12:46:09 -0400") References: <1f7befae0408071253544d2c31@mail.gmail.com> <1f7befae04080712597b9e7e0b@mail.gmail.com> <2misbu3cbp.fsf@starship.python.net> <20040807222357.GA27723@vicky.ecs.soton.ac.uk> <1f7befae04080717357814d502@mail.gmail.com> <1f7befae0408080946767c0a1c@mail.gmail.com> Message-ID: <2mbrhk3809.fsf@starship.python.net> Tim Peters writes: > [Tim] >>> Alas, test_compiler still fails (but in an ordinary "doesn't work" >>> way, no longer in an internal-error way), so testing with -uall is >>> still a bit broken. > > [Jeremy] >> I won't be able to fix it today, but I'll get on that asap. > > That's OK. I agree it's valuable to run this, so I fixed it. > test_compiler passes on Windows now. But I have a gigabyte of RAM on > this box to hold the stack . I actually worked out the fix on Saturday but succumbed to hunger before the test run finished :-) Cheers, mwh -- I never realized it before, but having looked that over I'm certain I'd rather have my eyes burned out by zombies with flaming dung sticks than work on a conscientious Unicode regex engine. -- Tim Peters, 3 Dec 1998 From mwh at python.net Mon Aug 9 13:05:04 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 9 13:05:06 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <41176596.5060209@hlabs.spb.ru> (Dmitry Vasiliev's message of "Mon, 09 Aug 2004 11:52:54 +0000") References: <41176596.5060209@hlabs.spb.ru> Message-ID: <2m7js83733.fsf@starship.python.net> Dmitry Vasiliev writes: > Is there a plan for implementing a base class for int and long (like > basestring for str and unicode): > > >>> issubclass(int, baseint) and issubclass(long, baseint) > True Not that I'm aware of. Personally, I think it's slightly more likely that we'll have: >>> int is long True but this is a long way off, and could cause upset with people who need to do bit bashing or interface with C. Of course, >>> baseint = (int, long) makes what you type accurate today :-) Cheers, mwh -- > so python will fork if activestate starts polluting it? I find it more relevant to speculate on whether Python would fork if the merpeople start invading our cities riding on the backs of giant king crabs. -- Brian Quinlan, comp.lang.python From lists at hlabs.spb.ru Mon Aug 9 17:38:18 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Mon Aug 9 13:32:26 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <2m7js83733.fsf@starship.python.net> References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> Message-ID: <41179A6A.90905@hlabs.spb.ru> Michael Hudson wrote: > Dmitry Vasiliev writes: > >>Is there a plan for implementing a base class for int and long (like >>basestring for str and unicode): >> >> >>> issubclass(int, baseint) and issubclass(long, baseint) >>True > > Not that I'm aware of. Personally, I think it's slightly more likely > that we'll have: > > >>>>int is long > > True > > but this is a long way off, and could cause upset with people who need > to do bit bashing or interface with C. Quote from PEP-237: """ A new type, integer, may be introduced that is an abstract base type of which both the int and long implementation types are subclassed. This is useful so that programs can check integer-ness with a single test: if isinstance(i, integer): ... """ So maybe correct question then: is there a plan for implementing the integer type in Python 2.4? > > Of course, > >>>>baseint = (int, long) > > makes what you type accurate today :-) -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From anthony at interlink.com.au Mon Aug 9 14:58:26 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 9 14:59:17 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <41179A6A.90905@hlabs.spb.ru> References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> <41179A6A.90905@hlabs.spb.ru> Message-ID: <411774F2.7040408@interlink.com.au> Dmitry Vasiliev wrote: > > Quote from PEP-237: > > """ > A new type, > integer, may be introduced that is an abstract base type of > which both the int and long implementation types are > subclassed. This is useful so that programs can check > integer-ness with a single test: > > if isinstance(i, integer): ... > """ > > So maybe correct question then: is there a plan for implementing the > integer type in Python 2.4? Well, we're a couple of weeks from what will hopefully be the last alpha. I'd _hope_ anything like this would be in before then (although we can have things that change behaviour up until b1, I'd prefer that a3->b1 is just fixing bugs that require changes, or minor changes, not major changes). OTOH, I'm not sure how deep a change like this would be. In any case, if it's a change that you (or anyone else) feels strongly about, a patch on SF would be a first step towards making this happen. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From s.percivall at chello.se Mon Aug 9 15:02:42 2004 From: s.percivall at chello.se (Simon Percivall) Date: Mon Aug 9 15:02:44 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <41175442.4010902@livinglogic.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> Message-ID: <6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se> On 2004-08-09, at 12.38, Walter D?rwald wrote: > I can't understand why we can't have a new keyword for decorators. > If I remember correctly the introduction of yield didn't result > in such a public outcry. We'd have to change our programs once > if a variable names collides with the new keyword, but that's better > than having to look at @s for the rest of our Python days. > > So how about: > > make classmethod > def foo(cls, bar): > ... I think "adorn" would make a better keyword in that case, as in "adorn with" this decorator the method foo. //Simon From bob at redivi.com Mon Aug 9 15:33:06 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 9 15:33:16 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> <6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se> Message-ID: On Aug 9, 2004, at 9:02 AM, Simon Percivall wrote: > On 2004-08-09, at 12.38, Walter D?rwald wrote: >> I can't understand why we can't have a new keyword for decorators. >> If I remember correctly the introduction of yield didn't result >> in such a public outcry. We'd have to change our programs once >> if a variable names collides with the new keyword, but that's better >> than having to look at @s for the rest of our Python days. >> >> So how about: >> >> make classmethod >> def foo(cls, bar): >> ... > > I think "adorn" would make a better keyword in that case, as in > "adorn with" this decorator the method foo. I don't think that's appropriate at all. Most decorators are transformations that change the function object into something else entirely, most likely by wrapping it but quite possibly not. I think the idea of adding a new keyword has already been dismissed, though. -bob From dave at boost-consulting.com Mon Aug 9 15:34:58 2004 From: dave at boost-consulting.com (David Abrahams) Date: Mon Aug 9 15:35:34 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <200408071617.i77GHgs10281@guido.python.org> <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> Message-ID: Bill Janssen writes: >> In the discussion on decorators months ago, solutions involving >> special syntax inside the block were ruled out early on. Basically, >> you shouldn't have to peek inside the block to find out important >> external properties of the function. > > Guido, could you expand on this principle a bit? Just stated as it > is, it sounds as pointless (to my untutored mind, at least :-), as the > arguments about "@ is ugly, so it shouldn't be used". After all, > isn't "what the function does" an important external property of the > function, and don't you have to peek inside the block to determine > that? For instance, how do you know whether a function is a generator > unless you look for yield statements inside it? I'm not Guido (obviously), but: You look at the documentation. The fact that Python doesn't divorce interface declarations from implementation is convenient for authors, but in some ways a trap for readers/maintainers (this is not a design criticism -- it was probably the right design choice). I often have to look inside Python functions to understand them, and find that their authors expect me to do so. I think a style that cleanly delimits interface and implementation as much as possible should be encouraged. The fact that docstrings sit inside the function block make that harder to achieve; that's why I'm glad to hear that prefix decorators are being considered for docstrings. That said, I'm not sure that "immediately following the opening colon" is such a bad place for interface-y things. It's nice that you can find the whole function's declaration by looking forward from "def ..." today [except for the decorators, of course ;-(]. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From dave at boost-consulting.com Mon Aug 9 15:42:31 2004 From: dave at boost-consulting.com (David Abrahams) Date: Mon Aug 9 15:50:28 2004 Subject: [Python-Dev] Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> <411610FB.9040201@v.loewis.de> <59e9fd3a04080807177dd446af@mail.gmail.com> <200408081525.i78FPJm20862@guido.python.org> <59e9fd3a04080822284be45e94@mail.gmail.com> Message-ID: Andrew Durdin writes: > In the general case, this is probably true; but this argument of mine > is valid for any level of Python user familiar with versions up to > 2.3, who has *only* not seen of or heard of the new decorator syntax. > This is because the @decorator syntax has a hidden side-effect, when > compared with normal Python statements. It is not at all consistent > with *any* other valid Python syntax that I know. > > Consistency may not always be desirable; but I would have thought it > generally agreed that introducing a severe inconsistency for the sake > of only one feature is not a good design choice. If you believe that > introducing 'new-style' statements/expressions that implicitly affect > what follows them is a good idea *in general*, and is well-suited to > many features other than decorator expressions, then by all means, > introduce it first for decorators, and subsequently for these other > features--but at least let us know *why* these 'new-style' > statements/expressions are a good idea in general (not just for > decorators). Because frankly, and with respect, I haven't seen any > compelling arguments that introducing new statements/expressions with > implicit, non-obvious effects is a good thing for Python. To me, such > a thing reminds me of the horrible implicitly-set $_ and $? (and other > such creatures) in Perl. Seeing as how we're now firmly back in the land of opinions, voting, and trying to influence the BDFL, I'd like to add: I agree with Andrew. Whether or not the implementation uses sys.settrace internally, it still feels like it has the same weird, stateful semantics of temporarily affecting the next executable statement and then evaporating. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From FBatista at uniFON.com.ar Mon Aug 9 15:48:50 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Mon Aug 9 15:52:56 2004 Subject: [Python-Dev] decimal.Context.copy() shallow or deep? Message-ID: [Raymond Hettinger] #- > > Do you guys think it should be left alone (shallow) or #- changed (made #- > > deep)? #- > #- > A user-visible copy certainly should not share mutable #- state with the #- > original. #- #- I was afraid of that. What a PITA. My opinion is that the important step here is to fix the behaviour before alpha-3 or beta (whatever cames first). So, it's nice your idea to rename context.copy() to context._shallow_copy(), make context.copy() deep, and then, in all the tests that fail, change copy for the shallow one. Later, we'll have time to maybe fix that and don't depend of a shallow copy (work around that WorkRep would be nice also). . Facundo From bkc at murkworks.com Mon Aug 9 16:27:05 2004 From: bkc at murkworks.com (Brad Clements) Date: Mon Aug 9 16:20:58 2004 Subject: [Python-Dev] PyRun_ with file name In-Reply-To: <41172302.6090001@v.loewis.de> References: <1f7befae040808174233fb20db@mail.gmail.com> Message-ID: <41175178.1484.ED32EF69@coal.murkworks.com> On 9 Aug 2004 at 9:08, Martin v. L?wis wrote: > The end result is that the _lock_file operation crashes. As MSDN > explains, you must not pass FILE* across CRTs. The proposed patch avoids > the problem by passing file names instead. I vaugely recall having a similar problem embedding Python perhaps two years ago. At that time I had contemplated adding a function to python.dll along the lines of FILE * Py_OpenFile(char *filename, char *mode) The sole purpose of this function was to allow embedding applications to use PyRun_SimpleFile and similar python.dll functions that expect a (crt) FILE object. I'm sorry I can't remember the exact use case. I'm not proposing Py_OpenFile as a solution to this problem. I just wanted to speak up and say "me too". +1 -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements From jason at tishler.net Mon Aug 9 16:52:42 2004 From: jason at tishler.net (Jason Tishler) Date: Mon Aug 9 16:48:46 2004 Subject: [Python-Dev] Cygwin and sockets In-Reply-To: <41171772.2050602@v.loewis.de> References: <4110AABA.80407@v.loewis.de> <20040808145442.GF55644@tishler.net> <41171772.2050602@v.loewis.de> Message-ID: <20040809145241.GA67536@tishler.net> Martin, On Mon, Aug 09, 2004 at 08:19:30AM +0200, "Martin v. L?wis" wrote: > Jason Tishler wrote: > >See attach for the details. OK to apply? > > Yes, go ahead. Done. > I'd like to find a better way though to determine whether we need to > include the emulated addrinfo.h Agreed, which was I checked before committing my simplistic patch. > >BTW, the socket module builds automatically for me. > > It might be that the bug reporter is mistaken in his claim that > building _socked.pyd was skipped. More likely, it failed, but setup.py > proceeded happily. I thought the above was a likely possibility too. > Please close the bug report when you are done, indicating that it now > works for you. Done. > Thanks for looking into this, You are quite welcome -- thanks for the heads up. BTW, I discovered that the following modules fail to build under Cygwin too: strop cPickle readline I will submit a patch to fix these problems shortly. Jason -- PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6 From guido at python.org Mon Aug 9 17:04:47 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 17:06:01 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: Your message of "Sun, 08 Aug 2004 18:31:49 PDT." <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> Message-ID: <200408091504.i79F4lD22945@guido.python.org> [Guido] > > In the discussion on decorators months ago, solutions involving > > special syntax inside the block were ruled out early on. Basically, > > you shouldn't have to peek inside the block to find out important > > external properties of the function. [Bill Janssen] > Guido, could you expand on this principle a bit? Just stated as it > is, it sounds as pointless (to my untutored mind, at least :-), as the > arguments about "@ is ugly, so it shouldn't be used". After all, > isn't "what the function does" an important external property of the > function, and don't you have to peek inside the block to determine > that? For instance, how do you know whether a function is a generator > unless you look for yield statements inside it? It's probably untenable as a formal principle. :-) Nevertheless I don't like to put decorators in the method body. I think it has something to do with the fact that normally one thinks of the body contents as relevalt to "call-time" while decorators happen at "definition-time". And before you bring up docstrings as a counterexample, yes, they should ideally have been designed to precede the method, too. --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Mon Aug 9 05:03:40 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 9 17:06:45 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: <1qygyzxyh5qnx.1r2aymzvv6xlw$.dlg@40tude.net> Message-ID: <002601c47dbd$79161dc0$191fc797@oemcomputer> > I am more interested now in hearing about people's feelings on the idea of > having the Decimal type "in-between" Int and Float. Assume a decent > Decimal implementation is used. FWIW, Lua used floating point for everything. With integers upto 53 bits being represented exactly, everything works out fine. Plus is has a nice side effect of simplifying the language and underlying code. Raymond From guido at python.org Mon Aug 9 17:06:49 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 17:07:07 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: Your message of "Mon, 09 Aug 2004 14:16:45 +1200." <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408091506.i79F6nc22962@guido.python.org> > If @decorator stays in, I fervently hope it won't be taken as a > precedent for using @ for other things all over the place. > Otherwise we really *will* end up with a language that resembles > Perl. That's dangerously close to rhetorical nonsense. @ will always be used to indicate meta-information (roughly speaking), while in Perl it indicates a (certain kind of) variable. Very different. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Mon Aug 9 17:11:45 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 9 17:10:40 2004 Subject: [Python-Dev] Re: Multi-line import implementation (was: 2.4a2, and @decorators) References: <410DDFE3.7010006@interlink.com.au><200408021850.i72IoCg28650@guido.python.org><410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org> Message-ID: Dima Dorfman wrote: > I took a stab at implementing this (the easy part of PEP 328--parens > around import lists). Are the exact semantics of what's allowed > documented somewhere? I think this syntax was originally proposed here: http://mail.python.org/pipermail/python-dev/2003-December/041072.html which mentions that since "a, b = c" is already equivalent to "(a, b) = c", so " in mind, I'd hand out the following scores (0=don't really care): > import (os, sys) +0 > from sys import (stdin, stdout, stderr) +1 > import (os) +0 > from sys import (*) -0 > from (sys) import stdin -0 > import (os), (sys) -1 > import (os,) -1 From jhylton at gmail.com Mon Aug 9 17:13:41 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Mon Aug 9 17:13:46 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu> References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: On Sat, 07 Aug 2004 00:35:17 -0400, Edward Loper wrote: > The new "vertical bar" syntax was proposed recently, and has gotten > almost unanimously positive feedback. In particular, of 17 responses > to the syntax, only one has been negative (see table, below). I dislike it. It's not as friendly or visible as the @. Jeremy From guido at python.org Mon Aug 9 17:16:38 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 17:16:42 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: Your message of "Mon, 09 Aug 2004 12:37:05 +1000." <019301c47db9$c269ec80$a9498890@neil> References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> <019301c47db9$c269ec80$a9498890@neil> Message-ID: <200408091516.i79FGc722993@guido.python.org> > A motivation for this was that one of the examples > > @accepts(int,int) > @returns(float) > def bar(low,high): > > which separates related information too much for me. If > decorators need to decorate parameters, they are more likely to be > understood correctly by a reader if the decorations are next to the > parameters. Let's make one thing clear; while decorators may be useful for some to experiment with attaching signatures to methods, mainstream Python will eventually have optional static typing built in, and it won't use decorator syntax. (More likely you would specify argument types by saying e.g. "def bar(low: int, high: int) -> float: ...") --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 9 17:25:21 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 17:25:26 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: Your message of "Mon, 09 Aug 2004 04:31:05 -0000." <20040809043104.GA82909@trit.org> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> <410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org> Message-ID: <200408091525.i79FPLM23023@guido.python.org> > > The 2.4 release schedule PEP now lists this as lacking someone > > to drive it. I won't/can't spend time on it. So, if one of the people > > who wants it could step forward, that'd be excellent. Even if it's > > just to do the multi-line import syntax of > > > > from foo import ( bar, baz, bar2, > > baz2, bar3, baz3 ) > > I took a stab at implementing this (the easy part of PEP 328--parens > around import lists). Great! > Are the exact semantics of what's allowed documented somewhere? Not unless it's in the PEP -- it usually takes a stab at implementing something to find what's left unspecified! > PEP 328 mostly talks about relative and absolute > imports, and it doesn't specify the exact semantics of where > parentheses should be allowed. My patch (attached) accepts > > import (os, sys) > from sys import (stdin, stdout, stderr) > import (os) > from sys import (*) > > but rejects > > from (sys) import stdin > import (os), (sys) > import (os,) > > Should any of those be allowed? Anything that I missed? This suggests (given the recent criticism of how we've been following the PEP process) that the PEP needs to be updated first. My own gut feeling is that it's not important to allow parentheses on "bare" (from-less) imports, since you can easily repeat the 'import' keyword on a new line; but that it is important on the from...import form, and then only after the "import" keyword. So that would accept your 2nd and 4th example but reject the 1st and 3rd. I guess one can argue about (*); I don't care about it and suggest that it be dropped. > The patch is incomplete in other ways; the docs haven't been updated > and neither have the parser module and compile package. If it's > decided that it would be okay to include this separately from > relative/absolute import support (they're different features, > really), I'll complete the patch. Please do, and please use the SF patch manager to upload the patch. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 9 17:41:11 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 17:41:18 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: Your message of "Mon, 09 Aug 2004 08:14:22 +0200." <4117163E.7050203@v.loewis.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <41167146.4070401@stackless.com> <4117163E.7050203@v.loewis.de> Message-ID: <200408091541.i79FfBl23117@guido.python.org> > [As any good list of 2 reasons, this list of course has 3] Cute. Perhaps there are only two good reasons amongst those three? :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 9 17:39:45 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 17:42:25 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: Your message of "Mon, 09 Aug 2004 15:28:52 +1000." <59e9fd3a04080822284be45e94@mail.gmail.com> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <200408061557.i76FvGo07481@guido.python.org> <4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de> <4115FF35.4030608@v.loewis.de> <411610FB.9040201@v.loewis.de> <59e9fd3a04080807177dd446af@mail.gmail.com> <200408081525.i78FPJm20862@guido.python.org> <59e9fd3a04080822284be45e94@mail.gmail.com> Message-ID: <200408091539.i79Fdjr23097@guido.python.org> > that the decorator can fundamentally alter the meaning of the def > statement is not at all clear -- and it ought to be! By contrast, > those proposed syntaxes which have the decorators in the def > statement (somewhere between the def and the colon) do not suffer > this opacity. I think that's too strict a rule. You're asking for something to be intuitively clear from looking at the syntax alone, without having learned it before. All I'm asking for is that it's easy to remember once explained. Very few things beyond 2+2 are intuitively clear, even in Python; almost everything is easy to explain and remember though, and I think that @decorators fall in that category, too. (And of course, to anyone who's programmed in Java 1.5 before, @decorators *will* be intuitively clear. But that's not the main motivation.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 9 17:47:10 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 17:47:17 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Mon, 09 Aug 2004 11:52:54 -0000." <41176596.5060209@hlabs.spb.ru> References: <41176596.5060209@hlabs.spb.ru> Message-ID: <200408091547.i79FlAe23178@guido.python.org> > Is there a plan for implementing a base class for int and long (like > basestring for str and unicode): > > >>> issubclass(int, baseint) and issubclass(long, baseint) > True > > ? I think this would be a good idea; maybe the name should be baseinteger? I think it's been suggested before, and can't remember why it wasn't implemented -- perhaps the use cases aren't as compelling as for basestring? --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Mon Aug 9 17:55:59 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 9 17:54:21 2004 Subject: [Python-Dev] Re: Multi-line import implementation (was: 2.4a2, and @decorators) References: <410DDFE3.7010006@interlink.com.au><200408021850.i72IoCg28650@guido.python.org><410EF963.40704@interlink.com.au><20040809043104.GA82909@trit.org> Message-ID: I wrote: > which mentions that since "a, b = c" is already equivalent to "(a, b) = c", so > " > in mind, I'd hand out the following scores (0=don't really care): JZ took control of my keyboard. sorry for that. the text above should have been: which mentions that since "a, b = c" is already equivalent to "(a, b) = c", so "from x import a, b" might as well made be equivalent to "from x import (a, b)". with that in mind, I'd hand out the following scores (0=don't really care) From fredrik at pythonware.com Mon Aug 9 18:02:28 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 9 18:00:53 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> <019301c47db9$c269ec80$a9498890@neil> <200408091516.i79FGc722993@guido.python.org> Message-ID: Guido van Rossum wrote: > Let's make one thing clear; while decorators may be useful for some to > experiment with attaching signatures to methods, mainstream Python > will eventually have optional static typing built in, and it won't use > decorator syntax. (More likely you would specify argument types by > saying e.g. "def bar(low: int, high: int) -> float: ...") that could, of course, be seen as rather strong argument against having decorators after the def but before the colon; let's reserve that space for information about the return type (or types). (more about decorators later) From tim.peters at gmail.com Mon Aug 9 18:05:01 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 18:05:13 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: <002601c47dbd$79161dc0$191fc797@oemcomputer> References: <002601c47dbd$79161dc0$191fc797@oemcomputer> Message-ID: <1f7befae04080909057a876605@mail.gmail.com> [Raymond Hettinger] > FWIW, Lua used floating point for everything. With integers upto 53 > bits being represented exactly, everything works out fine. Plus is has > a nice side effect of simplifying the language and underlying code. Perl also does, although it later added an actual integer type too. It doesn't help with 0.1 not being equal to one-tenth, etc. From tim.peters at gmail.com Mon Aug 9 18:23:51 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 18:23:57 2004 Subject: [Python-Dev] Decimal type question [Prothon] In-Reply-To: <4117193D.5030001@v.loewis.de> References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> <4117193D.5030001@v.loewis.de> Message-ID: <1f7befae04080909231d5b5bc@mail.gmail.com> [Martin v. L?wis] > ... > For example, would it be possible to automatically fall back > to binary floating point if the result cannot be represented > exactly (which would be most divide operations)? Would that > help? It's a puzzle. .NET Decimal is really more a fixed-point type than a floating-point type. It consists of a sign bit, a 96-bit binary integer, and a "scale factor" in 0..28, which is the power of 10 by which the integer is conceptually divided. The largest positive representable value is thus 2**96 == 79228162514264337593543950336. The smallest positive non-zero representable value is 1/10**28. So for something like 1/3, you get about 28 decimal digits of good result, which is much better than you can get with an IEEE double. OTOH, something like 1/300000000000000000000 starts to make the "gradual underflow" nature of Decimal apparent: for numbers with absolute value less than 1, the number of digits you get decreases the smaller the absolute value, until at 1e-28 you have only 1 bit of precision (and, e.g., 1.49999e-28 "rounds to" 1e-28). So it's a weird arithmetic as you approach its limits. But binary FP is too, and so is IBM's decimal spec. A primary difference is that binary FP has a much larger dynamic range, so you don't get near the limits nearly as often; and IBM's decimal has a gigantic dynamic range (the expectation is that essentially no real app will get anywhere near its limits, unless the app is buggy). From foom at fuhm.net Mon Aug 9 18:38:35 2004 From: foom at fuhm.net (James Y Knight) Date: Mon Aug 9 18:38:37 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> Message-ID: <8EBFDCAD-EA22-11D8-8D12-000A95A50FB2@fuhm.net> On Aug 8, 2004, at 9:31 PM, Bill Janssen wrote: > Guido, could you expand on this principle a bit? Just stated as it > is, it sounds as pointless (to my untutored mind, at least :-), as the > arguments about "@ is ugly, so it shouldn't be used". After all, > isn't "what the function does" an important external property of the > function, and don't you have to peek inside the block to determine > that? For instance, how do you know whether a function is a generator > unless you look for yield statements inside it? A function being a generator only* affects the return value, so not knowing what functions are generator functions are really just a special case of not knowing the return value of any function. That can be solved with documentation, or a decorator. e.g. @returns(types.GeneratorType) James *: well not really, but to the external caller it's equivalent. From janssen at parc.com Mon Aug 9 18:42:24 2004 From: janssen at parc.com (Bill Janssen) Date: Mon Aug 9 18:42:55 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: Your message of "Mon, 09 Aug 2004 08:04:47 PDT." <200408091504.i79F4lD22945@guido.python.org> Message-ID: <04Aug9.094230pdt."58612"@synergy1.parc.xerox.com> GvR writes: > Nevertheless I don't like to put decorators in the method body. I > think it has something to do with the fact that normally one thinks of > the body contents as relevalt to "call-time" while decorators happen > at "definition-time". > > And before you bring up docstrings as a counterexample, yes, they > should ideally have been designed to precede the method, too. Perhaps another way to think about this is that there are statements about properties of the function, method, or class which should precede the implementation of the whatever-it-is? That makes sense to me. But I'm less sure about the relative placement of the name of the whatever versus statements about the properties of the whatever. For example, should the argument list also occur before the name of the function? In a perfect world, would Python have the syntax: def (param1, param2) funcname: ... or perhaps: @args(param1, param2) def funcname: ... In thinking about this, I find the argument made a day or two ago about "cognitive binding" very powerful. That is, you should state the name of the thing you are defining before you start stating properties of it, so that the reader has the opportunity to mentally construct a tagged "bucket" to put those properties in. This is probably a useful thing for non-human tools, as well. Perhaps that's one reason that arguments to a function almost universally occur after the mention of the function name, in most programming languages: def foo (arg1, arg2, ...) So, by this principle, docstrings occur in the "right place", after the function name but before the implementation. Perhaps this is why docstrings are so successful in Python. Extending the principle to decorators would mean that they should occur somewhere between the function name and the function implementation. All of the C, D, and E variants in the Wiki seem to support this. Perhaps one of them is the right choice? Bill From bob at redivi.com Mon Aug 9 19:02:27 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 9 19:02:37 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <04Aug9.094230pdt."58612"@synergy1.parc.xerox.com> References: <04Aug9.094230pdt."58612"@synergy1.parc.xerox.com> Message-ID: On Aug 9, 2004, at 12:42 PM, Bill Janssen wrote: > GvR writes: >> Nevertheless I don't like to put decorators in the method body. I >> think it has something to do with the fact that normally one thinks of >> the body contents as relevalt to "call-time" while decorators happen >> at "definition-time". >> >> And before you bring up docstrings as a counterexample, yes, they >> should ideally have been designed to precede the method, too. > > Perhaps another way to think about this is that there are statements > about properties of the function, method, or class which should > precede the implementation of the whatever-it-is? That makes sense to > me. > > But I'm less sure about the relative placement of the name of the > whatever versus statements about the properties of the whatever. For > example, should the argument list also occur before the name of the > function? In a perfect world, would Python have the syntax: > > def (param1, param2) funcname: > ... > > or perhaps: > > @args(param1, param2) > def funcname: > ... > > In thinking about this, I find the argument made a day or two ago > about "cognitive binding" very powerful. That is, you should state > the name of the thing you are defining before you start stating > properties of it, so that the reader has the opportunity to mentally > construct a tagged "bucket" to put those properties in. This is > probably a useful thing for non-human tools, as well. > > Perhaps that's one reason that arguments to a function almost > universally occur after the mention of the function name, in most > programming languages: > > def foo (arg1, arg2, ...) > > So, by this principle, docstrings occur in the "right place", after > the function name but before the implementation. Perhaps this is why > docstrings are so successful in Python. Extending the principle to > decorators would mean that they should occur somewhere between the > function name and the function implementation. All of the C, D, and E > variants in the Wiki seem to support this. Perhaps one of them is the > right choice? Actually I would say that in a perfect world, functions would be expressions. Something like this: foo = staticmethod(def(*args, **kwargs)): .... -bob From mark at prothon.org Mon Aug 9 19:29:59 2004 From: mark at prothon.org (Mark Hahn) Date: Mon Aug 9 19:30:06 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: Forgive me if I'm being a pest, but no one has commented on the real reason I asked the question. What does everyone think of the idea of having these three built-in numeric types? 1) An Int implemented with infinite precision integer (Python Longs) with the constant/__str__ form of +-NN such as 0, -123, 173_394, etc. 2) A Decimal implemented with the .Net decimal float (or the IBM decimal if the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. 3) A binary Float implemented with the hardware floating point with the constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3, 2.35e-78, etc. There would be no automatic conversion except the / operator would convert from Int to Decimal and the Math module would convert Int and Decimal values to Float for almost all functions (except simple ones like abs, min, max). From fperez528 at yahoo.com Mon Aug 9 19:33:59 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Mon Aug 9 19:34:06 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> <200408071617.i77GHgs10281@guido.python.org> Message-ID: Paul Moore wrote: > Fernando Perez writes: [...] > But we have a couple of inconsistencies to backtrack and resolve. (And > the accepts/returns stuff is also inconsistent, but I'll stop picking > holes in what is clearly just an example...) > > I'm not saying that the same issues mightn't exist with the > decorator-before-def form, but I don't think the mental model for the > decorator-after-def form is quite as clean as you'd like either... Well, I'm not so naive as to think that there is a silver bullet here or a perfect syntax. Obviously in all cases the programmer will need to keep some context in his head. But this is true always: x = 1 ... many lines of code which can't change x in any way x.append('foo') # BUG! You spot something like this because you have a notion of local context, and you remember that x, being an integer which hasn't changed, can't be appended to as a list. In fact, I'd argue that one of (not the only one) the key qualities of a good programmer is the ability to maintain a fair amount of local and global context in his head. This is what allows you to realize the implications of a code change, see the potential for fruitful refactorings, etc. But that doesn't mean that one syntax choice can't make that process _easier_. It doesn't remove the need for having one's brain turned on, it is not a silver bullet, but it enhances the flow of processing this kind of information into an organized structure. > It's still subjective :-) I disagree, and I think I've given detailed arguments trying _precisely_ to cast this discussion in terms which are potentially more productive than 'its all subjective', which lead to endless running in circles. I very much appreciate a discussion of the merits of my ideas, but hopefully in terms which allow something which goes beyond "your shirt is blue, and I don't like blue". I've tried to reply here in such a vein, hoping to produce useful arguments for Guido and those ultimately making the decisions. Yes, there is always a degree of subjectivity in all choices, and in that sense I immensely trust Guido's intuition, ability and sense of aesthetics. But one's subjectivity works best when assisted by other arguments as well. Best, f From aahz at pythoncraft.com Mon Aug 9 19:34:15 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 9 19:34:18 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: <200408091525.i79FPLM23023@guido.python.org> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> <410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org> <200408091525.i79FPLM23023@guido.python.org> Message-ID: <20040809173415.GB28961@panix.com> On Mon, Aug 09, 2004, Guido van Rossum wrote: >Dima Dorfman: >> >> Are the exact semantics of what's allowed documented somewhere? > > Not unless it's in the PEP -- it usually takes a stab at implementing > something to find what's left unspecified! Yup! >> PEP 328 mostly talks about relative and absolute >> imports, and it doesn't specify the exact semantics of where >> parentheses should be allowed. My patch (attached) accepts >> >> import (os, sys) >> from sys import (stdin, stdout, stderr) >> import (os) >> from sys import (*) >> >> but rejects >> >> from (sys) import stdin >> import (os), (sys) >> import (os,) >> >> Should any of those be allowed? Anything that I missed? > > This suggests (given the recent criticism of how we've been following > the PEP process) that the PEP needs to be updated first. Enh; as you say, implementation is often needed to clarify, and I don't think anyone would dispute that this is a small portion of the PEP. > My own gut feeling is that it's not important to allow parentheses on > "bare" (from-less) imports, since you can easily repeat the 'import' > keyword on a new line; but that it is important on the from...import > form, and then only after the "import" keyword. So that would accept > your 2nd and 4th example but reject the 1st and 3rd. > > I guess one can argue about (*); I don't care about it and suggest > that it be dropped. I'd suggest that only ``from foo import (...)`` be permitted (where "..." does not include ``*``). But I don't care much, and I'll update the PEP as soon as it's clear which precise versions are allowed. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From fperez528 at yahoo.com Mon Aug 9 19:38:49 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Mon Aug 9 19:40:24 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> <019301c47db9$c269ec80$a9498890@neil> <200408091516.i79FGc722993@guido.python.org> Message-ID: Guido van Rossum wrote: > Let's make one thing clear; while decorators may be useful for some to > experiment with attaching signatures to methods, mainstream Python > will eventually have optional static typing built in, and it won't use I am immensely happy to hear this. I have read your old essay on the topic, but the long silence on this issue had me wondering whether it had been completely abandoned. I can't currently contribute to it, so I don't complain. But this is great news. Would you care to venture even a rough guess for when this might happen (and no, I'm not a lawyer, so you don't need a disclaimer in front :)? >From a scientific computing viewpoint, this is probably the single biggest stumbling point when using python today. We have scipy.weave, f2py and even manually written extension modules, which we all know how to use. But having optional static typing for the cases where python's dynamic nature is a cost with no benefit will be an incredible gain. Other methods always slow down the otherwise very fluid feel of using python, so having this as part of the language will be fantastic. Best, f From pf_moore at yahoo.co.uk Mon Aug 9 19:42:37 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Mon Aug 9 19:42:23 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: Jeremy Hylton writes: > On Sat, 07 Aug 2004 00:35:17 -0400, Edward Loper > wrote: >> The new "vertical bar" syntax was proposed recently, and has gotten >> almost unanimously positive feedback. In particular, of 17 responses >> to the syntax, only one has been negative (see table, below). > > I dislike it. It's not as friendly or visible as the @. I prefer @ but am willing to accept | as a compromise. I'd be interested to know if that sentiment (which I think I've expressed before) was one of those that Edward counted as "non-negative"... Paul. -- Home computers are being called upon to perform many new functions, including the consumption of homework formerly eaten by the dog -- Doug Larson From fperez528 at yahoo.com Mon Aug 9 19:50:19 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Mon Aug 9 19:50:27 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> <200408091504.i79F4lD22945@guido.python.org> Message-ID: Guido van Rossum wrote: > [Guido] >> > In the discussion on decorators months ago, solutions involving >> > special syntax inside the block were ruled out early on. Basically, >> > you shouldn't have to peek inside the block to find out important >> > external properties of the function. > > [Bill Janssen] >> Guido, could you expand on this principle a bit? Just stated as it > It's probably untenable as a formal principle. :-) > > Nevertheless I don't like to put decorators in the method body. I > think it has something to do with the fact that normally one thinks of > the body contents as relevalt to "call-time" while decorators happen > at "definition-time". I realize what your current position on this topic is, but I'd like to encourage you to at least reconsider it for a moment. In python we've come to consider, for better or worse, the section up to the end of the docstring, as a kind of 'header' or 'interface' zone for functions. I know you dislike the docstrings as an example of this, but in fact I don't view it as such a bad thing: the def indents and clearly separates the function, and for a while you have 'interface' information: function name, argument list, docstring, and perhaps now decorators. This actually feels very clean to me, and seems to nicely fit the mental model of how python works. Elsewhere in this thread: http://mail.python.org/pipermail/python-dev/2004-August/047326.html and below, I tried to make a reasoned argument in defense of this. I won't repeat all that here. I further worry what will happen to tools like inspect.getsource() with decorators outside the def: @foo @bar def baz(x,y): .... if I try to get the source for baz, will I see foo/bar? For introspection tools, this can lead to pretty misleading results. But perhaps this is easy to solve or has even already been taken care of. I imagine if @foo/@bar were inside the def, this wouldn't be an issue. Best, f From barry at python.org Mon Aug 9 20:00:54 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 9 20:00:43 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: <1092074454.8687.26.camel@localhost> On Mon, 2004-08-09 at 13:42, Paul Moore wrote: > I prefer @ but am willing to accept | as a compromise. I agree with Paul here. The only reason I suggested | was to make life easier for 3rd party tools. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/b8d87a29/attachment.pgp From tim.peters at gmail.com Mon Aug 9 20:20:15 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 20:20:18 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: <1f7befae04080911203b46485c@mail.gmail.com> [Mark Hahn] > Forgive me if I'm being a pest, Nope, not at all. > but no one has commented on the real reason I asked the question. What > does everyone think of the idea of having these three built-in numeric types? It depends on your goals for Prothon, and also on the relative performance of the various numeric implementations. Semantically, it would be nicest if Python used the IBM decimal type for everything by default (whether spelled 123, 123.0, or 123e0). That's not gonna happen, both because of backward compatibility, and because our implementation of that type is "too slow". I don't know how fast or slow the .NET Decimal type is on its own, let alone compared to whatever implementations of bigints and binary floats you're using. Speed may or may not matter to your target audience (although, in reality, you'll have multiple audiences, each with their own irrational demands ). > 1) An Int implemented with infinite precision integer (Python Longs) with > the constant/__str__ form of +-NN such as 0, -123, 173_394, etc. > > 2) A Decimal implemented with the .Net decimal float (or the IBM decimal if > the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN > such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. > > 3) A binary Float implemented with the hardware floating point with the > constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3, > 2.35e-78, etc. Sounds decent to me, although the IBM flavor of decimal is a full floating-point type, and some apps will definitely want to use it for 6.02e23 thingies instead of the native binary Float. Maybe use "d" as a "decimal float" exponent marker. > There would be no automatic conversion except the / operator would convert > from Int to Decimal and the Math module would convert Int and Decimal > values to Float for almost all functions (except simple ones like abs, min, > max). Int * Decimal will be very common in financial apps, and the latter are a prime use case for Decimal ("OK, they bought 200 copies of 'Programming Prothon' at 39.98 each", etc). Numeric auto-conversions that don't lose information are usually helpful. From FBatista at uniFON.com.ar Mon Aug 9 20:24:51 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Mon Aug 9 20:28:49 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] Message-ID: [Mark Hahn] #- 2) A Decimal implemented with the .Net decimal float (or the #- IBM decimal if #- the .Net decimal sucks too much) with the constant/__str__ #- form of +-NN.NN #- such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. I'm ok with all the rest and this but with a particular point: .Net decimal sucks too much if doesn't implement the IBM standard, ;) Basically, it'll be real nice if Python and Prothon behaves equally in decimal float. . Facundo From pf_moore at yahoo.co.uk Mon Aug 9 20:31:33 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Mon Aug 9 20:31:21 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: Mark Hahn writes: > Forgive me if I'm being a pest, but no one has commented on the real reason > I asked the question. What does everyone think of the idea of having these > three built-in numeric types? > > 1) An Int implemented with infinite precision integer (Python Longs) with > the constant/__str__ form of +-NN such as 0, -123, 173_394, etc. > > 2) A Decimal implemented with the .Net decimal float (or the IBM decimal if > the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN > such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. > > 3) A binary Float implemented with the hardware floating point with the > constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3, > 2.35e-78, etc. > > There would be no automatic conversion except the / operator would convert > from Int to Decimal and the Math module would convert Int and Decimal > values to Float for almost all functions (except simple ones like abs, min, > max). I think that having 2 different fractional types distinguished by something as subtle as the presence of an exponent (something no other language does to my knowledge) is an extremely bad idea. Worse still is the fact that the exponent in the e notation is a decimal exponent, but the type used is binary! Paul. -- It is better to be quotable than to be honest -- Tom Stoppard From eppstein at ics.uci.edu Mon Aug 9 20:45:06 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Mon Aug 9 20:45:18 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> <1092074454.8687.26.camel@localhost> Message-ID: In article <1092074454.8687.26.camel@localhost>, Barry Warsaw wrote: > > I prefer @ but am willing to accept | as a compromise. > > I agree with Paul here. The only reason I suggested | was to make life > easier for 3rd party tools. But does it? With @, a third party tool has an unambiguous indication of whether a line is a decorator. With |, it is not possible to recognize decorators with a regular expression, instead you have to do some context-free parsing to determine whether some previous line has an unclosed paren (in which case the | is a binop rather than a decorator). -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From mark at prothon.org Mon Aug 9 20:45:29 2004 From: mark at prothon.org (Mark Hahn) Date: Mon Aug 9 20:45:34 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> <1f7befae04080911203b46485c@mail.gmail.com> Message-ID: On Mon, 9 Aug 2004 14:20:15 -0400, Tim Peters wrote: > [Mark Hahn] >> Forgive me if I'm being a pest, > > Nope, not at all. > >> but no one has commented on the real reason I asked the question. What >> does everyone think of the idea of having these three built-in numeric types? > > It depends on your goals for Prothon, and also on the relative > performance of the various numeric implementations. Semantically, it > would be nicest if Python used the IBM decimal type for everything by > default (whether spelled 123, 123.0, or 123e0). Are you sure you mean that? I have been assuming that there is a fundamental need for an integer type that has no fraction. There are many places like indexing where you semantically want integers. I think if speed were no issue I'd still have integers and decimal floats. > That's not gonna > happen, both because of backward compatibility, and because our > implementation of that type is "too slow". > > I don't know how fast or slow the .NET Decimal type is on its own, let > alone compared to whatever implementations of bigints and binary > floats you're using. Speed may or may not matter to your target > audience (although, in reality, you'll have multiple audiences, each > with their own irrational demands ). I'm already planning on using Jim's IronPython implementation of your BigInts so I might as well port your Decimal code over for the Prothon Decimal also. The only thing I've ever stolen from Python for Prothon was your bigint code so I might as well be consistent in my thievery. :-) >> 1) An Int implemented with infinite precision integer (Python Longs) with >> the constant/__str__ form of +-NN such as 0, -123, 173_394, etc. >> >> 2) A Decimal implemented with the .Net decimal float (or the IBM decimal if >> the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN >> such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. >> >> 3) A binary Float implemented with the hardware floating point with the >> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3, >> 2.35e-78, etc. > > Sounds decent to me, although the IBM flavor of decimal is a full > floating-point type, and some apps will definitely want to use it for > 6.02e23 thingies instead of the native binary Float. Maybe use "d" as > a "decimal float" exponent marker. I would hate to mess up my "pure" scheme but it makes sense. Will do. >> There would be no automatic conversion except the / operator would convert >> from Int to Decimal and the Math module would convert Int and Decimal >> values to Float for almost all functions (except simple ones like abs, min, >> max). > > Int * Decimal will be very common in financial apps, and the latter > are a prime use case for Decimal ("OK, they bought 200 copies of > 'Programming Prothon' at 39.98 each", etc). Numeric auto-conversions > that don't lose information are usually helpful. When I wrote that I was thinking of no automatic conversion from Decimal to Float. Conversion from Int to Decimal would make sense as you suggest. Thanks very much. I think I have a handle on the issues now. To summarize, .Net sucks (no big surprise). :-) From chris.cavalaria at free.fr Mon Aug 9 20:43:05 2004 From: chris.cavalaria at free.fr (Christophe Cavalaria) Date: Mon Aug 9 20:50:15 2004 Subject: [Python-Dev] Re: Re: def fn (args) [dec,dec]: References: <200408061445.i76EjqT07026@guido.python.org> <200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Guido: > >> Just look at it when either the list of decorators or the argument >> list doesn't fit on one line: > > You don't *have* to format it quite that badly. > > def longMethodNameForEffect(longArgumentOne=None, > longArgumentTwo=42) \ > [staticmethod, > funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", > status="experimental", > author="BDFL")]: Am I the only one who finds that it's : - ugly - incredibly error prone due to a lot of special cases on each line of code - a back door for the "how to indent C code" flamewars in Python I guess it depends on the amount of decorators you will place on each function. If you place 3-4 decorators per function ( it could easily happen once we get a good and easy to use syntax ), we'll favor a syntax that makes multiline decorator declaration clear and easy to type. The [...] doesn't apply. On a readability point of view, the @ syntax scales much better with a big number of decorators than the [...] syntax. Also, the decorators don't have that much effet on what happens between the def and the return. For that reason I find it a mistake to place any decorator information after the def statement. From mark at prothon.org Mon Aug 9 20:45:51 2004 From: mark at prothon.org (Mark Hahn) Date: Mon Aug 9 20:50:54 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] References: Message-ID: On Mon, 9 Aug 2004 15:24:51 -0300, Batista, Facundo wrote: > [Mark Hahn] > > #- 2) A Decimal implemented with the .Net decimal float (or the > #- IBM decimal if > #- the .Net decimal sucks too much) with the constant/__str__ > #- form of +-NN.NN > #- such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. > > I'm ok with all the rest and this but with a particular point: .Net decimal > sucks too much if doesn't implement the IBM standard, ;) > > Basically, it'll be real nice if Python and Prothon behaves equally in > decimal float. OK, will do. From jack at performancedrivers.com Mon Aug 9 21:05:31 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Mon Aug 9 21:05:36 2004 Subject: [Python-Dev] A usability argument for list-after-def In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com> Message-ID: <20040809190531.GA23725@performancedrivers.com> On Thu, Aug 05, 2004 at 04:48:09PM -0400, Phillip J. Eby wrote: > Brackets are normally used in English to indicate an annotation or "aside" > of some sort; such as a commentary *about* the surrounding material. +1, glad this one is back from the dead. -Jack From mark at prothon.org Mon Aug 9 21:09:29 2004 From: mark at prothon.org (Mark Hahn) Date: Mon Aug 9 21:09:34 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: <6rdhjlf9tfol$.d6mi30mbjt80.dlg@40tude.net> On Mon, 09 Aug 2004 19:31:33 +0100, Paul Moore wrote: > I think that having 2 different fractional types distinguished by > something as subtle as the presence of an exponent (something no other > language does to my knowledge) is an extremely bad idea. > > Worse still is the fact that the exponent in the e notation is a > decimal exponent, but the type used is binary! Good point. Since Tim talked me into allowing the decimal type to use the floating point form anyway, I might as well drop that idea for differentiation. I've learned what I needed here. I've learned that the .Net decimal type sucks and that having an "in-between" decimal type isn't necessarily a bad idea. We can hammer out the syntax back on the Prothon mailing list. Of course I'd like to encourage any and all to help me out back there. The mailing list has light traffic since we have no pesky users. :-) Discussion is at: http://lists.prothon.org/mailman/listinfo/prothon-user Thanks everyone... Have a good day. From s.percivall at chello.se Mon Aug 9 21:20:35 2004 From: s.percivall at chello.se (Simon Percivall) Date: Mon Aug 9 21:20:38 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> <200408091504.i79F4lD22945@guido.python.org> Message-ID: <30C8BD17-EA39-11D8-8B30-0003934AD54A@chello.se> On 2004-08-09, at 19.50, Fernando Perez wrote: > I further worry what will happen to tools like inspect.getsource() with > decorators outside the def: > > @foo > @bar > def baz(x,y): > .... > > if I try to get the source for baz, will I see foo/bar? For > introspection > tools, this can lead to pretty misleading results. But perhaps this > is easy > to solve or has even already been taken care of. I imagine if > @foo/@bar were > inside the def, this wouldn't be an issue. Right now, you'll see the source of the function/method defined above the decorators or, if none, the top line/lines of the source file. Yes, it's pretty broken. From bsder at mail.allcaps.org Mon Aug 9 21:34:31 2004 From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.) Date: Mon Aug 9 21:34:37 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: <22FD7E0D-EA3B-11D8-B417-000A95C874EE@mail.allcaps.org> On Aug 9, 2004, at 11:31 AM, Paul Moore wrote: > I think that having 2 different fractional types distinguished by > something as subtle as the presence of an exponent (something no other > language does to my knowledge) is an extremely bad idea. Doesn't Common Lisp use 1.0e+10 to signify singles and 1.0d+10 to signify doubles? That doesn't make it a good idea, but there is precedent. -a From python at rcn.com Sun Aug 8 21:42:38 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 9 21:42:44 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] In-Reply-To: Message-ID: <004801c47d7f$dcedb120$191fc797@oemcomputer> > I have been assuming that there is a > fundamental need for an integer type that has no fraction. The IBM spec covers almost all needs except speed and space (and even those aren't horrible in a good C implementation). It is a superset of 754/854, of fixed point, and of high precision integer arithmetic. If you were bold, you would have this as the sole numeric type. This would spare you decimal representation issues, inadequate range/precision problems, type conversion issues, and compatibility challenges. Plus, it is darned nice to have something as thoroughly thought out and tested. Of course, if you want to roll your own from scratch, I hear that it is trivially easy and that everything always works out in the end, NOT! Raymond From bsder at mail.allcaps.org Mon Aug 9 21:46:41 2004 From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.) Date: Mon Aug 9 21:46:45 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] In-Reply-To: References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> <1f7befae04080911203b46485c@mail.gmail.com> Message-ID: On Aug 9, 2004, at 11:45 AM, Mark Hahn wrote: > On Mon, 9 Aug 2004 14:20:15 -0400, Tim Peters wrote: >> >> It depends on your goals for Prothon, and also on the relative >> performance of the various numeric implementations. Semantically, it >> would be nicest if Python used the IBM decimal type for everything by >> default (whether spelled 123, 123.0, or 123e0). > > Are you sure you mean that? I have been assuming that there is a > fundamental need for an integer type that has no fraction. Sure, but both binary floating point and Decimal happily inhabit that space. The issue in Days Gone By(tm) was the fact that a single precision float tended to be the default type and only had about 24 bits of precision (somewhere around 8-9 decimal digits). Consequently, people would often get surprised when accumulating small integers ceased to work around 10^8 or 10^9. With binary FP doubles, this is less of an issue as it has about 54 bits of precision (about 18 decimal digits or so). However, it still places an artificial restriction on maximum precision. Besides, the only reason programmers write loops as integers is because accumulating floats "doesn't work". With Decimal, that is no longer the case. -a From bsder at mail.allcaps.org Mon Aug 9 21:49:12 2004 From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.) Date: Mon Aug 9 21:49:16 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: <3030F1DE-EA3D-11D8-B417-000A95C874EE@mail.allcaps.org> On Aug 9, 2004, at 10:29 AM, Mark Hahn wrote: > 3) A binary Float implemented with the hardware floating point with the > constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3, > 2.35e-78, etc. Be careful defining string representations of numbers. It is a non-trivial problem to adhere to the correct internationalization standards. > There would be no automatic conversion except the / operator would > convert > from Int to Decimal Int goes to the default type of float (which is currently binary FP) already upon using the / operator. > and the Math module would convert Int and Decimal > values to Float for almost all functions (except simple ones like abs, > min, > max). This is not necessarily a good idea. Exponentials are very useful in financial situations and want to be decimals, not binary floats, in those cases. -a From barry at python.org Mon Aug 9 21:53:10 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 9 21:53:00 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: References: <41145C05.1020501@gradient.cis.upenn.edu> <1092074454.8687.26.camel@localhost> Message-ID: <1092081190.8687.40.camel@localhost> On Mon, 2004-08-09 at 14:45, David Eppstein wrote: > > I agree with Paul here. The only reason I suggested | was to make life > > easier for 3rd party tools. > > But does it? > > With @, a third party tool has an unambiguous indication of whether a > line is a decorator. With |, it is not possible to recognize decorators > with a regular expression, instead you have to do some context-free > parsing to determine whether some previous line has an unclosed paren > (in which case the | is a binop rather than a decorator). Oh ick. I hadn't thought of that, but if that's going to cause a problem given the current grammar (and I can't test that atm), then I'm going to have to retract my support for |. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/265fc754/attachment.pgp From python at rcn.com Sun Aug 8 21:57:17 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 9 21:57:22 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: <1092081190.8687.40.camel@localhost> Message-ID: <004f01c47d81$e861e740$191fc797@oemcomputer> > > > I agree with Paul here. The only reason I suggested | was to make > life > > > easier for 3rd party tools. > > > > But does it? > > > > With @, a third party tool has an unambiguous indication of whether a > > line is a decorator. With |, it is not possible to recognize decorators > > with a regular expression, instead you have to do some context-free > > parsing to determine whether some previous line has an unclosed paren > > (in which case the | is a binop rather than a decorator). > > Oh ick. I hadn't thought of that, but if that's going to cause a > problem given the current grammar (and I can't test that atm), then I'm > going to have to retract my support for |. An exclamation point would be better in this regard. Raymond From fredrik at pythonware.com Mon Aug 9 22:00:19 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 9 21:58:42 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> <1092074454.8687.26.camel@localhost> Message-ID: David Eppstein wrote: > With @, a third party tool has an unambiguous indication of whether a > line is a decorator. With |, it is not possible to recognize decorators > with a regular expression, instead you have to do some context-free > parsing to determine whether some previous line has an unclosed paren > (in which case the | is a binop rather than a decorator). "\ @decorator #" """ @decorator """ (etc) From chris.cavalaria at free.fr Mon Aug 9 22:04:41 2004 From: chris.cavalaria at free.fr (Christophe Cavalaria) Date: Mon Aug 9 22:04:45 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> <1092074454.8687.26.camel@localhost> Message-ID: Fredrik Lundh wrote: > David Eppstein wrote: > >> With @, a third party tool has an unambiguous indication of whether a >> line is a decorator. With |, it is not possible to recognize decorators >> with a regular expression, instead you have to do some context-free >> parsing to determine whether some previous line has an unclosed paren >> (in which case the | is a binop rather than a decorator). > > "\ > @decorator #" > > """ > @decorator > """ > > (etc) Let me continue the (etc) for you : """ def foo(): return 0 """ From s.percivall at chello.se Mon Aug 9 22:10:40 2004 From: s.percivall at chello.se (Simon Percivall) Date: Mon Aug 9 22:10:43 2004 Subject: [Python-Dev] tokenize.py (generate_tokens yielding @ as OP) Message-ID: <2FE44090-EA40-11D8-8B30-0003934AD54A@chello.se> This is a question regarding the tokenize.generate_tokens function. As it is in CVS HEAD, generate_tokens yields the '@' token as having type OP, not type AT. Is this how it should be? //Simon From guido at python.org Mon Aug 9 22:14:42 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 22:14:49 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: Your message of "Mon, 09 Aug 2004 11:50:19 MDT." References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> <200408091504.i79F4lD22945@guido.python.org> Message-ID: <200408092014.i79KEh123856@guido.python.org> > I realize what your current position on this topic is, but I'd like > to encourage you to at least reconsider it for a moment. Please remember the procedure. The community is to come up with *one* alternative that they universally prefer over @decorators; otherwise, @decorators-before-def will stay. (I'll even give you *two* if the community can agree that both are acceptable and really wishes to let me pick one.) Plenty of people agree with me on this particular issue (decorators inside or outside the def); you have to convince them first before pleading with me. I don't want to have to argue every single detail. As it stands now, it looks like the community is having a hard time agreeing on anything, which suggests that I might as well stick with @decorators-before-def. I try to limit my posts to (a) procedural adjustments; (b) clarifications of my previously stated position; (c) clarifications of other factual misunderstandings; (d) pointing out issues that nobody else has brought up yet. I try not to be pulled into the debate; I simply can't afford the time. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 9 22:17:46 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 22:17:52 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: Your message of "Mon, 09 Aug 2004 11:38:49 MDT." References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> <019301c47db9$c269ec80$a9498890@neil> <200408091516.i79FGc722993@guido.python.org> Message-ID: <200408092017.i79KHkW23877@guido.python.org> > Guido van Rossum wrote: > > Let's make one thing clear; while decorators may be useful for > > some to experiment with attaching signatures to methods, > > mainstream Python will eventually have optional static typing > > built in, and it won't use > > I am immensely happy to hear this. I have read your old essay on > the topic, but the long silence on this issue had me wondering > whether it had been completely abandoned. I can't currently > contribute to it, so I don't complain. But this is great news. > Would you care to venture even a rough guess for when this might > happen (and no, I'm not a lawyer, so you don't need a disclaimer in > front :)? Probably not before Python 3000, which is awaiting my retirement (or at least a sabbatical year), unless the community revisits the issue and comes up with a solid design that I can agree with. My retirement or sabbatical is at least a few years off... :-( > From a scientific computing viewpoint, this is probably the single > biggest stumbling point when using python today. We have > scipy.weave, f2py and even manually written extension modules, which > we all know how to use. But having optional static typing for the > cases where python's dynamic nature is a cost with no benefit will > be an incredible gain. Other methods always slow down the otherwise > very fluid feel of using python, so having this as part of the > language will be fantastic. I'm afraid that if you want something *now* in order to interface to 3rd party tools, an interim solution using decorators may be your best bet for the next few years (assuming Python 2.4 has some form of decorators). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 9 22:19:25 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 22:19:32 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: Your message of "Mon, 09 Aug 2004 13:34:15 EDT." <20040809173415.GB28961@panix.com> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> <410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org> <200408091525.i79FPLM23023@guido.python.org> <20040809173415.GB28961@panix.com> Message-ID: <200408092019.i79KJPw23893@guido.python.org> > I'd suggest that only ``from foo import (...)`` be permitted (where > "..." does not include ``*``). But I don't care much, and I'll > update the PEP as soon as it's clear which precise versions are > allowed. This seems to be the consensus so far, so why don't you add all that detail to the PEP. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 9 22:21:57 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 9 22:22:04 2004 Subject: [Python-Dev] tokenize.py (generate_tokens yielding @ as OP) In-Reply-To: Your message of "Mon, 09 Aug 2004 22:10:40 +0200." <2FE44090-EA40-11D8-8B30-0003934AD54A@chello.se> References: <2FE44090-EA40-11D8-8B30-0003934AD54A@chello.se> Message-ID: <200408092021.i79KLv823929@guido.python.org> > This is a question regarding the tokenize.generate_tokens function. > > As it is in CVS HEAD, generate_tokens yields the '@' token as > having type OP, not type AT. Is this how it should be? Without looking at the code, I believe tokenize treats all operators as OP, for a rather liberal definition of "operator". (A more serious problem that I just encountered in a different context: when you feed it input containing unsupported characters, the spaces before those unsupported characters are also returned as tokens. I think that's a bug.) --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Mon Aug 9 22:42:28 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 9 22:42:53 2004 Subject: [Python-Dev] Re: Recommended way to tell platform In-Reply-To: <4115CFFD.1000309@v.loewis.de> References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de> <200408071637.i77Gb0w10415@guido.python.org> <4115CFFD.1000309@v.loewis.de> Message-ID: <20040809204228.GA11199@rogue.amk.ca> On Sun, Aug 08, 2004 at 09:02:21AM +0200, "Martin v. L?wis" wrote: > Precisely that: impossible to say. In most cases, you should not test > for the platform, but just use the platform functionality, and fall back > to something else if it isn't present. Returning to the asyncore case that started all this: should I follow this principle and check if SO_EXCLUSIVEADDRUSE exists, and if it does use that constant instead of SO_REUSEADDR? --amk From eppstein at ics.uci.edu Mon Aug 9 22:49:20 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Mon Aug 9 22:49:28 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> <1092074454.8687.26.camel@localhost> Message-ID: In article , "Fredrik Lundh" wrote: > David Eppstein wrote: > > > With @, a third party tool has an unambiguous indication of whether a > > line is a decorator. With |, it is not possible to recognize decorators > > with a regular expression, instead you have to do some context-free > > parsing to determine whether some previous line has an unclosed paren > > (in which case the | is a binop rather than a decorator). > > "\ > @decorator #" > > """ > @decorator > """ > > (etc) > > I think quoted string recognition in Python is still regular, not context-free. Anyway, the syntax highlighters in editors that I've used don't seem to have a problem with it. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From tim.peters at gmail.com Mon Aug 9 22:54:00 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 22:54:02 2004 Subject: [Python-Dev] Another test_compiler mystery Message-ID: <1f7befae0408091354605dc58b@mail.gmail.com> I've noticed several times now, in both debug and release builds, that if I run regrtest.py with -uall, *sometimes* it just stops after running test_compiler: $ python_d regrtest.py -uall test_grammar test_opcodes ... test_compare test_compile test_compiler $ There's no indication of error, it just ends. It's not consistent. Happened once when I was running with -v, and test_compiler's output ended here: ... compiling C:\Code\python\lib\test\test_operator.py compiling C:\Code\python\lib\test\test_optparse.py compiling C:\Code\python\lib\test\test_os.py compiling C:\Code\python\lib\test\test_ossaudiodev.py compiling C:\Code\python\lib\test\test_parser.py In particular, there's no Ran M tests in Ns output, so it doesn't look like unittest (let alone regrtest) ever got control back. Hmm. os.listdir() is in sorted order on NTFS, so test_compiler should be chewing over a lot more files after test_parser.py. *This* I could blame on a blown C stack -- although I'd expect a much nastier symptom then than just premature termination. Anyone else? From amk at amk.ca Mon Aug 9 23:15:17 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 9 23:15:43 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <4115DD5A.2070108@interlink.com.au> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> Message-ID: <20040809211517.GB11199@rogue.amk.ca> On Sun, Aug 08, 2004 at 05:59:22PM +1000, Anthony Baxter wrote: > The AddlistClass in email is marked as deprecated, but is deprecated > in favour of a class in rfc822! Maybe deprecating rfc822 simply isn't a worthwhile goal? Anyway, here's a revised action list: * Remove TERMIOS, mpz, statcache, xreadlines, rotor. (Unchanged from my earlier proposal.) These modules have raised DeprecationWarning for a while, but only rotor is listed in PEP 4. I'll add them to the PEP in any event; is not being listed sufficient grounds for delaying their removal to 2.5? (I would say we don't need to wait until 2.5; they've been raising DeprecationWarnings for a long time, so users should be well aware the modules' days are numbered.) * Leave rfc822, mimetools alone; the stdlib will have to avoid their use first, and I'm not going to do that right now. * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the stdlib uses them. --amk From aahz at pythoncraft.com Mon Aug 9 23:23:46 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 9 23:23:48 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040809211517.GB11199@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> Message-ID: <20040809212346.GA2699@panix.com> On Mon, Aug 09, 2004, A.M. Kuchling wrote: > > Maybe deprecating rfc822 simply isn't a worthwhile goal? > > Anyway, here's a revised action list: > > * Remove TERMIOS, mpz, statcache, xreadlines, rotor. > (Unchanged from my earlier proposal.) > > These modules have raised DeprecationWarning for a while, but only > rotor is listed in PEP 4. I'll add them to the PEP in any event; > is not being listed sufficient grounds for delaying their removal to > 2.5? > > (I would say we don't need to wait until 2.5; they've been raising > DeprecationWarnings for a long time, so users should be well aware > the modules' days are numbered.) > > * Leave rfc822, mimetools alone; the stdlib will have to avoid their use > first, and I'm not going to do that right now. > > * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the > stdlib uses them. What about DeprecationWarning for whrandom? (Not that I care personally.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From s.percivall at chello.se Mon Aug 9 23:34:50 2004 From: s.percivall at chello.se (Simon Percivall) Date: Mon Aug 9 23:34:52 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: <30C8BD17-EA39-11D8-8B30-0003934AD54A@chello.se> References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> <200408091504.i79F4lD22945@guido.python.org> <30C8BD17-EA39-11D8-8B30-0003934AD54A@chello.se> Message-ID: On 2004-08-09, at 21.20, Simon Percivall wrote: > On 2004-08-09, at 19.50, Fernando Perez wrote: >> I further worry what will happen to tools like inspect.getsource() >> with >> decorators outside the def: >> >> @foo >> @bar >> def baz(x,y): >> .... >> >> if I try to get the source for baz, will I see foo/bar? For >> introspection >> tools, this can lead to pretty misleading results. But perhaps this >> is easy >> to solve or has even already been taken care of. I imagine if >> @foo/@bar were >> inside the def, this wouldn't be an issue. > > Right now, you'll see the source of the function/method defined above > the decorators or, if none, the top line/lines of the source file. Yes, > it's pretty broken. I uploaded a quick patch for this broken behaviour to SF, patch 1006219, if anyone cares. //Simon From tim.peters at gmail.com Mon Aug 9 23:35:14 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 9 23:35:16 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040809212346.GA2699@panix.com> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040809212346.GA2699@panix.com> Message-ID: <1f7befae04080914352cf5085d@mail.gmail.com> [Aahz] > What about DeprecationWarning for whrandom? (Not that I care > personally.) That's already been done. From amk at amk.ca Mon Aug 9 23:38:20 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 9 23:38:46 2004 Subject: [Python-Dev] Third Bug Day outcome Message-ID: <20040809213820.GC11199@rogue.amk.ca> The bug day on Saturday went well, as usual. 19 bugs and 12 patches were closed. That's not as good as the first bug day (30 bugs), but is competitive with the second (18 bugs, 21 patches). Lots of the easy bugs have been cleaned out, I expect, so each remaining bug takes more time to fix. The composition of the turnout was the surprising thing. My plaintive bleating on python-dev resulted in much higher participation by people with CVS committers, but there weren't many non-committer people around (Seo Sanghyeon and Mike Coleman were the two non-developers I noticed). This is a pity, because bug days seem like a good way to gently introduce more people to hacking on the source. During the day, Armin Rigo wrote a nifty IRC bot that takes an SF bug/patch ID and returns the title; taking an example from the transcript: 20:53:56 Looking at #777659. 20:53:57 * sf_number bug 777659 - Uninitialized variable used in Tools/faqwiz/faqwiz.py I won't be running a bug day in September; if someone else wants to arrange it, please feel free. Otherwise, they'll resume in October. --amk From python at rcn.com Sun Aug 8 23:41:48 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 9 23:41:54 2004 Subject: [Python-Dev] Third Bug Day outcome In-Reply-To: <20040809213820.GC11199@rogue.amk.ca> Message-ID: <006101c47d90$8228efa0$191fc797@oemcomputer> > A.M. Kuchling > I won't be running a bug day in September; if someone else wants to > arrange it, please feel free. Otherwise, they'll resume in October. What is involved beyond the Wiki page and announcing a day? learned-to-ask-questions-before-volunteering-ly yours, Raymond From fperez528 at yahoo.com Mon Aug 9 23:50:00 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Mon Aug 9 23:50:09 2004 Subject: [Python-Dev] Re: Re: Decorators: vertical bar syntax References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> <200408091504.i79F4lD22945@guido.python.org> <200408092014.i79KEh123856@guido.python.org> Message-ID: Guido van Rossum wrote: >> I realize what your current position on this topic is, but I'd like >> to encourage you to at least reconsider it for a moment. > > Please remember the procedure. The community is to come up with *one* > alternative that they universally prefer over @decorators; otherwise, > @decorators-before-def will stay. (I'll even give you *two* if the > community can agree that both are acceptable and really wishes to let > me pick one.) I was simply trying to outline, in a hopefully clear and useful form, an argument specifically pointing in a direction you'd personally called 'closed' from your side. I fully realize your bandwith is limited, and probably already well saturated by this debate. I also have the strong suspicion, given how difficult it is proving to reach significant consensus on almost anything on this, that it will probably come down to you making a final 'hard' decision. That's ok, and we can all live with it (and most of us trust your judgement because you've shown its qualities before). But if it comes to that, at least I figured it was worth arguing better this particular point, which I felt had been discussed from other angles but not this one. I feel like I've said as much as is reasonable on this topic, so I'll probably shut up and wait for something to happen. I'll then see if IPython needs fixing to live in a post-decorators world or not, and work accordingly. Best, f From bob at redivi.com Tue Aug 10 00:10:35 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 10 00:10:42 2004 Subject: [Python-Dev] Re: Re: Decorators: vertical bar syntax In-Reply-To: References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> <200408091504.i79F4lD22945@guido.python.org> <200408092014.i79KEh123856@guido.python.org> Message-ID: On Aug 9, 2004, at 5:50 PM, Fernando Perez wrote: > I feel like I've said as much as is reasonable on this topic, so I'll > probably > shut up and wait for something to happen. I'll then see if IPython > needs > fixing to live in a post-decorators world or not, and work accordingly. Wouldn't it be reasonable to just change the magic character that IPython (and maybe even Leo) uses from '@' to ':', '=', or '/'? As far as I know, it's practically[1] never valid Python to start a line with any of these symbols (this list of not definitive, of course). '/' has the advantage, for IPython, that it's easy to type (no shift involved, at least on my US keyboard) and should be familiar to many people as a "command" character from IRC clients, searching in vim and less, etc. [1] It can be done with a backslash line continuation or in a multi-line docstring, but I wouldn't really consider that the beginning of a line in the practical sense... -bob From foom at fuhm.net Tue Aug 10 00:14:12 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 10 00:14:14 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't Message-ID: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> There's a fair number of classes that claim they are defined in __builtin__, but do not actually appear there. For example: >>> def qual(clazz): ... return clazz.__module__ + '.' + clazz.__name__ ... >>> qual(types.GeneratorType) '__builtin__.generator' >>> qual(types.FunctionType) '__builtin__.function' >>> qual(types.MethodType) '__builtin__.instancemethod' >>> qual(types.NoneType) '__builtin__.NoneType' >>> qual(types.GeneratorType) '__builtin__.generator' >>> __builtin__.generator AttributeError: 'module' object has no attribute 'generator' [[[etc.]]] IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these?class names to use the appropriate object from the types module instead. Thoughts? If it isn't desirable to have these names appear in __builtin__, perhaps their '__module__' should be changed to another module where they are defined? James From jim at zope.com Tue Aug 10 00:16:53 2004 From: jim at zope.com (Jim Fulton) Date: Tue Aug 10 00:16:56 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <4117F7D5.3090907@zope.com> James Y Knight wrote: > There's a fair number of classes that claim they are defined in > __builtin__, but do not actually appear there. For example: > > >>> def qual(clazz): > ... return clazz.__module__ + '.' + clazz.__name__ > ... > >>> qual(types.GeneratorType) > '__builtin__.generator' > >>> qual(types.FunctionType) > '__builtin__.function' > >>> qual(types.MethodType) > '__builtin__.instancemethod' > >>> qual(types.NoneType) > '__builtin__.NoneType' > >>> qual(types.GeneratorType) > '__builtin__.generator' > >>> __builtin__.generator > AttributeError: 'module' object has no attribute 'generator' > [[[etc.]]] > > IMO classes ought to actually appear in __builtin__ if they claim they > are defined there. Doing otherwise breaks reflection, as you have to add > a special case for these class names to use the appropriate object from > the types module instead. Thoughts? I agree. > If it isn't desirable to have these > names appear in __builtin__, perhaps their '__module__' should be > changed to another module where they are defined? +1 Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From amk at amk.ca Tue Aug 10 00:16:42 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 10 00:17:08 2004 Subject: [Python-Dev] Third Bug Day outcome In-Reply-To: <006101c47d90$8228efa0$191fc797@oemcomputer> References: <20040809213820.GC11199@rogue.amk.ca> <006101c47d90$8228efa0$191fc797@oemcomputer> Message-ID: <20040809221642.GA11485@rogue.amk.ca> On Sun, Aug 08, 2004 at 05:41:48PM -0400, Raymond Hettinger wrote: > What is involved beyond the Wiki page and announcing a day? The most burdensome part is just hanging around all day so there's someone who can close bugs, commit changes, answer questions, etc. For the first two bug days, I was the only committer around so that meant I had to stay on IRC for 8 hours. This past bug day, there were lots of committers around, so I could have easily skipped out for a few hours to go grocery shopping, run errands, or whatever. Also, 9 to 5 is a good length for the bug day; by 3 or 4 PM I'm starting to get frazzled, but the day is almost over so that's OK. The first bug day was announced as being 12 hours long, which would have ben deadly, but luckily SF CVS stopped working around 6PM on that first day and saved me. --amk From FBatista at uniFON.com.ar Tue Aug 10 00:19:04 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Aug 10 00:23:01 2004 Subject: [Python-Dev] Third Bug Day outcome Message-ID: [A.M. Kuchling] #- Also, 9 to 5 is a good length for the bug day; by 3 or 4 PM I'm #- starting to get frazzled, but the day is almost over so that's OK. #- The first bug day was announced as being 12 hours long, which would #- have ben deadly, but luckily SF CVS stopped working around #- 6PM on that #- first day and saved me. 9 to 5 where? Maybe for the next Bug Day should be two "IRC hosts", one in America and other in Europe. so, the'll be 6 hs each (top) and the whole day will be of 12 hours. Not-being-able-to-be-him-because-of-56.6k-modem-ly yours, . Facundo From fperez528 at yahoo.com Tue Aug 10 00:23:43 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Tue Aug 10 00:23:48 2004 Subject: [Python-Dev] Re: Re: Re: Decorators: vertical bar syntax References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> <200408091504.i79F4lD22945@guido.python.org> <200408092014.i79KEh123856@guido.python.org> Message-ID: Bob Ippolito wrote: > > On Aug 9, 2004, at 5:50 PM, Fernando Perez wrote: > >> I feel like I've said as much as is reasonable on this topic, so I'll >> probably >> shut up and wait for something to happen. I'll then see if IPython >> needs >> fixing to live in a post-decorators world or not, and work accordingly. > > Wouldn't it be reasonable to just change the magic character that > IPython (and maybe even Leo) uses from '@' to ':', '=', or '/'? As > far as I know, it's practically[1] never valid Python to start a line > with any of these symbols (this list of not definitive, of course). > '/' has the advantage, for IPython, that it's easy to type (no shift > involved, at least on my US keyboard) and should be familiar to many > people as a "command" character from IRC clients, searching in vim and > less, etc. Indeed, if @ becomes the decorator character, I'll post to the ipython user/dev lists and request community feedback. Any of these is an option, and I've also considered using %. Note that in most cases in ipython, you don't really type the @ character explicitly, since ipython auto-detects magics when there is no chance of ambiguity. You only need the explicit @ when you have both a magic and a variable with the same name. As I said, this is techincally trivial to fix. It's just a bunch of busywork, and it will break user's _habits_. I'll keep your point of non-shifting characters in mind when the time comes, thanks for pointing that out. > [1] It can be done with a backslash line continuation or in a > multi-line docstring, but I wouldn't really consider that the beginning > of a line in the practical sense... For ipython these only make sense in single-lines, so this is not a problem. Thanks for the feedback. Best, f From amk at amk.ca Tue Aug 10 00:34:38 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 10 00:35:04 2004 Subject: [Python-Dev] Re: Python-dev get-togethers? In-Reply-To: <20040809221642.GA11485@rogue.amk.ca> References: <20040809213820.GC11199@rogue.amk.ca> <006101c47d90$8228efa0$191fc797@oemcomputer> <20040809221642.GA11485@rogue.amk.ca> Message-ID: <20040809223438.GB11485@rogue.amk.ca> BTW, Neal Norwitz recently suggested to me that having local, physical meetings of python-dev'vers would be useful for planning (and perhaps development) purposes. While Guido and Jeremy are no longer in the DC area, there's still a number of people here -- Tim, Fred, Barry, Jim, Neil Schemenauer, Neal Norwitz, Steve Holden, me -- who could usefully chew over issues or work on some particularly difficult task or bug. (I don't think it's very useful to congregate physically just to work on run-of-the-mill bugs and patches; as a group we've become pretty skilled at working remotely on simpler things.) We could meet at someone's house, do two or four hours of work or discussion, and then turn it into a party. Would anyone else in the DC metro area be interested in coming to such a thing? Please mail me privately if you'd be interested, or if you could offer your home as a location. (This wouldn't happen until later in the fall, October/Novemberish; for now I just want to see if there's any interest.) --amk From tim.peters at gmail.com Tue Aug 10 00:39:01 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 10 00:39:11 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae0408091354605dc58b@mail.gmail.com> References: <1f7befae0408091354605dc58b@mail.gmail.com> Message-ID: <1f7befae040809153946522f6d@mail.gmail.com> Here's a cute one: """ import compiler, sys f = open('../Lib/test/test_parser.py') guts = f.read() f.close() def ouch(n): if n == 0: return compiler.compile(guts, "", "exec") else: return ouch(n-1) for n in range(100, 250): try: ouch(n) msg = 'ok' except Exception, msg: msg = str(sys.exc_info()[0]) + ' ' + str(msg) print n, msg """ Under 2.3.4, that works as expected: when n hit a large enough value, from then on it was all repetitions of recursion-depth exceptions: ,,, 147 ok 148 ok 149 ok 150 exceptions.RuntimeError maximum recursion depth exceeded 151 exceptions.RuntimeError maximum recursion depth exceeded 152 exceptions.RuntimeError maximum recursion depth exceeded ... Under CVS, it's very different. Under a debug build, I get no output *at all*(!). Not even if I change the loop to start at 0. Under a release build: ... 120 ok 121 ok 122 ok 123 exceptions.RuntimeError maximum recursion depth exceeded 124 exceptions.RuntimeError maximum recursion depth exceeded 125 exceptions.RuntimeError maximum recursion depth exceeded 126 exceptions.KeyError 307 127 exceptions.RuntimeError maximum recursion depth exceeded 128 exceptions.KeyError 306 129 exceptions.RuntimeError maximum recursion depth exceeded 130 exceptions.KeyError 305 131 exceptions.RuntimeError maximum recursion depth exceeded 132 exceptions.KeyError 304 133 exceptions.RuntimeError maximum recursion depth exceeded 134 exceptions.KeyError 303 135 exceptions.KeyError 302 136 exceptions.RuntimeError maximum recursion depth exceeded 137 exceptions.KeyError 301 138 exceptions.RuntimeError maximum recursion depth exceeded 139 exceptions.KeyError 300 140 exceptions.RuntimeError maximum recursion depth exceeded 141 exceptions.RuntimeError maximum recursion depth exceeded 142 exceptions.KeyError 299 143 exceptions.RuntimeError maximum recursion depth exceeded 144 exceptions.KeyError 297 145 exceptions.KeyError 296 146 exceptions.RuntimeError maximum recursion depth exceeded 147 exceptions.KeyError 295 148 exceptions.RuntimeError maximum recursion depth exceeded 149 exceptions.KeyError 294 150 exceptions.RuntimeError maximum recursion depth exceeded 151 exceptions.RuntimeError maximum recursion depth exceeded 152 exceptions.RuntimeError maximum recursion depth exceeded 153 exceptions.KeyError 309 154 exceptions.RuntimeError maximum recursion depth exceeded 155 exceptions.RuntimeError maximum recursion depth exceeded 156 exceptions.KeyError 307 157 exceptions.RuntimeError maximum recursion depth exceeded 158 exceptions.KeyError 306 159 exceptions.RuntimeError maximum recursion depth exceeded 160 exceptions.KeyError 305 161 exceptions.RuntimeError maximum recursion depth exceeded 162 exceptions.KeyError 304 163 exceptions.RuntimeError maximum recursion depth exceeded 164 exceptions.KeyError 303 165 exceptions.KeyError 302 166 exceptions.RuntimeError maximum recursion depth exceeded ... and there's a seemingly non-random but hard-to-fathom jumble of recursion-depth and KeyError exceptions thereafter too. So something's really hosed in CVS, or in MSVC 7.1, or ... From tdelaney at avaya.com Tue Aug 10 00:40:59 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Tue Aug 10 00:41:05 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> Walter D?rwald wrote: > I can't understand why we can't have a new keyword for decorators. > If I remember correctly the introduction of yield didn't result > in such a public outcry. We'd have to change our programs once > if a variable names collides with the new keyword, but that's better > than having to look at @s for the rest of our Python days. Not such a large outcry, but lots of us were concerned about "having to look inside the function for 'yield' to determine if it were a generator or a function" and were calling for a 'gen' or 'generator' keyword. Strangely enough, exactly the *opposite* argument is now being used to prevent syntaxes such as: def func (args): @decorator as was used then to allow: def func (args): yield value Tim Delaney From aahz at pythoncraft.com Tue Aug 10 00:55:37 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Aug 10 00:55:40 2004 Subject: [Python-Dev] Python startup time? Message-ID: <20040809225537.GA23187@panix.com> I just remembered that shortly before release of Python 2.3 there was a big fuss about the startup time. Has anyone revisited the issue since then? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From mhammond at skippinet.com.au Tue Aug 10 01:14:50 2004 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue Aug 10 01:14:39 2004 Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows In-Reply-To: <20040807150229.GC2529@rogue.amk.ca> Message-ID: <05af01c47e66$b1ab6420$0200a8c0@eden> > OK; I just care about platforms where the alternative constant is > necessary. It sounds to me like win32 is therefore the only relevant > sys.platform string to check for. Thanks! I believe the Windows CE port uses "wince", and I see no reason not to think async code will end up running there (assuming the platform itself continues). Mark. From tdelaney at avaya.com Tue Aug 10 01:19:18 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Tue Aug 10 01:19:24 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CAC2@au3010avexu1.global.avaya.com> Guido van Rossum wrote: > Please remember the procedure. The community is to come up with *one* > alternative that they universally prefer over @decorators; otherwise, > @decorators-before-def will stay. (I'll even give you *two* if the > community can agree that both are acceptable and really wishes to let > me pick one.) And yet in the poll to gain support for one option, we've been actively discouraged by Anthony from participating on the grounds that "weight of opinion" is no good ... ;) (No - not an accusation - just a suggestion that the procedure is possibly not clear to all the major participants). Tim Delaney From nbastin at opnet.com Tue Aug 10 01:42:12 2004 From: nbastin at opnet.com (Nick Bastin) Date: Tue Aug 10 01:42:36 2004 Subject: [Python-Dev] Python startup time? In-Reply-To: <20040809225537.GA23187@panix.com> References: <20040809225537.GA23187@panix.com> Message-ID: On Aug 9, 2004, at 6:55 PM, Aahz wrote: > I just remembered that shortly before release of Python 2.3 there was a > big fuss about the startup time. Has anyone revisited the issue since > then? Experience from pycon sprint revealed several things to me: 1) 2.3 startup is faster than 2.2 if you don't import site 2) 2.2 didn't really import anything on startup, so it was always fast 3) Never benchmark without installing, as performance is really bad in CVS (distutils) If people who really care about startup time don't need site, then they're good. If they do, then the real slowdown comes from warnings and linecache, if I recall correctly. -- Nick From nbastin at opnet.com Tue Aug 10 01:45:06 2004 From: nbastin at opnet.com (Nick Bastin) Date: Tue Aug 10 01:45:14 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <4117F7D5.3090907@zope.com> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <4117F7D5.3090907@zope.com> Message-ID: <24A02222-EA5E-11D8-AAEA-000D932927FE@opnet.com> On Aug 9, 2004, at 6:16 PM, Jim Fulton wrote: > James Y Knight wrote: >> IMO classes ought to actually appear in __builtin__ if they claim >> they are defined there. Doing otherwise breaks reflection, as you >> have to add a special case for these class names to use the >> appropriate object from the types module instead. Thoughts? > > I agree. I think this should stand for functions as well. help() is a good example. It's a bit confusing for newbies that help.__module__ is 'site', but 'site' isn't in their local namespace. -- Nick From python at rcn.com Mon Aug 9 02:26:54 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 10 02:27:03 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae040809153946522f6d@mail.gmail.com> Message-ID: <007c01c47da7$92b71920$191fc797@oemcomputer> > and there's a seemingly non-random but hard-to-fathom jumble of > recursion-depth and KeyError exceptions thereafter too. > > So something's really hosed in CVS, or in MSVC 7.1, or ... This also occurs on MSVC++ 6.0. I'm working back in time and it still occurs on the 6/21/2004 checkout. That excludes my eval locals argument, Armin's string optimization, and Nick's C profiling re-org. Raymond From mkc at mathdogs.com Tue Aug 10 03:15:52 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Tue Aug 10 03:15:56 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <20040809033554.GA21340@panix.com> Message-ID: Aahz writes: > This section needs fleshing out to explain why re.structmatch works better > for some purposes than forcing people to switch to a full-blown parser. That's a good point. I spent a while thinking about this, but didn't document why I thought it was still worthwhile. (See my nearby reply to Stephen Turnbull for some thoughts.) Mike From tjreedy at udel.edu Tue Aug 10 03:18:14 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Tue Aug 10 03:18:18 2004 Subject: [Python-Dev] Decorator classification: location, format, details Message-ID: I found the recent discussion of decorators rather confusing until I noticed that objections and options to prefixed @deco fell into three categories: location, format, and details. Perhaps this structuring might help the debate. LOCATION (6 proposals/alternatives) Multiple def prefix - dedented relative to def and covering all indented under it. Prefix to single def - the alpha2 innovation. Infix in header, before or after function name. Suffix to header, after param list, before :. Top of body postfiX - the 2.3 status quo FORMAT The format depends somewhat but not completely on location, and there have been one to many proposed for each. By 'format' I here mean a symbolic template without literals (except possibly \n). In grammar terms, a production rhs without terminals. For alpha2 prefix, the format is, simplified, sequence of decos, where deco is pfix user_decorator. One alternative prefix form is open seq of user_decorators close. Another is a prefixed suite. For 2.3, the postfiX form is func_name = user_decorator(func_name). DETAILS These are the constant literals supplied by the designer (or grammar) rather than the symbolic variables filled in by the user/programmer. Grammatically, these are the terminal productions. For alpha2, pfix is '@'. Suggested alternatives include '=', '|', and 'deco'. I notice in the PEP, the wiki, and here that discussions of a particular option with all 3 aspects fixed tend to mix discussion of the pros and cons of its location, which actually apply to all formats in that location, and discussion the pros and cons of the format, which mostly would apply to the format in any position. I think greater separation would be less confusing. Is it possible to first pick a location, then a format, then fill in the details? Terry J. Reedy From greg at cosc.canterbury.ac.nz Tue Aug 10 03:27:52 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 10 03:27:58 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: <200408091525.i79FPLM23023@guido.python.org> Message-ID: <200408100127.i7A1RqAi007468@cosc353.cosc.canterbury.ac.nz> > My own gut feeling is that it's not important to allow parentheses on > "bare" (from-less) imports, since you can easily repeat the 'import' > keyword on a new line; It might not be strictly necessary, but for consistency I think it should be alllowed. By the way, rather than parentheses, I would prefer to be able to simply end the line with a comma and continue on to the next line: from Bubble import foo, spam, tomato, grail, arthur, camembert Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From aahz at pythoncraft.com Tue Aug 10 03:36:44 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Aug 10 03:36:46 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: <200408100127.i7A1RqAi007468@cosc353.cosc.canterbury.ac.nz> References: <200408091525.i79FPLM23023@guido.python.org> <200408100127.i7A1RqAi007468@cosc353.cosc.canterbury.ac.nz> Message-ID: <20040810013644.GB5503@panix.com> >> My own gut feeling is that it's not important to allow parentheses on >> "bare" (from-less) imports, since you can easily repeat the 'import' >> keyword on a new line; > > It might not be strictly necessary, but for consistency > I think it should be alllowed. Unless other support is forthcoming, this won't happen. > By the way, rather than parentheses, I would prefer to > be able to simply end the line with a comma and continue > on to the next line: > > from Bubble import foo, spam, tomato, > grail, arthur, camembert Rejected earlier on parsing grounds. I've updated the PEP. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From mkc at mathdogs.com Tue Aug 10 03:38:08 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Tue Aug 10 03:38:13 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: "Stephen J. Turnbull" writes: > >>>>> "Mike" == Mike Coleman writes: > Mike> m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz') > Sure, but regexp syntax is a horrible way to express that. Do you mean, horrible compared to spelling it out using a Python loop that walks through the array, or horrible compared to some more serious parsing package? For the former, I would disagree. I see code like this a lot and it drives me crazy. Reminds me of the bad old days of building 'while' loops out of 'if's and 'goto's. For the latter, I think it depends on the complexity of the matching, and the level of effort required to learn and distribute the "not-included" parsing package. I certainly wouldn't want to see someone try to write a language front-end with this, but for a lot of text-scraping activities, I think it would be very useful. > This feature would be an attractive nuisance, IMHO. I agree that, like list comprehensions (for example), it needs to be applied with good judgement. Turning it around, though, why *shouldn't* there be a good mechanism for returning the multiple matches for multiply matching groups? Why should this be an exception? If you agree that there should be a mechanism, it certainly doesn't have to be the one in the PEP, but what would be better? I'd welcome alternative ideas here. > Mike> p = r'((?:(?:^|:)([^:\n]*))*\n)*\Z' > > This is a _easy_ one, but even it absolutely requires being written > with (?xm) and lots of comments, don't you think? I think it's preferable--that's why I did it. :-) > If you're going to be writing a multiline, verbose regular expression, why > not write a grammar instead, which (assuming a modicum of library support) > will be shorter and self-documenting? If there were a suitable parsing package in the standard library, I agree that this would probably be a lot less useful. As things stand right now, though, it's a serious irritation that we have a standard mechanism that *almost* does this, but quits at the last moment. If I may wax anthropomorphic, the 're.match' function says to me as a programmer *You* know what structure this RE represents, and *I* know what structure it represents, too, because I had to figure it out to do the match. But too bad, sucker, I'm not going to tell you what I found! Irritating as hell. Mike From tim.peters at gmail.com Tue Aug 10 03:48:37 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 10 03:48:44 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <007c01c47da7$92b71920$191fc797@oemcomputer> References: <007c01c47da7$92b71920$191fc797@oemcomputer> Message-ID: <1f7befae04080918481e1f8053@mail.gmail.com> >> and there's a seemingly non-random but hard-to-fathom jumble of >> recursion-depth and KeyError exceptions thereafter too. >> >> So something's really hosed in CVS, or in MSVC 7.1, or ... [Raymond Hettinger] > This also occurs on MSVC++ 6.0. > > I'm working back in time and it still occurs on the 6/21/2004 checkout. > That excludes my eval locals argument, Armin's string optimization, and > Nick's C profiling re-org. Thank you for trying this stuff! You already ruled out my #1 guess, namely that the "catch an exception from alloca()" trick we use on Windows to detect impending stack overflow stopped working in 7.1. But if it sucks under 6.0 too ... Another possibility is that it's something new & evil in the CVS compiler package (when my driver does "import compiler" under 2.3.4, of course it picks up 2.3.4's compiler package). Do you also get no output at all under a 6.0 debug build? Has anyone on Linux tried this yet? From greg at cosc.canterbury.ac.nz Tue Aug 10 03:52:30 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 10 03:52:40 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: Message-ID: <200408100152.i7A1qUYD007511@cosc353.cosc.canterbury.ac.nz> > Worse still is the fact that the exponent in the e notation is a > decimal exponent, but the type used is binary! Indeed. If someone really wants a binary float, I think they should have to write it in hex. If they can't cope with that, they've no business using binary floats. <0xd37c*pow(2,-0x10) wink>, Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From d.holton at vanderbilt.edu Fri Aug 6 19:13:16 2004 From: d.holton at vanderbilt.edu (Doug Holton) Date: Tue Aug 10 03:58:56 2004 Subject: [Python-Dev] request: add keywords for static and class methods only [related to decorators discussion] Message-ID: <4113BC2C.1010206@vanderbilt.edu> After reading some posts here and the wiki page discussing the decorators: http://www.python.org/moin/PythonDecorators One idea that some people have suggested is to separate out standard uses for decorators like function metadata, vs. built-in features of python, like static methods and class methods. So I'm suggesting, don't use decorators for static and class methods. Perhaps for 2.5 or whenever, we should add keyword support for built-in features, just like java: These two keywords at least: def static mymethod (arg1, arg2): .... def classmethod mymethod( cls, arg1, arg2 ): .... And then you have to decide if you want to support other java built-in features and keywords like private, protected, public, synchronized. See the bottom of the wiki page for ideas for the long term future that combine these keywords with decorators and other possible features. From barnesc at engr.orst.edu Sat Aug 7 02:38:27 2004 From: barnesc at engr.orst.edu (barnesc@engr.orst.edu) Date: Tue Aug 10 03:59:00 2004 Subject: [Python-Dev] Another decorator syntax idea Message-ID: <1091839107.4114248325784@webmail.oregonstate.edu> Another decorator syntax idea Using Guido's example: -------------------- @ Symbols -------------------- class C(object): @staticmethod @funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", \ status="experimental", author="BDFL") def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42): """This method blah, blah. It supports the following arguments: - longArgumentOne -- a string giving ... - longArgumentTwo -- a number giving ... blah, blah. """ raise NotYetImplemented -------------------- decorate keyword -------------------- class C(object): def longMethodNameForEffect(longArgumentOne=None, longArgumentTwo=42): decorate: staticmethod funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", \ status="experimental", author="BDFL")] """This method blah, blah. It supports the following arguments: - longArgumentOne -- a string giving ... - longArgumentTwo -- a number giving ... blah, blah. """ raise NotYetImplemented Pros: - Doesn't hide the function name behind @ garbage. - 'decorate' is easier to look at than '@'. - 'decorate' is more meaningful to most readers than '@'. - Block structure isolates decorators from surrounding code, making them more readable. - Indentation rules are consistent with the rest of Python. Cons: - Not pear shaped. - Connelly From bokr at oz.net Sun Aug 8 01:51:58 2004 From: bokr at oz.net (Bengt Richter) Date: Tue Aug 10 03:59:04 2004 Subject: [Python-Dev] @decorator syntax is sugar, but for what exactly? Message-ID: <41153969.226164657@news.oz.net> ISTM that @limited_expression_producing_function @another def func(): pass is syntactic sugar for creating a hidden list of functions. (Using '|' in place of '@' doesn't change the picture much (except for people whose tools depend on '@' ;-)). I.e., (not having the source or time to delve) the apparent semantics of the above is something roughly like __funclist__ = [] __funclist__.append(limited_expression_producing_function) __funclist__.append(another) def func():pass while __funclist__: func = __funclist__.pop()(func) del __funclist__ Is this a special case of a more general idea? E.g., could it apply to right after ANY next name is bound, in general, not just a name bound by def? thus (untested) def censor(cls): cls.__repr__ = lambda self: '' return cls ... @censor class C(object): pass could have the expected effect (after metaclass effects, if any, presumably, BTW) (note that censor could instead e.g. wrap selected methods or add class variable data etc., though IIRC __metaclass__ can create some things that are read-only later) This is still very narrowly defined by prefix context. Is this context also a special case default of something more general? IOW the default choice for namespace is the lexically enclosing one. What about, e.g., being able to specify decoration in one place at the top of a module and decorate (in the same way, using the same function list) all methods of a specified (by name) list of classes? I.e., a more general control over what to do when what names are bound in what namespace could be envisaged. This begins to feel like event-driven processing. Could @deco1 @deco2 be sugar for the special case (don't take this literally, just illustrating semantics ;-) when(event='next_binding', namespace='immediate', symbols='any', funclist=(deco1,deco2)) def foo(): pass of something general that could also include other and possibly repeated events like on completion of an arg list for a particular function or method being called or any rebinding of particular symbols in a specified namespace (not just the immediate one), e.g., for debugging or profiling etc. Since I am copying this to python-dev, I'll re-ask whether @decorator has any peculiar thread safety pitfalls that should be warned against, just in case. Please take this as a probe into the intended semantics, not as a proposal for any particular functionality ;-) Regards, Bengt Richter From bsder at mail.allcaps.org Mon Aug 9 19:48:44 2004 From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.) Date: Tue Aug 10 03:59:07 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> Message-ID: <5B8ECC2E-EA2C-11D8-B30C-000A95C874EE@mail.allcaps.org> On Aug 9, 2004, at 10:29 AM, Mark Hahn wrote: > Forgive me if I'm being a pest, but no one has commented on the real > reason > I asked the question. What does everyone think of the idea of having > these > three built-in numeric types? > > 1) An Int implemented with infinite precision integer (Python Longs) > with > the constant/__str__ form of +-NN such as 0, -123, 173_394, etc. > > 2) A Decimal implemented with the .Net decimal float (or the IBM > decimal if > the .Net decimal sucks too much) with the constant/__str__ form of > +-NN.NN > such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. > > 3) A binary Float implemented with the hardware floating point with the > constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3, > 2.35e-78, etc. Be careful defining string representations of numbers. It is a non-trivial problem to adhere to the correct internationalization standards. Forgive me if I'm being dense, but don't we already have 1 & 3 as built in types? 2 is unlikely to be a builtin until folks get some experience using the Decimal module. Yes, the .Net Decimal sucks. In addition, the desired evolution of Python seems to be replacing the binary floating point as the default floating point type with the decimal floating point type by default rather than growing another builtin numeric type. This is obviously contingent upon the Decimal module getting some real-life use as well as it getting some speed improvements in the future. > There would be no automatic conversion except the / operator would > convert > from Int to Decimal Int goes to the default type of float (which is currently binary FP) already upon using the / operator. > and the Math module would convert Int and Decimal > values to Float for almost all functions (except simple ones like abs, > min, > max). This is not necessarily a good idea. Exponentials are very useful in financial situations and want to be decimals, not binary floats, in those cases. -a From greg at cosc.canterbury.ac.nz Tue Aug 10 04:00:55 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 10 04:00:59 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] In-Reply-To: Message-ID: <200408100200.i7A20tsq007540@cosc353.cosc.canterbury.ac.nz> > I have been assuming that there is a fundamental need for an integer > type that has no fraction. There are many places like indexing > where you semantically want integers. That doesn't necessarily mean you need an integer *type*. Indexing code can still reject numbers whose value isn't an integer. There may be other reasons for wanting an integer type, though. For instance, allowing integers to become abitrarily large doesn't usually cause any problem, whereas allowing the precision of fractional numbers to increase unboundedly can lead to unexpectedly high memory consumption and slowness. So you probably want arbitrary precision integers, but not arbitrary precision fractional numbers (at least by default). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From bob at redivi.com Tue Aug 10 04:10:37 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 10 04:10:45 2004 Subject: [Python-Dev] request: add keywords for static and class methods only [related to decorators discussion] In-Reply-To: <4113BC2C.1010206@vanderbilt.edu> References: <4113BC2C.1010206@vanderbilt.edu> Message-ID: <788747D0-EA72-11D8-B58B-000A95686CD8@redivi.com> On Aug 6, 2004, at 1:13 PM, Doug Holton wrote: > After reading some posts here and the wiki page discussing the > decorators: http://www.python.org/moin/PythonDecorators > One idea that some people have suggested is to separate out standard > uses for decorators like function metadata, vs. built-in features of > python, like static methods and class methods. > > So I'm suggesting, don't use decorators for static and class methods. > > Perhaps for 2.5 or whenever, we should add keyword support for > built-in features, just like java: Now that's a -1 if I've ever seen one. -bob From greg at cosc.canterbury.ac.nz Tue Aug 10 04:11:50 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 10 04:11:55 2004 Subject: [Python-Dev] Re: [Prothon-user] Re: Decimal type question [Prothon] (was: Re: Re: Decimal type question [Prothon]) In-Reply-To: <1pj8x4u2ekigy$.21gr6je8l10a$.dlg@40tude.net> Message-ID: <200408100211.i7A2BoqZ007551@cosc353.cosc.canterbury.ac.nz> > How about a basic built-in numeric default type called Decimal with all the > default numeric constant formats (123, 123.0, & 123e0) and a separate > binary double hardware type called Float with the constant format of 0f0, > 1f0, -1.3f-34, etc. where the letter "f" replaces the letter "e"? Since the base is the important distinction between these, if the decimal one is to be called Decimal, the binary one should be called Binary. The exponent character should be 'b' to reinforce this. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Tue Aug 10 04:17:55 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 10 04:18:18 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> Message-ID: <200408100217.i7A2Ht9X007573@cosc353.cosc.canterbury.ac.nz> > Strangely enough, exactly the *opposite* argument is now being used > to prevent syntaxes such as: > > def func (args): > @decorator > > as was used then to allow: > > def func (args): > yield value It's not quite the same argument, because 'yield' can appear anywhere inside the body, buried arbitrarily deeply. Decorators would only appear at the beginning. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Tue Aug 10 04:20:18 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 10 04:20:22 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] In-Reply-To: References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> <1f7befae04080911203b46485c@mail.gmail.com> Message-ID: <1f7befae040809192054372908@mail.gmail.com> [Mark Hahn] > ... > I'm already planning on using Jim's IronPython implementation of your > BigInts so I might as well port your Decimal code over for the Prothon > Decimal also. The only thing I've ever stolen from Python for Prothon was > your bigint code so I might as well be consistent in my thievery. :-) It may not be an easy port, as it freely uses lots of Python features, including Jim Fulton's new-in-2.4 thread-local storage API (just like the status and control registers in your HW FPU carry thread-local state, the IBM spec has thread-local status and control data too). A saving grace is that IBM's Mike Colishaw worked up thousands of exacting test cases distributed in a plain text file format, and so a broken implementation is remarkably easy to detect. From anthony at interlink.com.au Tue Aug 10 04:19:31 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 10 04:20:25 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CAC2@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CAC2@au3010avexu1.global.avaya.com> Message-ID: <411830B3.3050809@interlink.com.au> Delaney, Timothy C (Timothy) wrote: > And yet in the poll to gain support for one option, we've been actively > discouraged by Anthony from participating on the grounds that "weight of > opinion" is no good ... ;) Please don't misinterpret what I wrote. I wrote, on python-list/c.l.py that merely "collecting votes" was not a useful task - collecting useful arguments _is_. Many of the posts here (and even more so on c.l.py) are simply stating and restating the same arguments, or else people posting "I hate form blah". In the useful case, I'd point to the discussion on "|" vs "@". People are working through the issues. > (No - not an accusation - just a suggestion that the procedure is > possibly not clear to all the major participants). The goal here is to attempt to reach a concensus. -- Anthony Baxter It's never too late to have a happy childhood. From python at rcn.com Tue Aug 10 04:30:00 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 10 04:30:08 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] In-Reply-To: <1f7befae040809192054372908@mail.gmail.com> Message-ID: <001a01c47e81$efb03500$2e05a044@oemcomputer> > It may not be an easy port, as it freely uses lots of Python features, > including Jim Fulton's new-in-2.4 thread-local storage API FWIW, I made that part optional by putting in backwards compatibility code. I periodically verify that it runs fine under Py2.3. Raymond From greg at cosc.canterbury.ac.nz Tue Aug 10 04:44:21 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 10 04:44:47 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <41175442.4010902@livinglogic.de> Message-ID: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz> =?ISO-8859-1?Q?Walter_D=F6rwald?= : > I can't understand why we can't have a new keyword for decorators. > If I remember correctly the introduction of yield didn't result > in such a public outcry. The problem isn't so much the mechanics of adding a keyword, but choosing which keyword to use. The expected uses of decorators are so diverse that no single word would sound right for all of them. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From foom at fuhm.net Tue Aug 10 04:48:37 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 10 04:48:42 2004 Subject: [Python-Dev] @decorator syntax is sugar, but for what exactly? In-Reply-To: <41153969.226164657@news.oz.net> References: <41153969.226164657@news.oz.net> Message-ID: On Aug 7, 2004, at 7:51 PM, Bengt Richter wrote: > ISTM that > @limited_expression_producing_function > @another > def func(): pass > > is syntactic sugar for creating a hidden list of functions. (Using '|' > in place of '@' > doesn't change the picture much (except for people whose tools depend > on '@' ;-)). As I understand it, it's really syntax sugar for: func = limited_expression_producing_function(another( defbutdontbind func(): pass)) except that isn't actually valid python source code. ;) It's nearly, but not exactly, the same as def func(): pass func = another(func) func = limited_expression_producing_function(func) > Is this a special case of a more general idea? E.g., could it apply to > right after ANY next name is bound, in general, not just a name bound > by def? Right now decorators only work before def. I posted an idea earlier about a @public decorator, and wrote a quick proof-of-concept patch to allow decorators in front of funcdef, classdef, and "NAME '=' testlist". This much *can* work with little code change, if the functionality is deemed as desirable. However, as I mentioned, @public in front of a arbitrary binding expression (e.g. not class or function) isn't useful unless the decorator gets the name of the variable being bound. I didn't quite figure out how to manage that part. :) James From skip at pobox.com Tue Aug 10 05:00:21 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 10 05:00:43 2004 Subject: [Python-Dev] Decorators: vertical bar syntax In-Reply-To: <019301c47db9$c269ec80$a9498890@neil> References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> <019301c47db9$c269ec80$a9498890@neil> Message-ID: <16664.14917.365982.249725@montanaro.dyndns.org> Neil> A motivation for this was that one of the examples Neil> @accepts(int,int) Neil> @returns(float) Neil> def bar(low,high): Neil> which separates related information too much for me. Yes, but when I wrote that example, the list-after-def form was the current king-of-the-hill: def bar(low,high) [accepts(int,int), returns(float)]: ... Looks fine to me. ;-) Skip From foom at fuhm.net Tue Aug 10 05:16:39 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 10 05:16:41 2004 Subject: [Python-Dev] Decorator order implemented backwards? Message-ID: I haven't seen anyone remark upon this yet, but the order of decorator application appears to be implemented backwards. According to the PEP: > Proposed Syntax > > The current syntax for function decorators as implemented in Python > 2.4a2 is: > @dec2 > @dec1 > def func(arg1, arg2, ...): > pass > > > This is equivalent to: > def func(arg1, arg2, ...): > pass > func = dec2(dec1(func)) However, it seems the '@' form is really doing dec1(dec2(func)), not dec2(dec1(func)). The test_decorators.py reveals that it is, indeed, testing *for* the wrong behavior. test_decorators.py: > def test_order(self): > class C(object): > @funcattrs(abc=1) @staticmethod > def foo(): return 42 > # This wouldn't work if staticmethod was called first > self.assertEqual(C.foo(), 42) > self.assertEqual(C().foo(), 42) So, um...I assume this is a bug in the implementation, not the PEP, because the PEP's way makes sense and the way it's implemented doesn't. James From python at rcn.com Tue Aug 10 05:17:02 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 10 05:17:09 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae04080918481e1f8053@mail.gmail.com> Message-ID: <000001c47e88$81ef1d40$5635c797@oemcomputer> > >> and there's a seemingly non-random but hard-to-fathom jumble of > >> recursion-depth and KeyError exceptions thereafter too. > >> > >> So something's really hosed in CVS, or in MSVC 7.1, or ... > > [Raymond Hettinger] > > This also occurs on MSVC++ 6.0. > > > > I'm working back in time and it still occurs on the 6/21/2004 checkout. > > That excludes my eval locals argument, Armin's string optimization, and > > Nick's C profiling re-org. > > Thank you for trying this stuff! You already ruled out my #1 guess, > namely that the "catch an exception from alloca()" trick we use on > Windows to detect impending stack overflow stopped working in 7.1. > But if it sucks under 6.0 too ... Yes, the usual suspects are innocent. > Another possibility is that it's something new & evil in the CVS > compiler package (when my driver does "import compiler" under 2.3.4, > of course it picks up 2.3.4's compiler package). > > Do you also get no output at all under a 6.0 debug build? I've gone back to Jan 15, 2004 and can still reproduce the error on a release build. Haven't made a debug build yet. BTW, if you try this, remember, the opcodes were difference, so do a full rebuild and clear away the pyc files. Raymond From mark at prothon.org Tue Aug 10 05:24:00 2004 From: mark at prothon.org (Mark Hahn) Date: Tue Aug 10 05:23:53 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> <5B8ECC2E-EA2C-11D8-B30C-000A95C874EE@mail.allcaps.org> Message-ID: On Mon, 9 Aug 2004 10:48:44 -0700, Andrew P. Lentvorski, Jr. wrote: > On Aug 9, 2004, at 10:29 AM, Mark Hahn wrote: > >> Forgive me if I'm being a pest, but no one has commented on the real >> reason >> I asked the question. What does everyone think of the idea of having >> these >> three built-in numeric types? >> >> 1) An Int implemented with infinite precision integer (Python Longs) >> with >> the constant/__str__ form of +-NN such as 0, -123, 173_394, etc. >> >> 2) A Decimal implemented with the .Net decimal float (or the IBM >> decimal if >> the .Net decimal sucks too much) with the constant/__str__ form of >> +-NN.NN >> such as 0.0, -123.0, 173_394.0, 173394.912786765, etc. >> >> 3) A binary Float implemented with the hardware floating point with the >> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3, >> 2.35e-78, etc. > > Be careful defining string representations of numbers. It is a > non-trivial > problem to adhere to the correct internationalization standards. > > Forgive me if I'm being dense, but don't we already have 1 & 3 as built > in types? There is no "we". This thread is about Prothon which has nothing yet since it hasn't ever been released. Your feedback is still relevant and helpful though. > 2 is unlikely to be a builtin until folks get some experience using the > Decimal module. Yes, the .Net Decimal sucks. > > In addition, the desired evolution of Python seems to be replacing the > binary floating point as the default floating point type with the > decimal floating point type by default rather than growing another > builtin numeric type. This is obviously contingent upon the Decimal > module > getting some real-life use as well as it getting some speed improvements > in the future. > >> There would be no automatic conversion except the / operator would >> convert >> from Int to Decimal > > Int goes to the default type of float (which is currently binary FP) > already > upon using the / operator. > >> and the Math module would convert Int and Decimal >> values to Float for almost all functions (except simple ones like abs, >> min, >> max). > > This is not necessarily a good idea. Exponentials are very useful in > financial situations and want to be decimals, not binary floats, in > those > cases. Good point. From tdelaney at avaya.com Tue Aug 10 05:26:57 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Tue Aug 10 05:27:04 2004 Subject: [Python-Dev] Decorator order implemented backwards? Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF3D1F@au3010avexu1.global.avaya.com> James Y Knight wrote: > So, um...I assume this is a bug in the implementation, not the PEP, > because the PEP's way makes sense and the way it's implemented > doesn't. Especially since it will be most common to put @staticmethod or @classmethod first, which will play havoc with any decorator following that is expecting a function object ... Tim Delaney From barry at python.org Tue Aug 10 05:29:38 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 10 05:29:25 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz> References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz> Message-ID: <1092108578.18208.10.camel@localhost> On Mon, 2004-08-09 at 22:44, Greg Ewing wrote: > =?ISO-8859-1?Q?Walter_D=F6rwald?= : > > > I can't understand why we can't have a new keyword for decorators. > > If I remember correctly the introduction of yield didn't result > > in such a public outcry. > > The problem isn't so much the mechanics of adding a keyword, > but choosing which keyword to use. The expected uses of > decorators are so diverse that no single word would sound > right for all of them. Actually, that's an interesting case study. Myself and others advocated for a keyword other than 'def' for introducing generators. That wasn't how things turned out and in hindsight I think Guido made the right decision. I'm confident the same thing will happen with decorators. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/ca58ff72/attachment-0001.pgp From tim.peters at gmail.com Tue Aug 10 06:06:16 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 10 06:06:20 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <1092108578.18208.10.camel@localhost> References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz> <1092108578.18208.10.camel@localhost> Message-ID: <1f7befae040809210679c2d4a0@mail.gmail.com> [Barry Warsaw] > Actually, that's an interesting case study. Myself and others advocated > for a keyword other than 'def' for introducing generators. That wasn't > how things turned out and in hindsight I think Guido made the right > decision. I'm confident the same thing will happen with decorators. Ya, but it's sure taking Guido long enough to figure that out in this case! def staticmethod: def author(name='Barry'): def returns([int]): def method(self): return 42 a-keyword-for-all-seasons-ly y'rs - tim From martin at v.loewis.de Tue Aug 10 06:26:05 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 10 06:26:04 2004 Subject: [Python-Dev] Python startup time? In-Reply-To: <20040809225537.GA23187@panix.com> References: <20040809225537.GA23187@panix.com> Message-ID: <41184E5D.9090002@v.loewis.de> Aahz wrote: > I just remembered that shortly before release of Python 2.3 there was a > big fuss about the startup time. Has anyone revisited the issue since > then? As Nick says, we looked at it at PyCon. I found that a lot of calls to string interning are made, and changed the marshal format to share interned strings, instead of duplicating them. This causes good savings in disk space; whether it also saves startup time significantly is hard to prove. Regards, Martin From anthony at interlink.com.au Tue Aug 10 07:00:27 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 10 07:06:05 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: References: Message-ID: <4118566B.2090907@interlink.com.au> James Y Knight wrote: > I haven't seen anyone remark upon this yet, but the order of decorator > application appears to be implemented backwards. >> def test_order(self): >> class C(object): >> @funcattrs(abc=1) @staticmethod >> def foo(): return 42 >> # This wouldn't work if staticmethod was called first >> self.assertEqual(C.foo(), 42) >> self.assertEqual(C().foo(), 42) How odd. I'm positive at one point I tested this and it was the right way round. >>> def dec1(x): ... print "dec1" ... return x ... >>> def dec2(x): ... print "dec2" ... return x ... >>> >>> @dec1 ... @dec2 ... def foo(): pass ... dec1 dec2 >>> This is definitely wrong. Wrong wrong wrong. Anthony From anthony at interlink.com.au Tue Aug 10 07:08:20 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 10 07:08:53 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: <4118566B.2090907@interlink.com.au> References: <4118566B.2090907@interlink.com.au> Message-ID: <41185844.80906@interlink.com.au> Anthony Baxter wrote: > James Y Knight wrote: > >> I haven't seen anyone remark upon this yet, but the order of decorator >> application appears to be implemented backwards. >> >>> def test_order(self): >>> class C(object): >>> @funcattrs(abc=1) @staticmethod >>> def foo(): return 42 >>> # This wouldn't work if staticmethod was called first >>> self.assertEqual(C.foo(), 42) >>> self.assertEqual(C().foo(), 42) > > > How odd. I'm positive at one point I tested this and it was > the right way round. Mark, is it possible that this changed between the first and final versions of the decorator patch? The SF report doesn't list any of the older versions... Thanks, Anthony From guido at python.org Tue Aug 10 07:23:04 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 10 07:23:12 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: Your message of "Tue, 10 Aug 2004 13:26:57 +1000." <338366A6D2E2CA4C9DAEAE652E12A1DE01CF3D1F@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF3D1F@au3010avexu1.global.avaya.com> Message-ID: <200408100523.i7A5N4Z24708@guido.python.org> > > So, um...I assume this is a bug in the implementation, not the PEP, > > because the PEP's way makes sense and the way it's implemented > > doesn't. > > Especially since it will be most common to put @staticmethod or > @classmethod first, which will play havoc with any decorator following > that is expecting a function object ... Um, right, this is backwards from what I had wanted. Sorry, I should've caught that before 2.4a2 went out! --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Aug 10 07:52:43 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 10 07:52:50 2004 Subject: [Python-Dev] @decorator syntax is sugar, but for what exactly? In-Reply-To: <41153969.226164657@news.oz.net> Message-ID: <000001c47e9e$417c1540$5635c797@oemcomputer> > -----Original Message----- > From: python-dev-bounces+python=rcn.com@python.org [mailto:python-dev- [Bengt Richter] > Since I am copying this to python-dev, I'll re-ask whether @decorator has > any peculiar > thread safety pitfalls that should be warned against, just in case. I would think that no thread safety issues would arise unless there were a stateful decorator. Raymond From fredrik at pythonware.com Tue Aug 10 08:12:31 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 10 08:10:58 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: Mike Coleman wrote: > If I may wax anthropomorphic, the 're.match' function says to me as a programmer > > *You* know what structure this RE represents, and *I* know what > structure it represents, too, because I had to figure it out to > do the match. that only shows that you dont understand how regular expressions work. a regular expression defines a set of strings, and the RE engine is designed to tell you if a string you have is a member of this set. the engine's not a parser, and it has a very vague notion of "structure" (groups are implemented by "marks", which are registers that keeps track of where the engine last passed a given position; changing that to "lists of all possible matches" would require a major rewrite). you're probably better off using the scanner mechanism: http://effbot.org/zone/xml-scanner.htm or the sre.Scanner class (see the source code). the scanner concept could need some documentation, and it would be nice to be able to switch patterns during parsing. (time for a scanner PEP, anyone?) From ncoghlan at iinet.net.au Tue Aug 10 08:35:04 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Aug 10 08:35:17 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <000001c47e88$81ef1d40$5635c797@oemcomputer> References: <000001c47e88$81ef1d40$5635c797@oemcomputer> Message-ID: <41186C98.9050604@iinet.net.au> Raymond Hettinger wrote: > I've gone back to Jan 15, 2004 and can still reproduce the error on a > release build. Haven't made a debug build yet. BTW, if you try this, > remember, the opcodes were difference, so do a full rebuild and clear > away the pyc files. Thank you Raymond! You just solved my problem with the totally psychotic behaviour I was getting after building with the free toolkit. I must have had some broken .pyc's lying around, and they were sending ceval.c insane. . . Now maybe I can start being useful around here :) Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From fredrik at pythonware.com Tue Aug 10 08:42:25 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 10 08:40:50 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu><1092074454.8687.26.camel@localhost> Message-ID: David Eppstein wrote: > I think quoted string recognition in Python is still regular, not > context-free. Anyway, the syntax highlighters in editors that I've used > don't seem to have a problem with it. shouldn't a simple parenthesis counter suffice? you have to count parens in many cases anyway (e.g. to find function bodies). From tim.peters at gmail.com Tue Aug 10 08:48:15 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 10 08:48:18 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <000001c47e88$81ef1d40$5635c797@oemcomputer> References: <000001c47e88$81ef1d40$5635c797@oemcomputer> Message-ID: <1f7befae0408092348583f0953@mail.gmail.com> Well, this gets nasty. In a debug build, and starting the loop at 0, I can't get off the gound in the MS 7.1 debugger. It dies quickly with an access violation in the bowels of ntdll.dll, and I don't have source for that. PyOS_CheckStack on Windows does this to detect stack overflow (it catches an MS exception if the C runtime can't allocate enough room on the stack): alloca(PYOS_STACK_MARGIN * sizeof(void*)); It's trying to see whether there's still room for 2K pointers on the stack. If I multiply that by 2, or by 3, nothing changes. But if I multiply it by 4, everything changes. Then the "oops! we're gonna blow the stack!" exit from PyOS_CheckStack is taken. It returns 1 to _Py_CheckRecursiveCall, which sets a "stack overflow" MemoryError and returns -1 to its caller. That's if (Py_EnterRecursiveCall(" in cmp")) return NULL; in PyObject_RichCompare. That's just trying to compare two ints. So NULL gets returned to PyObject_RichCompareBool, which in turn returns -1 to lookdict. AAAARGHGH! lookdict "isn't allowed" to raise exceptions, so it does a PyErr_Clear(), goes on to futilely chase the entire dict looking for another match on the hash code, and we've effectively turned a MemoryError into a KeyError. I expect that explains a lot about what we see in the release-build runs. If I multiply the stack check by 20, I can finally get some results out of the debug build: 0 exceptions.KeyError 299 1 exceptions.MemoryError Stack overflow 2 exceptions.MemoryError Stack overflow 3 exceptions.MemoryError Stack overflow 4 exceptions.KeyError 295 5 exceptions.MemoryError Stack overflow 6 exceptions.KeyError 294 7 exceptions.MemoryError Stack overflow 8 exceptions.MemoryError Stack overflow 9 exceptions.MemoryError Stack overflow 10 exceptions.MemoryError Stack overflow 11 exceptions.MemoryError Stack overflow 12 exceptions.MemoryError Stack overflow 13 exceptions.MemoryError Stack overflow 14 exceptions.KeyError 309 15 exceptions.KeyError 296 ... So we're blowing the C stack left and right in this test case, and sometimes dict lookup turns that into a KeyError. The question is what we did since 2.3.4 that apparently increases our stack demands, and grossly increases them in a debug build(!). Could be that the compile package is more heavily recursive now too (no idea). test_parser.py in 2,3.4 contained the same deeply nested tuples, so that's not what changed. Back in a release build, and restoring the original Windows stack-check code, but leaving the driver loop starting at 0, I have to sys.setrecursionlimit(16) to avoid getting any KeyErrors. sys.setrecursionlimit(878) is the minimum that allows at least one "ok" to show up: 0 ok 1 exceptions.RuntimeError maximum recursion depth exceeded 2 exceptions.RuntimeError maximum recursion depth exceeded 3 exceptions.RuntimeError maximum recursion depth exceeded 4 exceptions.KeyError 307 5 exceptions.RuntimeError maximum recursion depth exceeded 6 exceptions.KeyError 306 7 exceptions.RuntimeError maximum recursion depth exceeded 8 exceptions.KeyError 305 ... Misc/find_recursionlimit.py in CVS manages to print Limit of 1000 is fine before it craps out in a release build; in a debug build, it doesn't produce *any* output. If I change the limit it starts with to 100, it manages to get to Limit of 400 is fine in a debug build before stopping without a clue. Hmm! But over in 2.3.4 build, a release build also stopped with 1000, and a debug build also exited mysteriously. But after reducing its starting point to 100, it got to Limit of 700 is fine before crapping out. BTW, in 2.3.4 and CVS, when a debug run craps out mysteriously like this, it has an exit code of 128. That's scary: http://support.microsoft.com/support/kb/articles/q184/8/02.asp From eppstein at ics.uci.edu Tue Aug 10 09:15:35 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Tue Aug 10 09:15:40 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: In article , "Fredrik Lundh" wrote: > > I think quoted string recognition in Python is still regular, not > > context-free. Anyway, the syntax highlighters in editors that I've used > > don't seem to have a problem with it. > > shouldn't a simple parenthesis counter suffice? you have to count > parens in many cases anyway (e.g. to find function bodies). I think it would suffice (with some care to ignore parens in comments and strings, and assuming there aren't any mismatches), but counting is not something you can do in a regexp. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From fredrik at pythonware.com Tue Aug 10 09:34:08 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 10 09:32:31 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax References: <41145C05.1020501@gradient.cis.upenn.edu> Message-ID: David Eppstein wrote: >> shouldn't a simple parenthesis counter suffice? you have to count >> parens in many cases anyway (e.g. to find function bodies). > > I think it would suffice (with some care to ignore parens in comments > and strings, and assuming there aren't any mismatches), but counting is > not something you can do in a regexp. no, but adding a single integer register to the scanning loop isn't that much work. From Paul.Moore at atosorigin.com Tue Aug 10 10:21:06 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Tue Aug 10 10:21:11 2004 Subject: [Python-Dev] Decorator order implemented backwards? Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB386@UKDCX001.uk.int.atosorigin.com> From: James Y Knight > So, um...I assume this is a bug in the implementation, > not the PEP, because the PEP's way makes sense and the > way it's implemented doesn't. Interestingly, I found the PEP's description felt "backwards" to me, but didn't mention it because I didn't want to start another long discussion over something which has proved to be subjective in past discussions. I'd vote for correcting the PEP rather than the code (not least because it's probably easier :-)) But consistency is the only real requirement here. Paul. __________________________________________________________________________ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. __________________________________________________________________________ From fredrik at pythonware.com Tue Aug 10 11:00:08 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 10 11:00:17 2004 Subject: [Python-Dev] Re: re.split on empty patterns References: <20040807145142.GB2529@rogue.amk.ca> <1f7befae040807105878b47c61@mail.gmail.com> Message-ID: Tim Peters wrote: > > Anyway, we therefore can't just make this the default in 2.4. We > > could trigger a warning when emptyok is not supplied and a split > > pattern results in a zero-length match; users could supply > > emptyok=False to avoid the warning. Patterns that never have a > > zero-length match would never get the warning. 2.5 could then set > > emptyok to True. > > > > Note: raising the warning might cause a serious performance hit for > > patterns that get zero-length matches a lot, which would make 2.4 > > slower in certain cases. > > If you don't intend to change the default, there's no problem. I like > "no problem". This isn't so useful so often that it can't afford to > wait for Python 3 to change. In the meantime, "emptyok" is an odd > name since it's always "ok" to have an empty match. "split0=True" > reads better to me, since the effect is to split on a 0-length match. > split_on_empty_match would be too wordy. the RE module already have an established method for specifying flags, either with (?x) markers in the pattern strings, or via the "flags" argument to re.compile and others. any reason why you cannot use a new standard flag for this purpose? (e.g. (?e) and re.EMPTY_OK) adding a flags argument to re.split is less important; if someone really needs this feature, they can use (?e) or re.compile(..., re.EMPTY_OK).split() for extra credit, add EMPTY_OK behaviour to find, finditer, and sub/subn. From gmccaughan at synaptics-uk.com Tue Aug 10 11:47:50 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue Aug 10 11:48:22 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] In-Reply-To: <22FD7E0D-EA3B-11D8-B417-000A95C874EE@mail.allcaps.org> References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net> <22FD7E0D-EA3B-11D8-B417-000A95C874EE@mail.allcaps.org> Message-ID: <200408101047.50275.gmccaughan@synaptics-uk.com> On Monday 2004-08-09 20:34, Andrew P. Lentvorski, Jr. wrote: > > On Aug 9, 2004, at 11:31 AM, Paul Moore wrote: > > > I think that having 2 different fractional types distinguished by > > something as subtle as the presence of an exponent (something no other > > language does to my knowledge) is an extremely bad idea. > > Doesn't Common Lisp use 1.0e+10 to signify singles and 1.0d+10 to > signify doubles? > > That doesn't make it a good idea, but there is precedent. What CL does is a bit hairier than that. CL has, potentially, *four* different float formats. (An implementation is allowed to coalesce some or all of them.) s,f,d,l are used as exponent letters for short, single, double, and long floats, respectively; e means "use the default". The default default is single-float, but users can change that if they prefer (say) doubles. (I'm sure that if CL were being designed now rather than N years ago double-float would be the default default.) At least one implementation known to me (CLISP) does have four different kinds of float. CLISP's long-floats have user-settable precision, which can be as big as you like, which makes CLISP a very handy platform for some kinds of numerical experiments... Fortran also distinguishes between 1.0e10 and 1.0d10, in much the same way. In Fortran 90 and later, where there's potentially an even greater plethora of floating types than in CL, there's a more general mechanism for specifying the type of a floating-point constant. It's very nasty. Floating-point types are represented as integers. You define a parameter (i.e., a variable that isn't allowed to change) that holds the type you're interested in, and then you suffix your constants with "_" plus the parameter name. Thus: integer, parameter my_real_type = selected_real_kind(15,100) real(my_real_type) x x = 1.234_my_real_type It's not clear to me that any of this is very valuable in a statically-typed language without type inference: the type that a constant should be is almost always deducible from context. (More precisely: there's always a way to deduce a type that's almost always the one wanted.) -- g From skip at pobox.com Tue Aug 10 14:55:53 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 10 14:56:10 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <2m7js83733.fsf@starship.python.net> References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> Message-ID: <16664.50649.835422.998574@montanaro.dyndns.org> >> Is there a plan for implementing a base class for int and long (like >> basestring for str and unicode): Michael> Not that I'm aware of. We've discussed the numeric type hierarchy in the past. I think it would be worth considering all the numeric types at once and come up with a reasonable hierarchy that includes float, complex and decimal types as well. (It's clear there is no "best" way to do that.) Skip From skip at pobox.com Tue Aug 10 14:56:09 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 10 14:56:29 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <41175442.4010902@livinglogic.de> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> Message-ID: <16664.50665.170916.600964@montanaro.dyndns.org> Walter> So how about: Walter> make classmethod Walter> def foo(cls, bar): Walter> ... I don't think anybody's come up with a candidate keyword that reads well in most/all situations. Consider: make accepts(int, (int, float)) make returns(int) def foo(arg1, arg2): return arg1 * arg2 While you can contort "make" to mean what you want in this case it's clearly a much different meaning ("ensure that (or force) this function to accept and return these types) than "make classmethod" ("turn this function into a class method"). Other potential keywords have similar problems. Skip From skip at pobox.com Tue Aug 10 14:56:27 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 10 14:56:40 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> <6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se> Message-ID: <16664.50683.86050.621985@montanaro.dyndns.org> >> I think "adorn" would make a better keyword in that case, as in >> "adorn with" this decorator the method foo. Bob> I don't think that's appropriate at all. Most decorators are Bob> transformations that change the function object into something else Bob> entirely, most likely by wrapping it but quite possibly not. Correctamundo. In fact, there's no need that the decorator return a function (or callable) object at all. You could define a function to compute a lookup table of some sort. The decorator might simply invoke the function and return its value. Perfectly legal. Slightly more than adornment. Somewhat silly example: def define(func): return func() @ define def grayscale(): table = [] for i in range(256): table.append((i,i,i)) return table Skip From erik at heneryd.com Tue Aug 10 14:57:10 2004 From: erik at heneryd.com (Erik Heneryd) Date: Tue Aug 10 14:57:15 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching In-Reply-To: References: <411429FB.6080603@gradient.cis.upenn.edu> Message-ID: <4118C626.4070203@heneryd.com> Mike Coleman wrote: > Paul Moore writes: >>The only other comment I have is that the semantics seem pretty >>complex - I think that in practice, they do more or less what you want >>them to, but the description is pretty obscure. I'd say that is probably a warning flag, but I don't know, at the same time it looks like it could be kind of useful. >>And although I can see that the error return has some value, I suspect that >>it might actually complicate real use. > > > Someone else suggested throwing an exception instead, which I'm now leaning in > favor of, too. Exceptions are the way to go. You should get what you ask for, or a heads up. Errors should never pass silently. > > >>A suggestion - would it be possible to implement re.structmatch as a >>pure Python prototype, to thrash out some of the usability questions? >>If the function appears valuable in such a form, arguing for >>incorporation into the re module would be a lot easier. > > > I suspect that in this case a pure Python implementation might actually be a > lot harder than implementing it as a C patch. This is because so much of the > work is already done in the current re module; the C patch would probably not > be that big. In order to do it in Python I'd end up reimplementing re.match > in Python, which I wouldn't look forward to. I hacked up a Python prototype, mostly as a challenge. It's brittle and it's ugly, but it kind of works. Though, if /F says it would require a major rewrite of the C-engine, that's probably true... Erik -------------- next part -------------- A non-text attachment was scrubbed... Name: structmatch.py Type: text/x-python Size: 5760 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/a5e602ab/structmatch.py From erik at heneryd.com Tue Aug 10 15:00:50 2004 From: erik at heneryd.com (Erik Heneryd) Date: Tue Aug 10 15:00:56 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching In-Reply-To: References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <4118C702.8020707@heneryd.com> Fredrik Lundh wrote: > a regular expression defines a set of strings, and the RE engine is designed to > tell you if a string you have is a member of this set. the engine's not a parser, Well, as you know (and point out now and then), regexes aren't regular expressions in the CS sense. And most people don't use em just to see if a string matches or not, but to see what parts matches waht. > you're probably better off using the scanner mechanism: > > http://effbot.org/zone/xml-scanner.htm > > or the sre.Scanner class (see the source code). the scanner concept could > need some documentation, and it would be nice to be able to switch patterns > during parsing. (time for a scanner PEP, anyone?) I'd love to see scanners as an official part of the stdlib. Isn't this just a matter of documentation? Erik From fredrik at pythonware.com Tue Aug 10 15:06:36 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 10 15:06:41 2004 Subject: [Python-Dev] Re: Re: pre-PEP: Complete, Structured Regular ExpressionGroup Matching References: <411429FB.6080603@gradient.cis.upenn.edu> <4118C626.4070203@heneryd.com> Message-ID: Erik Heneryd wrote: > I hacked up a Python prototype, mostly as a challenge. It's brittle and > it's ugly, but it kind of works. Though, if /F says it would require a > major rewrite of the C-engine, that's probably true... if done on the C level, that is. partial compilation on the Python level is slightly easier, as you've just shown. clever! (but probably rather slow; I still recommend building something on top of the scanner object) From fredrik at pythonware.com Tue Aug 10 15:08:53 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 10 15:10:31 2004 Subject: [Python-Dev] Re: Re: Re: Call for defense of @decorators References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost><4113D5E1.6040006@interlink.com.au><41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> <16664.50665.170916.600964@montanaro.dyndns.org> Message-ID: Skip Montanaro wrote: > I don't think anybody's come up with a candidate keyword that reads well in > most/all situations. Consider: > > make accepts(int, (int, float)) > make returns(int) > def foo(arg1, arg2): > return arg1 * arg2 how about "at", pronounced "annotation type" ? (duck) From lists at hlabs.spb.ru Tue Aug 10 19:25:07 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Tue Aug 10 15:19:10 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <16664.50649.835422.998574@montanaro.dyndns.org> References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> <16664.50649.835422.998574@montanaro.dyndns.org> Message-ID: <411904F3.1080703@hlabs.spb.ru> Skip Montanaro wrote: > >> Is there a plan for implementing a base class for int and long (like > >> basestring for str and unicode): > > Michael> Not that I'm aware of. > > We've discussed the numeric type hierarchy in the past. I think it would be > worth considering all the numeric types at once and come up with a > reasonable hierarchy that includes float, complex and decimal types as well. > (It's clear there is no "best" way to do that.) Something like following? object numeric integer int long float complex PS I've already have implementation for baseinteger, but can't made a patch since cvs.python.sourceforge.net always told me 'Connection timed out'... :-/ -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From jepler at unpythonic.net Tue Aug 10 15:26:13 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Tue Aug 10 15:26:27 2004 Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows In-Reply-To: <20040807142327.GA2529@rogue.amk.ca> References: <20040807142327.GA2529@rogue.amk.ca> Message-ID: <20040810132612.GB28542@unpythonic.net> Do any systems *but* win32 have SO_EXLUSIVEADDRUSE ? (and if they do, is it better than SO_REUSEADDR for the same reasons it is on win32?) .. so why not write if hasattr(socket, 'SO_EXCLUSIVEADDRUSE'): reuse_constant = socket.SO_EXCLUSIVEADDRUSE else: reuse_constant = socket.SO_REUSEADDR SO_EXCLUSIVEADDRUSE sounds like the exact opposite of SO_REUSEADDR, according to a page I found when trying to find out whether any other platform has SO_EXCLUSEIVEADDRUSE: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp .. SO_REUSEADDR's semantics on Windows are apparently different from Unix, but SO_EXCLUSEIVEADDRUSE is different still. I'm no win32-platform programmer, just trying to make sense of what I read on the web. Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/2fd33ee5/attachment.pgp From walter at livinglogic.de Tue Aug 10 15:28:23 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Aug 10 15:28:34 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <16664.50665.170916.600964@montanaro.dyndns.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> <16664.50665.170916.600964@montanaro.dyndns.org> Message-ID: <4118CD77.30204@livinglogic.de> Skip Montanaro wrote: > Walter> So how about: > > Walter> make classmethod > Walter> def foo(cls, bar): > Walter> ... > > I don't think anybody's come up with a candidate keyword that reads well in > most/all situations. Consider: > > make accepts(int, (int, float)) > make returns(int) > def foo(arg1, arg2): > return arg1 * arg2 > > While you can contort "make" to mean what you want in this case it's clearly > a much different meaning ("ensure that (or force) this function to accept > and return these types) than "make classmethod" ("turn this function into a > class method"). Other potential keywords have similar problems. If you know what the keyword is, you can rename the decorator accordingly: make argcheck(int, (int, float)) make returncheck(int) But we could choose a keyword that doesn't mean anything, e.g. 'dec' or 'decorate'. Bye, Walter D?rwald From mwh at python.net Tue Aug 10 15:35:16 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 10 15:35:18 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <411904F3.1080703@hlabs.spb.ru> (Dmitry Vasiliev's message of "Tue, 10 Aug 2004 17:25:07 +0000") References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> <16664.50649.835422.998574@montanaro.dyndns.org> <411904F3.1080703@hlabs.spb.ru> Message-ID: <2mpt5z15gr.fsf@starship.python.net> Dmitry Vasiliev writes: > Skip Montanaro wrote: >> >> Is there a plan for implementing a base class for int and long (like >> >> basestring for str and unicode): >> Michael> Not that I'm aware of. >> We've discussed the numeric type hierarchy in the past. I think it >> would be >> worth considering all the numeric types at once and come up with a >> reasonable hierarchy that includes float, complex and decimal types as well. >> (It's clear there is no "best" way to do that.) > > Something like following? > > object > numeric > integer > int > long > float > complex > > > PS I've already have implementation for baseinteger, but can't made a > patch since cvs.python.sourceforge.net always told me 'Connection > timed out'... :-/ I think you want cvs.sourceforge.net these days. Cheers, mwh -- I located the link but haven't bothered to re-read the article, preferring to post nonsense to usenet before checking my facts. -- Ben Wolfson, comp.lang.python From skip at pobox.com Tue Aug 10 15:35:14 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 10 15:35:37 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <411904F3.1080703@hlabs.spb.ru> References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> <16664.50649.835422.998574@montanaro.dyndns.org> <411904F3.1080703@hlabs.spb.ru> Message-ID: <16664.53010.746248.733772@montanaro.dyndns.org> >> I think it would be worth considering all the numeric types at once >> and come up with a reasonable hierarchy that includes float, complex >> and decimal types as well. (It's clear there is no "best" way to do >> that.) Dmitry> Something like following? Dmitry> object Dmitry> numeric Dmitry> integer Dmitry> int Dmitry> long Dmitry> float Dmitry> complex At some level it seems that float is a specialization of complex (w/ imag field == 0). Also, it's not clear that float or complex don't have something other than a sibling relationship to integers. I don't have the time to dig up the previous discussions. Three days out of town left me we nearly a thousand Python-related mail messages. I have no idea how (if at all) I'll be able to boil any of that down into reasonable edits to PEP 318. Skip From mwh at python.net Tue Aug 10 15:37:46 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 10 15:37:48 2004 Subject: [Python-Dev] Third Bug Day outcome In-Reply-To: <20040809213820.GC11199@rogue.amk.ca> (A. M. Kuchling's message of "Mon, 9 Aug 2004 17:38:20 -0400") References: <20040809213820.GC11199@rogue.amk.ca> Message-ID: <2mllgn15cl.fsf@starship.python.net> "A.M. Kuchling" writes: > The bug day on Saturday went well, as usual. 19 bugs and 12 patches > were closed. That's not as good as the first bug day (30 bugs), but > is competitive with the second (18 bugs, 21 patches). Lots of the > easy bugs have been cleaned out, I expect, so each remaining bug takes > more time to fix. > > The composition of the turnout was the surprising thing. My plaintive > bleating on python-dev resulted in much higher participation by people > with CVS committers, but there weren't many non-committer people > around (Seo Sanghyeon and Mike Coleman were the two non-developers I > noticed). Well apart from jlgijsbers who you mentioned in your post :-) I don't know how much longer we should let him get away with not being a commiter, though . Cheers, mwh -- ... so the notion that it is meaningful to pass pointers to memory objects into which any random function may write random values without having a clue where they point, has _not_ been debunked as the sheer idiocy it really is. -- Erik Naggum, comp.lang.lisp From amk at amk.ca Tue Aug 10 15:40:23 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 10 15:40:52 2004 Subject: [Python-Dev] More PEPs to mark as final Message-ID: <20040810134023.GA13993@rogue.amk.ca> The following PEPs all seem to be completed; can their authors (or someone) please confirm, and mark them as 'Final' if the PEPs are in fact completed? 273 Import Modules from Zip Archives 278 Universal Newline Support 282 A Logging System 302 New Import Hooks 305 CSV File API --amk From lists at hlabs.spb.ru Tue Aug 10 19:58:23 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Tue Aug 10 15:52:32 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <2mpt5z15gr.fsf@starship.python.net> References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> <16664.50649.835422.998574@montanaro.dyndns.org> <411904F3.1080703@hlabs.spb.ru> <2mpt5z15gr.fsf@starship.python.net> Message-ID: <41190CBF.6010405@hlabs.spb.ru> Michael Hudson wrote: > Dmitry Vasiliev writes: [SKIP] >>PS I've already have implementation for baseinteger, but can't made a >>patch since cvs.python.sourceforge.net always told me 'Connection >>timed out'... :-/ > > I think you want cvs.sourceforge.net these days. You've right, thanks. -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From mwh at python.net Tue Aug 10 15:55:19 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 10 15:55:21 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> (James Y. Knight's message of "Mon, 9 Aug 2004 18:14:12 -0400") References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> Message-ID: <2mekmf14jc.fsf@starship.python.net> James Y Knight writes: > There's a fair number of classes that claim they are defined in > __builtin__, but do not actually appear there. For example: "__builtin__" is the "I don't know" answer in type_module for non-HEAPTYPEs. I'm certainly not sure that's totally wise... > IMO classes ought to actually appear in __builtin__ if they claim they > are defined there. Doing otherwise breaks reflection, as you have to > add a special case for these?class names to use the appropriate object > from the types module instead. Thoughts? If it isn't desirable to have > these names appear in __builtin__, perhaps their '__module__' should > be changed to another module where they are defined? Such as? There really isn't a module where e.g. GeneratorType is defined. Cheers, mwh -- 3. Syntactic sugar causes cancer of the semicolon. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From exarkun at divmod.com Tue Aug 10 16:02:36 2004 From: exarkun at divmod.com (Jp Calderone) Date: Tue Aug 10 16:02:46 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <2mekmf14jc.fsf@starship.python.net> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> Message-ID: <4118D57C.9090306@divmod.com> Michael Hudson wrote: > James Y Knight writes: >>IMO classes ought to actually appear in __builtin__ if they claim they >>are defined there. Doing otherwise breaks reflection, as you have to >>add a special case for these class names to use the appropriate object >>from the types module instead. Thoughts? If it isn't desirable to have >>these names appear in __builtin__, perhaps their '__module__' should >>be changed to another module where they are defined? > > > Such as? There really isn't a module where e.g. GeneratorType is > defined. > Seems perfectly reasonable and useful to add GeneratorType and others to the types module. I have code, for example, like this, in a couple places: def _getACell(o): x = o def y(): y = x return [o for o in gc.get_referrers(x) if type(o).__name__ == 'cell'][0] def _f(): types.CellType = type(_getACell(object())) _f() Jp From erik at heneryd.com Tue Aug 10 16:04:39 2004 From: erik at heneryd.com (Erik Heneryd) Date: Tue Aug 10 16:04:44 2004 Subject: [Python-Dev] Re: Re: pre-PEP: Complete, Structured Regular ExpressionGroup Matching In-Reply-To: References: <411429FB.6080603@gradient.cis.upenn.edu> <4118C626.4070203@heneryd.com> Message-ID: <4118D5F7.1090709@heneryd.com> Fredrik Lundh wrote: > if done on the C level, that is. partial compilation on the Python level > is slightly easier, as you've just shown. clever! > > (but probably rather slow; I still recommend building something on top > of the scanner object) Yes, ofcourse it's slow, as I recursivly compile subpatterns and try them out, but as things are, it's probably much easier to implement it this way than using scanners, which I suspect is a lot of work (given the structmatch interface)... Erik From mwh at python.net Tue Aug 10 16:05:40 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 10 16:05:42 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <4118D57C.9090306@divmod.com> (Jp Calderone's message of "Tue, 10 Aug 2004 10:02:36 -0400") References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> Message-ID: <2macx31423.fsf@starship.python.net> Jp Calderone writes: > Michael Hudson wrote: >> James Y Knight writes: >>>IMO classes ought to actually appear in __builtin__ if they claim they >>>are defined there. Doing otherwise breaks reflection, as you have to >>>add a special case for these class names to use the appropriate object >>>from the types module instead. Thoughts? If it isn't desirable to have >>>these names appear in __builtin__, perhaps their '__module__' should >>>be changed to another module where they are defined? >> Such as? There really isn't a module where e.g. GeneratorType is >> defined. >> > > Seems perfectly reasonable and useful to add GeneratorType and > others to the types module. I have code, for example, like this, > in a couple places: Well, it's already there, but types.GeneratorType.__name__ is 'generator'... it could be changed to 'GeneratorType', I guess. Cheers, mwh -- . <- the point your article -> . |------------------------- a long way ------------------------| -- Cristophe Rhodes, ucam.chat From gmccaughan at synaptics-uk.com Tue Aug 10 16:21:55 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue Aug 10 16:22:28 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <16664.53010.746248.733772@montanaro.dyndns.org> References: <41176596.5060209@hlabs.spb.ru> <411904F3.1080703@hlabs.spb.ru> <16664.53010.746248.733772@montanaro.dyndns.org> Message-ID: <200408101521.55005.gmccaughan@synaptics-uk.com> On Tuesday 2004-08-10 14:35, Skip Montanaro wrote: > > >> I think it would be worth considering all the numeric types at once > >> and come up with a reasonable hierarchy that includes float, complex > >> and decimal types as well. (It's clear there is no "best" way to do > >> that.) > > Dmitry> Something like following? > > Dmitry> object > Dmitry> numeric > Dmitry> integer > Dmitry> int > Dmitry> long > Dmitry> float > Dmitry> complex > > At some level it seems that float is a specialization of complex (w/ imag > field == 0). Also, it's not clear that float or complex don't have > something other than a sibling relationship to integers. Start by thinking about the mathematical types, which have nice clear subset relationships between them: number complex real integer Python doesn't have anything between "number" and "complex" and there's no particular reason to think it ever will, so let's simplify: number real integer Now consider the different implementation types and fit them in: number complex real (Decimal) float integer int long I've reintroduced "complex" here as a name for the implementation type, which in some sense should really be called complex_float or something like that. By way of clarification of the principles here, this is what would happen to the hierarchy if Python were to acquire a type for ratios of integers. I'll call that type "fraction" and the underlying mathematical type "rational". Then we'd get: number complex real (Decimal) float rational fraction integer int long (You could argue that floats are rationals really, but that's misleading even though it's true. The *structure* of arithmetic operations on floats isn't the same as that on rationals.) -- g From fredrik at pythonware.com Tue Aug 10 16:41:12 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 10 16:41:22 2004 Subject: [Python-Dev] Re: Unifying Long Integers and Integers: baseint References: <41176596.5060209@hlabs.spb.ru><2m7js83733.fsf@starship.python.net><16664.50649.835422.998574@montanaro.dyndns.org><411904F3.1080703@hlabs.spb.ru> <16664.53010.746248.733772@montanaro.dyndns.org> Message-ID: Skip Montanaro wrote: > I don't have the time to dig up the previous discussions. why not start with the PEPs? http://www.python.org/peps/pep-0228.html http://www.python.org/peps/pep-0242.html http://www.python.org/peps/pep-0313.html (etc) From pje at telecommunity.com Tue Aug 10 17:03:52 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 10 17:00:39 2004 Subject: [Python-Dev] More PEPs to mark as final In-Reply-To: <20040810134023.GA13993@rogue.amk.ca> Message-ID: <5.1.1.6.0.20040810105316.02a51230@mail.telecommunity.com> At 09:40 AM 8/10/04 -0400, A.M. Kuchling wrote: >The following PEPs all seem to be completed; can their authors (or >someone) please confirm, and mark them as 'Final' if the PEPs are in >fact completed? > >273 Import Modules from Zip Archives >278 Universal Newline Support >282 A Logging System >302 New Import Hooks >305 CSV File API If you're asking whether these are implemented, at least PEP 302 isn't quite finished. If I understand it correctly, there should be an 'imp.get_loader', and its "phase 2" plan calls for putting "frozen, built-in and sys.path importer objects on sys.meta_path, allowing for some extra flexibility". From guido at python.org Tue Aug 10 17:01:17 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 10 17:01:27 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: Your message of "Tue, 10 Aug 2004 15:05:40 BST." <2macx31423.fsf@starship.python.net> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> Message-ID: <200408101501.i7AF1HZ25725@guido.python.org> > Well, it's already there, but types.GeneratorType.__name__ is > 'generator'... it could be changed to 'GeneratorType', I guess. No, no, no! The "new" convention for built-in type names is all lowercase. CamelCaseType is old and only used in the context of types.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Tue Aug 10 17:03:03 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 10 17:03:22 2004 Subject: [Python-Dev] use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings In-Reply-To: <4113E461.20600@v.loewis.de> References: <16658.42673.542930.749915@montanaro.dyndns.org> <4112A968.2030401@v.loewis.de> <16659.57319.931941.158458@montanaro.dyndns.org> <4113E461.20600@v.loewis.de> Message-ID: <16664.58279.489645.778551@montanaro.dyndns.org> Martin> I don't think it is a goal that Python generates no warnings if Martin> compiled with a C++ compiler (I don't think you actually *can* Martin> compile all of Python with a C++ compiler). Our goal is less ambitious than that. It is to build locally written extension modules and other stuff (both python-related and otherwise) without warnings, not necessarily all software we build for internal use. >> This suggests that Sun expects the application or its build tools to >> define _XOPEN_SOURCE, not the compiler or its header files. That >> suggests a bug in g++. Martin> Indeed, and I believe this is a known bug. g++ needs to define Martin> _XOPEN_SOURCE so that some of Sun's system headers declare a Martin> number of prototypes which are then referred-to in inline Martin> functions of libstdc++ header files; ... Martin> This gives us a number of options: Martin> 1. On Solaris, define _XOPEN_SOURCE to the same value that g++ Martin> uses. This should then cause no redefinition, but might Martin> cause some functions go away. We would need an autoconf test Martin> to find out what the value of _XOPEN_SOURCE is that g++ uses. Of the three this seemed most plausible to me. A first pass at a configure.in mod is available at http://python.org/sf/1006629 Assigned to Martin. Skip From jim at zope.com Tue Aug 10 17:19:04 2004 From: jim at zope.com (Jim Fulton) Date: Tue Aug 10 17:19:09 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <24A02222-EA5E-11D8-AAEA-000D932927FE@opnet.com> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <4117F7D5.3090907@zope.com> <24A02222-EA5E-11D8-AAEA-000D932927FE@opnet.com> Message-ID: <4118E768.4050104@zope.com> Nick Bastin wrote: > > On Aug 9, 2004, at 6:16 PM, Jim Fulton wrote: > >> James Y Knight wrote: >> >>> IMO classes ought to actually appear in __builtin__ if they claim >>> they are defined there. Doing otherwise breaks reflection, as you >>> have to add a special case for these class names to use the >>> appropriate object from the types module instead. Thoughts? >> >> >> I agree. > > > I think this should stand for functions as well. help() is a good > example. It's a bit confusing for newbies that help.__module__ is > 'site', but 'site' isn't in their local namespace. Yup. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From skip at pobox.com Tue Aug 10 17:25:53 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 10 17:26:13 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: References: <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> Message-ID: <16664.59649.633382.860519@montanaro.dyndns.org> (still catching up...) Simon> I just wanted to say that I believe it should be allowed to Simon> decorate classes. There's not reason enough (reading the thread Simon> linked to by the PEP) to limit decorators this way. Like said Simon> several times in that thread, metaclasses are harder (to write Simon> and to understand) than decorators. While it might be perceived as a limitation there's nothing preventing its addition in a later version if warranted. I believe most/all the various proposals on the PythonDecorators wiki, not just the pie syntax, could be easily adapted to class decoration. Skip From colanderman at gmail.com Tue Aug 10 18:07:20 2004 From: colanderman at gmail.com (Chris King) Date: Tue Aug 10 18:07:34 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> Message-ID: <875c7e0704081009076cd1cc77@mail.gmail.com> On Tue, 10 Aug 2004 08:40:59 +1000, Delaney, Timothy C (Timothy) wrote: > Not such a large outcry, but lots of us were concerned about "having to look inside the function for 'yield' to determine if it were a generator or a function" and were calling for a 'gen' or 'generator' keyword. Building off of this: It's not really that important to a user to know whether a function is a generator or not: the difference (to a user) is the same difference as whether a function returns a single item or a list. A function's return type isn't visibly declared at the top of the function, it (along with everything else you need to know when calling a function) is in the docstring. Now, is it really that important to a user whether a method is a class or static method? They're all called the same way (so long as they're bound), and most users couldn't care less how a function is implemented. If the distinction is really important to the user, then they will presumably be prudent enough to check for this by looking either just below the function name (or wherever decorators end up), or by reading the docstring if the function author decided to document that. What's more, aside from classmethod and staticmethod, most other decorations will be for the module's own use. Placing such insignificant (to the user!) details such as these above the function's name and signature (what I think is the most important aspect of a function!) just seems like we're asking for obfuscation. A counter argument may be that decorators need to be readily visible to module authors. Well, to the author of a module, I'd say the code is the most important aspect of a function -- module authors should be worrying more about why their calculate_pi() funcion is returning 3.24895, rather than whether it's a class or static method. Of course, the error could have been caused by a misdeclaration of method type, but I'm sure that would be the cause much less often than, say, an off-by-one error in the code. +1 for decorators anywhere after the function name and signature, even if it involves pies. From foom at fuhm.net Tue Aug 10 18:17:14 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 10 18:17:16 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <200408101501.i7AF1HZ25725@guido.python.org> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> <200408101501.i7AF1HZ25725@guido.python.org> Message-ID: On Aug 10, 2004, at 11:01 AM, Guido van Rossum wrote: > No, no, no! The "new" convention for built-in type names is all > lowercase. CamelCaseType is old and only used in the context of > types.py. Sooo should (for 'generator' in objects that claim to be in __builtins__ but aren't), 1) 'generator' be added to __builtins__ 2) 'generator' be added to types.py and its __module__ be set to 'types' 3) 'generator' be added to .py and its __module__ be set to '' (and a name for the module chosen) James From pje at telecommunity.com Tue Aug 10 19:21:42 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 10 19:17:26 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <875c7e0704081009076cd1cc77@mail.gmail.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> Message-ID: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com> At 12:07 PM 8/10/04 -0400, Chris King wrote: >Now, is it really that important to a user whether a method is a class >or static method? They're all called the same way (so long as they're >bound), and most users couldn't care less how a function is >implemented. If the distinction is really important to the user, then >they will presumably be prudent enough to check for this by looking >either just below the function name (or wherever decorators end up), >or by reading the docstring if the function author decided to document >that. This doesn't apply to other decorators. For example, some decorators turn a function into a property. Others change the (effective) call signature. Indeed, staticmethod changes the call signature, since there is no special first argument. In frameworks where methods can be wrapped in transactions, lock synchronization, security checks, remote communication, etc., these are all decorations that are potentially part of the interface and important for the reader to know, even if they don't look at the method body (apart from the doc string). From bob at redivi.com Tue Aug 10 19:34:57 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 10 19:35:29 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com> Message-ID: <991C2154-EAF3-11D8-91E7-000A95686CD8@redivi.com> On Aug 10, 2004, at 1:21 PM, Phillip J. Eby wrote: > At 12:07 PM 8/10/04 -0400, Chris King wrote: > >> Now, is it really that important to a user whether a method is a class >> or static method? They're all called the same way (so long as they're >> bound), and most users couldn't care less how a function is >> implemented. If the distinction is really important to the user, then >> they will presumably be prudent enough to check for this by looking >> either just below the function name (or wherever decorators end up), >> or by reading the docstring if the function author decided to document >> that. > > This doesn't apply to other decorators. For example, some decorators > turn a function into a property. Others change the (effective) call > signature. > > Indeed, staticmethod changes the call signature, since there is no > special first argument. > > In frameworks where methods can be wrapped in transactions, lock > synchronization, security checks, remote communication, etc., these > are all decorations that are potentially part of the interface and > important for the reader to know, even if they don't look at the > method body (apart from the doc string). Also, most people that call static methods or class methods probably aren't calling them bound to an *instance*. At least I know that when I use classmethods, it's usually to *create* instances. -bob From colanderman at gmail.com Tue Aug 10 20:03:33 2004 From: colanderman at gmail.com (Chris King) Date: Tue Aug 10 20:03:36 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com> Message-ID: <875c7e070408101103710836f6@mail.gmail.com> On Tue, 10 Aug 2004 13:21:42 -0400, Phillip J. Eby wrote: > This doesn't apply to other decorators. For example, some decorators turn > a function into a property. Ew. If you did that, you'd be declaring get properties one way (with decorators) and get/set properties another (using property()). > Others change the (effective) call signature. But will a casual observer be able to see exactly how a given decorator is mutating the call signature? If such a function is meant for external use, its effective signature should be in the docstring, not unlike those of C extension functions. On the other hand, the developer will know which functions are using such a decorator, or will know to look for whether such a decorator is being used in a function. > Indeed, staticmethod changes the call signature, since there is no special > first argument. No, static methods are called with the same signature you see defined, just like a normal function. Not seeing 'self' as the first argument should be enough to tip you off that this isn't a bound method. Concerning users, whether a method is a static or class method should be indicated in the method's docstring. I should hope any user reads a function's docstring for usage information before using it. > In frameworks where methods can be wrapped in transactions, lock > synchronization, security checks, remote communication, etc., these are all > decorations that are potentially part of the interface and important for > the reader to know, even if they don't look at the method body (apart from > the doc string). Assuming they look at the docstring, perhaps the method author would be kind enough to specify "This function performs an exclusively locked record transaction.", or, better yet, the decorator itself could extend the docstring with its own "record-locking method" indicator. Making decorators more visible in the source doesn't make then more visible in the docstring. If the user can't be bothered to read a three-line description that is a docstring, I doubt they will be bothered to read (and possibly decipher) the information contained in decorators, no matter where they're placed. From dave at boost-consulting.com Tue Aug 10 20:12:36 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue Aug 10 20:13:15 2004 Subject: [Python-Dev] Re: pep 318, Decorators for Functions, Methods and Classes References: <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> <16664.59649.633382.860519@montanaro.dyndns.org> Message-ID: Skip Montanaro writes: > (still catching up...) > > Simon> I just wanted to say that I believe it should be allowed to > Simon> decorate classes. There's not reason enough (reading the thread > Simon> linked to by the PEP) to limit decorators this way. Like said > Simon> several times in that thread, metaclasses are harder (to write > Simon> and to understand) than decorators. > > While it might be perceived as a limitation there's nothing preventing its > addition in a later version if warranted. I believe most/all the various > proposals on the PythonDecorators wiki, not just the pie syntax, could be > easily adapted to class decoration. I was going for replacing the module's dict with a dict subclass, and hooking __setitem__ it as an alternative to using settrace to implement this. It's not allowed, though :(. We also need a way to say, "replace the dict into which new definitions are falling" so we can implement decorators inside of class definitions. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From mark at hahnca.com Tue Aug 10 20:56:05 2004 From: mark at hahnca.com (Mark Hahn) Date: Tue Aug 10 20:56:19 2004 Subject: [Python-Dev] Prothon on CPython intrepreter? [PROTHON] Message-ID: <000a01c47f0b$b3cf4690$0b01a8c0@mark> I got such excellent help from Python-dev on the Prothon decimal question that I've come back for a really big question. :-) I've announced recently that Prothon was switching from a homegrown (CProthon) interpreter to a .Net implementation like IronPython. The logic was that Prothon would get a library and a community much faster. What it got very quickly was a backlash from people, including Guido, who wanted Prothon to stay away from the dark side and remain pure like CPython. I've said in the past that Prothon couldn't use the CPython interpreter. I made this determination late last year when I first started the Prothon project because CPython was incompatible with my goals of pre-emptive threads with no GIL, object locking, etc. Now I find myself compromising my goals for .Net in that I cannot implement object-locking and lightweight threads. I am also going to be mixing real classes with my prototypes, although this is something I have determined I want to do for other reasons (it's a long story). I'm sure you've heard the joke that ends with "we've established you're a whore, now we are just haggling over the price". If I can compromise Prothon for .Net, why can't I compromise it some more for CPython? So I am thinking of doing a Prothon implementation in CPython before doing the .Net version. I will get a library and community fast like I would have with .Net. This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly easy my wild ideas for the interpreter into Prothon. It will also make CPython a multiple language VM for the first time. So, I'll finally get to some questions: 1) Guido: Does this address the reason you asked me not to switch to .Net, or did you specifically want me to do CProthon? I think this will still be a good sandbox for Python. It should be quite interesting to see how far I can stretch the Python interpreter. Who knows how many of my CProthon engine features I might be able to add to CPython. 2) Lenard Lindstrom said he thought I could "Add some extra byte code instructions to handle Prothon's distinct scoping rules and implement prototypes using PyType_Type". He also said "All attribute accesses would be through a special attribute handler method". I don't know the Python interpreter code yet, but does this sound workable to everyone? It doesn't have to be Python compatible. 3) Could Tim Peter's welcome offer to Prothon here on Python-dev be stretched to include the many questions I would have when modifying CPython? In other words would this project be welcomed as a sandbox for Python or would it be an unwelcome drain on your limited resources? If any of you want to see background material on Prothon, my VanPy talk is at: http://prothon.org/pub/prothon/docs/vanpy/ProthonTalk.htm. The section titled "engine" is about the homegrown engine and wouldn't be applicable to anything on .Net or CPython. Thanks for you attention. I would love to find a solution that lets me work with the wonderful Python community. From jjl at pobox.com Tue Aug 10 21:11:34 2004 From: jjl at pobox.com (John J Lee) Date: Tue Aug 10 21:11:38 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae04080918481e1f8053@mail.gmail.com> References: <007c01c47da7$92b71920$191fc797@oemcomputer> <1f7befae04080918481e1f8053@mail.gmail.com> Message-ID: On Mon, 9 Aug 2004, Tim Peters wrote: [...] > Has anyone on Linux tried this yet? On Linux, I get precisely the same output from Python CVS that you posted, at least as far as 148 (didn't check any further): [...] 118 ok 119 ok 120 ok 121 ok 122 ok 123 exceptions.RuntimeError maximum recursion depth exceeded 124 exceptions.RuntimeError maximum recursion depth exceeded 125 exceptions.RuntimeError maximum recursion depth exceeded 126 exceptions.KeyError 307 127 exceptions.RuntimeError maximum recursion depth exceeded 128 exceptions.KeyError 306 129 exceptions.RuntimeError maximum recursion depth exceeded 130 exceptions.KeyError 305 131 exceptions.RuntimeError maximum recursion depth exceeded 132 exceptions.KeyError 304 133 exceptions.RuntimeError maximum recursion depth exceeded 134 exceptions.KeyError 303 135 exceptions.KeyError 302 136 exceptions.RuntimeError maximum recursion depth exceeded 137 exceptions.KeyError 301 138 exceptions.RuntimeError maximum recursion depth exceeded 139 exceptions.KeyError 300 140 exceptions.RuntimeError maximum recursion depth exceeded 141 exceptions.RuntimeError maximum recursion depth exceeded 142 exceptions.KeyError 299 143 exceptions.RuntimeError maximum recursion depth exceeded 144 exceptions.KeyError 297 145 exceptions.KeyError 296 146 exceptions.RuntimeError maximum recursion depth exceeded 147 exceptions.KeyError 295 148 exceptions.RuntimeError maximum recursion depth exceeded [...] John From arigo at tunes.org Tue Aug 10 21:16:38 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Aug 10 21:21:15 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae04080918481e1f8053@mail.gmail.com> References: <007c01c47da7$92b71920$191fc797@oemcomputer> <1f7befae04080918481e1f8053@mail.gmail.com> Message-ID: <20040810191638.GA24820@vicky.ecs.soton.ac.uk> Hello Tim, On Mon, Aug 09, 2004 at 09:48:37PM -0400, Tim Peters wrote: > Has anyone on Linux tried this yet? I get this: in a release build, ... 120 ok 121 ok 122 ok 123 exceptions.RuntimeError maximum recursion depth exceeded 124 exceptions.RuntimeError maximum recursion depth exceeded 125 exceptions.RuntimeError maximum recursion depth exceeded 126 exceptions.RuntimeError maximum recursion depth exceeded 127 exceptions.RuntimeError maximum recursion depth exceeded 128 exceptions.RuntimeError maximum recursion depth exceeded ... In a release 21st of March version with the fresh generator expression patch applied, ... 120 ok 121 ok 122 ok 123 exceptions.RuntimeError maximum recursion depth exceeded 124 exceptions.RuntimeError maximum recursion depth exceeded 125 exceptions.RuntimeError maximum recursion depth exceeded 126 exceptions.KeyError 305 127 exceptions.RuntimeError maximum recursion depth exceeded 128 exceptions.KeyError 304 129 exceptions.RuntimeError maximum recursion depth exceeded ... In a recent debug build, ... 120 ok 121 ok 122 ok 123 exceptions.RuntimeError maximum recursion depth exceeded 124 exceptions.RuntimeError maximum recursion depth exceeded 125 exceptions.RuntimeError maximum recursion depth exceeded 126 exceptions.KeyError 307 127 exceptions.RuntimeError maximum recursion depth exceeded 128 exceptions.KeyError 306 129 exceptions.RuntimeError maximum recursion depth exceeded 130 exceptions.KeyError 305 131 exceptions.RuntimeError maximum recursion depth exceeded ... So there is no particular problem running debug builds under Linux, but still why we get this behavior is anybody's guess. Also note that even though the compiler package may only have changed a bit, the parser C module definitely has, e.g. when genexprs and decorators were introduced. I wonder what effect this has on the depth of concrete syntax trees. I also wonder if the parser module tries to be safe again C stack overflow when building hugely nested data structures. Armin From mwh at python.net Tue Aug 10 21:00:14 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 10 21:22:39 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae0408092348583f0953@mail.gmail.com> (Tim Peters's message of "Tue, 10 Aug 2004 02:48:15 -0400") References: <000001c47e88$81ef1d40$5635c797@oemcomputer> <1f7befae0408092348583f0953@mail.gmail.com> Message-ID: <2mvffqzum9.fsf@starship.python.net> Tim Peters writes: > Well, this gets nasty. You're certainly not wrong about that! [snip Windows details] > That's > > if (Py_EnterRecursiveCall(" in cmp")) > return NULL; > > in PyObject_RichCompare. That's just trying to compare two ints. > > So NULL gets returned to PyObject_RichCompareBool, which in turn > returns -1 to lookdict. AAAARGHGH! lookdict "isn't allowed" to raise > exceptions, so it does a PyErr_Clear(), goes on to futilely chase the > entire dict looking for another match on the hash code, and we've > effectively turned a MemoryError into a KeyError. I expect that > explains a lot about what we see in the release-build runs. This part happens on Linux, too, it seems. Running your script: 114 ok 115 ok 116 ok 117 ok 118 ok 119 ok 120 ok 121 ok 122 ok 123 exceptions.RuntimeError maximum recursion depth exceeded 124 exceptions.RuntimeError maximum recursion depth exceeded 125 exceptions.RuntimeError maximum recursion depth exceeded 126 exceptions.KeyError 307 127 exceptions.RuntimeError maximum recursion depth exceeded 128 exceptions.KeyError 306 129 exceptions.RuntimeError maximum recursion depth exceeded 130 exceptions.KeyError 305 131 exceptions.RuntimeError maximum recursion depth exceeded 132 exceptions.KeyError 304 133 exceptions.RuntimeError maximum recursion depth exceeded 134 exceptions.KeyError 303 135 exceptions.KeyError 302 136 exceptions.RuntimeError maximum recursion depth exceeded 137 exceptions.KeyError 301 This is a debug build, so we're not abusing the stack quite so much on linux as apparently seems to happen in debug builds on windows. This isn't going to be something we can pin on the new Windows tools, is it? It doesn't seem like a feature that a debug build gets a tiny stack, but, well... > The question is what we did since 2.3.4 that apparently increases our > stack demands, and grossly increases them in a debug build(!). ^ on windows; I don't see anything so scary on Linux. Hmm, maybe that's a lie, I don't see the crazy KeyErrors with 2.3. > Could be that the compile package is more heavily recursive now too > (no idea). Really don't think so. > test_parser.py in 2,3.4 contained the same deeply nested tuples, so > that's not what changed. Something that *has* changed, however, is that we lost the code that compared objects recursively and now PyObject_RichCompare uses the more general Py_EnterRecursiveCall mechanism and not its own counter. So now we're grossly more likely to hit one of these faux KeyErrors in 2.3 than 2.4 (you'd need to invoke a Python __eq__ method or so). Would it make more sense for PyObject_RichCompare to still use a private counter but raise RuntimeError if it is exceeded instead of trying to be clever? That would make this problem /less/ likely, though obviously still possible (functions that claim they 'can't fail' are lying). Running Misc/find_recursionlimit.py with a debug build yields ... well, I gave up and killed it at 22000 (hmm, seems the default 'ulimit -s' is "unlimited"... I'm sure this hasn't always had this effect with a threaded build). Cheers, mwh -- > Emacs is a fashion statement. No, Gnus is a fashion statement. Emacs is clothing. Everyone else is running around naked. -- Karl Kleinpaste & Jonadab the Unsightly One, gnu.emacs.gnus From walter at livinglogic.de Tue Aug 10 21:24:20 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Aug 10 21:24:23 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4107F0E8.3010005@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> Message-ID: <411920E4.9020404@livinglogic.de> OK, here a my current thoughts on the codec problem: The optimal solution (ignoring backwards compatibility) would look like this: codecs.lookup() would return the following stuff (this could be done by replacing the 4 entry tuple with a real object): * decode: The stateless decoding function * encode: The stateless encocing function * chunkdecoder: The stateful chunk decoder * chunkencoder: The stateful chunk encoder * streamreader: The stateful stream decoder * streamwriter: The stateful stream encoder The functions and classes look like this: Stateless decoder: decode(input, errors='strict'): Function that decodes the (str) input object and returns a (unicode) output object. The decoder must decode the complete input without any remaining undecoded bytes. Stateless encoder: encode(input, errors='strict'): Function that encodes the complete (unicode) input object and returns a (str) output object. Stateful chunk decoder: chunkdecoder(errors='strict'): A factory function that returns a stateful decoder with the following method: decode(input, final=False): Decodes a chunk of input and return the decoded unicode object. This method can be called multiple times and the state of the decoder will be kept between calls. This includes trailing incomplete byte sequences that will be retained until the next call to decode(). When the argument final is true, this is the last call to decode() and trailing incomplete byte sequences will not be retained, but a UnicodeError will be raised. Stateful chunk encoder: chunkencoder(errors='strict'): A factory function that returns a stateful encoder with the following method: encode(input, final=False): Encodes a chunk of input and returns the encoded str object. When final is true this is the last call to encode(). Stateful stream decoder: streamreader(stream, errors='strict'): A factory function that returns a stateful decoder for reading from the byte stream stream, with the following methods: read(size=-1, chars=-1, final=False): Read unicode characters from the stream. When data is read from the stream it should be done in chunks of size bytes. If size == -1 all the remaining data from the stream is read. chars specifies the number of characters to read from the stream. read() may return less then chars characters if there's not enough data available in the byte stream. If chars == -1 as much characters are read as are available in the stream. Transient errors are ignored and trailing incomplete byte sequence are retained when final is false. Otherwise a UnicodeError is raised in the case of incomplete byte sequences. readline(size=-1): ... next(): ... __iter__(): ... Stateful stream encoder: streamwriter(stream, errors='strict'): A factory function that returns a stateful encoder for writing unicode data to the byte stream stream, with the following methods: write(data, final=False): Encodes the unicode object data and writes it to the stream. If final is true this is the last call to write(). writelines(data): ... I know that this is quite a departure from the current API, and I'm not sure if we can get all of the functionality without sacrificing backwards compatibility. I don't particularly care about the "bytes consumed" return value from the stateless codec. The codec should always have returned only the encoded/decoded object, but I guess fixing this would break too much code. And users who are only interested in the stateless functionality will probably use unicode.encode/str.decode anyway. For the stateful API it would be possible to combine the chunk and stream decoder/encode into one class with the following methods (for the decoder): __init__(stream, errors='strict'): Like the current StreamReader constructor, but stream may be None, if only the chunk API is used. decode(input, final=False): Like the current StreamReader (i.e. it returns a (unicode, int) tuple.) This does not keep the remaining bytes in a buffer. This is the job of the caller. feed(input, final=False): Decodes input and returns a decoded unicode object. This method calls decode() internally and manages the byte buffer. read(size=-1, chars=-1, final=False): readline(size=-1): next(): __iter__(): See above. As before implementers of decoders only need to implement decode(). To be able to support the final argument the decoding functions in _codecsmodule.c could get an additional argument. With this they could be used for the stateless codecs too and we can reduce the number of functions again. Unfortunately adding the final argument breaks all of the current codecs, but dropping the final argument requires one of two changes: 1) When the input stream is exhausted, the bytes read are parsed as if final=True. That's the way the CJK codecs currently handle it, but unfortunately this doesn't work with the feed decoder. 2) Simply ignore any remaing undecoded bytes at the end of the stream. If we really have to drop the final argument, I'd prefer 2). I've uploaded a second version of the patch. It implements the final argument, adds the feed() method to StreamReader and again merges the duplicate decoding functions in the codecs module. Note that the patch isn't really finished (the final argument isn't completely supported in the encoders and the CJK and escape codecs are unchanged), but it should be sufficient as a base for discussion. Bye, Walter D?rwald From fdrake at acm.org Tue Aug 10 21:25:57 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue Aug 10 21:26:09 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <20040810191638.GA24820@vicky.ecs.soton.ac.uk> References: <007c01c47da7$92b71920$191fc797@oemcomputer> <1f7befae04080918481e1f8053@mail.gmail.com> <20040810191638.GA24820@vicky.ecs.soton.ac.uk> Message-ID: <200408101525.57934.fdrake@acm.org> On Tuesday 10 August 2004 03:16 pm, Armin Rigo wrote: > syntax trees. I also wonder if the parser module tries to be safe again C > stack overflow when building hugely nested data structures. No, it really doesn't, though it probably should be a little careful. Switching to a less recursive approach may be a good idea, but not sure what would cause the most pain in implementation. -Fred -- Fred L. Drake, Jr. From arigo at tunes.org Tue Aug 10 21:23:03 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Aug 10 21:27:40 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <20040810191638.GA24820@vicky.ecs.soton.ac.uk> References: <007c01c47da7$92b71920$191fc797@oemcomputer> <1f7befae04080918481e1f8053@mail.gmail.com> <20040810191638.GA24820@vicky.ecs.soton.ac.uk> Message-ID: <20040810192303.GB24820@vicky.ecs.soton.ac.uk> Hello again, On Tue, Aug 10, 2004 at 08:16:38PM +0100, Armin Rigo wrote: > I get this: in a release build, > > ... > 120 ok > 121 ok > 122 ok > 123 exceptions.RuntimeError maximum recursion depth exceeded > 124 exceptions.RuntimeError maximum recursion depth exceeded > 125 exceptions.RuntimeError maximum recursion depth exceeded > 126 exceptions.RuntimeError maximum recursion depth exceeded > 127 exceptions.RuntimeError maximum recursion depth exceeded > 128 exceptions.RuntimeError maximum recursion depth exceeded > ... Sorry, my mistake. This was after I removed from object.c the two Py_Enter/LeaveRecursiveCall(" in cmp"). Not suprizingly, I then only get "regular" recursion errors. A pure CVS version gets me the same results as you and John. Armin From jhylton at gmail.com Tue Aug 10 21:31:39 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Tue Aug 10 21:31:42 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <20040810191638.GA24820@vicky.ecs.soton.ac.uk> References: <007c01c47da7$92b71920$191fc797@oemcomputer> <1f7befae04080918481e1f8053@mail.gmail.com> <20040810191638.GA24820@vicky.ecs.soton.ac.uk> Message-ID: On Tue, 10 Aug 2004 20:16:38 +0100, Armin Rigo wrote: > Also note that even though the compiler package may only have changed a bit, > the parser C module definitely has, e.g. when genexprs and decorators were > introduced. I wonder what effect this has on the depth of concrete syntax > trees. I also wonder if the parser module tries to be safe again C stack > overflow when building hugely nested data structures. I don't think the compiler package has changed much at all, but it consumes the deeply nested concrete syntax trees created by the parser module. The compiler.transformer module uses recursion a lot. Several releases ago I added some hacks to reduce the number of frames (basically did tail call optimization by hand in a few spots). It may be that those hacks are no longer sufficient. where's-that-ast-branch-when-you-need-it-ly y'rs, Jeremy From tim.peters at gmail.com Tue Aug 10 21:58:22 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 10 21:58:24 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <2mvffqzum9.fsf@starship.python.net> References: <000001c47e88$81ef1d40$5635c797@oemcomputer> <1f7befae0408092348583f0953@mail.gmail.com> <2mvffqzum9.fsf@starship.python.net> Message-ID: <1f7befae0408101258431039a1@mail.gmail.com> FYI, "the stack" under an MS compiler defaults to 1MB. It's easy to experiment with different stack allocations from the cmdline, like this example: editbin /stack:2000000 python.exe The ~2MB there is enough that test_compiler runs to completion normally under a release-build Python, and under a debug-build Python. In a debug-build Windows Python, each MB allocated to the stack allows about 400 recursion levels (according to Misc/find_recursionlimit.py), and running compiler.compile() directly on test_parser.py requires more than 800 levels, so a 2MB stack must be near the insanity limit for the debug build on this test. So the easiest way to get the -uall test suite running on Windows again is to fiddle linker flags to boost the stack size. I'm not sure real apps need it. If they do and we don't "fix it", what happens is that the process mysteriously just vanishes (no error msg, nothing in the system or application event logs either) with a 128 exit code. That's a disaster. Alas, I don't have a theory for how we could be managing to screw up the OS so badly -- but then I guess you really can't spell Windows XP without Windows <0.9 wink>. From python at rcn.com Tue Aug 10 22:18:17 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 10 22:18:30 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae0408101258431039a1@mail.gmail.com> Message-ID: <008301c47f17$2c6af940$e622c797@oemcomputer> Another datapoint: the problem existed prior to 1/15/2004. That eliminates many possibilities. Raymond From pje at telecommunity.com Tue Aug 10 22:31:16 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 10 22:27:01 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <875c7e070408101103710836f6@mail.gmail.com> References: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com> <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com> <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040810162646.05242aa0@mail.telecommunity.com> At 02:03 PM 8/10/04 -0400, Chris King wrote: >On Tue, 10 Aug 2004 13:21:42 -0400, Phillip J. Eby > wrote: > > > This doesn't apply to other decorators. For example, some decorators turn > > a function into a property. > >Ew. If you did that, you'd be declaring get properties one way (with >decorators) and get/set properties another (using property()). Google "property_getter", and you'll find an implementation for this: @property_getter def foo(self): return self.__foo @property_setter def foo(self,value): self.__foo = value @property_deleter def foo(self): del self.__foo From marktrussell at btopenworld.com Tue Aug 10 23:07:20 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Tue Aug 10 23:07:19 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: <41185844.80906@interlink.com.au> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> Message-ID: <1092172040.1681.9.camel@localhost> On Tue, 2004-08-10 at 06:08, Anthony Baxter wrote: > Mark, is it possible that this changed between the first and > final versions of the decorator patch? The SF report doesn't > list any of the older versions... No, it has always been this way round. In fact test_order was inherited from Guido's original version of test_decorators.py (from patch #926860) where it read: def test_order(self): class C(object): [funcattrs(abc=1), staticmethod] def foo(): return 42 # This wouldn't work if staticmethod was called first self.assertEqual(C.foo(), 42) self.assertEqual(C().foo(), 42) (i.e. identical to the current version except for the change in syntax). In fact I relied on the fact that this test passed to convince me I had the order right! But I should have spotted the inconsistency between that and the documentation that I wrote for the reference manual. I'll do a patch to fix the order and the corresponding tests. While I'm at it, do we want to drop support for multiple decorators on a single line? Nobody has spoken up for it, and in fact forcing one-per-line would simplify the grammar (as well as making it easier for toy parsers to find decorators). Mark From bob at redivi.com Tue Aug 10 23:24:54 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 10 23:25:00 2004 Subject: [Python-Dev] Drop support for multiple decorators per line? (was: Decorator order implemented backwards?) In-Reply-To: <1092172040.1681.9.camel@localhost> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> Message-ID: On Aug 10, 2004, at 5:07 PM, Mark Russell wrote: > While I'm at it, do we want to drop support for multiple decorators on > a > single line? Nobody has spoken up for it, and in fact forcing > one-per-line would simplify the grammar (as well as making it easier > for > toy parsers to find decorators). +1 -bob From andrew at indranet.co.nz Tue Aug 10 23:30:42 2004 From: andrew at indranet.co.nz (Andrew McGregor) Date: Tue Aug 10 23:30:59 2004 Subject: [Python-Dev] Re: Decorators: vertical bar syntax In-Reply-To: References: <004501c47cdb$355743d0$6602a8c0@arkdesktop> Message-ID: --On Sunday, 8 August 2004 2:16 p.m. +0400 Roman Suzi wrote: > I just anted to rewrite those "real-life-looking" examples with my > proposed '%' syntax: > > def foo() % (classmethod, accepts(int,int), returns(float),): > > or formatted: > > def foo() % ( > classmethod, > accepts(int,int), > returns(float),): > > Or it can be done the this way: > > def foo(): > ... > > foo %= (classmethod, accepts(int,int), returns(float),) > > The operation could be called "decorative apply", if the LHS > has __call__ attribute. Just a new form of % operation with > special syntax for "def" operator. (unlurking for a moment...) Now that actually looks like Python! Great suggestion. --------- Andrew McGregor Director, Scientific Advisor IndraNet Technologies Ltd http://www.indranet-technologies.com/ -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GS/E/B/PA/SS d+(++) s+:+ a C++$ ULS++++ !P+++(---)$ L++++$ E++ W++ !N w(+++) !O() M++ V--() Y+$ PGP+ t- !5? X- !R !tv@ b++(++++) DI++ D+++@ G e+++ h(*)@ r% ------END GEEK CODE BLOCK------ From barry at python.org Wed Aug 11 00:00:56 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 11 00:00:45 2004 Subject: [Python-Dev] Drop support for multiple decorators per line? (was: Decorator order implemented backwards?) In-Reply-To: References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> Message-ID: <1092175256.21841.20.camel@localhost> On Tue, 2004-08-10 at 17:24, Bob Ippolito wrote: > On Aug 10, 2004, at 5:07 PM, Mark Russell wrote: > > > While I'm at it, do we want to drop support for multiple decorators on > > a > > single line? Nobody has spoken up for it, and in fact forcing > > one-per-line would simplify the grammar (as well as making it easier > > for > > toy parsers to find decorators). > > +1 +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/2fafa2c2/attachment.pgp From andrew at indranet.co.nz Wed Aug 11 00:19:20 2004 From: andrew at indranet.co.nz (Andrew McGregor) Date: Wed Aug 11 00:20:33 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <16664.50665.170916.600964@montanaro.dyndns.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> <16664.50665.170916.600964@montanaro.dyndns.org> Message-ID: --On Tuesday, 10 August 2004 7:56 a.m. -0500 Skip Montanaro wrote: > I don't think anybody's come up with a candidate keyword that reads well > in most/all situations. Consider: > > make accepts(int, (int, float)) > make returns(int) > def foo(arg1, arg2): > return arg1 * arg2 make accepts(int, (int, float)): make returns(int): def foo(arg1, arg2): return arg1 * arg2 looks more like Python to me. And I know it's nested, but it reads better IMO. Aren't we really trying to recreate def as a multi-line lambda that binds when it's in a context that is not expecting a function argument in block form? One way of looking at def or any of the other keywords that introduce blocks is as operators that take block(s) as argument(s). --------- Andrew McGregor Director, Scientific Advisor IndraNet Technologies Ltd http://www.indranet-technologies.com/ -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GS/E/B/PA/SS d+(++) s+:+ a C++$ ULS++++ !P+++(---)$ L++++$ E++ W++ !N w(+++) !O() M++ V--() Y+$ PGP+ t- !5? X- !R !tv@ b++(++++) DI++ D+++@ G e+++ h(*)@ r% ------END GEEK CODE BLOCK------ From barry at python.org Wed Aug 11 00:42:51 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 11 00:42:43 2004 Subject: [Python-Dev] Update PEP 292 Message-ID: <1092177771.21839.54.camel@localhost> Raymond and I had a chance to talk about PEP 292 on bug day, and we've exchanged a few private emails and implementations. I think we've finally knocked everything into shape for inclusion in Python 2.4's next alpha. In the sandbox you will find the updated implementation, along with updated tests and documentation (libstring.tex). The updated PEP is at: http://www.python.org/peps/pep-0292.html A summary of the most important changes: * Renamed 'dstring' to 'template', and removed the other ?string classes * Added a 'safe_template' subclass for non KeyError-throwing substitutions * Got rid of safedicts; Raymond figured out the trick to retain optional braces in the safe substitution case without needing that class. * Renamed the 'cre' class attribute for subclasses to 'pattern' Now for the controversial bit . I still think it's worthwhile to turn the string module into a package. The sandbox has an implementation of this that is completely backward compatible. I like it because it segregates the deprecated functions into a separate sub-module and makes the whole shebang easier to maintain. It also provides an existing namespace for future expansion, as has been discussed before. Please note: PEP 292 is not dependent on the string module -> package reorganization, so it isn't documented in that PEP. I don't think it's worth writing a PEP just for this library reorganization. Take a look at the sandbox if you're interested. http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/string/ As far as this aspect of the changes go, I would like BDFL pronouncement on it. I'll abide by Guido's decision here. If he nixes the re-org, then I propose to put the template and safe_template classes in Lib/string.py. The libstring.tex changes will still be relevant. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/15fbe4b9/attachment.pgp From john at grulic.org.ar Wed Aug 11 00:46:44 2004 From: john at grulic.org.ar (John Lenton) Date: Wed Aug 11 00:47:09 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz> References: <200408061445.i76EjqT07026@guido.python.org> <200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz> Message-ID: <20040810224644.GA9758@grulic.org.ar> On Mon, Aug 09, 2004 at 01:05:48PM +1200, Greg Ewing wrote: > > Perhaps someone can post some real-life use cases written with this > syntax, so we can see what it would *really* look like in typical use, > as opposed to made-up worst-case examples? well, how about this: def someMethod(klass, anarg, otharg, lastarg): """ Summary. Long description that explains the details about someMethod. Aliquam venenatis orci in risus. Nunc ornare aliquam lectus. Integer ligula turpis, posuere id, commodo ut, molestie id, est. Donec eu odio. Fusce at tellus in erat iaculis suscipit. Nulla metus dui, tristique vel, posuere sed, consectetuer sed, pede. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Mauris in orci sed metus porta auctor. @pre some_validator(lastarg) @post rv_validator(__return__) @type anarg: aClass @param anarg: desc of anarg @type otharg: othClass @param otharg: desc of otharg @type lastarg: lastClass @param lastarg: longer desc of lastarg because it's special @rtype: rClass @return: return value description """ pass someMethod = classmethod(someMethod) (yes, I have code that looks like this, roughly, from memory; I make no claims as to typicality) this could be rendered as (inventing some stuff here, probably suboptimally; also, I assume you can access the args of the function by name---I'm unable to verify that at this time): @param(anarg, aClass, 'desc of anarg') @param(otharg, othClass, 'desc of otharg') @param(lastarg, lastClass, "longer desc of lastarg because it's special") @return(rClass, 'return value description') @precondition(some_validator, lastarg) @postcondition(rv_validator) @classmethod def someMethod(klass, anarg, otharg, lastarg): """ Summary. Long description that explains the details about someMethod. Aliquam venenatis orci in risus. Nunc ornare aliquam lectus. Integer ligula turpis, posuere id, commodo ut, molestie id, est. Donec eu odio. Fusce at tellus in erat iaculis suscipit. Nulla metus dui, tristique vel, posuere sed, consectetuer sed, pede. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Mauris in orci sed metus porta auctor. """ pass versus def someMethod(klass, anarg, otharg, lastarg) \ [param(anarg, aClass, 'desc of anarg'), param(otharg, othClass, 'desc of otharg'), param(lastarg, lastClass, "longer desc of lastarg because it's special"), return(rClass, 'return value description'), precondition(some_validator, lastarg), postcondition(rv_validator), classmethod]: """ Summary. Long description that explains the details about someMethod. Aliquam venenatis orci in risus. Nunc ornare aliquam lectus. Integer ligula turpis, posuere id, commodo ut, molestie id, est. Donec eu odio. Fusce at tellus in erat iaculis suscipit. Nulla metus dui, tristique vel, posuere sed, consectetuer sed, pede. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Mauris in orci sed metus porta auctor. """ pass I must say I found decorators' '@' ugly until I thought about using it for doing what epydoc and contract do in docstrings... -- John Lenton (john@grulic.org.ar) -- Random fortune: Se olvida una buena acci?n, y no un buen bofet?n. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/d4de6078/attachment.pgp From guido at python.org Wed Aug 11 02:02:42 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 11 02:02:50 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: Your message of "Tue, 10 Aug 2004 12:17:14 EDT." References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> <200408101501.i7AF1HZ25725@guido.python.org> Message-ID: <200408110002.i7B02gu26415@guido.python.org> > Sooo should (for 'generator' in objects that claim to be in > __builtins__ but aren't), > 1) 'generator' be added to __builtins__ > 2) 'generator' be added to types.py and its __module__ be set to 'types' > 3) 'generator' be added to .py and its __module__ be set to > '' (and a name for the module chosen) I guess (1). Please submit a patch to SF... --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Aug 11 02:09:54 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 11 02:10:01 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: Your message of "Tue, 10 Aug 2004 22:07:20 BST." <1092172040.1681.9.camel@localhost> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> Message-ID: <200408110009.i7B09s626481@guido.python.org> > No, it has always been this way round. In fact test_order was inherited > from Guido's original version of test_decorators.py (from patch #926860) > where it read: > > def test_order(self): > class C(object): > [funcattrs(abc=1), staticmethod] > def foo(): return 42 > # This wouldn't work if staticmethod was called first > self.assertEqual(C.foo(), 42) > self.assertEqual(C().foo(), 42) > > (i.e. identical to the current version except for the change in > syntax). In fact I relied on the fact that this test passed to > convince me I had the order right! But I should have spotted the > inconsistency between that and the documentation that I wrote for > the reference manual. Oops. When using a list, it makes sense to apply the decorators left-to-right. But when using @deco, I think it makes more sense that the decorators closer to the def are applied first. IOW the syntax is conceptually right-recursive: definition ::= '@' decorator definition | 'def' ... > I'll do a patch to fix the order and the corresponding tests. Great. > While I'm at it, do we want to drop support for multiple decorators > on a single line? Nobody has spoken up for it, and in fact forcing > one-per-line would simplify the grammar (as well as making it easier > for toy parsers to find decorators). +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From jack at performancedrivers.com Wed Aug 11 03:11:16 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Wed Aug 11 03:11:22 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <200408072105.i77L5Ea10965@guido.python.org> References: <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> <200408072105.i77L5Ea10965@guido.python.org> Message-ID: <20040811011116.GF23725@performancedrivers.com> On Sat, Aug 07, 2004 at 02:05:14PM -0700, Guido van Rossum wrote: > > I just wanted to say that I believe it should be allowed to decorate > > classes. There's not reason enough (reading the thread linked to by the > > PEP) to limit decorators this way. Like said several times in that > > thread, metaclasses are harder (to write and to understand) than > > decorators. > > > > Also, considering that you said you would have actually wanted > > docstrings above the definition of a function, wouldn't this apply > > to classes as well? > > > > Anyway, it's an artificial limit and it would be better to be able > > to test it out during the alpha. > > Are you volunteering to implement it? > I'm game, I would prefer to use class decorators in most of the places I currently use metaclasses. I dropped the original patch author (Mark Russel) an email yesterday after looking through the 4kloc patch fishing for advice. If someone would clue me in on which files are generated and which ones need to be done by hand it would be a big help. It wasn't obvious to me (other than compile.c and tokenizer.c) on first reading. TIA, -Jack From mkc at mathdogs.com Wed Aug 11 03:29:32 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Wed Aug 11 03:29:37 2004 Subject: [Python-Dev] Re: re.split on empty patterns References: <20040807145142.GB2529@rogue.amk.ca> <1f7befae040807105878b47c61@mail.gmail.com> Message-ID: "Fredrik Lundh" writes: > the RE module already have an established method for specifying flags, > either with (?x) markers in the pattern strings, or via the "flags" argument > to re.compile and others. > > any reason why you cannot use a new standard flag for this purpose? > > (e.g. (?e) and re.EMPTY_OK) I considered this when I was writing the patch, but it seemed like the "(?x)"-style flags all have to do with the meaning and interpretation of the pattern. They're quite general and make sense for all of the methods that use patterns. The "empty okay" concept, on the other hand, seems to have to do specifically with how re.split uses the pattern. That is, in the process of finding matches decides (or not) to discard the empty matches. Would you even want the other methods to have this "do what I say but also discard empty matches" wart? IMO, the only reason for the existence of the option is to preserve backward compatibility with previous code that assumes empties will be discarded. This is something we want to go away eventually, IMO. > for extra credit, add EMPTY_OK behaviour to find, finditer, and sub/subn. I'd vote -1, since there's no backward compatibility issue for these. If someone writing new code doesn't want to match empty strings, they can easily write a pattern that doesn't match empty strings. Mike From bac at OCF.Berkeley.EDU Wed Aug 11 03:30:24 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 11 03:30:58 2004 Subject: [Python-Dev] Prothon on CPython intrepreter? [PROTHON] In-Reply-To: <000a01c47f0b$b3cf4690$0b01a8c0@mark> References: <000a01c47f0b$b3cf4690$0b01a8c0@mark> Message-ID: <411976B0.2020705@ocf.berkeley.edu> Mark Hahn wrote: [SNIP] > I'm sure you've heard the joke that ends with "we've established you're a > whore, now we are just haggling over the price". If I can compromise > Prothon for .Net, why can't I compromise it some more for CPython? So I am > thinking of doing a Prothon implementation in CPython before doing the .Net > version. I will get a library and community fast like I would have with > .Net. > I am personally very glad to hear this! > This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly > easy my wild ideas for the interpreter into Prothon. It will also make > CPython a multiple language VM for the first time. > > So, I'll finally get to some questions: > [SNIP - question for Guido] > 2) Lenard Lindstrom said he thought I could "Add some extra byte code > instructions to handle Prothon's distinct scoping rules and implement > prototypes using PyType_Type". That is definitely possible. If you look at the opcodes that currently exist (http://www.python.org/dev/doc/devel/lib/bytecodes.html for the in-dev docs; doesn't list LIST_APPEND or NOP, though ... Raymond, you planning on documenting these, or should I file a bug report?), there are opcodes for loading from locals, globals, names (when you don't know which scope), and cells (closure stuff). Basically the opcode just specifies where to look in the code object for the value. So yes, you could emit another opcode if you needed to for scope and you made the appropriate changes to the code object if you need other fields. > He also said "All attribute accesses would be > through a special attribute handler method". I don't know the Python > interpreter code yet, but does this sound workable to everyone? It doesn't > have to be Python compatible. > Well, all attribute access goes through descriptors already (if I remember correctly from PyCON 2003's sprint). You could just change the default descriptor to do what you want. > 3) Could Tim Peter's welcome offer to Prothon here on Python-dev be > stretched to include the many questions I would have when modifying CPython? > In other words would this project be welcomed as a sandbox for Python or > would it be an unwelcome drain on your limited resources? > =) You are are free to ask, just don't be floored if you are warnocked on occasion (warnocking, since I have only heard this in the Perl world form the perl-6-internals list, is when a message gets no response). I personally have no issue and actually welcome the discussion since it could help with improving CPython with new ideas. And this is coming from someone who will have to read the emails. =) I know I am personally willing to answer your questions. I do have a piece of advice before you delve into this too deeply, and that is to consider looking at the AST branch (tagged as ast-branch in CVS). Jeremy Hylton has been leading a rewrite of the compiler to take the concrete syntax tree, generate an AST for it, and then emit opcode from that. It is *much* easier to use than how compile.c works now. It also, in my opinion, makes things clearer by breaking up compilation into more, smaller steps. It isn't in the main trunk yet, but it will be someday (I personally plan to work on it once my thesis is done so that I can hopefully integrate my work into the core more easily and it is now tradition for some people to sprint on it at PyCON =) . Who knows, you might be able to just generate your own version of the AST or just have a different compiler for the AST itself and thus cut down and code duplication. -Brett From bac at OCF.Berkeley.EDU Wed Aug 11 03:36:47 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 11 03:36:53 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1092177771.21839.54.camel@localhost> References: <1092177771.21839.54.camel@localhost> Message-ID: <4119782F.8040902@ocf.berkeley.edu> Barry Warsaw wrote: [SNIP] > Now for the controversial bit . > Can't leave well enough alone, can you, Barry? =) > I still think it's worthwhile to turn the string module into a package. > The sandbox has an implementation of this that is completely backward > compatible. I like it because it segregates the deprecated functions > into a separate sub-module and makes the whole shebang easier to > maintain. It also provides an existing namespace for future expansion, > as has been discussed before. > In case Guido cares, I personally am (and have been) +1 on the re-organization. -Brett From greg at cosc.canterbury.ac.nz Wed Aug 11 03:46:59 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 11 03:47:07 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <2mvffqzum9.fsf@starship.python.net> Message-ID: <200408110146.i7B1kxua009746@cosc353.cosc.canterbury.ac.nz> > AAAARGHGH! lookdict "isn't allowed" to raise > exceptions, so it does a PyErr_Clear(), goes on to futilely chase the > entire dict looking for another match on the hash code, and we've > effectively turned a MemoryError into a KeyError. This suggests to me that the very concept of a function that's not allowed to return errors is dubious, since even just calling another function could provoke a MemoryError. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From mark at prothon.org Wed Aug 11 03:54:38 2004 From: mark at prothon.org (Mark Hahn) Date: Wed Aug 11 03:54:28 2004 Subject: [Python-Dev] Re: Prothon on CPython intrepreter? [PROTHON] References: <000a01c47f0b$b3cf4690$0b01a8c0@mark> <411976B0.2020705@ocf.berkeley.edu> Message-ID: <2przc3fdgcvh.1swvqarmq0ngp.dlg@40tude.net> On Tue, 10 Aug 2004 18:30:24 -0700, Brett C. wrote: ...... Thanks for your words of support. > I do have a piece of advice before you delve into this too deeply, and > that is to consider looking at the AST branch (tagged as ast-branch in > CVS). Jeremy Hylton has been leading a rewrite of the compiler to take > the concrete syntax tree, generate an AST for it, and then emit opcode > from that. It is *much* easier to use than how compile.c works now. It > also, in my opinion, makes things clearer by breaking up compilation > into more, smaller steps. It isn't in the main trunk yet, but it will > be someday (I personally plan to work on it once my thesis is done so > that I can hopefully integrate my work into the core more easily and it > is now tradition for some people to sprint on it at PyCON =) . > > Who knows, you might be able to just generate your own version of the > AST or just have a different compiler for the AST itself and thus cut > down and code duplication. I would like to use the AST since IronPython does. This might make moving from CPython to .Net easier later. Also Paul Prescod has been bugging me to use the AST in general. I am planning on using stackless to get the lightweight threads so I don't know how much trouble it will be to integrate all these versions. Clearly my codebase will be separate from everyone elses so I don't have to worry about compatibility, but it would be nice to somehow make it easy to get fixes from the Python codebase into mine easily. From greg at cosc.canterbury.ac.nz Wed Aug 11 03:57:03 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 11 03:57:09 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <20040810224644.GA9758@grulic.org.ar> Message-ID: <200408110157.i7B1v30u009783@cosc353.cosc.canterbury.ac.nz> > > Perhaps someone can post some real-life use cases written with this > > syntax, > > well, how about this: > > def someMethod(klass, anarg, otharg, lastarg): > """ > Summary. > > Long description that explains the details about someMethod. > Aliquam venenatis orci in risus. Nunc ornare aliquam I was hoping for some real live code, e.g. an excerpt from PyObjC. This is still a made-up example. > @param(anarg, aClass, 'desc of anarg') > @param(otharg, othClass, 'desc of otharg') > @param(lastarg, lastClass, "longer desc of lastarg because it's special") > @return(rClass, 'return value description') > @precondition(some_validator, lastarg) > @postcondition(rv_validator) > @classmethod > def someMethod(klass, anarg, otharg, lastarg): > """ > Summary. And this looks awful - the method name is almost completely lost under the deluge of preceding junk! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From michael.walter at gmail.com Wed Aug 11 04:14:30 2004 From: michael.walter at gmail.com (Michael Walter) Date: Wed Aug 11 04:14:33 2004 Subject: [Python-Dev] Re: def fn (args) [dec,dec]: In-Reply-To: <200408110157.i7B1v30u009783@cosc353.cosc.canterbury.ac.nz> References: <200408110157.i7B1v30u009783@cosc353.cosc.canterbury.ac.nz> Message-ID: <877e9a1704081019142dcaa0f6@mail.gmail.com> Let's see how we could reduce that in a (hypothetical) optionally-typed Python, supposed that we use meaningful variable+classnames (thus usually avoiding the need for parameter descriptions): @precondition(some_validator, lastarg) @postcondition(rv_validator) @classmethod def someMethod(klass, anarg : aClass, otharg : othClass, lastarg : lastClass) -> rClass: # foo By moving preconditions into the type (which is often possible -- in the other case we often don't want to expose the precondition publically, anyway, at least in my limited experience), we can reduce this some more: @postcondition(rv_validator) @classmethod def someMethod(klass, anarg : aClass, otharg : othClass, lastarg : lastClassSubType) -> rClass: # foo The only remaining question would be whether the postcondition should be revealed in the method's "interface" (which I consider the decorations to be part of) - I would also think that usually this isn't the case (question: would you put them into the code in the form of asserts instead? or use the old someMethod = postconditioned(someMethod, rv_validator) syntax?). I'm pretty sure I've made too many assumptions for too many people (and I don't really see the point of my post yet), but oh well :) Cheers, Michael On Wed, 11 Aug 2004 13:57:03 +1200, Greg Ewing wrote: > > > Perhaps someone can post some real-life use cases written with this > > > syntax, > > > > well, how about this: > > > > def someMethod(klass, anarg, otharg, lastarg): > > """ > > Summary. > > > > Long description that explains the details about someMethod. > > Aliquam venenatis orci in risus. Nunc ornare aliquam > > I was hoping for some real live code, e.g. an excerpt from > PyObjC. This is still a made-up example. > > > @param(anarg, aClass, 'desc of anarg') > > @param(otharg, othClass, 'desc of otharg') > > @param(lastarg, lastClass, "longer desc of lastarg because it's special") > > @return(rClass, 'return value description') > > @precondition(some_validator, lastarg) > > @postcondition(rv_validator) > > @classmethod > > def someMethod(klass, anarg, otharg, lastarg): > > """ > > Summary. > > And this looks awful - the method name is almost completely > lost under the deluge of preceding junk! > > > > Greg Ewing, Computer Science Dept, +--------------------------------------+ > University of Canterbury, | A citizen of NewZealandCorp, a | > Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | > greg@cosc.canterbury.ac.nz +--------------------------------------+ > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com > From tim.peters at gmail.com Wed Aug 11 04:26:31 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 11 04:26:34 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <200408110146.i7B1kxua009746@cosc353.cosc.canterbury.ac.nz> References: <200408110146.i7B1kxua009746@cosc353.cosc.canterbury.ac.nz> Message-ID: <1f7befae04081019262cf60167@mail.gmail.com> [Tim Peters] >> AAAARGHGH! lookdict "isn't allowed" to raise >> exceptions, so it does a PyErr_Clear(), goes on to futilely chase the >> entire dict looking for another match on the hash code, and we've >> effectively turned a MemoryError into a KeyError. [Greg Ewing] > This suggests to me that the very concept of a function > that's not allowed to return errors is dubious, since > even just calling another function could provoke a > MemoryError. I believe I had a more elegant way to say that: AAAARGHGH! Alas, 10 billion call sites, including in external extensions, rely on that a NULL return from PyDict_GetItem() or PyDict_GetItemString() is the unexceptional "nope, the key's not there". So there's no realistic chance of repairing this before Python 3.0 (if even then). In the meantime, KeyError is confusable with almost all exceptions. For some real fun, ask on a Zope list about hasattr() suppressing all exceptions. Not that Guido ever expected a hasttr() call to try to mess with persistent objects across a networked database as side effect. Python originally had only string-keyed dicts, and the basic dict API still reflects that a dict lookup simply "can't fail" for any reason other than "key ain't there". That used to be true. For that matter, hasattr() was originally no worse than a string-keyed dict lookup too, so "can't fail" also made good sense there at the time. From mkc at mathdogs.com Wed Aug 11 05:01:21 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Wed Aug 11 05:01:27 2004 Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression Group Matching References: <411429FB.6080603@gradient.cis.upenn.edu> <4118C626.4070203@heneryd.com> Message-ID: Erik Heneryd writes: > Mike Coleman wrote: > > Paul Moore writes: > > >>The only other comment I have is that the semantics seem pretty > >>complex - I think that in practice, they do more or less what you want > >>them to, but the description is pretty obscure. > > I'd say that is probably a warning flag, but I don't know, at the same time it > looks like it could be kind of useful. It concerns me a little, too. No doubt the documentation could be better. But if you're already familiar with re.match and structure interpolation (or whatever it's called): x = 1 y = [3, 4, 5] z = [x, y, [x]] z -> [1, [3 4, 5], [1]] then to me it has a certain ring of familiarity. > Exceptions are the way to go. You should get what you ask for, or a heads up. > Errors should never pass silently. I absolutely agree on not being silent on error. The alternative would be to return an error index (i.e., an integer) on error and then have the caller observe the type of the return value (integer on failuer, list on success) to determine success. But again, throwing an exception does seem more natural. > I hacked up a Python prototype, mostly as a challenge. It's brittle and it's > ugly, but it kind of works. Though, if /F says it would require a major > rewrite of the C-engine, that's probably true... Impressive--I thought that'd take a lot more code. I'll study it. Thanks for taking up the challenge! Mike From kbk at shore.net Wed Aug 11 05:15:37 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Aug 11 05:15:42 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200408110315.i7B3FbIe016888@h006008a7bda6.ne.client2.attbi.com> Patch / Bug Summary ___________________ Patches : 284 open ( +8) / 2514 closed (+21) / 2798 total (+29) Bugs : 744 open (-22) / 4361 closed (+44) / 5105 total (+22) RFE : 149 open ( +3) / 130 closed ( +0) / 279 total ( +3) New / Reopened Patches ______________________ Simple File fix for Windows Runtime incompatability (2004-08-04) http://python.org/sf/1003535 opened by Jeremy Schwartz fix for bugs 976878, 926369, 875404 (pdb bkpt handling) (2004-08-04) http://python.org/sf/1003640 opened by Ilya Sandler Fix for 1002475 (Feedparser not handling \r\n correctly) (2004-08-05) CLOSED http://python.org/sf/1003693 opened by Tony Meyer add socketpair function to socket module (2004-08-05) CLOSED http://python.org/sf/1003700 opened by Dave Cole Fix markup in concrete.tex (2004-08-05) CLOSED http://python.org/sf/1003861 opened by Dima Dorfman adds an index entry for __all__ to the tutorial (2004-08-05) CLOSED http://python.org/sf/1003863 opened by Clinton Roy comparison of Decimal instance with None fails (2004-08-05) CLOSED http://python.org/sf/1004018 opened by Anthony Tuininga big code objects (>64K) may be optimized incorrectly (2004-08-05) CLOSED http://python.org/sf/1004088 opened by Cezary Biernacki untested patch for compile.c (bug 1004088) (2004-08-05) CLOSED http://python.org/sf/1004095 opened by Cezary Biernacki Make func_name writable (2004-08-06) http://python.org/sf/1004703 opened by Michael Hudson curses.wrapper should also forward keyword args (2004-08-07) CLOSED http://python.org/sf/1005008 opened by Rob Nichols test_curses fails on MacOS X 10.3 (2004-08-07) CLOSED http://python.org/sf/1005123 opened by Ronald Oussoren property to get the docstring from fget (2004-08-08) http://python.org/sf/1005461 opened by Dima Dorfman Markup tweaks to @deco examples in libref (2004-08-08) CLOSED http://python.org/sf/1005465 opened by Dima Dorfman fix inspect.getargs() crash on def foo((bar)) (bug #891637) (2004-08-08) http://python.org/sf/1005466 opened by Johannes Gijsbers Disambiguate "min() or max()" exception string (2004-08-08) http://python.org/sf/1005468 opened by Dima Dorfman use __name__ = '__main__' in tools/scripts (bug #624893) (2004-08-08) CLOSED http://python.org/sf/1005491 opened by Johannes Gijsbers Avoid seg fault if list object is modified during list.index (2004-08-09) http://python.org/sf/1005778 opened by Dima Dorfman replacing rfc822.formatdate (2004-08-09) http://python.org/sf/1005857 opened by Anthony Baxter support --with-tsc on PPC (2004-08-09) http://python.org/sf/1005891 opened by Michael Hudson test_curses bugs (2004-08-09) http://python.org/sf/1005892 opened by Alexandr Zamaraev curses for win32 (2004-08-09) http://python.org/sf/1005895 opened by Alexandr Zamaraev Patch to allow building of paper-*/dist.pdf (2004-08-09) http://python.org/sf/1005913 opened by Jeff Epler Cygwin standard module build problems (2004-08-09) http://python.org/sf/1006003 opened by Jason Tishler Cygwin standard module build problems (2004-08-09) http://python.org/sf/1006003 opened by Jason Tishler make inspect.getsource show @decorators (2004-08-09) CLOSED http://python.org/sf/1006219 opened by Simon Percivall cross compile patch (2004-08-09) http://python.org/sf/1006238 opened by Daniel Goertzen Match Solaris def'n of _XOPEN_SOURCE (2004-08-10) http://python.org/sf/1006629 opened by Skip Montanaro Remove ODBC library references from Windows build (2004-08-11) http://python.org/sf/1006916 opened by Nick Coghlan Remove ODBC library references from Windows build (2004-08-11) http://python.org/sf/1006916 opened by Nick Coghlan Build Patch #941881 (PEP 309 in C) on windows (2004-08-11) http://python.org/sf/1006948 opened by Nick Coghlan Patches Closed ______________ O(1) xrange membership testing (2004-08-02) http://python.org/sf/1002085 closed by loewis Allow any encodings other than latin-1 in interactive interp (2004-04-24) http://python.org/sf/941229 closed by perky Fix for 1002475 (Feedparser not handling \r\n correctly) (2004-08-04) http://python.org/sf/1003693 closed by bwarsaw add socketpair function to socket module (2004-08-05) http://python.org/sf/1003700 closed by davecole fix build of docs in info format (2004-06-15) http://python.org/sf/972991 closed by doko Fix markup in concrete.tex (2004-08-05) http://python.org/sf/1003861 closed by akuchling adds an index entry for __all__ to the tutorial (2004-08-05) http://python.org/sf/1003863 closed by mwh efficient string concatenation (2004-06-27) http://python.org/sf/980695 closed by rhettinger comparison of Decimal instance with None fails (2004-08-05) http://python.org/sf/1004018 closed by rhettinger big code objects (>64K) may be optimized incorrectly (2004-08-05) http://python.org/sf/1004088 closed by rhettinger untested patch for compile.c (bug 1004088) (2004-08-05) http://python.org/sf/1004095 closed by cezary curses.wrapper should also forward keyword args (2004-08-07) http://python.org/sf/1005008 closed by mwh test_curses fails on MacOS X 10.3 (2004-08-07) http://python.org/sf/1005123 closed by mwh Nails down the semantics of dict setitem (2003-06-03) http://python.org/sf/748126 closed by mwh httplib.HTTPConnection.request() bug (2004-07-25) http://python.org/sf/997626 closed by jhylton assert should not generate code if optimized (2003-11-05) http://python.org/sf/836879 closed by jhylton Update kwargs in pickle docs to match implementations (2004-07-28) http://python.org/sf/999280 closed by akuchling installing modules are explained in old filenames (2003-12-18) http://python.org/sf/862531 closed by akuchling Markup tweaks to @deco examples in libref (2004-08-08) http://python.org/sf/1005465 closed by akuchling use __name__ = '__main__' in tools/scripts (bug #624893) (2004-08-08) http://python.org/sf/1005491 closed by akuchling proposed patch for posixpath.py: getctime() (2003-04-12) http://python.org/sf/720188 closed by kbk Cygwin standard module build problems (2004-08-09) http://python.org/sf/1006003 closed by jlt63 make inspect.getsource show @decorators (2004-08-09) http://python.org/sf/1006219 closed by percivall New / Reopened Bugs ___________________ segfault when running smtplib example (2004-08-04) http://python.org/sf/1003195 opened by Bertram Scharpf telnetlib on Windows: read_until() freezes (2004-08-04) CLOSED http://python.org/sf/1003323 opened by Yannick Turgeon Python 1.5.2 security vulnerability (2004-08-04) http://python.org/sf/1003471 opened by Kirby Kuehl xrange overflows (2004-08-05) http://python.org/sf/1003935 reopened by hfuru xrange overflows (2004-08-05) http://python.org/sf/1003935 opened by Hallvard B Furuseth Shortcut keys don't work when CAPS LOCK is on (2004-08-05) CLOSED http://python.org/sf/1004217 opened by otto imaplib.IMAP4.select doesn't return critical data (2004-08-05) CLOSED http://python.org/sf/1004271 opened by Charles Type returned from .keys() is not checked (2004-08-06) CLOSED http://python.org/sf/1004629 opened by ben handley Type returned from .keys() is not checked (2004-08-06) CLOSED http://python.org/sf/1004669 opened by ben handley Int Type Documentation incomplete (2004-08-06) CLOSED http://python.org/sf/1004698 opened by Colin J. Williams Carbon.File fails if server vol is mounted after launch (2004-08-06) http://python.org/sf/1004810 opened by Paul Pharr Limited User gets error in 2.4a2 MSI (2004-08-06) CLOSED http://python.org/sf/1004837 opened by Martin v. L?wis iso-2022-jp crashes (2004-08-07) CLOSED http://python.org/sf/1005078 opened by Martin v. L?wis test__locale fails on MacOS X (2004-08-07) http://python.org/sf/1005113 opened by Ronald Oussoren new.code() not cleanly checking its arguments (2004-08-07) http://python.org/sf/1005248 opened by Armin Rigo _XOPEN_SOURCE issue on IRIX 5.3 (2004-08-08) http://python.org/sf/1005308 opened by Georg Schwarz inconsistent acceptance of floats for range() (2004-08-07) http://python.org/sf/1005325 opened by Brett Cannon _SC_PAGE_SIZE unknown on IRIX 5.3 (2004-08-08) http://python.org/sf/1005568 opened by Georg Schwarz compile errors on _codecs_iso2022.c (2004-08-09) http://python.org/sf/1005737 opened by roadkill imaplib.IMAP4.getquotaroot error (2004-08-09) CLOSED http://python.org/sf/1005933 opened by Thierry FLORAC Index entries with "_" are wrong in pdf file (2004-08-09) http://python.org/sf/1005936 opened by Jeff Epler leaking snippet (2003-09-18) http://python.org/sf/808596 reopened by mwh FutureWarning when running regression tests (2004-08-09) http://python.org/sf/1006001 opened by Viktor Ferenczi odbc queries with very long strings fail with no exceptions (2004-08-09) CLOSED http://python.org/sf/1006056 opened by leve distutils builds too many rpms (2004-08-10) CLOSED http://python.org/sf/1006607 opened by Jeff MacDonald Possible memory leak in output console (2004-08-10) http://python.org/sf/1006740 opened by Kirk Strauser Bugs Closed ___________ socketmodule does not build under cygwin (2004-08-02) http://python.org/sf/1001857 closed by jlt63 unicode.width broken for combining characters (2004-07-12) http://python.org/sf/989185 closed by lemburg test_logging fails if run twice (2004-08-03) http://python.org/sf/1002537 closed by vsajip variable reuse in the logging module (2004-07-16) http://python.org/sf/992397 closed by vsajip telnetlib on Windows: read_until() freezes (2004-08-04) http://python.org/sf/1003323 closed by mwh email message parser doesn't handle \r\n correctly (2004-08-03) http://python.org/sf/1002475 closed by bwarsaw Compiler warnings for Modules/readline.c:flex_complete() (2004-07-09) http://python.org/sf/988300 closed by bcannon xrange overflows (2004-08-05) http://python.org/sf/1003935 closed by tim_one strftime() backwards incompatibility (2004-07-24) http://python.org/sf/997368 closed by bwarsaw Shortcut keys don't work when CAPS LOCK is on (2004-08-05) http://python.org/sf/1004217 closed by kbk bdist_rpm license documented, but not accepted as valid (2004-07-05) http://python.org/sf/985478 closed by melicertes imaplib.IMAP4.select doesn't return critical data (2004-08-05) http://python.org/sf/1004271 closed by melicertes Type returned from .keys() is not checked (2004-08-06) http://python.org/sf/1004629 closed by rhettinger Type returned from .keys() is not checked (2004-08-06) http://python.org/sf/1004669 closed by rhettinger Int Type Documentation incomplete (2004-08-06) http://python.org/sf/1004698 closed by mwh Limited User gets error in 2.4a2 MSI (2004-08-06) http://python.org/sf/1004837 closed by loewis test_decimal fails if repeated (2004-08-03) http://python.org/sf/1002530 closed by rhettinger doctest.DocTestSuite does not support __test__ (2003-07-16) http://python.org/sf/772091 closed by tim_one iso-2022-jp crashes (2004-08-07) http://python.org/sf/1005078 closed by loewis replace file() with open() in libcvs.tex (2004-07-26) http://python.org/sf/998307 closed by akuchling httplib fails on Akamai URLs (2004-01-11) http://python.org/sf/874842 closed by jhylton pickletools module is not documented (2004-01-08) http://python.org/sf/873146 closed by akuchling readline function pointer with not tty's (2004-07-23) http://python.org/sf/996259 closed by mwh test_tcl fails -- (2004-02-26) http://python.org/sf/905046 closed by mwh AIX POLLNVAL definition causes problems (2004-03-25) http://python.org/sf/923315 closed by akuchling typo in os.popen4 documentation (2004-07-26) http://python.org/sf/998066 closed by akuchling sre.sub documentation is incomplete (2004-07-14) http://python.org/sf/990792 closed by akuchling PyObject_CallMethod docs unclear (with dangerous results) (2004-07-15) http://python.org/sf/991883 closed by akuchling PyArg_ParseTuple can miss errors with warnings as exceptions (2004-07-15) http://python.org/sf/991812 closed by mwh Pyton 2.3.4 Make Test Fails on Mac OS X (2004-06-10) http://python.org/sf/970799 closed by bcannon SimpleHTTPServer docs out of date (2003-12-27) http://python.org/sf/866222 closed by akuchling List comprehensions leaking control variable name deprecated (2003-10-20) http://python.org/sf/827209 closed by akuchling 2.3a2 site.py non-existing dirs (2003-02-25) http://python.org/sf/693255 closed by bcannon Webchecker error on http://www.naleo.org (2002-08-08) http://python.org/sf/592441 closed by mwh Uninitialized variable used in Tools/faqwiz/faqwiz.py (2003-07-25) http://python.org/sf/777659 closed by mwh On HPUX 11i universal newlines seems to cause readline(s) to (2004-06-25) http://python.org/sf/979872 closed by mwh doctest fails with TypeError (2003-07-02) http://python.org/sf/764504 closed by tim_one doctest chokes on recursive members (2003-12-23) http://python.org/sf/864944 closed by tim_one doctest confused by super-instances in class-dicts (2004-02-23) http://python.org/sf/902628 closed by tim_one texi2html.py not working (2004-05-06) http://python.org/sf/948881 closed by anthonybaxter imaplib.IMAP4.getquotaroot error (2004-08-09) http://python.org/sf/1005933 closed by pierslauder odbc queries with very long strings fail with no exceptions (2004-08-09) http://python.org/sf/1006056 closed by loewis pydoc shows tools/scripts but blows up (2002-10-17) http://python.org/sf/624893 closed by akuchling Dubious use of Unicode literals in tutorial (2003-07-18) http://python.org/sf/773351 closed by ghaering distutils builds too many rpms (2004-08-10) http://python.org/sf/1006607 closed by jonez Ctrl+key combos stop working in IDLE (2003-10-31) http://python.org/sf/833957 closed by kbk New / Reopened RFE __________________ Adding missing ISO 8859 codecs, especially Thai (2004-08-02) http://python.org/sf/1001895 reopened by loewis PEP 263: help locating the offending character (2004-08-06) http://python.org/sf/1004533 opened by Michael Hudson translate Windows cr-lf when installing scripts on Linux (2004-08-06) http://python.org/sf/1004696 opened by Tim Cera extend strptime to understand logging timestamps (2004-08-10) http://python.org/sf/1006786 opened by Skip Montanaro RFE Closed __________ Adding missing ISO 8859 codecs, especially Thai (2004-08-02) http://python.org/sf/1001895 closed by lemburg From mkc at mathdogs.com Wed Aug 11 05:27:21 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Wed Aug 11 05:27:26 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: "Fredrik Lundh" writes: > Mike Coleman wrote: > > If I may wax anthropomorphic, the 're.match' function says to me as a programmer > > > > *You* know what structure this RE represents, and *I* know what > > structure it represents, too, because I had to figure it out to > > do the match. > > that only shows that you dont understand how regular expressions work. Ouch. > a regular expression defines a set of strings, and the RE engine is designed to > tell you if a string you have is a member of this set. Yes, I agree. In particular, the re.match function "proves" that a particular regular expression pattern matches a string or that it does not. In the process of doing that, each subexpression in the pattern must be matched against some (possibly empty) substring of the string. The re.match function has all of this knowledge implicitly and ephemerally, and for parenthesized groups, it even remembers this information and allows the caller access to it. Now given all that, doesn't it seem odd that I can get complete information about concatenated subexpressions >>> re.match(r'(...)(...)', 'abcdef').groups() ('abc', 'def') and alternating subexpressions >>> re.match(r'(a..)|(z..)', 'abcdef').groups() ('abc', None) but not repeating subexpressions >>> re.match(r'(..)*', 'abcdef').groups() ('ef',) ? What's the logic of that? That's the limitation I'd like to remove. Now, I understand that doing this will be less efficient, which is why this is a separate method and might not be able to share as much implementation with re.match as would otherwise be desirable. But I still think this is a gap that really needs to be filled. So, is my proposal the best way to do it? Not necessarily. I'd love to see a better alternative. > the engine's not a parser, and it has a very vague notion of "structure" > (groups are implemented by "marks", which are registers that keeps track of > where the engine last passed a given position; changing that to "lists of > all possible matches" would require a major rewrite). I was thinking that the modifications necessary wouldn't be that bad, since the search algorithm itself is not being disturbed. I could be wrong of course. (Note that we're not saving "all possible matches", just the multiple matches involved in the successful match of a repeat group.) > you're probably better off using the scanner mechanism: > > http://effbot.org/zone/xml-scanner.htm > > or the sre.Scanner class (see the source code). the scanner concept could > need some documentation, and it would be nice to be able to switch patterns > during parsing. (time for a scanner PEP, anyone?) I saw this but thought the code was dead. An example or two of usage in a docstring would be worth its weight in gold. It looks like a useful feature--I'd be +1 for it. I'm not sure straightaway how to use the scanner mechanism to implement re.structmatch, though. I'll look at it. Mike From ncoghlan at iinet.net.au Wed Aug 11 07:46:10 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 11 07:46:23 2004 Subject: [Python-Dev] Removing ODBC references In-Reply-To: <4114594E.2030606@v.loewis.de> References: <4111DC2C.8070302@ocf.berkeley.edu> <41140C77.1040802@iinet.net.au> <4114594E.2030606@v.loewis.de> Message-ID: <4119B2A2.6060902@iinet.net.au> Martin v. L?wis wrote: > Feel free to contribute patches. I think I already said that I see no > problem with the Python script being distributed along with Python, > provided its author(s) are willing to contribute it (it would be good > if there was a single dedicated contact for maintaining it, though). > > For the odbc references, please contribute patches to take the > references out of the projects. We really don't need odbc in Python. Done - posted to SF as Patch #1006916. Should definitely be checked by someone with Visual Studio, since I just used sed to get rid of them. Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From ncoghlan at iinet.net.au Wed Aug 11 07:46:25 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 11 07:46:35 2004 Subject: [Python-Dev] Re: re.split on empty patterns In-Reply-To: References: <20040807145142.GB2529@rogue.amk.ca> <1f7befae040807105878b47c61@mail.gmail.com> Message-ID: <4119B2B1.9020401@iinet.net.au> Mike Coleman wrote: > The "empty okay" concept, on the other hand, seems to have to do specifically > with how re.split uses the pattern. That is, in the process of finding > matches decides (or not) to discard the empty matches. Hmm, to me, this suggests 'split_empty = False' as your new parameter for re.split. Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From ncoghlan at iinet.net.au Wed Aug 11 07:47:20 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 11 07:47:29 2004 Subject: [Python-Dev] Decorators with arguments are curries! In-Reply-To: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com> References: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com> Message-ID: <4119B2E8.7020809@iinet.net.au> Phillip J. Eby wrote: > At 10:36 PM 8/7/04 +1000, Andrew Durdin wrote: > >> def bar_decorator(func, param): >> print param >> return func >> > Decorator syntax does *not* provide currying. You have to write > something like this: > Or, if the 'partial' function is in the standard lib by that point, you can use it to get your decorator. . . X>>> @partial(bar_decorator(param_val)) X... def myFoo(): pass Cheers, Nick. (P.S. Does the interactive interpreter currently do a line continuation after the decorator line?) -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From ncoghlan at iinet.net.au Wed Aug 11 07:52:08 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 11 07:52:18 2004 Subject: [Python-Dev] Decorators with arguments are curries! In-Reply-To: <4119B2E8.7020809@iinet.net.au> References: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com> <4119B2E8.7020809@iinet.net.au> Message-ID: <4119B408.2040700@iinet.net.au> Nick Coghlan wrote: > (P.S. Does the interactive interpreter currently do a line continuation > after the decorator line?) Never mind. It occurred to me that my build enivironment works now, so I just went and checked what current CVS does ;) Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From stephen at xemacs.org Wed Aug 11 08:06:31 2004 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed Aug 11 08:07:52 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae0408101258431039a1@mail.gmail.com> (Tim Peters's message of "Tue, 10 Aug 2004 15:58:22 -0400") References: <000001c47e88$81ef1d40$5635c797@oemcomputer> <1f7befae0408092348583f0953@mail.gmail.com> <2mvffqzum9.fsf@starship.python.net> <1f7befae0408101258431039a1@mail.gmail.com> Message-ID: <877js6gqe0.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Tim" == Tim Peters writes: Tim> I guess you really can't spell Windows XP without Windows Tim> <0.9 wink>. Gee, and on your consistent recommendation, I was just about to upgrade from Linux and Mac OS X! -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From lists at hlabs.spb.ru Wed Aug 11 13:31:58 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Wed Aug 11 09:26:01 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <411774F2.7040408@interlink.com.au> References: <41176596.5060209@hlabs.spb.ru> <2m7js83733.fsf@starship.python.net> <41179A6A.90905@hlabs.spb.ru> <411774F2.7040408@interlink.com.au> Message-ID: <411A03AE.40606@hlabs.spb.ru> Anthony Baxter wrote: > Dmitry Vasiliev wrote: > >> >> Quote from PEP-237: >> >> """ >> A new type, >> integer, may be introduced that is an abstract base type of >> which both the int and long implementation types are >> subclassed. This is useful so that programs can check >> integer-ness with a single test: >> >> if isinstance(i, integer): ... >> """ >> >> So maybe correct question then: is there a plan for implementing the >> integer type in Python 2.4? > > Well, we're a couple of weeks from what will hopefully be the last > alpha. I'd _hope_ anything like this would be in before then (although > we can have things that change behaviour up until b1, I'd prefer that > a3->b1 is just fixing bugs that require changes, or minor changes, not > major changes). OTOH, I'm not sure how deep a change like this would be. > > In any case, if it's a change that you (or anyone else) feels strongly > about, a patch on SF would be a first step towards making this happen. So here it is: http://sourceforge.net/tracker/index.php?func=detail&aid=1007068&group_id=5470&atid=305470 -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From ncoghlan at iinet.net.au Wed Aug 11 10:24:24 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 11 10:24:36 2004 Subject: [Python-Dev] Request for bug closures Message-ID: <4119D7B8.8030507@iinet.net.au> There are a couple of bugs which can probably be closed: #1006001 (Future warning in test_format.py): Looks like a problem with the user accidentally running the 2.4 regrtest.py with the 2.3 interpreter #1006740 ('Memory leak' in long running console window): Looks like the user has encountered a Windows 'feature', not a Python bug. Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From stephen at xemacs.org Wed Aug 11 10:51:29 2004 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed Aug 11 10:51:38 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching In-Reply-To: (Mike Coleman's message of "09 Aug 2004 21:38:08 -0400") References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Mike" == Mike Coleman writes: Mike> "Stephen J. Turnbull" writes: >> >>>>> "Mike" == Mike Coleman writes: Mike> m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz') >> Sure, but regexp syntax is a horrible way to express that. Mike> Do you mean, horrible compared to spelling it out using a Mike> Python loop that walks through the array, I didn't have that in mind, but I can live with it. Regexp doesn't compare well even with that when you consider error-checking and program maintenance. Mike> or horrible compared to some more serious parsing package? That's more like it. It doesn't have to be all that serious, either. It's not obvious to me how to make grammar rules pretty in Python, but implementing an SLR parser-generator should be at most a couple of days' work to get something you could live with. Note that one thing about the LR parser itself is that the stack machine is quite generic; you can start with SLR, and go on to add more powerful parser table generators later. Mike> I certainly wouldn't want to see someone try to write a Mike> language front-end with this, but for a lot of text-scraping Mike> activities, I think it would be very useful. I agree it would be useful. I think that a parser generator would be equally useful for this, as easy to learn as regexps are (assuming somebody comes up with a tasteful way to express grammar rules), and more powerful with even a basic implementation. BTW, do you have a sample implementation for re.structmatch? Fredrik seems to have some doubt about ease of implementation. >> This feature would be an attractive nuisance, IMHO. Mike> I agree that, like list comprehensions (for example), it Mike> needs to be applied with good judgement. I don't see the analogy to list comprehensions. My objection is not that re.structmatch can express a request for a huge amount of data in compact form. If you've got the RAM, go right ahead. My objection is that it throws away a lot of structure, and therefore is liable to return the _wrong_ parse, or simply an error with no hint as to where the data is malformed. (AFAICS this is the practical implication of Fredrik's "theoretical" statement "the RE engine is designed to tell you if a string you have is a member of this set". Yes, the Python re engine can provide some annotations for a well-formed string, but in the end, well-formedness of the string is yes/no, with no error diagnosis.) Sure, there are applications like reading a matrix where the structure is as uniform as the "*" operator implies. But what do you get from that? A list of lists of _strings_ ... and you need a nested loop to turn those strings into numbers that your linear algebra package can use. No? I just don't see the benefits in code reduction you claim. Mike> Turning it around, though, why *shouldn't* there be a good Mike> mechanism for returning the multiple matches for multiply Mike> matching groups? Because they're all too often not "multiple matches". They're matches for different objects that happen to match the same regular expression, and the notation for the parser should express that without requiring comments. re.structmatch _encourages_ a style where users deliberately throw away structure; your /etc/group parser is an example. (See below.) Furthermore, since this is going to be recursive, users are going to end up with "list of list of ..." nested to arbitrary depth, depending on how complex the regexp is. Isn't that what we have Lisp for? Mike> As things stand right now, though, it's a serious irritation Mike> that we have a standard mechanism that *almost* does this, Mike> but quits at the last moment. If I may wax anthropomorphic, Mike> the 're.match' function says to me as a programmer Mike> *You* know what structure this RE represents, and *I* Mike> know what structure it represents, too, because I had to Mike> figure it out to do the match. But too bad, sucker, I'm not Mike> going to tell you what I found! Mike> Irritating as hell. Problem is, you (for some values of "you") and the program both _know_ what structure it represents, but you have different structures in mind. This is a standard source of regexp bugs. The structmatch extension will exacerbate the problem. I suppose you would be satisfied if you could represent file := file line line := group ':' passwd ':' number ':' any '\n' group := '[A-Za-z0-9]+' passwd := '[A-Za-z0-9]+' number := '[0-9]+' any := '.*' by '(([A-Za-z0-9]+:)*.*\n)*' in which case I agree it's pretty harmless. But note well, you've already allowed arbitrary words instead of numbers for 'number', and improving 'passwd' to check for the characteristic signature of a crypted password or a "no password authentication" flag would be annoying with re.structmatch, while it's trivial with a parser. You say, "that's a matter of application of good judgement". My experience is that when it comes to regexps, application of good judgement is pretty rare. I have a lot of experience with parsing done by regexp, none of it pleasant. Have some pity on the c.l.py regulars who will be debugging the monstrosities that some people will come up with! <0.5 wink> I grant that a full-blown parser generator is overkill for your target set of applications. But I stick with my judgement that regexps as they are are an attractive nuisance, and re.structmatch would make the situation worse in proportion to its usage. I hope your proposal will be superseded either by a scanner package, or even by a parser package. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From mwh at python.net Wed Aug 11 13:22:25 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 11 13:22:27 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1092177771.21839.54.camel@localhost> (Barry Warsaw's message of "Tue, 10 Aug 2004 18:42:51 -0400") References: <1092177771.21839.54.camel@localhost> Message-ID: <2mr7qdzzpq.fsf@starship.python.net> Barry Warsaw writes: > Raymond and I had a chance to talk about PEP 292 on bug day, and we've > exchanged a few private emails and implementations. I think we've > finally knocked everything into shape for inclusion in Python 2.4's next > alpha. > > In the sandbox you will find the updated implementation, along with > updated tests and documentation (libstring.tex). > > The updated PEP is at: > > http://www.python.org/peps/pep-0292.html It's a silly point, but my instant impression on reading This PEP describes a simpler string substitution feature, also known as string interpolation. is "simpler than *what*?". Perhaps a better title for the PEP would just be "simple string interpolation". Otherwise, I like it. The only issue I have with putting it in string is that this makes it a touch harder to bundle up for use with scripts that work with older Python's (I take it the implementation is pure Python and works or can be fairly easily be made to work with 2.2?). But I can live with try: from string import template except ImportError: from string_compat import template Cheers, mwh -- Exam invigilation - it doesn't come much harder than that, esp if the book you're reading turns out to be worse than expected. -- Dirk Bruere, sci.physics.research From dima at trit.org Wed Aug 11 13:24:41 2004 From: dima at trit.org (Dima Dorfman) Date: Wed Aug 11 13:27:13 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: <200408092019.i79KJPw23893@guido.python.org> References: <410DDFE3.7010006@interlink.com.au> <200408021850.i72IoCg28650@guido.python.org> <410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org> <200408091525.i79FPLM23023@guido.python.org> <20040809173415.GB28961@panix.com> <200408092019.i79KJPw23893@guido.python.org> Message-ID: <20040811112441.GH82909@trit.org> Guido van Rossum wrote: > > I'd suggest that only ``from foo import (...)`` be permitted (where > > "..." does not include ``*``). But I don't care much, and I'll > > update the PEP as soon as it's clear which precise versions are > > allowed. > > This seems to be the consensus so far, so why don't you add all that > detail to the PEP. I just uploaded the updated patch as SF #1007189. That version is complete and, as agreed, only accepts the from/import form. It could use a good review, but otherwise it should be ready to go. Thanks, Dima. From fredrik at pythonware.com Wed Aug 11 13:38:25 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed Aug 11 13:38:52 2004 Subject: [Python-Dev] Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> <2mr7qdzzpq.fsf@starship.python.net> Message-ID: Michael Hudson wrote: > It's a silly point, but my instant impression on reading > > This PEP describes a simpler string substitution feature, also > known as string interpolation. > > is "simpler than *what*?". that's explained in the next sentence, of course. This PEP is "simpler" in two respects: [...] > The only issue I have with putting it in string is that this makes it > a touch harder to bundle up for use with scripts that work with older > Python's (I take it the implementation is pure Python and works or can > be fairly easily be made to work with 2.2?). here's code that implements the "simpler" syntax, and works with 1.6 and later: http://effbot.org/zone/re-sub.htm#simple-templating (from what I saw from the checkins, the PEP implementation isn't exactly optimal, but I guess we can leave that for the next Optimization Day ;-) From mwh at python.net Wed Aug 11 13:40:09 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 11 13:40:12 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: (Fredrik Lundh's message of "Wed, 11 Aug 2004 13:38:25 +0200") References: <1092177771.21839.54.camel@localhost> <2mr7qdzzpq.fsf@starship.python.net> Message-ID: <2mn011zyw6.fsf@starship.python.net> "Fredrik Lundh" writes: > Michael Hudson wrote: > >> It's a silly point, but my instant impression on reading >> >> This PEP describes a simpler string substitution feature, also >> known as string interpolation. >> >> is "simpler than *what*?". > > that's explained in the next sentence, of course. > > This PEP is "simpler" in two respects: > [...] Yes, I know. It still annoyed me :-) Cheers, mwh -- ZAPHOD: Listen three eyes, don't try to outwierd me, I get stranger things than you free with my breakfast cereal. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From fperez528 at yahoo.com Wed Aug 11 14:06:12 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Wed Aug 11 14:06:23 2004 Subject: [Python-Dev] Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> Message-ID: Barry Warsaw wrote: > Raymond and I had a chance to talk about PEP 292 on bug day, and we've > exchanged a few private emails and implementations. I think we've > finally knocked everything into shape for inclusion in Python 2.4's next > alpha. > > In the sandbox you will find the updated implementation, along with > updated tests and documentation (libstring.tex). > > The updated PEP is at: > > http://www.python.org/peps/pep-0292.html If I understood things correctly, this proposal will not allow the evaluation of any kind of expression, including attributes. With the existing interpolation system, the following happens: In [2]: print 'sys.platform is: %(sys.platform)s' % locals() --------------------------------------------------------------------------- KeyError Traceback (most recent call last) ? KeyError: sys.platform I understand why this is the case, but it has always struck me as an unpleasant aspect of string interpolation, since it often forces me to create throw-away locals when I want to print data stored as attributes of an object. It is easy to end up with code that looks like: _x,_y,_z = self.x, self.y,self.z # throwaways for printing print """\ The coordinates are: x = %(_x)s y = %(_y)s z = %(_z)s """ % locals() Simply because trying to do %(self.x)s won't work. This just feels ugly, IMHO. I wonder if it would be possible to at least allow this kind of expansion (without going to the full-blown possibilities of PEP 215) in pep-292. I know there are tricks to produce this kind of evaluation already (which I sometimes use). But since pep-292 is trying to introduce a standard interpolation mechanism which is easy to use for the common cases, and object attributes are such a pervasive part of the language, I hope this addition can be considered. Best, f ps. Forgive me if the above is already allowed; if that is the case, it might be worth pointing it out in an example in the PEP. From fperez528 at yahoo.com Wed Aug 11 14:16:59 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Wed Aug 11 14:17:08 2004 Subject: [Python-Dev] Re: [SPAM: 3.611] IPython, @, and option E from the wiki References: <200408090202.i7922diB005813@cosc353.cosc.canterbury.ac.nz> Message-ID: Greg Ewing wrote: > 3. In this kind of setup, using | instead of @ would be ok as well, I guess: > Fernando Perez : > >> def bar(low,high): >> |accepts(int,int) >> |returns(float) > > How about > > def bar(low,high): > ...accepts(int,int) > ...returns(float) The ellipsis is alredy valid python syntax elsewhere: In [7]: a=arange(3**3) In [8]: a.shape=(3,3,3) In [9]: a[:,...,:] Out[9]: array([[[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8]], [[ 9, 10, 11], [12, 13, 14], [15, 16, 17]], [[18, 19, 20], [21, 22, 23], [24, 25, 26]]]) I don't think it would be a good idea to make it do this kind of double-duty. best, f From jlgijsbers at planet.nl Wed Aug 11 14:17:27 2004 From: jlgijsbers at planet.nl (Johannes Gijsbers) Date: Wed Aug 11 14:20:55 2004 Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome) References: <20040809213820.GC11199@rogue.amk.ca> <2mllgn15cl.fsf@starship.python.net> Message-ID: Michael Hudson python.net> writes: > Well apart from jlgijsbers who you mentioned in your post > > I don't know how much longer we should let him get away with not being > a committer, though . > > Cheers, > mwh Not very long . Being a committer would be great, though how much time I can spend on Python really depends on the rest of my life. Consider this a request for checkin privileges. Some of you may have seen me on the wiki or the pydotorg list, but a short introduction seems in order. As you've probably noticed, my name is Johannes Gijsbers. I run a small company with a friend; we use Python whenever we can. Also, I've just finished the first year of my philosophy bachelor. This year the plan is not to continue studying philosophy, but to work for six months and then to travel around in South America until my money runs out. I'll see what happens after that. I discovered Python in March 2001 and haven't really looked back since. I admire other languages (e.g. Lisp, Smalltalk) in some ways, but I can't use them for real work. I've been helping out with the wiki and pydotorg for some time now and the Python Bug Days sucked me into helping out with Python itself as well. Besides programming, my main interests are literature and its relation with philosophy, juggling, communities in general and wiki culture in particular. Johannes From adurdin at gmail.com Wed Aug 11 14:21:57 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Wed Aug 11 14:21:59 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: References: <1092177771.21839.54.camel@localhost> Message-ID: <59e9fd3a040811052152548b6b@mail.gmail.com> On Wed, 11 Aug 2004 06:06:12 -0600, Fernando Perez wrote: > Barry Warsaw wrote: > > > Raymond and I had a chance to talk about PEP 292 on bug day, and we've > > exchanged a few private emails and implementations. I think we've > > finally knocked everything into shape for inclusion in Python 2.4's next > > alpha. > > > > In the sandbox you will find the updated implementation, along with > > updated tests and documentation (libstring.tex). > > > > The updated PEP is at: > > > > http://www.python.org/peps/pep-0292.html Forgive me for being obtuse, but I can't see anything significantly simpler in this PEP than the standard string interpolation, except for the case where the identifier doesn't need braces: template("Hello, $name, how are you?") % {name: "Jim"} versus "Hello, %(name)s, how are you?") % {name: "Jim"} Apart from removing the need for the parentheses/braces in this case, all it does is consistently remove the "s", and replace the % with $, and then limiting the interpolated values to strings (or will it automatically use str(x) for any non-string variable x?). Is there any other benefit to this that I'm overlooking? From barry at python.org Wed Aug 11 15:50:35 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 11 15:50:21 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <2mr7qdzzpq.fsf@starship.python.net> References: <1092177771.21839.54.camel@localhost> <2mr7qdzzpq.fsf@starship.python.net> Message-ID: <1092232235.9941.15.camel@localhost> On Wed, 2004-08-11 at 07:22, Michael Hudson wrote: > It's a silly point, but my instant impression on reading > > This PEP describes a simpler string substitution feature, also > known as string interpolation. > > is "simpler than *what*?". Perhaps a better title for the PEP would > just be "simple string interpolation". As others have pointed out, that's described in the text of the PEP. > Otherwise, I like it. Cool! > The only issue I have with putting it in string is that this makes it > a touch harder to bundle up for use with scripts that work with older > Python's (I take it the implementation is pure Python and works or can > be fairly easily be made to work with 2.2?). But I can live with > > try: > from string import template > except ImportError: > from string_compat import template To be brutally honest (about our work, that is ;), the template class is so damn simple you could stick its definition in the except clause. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040811/3c44549a/attachment.pgp From barry at python.org Wed Aug 11 15:55:37 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 11 15:55:23 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: References: <1092177771.21839.54.camel@localhost> Message-ID: <1092232536.9945.21.camel@localhost> On Wed, 2004-08-11 at 08:06, Fernando Perez wrote: > If I understood things correctly, this proposal will not allow the evaluation of > any kind of expression, including attributes. With the existing interpolation > system, the following happens: > > In [2]: print 'sys.platform is: %(sys.platform)s' % locals() > --------------------------------------------------------------------------- > KeyError Traceback (most recent call last) > > ? > > KeyError: sys.platform You don't mean the 'existing' interpolation system, because this obviously doesn't work in Python today, although with a fancy enough implementation of 'locals()', it could. That actually doesn't change much with PEP 292. I specifically designed it so that you could subclass from template, so as to accept the extended placeholder syntax (i.e. dots), and then implement a mapping-like class to do the lookups in any namespace you want. In fact, my earlier implementation contained such a class, but it was deemed too complex to win the name 'simpler' so it was left out of the library. It's a trivial amount of code and my own personal use case has a need for such a feature. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040811/ca79d64c/attachment-0001.pgp From mcherm at mcherm.com Wed Aug 11 15:57:41 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Aug 11 15:56:37 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) Message-ID: <1092232661.411a25d55a567@mcherm.com> My patch (attached) accepts > > import (os, sys) > from sys import (stdin, stdout, stderr) > import (os) > from sys import (*) > > but rejects > > from (sys) import stdin > import (os), (sys) > import (os,) > > Should any of those be allowed? Anything that I missed? Others have already mentioned that "from sys import xxx, xxx, xxx" is the case that badly needs line wrapping. I would, however, like to suggest that (if it's easy to do) you allow a trailing comma when parenthesees are used. Of course this looks silly: from mymodule import (xxx, xxx, xxx, xxx,) But in the most extreme case, like this: from myBigModule import ( xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, ) it's awefully nice on cut-and-paste to allow the trailing comma. But the most important reason for doing so is that Python already allows (and disregards) an extra trailing comma in similar lists: >>> [1,2,3,] [1, 2, 3] >>> {1:1,2:2,3:3,} {1: 1, 2: 2, 3: 3} >>> def func(a,b,c,): ... print a, b, c ... >>> func(1,2,3) 1 2 3 >>> (1,2,3,) (1, 2, 3) -- Michael Chermside PS: The length-one-tuple is a special case (and a bit of a wart) in that it MANDATES the use of the trailing comma. But that's got nothing to do with the basic principle. From barry at python.org Wed Aug 11 15:58:03 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 11 15:57:49 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <59e9fd3a040811052152548b6b@mail.gmail.com> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> Message-ID: <1092232683.9945.26.camel@localhost> On Wed, 2004-08-11 at 08:21, Andrew Durdin wrote: > Forgive me for being obtuse, but I can't see anything significantly > simpler in this PEP than the standard string interpolation, except for > the case where the identifier doesn't need braces: > > template("Hello, $name, how are you?") % {name: "Jim"} > versus > "Hello, %(name)s, how are you?") % {name: "Jim"} > > Apart from removing the need for the parentheses/braces in this case, > all it does is consistently remove the "s", and replace the % with $, > and then limiting the interpolated values to strings (or will it > automatically use str(x) for any non-string variable x?). > Is there any other benefit to this that I'm overlooking? The elimination of the trailing 's' is definitely a benefit, as is the adoption of $-placeholders, since that tends to be the placeholder introducing character in many other languages. In terms of usability, it's a win. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040811/9ab236ac/attachment.pgp From mcherm at mcherm.com Wed Aug 11 16:18:47 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Aug 11 16:17:46 2004 Subject: [Python-Dev] Re: Decimal type question [Prothon] Message-ID: <1092233927.411a2ac77a160@mcherm.com> Mark Hahn writes: > I am more interested now in hearing about people's feelings > on the idea of > having the Decimal type "in-between" Int and Float. Assume a decent > Decimal implementation is used. All right. I think that "in-between" Int and Float is a bad idea -- or at least that it's a bad way to think about a decent design. "in-between" suggests that Float has greater range than Decimal which has greater range than Int... but in your design that simply isn't true. Your Int, for instance, can represent values that neither Decimal nor Float can handle. So the "right" way (IMHO) to think about it is as three *different* classes of numbers. There are Ints (can only handle integers, but able to manage any of those), Decimals (can only handle certain rational numbers but it happens to be those that humans often find useful; also has an upper bound and a precision limit (which may vary with size depending on what implementation you use)), and Floats (can only handle certain rational numbers most of which are NOT the same as those handled by Decimal; also has upper bounds and precision limits; runs real fast). I would recomend that you AVOID "automatic" conversion (just like Python does) but that you make sure that when a (pure) number can be represented in two ways that they compare equal. So the following would be true: Int(1) == Decimal(1.0) == Float(1.e0) Decimal(2.5) == Float(2.5e0) This is pretty much what you are already doing... for instance, Int(1) / Int(3) ---> Decimal(0.3333333333333333) because there is a "/" operator which accepts two Ints or Decimals and returns a Decimal. I'd propose that there's also another "/" operator which accepts two Floats and returns a Float. And that passing one Float and one (Int or Decimal) to "/" results in a type error. The cos() function would accept one number (of any type) and return a Float. There's also the question (and it's an important one!) of what literals to use. I agree (and everyone else would too) that literals like "1234" should create Ints. I also agree (as would everyone else) that literals like "123.45e6" should create Floats. The contraversial bit is whether literals like "123.45" should create Decimals... but I think that they should. My reasons are (1) for newbies, matching what people expect is more important than speed... and that's exactly why Decimal is better than Float, and (2) if not, then what form WOULD you use for Decimal literals? (I suppose there's always the trailing "d".) From amk at amk.ca Wed Aug 11 16:39:20 2004 From: amk at amk.ca (A.M. Kuchling) Date: Wed Aug 11 16:39:50 2004 Subject: [Python-Dev] Request for bug closures In-Reply-To: <4119D7B8.8030507@iinet.net.au> References: <4119D7B8.8030507@iinet.net.au> Message-ID: <20040811143920.GB18194@rogue.amk.ca> On Wed, Aug 11, 2004 at 01:24:24AM -0700, Nick Coghlan wrote: > There are a couple of bugs which can probably be closed: > #1006001 (Future warning in test_format.py): > #1006740 ('Memory leak' in long running console window): I've closed both of these. --amk From guido at python.org Wed Aug 11 16:46:17 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 11 16:46:23 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: Your message of "Wed, 11 Aug 2004 06:57:41 PDT." <1092232661.411a25d55a567@mcherm.com> References: <1092232661.411a25d55a567@mcherm.com> Message-ID: <200408111446.i7BEkHI27825@guido.python.org> > Others have already mentioned that "from sys import xxx, xxx, xxx" > is the case that badly needs line wrapping. I would, however, like > to suggest that (if it's easy to do) you allow a trailing comma when > parentheses are used. [...] +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Wed Aug 11 16:52:14 2004 From: aahz at pythoncraft.com (Aahz) Date: Wed Aug 11 16:52:22 2004 Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome) In-Reply-To: References: <20040809213820.GC11199@rogue.amk.ca> <2mllgn15cl.fsf@starship.python.net> Message-ID: <20040811145214.GA20280@panix.com> On Wed, Aug 11, 2004, Johannes Gijsbers wrote: > > I discovered Python in March 2001 and haven't really looked back > since. I admire other languages (e.g. Lisp, Smalltalk) in some ways, > but I can't use them for real work. I've been helping out with the > wiki and pydotorg for some time now and the Python Bug Days sucked me > into helping out with Python itself as well. For those who don't translate automatically, "help with pydotorg" means that he already has checkin privs on creosote (the machine that runs www.python.org). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From fperez528 at yahoo.com Wed Aug 11 16:54:27 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Wed Aug 11 16:54:37 2004 Subject: [Python-Dev] Re: Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> Message-ID: Barry Warsaw wrote: > On Wed, 2004-08-11 at 08:06, Fernando Perez wrote: > >> If I understood things correctly, this proposal will not allow the evaluation >> of >> any kind of expression, including attributes. With the existing >> interpolation system, the following happens: >> >> In [2]: print 'sys.platform is: %(sys.platform)s' % locals() >> --------------------------------------------------------------------------- >> KeyError Traceback (most recent call last) >> >> ? >> >> KeyError: sys.platform > > You don't mean the 'existing' interpolation system, because this > obviously doesn't work in Python today, although with a fancy enough > implementation of 'locals()', it could. That actually doesn't change > much with PEP 292. I specifically designed it so that you could > subclass from template, so as to accept the extended placeholder syntax > (i.e. dots), and then implement a mapping-like class to do the lookups > in any namespace you want. In fact, my earlier implementation contained > such a class, but it was deemed too complex to win the name 'simpler' so > it was left out of the library. It's a trivial amount of code and my > own personal use case has a need for such a feature. I guess my post wasn't worded too clearly. I did mean existing, in the sense that the error above occurs with current python (it's cut-and-paste from python 2.2). My point was precisely to illustrate this failure with a simple example, to then ask whether the new scheme could be made, _as shipped_, to accept this kind of expansion: print template('sys.platform is: $sys.platform') % locals() It may be trivial to extend pep-292 for this, but not having to do it in every small utility you write/ship would be very useful, IMHO. If there are no technical reasons to prevent it, is the 'too complex' argument really that strong (especially since you state it's a trivial amount of code)? I hope this clarifies better my question. Regards, f From skip at pobox.com Wed Aug 11 16:55:23 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Aug 11 16:55:41 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1092232235.9941.15.camel@localhost> References: <1092177771.21839.54.camel@localhost> <2mr7qdzzpq.fsf@starship.python.net> <1092232235.9941.15.camel@localhost> Message-ID: <16666.13147.274810.410339@montanaro.dyndns.org> Barry> On Wed, 2004-08-11 at 07:22, Michael Hudson wrote: >> It's a silly point, but my instant impression on reading >> >> This PEP describes a simpler string substitution feature, also >> known as string interpolation. >> >> is "simpler than *what*?". Perhaps a better title for the PEP would >> just be "simple string interpolation". Barry> As others have pointed out, that's described in the text of the Barry> PEP. >> Otherwise, I like it. Barry> Cool! ... I would personally like to thank Barry for stepping up to the plate and providing a much-needed distraction from the D-word. ;-) Skip From skip at pobox.com Wed Aug 11 16:55:41 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Aug 11 16:56:28 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> <16664.50665.170916.600964@montanaro.dyndns.org> Message-ID: <16666.13165.619920.756435@montanaro.dyndns.org> Andrew> Aren't we really trying to recreate def as a multi-line lambda Andrew> that binds when it's in a context that is not expecting a Andrew> function argument in block form? I suppose you can look at it that way. The output of a function definition plus any decorator applications to it need not be a function though. I posted a simpleminded example of generating a greyscale ramp using a decorator. If I thought about it for more than a few seconds I could probably come up with some other examples as well where the output wasn't a function. Some of them might even be useful. ;-) Skip From edcjones at erols.com Wed Aug 11 17:05:02 2004 From: edcjones at erols.com (Edward C. Jones) Date: Wed Aug 11 17:11:07 2004 Subject: [Python-Dev] Semantics of decorators? Message-ID: <411A359E.3080309@erols.com> In the development docs, Python Reference Manual, 7.5, it says: A function definition may be wrapped by one or more decorator expressions. Decorator expressions are evaluated when the function is defined, in the scope that contains the function definition. The result must be a callable, which is invoked with the function object as the only argument. The returned value is bound to the function name instead of the function object. If there are multiple decorators, they are applied in reverse order. Does this completely describe the semantics of decorators? From amk at amk.ca Wed Aug 11 17:13:47 2004 From: amk at amk.ca (A.M. Kuchling) Date: Wed Aug 11 17:14:18 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1092177771.21839.54.camel@localhost> References: <1092177771.21839.54.camel@localhost> Message-ID: <20040811151347.GA18312@rogue.amk.ca> On Tue, Aug 10, 2004 at 06:42:51PM -0400, Barry Warsaw wrote: > The updated PEP is at: > http://www.python.org/peps/pep-0292.html Are you sure that people aren't going to want to format numbers as they do with % (e.g. %4.2f)? You should post the PEP to c.l.py.announce, too. (Perhaps you have and it's still sitting in the queue.) --amk From John.Marshall at ec.gc.ca Wed Aug 11 17:16:57 2004 From: John.Marshall at ec.gc.ca (John Marshall) Date: Wed Aug 11 17:17:07 2004 Subject: [Python-Dev] A decorator syntax not yet mentioned (I think!) Message-ID: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca> How about the following, which I am almost positive has not been suggested: ----- class Klass: def __init__(self, name): self.name = name deco meth0: staticmethod def meth0(x): return x deco meth1: classmethod def meth1(cls): return cls deco sayhello: funcattrs(name='GvR', language='python') log(file='func.log') def sayhello(self): print 'hello python world' ----- 1) The decorators clearly apply to a specific method/function, therefore there is no need to do any stack pushing in memory until the method/function definition is done. 2) The decorators are "outside" of the method/function they decorate: a) which will please those who want the outside location b) will not be folded within the function c) folding on the decorators can be done so that the def is not obfuscated d) can be located anywhere in the code--but most likely before the "def ...()" 3) The sequence in which the decorators are applied is just like code would be--this is certainly intuitive given that we are writing code. This approach could also be applied to classes in case decorators should ever be extended to them: ----- deco Klass: doc("This is a class to ...") class Klass: : : ----- Any comments? John From edcjones at erols.com Wed Aug 11 17:11:28 2004 From: edcjones at erols.com (Edward C. Jones) Date: Wed Aug 11 17:17:32 2004 Subject: [Python-Dev] Long term plans for Python? Message-ID: <411A3720.8050301@erols.com> What are the long term plans for changes in Python from 2.5 to 3.0 and beyond? From s.percivall at chello.se Wed Aug 11 17:23:12 2004 From: s.percivall at chello.se (Simon Percivall) Date: Wed Aug 11 17:23:15 2004 Subject: [Python-Dev] Semantics of decorators? In-Reply-To: <411A359E.3080309@erols.com> References: <411A359E.3080309@erols.com> Message-ID: <5BEEA740-EBAA-11D8-8A1D-0003934AD54A@chello.se> On 2004-08-11, at 17.05, Edward C. Jones wrote: > In the development docs, Python Reference Manual, 7.5, it says: > > A function definition may be wrapped by one or more decorator > expressions. > Decorator expressions are evaluated when the function is defined, in > the > scope that contains the function definition. The result must be a > callable, > which is invoked with the function object as the only argument. The > returned > value is bound to the function name instead of the function object. If > there > are multiple decorators, they are applied in reverse order. > > Does this completely describe the semantics of decorators? "applied in reverse order" might be a bit ambiguous. Reverse order to what? To their definition?, to their proximity to the function? //Simon From s.percivall at chello.se Wed Aug 11 17:30:04 2004 From: s.percivall at chello.se (Simon Percivall) Date: Wed Aug 11 17:30:08 2004 Subject: [Python-Dev] A decorator syntax not yet mentioned (I think!) In-Reply-To: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca> References: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca> Message-ID: <51A1B43B-EBAB-11D8-8A1D-0003934AD54A@chello.se> On 2004-08-11, at 17.16, John Marshall wrote: > How about the following, which I am almost positive > has not been suggested: > ----- > class Klass: > def __init__(self, name): > self.name = name > > deco meth0: > staticmethod > def meth0(x): > return x > > deco meth1: > classmethod > def meth1(cls): > return cls > > deco sayhello: > funcattrs(name='GvR', language='python') > log(file='func.log') > def sayhello(self): > print 'hello python world' > > Any comments? There is the problem that you have to repeat the name of the function you define. Repetition of this kind leads to error. It also takes more place and distracts more from the actual function than most of the other syntax. Also, there is the "new keyword" problem, leading to breakage if older code uses the "deco" name. //Simon From John.Marshall at ec.gc.ca Wed Aug 11 17:52:26 2004 From: John.Marshall at ec.gc.ca (John Marshall) Date: Wed Aug 11 17:52:33 2004 Subject: [Python-Dev] A decorator syntax not yet mentioned (I think!) In-Reply-To: <492EA476-EBAB-11D8-87F8-000A95686CD8@redivi.com> References: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca> <492EA476-EBAB-11D8-87F8-000A95686CD8@redivi.com> Message-ID: <1092239547.22674.39.camel@mango.cmc.ec.gc.ca> On Wed, 2004-08-11 at 15:29, Bob Ippolito wrote: > On Aug 11, 2004, at 11:16 AM, John Marshall wrote: > > > How about the following, which I am almost positive > > has not been suggested: > > ----- > > class Klass: > > def __init__(self, name): > > self.name = name > > > > deco meth0: > > staticmethod > > def meth0(x): > > return x > > > > deco meth1: > > classmethod > > def meth1(cls): > > return cls > > > > deco sayhello: > > funcattrs(name='GvR', language='python') > > log(file='func.log') > > def sayhello(self): > > print 'hello python world' > > > > ----- > > 1) The decorators clearly apply to a specific method/function, > > therefore there is no need to do any stack pushing in memory > > until the method/function definition is done. > > None of that code can actually execute until the method/function > definition is done, how is this different at all? > I am not refering to actually executing the code but addressing people's point that when they read through the code, they need to keep all the decorator items in memory until they come to the "def ...". One person described the process as push each decorator onto a stack (in his head) until he found the def. > > 2) The decorators are "outside" of the method/function they > > decorate: > > a) which will please those who want the outside location > > b) will not be folded within the function > > Doesn't a) imply b)? > Sure. But this addresses two concerns explicitly. > > c) folding on the decorators can be done so that the > > def is not obfuscated > > This could probably be done with the @syntax too if the editor was > smart enough I imagine it is possible. But the point was to make the signature clearly visible/prominent but also indicate that there _are_ decorators associated with the method/function. > > > d) can be located anywhere in the code--but most likely > > before the "def ...()" > > I don't think that is a bonus at all. > I am not claiming this as a bonus. Rather, I am making it clear that the mention of the method/function name does not require any special/forced placement. > > 3) The sequence in which the decorators are applied is just > > like code would be--this is certainly intuitive given > > that we are writing code. > > That's not really obvious. > I am not claiming that it is obvious but that it could be understood that way (rather than leave it to interpretation, I made it explicit as an option). Note how the sequence issue has come up recently with respect to the @-syntax and the list syntax, which apparently have been understood to apply in the opposite order. > > This approach could also be applied to classes in case > > decorators should ever be extended to them: > > So can any of the other proposals. That may be, but to be able to do so this way makes it an viable option. > > > ----- > > deco Klass: > > doc("This is a class to ...") > > class Klass: > > : > > : > > ----- > > > > Any comments? > > Yeah, this is no less than twice as bad as any of the current decorator > proposals because you have to repeat the function name :) > Personally, I might opt for something else, but having the decorators outside have raised concerns for some. With this approach the tagging makes things explicit, which is often a good thing. The current @-syntax _implicitly_ identifies the decorators with the method/function/class which people have said they don't like. Thanks for the comments. John > -bob From pje at telecommunity.com Wed Aug 11 18:22:04 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 11 18:22:09 2004 Subject: [Python-Dev] Semantics of decorators? In-Reply-To: <411A359E.3080309@erols.com> Message-ID: <5.1.1.6.0.20040811121756.039ef9e0@mail.telecommunity.com> At 11:05 AM 8/11/04 -0400, Edward C. Jones wrote: >In the development docs, Python Reference Manual, 7.5, it says: > >A function definition may be wrapped by one or more decorator expressions. >Decorator expressions are evaluated when the function is defined, in the >scope that contains the function definition. The result must be a callable, >which is invoked with the function object as the only argument. The returned >value is bound to the function name instead of the function object. If there >are multiple decorators, they are applied in reverse order. > >Does this completely describe the semantics of decorators? It should also be noted that when the decorator expressions are applied, the function has not yet been bound to its name in the appropriate scope. Thus, if there is a previous binding of that name in the scope, the decorator expression may be able to access it. This aspect of the semantics is important for additive decorators like Ed Loper's 'property_getter' and 'property_setter', or my generic function decorators. E.g.: @property_getter def foo(self): return self.__foo @property_setter def foo(self,value): self.__foo = value # etc. >_______________________________________________ >Python-Dev mailing list >Python-Dev@python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com From nbastin at opnet.com Wed Aug 11 19:48:42 2004 From: nbastin at opnet.com (Nick Bastin) Date: Wed Aug 11 19:49:05 2004 Subject: [Python-Dev] 2.3.4 on Win64? Message-ID: Has anybody built 2.3.4 on Win64, and if so, is there a preferred way? We built it by exporting the makefiles and twiddling with the linker options and pointer size in pyconfig.h, but that seems a bit kludgy. -- Nick From martin at v.loewis.de Wed Aug 11 20:20:43 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 11 20:20:38 2004 Subject: [Python-Dev] 2.3.4 on Win64? In-Reply-To: References: Message-ID: <411A637B.6080109@v.loewis.de> Nick Bastin wrote: > Has anybody built 2.3.4 on Win64, and if so, is there a preferred way? > We built it by exporting the makefiles and twiddling with the linker > options and pointer size in pyconfig.h, but that seems a bit kludgy. I did (assuming you mean Itanium). I first tried it with that approach, then I talked Peter Tr?ger into developing vsextcomp, which is now used in the CVS. So unless you have a specific reason to use 2.3.4, I recommend to use 2.4. If I had to do it again, I would use a vsextcomp procedure now: Import the VC6 project files into VC7, then, using vsextcomp, generate Itanium targets for them. Regards, Martin P.S. sf.net/projects/vsextcomp From nbastin at opnet.com Wed Aug 11 20:26:33 2004 From: nbastin at opnet.com (Nick Bastin) Date: Wed Aug 11 20:26:56 2004 Subject: [Python-Dev] 2.3.4 on Win64? In-Reply-To: <411A637B.6080109@v.loewis.de> References: <411A637B.6080109@v.loewis.de> Message-ID: On Aug 11, 2004, at 2:20 PM, Martin v. L?wis wrote: > Nick Bastin wrote: >> Has anybody built 2.3.4 on Win64, and if so, is there a preferred >> way? We built it by exporting the makefiles and twiddling with the >> linker options and pointer size in pyconfig.h, but that seems a bit >> kludgy. > > I did (assuming you mean Itanium). I first tried it with that approach, > then I talked Peter Tr?ger into developing vsextcomp, which is now used > in the CVS. So unless you have a specific reason to use 2.3.4, I > recommend to use 2.4. I forgot to mention that the platform in question is AMD64, although we will be building on Itanium as well. Using 2.4 is not an option at this point. -- Nick From fredrik at pythonware.com Wed Aug 11 21:16:58 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed Aug 11 21:15:19 2004 Subject: [Python-Dev] Re: Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> Message-ID: Barry Warsaw wrote: > I specifically designed it so that you could subclass from template, so as to > accept the extended placeholder syntax (i.e. dots), and then implement a > mapping-like class to do the lookups in any namespace you want. on the other hand, to do that, you need to replace the pattern, and make sure that you have the same number of groups, with the same meaning. it's doable, but it's rather fragile. two suggestions: - refactor the RE, and use lastindex to extract the name (see the link I posted earlier). This would let users add any number of patterns, without having to rewrite the __mod__ method. And indexing by lastindex should be faster than tuple unpacking, too. - Consider changing the ${} to accept arbitrary patterns, instead of just Python-style identifiers. \${([^}]+)} should do the trick. - Consider using character classes instead of re.IGNORECASE (easier to read, and IIRC, slightly faster). From mark at prothon.org Wed Aug 11 20:53:21 2004 From: mark at prothon.org (Mark Hahn) Date: Wed Aug 11 21:20:39 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] References: <1092233927.411a2ac77a160@mcherm.com> Message-ID: Michael Chermside wrote: >Mark Hahn writes: >> I am more interested now in hearing about people's feelings on the >> idea of having the Decimal type "in-between" Int and Float. >Assume a >> decent Decimal implementation is used. > >All right. I think that "in-between" Int and Float is a bad >idea -- or at least that it's a bad way to think about a decent design. > >"in-between" suggests that Float has greater range than >Decimal which has greater range than Int... but in your design >that simply isn't true. Your Int, for instance, can represent >values that neither Decimal nor Float can handle. What do you think of the newer idea of using the IBM floating decimal for all numerics? Tim Peters and others pushed this on me. Performance would be handled somewhat by automatically switching between ints, bigints, and decimal internally, but with no loss of precision except when a divide or other such operation happens. I would still have a hardware based binary floating point type that you have to manually specify if you want hardware performance for floats. I am thinking of having all current constant formats default to the "Decimal" type (0, 123, 123.456, 123e3) and have a new constant type using the letter 'f' for the binary "Float" type (0f0, 123f0, 123.456f0, 123.f3). From mzarate at uoguelph.ca Tue Aug 10 18:55:19 2004 From: mzarate at uoguelph.ca (Martin Zarate) Date: Wed Aug 11 21:43:06 2004 Subject: [Python-Dev] Another approach to decorators. Message-ID: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca> Hello. I've already thrown a few cents into this debate, and have read every post on it. I've used Ruby, so I love being able to make blocks for whatever I want - which is why Python's decorator syntax interests me. Its not as powerful as Ruby blocks, but its a close approach for some things. I can see the problems are legion. I think the fundamental difficulty is that Python is line-oriented, and decorators break this - they either make a gigantic line that the coder remakes as they see fit (very unpython) or a multi-line mess, which Python does not have for any statements. Second is that Python is an extremely legible language, and most decorator syntaxes are not. @ means nothing to an uninformed eye. This violates the most important feature of Python (imho) which is that it is "runnable pseudocode". A coder will see what "def" or "class" or "import" means just by looking at it. @ is meaningless. I submit that a reserved keyword would fit much better in the @ place. Second, that comma-based delimiting (like the global statement) would also be nicer, to avoid line-after-line of decoration. For example, lets say our keyword is "remake" - a bad word, a better one should be used, but I'm tired of thinking of one. remake functor(foo, bar) remake staticmethod def baz(): pass or, alternately, remake functor(foo,bar), staticmethod def baz(): pass but either way, the system doesn't look very Python, as you've a multi-line statement. I submit that the most Python solution, that would also be legible (but not necessarily very pretty) would be to actually make the decorator a bona-fide block that you nest your class or function into. remake functor(foo, bar), staticmethod: def baz(): pass This concretely shows the relationship of the original define statement to its wrapper objects. The principle is simple - the remake block will close with only one object in its immediate namespace, and that object will be wrapped by each of its argument objects in turn, and place it in its surrounding namespace with the same name. Thus, more general-purpose oddball approaches could be used, such as, lambdas or even rewrapping an external object. For example, in order to import a C-based extension module as a member method in a class... class foo(number): remake instancemethod: sin = math.sin bar = foo() num = bar.sin or something like that. other good keywords that could be used instead of remake: wrap, decorate, as, etc. I'm sure you can all think of something better than @. From barnesc at engr.orst.edu Wed Aug 11 01:12:58 2004 From: barnesc at engr.orst.edu (barnesc@engr.orst.edu) Date: Wed Aug 11 21:43:09 2004 Subject: [Python-Dev] Decorators after 'def' should be reconsidered Message-ID: <1092179578.4119567af011d@webmail.oregonstate.edu> >In the discussion on decorators months ago, solutions involving >special syntax inside the block were ruled out early on. Basically, >you shouldn't have to peek inside the block to find out important >external properties of the function. >--Guido van Rossum (home page: http://www.python.org/~guido/) I disagree. The most important part of a function is the first def line. If you place other stuff before the def line, you make it harder to grep for the def line. Compare: ----------------------------------------------- Option A ----------------------------------------------- """ Hi, Performs a binary operation. Docstring. """ @staticmethod @synchronized(lockvar) def f(a, b): return a + b """ Now, Performs a simple operation. """ @classmethod def g(a): return a ----------------------------------------------- Option B ----------------------------------------------- def f(a, b): @staticmethod @synchronized(lockvar) """ Hi, Performs a binary operation. Docstring. """ return a + b def g(a): @classmethod """ Now, Performs a simple operation. """ return a Now, forget everything you've learned about Python. Forget that @ symbols are ugly. - For which method is it visually easier to find the function def? - For which method is the code in the most logical order? Note that Option A causes you to skim all the way through the docstring and the decorators to find out which function is being defined. At this point, you have to start over at the docstring, to find out what the function actually does. Option A is especially painful when a large number of functions are thrown into a module. It is almost impossible to grep efficiently for the function names. Thus the code appears disorganized, and messy. It seems strange that Option B was "ruled out early on." Option B is clearly more readable. - Connelly From bokr at oz.net Wed Aug 11 21:26:46 2004 From: bokr at oz.net (Bengt Richter) Date: Wed Aug 11 21:43:13 2004 Subject: [Python-Dev] deco_fun_expr(= vs. @deco_fun_expr Message-ID: <411a5943.561998230@news.oz.net> Just wanted to make sure this got into python-dev as an alternative ;-) Example using post-fixed '(=' instead of prefixed '@': deco1(= decomodule.deco2(= def foo(): pass effect is same as with prefixed-@ syntax, i.e., foo = deco1(decomodule.deco2(foo)) (except either way there's no intermediate foo binding during evaluation, I gather?) Advantages: 1. currently illegal syntax, UIAM 2. lets callable-returning expression have unchanged grammar up to new (= trailer part 2. doesn't use up '@' or interfere with tools that use it 3. expresses calling in the proper nested way (inner before outer). No confusion about "backwards." 4. Leaves the door open to callable-calls operating on other code-block-derived arguments than from def and its suite (e.g. class and suite). Leaves door open for explicit =) end-of-block-arg mark if that should turn out useful for demarcation of otherwise ambiguous block endings, for some future use. Unfortunately vexing personal matters will demand priority attention for some time again :-( Bye for now. Regards, Bengt Richter From jack at performancedrivers.com Wed Aug 11 21:44:44 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Wed Aug 11 21:44:49 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <20040811011116.GF23725@performancedrivers.com> References: <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> <200408072105.i77L5Ea10965@guido.python.org> <20040811011116.GF23725@performancedrivers.com> Message-ID: <20040811194444.GG23725@performancedrivers.com> On Tue, Aug 10, 2004 at 09:11:16PM -0400, Jack Diederich wrote: > On Sat, Aug 07, 2004 at 02:05:14PM -0700, Guido van Rossum wrote: > > > I just wanted to say that I believe it should be allowed to decorate > > > classes. There's not reason enough (reading the thread linked to by the > > > PEP) to limit decorators this way. Like said several times in that > > > thread, metaclasses are harder (to write and to understand) than > > > decorators. > > > > Are you volunteering to implement it? > > > I'm game, I would prefer to use class decorators in most of the places > I currently use metaclasses. I have a mostly working patch for class decorators but it gets an XXX ambiguity between funcdef: [decorators] 'def' NAME parameters ':' suite and classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite It works as advertised with the pre-decorators funcdef definition (which removes the ambiguity). Help? -Jack # --- quicktst.py ---- class Tom(object): pass def decor(cls): print "decord'd", cls return Tom @decor class Dick(object): """ some classs """ def bar(self): print "BAR!" return @decor class Harry: def bar(self): print "BAR!" return print Dick print Dick.__class__ print Harry print Harry() print Harry().bar() ./python Python 2.4a2 (#13, Aug 11 2004, 15:25:00) [GCC 3.3.4 (Debian 1:3.3.4-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import quicktst decord'd decord'd tst.Harry Traceback (most recent call last): File "", line 1, in ? File "tst.py", line 25, in ? print Harry().bar() AttributeError: 'Tom' object has no attribute 'bar' >>> From python at rcn.com Wed Aug 11 21:48:20 2004 From: python at rcn.com (Raymond Hettinger) Date: Wed Aug 11 21:49:07 2004 Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome) In-Reply-To: <20040811145214.GA20280@panix.com> Message-ID: <002f01c47fdc$290bfa40$e622c797@oemcomputer> > On Wed, Aug 11, 2004, Johannes Gijsbers wrote: > > > > I discovered Python in March 2001 and haven't really looked back > > since. I admire other languages (e.g. Lisp, Smalltalk) in some ways, > > but I can't use them for real work. I've been helping out with the > > wiki and pydotorg for some time now and the Python Bug Days sucked me > > into helping out with Python itself as well. > > For those who don't translate automatically, "help with pydotorg" means > that he already has checkin privs on creosote (the machine that runs > www.python.org). Does that exempt him from the rule that the new guy has to fix bugs and buy all the current committers a round of beer? Raymond From raymond.hettinger at verizon.net Wed Aug 11 21:51:37 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed Aug 11 21:52:27 2004 Subject: [Python-Dev] Naming nit Message-ID: <003401c47fdc$a0f722a0$e622c797@oemcomputer> For PEP 292, should the class be Template() or template()? Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040811/8649f842/attachment.htm From pje at telecommunity.com Wed Aug 11 21:54:23 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 11 21:54:36 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <20040811194444.GG23725@performancedrivers.com> References: <20040811011116.GF23725@performancedrivers.com> <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> <200408072105.i77L5Ea10965@guido.python.org> <20040811011116.GF23725@performancedrivers.com> Message-ID: <5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com> At 03:44 PM 8/11/04 -0400, Jack Diederich wrote: >I have a mostly working patch for class decorators but it gets an XXX >ambiguity >between > >funcdef: [decorators] 'def' NAME parameters ':' suite >and >classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite > >It works as advertised with the pre-decorators funcdef definition (which >removes the ambiguity). >Help? How about this? decorated_assignment: [decorators] (funcdef | classdef) classdef: 'class' NAME [ '(' testlist ')' ] ':' suite funcdef: 'def' NAME parameters ':' suite From bac at OCF.Berkeley.EDU Wed Aug 11 21:55:22 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 11 21:55:35 2004 Subject: [Python-Dev] Naming nit In-Reply-To: <003401c47fdc$a0f722a0$e622c797@oemcomputer> References: <003401c47fdc$a0f722a0$e622c797@oemcomputer> Message-ID: <411A79AA.4020505@ocf.berkeley.edu> Raymond Hettinger wrote: > For PEP 292, should the class be Template() or template()? > Template as per PEP 8. -Brett From martin at v.loewis.de Wed Aug 11 22:04:06 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 11 22:04:02 2004 Subject: [Python-Dev] 2.3.4 on Win64? In-Reply-To: References: <411A637B.6080109@v.loewis.de> Message-ID: <411A7BB6.4040909@v.loewis.de> Nick Bastin wrote: > I forgot to mention that the platform in question is AMD64, although we > will be building on Itanium as well. Using 2.4 is not an option at this > point. I'd still recommend vsextcomp. Copy the 2.4 project files to 2.3, and replace all occurrences of MS_ITANIUM with MS_OPTERON. Alternatively, create new AMD64 configurations, and contribute that to Python 2.4, then use the project files on 2.3. Regards, Martin From bac at OCF.Berkeley.EDU Wed Aug 11 22:03:54 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 11 22:04:12 2004 Subject: [Python-Dev] Drop support for multiple decorators per line? (was: Decorator order implemented backwards?) In-Reply-To: <1092175256.21841.20.camel@localhost> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <1092175256.21841.20.camel@localhost> Message-ID: <411A7BAA.80703@ocf.berkeley.edu> Barry Warsaw wrote: > On Tue, 2004-08-10 at 17:24, Bob Ippolito wrote: > >>On Aug 10, 2004, at 5:07 PM, Mark Russell wrote: >> >> >>>While I'm at it, do we want to drop support for multiple decorators on >>>a >>>single line? Nobody has spoken up for it, and in fact forcing >>>one-per-line would simplify the grammar (as well as making it easier >>>for >>>toy parsers to find decorators). >> >>+1 > > > +1 > +1 = +3 from me! =) trying-to-get-through-over-1,000-emails-for-the-next-summary-yr's, Brett From martin at v.loewis.de Wed Aug 11 22:11:37 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 11 22:11:32 2004 Subject: [Python-Dev] Another approach to decorators. In-Reply-To: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca> References: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca> Message-ID: <411A7D79.1040302@v.loewis.de> Martin Zarate wrote: > @ means nothing to an uninformed eye. This violates the most important > feature of Python (imho) which is that it is "runnable pseudocode". A coder > will see what "def" or "class" or "import" means just by looking at it. @ is > meaningless. I think it is irrelevant that @ means nothing in this context. Python will still be runnable pseudo-code. Remember that the decorator will read @staticmethod So the eye unfamiliar with Python may wonder what the @ means. As you suggest, it means nothing, and the reader will see the staticmethod after it, and guess (correctly): "this somehow make a static method". > I submit that a reserved keyword would fit much better in the @ place. To the uninformed eye, this would be harmful. It may have a grasp what a static method is, and that you have to declare it. However, what does the additional keyword mean? The uninformed eye will try to put a meaning into it, but there is none. The unformed eye will never guess how static methods work in Python - that the method is actually an object, that "staticmethod" is actually a type, and this all is a constructor call, assigning to the name of the function. > I'm sure you can all think of something better than @. Actually, I can't. Your post just convinced me that a keyword can never be better. Regards, Martin From michel at dialnetwork.com Wed Aug 11 22:37:11 2004 From: michel at dialnetwork.com (Michel Pelletier) Date: Wed Aug 11 22:42:15 2004 Subject: [Python-Dev] Another approach to decorators. In-Reply-To: <20040811195454.9BD311E400A@bag.python.org> References: <20040811195454.9BD311E400A@bag.python.org> Message-ID: <20040811133711.5f1986b6.michel@dialnetwork.com> > Message: 2 > Date: Tue, 10 Aug 2004 12:55:19 -0400 > From: Martin Zarate > Subject: [Python-Dev] Another approach to decorators. > To: python-dev@python.org > Message-ID: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca> > Content-Type: text/plain; charset=ISO-8859-1 > > @ means nothing to an uninformed eye. This violates the most important > feature of Python (imho) which is that it is "runnable pseudocode". Yes! > I submit that the most Python solution, that would also be legible (but not > necessarily very pretty) would be to actually make the decorator a bona-fide > block that you nest your class or function into. > > remake functor(foo, bar), staticmethod: > def baz(): > pass > > This concretely shows the relationship of the original define statement to its > wrapper objects. The principle is simple - the remake block will close with > only one object in its immediate namespace I like your idea a lot, buy why just one? Your scheme of making decorators a block could be applied to several methods in a class: class Foo: decorate static: def static1(blah...): pass def static2(blah...): pass To me, this idea looks more like Python than all the rest, and allows you to consolidate related decorated methods and classes. Nest them to apply decorators "aquisition style": class Foo: decorate static: def static1(blah...): pass decorate locksFile: def static2andLocks(blah...): # both decorators appy pass -Michel From mcherm at mcherm.com Wed Aug 11 22:53:31 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Aug 11 22:52:24 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] Message-ID: <1092257611.411a874b5094e@mcherm.com> > the only reason programmers write loops as integers is > because accumulating floats "doesn't work". With Decimal, that > is no longer the case. That is completely untrue. Binary floating point does just as good a job of handling integers as decimal floating point does. For instance: >>> accumulator = 0.0 >>> for i in range(25000): ... accumulator += 1.0 ... >>> print accumulator 25000.0 Notice, perfect accuracy. Yet people don't tend to use floating point for loops... why not? I'd argue there are at LEAST two other good reasons. Performance (floating point arithmatic is fast, but the equivalent integer arithmatic is normally faster), and clarity of intent (using an int type makes it clear that you intend only to use integers. But lots of languages make other choices. The number of languages that have a single numeric type is pretty large... perl comes to mind, but I bet I could list 10 others without too much trouble. -- Michael Chermside From mcherm at mcherm.com Wed Aug 11 22:55:39 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Aug 11 22:54:32 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] Message-ID: <1092257739.411a87cb46d65@mcherm.com> > What do you think of the newer idea of using the IBM floating > decimal for > all numerics? Tim Peters and others pushed this on me. [...] > I am thinking of having all current constant formats default to the > "Decimal" type (0, 123, 123.456, 123e3) and have a new > constant type using > the letter 'f' for the binary "Float" type (0f0, 123f0, > 123.456f0, 123.f3). +1 -- Michael Chermside From mcherm at mcherm.com Wed Aug 11 23:02:26 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Aug 11 23:01:18 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint Message-ID: <1092258146.411a896246d1a@mcherm.com> > > Is there a plan for implementing a base class for int and long (like > > basestring for str and unicode): > > > > >>> issubclass(int, baseint) and issubclass(long, baseint) > > True > > > > ? > I think this would be a good idea; maybe the name should be > baseinteger? I would like to urge caution before making this change. Despite what the PEP may say, I actually think that creating a 'baseint' type is the WRONG design choice for the long term. I envision an eventual Python which has just one type, called 'int'. The fact that an efficient implementation is used when the ints are small and an arbitrary-precision version when they get too big would be hidden from the user by automatic promotion of overflow. (By "hidden" I mean the user doesn't need to care, not that they can't find out if they want to.) We are almost there already, but if people start coding to 'baseinteger' it takes us down a different path entirely. 'basestring' is a completely different issue -- there will always be a need for both unicode and 8-bit-strings as separate types. -- Michael Chermside From bac at OCF.Berkeley.EDU Wed Aug 11 23:03:11 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 11 23:03:23 2004 Subject: [Python-Dev] Another approach to decorators. In-Reply-To: <20040811133711.5f1986b6.michel@dialnetwork.com> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> Message-ID: <411A898F.3060004@ocf.berkeley.edu> Michel Pelletier wrote: >>Message: 2 >>Date: Tue, 10 Aug 2004 12:55:19 -0400 >>From: Martin Zarate >>Subject: [Python-Dev] Another approach to decorators. >>To: python-dev@python.org >>Message-ID: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca> >>Content-Type: text/plain; charset=ISO-8859-1 >> >>@ means nothing to an uninformed eye. This violates the most important >>feature of Python (imho) which is that it is "runnable pseudocode". > > > Yes! > > >>I submit that the most Python solution, that would also be legible (but not >>necessarily very pretty) would be to actually make the decorator a bona-fide >>block that you nest your class or function into. >> >>remake functor(foo, bar), staticmethod: >> def baz(): >> pass >> >>This concretely shows the relationship of the original define statement to its >>wrapper objects. The principle is simple - the remake block will close with >>only one object in its immediate namespace > > > I like your idea a lot, buy why just one? Your scheme of making decorators a > block could be applied to several methods in a class: > > class Foo: > > decorate static: > > def static1(blah...): > pass > > def static2(blah...): > pass > > To me, this idea looks more like Python than all the rest, and allows > you to consolidate related decorated methods and classes. Nest them to > apply decorators "aquisition style": But then how are you supposed to do multiple decorators for the same method? I am with Martin that a keyword argument is not going to work here. Going that route leads to ``public static painInTheAssAndClutteredSyntax(That can) implements Lots, Of, Stuff, That throws Things, You, Can, Learn, From, The, Docs`` hell. -Brett From bac at OCF.Berkeley.EDU Wed Aug 11 23:09:46 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 11 23:09:57 2004 Subject: [Python-Dev] request: add keywords for static and class methods only [related to decorators discussion] In-Reply-To: <4113BC2C.1010206@vanderbilt.edu> References: <4113BC2C.1010206@vanderbilt.edu> Message-ID: <411A8B1A.7000806@ocf.berkeley.edu> Doug Holton wrote: > After reading some posts here and the wiki page discussing the > decorators: http://www.python.org/moin/PythonDecorators > One idea that some people have suggested is to separate out standard > uses for decorators like function metadata, vs. built-in features of > python, like static methods and class methods. > > So I'm suggesting, don't use decorators for static and class methods. > > Perhaps for 2.5 or whenever, we should add keyword support for built-in > features, just like java: > > These two keywords at least: > > def static mymethod (arg1, arg2): > .... > > def classmethod mymethod( cls, arg1, arg2 ): > .... > > And then you have to decide if you want to support other java built-in > features and keywords like private, protected, public, synchronized. > This is in no way meant to be taken personally, but I am going make the understated statement, "hell no", for this. That means -Inf from me. =) We do not need more keywords for just static and class methods. They don't get used enough to warrant that. -Brett From michel at dialnetwork.com Wed Aug 11 23:28:02 2004 From: michel at dialnetwork.com (Michel Pelletier) Date: Wed Aug 11 23:25:42 2004 Subject: [Python-Dev] Another approach to decorators. In-Reply-To: <20040811142354.661970e0.michel@dialnetwork.com> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> Message-ID: <20040811142802.0386bd60.michel@dialnetwork.com> On Wed, 11 Aug 2004 14:23:54 -0700 Michel Pelletier wrote: > > Put more than one of them in the same decorate block: > > class ... > > decorate staticmethod, locksFile(arg1): > > def meth1(...) > ... > > def meth2(...) > ... Oh, and also for many decorators that look better spanning several lines, use the same syntax proposed for multi-line imports: class ... decorate (staticmethod, blah, blah, locksFile(arg1)): def meth1(...) ... def meth2(...) ... -Michel From michel at dialnetwork.com Wed Aug 11 23:23:54 2004 From: michel at dialnetwork.com (Michel Pelletier) Date: Wed Aug 11 23:27:36 2004 Subject: [Python-Dev] Another approach to decorators. In-Reply-To: <411A898F.3060004@ocf.berkeley.edu> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> Message-ID: <20040811142354.661970e0.michel@dialnetwork.com> On Wed, 11 Aug 2004 14:03:11 -0700 "Brett C." wrote: I said: > > > > class Foo: > > > > decorate static: > > > > def static1(blah...): > > pass > > > > def static2(blah...): > > pass > > > > To me, this idea looks more like Python than all the rest, and allows > > you to consolidate related decorated methods and classes. Nest them to > > apply decorators "aquisition style": > > But then how are you supposed to do multiple decorators for the same method? Put more than one of them in the same decorate block: class ... decorate staticmethod, locksFile(arg1): def meth1(...) ... def meth2(...) ... > I am with Martin that a keyword argument is not going to work here. I presume you mean MvL and not the orginal poster, Martin Zarate. > Going that route leads to ``public static > painInTheAssAndClutteredSyntax(That can) implements Lots, Of, Stuff, > That throws Things, You, Can, Learn, From, The, Docs`` hell. How does Martin Z's idea lead to this? -Michel From fredrik at pythonware.com Thu Aug 12 00:29:02 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu Aug 12 00:30:22 2004 Subject: [Python-Dev] Re: Re: Re: pre-PEP: Complete, Structured Regular ExpressionGroup Matching References: <411429FB.6080603@gradient.cis.upenn.edu> <4118C626.4070203@heneryd.com> <4118D5F7.1090709@heneryd.com> Message-ID: Erik Heneryd wrote: > Yes, of course it's slow, as I recursivly compile subpatterns and try them out, but as things are, > it's probably much easier to implement it this way than using scanners, which I suspect is a lot > of work (given the structmatch interface)... sure, which is why it's a good idea to design a better interface. more on this later. From dave at boost-consulting.com Thu Aug 12 00:38:12 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 12 00:38:56 2004 Subject: [Python-Dev] Re: Another approach to decorators. References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> Message-ID: Michel Pelletier writes: >> From: Martin Zarate > >> I submit that the most Python solution, that would also be legible (but not >> necessarily very pretty) would be to actually make the decorator a bona-fide >> block that you nest your class or function into. >> >> remake functor(foo, bar), staticmethod: >> def baz(): >> pass >> >> This concretely shows the relationship of the original define statement to its >> wrapper objects. The principle is simple - the remake block will close with >> only one object in its immediate namespace > > I like your idea a lot, buy why just one? Your scheme of making decorators a > block could be applied to several methods in a class: > > class Foo: > > decorate static: > > def static1(blah...): > pass > > def static2(blah...): > pass > > To me, this idea looks more like Python than all the rest, and allows > you to consolidate related decorated methods and classes. Nest them to > apply decorators "aquisition style": > > > class Foo: > > decorate static: > > def static1(blah...): > pass > > decorate locksFile: > > def static2andLocks(blah...): # both decorators appy > pass Wow, Martin Z's idea (using your keyword) really went "thunk" for me. What decorate does would be very much like what "class" does in some ways. I wonder if there's a way, using metaclasses, to get this syntax today: class Foo: class d1(static): def static1(blah): pass class d2(locksFile): def static2andLocks(blah): # both apply pass for suitably defined "static" and "locksFile" classes? It's not beautiful, but it would be a good proof of the generality/implementability of the decorator->class mapping. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From mcherm at mcherm.com Thu Aug 12 00:46:59 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Thu Aug 12 00:45:52 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint Message-ID: <1092264419.411aa1e38b2fd@mcherm.com> I previously wrote: > I envision an eventual Python which has just one type, called 'int'. Oops, didn't mean to let the cat out of the bag about my evil plan to make Python fast by turning it into a simple machine langauge. Hmm... let's pretend I said "...just one integral numeric type..." instead. That way the PSU won't be able to figure out that From skip at pobox.com Thu Aug 12 00:03:59 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 01:06:46 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <1092258146.411a896246d1a@mcherm.com> References: <1092258146.411a896246d1a@mcherm.com> Message-ID: <16666.38863.339305.573591@montanaro.dyndns.org> >> I think this would be a good idea; maybe the name should be >> baseinteger? Michael> I would like to urge caution before making this change. Despite Michael> what the PEP may say, I actually think that creating a Michael> 'baseint' type is the WRONG design choice for the long term. I Michael> envision an eventual Python which has just one type, called Michael> 'int'. I agree. I made a suggestion that we consider the entire tree of numeric types, but I had int/long unification in the back of my mind as well. I will take /F's suggestion and poke around the peps when I have some time, but I see no pressing reason a base integer class, however it's spelled, needs to be added for 2.4. Skip From ncoghlan at iinet.net.au Thu Aug 12 01:26:38 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Aug 12 01:26:47 2004 Subject: [Python-Dev] Decorator keyword possibility? Message-ID: <411AAB2E.1050200@iinet.net.au> Someone recently suggested 'remake' as a possible replacement for '@' I don't like that name because Python doesn't use the word 'make' for anything. However, it immediately prompted the thought of 'redef' After all, decorator's effectively 'redefine' the original 'define' block, to mean something else. And the old decorator syntax is *definitely* a 'redefinition' syntax. However, the problem with that word is that you would expect it to come _after_ the definition (like the old syntax does). Which leads to the actual suggestion of a 'predefinition'. That is, we're going to tell you about the definition before we actually get to it. And so, on with the examples: class demo(object): predef classmethod def foo(cls): pass predef staticmethod def bar(): pass predef accepts(int, int) predef returns(bool) predef someOtherDecorator(anarg, anotherarg, yetanotherarg) def foobar(self, apples, oranges): """Compares apples and oranges""" return apples < oranges Regards, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From mcherm at mcherm.com Thu Aug 12 01:31:12 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Thu Aug 12 01:30:05 2004 Subject: [Python-Dev] Re: Re: Update PEP 292 Message-ID: <1092267072.411aac40ecd4a@mcherm.com> Fredrik writes: > - Consider changing the ${} to accept arbitrary patterns, instead of > just Python-style identifiers. \${([^}]+)} should do the trick. +1 And hey, I'd be +1 on releasing a less-simple template that allowed arbitrary Python expressions (but no '}' characters) instead of just identifiers. But that's another issue -- and my liking it doesn't detract from my liking SIMPLE templates too. As Barry said elsewhere... even if all it does is to replace '%()s' with '${}' it's still an improvement. -- Michael Chermside From chris.cavalaria at free.fr Thu Aug 12 01:04:43 2004 From: chris.cavalaria at free.fr (Christophe Cavalaria) Date: Thu Aug 12 01:40:15 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered References: <1092179578.4119567af011d@webmail.oregonstate.edu> Message-ID: barnesc@engr.orst.edu wrote: > ----------------------------------------------- > Option A > ----------------------------------------------- > > """ > Hi, > > Performs a binary operation. > > Docstring. > """ > @staticmethod > @synchronized(lockvar) > def f(a, b): > return a + b > > """ > Now, > > Performs a simple operation. > """ > @classmethod > def g(a): > return a > > ----------------------------------------------- > Option B > ----------------------------------------------- > > def f(a, b): > @staticmethod > @synchronized(lockvar) > """ > Hi, > > Performs a binary operation. > > Docstring. > """ > return a + b > > def g(a): > @classmethod > """ > Now, > > Performs a simple operation. > """ > return a > > > Now, forget everything you've learned about Python. > Forget that @ symbols are ugly. > > - For which method is it visually easier to find the function def? None of them. A good syntax coloring would even make it easier in fact. On the second hand, the Option B makes it much harder to find the function code once you've found the function def. > - For which method is the code in the most logical order? Option A of course. Since the decorator can be seen as a function that takes the defined function as it's first parameter, it's logical to place the decorator before the definition. @staticmethod def f(): pass is a short version of f = staticmethod( def f(): pass ) after all > Note that Option A causes you to skim all the way through > the docstring and the decorators to find out which function > is being defined. At this point, you have to start over > at the docstring, to find out what the function actually does. This is false of course, any good syntax coloring would give you a good contrast between the function itself and the decorators. That way, it'll be easy to find the first line that isn't colored like a decorator. On the Option B, you'll have to identify 3 blocs of data : the function def, the decorator bloc and the function body. > Option A is especially painful when a large number of functions > are thrown into a module. It is almost impossible to grep > efficiently for the function names. Thus the code appears > disorganized, and messy. I don't understand how grep can be "confused" by the decorators before the def. Are we talking about the same program ??? From andrew at indranet.co.nz Thu Aug 12 03:21:33 2004 From: andrew at indranet.co.nz (Andrew McGregor) Date: Thu Aug 12 03:22:54 2004 Subject: [Python-Dev] Re: Re: Call for defense of @decorators In-Reply-To: <16666.13165.619920.756435@montanaro.dyndns.org> References: <200408051636.i75Gaak03654@guido.python.org> <20040805174252.GA27820@burma.localdomain> <20040805112318.B13062@ActiveState.com> <20040805183444.GA29796@burma.localdomain> <20040805190549.GC29796@burma.localdomain> <4112A18D.5090801@v.loewis.de> <20040805215331.GB31288@burma.localdomain> <1091806774.8414.63.camel@localhost> <1f7befae040806090644be9c04@mail.gmail.com> <1091818207.7529.34.camel@localhost> <4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de> <16664.50665.170916.600964@montanaro.dyndns.org> <16666.13165.619920.756435@montanaro.dyndns.org> Message-ID: <34A2826306EF5636150168FE@[192.168.1.248]> --On Wednesday, 11 August 2004 9:55 a.m. -0500 Skip Montanaro wrote: > > Andrew> Aren't we really trying to recreate def as a multi-line lambda > Andrew> that binds when it's in a context that is not expecting a > Andrew> function argument in block form? > > I suppose you can look at it that way. The output of a function > definition plus any decorator applications to it need not be a function > though. I posted a simpleminded example of generating a greyscale ramp > using a decorator. If I thought about it for more than a few seconds I > could probably come up with some other examples as well where the output > wasn't a function. Some of them might even be useful. ;-) > > Skip > > And that's exactly my point: (relatively) big one-time-use functions can be really useful, especially if the function definition is somewhat dependant on context. Andrew From greg at cosc.canterbury.ac.nz Thu Aug 12 03:24:58 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 12 03:25:05 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <59e9fd3a040811052152548b6b@mail.gmail.com> Message-ID: <200408120124.i7C1OwMk011645@cosc353.cosc.canterbury.ac.nz> > Apart from removing the need for the parentheses/braces in this case, > all it does is consistently remove the "s", I believe that just removing the 's' alone would be a substantial benefit. It's easy to forget when writing, and tends to get lost or confused with the surrounding text when reading. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From bac at OCF.Berkeley.EDU Thu Aug 12 03:36:27 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 12 03:36:40 2004 Subject: [Python-Dev] Another approach to decorators. In-Reply-To: <20040811142354.661970e0.michel@dialnetwork.com> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> Message-ID: <411AC99B.2040906@ocf.berkeley.edu> Michel Pelletier wrote: > On Wed, 11 Aug 2004 14:03:11 -0700 > "Brett C." wrote: > > I said: > >>>class Foo: >>> >>> decorate static: >>> >>> def static1(blah...): >>> pass >>> >>> def static2(blah...): >>> pass >>> >>>To me, this idea looks more like Python than all the rest, and allows >>>you to consolidate related decorated methods and classes. Nest them to >>>apply decorators "aquisition style": >> >>But then how are you supposed to do multiple decorators for the same method? > > > Put more than one of them in the same decorate block: > > class ... > > decorate staticmethod, locksFile(arg1): > > def meth1(...) > ... > > def meth2(...) > ... > > I still don't like it. It feels like I am classifying my methods based on the applied decorators instead of thinking of decorators as, well, decorators of methods. I feel methods are the major thing and decorators are just extra fluff on them, not the other way around as this syntax says to me. Changing the indentation delineates scope and I don't feel decorators truly represent a change in scope. >>I am with Martin that a keyword argument is not going to work here. > > > I presume you mean MvL and not the orginal poster, Martin Zarate. > Yep, sorry for not being clearer. > >>Going that route leads to ``public static >>painInTheAssAndClutteredSyntax(That can) implements Lots, Of, Stuff, >>That throws Things, You, Can, Learn, From, The, Docs`` hell. > > > How does Martin Z's idea lead to this? > A keyword followed by a comma-separated list of what that keyword affects. -Brett From greg at cosc.canterbury.ac.nz Thu Aug 12 03:56:42 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 12 03:56:52 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <1092258146.411a896246d1a@mcherm.com> Message-ID: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> Michael Chermside : > 'basestring' is a completely different issue -- there will always be > a need for both unicode and 8-bit-strings as separate types. I'm not so sure about that. There will certainly be a need for something holding an arbitrary sequence of bytes, but it's not clear that it needs to be called anything with 'string' in it. I can enivisage a future in which 'string' means unicode string, and byte sequences are called something else entirely, such as 'bytevector' or 'bytearray' (or maybe just 'array' :-). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From eppstein at ics.uci.edu Thu Aug 12 04:02:14 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Thu Aug 12 04:02:21 2004 Subject: [Python-Dev] Re: Another approach to decorators. References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> Message-ID: In article , David Abrahams wrote: > > class Foo: > > > > decorate static: > > > > def static1(blah...): > > pass > > > > decorate locksFile: > > > > def static2andLocks(blah...): # both decorators appy > > pass > > Wow, Martin Z's idea (using your keyword) really went "thunk" for me. > What decorate does would be very much like what "class" does in some > ways. class: (and other something: constructs) start a block that can contain any code. Does this decorate keyword allow e.g. assignments as well as defs and other decorates? Or loops? If so what should it mean? Is it like that locals() gets replaced by a special dictionary-like-object that calls the decorator whenever any of its contents gets set? -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From python at rcn.com Thu Aug 12 04:21:29 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 12 04:22:15 2004 Subject: [Python-Dev] Naming nit In-Reply-To: <411A79AA.4020505@ocf.berkeley.edu> Message-ID: <006a01c48013$13eab7a0$e622c797@oemcomputer> [Raymond Hettinger] > > For PEP 292, should the class be Template() or template()? > > [Brett] > Template as per PEP 8. Does anyone else want to chime in (I'm not the one who needs to be convinced)? Raymond From skip at pobox.com Thu Aug 12 04:37:47 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 04:38:01 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> Message-ID: <16666.55291.734287.170753@montanaro.dyndns.org> Greg> I can enivisage a future in which 'string' means unicode string, Greg> and byte sequences are called something else entirely, such as Greg> 'bytevector' or 'bytearray' (or maybe just 'array' :-). Or just 'bytes'? Skip From skip at pobox.com Thu Aug 12 04:50:09 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 04:50:20 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> Message-ID: <16666.56033.65067.241794@montanaro.dyndns.org> Greg> I can enivisage a future in which 'string' means unicode string, Greg> and byte sequences are called something else entirely, such as Greg> 'bytevector' or 'bytearray' (or maybe just 'array' :-). After another couple minutes of thought: 1. Maybe in 2.5 introduce a "bytes" builtin as a synonym for "str" and recommend its use when the intent is an arbitrary sequence of bytes. Add 'b' as a string literal prefix to generate a bytes object (which will really just be a string). At the same time, start raising exceptions when non-ascii string literals are used without a coding cookie. (Warnings are already issued for that.) 2. In 2.6, make str a synonym for unicode, remove basestring (or also make it a synonym for unicode), make bytes its own distinct (mutable? immutable?) type, and have b"..." literals generate bytes objects. Then again, maybe this can't happen until Py3K. Before then we could do the following though: 1. Make bytes a synonuym for str. 2. Warn about the use of bytes as a variable name. 3. Introduce b"..." literals as a synonym for current string literals, and have them *not* generate warnings if non-ascii characters were used in them without a coding cookie. PEP time? Skip From barnesc at engr.orst.edu Thu Aug 12 04:38:25 2004 From: barnesc at engr.orst.edu (barnesc@engr.orst.edu) Date: Thu Aug 12 04:51:14 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered Message-ID: <1092278305.411ad82141c53@webmail.oregonstate.edu> >> ----------------------------------------------- >> Option A >> ----------------------------------------------- >> >> """ >> Hi, >> >> Performs a binary operation. >> >> Docstring. >> """ >> @staticmethod >> @synchronized(lockvar) >> def f(a, b): >> return a + b >> >> """ >> Now, >> >> Performs a simple operation. >> """ >> @classmethod >> def g(a): >> return a >> >> ----------------------------------------------- >> Option B >> ----------------------------------------------- >> >> def f(a, b): >> @staticmethod >> @synchronized(lockvar) >> """ >> Hi, >> >> Performs a binary operation. >> >> Docstring. >> """ >> return a + b >> >> def g(a): >> @classmethod >> """ >> Now, >> >> Performs a simple operation. >> """ >> return a >> >> >> Now, forget everything you've learned about Python. >> Forget that @ symbols are ugly. >> >> - For which method is it visually easier to find the function def? > >None of them. A good syntax coloring would even make it easier in fact. On >the second hand, the Option B makes it much harder to find the function >code once you've found the function def. Option B makes it easier to find the function def, because the def is the only line that is not indented. I can scan straight down the page and instantly find the two function 'def's. For Option A, I have to scan line by line. >> - For which method is the code in the most logical order? > >Option A of course. Since the decorator can be seen as a function that takes >the defined function as it's first parameter, it's logical to place the >decorator before the definition. I consider the function def to be the most important line of the entire function. Thus it follows that it should be the first line. Thinking further, it makes sense to define a function, then define various properties of the function in indented blocks, since the properties are 'contained' in the function. I do agree with your assessment for decorators. However, you have to weigh the 'priority' of the decorator against the priority of the def line. In my mind, the def line wins, and so should be first. Again, of course, there's the advantage of being able to skim down the left column (which is mostly whitespace) and locate function entrance points quickly. For me, this increases the perceived organization of the code, much like labelled boxes help organize the papers in my room (locate the box quickly, then sort through it). See: http://oregonstate.edu/~barnesc/temp/option/optiona.html http://oregonstate.edu/~barnesc/temp/option/optionb.html Consider the difficulty of scanning such code, when there is a long module, with nested functions inside a class. >I don't understand how grep can be "confused" by the decorators before the >def. Are we talking about the same program ??? I refer to searching done by the eye. - Connelly From steven at zensaki.com Thu Aug 12 03:20:20 2004 From: steven at zensaki.com (Steven Kah Hien Wong) Date: Thu Aug 12 04:51:18 2004 Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence Message-ID: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> Hi, From: http://docs.python.org/tut/node11.html#SECTION0011510000000000000000 This paragraph (last two sentences): It is clear that indiscriminate use of multiple inheritance is a maintenance nightmare, given the reliance in Python on conventions to avoid accidental name conflicts. A well-known problem with multiple inheritance is a class derived from two classes that happen to have a common base class. While it is easy enough to figure out what happens in this case (the instance will have a single copy of ``instance variables'' or data attributes used by the common base class), it is not clear that these semantics are in any way useful. Maybe I have misread, but I have interpreted that as meaning the two children will share the same instance of the base class. So if one child changes its parent, the other child will pick up that change, too. But I did some quick tests, and it turns out this isn't so. class CommonBase: x = 0 class ChildOne(CommonBase): None class ChildTwo(CommonBase): None class ChildOneAndTwo(ChildOne, ChildTwo): def run(self): ChildOne.x = 1 print "one =", ChildOne.x print "two =", ChildTwo.x ChildOneAndTwo().run() And the output was: $ python multi.py one = 1 two = 0 According to the documentation, I thought it should be: one = 1 two = 1 Mind you, I find the current behaviour is much more useful, so I am saying the documentation is in error, not the implementation. :) Have I simply misread? If so, maybe that paragraph should be rewritten to make it more clear. With an example like the above, perhaps? Steven Wong From exarkun at divmod.com Thu Aug 12 04:59:06 2004 From: exarkun at divmod.com (Jp Calderone) Date: Thu Aug 12 04:59:11 2004 Subject: [Python-Dev] Re: Another approach to decorators. In-Reply-To: References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> Message-ID: <411ADCFA.8050305@divmod.com> David Eppstein wrote: > In article , > David Abrahams wrote: > > >>>class Foo: >>> >>> decorate static: >>> >>> def static1(blah...): >>> pass >>> >>> decorate locksFile: >>> >>> def static2andLocks(blah...): # both decorators appy >>> pass >> >>Wow, Martin Z's idea (using your keyword) really went "thunk" for me. >>What decorate does would be very much like what "class" does in some >>ways. > > > class: (and other something: constructs) start a block that can contain > any code. Does this decorate keyword allow e.g. assignments as well as > defs and other decorates? Or loops? If so what should it mean? Is it > like that locals() gets replaced by a special dictionary-like-object > that calls the decorator whenever any of its contents gets set? > (I have no idea what the original poster intended, however....) 'decorate' expression ':' suite could create a nested scope, the locals of which could be passed to whatever "expression" evaluates to (or if it is a tuple, do the looping thing, since people seem to like that). The call can return a dict with which the class dict is updated. For that matter, 'decorate' could even be dropped, to avoid introducing a new keyword. I don't think expression ':' suite means anything in the current grammar. This would mean staticmethod and classmethod would no longer work (unless we really want them to - they can do type checking on their argument and modify their behavior accordingly). Replacement implementations would be simple. In fact, making any current decorator work with this new mechanism is easy: def forall(decorator): def new_decorator(namespace): for k, v in namespace.items(): namespace[k] = decorator(v) return namespace return new_decorator class Foo(object): forall(staticmethod): def bar(x): print 'spam', x Many decorators would probably be used with something like forall, or simply support dicts directly. Of course, many would want to work some other way. For the PyObjC crowd, I think this would remove a major use of metaclasses (as people have argued that metaclasses are hard to understand and best avoided for simple decoration, this seems to be a good thing). I can also think of some terrible, terrible metaclass hackery in my own code that this would obviate the need for. This covers a use case James Knight mentioned as well. As he pointed out, '@' and other proposals only decorate functions (and maybe classes, soon). The example he gave, I believe, was: @public x = 1 This doesn't work, but: public: x = 1 would. Now, swallow that scream of horror (I did, barely ;). "public" here is intended to be defined as something like: def public(namespace): __all__.extend(namespace.keys()) return namespace I can see the potential for confusion to new Python programmers here, though, if they come from certain other languages. Perhaps leaving the keyword prefix would be best after all. Another possibility would be to pass a code object for the suite, instead of a namespace. I am inclined to prefer this, since it more closely resembles the other construct which can create a nested scope. It adds the burden of exec'ing the code to the decorator, though. This isn't a problem if something like "forall" above is included in the stdlib, since the common case is dealt with already (thus people desiring class or static methods need not know or care about it), and more advanced users will value the power it affords over the minor additional effort required to use it. Jp From guido at python.org Thu Aug 12 05:00:30 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 05:00:40 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] In-Reply-To: Your message of "Wed, 11 Aug 2004 13:53:31 PDT." <1092257611.411a874b5094e@mcherm.com> References: <1092257611.411a874b5094e@mcherm.com> Message-ID: <200408120300.i7C30Uk28852@guido.python.org> > For instance: > > >>> accumulator = 0.0 > >>> for i in range(25000): > ... accumulator += 1.0 > ... > >>> print accumulator > 25000.0 > > Notice, perfect accuracy. Yet people don't tend to use floating > point for loops... why not? Because the generalization to using fractions doesn't work. Try the same with adding 0.1, 25000 times. Better not be tempted. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 12 05:03:11 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 05:03:19 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Wed, 11 Aug 2004 14:02:26 PDT." <1092258146.411a896246d1a@mcherm.com> References: <1092258146.411a896246d1a@mcherm.com> Message-ID: <200408120303.i7C33BG28874@guido.python.org> > I would like to urge caution before making this change. Despite > what the PEP may say, I actually think that creating a 'baseint' > type is the WRONG design choice for the long term. I envision > an eventual Python which has just one type, called 'int'. The > fact that an efficient implementation is used when the ints > are small and an arbitrary-precision version when they get too > big would be hidden from the user by automatic promotion of > overflow. (By "hidden" I mean the user doesn't need to care, not > that they can't find out if they want to.) We are almost there > already, but if people start coding to 'baseinteger' it takes > us down a different path entirely. 'basestring' is a completely > different issue -- there will always be a need for both unicode > and 8-bit-strings as separate types. Not so sure. I expect that, like Jython and IronPython, Python 3000 will use unicode for strings, and have a separate mutable byte array for 8-bit bytes. In Python 3000 I expect that indeed somehow the existence of long is completely hidden from the user, but that's a long time away, and until then baseinteger might be a better solution than requiring people to write isinstance(x, (int, long)). --Guido van Rossum (home page: http://www.python.org/~guido/) From exarkun at divmod.com Thu Aug 12 05:07:21 2004 From: exarkun at divmod.com (Jp Calderone) Date: Thu Aug 12 05:07:26 2004 Subject: [Python-Dev] Semantics of decorators? In-Reply-To: <411A359E.3080309@erols.com> References: <411A359E.3080309@erols.com> Message-ID: <411ADEE9.4000105@divmod.com> Edward C. Jones wrote: > In the development docs, Python Reference Manual, 7.5, it says: > > A function definition may be wrapped by one or more decorator expressions. > Decorator expressions are evaluated when the function is defined, in the > scope that contains the function definition. The result must be a callable, > which is invoked with the function object as the only argument. The > returned > value is bound to the function name instead of the function object. If > there > are multiple decorators, they are applied in reverse order. > > Does this completely describe the semantics of decorators? > One thing it omits is what "decorator expressions" means. How about including a note regarding the kind of expressions which are acceptable? Jp From bac at OCF.Berkeley.EDU Thu Aug 12 05:47:31 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 12 05:47:43 2004 Subject: [Python-Dev] status of statistics.py? In-Reply-To: <2mwu0c476t.fsf@starship.python.net> References: <2mwu0c476t.fsf@starship.python.net> Message-ID: <411AE853.10404@ocf.berkeley.edu> Michael Hudson wrote: > I see statistics.py is still in nondist/sandbox/. Is there any deep > reason for this? I see the file ends with > > XXX = """ > Other possible functions include data groupers for > binning, counting, and splitting into equivalence classes. > """ > > but that hardly seems a blocker. I'd like to see the module in 2.4. > Just so that it is documented here and the people not at the last Bug Day know, Michael asked Raymond about this. Raymond said that he didn't really see a need for the module anymore since most of the important functionality had already made it into the stdlib in one form or another. He also said that time would be better spent on a calculator module that could take on any functionality that was needed that 'statistics' provided. -Brett From aahz at pythoncraft.com Thu Aug 12 05:54:55 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 12 05:54:59 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <200408120124.i7C1OwMk011645@cosc353.cosc.canterbury.ac.nz> References: <59e9fd3a040811052152548b6b@mail.gmail.com> <200408120124.i7C1OwMk011645@cosc353.cosc.canterbury.ac.nz> Message-ID: <20040812035455.GA12140@panix.com> On Thu, Aug 12, 2004, Greg Ewing wrote: > >> Apart from removing the need for the parentheses/braces in this case, >> all it does is consistently remove the "s", > > I believe that just removing the 's' alone would be a substantial > benefit. It's easy to forget when writing, and tends to get lost or > confused with the surrounding text when reading. I'll say. I've been doing a lot of work with interpolated strings lately, and even though I've been bitten several times, I keep making the same mistake. Maybe I'm just stupid, though... -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From tim.hochberg at ieee.org Thu Aug 12 06:08:37 2004 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu Aug 12 06:11:02 2004 Subject: [Python-Dev] Re: Another approach to decorators. In-Reply-To: <411ADCFA.8050305@divmod.com> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411ADCFA.8050305@divmod.com> Message-ID: <411AED45.1070300@ieee.org> Jp Calderone wrote: > David Eppstein wrote: > >> In article , >> David Abrahams wrote: >> >> >>>> class Foo: >>>> >>>> decorate static: >>>> >>>> def static1(blah...): >>>> pass >>>> >>>> decorate locksFile: >>>> >>>> def static2andLocks(blah...): # both decorators appy >>>> pass >>> >>> >>> Wow, Martin Z's idea (using your keyword) really went "thunk" for me. >>> What decorate does would be very much like what "class" does in some >>> ways. >> >> >> >> class: (and other something: constructs) start a block that can >> contain any code. Does this decorate keyword allow e.g. assignments >> as well as defs and other decorates? Or loops? If so what should it >> mean? Is it like that locals() gets replaced by a special >> dictionary-like-object that calls the decorator whenever any of its >> contents gets set? >> > > (I have no idea what the original poster intended, however....) > > 'decorate' expression ':' > suite > > could create a nested scope, the locals of which could be passed to > whatever "expression" evaluates to (or if it is a tuple, do the looping > thing, since people seem to like that). > > The call can return a dict with which the class dict is updated. FWIW, I proposed something very similar to this on python-list (http://mail.python.org/pipermail/python-list/2004-August/233262.html). It didn't get a very excited reception there. I think deafening silence is the phrase I'd use. But perhaps it'll inspire some new thoughts on your proposal. The gist of my proposal was that: 'def' name 'as' expression ':' suite would bind name to the result of expression where expression was evaluated in the scope of suite, after suite was executed. So, to reuse the example below: class Foo(object): def bar as staticmethod(bar): def bar(x): print 'spam', x For several more examples, see the original post. -tim > > For that matter, 'decorate' could even be dropped, to avoid > introducing a new keyword. I don't think > > expression ':' > suite > > means anything in the current grammar. This would mean staticmethod > and classmethod would no longer work (unless we really want them to - > they can do type checking on their argument and modify their behavior > accordingly). Replacement implementations would be simple. In fact, > making any current decorator work with this new mechanism is easy: > > def forall(decorator): > def new_decorator(namespace): > for k, v in namespace.items(): > namespace[k] = decorator(v) > return namespace > return new_decorator > > class Foo(object): > forall(staticmethod): > def bar(x): > print 'spam', x > > Many decorators would probably be used with something like forall, or > simply support dicts directly. Of course, many would want to work some > other way. > > For the PyObjC crowd, I think this would remove a major use of > metaclasses (as people have argued that metaclasses are hard to > understand and best avoided for simple decoration, this seems to be a > good thing). I can also think of some terrible, terrible metaclass > hackery in my own code that this would obviate the need for. > > This covers a use case James Knight mentioned as well. As he pointed > out, '@' and other proposals only decorate functions (and maybe classes, > soon). The example he gave, I believe, was: > > @public > x = 1 > > This doesn't work, but: > > public: > x = 1 > > would. Now, swallow that scream of horror (I did, barely ;). "public" > here is intended to be defined as something like: > > def public(namespace): > __all__.extend(namespace.keys()) > return namespace > > I can see the potential for confusion to new Python programmers here, > though, if they come from certain other languages. Perhaps leaving the > keyword prefix would be best after all. > > Another possibility would be to pass a code object for the suite, > instead of a namespace. I am inclined to prefer this, since it more > closely resembles the other construct which can create a nested scope. > It adds the burden of exec'ing the code to the decorator, though. This > isn't a problem if something like "forall" above is included in the > stdlib, since the common case is dealt with already (thus people > desiring class or static methods need not know or care about it), and > more advanced users will value the power it affords over the minor > additional effort required to use it. > > Jp > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > > From guido at python.org Thu Aug 12 06:14:53 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 06:15:01 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Wed, 11 Aug 2004 21:50:09 CDT." <16666.56033.65067.241794@montanaro.dyndns.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> Message-ID: <200408120414.i7C4Er429003@guido.python.org> > 1. Make bytes a synonuym for str. Hmm... I worry that a simple alias would just encourage confused usage, since the compiler won't check. I'd rather see bytes an alias for a bytes array as defined by the array module. > 2. Warn about the use of bytes as a variable name. Is this really needed? Builtins don't byte variable names. > 3. Introduce b"..." literals as a synonym for current string > literals, and have them *not* generate warnings if non-ascii > characters were used in them without a coding cookie. I expecet all sorts of problems with that, such as what it would mean if Unicode or multibyte characters are used in the source. Do we really need byte array literals at all? I don't expect there to be much of a demand. Rather, byte arrays would eventually be returned by the read() method when a file is opened in binary mode. (Isn't this roughly how Java does this?) We could start doing this relatively soon if we used a new mode character ("B" anyone?). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 12 06:17:07 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 06:17:15 2004 Subject: [Python-Dev] Naming nit In-Reply-To: Your message of "Wed, 11 Aug 2004 22:21:29 EDT." <006a01c48013$13eab7a0$e622c797@oemcomputer> References: <006a01c48013$13eab7a0$e622c797@oemcomputer> Message-ID: <200408120417.i7C4H7529029@guido.python.org> > [Raymond Hettinger] > > > For PEP 292, should the class be Template() or template()? > > > > > [Brett] > > Template as per PEP 8. > > Does anyone else want to chime in (I'm not the one who needs to be > convinced)? Who is? In general, classes imported from modules are Capitalized (CamelCase, really). Only builtins aren't. (Counter-examples in the standard library are mostly historic relics.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 12 06:26:30 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 06:26:38 2004 Subject: [Python-Dev] Another approach to decorators. In-Reply-To: Your message of "Wed, 11 Aug 2004 14:28:02 PDT." <20040811142802.0386bd60.michel@dialnetwork.com> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> <20040811142802.0386bd60.michel@dialnetwork.com> Message-ID: <200408120426.i7C4QU829083@guido.python.org> [Various proposals suggesting the use of a single decorator block to decorate multiple method definitions] I'm not keen on this. I expect that beyond classmethod and staticmethod there aren't many use cases for repeating the same decorator exactly multiple times. It also requires indenting the method, which I've already argued against. And lastly it removes the decorator from the method definition again, thereby defeating the readability goals: when you look at a def in isolation, you may forget to look several pages up for a decorator. I think this is too far-fetched to consider as an alternative to the humble @decorator. --Guido van Rossum (home page: http://www.python.org/~guido/) From michel at dialnetwork.com Thu Aug 12 06:18:02 2004 From: michel at dialnetwork.com (Michel Pelletier) Date: Thu Aug 12 06:28:05 2004 Subject: [Python-Dev] An indentation-based approach to decorators (was Another aproach to decorators.) In-Reply-To: <411AC99B.2040906@ocf.berkeley.edu> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> <411AC99B.2040906@ocf.berkeley.edu> Message-ID: <20040811211802.55a7276e.michel@dialnetwork.com> On Wed, 11 Aug 2004 18:36:27 -0700 "Brett C." wrote: > >>But then how are you supposed to do multiple decorators for the same method? > > > > > > Put more than one of them in the same decorate block: > > > > class ... > > > > decorate staticmethod, locksFile(arg1): > > > > def meth1(...) > > ... > > > > def meth2(...) > > ... > > I still don't like it. It feels like I am classifying my methods based > on the applied decorators instead of thinking of decorators as, well, > decorators of methods. True, it is whole 'nother pair o' dimes, a different way of looking at it than has been discussed so far. But by decorating a whole indentation block, new ideas can surface beyond decorating methods. An indentation based decoration syntax can implement quite a few new features by decorating a whole block including, but not limited to, methods inside that block. This could possible implement several PEPs that are really their own subset of a generic block decoration syntax. Consider the followign decoration blocks: # PEP 319 (disclaimer: my PEP) def foo(): # asynchronous code decorate synchronized: # sychronous code # back to async # PEP 310 def foo(): # undecorated decorate with(expr): # acquired block def bar(): # decorated ... # released # PEP 316 (method inside) decorate inv(expr), pre(expr): # invariant expressions def foo(): # decorated method ... (although I'm a little fuzzy on that last one) Possible more, what about __future__ features? decorate nested_scopes: ... that's a bit of a stretch, but the point is decorated blocks can introduce different language features or semantics on a per-block basis, explicitly. Imagine Guido decided nested scopes were a bad idea, and decided to get in the time machine and change the __future__ so the feature would never be in the language. Those who chose to go back to the __future__ would have to stay there, leaving that brilliant turd of an import in all their code. Decorated blocks would allow you to experiment explicitly with new language features or semantics without having to guarantee that they will be in the __future__. They never even need to be considered for the __future__, and if they were, the decoration would then become a noop. > I feel methods are the major thing and > decorators are just extra fluff on them, not the other way around as > this syntax says to me. > > Changing the indentation delineates scope and I > don't feel decorators truly represent a change in scope. I'm no language lawyer, but do all changes in indentation represent changes in scope? What about if, for, while, try, etc? To me, methods are not the "major thing", it's indentation. Of all the other people I've met who have heard of Python but never used it, almost all of them know nothing about the langauge but for these facts: it's based on indentation and few people go back once they get a taste of it. Decorators preceding a method definition on their own line is the first thing in Python I can think of that breaks wide of Python's indentation style, which is what has made me so uncomfortable with it from the beginning. -Michel From mkc at mathdogs.com Thu Aug 12 06:25:38 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Thu Aug 12 07:10:34 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> <873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: "Stephen J. Turnbull" writes: > I didn't have that in mind, but I can live with it. Regexp doesn't > compare well even with that when you consider error-checking and > program maintenance. It's true that regexp matching is pretty much a binary thing--either it does match or it doesn't. For human generated files, this isn't so good, but my domain of interest is machine-generated files, where I "know" that the pattern will match. If the match fails in this case, I'm happy enough to just detect it and bail, possibly with an error message that says "looks like there's a problem somewhere around char x in line y". Re maintenance, yeah regexp is pretty terse and ugly. Generally, though, I'd rather deal with a reasonably well-considered 80 char regexp than 100 lines of code that does the same thing. > It's not obvious to me how to make grammar rules pretty in Python, but > implementing an SLR parser-generator should be at most a couple of > days' work to get something you could live with. There are several of these packages available and I'm all in favor of their use, particularly for problems of sufficient complexity. These are currently hindered, though, by the fact that none have been elected for inclusion in the standard library. Furthermore, since we have 're', and it's not going away, I'd really like to fix this repeated match deficiency, one way or another. > I agree it would be useful. I think that a parser generator would be > equally useful for this, as easy to learn as regexps are (assuming > somebody comes up with a tasteful way to express grammar rules), and > more powerful with even a basic implementation. I will be +1 in favor if someone wants to pursue it. > BTW, do you have a sample implementation for re.structmatch? Fredrik > seems to have some doubt about ease of implementation. Erik Heneryd posted an intriguing Python prototype, which I'm still looking at. > Mike> I agree that, like list comprehensions (for example), it > Mike> needs to be applied with good judgement. > > I don't see the analogy to list comprehensions. Just that although list comprehensions are cool, you can still make an unreadable mess with them, as I've found myself doing occasionally. (nesting complex comprehensions four levels deep, etc.) > My objection is that it throws away a lot of structure, and therefore > is liable to return the _wrong_ parse, or simply an error with no hint > as to where the data is malformed. Hmm. Regarding the lack of error diagnosis, I'm not too concerned about this, for the reason I mention above. When 're.structmatch' does fail, though, it returns a "farthest match position", which will usually be of some aid, I would think. Regarding getting the parse wrong, sure, this could happen. Users will have to be judicious. The answer here is to write RE's with some care, realize that some matches may require further checking after being returned by re.structmatch, and further realize that some parsing problems are too complex for this method and require a grammar-based approach instead. This doesn't really seem worse than the situation for re.match, though, to me. > Because they're all too often not "multiple matches". They're matches > for different objects that happen to match the same regular > expression, and the notation for the parser should express that > without requiring comments. re.structmatch _encourages_ a style where > users deliberately throw away structure; your /etc/group parser is an > example. (See below.) Hmm. You're quibbling with my example. Yes, I could have written the expression to "know" that there were only four fields and that the third was a number, etc. I chose not to for pedagogical purposes, but it could easily have been done the other way. And in practice the programmer can choose the specificity of the match depending on exactly what they're trying to accomplish. Is this really different than what would happen with grammars? > Furthermore, since this is going to be recursive, users are going to > end up with "list of list of ..." nested to arbitrary depth, depending > on how complex the regexp is. Isn't that what we have Lisp for? The shape of the result follows directly from the shape of the grouping in the RE, yes. This is construed as a benefit. And yes, Lisp's backtick (? it's been a while) expressions were part of what inspired me on this. > I suppose you would be satisfied if you could represent > > file := file line > line := group ':' passwd ':' number ':' any '\n' > group := '[A-Za-z0-9]+' > passwd := '[A-Za-z0-9]+' > number := '[0-9]+' > any := '.*' > > by > > '(([A-Za-z0-9]+:)*.*\n)*' This isn't really fair. If you know that that's what you want to parse, then you could use a more appropriate re.structmatch equivalent like p = r'''(?xm) # VERBOSE, MULTILINE ( (?P [A-Za-z0-9]+) (?P [A-Za-z0-9]+) (?P [0-9]+) (?P .*) \n )* \Z # assert that the entire # input was matched ''' which will return the submatches tagged by those group names. > improving 'passwd' to check for the characteristic signature of a > crypted password or a "no password authentication" flag would be > annoying with re.structmatch, while it's trivial with a parser. If this signature can be expressed as an RE, then I suspect it would be easy enough to add. If not, then probably you'd have to check with code after the fact, even if you're using a grammar parser. Or am I missing something? > I grant that a full-blown parser generator is overkill for your target > set of applications. But I stick with my judgement that regexps as > they are are an attractive nuisance, and re.structmatch would make the > situation worse in proportion to its usage. I hope your proposal will > be superseded either by a scanner package, or even by a parser package. I think you're saying that you just don't care for regexps, period. This reminds me of jwz's quip ("now you have two problems"), and I have some sympathy for this point of view, but practically speaking 're' is here and it's not going away any decade soon. Given that, I'd like to fix this multiple match deficiency. I'm in favor of the scanner package, but it doesn't cover the same problem space as re.structmatch, so I don't see it as a replacement. A parser package *might* be, but I don't see any sign that that's coming any time soon. (I'd be happy enough to be wrong about this.) Cheers, Mike From martin at v.loewis.de Thu Aug 12 07:10:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 12 07:10:44 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408120414.i7C4Er429003@guido.python.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> Message-ID: <411AFBD4.5080903@v.loewis.de> Guido van Rossum wrote: > Do we really need byte array literals at all? I don't expect there to > be much of a demand. I'm not so sure. For example, in httplib, in the line h.putrequest('GET', selector) I would claim that 'GET' is a byte string, not a character string: it is sent as-is onto the wire, which is a byte-oriented wire. Now, it would also "work" if it were a character string, which then gets converted to a byte string using the system default encoding - but I believe that an application that relies on the system default encoding is somewhat broken: Explicit is better than implicit. > Rather, byte arrays would eventually be returned > by the read() method when a file is opened in binary mode. (Isn't > this roughly how Java does this?) Java also supports byte arrays in the source, although they are difficult to type: byte[] request = {'G', 'E', 'T'}; As for reading from streams: Java has multiple reader APIs; some return byte strings, some character strings. Regards, Martin From martin at v.loewis.de Thu Aug 12 07:15:30 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 12 07:15:40 2004 Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence In-Reply-To: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> Message-ID: <411AFCF2.1050208@v.loewis.de> Steven Kah Hien Wong wrote: > Mind you, I find the current behaviour is much more useful, so I am > saying the documentation is in error, not the implementation. :) Have I > simply misread? Yes. You are confusing instance attributes and class attributes. > If so, maybe that paragraph should be rewritten to make > it more clear. With an example like the above, perhaps? Can you provide a patch? Please upload to sf.net/projects/python. Regards, Martin From adurdin at gmail.com Thu Aug 12 07:16:32 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Thu Aug 12 07:16:37 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered In-Reply-To: References: <1092179578.4119567af011d@webmail.oregonstate.edu> Message-ID: <59e9fd3a0408112216345f70b9@mail.gmail.com> On Thu, 12 Aug 2004 01:04:43 +0200, Christophe Cavalaria wrote: > barnesc@engr.orst.edu wrote: > > > > - For which method is it visually easier to find the function def? > > None of them. A good syntax coloring would even make it easier in fact. One very nice thing with Python to date is that it is very easy to read even without syntax coloring--I think that's a feature worth keeping. > On the second hand, the Option B makes it much harder to find the function > code once you've found the function def. Barely harder at all than the usual function with a docstring -- you just look below the docstring. > > - For which method is the code in the most logical order? > > Option A of course. Since the decorator can be seen as a function that takes > the defined function as it's first parameter, it's logical to place the > decorator before the definition. > > @staticmethod > def f(): > pass > > is a short version of > > f = staticmethod( > def f(): > pass > ) Except that in actual fact it is a short version of def f(): pass f = staticmethod(f) > > Note that Option A causes you to skim all the way through > > the docstring and the decorators to find out which function > > is being defined. At this point, you have to start over > > at the docstring, to find out what the function actually does. > > This is false of course, any good syntax coloring would give you a good > contrast between the function itself and the decorators. That way, it'll be > easy to find the first line that isn't colored like a decorator. On the > Option B, you'll have to identify 3 blocs of data : the function def, the > decorator bloc and the function body. Relying on syntax coloring is a Bad Thing. :) From tim.peters at gmail.com Thu Aug 12 07:19:00 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 12 07:19:03 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1092232683.9945.26.camel@localhost> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> Message-ID: <1f7befae040811221916a955fd@mail.gmail.com> [Barry Warsaw] > The elimination of the trailing 's' is definitely a benefit, as is the > adoption of $-placeholders, since that tends to be the placeholder > introducing character in many other languages. In terms of usability, > it's a win. Allowing to skip the {} brackets when they aren't needed is also a win -- and I expect the most common case. I do object to this part: If the $ character appears at the end of the line, or is followed by any other character than those described above, it is treated as if it had been escaped, appearing in the resulting string unchanged. There's already a facility to escape $, and it's dead easy to use. $ isn't a frequently needed character either in most apps. So escaping $ "by magic" too is, I think, more likely to make typing errors harder to find than to do anyone real good. Better to reserve that magic for the Sloppy^H^H^H^H^H^HSafeTemplate() class. From greg at cosc.canterbury.ac.nz Thu Aug 12 07:19:58 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 12 07:20:08 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <411AFBD4.5080903@v.loewis.de> Message-ID: <200408120519.i7C5JwQV011984@cosc353.cosc.canterbury.ac.nz> Martin: > Now, it would also "work" if it were a character string, which > then gets converted to a byte string using the system default > encoding Or the encoding associated with the file object, which could be set to ascii, either by default or explicitly. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Thu Aug 12 07:22:21 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 12 07:22:28 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered In-Reply-To: <59e9fd3a0408112216345f70b9@mail.gmail.com> Message-ID: <200408120522.i7C5MLl8011994@cosc353.cosc.canterbury.ac.nz> Andrew Durdin : > One very nice thing with Python to date is that it is very easy to > read even without syntax coloring--I think that's a feature worth > keeping. So do I. Currently I edit Python on my Mac with BBEdit Lite, without syntax colouring, and I haven't missed it. I wouldn't enjoy starting to miss it now. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido at python.org Thu Aug 12 07:39:52 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 07:40:00 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: Your message of "Thu, 12 Aug 2004 01:19:00 EDT." <1f7befae040811221916a955fd@mail.gmail.com> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> Message-ID: <200408120539.i7C5dqL29226@guido.python.org> This PEP reminded me of the "boo" language (http://boo.codehaus.org/), a new scripting language for the CLI strongly inspired by Python (especially the Python 3000 variant :-). It has string interpolation like this (http://boo.codehaus.org/String+Interpolation): print("Now is ${date.Now}.") print("Tomorrow will be ${date.Now + 1d}") They require use of {...} after the $, and you don't need to escape the $ if not followed by {. Just another design variant (of course they may change it all tomorrow :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 12 07:45:04 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 07:45:12 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Thu, 12 Aug 2004 07:10:44 +0200." <411AFBD4.5080903@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> Message-ID: <200408120545.i7C5j4W29250@guido.python.org> > Guido van Rossum wrote: > > Do we really need byte array literals at all? I don't expect there to > > be much of a demand. > > I'm not so sure. For example, in httplib, in the line > > h.putrequest('GET', selector) > > I would claim that 'GET' is a byte string, not a character string: > it is sent as-is onto the wire, which is a byte-oriented wire. > > Now, it would also "work" if it were a character string, which > then gets converted to a byte string using the system default > encoding - but I believe that an application that relies on the > system default encoding is somewhat broken: Explicit is better > than implicit. Alternatively, we could postulate that the stream to which the string is written determines the encoding. This would still be explicit. Anyway, if we really do have enough use cases for byte array literals, we might add them. I still think that would be confusing though, because byte arrays are most useful if they are mutable: and then we'd have mutable literals -- blechhhh! --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Thu Aug 12 07:41:42 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Aug 12 07:47:48 2004 Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence In-Reply-To: <411AFCF2.1050208@v.loewis.de> References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> <411AFCF2.1050208@v.loewis.de> Message-ID: <20040811224013.BDE8.JCARLSON@uci.edu> > > If so, maybe that paragraph should be rewritten to make > > it more clear. With an example like the above, perhaps? > > Can you provide a patch? Please upload to sf.net/projects/python. Perhaps the following example (or a variant) should be included in the tutorial as an example. >>> class a: ... x = 0 ... >>> class b(a): ... x = 1 ... >>> class c(a): ... pass ... >>> class d(a,b): ... pass ... >>> class e(b,a): ... pass ... >>> d().x 0 >>> e().x 1 - Josiah From rnd at onego.ru Thu Aug 12 07:51:13 2004 From: rnd at onego.ru (Roman Suzi) Date: Thu Aug 12 07:51:24 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered In-Reply-To: <200408120522.i7C5MLl8011994@cosc353.cosc.canterbury.ac.nz> References: <200408120522.i7C5MLl8011994@cosc353.cosc.canterbury.ac.nz> Message-ID: On Thu, 12 Aug 2004, Greg Ewing wrote: > Andrew Durdin : > > > One very nice thing with Python to date is that it is very easy to > > read even without syntax coloring--I think that's a feature worth > > keeping. > > So do I. Currently I edit Python on my Mac with BBEdit Lite, > without syntax colouring, and I haven't missed it. I wouldn't > enjoy starting to miss it now. > > Greg Ewing, Computer Science Dept, +--------------------------------------+ The only useful sense for syntax colouring right now is to highlight reserved keywords and built-in names. There are so many of them, that it is already hard to find a name for a variable ;-) Probably, Python should have an option to protect built-in names. Sincerely yours, Roman A.Suzi -- - Petrozavodsk - Karelia - Russia - mailto:rnd@onego.ru - From adurdin at gmail.com Thu Aug 12 08:01:10 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Thu Aug 12 08:01:12 2004 Subject: [Python-Dev] Similarity of docstrings and decorators Message-ID: <59e9fd3a040811230159e61ff3@mail.gmail.com> It recently struck me that both decorators and docstrings are syntactic sugar for code that would otherwise follow the function definition, vide: @staticmethod def foo(): """Frobs the baz""" pass versus def foo(): pass foo.__doc__ = """Frobs the baz""" foo = staticmethod(foo) This suggests two things to me: firstly, that the proposed decorator syntaxes which place the decorators after the def, beside the docstring, are the most consistent with existing syntax (that for docstrings); and secondly, that this suggests that it might be better to have an explicit function "header" section where both decorators and docstrings will reside. For a conceptual example (not a syntax proposal): def foo(): header: staticmethod """Frobs the baz""" pass I realise there would be some issues with this--backward compatibility with existing docstrings (which could probably be resolved by a gradual deprecation), and Guido's concern about having to "peek inside the block"--but I thought it worth mentioning this generalised conception of both features, that highlights their similarity. (As for peeking inside the function, if a separate header area existed for the "important external properties", then it, together with the signature (which is arguably the most important part of a function definition) would comprise all that you would need to know externally.) From adurdin at gmail.com Thu Aug 12 08:05:56 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Thu Aug 12 08:06:00 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <200408120539.i7C5dqL29226@guido.python.org> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <200408120539.i7C5dqL29226@guido.python.org> Message-ID: <59e9fd3a0408112305fa1b667@mail.gmail.com> One thing the PEP doesn't make clear: do these templates only interpolate string variables, or will they handle anything with a __str__ method? e.g., will the following raise an exception: print Template("I am ${height} centimetres tall.") % {height: 183} From greg at cosc.canterbury.ac.nz Thu Aug 12 08:11:53 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 12 08:11:59 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org> Message-ID: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz> Guido: > Anyway, if we really do have enough use cases for byte array > literals, we might add them. I still think that would be confusing > though, because byte arrays are most useful if they are mutable: and > then we'd have mutable literals -- blechhhh! Perhaps the constructor for a byte array could accept a string argument as long as it contained only ascii characters? h.putrequest(bytes('GET'), selector) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Thu Aug 12 08:25:21 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 12 08:25:29 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae0408101258431039a1@mail.gmail.com> References: <000001c47e88$81ef1d40$5635c797@oemcomputer> <1f7befae0408092348583f0953@mail.gmail.com> <2mvffqzum9.fsf@starship.python.net> <1f7befae0408101258431039a1@mail.gmail.com> Message-ID: <1f7befae04081123253f08bb71@mail.gmail.com> Here's some puzzling evidence (WinXP Pro SP1). The test driver, for convenience: """ import compiler, sys f = open('../Lib/test/test_parser.py') guts = f.read() f.close() def ouch(n): if n == 0: return compiler.compile(guts, "", "exec") else: return ouch(n-1) for n in range(0, 50): try: ouch(n) msg = 'ok' except KeyboardInterrupt: raise except Exception, msg: msg = str(sys.exc_info()[0]) + ' ' + str(msg) print n, msg """ The loop starts at 0 this time, and I'm only looking at a debug-build Python. The stack size for an .exe can be specified to 4-byte granularity. Here's the largest stack size at which that produces no output at all: C:\Code\python\PCbuild>editbin /stack:1310720 python_d.exe Microsoft (R) COFF Binary File Editor Version 6.00.8447 Copyright (C) Microsoft Corp 1992-1998. All rights reserved. C:\Code\python\PCbuild>python_d temp.py C:\Code\python\PCbuild>echo %ERRORLEVEL% 128 Add 4 measly bytes, and it's a world of difference: C:\Code\python\PCbuild>editbin /stack:1310724 python_d.exe Microsoft (R) COFF Binary File Editor Version 6.00.8447 Copyright (C) Microsoft Corp 1992-1998. All rights reserved. C:\Code\python\PCbuild>python_d temp.py 0 ok 1 ok ... 41 ok 42 ok C:\Code\python\PCbuild>echo %ERRORLEVEL% 128 So it still vanishes early, but gets thru the entire compiler.compile() business + 42 additional stacked Python calls! That's awfully impressive for four bytes. That suggests "the problem" isn't in detecting Python-level recursion. Under the debugger, it dies with an access violation at that point. Alas, the C-level Python call stack is no longer anywhere in evidence then. Instead it's 7 levels deep in ntdll.dll, and is "in the middle" of four consecutive Pentium PUSH instructions. That's evidence that the stack has been blown . The debugger Output window does show ntdll.dll suffering a Stack Overflow exception. It then shows a variety of Access Violations in ntdll.dll trying to read and write various locations, presumably in a failing attempt to report the stack overflow. From martin at v.loewis.de Thu Aug 12 08:26:40 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 12 08:26:34 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> Message-ID: <411B0DA0.4030109@v.loewis.de> Guido van Rossum wrote: > Anyway, if we really do have enough use cases for byte array literals, > we might add them. I still think that would be confusing though, > because byte arrays are most useful if they are mutable: and then we'd > have mutable literals -- blechhhh! I see. How would you like byte array displays then? This is the approach taken in the other languages: Everytime the array display is executed, a new array is created. There is then no problem with that being mutable. Of course, if the syntax is too similar to string literals, people might be tricked into believing they are actually literals. Perhaps bytes('G','E','T') would be sufficient, or even bytes("GET") which would implicitly convert each character to Latin-1. Regards, Martin From martin at v.loewis.de Thu Aug 12 08:29:00 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 12 08:28:56 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz> References: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz> Message-ID: <411B0E2C.5000608@v.loewis.de> Greg Ewing wrote: > Perhaps the constructor for a byte array could accept a string > argument as long as it contained only ascii characters? > > h.putrequest(bytes('GET'), selector) That is probably most reasonable. It would be harmless to extend it to Latin-1, which would allow to represent all byte values in a string literal - even using the appropriate \x escapes. Regards, Martin From ncoghlan at iinet.net.au Thu Aug 12 08:49:11 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Aug 12 08:49:23 2004 Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence In-Reply-To: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> Message-ID: <411B12E7.6060808@iinet.net.au> Steven Kah Hien Wong wrote: > Maybe I have misread, but I have interpreted that as meaning the two > children will share the same instance of the base class. So if one child > changes its parent, the other child will pick up that change, too. This interpretation seems correct to me. > But I did some quick tests, and it turns out this isn't so. It *is* so if you actually change the common parent, rather than one of the siblings (see modified example below). In your original example, the new 'x' was stored inside ChildOne's class dictionary, and so ChildTwo was not affected. To see the effect of the shared parent, you need to actually change the value of 'x' that is stored in CommonBase (as happens in the example below). ===================== class CommonBase: x = 0 class ChildOne(CommonBase): def foo(self): CommonBase.x = 1 ##### Change here class ChildTwo(CommonBase): pass class ChildOneAndTwo(ChildOne, ChildTwo): def run(self): ChildOne.foo(self) ##### Change Here print "one =", ChildOne.x print "two =", ChildTwo.x ChildOneAndTwo().run() ===================== Output is now: $ python multi.py one = 1 two = 1 ===================== In relation to Martin's post, the text is really talking about a problem with 'instance' variables in the presence of multiple inheritance: ===================== class Base(object): def __init__(self): super(Base, self).__init__() self.x = 0 print "Base initialised" def run(self): print "I am", self.x class One(Base): def __init__(self): super(One, self).__init__() self.x = 1 print "One initialised" class Two(Base): def __init__(self): super(Two, self).__init__() self.x = 2 print "Two initialised" class OneTwo(One, Two): def __init__(self): super(OneTwo, self).__init__() OneTwo().run() ===================== Output is: Base initialised Two initialised One initialised I am 1 ===================== -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From lists at hlabs.spb.ru Thu Aug 12 12:58:11 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Thu Aug 12 08:52:10 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <16666.38863.339305.573591@montanaro.dyndns.org> References: <1092258146.411a896246d1a@mcherm.com> <16666.38863.339305.573591@montanaro.dyndns.org> Message-ID: <411B4D43.70308@hlabs.spb.ru> Skip Montanaro wrote: > >> I think this would be a good idea; maybe the name should be > >> baseinteger? > > Michael> I would like to urge caution before making this change. Despite > Michael> what the PEP may say, I actually think that creating a > Michael> 'baseint' type is the WRONG design choice for the long term. I > Michael> envision an eventual Python which has just one type, called > Michael> 'int'. > > I agree. I made a suggestion that we consider the entire tree of numeric > types, Is it a good time to consider the entrie tree of ALL types then? For example: object (Tree of numeric types as suggested by Gareth) number complex real (Decimal) float rational fraction integer int bool long sequence buffer basestring str unicode list tuple mapping dict set frozenset file > but I had int/long unification in the back of my mind as well. I > will take /F's suggestion and poke around the peps when I have some time, > but I see no pressing reason a base integer class, however it's spelled, > needs to be added for 2.4. -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From ncoghlan at iinet.net.au Thu Aug 12 09:10:55 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Aug 12 09:11:05 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <411B4D43.70308@hlabs.spb.ru> References: <1092258146.411a896246d1a@mcherm.com> <16666.38863.339305.573591@montanaro.dyndns.org> <411B4D43.70308@hlabs.spb.ru> Message-ID: <411B17FF.7060607@iinet.net.au> Dmitry Vasiliev wrote: > Is it a good time to consider the entrie tree of ALL types then? Please, no. One of the best things about Python is that, for most purposes, you can get the appropriate reaction from the standard library simply by providing the relevant magic methods (e.g. __iter__ if you want to be treated like a sequence). Normal class inheritance means you are inheriting the *whole* interface, as well as the implementation of any parts you don't override. So 'sequence' for instance, couldn't have any methods or do much of anything, without causing potentially undesirable behaviour. About the only exception that makes any sense to me is for exceptions, where using inheritance for classification seems to be universal practice in all languages with exceptions (since 'catch X' generally means catch X, or any of its subclasses). (Bad pun noted, and not avoided). Localised type hierarchies where it is useful, OK. Inheriting from object as a way to say 'new-style class please', OK. But please, please, please, lets keep interface introspection the dominant way of determining what can be done with a particular object. Creating a 'grand hierarchy of everything' runs directly counter to that goal. Regards, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From sdm7g at mac.com Thu Aug 12 09:15:50 2004 From: sdm7g at mac.com (Steven Majewski) Date: Thu Aug 12 09:16:01 2004 Subject: [Python-Dev] CGI module [was: New Subscriber] In-Reply-To: <20040507043316.GB45563@unixbeast.com> References: <20040507043316.GB45563@unixbeast.com> Message-ID: <70C26872-EC2F-11D8-B23F-000A957D4F2A@mac.com> [ OK. This is an old thread, but it's a new concern for me. I was researching to see what the current state of things was before making any public comments. I though I might as well provide this background to Dave Hastings, and also ask anyone involved in those earlier threads if there has been any more recent work on this that I might have missed. It looks like there was a flurry of discussion about this last fall but nothing seems to have come of it. ] On May 7, 2004, at 12:33 AM, Dave Hastings wrote: > [ ... ] The project that was decided upon, with the advice of the > Professor, was to make a CGIpm python module that reflected CGI.pm for > perl. We noticed that there is a current cgi module for python and > even a cgipm module. These modules are very useful, but did not > include all the functions that CGI.pm for perl provides. It turns out > that my Professor really likes python and would have loved to see a > CGIpm python module for python that was like the CGI.pm module for > perl. [ ... ] Dave, It's been a few years since I needed to do any cgi programming in Python. When I recently had to dig into it, I was SHOCKED to discover what a mess the current cgi module is. ( My second shock was when I looked at the source to try to figure things out, and discovered my name there at the top! ;-( It seems to be one of those things that just grew wild and didn't get much pruning. Mike McLay at NIST did the first version. I made some changes, but tried to keep backwards compatibility. Someone else came along and created a new set of interfaces, which both added and lost some functionality, but kept the older versions around for compatibility. ( I couldn't seem to get the new, improved version to do what I wanted, so I had to fall back on using the old compatability version! ) None of them seem very nice to use. Probably one of the few places where Python is 'uglier' than perl. And searching various Python archives, I find a history of similar complaints. Among the "Deferred, Abandoned, Withdrawn, and Rejected PEPs" is: PEP 222 (Dec 2000): http://www.python.org/peps/pep-0222.html which first proposes dumping the cgi module and starting over again: > The cgi.py module would be deprecated. (XXX A new module or > package name hasn't been chosen yet: 'web'? 'cgilib'?) ... and a partial outline of a proposed interface. ( along with: Extended HTTP functionality and WEBDAV: http://www.python.org/peps/pep-0268.html ) In the Sept. 2003 Python-Dev archives: See the thread that begins with Subject: "[Python-Dev] Improving the CGI module": http://mail.python.org/pipermail/python-dev/2003-September/038128.html And the next month, a similar discussion picked up in the Web-SIG mailing list. See Web-SIG October 2003 Archives: http://mail.python.org/pipermail/web-sig/2003-October/thread.html See the many messages concerning "Form field dictionaries" , as well as several other threads in that months archive. As far as I can tell, all of those threads died out without any actual code being produced. However, they do give a good enumeration of what's wrong with the current cgi module and what's needed in a new module. Please let me know if there is any more current history I've missed in my search. ( I've been out of the python-dev loop for a while! ) I guess the Web-SIG is the best place for any further discussion. http://mail.python.org/mailman/listinfo/web-sig Dave: I'm not sure that following CGI.pm is the best possible design, but I'ld like to see what you're planning. If nothing else is happening on this front, I may be able to take a swing at it when I return from my vacation. -- Steve Majewski From mwh at python.net Thu Aug 12 12:33:48 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 12 12:33:50 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae04081123253f08bb71@mail.gmail.com> (Tim Peters's message of "Thu, 12 Aug 2004 02:25:21 -0400") References: <000001c47e88$81ef1d40$5635c797@oemcomputer> <1f7befae0408092348583f0953@mail.gmail.com> <2mvffqzum9.fsf@starship.python.net> <1f7befae0408101258431039a1@mail.gmail.com> <1f7befae04081123253f08bb71@mail.gmail.com> Message-ID: <2mekmczlv7.fsf@starship.python.net> Tim Peters writes: > Here's some puzzling evidence (WinXP Pro SP1). The test driver, for > convenience: > > """ > import compiler, sys > > f = open('../Lib/test/test_parser.py') > guts = f.read() > f.close() > > def ouch(n): > if n == 0: > return compiler.compile(guts, "", "exec") > else: > return ouch(n-1) > > for n in range(0, 50): > try: > ouch(n) > msg = 'ok' > except KeyboardInterrupt: > raise > except Exception, msg: > msg = str(sys.exc_info()[0]) + ' ' + str(msg) > print n, msg > """ > > The loop starts at 0 this time, and I'm only looking at a debug-build Python. > > The stack size for an .exe can be specified to 4-byte granularity. > Here's the largest stack size at which that produces no output at all: > > C:\Code\python\PCbuild>editbin /stack:1310720 python_d.exe > Microsoft (R) COFF Binary File Editor Version 6.00.8447 > Copyright (C) Microsoft Corp 1992-1998. All rights reserved. > > C:\Code\python\PCbuild>python_d temp.py > > C:\Code\python\PCbuild>echo %ERRORLEVEL% > 128 > > > Add 4 measly bytes, and it's a world of difference: > > C:\Code\python\PCbuild>editbin /stack:1310724 python_d.exe > Microsoft (R) COFF Binary File Editor Version 6.00.8447 > Copyright (C) Microsoft Corp 1992-1998. All rights reserved. > > > > C:\Code\python\PCbuild>python_d temp.py > 0 ok > 1 ok > ... > 41 ok > 42 ok > > C:\Code\python\PCbuild>echo %ERRORLEVEL% > 128 > > So it still vanishes early, but gets thru the entire > compiler.compile() business + 42 additional stacked Python calls! > That's awfully impressive for four bytes. That suggests "the problem" > isn't in detecting Python-level recursion. Has the failure mode of alloca() changed? I take it you're building with VC++ 7.1? What happens for a VC++ 6 build? Hmm, a moment with msdn suggests that there's been no significant changes here, although the documentation is for _alloca(), and Python calls alloca(). That can't make any difference, can it? It still smells like a tool change to me. Cheers, mwh -- 58. Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From mwh at python.net Thu Aug 12 12:39:35 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 12 12:39:36 2004 Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes In-Reply-To: <5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com> (Phillip J. Eby's message of "Wed, 11 Aug 2004 15:54:23 -0400") References: <20040811011116.GF23725@performancedrivers.com> <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> <200408072105.i77L5Ea10965@guido.python.org> <20040811011116.GF23725@performancedrivers.com> <5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com> Message-ID: <2macx0zllk.fsf@starship.python.net> "Phillip J. Eby" writes: > At 03:44 PM 8/11/04 -0400, Jack Diederich wrote: >> I have a mostly working patch for class decorators but it gets an >> XXX ambiguity >>between >> >>funcdef: [decorators] 'def' NAME parameters ':' suite >>and >>classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite >> >> It works as advertised with the pre-decorators funcdef definition >> (which removes the ambiguity). >>Help? > > > How about this? > > decorated_assignment: [decorators] (funcdef | classdef) decorated_definition or something would be better surely? > classdef: 'class' NAME [ '(' testlist ')' ] ':' suite > funcdef: 'def' NAME parameters ':' suite But yes, the answer to this problem is "left factoring", which google can tell you all about... Cheers, mwh -- For their next act, they'll no doubt be buying a firewall running under NT, which makes about as much sense as building a prison out of meringue. -- -:Tanuki:- -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From erik at heneryd.com Thu Aug 12 12:44:26 2004 From: erik at heneryd.com (Erik Heneryd) Date: Thu Aug 12 12:44:32 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching In-Reply-To: References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> <873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <411B4A0A.1030201@heneryd.com> Mike Coleman wrote: > "Stephen J. Turnbull" writes: > Re maintenance, yeah regexp is pretty terse and ugly. Generally, though, I'd > rather deal with a reasonably well-considered 80 char regexp than 100 lines of > code that does the same thing. ditto >>It's not obvious to me how to make grammar rules pretty in Python, but >>implementing an SLR parser-generator should be at most a couple of >>days' work to get something you could live with. > > > There are several of these packages available and I'm all in favor of their > use, particularly for problems of sufficient complexity. These are currently > hindered, though, by the fact that none have been elected for inclusion in the > standard library. > > Furthermore, since we have 're', and it's not going away, I'd really like to > fix this repeated match deficiency, one way or another. Well, I guess that if you want structmatch into the stdlib you'll have to show that it's better than it's alternatives. Including those parser packages. >>BTW, do you have a sample implementation for re.structmatch? Fredrik >>seems to have some doubt about ease of implementation. > > > Erik Heneryd posted an intriguing Python prototype, which I'm still looking > at. You'd still have to do a real implementation. If it can't be done without rewriting a whole lot of code, that would be a problem. >>My objection is that it throws away a lot of structure, and therefore >>is liable to return the _wrong_ parse, or simply an error with no hint >>as to where the data is malformed. > > > Hmm. Regarding the lack of error diagnosis, I'm not too concerned about this, > for the reason I mention above. When 're.structmatch' does fail, though, it > returns a "farthest match position", which will usually be of some aid, I > would think. > > Regarding getting the parse wrong, sure, this could happen. Users will have > to be judicious. The answer here is to write RE's with some care, realize > that some matches may require further checking after being returned by > re.structmatch, and further realize that some parsing problems are too complex > for this method and require a grammar-based approach instead. This doesn't > really seem worse than the situation for re.match, though, to me. Hmm... think this is the wrong approach. Your PEP is not just about "structured matching", it tries to deal with a couple of issues and I think it would be better to address them separately, one by one: * Parsing/scanning - this is mostly what's been discussed so far... * Capturing repeated groups - IMO nice-to-have (tm) but not something I would lose sleep over. Hard to do. * Partial matches - would be great for debugging more complex regexes. Why not a general re.RAISE flag raising an exception on failure? * ... ? Erik From mak at trisoft.com.pl Thu Aug 12 13:08:45 2004 From: mak at trisoft.com.pl (Grzegorz Makarewicz) Date: Thu Aug 12 13:08:46 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae04081123253f08bb71@mail.gmail.com> References: <000001c47e88$81ef1d40$5635c797@oemcomputer> <1f7befae0408092348583f0953@mail.gmail.com> <2mvffqzum9.fsf@starship.python.net> <1f7befae0408101258431039a1@mail.gmail.com> <1f7befae04081123253f08bb71@mail.gmail.com> Message-ID: <411B4FBD.9090100@trisoft.com.pl> Tim Peters wrote: > > So it still vanishes early, but gets thru the entire > compiler.compile() business + 42 additional stacked Python calls! > That's awfully impressive for four bytes. That suggests "the problem" > isn't in detecting Python-level recursion. > > Under the debugger, it dies with an access violation at that point. > Alas, the C-level Python call stack is no longer anywhere in evidence > then. Instead it's 7 levels deep in ntdll.dll, and is "in the middle" > of four consecutive Pentium PUSH instructions. That's evidence that > the stack has been blown . > > The debugger Output window does show ntdll.dll suffering a Stack > Overflow exception. It then shows a variety of Access Violations in > ntdll.dll trying to read and write various locations, presumably in a > failing attempt to report the stack overflow. I have copied stack check into PyObject_Malloc and after X calls to PyEval_EvalFrame, fast_function python dies in malloc(msvcrt) and HeapAlloc (ntdll) as described by Tim. After this patch most ocurring exception is MemoryError, but for 141, 171, 201 and 231 python raises SyntaxError - always witout traceback :( mak relevant part from obmalloc.c: PyObject_Malloc(size_t nbytes) { block *bp; poolp pool; poolp next; uint size; #ifdef USE_STACKCHECK if (PyOS_CheckStack()) { return NULL; } #endif From arigo at tunes.org Thu Aug 12 11:53:13 2004 From: arigo at tunes.org (Armin Rigo) Date: Thu Aug 12 13:44:12 2004 Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence In-Reply-To: <20040811224013.BDE8.JCARLSON@uci.edu> References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> <411AFCF2.1050208@v.loewis.de> <20040811224013.BDE8.JCARLSON@uci.edu> Message-ID: <20040812095313.GA5276@vicky.ecs.soton.ac.uk> Hello, On Wed, Aug 11, 2004 at 10:41:42PM -0700, Josiah Carlson wrote: > Perhaps the following example (or a variant) should be included in the > tutorial as an example. Including this example would be confusing for two reasons. One is that it doesn't answer the original concern, which was not about class attributes but instance attributes, e.g.: class point: def move_to_the_right(self): x, y = self.pos self.pos = x+1, y class myfile: def seek(self, newpos): self.pos = newpos def read(self): return self.data[self.pos:] In this example, a class inheriting from both 'point' and 'myfile' will be in trouble because the same instance attribute 'pos' is used with different meanings by the methods of the base classes. Another concern with your example is that some of the behavior it relies upon is specific to old-style classes (which might be dropped at some time long in the future). Try to do the same with new-style classes: class a(object): # forces new-style classes pass class b(a): pass class d(a,b): TypeError: Cannot create a consistent method resultion order (MRO) for bases b, a A saner example along those lines would be something like: class a: pass class b(a): x = 1 class c(a): x = 2 class d(b,c): pass class e(c,b): pass d().x # 1 e().x # 2 A bient?t, Armin. From gmccaughan at synaptics-uk.com Thu Aug 12 14:06:58 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu Aug 12 14:07:30 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <1092258146.411a896246d1a@mcherm.com> References: <1092258146.411a896246d1a@mcherm.com> Message-ID: <200408121306.58446.gmccaughan@synaptics-uk.com> On Wednesday 2004-08-11 22:02, Michael Chermside wrote: > I would like to urge caution before making this change. Despite > what the PEP may say, I actually think that creating a 'baseint' > type is the WRONG design choice for the long term. I envision > an eventual Python which has just one type, called 'int'. The > fact that an efficient implementation is used when the ints > are small and an arbitrary-precision version when they get too > big would be hidden from the user by automatic promotion of > overflow. (By "hidden" I mean the user doesn't need to care, not > that they can't find out if they want to.) We are almost there > already, but if people start coding to 'baseinteger' it takes > us down a different path entirely. 'basestring' is a completely > different issue -- there will always be a need for both unicode > and 8-bit-strings as separate types. This is why "integer" is a better name than "baseinteger". For now it can be the common supertype of int and long. In the future, it can be the name of the single integer type. -- g From john at grulic.org.ar Thu Aug 12 14:47:38 2004 From: john at grulic.org.ar (John Lenton) Date: Thu Aug 12 14:47:45 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] In-Reply-To: <1092257611.411a874b5094e@mcherm.com> References: <1092257611.411a874b5094e@mcherm.com> Message-ID: <20040812124738.GC8843@grulic.org.ar> On Wed, Aug 11, 2004 at 01:53:31PM -0700, Michael Chermside wrote: > > But lots of languages make other choices. The number of languages > that have a single numeric type is pretty large... perl comes to > mind, but I bet I could list 10 others without too much trouble. perl doesn't have a single numeric type. Depending on how deep you look into the issue, it has none (only a scalar type), or several (scalars that are assigned integers are integers until they are promoted to float when used floatishly). -- John Lenton (john@grulic.org.ar) -- Random fortune: What's so funny? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/a81e7074/attachment.pgp From mal at egenix.com Thu Aug 12 15:15:37 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 12 15:15:41 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <411920E4.9020404@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> Message-ID: <411B6D79.2040503@egenix.com> Hi Walter, I don't have time to comment on this this week, I'll respond next week. Overall, I don't like the idea of adding extra APIs breaking the existing codec API. I believe that we can extend stream codecs to also work in a feed mode without breaking the API. Walter D?rwald wrote: > OK, here a my current thoughts on the codec problem: > > The optimal solution (ignoring backwards compatibility) > would look like this: codecs.lookup() would return the > following stuff (this could be done by replacing the > 4 entry tuple with a real object): > > * decode: The stateless decoding function > * encode: The stateless encocing function > * chunkdecoder: The stateful chunk decoder > * chunkencoder: The stateful chunk encoder > * streamreader: The stateful stream decoder > * streamwriter: The stateful stream encoder > > The functions and classes look like this: > > > Stateless decoder: > decode(input, errors='strict'): > Function that decodes the (str) input object and returns > a (unicode) output object. The decoder must decode the > complete input without any remaining undecoded bytes. > > Stateless encoder: > encode(input, errors='strict'): > Function that encodes the complete (unicode) input object and > returns a (str) output object. > > Stateful chunk decoder: > chunkdecoder(errors='strict'): > A factory function that returns a stateful decoder with the > following method: > > decode(input, final=False): > Decodes a chunk of input and return the decoded unicode > object. This method can be called multiple times and > the state of the decoder will be kept between calls. > This includes trailing incomplete byte sequences > that will be retained until the next call to decode(). > When the argument final is true, this is the last call > to decode() and trailing incomplete byte sequences will > not be retained, but a UnicodeError will be raised. > > Stateful chunk encoder: > chunkencoder(errors='strict'): > A factory function that returns a stateful encoder > with the following method: > encode(input, final=False): > Encodes a chunk of input and returns the encoded > str object. When final is true this is the last > call to encode(). > > Stateful stream decoder: > streamreader(stream, errors='strict'): > A factory function that returns a stateful decoder > for reading from the byte stream stream, with the > following methods: > > read(size=-1, chars=-1, final=False): > Read unicode characters from the stream. When data > is read from the stream it should be done in chunks of > size bytes. If size == -1 all the remaining data > from the stream is read. chars specifies the number > of characters to read from the stream. read() may return > less then chars characters if there's not enough data > available in the byte stream. If chars == -1 as much > characters are read as are available in the stream. > Transient errors are ignored and trailing incomplete > byte sequence are retained when final is false. Otherwise > a UnicodeError is raised in the case of incomplete byte > sequences. > readline(size=-1): > ... > next(): > ... > __iter__(): > ... > > Stateful stream encoder: > streamwriter(stream, errors='strict'): > A factory function that returns a stateful encoder > for writing unicode data to the byte stream stream, > with the following methods: > > write(data, final=False): > Encodes the unicode object data and writes it > to the stream. If final is true this is the last > call to write(). > writelines(data): > ... > > > I know that this is quite a departure from the current API, and > I'm not sure if we can get all of the functionality without > sacrificing backwards compatibility. > > I don't particularly care about the "bytes consumed" return value > from the stateless codec. The codec should always have returned only > the encoded/decoded object, but I guess fixing this would break too > much code. And users who are only interested in the stateless > functionality will probably use unicode.encode/str.decode anyway. > > For the stateful API it would be possible to combine the chunk and > stream decoder/encode into one class with the following methods > (for the decoder): > > __init__(stream, errors='strict'): > Like the current StreamReader constructor, but stream may be > None, if only the chunk API is used. > decode(input, final=False): > Like the current StreamReader (i.e. it returns a (unicode, int) > tuple.) This does not keep the remaining bytes in a buffer. > This is the job of the caller. > feed(input, final=False): > Decodes input and returns a decoded unicode object. This method > calls decode() internally and manages the byte buffer. > read(size=-1, chars=-1, final=False): > readline(size=-1): > next(): > __iter__(): > See above. > > As before implementers of decoders only need to implement decode(). > > To be able to support the final argument the decoding functions > in _codecsmodule.c could get an additional argument. With this > they could be used for the stateless codecs too and we can reduce > the number of functions again. > > Unfortunately adding the final argument breaks all of the current > codecs, but dropping the final argument requires one of two > changes: > 1) When the input stream is exhausted, the bytes read are parsed > as if final=True. That's the way the CJK codecs currently > handle it, but unfortunately this doesn't work with the feed > decoder. > 2) Simply ignore any remaing undecoded bytes at the end of the > stream. > > If we really have to drop the final argument, I'd prefer 2). > > I've uploaded a second version of the patch. It implements > the final argument, adds the feed() method to StreamReader and > again merges the duplicate decoding functions in the codecs > module. Note that the patch isn't really finished (the final > argument isn't completely supported in the encoders and the > CJK and escape codecs are unchanged), but it should be sufficient > as a base for discussion. > > Bye, > Walter D?rwald > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/mal%40egenix.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 12 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From amk at amk.ca Thu Aug 12 15:27:06 2004 From: amk at amk.ca (A.M. Kuchling) Date: Thu Aug 12 15:27:39 2004 Subject: [Python-Dev] Third Bug Day outcome In-Reply-To: References: Message-ID: <20040812132706.GF21744@rogue.amk.ca> On Mon, Aug 09, 2004 at 07:19:04PM -0300, Batista, Facundo wrote: > Maybe for the next Bug Day should be two "IRC hosts", one in America and > other in Europe. so, the'll be 6 hs each (top) and the whole day will be of > 12 hours. That would be a good idea, if a volunteer in Europe can be found. --amk From amk at amk.ca Thu Aug 12 15:36:55 2004 From: amk at amk.ca (A.M. Kuchling) Date: Thu Aug 12 15:37:28 2004 Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows In-Reply-To: <20040810132612.GB28542@unpythonic.net> References: <20040807142327.GA2529@rogue.amk.ca> <20040810132612.GB28542@unpythonic.net> Message-ID: <20040812133655.GG21744@rogue.amk.ca> On Tue, Aug 10, 2004 at 08:26:13AM -0500, Jeff Epler wrote: > SO_EXCLUSIVEADDRUSE sounds like the exact opposite of SO_REUSEADDR, > according to a page I found when trying to find out whether any other > platform has SO_EXCLUSEIVEADDRUSE: > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp Hm; this change came from patch #982681; I'm not a Windows person either, and just assumed the patch was correct for Winsock's semantics. Unless someone with Windows experience can suggest otherwise, I'll back out this change. --amk From jimjjewett at yahoo.com Thu Aug 12 15:39:52 2004 From: jimjjewett at yahoo.com (Jim J Jewett) Date: Thu Aug 12 15:39:56 2004 Subject: [Python-Dev] Re: Re: Decimal type question [Prothon] Message-ID: <20040812133952.69390.qmail@web50702.mail.yahoo.com> Mark Hahn: > What do you think of the newer idea of using > the IBM floating decimal for all numerics? > Tim Peters and others pushed this on me. In theory, it is a great idea, and removes a lot of useless clutter. C often has signed and unsigned versions of char, short, int, long, and long long. Python has just int and long. The "missing" types aren't really missed. For Python itself, the unification probably shouldn't happen until 3.0, because of all the programs that use an "integer" as a bitfield, and because performance won't really be equal. For a new language -- yes, you might as well get it right from the start. -jJ __________________________________ Do you Yahoo!? Yahoo! Mail is new and improved - Check it out! http://promotions.yahoo.com/new_mail From guido at python.org Thu Aug 12 16:20:39 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 16:20:43 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Thu, 12 Aug 2004 18:11:53 +1200." <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz> References: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408121420.i7CEKdo30140@guido.python.org> > Guido: > > Anyway, if we really do have enough use cases for byte array > > literals, we might add them. I still think that would be confusing > > though, because byte arrays are most useful if they are mutable: and > > then we'd have mutable literals -- blechhhh! Greg: > Perhaps the constructor for a byte array could accept a string > argument as long as it contained only ascii characters? > > h.putrequest(bytes('GET'), selector) Yeah, but that's what Martin called depending on the default encoding. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 12 16:30:06 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 16:30:12 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Thu, 12 Aug 2004 08:26:40 +0200." <411B0DA0.4030109@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> Message-ID: <200408121430.i7CEU6F30168@guido.python.org> > Guido van Rossum wrote: > > Anyway, if we really do have enough use cases for byte array > > literals, we might add them. I still think that would be > > confusing though, because byte arrays are most useful if they are > > mutable: and then we'd have mutable literals -- blechhhh! Martin: > I see. How would you like byte array displays then? Not as characters (not by default anyway), because more often than not they will contain binary or encoded gibberish! > This is the approach taken in the other languages: Everytime the > array display is executed, a new array is created. There is then no > problem with that being mutable. The downside of that is that then for performance reasons you might end up having to move bytes literals out of expressions if they are in fact used read-only (which the compiler can't know but the user can). > Of course, if the syntax is too similar to string literals, people > might be tricked into believing they are actually literals. Perhaps > > bytes('G','E','T') > > would be sufficient, or even > > bytes("GET") > > which would implicitly convert each character to Latin-1. The first form would also need an encoding, since Python doesn't have character literals! I don't think we should use Latin-1, for the same reasons that the default encoding is ASCII. Better would be to have a second argument that's the encoding, so you can write bytes(u"", "utf-8") Hm, u"".encode("utf-8") should probably return a bytes array, and that might be sufficient. Perhaps bytes should by default be considered as arrays of tiny unsigned ints, so we could use bytes(map(ord, "GET")) and it would display itself as bytes([71, 69, 84]) Very long ones should probably use ellipses rather than print a million numbers. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 12 16:44:05 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 16:44:09 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Thu, 12 Aug 2004 13:06:58 BST." <200408121306.58446.gmccaughan@synaptics-uk.com> References: <1092258146.411a896246d1a@mcherm.com> <200408121306.58446.gmccaughan@synaptics-uk.com> Message-ID: <200408121444.i7CEi5c30221@guido.python.org> > This is why "integer" is a better name than "baseinteger". > For now it can be the common supertype of int and long. > In the future, it can be the name of the single integer > type. No, that will be int, of course! Like 'basestring', 'baseinteger' is intentionally cumbersome, because it is only the base class of all *built-in* integral types. Note that UserString is *not* subclassing basestring, and likewise if you wrote an integer-like class from scratch it should not inherit from baseinteger. Code testing for these types is interested in knowing whether something is a member of one of the *built-in* types, which is often needed because other built-in operations (e.g. many extension modules) only handle the built-in types. If you want to test for integer-like or string-like behavior, you won't be able to use isinstance(), but instead you'll have to check for the presence of certain methods. I know this is not easy in the case of integers, but I don't want to start requiring inheritance from a marker base type now. Python is built on duck typing. (Google for it.) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Thu Aug 12 16:54:13 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 12 16:53:57 2004 Subject: [Python-Dev] Naming nit In-Reply-To: <411A79AA.4020505@ocf.berkeley.edu> References: <003401c47fdc$a0f722a0$e622c797@oemcomputer> <411A79AA.4020505@ocf.berkeley.edu> Message-ID: <1092322453.8672.67.camel@localhost> On Wed, 2004-08-11 at 15:55, Brett C. wrote: > Raymond Hettinger wrote: > > > For PEP 292, should the class be Template() or template()? > > > > Template as per PEP 8. Ok, I'll change that when I make my next round of mods to PEP 292. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/67d0d9fc/attachment.pgp From pje at telecommunity.com Thu Aug 12 17:00:15 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 12 17:00:22 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> Message-ID: <5.1.1.6.0.20040812105836.0321fec0@mail.telecommunity.com> At 10:45 PM 8/11/04 -0700, Guido van Rossum wrote: >Anyway, if we really do have enough use cases for byte array literals, >we might add them. I still think that would be confusing though, >because byte arrays are most useful if they are mutable: and then we'd >have mutable literals -- blechhhh! Not if they work like list or dictionary "literals". That is, if they're just short for 'array("B","literal here")'. From jack at performancedrivers.com Thu Aug 12 17:03:54 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Thu Aug 12 17:03:58 2004 Subject: [Python-Dev] @decorator now w/ class support [was: pep 318, Decorators for Functions, Methods and Classes] In-Reply-To: <2macx0zllk.fsf@starship.python.net> References: <20040811011116.GF23725@performancedrivers.com> <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> <200408072105.i77L5Ea10965@guido.python.org> <20040811011116.GF23725@performancedrivers.com> <5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com> <2macx0zllk.fsf@starship.python.net> Message-ID: <20040812150354.GN23725@performancedrivers.com> On Thu, Aug 12, 2004 at 11:39:35AM +0100, Michael Hudson wrote: > "Phillip J. Eby" writes: > > > At 03:44 PM 8/11/04 -0400, Jack Diederich wrote: > >> I have a mostly working patch for class decorators but it gets an > >> XXX ambiguity > >>between > >> > >>funcdef: [decorators] 'def' NAME parameters ':' suite > >>and > >>classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite > >> > >> It works as advertised with the pre-decorators funcdef definition > >> (which removes the ambiguity). > >>Help? > > > > > > How about this? > > > > decorated_assignment: [decorators] (funcdef | classdef) > > decorated_definition or something would be better surely? I ended up going with 'decorated_thing' http://python.org/sf/1007991 decorator code moved out of the function code and will decorate both functions and classes. -Jack From tim.peters at gmail.com Thu Aug 12 17:23:23 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 12 17:23:27 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <2mekmczlv7.fsf@starship.python.net> References: <000001c47e88$81ef1d40$5635c797@oemcomputer> <1f7befae0408092348583f0953@mail.gmail.com> <2mvffqzum9.fsf@starship.python.net> <1f7befae0408101258431039a1@mail.gmail.com> <1f7befae04081123253f08bb71@mail.gmail.com> <2mekmczlv7.fsf@starship.python.net> Message-ID: <1f7befae04081208237cd173ef@mail.gmail.com> [Michael Hudson Has the failure mode of alloca() changed? No, and the Windows stack-check code works fine. I showed results before from boosting the amount of padding the Windows stack-check code checks for, and it if checks for 20x more (which is ridiculously large) padding than it checks for now, it reliably generates Python stack-overflow MemoryErrors. Indeed, the KeyError exceptions were traced specifically to this: a stack-overflow MemoryError, due to the Windows stack-check code, getting wiped out by lookdict (whose caller took lookdict's NULL return as meaning the key wasn't present in the dict -- although it actually was). > I take it you're building with VC++ 7.1? Right. > What happens for a VC++ 6 build? Raymond reported on that earlier. Appeared to be the same as I saw in a release build. He didn't report on a debug build. He's running WinME, so a "hard" failure may look quite different for him. > Hmm, a moment with msdn suggests that there's been no significant > changes here, although the documentation is for _alloca(), and Python > calls alloca(). That can't make any difference, can it? Right, no difference. > It still smells like a tool change to me. Not to me. Although it does smell. From skip at pobox.com Thu Aug 12 16:29:29 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 17:33:16 2004 Subject: [Python-Dev] adding a bytes sequence type to Python In-Reply-To: <411B0DA0.4030109@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> Message-ID: <16667.32457.28553.468008@montanaro.dyndns.org> Martin> bytes("GET") Martin> which would implicitly convert each character to Latin-1. That's why I think a special literal is necessary. There'd be no unicode foolishness involved. ;-) They'd just be raw uninterpreted bytes. Skip From skip at pobox.com Thu Aug 12 16:27:34 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 17:33:20 2004 Subject: [Python-Dev] adding a bytes sequence type to Python In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> Message-ID: <16667.32342.640469.913208@montanaro.dyndns.org> Guido> Anyway, if we really do have enough use cases for byte array Guido> literals, we might add them. I still think that would be Guido> confusing though, because byte arrays are most useful if they are Guido> mutable: and then we'd have mutable literals -- blechhhh! Today I can initialize mutable objects from immutable strings: >>> print list("abc") ['a', 'b', 'c'] >>> print set("abc") set(['a', 'c', 'b']) I see no reason that mutable bytes objects couldn't be created from otherwise immutable sequences either. Would it be a problem to ensure that a = b"abc" b = b"abc" print a is b prints False? The main difference as I see it is that byte literals would be completely devoid of any sort of interpretation as unicode sequences. It would be nice if this was possible: # -*- coding: utf-8 -*- b = b"?" though that would probably wreak havoc with editors and hex escapes would have to be used in most situations. Skip From skip at pobox.com Thu Aug 12 16:09:05 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 17:33:27 2004 Subject: [Python-Dev] adding a bytes sequence type to Python In-Reply-To: <200408120414.i7C4Er429003@guido.python.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> Message-ID: <16667.31233.264061.518559@montanaro.dyndns.org> (changing the subject - sorry i didn't do it in my first two messages...) >> 1. Make bytes a synonuym for str. Guido> Hmm... I worry that a simple alias would just encourage confused Guido> usage, since the compiler won't check. I'd rather see bytes an Guido> alias for a bytes array as defined by the array module. You're right. This could probably be added now ("now" being 2.5) with little or no problem. My thought was to get the name in there quickly (could be done in 2.4) with some supporting documentation so people could begin modifying their code. >> 2. Warn about the use of bytes as a variable name. Guido> Is this really needed? Builtins don't byte variable names. I suppose this could be dispensed with. Let pychecker handle it. >> 3. Introduce b"..." literals as a synonym for current string >> literals, and have them *not* generate warnings if non-ascii >> characters were used in them without a coding cookie. Guido> I expecet all sorts of problems with that, such as what it would Guido> mean if Unicode or multibyte characters are used in the source. My intent in proposing b"..." literals was that they would be allowed in any source file. Their contents would not be interpreted in any way. One simple use case: identifying the magic number of a binary file type of some sort. That might well be a constant to programmers manipulating that sort of file and have nothing to do with Unicode at all. Guido> Do we really need byte array literals at all? I think so. Martin already pointed out an example where a string literal is used today for a sequences of bytes that's put out on the wire as-is. It's just convenient that the protocol was developed in such a way that most of its meta-data is plain ASCII. Skip From skip at pobox.com Thu Aug 12 16:12:27 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 17:33:31 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <411AFBD4.5080903@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> Message-ID: <16667.31435.566032.294560@montanaro.dyndns.org> Martin> Java also supports byte arrays in the source, although they are Martin> difficult to type: Martin> byte[] request = {'G', 'E', 'T'}; Seems to me that b"GET" would be more Pythonic given existing Python string literals. Martin> As for reading from streams: Java has multiple reader APIs; some Martin> return byte strings, some character strings. I think Guido's proposed 'B' might make sense here. OTOH, today's 'b' might work as well, though a switch of that magnitude could probably not be made until 3.0 if bytes objects are not synonyms for strings. Skip From python at rcn.com Thu Aug 12 17:46:58 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 12 17:47:16 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae04081208237cd173ef@mail.gmail.com> Message-ID: <002d01c48083$9a89a6c0$5229c797@oemcomputer> > > What happens for a VC++ 6 build? > > Raymond reported on that earlier. Appeared to be the same as I saw in > a release build. He didn't report on a debug build. He's running > WinME, so a "hard" failure may look quite different for him. On a debug build with VC++ 6 on WinMe, the script faults immediately and the program terminates with no useful messages of any sort. Raymond From guido at python.org Thu Aug 12 18:10:48 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 18:10:55 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Thu, 12 Aug 2004 09:09:05 CDT." <16667.31233.264061.518559@montanaro.dyndns.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <16667.31233.264061.518559@montanaro.dyndns.org> Message-ID: <200408121610.i7CGAmX30436@guido.python.org> > >> 1. Make bytes a synonuym for str. > > Guido> Hmm... I worry that a simple alias would just encourage confused > Guido> usage, since the compiler won't check. I'd rather see bytes an > Guido> alias for a bytes array as defined by the array module. > > You're right. This could probably be added now ("now" being 2.5) with > little or no problem. My thought was to get the name in there quickly > (could be done in 2.4) with some supporting documentation so people could > begin modifying their code. I think very few people would do so until the semantics of bytes were clearer. Let's just put it in meaning byte array when we're ready. > >> 2. Warn about the use of bytes as a variable name. > > Guido> Is this really needed? Builtins don't byte variable names. > > I suppose this could be dispensed with. Let pychecker handle it. > > >> 3. Introduce b"..." literals as a synonym for current string > >> literals, and have them *not* generate warnings if non-ascii > >> characters were used in them without a coding cookie. > > Guido> I expecet all sorts of problems with that, such as what it would > Guido> mean if Unicode or multibyte characters are used in the source. > > My intent in proposing b"..." literals was that they would be > allowed in any source file. Their contents would not be interpreted > in any way. But they would be manipulated if they were non-ASCII and the source file was converted to a different encoding. Better be safe and only allow printable ASCII and hex escapes there. > One simple use case: identifying the magic number of a > binary file type of some sort. That might well be a constant to > programmers manipulating that sort of file and have nothing to do > with Unicode at all. Not a very strong use case, this could easily be done using just hex. > Guido> Do we really need byte array literals at all? > > I think so. Martin already pointed out an example where a string > literal is used today for a sequences of bytes that's put out on the > wire as-is. It's just convenient that the protocol was developed in > such a way that most of its meta-data is plain ASCII. See my response to that. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Thu Aug 12 18:17:18 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 12 18:17:21 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <002d01c48083$9a89a6c0$5229c797@oemcomputer> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> Message-ID: <1f7befae040812091754035bcb@mail.gmail.com> More clues. As expected, commenting out the tests in test_parser.py that contain deeply nested tuples makes the problems go away. A surprise: adding code to every function in parsermodule.c to do Py_FatalError() if the Windows stack-check fails doesn't make any difference. So it's not C recursion in parsermodule.c that causes this (although I bet it *could*, it simply doesn't in this test case). Not a surprise: Grzegorz Makarewicz's patch to call the stack-check code in pymalloc catches the problem "in time". But we could put the same patch anywhere in Python that gets called frequently, and catch it in time. That's too expensive to bear. The Windows stack-check code just isn't called often enough, and the recursion limit is "too large" now to live without a reliable stack check. Py_EnterRecursiveCall() in fact rarely calls the Windows stack-check code. At the time the Windows stack-check code first *would* have complained (had it been called then), the tstate recursion depth was 694, and _Py_CheckRecursionLimit was 973. The funky way this code works, that means we can do ~150 more levels of Python call before the Windows stack-check code is called again, and that's too late. We only checked that there's still room for 2K pointers on the stack, and 150 Python-level calls means about 450 C-level calls in this test: the stack at this point is hundreds (& hundreds) of repetitions of PyEval_EvalFrame -> fast_function -> call_function -> PyEval_Frame -> etc etc etc and PyEval_EvalFrame C frames are fat enough on their own for 150 of them to blow what remains of the stack. The easiest workaround for Windows is still to boost the stack size. I'm going to do that if nobody screams. Looks like nobody has an explanation yet for why 2.3.4 consistently yielded MemoryError but 2.4a2 mixes those with spurious KeyError and SyntaxError exceptions. That could be coincidence in this specific test driver, though -- lookdict has always wiped out exceptions. where's-stackless-when-you-need-it-ly y'rs - tim From janssen at parc.com Thu Aug 12 18:23:10 2004 From: janssen at parc.com (Bill Janssen) Date: Thu Aug 12 18:23:31 2004 Subject: [Python-Dev] CGI module [was: New Subscriber] In-Reply-To: Your message of "Thu, 12 Aug 2004 00:15:50 PDT." <70C26872-EC2F-11D8-B23F-000A957D4F2A@mac.com> Message-ID: <04Aug12.092313pdt."58612"@synergy1.parc.xerox.com> Steve Majewski writes: > As far as I can tell, all of those threads died out without any actual > code being produced. However, they do give a good enumeration of what's > wrong with the current cgi module and what's needed in a new module. > > I guess the Web-SIG is the best place for any further discussion. > http://mail.python.org/mailman/listinfo/web-sig As Steve notes, we've discussed this subject quite a bit. The next step is to form some more structured proposals, in the form of PEP submissions. These can then properly be considered on c.l.p. and python-dev. If folks want to contribute, they can review the past discussion, pick an area to write a PEP on (it would be polite to announce that on Web-SIG), write a proposal, and continue on with the process. This is starting to happen. Those on Web-SIG will see that Phillip Eby has submitted a pre-PEP on a Web Server Gateway Interface (WSGI), which is being discussed right now. [http://mail.python.org/pipermail/web-sig/2004-August/000518.html] Remember that having working code sometimes helps people evaluate a PEP proposal :-). Bill From gmccaughan at synaptics-uk.com Thu Aug 12 18:23:33 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu Aug 12 18:24:05 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408121444.i7CEi5c30221@guido.python.org> References: <1092258146.411a896246d1a@mcherm.com> <200408121306.58446.gmccaughan@synaptics-uk.com> <200408121444.i7CEi5c30221@guido.python.org> Message-ID: <200408121723.33597.gmccaughan@synaptics-uk.com> [I said:] >> This is why "integer" is a better name than "baseinteger". >> For now it can be the common supertype of int and long. >> In the future, it can be the name of the single integer >> type. [Guido:] > No, that will be int, of course! Dang. I suppose it has to be, for hysterical raisins. > Like 'basestring', 'baseinteger' is intentionally cumbersome, because > it is only the base class of all *built-in* integral types. Note that > UserString is *not* subclassing basestring, and likewise if you wrote > an integer-like class from scratch it should not inherit from > baseinteger. Code testing for these types is interested in knowing > whether something is a member of one of the *built-in* types, which is > often needed because other built-in operations (e.g. many extension > modules) only handle the built-in types. > > If you want to test for integer-like or string-like behavior, you > won't be able to use isinstance(), but instead you'll have to check > for the presence of certain methods. > > I know this is not easy in the case of integers, but I don't want to > start requiring inheritance from a marker base type now. Python is > built on duck typing. (Google for it.) I don't think this is a good reason for rejecting "integer" as the name of the common supertype of int and long. You'd use isinstance(x,integer) to check whether x is an integer of built-in type, just as you currently use isinstance(x,float) to check whether x is a floating-point number of built-in type. I see no reason why a cumbersome name is much advantage. I am, however, convinced by the other argument: All the steps towards int/long unification that have been taken so far assume that the endpoint is having a single type called "int", and that would be derailed by changing the target name to "integer". Not to mention that there's any amount of code out there that uses "int" for conversions and the like, which there's no reason to break. I can't get away from the feeling that it would, in some possibly over-abstract sense, have been neater to introduce "integer" as the supertype and encourage people to *change* from using "int" (and "long") to using "integer", so that the meanings of "int" and "long" never change in the unification process. But, even if I could convince you, it's too late for that :-). By the way, I wasn't at any point proposing that inheritance from a "marker" type should be required for anything, and I'm not sure why you thought I were. I expect I was unclear, and I'm sorry about that. -- g From janssen at parc.com Thu Aug 12 18:25:49 2004 From: janssen at parc.com (Bill Janssen) Date: Thu Aug 12 18:26:32 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Thu, 12 Aug 2004 07:30:06 PDT." <200408121430.i7CEU6F30168@guido.python.org> Message-ID: <04Aug12.092558pdt."58612"@synergy1.parc.xerox.com> > Hm, u"".encode("utf-8") should probably return a bytes > array, and that might be sufficient. +1. Bill From python at discworld.dyndns.org Thu Aug 12 18:38:38 2004 From: python at discworld.dyndns.org (Charles Cazabon) Date: Thu Aug 12 18:31:11 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae040812091754035bcb@mail.gmail.com> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> Message-ID: <20040812163838.GA9343@discworld.dyndns.org> Tim Peters wrote: > > The Windows stack-check code just isn't called often enough, and the > recursion limit is "too large" now to live without a reliable stack > check. [...] > The easiest workaround for Windows is still to boost the stack size. > I'm going to do that if nobody screams. Would reducing the recursion limit (for Windows) be a reasonable approach? Charles P.S. Please don't cc: me on list messages. -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://www.qcc.ca/~charlesc/software/ ----------------------------------------------------------------------- From l at lrowe.co.uk Thu Aug 12 18:31:10 2004 From: l at lrowe.co.uk (Laurence Rowe) Date: Thu Aug 12 18:40:32 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered In-Reply-To: <1092179578.4119567af011d@webmail.oregonstate.edu> References: <1092179578.4119567af011d@webmail.oregonstate.edu> Message-ID: I must say I like this idea a lot. It seems much more in keeping with the rest of the python block structure. You could even think of decorators as bullet points. Now if only someone hadn't already found a use for '*' character... For those of us who like code-folding editors, this option also allows them to continue folding the whole code block down to one line. - Laurence > ----------------------------------------------- > Option B > ----------------------------------------------- > > def f(a, b): > @staticmethod > @synchronized(lockvar) > """ > Hi, > > Performs a binary operation. > > Docstring. > """ > return a + b > - Connelly From mwh at python.net Thu Aug 12 19:36:30 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 12 19:36:32 2004 Subject: [Python-Dev] Third Bug Day outcome In-Reply-To: <20040812132706.GF21744@rogue.amk.ca> (A. M. Kuchling's message of "Thu, 12 Aug 2004 09:27:06 -0400") References: <20040812132706.GF21744@rogue.amk.ca> Message-ID: <2mllgkxnq9.fsf@starship.python.net> "A.M. Kuchling" writes: > On Mon, Aug 09, 2004 at 07:19:04PM -0300, Batista, Facundo wrote: >> Maybe for the next Bug Day should be two "IRC hosts", one in America and >> other in Europe. so, the'll be 6 hs each (top) and the whole day will be of >> 12 hours. > > That would be a good idea, if a volunteer in Europe can be found. I'm game, but have quite a tendency to be away at weekends, so it'll depend if I'm around for any given bug day. Cheers, mwh -- MARVIN: Oh dear, I think you'll find reality's on the blink again. -- The Hitch-Hikers Guide to the Galaxy, Episode 12 From mwh at python.net Thu Aug 12 19:40:03 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 12 19:40:10 2004 Subject: [Python-Dev] adding a bytes sequence type to Python In-Reply-To: <16667.31233.264061.518559@montanaro.dyndns.org> (Skip Montanaro's message of "Thu, 12 Aug 2004 09:09:05 -0500") References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <16667.31233.264061.518559@montanaro.dyndns.org> Message-ID: <2mekmcxnkc.fsf@starship.python.net> Skip Montanaro writes: > My intent in proposing b"..." literals was that they would be allowed in any > source file. Their contents would not be interpreted in any way. I think this is a bad idea. If a coding cookie says a file is in utf-8, then the file really has to be valid utf-8 data, for implementation sanity and not freaking out editors. The use-case for including arbitrary chunks of binary data in a source file seems stretched in the extreme. Cheers, mwh -- My hat is lined with tinfoil for protection in the unlikely event that the droid gets his PowerPoint presentation working. -- Alan W. Frame, alt.sysadmin.recovery From tim.peters at gmail.com Thu Aug 12 19:54:01 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 12 19:54:08 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <20040812163838.GA9343@discworld.dyndns.org> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812163838.GA9343@discworld.dyndns.org> Message-ID: <1f7befae04081210541b570143@mail.gmail.com> [Charles Cazabon] > Would reducing the recursion limit (for Windows) be a reasonable > approach? Not if that means Windows can't run the standard test suite. Boosting the stack size on Windows is a VM reservation operation, BTW -- it doesn't actually increase the RAM needed, unless the stack actually needs to grow that big. From s.percivall at chello.se Thu Aug 12 19:56:14 2004 From: s.percivall at chello.se (Simon Percivall) Date: Thu Aug 12 19:56:18 2004 Subject: [Python-Dev] @decorator now w/ class support [was: pep 318, Decorators for Functions, Methods and Classes] In-Reply-To: <20040812150354.GN23725@performancedrivers.com> References: <20040811011116.GF23725@performancedrivers.com> <200408061555.i76FtV707448@guido.python.org> <4113BACA.6040801@ieee.org> <200408072105.i77L5Ea10965@guido.python.org> <20040811011116.GF23725@performancedrivers.com> <5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com> <2macx0zllk.fsf@starship.python.net> <20040812150354.GN23725@performancedrivers.com> Message-ID: On 2004-08-12, at 17.03, Jack Diederich wrote: > I ended up going with 'decorated_thing' > http://python.org/sf/1007991 > > decorator code moved out of the function code and will decorate > both functions and classes. That's wonderful! +1 for inclusion from me, when the patch is final, etc. //Simon From dave at boost-consulting.com Thu Aug 12 20:05:04 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 12 20:05:47 2004 Subject: [Python-Dev] Re: Another approach to decorators. References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> Message-ID: David Eppstein writes: > In article , > David Abrahams wrote: > >> > class Foo: >> > >> > decorate static: >> > >> > def static1(blah...): >> > pass >> > >> > decorate locksFile: >> > >> > def static2andLocks(blah...): # both decorators appy >> > pass >> >> Wow, Martin Z's idea (using your keyword) really went "thunk" for me. >> What decorate does would be very much like what "class" does in some >> ways. > > class: (and other something: constructs) start a block that can contain > any code. Does this decorate keyword allow e.g. assignments as well as > defs and other decorates? Or loops? Why not? class: starts a block that establishes a new dict as the execution namespace and then passes the result off to the metaclass. decorate might do the same. > If so what should it mean? Whatever the user wants, I guess? > Is it like that locals() gets replaced by a special > dictionary-like-object that calls the decorator whenever any of its > contents gets set? I was thinking it calls the decorator afterwards, just like class calls the metaclass afterwards. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From dave at boost-consulting.com Thu Aug 12 20:07:45 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 12 20:10:20 2004 Subject: [Python-Dev] Re: Another approach to decorators. References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411ADCFA.8050305@divmod.com> Message-ID: Jp Calderone writes: > David Eppstein wrote: >> In article , >> David Abrahams wrote: >> >>>>class Foo: >>>> >>>> decorate static: >>>> >>>> def static1(blah...): >>>> pass >>>> >>>> decorate locksFile: >>>> >>>> def static2andLocks(blah...): # both decorators appy >>>> pass >>> >>>Wow, Martin Z's idea (using your keyword) really went "thunk" for me. >>>What decorate does would be very much like what "class" does in some >>>ways. >> class: (and other something: constructs) start a block that can >> contain any code. Does this decorate keyword allow e.g. assignments >> as well as defs and other decorates? Or loops? If so what should >> it mean? Is it like that locals() gets replaced by a special >> dictionary-like-object that calls the decorator whenever any of its >> contents gets set? >> > > (I have no idea what the original poster intended, however....) > > 'decorate' expression ':' > suite > > could create a nested scope, the locals of which could be passed to > whatever "expression" evaluates to (or if it is a tuple, do the > looping thing, since people seem to like that). > > The call can return a dict with which the class dict is updated. That's almost exactly what I intended. > This covers a use case James Knight mentioned as well. As he > pointed out, '@' and other proposals only decorate functions (and > maybe classes, soon). The example he gave, I believe, was: > > @public > x = 1 > > This doesn't work, but: > > public: > x = 1 > > would. Now, swallow that scream of horror (I did, barely > ;). No screams; I love it. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From fredrik at pythonware.com Thu Aug 12 20:18:27 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu Aug 12 20:16:48 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp><873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: Mike Coleman wrote: > Re maintenance, yeah regexp is pretty terse and ugly. Generally, though, I'd > rather deal with a reasonably well-considered 80 char regexp than 100 lines of > code that does the same thing. well, the examples in your PEP can be written as: data = [line[:-1].split(":") for line in open(filename)] and import ConfigParser c = ConfigParser.ConfigParser() c.read(filename) data = [] for section in c.sections(): data.append((section, c.items(section))) both of which are shorter than your structparse examples. and most of the one-liners in your pre-PEP can be handled with a combination of "match" and "finditer". here's a 16-line helper that parses strings matching the "a(b)*c" pattern into a prefix/list/tail tuple. import re def parse(string, pat1, pat2): """Parse a string having the form pat1(pat2)*""" m = re.match(pat1, string) i = m.end() a = m.group(1) b = [] for m in re.compile(pat2 + "|.").finditer(string, i): try: token = m.group(m.lastindex) except IndexError: break b.append(token) i = m.end() return a, b, string[i:] >>> parse("hello 1 2 3 4 # 5", "(\w+)", "\s*(\d+)") ('hello', ['1', '2', '3', '4'], ' # 5') tweak as necessary. From dave at boost-consulting.com Thu Aug 12 20:10:37 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 12 20:20:21 2004 Subject: [Python-Dev] Re: Another approach to decorators. References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> <20040811142802.0386bd60.michel@dialnetwork.com> <200408120426.i7C4QU829083@guido.python.org> Message-ID: Guido van Rossum writes: > [Various proposals suggesting the use of a single decorator block to > decorate multiple method definitions] > > I'm not keen on this. I expect that beyond classmethod and > staticmethod there aren't many use cases for repeating the same > decorator exactly multiple times. It also requires indenting the > method, which I've already argued against. And lastly it removes the > decorator from the method definition again, thereby defeating the > readability goals: when you look at a def in isolation, you may forget > to look several pages up for a decorator. > > I think this is too far-fetched to consider as an alternative to the > humble @decorator. Maybe, but if you're still serious about improving support for domain-specific embedded languages in Python you ought to give the ideas another look. If a reasonable syntax for decorators falls out of a more general and useful mechanism, so much the better. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From martin at v.loewis.de Thu Aug 12 20:21:00 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 12 20:20:56 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16667.32457.28553.468008@montanaro.dyndns.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <16667.32457.28553.468008@montanaro.dyndns.org> Message-ID: <411BB50C.8000804@v.loewis.de> Skip Montanaro wrote: > That's why I think a special literal is necessary. There'd be no unicode > foolishness involved. ;-) They'd just be raw uninterpreted bytes. But you'd spell them b"GET", no? If so, which numeric value has "G"? Regards, Martin From arigo at tunes.org Thu Aug 12 20:55:21 2004 From: arigo at tunes.org (Armin Rigo) Date: Thu Aug 12 21:00:24 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae040812091754035bcb@mail.gmail.com> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> Message-ID: <20040812185521.GA2277@vicky.ecs.soton.ac.uk> Hello Tim, On Thu, Aug 12, 2004 at 12:17:18PM -0400, Tim Peters wrote: > Looks like nobody has an explanation yet for why 2.3.4 consistently > yielded MemoryError but 2.4a2 mixes those with spurious KeyError and > SyntaxError exceptions. The reason is that Py_EnterRecursiveCall() was only introduced in 2.4. Comparisons didn't throw RuntimeErrors that easily in 2.3. Armin From skip at pobox.com Thu Aug 12 21:14:35 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 21:14:46 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered In-Reply-To: References: <1092179578.4119567af011d@webmail.oregonstate.edu> Message-ID: <16667.49563.652742.680876@montanaro.dyndns.org> Laurence> I must say I like this idea a lot. It seems much more in Laurence> keeping with the rest of the python block structure. You could Laurence> even think of decorators as bullet points. Now if only someone Laurence> hadn't already found a use for '*' character... I think '*' is syntactically available for this use. It's not used as a unary operator I don't believe. The *args thing isn't available in the context where decorators would be used. That also has the nice side effect of turning @ back over to tools like leo and ipython. Unlike '|' it's not likely to be mistaken for other commonly used characters. Also, a unary '*' or '**' already have some special meaning when used in and around function definitions, so seeing *deco def func(...): ... or def func(...): *deco ... might be a small hint to readers of the code that deco is somehow involved in the function's definition. Skip From skip at pobox.com Thu Aug 12 21:19:22 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 21:19:30 2004 Subject: [Python-Dev] adding a bytes sequence type to Python In-Reply-To: <200408121430.i7CEU6F30168@guido.python.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <200408121430.i7CEU6F30168@guido.python.org> Message-ID: <16667.49850.156147.876146@montanaro.dyndns.org> >> This is the approach taken in the other languages: Everytime the >> array display is executed, a new array is created. There is then no >> problem with that being mutable. Guido> The downside of that is that then for performance reasons you Guido> might end up having to move bytes literals out of expressions if Guido> they are in fact used read-only (which the compiler can't know Guido> but the user can). Wouldn't the compiler be able to tell it was to be treated specially if it saw b"GET"? In that case, the code generated for x = b"GET" would be something like LOAD_CONST "GET" LOAD_NAME bytes CALL_FUNCTION 1 STORE_FAST x Skip From tim.peters at gmail.com Thu Aug 12 21:41:37 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 12 21:41:53 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <20040812185521.GA2277@vicky.ecs.soton.ac.uk> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> Message-ID: <1f7befae04081212414007274f@mail.gmail.com> [Tim] >> Looks like nobody has an explanation yet for why 2.3.4 consistently >> yielded MemoryError but 2.4a2 mixes those with spurious KeyError and >> SyntaxError exceptions. [Armin] > The reason is that Py_EnterRecursiveCall() was only introduced in 2.4. > Comparisons didn't throw RuntimeErrors that easily in 2.3. Doh! Of course. Recursive compares aren't implicated in the test programs that failed here. Under 2.3.4, all the compares return normally because they're not stack-checking at all, and the program "gets to" recurse deeper then, until a "recursion depth exceeded" exception gets thrown. But in 2.4 it's a crap shoot whether a comparison or a recursive Python call notices first that we're nearing the end of the stack. The recursive Python calls make real stack demands in these tests, but the comparisons are just int-vs-int and string-vs-string, so the stack-check in compare now is "almost always" a nuisance check in these tests. Sounds like a good idea not to run out to stack . From barry at python.org Thu Aug 12 21:47:41 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 12 21:47:25 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered In-Reply-To: <16667.49563.652742.680876@montanaro.dyndns.org> References: <1092179578.4119567af011d@webmail.oregonstate.edu> <16667.49563.652742.680876@montanaro.dyndns.org> Message-ID: <1092340061.8681.109.camel@localhost> On Thu, 2004-08-12 at 15:14, Skip Montanaro wrote: > I think '*' is syntactically available for this use. That was one of the characters I tried back when I was looking for @ alternatives. For some reason I can't explain, this one just rubs me the wrong way. i'll-let-tim-psychoanalyze-the-reasons-why-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/5f6b69ed/attachment.pgp From skip at pobox.com Thu Aug 12 22:40:58 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 12 22:41:06 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <411BB50C.8000804@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <16667.32457.28553.468008@montanaro.dyndns.org> <411BB50C.8000804@v.loewis.de> Message-ID: <16667.54746.535997.120878@montanaro.dyndns.org> >> That's why I think a special literal is necessary. There'd be no >> unicode foolishness involved. ;-) They'd just be raw uninterpreted >> bytes. Martin> But you'd spell them b"GET", no? If so, which numeric value has Martin> "G"? Good point... From arigo at tunes.org Thu Aug 12 22:44:31 2004 From: arigo at tunes.org (Armin Rigo) Date: Thu Aug 12 22:49:10 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae04081212414007274f@mail.gmail.com> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> Message-ID: <20040812204431.GA31884@vicky.ecs.soton.ac.uk> Hello, On Thu, Aug 12, 2004 at 03:41:37PM -0400, Tim Peters wrote: > Sounds like a good idea not to run out to stack . I don't remember if it was mentioned here, but maybe we'd better check directly whether the C stack is too large rather than (or possibly in addition to) using a limit on the number of Python iterations. This is not really ANSI C, but I can't think of a setting in which the following trick would fail to be a good indication of the amount of stack space consumed: save a pointer to a local variable "early", e.g. in Py_Initialize(); then in any other function call, the distance between this pointer and a pointer to some local equals the amount of stack space consumed by Python minus a few bytes. If this sounds too much of a hack, the (usual) recursion limit could be kept to limit nested Python frames; but all C-level recursions (including eval_frame) could additionally use the above trick. Its advantage is that it is an extremely fast check. If the total stack size is difficult to know in advance, we can still use PyOS_CheckStack(), but more efficiently and less randomly than now, by maintaining a "high tide" pointer that records how much stack we are sure we have, and calling PyOS_CheckStack() only to attempt to push the "high tide" mark further. While I'm in a hacking mood, there might be a way to prevent PyErr_Clear() to clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and KeyboardInterrupt: for these exceptions, let's just store them away instead of clearing them, and re-raise them at the next occasion. The "next occasion" would have to be defined more precisely, but there is probably a way to ensure that it will at least be before the next non-trivial opcode operation starts. It would overwrite an exception set later, like those spurious KeyError we get for dict lookups. It might be a better-than-nothing quick fix to all these PyErr_Clear() all around the code base. A bient?t, Armin. From guido at python.org Thu Aug 12 22:56:07 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 22:56:13 2004 Subject: [Python-Dev] adding a bytes sequence type to Python In-Reply-To: Your message of "Thu, 12 Aug 2004 14:19:22 CDT." <16667.49850.156147.876146@montanaro.dyndns.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <200408121430.i7CEU6F30168@guido.python.org> <16667.49850.156147.876146@montanaro.dyndns.org> Message-ID: <200408122056.i7CKu7n30960@guido.python.org> > Wouldn't the compiler be able to tell it was to be treated specially if it > saw b"GET"? In that case, the code generated for > > x = b"GET" > > would be something like > > LOAD_CONST "GET" > LOAD_NAME bytes > CALL_FUNCTION 1 > STORE_FAST x This is actually an illustration of what I meant: a performance-aware person might want to move that CALL_FUNCTION out of an inner loop if they knew the result was never modified inside the loop. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 12 22:53:27 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 12 22:57:56 2004 Subject: [Python-Dev] Re: Another approach to decorators. In-Reply-To: Your message of "Thu, 12 Aug 2004 14:10:37 EDT." References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> <20040811142802.0386bd60.michel@dialnetwork.com> <200408120426.i7C4QU829083@guido.python.org> Message-ID: <200408122053.i7CKrRg30941@guido.python.org> > > I think this is too far-fetched to consider as an alternative to the > > humble @decorator. > > Maybe, but if you're still serious about improving support for > domain-specific embedded languages in Python you ought to give the > ideas another look. If a reasonable syntax for decorators falls out > of a more general and useful mechanism, so much the better. How does decorate , ...: improve support for domain-specific embedded languages? --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Aug 12 23:12:10 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 12 23:12:27 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1092177771.21839.54.camel@localhost> Message-ID: <007301c480b1$0867bec0$5229c797@oemcomputer> [Barry] > Now for the controversial bit . > > I still think it's worthwhile to turn the string module into a package. > The sandbox has an implementation of this that is completely backward > compatible. I like it because it segregates the deprecated functions > into a separate sub-module and makes the whole shebang easier to > maintain. It also provides an existing namespace for future expansion, > as has been discussed before. > > Please note: PEP 292 is not dependent on the string module -> package > reorganization, so it isn't documented in that PEP. I don't think it's > worth writing a PEP just for this library reorganization. Take a look > at the sandbox if you're interested. > > http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/stri ng > / > > As far as this aspect of the changes go, I would like BDFL pronouncement > on it. I'll abide by Guido's decision here. If he nixes the re-org, > then I propose to put the template and safe_template classes in > Lib/string.py. The libstring.tex changes will still be relevant. After more thought, I would like to voice a strong -1 on the packaging of string. At the outset, I felt that it introduced unnecessary complexity. Adding a new directory, an __init__.py, and two modules just to add a few lines of code is overkill. Package logic should be reserved for organizing voluminous code like the email package. Arguably, the logging package should have been just a single module -- putting it in a package just made it more difficult to read and maintain. To test the existing sandbox code, I moved it into the library and found a circular import issue: importing the string package, causes template.py to be imported, which in turn imports re which uses sre_parse.py, needs to import string. So, the whole thing fails. It is fixable, but it sucks to have to start doing work-arounds from the outset just to compensate for something that adds no value to begin with. The package has already complicated review and updates. I'm having to re-write it in non-package form just to add doctest (which fails by the way) and other small fix-ups. My recommendation: Create a stringtools module with the new template stuff in it and with room for growth. Leave string.py alone and let it die in peace. The business end of the template code is only a few lines long. Wrapping it in a package is like using cannon to kill a mosquito. Let's not make something hard out of something trivially simple. Raymond P.S. Side rant: Also, in the last few months, the #ifdefs have multiplied. While I understand that some things like time stamping are worthwhile, the code is getting harder to read and maintain because of all this b.s. At some point, someone changed a definition and I had to add an #ifdef to import addrinfo.h for VC++ 6.0. I noticed the same had to be done for Cygwin and there was a similar GCC entry. Each checkin was done by a separate person fixing a crash in their own part of the universe. The right answer would have likely been to refactor the original change that mucked up everyone else's compile. Instead we ended up with a pile of conditionals that will be hard to maintain going forward because the surrounding code has become so stiff. From bob at redivi.com Thu Aug 12 23:25:24 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Aug 12 23:25:58 2004 Subject: [Python-Dev] Re: Another approach to decorators. In-Reply-To: <200408122053.i7CKrRg30941@guido.python.org> References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> <20040811142802.0386bd60.michel@dialnetwork.com> <200408120426.i7C4QU829083@guido.python.org> <200408122053.i7CKrRg30941@guido.python.org> Message-ID: <1F6D2F67-ECA6-11D8-B20E-000A95686CD8@redivi.com> On Aug 12, 2004, at 4:53 PM, Guido van Rossum wrote: >>> I think this is too far-fetched to consider as an alternative to the >>> humble @decorator. >> >> Maybe, but if you're still serious about improving support for >> domain-specific embedded languages in Python you ought to give the >> ideas another look. If a reasonable syntax for decorators falls out >> of a more general and useful mechanism, so much the better. > > How does > > decorate , ...: > > > improve support for domain-specific embedded languages? I guess it depends on what is passed to the decorators. If enough information is passed to the decorators it might almost be like a macro. Practically speaking, I don't see how it would help very much considering that the block would be executed before the expression. If it wasn't executed before the decorators, then I don't see how it would be any better than @ def throwawayname(): I suppose that decorators in the decorate , ...: syntax would take a dict of the block's local namespace and return a dict of anything that should be set in the local namespace of the decorate statement. But now that I think about it some more, it would probably be more useful to domain-specific languages if the decorators were allowed to modify the namespace *before* and *after* the block. For example, to import a whole bunch of stuff for a domain specific purpose (say, writing SQL code, doing AppleEvents, etc.) and then get rid of it afterwards? Anyway, I think that the decorate , ...: syntax might be useful someday, but I don't think that it could usefully have the same semantics as the current decorator PEP. -bob From foom at fuhm.net Thu Aug 12 23:28:05 2004 From: foom at fuhm.net (James Y Knight) Date: Thu Aug 12 23:28:10 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <200408110002.i7B02gu26415@guido.python.org> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> <200408101501.i7AF1HZ25725@guido.python.org> <200408110002.i7B02gu26415@guido.python.org> Message-ID: <7FA48FA3-ECA6-11D8-BF5E-000A95A50FB2@fuhm.net> On Aug 10, 2004, at 8:02 PM, Guido van Rossum wrote: >> Sooo should (for 'generator' in objects that claim to be in >> __builtins__ but aren't), >> 1) 'generator' be added to __builtins__ >> 2) 'generator' be added to types.py and its __module__ be set to >> 'types' >> 3) 'generator' be added to .py and its __module__ be set to >> '' (and a name for the module chosen) > > I guess (1). > > Please submit a patch to SF... Okay, so, I don't have a patch, but I have made a script to examine the situation. It seems the problem is a little more widespread than I had realized. I grepped through the python C source to find all the type objects, and then verified their existence. Just looking at Objects/*.c, there are 31 classes without appropriate bindings. Additionally, some aren't even valid identifier names. It seems like there's 4 categories here: iterators, descrobject stuffs, list comparison wrappers, and other. I'm not sure it's a good idea to add all these names to the builtins -- perhaps only the ones in the 'other' group? For {Modules,Mac,PC,RISCOS,Python}/*.c, most seem like they could relatively straightforwardly be added to their module, but I haven't examined them too closely. Some likely ought to have constructors added (e.g. the dl.dl('file') could be equivalent to dl.open('file')). __builtin__.dictionary-keyiterator not found (./Objects/dictobject.c) __builtin__.dictionary-valueiterator not found (./Objects/dictobject.c) __builtin__.dictionary-itemiterator not found (./Objects/dictobject.c) __builtin__.tupleiterator not found (./Objects/tupleobject.c) __builtin__.rangeiterator not found (./Objects/rangeobject.c) __builtin__.iterator not found (./Objects/iterobject.c) __builtin__.callable-iterator not found (./Objects/iterobject.c) __builtin__.listiterator not found (./Objects/listobject.c) __builtin__.listreverseiterator not found (./Objects/listobject.c) __builtin__.method_descriptor not found (./Objects/descrobject.c) __builtin__.classmethod_descriptor not found (./Objects/descrobject.c) __builtin__.member_descriptor not found (./Objects/descrobject.c) __builtin__.getset_descriptor not found (./Objects/descrobject.c) __builtin__.wrapper_descriptor not found (./Objects/descrobject.c) __builtin__.method-wrapper not found (./Objects/descrobject.c) __builtin__.sortwrapper not found (./Objects/listobject.c) __builtin__.cmpwrapper not found (./Objects/listobject.c) __builtin__.ellipsis not found (./Objects/sliceobject.c) types.EllipsisType __builtin__.builtin_function_or_method not found (./Objects/methodobject.c) types.BuiltinFunctionType types.BuiltinMethodType __builtin__.dictproxy not found (./Objects/descrobject.c) types.DictProxyType __builtin__.generator not found (./Objects/genobject.c) types.GeneratorType __builtin__.PyCObject not found (./Objects/cobject.c) __builtin__.classobj not found (./Objects/classobject.c) types.ClassType __builtin__.instance not found (./Objects/classobject.c) types.InstanceType __builtin__.instancemethod not found (./Objects/classobject.c) types.MethodType types.UnboundMethodType __builtin__.cell not found (./Objects/cellobject.c) __builtin__.NoneType not found (./Objects/object.c) types.NoneType __builtin__.NotImplementedType not found (./Objects/object.c) types.NotImplementedType __builtin__.frame not found (./Objects/frameobject.c) types.FrameType __builtin__.function not found (./Objects/funcobject.c) types.FunctionType types.LambdaType __builtin__.module not found (./Objects/moduleobject.c) types.ModuleType (the Mac ones I checked on my mac w/ python 2.3.0) _Qt.IdleManager not found (./Mac/Modules/qt/_Qtmodule.c) _Qt.SGOutput not found (./Mac/Modules/qt/_Qtmodule.c) module _OSA not installed (./Mac/Modules/osa/_OSAmodule.c) Nav.NavReplyRecord not found (./Mac/Modules/Nav.c) _Scrap.Scrap not found (./Mac/Modules/scrap/_Scrapmodule.c) module waste not installed (./Mac/Modules/waste/wastemodule.c) MacOS.ResourceFork not found (./Mac/Modules/macosmodule.c) icglue.ic_instance not found (./Mac/Modules/icgluemodule.c) __builtin__.PyHKEY not found (./PC/_winreg.c) __builtin__.drawf not found (./RISCOS/Modules/drawfmodule.c) __builtin__.block not found (./RISCOS/Modules/swimodule.c) __builtin__.traceback not found (./Python/traceback.c) types.TracebackType __builtin__.code not found (./Python/compile.c) types.CodeType __builtin__.symtable entry not found (./Python/symtable.c) __builtin__.tktimertoken not found (./Modules/_tkinter.c) __builtin__.tkapp not found (./Modules/_tkinter.c) __builtin__.arrayiterator not found (./Modules/arraymodule.c) _curses_panel.curses panel not found (./Modules/_curses_panel.c) linuxaudiodev.linux_audio_device not found (./Modules/linuxaudiodev.c) module fl not installed (./Modules/flmodule.c) __builtin__.DB not found (./Modules/_bsddb.c) __builtin__.DBCursor not found (./Modules/_bsddb.c) __builtin__.DBEnv not found (./Modules/_bsddb.c) __builtin__.DBTxn not found (./Modules/_bsddb.c) __builtin__.DBLock not found (./Modules/_bsddb.c) sha.SHA not found (./Modules/shamodule.c) module sv not installed (./Modules/svmodule.c) itertools._grouper not found (./Modules/itertoolsmodule.c) itertools.tee_dataobject not found (./Modules/itertoolsmodule.c) rotor.rotor not found (./Modules/rotormodule.c) module cl not installed (./Modules/clmodule.c) _sre.SRE_Pattern not found (./Modules/_sre.c) _sre.SRE_Match not found (./Modules/_sre.c) _sre.SRE_Scanner not found (./Modules/_sre.c) socket.SSL not found (./Modules/_ssl.c) _curses.curses window not found (./Modules/_cursesmodule.c) parser.st not found (./Modules/parsermodule.c) cStringIO.StringO not found (./Modules/cStringIO.c) cStringIO.StringI not found (./Modules/cStringIO.c) module sunaudiodev not installed (./Modules/sunaudiodev.c) module dbm not installed (./Modules/dbmmodule.c) dl.dl not found (./Modules/dlmodule.c) module fm not installed (./Modules/fmmodule.c) regex.regex not found (./Modules/regexmodule.c) pyexpat.xmlparser not found (./Modules/pyexpat.c) __builtin__.MultibyteCodec not found (./Modules/cjkcodecs/multibytecodec.c) __builtin__.MultibyteStreamReader not found (./Modules/cjkcodecs/multibytecodec.c) __builtin__.MultibyteStreamWriter not found (./Modules/cjkcodecs/multibytecodec.c) bsddb.bsddb not found (./Modules/bsddbmodule.c) module cd not installed (./Modules/cdmodule.c) cPickle.Pdata not found (./Modules/cPickle.c) module al not installed (./Modules/almodule.c) __builtin__.deque_iterator not found (./Modules/collectionsmodule.c) __builtin__.deque_reverse_iterator not found (./Modules/collectionsmodule.c) thread.lock not found (./Modules/threadmodule.c) zlib.Compress not found (./Modules/zlibmodule.c) zlib.Decompress not found (./Modules/zlibmodule.c) gdbm.gdbm not found (./Modules/gdbmmodule.c) ossaudiodev.oss_audio_device not found (./Modules/ossaudiodev.c) ossaudiodev.oss_mixer_device not found (./Modules/ossaudiodev.c) From barry at python.org Fri Aug 13 00:16:10 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 13 00:15:56 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/sandbox/string/string __init__.py, 1.2, 1.3 template.py, 1.1, 1.2 In-Reply-To: References: Message-ID: <1092348970.8679.120.camel@localhost> On Thu, 2004-08-12 at 17:20, rhettinger@users.sourceforge.net wrote: > Update of /cvsroot/python/python/nondist/sandbox/string/string > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7407 > > Modified Files: > __init__.py template.py > Log Message: > Capitalized the class names. > Fixed a broken example > Cleaned-up the namespace. > Added __slots__ for compactness. > Doctest the example. Thanks! pining-for-my-page-breaks-ly y'rs, -Barry :) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/454a8c6f/attachment.pgp From mike at skew.org Fri Aug 13 00:26:23 2004 From: mike at skew.org (Mike Brown) Date: Fri Aug 13 00:26:21 2004 Subject: [Python-Dev] Could getpass use streams other than stdin? Message-ID: <200408122226.i7CMQNvQ017619@chilled.skew.org> On posix, it would be useful if I could have getpass.getpass() write the prompt to stderr or to an arbitrary stream (file-like object), rather than always writing to stdin. In my use case, I need to send the results of a command-line script to stdout after the user authenticates, and the user may be redirecting stdout, so I don't want the password prompt to pollute the output. If I could have the "Password: " show up in stderr, that would be ideal. I don't think this would be terribly difficult to implement an API like def getpass(prompt='', stream=None) if stream is None: stream = sys.stdin ... but I'm unsure of the nuances of termios and arbitrary streams. Would it be better to just make the choice be between stdin and stderr? -- in which case the API would be something like def getpass(prompt='', use_stderr=False) Or is the whole exercise a waste of time? Thanks From mike at skew.org Fri Aug 13 00:35:00 2004 From: mike at skew.org (Mike Brown) Date: Fri Aug 13 00:34:56 2004 Subject: [Python-Dev] Could getpass use streams other than stdin? In-Reply-To: <200408122226.i7CMQNvQ017619@chilled.skew.org> "from Mike Brown at Aug 12, 2004 04:26:23 pm" Message-ID: <200408122235.i7CMZ0Eg017699@chilled.skew.org> > def getpass(prompt='', stream=None) > if stream is None: > stream = sys.stdin > ... Of course I meant sys.stdout there. From arigo at tunes.org Fri Aug 13 00:59:41 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri Aug 13 01:04:21 2004 Subject: [Python-Dev] Third Bug Day outcome In-Reply-To: <20040809213820.GC11199@rogue.amk.ca> References: <20040809213820.GC11199@rogue.amk.ca> Message-ID: <20040812225941.GA24248@vicky.ecs.soton.ac.uk> Hi, On Mon, Aug 09, 2004 at 05:38:20PM -0400, A.M. Kuchling wrote: > During the day, Armin Rigo wrote a nifty IRC bot that takes an SF > bug/patch ID and returns the title; taking an example from the transcript: > > 20:53:56 Looking at #777659. > 20:53:57 * sf_number bug 777659 - Uninitialized variable used in Tools/faqwiz/faqwiz.py Source code is at http://codespeak.net/svn/user/arigo/pypy-hack/ , files pysf*.py. Armin From tdelaney at avaya.com Fri Aug 13 01:20:43 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Fri Aug 13 01:22:04 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF460F@au3010avexu1.global.avaya.com> Gareth McCaughan wrote: > I don't think this is a good reason for rejecting "integer" > as the name of the common supertype of int and long. You'd use > isinstance(x,integer) to check whether x is an integer of > built-in type, just as you currently use isinstance(x,float) > to check whether x is a floating-point number of built-in > type. I see no reason why a cumbersome name is much advantage. All this brings to mind - is there actually a good reason to need a base type? Why not just define baseinteger as: baseinteger = int, long if the only reason for it is to use isinstance? Tim Delaney From tdelaney at avaya.com Fri Aug 13 01:25:36 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Fri Aug 13 01:27:00 2004 Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF4617@au3010avexu1.global.avaya.com> Barry Warsaw wrote: > On Thu, 2004-08-12 at 15:14, Skip Montanaro wrote: > >> I think '*' is syntactically available for this use. > > That was one of the characters I tried back when I was looking for @ > alternatives. For some reason I can't explain, this one just rubs me > the wrong way. > > i'll-let-tim-psychoanalyze-the-reasons-why-ly y'rs, Well, not the Tim you were thinking of, but ... I think the reason * doesn't work is that it is partially superscripted in most fonts. For example, compare: *deco @deco -deco @ and - are vertically centred with the text. However, * is positioned slightly above the text. Tim Delaney From guido at python.org Fri Aug 13 01:30:16 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 13 01:30:22 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Fri, 13 Aug 2004 09:20:43 +1000." <338366A6D2E2CA4C9DAEAE652E12A1DE01CF460F@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF460F@au3010avexu1.global.avaya.com> Message-ID: <200408122330.i7CNUGG31292@guido.python.org> > All this brings to mind - is there actually a good reason to need a base > type? Why not just define baseinteger as: > > baseinteger = int, long > > if the only reason for it is to use isinstance? So that an extension author *could* write an int-like type deriving from it? --Guido van Rossum (home page: http://www.python.org/~guido/) From tdelaney at avaya.com Fri Aug 13 01:33:41 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Fri Aug 13 01:34:49 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF461C@au3010avexu1.global.avaya.com> guido@python.org wrote: >> All this brings to mind - is there actually a good reason to need a >> base type? Why not just define baseinteger as: >> >> baseinteger = int, long >> >> if the only reason for it is to use isinstance? > > So that an extension author *could* write an int-like type deriving > from it? Fair 'nuff. Tim Delaney From greg at cosc.canterbury.ac.nz Fri Aug 13 03:21:22 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 13 03:21:29 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408121420.i7CEKdo30140@guido.python.org> Message-ID: <200408130121.i7D1LMqZ013619@cosc353.cosc.canterbury.ac.nz> Guido: > > Perhaps the constructor for a byte array could accept a string > > argument as long as it contained only ascii characters? > > > > h.putrequest(bytes('GET'), selector) > > Yeah, but that's what Martin called depending on the default encoding. I don't see anything wrong with that. It would be a fixed default, defined by the language, not something site-dependent that could shift under you. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From mkc at mathdogs.com Fri Aug 13 03:43:01 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Fri Aug 13 03:43:06 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> <873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp> <411B4A0A.1030201@heneryd.com> Message-ID: Erik Heneryd writes: > Well, I guess that if you want structmatch into the stdlib you'll have to show > that it's better than it's alternatives. Including those parser packages. As a practical matter, this may well be the case. I look at it kind of like 'structmatch' is to (a grammar parser) as 'sed' is to (a full scripting language) That is, it has its niche, but it certainly not as general or powerful as a full parsing package. I'm not sure, either, exactly how to show that it's better. That said, I use sed all the time, not because it's a better than full scripting languages, but because it nicely fits the problem I'm addressing better. > You'd still have to do a real implementation. If it can't be done without > rewriting a whole lot of code, that would be a problem. I agree. Being busy and lazy, I'm trying to get a bead first on whether this would be a wasted effort. > Hmm... think this is the wrong approach. Your PEP is not just about > "structured matching", it tries to deal with a couple of issues and I think it > would be better to address them separately, one by one: > > * Parsing/scanning - this is mostly what's been discussed so far... > > * Capturing repeated groups - IMO nice-to-have (tm) but not something I would > lose sleep over. Hard to do. > > * Partial matches - would be great for debugging more complex regexes. Why not > a general re.RAISE flag raising an exception on failure? This is true. For me, the second is fundamental--it's why I'm bothering with this. The third is a useful add-on, and as you suggest could probably be added orthogonally to several of the existing methods. The first I'm not sure about. I don't think re.structmatch does scanning--that's not really the problem it tries to solve. As for "parsing", I guess it depends on what you mean by that. Certainly it would be possible to address the "repeated groups" point without the whole structured return value thing, but I'm not seeing what would be better. Mike From greg at cosc.canterbury.ac.nz Fri Aug 13 03:57:45 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 13 03:57:49 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <200408122330.i7CNUGG31292@guido.python.org> Message-ID: <200408130157.i7D1vjoT013689@cosc353.cosc.canterbury.ac.nz> Guido: > > if the only reason for it is to use isinstance? > > So that an extension author *could* write an int-like type deriving > from it? But didn't you just say that people shouldn't be deriving their own int-like types from baseinteger? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido at python.org Fri Aug 13 04:35:24 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 13 04:35:34 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: Your message of "Fri, 13 Aug 2004 13:57:45 +1200." <200408130157.i7D1vjoT013689@cosc353.cosc.canterbury.ac.nz> References: <200408130157.i7D1vjoT013689@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408130235.i7D2ZOE31624@guido.python.org> > Guido: > > > > if the only reason for it is to use isinstance? > > > > So that an extension author *could* write an int-like type deriving > > from it? > > But didn't you just say that people shouldn't be > deriving their own int-like types from baseinteger? Indeed, in general they shouldn't. But for specialized purposes it might be needed (that's why I emphasized *could*). Unfortunately, for the purpose of having one's ersatz integers accepted as the real thing by the Python core, you pretty much have to inherit from int. So my only reason to make baseinteger a class rather than a tuple is consistency with basestring. --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Fri Aug 13 04:46:10 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 13 04:46:15 2004 Subject: [Python-Dev] Re: re.split on empty patterns In-Reply-To: References: <20040807145142.GB2529@rogue.amk.ca> Message-ID: <411C2B72.4090809@ocf.berkeley.edu> Mike Coleman wrote: [SNIP] > # alternative 2: > re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', 'bbczz'] ^ > re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz'] > > # alternative 3: > re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', '', 'bbczz'] ^ > re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz'] > I take it the first 'b' in both of the first examples for each alternative were supposed to be 'a'? And as for which version, I actually like Mike's version more than the one AMK and Tim like. The reason is that the '' in the middle of the example in question in the OP tells you where the split would have occurred had split0 (I like that or 'split_empty') not been turned on. That way there is no real loss of data between the two, but a gain with the new feature being used. -Brett From klm at zope.com Fri Aug 13 00:28:57 2004 From: klm at zope.com (Ken Manheimer) Date: Fri Aug 13 06:00:31 2004 Subject: [Python-Dev] Re: Call for defense of @decorators References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz> <1092108578.18208.10.camel@localhost> <1f7befae040809210679c2d4a0@mail.gmail.com> Message-ID: <87n0109eja.fsf@smtp.zope.com> Tim Peters writes: > [Barry Warsaw] > > Actually, that's an interesting case study. Myself and others advocated > > for a keyword other than 'def' for introducing generators. That wasn't > > how things turned out and in hindsight I think Guido made the right > > decision. I'm confident the same thing will happen with decorators. > > Ya, but it's sure taking Guido long enough to figure that out in this case! > > def staticmethod: > def author(name='Barry'): > def returns([int]): > def method(self): > return 42 Oh my. I'm in the unfortunate position of liking this more than any of the other options i've seen - while strongly suspecting you were not serious, and would think a fool anyone that would consider it for a moment. Ah well. The truth is out. The only adjustment i would make is to not include the ":" on the modifier lines: def staticmethod def author(name='yarrB') def returns([int]) def plugh(self): return 42 > a-keyword-for-all-seasons-ly y'rs - tim Could be! Ken klm@zope.com From mark at prothon.org Thu Aug 12 23:20:26 2004 From: mark at prothon.org (Mark Hahn) Date: Fri Aug 13 06:26:09 2004 Subject: [Python-Dev] Re: Prothon on CPython intrepreter? [PROTHON] References: <000a01c47f0b$b3cf4690$0b01a8c0@mark> Message-ID: On Tue, 10 Aug 2004 11:56:05 -0700, Mark Hahn wrote: > This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly > ease my wild ideas for the interpreter into Prothon. It will also make > CPython a multiple language VM for the first time. I've gotten a cold reception to this idea of implementing Prothon on the CPython interpreter. Not only did Christian Tismer (and he should know) and others warn me of the technnical challenge I was signing up for, but there wasn't exactly a groundswell of enthusiasm for the idea overall. So forget I bothered you about this crazy idea. I'll be sticking with my public announcement about the first implementation of Prothon being on .Net and Mono. The native CProthon interpreter will be next in line in 2005. I will of course always be interested in any and all ideas for alternatives to .Net for getting libraries for Prothon. Besides CPython, I have considered Parrot, Mozilla, PyPy, and Java and ruled them out for alternatives to .Net for the first implementation. From ncoghlan at iinet.net.au Fri Aug 13 07:50:29 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Aug 13 07:50:38 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <87n0109eja.fsf@smtp.zope.com> References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz> <1092108578.18208.10.camel@localhost> <1f7befae040809210679c2d4a0@mail.gmail.com> <87n0109eja.fsf@smtp.zope.com> Message-ID: <411C56A5.8070404@iinet.net.au> Ken Manheimer wrote: > Oh my. I'm in the unfortunate position of liking this more than any of the > other options i've seen - while strongly suspecting you were not serious, > and would think a fool anyone that would consider it for a moment. Ah > well. The truth is out. > > The only adjustment i would make is to not include the ":" on the modifier > lines: > > def staticmethod > def author(name='yarrB') > def returns([int]) > def plugh(self): > return 42 Did you see the 'predef' proposal I posted the other day? It seemed perfect to me, but given the staggering lack of response to the post, I suspect others were rather underwhelmed. The above would become: predef staticmethod predef author(name='yarrB') predef returns([int]) def plugh(self): return 42 To me, 'predef' seems to shout "Look, I'm telling you something about the 'def' that you are about to encounter in a line or two." Ah well, guess I'll go back to being a lurker on the deco issue. . . Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From fredrik at pythonware.com Fri Aug 13 08:29:58 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri Aug 13 08:28:18 2004 Subject: [Python-Dev] Re: Could getpass use streams other than stdin? References: <200408122226.i7CMQNvQ017619@chilled.skew.org> Message-ID: Mike Brown wrote: > In my use case, I need to send the results of a command-line script to stdout > after the user authenticates, and the user may be redirecting stdout, so I > don't want the password prompt to pollute the output. If I could have the > "Password: " show up in stderr, that would be ideal. you can redirect sys.stdin/sys.stdout before you call getpass: try: old_stdout = sys.stdout sys.stdout = sys.stderr password = getpass.getpass(prompt) finally: sys.stdout = old_stdout From anthony at interlink.com.au Fri Aug 13 10:15:07 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 13 10:15:39 2004 Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence In-Reply-To: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au> Message-ID: <411C788B.8030105@interlink.com.au> Steven Kah Hien Wong wrote: > class CommonBase: > x = 0 > > class ChildOne(CommonBase): > None > > class ChildTwo(CommonBase): > None > > class ChildOneAndTwo(ChildOne, ChildTwo): > def run(self): > ChildOne.x = 1 > print "one =", ChildOne.x > print "two =", ChildTwo.x > > ChildOneAndTwo().run() > > And the output was: > > $ python multi.py > one = 1 > two = 0 > > According to the documentation, I thought it should be: > > one = 1 > two = 1 Nope. In the code, you set a new attribute 'x' on the ChildOne class. Change the end of your code to ChildOneAndTwo().run() print "CommonBase.x", CommonBase.x print "ChildOne.x", ChildOne.x print "ChildTwo.x", ChildTwo.x Anthony From mal at egenix.com Fri Aug 13 11:11:18 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 13 11:11:31 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16667.54746.535997.120878@montanaro.dyndns.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <16667.32457.28553.468008@montanaro.dyndns.org> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> Message-ID: <411C85B6.2060104@egenix.com> Skip Montanaro wrote: > >> That's why I think a special literal is necessary. There'd be no > >> unicode foolishness involved. ;-) They'd just be raw uninterpreted > >> bytes. > > Martin> But you'd spell them b"GET", no? If so, which numeric value has > Martin> "G"? > > Good point... I don't think I understand the example... What's binary about 'GET' ? Why would you want to put non-ASCII into a binary literal definition ? If we switch the binding of 'yyy' to mean unicode('yyy') some day, why can't we just continue to use the existing implementation for 8-bit strings for b'xxx' (the current implementation is already doing the right thing, meaning that it is 8-bit safe regardeless of the source code encoding) ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ncoghlan at iinet.net.au Fri Aug 13 12:39:26 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Aug 13 12:39:33 2004 Subject: [Python-Dev] 'commands' module on Win32 Message-ID: <411C9A5E.3070109@iinet.net.au> Bug #1008275 refers to attempting to run commands.getstatusoutput() on win32, and the fact that it breaks. The reason it breaks is because it expects a posix shell, and this is noted in the documentation for the module (or rather, the module is noted as only supported on Unix). commands.getstatusoutput() and commands.getoutput() can easily be fixed to run properly on Windows (using "if os.name == 'posix'" to select the correct formatting for the shell command). Only the first one actually needs to be fixed, since getoutput() just returns the second element of the tuple returned by getstatusoutput() The dirty holdout is commands.getstatus(). This function does *not* simply return the first element of the tuple returned by commands.getstatusoutput() as you might expect. Instead of returning the status code resulting from executing its argument as a shell command, it insteads calls 'ls -ld ' to get the file information. (And this is a function in the commands module exactly why?) This actually works for me, but only because I have MSYS & MINGW installed :) Anyway, getstatusoutput() and getoutput() seem like very handy ways to make a command line call and get its results. I'd never seen them before because they're squirelled away as Unix specific modules. Are there any objections to changing this module so that only getstatus() is marked as Unix specific? I ask, because I don't want to start work on a documentation patch if the idea is a non-starter (the code & test changes are already done, butI think the hardest part of the exercise will be moving the documentation out of the 'Unix-specific' part of the doc tree). Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From mwh at python.net Fri Aug 13 13:05:20 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 13 13:05:22 2004 Subject: [Python-Dev] #ifdeffery In-Reply-To: <007301c480b1$0867bec0$5229c797@oemcomputer> (Raymond Hettinger's message of "Thu, 12 Aug 2004 17:12:10 -0400") References: <007301c480b1$0867bec0$5229c797@oemcomputer> Message-ID: <2mwu03wb67.fsf@starship.python.net> "Raymond Hettinger" writes: > P.S. Side rant: Also, in the last few months, the #ifdefs have > multiplied. While I understand that some things like time stamping are > worthwhile, the code is getting harder to read and maintain because of > all this b.s. Is it really getting worse? I think there are two qualitatively different sorts of #ifdef mess: feature ones, like the WITH_TSC I've recently been fiddling with recently and portability ones. The feature #ifdefs aren't (or shouldn't be) that much of a big deal. Ideally, they are documented in Misc/SpecialBuilds.txt and depending on whether they are defined or not, bits and pieces of code are or are not included. Platform #ifdefs, on the other hand, are pain in a tin. > At some point, someone changed a definition and I had to add an #ifdef > to import addrinfo.h for VC++ 6.0. I noticed the same had to be done > for Cygwin and there was a similar GCC entry. Each checkin was done by > a separate person fixing a crash in their own part of the universe. The > right answer would have likely been to refactor the original change that > mucked up everyone else's compile. Instead we ended up with a pile of > conditionals that will be hard to maintain going forward because the > surrounding code has become so stiff. Of course the problem here is that it's takes a lot of courage to make changes that affect systems you don't have access to. One of the goals of Include/pyport.h is that this localizes the mess: think of a sensibly named preprocessor symbol (PY_WE_NEED_ADDRINFO_H?) and put all the bullshit needed to work out whether to define it in one place. But you really want to get things like this right, especially if one of the affected platforms is something a little obscure. Break things on linux/x86 and you'll find out pretty quickly, but break them on Irix and it may take months and it won't be obvious which change is at fault. Still, I want at some point to do test builds on all the test drive machines at some point before 2.4 final, so any changes will hopefully get some amount of testing on these systems at least. Cheers, mwh -- (Unfortunately, while you get Tom Baker saying "then we were attacked by monsters", he doesn't flash and make "neeeeooww-sploot" noises.) -- Gareth Marlow, ucam.chat, from Owen Dunn's review of the year From jlgijsbers at planet.nl Fri Aug 13 14:54:30 2004 From: jlgijsbers at planet.nl (Johannes Gijsbers) Date: Fri Aug 13 14:53:14 2004 Subject: [Python-Dev] Re: 'commands' module on Win32 Message-ID: <20040813125429.GA22786@mail.planet.nl> Nick Coghlan wrote: > Are there any objections to changing this module so that only > getstatus() is marked as Unix specific? Not from me. > I ask, because I don't want to start work on a documentation patch if > the idea is a non-starter (the code & test changes are already done, > butI think the hardest part of the exercise will be moving the > documentation out of the 'Unix-specific' part of the doc tree). This is pretty easy actually. In fact, while I was checking whether it really was easy, I created a patch you could use. It's attached. Johannes -------------- next part -------------- cvs diff: Diffing . Index: lib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/lib.tex,v retrieving revision 1.229 diff -B -c -r1.229 lib.tex *** lib.tex 7 Aug 2004 16:53:59 -0000 1.229 --- lib.tex 13 Aug 2004 12:27:42 -0000 *************** *** 189,194 **** --- 189,195 ---- \input{libtarfile} \input{libreadline} \input{librlcompleter} + \input{libcommands} \input{libunix} % UNIX Specific Services \input{libposix} *************** *** 207,213 **** \input{libresource} \input{libnis} \input{libsyslog} - \input{libcommands} \input{libpdb} % The Python Debugger --- 208,213 ---- Index: libcommands.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcommands.tex,v retrieving revision 1.12 diff -B -c -r1.12 libcommands.tex *** libcommands.tex 22 Apr 1999 15:53:35 -0000 1.12 --- libcommands.tex 13 Aug 2004 12:27:42 -0000 *************** *** 2,8 **** Utilities for running commands} \declaremodule{standard}{commands} ! \platform{Unix} \modulesynopsis{Utility functions for running external commands.} \sectionauthor{Sue Williams}{sbw@provis.com} --- 2,8 ---- Utilities for running commands} \declaremodule{standard}{commands} ! \platform{Unix, Windows} \modulesynopsis{Utility functions for running external commands.} \sectionauthor{Sue Williams}{sbw@provis.com} *************** *** 33,39 **** \begin{funcdesc}{getstatus}{file} Return the output of \samp{ls -ld \var{file}} as a string. This function uses the \function{getoutput()} function, and properly ! escapes backslashes and dollar signs in the argument. \end{funcdesc} Example: --- 33,39 ---- \begin{funcdesc}{getstatus}{file} Return the output of \samp{ls -ld \var{file}} as a string. This function uses the \function{getoutput()} function, and properly ! escapes backslashes and dollar signs in the argument. Availability: \UNIX. \end{funcdesc} Example: From aahz at pythoncraft.com Fri Aug 13 16:10:42 2004 From: aahz at pythoncraft.com (Aahz) Date: Fri Aug 13 16:10:46 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <411C85B6.2060104@egenix.com> References: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <16667.32457.28553.468008@montanaro.dyndns.org> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> <411C85B6.2060104@egenix.com> Message-ID: <20040813141042.GB21742@panix.com> On Fri, Aug 13, 2004, M.-A. Lemburg wrote: > > What's binary about 'GET' ? It's an ASCII, human-readable representation of a set of bytes sent over a network interface to command a server. It could just as easily have been "\010\011\012", but it was selected for the convenience of English-speaking humans. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From dave at boost-consulting.com Fri Aug 13 16:11:21 2004 From: dave at boost-consulting.com (David Abrahams) Date: Fri Aug 13 16:12:04 2004 Subject: [Python-Dev] Re: Another approach to decorators. In-Reply-To: <200408122053.i7CKrRg30941@guido.python.org> (Guido van Rossum's message of "Thu, 12 Aug 2004 13:53:27 -0700") References: <20040811195454.9BD311E400A@bag.python.org> <20040811133711.5f1986b6.michel@dialnetwork.com> <411A898F.3060004@ocf.berkeley.edu> <20040811142354.661970e0.michel@dialnetwork.com> <20040811142802.0386bd60.michel@dialnetwork.com> <200408120426.i7C4QU829083@guido.python.org> <200408122053.i7CKrRg30941@guido.python.org> Message-ID: Guido van Rossum writes: >> > I think this is too far-fetched to consider as an alternative to the >> > humble @decorator. >> >> Maybe, but if you're still serious about improving support for >> domain-specific embedded languages in Python you ought to give the >> ideas another look. If a reasonable syntax for decorators falls out >> of a more general and useful mechanism, so much the better. > > How does > > decorate , ...: > > > improve support for domain-specific embedded languages? I was thinking -- as well as I could through yesterday's allergy attack -- more of the some-decorator: form, which allows people to create pseudo-statements of various kinds. Probably Bob I. is right that it doesn't help as much as it might if is executed before some-decorator is invoked. That said, maybe it's possible to pass the *code* for the block to some-decorator in this scenario. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From guido at python.org Fri Aug 13 16:30:42 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 13 16:30:47 2004 Subject: [Python-Dev] 'commands' module on Win32 In-Reply-To: Your message of "Fri, 13 Aug 2004 03:39:26 PDT." <411C9A5E.3070109@iinet.net.au> References: <411C9A5E.3070109@iinet.net.au> Message-ID: <200408131430.i7DEUgK32685@guido.python.org> > Bug #1008275 refers to attempting to run commands.getstatusoutput() on > win32, and the fact that it breaks. Hm... That module's functionality is such a small improvement over raw os.popen, it would never have been admitted into the standard library these days. I'd rather see this added as a higher-level API to one of the process management modules that are floating around, one of which will eventually land in the library, at which point we can deprecate commands.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Fri Aug 13 16:52:10 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 13 16:52:20 2004 Subject: [Python-Dev] 'commands' module on Win32 In-Reply-To: <411C9A5E.3070109@iinet.net.au> References: <411C9A5E.3070109@iinet.net.au> Message-ID: <1f7befae040813075216398c46@mail.gmail.com> [Nick Coghlan] > ... > Are there any objections to changing this module so that only > getstatus() is marked as Unix specific? -0. It works on Unix. On Windows you have to distinguish among command.com systems (95/98/ME), cmd.exe systems (NT/2K/XP), and systems where the user installed their own COMSPEC shell. The command.com systems can't redirect stderr, and always return a status of 0 (command.com's own exit code, which has nothing to do with the exit code of the command being run). "It should work" on cmd.exe systems. Can't say anything definitive about use-my-own-shell systems. IOW, telling the truth would complicate the docs, and the result wouldn't be portable across Windows boxes anyway. So I'd rather leave commands.py in peace, short, simple, obvious, but limited to Unix. It's not a suitable basis for a *good* solution to x-platform command-line work (Tcl's "exec" subsystem is an example of a good solution, but it's a lot of work). From skip at pobox.com Fri Aug 13 17:44:17 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 13 17:44:23 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: References: Message-ID: <16668.57809.362313.36355@montanaro.dyndns.org> (posting to python-dev separately from c.l.py so the thread can live or die on its own in both places) I responded on c.l.py to a couple proposals: Steven> decorate: Steven> grammarrule('statement : expression') Steven> versioninfo("Added in 2.4") Steven> deprecated Steven> typeinfo(None) Steven> def p_statement_expr(self, p): Steven> print p[1] Nick> as: Nick> staticmethod Nick> grammarrule('statement : expression') Nick> version("Added in 2.4") Nick> deprecatedmethod Nick> type_(None) Nick> def p_statement_expr(self, p): Nick> print p[1] with def p_statement_expr: staticmethod grammarrule('statement : expression') version("Added in 2.4") deprecatedmethod type_(None) decorate (self, p): """docstring here""" print p[1] It seems different enough from other solutions that I thought it worth tossing out. I didn't see it mentioned in the PythonDecorators moin page. Read it something like "define a function named p_statement_expr using a bunch of functions to decorate the basic function". It solves a couple problems: 1. "def" introduces the function definition instead of an arbitrary number of @-expressions. 2. There is no extra indentation of the main body. 3. The name of the function is known early on. 4. "def"/"decorate" pair up visually much the same as "try"/"except" or "if"/"then", though they don't represent alternative blocks of code to be executed. On the minus side it introduces a vertical separation between the function name and parameter list and introduces a new keyword, "decorate". >From a parsing standpoint I think it will work. You'll see either a colon or a left paren after the function name to distinguish between the two types of function definition. I'm not sure if a token needs to be used to separate the various decorator functions or if requiring a newline and indentation is sufficient. Skip From rnd at onego.ru Fri Aug 13 18:24:21 2004 From: rnd at onego.ru (Roman Suzi) Date: Fri Aug 13 18:23:30 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <16668.57809.362313.36355@montanaro.dyndns.org> References: <16668.57809.362313.36355@montanaro.dyndns.org> Message-ID: On Fri, 13 Aug 2004, Skip Montanaro wrote: >with > > def p_statement_expr: > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > decorate (self, p): > """docstring here""" > print p[1] Bingo! Just replace decorate with "from" and the nice syntax is found: def f: staticmethod grammarrule('statement : expression') version("Added in 2.4") deprecatedmethod type_(None) from self, p: """docstring here""" print p[1] + doesn't surprise Python programmer, because it is like try-except, etc + reads a natural language (with implicit "with" after "f") + doesn't require any new keywords or symbols and "prefix" operators + is explicit about transformation + no additional indentation - grepping for defs now require more effort, so probably second name should be allowed after "from" or exactly this. As a variant, docstring could be moved to the upper part. I'd also liked being able to write: def f: staticmethod; grammarrule('statement : expression') version("Added in 2.4"); deprecatedmethod; type_(None) from x, y: pass #... and: def f: staticmethod from x, y: return x+y >It seems different enough from other solutions that I thought it worth >tossing out. I didn't see it mentioned in the PythonDecorators moin page. > >Read it something like "define a function named p_statement_expr using a >bunch of functions to decorate the basic function". > >It solves a couple problems: > >1. "def" introduces the function definition instead of an arbitrary number > of @-expressions. > >2. There is no extra indentation of the main body. > >3. The name of the function is known early on. > >4. "def"/"decorate" pair up visually much the same as "try"/"except" or > "if"/"then", though they don't represent alternative blocks of code to be > executed. > >On the minus side it introduces a vertical separation between the function >name and parameter list and introduces a new keyword, "decorate". > >>From a parsing standpoint I think it will work. You'll see either a colon >or a left paren after the function name to distinguish between the two types >of function definition. I'm not sure if a token needs to be used to >separate the various decorator functions or if requiring a newline and >indentation is sufficient. > >Skip >_______________________________________________ >Python-Dev mailing list >Python-Dev@python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: http://mail.python.org/mailman/options/python-dev/rnd%40onego.ru > Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From jcarlson at uci.edu Fri Aug 13 18:54:14 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri Aug 13 19:01:13 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: References: <16668.57809.362313.36355@montanaro.dyndns.org> Message-ID: <20040813093137.63C0.JCARLSON@uci.edu> I've kept my nose out of the decorator discussion, but I thought I would give my opinion on this one... > On Fri, 13 Aug 2004, Skip Montanaro wrote: > > >with > > > > def p_statement_expr: > > staticmethod > > grammarrule('statement : expression') > > version("Added in 2.4") > > deprecatedmethod > > type_(None) > > decorate (self, p): > > """docstring here""" > > print p[1] > > Bingo! Oh god no. > Just replace decorate with "from" and the nice syntax is found: > > def f: > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > from self, p: > """docstring here""" > print p[1] Gah, the horror. > + doesn't surprise Python programmer, because it is like try-except, etc It surprises anyone who expects the function signature immediately after the name. > + reads a natural language (with implicit "with" after "f") But it is so radically different from current definitions that it breaks current understanding about the language. It also requires people to learn two significantly different semantics for defining a function. The good proposals (IMO) keep the function name and signature unchanged. So let us read this "natural language" (my interpretation of the syntax given above): define f as a staticmethod... from the arguments self and p with function body... Not bad, though I don't like the visual aesthetics. > + doesn't require any new keywords or symbols and "prefix" operators Prefix operators don't seem that bad to me, nor to Guido (who offered them in the first place), nor to the dozen+ others who have stated that "@, !, %, *, -, +, | or & is cool". As for keywords, there has been an offered solution; the use of a future import. Similar to the way yield was used in 2.3. From is an import semantic, not function definition. Overloading 'from' does remove the necessity of a new keyword, but so does the use of the symbols previously offered. > + is explicit about transformation > + no additional indentation Ahh, but you indent the block of decorators before the function definition must /always/ be read in order to figure out . While this has been done before (try, except, finally) > - grepping for defs now require more effort, so probably second name should > be allowed after "from" or exactly this. > > As a variant, docstring could be moved to the upper part. > > I'd also liked being able to write: > > def f: > staticmethod; grammarrule('statement : expression') > version("Added in 2.4"); deprecatedmethod; type_(None) > from x, y: > pass #... > > and: > > def f: staticmethod > from x, y: return x+y Yikes. If there were voting, I'd like to put my vote in for: @deco def fun(arg1): #body or def fun(arg1): @deco #body ...with a preference towards the second one, and a slight prefence toward some other symbol than @ (for compatibility reasons). |, &, ! are all aesthetically pleasing to me, in that order. Reading the second one: define fun with arguments arg decorated with deco and body... To me it reads better than the three syntaxes offered by Skip and Roman this morning. - Josiah From rnd at onego.ru Fri Aug 13 19:04:34 2004 From: rnd at onego.ru (Roman Suzi) Date: Fri Aug 13 19:03:42 2004 Subject: [Python-Dev] Re: Call for defense of @decorators In-Reply-To: <411C56A5.8070404@iinet.net.au> References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz> <1092108578.18208.10.camel@localhost> <1f7befae040809210679c2d4a0@mail.gmail.com> <87n0109eja.fsf@smtp.zope.com> <411C56A5.8070404@iinet.net.au> Message-ID: On Thu, 12 Aug 2004, Nick Coghlan wrote: > predef staticmethod > predef author(name='yarrB') > predef returns([int]) > def plugh(self): > return 42 > > >To me, 'predef' seems to shout "Look, I'm telling you something about >the 'def' that you are about to encounter in a line or two." This is too much repetition to please programmers ;) They like to write things ONCE. This is exactly the reason why all this decorator hell raised: def X: ... X = decor(X) uses X three times... >Ah well, guess I'll go back to being a lurker on the deco issue. . . > >Cheers, >Nick. > > Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From rnd at onego.ru Fri Aug 13 19:27:23 2004 From: rnd at onego.ru (Roman Suzi) Date: Fri Aug 13 19:26:33 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <20040813093137.63C0.JCARLSON@uci.edu> References: <16668.57809.362313.36355@montanaro.dyndns.org> <20040813093137.63C0.JCARLSON@uci.edu> Message-ID: On Fri, 13 Aug 2004, Josiah Carlson wrote: >I've kept my nose out of the decorator discussion, but I thought I would >give my opinion on this one... >> Bingo! >Oh god no. ;) >> Just replace decorate with "from" and the nice syntax is found: >> >> def f: >> staticmethod >> grammarrule('statement : expression') >> version("Added in 2.4") >> deprecatedmethod >> type_(None) >> from self, p: >> """docstring here""" >> print p[1] > >Gah, the horror. > >> + doesn't surprise Python programmer, because it is like try-except, etc > >It surprises anyone who expects the function signature immediately after >the name. I do disagree. In my suggestion function is being defined in natural order hich is illustratable by use of lambda: f = \ decor( \ lambda arg: \ return_val) Decorators could be destructive and change function completely, so it's pitiful signature doesn't mean much. >> + reads a natural language (with implicit "with" after "f") > >But it is so radically different from current definitions that it breaks >current understanding about the language. Why? IMHO @s are much more radical. >It also requires people to >learn two significantly different semantics for defining a function. No more "significantly different" than learning import m and from m import something >The good proposals (IMO) keep the function name and signature unchanged. > >So let us read this "natural language" (my interpretation of the syntax >given above): > >define f as a staticmethod... from the arguments self and p with >function body... Not bad, though I don't like the visual aesthetics. Ok. Now I see our disagrement. I am thinking in terms of applying decorators to make some other function FROM a given function while you are reasoning about "adding" attributes to the defined function. >> + doesn't require any new keywords or symbols and "prefix" operators > >Prefix operators don't seem that bad to me, nor to Guido (who offered >them in the first place), nor to the dozen+ others who have stated that >"@, !, %, *, -, +, | or & is cool". (ooops! I mean "prefix statements" - statements sticking to other statements, modifying their semantics) >As for keywords, there has been an offered solution; the use of a future >import. Similar to the way yield was used in 2.3. IMO, decorators deserve no keywords... >>From is an import semantic, not function definition. Overloading 'from' >does remove the necessity of a new keyword, but so does the use of the >symbols previously offered. I see no problem overloading preposition like "from", "as", "to", etc, if it adds natural readability to language statements. >> + is explicit about transformation >> + no additional indentation > >Ahh, but you indent the block of decorators before the function >definition must /always/ be read in order to figure out . While this has >been done before (try, except, finally) I mean "no additional" level, like other group decoration proposals. >> - grepping for defs now require more effort, so probably second name should >> be allowed after "from" or exactly this. However, I am not sure Guido will change his mind and replace '@' with something else. Probably, Python became too boring without fresh syntax solutions like pies. If so, I'd added a possibility to use ";" to delimit decorators: @ decor1; decor2 def f(x): ... Or: @decor1; @decor2 def f(x): ... May double pie mean "repeat last time used pie": @staticmethod def m(self, a): ... @@ def n(self, a): ... Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From bac at OCF.Berkeley.EDU Fri Aug 13 19:26:18 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 13 19:26:49 2004 Subject: [Python-Dev] Re: Prothon on CPython intrepreter? [PROTHON] In-Reply-To: References: <000a01c47f0b$b3cf4690$0b01a8c0@mark> Message-ID: <411CF9BA.1070102@ocf.berkeley.edu> Mark Hahn wrote: > On Tue, 10 Aug 2004 11:56:05 -0700, Mark Hahn wrote: > > >>This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly >>ease my wild ideas for the interpreter into Prothon. It will also make >>CPython a multiple language VM for the first time. > > > I've gotten a cold reception to this idea of implementing Prothon on the > CPython interpreter. Not only did Christian Tismer (and he should know) > and others warn me of the technnical challenge I was signing up for, but > there wasn't exactly a groundswell of enthusiasm for the idea overall. > Sorry to hear about this. Hopefully you can end up with a nice enough AST for Prothon that targetting other VMs won't be any more difficult than just writing walker for the AST. -Brett From ws-news at gmx.at Fri Aug 13 19:52:48 2004 From: ws-news at gmx.at (Werner Schiendl) Date: Fri Aug 13 19:52:59 2004 Subject: [Python-Dev] Re: def ... decorate References: <16668.57809.362313.36355@montanaro.dyndns.org> Message-ID: [Skip] > > def p_statement_expr: > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > decorate (self, p): > """docstring here""" > print p[1] > was the following suggested already? (don't think so): def p_statement_expr(self, p): staticmethod grammarrule('statement : expression') version("Added in 2.4") deprecatedmethod type_(None) body: """docstring here""" print p[1] This would eliminate the problem of the moved parameters, yet keep all the pros you listed. The docstring could allowed to be with the decorators so that all "meta-data" (from the perspective of a user) would be together. / Werner From gmccaughan at synaptics-uk.com Fri Aug 13 20:06:13 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Fri Aug 13 20:06:45 2004 Subject: [Python-Dev] Re: def ... decorate In-Reply-To: References: <16668.57809.362313.36355@montanaro.dyndns.org> Message-ID: <200408131906.13149.gmccaughan@synaptics-uk.com> On Friday 2004-08-13 18:52, Werner Schiendl wrote: > [Skip] > > > > def p_statement_expr: > > staticmethod > > grammarrule('statement : expression') > > version("Added in 2.4") > > deprecatedmethod > > type_(None) > > decorate (self, p): > > """docstring here""" > > print p[1] > > > > was the following suggested already? (don't think so): > > def p_statement_expr(self, p): > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > body: > """docstring here""" > print p[1] So you see "def name(args):" followed by what looks at first glance like a function body ... and then it turns out N lines later that it was really a set of decorators. Surely if this sort of two-suite approach is going to be used, there is only one sane way to do it (modulo choice of keyword): def p_statement_expr(self, p): """docstring goes here""" print p[1] decorated: staticmethod grammarrule('statement : expression') version("Added in 2.4") deprecatedmethod type_(None) which keeps the arguments with the function name, keeps the body right after the name and arguments, and puts the decoration after the body which corresponds with the order in which things actually happen (though not necessarily the best order for understanding the code). Another merit to "post-position" decoration (which applies here, where decoration is after the body, and also to the original after-arglist proposal) is that it's fairly obvious what order the decorators are applied in: the two criteria are order of appearance and order of proximity to the thing the decorators seem visually to be attached to, and those agree. For the @foo syntax they don't. I actually quite like this. It seems more Pythonic than the @foo proposal. Its obvious problem is that it involves something that looks at first glance like an ordinary suite of statements or expressions, but whose interpretation is substantially different. At least the @foo proposal avoids that. I'm +1 on the original []-after-def proposal (which is to be expected, since IIRC I was the first person to propose it), +0.5 on what I've suggested above, +0 on Guido's @foo, and a big hairy -1 on the weird distortions of "def" I'm replying to :-). -- g From python at rcn.com Fri Aug 13 20:11:06 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 13 20:11:22 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <20040813093137.63C0.JCARLSON@uci.edu> Message-ID: <004d01c48160$e7602d60$0bb0958d@oemcomputer> > I've kept my nose out of the decorator discussion, but I thought I would > give my opinion on this one... > > > > On Fri, 13 Aug 2004, Skip Montanaro wrote: > > > > >with > > > > > > def p_statement_expr: > > > staticmethod > > > grammarrule('statement : expression') > > > version("Added in 2.4") > > > deprecatedmethod > > > type_(None) > > > decorate (self, p): > > > """docstring here""" > > > print p[1] > > > > Bingo! > > Oh god no. > > > > Just replace decorate with "from" and the nice syntax is found: > > > > def f: > > staticmethod > > grammarrule('statement : expression') > > version("Added in 2.4") > > deprecatedmethod > > type_(None) > > from self, p: > > """docstring here""" > > print p[1] > > Gah, the horror. I think we should take maximum advantage of prior art. Other languages have proven the efficacy of backwards spellings. Likewise, the reST module has proven the utility of ASCII markup. dothemcitats:: elurrammarg ``statement : expression'':: dohtemdetacerped:: `````````````````` def f(arg): . . . Advantages: * Won't delude people into thinking they can understand a python program just by reading it. * Highly adaptable to electronic parsing while discouraging the adverse trend towards human readability. * Accomplishes the life saving core goal of moving decorators up a few lines while only incurring the minor cost of complicating the snot out of the language. * Unlikely to be confused with any existing python constructs. * Takes less effort to implement, document, and test than to read all of the decorator syntax emails to date. Alternative: A further application of prior art would be to hypergeneralize the solution to this problem and design a solution using XML. This takes maximum advantage of existing tools while providing a framework for all potential future proposals to add a second or third way to accomplish things we can already do now. citsacras-ly yours, Raymond From python at discworld.dyndns.org Fri Aug 13 20:27:51 2004 From: python at discworld.dyndns.org (Charles Cazabon) Date: Fri Aug 13 20:20:29 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <004d01c48160$e7602d60$0bb0958d@oemcomputer> References: <20040813093137.63C0.JCARLSON@uci.edu> <004d01c48160$e7602d60$0bb0958d@oemcomputer> Message-ID: <20040813182751.GB20947@discworld.dyndns.org> Raymond Hettinger wrote: > > I've kept my nose out of the decorator discussion, but I thought I would > > give my opinion on this one... [...] > Advantages: > > * Won't delude people into thinking they can understand a python program > just by reading it. :) I've lost track of all the decorator proposals. Anyone proposed something like this? def f { 'accepts' : (type1, type2), 'returns' : (int, str)} funcname (args): ? The decorator dictionary could span multiple lines, of course. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://www.qcc.ca/~charlesc/software/ ----------------------------------------------------------------------- From pje at telecommunity.com Fri Aug 13 20:26:14 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 13 20:26:34 2004 Subject: [Python-Dev] @-after-def-before-colon (was Re: def ... decorate) In-Reply-To: References: <16668.57809.362313.36355@montanaro.dyndns.org> Message-ID: <5.1.1.6.0.20040813140145.026ddec0@mail.telecommunity.com> At 07:52 PM 8/13/04 +0200, Werner Schiendl wrote: >was the following suggested already? (don't think so): > > def p_statement_expr(self, p): > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > body: > """docstring here""" > print p[1] hmmmm.... that's really not much different from: def p_statement_expr(self, p) [ staticmethod, grammarrule('statement : expression'), version("Added in 2.4"), returns(None) ]: """Docstring""" body is it? But then, how about *this*: def p_statement_expr(self, p) @staticmethod @grammarrule('statement : expression') @version("Added in 2.4") @returns(None) as: """Docstring""" body The decorators are much more visible with the @ signs. Also, we could possibly allow the docstring to appear in the def statement, rather than in the body, e.g.: def p_statement_expr(self, p) @staticmethod @grammarrule('statement : expression') @version("Added in 2.4") @returns(None) """Docstring""" as: body Some advantages: * Tools that just only look at 'def funcname(signature)' still work * Tools that only look for '@something' at the start of the line will work (e.g. I believe this means IPython will still work, dunno about Leo) * Decorators are much more visible than in the list-after-def forms, but are still visibly part of the function definition * You know the name/signature of the thing being defined before wading through decorators Disadvantages: * line continuation isn't standard; there ought to be brackets or parentheses, otherwise tokenizer will say there's an INDENT, and it's not standard to have an INDENT without a colon. * How would we define the syntax? Should it be something like this: classdef: 'class' NAME ['(' testlist ')'] [decorations] ':' suite funcdef: 'def' NAME parameters [decorations] ':' suite decorations: NEWLINE INDENT decorator* [docstring NEWLINE] DEDENT 'as' decorator: '@' test NEWLINE In other words, should indent and one-decorator-per-line be enforced? * Guido has argued that list-after-def is bad in the case of a long argument list, because of poor visibility. Does this approach fix that? From amk at amk.ca Fri Aug 13 20:51:31 2004 From: amk at amk.ca (A.M. Kuchling) Date: Fri Aug 13 20:52:08 2004 Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome) In-Reply-To: References: <20040809213820.GC11199@rogue.amk.ca> <2mllgn15cl.fsf@starship.python.net> Message-ID: <20040813185131.GB28569@rogue.amk.ca> On Wed, Aug 11, 2004 at 12:17:27PM +0000, Johannes Gijsbers wrote: > Consider this a request for checkin privileges. Seconded. --amk From jcarlson at uci.edu Fri Aug 13 21:02:22 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri Aug 13 21:09:14 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <20040813182751.GB20947@discworld.dyndns.org> References: <004d01c48160$e7602d60$0bb0958d@oemcomputer> <20040813182751.GB20947@discworld.dyndns.org> Message-ID: <20040813120041.63C6.JCARLSON@uci.edu> > I've lost track of all the decorator proposals. Anyone proposed something > like this? > > def f { 'accepts' : (type1, type2), 'returns' : (int, str)} funcname (args): > > ? The decorator dictionary could span multiple lines, of course. Basically yes. See Ka-Ping's listing of options here: http://mail.python.org/pipermail/python-dev/2004-March/043685.html - Josiah From jcarlson at uci.edu Fri Aug 13 21:06:35 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri Aug 13 21:13:13 2004 Subject: [Python-Dev] Re: def ... decorate In-Reply-To: <200408131906.13149.gmccaughan@synaptics-uk.com> References: <200408131906.13149.gmccaughan@synaptics-uk.com> Message-ID: <20040813120226.63C9.JCARLSON@uci.edu> On Fri, 13 Aug 2004 19:06:13 +0100 Gareth McCaughan wrote: > def p_statement_expr(self, p): > """docstring goes here""" > print p[1] > decorated: > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > > which keeps the arguments with the function name, > keeps the body right after the name and arguments, and > puts the decoration after the body which corresponds > with the order in which things actually happen (though > not necessarily the best order for understanding the > code). You seem to have missed the point of function decorations entirely. We already have what you offer in current Python syntax. The point was to move decorations to near/next to the function signature. Read the PEP: http://www.python.org/peps/pep-0318.html > I actually quite like this. It seems more Pythonic > than the @foo proposal. Its obvious problem is that > it involves something that looks at first glance like > an ordinary suite of statements or expressions, but > whose interpretation is substantially different. At > least the @foo proposal avoids that. What you like is what has existed with Python since the beginning. - Josiah From jcarlson at uci.edu Fri Aug 13 21:13:55 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri Aug 13 21:20:14 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: References: <20040813093137.63C0.JCARLSON@uci.edu> Message-ID: <20040813104200.63C3.JCARLSON@uci.edu> On Fri, 13 Aug 2004 21:27:23 +0400 (MSD) Roman Suzi wrote: > >> + doesn't surprise Python programmer, because it is like try-except, etc > > > >It surprises anyone who expects the function signature immediately after > >the name. > > I do disagree. In my suggestion function is being defined in natural order > hich is illustratable by use of lambda: > > > f = \ > decor( \ > lambda arg: \ > return_val) > > Decorators could be destructive and change function completely, so it's > pitiful signature doesn't mean much. That is funny, because in the 100,000+ lines of Python I've written over the years, function signatures are, in fact, important. Knowing a signature allows you to call a function, and I believe that any decorator that destroys a signature willingly should be considered broken. One could make the claim that function signatures are at least as important as the function body; if you don't know how to call a function, then its body is nearly worthless. Interestingly enough, decorators are important to PyObjC developers because they use decorators to specify signatures. As for the "natural order" of definitions, in Python, the natural order of definitions uses 'def', 'lambda' is considered by many (Guido included) to be a mistake. I believe your argument using lambda is poor. > >> + reads a natural language (with implicit "with" after "f") > > > >But it is so radically different from current definitions that it breaks > >current understanding about the language. > > Why? IMHO @s are much more radical. Seemingly in Guido's opinion, @s (or other symbols) were minimal. You can have a humble (or not so humble) opinion, but unless you can convince Guido and/or the community, it is not going to happen (I've been bitten by this myself). > >It also requires people to > >learn two significantly different semantics for defining a function. > > No more "significantly different" than learning > > import m > > and > > from m import something While the use of 'from' implies something has changed in imports, and it would imply something has changed in the decorator syntax, the use of from differs. Also, with your proposed decorator syntax, one inserts 'from' into the middle of the defintion of the function with decorators, basically making 'from' into noise. > >The good proposals (IMO) keep the function name and signature unchanged. > > > >So let us read this "natural language" (my interpretation of the syntax > >given above): > > > >define f as a staticmethod... from the arguments self and p with > >function body... Not bad, though I don't like the visual aesthetics. > > Ok. Now I see our disagrement. I am thinking in terms of > applying decorators to make some other function FROM > a given function while you are reasoning about "adding" attributes to the > defined function. No, our disagreement isn't about how we understand decorators. I never argue it is about adding attributes, as I have actually read the 1800+ postings on decorators in the last 8 months, where attributes are basically stated as being not arguable. Our disagreement is about what is a reasonable decorator syntax. I don't believe that anything that changes the look of a function signature to be a reasonable decorator syntax. > >> + doesn't require any new keywords or symbols and "prefix" operators > > > >Prefix operators don't seem that bad to me, nor to Guido (who offered > >them in the first place), nor to the dozen+ others who have stated that > >"@, !, %, *, -, +, | or & is cool". > > (ooops! I mean "prefix statements" - statements sticking to > other statements, modifying their semantics) What, like the following? decorate(memoize) def fun(arg): #body I think the fact that the above is actually implemented suggests that it makes sense for many people. Some have even stated "Just stick it in the standard library and call it good." If it didn't rely on implementation-specific CPython hooks, I believe that Guido would have preferred it to '@'. > >As for keywords, there has been an offered solution; the use of a future > >import. Similar to the way yield was used in 2.3. > > IMO, decorators deserve no keywords... I wouldn't say anything of the sort. I don't believe keywords are /necessary/ for decorators, but if a new keyword makes it click for Guido (very unlikely), then so be it. > >>From is an import semantic, not function definition. Overloading 'from' > >does remove the necessity of a new keyword, but so does the use of the > >symbols previously offered. > > I see no problem overloading preposition like "from", "as", "to", etc, > if it adds natural readability to language statements. Ahh, you seem to have missed that neither "as" nor "to" are keywords: >>> def as(arg): ... print "as", arg ... >>> as(1) as 1 >>> def to(arg): ... print "to", arg ... >>> to(1) to 1 Personally, I have no issue with keyword reuse, as long as they make sense. Considering the logical separation of the use of from during imports, and your use of from, I think it could confuse new users of the language. > >> + is explicit about transformation > >> + no additional indentation > > > >Ahh, but you indent the block of decorators before the function > >definition must /always/ be read in order to figure out . While this has > >been done before (try, except, finally) > > I mean "no additional" level, like other group decoration proposals. Oops, I didn't finish my argument. I know what you meant. In order to understand what a function does, one must read the decorators. While associated indented blocks have been done before (try, except, finally, else), one doesn't need to read the try block to understand that if an exception were to occur, the except block would be executed. In order to understand the semantics of a function definition, you need to read the decorators. Having an indented block previous to the function definition, which includes the decorators for the function below, only makes sense if the name is given before the decorators (the brain-stack argument). However, since the function signature is as important as any decorator, then the signature should be prominant. I believe your arguments are better focused toward a different syntax, namely: def fun(args): @deco #body I think it beats your offered syntax hands-down due to its conciseness and lack of need to include a separate indented block for decorations. It also doesn't require re-learning the meaning of "from". > However, I am not sure Guido will change his mind and replace > '@' with something else. Probably, Python became too boring without > fresh syntax solutions like pies. Guido has stated, quite clearly, that if we can come up with a better symbol than "@", then he will go with it. > If so, I'd added a possibility to use ";" to delimit decorators: With the @ syntax, I believe it has been stated that multiple decorators on one line is not to be supported. > May double pie mean "repeat last time used pie": Good for typing reduction, bad for understanding. While a user can know that the function is being decorated, the decorators are not immediately available, nor are named. Since we can name things, perhaps we should. In this case, I believe copy+paste is preferable, or even named multiple application of decorators. def multi_decorate(*decorators): def decorate(function): a = decorators[:] while a: function = a.pop()(function) return function return decorate my_deco = multi_decorate(staticmethod, returns(int)) @my_deco def foo(args): #body @my_deco def goo(args): #body - Josiah From skip at pobox.com Fri Aug 13 21:51:37 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 13 21:51:51 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <004d01c48160$e7602d60$0bb0958d@oemcomputer> References: <20040813093137.63C0.JCARLSON@uci.edu> <004d01c48160$e7602d60$0bb0958d@oemcomputer> Message-ID: <16669.7113.531528.462699@montanaro.dyndns.org> I understand that not everyone will like every idea and that in this particular arena it seems that no one idea is acceptable to everyone, but can we return to my proposal and discuss its pros and cons? I didn't pick the example out of thin air. I cast an existing example from c.l.py in this form (thus the function name and somewhat unusual collection of decorator names). The only thing I added besides the syntax change was to identify where I thought the docstring should go. def p_statement_expr: staticmethod grammarrule('statement : expression') version("Added in 2.4") deprecatedmethod type_(None) decorate (self, p): """docstring here""" print p[1] Roman Suzi seemed to like it, given his "Bingo!" remark, but felt immediately moved to alter it with no explanation. Josiah replied to that with "Oh god no". It went downhill from there, to the point where Raymond posted a sarcastic note about using reST and XML. I don't know if Roman's immediate modification implies his "Bingo!" was just lukewarm. I have no idea what Josiah was saying "Oh god no" to. It seems to satisfy most obvious requirements: * It's clear from the start what function we are modifying, and you know early on that it's not your grandfather's Oldsmobile, err... function. * All decorators are listed up near the top (parseable I think using grammar bits requiring one decorator per line, similar to the current @-syntax). * It doesn't put the cart before the horse by introducing the decorators before the def. * If someone looks at it and thinks, "WTF!", it's readily apparent they should seek help with the "def" or "decorate" keywords. * It doesn't add an extra level of indentation to the function body. * It leaves '@' alone, implying that at some point in the future Guido (if he so chooses) proclaim that "'@' will never be used in Python at the start of a line, so it is free for external tools to use as a markup character". * It looks more like current Python block structure (at least to me) than the @-syntax proposal. I understand that it has obvious weaknesses as well: * It separates the function name from its parameter list. That does allow us to aboid parsing ambiguities though. * It introduces a new 'decorate' keyword, though I think it could sneak by as a pseudo-keyword like 'as'. * It can't readily be extended to class definitions, though the demand for that seems minor and mostly argued for symmetry reasons. One or more of those weaknesses may well be a show stopper. There may be other pros and cons I'm missing. That's fine, but can people please be specific in their criticism? I purposely posted twice, once here and once on c.l.py, to avoid random smartass responses on c.l.py from leaking over to python-dev. Little did I realize I need not have worried about that. Skip From tim.peters at gmail.com Fri Aug 13 21:58:28 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 13 21:58:32 2004 Subject: [Python-Dev] 2.4 asyncore headache Message-ID: <1f7befae040813125824acdb87@mail.gmail.com> asyncore gives me a headache. Windows sockets give me a headache. Zope's ZEO gives me a headache. The ZEO test suite gives me nightmares. With that encouraging background, this ZEO test in ZODB 3.3 fails every time when run on Windows under Python 2.4, but never under Python 2.3.4: checkReconnection (ZEO.tests.testConnection.MappingStorageConnectionTests) ... Traceback (most recent call last): File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\zeoserver.py", line 217, in ? main() File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\zeoserver.py", line 194, in main auth_realm=zo.auth_realm) File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\StorageServer.py", line 738, in __init__ factory=self.new_connection) File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\zrpc\server.py", line 34, in __init__ self._open_socket() File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\zrpc\server.py", line 43, in _open_socket self.bind(self.addr) File "C:\Code\python\lib\asyncore.py", line 304, in bind return self.socket.bind(addr) File "", line 1, in bind socket.error: (10048, 'Address already in use') unittest doesn't notice that failure, but the test hangs there for a minute or so, and unittest does notice the subsequent: Traceback (most recent call last): File "C:\Code\python\lib\unittest.py", line 260, in run testMethod() File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\ConnectionTests.py", line 458, in checkReconnection self.startServer(create=0) File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\ConnectionTests.py", line 202, in startServer zeoport, adminaddr, pid, path = forker.start_zeo_server( File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\forker.py", line 149, in start_zeo_server s.connect(adminaddr) File "", line 1, in connect error: (10061, 'Connection refused') failure. Now somebody, for some reason, added this to 2.4's asyncore.py: if sys.platform == 'win32': reuse_constant = socket.SO_EXCLUSIVEADDRUSE else: reuse_constant = socket.SO_REUSEADDR If I take away the new special case for Windows there, the checkReconnection test works fine again. Now I don't understand any of this. Does anyone? From the *looks* of it, the error from bind() comes from trying to reuse an address, which used to work fine, but no longer works at all on Windows -- at least not in the way ZEO needs it to work. ZEO calls set_reuse_addr() because it needs to reuse ports quickly, and offhand it makes no sense to me to change the meaing of set_reuse_addr() on Windows to something that refuses to allow port reuse. From walter at livinglogic.de Fri Aug 13 22:04:04 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri Aug 13 22:04:09 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <411B6D79.2040503@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> Message-ID: <411D1EB4.10103@livinglogic.de> M.-A. Lemburg wrote: > Hi Walter, > > I don't have time to comment on this this week, I'll respond > next week. OK. > Overall, I don't like the idea of adding extra > APIs breaking the existing codec API. Adding a final argument that defaults to False doesn't break the API for the callers, only for the implementors. And if we drop the final argument, the API is completely backwards compatible both for users and implementors. The only thing that gets added is the feed() method, that implementors don't have to overwrite. > I believe that we can > extend stream codecs to also work in a feed mode without > breaking the API. Abandoning the final argument and adding a feed() method would IMHO be the simplest way to do this. But then there's no way to make sure that every byte from the input stream is really consumed. Bye, Walter D?rwald From amk at amk.ca Fri Aug 13 22:03:47 2004 From: amk at amk.ca (A.M. Kuchling) Date: Fri Aug 13 22:04:24 2004 Subject: [Python-Dev] 2.4 asyncore headache In-Reply-To: <1f7befae040813125824acdb87@mail.gmail.com> References: <1f7befae040813125824acdb87@mail.gmail.com> Message-ID: <20040813200347.GB29067@rogue.amk.ca> On Fri, Aug 13, 2004 at 03:58:28PM -0400, Tim Peters wrote: > Now somebody, for some reason, added this to 2.4's asyncore.py: > if sys.platform == 'win32': > reuse_constant = socket.SO_EXCLUSIVEADDRUSE I'll back out this change. --amk From rnd at onego.ru Fri Aug 13 22:21:03 2004 From: rnd at onego.ru (Roman Suzi) Date: Fri Aug 13 22:20:13 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <20040813104200.63C3.JCARLSON@uci.edu> References: <20040813093137.63C0.JCARLSON@uci.edu> <20040813104200.63C3.JCARLSON@uci.edu> Message-ID: On Fri, 13 Aug 2004, Josiah Carlson wrote: >> f = \ >> decor( \ >> lambda arg: \ >> return_val) >> >> Decorators could be destructive and change function completely, so it's >> pitiful signature doesn't mean much. > >That is funny, because in the 100,000+ lines of Python I've written over >the years, function signatures are, in fact, important. Of course, they are! However, with a possibility to apply decorators function signature could change! >Knowing a >signature allows you to call a function, and I believe that any >decorator that destroys a signature willingly should be considered >broken. Hmmmm... It's up to discipline of a programmer... >As for the "natural order" of definitions, in Python, the natural order >of definitions uses 'def', 'lambda' is considered by many (Guido >included) to be a mistake. I believe your argument using lambda is poor. def f (args): body f = lambda args: body Hmmm... slightly diferent order, indeed. But quite understandable, because lambda creates anonymous function in an expression, while def does namebinding. I always thought the natural order of definition is value to define first, definition to follow it. lambda a mistake? lambda a mistake... -OX >> >> + reads a natural language (with implicit "with" after "f") >> >But it is so radically different from current definitions that it breaks >> >current understanding about the language. >> >> Why? IMHO @s are much more radical. > >Seemingly in Guido's opinion, @s (or other symbols) were minimal. You >can have a humble (or not so humble) opinion, but unless you can >convince Guido and/or the community, it is not going to happen (I've >been bitten by this myself). I am pretty sure @-syntax will do it into Python 2.4 and I am also sure that nobody will remeber these discussions 1-2 years from now. I wonder WHAT Python looked like if it's community discussed EVERY syntax feature it had before 1.5.2. My feeling is Python still missed IF-THEN-ELSE... GvR already decided on the syntax. His brain has already calculated all those millions of possibilities (moves), community response included. We have almost zero chance to find better syntax for decorators. >> >It also requires people to >> >learn two significantly different semantics for defining a function. >> >> >define f as a staticmethod... from the arguments self and p with >> >function body... Not bad, though I don't like the visual aesthetics. >> >> Ok. Now I see our disagrement. I am thinking in terms of >> applying decorators to make some other function FROM >> a given function while you are reasoning about "adding" attributes to the >> defined function. > >No, our disagreement isn't about how we understand decorators. I never >argue it is about adding attributes, as I have actually read the 1800+ I hope you do not receive treatment at mental hospital right now? ;-) (Disclaimer: no pun intended. Just expressing my surprise) >postings on decorators in the last 8 months, where attributes are >basically stated as being not arguable. >Our disagreement is about what is a reasonable decorator syntax. I >don't believe that anything that changes the look of a function >signature to be a reasonable decorator syntax. In fact, I've already proposed 2 syntaxes: C4 and J3. But aren't decorators part of a signature? Are there any languages with decorators similar to being implemented in Python? (Full-fledged, not things like "static volatile ..." in C/Java/C++ like languages.) Aren't decorators functional programming everyday feature, when (in some programming languages) it is normal to define function as a superposition of simpler ones? [ neither "as" nor "to" are keywords ] ok >> I mean "no additional" level, like other group decoration proposals. > >Oops, I didn't finish my argument. > >I know what you meant. In order to understand what a function does, one >must read the decorators. While associated indented blocks have been >done before (try, except, finally, else), one doesn't need to read the >try block to understand that if an exception were to occur, the except >block would be executed. In order to understand the semantics of a >function definition, you need to read the decorators. Having an >indented block previous to the function definition, which includes the >decorators for the function below, only makes sense if the name is given >before the decorators (the brain-stack argument). > However, since the >function signature is as important as any decorator, then the signature >should be prominant. >I believe your arguments are better focused toward a different syntax, >namely: > >def fun(args): > @deco > #body > >I think it beats your offered syntax hands-down due to its conciseness >and lack of need to include a separate indented block for decorations. >It also doesn't require re-learning the meaning of "from". >> However, I am not sure Guido will change his mind and replace >> '@' with something else. Probably, Python became too boring without >> fresh syntax solutions like pies. > >Guido has stated, quite clearly, that if we can come up with a better >symbol than "@", then he will go with it. The ord "better" is subjective. In this case it means "better" in Guido's view. Probably, we can even write a syntax-proposing program which could give thousands of variants (and probably even validate them with existing Python grammar to exclude clashes) from which Guido could choose. Wiki already has enough material to do this sort of machine. Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From tim.peters at gmail.com Fri Aug 13 22:25:07 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 13 22:25:12 2004 Subject: [Python-Dev] 2.4 asyncore headache In-Reply-To: <20040813200347.GB29067@rogue.amk.ca> References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> Message-ID: <1f7befae04081313252cb3304a@mail.gmail.com> [Tim] >> Now somebody, for some reason, added this to 2.4's asyncore.py: >> if sys.platform == 'win32': >> reuse_constant = socket.SO_EXCLUSIVEADDRUSE [Andrew] > I'll back out this change. But only if that's the right thing to do <0.9 wink>. I gave up reading the Windows SO_EXCLUSIVEADDRUSE docs after it got to the "This situation can be quite complicated; even though the socket has been closed, the underlying transport may not terminate its connection ..." part. But by that point, it sure didn't seem like SO_EXCLUSIVEADDRUSE was a plausible Windowsy way to spell the Unixy SO_REUSEADDR ... From jack at performancedrivers.com Fri Aug 13 23:31:00 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Fri Aug 13 23:31:04 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <16669.7113.531528.462699@montanaro.dyndns.org> References: <20040813093137.63C0.JCARLSON@uci.edu> <004d01c48160$e7602d60$0bb0958d@oemcomputer> <16669.7113.531528.462699@montanaro.dyndns.org> Message-ID: <20040813213100.GQ23725@performancedrivers.com> On Fri, Aug 13, 2004 at 02:51:37PM -0500, Skip Montanaro wrote: > > I understand that not everyone will like every idea and that in this > particular arena it seems that no one idea is acceptable to everyone, but > can we return to my proposal and discuss its pros and cons? I didn't pick > the example out of thin air. I cast an existing example from c.l.py in this > form (thus the function name and somewhat unusual collection of decorator > names). The only thing I added besides the syntax change was to identify > where I thought the docstring should go. > > def p_statement_expr: > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > decorate (self, p): > """docstring here""" > print p[1] > > I understand that it has obvious weaknesses as well: > > * It separates the function name from its parameter list. That does > allow us to aboid parsing ambiguities though. This is huge, it would mean two ways to do it. When I want to decorate the function with '@' I add an extra line above the existing definition. With this suggestion I have to pick the line apart and shuffle stuff around. Guido has said being able to add/remove a decorator should be as simple as cut/pasting a line and I agree. > * It can't readily be extended to class definitions, though the demand > for that seems minor and mostly argued for symmetry reasons. Class decorators are pretty important to me. I would replace most of my meta class usages with decorators. I care enough I spent two days writing a patch and fishing through the bowels of compile.c (*plug plug*). * It uses a suite in a way different from all other suites. A group of points * it breaks grepability badly * it breaks existing tools by radically changing the grammar * breaking machine tools also means breaking meat tools. I now have to fit two ways of recognizing functions in my head all the time. This is the showstopper of all showstoppers right there. -Jack From rnd at onego.ru Sat Aug 14 00:05:08 2004 From: rnd at onego.ru (Roman Suzi) Date: Sat Aug 14 00:04:29 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <16669.7113.531528.462699@montanaro.dyndns.org> References: <20040813093137.63C0.JCARLSON@uci.edu> <004d01c48160$e7602d60$0bb0958d@oemcomputer> <16669.7113.531528.462699@montanaro.dyndns.org> Message-ID: On Fri, 13 Aug 2004, Skip Montanaro wrote: >I understand that not everyone will like every idea and that in this >particular arena it seems that no one idea is acceptable to everyone, but >can we return to my proposal and discuss its pros and cons? I didn't pick >the example out of thin air. I cast an existing example from c.l.py in this >form (thus the function name and somewhat unusual collection of decorator >names). The only thing I added besides the syntax change was to identify >where I thought the docstring should go. > > def p_statement_expr: > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > decorate (self, p): > """docstring here""" > print p[1] > >Roman Suzi seemed to like it, given his "Bingo!" remark, but felt >immediately moved to alter it with no explanation. Josiah replied to that >with "Oh god no". It went downhill from there, to the point where Raymond >posted a sarcastic note about using reST and XML. I don't know if Roman's >immediate modification implies his "Bingo!" was just lukewarm. Well, your example lead me to my syntax and I liked it very much at the moment. > I have no >idea what Josiah was saying "Oh god no" to. Now it seems less wonderful to me, because I forgot about decorating classes. class lalalal: decor from (A,B): @definition doesn't have properties I'd liked to see. Same applies to Skip's syntax. >It seems to satisfy most obvious requirements: > > * It's clear from the start what function we are modifying, and you know > early on that it's not your grandfather's Oldsmobile, err... function. > > * All decorators are listed up near the top (parseable I think using > grammar bits requiring one decorator per line, similar to the current > @-syntax). > > * It doesn't put the cart before the horse by introducing the decorators > before the def. I agree with this. > * If someone looks at it and thinks, "WTF!", it's readily apparent they > should seek help with the "def" or "decorate" keywords. Ummm.... Me thinks it's much more important to facilitate those who write lots of code than those who looks at it sheepishly (sp?) > * It doesn't add an extra level of indentation to the function body. > > * It leaves '@' alone, implying that at some point in the future Guido > (if he so chooses) proclaim that "'@' will never be used in Python at > the start of a line, so it is free for external tools to use as a > markup character". > * It looks more like current Python block structure (at least to me) > than the @-syntax proposal. > >I understand that it has obvious weaknesses as well: > > * It separates the function name from its parameter list. That does > allow us to aboid parsing ambiguities though. > > * It introduces a new 'decorate' keyword, though I think it could sneak > by as a pseudo-keyword like 'as'. ... or no keyword at alll, as some suggested. > * It can't readily be extended to class definitions, though the demand > for that seems minor and mostly argued for symmetry reasons. > >One or more of those weaknesses may well be a show stopper. There may be >other pros and cons I'm missing. That's fine, but can people please be >specific in their criticism? I purposely posted twice, once here and once >on c.l.py, to avoid random smartass responses on c.l.py from leaking over to >python-dev. Little did I realize I need not have worried about that. IMO, "decorate" is too wordy. "dec" is associated with "decimal", so your proposal has no chance to fly :-( And mine too as Josiah showed above. This @discussion reminds me of inline-if one. With the difference that at that time GvR did not favor ANY syntax and now he favors the @syntax. Lets tell to themselves that there are no better choices than @. Amen. >Skip Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From jcarlson at uci.edu Sat Aug 14 00:12:14 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat Aug 14 00:18:20 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <16669.7113.531528.462699@montanaro.dyndns.org> References: <004d01c48160$e7602d60$0bb0958d@oemcomputer> <16669.7113.531528.462699@montanaro.dyndns.org> Message-ID: <20040813132721.63CC.JCARLSON@uci.edu> > I understand that not everyone will like every idea and that in this > particular arena it seems that no one idea is acceptable to everyone, but > can we return to my proposal and discuss its pros and cons? I didn't pick > the example out of thin air. I cast an existing example from c.l.py in this > form (thus the function name and somewhat unusual collection of decorator > names). The only thing I added besides the syntax change was to identify > where I thought the docstring should go. > > def p_statement_expr: > staticmethod > grammarrule('statement : expression') > version("Added in 2.4") > deprecatedmethod > type_(None) > decorate (self, p): > """docstring here""" > print p[1] > > Roman Suzi seemed to like it, given his "Bingo!" remark, but felt > immediately moved to alter it with no explanation. Josiah replied to that > with "Oh god no". It went downhill from there, to the point where Raymond > posted a sarcastic note about using reST and XML. I don't know if Roman's > immediate modification implies his "Bingo!" was just lukewarm. I have no > idea what Josiah was saying "Oh god no" to. Mostly the separating the function name from the arguments. > * It doesn't put the cart before the horse by introducing the decorators > before the def. Great. > * It doesn't add an extra level of indentation to the function body. Great. > * It leaves '@' alone, implying that at some point in the future Guido > (if he so chooses) proclaim that "'@' will never be used in Python at > the start of a line, so it is free for external tools to use as a > markup character". !|&-+* also leave the '@' alone. I think that if it werent for '|' being mistakable for I,l or 1, it would already have been chosen. > * It looks more like current Python block structure (at least to me) > than the @-syntax proposal. Check my P.S. at the end. > I understand that it has obvious weaknesses as well: > > * It separates the function name from its parameter list. That does > allow us to avoid parsing ambiguities though. Separating the argument list from the function name is enough to kill it for me. > * It can't readily be extended to class definitions, though the demand > for that seems minor and mostly argued for symmetry reasons. Why can't it be extended? class decorated_class: decorators decorate: #or (subcls1, subcls2) before the colon with subclasses "docstring" body Personally, I don't desire decorators for classes for symmetry reasons, I desire it to replace metaclasses, at least in those things that I could conceivably want metaclasses to do. > One or more of those weaknesses may well be a show stopper. There may be > other pros and cons I'm missing. That's fine, but can people please be > specific in their criticism? I purposely posted twice, once here and once > on c.l.py, to avoid random smartass responses on c.l.py from leaking over to > python-dev. Little did I realize I need not have worried about that. My major issue is that I think we should be making a small change to the language. I believe that any change that could be considered a large change should be avoided. Introducing an additional block structure to specify decorators seems to be a large change. Separating the name of a function from the arguments seems to be a large change. Other proposals attempt to indent function definitions inside decorators, which is also a large change (which also destroy readability). Let us try to keep the change small. Sound reasonable? - Josiah P.S. One could make the claim that assigning directly to __slots__ and __metaclass__ are a bit of magic that could have been avoided with proper decorators (or even that metaclasses could have been avoided with decorators). @__metaclass__(metaclass) @__slots__("slot1", "slot2") class classname: #etc. Heck, it almost looks like what we already have in Python when we do the following... class classname(object): @__metaclass__(metaclass) #or @metaclass_like_decorator @__slots__("slot1", "slot2") @__doc__("docstring") #etc. Goodness, there's a syntax to rule them all. Not only does it kind of look like what we already have, but it is further extendable with other __name__ decorators. Choose any @ replacement if desireable. From bob at redivi.com Sat Aug 14 00:27:37 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat Aug 14 00:27:40 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <20040813132721.63CC.JCARLSON@uci.edu> References: <004d01c48160$e7602d60$0bb0958d@oemcomputer> <16669.7113.531528.462699@montanaro.dyndns.org> <20040813132721.63CC.JCARLSON@uci.edu> Message-ID: On Aug 13, 2004, at 6:12 PM, Josiah Carlson wrote: > Personally, I don't desire decorators for classes for symmetry reasons, > I desire it to replace metaclasses, at least in those things that I > could conceivably want metaclasses to do. In some cases, what you actually want ARE metaclasses. I wouldn't say get rid of them, because I use them in cases where what I actually want ARE metaclasses, but I do concede that many of the (ab)uses of metaclasses could be replaced by decorators (assuming that the decorator syntax applied to classes as well). -bob From jcarlson at uci.edu Sat Aug 14 00:53:06 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat Aug 14 00:59:21 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: References: <20040813104200.63C3.JCARLSON@uci.edu> Message-ID: <20040813135102.63CF.JCARLSON@uci.edu> > >> Decorators could be destructive and change function completely, so it's > >> pitiful signature doesn't mean much. > > > >That is funny, because in the 100,000+ lines of Python I've written over > >the years, function signatures are, in fact, important. > > Of course, they are! However, with a possibility to apply decorators > function signature could change! You must be joking. Just because a signature can change doesn't mean that the signature is not important. I'll say it again, just in case you missed it the first time, the first major use of decorators are going to be in PyObjC, WHICH USES DECORATORS TO SPECIFY SIGNATURES. If you continue to claim that signatures are unimportant, I can't imagine how this discussion could continue in any sort of a productive fashion. > >Knowing a > >signature allows you to call a function, and I believe that any > >decorator that destroys a signature willingly should be considered > >broken. > > Hmmmm... It's up to discipline of a programmer... How does the discipline of a programmer have anything to do with the use of signature-changing decorators? When I (and I imagine most everyone else) define a function or class, its signature tells me almost everything I need to know about them. What you are suggesting is that this method of using signatures to describe functionality should not be relied upon, because signatures can change. That's like saying that I shouldn't rely on the amount of money in my checking account, because that amount changes. > >As for the "natural order" of definitions, in Python, the natural order > >of definitions uses 'def', 'lambda' is considered by many (Guido > >included) to be a mistake. I believe your argument using lambda is poor. > > def f (args): body > f = lambda args: body > > Hmmm... slightly diferent order, indeed. But quite understandable, > because lambda creates anonymous function in an expression, while > def does namebinding. You don't need to explain what it does. Generally, describing decoration in terms of a functional approach is pointless, we already have a functional approach to decorations, which is what we are trying to remove. def f(args): body f = decorate(f) Also, if we were to give a language description of the lambda form you describe, it would be: Define an anonymous function with arguments args and body bdy, decorating it with decorate, and naming it name. Ick. I much prefer: define a function called name, with arguments args, decorated by decorator, with body bdy. Which surprisingly enough looks like... def fun(args): @decorator #body Again, replace @ if desired. > I always thought the natural order of definition is value to define first, > definition to follow it. I don't know where this came from, but it is what we already have. def fun(args): body > lambda a mistake? lambda a mistake... -OX Yes, lambda a mistake: http://www.linuxjournal.com/article.php?sid=2959 > >Seemingly in Guido's opinion, @s (or other symbols) were minimal. You > >can have a humble (or not so humble) opinion, but unless you can > >convince Guido and/or the community, it is not going to happen (I've > >been bitten by this myself). > > I am pretty sure @-syntax will do it into Python 2.4 and I am also > sure that nobody will remeber these discussions 1-2 years from now. If you are so sure, why continue arguing towards your "J3" syntax (which is a variation on the syntax Skip proposed)? > I wonder WHAT Python looked like if it's community discussed > EVERY syntax feature it had before 1.5.2. My feeling is Python > still missed IF-THEN-ELSE... You are moving off-topic. > GvR already decided on the syntax. His brain has already > calculated all those millions of possibilities (moves), > community response included. We have almost zero chance to > find better syntax for decorators. Considering that you were, earlier today, offering a different syntax, this seems quite out of character for you. > >No, our disagreement isn't about how we understand decorators. I never > >argue it is about adding attributes, as I have actually read the 1800+ > > I hope you do not receive treatment at mental hospital right now? ;-) > (Disclaimer: no pun intended. Just expressing my surprise) No, I'm working on a PhD in computer science theory. A few thousand python-dev messages are just personal research into my favorite programming language. > >postings on decorators in the last 8 months, where attributes are > >basically stated as being not arguable. > > >Our disagreement is about what is a reasonable decorator syntax. I > >don't believe that anything that changes the look of a function > >signature to be a reasonable decorator syntax. > > In fact, I've already proposed 2 syntaxes: C4 and J3. C4 is a derivative of what has been known since before March. J3 has serious problems, as stated by Jack and myself. > But aren't decorators part of a signature? Are there any languages with > decorators similar to being implemented in Python? > (Full-fledged, not things like "static volatile ..." in C/Java/C++ like > languages.) Decorators are only part of the signature if you want them to be. I believe the vast majority of decorators will not change function signatures. I am not aware of decorators in other languages, but I have kept my nose out of most langauges developed since I discovered Python in 2000. > Aren't decorators functional programming everyday feature, when > (in some programming languages) it is normal to define function as > a superposition of simpler ones? That is like saying that all programming is functional. Is prolog a functional programming language? No, it is a logic programming language, yet it is common in Prolog to see the following: my_and(a, b) = my_bool(a) and my_bool(b) > >Guido has stated, quite clearly, that if we can come up with a better > >symbol than "@", then he will go with it. > > The ord "better" is subjective. In this case it means "better" > in Guido's view. Probably, we can even write a syntax-proposing > program which could give thousands of variants (and probably even > validate them with existing Python grammar to exclude clashes) > from which Guido could choose. Wiki already has enough material to > do this sort of machine. So then, it seems to me like the community should take a vote on what symbol (if any) is better than @, and commit to sticking behind that one. If Guido likes it, great, we have a symbol; then we can argue about any slight variation of Guido's preferred syntax (if he is willing to listen), and call it good. - Josiah From zathras at thwackety.com Sat Aug 14 01:31:24 2004 From: zathras at thwackety.com (Michael Sparks) Date: Sat Aug 14 01:11:16 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation Message-ID: Hi, I suspect people are getting sick of the decorator discussion by now, so I'll try to be brief, and useful. Syntax option J2 from the wiki seems to have provoked an interesting discussion[1] at least by a small subset of c.l.p, so I wondered how difficult it would be to implement - this turned out to be relatively simple [2]. [1] http://mail.python.org/pipermail/python-list/2004-August/233413.html [2] http://mail.python.org/pipermail/python-list/2004-August/233591.html (implementation) Example of the syntax... (I'm sure it's familiar...) #Syntax J2 from http://www.python.org/moin/PythonDecorators #Some random comment #More random comment decorate: staticmethod grammarrule('statement : expression') versioninfo("Added in 2.4") deprecated typeinfo(None) def p_statement_expr(self, p): print p[1] For comparison, this is intended as equivalent to: #Current syntax in 2.4a2 #Some random comment #More random comment @staticmethod @grammarrule('statement : expression') @versioninfo("Added in 2.4") @deprecated @typeinfo(None) def p_statement_expr(self, p): print p[1] With the exact semantics and restrictions remaining the same - this is just syntactic sugar. This was my first time looking at the python source to see how easy it would be to implement - so it's not perfect, and probably far from ideal. My reasons I looking and giving it a go though were: * To me it looks much clearer - you can instantly see where the method (and decorator block) starts. * Lack of repetition of the decoration hint * It hints to a non-python programmer very clearly that this is something different. If they're aware of the decorator patterm, they will realise (perhaps) what it means, and if they don't they have something to google for. I have read most of the arguments against this syntax, and if the decision ends up to stay with @ or take a different syntax, I'm sure it'll work out fine :) (This isn't my favourite syntax, my favourite has a keyword inside the function suite, which I'm well aware is dead in the water!) The post I made to c.l.p with a first attempt at an implementation (has minor issues regarding scopes, but does work) can be found here: * http://mail.python.org/pipermail/python-list/2004-August/233591.html If there is interest in this syntax, then I suspect/hope making it suitable for a "proper" patch would be relatively simple. In case people are wondering what the grammar change I made for this is, I'll include here: -decorator: '@' dotted_name [ '(' [arglist] ')' ] +decorator: dotted_name [ '(' [arglist] ')' ] decorators: decorator ([NEWLINE] decorator)* NEWLINE -funcdef: [decorators] 'def' NAME parameters ':' suite +funcdef: ['decorate' ':' NEWLINE INDENT decorators DEDENT ] 'def' NAME parameters ':' suite parameters: '(' [varargslist] ')' Downsides here: * Looks like a suite, implying it allows arbitrary statements, but isn't * Introduces a new keyword * Some people have expressed reservations about the syntax, some of which also apply to the @pie syntax, and some which don't. * The patch above has scope issues, which means currently it requires some cruft to force functions into scope. Biggest upside from a personal perspective though is this: (aside from others) * Forces things like the common cases of classmethod and staticmethod to jump out at a maintainer. As the wiki mentions, this two line syntax is relatively heavyweight for these cases, and I agree. I disagree about this being bad - static methods and class methods are rare. Having a BIG indication they exist when you have (say) 20 methods in a class strikes me as a very positive thing, and is surely 90% of the reason for moving from the existing mechanism of: foo = staticmethod(foo) Example tested: class Foo: staticmethod # This is cruft to force staticmethod into scope decorate: staticmethod def hello(who): print "woo?", who Anyway, hope this is considered useful, rather than extending the headache! (However, I am for one very happy that decorators are now available in something clearer than we had in pre(2.4) and it certainly made implementing the syntax/semantics above relatively trivial :) Regards, Michael. -- "The 'nice' thing about decorator syntaxes is there are so many to choose from." From bob at redivi.com Sat Aug 14 01:24:24 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat Aug 14 01:24:26 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation In-Reply-To: References: Message-ID: On Aug 13, 2004, at 7:31 PM, Michael Sparks wrote: > Syntax option J2 from the wiki seems to have provoked an interesting > discussion[1] at least by a small subset of c.l.p, so I wondered how > difficult it would be to implement - this turned out to be relatively > simple [2]. > [1] > http://mail.python.org/pipermail/python-list/2004-August/233413.html > [2] > http://mail.python.org/pipermail/python-list/2004-August/233591.html > (implementation) > > #Syntax J2 from http://www.python.org/moin/PythonDecorators > decorate: > staticmethod > grammarrule('statement : expression') > versioninfo("Added in 2.4") > deprecated > typeinfo(None) > def p_statement_expr(self, p): > print p[1] My only problem with this syntax is that I expect the common cases (mine, anyway) to be zero or one decorators per function, so the extra block and indent seems a bit excessive compared to the current @prefix-symbol-decorator proposal(s). It is, of course, far better than typing the function name three times though! :) -0 -bob From ncoghlan at iinet.net.au Sat Aug 14 01:30:47 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Aug 14 01:30:55 2004 Subject: [Python-Dev] 2.4 asyncore headache In-Reply-To: <1f7befae04081313252cb3304a@mail.gmail.com> References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> <1f7befae04081313252cb3304a@mail.gmail.com> Message-ID: <411D4F27.3030605@iinet.net.au> Tim Peters wrote: > [Tim] > >>>Now somebody, for some reason, added this to 2.4's asyncore.py: >>> if sys.platform == 'win32': >>> reuse_constant = socket.SO_EXCLUSIVEADDRUSE > > > [Andrew] > >>I'll back out this change. > > > But only if that's the right thing to do <0.9 wink>. I gave up > reading the Windows SO_EXCLUSIVEADDRUSE docs after it got to the "This > situation can be quite complicated; even though the socket has been > closed, the underlying transport may not terminate its connection ..." > part. But by that point, it sure didn't seem like SO_EXCLUSIVEADDRUSE > was a plausible Windowsy way to spell the Unixy SO_REUSEADDR ... Is test_socketserver currently passing for you? A couple of days ago I ran the test suite with -unetwork, and that test failed, but I haven't got around to looking into it yet. Could this be related? [rambling discussion of what I found out about this so far follows] Anyway, after some hunting, the difference between Windows and Unix SO_REUSEADDR, is that on Unix, this only allows local address reuse when the existing socket is in a TIME_WAIT state. On Windows, it allows local address reuse _anytime_. See http://hea-www.harvard.edu/~fine/Tech/addrinuse.html. Look for the phrase 'security stigma', then read the bit starting with 'On some operating systems. . .' The issue is that if we *don't* set SO_REUSEADDR on Windows, then you get into problems with getting EADDRINUSE when you shutdown and restart a TCP/IP server on the same local address (generally if the socket has gone into TIME_WAIT while shutting down). Looking at MSDN, SO_EXCLUSIVEADDRUSE is a way to say 'make my socket behave like a UNIX socket without SO_REUSEADDR specified'. That is, with no options specified, a Windows socket is vulnerable to having another process come along, specify SO_REUSEADDR for their socket options, and succeed in reusing the same local port. (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp) What I can't find is a way to get Unix-like SO_REUSEADDR behaviour on Windows. Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From ncoghlan at iinet.net.au Sat Aug 14 01:31:08 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Aug 14 01:31:15 2004 Subject: [Python-Dev] 'commands' module on Win32 In-Reply-To: <200408131430.i7DEUgK32685@guido.python.org> References: <411C9A5E.3070109@iinet.net.au> <200408131430.i7DEUgK32685@guido.python.org> Message-ID: <411D4F3C.9010608@iinet.net.au> Guido van Rossum wrote: >>Bug #1008275 refers to attempting to run commands.getstatusoutput() on >>win32, and the fact that it breaks. > > > Hm... That module's functionality is such a small improvement over > raw os.popen, it would never have been admitted into the standard > library these days. > > I'd rather see this added as a higher-level API to one of the process > management modules that are floating around, one of which will > eventually land in the library, at which point we can deprecate > commands.py. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > OK - I'll add something along these lines to the bug report. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From paul.dubois at gmail.com Sat Aug 14 01:35:32 2004 From: paul.dubois at gmail.com (Paul Du Bois) Date: Sat Aug 14 01:35:35 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation In-Reply-To: References: Message-ID: <85f6a31f04081316356c8a49ce@mail.gmail.com> [bob, apologies for the re-send... I'm a gmail newbie] On Fri, 13 Aug 2004 19:24:24 -0400, Bob Ippolito wrote: > My only problem with this syntax is that I expect the common cases > (mine, anyway) to be zero or one decorators per function, so the extra > block and indent seems a bit excessive compared to the current > @prefix-symbol-decorator proposal(s). It is, of course, far better For the case of one decorator, one could allow: decorate: staticmethod def p_statement_expr(self, p): pass Which is admittedly more typing than @staticmethod -- but is brevity a feature? Even if "decorate" is an inaccurate description of what the decorator is doing (it may registering, creating grayscale ramps, and so on) it at least matches the name of the language feature: decorators. Good for documentation! This is my favorite proposal so far. From tim.peters at gmail.com Sat Aug 14 03:22:32 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 14 03:22:38 2004 Subject: [Python-Dev] 2.4 asyncore headache In-Reply-To: <411D4F27.3030605@iinet.net.au> References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> <1f7befae04081313252cb3304a@mail.gmail.com> <411D4F27.3030605@iinet.net.au> Message-ID: <1f7befae04081318224503fb44@mail.gmail.com> [Nick Coghlan] > Is test_socketserver currently passing for you? Yes, on WinXP, and regardless of whether "currently" means before or after Andrew reverted the change (& all the ZODB 3.3 tests pass under current CVS Python on Windows after the reversion). > A couple of days ago I ran the test suite with -unetwork, and that test > failed, but I haven't got around to looking into it yet. Could this be > related? Which OS? asyncore isn't involved in that test, so the patch could not have affected it. SocketServer.py has its own, separate code using SO_REUSEADDR in some cases. test_socketserver.pickport() never reuses a port number anyway. However, it may pick a port number that happens to be already in use outside the test, so that could fail. The port numbers it picks depend on the process id, so a "picked a port # already in use" failure mode may disappear on the next run. > [rambling discussion of what I found out about this so far follows] > > Anyway, after some hunting, the difference between Windows and Unix > SO_REUSEADDR, is that on Unix, this only allows local address reuse when > the existing socket is in a TIME_WAIT state. On Windows, it allows local > address reuse _anytime_. > > See http://hea-www.harvard.edu/~fine/Tech/addrinuse.html. Look for the > phrase 'security stigma', then read the bit starting with 'On some > operating systems. . .' It also says: This is only a problem on multi-user machines that don't have restricted logins, it is NOT a vulnerability from outside the machine. And it is easily avoided by binding your server to the machine's address. If you're running a server on an open Windows machine, worrying about details of socket reuse may not be the best use of your security time . > The issue is that if we *don't* set SO_REUSEADDR on Windows, then you > get into problems with getting EADDRINUSE when you shutdown and restart > a TCP/IP server on the same local address (generally if the socket has > gone into TIME_WAIT while shutting down). That's why ZEO calls asyncore's set_reuse_addr(), so that it can shutdown/restart rapidly. This has worked fine for years on Windows. > Looking at MSDN, SO_EXCLUSIVEADDRUSE is a way to say 'make my socket > behave like a UNIX socket without SO_REUSEADDR specified'. That is, with > no options specified, a Windows socket is vulnerable to having another > process come along, specify SO_REUSEADDR for their socket options, and > succeed in reusing the same local port. > > (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp) asyncore shouldn't use this. > What I can't find is a way to get Unix-like SO_REUSEADDR behaviour on > Windows. Well, the more I learn about this one, the less I care about it. ZEO can live with Unix-like SO_REUSEADDR behavior too -- I think. I have to wonder, because reports of "connection refused" kinds of ZEO test failures have always been much more common on Linux boxes. From foom at fuhm.net Sat Aug 14 03:52:04 2004 From: foom at fuhm.net (James Y Knight) Date: Sat Aug 14 03:52:11 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <20040813135102.63CF.JCARLSON@uci.edu> References: <20040813104200.63C3.JCARLSON@uci.edu> <20040813135102.63CF.JCARLSON@uci.edu> Message-ID: <8ACD1FFD-ED94-11D8-8449-000A95A50FB2@fuhm.net> On Aug 13, 2004, at 6:53 PM, Josiah Carlson wrote: >> lambda a mistake? lambda a mistake... -OX > > Yes, lambda a mistake: > http://www.linuxjournal.com/article.php?sid=2959 Of course, in that article, Guido says lambda is a mistake because the scoping rules of Python were broken at that point. As they are no longer broken, that particular argument doesn't hold much sway... James From raymond.hettinger at verizon.net Sat Aug 14 04:09:28 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat Aug 14 04:09:54 2004 Subject: [Python-Dev] New Developer Message-ID: <008701c481a3$bb7250a0$0bb0958d@oemcomputer> [Johannes Gijsbers] > > Consider this a request for checkin privileges. [amk] > Seconded. Johannes, your permissions are enabled. Welcome. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040813/ee8a7999/attachment.html From python at rcn.com Sat Aug 14 03:30:17 2004 From: python at rcn.com (Raymond Hettinger) Date: Sat Aug 14 04:16:35 2004 Subject: [Python-Dev] New Developer In-Reply-To: <20040813185131.GB28569@rogue.amk.ca> Message-ID: <008001c4819e$4180ea40$0bb0958d@oemcomputer> [Johannes Gijsbers] > > Consider this a request for checkin privileges. [amk] > Seconded. Johannes, your permissions are enabled. Welcome. Raymond From jiangyu at adtec.com.cn Fri Aug 13 12:16:05 2004 From: jiangyu at adtec.com.cn (Jiang Yu) Date: Sat Aug 14 05:43:31 2004 Subject: [Python-Dev] Question about win32com in MS-Excel Message-ID: <000001c481b1$4acf2360$e7100a0a@jiangyu> Hello, My name is Jiangyu, a Chinese man. You can call me Joe. I learn Python for only one year. I think it's a simple tools in my job some time. But in my country, Python is a new tools for many people. So If i have some problems on it, I found it's difficult to look for the answers?So I add the this mail list. Now, I have a question about use win32com moudle for MS-Excel. Question: I want to insert a file into a Excel sheet. But a exception occured in my code. Using VBA, I insert a file OK. And the processing in Python is equivalent in VBA. My environment: Python2.2.3 + win32com + Win2000 Server + Excel(Office XP) My Python code: >>> from win32com.client import Dispatch >>> excel = Dispatch("Excel.Application") >>> excel.Visible = 1 >>> excel.WorkBooks.Add() >>> cursheet = excel.ActiveWorkBook.ActiveSheet >>> oleobjs = cursheet.OLEObjects() >>> oleobjs.Add(Filename = r"c:\temp.txt") My VBA code: ActiveWorkbook.ActiveSheet.OLEObjects.Add Filename:="c:\temp.txt", Link:=False, DisplayAsIcon:=False So, If you know how to do it. please answer me. Thanks! joe 2004/8/13 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040813/12d32d08/attachment.html From tim.peters at gmail.com Sat Aug 14 06:37:49 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 14 06:37:55 2004 Subject: [Python-Dev] Another 2.4 asyncore headache In-Reply-To: <20040813200347.GB29067@rogue.amk.ca> References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> Message-ID: <1f7befae040813213728e50a44@mail.gmail.com> Another change to asyncore broke Zope: http://collector.zope.org/Zope3-dev/261 I may have mentioned before that asyncore gives me a headache. The checkin comment for rev 1.57 just says: In poll(), check connections for exceptional conditions and it's not mentioned in NEWS or in any doc changes. I don't want to guess what the intent was. It definitely has a bad effect on Zope. I can't guess what asyncore's intent is in the presence of threads either. It appears to be robust against other threads mutating the socket map *across* select() calls. But Zope appears to mutate the socket map between the time map.items() captures a snapshot and the time select() is called. There's a thread race here when an object in the socket map is closing, and removing itself from the socket map. In 2.3.4 it apparently didn't matter because the exception set passed to select() was always empty. In 2.4 it's passing a non-readable non-writable object in select's exception set, and Zope manages to close the underlying socket before select() is called. The result is a select() error that (a) couldn't happen in previous Pythons, and (b) Zope sure isn't expecting. More generally, in 2.3.4 it didn't hurt to leave all kinds of crap in the socket map, provided that any such crap didn't call itself ready to read or ready to write. select() would never see it then. In 2.4, select() always sees it. That's a big change, particularly because everyone I've seen wrestling with asyncore works via iterative poke-and-hope, fiddling their code more-or-less randomly, until mysterious errors stop appearing. So if it's of real value to you (I don't know the motivating use case for the change) to pass stuff in the exception set, would it be good enough to do e = r + w instead of (in effect) e = map.keys() ? Then stuff that doesn't call itself readable or writable would again, as in 2.3.4, not get passed to select(). Or Zope has to change. From exarkun at divmod.com Sat Aug 14 07:32:04 2004 From: exarkun at divmod.com (Jp Calderone) Date: Sat Aug 14 07:32:09 2004 Subject: [Python-Dev] def ... decorate In-Reply-To: <20040813213100.GQ23725@performancedrivers.com> References: <20040813093137.63C0.JCARLSON@uci.edu> <004d01c48160$e7602d60$0bb0958d@oemcomputer> <16669.7113.531528.462699@montanaro.dyndns.org> <20040813213100.GQ23725@performancedrivers.com> Message-ID: <411DA3D4.4030007@divmod.com> Jack Diederich wrote: > On Fri, Aug 13, 2004 at 02:51:37PM -0500, Skip Montanaro wrote: > >> [snip] >> >> def p_statement_expr: >> staticmethod >> grammarrule('statement : expression') >> version("Added in 2.4") >> deprecatedmethod >> type_(None) >> decorate (self, p): >> """docstring here""" >> print p[1] >> > [snip] > > A group of points > * it breaks grepability badly "@" isn't all that greppable. exarkun@boson:~/Twisted$ grep -E '^[ \t]+@\w+' ./ -r | wc -l 5025 Personally, greppability doesn't seem particular important either way. > * it breaks existing tools by radically changing the grammar Existing tools are going to break. That's what happens when you change the grammar. Jp From ncoghlan at iinet.net.au Sat Aug 14 09:40:18 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Aug 14 09:40:26 2004 Subject: [Python-Dev] 2.4 asyncore headache In-Reply-To: <1f7befae04081318224503fb44@mail.gmail.com> References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> <1f7befae04081313252cb3304a@mail.gmail.com> <411D4F27.3030605@iinet.net.au> <1f7befae04081318224503fb44@mail.gmail.com> Message-ID: <411DC1E2.9080200@iinet.net.au> Tim Peters wrote: > [Nick Coghlan] > >>A couple of days ago I ran the test suite with -unetwork, and that test >>failed, but I haven't got around to looking into it yet. Could this be >>related? > > > Which OS? asyncore isn't involved in that test, so the patch could > not have affected it. SocketServer.py has its own, separate code > using SO_REUSEADDR in some cases. Ah, OK, it was just a thought. I'm on Windows XP SP1, and it's consistently choking on the second attempt to test the UDP Server: ==Test=Output== [snipped output from TCP tests] ADDR = ('localhost', 12963) CLASS = SocketServer.UDPServer server created thread: creating server server running thread: serving three times test client 0 test client 1 test test_socketserver failed -- test failed 1 test failed: test_socketserver ^C ==End=Test=Output== The Ctrl-Break comes from the fact that the test hangs at that point without exiting. 'netstat -a' shows that there isn't anything using anything even close to those port numbers. I plan to do some more poking to try and narrow down a bit further what is going on before going to SF with it. (There are 5 different network connections on this laptop, including a virtual one to the Linux install, so I wouldn't be surprised if there is something odd going on with the network interface) > It also says: > > This is only a problem on multi-user machines that don't have restricted > logins, it is NOT a vulnerability from outside the machine. And it is easily > avoided by binding your server to the machine's address. > > If you're running a server on an open Windows machine, worrying about > details of socket reuse may not be the best use of your security time > . Indeed. I think it suggests that continuing to use SO_REUSEADDR normally on Windows is the best option for ayncore, though. Windows users just need to be aware that it doesn't mean quite the same thing as it does on a Unix-like system (actually, some of the other Google hits made me wonder if it really means the same thing in all the different Unix variants in the first place. . .) Regards, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From martin at v.loewis.de Sat Aug 14 09:48:45 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 14 09:48:52 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <411C85B6.2060104@egenix.com> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <16667.32457.28553.468008@montanaro.dyndns.org> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> <411C85B6.2060104@egenix.com> Message-ID: <411DC3DD.3080900@v.loewis.de> M.-A. Lemburg wrote: > If we switch the binding of 'yyy' to mean unicode('yyy') > some day, why can't we just continue to use the existing implementation > for 8-bit strings for b'xxx' (the current implementation is already > doing the right thing, meaning that it is 8-bit safe regardeless > of the source code encoding) ? Not exactly - the current implementation is not safe with respect to re-encoding source in a different encoding. Regards, Martin From dima at trit.org Sat Aug 14 12:31:50 2004 From: dima at trit.org (Dima Dorfman) Date: Sat Aug 14 12:34:33 2004 Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators) In-Reply-To: <200408111446.i7BEkHI27825@guido.python.org> References: <1092232661.411a25d55a567@mcherm.com> <200408111446.i7BEkHI27825@guido.python.org> Message-ID: <20040814103150.GR82909@trit.org> Guido van Rossum wrote: > Michael Chermside wrote: > > Others have already mentioned that "from sys import xxx, xxx, xxx" > > is the case that badly needs line wrapping. I would, however, like > > to suggest that (if it's easy to do) you allow a trailing comma when > > parentheses are used. [...] > > +1 Certainly easy to do (in fact, changing the grammar was almost sufficient). I've uploaded a new patch to SF, with the comment saying, in part, - Support trailing comma in from list (suggested by Michael Chermside on python-dev). This form is now acceptable: from os import (path,) but this is not: from os import path, I originally wanted to support the latter for consistency, but I don't think it's necessary (who would want to do that but not use parens?) and it's potentially confusing in a case like this: from sys import stdin, stdout which looks like it was intended to be one statement, but the second line is actually separate. The way the grammar is written, the case with a trailing comma but no parentheses parses okay, but it is intentionally rejected by the compiler. If we want to support that case for some reason, it's simple enough to remove the rejection test from the compiler. Dima. From skip at pobox.com Sat Aug 14 15:02:59 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Aug 14 15:03:07 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <411DC3DD.3080900@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <16667.32457.28553.468008@montanaro.dyndns.org> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> <411C85B6.2060104@egenix.com> <411DC3DD.3080900@v.loewis.de> Message-ID: <16670.3459.285216.939504@montanaro.dyndns.org> >> ... 8-bit strings for b'xxx' (the current implementation is already >> doing the right thing, meaning that it is 8-bit safe regardeless of >> the source code encoding) ? Martin> Not exactly - the current implementation is not safe with Martin> respect to re-encoding source in a different encoding. Can't such re-encoding tools (whatever they are) be intelligent about b"..."? If they are Python-aware that seems fairly trivial. If not, the job is more difficult. Skip From martin at v.loewis.de Sat Aug 14 17:21:27 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 14 17:21:33 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16670.3459.285216.939504@montanaro.dyndns.org> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <200408120414.i7C4Er429003@guido.python.org> <411AFBD4.5080903@v.loewis.de> <200408120545.i7C5j4W29250@guido.python.org> <411B0DA0.4030109@v.loewis.de> <16667.32457.28553.468008@montanaro.dyndns.org> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> <411C85B6.2060104@egenix.com> <411DC3DD.3080900@v.loewis.de> <16670.3459.285216.939504@montanaro.dyndns.org> Message-ID: <411E2DF7.3010705@v.loewis.de> Skip Montanaro wrote: > Can't such re-encoding tools (whatever they are) be intelligent about > b"..."? If they are Python-aware that seems fairly trivial. If not, the > job is more difficult. What precisely should they do, though? A byte sequence in one encoding might be invalid in another. Regards, Martin From tim.peters at gmail.com Sat Aug 14 18:15:35 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Aug 14 18:15:41 2004 Subject: [Python-Dev] 2.4 asyncore headache In-Reply-To: <411DC1E2.9080200@iinet.net.au> References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> <1f7befae04081313252cb3304a@mail.gmail.com> <411D4F27.3030605@iinet.net.au> <1f7befae04081318224503fb44@mail.gmail.com> <411DC1E2.9080200@iinet.net.au> Message-ID: <1f7befae04081409157340b56e@mail.gmail.com> [Tim] >> Which OS? asyncore isn't involved in that test, so the patch could >> not have affected it. SocketServer.py has its own, separate code >> using SO_REUSEADDR in some cases. [Nick Coghlan] > Ah, OK, it was just a thought. I'm on Windows XP SP1, Same here. > and it's consistently choking on the second attempt to test the UDP Server: Not here, but there are many ways to run the tests, and how you run them may also be important. I don't know of a way that "should" fail in this test, but I'll show you one way that works on my SP1 box: C:\Code\python\PCbuild>python ../lib/test/test_socketserver.py ADDR = ('localhost', 16401) CLASS = SocketServer.TCPServer server created server running thread: creating server thread: serving three times test client 0 test client 1 test client 2 thread: done waiting for server done ADDR = ('localhost', 16402) CLASS = SocketServer.ThreadingTCPServer server created server running thread: creating server thread: serving three times test client 0 test client 1 test client 2 thread: done waiting for server done ADDR = ('localhost', 16403) CLASS = SocketServer.UDPServer server created server running thread: creating server thread: serving three times test client 0 test client 1 test client 2 thread: done waiting for server done ADDR = ('localhost', 16404) CLASS = SocketServer.ThreadingUDPServer server created server running thread: creating server thread: serving three times test client 0 test client 1 test client 2 thread: done waiting for server done C:\Code\python\PCbuild> > ... > I plan to do some more poking to try and narrow down a bit further what > is going on before going to SF with it. (There are 5 different network > connections on this laptop, including a virtual one to the Linux > install, so I wouldn't be surprised if there is something odd going on > with the network interface) Neither would I . ... > Indeed. I think it suggests that continuing to use SO_REUSEADDR normally > on Windows is the best option for ayncore, though. "normally" is loaded: *normally* (by default), asyncore doesn't use SO_REUSEADDR at all, on any box. You have to call asyncore.dispatcher.set_reuse_addr() explicitly if you *want* to get into the reuse game (and, e.g., ZEO does). The only asyncore client in the Python core that does this is smtpd.SMTPServer. > Windows users just need to be aware that it doesn't mean quite the same > thing as it does on a Unix-like system In general, they aren't aware of such subtle distinctions -- and most would simply be annoyed if you insisted that they should be <0.7 wink>. > (actually, some of the other Google hits made me wonder if it really means > the same thing in all the different Unix variants in the first place. . .) Alas, portability across Unix variants is often spelled #ifdef. From jhylton at gmail.com Sat Aug 14 19:54:06 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Sat Aug 14 19:54:12 2004 Subject: [Python-Dev] Another 2.4 asyncore headache In-Reply-To: <1f7befae040813213728e50a44@mail.gmail.com> References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> <1f7befae040813213728e50a44@mail.gmail.com> Message-ID: On Sat, 14 Aug 2004 00:37:49 -0400, Tim Peters wrote: > More generally, in 2.3.4 it didn't hurt to leave all kinds of crap in > the socket map, provided that any such crap didn't call itself ready > to read or ready to write. select() would never see it then. In 2.4, > select() always sees it. That's a big change, particularly because > everyone I've seen wrestling with asyncore works via iterative > poke-and-hope, fiddling their code more-or-less randomly, until > mysterious errors stop appearing. > > So if it's of real value to you (I don't know the motivating use case > for the change) to pass stuff in the exception set, would it be good > enough to do > > e = r + w > > instead of (in effect) > > e = map.keys() > > ? Then stuff that doesn't call itself readable or writable would > again, as in 2.3.4, not get passed to select(). I don't understand the use case or the correct intended use of asyncore , but e = r + w looks appealing. It allows object to be in the socket map without requiring that they be ready to do some kind of I/O at any instant that select is called. Jeremy From bac at OCF.Berkeley.EDU Sun Aug 15 01:01:30 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 15 01:01:41 2004 Subject: [Python-Dev] Question about win32com in MS-Excel In-Reply-To: <000001c481b1$4acf2360$e7100a0a@jiangyu> References: <000001c481b1$4acf2360$e7100a0a@jiangyu> Message-ID: <411E99CA.5040403@ocf.berkeley.edu> Jiang Yu wrote: [SNIP] > Now, I have a question about use win32com moudle for MS-Excel. [SNIP] This is the wrong place to ask about win32com (and the win32all package), Jiang. This list is to discuss the development of Python the language and not any extension modules. Try comp.lang.python, the newsgroup, or the win32all mailing list. -Brett From hans at zephyrfalcon.org Sun Aug 15 04:26:09 2004 From: hans at zephyrfalcon.org (Hans Nowak) Date: Sun Aug 15 04:30:23 2004 Subject: [Python-Dev] Decorators and docstrings Message-ID: Hi, When playing with the decorators in 2.4a2, I saw the following behavior: # wraps an object with the function, and prints this object # when the function is called. >>> def wrapwith(obj): ... def decorator(f): ... def _wrapper(*args, **kwargs): ... print "##", obj ... return f(*args, **kwargs) ... return _wrapper ... return decorator ... >>> @wrapwith("hello") ... def foo(x, y): ... """ foo! """ ... return 2*x + y ... >>> foo(4, 5) ## hello 13 >>> foo.__doc__ >>> I.e. if you wrap the function foo with the decorator, it loses the docstring. foo.__doc__ is empty. Is this behavior intended? Should the decorator in question (in this case, 'wrapwith') take care of copying of the docstring itself? Or should the decorator mechanism do this automatically (if possible)? Thanks, -- Hans Nowak (hans@zephyrfalcon.org) http://zephyrfalcon.org/ From bob at redivi.com Sun Aug 15 04:43:30 2004 From: bob at redivi.com (Bob Ippolito) Date: Sun Aug 15 04:43:30 2004 Subject: [Python-Dev] Decorators and docstrings In-Reply-To: References: Message-ID: On Aug 14, 2004, at 10:26 PM, Hans Nowak wrote: > When playing with the decorators in 2.4a2, I saw the following > behavior: > > # wraps an object with the function, and prints this object > # when the function is called. > >>> def wrapwith(obj): > ... def decorator(f): > ... def _wrapper(*args, **kwargs): > ... print "##", obj > ... return f(*args, **kwargs) > ... return _wrapper > ... return decorator > ... > >>> @wrapwith("hello") > ... def foo(x, y): > ... """ foo! """ > ... return 2*x + y > ... > >>> foo(4, 5) > ## hello > 13 > >>> foo.__doc__ > >>> > > I.e. if you wrap the function foo with the decorator, it loses the > docstring. foo.__doc__ is empty. > > Is this behavior intended? Should the decorator in question (in this > case, 'wrapwith') take care of copying of the docstring itself? Or > should the decorator mechanism do this automatically (if possible)? Decorators aren't that special. If you want to return a function object that has the same doc string as the original then you're going to have to return the original object or explicitly make sure your new function object has the same doc string. _wrapper.__doc__ = decorator.__doc__ before the return should probably do it. I don't think that automatically setting a __doc__ string on the result object is a good idea at all because it could cause unnecessary problems for some result objects (instances of a class, for example). -bob From bac at OCF.Berkeley.EDU Sun Aug 15 08:09:09 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 15 08:09:17 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <200408110002.i7B02gu26415@guido.python.org> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> <200408101501.i7AF1HZ25725@guido.python.org> <200408110002.i7B02gu26415@guido.python.org> Message-ID: <411EFE05.7090507@ocf.berkeley.edu> Guido van Rossum wrote: >>Sooo should (for 'generator' in objects that claim to be in >>__builtins__ but aren't), >>1) 'generator' be added to __builtins__ >>2) 'generator' be added to types.py and its __module__ be set to 'types' >>3) 'generator' be added to .py and its __module__ be set to >>'' (and a name for the module chosen) > > > I guess (1). > The problem I see with explicitly adding this stuff to __builtins__ is that tab-completion in the interpreter is suddenly going to have all of this extra stuff that people are not going want to really see that often. The other point I would like to make is that almost everything in __builtins__ is either an exception, a singleton (thinking of True and False), or a function (even thinking of list and dict since you can use their factory methods). The only exceptions I can think of are __name__ and that is just part of the design. Throwing in generator and any of the other types that are defined by the built-in types will go against all of this unless we create factory methods for them which is not really desired since they are returned only in certain situations. I personally prefer option 2. -Brett From guido at python.org Sun Aug 15 08:33:13 2004 From: guido at python.org (Guido van Rossum) Date: Sun Aug 15 08:33:22 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: Your message of "Sat, 14 Aug 2004 23:09:09 PDT." <411EFE05.7090507@ocf.berkeley.edu> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> <200408101501.i7AF1HZ25725@guido.python.org> <200408110002.i7B02gu26415@guido.python.org> <411EFE05.7090507@ocf.berkeley.edu> Message-ID: <200408150633.i7F6XE203096@guido.python.org> > Guido van Rossum wrote: > > >>Sooo should (for 'generator' in objects that claim to be in > >>__builtins__ but aren't), > >>1) 'generator' be added to __builtins__ > >>2) 'generator' be added to types.py and its __module__ be set to 'types' > >>3) 'generator' be added to .py and its __module__ be set to > >>'' (and a name for the module chosen) > > > > I guess (1). [Note: it should be added to __builtin__, not __builtins__ -- the two are related but __builtin__ is the module and __builtins__ is a secret attribute that references the module or its __dict__ for purposes of sandboxing. [Brett C] > The problem I see with explicitly adding this stuff to __builtins__ > is that tab-completion in the interpreter is suddenly going to have > all of this extra stuff that people are not going want to really see > that often. There's already lots of stuff there that's rarely used. > The other point I would like to make is that almost everything in > __builtins__ is either an exception, a singleton (thinking of True > and False), or a function (even thinking of list and dict since you > can use their factory methods). But list and dict are *not* functions, they are types, and they can be used for subclassing and type checking too. These are the precedent for placing generator etc. in __builtin__. > The only exceptions I can think of are __name__ and that is just > part of the design. Throwing in generator and any of the other > types that are defined by the built-in types will go against all of > this unless we create factory methods for them which is not really > desired since they are returned only in certain situations. They should not be made factory functions if they aren't already. > I personally prefer option 2. Bah. The types module shouldn't be the "home" for any types -- it's just a collection of convenient aliases, and mostly they have the wrong names. It should be deprecated. Also note that at least 6 of these "anonymous" built-in types currently have another alias in the 'new' module, which means that they *are* factories already. It would be nice if that module could finally be deprecated. --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Sun Aug 15 09:22:18 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 15 09:22:35 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: <1092172040.1681.9.camel@localhost> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> Message-ID: <411F0F2A.2070709@ocf.berkeley.edu> Mark Russell wrote: [SNIP - stuff about the order] > I'll do a patch to fix the order and the corresponding tests. > I hope I am not stepping on Mark's toes but I went ahead and fixed it to be bottom-up like the PEP. Applied as rev. 2.318 for Python/compile.c and rev. 1.4 for Lib/test/test_decorators.py (rewrote the order test to not rely on causing an error to signify an error to but actually lead to a failure). I did not make any changes to to force decorators on to a single line, though. Mark can still have that one. =) -Brett From rnd at onego.ru Sun Aug 15 10:50:53 2004 From: rnd at onego.ru (Roman Suzi) Date: Sun Aug 15 10:49:55 2004 Subject: [Python-Dev] Decorators and docstrings In-Reply-To: References: Message-ID: On Sat, 14 Aug 2004, Bob Ippolito wrote: >On Aug 14, 2004, at 10:26 PM, Hans Nowak wrote: > >> When playing with the decorators in 2.4a2, I saw the following >> behavior: >> >> # wraps an object with the function, and prints this object >> # when the function is called. >> >>> def wrapwith(obj): >> ... def decorator(f): >> ... def _wrapper(*args, **kwargs): >> ... print "##", obj >> ... return f(*args, **kwargs) >> ... return _wrapper >> ... return decorator >> ... >> >>> @wrapwith("hello") >> ... def foo(x, y): >> ... """ foo! """ >> ... return 2*x + y >> ... >> >>> foo(4, 5) >> ## hello >> 13 >> >>> foo.__doc__ >> >>> >> >> I.e. if you wrap the function foo with the decorator, it loses the >> docstring. foo.__doc__ is empty. >> >> Is this behavior intended? Should the decorator in question (in this >> case, 'wrapwith') take care of copying of the docstring itself? Or >> should the decorator mechanism do this automatically (if possible)? > >Decorators aren't that special. If you want to return a function >object that has the same doc string as the original then you're going >to have to return the original object or explicitly make sure your new >function object has the same doc string. _wrapper.__doc__ = >decorator.__doc__ before the return should probably do it. > >I don't think that automatically setting a __doc__ string on the result >object is a good idea at all because it could cause unnecessary >problems for some result objects (instances of a class, for example). Probably a decorator could be written to facilitate writing docstring- and other such info saving decorators... def wrapwith(obj): @docsafe def decorator(f): def _wrapper(*args, **kwargs): print "##", obj return f(*args, **kwargs) return _wrapper return decorator >-bob >_______________________________________________ >Python-Dev mailing list >Python-Dev@python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: http://mail.python.org/mailman/options/python-dev/rnd%40onego.ru > Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From rnd at onego.ru Sun Aug 15 11:00:32 2004 From: rnd at onego.ru (Roman Suzi) Date: Sun Aug 15 10:59:34 2004 Subject: [Python-Dev] Decorators are function/method subclassing technique? Message-ID: After considering the docstring problem of decorators, it occured to me that decorators could be seen as an inheritance mechanism of functions. class docsafe(function): ... @ docsafe def f(x, y, z): """ To be saved """ ... print f <__main__.docsafe object at 0x401e074c> Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From marktrussell at btopenworld.com Sun Aug 15 17:27:27 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Sun Aug 15 17:27:31 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: <411F0F2A.2070709@ocf.berkeley.edu> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> Message-ID: <1092583647.1670.18.camel@localhost> On Sun, 2004-08-15 at 08:22, Brett C. wrote: > Mark Russell wrote: > > [SNIP - stuff about the order] > > I'll do a patch to fix the order and the corresponding tests. > > > > I hope I am not stepping on Mark's toes but I went ahead and fixed it to > be bottom-up like the PEP. Now that's what I call timing :-) I finished the patch to fix this yesterday and decided to leave it until today to upload (I've been swamped this week, hence the delay getting time to do this). I've uploaded it now (as patch #1009560) because I think it is a more correct fix (as well as also fixing the Lib/compiler package). It turns out that the order of evaluation of issue is more subtle than it first appears. Here's the comment from the start of the new test_eval_order() in test_decorators.py: # Evaluating a decorated function involves four steps for each # decorator-maker (the function that returns a decorator): # # 1: Evaluate the decorator-maker name # 2: Evaluate the decorator-maker arguments (if any) # 3: Call the decorator-maker to make a decorator # 4: Call the decorator # # When there are multiple decorators, these steps should be # performed in the above order for each decorator, but we should # iterate through the decorators in the reverse of the order # they appear in the source. Your patch results in the evaluation order: evalname1 evalargs1 makedec1 evalname2 evalargs2 makedec2 evalname3 evalargs3 makedec3 calldec3 calldec2 calldec1 Mine (#1009560) gives: evalname3 evalargs3 makedec3 calldec3 evalname2 evalargs2 makedec2 calldec2 evalname1 evalargs1 makedec1 calldec1 I would defend this as the correct order because it is the same as the order you get with the current method (func = staticmethod(func)). > I did not make any changes to to force decorators on to a single > line, though. Mark can still have that one. =) Thanks :-) My patch implements that change as well (plus doc and test updates to match). Mark From eppstein at ics.uci.edu Sun Aug 15 19:11:08 2004 From: eppstein at ics.uci.edu (David Eppstein) Date: Sun Aug 15 19:11:20 2004 Subject: [Python-Dev] Re: Decorator order implemented backwards? References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> Message-ID: In article <1092583647.1670.18.camel@localhost>, Mark Russell wrote: [Brett C's] > patch results in the evaluation order: > > evalname1 evalargs1 makedec1 > evalname2 evalargs2 makedec2 > evalname3 evalargs3 makedec3 > calldec3 calldec2 calldec1 > > Mine (#1009560) gives: > > evalname3 evalargs3 makedec3 calldec3 > evalname2 evalargs2 makedec2 calldec2 > evalname1 evalargs1 makedec1 calldec1 It would probably be bad style to have any order dependencies in the evalname evalargs makedec parts, but Brett's order makes more sense to me. That is, I like the actual application of decorators to be in nested order, but it makes more sense to me for the lines constructing the decorators to be evaluated in the order they appear in the code. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From marktrussell at btopenworld.com Sun Aug 15 21:46:03 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Sun Aug 15 21:46:06 2004 Subject: [Python-Dev] Re: Decorator order implemented backwards? In-Reply-To: References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> Message-ID: <1092599162.29945.51.camel@localhost> On Sun, 2004-08-15 at 18:11, David Eppstein wrote: > It would probably be bad style to have any order dependencies in the > evalname evalargs makedec parts, but Brett's order makes more sense to > me. That is, I like the actual application of decorators to be in nested > order, but it makes more sense to me for the lines constructing the > decorators to be evaluated in the order they appear in the code. On second thoughts I think you are right. I was concerned about the difficulty of explaining the two different evaluation orders, but that is outweighed by the non-intuitiveness of having decorator expressions evaluated in the opposite of source code order. I've updated the patch. Here's my current attempt at describing the semantics in the reference manual: A function definition may be wrapped by one or more decorator expressions. Decorator expressions are evaluated when the function is defined, in the scope that contains the function definition. The result must be a callable, which is invoked with the function object as the only argument. The returned value is bound to the function name instead of the function object. Multiple decorators are applied in nested fashion. For example, the following code: @f1(arg) @f2 def func(): pass is equivalent to: def func(): pass func = f1(arg)(f2(func)) Mark From bac at OCF.Berkeley.EDU Sun Aug 15 22:05:28 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 15 22:05:44 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: <1092583647.1670.18.camel@localhost> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> Message-ID: <411FC208.3040800@ocf.berkeley.edu> Mark Russell wrote: > On Sun, 2004-08-15 at 08:22, Brett C. wrote: > >>Mark Russell wrote: >> >>[SNIP - stuff about the order] >> >>>I'll do a patch to fix the order and the corresponding tests. >>> >> >>I hope I am not stepping on Mark's toes but I went ahead and fixed it to >>be bottom-up like the PEP. > > > Now that's what I call timing :-) I finished the patch to fix this > yesterday and decided to leave it until today to upload (I've been > swamped this week, hence the delay getting time to do this). I've > uploaded it now (as patch #1009560) because I think it is a more > correct fix (as well as also fixing the Lib/compiler package). > =) I kept doing cvs update, just waiting for someone to fix this right in the middle of me doing all of this. > It turns out that the order of evaluation of issue is more subtle than > it first appears. Here's the comment from the start of the new > test_eval_order() in test_decorators.py: > > # Evaluating a decorated function involves four steps for each > # decorator-maker (the function that returns a decorator): > # > # 1: Evaluate the decorator-maker name > # 2: Evaluate the decorator-maker arguments (if any) > # 3: Call the decorator-maker to make a decorator > # 4: Call the decorator > # > # When there are multiple decorators, these steps should be > # performed in the above order for each decorator, but we should > # iterate through the decorators in the reverse of the order > # they appear in the source. > > Your patch results in the evaluation order: > > evalname1 evalargs1 makedec1 > evalname2 evalargs2 makedec2 > evalname3 evalargs3 makedec3 > calldec3 calldec2 calldec1 > > Mine (#1009560) gives: > > evalname3 evalargs3 makedec3 calldec3 > evalname2 evalargs2 makedec2 calldec2 > evalname1 evalargs1 makedec1 calldec1 > > I would defend this as the correct order because it is the same as the > order you get with the current method (func = staticmethod(func)). > I am leaving this up to Guido (seems to have become the default PEP 318 champion) to make the decision on the order. Personally I like my order more since I would expect evaluation of arguments to be in the order listed, just the application order to be reversed. Either way I won't be able to deal with this patch if it is accepted; with the python-dev Summary coverage period ending today I still have 728 emails to summarize on top of any sent in today so I will be swamped with that for the rest of the week. I am sure someone else can step up and applying it if Guido okays the application order. -Brett From bac at OCF.Berkeley.EDU Sun Aug 15 22:09:48 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 15 22:09:54 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <200408150633.i7F6XE203096@guido.python.org> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> <200408101501.i7AF1HZ25725@guido.python.org> <200408110002.i7B02gu26415@guido.python.org> <411EFE05.7090507@ocf.berkeley.edu> <200408150633.i7F6XE203096@guido.python.org> Message-ID: <411FC30C.2000200@ocf.berkeley.edu> Guido van Rossum wrote: [SNIP] > Bah. The types module shouldn't be the "home" for any types -- it's > just a collection of convenient aliases, and mostly they have the > wrong names. It should be deprecated. > Well that settles that one. Didn't know you wanted to deprecate 'types'. Then __builtin__ does seem to be the only logical place. -Brett From mzarate at uoguelph.ca Fri Aug 13 14:20:22 2004 From: mzarate at uoguelph.ca (Martin Zarate) Date: Sun Aug 15 22:40:52 2004 Subject: [Python-Dev] block-based decorators and other block issues Message-ID: <1092399622.411cb206345b5@webmail.uoguelph.ca> Our discussion has taken an interesting turn, leading to this concept of "super-decorators" that are more versatile than normal decorators, because they can accept multiple objects, and because they can play with namespaces. First of all, it is important not to get too far ahead of ourselves - decorators should not be full "custom blocks" like having a "threadsafe" decorator block and a "fork function" block wherein the decoratotor is changing the fundamental behaviour of a block of code. Concepts that we see in Ruby (I highly, highly recommend that people interested in this subject read over the Ruby tutorial, it is very insightful on the concept of versatile block constructs) could be brought to Python, but it would have to be done properly through a good general-purpose mechanism. Decorators are simply a filtering mechanism - they take an object defined with the decorator, and then either a) return a substitute to the parent scope or, possibly b) throw it out. They are not blocking concepts. The reason is simple - speed. A decorator shouldn't have to perform fasttolocals and vice versa. It doesn't need to be its own variable namespace - it can sit within the external namespace. To do otherwise is suboptimal. The decorator should not have its own scope. Instead, it should provide a simple filter. Now, it would be nice if Python had another versatile block structure usable for custom stuff that does involve having its own full namespace (hell, just a "block" keyword would do) but I'll get to that later. First, I need a keyword. I will use "turn", just to keep things different, but anything will do - has has been remarked elsewhere, a naked tuple with a ":" suffix would be enough - it would differentiate the conversion block from standard comma-delimited lists, while not looking like any other block. At any rate, people advocating the "decorator block" concept that I suggested keep assuming that the descriptor functions are functions that take a whole namespace in and put a whole other namespace out, which is then imported into the parent namespace - horribly inefficient. Use that later for custom blocks. The call signature of a descriptor could simply be descriptor(object, name, holder, doc, args) where object is the object being described, name is the string with its name (immutable, as always) and holder is the current contents of the holder. The return value of the descriptor function is assigned into the holder at the end of the operation - if no change is desired and the nested objects are to be thrown out, then simply return the holder parameter. The name parameter exists for the sake of naming objects. With the holder parameter, inspection of the existing object is allowed, so that the "property_get" descriptor (which actually alters an existing property or creates a new one if none exists) can work. The reason for the longer function signature is so that more complicated syntax is available to the decorator function - such as a docstring that can be prepended to the nested docstrings. This seems silly for functions, but would be appropriate when decorating non-function objects (such as data members). Creating a new scope block for decorators would be too slow, but still would be very useful for more bizarre custom linguistic constructs. A general purpose "scope" command with access to external and internal namespaces would be extremely useful. Imagine the general purpose "scope" statement: scope handler1(args), handler2(args), handler3(args): where the contained code within the block would be run, then its local namespace harvested, as well as the external namespace, both passed in to handlerN(inner, outer, args) this would pretty much let you do whatever you want within a random scope: block, allowing for some bizzarro macroesque stuff that would make descriptors look simple by comparison. Personally I think such a structure would be silly and dangerous, but its what many coders sound like they want. A better, more general-purpose structure would be an "object" block like object name handler(args): wherein the contained code is run on the spot and the local namespace is converted into a dictionary, which is then called with the signature handler(namespace, name, args) where namespace is the namespace is the local namespace, name is the string of the name, and args is the argument tuple. You could use this to code your own analogs to the class structure of Python. Say you want a forced single- inheritence-tree class system? You could have one with such custom blocks, or any other alternate-class-structure. Hell, with such a system the class itself wouldn't need to be a true keyword as it is now, but instead could behave as simply just one more "object" expression handler - as I understand it, all it does is take that namespace and the argument tuple and do some fun stuff with it and then plonk that info (plus the name of the class) into the "type" constructor. That doesn't need to be interpreter level. So instead of class foo(bar baz): we get object foo class(bar baz): which could be substituted with any wierd thing you want instead of "class" - like a quick module object creator with none of the overhead of a class, for example. Or you could do the much coveted "instance" block that many people have clamored for, in which you subclass an instance instead of subclassing a class. Or lets say you're making a game engine and you need to make a separate class system for engine objects (the situation I was in, actually)and you want to be able to work with them as if they were classes, but also want it to inherit some invisible engine settings not apparent to the python interpreter from other engine objects, as well as instead of making some kinds of functions visible to the interpreter they're loaded into the event handler automatically (this is personal experience here - I wanted these features very badly). Or you could just make a quick dict maker def dictblock(namespace, name, args): return namespace; Like sort of the reverse of a "with" block. Take note that this system would be by far the best way to handle the property object, rather than mucking about with descriptors. Works as such... object foo propertyblock: def fget(self): return "bar" def fset(self): pass def fdel(self): del self.foo doc = "baz" (alternately, docstrings could be implemented like functions, filling the __doc__ entry of the dictionary) and implemented like... propertyblock(space, name, args): return property(space["fget"], space["fset"], space["fdel"], space["doc"]) You could do a nice shortcut involving passing the namespace dict into the keywords dict while calling "property", but that's the more legible way. The point is that this would be distinct from the descriptor block concept, as descriptors don't need to play with the namespaces beyond one object and one name at a time. But none of this explains why I like using a block for descriptors - I'm just trying to explain to those who thought of the possibilites of a block where you could play with the local and surrounding namespace how yes, that stuff would be useful, but would have a lot more overhead than simply a descriptor block. There are places a multi-line descriptor block is ugly, but there are also places where it is goregeous. Public and Private are good examples that C++ devs would love, rather than saying public and private for each function its turn private: foo = 1 bar = 2 baz = 3 Hell, if you really don't like the keyword, we could get even wierder and play with a new syntax: use the @ with a bracket and nest the block within Guido's syntax... @[static, private, functor]: def foo: pass Wow, I covered way, way too much ground with that post. Seriously people - read the Ruby tutorial "Programming Ruby" http://www.rubycentral.com/book/ to see the gorgeous way that Ruby handles the whole concept of blocks. Unfortunately the rest of the language is thoroughly illegible, but the block concept is cool. From gareth.mccaughan at pobox.com Sat Aug 14 03:18:36 2004 From: gareth.mccaughan at pobox.com (Gareth McCaughan) Date: Sun Aug 15 22:40:56 2004 Subject: [Python-Dev] Re: def ... decorate Message-ID: <200408140218.36912.gareth.mccaughan@pobox.com> Josiah Carlson wrote, in reply to me: > You seem to have missed the point of function decorations entirely. We > already have what you offer in current Python syntax. The point was to > move decorations to near/next to the function signature. No, that was only part of the point. That it's part of the point is why I still prefer the list-before-colon syntax, which seems to me to go furthest towards keeping all information about the interface of a function together. And yes, I know that Guido finds that syntax unacceptable. > Read the PEP: > http://www.python.org/peps/pep-0318.html I have read the PEP, thanks. Heck, I'm even named in it. (I think I was the first person to propose a generalized decoration facility for Python, rather than just special syntax for a few special cases such as static methods. And, for what it's worth, the list-before-colon syntax was my proposal. Not that it matters much who thought of what, but I do think it's a bit off to tell me that I've "completely missed the point" of something I *invented*.) >> I actually quite like this. It seems more Pythonic >> than the @foo proposal. Its obvious problem is that >> it involves something that looks at first glance like >> an ordinary suite of statements or expressions, but >> whose interpretation is substantially different. At >> least the @foo proposal avoids that. > > What you like is what has existed with Python since the beginning. No, it is not. What has existed with Python since the beginning involves redundancy: def f(...): ... f = decorate_somehow(f) requires three instances of the name of f. If you have N decorators it requires 2N+1 instances of the name of f. Not a big deal when the name is "f", but not so good if it's "build_interface_subgraph" or something. Not even great when the name is "f": what if you decide to rename it "g"? *And* it's ugly. These are reasons to want something else, even without the issue of putting all the "interface" information together. -- g From bob at redivi.com Sun Aug 15 22:59:35 2004 From: bob at redivi.com (Bob Ippolito) Date: Sun Aug 15 22:59:34 2004 Subject: [Python-Dev] block-based decorators and other block issues In-Reply-To: <1092399622.411cb206345b5@webmail.uoguelph.ca> References: <1092399622.411cb206345b5@webmail.uoguelph.ca> Message-ID: <034FE97C-EEFE-11D8-BF62-000A95686CD8@redivi.com> On Aug 13, 2004, at 8:20 AM, Martin Zarate wrote: > Our discussion has taken an interesting turn, leading to this concept > of "super-decorators" that are more versatile than normal decorators, > because > they can accept multiple objects, and because they can play with > namespaces. --- > The reason is simple - speed. A decorator shouldn't have to perform > fasttolocals and vice versa. It doesn't need to be its own variable > namespace - it can sit within the external namespace. To do otherwise > is > suboptimal. The decorator should not have its own scope. Instead, it > should > provide a simple filter. Ok, I agree with you throughout this post except for this "speed" business. Decorators are largely considered a compile-time feature and will almost exclusively be executed as module-level code. Each 'decorator statement' will likely only be executed once in its entire lifetime upon module import. I don't think speed is too much of a concern at this point and worrying about it definitely seems like one heck of a premature optimization. -bob From raymond.hettinger at verizon.net Sun Aug 15 23:41:26 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun Aug 15 23:42:40 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? Message-ID: <001501c48310$bf15aa40$e841fea9@oemcomputer> The documentation for the atexit module (introduced in Py2.0) states: """Note: This module is unlikely to work correctly when used with other code that sets sys.exitfunc. In particular, other core Python modules are free to use atexit without the programmer's knowledge. Authors who use sys.exitfunc should convert their code to use atexit instead. The simplest way to convert code that sets sys.exitfunc is to import atexit and register the function that had been bound to sys.exitfunc.""" Can we strengthen this by deprecating sys.exitfunc? The atexit module does attempt to co-exist by introducing code to register a function in sys.exitfunc if it were defined before "import atexit" was called. However, this is unreliable because it depends on import order and the library is free to introduce an earlier "import atexit" which would break code relying on the co-existance mechanism. Raymond From skip at pobox.com Mon Aug 16 00:49:21 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 00:49:23 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <001501c48310$bf15aa40$e841fea9@oemcomputer> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> Message-ID: <16671.59505.127669.502898@montanaro.dyndns.org> Raymond> Can we strengthen this by deprecating sys.exitfunc? What would you have the atexit module use? Skip From barry at python.org Mon Aug 16 01:44:13 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 16 01:43:51 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation In-Reply-To: References: Message-ID: <1092613453.8173.108.camel@localhost> On Fri, 2004-08-13 at 19:24, Bob Ippolito wrote: > My only problem with this syntax is that I expect the common cases > (mine, anyway) to be zero or one decorators per function, so the extra > block and indent seems a bit excessive compared to the current > @prefix-symbol-decorator proposal(s). It is, of course, far better > than typing the function name three times though! :) Same here. It trades one line at the end of the function for two (or more) before the beginning of the function. -0 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040815/23f479b1/attachment.pgp From python at rcn.com Mon Aug 16 01:57:18 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 16 01:57:38 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <16671.59505.127669.502898@montanaro.dyndns.org> Message-ID: <001d01c48323$99a2e4e0$e841fea9@oemcomputer> > Raymond> Can we strengthen this by deprecating sys.exitfunc? > > What would you have the atexit module use? Of course, it would have to stick around in some form accessible by atexit but not by everyone else. I'm suggesting deprecating it from the published API and possibly changing its name to an underscore or some such. Right now, the docs for sys.exitfunc mention that 'atexit' is a way to handle multiple exit functions, but it stops short of saying not to use sys.exitfunc at all. Raymond From ncoghlan at iinet.net.au Mon Aug 16 01:58:55 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Aug 16 01:58:59 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation In-Reply-To: <1092613453.8173.108.camel@localhost> References: <1092613453.8173.108.camel@localhost> Message-ID: <411FF8BF.70702@iinet.net.au> Barry Warsaw wrote: > On Fri, 2004-08-13 at 19:24, Bob Ippolito wrote: > > >>My only problem with this syntax is that I expect the common cases >>(mine, anyway) to be zero or one decorators per function, so the extra >>block and indent seems a bit excessive compared to the current >>@prefix-symbol-decorator proposal(s). It is, of course, far better >>than typing the function name three times though! :) > > > Same here. It trades one line at the end of the function for two (or > more) before the beginning of the function. > > -0 What about the variant which makes the block optional when there is only one decorator? i.e. decorate: staticmethod def foobar(): pass The other thing I like about this version is that a folding editor like PythonWin can easily compress a decorated definition down to two lines, while leaving it clear that the function *is* decorated. That's not a huge point, though. Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From python at rcn.com Mon Aug 16 02:09:13 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 16 02:09:33 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II Message-ID: <002001c48325$436d9460$e841fea9@oemcomputer> Aahz had suggested that the threading section of the tutorial's Standard Library Tour Part II be re-written with the idea of making people smarter about what Python threading can and cannot do, and about approaches most likely to assure success. Please comment on the proposed revision listed below. Raymond ------------------------------------------------------------------------ --- --------------- Multi-threading --------------- Threading is a technique for decoupling tasks which are not sequentially dependent and creating the illusion of concurrency. Threads can be used to improve the responsiveness of applications that accept user input while other tasks run in the background. The following code shows how the high level threading module can run tasks in background while the main program continues to run: import threading, zipfile class AsyncZip(threading.Thread): def __init__(self, infile, outfile): threading.Thread.__init__(self) self.infile = infile self.outfile = outfile def run(self): f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED) f.write(self.infile) f.close() print 'Finished background zip of: %s' % self.infile background = AsyncZip('mydata.txt', 'myarchive.zip') background.start() print 'The main program continues to run in foreground.' background.join() # Wait for the background task to finish print 'Main program waited until background was done.' The principal challenge of multi-thread applications is coordinating threads that share data or other resources. To that end, the threading module provides a number of synchronization primitives including locks, events, condition variables, and semaphores. While those tools are powerful, minor design errors can result in problems that are difficult to reproduce. Hence, the preferred approach to task coordination is to concentrate all access to a resource in a single thread and then use the Queue module to feed that thread with requests from other threads. Applications using Queue objects for inter-thread communication and coordination tend to be easier to design, more readable, and more reliable. All that being said, a few cautions are in order. Thread programming is difficult to get right. And, its overhead decreases total application performance. Also, multiple processors cannot boost performance because Python's Global Interpreter Lock (GIL) precludes more than one thread from running in the interpreter at the same time (this was done to simplify re-entrancy issues). Another issue is that threading doesn't work with the event driven model used by most GUIs. From barry at python.org Mon Aug 16 02:19:22 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 16 02:19:02 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation In-Reply-To: <411FF8BF.70702@iinet.net.au> References: <1092613453.8173.108.camel@localhost> <411FF8BF.70702@iinet.net.au> Message-ID: <1092615562.8192.168.camel@localhost> On Sun, 2004-08-15 at 19:58, Nick Coghlan wrote: > What about the variant which makes the block optional when there is only > one decorator? Better, but I'm still not crazy about it, mostly because of the new keyword. I think you'd have to add an __future__ to get it into Python 2.4. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040815/bcc452ac/attachment.pgp From greg at cosc.canterbury.ac.nz Mon Aug 16 03:09:31 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 16 03:10:00 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <200408150633.i7F6XE203096@guido.python.org> Message-ID: <200408160109.i7G19VHN019296@cosc353.cosc.canterbury.ac.nz> Guido: > But list and dict are *not* functions, they are types, and they can be > used for subclassing and type checking too. These are the precedent > for placing generator etc. in __builtin__. The real question seems to be whether __builtin__ should contain *all* built-in types, including internal ones, or only those intended for public use. Do you have an opinion about that, Guido? I suppose it's reasonable to put them all in __builtin__, since as you say, they can be useful for type checking even if you can't or rarely want to instantiate them yourself. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From pje at telecommunity.com Mon Aug 16 03:26:13 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 16 03:26:47 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation In-Reply-To: <1092615562.8192.168.camel@localhost> References: <411FF8BF.70702@iinet.net.au> <1092613453.8173.108.camel@localhost> <411FF8BF.70702@iinet.net.au> Message-ID: <5.1.1.6.0.20040815212353.02d49300@mail.telecommunity.com> At 08:19 PM 8/15/04 -0400, Barry Warsaw wrote: >On Sun, 2004-08-15 at 19:58, Nick Coghlan wrote: > > > What about the variant which makes the block optional when there is only > > one decorator? > >Better, but I'm still not crazy about it, mostly because of the new >keyword. I think you'd have to add an __future__ to get it into Python >2.4. A slight variation of the J4 syntax could allow a single-decorator variant to look like: def foo(cls,bar) @classmethod as: # code here versus this for multiple decorators: def foo(cls,bar) @classmethod @synchronized as: # code here Neither variant needs a new keyword or __future__. From tim.peters at gmail.com Mon Aug 16 03:44:19 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 16 03:44:25 2004 Subject: [Python-Dev] Another 2.4 asyncore headache In-Reply-To: References: <1f7befae040813125824acdb87@mail.gmail.com> <20040813200347.GB29067@rogue.amk.ca> <1f7befae040813213728e50a44@mail.gmail.com> Message-ID: <1f7befae04081518441fd2c797@mail.gmail.com> [Jeremy Hylton] > I don't understand the use case or the correct intended use of > asyncore , but > e = r + w > looks appealing. It allows object to be in the socket map without > requiring that they be ready to do some kind of I/O at any instant > that select is called. It's enough that the Zope tests pass again, but I don't know Andrew's intended use case. The patch certainly changed visible semantics in one of the major apps indulging major asyncore abuse. Playing along with asyncore's emergent design, all along it passed something X in the read set iff X.readable() said "yup", and likewise for the write set wrt X.writable(). If there's a use case for passing things in the exception set too, the backward-compatible way would have been to do that iff a new X.exceptionable() said "yup", with a default implementation that always said "no". That would have slowed asyncore some for everyone, but if the use case isn't strong enough to justify a slowdown, perhaps the use case isn't strong enough period. From tim.peters at gmail.com Mon Aug 16 04:05:00 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 16 04:05:11 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <002001c48325$436d9460$e841fea9@oemcomputer> References: <002001c48325$436d9460$e841fea9@oemcomputer> Message-ID: <1f7befae0408151905594f785c@mail.gmail.com> I agree it would help to say more about threading pragmatics. It's hard to say anything true, though . I'll just pick on this: [Raymond Hettinger] > ... > And, its overhead decreases total application performance. Also, > multiple processors cannot boost performance because Python's > Global Interpreter Lock (GIL) precludes more than one thread > from running in the interpreter at the same time (this was done to > simplify re-entrancy issues). CPython's threading was really designed to allow I/O to proceed in parallel with one thread of Python computation. If used for that purpose, it's not true that total app performance decreases, nor is it true that performance can't improve on a multi-CPU box. Performance can increase significantly in that case, and that's the prime use case for *practical* threading in Python (that's why the GIL is released around virtually all C-level I/O calls -- I/O concurrency is the use case Guido cared about). Given experience with Zope and ZEO, I'd also add about 10 pages explaining that trying to mix asyncore with threads is legally sufficient evidence of insanity <0.9 wink>. From tim.peters at gmail.com Mon Aug 16 04:50:25 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 16 04:50:27 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <20040812204431.GA31884@vicky.ecs.soton.ac.uk> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> <20040812204431.GA31884@vicky.ecs.soton.ac.uk> Message-ID: <1f7befae0408151950361f0cb4@mail.gmail.com> [Armin Rigo] > I don't remember if it was mentioned here, but maybe we'd better check > directly whether the C stack is too large rather than (or possibly in addition > to) using a limit on the number of Python iterations. How large is too large? The Windows way checks the stack for impending overflow directly. A long "stack check on Unix" thread here in 2000 fizzled out in the expected way: no two Unices appear to have anything in common <0.9 wink>. > This is not really ANSI C, but I can't think of a setting in which the > following trick would fail to be a good indication of the amount of stack > space consumed: save a pointer to a local variable "early", e.g. in > Py_Initialize(); then in any other function call, the distance between this > pointer and a pointer to some local equals the amount of stack space consumed > by Python minus a few bytes. It fails for threads, whose stacks may be anywhere in relation to the main thread's. So any gimmick like this has to have a thread-local "starting point". If "a stack" is always a contiguous slice of memory, then it can work. > If this sounds too much of a hack, No, but I'm not sure how much additional hackery it requires to make it work in all cases. > the (usual) recursion limit could be kept to limit nested Python frames; but all > C-level recursions (including eval_frame) could additionally use the above trick. > Its advantage is that it is an extremely fast check. The ways to do it that don't work are indeed extremely fast . > If the total stack size is difficult to know in advance, That's apparently the case across platforms -- and someimes the main-thread stack is much larger than secondary-thread stacks. The Windows check doesn't even try to know the current stack's size, but relies on MS-specific C exception extensions. > we can still use PyOS_CheckStack(), but more efficiently and less randomly > than now, by maintaining a "high tide" pointer that records how much stack we > are sure we have, and calling PyOS_CheckStack() only to attempt to push > the "high tide" mark further. Threads again complicate this. AFAICT, the only platform that defines USE_STACKCHECK now (and actually implements PyOS_CheckStack()) is 32-bit Windows using an MS compiler. I did boost the stack reservation for that platform in CVS, which can't solve it, but at least hides it better . > While I'm in a hacking mood, there might be a way to prevent PyErr_Clear() to > clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and > KeyboardInterrupt: for these exceptions, let's just store them away instead of > clearing them, and re-raise them at the next occasion. The "next occasion" > would have to be defined more precisely, but there is probably a way to ensure > that it will at least be before the next non-trivial opcode operation starts. > It would overwrite an exception set later, like those spurious KeyError we get > for dict lookups. It might be a better-than-nothing quick fix to all these > PyErr_Clear() all around the code base. Threads probably complicate that too. It's dreadful that serious problems can get transformed into bogus KeyErrors (in the test driver I posted, I'm not sure I spelled this out, but the key always *was* in the dict when a MemoryError got turned into a KeyError; we called a comparison function to begin with because the hash codes matched, and since these were mostly small-integer keys, they were identical to their hash codes -- so these keys were in the dicts, and the KeyErrors were lying about that). That's just nuts. I don't have spare cycles to give to it, though, so you'll have to save the world on your own again. From stephen at xemacs.org Mon Aug 16 05:38:30 2004 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon Aug 16 05:38:38 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <002001c48325$436d9460$e841fea9@oemcomputer> (Raymond Hettinger's message of "Sun, 15 Aug 2004 20:09:13 -0400") References: <002001c48325$436d9460$e841fea9@oemcomputer> Message-ID: <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Raymond" == Raymond Hettinger writes: Raymond> Threading is a technique for decoupling tasks which are Raymond> not sequentially dependent and creating the illusion of Raymond> concurrency. All simultaneity is an illusion, but the concurrency of Python threads is not. Raymond> Also, multiple processors cannot boost performance Raymond> because Python's Global Interpreter Lock (GIL) precludes Raymond> more than one thread from running in the interpreter at Raymond> the same time (this was done to simplify re-entrancy Raymond> issues). This confused me because "running" is ambiguous. How about something like: The Python interpreter itself is not fully reentrant, so threading is accomplished by interleaving the execution of Python code from different threads. Thus, use of multiple processors cannot boost performance of threaded Python code, because a single instance of the interpreter can only execute code on one processor at a time. (Season to taste, fix technical inaccuracies.) I see no reason to mention the GIL, an implementation detail, at all. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From bob at redivi.com Mon Aug 16 05:44:23 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 05:45:00 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> References: <002001c48325$436d9460$e841fea9@oemcomputer> <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> On Aug 15, 2004, at 11:38 PM, Stephen J. Turnbull wrote: >>>>>> "Raymond" == Raymond Hettinger writes: > > Raymond> Threading is a technique for decoupling tasks which are > Raymond> not sequentially dependent and creating the illusion of > Raymond> concurrency. > > All simultaneity is an illusion, but the concurrency of Python threads > is not. > > Raymond> Also, multiple processors cannot boost performance > Raymond> because Python's Global Interpreter Lock (GIL) precludes > Raymond> more than one thread from running in the interpreter at > Raymond> the same time (this was done to simplify re-entrancy > Raymond> issues). > > This confused me because "running" is ambiguous. How about something > like: > > The Python interpreter itself is not fully reentrant, so threading > is accomplished by interleaving the execution of Python code from > different threads. Thus, use of multiple processors cannot boost > performance of threaded Python code, because a single instance of > the interpreter can only execute code on one processor at a time. > > (Season to taste, fix technical inaccuracies.) I see no reason to > mention the GIL, an implementation detail, at all. I think it's worth mentioning. It's absolutely necessary to know all about the GIL when writing an extension that can take advantage of threading or embed Python correctly in a threaded application. -bob From jack at performancedrivers.com Mon Aug 16 05:45:43 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Mon Aug 16 05:45:49 2004 Subject: [Python-Dev] Re: Decorator order implemented backwards? In-Reply-To: <1092599162.29945.51.camel@localhost> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> <1092599162.29945.51.camel@localhost> Message-ID: <20040816034543.GR23725@performancedrivers.com> On Sun, Aug 15, 2004 at 08:46:03PM +0100, Mark Russell wrote: > Here's my current attempt at describing the semantics in the reference > manual: > > A function definition may be wrapped by one or more decorator > expressions. Decorator expressions are evaluated when the function > is defined, in the scope that contains the function definition. > The result must be a callable, which is invoked with the function > object as the only argument. The returned value is bound to the > function name instead of the function object. Multiple decorators > are applied in nested fashion. For example, the following code: > > @f1(arg) > @f2 > def func(): pass > > is equivalent to: > > def func(): pass > func = f1(arg)(f2(func)) Is func actually defined when f1 and f2 are called? Looking at the code the VAR_STORE doesn't happen until after the decorators are added. [but I'm no expert on bytecode]. This won't matter for sane decorators but we're talking about the strange cases already. Reading the @ code top to bottom you would not expect func to be defined, but if you want it to be strictly like the 'func = decorate(func)' translation is should be. My patch to add support for class decorators defines func before the decorators as a side effect of moving Mark's decorator code out of com_funcdef (and yes, that was a plug). -Jack From aahz at pythoncraft.com Mon Aug 16 05:48:17 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 16 05:48:20 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <002001c48325$436d9460$e841fea9@oemcomputer> References: <002001c48325$436d9460$e841fea9@oemcomputer> Message-ID: <20040816034817.GA25038@panix.com> On Sun, Aug 15, 2004, Raymond Hettinger wrote: > > All that being said, a few cautions are in order. Thread programming is > difficult to get right. And, its overhead decreases total application > performance. Also, multiple processors cannot boost performance because > Python's Global Interpreter Lock (GIL) precludes more than one thread > from running in the interpreter at the same time (this was done to > simplify re-entrancy issues). Another issue is that threading doesn't > work with the event driven model used by most GUIs. Python threading certainly does work with GUIs, and while the GIL adds some difficulty, the overall simplicity of Python's thread model makes it easier to get threaded GUI programs correct. See the Tkinter example from my OSCON slides. I'll also emphasize what Tim said: the GIL allows only one Python thread, but any external library can release the GIL to perform threaded operations in parallel. In particular, I/O libraries almost always release the GIL (I believe this includes mxODBC, for example). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From bob at redivi.com Mon Aug 16 05:55:05 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 05:55:40 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <20040816034817.GA25038@panix.com> References: <002001c48325$436d9460$e841fea9@oemcomputer> <20040816034817.GA25038@panix.com> Message-ID: <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> On Aug 15, 2004, at 11:48 PM, Aahz wrote: > On Sun, Aug 15, 2004, Raymond Hettinger wrote: >> >> All that being said, a few cautions are in order. Thread programming >> is >> difficult to get right. And, its overhead decreases total application >> performance. Also, multiple processors cannot boost performance >> because >> Python's Global Interpreter Lock (GIL) precludes more than one thread >> from running in the interpreter at the same time (this was done to >> simplify re-entrancy issues). Another issue is that threading doesn't >> work with the event driven model used by most GUIs. > > Python threading certainly does work with GUIs, and while the GIL adds > some difficulty, the overall simplicity of Python's thread model makes > it easier to get threaded GUI programs correct. See the Tkinter > example > from my OSCON slides. Many platform GUIs require that all or most all GUI activities be consolidated to one and only one thread. I don't think any of them 'care' if another thread is also running, but it can't communicate directly with the GUI. -bob From stephen at xemacs.org Mon Aug 16 05:58:27 2004 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon Aug 16 05:58:34 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> (Bob Ippolito's message of "Sun, 15 Aug 2004 23:44:23 -0400") References: <002001c48325$436d9460$e841fea9@oemcomputer> <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> Message-ID: <874qn33fa4.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Bob" == Bob Ippolito writes: Bob> I think it's worth mentioning. It's absolutely necessary to Bob> know all about the GIL when writing an extension that can Bob> take advantage of threading or embed Python correctly in a Bob> threaded application. Different audience, no? -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From aahz at pythoncraft.com Mon Aug 16 05:59:36 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 16 05:59:38 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> References: <002001c48325$436d9460$e841fea9@oemcomputer> <20040816034817.GA25038@panix.com> <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> Message-ID: <20040816035936.GB7737@panix.com> On Sun, Aug 15, 2004, Bob Ippolito wrote: > > Many platform GUIs require that all or most all GUI activities be > consolidated to one and only one thread. I don't think any of them > 'care' if another thread is also running, but it can't communicate > directly with the GUI. That's true, but it also applies to other external resources, such as files. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From bob at redivi.com Mon Aug 16 06:33:48 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 06:34:26 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <20040816035936.GB7737@panix.com> References: <002001c48325$436d9460$e841fea9@oemcomputer> <20040816034817.GA25038@panix.com> <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> <20040816035936.GB7737@panix.com> Message-ID: <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com> On Aug 15, 2004, at 11:59 PM, Aahz wrote: > On Sun, Aug 15, 2004, Bob Ippolito wrote: >> >> Many platform GUIs require that all or most all GUI activities be >> consolidated to one and only one thread. I don't think any of them >> 'care' if another thread is also running, but it can't communicate >> directly with the GUI. > > That's true, but it also applies to other external resources, such as > files. Maybe I don't understand what you're saying.. but I don't know any platform that forces you to work with files from one and only one specific thread. But I definitely do know at least one platform where you may only perform most GUI operations from the main thread of execution. -bob From foom at fuhm.net Mon Aug 16 06:38:06 2004 From: foom at fuhm.net (James Y Knight) Date: Mon Aug 16 06:38:19 2004 Subject: [Python-Dev] Re: Decorator order implemented backwards? In-Reply-To: <20040816034543.GR23725@performancedrivers.com> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> <1092599162.29945.51.camel@localhost> <20040816034543.GR23725@performancedrivers.com> Message-ID: <117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net> On Aug 15, 2004, at 11:45 PM, Jack Diederich wrote: > My patch to add support for class decorators defines func before the > decorators as a side effect of moving Mark's decorator code out of > com_funcdef (and yes, that was a plug). That's definitely worse. It causes a possibly incorrect temporary value to be bound, even if only for a short amount of time. The current behavior of only doing the store after executing the decorators is cleaner. I'd rewrite the translation as: """ The following code: @f1(arg) @f2 def func(): return True is nearly* equivalent to: func = f1(arg)(f2(lambda : True)) * However, you don't have the syntactical restrictions of a lambda, and the function name gets set to "func" instead of "". """ James From rnd at onego.ru Mon Aug 16 06:41:43 2004 From: rnd at onego.ru (Roman Suzi) Date: Mon Aug 16 06:40:45 2004 Subject: [Python-Dev] More concerns about decorators Message-ID: After playing with decorators alittle I found that @-syntax is nearly optimal especially that it comes before def, but there are other things to keep in mind. 1. Decorators must transfer __doc__-strings and probably even add information about functions 2. As now functions could have arbitrary attributes, some mechanism must be in place to ensure this information is not lost. If it is purely up to the decorator to decide, we will end up with undocumented functions... Is it possible to change function inplace? Is it even an option? Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From bob at redivi.com Mon Aug 16 06:52:43 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 06:53:18 2004 Subject: [Python-Dev] More concerns about decorators In-Reply-To: References: Message-ID: <1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com> On Aug 16, 2004, at 12:41 AM, Roman Suzi wrote: > > After playing with decorators alittle I found that @-syntax is nearly > optimal > especially that it comes before def, but there are other things to > keep in > mind. > > 1. Decorators must transfer __doc__-strings and probably even > add information about functions > > 2. As now functions could have arbitrary attributes, some > mechanism must be in place to ensure this information is not lost. > If it is purely up to the decorator to decide, we will end up with > undocumented functions... def carryover(new, orig): if orig.__doc__: new.__doc__ = orig.__doc__ if orig.__dict__: d = dict(orig.__dict__) d.update(new.__dict__) new.__dict__ = d return new A built-in function like this would make it easy to preserve such information... > Is it possible to change function inplace? Is it even an option? Not sure what you mean. Functions have a __doc__ slot and a __dict__ slot that are read/write. __name__ is read-only. I have no idea if that answers your question or not (there are other slots, that is not a definitive list, but should cover all of the relevant ones). -bob From foom at fuhm.net Mon Aug 16 07:11:48 2004 From: foom at fuhm.net (James Y Knight) Date: Mon Aug 16 07:11:54 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: <7FA48FA3-ECA6-11D8-BF5E-000A95A50FB2@fuhm.net> References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> <2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> <2macx31423.fsf@starship.python.net> <200408101501.i7AF1HZ25725@guido.python.org> <200408110002.i7B02gu26415@guido.python.org> <7FA48FA3-ECA6-11D8-BF5E-000A95A50FB2@fuhm.net> Message-ID: On Aug 12, 2004, at 5:28 PM, James Y Knight wrote: > Okay, so, I don't have a patch Submitted, http://www.python.org/sf/1009811 I'll note here, too, that I didn't add all the missing core types, only the ones that weren't iterators, descriptor internal stuff, or list comparison wrappers. For any of those that are at all possible to access from python code, I think they ought to show up somewhere, but I'm not convinced __builtin__ is the right place. Also the ones with "-" in their name need to be renamed. > I have made a script to examine the situation. Improved it slightly, and found another set of problems. Some modules have a function bound in place of the type: E.g. >>> select.poll() >>> isinstance(select.poll(), select.poll) TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types >>> select.poll >>> "D'oh!" Here's the list of these: select.poll not a type. mpz.mpz not a type. itertools.tee not a type. _csv.reader not a type. _csv.writer not a type. md5.md5 not a type. cPickle.Pickler not a type. cPickle.Unpickler not a type. mmap.mmap not a type. xreadlines.xreadlines not a type. James From python at rcn.com Mon Aug 16 07:29:52 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 16 07:30:20 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <1f7befae0408151905594f785c@mail.gmail.com> Message-ID: <000b01c48352$0ed1bd80$e841fea9@oemcomputer> > I agree it would help to say more about threading pragmatics. It's > hard to say anything true, though . After more thought, I'm leaving the existing tour mostly as-is and just making minor fixups such as expanding the example. The tour is supposed to provide a brief introduction and not try to cover all that is known. Instead, the Library Reference on threading should have extra sections on threading pragmatics (especially with respect to GUIs) and some non-toy examples to get someone started. Also, Extending and Embedding ought to cover the GIL, its significance, how to release/acquire, and the implications for multi-processors, etc. Several of the topics exceed my experience and some really need the collective experiences of various implementers, teachers, etc. So, it might be worthwhile to have a wiki to collect the useful thoughts and use that as a basis for improving the rest of the docs. The threading docs are rather like a car manual that presumes that you already know how to drive. Raymond From jack at performancedrivers.com Mon Aug 16 07:35:40 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Mon Aug 16 07:35:45 2004 Subject: [Python-Dev] Re: Decorator order implemented backwards? In-Reply-To: <117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> <1092599162.29945.51.camel@localhost> <20040816034543.GR23725@performancedrivers.com> <117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net> Message-ID: <20040816053540.GS23725@performancedrivers.com> On Mon, Aug 16, 2004 at 12:38:06AM -0400, James Y Knight wrote: > On Aug 15, 2004, at 11:45 PM, Jack Diederich wrote: > >My patch to add support for class decorators defines func before the > >decorators as a side effect of moving Mark's decorator code out of > >com_funcdef (and yes, that was a plug). > > That's definitely worse. It causes a possibly incorrect temporary value > to be bound, even if only for a short amount of time. The current > behavior of only doing the store after executing the decorators is > cleaner. I agree it would be unexpected if a function defined two lines down happens before the @mutate above it. > I'd rewrite the translation as: > """ > The following code: > @f1(arg) > @f2 > def func(): return True > > is nearly* equivalent to: > func = f1(arg)(f2(lambda : True)) > > * However, you don't have the syntactical restrictions of a lambda, and > the function name gets set to "func" instead of "". > """ Yes, the docs just need some kind of asterisk about the evaluation order. I was just pointing out a minor documentation bug. Mainly I just wanted to say "class decorators please" and to say I've submitted and would be happy to submit more patches to make it happen (for '@' or anything else). I currently write: class FooOverTimeReport(object): __metaclass__ = RunDaily # works OK unless you need more than one ... when I mean @run_daily class FooOverTimeReport(object): ... -Jack From jack at performancedrivers.com Mon Aug 16 07:42:58 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Mon Aug 16 07:43:03 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <000b01c48352$0ed1bd80$e841fea9@oemcomputer> References: <1f7befae0408151905594f785c@mail.gmail.com> <000b01c48352$0ed1bd80$e841fea9@oemcomputer> Message-ID: <20040816054258.GT23725@performancedrivers.com> On Mon, Aug 16, 2004 at 01:29:52AM -0400, Raymond Hettinger wrote: > > I agree it would help to say more about threading pragmatics. It's > > hard to say anything true, though . > > After more thought, I'm leaving the existing tour mostly as-is and just > making minor fixups such as expanding the example. The tour is supposed > to provide a brief introduction and not try to cover all that is known. > > Several of the topics exceed my experience and some really need the > collective experiences of various implementers, teachers, etc. So, it > might be worthwhile to have a wiki to collect the useful thoughts and > use that as a basis for improving the rest of the docs. One of many GUI examples, lyntin uses a Queue.Queue for synchronization between the tk/wx/whatever GUI thread, the telnet thread, and execution including interactive debugger goodness. http://sf.net/projects/lyntin/ -Jack From fumanchu at amor.org Mon Aug 16 09:48:31 2004 From: fumanchu at amor.org (Robert Brewer) Date: Mon Aug 16 09:53:46 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DFC@exchange.hqamor.amorhq.net> Barry Warsaw wrote: > On Sun, 2004-08-15 at 19:58, Nick Coghlan wrote: > > > What about the variant which makes the block optional when > > there is only one decorator? > > Better, but I'm still not crazy about it, mostly because of the new > keyword. I think you'd have to add an __future__ to get it > into Python 2.4. ...which Guido hasn't completely ruled out (unlike most of the other 'potential' alternatives): http://mail.python.org/pipermail/python-dev/2004-August/047001.html . Personally, I feel decorators are powerful enough to warrant a more structured change process like __future__; it seemed to work quite well for generators. Perhaps any new feature which ends in "-rators" warrants it. ;) Robert Brewer MIS Amor Ministries fumanchu@amor.org From gjc at inescporto.pt Mon Aug 16 12:35:33 2004 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Mon Aug 16 12:35:41 2004 Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called? Message-ID: <1092652532.1820.8.camel@emperor> Short version: Can I add a variable to python to indicate whether PyEval_InitThreads has been called? Something like: extern int PyEval_ThreadsInitialized; Motivation: I have discovered that, in pygtk, performance decreases a lot (50-90% higher exectution time) when we use python locking. We have an internal boolean flag in pygtk to indicate whether pygtk threading was enabled or not, thus avoiding the GIL locking functions in single threaded programs. The problem is that this flag is local to pygtk. If another module activates threading in python, we would like to know it in order to start acquiring the GIL. This should be a very simple patch, with zero impact in python core, but it will help tremendously. Is likely to be accepted for python 2.4? Regards. PS: I already asked python-help about this, sorry for the duplicate mails. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic From gjc at inescporto.pt Mon Aug 16 12:43:09 2004 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Mon Aug 16 12:43:23 2004 Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called? In-Reply-To: <1092652532.1820.8.camel@emperor> References: <1092652532.1820.8.camel@emperor> Message-ID: <1092652989.1814.13.camel@emperor> A Seg, 2004-08-16 ?s 11:35, Gustavo J. A. M. Carneiro escreveu: > Short version: > > Can I add a variable to python to indicate whether PyEval_InitThreads > has been called? Something like: > > extern int PyEval_ThreadsInitialized; > > Motivation: > > I have discovered that, in pygtk, performance decreases a lot (50-90% > higher exectution time) when we use python locking. We have an internal > boolean flag in pygtk to indicate whether pygtk threading was enabled or > not, thus avoiding the GIL locking functions in single threaded > programs. > > The problem is that this flag is local to pygtk. If another module > activates threading in python, we would like to know it in order to > start acquiring the GIL. > > This should be a very simple patch, with zero impact in python core, > but it will help tremendously. Is likely to be accepted for python 2.4? Forget it. Why not go one step further and make all locking operations in python no-ops if PyEval_InitThreads has never been called? Currently, we risk python crashing if we call such functions without calling PyEval_InitThreads. This is also trivial to implement. Regards. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic From Michael.Sparks at rd.bbc.co.uk Mon Aug 16 12:55:57 2004 From: Michael.Sparks at rd.bbc.co.uk (Michael Sparks) Date: Mon Aug 16 13:01:37 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation Message-ID: <200408161155.57554.Michael.Sparks@rd.bbc.co.uk> Nick Coghlan> [make block optional for one decorator ?] Barry Warsaw> [Better, but requires keyword, and hence (?) __future__ ] Robert Brewer wrote: > ...which Guido hasn't completely ruled out (unlike most of the other > 'potential' alternatives): > http://mail.python.org/pipermail/python-dev/2004-August/047001.html . It was (partly) reading this post that made me think it worthwhile implementing this syntax. I actually prefer keyword inside the function block, but I know that's ruled out, and can see why. As a result for me this is my second choice. Sure, I can live with the @pie syntax, but *personally* I prefer having a keyword. As previously noted the arguments against the length of the decorator in the single case, I view as advantages when showing code to new users or maintainers. As for whether it's the right keyword or not, I'm not sure I'm qualified to say, but decorators seem much more like declarations of function properties in all the use cases I've seen. > Personally, I feel decorators are powerful enough to warrant a more > structured change process like __future__; it seemed to work quite well > for generators. Perhaps any new feature which ends in "-rators" warrants > it. ;) As a circular use case, I realised that the generator implementation in test3.py from Armin Rigo's greenlets package would allow one to write: decorate: generator def some function(bla): for i in xrange(1): Yield(X) So you *could* argue the case for "swapping" keywords... ;-) (not serious :) More seriously though, if there is interest I'll see what I can do about cleaning up the patch to resolve the scoping issue. Also I just realised that I didn't introduce myself (as per list instructions...). I don't know if this is still in vogue or not, but here goes... I've been using python for 2 years now, and prior to that used perl/C/C++ for around 4 years as my main languages. I've worked on large scale network systems for most of my working life, but also worked on 3D graphics, and translation between a high level language to asynchronous hardware (handshake circuits) code generator. I've also contributed to a variety of F/OSS projects (in different ways!) in the past including squid, linux virtual servers, and Vorbis, largely because I think it's one of the nicest way of saying "thank you" :) I also like B5, Cordwainer Smith and I'm scared of spiders :) Regards, Michael. -- Michael Sparks, Senior R&D Engineer, Digital Media Group Michael.Sparks@rd.bbc.co.uk, British Broadcasting Corporation, Research and Development Kingswood Warren, Surrey KT20 6NP This e-mail may contain personal views which are not the views of the BBC. From arigo at tunes.org Mon Aug 16 13:29:16 2004 From: arigo at tunes.org (Armin Rigo) Date: Mon Aug 16 13:35:36 2004 Subject: [Python-Dev] Re: Another test_compiler mystery In-Reply-To: <1f7befae0408151950361f0cb4@mail.gmail.com> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> <20040812204431.GA31884@vicky.ecs.soton.ac.uk> <1f7befae0408151950361f0cb4@mail.gmail.com> Message-ID: <20040816112916.GA19969@vicky.ecs.soton.ac.uk> Hello Tim, On Sun, Aug 15, 2004 at 10:50:25PM -0400, Tim Peters wrote: > > While I'm in a hacking mood, there might be a way to prevent PyErr_Clear() > > clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and > > KeyboardInterrupt: (...) > > Threads probably complicate that too. It's dreadful that serious > problems can get transformed into bogus KeyErrors Yes. Here is a patch attempting to do what I described: http://www.python.org/sf/1009929 It's an extension of the asynchronous exception mecanism used to signal between threads. PyErr_Clear() can send some exceptions to its own thread using this mecanism. (So it is thread-safe.) Armin From arigo at tunes.org Mon Aug 16 13:33:57 2004 From: arigo at tunes.org (Armin Rigo) Date: Mon Aug 16 13:38:44 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <001501c48310$bf15aa40$e841fea9@oemcomputer> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> Message-ID: <20040816113357.GB19969@vicky.ecs.soton.ac.uk> Hello Raymond, On Sun, Aug 15, 2004 at 05:41:26PM -0400, Raymond Hettinger wrote: > The atexit module does attempt to co-exist by introducing code to > register a function in sys.exitfunc if it were defined before "import > atexit" was called. However, this is unreliable because it depends on > import order and the library is free to introduce an earlier "import > atexit" which would break code relying on the co-existance mechanism. Multiple well-written modules can use sys.exitfunc in a way that is compatible. atexit is merely a glue to discourage non-well-written modules. I'd say that atexit should be encouraged but there is nothing wrong with using sys.exitfunc directly, if one is aware of the common patching pattern: "save-old-value, replace-with-my-function, upon-call-invoke-saved-old-value". A bient?t, Armin. From mcherm at mcherm.com Mon Aug 16 14:37:47 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Aug 16 14:35:53 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint Message-ID: <1092659867.4120aa9ba1239@mcherm.com> > > > > if the only reason for it is to use isinstance? > > > > > > So that an extension author *could* write an int-like type deriving > > > from it? > > > > But didn't you just say that people shouldn't be > > deriving their own int-like types from baseinteger? > > Indeed, in general they shouldn't. But for specialized purposes it > might be needed (that's why I emphasized *could*). I call YAGNI. We're talking about creating the class baseinteger which might be useful ONLY for people creating new kinds of integers in Python which will NOT extend int or long but WILL need to be treated just like integers. Who is really likely to do that? And if in the process we introduce a new class which won't be needed in the long run (ie Python 3000 has just one type, called "int" and has no need for baseinteger). So I maintain that it's not needed (and is, in fact, confusing to users) unless someone has a real use case. -- Michael Chermside From mwh at python.net Mon Aug 16 14:40:50 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 16 14:40:52 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules readline.c, 2.72, 2.73 In-Reply-To: (montanaro@users.sourceforge.net's message of "Sun, 15 Aug 2004 07:32:09 -0700") References: Message-ID: <2md61rcl2l.fsf@starship.python.net> montanaro@users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Modules > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28578/Modules > > Modified Files: > readline.c > Log Message: > Add get_history_item and replace_history_item functions to the readline > module. Closes patch #675551. My apologies to Michal Vitecek for taking so > long to process this. A few small quibbles... > > Index: readline.c > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Modules/readline.c,v > retrieving revision 2.72 > retrieving revision 2.73 > diff -C2 -d -r2.72 -r2.73 > *** readline.c 8 Jul 2004 15:28:18 -0000 2.72 > --- readline.c 15 Aug 2004 14:31:57 -0000 2.73 > *************** > *** 295,298 **** > --- 295,363 ---- > set the readline word delimiters for tab-completion"); > > + static PyObject * > + py_remove_history(PyObject *self, PyObject *args) > + { > + int entry_number; > + HIST_ENTRY *entry; > + > + if (!PyArg_ParseTuple(args, "i:remove_history", &entry_number)) > + return NULL; > + entry = remove_history(entry_number); > + if (!entry) { > + char buf[80]; > + PyOS_snprintf(buf, sizeof(buf), > + "No history item at position %i", > + entry_number); > + PyErr_SetString(PyExc_ValueError, buf); > + return NULL; > + } What's wrong with PyErr_Format()? > + /* free memory allocated for the history entry */ > + if (entry->line) > + free(entry->line); > + if (entry->data) > + free(entry->data); > + free(entry); > + > + Py_INCREF(Py_None); > + return Py_None; > + } > + > + PyDoc_STRVAR(doc_remove_history, > + "remove_history(pos) -> None\n\ ^^^^^^^^^^^^^^ This isn't the name in the PyMethodDef at the end... [snippity] The same comments apply to the other function, too. > /* Add a line to the history buffer */ > *************** > *** 494,497 **** > --- 559,564 ---- > METH_VARARGS, doc_set_completer_delims}, > {"add_history", py_add_history, METH_VARARGS, doc_add_history}, > + {"remove_history_item", py_remove_history, METH_VARARGS, doc_remove_history}, ^^^^^^^^^^^^^^^^^^^ > + {"replace_history_item", py_replace_history, METH_VARARGS, doc_replace_history}, > {"get_completer_delims", get_completer_delims, > METH_NOARGS, doc_get_completer_delims}, Cheers, mwh -- it's like a bicycle but with internet -- from Twisted.Quotes From mwh at python.net Mon Aug 16 15:05:38 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 16 15:05:40 2004 Subject: [Python-Dev] More concerns about decorators In-Reply-To: <1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com> (Bob Ippolito's message of "Mon, 16 Aug 2004 00:52:43 -0400") References: <1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com> Message-ID: <2m8ycfcjx9.fsf@starship.python.net> Bob Ippolito writes: >> Is it possible to change function inplace? Is it even an option? > > Not sure what you mean. Functions have a __doc__ slot and a __dict__ > slot that are read/write. __name__ is read-only. __name__ is writable as of a few days ago. Cheers, mwh -- I believe C++ instills fear in programmers, fear that the interaction of some details causes unpredictable results. -- Erik Naggum, comp.lang.lisp From fumanchu at amor.org Mon Aug 16 17:19:10 2004 From: fumanchu at amor.org (Robert Brewer) Date: Mon Aug 16 17:24:33 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DFD@exchange.hqamor.amorhq.net> Michael Sparks wrote: > Nick Coghlan> [make block optional for one decorator ?] > Barry Warsaw> [Better, but requires keyword, and hence (?) > __future__ ] > > As previously noted the arguments against the length of the > decorator in the > single case, I view as advantages when showing code to new users or > maintainers. If you allow the keyword and the single decorator on the same line, this argument comes down to "how long is the keyword?" >>> try: a = 3 ... except Exception, x: ... print "Error", x Nobody seems to have a problem typing "try:" again and again. I'm not sure why "decorate:" is receiving such pessimistic scrutiny. But then, I wasn't hanging about when try/except came into being--perhaps it had similar difficulties. > As for whether it's the right keyword or not, I'm not sure > I'm qualified to > say, but decorators seem much more like declarations of > function properties > in all the use cases I've seen. The proposal, in effect, creates a declarative suite as opposed to the more traditional imperative suite. Perhaps "declare:" would be more appropriate (but give your brain a week to mull it over ;). > More seriously though, if there is interest I'll see what I > can do about cleaning up the patch to resolve the scoping issue. I'm certainly interested; thanks for churning out an implementation. Robert Brewer MIS Amor Ministries fumanchu@amor.org From anthony at interlink.com.au Mon Aug 16 17:31:11 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 16 17:31:45 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com> References: <002001c48325$436d9460$e841fea9@oemcomputer> <20040816034817.GA25038@panix.com> <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> <20040816035936.GB7737@panix.com> <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com> Message-ID: <4120D33F.8080900@interlink.com.au> Bob Ippolito wrote: > > Maybe I don't understand what you're saying.. but I don't know any > platform that forces you to work with files from one and only one > specific thread. But I definitely do know at least one platform where > you may only perform most GUI operations from the main thread of execution. There's a number of GUIs in this boat. It's not too much hassle - you just have to make sure you handle events to and from the GUI correctly through some sort of intermediary that posts the events to the correct thread. An example of this is wx and twisted - the only sane path is to run each in it's own thread. wx also won't work unless the "main" thread is the one running the GUI. all-guis-suck, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From pje at telecommunity.com Mon Aug 16 17:38:58 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 16 17:39:37 2004 Subject: [Python-Dev] Re: Decorator order implemented backwards? In-Reply-To: <117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net> References: <20040816034543.GR23725@performancedrivers.com> <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> <1092599162.29945.51.camel@localhost> <20040816034543.GR23725@performancedrivers.com> Message-ID: <5.1.1.6.0.20040816113536.01ee7260@mail.telecommunity.com> At 12:38 AM 8/16/04 -0400, James Y Knight wrote: >On Aug 15, 2004, at 11:45 PM, Jack Diederich wrote: >>My patch to add support for class decorators defines func before the >>decorators as a side effect of moving Mark's decorator code out of >>com_funcdef (and yes, that was a plug). > >That's definitely worse. It causes a possibly incorrect temporary value to >be bound, even if only for a short amount of time. The current behavior of >only doing the store after executing the decorators is cleaner. Not only that, it's the behavior *required* by the PEP. Note that it states "*without* the intermediate assignment to the variable func". (Emphasis added.) This choice of semantics is intentional, to allow for things like Ed Loper's property_getter/property_setter and generic functions to be possible. That is, it's intended to allow for decorators that construct an object from multiple function definitions. Such decorators need to be able to get the previous object bound to that name, and that's not possible if the new function has been saved under that name. There should probably be some kind of test added to the test suite to verify this behavior, and adding an explanation to the PEP is probably in order as well. From bob at redivi.com Mon Aug 16 17:44:37 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 17:45:30 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <4120D33F.8080900@interlink.com.au> References: <002001c48325$436d9460$e841fea9@oemcomputer> <20040816034817.GA25038@panix.com> <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> <20040816035936.GB7737@panix.com> <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com> <4120D33F.8080900@interlink.com.au> Message-ID: <2D8666C4-EF9B-11D8-A746-000A95686CD8@redivi.com> On Aug 16, 2004, at 11:31 AM, Anthony Baxter wrote: > Bob Ippolito wrote: >> Maybe I don't understand what you're saying.. but I don't know any >> platform that forces you to work with files from one and only one >> specific thread. But I definitely do know at least one platform >> where you may only perform most GUI operations from the main thread >> of execution. > > There's a number of GUIs in this boat. It's not too much hassle - you > just have to make sure you handle events to and from the GUI correctly > through some sort of intermediary that posts the events to the correct > thread. An example of this is wx and twisted - the only sane path is > to run each in it's own thread. wx also won't work unless the "main" > thread is the one running the GUI. But in those cases, Twisted only needs the mainloop to do two things: fire timers, watch file descriptors. It's possible to write a reactor for many or most GUIs that does that while still remaining single threaded (as far as Python is concerned). For GUIs that don't natively "watch sockets" as part of their runloop, you can start a thread that does that and only that and send asynchronous notifications to the GUI mainloop (which is what Cocoa/CoreFoundation does behind your back with their networking APIs). -bob From anthony at interlink.com.au Mon Aug 16 17:52:45 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 16 17:52:56 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <2D8666C4-EF9B-11D8-A746-000A95686CD8@redivi.com> References: <002001c48325$436d9460$e841fea9@oemcomputer> <20040816034817.GA25038@panix.com> <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> <20040816035936.GB7737@panix.com> <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com> <4120D33F.8080900@interlink.com.au> <2D8666C4-EF9B-11D8-A746-000A95686CD8@redivi.com> Message-ID: <4120D84D.70703@interlink.com.au> Bob Ippolito wrote: > But in those cases, Twisted only needs the mainloop to do two things: > fire timers, watch file descriptors. It's possible to write a reactor > for many or most GUIs that does that while still remaining single > threaded (as far as Python is concerned). For GUIs that don't natively > "watch sockets" as part of their runloop, you can start a thread that > does that and only that and send asynchronous notifications to the GUI > mainloop (which is what Cocoa/CoreFoundation does behind your back with > their networking APIs). Unfortunately in the wx case, the timer granularity sucks. The wx docs guarantee "no worse than 1s granularity". It's not _that_ bad, but on Windows at least, it's pretty terrible - at best, only 20-30Hz. I'm depressingly sure that there's other GUI toolkits with similar levels of suck. (Short (off-topic) summary - there's two ways to make the twisted and wx event loops cooperate - either the wx event loop runs everything, and fires timers to let the twisted event loop do it's thing - see above. The second is to let the twisted event loop regularly call the wx event loop to let it iterate any outstanding events. When a popup or menu comes up, wx creates a new sub-event-loop and takes control until that's dismissed... so we're back to needing to run the two event loops alongside each other. And yes, Tim's comment about insanity resulting from using asyncore and threads together also apply to twisted - the only sane way to do it is to have all the event loop stuff handled in the same thread. Madness follows, otherwise.) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From rnd at onego.ru Mon Aug 16 17:39:54 2004 From: rnd at onego.ru (Roman Suzi) Date: Mon Aug 16 18:04:17 2004 Subject: [Python-Dev] More concerns about decorators In-Reply-To: <2m8ycfcjx9.fsf@starship.python.net> References: <1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com> <2m8ycfcjx9.fsf@starship.python.net> Message-ID: On Mon, 16 Aug 2004, Michael Hudson wrote: >Bob Ippolito writes: > >>> Is it possible to change function inplace? Is it even an option? >> >> Not sure what you mean. Functions have a __doc__ slot and a __dict__ >> slot that are read/write. __name__ is read-only. > >__name__ is writable as of a few days ago. I mean that decorators are... decorators. They by their meaning must not change the function but add some twists to their behaviour. Many of decorator examples are destructing function attributes. Good example of non-destructing decorator is staticmethod: >>> class A: ... def m(x): ... """I am method m""" ... m.__security__ = "low" ... m = staticmethod(m) ... >>> A.m.__doc__ 'I am method m' >>> A.m.__security__ 'low' So, my concern was that it requires additional efforts to save function attributes. My thought was that probably decorators could work directly on the function, not changing id of the function? (staticmethod changes id, of course). Why? Because the function is not accessible any more. while the old way it was possible to store intermediate result: class A: def m(x): """I am method m""" mm = m m = staticmethod(m) And use it somehow. But with decorator function changes. Yes, it is possible that some decorators could store function internally for the purpose of calling it, but others will not. The transfer of function attributes is on the developer. And he/she could be clueless that docstrings, etc, could be lost in transformation. Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From skip at pobox.com Mon Aug 16 17:51:06 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 18:09:59 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX Message-ID: <16672.55274.689761.740267@montanaro.dyndns.org> I think I might have reported this previously (long time ago), but I don't recall if it was ever resolved. (Apparently not, since I'm still seeing it.) I'm seeing a test failure in test_tempfile on Mac OSX (I'm running 10.2.8/Jaguar) with a unix (not framework) build. I've whittled the tests down to sundry, tcl and tempfile. Here's the full run: % ./python.exe Lib/test/regrtest.py -f testrun test_sundry test_tcl test_tempfile *** malloc[17153]: error for object 0x2d70808: Pointer being reallocated was not allocated [5999 refs] test test_tempfile failed -- Traceback (most recent call last): File "/Users/skip/tmp/python/dist/src/Lib/test/test_tempfile.py", line 155, in test_wanted_dirs os.environ[envname] = os.path.abspath(envname) File "/Users/skip/tmp/python/dist/src/Lib/os.py", line 447, in __setitem__ putenv(key, item) OSError: (0, 'Error') 2 tests OK. 1 test failed: test_tempfile [39478 refs] This is a debug build linked with /usr/lib/libMallocDebug.a. Running under gdb's control with a breakpoint in malloc_printf yields this traceback info: (gdb) bt 10 #0 0x90070ddc in malloc_printf () #1 0x9000de5c in szone_realloc () #2 0x9000dd70 in malloc_zone_realloc () #3 0x9000dce8 in realloc () #4 0x90010c24 in setenv () #5 0x90010a74 in putenv () #6 0x00131018 in posix_putenv (self=0x0, args=0x12ac678) at Modules/posixmodule.c:5695 #7 0x000db4fc in PyCFunction_Call (func=0x3e25f8, arg=0x12ac678, kw=0x0) at Objects/methodobject.c:73 #8 0x000ce2f4 in call_function (pp_stack=0xbfff94f0, oparg=2) at Python/ceval.c:3566 #9 0x000c8624 in PyEval_EvalFrame (f=0xa05108) at Python/ceval.c:2170 ... In frame 6, putenv() is being called: (gdb) fr 6 #6 0x00131018 in posix_putenv (self=0x0, args=0x12ac678) at Modules/posixmodule.c:5695 5695 if (putenv(new)) { (gdb) p new $1 = 0x109a5e4 "TMPDIR=/Users/skip/tmp/python/dist/src/TMPDIR" This looks reasonable to me, so it doesn't look like the problem is within Python. Should I just skip the tempfile test on Mac OSX? Can someone check this out on 10.3/Panther? Skip From skip at pobox.com Mon Aug 16 18:00:22 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 18:10:05 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules readline.c, 2.72, 2.73 In-Reply-To: <2md61rcl2l.fsf@starship.python.net> References: <2md61rcl2l.fsf@starship.python.net> Message-ID: <16672.55830.787676.951897@montanaro.dyndns.org> Michael> A few small quibbles... >> + PyErr_SetString(PyExc_ValueError, buf); Michael> What's wrong with PyErr_Format()? Probably nothing. Michal's original patch was against 2.2.something. I don't know if that makes a difference or not. I'll fix it. >> + PyDoc_STRVAR(doc_remove_history, >> + "remove_history(pos) -> None\n\ Michael> ^^^^^^^^^^^^^^ This isn't the name in the PyMethodDef at the Michael> end... That too. Michal had originally called them remove_history and replace_history. I added "_item" to the end of each so they were structurally more like the preexisting get_history_item, then conveniently missed the doc strings. Skip From mwh at python.net Mon Aug 16 18:19:15 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 16 18:19:17 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules readline.c, 2.72, 2.73 In-Reply-To: <16672.55830.787676.951897@montanaro.dyndns.org> (Skip Montanaro's message of "Mon, 16 Aug 2004 11:00:22 -0500") References: <2md61rcl2l.fsf@starship.python.net> <16672.55830.787676.951897@montanaro.dyndns.org> Message-ID: <2mzn4vawe4.fsf@starship.python.net> Skip Montanaro writes: > Michael> A few small quibbles... > >> + PyErr_SetString(PyExc_ValueError, buf); > > Michael> What's wrong with PyErr_Format()? > > Probably nothing. Michal's original patch was against 2.2.something. I > don't know if that makes a difference or not. I'll fix it. Thanks. > >> + PyDoc_STRVAR(doc_remove_history, > >> + "remove_history(pos) -> None\n\ > Michael> ^^^^^^^^^^^^^^ This isn't the name in the PyMethodDef at the > Michael> end... > > That too. Michal had originally called them remove_history and > replace_history. I added "_item" to the end of each so they were > structurally more like the preexisting get_history_item, then conveniently > missed the doc strings. Fast work! Cheers, mwh -- Never meddle in the affairs of NT. It is slow to boot and quick to crash. -- Stephen Harris -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From bob at redivi.com Mon Aug 16 18:19:09 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 18:19:50 2004 Subject: [Python-Dev] More concerns about decorators In-Reply-To: References: <1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com> <2m8ycfcjx9.fsf@starship.python.net> Message-ID: <008A3A96-EFA0-11D8-A746-000A95686CD8@redivi.com> On Aug 16, 2004, at 11:39 AM, Roman Suzi wrote: > On Mon, 16 Aug 2004, Michael Hudson wrote: > >> Bob Ippolito writes: >> >>>> Is it possible to change function inplace? Is it even an option? >>> >>> Not sure what you mean. Functions have a __doc__ slot and a __dict__ >>> slot that are read/write. __name__ is read-only. >> >> __name__ is writable as of a few days ago. > > I mean that decorators are... decorators. They by their meaning must > not > change the function but add some twists to their behaviour. Many of > decorator > examples are destructing function attributes. Some decorators are "decorators", other decorators are transformations :) staticmethod and classmethod certainly aren't "decorators" as they change what the function does and return a different object, from an implementation standpoint anyway. > Good example of non-destructing decorator is staticmethod: Not all decorators are non-destructive. > So, my concern was that it requires additional efforts to save > function attributes. My thought was that probably decorators could > work directly on the function, not changing id of the function? > (staticmethod changes id, of course). Why? Because the function > is not accessible any more. while the old way it was possible to store > intermediate result: Most, if not all, useful decorators will need to change the id of the function. Even a decorator that optimizes bytecode would have to return a new object. The only things you can do to a function without changing its identity (IIRC) are changing the slots __name__, __doc__, and __dict__. > class A: > def m(x): > """I am method m""" > mm = m > m = staticmethod(m) > > > And use it somehow. But with decorator function changes. Yes, it is > possible > that some decorators could store function internally for the purpose of > calling it, but others will not. I think that most decorators are going to store the function object one way or another. I'm not sure why that matters though? I've never seen production code that uses this pattern (save out an instance method and a staticmethod of separate names), except for properties (which is somewhat different). > The transfer of function attributes is on the developer. And he/she > could be > clueless that docstrings, etc, could be lost in transformation. That already happens if you do decoration "manually" the pre-2.4 way. As long as its documented, it shouldn't be a problem. A "carryover" function as I offered up in my last email (modified to also bring over __name__) would help considerably, I think. -bob From bob at redivi.com Mon Aug 16 18:47:06 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 18:47:16 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16672.55274.689761.740267@montanaro.dyndns.org> References: <16672.55274.689761.740267@montanaro.dyndns.org> Message-ID: On Aug 16, 2004, at 11:51 AM, Skip Montanaro wrote: > > I think I might have reported this previously (long time ago), but I > don't > recall if it was ever resolved. (Apparently not, since I'm still > seeing > it.) > > I'm seeing a test failure in test_tempfile on Mac OSX (I'm running > 10.2.8/Jaguar) with a unix (not framework) build. I've whittled the > tests > down to sundry, tcl and tempfile. Here's the full run: -- > This looks reasonable to me, so it doesn't look like the problem is > within > Python. Should I just skip the tempfile test on Mac OSX? Can someone > check > this out on 10.3/Panther? % sw_vers ProductName: Mac OS X ProductVersion: 10.3.5 BuildVersion: 7M34 % mkdir _framework % cd _framework % ../configure --enable-framework % make % setenv DYLD_FRAMEWORK_PATH . % ./python.exe ../Lib/test/regrtest.py -f testrun test_sundry test_tcl test_tempfile All 3 tests OK. -bob From mwh at python.net Mon Aug 16 18:52:53 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 16 18:52:56 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16672.55274.689761.740267@montanaro.dyndns.org> (Skip Montanaro's message of "Mon, 16 Aug 2004 10:51:06 -0500") References: <16672.55274.689761.740267@montanaro.dyndns.org> Message-ID: <2mvffjauu2.fsf@starship.python.net> Skip Montanaro writes: > I think I might have reported this previously (long time ago), but I don't > recall if it was ever resolved. (Apparently not, since I'm still seeing > it.) > > I'm seeing a test failure in test_tempfile on Mac OSX (I'm running > 10.2.8/Jaguar) with a unix (not framework) build. I've whittled the tests > down to sundry, tcl and tempfile. Here's the full run: It's odd that test_tcl is on the list. Where is your tcl from? Fink? Even so, it's hard to see how it could be interfering. I don't see this behaviour on a similar system, but then I don't have tcl installed. Cheers, mwh -- All obscurity will buy you is time enough to contract venereal diseases. -- Tim Peters, python-dev From michel at dialnetwork.com Mon Aug 16 13:35:06 2004 From: michel at dialnetwork.com (Michel Pelletier) Date: Mon Aug 16 19:16:27 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <20040816162011.424A11E4005@bag.python.org> References: <20040816162011.424A11E4005@bag.python.org> Message-ID: <20040816043506.0bb8542f.michel@dialnetwork.com> > (Short (off-topic) summary - there's two ways to make the twisted and > wx event loops cooperate - either the wx event loop runs everything, > and fires timers to let the twisted event loop do it's thing - see > above. The second is to let the twisted event loop regularly call > the wx event loop to let it iterate any outstanding events. When > a popup or menu comes up, wx creates a new sub-event-loop and takes > control until that's dismissed... so we're back to needing to run > the two event loops alongside each other. And yes, Tim's comment > about insanity resulting from using asyncore and threads together > also apply to twisted - the only sane way to do it is to have all > the event loop stuff handled in the same thread. Madness follows, > otherwise.) What if asyncore encapsulated the socket map in a selection object (like Java NIO) instead of a thread-global, module-level dictionary?. Selections could then be made by calling methods on an object (again, like NIO's select() (blocking), selectNow() (nonblocking) and select(timeout)). This doesn't really address the wx/event granularity issue, but it would reduce the thread madness considerably and allow programmers to see beyond asyncore's mainloop() by rolling their own, possible one taking into consideration other event loops like GUIs. Another benefit would be more than one selection loop per process, allowing multiple threads to handle multiple selection sets. -Michel From bac at OCF.Berkeley.EDU Mon Aug 16 19:28:22 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 16 19:28:32 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16672.55274.689761.740267@montanaro.dyndns.org> References: <16672.55274.689761.740267@montanaro.dyndns.org> Message-ID: <4120EEB6.6040505@ocf.berkeley.edu> Skip Montanaro wrote: [SNIP - test_tempfile failure] > This looks reasonable to me, so it doesn't look like the problem is within > Python. Should I just skip the tempfile test on Mac OSX? Can someone check > this out on 10.3/Panther? > drifty@Bretts-Computer>./python.exe Lib/test/regrtest.py test_sundry test_tcl test_tempfile test_sundry test_tcl test_tempfile [7731 refs] All 3 tests OK. [31661 refs] This UNIX pydebug build was compiled on Aug 14 for OS X 10.3.5 . -Brett From skip at pobox.com Mon Aug 16 19:54:17 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 19:54:36 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <2mvffjauu2.fsf@starship.python.net> References: <16672.55274.689761.740267@montanaro.dyndns.org> <2mvffjauu2.fsf@starship.python.net> Message-ID: <16672.62665.797132.507635@montanaro.dyndns.org> Michael> It's odd that test_tcl is on the list. Where is your tcl from? Michael> Fink? No, it's the TclAqua thing I think: _tkinter.so: /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.0) /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 63.0.0) Michael> Even so, it's hard to see how it could be interfering. I don't Michael> see this behaviour on a similar system, but then I don't have Michael> tcl installed. Dunno. It was required to provoke the failure for me. I was encouraged that Bob's box didn't fail. How is your _tkinter.so linked? Skip From mwh at python.net Mon Aug 16 19:56:02 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 16 19:56:04 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16672.62665.797132.507635@montanaro.dyndns.org> (Skip Montanaro's message of "Mon, 16 Aug 2004 12:54:17 -0500") References: <16672.55274.689761.740267@montanaro.dyndns.org> <2mvffjauu2.fsf@starship.python.net> <16672.62665.797132.507635@montanaro.dyndns.org> Message-ID: <2mr7q7arwt.fsf@starship.python.net> Skip Montanaro writes: > Michael> It's odd that test_tcl is on the list. Where is your tcl from? > Michael> Fink? > > No, it's the TclAqua thing I think: > > _tkinter.so: > /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.0) > /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.0) > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 63.0.0) > > Michael> Even so, it's hard to see how it could be interfering. I don't > Michael> see this behaviour on a similar system, but then I don't have > Michael> tcl installed. > > Dunno. It was required to provoke the failure for me. I was encouraged > that Bob's box didn't fail. How is your _tkinter.so linked? It's not, I don't have one :-) test_tcl skips on my system. Cheers, mwh -- One of the great skills in using any language is knowing what not to use, what not to say. ... There's that simplicity thing again. -- Ron Jeffries From skip at pobox.com Mon Aug 16 19:57:27 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 19:57:42 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <4120EEB6.6040505@ocf.berkeley.edu> References: <16672.55274.689761.740267@montanaro.dyndns.org> <4120EEB6.6040505@ocf.berkeley.edu> Message-ID: <16672.62855.287221.180403@montanaro.dyndns.org> Brett> drifty@Bretts-Computer>./python.exe Lib/test/regrtest.py test_sundry Brett> test_tcl test_tempfile Brett> test_sundry Brett> test_tcl Brett> test_tempfile Brett> [7731 refs] Brett> All 3 tests OK. Brett> [31661 refs] Brett> This UNIX pydebug build was compiled on Aug 14 for OS X 10.3.5 . Thanks. I'm going to let it slide on the assumption that it's a fixed bug in Jaguar. On the outside chance someone wants to truly mimic my build environment, here's how I configured: ./configure '--with-pydebug' '--with-libs=/usr/lib/libMallocDebug.a' I suppose it's possible other folks trying to reproduce this problem encounter the same behavior but aren't linked with MallocDebug, so the problem passes silently. Skip From skip at pobox.com Mon Aug 16 20:07:32 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 20:07:48 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <2mr7q7arwt.fsf@starship.python.net> References: <16672.55274.689761.740267@montanaro.dyndns.org> <2mvffjauu2.fsf@starship.python.net> <16672.62665.797132.507635@montanaro.dyndns.org> <2mr7q7arwt.fsf@starship.python.net> Message-ID: <16672.63460.103743.36579@montanaro.dyndns.org> >> Dunno. It was required to provoke the failure for me. I was >> encouraged that Bob's box didn't fail. How is your _tkinter.so >> linked? Michael> It's not, I don't have one :-) test_tcl skips on my system. Sorry about that. I meant, "Bob, how is your _tkinter.so linked?"... Skip From bac at OCF.Berkeley.EDU Mon Aug 16 20:10:33 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 16 20:10:43 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16672.62855.287221.180403@montanaro.dyndns.org> References: <16672.55274.689761.740267@montanaro.dyndns.org> <4120EEB6.6040505@ocf.berkeley.edu> <16672.62855.287221.180403@montanaro.dyndns.org> Message-ID: <4120F899.7090008@ocf.berkeley.edu> Skip Montanaro wrote: > Brett> drifty@Bretts-Computer>./python.exe Lib/test/regrtest.py test_sundry > Brett> test_tcl test_tempfile > Brett> test_sundry > Brett> test_tcl > Brett> test_tempfile > Brett> [7731 refs] > Brett> All 3 tests OK. > Brett> [31661 refs] > > Brett> This UNIX pydebug build was compiled on Aug 14 for OS X 10.3.5 . > > Thanks. I'm going to let it slide on the assumption that it's a fixed bug > in Jaguar. On the outside chance someone wants to truly mimic my build > environment, here's how I configured: > > ./configure '--with-pydebug' '--with-libs=/usr/lib/libMallocDebug.a' > > I suppose it's possible other folks trying to reproduce this problem > encounter the same behavior but aren't linked with MallocDebug, so the > problem passes silently. > And just to be thorough with my build environment (maybe we should have some command-line switch that outputs everything about the build of the interpreter?): CONFIG_ARGS= '--prefix=/Users/drifty/Code/compiled_CVS' '--disable-framework' ' --disable-toolbox-glue' '--without-cxx' '--with-pydebug' '-enable-unicode=ucs2' 'CPPFLAGS=-I/sw/include -I/opt/include' 'LDFLAGS=-L/sw/lib -L/opt/lib' -Brett From bob at redivi.com Mon Aug 16 20:17:12 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 20:17:53 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16672.63460.103743.36579@montanaro.dyndns.org> References: <16672.55274.689761.740267@montanaro.dyndns.org> <2mvffjauu2.fsf@starship.python.net> <16672.62665.797132.507635@montanaro.dyndns.org> <2mr7q7arwt.fsf@starship.python.net> <16672.63460.103743.36579@montanaro.dyndns.org> Message-ID: <7EE39EE7-EFB0-11D8-A746-000A95686CD8@redivi.com> On Aug 16, 2004, at 2:07 PM, Skip Montanaro wrote: > >>> Dunno. It was required to provoke the failure for me. I was >>> encouraged that Bob's box didn't fail. How is your _tkinter.so >>> linked? > > Michael> It's not, I don't have one :-) test_tcl skips on my > system. > > Sorry about that. I meant, "Bob, how is your _tkinter.so linked?"... % otool -L build/lib.darwin-7.3.0-Power_Macintosh-2.4/_tkinter.so build/lib.darwin-7.3.0-Power_Macintosh-2.4/_tkinter.so: /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.0) /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.0.0) presumably this is: -bundle -undefined dynamic_lookup -framework Tcl -framework Tk (using TclTkAqua framework builds...) -undefined dynamic_lookup is new to Mac OS X 10.3, and allows bundles to use Python without linking directly to it. 10.2 would look largely the same except it would probably have a -F. -framework Python in there. -bob From mal at egenix.com Mon Aug 16 20:27:50 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Aug 16 20:27:52 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <411DC3DD.3080900@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <411AFBD4.5080903@v.loewis.de><411B0DA0.4030109@v.loewis.de> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> <411C85B6.2060104@egenix.com> <411DC3DD.3080900@v.loewis.de> Message-ID: <4120FCA6.2070104@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> If we switch the binding of 'yyy' to mean unicode('yyy') >> some day, why can't we just continue to use the existing implementation >> for 8-bit strings for b'xxx' (the current implementation is already >> doing the right thing, meaning that it is 8-bit safe regardeless >> of the source code encoding) ? > > > Not exactly - the current implementation is not safe with respect to > re-encoding source in a different encoding. It is if you stick to writing your binary data using an ASCII compatible encoding -- I wouldn't expect any other encoding for binary data anyway. The most common are ASCII + escape sequences, base64 or hex, all of which are ASCII compatible. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 16 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From skip at pobox.com Mon Aug 16 21:14:38 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 21:14:55 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <4120F899.7090008@ocf.berkeley.edu> References: <16672.55274.689761.740267@montanaro.dyndns.org> <4120EEB6.6040505@ocf.berkeley.edu> <16672.62855.287221.180403@montanaro.dyndns.org> <4120F899.7090008@ocf.berkeley.edu> Message-ID: <16673.1950.317156.261551@montanaro.dyndns.org> Brett> ... (maybe we should have some command-line switch that outputs Brett> everything about the build of the interpreter?): The time machine strikes again: % egrep 'exec /bin/sh ./configure' config.status exec /bin/sh ./configure '--with-pydebug' '--with-libs=/usr/lib/libMallocDebug.a' $ac_configure_extra_args --no-create --no-recursion though in this case I think it may be Dennis Ritchie's or Ken Thompson's time machines, not Guido's. Skip From bob at redivi.com Mon Aug 16 21:23:01 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Aug 16 21:23:39 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16673.1950.317156.261551@montanaro.dyndns.org> References: <16672.55274.689761.740267@montanaro.dyndns.org> <4120EEB6.6040505@ocf.berkeley.edu> <16672.62855.287221.180403@montanaro.dyndns.org> <4120F899.7090008@ocf.berkeley.edu> <16673.1950.317156.261551@montanaro.dyndns.org> Message-ID: On Aug 16, 2004, at 3:14 PM, Skip Montanaro wrote: > > Brett> ... (maybe we should have some command-line switch that > outputs > Brett> everything about the build of the interpreter?): > > The time machine strikes again: > > % egrep 'exec /bin/sh ./configure' config.status > exec /bin/sh ./configure '--with-pydebug' > '--with-libs=/usr/lib/libMallocDebug.a' $ac_configure_extra_args > --no-create --no-recursion > > though in this case I think it may be Dennis Ritchie's or Ken > Thompson's > time machines, not Guido's. Even better (and I think this is what he was actually asking for): % python -c 'import distutils.sysconfig; print distutils.sysconfig.get_config_var("CONFIG_ARGS")' '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-ipv6' '--with-threads' '--enable-framework=/System/Library/Frameworks' '--enable-toolbox-glue' 'CFLAGS=-g -Os -pipe -fno-common -Wno-long-double -mno-fused-madd -pipe' 'LDFLAGS=-Wl,-F.' % ./python.exe -c 'import distutils.sysconfig; print distutils.sysconfig.get_config_var("CONFIG_ARGS")' '--enable-framework' -bob From phil at riverbankcomputing.co.uk Mon Aug 16 21:57:41 2004 From: phil at riverbankcomputing.co.uk (Phil Thompson) Date: Mon Aug 16 21:57:43 2004 Subject: [Python-Dev] Problem with PyGILState_Ensure() and the thread Module Message-ID: <200408162057.41113.phil@riverbankcomputing.co.uk> I'm having a problem calling functions in a C extension module that uses PyGILState_Ensure() and PyGILState_Release() from a thread created with thread.start_new_thread(). A deadlock occurs in PyGILState_Ensure() because it doesn't know anything about the thread state created by start_new_thread(). Therefore it creates a new one and then calls PyEval_RestoreThread() - but it already holds the GIL and so deadlocks. I would expect the threading PyGILState_Ensure() to know about all thread states - not just the ones it creates itself. Is this a bug, feature, or misunderstanding on my part? This is with Python 2.4a2. Thanks, Phil From aahz at pythoncraft.com Mon Aug 16 23:19:07 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 16 23:19:09 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com> References: <002001c48325$436d9460$e841fea9@oemcomputer> <20040816034817.GA25038@panix.com> <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com> <20040816035936.GB7737@panix.com> <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com> Message-ID: <20040816211907.GA1471@panix.com> On Mon, Aug 16, 2004, Bob Ippolito wrote: > On Aug 15, 2004, at 11:59 PM, Aahz wrote: >>On Sun, Aug 15, 2004, Bob Ippolito wrote: >>> >>>Many platform GUIs require that all or most all GUI activities be >>>consolidated to one and only one thread. I don't think any of them >>>'care' if another thread is also running, but it can't communicate >>>directly with the GUI. >> >>That's true, but it also applies to other external resources, such as >>files. > > Maybe I don't understand what you're saying.. but I don't know any > platform that forces you to work with files from one and only one > specific thread. But I definitely do know at least one platform where > you may only perform most GUI operations from the main thread of > execution. For any one file handle, you use only one thread. It just happens in the case of GUIs that there is only one GUI handle. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From skip at pobox.com Mon Aug 16 23:55:41 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 16 23:55:55 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: References: <16672.55274.689761.740267@montanaro.dyndns.org> <4120EEB6.6040505@ocf.berkeley.edu> <16672.62855.287221.180403@montanaro.dyndns.org> <4120F899.7090008@ocf.berkeley.edu> <16673.1950.317156.261551@montanaro.dyndns.org> Message-ID: <16673.11613.356961.874826@montanaro.dyndns.org> Bob> Even better (and I think this is what he was actually asking for): Bob> % python -c 'import distutils.sysconfig; print Bob> distutils.sysconfig.get_config_var("CONFIG_ARGS")' Thanks. Not as easy to remember as "grep configure config.status", but useful nonetheless. Skip From bob at redivi.com Tue Aug 17 00:09:25 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 17 00:10:34 2004 Subject: [Python-Dev] test_tempfile failure on Mac OSX In-Reply-To: <16673.11613.356961.874826@montanaro.dyndns.org> References: <16672.55274.689761.740267@montanaro.dyndns.org> <4120EEB6.6040505@ocf.berkeley.edu> <16672.62855.287221.180403@montanaro.dyndns.org> <4120F899.7090008@ocf.berkeley.edu> <16673.1950.317156.261551@montanaro.dyndns.org> <16673.11613.356961.874826@montanaro.dyndns.org> Message-ID: On Aug 16, 2004, at 5:55 PM, Skip Montanaro wrote: > > Bob> Even better (and I think this is what he was actually asking > for): > > Bob> % python -c 'import distutils.sysconfig; print > Bob> distutils.sysconfig.get_config_var("CONFIG_ARGS")' > > Thanks. Not as easy to remember as "grep configure config.status", but > useful nonetheless. Yeah, but I don't have a hard time remembering that the Makefile (that gets distributed w/ Python) knows everything, and that distutils knows how to read the Makefile. It's hard to remember the names of the make variables and the distutils incantation, but neither are hard to look up (if you've been through it a couple times anyway). I prefer not to depend on your solution because it's only useful if you compiled Python and still have access to that build tree (and remember which one it was). The first CONFIG_ARGS I spit out was the one used to build Python 2.3.0 on Mac OS X 10.3, and I certainly didn't do that here :) -bob From martin at v.loewis.de Tue Aug 17 01:04:13 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 17 01:04:16 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4120FCA6.2070104@egenix.com> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <411AFBD4.5080903@v.loewis.de><411B0DA0.4030109@v.loewis.de> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> <411C85B6.2060104@egenix.com> <411DC3DD.3080900@v.loewis.de> <4120FCA6.2070104@egenix.com> Message-ID: <41213D6D.50302@v.loewis.de> M.-A. Lemburg wrote: > It is if you stick to writing your binary data using an ASCII > compatible encoding -- I wouldn't expect any other encoding for > binary data anyway. The most common are ASCII + escape sequences, > base64 or hex, all of which are ASCII compatible. We probably have a different notion of "ASCII compatible" then. I would define it as: An encoding E is "ASCII compatbible" if strings that only consist of ASCII characters use the same byte representation in E that they use in ASCII. In that sense, ISO-8859-1 and UTF-8 are also ASCII compatible. Notice that this is also the definition that PEP 263 assumes. However, byte strings used in source code are not "safe" if they are encoded in ISO-8859-1 under recoding: If the source code is converted to UTF-8 (including the encoding declaration), then the length of the strings changes, as do the byte values inside the string. Regards, Martin From ncoghlan at iinet.net.au Tue Aug 17 01:15:48 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Aug 17 01:15:51 2004 Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with implementation In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E3022DFD@exchange.hqamor.amorhq.net> References: <3A81C87DC164034AA4E2DDFE11D258E3022DFD@exchange.hqamor.amorhq.net> Message-ID: <41214024.2030005@iinet.net.au> Robert Brewer wrote: > Michael Sparks wrote: > >>As for whether it's the right keyword or not, I'm not sure >>I'm qualified to >>say, but decorators seem much more like declarations of >>function properties >>in all the use cases I've seen. > > The proposal, in effect, creates a declarative suite as opposed to the > more traditional imperative suite. Perhaps "declare:" would be more > appropriate (but give your brain a week to mull it over ;). I'm still a fan of 'predef:' if a keyword is going to be involved. The only thing I've seen so far that is common to all decorators is that they are predefining things about a 'def' we haven't encountered yet :) Cheers, Nick. -- Nick Coghlan | Eugene, Oregon Email: ncoghlan@email.com | USA From foom at fuhm.net Tue Aug 17 02:07:14 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 17 02:07:25 2004 Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler mystery] In-Reply-To: <20040816112916.GA19969@vicky.ecs.soton.ac.uk> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> <20040812204431.GA31884@vicky.ecs.soton.ac.uk> <1f7befae0408151950361f0cb4@mail.gmail.com> <20040816112916.GA19969@vicky.ecs.soton.ac.uk> Message-ID: <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> On Aug 16, 2004, at 7:29 AM, Armin Rigo wrote: > Yes. Here is a patch attempting to do what I described: > http://www.python.org/sf/1009929 From the patch description: > Some more thinking about [the exception hierarchy] would be welcome. > Maybe AsynchronousException and a few others should not subclass > Exception at all, so that "except Exception" statements don't catch > them. Anyway, this patch alreaddy improves the situation, because you > can catch and re-raise AsynchronousException (instead of, say, just > KeyboardInterrupt). It seems to me that something similar to what Java has would be a good idea. Namely, a new top level exception (from which all others would derive) called "Raisable", analogous to Java's Throwable. This then has two subclasses: "Exception", and "FatalError". I'm not sure FatalError is a good name, but *some* new name needs to be thought up for Java's "Error" class, because lots of python exceptions end in "Error" but belong under the "Exception" hierarchy (for example "KeyError"). The criteria for whether a given exception should go under "Exception" or "FatalError" is whether users' code should generally catch the exception. Thus, at least "SystemExit", "KeyboardInterrupt", and "MemoryError" should go under "FatalError". Catching those is nearly never what you wanted to do when you write "except Exception:". There's likely more -- I have not gone through all the exceptions in Python to classify them. One issue is that creating a new category of Exceptions doesn't help people who do "except:" instead of "except Exception:". It is unlikely that person meant to catch things like MemoryError, rather, they were just being lazy. I suppose that syntax could be deprecated, at least in example code and documentation, in favor of "except Exception" for the usual case, and "except Raisable" for the cases where you do actually want to catch everything*. James * Except, of course, old string exceptions which have been deprecated for ages. From bob at redivi.com Tue Aug 17 02:33:51 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 17 02:34:27 2004 Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler mystery] In-Reply-To: <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> <20040812204431.GA31884@vicky.ecs.soton.ac.uk> <1f7befae0408151950361f0cb4@mail.gmail.com> <20040816112916.GA19969@vicky.ecs.soton.ac.uk> <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> Message-ID: <1CC8AA0A-EFE5-11D8-AA0D-000A95686CD8@redivi.com> On Aug 16, 2004, at 8:07 PM, James Y Knight wrote: > On Aug 16, 2004, at 7:29 AM, Armin Rigo wrote: >> Yes. Here is a patch attempting to do what I described: >> http://www.python.org/sf/1009929 > > > From the patch description: >> Some more thinking about [the exception hierarchy] would be welcome. >> Maybe AsynchronousException and a few others should not subclass >> Exception at all, so that "except Exception" statements don't catch >> them. Anyway, this patch alreaddy improves the situation, because you >> can catch and re-raise AsynchronousException (instead of, say, just >> KeyboardInterrupt). > > It seems to me that something similar to what Java has would be a good > idea. Namely, a new top level exception (from which all others would > derive) called "Raisable", analogous to Java's Throwable. This then > has two subclasses: "Exception", and "FatalError". I'm not sure > FatalError is a good name, but *some* new name needs to be thought up > for Java's "Error" class, because lots of python exceptions end in > "Error" but belong under the "Exception" hierarchy (for example > "KeyError"). > > The criteria for whether a given exception should go under "Exception" > or "FatalError" is whether users' code should generally catch the > exception. Thus, at least "SystemExit", "KeyboardInterrupt", and > "MemoryError" should go under "FatalError". Catching those is nearly > never what you wanted to do when you write "except Exception:". > There's likely more -- I have not gone through all the exceptions in > Python to classify them. > > One issue is that creating a new category of Exceptions doesn't help > people who do "except:" instead of "except Exception:". It is unlikely > that person meant to catch things like MemoryError, rather, they were > just being lazy. I suppose that syntax could be deprecated, at least > in example code and documentation, in favor of "except Exception" for > the usual case, and "except Raisable" for the cases where you do > actually want to catch everything*. > > James > > * Except, of course, old string exceptions which have been deprecated > for ages. * basestr could inherit from "Raisable" ;) The big problem with "Raisable" is that both raiseable and raisable seem to be correct spellings, and I don't think either are in many abridged dictionaries (the OS X spell checker doesn't like either, for example). -bob From aahz at pythoncraft.com Tue Aug 17 03:01:56 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Aug 17 03:02:12 2004 Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler mystery] In-Reply-To: <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> <20040812204431.GA31884@vicky.ecs.soton.ac.uk> <1f7befae0408151950361f0cb4@mail.gmail.com> <20040816112916.GA19969@vicky.ecs.soton.ac.uk> <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> Message-ID: <20040817010156.GA23702@panix.com> On Mon, Aug 16, 2004, James Y Knight wrote: > > The criteria for whether a given exception should go under "Exception" > or "FatalError" is whether users' code should generally catch the > exception. Thus, at least "SystemExit", "KeyboardInterrupt", and > "MemoryError" should go under "FatalError". Catching those is nearly > never what you wanted to do when you write "except Exception:". There's > likely more -- I have not gone through all the exceptions in Python to > classify them. > > One issue is that creating a new category of Exceptions doesn't help > people who do "except:" instead of "except Exception:". It is unlikely > that person meant to catch things like MemoryError, rather, they were > just being lazy. I suppose that syntax could be deprecated, at least in > example code and documentation, in favor of "except Exception" for the > usual case, and "except Raisable" for the cases where you do actually > want to catch everything*. We've already got StandardError; I think it makes more sense to rearrange the exception hierarchy a bit to support your suggestion rather than creating a whole new base class. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From guido at python.org Tue Aug 17 03:06:19 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 03:06:40 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: Your message of "Sun, 15 Aug 2004 13:05:28 PDT." <411FC208.3040800@ocf.berkeley.edu> References: <4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au> <1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu> <1092583647.1670.18.camel@localhost> <411FC208.3040800@ocf.berkeley.edu> Message-ID: <200408170106.i7H16J614260@guido.python.org> > 728 emails to summarize on top of any sent in today so I will be swamped > with that for the rest of the week. I am sure someone else can step up > and applying it if Guido okays the application order. Since Mark ended up agreeing with your order, the OK is given by default. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 03:08:56 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 03:09:08 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: Your message of "Sun, 15 Aug 2004 17:41:26 EDT." <001501c48310$bf15aa40$e841fea9@oemcomputer> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> Message-ID: <200408170108.i7H18vs14287@guido.python.org> > The documentation for the atexit module (introduced in Py2.0) states: > > """Note: This module is unlikely to work correctly when used with other > code that sets sys.exitfunc. In particular, other core Python modules > are free to use atexit without the programmer's knowledge. Authors who > use sys.exitfunc should convert their code to use atexit instead. The > simplest way to convert code that sets sys.exitfunc is to import atexit > and register the function that had been bound to sys.exitfunc.""" > > Can we strengthen this by deprecating sys.exitfunc? > > The atexit module does attempt to co-exist by introducing code to > register a function in sys.exitfunc if it were defined before "import > atexit" was called. However, this is unreliable because it depends on > import order and the library is free to introduce an earlier "import > atexit" which would break code relying on the co-existance mechanism. +1 (I think the quoted words were written before we had introduced a formal concept of deprecation -- atexit is quite old. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tdelaney at avaya.com Tue Aug 17 03:11:17 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Tue Aug 17 03:11:25 2004 Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compilermystery] Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D588B3@au3010avexu1.global.avaya.com> James Y Knight wrote: > One issue is that creating a new category of Exceptions doesn't help > people who do "except:" instead of "except Exception:". It is unlikely > that person meant to catch things like MemoryError, rather, they were > just being lazy. I suppose that syntax could be deprecated, at least > in > example code and documentation, in favor of "except Exception" for the > usual case, and "except Raisable" for the cases where you do actually > want to catch everything*. Personally, I would be in favour of a bare except being equivalent in new code to 'except Exception'. Although this does violate 'in the presence of ambiguity, resist the temptation to guess'. Tim Delaney From guido at python.org Tue Aug 17 03:11:34 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 03:11:42 2004 Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but aren't In-Reply-To: Your message of "Mon, 16 Aug 2004 13:09:31 +1200." <200408160109.i7G19VHN019296@cosc353.cosc.canterbury.ac.nz> References: <200408160109.i7G19VHN019296@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408170111.i7H1BYJ14320@guido.python.org> > The real question seems to be whether __builtin__ should contain *all* > built-in types, including internal ones, or only those intended for > public use. Do you have an opinion about that, Guido? > > I suppose it's reasonable to put them all in __builtin__, since as you > say, they can be useful for type checking even if you can't or rarely > want to instantiate them yourself. That's where I stand, at about +0.5. I don't think it's super important that all builtins be particularly well-known or useful -- it's easy enough to have a section in the documentation for "lesser-known builtins". --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at cosc.canterbury.ac.nz Tue Aug 17 03:13:32 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 17 03:13:37 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41213D6D.50302@v.loewis.de> Message-ID: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> > However, byte strings used in source code are not "safe" if they > are encoded in ISO-8859-1 under recoding: If the source code is > converted to UTF-8 (including the encoding declaration), then > the length of the strings changes, as do the byte values inside > the string. This suggests that byte string literals should be restricted to ASCII characters and \x escapes. Would that be safe enough? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido at python.org Tue Aug 17 03:14:47 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 03:14:55 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: Your message of "Sun, 15 Aug 2004 23:44:23 EDT." <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> References: <002001c48325$436d9460$e841fea9@oemcomputer> <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> Message-ID: <200408170114.i7H1Elk14352@guido.python.org> > > (Season to taste, fix technical inaccuracies.) I see no reason to > > mention the GIL, an implementation detail, at all. > > I think it's worth mentioning. It's absolutely necessary to know all > about the GIL when writing an extension that can take advantage of > threading or embed Python correctly in a threaded application. But does that mention beling in a threading *tutorial*? (In addition, I find that the GIL often causes people to worry unnecessarily about the GIL. Sort of like worrying about relativistic effects when driving a car.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 03:21:55 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 03:22:02 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: Your message of "Mon, 16 Aug 2004 12:33:57 BST." <20040816113357.GB19969@vicky.ecs.soton.ac.uk> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> Message-ID: <200408170121.i7H1Ltn14417@guido.python.org> > Multiple well-written modules can use sys.exitfunc in a way that is > compatible. atexit is merely a glue to discourage non-well-written > modules. I'd say that atexit should be encouraged but there is > nothing wrong with using sys.exitfunc directly, if one is aware of > the common patching pattern: "save-old-value, > replace-with-my-function, upon-call-invoke-saved-old-value". But why offer two ways of doing the same thing? Apart from b/w compatibility, what's the advantage of using sys.exitfunc directly? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 03:32:59 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 03:33:08 2004 Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler mystery] In-Reply-To: Your message of "Mon, 16 Aug 2004 20:07:14 EDT." <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> <20040812204431.GA31884@vicky.ecs.soton.ac.uk> <1f7befae0408151950361f0cb4@mail.gmail.com> <20040816112916.GA19969@vicky.ecs.soton.ac.uk> <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> Message-ID: <200408170133.i7H1WxC14538@guido.python.org> > From the patch description: > > Some more thinking about [the exception hierarchy] would be welcome. > > Maybe AsynchronousException and a few others should not subclass > > Exception at all, so that "except Exception" statements don't catch > > them. Anyway, this patch alreaddy improves the situation, because you > > can catch and re-raise AsynchronousException (instead of, say, just > > KeyboardInterrupt). > > It seems to me that something similar to what Java has would be a good > idea. Namely, a new top level exception (from which all others would > derive) called "Raisable", analogous to Java's Throwable. This then has > two subclasses: "Exception", and "FatalError". I'm not sure FatalError > is a good name, but *some* new name needs to be thought up for Java's > "Error" class, because lots of python exceptions end in "Error" but > belong under the "Exception" hierarchy (for example "KeyError"). Hm, Java needs the distinction because some exceptions must be declared and others mustn't. But Python doesn't have that distinction. I'm not sure that you can always treat the same set of exceptions as fatal. E.g. in many situations, AttributeError, TypeError and NameError are all indicative of programming bugs (typically typos), but in other contexts these are recoverable. So rather than giving an arbitrary definition of fatality, let's refrain from defining the concept. > The criteria for whether a given exception should go under "Exception" > or "FatalError" is whether users' code should generally catch the > exception. Thus, at least "SystemExit", "KeyboardInterrupt", and > "MemoryError" should go under "FatalError". Catching those is nearly > never what you wanted to do when you write "except Exception:". There's > likely more -- I have not gone through all the exceptions in Python to > classify them. Calling SystemExit and KeyboardInterrupt fatal strikes me as particularly odd, as I routinely catch these. > One issue is that creating a new category of Exceptions doesn't help > people who do "except:" instead of "except Exception:". It is unlikely > that person meant to catch things like MemoryError, rather, they were > just being lazy. I suppose that syntax could be deprecated, at least in > example code and documentation, in favor of "except Exception" for the > usual case, and "except Raisable" for the cases where you do actually > want to catch everything*. > > James > > * Except, of course, old string exceptions which have been deprecated > for ages. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 03:34:13 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 03:34:19 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 13:13:32 +1200." <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408170134.i7H1YDR14560@guido.python.org> > This suggests that byte string literals should be > restricted to ASCII characters and \x escapes. > Would that be safe enough? Make that printable ASCII and I'm +0. (I'm still not sold on the concept of bytes literals at all.) --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Tue Aug 17 04:52:35 2004 From: foom at fuhm.net (James Y Knight) Date: Tue Aug 17 04:52:40 2004 Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler mystery] In-Reply-To: <200408170133.i7H1WxC14538@guido.python.org> References: <002d01c48083$9a89a6c0$5229c797@oemcomputer> <1f7befae040812091754035bcb@mail.gmail.com> <20040812185521.GA2277@vicky.ecs.soton.ac.uk> <1f7befae04081212414007274f@mail.gmail.com> <20040812204431.GA31884@vicky.ecs.soton.ac.uk> <1f7befae0408151950361f0cb4@mail.gmail.com> <20040816112916.GA19969@vicky.ecs.soton.ac.uk> <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> <200408170133.i7H1WxC14538@guido.python.org> Message-ID: <7E3ABDEE-EFF8-11D8-8449-000A95A50FB2@fuhm.net> On Aug 16, 2004, at 9:32 PM, Guido van Rossum wrote: > Hm, Java needs the distinction because some exceptions must be > declared and others mustn't. But Python doesn't have that > distinction. I'm not sure that you can always treat the same set of > exceptions as fatal. E.g. in many situations, AttributeError, > TypeError and NameError are all indicative of programming bugs > (typically typos), but in other contexts these are recoverable. So > rather than giving an arbitrary definition of fatality, let's refrain > from defining the concept. Well, actually, java has three categories of exceptions: 1) Error - not necessary to declare as thrown, only *serious* errors. Contains: a) things that can "never happen" with a program compiled by the java compiler (no such field, unknown class file format, etc), b) machine errors (out of memory, stack overflow, etc) c) assert() failure d) ThreadDeath (similar to a KeyboardInterrupt/SystemExit) 2) Exception - normal run of the mill exception, needs to be declared as thrown. 3) RuntimeException - subclass of Exception, does not need to be declared as thrown. (e.g. IndexOutOfBoundsException, NoSuchFieldException, ClassCastException) The distinction you refer to above is really the difference between Exception and RuntimeException. Translated to java, AttributeError, TypeError and NameError would be RuntimeExceptions. So, I agree with you - I don't believe python needs that distinction. I do believe python needs the distinction between Exception and Error. > Calling SystemExit and KeyboardInterrupt fatal strikes me as > particularly odd, as I routinely catch these. I'll agree: I don't think the name "FatalError" is particularly great. However, I hope it gets the idea across better than "XXXErrorXXXRenameMeXXX" which was my other choice of name. ;) I do think the categorization is correct. While you may sometimes catch KeyboardInterrupt/SystemExit, most of the time you really do not want to, even if you are catching "everything". If you do want to catch KeyboardInterrupt, you are also likely to be catching it explicitly by its name, anyhow. James From martin at v.loewis.de Tue Aug 17 08:30:05 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 17 08:30:13 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> Message-ID: <4121A5ED.9030001@v.loewis.de> Greg Ewing wrote: > This suggests that byte string literals should be > restricted to ASCII characters and \x escapes. > Would that be safe enough? Yes, that should work fine (Guidos restriction to *printable* characters is also useful, as line endings are easily changed, too, when moving from one system to another). Regards, Martin From anthony at interlink.com.au Tue Aug 17 08:54:35 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 17 08:54:54 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408170134.i7H1YDR14560@guido.python.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> Message-ID: <4121ABAB.9060605@interlink.com.au> Guido van Rossum wrote: > (I'm still not sold on the > concept of bytes literals at all.) Ok. Here's a case - in shtoom, I generate audio data. Lots of audio data. This is broken into packets, then gets a small header put onto each RTP packet. Right now, I'm using strings for this. If there was a 'byte literal', I'd use it. This isn't a huge problem right now, because strings are good enough. But if we end up in an 'all the strings are unicode', I'll need _some_ way to construct these packets. -- Anthony Baxter It's never too late to have a happy childhood. From martin at v.loewis.de Tue Aug 17 09:06:23 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 17 09:06:35 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4121ABAB.9060605@interlink.com.au> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> Message-ID: <4121AE6F.4090904@v.loewis.de> Anthony Baxter wrote: > Ok. Here's a case - in shtoom, I generate audio data. Lots > of audio data. This is broken into packets, then gets a small > header put onto each RTP packet. Right now, I'm using strings > for this. If there was a 'byte literal', I'd use it. This isn't > a huge problem right now, because strings are good enough. But > if we end up in an 'all the strings are unicode', I'll need > _some_ way to construct these packets. Maybe you are missing the point here, maybe not: there is no debate that Python should always have a byte string type (although there is debate on whether that type should be mutable). The current question is whether you want to denote objects of the byte string type *in source code*. I.e. do you have the "Lots of audio data" stored in .py files? Regards, Martin From anthony at interlink.com.au Tue Aug 17 09:17:13 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 17 09:17:30 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4121AE6F.4090904@v.loewis.de> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <4121AE6F.4090904@v.loewis.de> Message-ID: <4121B0F9.4080309@interlink.com.au> Martin v. L?wis wrote: > The current question is whether you want to denote objects of > the byte string type *in source code*. I.e. do you have the "Lots > of audio data" stored in .py files? Generally, no - with the exception of test cases. In that case, I often end up with byte literals in the source code. (To check that a particular en/de coding operation Did The Right Thing). Anthony From t-meyer at ihug.co.nz Tue Aug 17 09:24:26 2004 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Tue Aug 17 09:24:33 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Message-ID: [Martin v. L?wis] > Maybe you are missing the point here, maybe not: there is no > debate that Python should always have a byte string type > (although there is debate on whether that type should be mutable). > > The current question is whether you want to denote objects of > the byte string type *in source code*. I.e. do you have the > "Lots of audio data" stored in .py files? I don't know about shtoom, but Mike Fletcher's resourcepackage does. (It packages up non-*.py files as *.py files). I believe that the main point of resourcepackage is for ease of distribution (but could be wrong), so perhaps better distribution tools make this unnecessary. It is an example use-case, though. =Tony Meyer From mal at egenix.com Tue Aug 17 09:32:38 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 17 09:32:33 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41213D6D.50302@v.loewis.de> References: <1092258146.411a896246d1a@mcherm.com> <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> <16666.56033.65067.241794@montanaro.dyndns.org> <411AFBD4.5080903@v.loewis.de><411B0DA0.4030109@v.loewis.de> <411BB50C.8000804@v.loewis.de> <16667.54746.535997.120878@montanaro.dyndns.org> <411C85B6.2060104@egenix.com> <411DC3DD.3080900@v.loewis.de> <4120FCA6.2070104@egenix.com> <41213D6D.50302@v.loewis.de> Message-ID: <4121B496.6050207@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> It is if you stick to writing your binary data using an ASCII >> compatible encoding -- I wouldn't expect any other encoding for >> binary data anyway. The most common are ASCII + escape sequences, >> base64 or hex, all of which are ASCII compatible. > > > We probably have a different notion of "ASCII compatible" then. > I would define it as: > > An encoding E is "ASCII compatbible" if strings that only consist > of ASCII characters use the same byte representation in E that > they use in ASCII. > > In that sense, ISO-8859-1 and UTF-8 are also ASCII compatible. Notice > that this is also the definition that PEP 263 assumes. Sorry, wrong wording on my part: I meant a string literal that only uses ASCII characters for the literal definition, i.e. literaldefinition.decode('ascii').encode('ascii') == literaldefinition. > However, byte strings used in source code are not "safe" if they > are encoded in ISO-8859-1 under recoding: If the source code is > converted to UTF-8 (including the encoding declaration), then > the length of the strings changes, as do the byte values inside > the string. Agreed. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From guido at python.org Tue Aug 17 12:45:26 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 12:45:38 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 16:54:35 +1000." <4121ABAB.9060605@interlink.com.au> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> Message-ID: <200408171045.i7HAjQZ15411@guido.python.org> > Guido van Rossum wrote: > > (I'm still not sold on the concept of bytes literals at all.) [Anthony] > Ok. Here's a case - in shtoom, I generate audio data. Lots > of audio data. This is broken into packets, then gets a small > header put onto each RTP packet. Right now, I'm using strings > for this. If there was a 'byte literal', I'd use it. This isn't > a huge problem right now, because strings are good enough. But > if we end up in an 'all the strings are unicode', I'll need > _some_ way to construct these packets. I see that as a huge case for a bytes type, which I've proposed myself; but what's the use case for bytes literals, assuming you can write bytes("foo")? Does b"foo" really make much of a difference? Is it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of b"\x66\x6f\x6f"? IOW, how many *literal* packet fragments are in shtoom? --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony at interlink.com.au Tue Aug 17 13:13:29 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 17 13:13:45 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408171045.i7HAjQZ15411@guido.python.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> Message-ID: <4121E859.8000908@interlink.com.au> Guido van Rossum wrote: > I see that as a huge case for a bytes type, which I've proposed > myself; but what's the use case for bytes literals, assuming you can > write bytes("foo")? Does b"foo" really make much of a difference? Is > it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of > b"\x66\x6f\x6f"? It's a pretty marginal case for it. I just played with it a bit, and I think after playing with it, I actually prefer the non b'' case. A big +1 for a bytes() type, though. I'm not sure on the details, but it'd be nice if it was possible to pass a bytes() object to, for instance, write() directly. From arigo at tunes.org Tue Aug 17 13:16:22 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Aug 17 13:21:02 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <200408170121.i7H1Ltn14417@guido.python.org> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> <200408170121.i7H1Ltn14417@guido.python.org> Message-ID: <20040817111622.GA12954@vicky.ecs.soton.ac.uk> Hello, On Mon, Aug 16, 2004 at 06:21:55PM -0700, Guido van Rossum wrote: > But why offer two ways of doing the same thing? Apart from b/w > compatibility, what's the advantage of using sys.exitfunc directly? Well, none, really. But let's not change the name sys.exitfunc just for the sake of deprecation, because it will probably break existing and well-behaving modules (not just non-well-behaving ones). Armin From gmccaughan at synaptics-uk.com Tue Aug 17 14:19:24 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue Aug 17 14:20:07 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4121E859.8000908@interlink.com.au> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> Message-ID: <200408171319.24084.gmccaughan@synaptics-uk.com> On Tuesday 2004-08-17 12:13, Anthony Baxter wrote: > Guido van Rossum wrote: > > I see that as a huge case for a bytes type, which I've proposed > > myself; but what's the use case for bytes literals, assuming you can > > write bytes("foo")? Does b"foo" really make much of a difference? Is > > it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of > > b"\x66\x6f\x6f"? > > It's a pretty marginal case for it. I just played with it a bit, and > I think after playing with it, I actually prefer the non b'' case. Another option, with pros and cons of its own: something along the lines of b"666f6f". -- g From mwh at python.net Tue Aug 17 14:49:21 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 17 14:49:23 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4121E859.8000908@interlink.com.au> (Anthony Baxter's message of "Tue, 17 Aug 2004 21:13:29 +1000") References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> Message-ID: <2mbrhaaq0e.fsf@starship.python.net> Anthony Baxter writes: > Guido van Rossum wrote: >> I see that as a huge case for a bytes type, which I've proposed >> myself; but what's the use case for bytes literals, assuming you can >> write bytes("foo")? Does b"foo" really make much of a difference? Is >> it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of >> b"\x66\x6f\x6f"? > > It's a pretty marginal case for it. I just played with it a bit, and > I think after playing with it, I actually prefer the non b'' case. Is this getting to (hopefully uncontroverisal!) PEP time? Is there any consensus forming on whether bytes() instances are mutable or not? > A big +1 for a bytes() type, though. I'm not sure on the details, > but it'd be nice if it was possible to pass a bytes() object to, > for instance, write() directly. If bytes() doesn't implement the read buffer interface, someone somewhere is going to need shooting :-) Cheers, mwh -- rasterman is the millionth monkey -- from Twisted.Quotes From barry at python.org Tue Aug 17 16:25:52 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 17 16:25:28 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: References: Message-ID: <1092752752.8622.32.camel@localhost> On Tue, 2004-08-17 at 03:24, Tony Meyer wrote: > I don't know about shtoom, but Mike Fletcher's resourcepackage does. (It > packages up non-*.py files as *.py files). I believe that the main point of > resourcepackage is for ease of distribution (but could be wrong), so perhaps > better distribution tools make this unnecessary. It is an example use-case, > though. I'd think that in most cases, huge globs of binary byte data would better live in non-py data files. I personally don't see much of a use case for byte literals (although, +1 on a bytes type). Byte literals just aren't going to be commonly written by people I think, and any programmatic generation of byte data should just as easily live outside py files. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040817/3586cfdb/attachment.pgp From skip at pobox.com Tue Aug 17 15:26:31 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 17 16:41:41 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <2mbrhaaq0e.fsf@starship.python.net> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> Message-ID: <16674.1927.230587.346903@montanaro.dyndns.org> Michael> Is this getting to (hopefully uncontroverisal!) PEP time? One would hope. I sent in a skeletal PEP last week asking for a number but haven't heard back. Michael> Is there any consensus forming on whether bytes() instances are Michael> mutable or not? ISTR Guido thought mutable was the way to go. I don't think that efficiency concerns will be a huge deal since it won't be used all over the place the way strings are. Will they need to be used as dict keys? Doesn't seem likely to me. Skip From anthony at interlink.com.au Tue Aug 17 16:49:54 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 17 16:50:09 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16674.1927.230587.346903@montanaro.dyndns.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <16674.1927.230587.346903@montanaro.dyndns.org> Message-ID: <41221B12.4080808@interlink.com.au> > ISTR Guido thought mutable was the way to go. I don't think that efficiency > concerns will be a huge deal since it won't be used all over the place the > way strings are. Will they need to be used as dict keys? Doesn't seem > likely to me. Mutable is far far more useful. I can't see them being used commonly for dict keys, but I do know of a lot of protocols where you have to go back and patch a 'length' field in a packet header after you've finished the packet construction. Maybe we could use the @ symbol for the byte literal? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mwh at python.net Tue Aug 17 16:57:15 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 17 16:57:17 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16674.1927.230587.346903@montanaro.dyndns.org> (Skip Montanaro's message of "Tue, 17 Aug 2004 08:26:31 -0500") References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <16674.1927.230587.346903@montanaro.dyndns.org> Message-ID: <2mvffhak38.fsf@starship.python.net> Skip Montanaro writes: > Michael> Is this getting to (hopefully uncontroverisal!) PEP time? > > One would hope. I sent in a skeletal PEP last week asking for a number but > haven't heard back. Goody. > Michael> Is there any consensus forming on whether bytes() instances are > Michael> mutable or not? > > ISTR Guido thought mutable was the way to go. OK. > I don't think that efficiency concerns will be a huge deal since it > won't be used all over the place the way strings are. Who cares? :-) > Will they need to be used as dict keys? Doesn't seem likely to me. This is indeed the question. I agree it's unlikely, and you can presumably do something like convert a bytes() instance into a tuple of small integers if you really want to key into a dict. Cheers, mwh -- Clue: You've got the appropriate amount of hostility for the Monastery, however you are metaphorically getting out of the safari jeep and kicking the lions. -- coonec -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From guido at python.org Tue Aug 17 17:27:20 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 17:27:26 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 21:13:29 +1000." <4121E859.8000908@interlink.com.au> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> Message-ID: <200408171527.i7HFRKV16082@guido.python.org> > A big +1 for a bytes() type, though. I'm not sure on the details, > but it'd be nice if it was possible to pass a bytes() object to, > for instance, write() directly. That should already possible with the array module, but somehow it doesn't quite work, even though the array type appears to support the buffer API. (Does anybody understand why not?) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 17:28:03 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 17:28:08 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: Your message of "Tue, 17 Aug 2004 12:16:22 BST." <20040817111622.GA12954@vicky.ecs.soton.ac.uk> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> <200408170121.i7H1Ltn14417@guido.python.org> <20040817111622.GA12954@vicky.ecs.soton.ac.uk> Message-ID: <200408171528.i7HFS3t16093@guido.python.org> > > But why offer two ways of doing the same thing? Apart from b/w > > compatibility, what's the advantage of using sys.exitfunc directly? > > Well, none, really. But let's not change the name sys.exitfunc just > for the sake of deprecation, because it will probably break existing > and well-behaving modules (not just non-well-behaving ones). That's never been a reason not to deprecate something. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 17:31:30 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 17:31:36 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 13:49:21 BST." <2mbrhaaq0e.fsf@starship.python.net> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> Message-ID: <200408171531.i7HFVUx16139@guido.python.org> > Is this getting to (hopefully uncontroverisal!) PEP time? Sure. > Is there any consensus forming on whether bytes() instances are > mutable or not? Mutable! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 17:33:07 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 17:33:13 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 15:57:15 BST." <2mvffhak38.fsf@starship.python.net> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <16674.1927.230587.346903@montanaro.dyndns.org> <2mvffhak38.fsf@starship.python.net> Message-ID: <200408171533.i7HFX7416165@guido.python.org> > This is indeed the question. I agree it's unlikely, and you can > presumably do something like convert a bytes() instance into a tuple > of small integers if you really want to key into a dict. Or even a Unicode string using the Latin-1 encoding. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Aug 17 17:48:04 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 17 17:49:11 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408171531.i7HFVUx16139@guido.python.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> Message-ID: <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> At 08:31 AM 8/17/04 -0700, Guido van Rossum wrote: > > Is this getting to (hopefully uncontroverisal!) PEP time? > >Sure. > > > Is there any consensus forming on whether bytes() instances are > > mutable or not? > >Mutable! So, how will it be different from: from array import array def bytes(*initializer): return array('B',*initializer) Even if it's desirable for 'bytes' to be an actual type (e.g. subclassing ArrayType), it might help the definition process to describe the difference between the new type and a byte array. From mal at egenix.com Tue Aug 17 18:20:30 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 17 18:20:34 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <2mbrhaaq0e.fsf@starship.python.net> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> Message-ID: <4122304E.1070907@egenix.com> Michael Hudson wrote: > Anthony Baxter writes: >>A big +1 for a bytes() type, though. I'm not sure on the details, >>but it'd be nice if it was possible to pass a bytes() object to, >>for instance, write() directly. > > If bytes() doesn't implement the read buffer interface, someone > somewhere is going to need shooting :-) Is there any reason you cannot use buffer() ?! It already implements all the necessary things and has been available for many years. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From arigo at tunes.org Tue Aug 17 18:18:33 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Aug 17 18:23:05 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <200408171528.i7HFS3t16093@guido.python.org> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> <200408170121.i7H1Ltn14417@guido.python.org> <20040817111622.GA12954@vicky.ecs.soton.ac.uk> <200408171528.i7HFS3t16093@guido.python.org> Message-ID: <20040817161833.GA29635@vicky.ecs.soton.ac.uk> Hello Guido, On Tue, Aug 17, 2004 at 08:28:03AM -0700, Guido van Rossum wrote: > > Well, none, really. But let's not change the name sys.exitfunc just > > for the sake of deprecation, because it will probably break existing > > and well-behaving modules (not just non-well-behaving ones). > > That's never been a reason not to deprecate something. Sorry. Sure. What I was opposing here is Raymond's original claim in this thread, which was apparently the reason he wanted to deprecate sys.exitfunc: """ The atexit module does attempt to co-exist by introducing code to register a function in sys.exitfunc if it were defined before "import atexit" was called. However, this is unreliable because it depends on import order and the library is free to introduce an earlier "import atexit" which would break code relying on the co-existance mechanism. """ I claim that there is nothing unreliable or depending on import order here, as long as all concerned parties do the right thing. Now if there are other good reasons to deprecate sys.exitfunc, like it being another way of doing something for which a better interface is provided, then fine. Armin From skip at pobox.com Tue Aug 17 18:23:10 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 17 18:23:25 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <200408171528.i7HFS3t16093@guido.python.org> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> <200408170121.i7H1Ltn14417@guido.python.org> <20040817111622.GA12954@vicky.ecs.soton.ac.uk> <200408171528.i7HFS3t16093@guido.python.org> Message-ID: <16674.12526.711803.480295@montanaro.dyndns.org> >> > But why offer two ways of doing the same thing? Apart from b/w >> > compatibility, what's the advantage of using sys.exitfunc directly? >> >> Well, none, really. But let's not change the name sys.exitfunc just >> for the sake of deprecation, because it will probably break existing >> and well-behaving modules (not just non-well-behaving ones). Guido> That's never been a reason not to deprecate something. Care to reword that without the double negative? I tried and came up with That's sometimes a reason to deprecate something. which seems like you're saying that breaking existing well-behaving modules is sometimes a good reason to deprecate something. not-sure-if-i-should-wink-ly, y'rs, Skip From mcherm at mcherm.com Tue Aug 17 18:40:02 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 17 18:37:57 2004 Subject: [Python-Dev] Decorator order implemented backwards? Message-ID: <1092760802.412234e2cb4d5@mcherm.com> Mark writes: > Your patch results in the evaluation order: > > evalname1 evalargs1 makedec1 > evalname2 evalargs2 makedec2 > evalname3 evalargs3 makedec3 > calldec3 calldec2 calldec1 > > Mine (#1009560) gives: > > evalname3 evalargs3 makedec3 calldec3 > evalname2 evalargs2 makedec2 calldec2 > evalname1 evalargs1 makedec1 calldec1 Guido writes: > Since Mark ended up agreeing with your order, the OK is given by > default. :-) Wait... I'm confused. What was the final decision? I favor evalname1 evalargs1 makedec1 evalname2 evalargs2 makedec2 evalname3 evalargs3 makedec3 calldec3 calldec2 calldec1 becase of left-to-right-top-to-bottom evaluation. Is this what it actually does? I imagine this: >>> # Warning - not actual code >>> def dummyDecorator1(x): ... print 'one' ... return x ... >>> def dummyDecorator2(x): ... print 'two' ... return x ... >>> @dummyDecorator1 >>> @dummyDecorator2 >>> def func(): pass two one which somehow seems wrong to me. -- Michael Chermside From python at rcn.com Tue Aug 17 18:38:15 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 17 18:39:38 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <20040817161833.GA29635@vicky.ecs.soton.ac.uk> Message-ID: <002f01c48478$98b112e0$e841fea9@oemcomputer> [Armin] > I claim that there is nothing unreliable or depending on import order > here, as > long as all concerned parties do the right thing. Right. It's only unreliable if a party takes an obvious, but not wise approach. > Now if there are other > good > reasons to deprecate sys.exitfunc, like it being another way of doing > something for which a better interface is provided, then fine. That is the case. The API for at exit accepts function arguments and automatically handles multiple exit functions. It is careful to handle the multiple exits using pop() in case another exit function alters the function list or in case multiple threads are using atexit. So, yes there is a better API that should be the one preferred way to do it. Raymond From mcherm at mcherm.com Tue Aug 17 18:47:37 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 17 18:45:27 2004 Subject: [Python-Dev] Decorator order implemented backwards? Message-ID: <1092761257.412236a9e7f9f@mcherm.com> I wrote: > Wait... I'm confused. What was the final decision? I favor [...] Never mind. I somehow missed Mark's second email. -- Michael Chermside From pje at telecommunity.com Tue Aug 17 18:52:59 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 17 18:53:56 2004 Subject: [Python-Dev] Decorator order implemented backwards? In-Reply-To: <1092760802.412234e2cb4d5@mcherm.com> Message-ID: <5.1.1.6.0.20040817124720.02d28640@mail.telecommunity.com> At 09:40 AM 8/17/04 -0700, Michael Chermside wrote: >Mark writes: > > Your patch results in the evaluation order: > > > > evalname1 evalargs1 makedec1 > > evalname2 evalargs2 makedec2 > > evalname3 evalargs3 makedec3 > > calldec3 calldec2 calldec1 > > > > Mine (#1009560) gives: > > > > evalname3 evalargs3 makedec3 calldec3 > > evalname2 evalargs2 makedec2 calldec2 > > evalname1 evalargs1 makedec1 calldec1 > >Guido writes: > > Since Mark ended up agreeing with your order, the OK is given by > > default. :-) > >Wait... I'm confused. What was the final decision? I favor > > evalname1 evalargs1 makedec1 > evalname2 evalargs2 makedec2 > evalname3 evalargs3 makedec3 > calldec3 calldec2 calldec1 > >becase of left-to-right-top-to-bottom evaluation. Is this what it >actually does? I imagine this: Hm. Consider this code: func = a(b)(c(d)(e(func))) Per the PEP, it's the equivalent of: @a(b) @c(d) @e def func(...): ... Here's an annotated disassembly: >>> dis.dis(lambda: a(b)(c(d)(e(func)))) 0 SET_LINENO 1 3 LOAD_GLOBAL 0 (a) #evalname1 6 LOAD_GLOBAL 1 (b) #evalargs1 9 CALL_FUNCTION 1 #makedec1 12 LOAD_GLOBAL 2 (c) #evalname2 15 LOAD_GLOBAL 3 (d) #evalargs2 18 CALL_FUNCTION 1 #makedec2 21 LOAD_GLOBAL 4 (e) #evalname3 24 LOAD_GLOBAL 5 (func)#defun 27 CALL_FUNCTION 1 #calldec3 30 CALL_FUNCTION 1 #calldec2 33 CALL_FUNCTION 1 #calldec1 36 RETURN_VALUE Therefore, this is the evaluation order currently prescribed by the PEP. From nas at arctrix.com Tue Aug 17 19:00:05 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Tue Aug 17 19:00:10 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: <16674.12526.711803.480295@montanaro.dyndns.org> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> <200408170121.i7H1Ltn14417@guido.python.org> <20040817111622.GA12954@vicky.ecs.soton.ac.uk> <200408171528.i7HFS3t16093@guido.python.org> <16674.12526.711803.480295@montanaro.dyndns.org> Message-ID: <20040817170005.GA11751@mems-exchange.org> On Tue, Aug 17, 2004 at 11:23:10AM -0500, Skip Montanaro wrote: > Care to reword that without the double negative? I tried and came > up with > > That's sometimes a reason to deprecate something. > > which seems like you're saying that breaking existing > well-behaving modules is sometimes a good reason to deprecate > something. I think Guido's point was that deprecating something is not the same as making a backwards incompatible change. It probably would be good to deprecate features that will be removed in 3.0. IMO, the sys.exitfunc should not be renamed to sys._exitfunc before 3.0. Neil From amk at amk.ca Tue Aug 17 19:16:22 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 17 19:17:11 2004 Subject: [Python-Dev] Python 3.0 list of goals Message-ID: <20040817171622.GA10200@rogue.amk.ca> The thread about bytes() is about a Python 3.0 feature. Guido's presentations have mentioned various changes he'd like to make in 3.0, but there's no master list of things that would change. I think it would be useful to have such a list, because it would focus our language development effort on ones that are steps to 3.0, and maybe people can invent ways to smooth the transition. I've started a list in the Wiki at http://www.python.org/moin/Python3.0 , but should it be a PEP? (PEP 3000, perhaps?) --amk From mcherm at mcherm.com Tue Aug 17 19:19:37 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 17 19:17:26 2004 Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called? Message-ID: <1092763177.41223e29382cb@mcherm.com> > Why not go one step further and make all locking > operations in python no-ops if PyEval_InitThreads has never been > called? Currently, we risk python crashing if we call such functions > without calling PyEval_InitThreads. +1. Seems like it can't possibly hurt, and as you say, it's trivial to implement. -- Michael Chermside From bac at OCF.Berkeley.EDU Tue Aug 17 19:24:35 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 17 19:24:44 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <20040817171622.GA10200@rogue.amk.ca> References: <20040817171622.GA10200@rogue.amk.ca> Message-ID: <41223F53.5060501@ocf.berkeley.edu> A.M. Kuchling wrote: > The thread about bytes() is about a Python 3.0 feature. Guido's > presentations have mentioned various changes he'd like to make in 3.0, > but there's no master list of things that would change. > > I think it would be useful to have such a list, because it would focus > our language development effort on ones that are steps to 3.0, and > maybe people can invent ways to smooth the transition. I've started a > list in the Wiki at http://www.python.org/moin/Python3.0 , but should > it be a PEP? (PEP 3000, perhaps?) > I say a PEP. While it can say that these features are just possibilities, I would rather have a controlled place to list this stuff so it is a little more official than having someone randomly come in and start throwing in everything they wished Python 3.0 will have. When it is decided where all of this info will end up I will go through the python-dev Summaries and see if I can find anything there (tried to make it a habit to doc stuff that Guido said would be a Python 3.0 feature. There is also another wiki page on Python 3.0 at http://www.python.org/cgi-bin/moinmoin/PythonThreeDotOh . I also have an initial list going. At PyCON Michael McLay and I discussed getting a PEP written in hopes of getting grant funding from outside sources (read: not the PSF) to implement some of the features. And we figured a PEP was probably as official as we could get short of a specific grant proposal and thus be more acceptable to people considering providing funds. -Brett From bac at OCF.Berkeley.EDU Tue Aug 17 19:27:46 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 17 19:27:56 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <41223F53.5060501@ocf.berkeley.edu> References: <20040817171622.GA10200@rogue.amk.ca> <41223F53.5060501@ocf.berkeley.edu> Message-ID: <41224012.9050500@ocf.berkeley.edu> Brett C. wrote: > A.M. Kuchling wrote: > >> The thread about bytes() is about a Python 3.0 feature. Guido's >> presentations have mentioned various changes he'd like to make in 3.0, >> but there's no master list of things that would change. >> I think it would be useful to have such a list, because it would focus >> our language development effort on ones that are steps to 3.0, and >> maybe people can invent ways to smooth the transition. I've started a >> list in the Wiki at http://www.python.org/moin/Python3.0 , but should >> it be a PEP? (PEP 3000, perhaps?) >> > > I say a PEP. While it can say that these features are just > possibilities, I would rather have a controlled place to list this stuff > so it is a little more official than having someone randomly come in and > start throwing in everything they wished Python 3.0 will have. And I forgot to mention I would like to help out with this. -Brett From rnd at onego.ru Tue Aug 17 19:43:50 2004 From: rnd at onego.ru (Roman Suzi) Date: Tue Aug 17 19:42:53 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4122304E.1070907@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> Message-ID: On Tue, 17 Aug 2004, M.-A. Lemburg wrote: >Michael Hudson wrote: >> Anthony Baxter writes: >>>A big +1 for a bytes() type, though. I'm not sure on the details, >>>but it'd be nice if it was possible to pass a bytes() object to, >>>for instance, write() directly. >> >> If bytes() doesn't implement the read buffer interface, someone >> somewhere is going to need shooting :-) > >Is there any reason you cannot use buffer() ?! Is it mutable? My guess: no: >>> d = u'123124' >>> ddd[0] '1' >>> ddd[1] '\x00' >>> ddd[1] = '1' Traceback (most recent call last): File "", line 1, in ? TypeError: buffer is read-only >It already implements all the necessary things and has been >available for many years. It was in the shadows because we had byte-strings. Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From janssen at parc.com Tue Aug 17 21:09:22 2004 From: janssen at parc.com (Bill Janssen) Date: Tue Aug 17 21:10:48 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 03:45:26 PDT." <200408171045.i7HAjQZ15411@guido.python.org> Message-ID: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> > Does b"foo" really make much of a difference? Yes. My guess is that if you leave it out, you'll see var = u"foo".encode("ASCII") all over the place (assuming that encode() will produce a bytes type). Wouldn't b"foo" be more readable all around? > Is it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of > b"\x66\x6f\x6f"? No, that's true. But if you have a bytes literal syntax, might as well allow \x in it. Bill From barry at python.org Tue Aug 17 21:16:29 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 17 21:16:06 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <20040817171622.GA10200@rogue.amk.ca> References: <20040817171622.GA10200@rogue.amk.ca> Message-ID: <1092770189.9170.89.camel@localhost> On Tue, 2004-08-17 at 13:16, A.M. Kuchling wrote: > it be a PEP? (PEP 3000, perhaps?) +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040817/060d410e/attachment.pgp From martin at v.loewis.de Tue Aug 17 22:07:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 17 22:07:48 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> Message-ID: <41226592.1040109@v.loewis.de> Bill Janssen wrote: > Yes. My guess is that if you leave it out, you'll see > > var = u"foo".encode("ASCII") > > all over the place (assuming that encode() will produce a bytes type). If you also had var = bytes(u"foo") then I guess people would prefer that. People who want to save typing can do b = bytes and, given that the u prefix will be redundant, write var = b("foo") Regards, Martin From bob at redivi.com Tue Aug 17 22:20:49 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 17 22:21:30 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41226592.1040109@v.loewis.de> References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> <41226592.1040109@v.loewis.de> Message-ID: On Aug 17, 2004, at 4:07 PM, Martin v. L?wis wrote: > Bill Janssen wrote: >> Yes. My guess is that if you leave it out, you'll see >> var = u"foo".encode("ASCII") >> all over the place (assuming that encode() will produce a bytes type). > > If you also had > > var = bytes(u"foo") > > then I guess people would prefer that. People who want to save typing > can do > > b = bytes > > and, given that the u prefix will be redundant, write > > var = b("foo") How would you embed raw bytes if the string was unicode? Maybe there should be something roughly equivalent to this: bytesvar = r"delimited packet\x00".decode("string_escape") "string_escape" would probably be a bad name for it, of course. -bob From pgimpelj at sympatico.ca Tue Aug 17 22:41:56 2004 From: pgimpelj at sympatico.ca (Paul Gimpelj) Date: Tue Aug 17 22:41:11 2004 Subject: [Python-Dev] multiple instances of python on XP Message-ID: <029201c4849a$a43763e0$c901010a@zoom> hi, I was wondering how to have multiple instances of python running on XP. I have ZOPE and TSW installed, but ZOPE's python is grabbing paths from TSW's python. specifically I changed to dir "...\Zope-2.7.2-0\bin" and in cmd window ran python did help() , sys and executable= r'...\Zope-2.7.2-0\bin\python.exe' but exec_prefix = r'...\TSW\Apache2\python' path_importer_cache = ...\TSW|Apache2\python... TSW's pyhon is ver 2.3.4 ZOPE python is 2.3.3 so Is there a way to prevent this mixup? I would like to have ZOPE installed as a service, and that means a python is running as a service. I would like to multiple python services running, and have them stay in their own installed areas. Is there a way to define the internal paths, and not to use XP's search PATH Would I have to rename python to python_1, Python_2, etc? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040817/296e39b1/attachment.htm From martin at v.loewis.de Tue Aug 17 23:11:06 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 17 23:11:24 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> <41226592.1040109@v.loewis.de> Message-ID: <4122746A.2010106@v.loewis.de> Bob Ippolito wrote: > How would you embed raw bytes if the string was unicode? The most direct notation would be bytes("delimited packet\x00") However, people might not understand what is happening, and Guido doesn't like it if the bytes are >127. Regards, Martin From martin at v.loewis.de Tue Aug 17 23:14:56 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 17 23:14:55 2004 Subject: [Python-Dev] multiple instances of python on XP In-Reply-To: <029201c4849a$a43763e0$c901010a@zoom> References: <029201c4849a$a43763e0$c901010a@zoom> Message-ID: <41227550.6030907@v.loewis.de> Paul Gimpelj wrote: > I was wondering how to have multiple instances of python running on XP. python-dev is for the development *of* Python, not for the development *with* Python; please post your question to an appropriate forum. I see that you have also posted the question to python-help. I personally think it is impolite to cross-post questions without indicating so - people might try to research answers for you when you already might have found them elsewhere. Regards, Martin From bob at redivi.com Tue Aug 17 23:18:38 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 17 23:19:15 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4122746A.2010106@v.loewis.de> References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> <41226592.1040109@v.loewis.de> <4122746A.2010106@v.loewis.de> Message-ID: <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> On Aug 17, 2004, at 5:11 PM, Martin v. L?wis wrote: > Bob Ippolito wrote: >> How would you embed raw bytes if the string was unicode? > > The most direct notation would be > > bytes("delimited packet\x00") > > However, people might not understand what is happening, and > Guido doesn't like it if the bytes are >127. I guess that was a bad example, what if the delimiter was \xff? I know that map(ord, u'delimited packet\xff') would get correct results.. but I don't think I like that either. -bob From martin at v.loewis.de Tue Aug 17 23:31:09 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 17 23:31:16 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> <41226592.1040109@v.loewis.de> <4122746A.2010106@v.loewis.de> <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> Message-ID: <4122791D.7050003@v.loewis.de> Bob Ippolito wrote: > I guess that was a bad example, what if the delimiter was \xff? I would write bytes("delimited packet\xff") Repeating myself: Guido wouldn't. Regards, Martin From guido at python.org Tue Aug 17 23:33:54 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 23:34:01 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 11:48:04 EDT." <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> Message-ID: <200408172133.i7HLXsS16658@guido.python.org> > So, how will it be different from: > > from array import array > > def bytes(*initializer): > return array('B',*initializer) > > Even if it's desirable for 'bytes' to be an actual type (e.g. subclassing > ArrayType), it might help the definition process to describe the difference > between the new type and a byte array. Not a whole lot different, except for the ability to use a string as alternate argument to the constructor, and the fact that it's going to be an actual type, and that it should support the buffer API (which array mysteriously doesn't?). The string argument support may not even be necessary -- an alternative way to spell that would be to let s.decode() return a bytes object, which has the advantage of being explicit about the encoding; there's even a base64 encoding already! But it would be a bigger incompatibility, more likely to break existing code using decode() and expecting to get a string. --Guido van Rossum (home page: http://www.python.org/~guido/) From pgimpelj at sympatico.ca Tue Aug 17 23:48:47 2004 From: pgimpelj at sympatico.ca (Paul Gimpelj) Date: Tue Aug 17 23:48:00 2004 Subject: [Python-Dev] multiple instances of python on XP References: <029201c4849a$a43763e0$c901010a@zoom> <41227550.6030907@v.loewis.de> Message-ID: <02e001c484a3$faed7c20$c901010a@zoom> I have searched all the documentation online and google. I thought that developers would interested in public input. and I got email back that everybody is on vacation till sept 6. Since it is open source, I would think that this an import issue since a python advocate would want python's user base to increase. And if python , by design, is incapable of coexisting with all the other user programs and itself, what use is it? on the most popular operating system in the entire world. I do work with linux. regards, Paul ----- Original Message ----- From: "Martin v. L?wis" To: "Paul Gimpelj" Cc: Sent: Tuesday, August 17, 2004 5:14 PM Subject: Re: [Python-Dev] multiple instances of python on XP > Paul Gimpelj wrote: > > I was wondering how to have multiple instances of python running on XP. > > python-dev is for the development *of* Python, not for the development > *with* Python; please post your question to an appropriate forum. > > I see that you have also posted the question to python-help. I > personally think it is impolite to cross-post questions without > indicating so - people might try to research answers for you when > you already might have found them elsewhere. > > Regards, > Martin > From guido at python.org Tue Aug 17 23:48:35 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 23:48:42 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: Your message of "Tue, 17 Aug 2004 17:18:33 BST." <20040817161833.GA29635@vicky.ecs.soton.ac.uk> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> <200408170121.i7H1Ltn14417@guido.python.org> <20040817111622.GA12954@vicky.ecs.soton.ac.uk> <200408171528.i7HFS3t16093@guido.python.org> <20040817161833.GA29635@vicky.ecs.soton.ac.uk> Message-ID: <200408172148.i7HLmZx16688@guido.python.org> > On Tue, Aug 17, 2004 at 08:28:03AM -0700, Guido van Rossum wrote: > > > Well, none, really. But let's not change the name sys.exitfunc just > > > for the sake of deprecation, because it will probably break existing > > > and well-behaving modules (not just non-well-behaving ones). > > > > That's never been a reason not to deprecate something. > > Sorry. Sure. What I was opposing here is Raymond's original claim in this > thread, which was apparently the reason he wanted to deprecate sys.exitfunc: > > """ > The atexit module does attempt to co-exist by introducing code to > register a function in sys.exitfunc if it were defined before "import > atexit" was called. However, this is unreliable because it depends on > import order and the library is free to introduce an earlier "import > atexit" which would break code relying on the co-existance mechanism. > """ > > I claim that there is nothing unreliable or depending on import > order here, as long as all concerned parties do the right thing. > Now if there are other good reasons to deprecate sys.exitfunc, like > it being another way of doing something for which a better interface > is provided, then fine. Raymond may have overstated his case somewhat. But the fact is that it is easy to abuse sys.exitfunc with a poorly written handler, and the atexit module avoids this. This in itself to me looks like a classic "deprecate the old solution, recommend the new way". TOOWTDI! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 17 23:50:13 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 17 23:50:20 2004 Subject: [Python-Dev] Deprecate sys.exitfunc? In-Reply-To: Your message of "Tue, 17 Aug 2004 13:00:05 EDT." <20040817170005.GA11751@mems-exchange.org> References: <001501c48310$bf15aa40$e841fea9@oemcomputer> <20040816113357.GB19969@vicky.ecs.soton.ac.uk> <200408170121.i7H1Ltn14417@guido.python.org> <20040817111622.GA12954@vicky.ecs.soton.ac.uk> <200408171528.i7HFS3t16093@guido.python.org> <16674.12526.711803.480295@montanaro.dyndns.org> <20040817170005.GA11751@mems-exchange.org> Message-ID: <200408172150.i7HLoEL16716@guido.python.org> > IMO, the sys.exitfunc should not be renamed to sys._exitfunc before > 3.0. Agreed. But it can be deprecated right away. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Tue Aug 17 23:58:27 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 17 23:58:31 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> Message-ID: <41227F83.4060403@egenix.com> Roman Suzi wrote: > On Tue, 17 Aug 2004, M.-A. Lemburg wrote: > > >>Michael Hudson wrote: >> >>>Anthony Baxter writes: >>> >>>>A big +1 for a bytes() type, though. I'm not sure on the details, >>>>but it'd be nice if it was possible to pass a bytes() object to, >>>>for instance, write() directly. >>> >>>If bytes() doesn't implement the read buffer interface, someone >>>somewhere is going to need shooting :-) >> >>Is there any reason you cannot use buffer() ?! > > Is it mutable? > My guess: no: The buffer object itself can be read-only or read-write. Unfortunately, the buffer() built-in always returns read-only buffers. At C level it is easy to create a buffer object from a read-write capable object. >>>>d = u'123124' >>>>ddd[0] > > '1' > >>>>ddd[1] > > '\x00' > >>>>ddd[1] = '1' > > Traceback (most recent call last): > File "", line 1, in ? > TypeError: buffer is read-only > > >>It already implements all the necessary things and has been >>available for many years. > > > It was in the shadows because we had byte-strings. Right, so why not revive it ?! Anyway, this whole discussion about a new bytes type doesn't really solve the problem that the b'...' literal was intended for: that of having a nice way to define (read-only) 8-bit binary string literals. We already have a number of read-write types for storing binary data, e.g. arrays, cStringIO and buffers. Inventing yet another way to spell binary data won't make life easier. However, what will be missing is a nice way to spell read-only binary data. Since 'tada' will return a Unicode object in Py3k, I think we should reuse the existing 8-bit string object under the new literal constructor b'tada\x00' (and apply the same source code encoding semantics we apply today for 'tada\x00'). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bob at redivi.com Tue Aug 17 23:59:19 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Aug 17 23:59:58 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408172133.i7HLXsS16658@guido.python.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <200408172133.i7HLXsS16658@guido.python.org> Message-ID: On Aug 17, 2004, at 5:33 PM, Guido van Rossum wrote: >> So, how will it be different from: >> >> from array import array >> >> def bytes(*initializer): >> return array('B',*initializer) >> >> Even if it's desirable for 'bytes' to be an actual type (e.g. >> subclassing >> ArrayType), it might help the definition process to describe the >> difference >> between the new type and a byte array. > > Not a whole lot different, except for the ability to use a string as > alternate argument to the constructor, and the fact that it's going to > be an actual type, and that it should support the buffer API (which > array mysteriously doesn't?). > > The string argument support may not even be necessary -- an > alternative way to spell that would be to let s.decode() return a > bytes object, which has the advantage of being explicit about the > encoding; there's even a base64 encoding already! But it would be a > bigger incompatibility, more likely to break existing code using > decode() and expecting to get a string. IMHO current uses of decode and encode are really confusing. Many decodes are from str -> unicode, and many encodes are from unicode -> str (or str -> unicode -> str implicitly, which is usually going to fail miserably)... while yet others like zlib, base64, etc. are str <-> str. Technically unicode.decode(base64) should certainly work, but it doesn't because unicode doesn't have a decode method. I don't have a proposed solution at the moment, but perhaps these operations should either be outside of the data types altogether (i.e. use codecs only) or there should be separate methods for doing separate things (character translations versus data->data transformations). -bob From guido at python.org Wed Aug 18 00:01:12 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 18 00:01:21 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 17:18:38 EDT." <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> <41226592.1040109@v.loewis.de> <4122746A.2010106@v.loewis.de> <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> Message-ID: <200408172201.i7HM1CT16805@guido.python.org> > >> How would you embed raw bytes if the string was unicode? > > > > The most direct notation would be > > > > bytes("delimited packet\x00") > > > > However, people might not understand what is happening, and > > Guido doesn't like it if the bytes are >127. > > I guess that was a bad example, what if the delimiter was \xff? I know > that map(ord, u'delimited packet\xff') would get correct results.. but > I don't think I like that either. Maybe the constructor could be bytes([, ])? --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Wed Aug 18 00:04:45 2004 From: foom at fuhm.net (James Y Knight) Date: Wed Aug 18 00:04:49 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com> <41226592.1040109@v.loewis.de> <4122746A.2010106@v.loewis.de> <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> Message-ID: <72B8535D-F099-11D8-8449-000A95A50FB2@fuhm.net> On Aug 17, 2004, at 5:18 PM, Bob Ippolito wrote: > On Aug 17, 2004, at 5:11 PM, Martin v. L?wis wrote: > >> Bob Ippolito wrote: >>> How would you embed raw bytes if the string was unicode? >> >> The most direct notation would be >> >> bytes("delimited packet\x00") >> >> However, people might not understand what is happening, and >> Guido doesn't like it if the bytes are >127. > > I guess that was a bad example, what if the delimiter was \xff? Indeed, if all strings are unicode, the question becomes: what encoding does bytes() use to translate unicode characters to bytes. Two alternatives have been proposed so far: 1) ASCII (translate chars as their codepoint if < 128, else error) 2) ISO-8859-1 (translate chars as their codepoint if < 256, else error) I think I'd choose #2, myself. > I know that map(ord, u'delimited packet\xff') would get correct > results.. but I don't think I like that either. Why would you consider that wrong? ord(u'\xff') *should* return 255. Just as ord(u'\u1000') returns 4096. There's nothing mysterious there. James From mal at egenix.com Wed Aug 18 00:25:21 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 00:25:27 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <411D1EB4.10103@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> Message-ID: <412285D1.7010907@egenix.com> Walter D?rwald wrote: > M.-A. Lemburg wrote: >> Overall, I don't like the idea of adding extra >> APIs breaking the existing codec API. > > Adding a final argument that defaults to False doesn't > break the API for the callers, only for the implementors. > > And if we drop the final argument, the API is completely > backwards compatible both for users and implementors. > The only thing that gets added is the feed() method, > that implementors don't have to overwrite. > >> I believe that we can >> extend stream codecs to also work in a feed mode without >> breaking the API. > > Abandoning the final argument and adding a feed() method > would IMHO be the simplest way to do this. > > But then there's no way to make sure that every byte from > the input stream is really consumed. I've thought about this some more. Perhaps I'm still missing something, but wouldn't it be possible to add a feeding mode to the existing stream codecs by creating a new queue data type (much like the queue you have in the test cases of your patch) and using the stream codecs on these ? I think such a queue would be generally useful in other contexts as well, e.g. for implementing fast character based pipes between threads, non-Unicode feeding parsers, etc. Using such a type you could potentially add a feeding mode to stream or file-object API based algorithms very easily. We could then have a new class, e.g. FeedReader, which wraps the above in a nice API, much like StreamReaderWriter and StreamRecoder. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bac at OCF.Berkeley.EDU Wed Aug 18 00:29:25 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 18 00:29:33 2004 Subject: [Python-Dev] multiple instances of python on XP In-Reply-To: <02e001c484a3$faed7c20$c901010a@zoom> References: <029201c4849a$a43763e0$c901010a@zoom> <41227550.6030907@v.loewis.de> <02e001c484a3$faed7c20$c901010a@zoom> Message-ID: <412286C5.9010802@ocf.berkeley.edu> Paul Gimpelj wrote: > I have searched all the documentation online and google. > > I thought that developers would interested in public input. > We do care about public input, but not on possible bugs delivered this way. It is easier for everyone if bugs are reported through the SourceForge bug tracker at http://www.sf.net/projects/python . > and I got email back that everybody is on vacation till sept 6. > Don't know who that is from but is most likely a single person who forgot to turn off their mailing list subscriptions. > Since it is open source, > > I would think that this an import issue since a python advocate would want > python's user base to increase. > I am going to assume you meant "important" instead of "import". Yes, we would like to see usage grow, but we just don't have time to try to personally fix every bug through python-dev. That is why we have the SF bug tracker. > And if python , by design, is incapable of coexisting with all the other > user programs and itself, > what use is it? Python can coexist, just don't expect it to come out in a hemp-based tie-die shirt holding a bunch of posies. =) I would wait and see if python-help can turn up some leads. If not you can try comp.lang.python since I suspect this is just a configuration error. -Brett From mal at egenix.com Wed Aug 18 00:30:27 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 00:30:29 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <200408172133.i7HLXsS16658@guido.python.org> Message-ID: <41228703.40207@egenix.com> Bob Ippolito wrote: > > On Aug 17, 2004, at 5:33 PM, Guido van Rossum wrote: > >>> So, how will it be different from: >>> >>> from array import array >>> >>> def bytes(*initializer): >>> return array('B',*initializer) >>> >>> Even if it's desirable for 'bytes' to be an actual type (e.g. >>> subclassing >>> ArrayType), it might help the definition process to describe the >>> difference >>> between the new type and a byte array. >> >> >> Not a whole lot different, except for the ability to use a string as >> alternate argument to the constructor, and the fact that it's going to >> be an actual type, and that it should support the buffer API (which >> array mysteriously doesn't?). >> >> The string argument support may not even be necessary -- an >> alternative way to spell that would be to let s.decode() return a >> bytes object, which has the advantage of being explicit about the >> encoding; there's even a base64 encoding already! But it would be a >> bigger incompatibility, more likely to break existing code using >> decode() and expecting to get a string. > > > IMHO current uses of decode and encode are really confusing. Many > decodes are from str -> unicode, and many encodes are from unicode -> > str (or str -> unicode -> str implicitly, which is usually going to fail > miserably)... while yet others like zlib, base64, etc. are str <-> str. > Technically unicode.decode(base64) should certainly work, but it doesn't > because unicode doesn't have a decode method. They do in 2.4. Note that in 2.4 .decode() and .encode() guarantee that you get a basestring instance. If you want more flexibility in terms of return type, the new codecs.encode() and codecs.decode() will allow arbitrary types as return value. > I don't have a proposed solution at the moment, but perhaps these > operations should either be outside of the data types altogether (i.e. > use codecs only) or there should be separate methods for doing separate > things (character translations versus data->data transformations). It all depends on whether you are discussing placing binary data into the Python source file (by some means of using literals) or just working with bytes you got from a file, generator, socket, etc. The current discussion is mixing these contexts a bit too much, I believe, which is probably why people keep misunderstanding each other (at least that's how I perceive the debate). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From tim.peters at gmail.com Wed Aug 18 00:39:22 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 18 00:39:31 2004 Subject: [Python-Dev] multiple instances of python on XP In-Reply-To: <029201c4849a$a43763e0$c901010a@zoom> References: <029201c4849a$a43763e0$c901010a@zoom> Message-ID: <1f7befae0408171539138aa997@mail.gmail.com> [Paul Gimpelj] > I was wondering how to have multiple instances of python running on > XP. Multiple instances of Python are routinely used on all flavors of Windows without problems (at the moment, I have 4 on my XP box, and often run 3 simultaneously). > I have ZOPE and TSW installed, but ZOPE's python is grabbing paths > from TSW's python. Then you need to ask on Zope and TSW (unsure what that is) lists about why those applications are fighting with each other. Asking here is like asking on a bash developer's list because you run two apps from the shell that insist on having different PATHs. There's nothing bash can do about that -- and the question is better addressed to a bash user's list even if you think there should be. > specifically I > > changed to dir "...\Zope-2.7.2-0\bin" > and in cmd window ran python > > did help() , sys > and executable= r'...\Zope-2.7.2-0\bin\python.exe' > but exec_prefix = r'...\TSW\Apache2\python' > path_importer_cache = ....\TSW|Apache2\python... Well, Python isn't forcing the use of TSW, and Zope certainly isn't, so if I were you I'd ask the TSW developers what they're up to. I have no idea how they manage to change Zope's Python's idea of what exec_prefix should be. Zope sets no environment variables or registry entries related to Python, and Zope runs happily without interfering with a user-installed Python, so TSW is your hangup. From janssen at parc.com Wed Aug 18 01:13:21 2004 From: janssen at parc.com (Bill Janssen) Date: Wed Aug 18 01:14:00 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Tue, 17 Aug 2004 13:07:46 PDT." <41226592.1040109@v.loewis.de> Message-ID: <04Aug17.161328pdt."58612"@synergy1.parc.xerox.com> Yes, that works for me. Martin writes: > If you also had > > var = bytes(u"foo") > > then I guess people would prefer that. People who want to save typing > can do > > b = bytes > > and, given that the u prefix will be redundant, write > > var = b("foo") Bill From guido at python.org Wed Aug 18 01:21:06 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 18 01:21:13 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: Your message of "Tue, 17 Aug 2004 10:24:35 PDT." <41223F53.5060501@ocf.berkeley.edu> References: <20040817171622.GA10200@rogue.amk.ca> <41223F53.5060501@ocf.berkeley.edu> Message-ID: <200408172321.i7HNL6L17005@guido.python.org> > A.M. Kuchling wrote: > > > The thread about bytes() is about a Python 3.0 feature. Guido's > > presentations have mentioned various changes he'd like to make in 3.0, > > but there's no master list of things that would change. > > > > I think it would be useful to have such a list, because it would focus > > our language development effort on ones that are steps to 3.0, and > > maybe people can invent ways to smooth the transition. I've started a > > list in the Wiki at http://www.python.org/moin/Python3.0 , but should > > it be a PEP? (PEP 3000, perhaps?) Good idea to start collecting all this. I'm currently too busy to spend a lot of time thinking about all this, but eventually I will be back and then it would be nice to find a list of things I said in the past. :-) > I say a PEP. While it can say that these features are just > possibilities, I would rather have a controlled place to list this stuff > so it is a little more official than having someone randomly come in and > start throwing in everything they wished Python 3.0 will have. Right, but the Wiki is a good start while you consider a PEP. > When it is decided where all of this info will end up I will go through > the python-dev Summaries and see if I can find anything there (tried to > make it a habit to doc stuff that Guido said would be a Python 3.0 feature. > > There is also another wiki page on Python 3.0 at > http://www.python.org/cgi-bin/moinmoin/PythonThreeDotOh . Hrm. That is mostly Mike McLay's wish list; best to ignore it. > I also have an initial list going. At PyCON Michael McLay and I > discussed getting a PEP written in hopes of getting grant funding > from outside sources (read: not the PSF) to implement some of the > features. And we figured a PEP was probably as official as we could > get short of a specific grant proposal and thus be more acceptable > to people considering providing funds. A PEP along the lines of Mike's Wiki page doesn't really have my support. It's a long story that I'll have to save for another time, but basically it seems to me that Mike's agenda is driven by the need of a very specific subcommunity. That's fine, but let's be clear about what it is -- it's not my idea for Python 3000. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Wed Aug 18 04:03:03 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Aug 18 04:03:26 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408172133.i7HLXsS16658@guido.python.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <200408172133.i7HLXsS16658@guido.python.org> Message-ID: <16674.47319.319502.989563@montanaro.dyndns.org> Guido> The string argument support may not even be necessary -- an Guido> alternative way to spell that would be to let s.decode() return a Guido> bytes object, which has the advantage of being explicit about the Guido> encoding; there's even a base64 encoding already! I'm sorry folks, but I still don't understand all this discussion overlap between unicode/string objects (which require explicit or implicit decoding) and bytes objects (which clearly must not). Everyone keeps talking about decoding stuff into bytes objects and whether or not bytes literals would be compatible with the current source encoding. My understanding is that bytes objects are just that, raw sequences of bytes in the range 0x00 to 0xff, inclusive, with no interpretation of any type. Skip From paul at prescod.net Wed Aug 18 04:15:54 2004 From: paul at prescod.net (Paul Prescod) Date: Wed Aug 18 04:16:05 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41227F83.4060403@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> Message-ID: <4122BBDA.4050401@prescod.net> M.-A. Lemburg wrote: >... > > > We already have a number of read-write types for storing binary > data, e.g. arrays, cStringIO and buffers. Inventing yet another > way to spell binary data won't make life easier. The point is canonicalize one and start to make APIs that expect it. Otherwise we will never make the leap from the bad habit of using strings as byte arrays. Can you pass arrays to the write() function? Can you decode buffers to strings? A byte array type would have a certain mix of functions and API compatibility that is missing from the plethora of similar thingees. Paul Prescod From greg at cosc.canterbury.ac.nz Wed Aug 18 05:01:34 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 18 05:01:39 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <20040817171622.GA10200@rogue.amk.ca> Message-ID: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> "A.M. Kuchling" : > I think it would be useful to have such a list, because it would focus > our language development effort on ones that are steps to 3.0, and > maybe people can invent ways to smooth the transition. I've started a > list in the Wiki at http://www.python.org/moin/Python3.0 , but should > it be a PEP? (PEP 3000, perhaps?) As a point of interest, are Python 3.0 and Python 3000 actually the same thing? Python 3000 is clearly some hypothetical version at some unspecified time in the future, which may or may not ever exist. But at the rate version numbers are going, we're going to actually get to 3.0 before very long. So, when we talk about 3.0, are we talking about a wish version or a real version? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido at python.org Wed Aug 18 05:16:46 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 18 05:16:55 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: Your message of "Wed, 18 Aug 2004 15:01:34 +1200." <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> References: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408180316.i7I3Gk217306@guido.python.org> > As a point of interest, are Python 3.0 and Python 3000 actually the > same thing? In my mind, yes. > Python 3000 is clearly some hypothetical version at some > unspecified time in the future, which may or may not ever exist. But > at the rate version numbers are going, we're going to actually get to > 3.0 before very long. So, when we talk about 3.0, are we talking about > a wish version or a real version? Actually, with one minor version number bump per 12-18 months, we should have at least 5 years before we're at 2.9. So I think we can continue to equate 3.0 and 3000. It would be nice PR to actually call it "Python 3000" rather than "Python 3.0" when it comes out -- showing we can play the marketing game as well as anybody without losing our sense of humor. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Wed Aug 18 05:52:46 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 18 05:52:49 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <200408170114.i7H1Elk14352@guido.python.org> References: <002001c48325$436d9460$e841fea9@oemcomputer> <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> <200408170114.i7H1Elk14352@guido.python.org> Message-ID: <1f7befae040817205233d70086@mail.gmail.com> [Guido] > (In addition, I find that the GIL often causes people to worry > unnecessarily about the GIL. Sort of like worrying about relativistic > effects when driving a car.) LOL! OTOH, well-endowed gentlemen have always driven as fast as possible, in order to avoid frightening their admiring lady onlookers. To be effective, I have to scrunch over rather uncomfortably, though, since Lorentz contraction only occurs along the direction of observed motion. done-right-it-only-takes-0.97c-to-give-the-appearance-of-6-inches-ly y'rs - tim From barry at python.org Wed Aug 18 06:23:00 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 18 06:22:35 2004 Subject: [Python-Dev] Re: Re: Update PEP 292 In-Reply-To: References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> Message-ID: <1092802980.7810.29.camel@geddy.wooz.org> On Wed, 2004-08-11 at 10:54, Fernando Perez wrote: > I guess my post wasn't worded too clearly. I did mean existing, in the sense > that the error above occurs with current python (it's cut-and-paste from python > 2.2). My point was precisely to illustrate this failure with a simple example, > to then ask whether the new scheme could be made, _as shipped_, to accept this > kind of expansion: > > print template('sys.platform is: $sys.platform') % locals() Not as shipped. You'd have you derive from the Template class and write a regular expression (or adapt the existing pattern). > It may be trivial to extend pep-292 for this, but not having to do it in every > small utility you write/ship would be very useful, IMHO. If there are no > technical reasons to prevent it, is the 'too complex' argument really that > strong (especially since you state it's a trivial amount of code)? Well, it's about 3 lines of Python, but what would you call the class, AttributeTemplate? I wouldn't be opposed to adding the class, since I think it would be pretty useful. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040818/62a441f5/attachment.pgp From barry at python.org Wed Aug 18 06:24:44 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 18 06:24:19 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <20040811151347.GA18312@rogue.amk.ca> References: <1092177771.21839.54.camel@localhost> <20040811151347.GA18312@rogue.amk.ca> Message-ID: <1092803084.7834.32.camel@geddy.wooz.org> On Wed, 2004-08-11 at 11:13, A.M. Kuchling wrote: > On Tue, Aug 10, 2004 at 06:42:51PM -0400, Barry Warsaw wrote: > > The updated PEP is at: > > http://www.python.org/peps/pep-0292.html > > Are you sure that people aren't going to want to format numbers > as they do with % (e.g. %4.2f)? I'm sure they will, but I think they should just use %-interpolation then. ;) I'm personally not interested in supporting that for (default) $-strings. > You should post the PEP to c.l.py.announce, too. (Perhaps you have > and it's still sitting in the queue.) I hadn't, but good point. I'll do that. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040818/5125de4b/attachment.pgp From rnd at onego.ru Wed Aug 18 06:30:58 2004 From: rnd at onego.ru (Roman Suzi) Date: Wed Aug 18 06:30:01 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41227F83.4060403@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> Message-ID: On Tue, 17 Aug 2004, M.-A. Lemburg wrote: >Roman Suzi wrote: >> On Tue, 17 Aug 2004, M.-A. Lemburg wrote: >> It was in the shadows because we had byte-strings. > >Right, so why not revive it ?! > >Anyway, this whole discussion about a new bytes type doesn't >really solve the problem that the b'...' literal was >intended for: that of having a nice way to define (read-only) >8-bit binary string literals. I think new _mutable_ bytes() type is better than old 8-bit binary strings for binary data processing purposes. Or do we need them for legacy text-procesing software? >We already have a number of read-write types for storing binary >data, e.g. arrays, cStringIO and buffers. Inventing yet another >way to spell binary data won't make life easier. > >However, what will be missing is a nice way to spell read-only >binary data. > >Since 'tada' will return a Unicode object in Py3k, I think we >should reuse the existing 8-bit string object under the new >literal constructor b'tada\x00' (and apply the same source code >encoding semantics we apply today for 'tada\x00'). > > Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From martin at v.loewis.de Wed Aug 18 06:57:02 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 06:57:04 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412285D1.7010907@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> Message-ID: <4122E19E.9050906@v.loewis.de> M.-A. Lemburg wrote: > I've thought about this some more. Perhaps I'm still missing > something, but wouldn't it be possible to add a feeding > mode to the existing stream codecs by creating a new queue > data type (much like the queue you have in the test cases of > your patch) and using the stream codecs on these ? Here is the problem. In UTF-8, how does the actual algorithm tell (the application) that the bytes it got on decoding provide for three fully decodable characters, and that 2 bytes are left undecoded, and that those bytes are not inherently ill-formed, but lack a third byte to complete the multi-byte sequence? On top of that, you can implement whatever queuing or streaming APIs you want, but you *need* an efficient way to communicate incompleteness. Regards, Martin From greg at cosc.canterbury.ac.nz Wed Aug 18 06:58:28 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 18 06:58:33 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <1f7befae040817205233d70086@mail.gmail.com> Message-ID: <200408180458.i7I4wS2a023568@cosc353.cosc.canterbury.ac.nz> > done-right-it-only-takes-0.97c-to-give-the-appearance-of-6-inches-ly > y'rs - tim Hmmm, so Tim is claiming to have an actual length of >>> 6 / sqrt(1 - (0.97 * 0.97)) 24.680702093691806 inches... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From martin at v.loewis.de Wed Aug 18 07:01:50 2004 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed Aug 18 07:01:47 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41227F83.4060403@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> Message-ID: <4122E2BE.1070101@v.loewis.de> M.-A. Lemburg wrote: > Anyway, this whole discussion about a new bytes type doesn't > really solve the problem that the b'...' literal was > intended for: that of having a nice way to define (read-only) > 8-bit binary string literals. But why do you need a way to spell 8-bit string literals? You can always do "string".encode("L1") If that is too much typing, do def b(s):return s.encode("L1") b("string") Regards, Martin From martin at v.loewis.de Wed Aug 18 07:02:45 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 07:02:42 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <200408172133.i7HLXsS16658@guido.python.org> Message-ID: <4122E2F5.5080508@v.loewis.de> Bob Ippolito wrote: > > IMHO current uses of decode and encode are really confusing. I completely agree. This overgeneralization is insane. Regards, Martin From martin at v.loewis.de Wed Aug 18 07:09:18 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 07:09:17 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16674.47319.319502.989563@montanaro.dyndns.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <200408172133.i7HLXsS16658@guido.python.org> <16674.47319.319502.989563@montanaro.dyndns.org> Message-ID: <4122E47E.1040806@v.loewis.de> Skip Montanaro wrote: > My understanding is that bytes > objects are just that, raw sequences of bytes in the range 0x00 to 0xff, > inclusive, with no interpretation of any type. Yes, but your understanding is limited :-) This idea is good, but it falls short once we talk about source code, because source code does have an encoding. So if you don't want to incorporate the notion of encodings into the byte string types, yet be able to declare them in source code, you have to go for a numeric representation. I.e. you write bytes(71,69, 84) instead of b"GET" As soon as you use some kind of string notation for denoting byte code values, you immediately *have* to deal with encodings. Regards, Martin From martin at v.loewis.de Wed Aug 18 07:13:25 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 07:13:21 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> References: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> Message-ID: <4122E575.9020108@v.loewis.de> Greg Ewing wrote: > As a point of interest, are Python 3.0 and Python 3000 actually the > same thing? Python 3000 is clearly some hypothetical version at some > unspecified time in the future, which may or may not ever exist. But > at the rate version numbers are going, we're going to actually get to > 3.0 before very long. I can't see that :-) After 2.9, we get 2.10, then 2.11. After 2.99, we get 2.100, then 2.101. At that rate, we won't reach 3.0 before that heath death (sp?) of the universe. Regards, Martin From allison at sumeru.stanford.EDU Wed Aug 18 07:33:54 2004 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Wed Aug 18 07:34:11 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <4122E575.9020108@v.loewis.de> Message-ID: Ahhhh... Zeno's paradox again. We'll get there eventually, for all practical purposes. On Wed, 18 Aug 2004, [ISO-8859-1] "Martin v. L=F6wis" wrote: > Greg Ewing wrote: > > As a point of interest, are Python 3.0 and Python 3000 actually the > > same thing? Python 3000 is clearly some hypothetical version at some > > unspecified time in the future, which may or may not ever exist. But > > at the rate version numbers are going, we're going to actually get to > > 3.0 before very long. >=20 > I can't see that :-) After 2.9, we get 2.10, then 2.11. After 2.99, > we get 2.100, then 2.101. At that rate, we won't reach 3.0 before > that heath death (sp?) of the universe. >=20 > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40= sumeru.stanford.edu >=20 From mal at egenix.com Wed Aug 18 10:36:06 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 10:36:09 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4122E19E.9050906@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> Message-ID: <412314F6.5060502@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> I've thought about this some more. Perhaps I'm still missing >> something, but wouldn't it be possible to add a feeding >> mode to the existing stream codecs by creating a new queue >> data type (much like the queue you have in the test cases of >> your patch) and using the stream codecs on these ? > > Here is the problem. In UTF-8, how does the actual algorithm > tell (the application) that the bytes it got on decoding provide > for three fully decodable characters, and that 2 bytes are left > undecoded, and that those bytes are not inherently ill-formed, > but lack a third byte to complete the multi-byte sequence? This state can be stored in the stream codec instance, e.g. by using a special state object that is stored in the instance and passed to the encode/decode APIs of the codec or by implementing the stream codec itself in C. We do need to extend the API between the stream codec and the encode/decode functions, no doubt about that. However, this is an extension that is well hidden from the user of the codec and won't break code. > On top of that, you can implement whatever queuing or streaming > APIs you want, but you *need* an efficient way to communicate > incompleteness. Agreed. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 10:41:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 10:41:39 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4122E2BE.1070101@v.loewis.de> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122E2BE.1070101@v.loewis.de> Message-ID: <41231640.2020706@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> Anyway, this whole discussion about a new bytes type doesn't >> really solve the problem that the b'...' literal was >> intended for: that of having a nice way to define (read-only) >> 8-bit binary string literals. > > But why do you need a way to spell 8-bit string literals? > > You can always do > > "string".encode("L1") > > If that is too much typing, do > > def b(s):return s.encode("L1") > > b("string") You need to think about the important use-case of having to convert Py2 applications to Py3 style. In many cases, the application can be made to run under Py3 be adding the small 'b' in front of the used string literals. Even better: if we add the b'xxx' notation now, we could start working towards the switch-over by slowly converting the Python standard library to actually work in -U mode (which basically implements the switch-over by making 'abc' behave as u'abc'). Since the code is already in place and the change is minimal, I don't see any reason not to use it. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 10:46:55 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 10:46:57 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4122E47E.1040806@v.loewis.de> References: <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <200408172133.i7HLXsS16658@guido.python.org> <16674.47319.319502.989563@montanaro.dyndns.org> <4122E47E.1040806@v.loewis.de> Message-ID: <4123177F.7050306@egenix.com> Martin v. L?wis wrote: > Skip Montanaro wrote: > >> My understanding is that bytes >> objects are just that, raw sequences of bytes in the range 0x00 to 0xff, >> inclusive, with no interpretation of any type. > > > Yes, but your understanding is limited :-) This idea is good, but it > falls short once we talk about source code, because source code does > have an encoding. So if you don't want to incorporate the notion of > encodings into the byte string types, yet be able to declare them > in source code, you have to go for a numeric representation. I.e. > you write > bytes(71,69, 84) > instead of > b"GET" > > As soon as you use some kind of string notation for denoting byte > code values, you immediately *have* to deal with encodings. Of course you do, but aren't you making things too complicated, Martin ? If you write your string literal using just ASCII characters and escapes, I don't see much of a problem with different source code encodings. If it makes you feel better, we could even enforce this by only allowing these characters in binary string literals. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 10:49:52 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 10:49:55 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> Message-ID: <41231830.2090903@egenix.com> Roman Suzi wrote: > On Tue, 17 Aug 2004, M.-A. Lemburg wrote: > > >>Roman Suzi wrote: >> >>>On Tue, 17 Aug 2004, M.-A. Lemburg wrote: > > >>>It was in the shadows because we had byte-strings. >> >>Right, so why not revive it ?! >> >>Anyway, this whole discussion about a new bytes type doesn't >>really solve the problem that the b'...' literal was >>intended for: that of having a nice way to define (read-only) >>8-bit binary string literals. > > I think new _mutable_ bytes() type is better than old 8-bit binary strings > for binary data processing purposes. > Or do we need them for legacy text-procesing software? Hmm, who ever said that we are going to drop the current 8-bit string implementation ? I'm only suggesting to look at what's there instead of trying to redo everything in slightly different way, e.g. you can already get the bytes() functionality from buffer type at C level - it's just that this functionality is not exposed at Python level. >>We already have a number of read-write types for storing binary >>data, e.g. arrays, cStringIO and buffers. Inventing yet another >>way to spell binary data won't make life easier. >> >>However, what will be missing is a nice way to spell read-only >>binary data. >> >>Since 'tada' will return a Unicode object in Py3k, I think we >>should reuse the existing 8-bit string object under the new >>literal constructor b'tada\x00' (and apply the same source code >>encoding semantics we apply today for 'tada\x00'). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 11:02:40 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 11:02:41 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4122BBDA.4050401@prescod.net> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net> Message-ID: <41231B30.2060307@egenix.com> Paul Prescod wrote: > M.-A. Lemburg wrote: > >> ... >> >> >> We already have a number of read-write types for storing binary >> data, e.g. arrays, cStringIO and buffers. Inventing yet another >> way to spell binary data won't make life easier. > > The point is canonicalize one and start to make APIs that expect it. > Otherwise we will never make the leap from the bad habit of using > strings as byte arrays. > > Can you pass arrays to the write() function? Can > you decode buffers to strings? A byte array type would have a certain > mix of functions and API compatibility that is missing from the plethora > of similar thingees. Wouldn't it be possible to extend the existing buffer type to meet those standards ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From anthony at interlink.com.au Wed Aug 18 08:50:49 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Aug 18 13:12:24 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <200408180316.i7I3Gk217306@guido.python.org> References: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> <200408180316.i7I3Gk217306@guido.python.org> Message-ID: <4122FC49.5040008@interlink.com.au> Guido van Rossum wrote: > Actually, with one minor version number bump per 12-18 months, we > should have at least 5 years before we're at 2.9. And if we still need time, there's 2.X - just like 2.9, but with multicoloured buttons! Anthony -- Anthony Baxter It's never too late to have a happy childhood. From gjc at inescporto.pt Wed Aug 18 14:11:07 2004 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Wed Aug 18 14:11:18 2004 Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called? In-Reply-To: <1092763177.41223e29382cb@mcherm.com> References: <1092763177.41223e29382cb@mcherm.com> Message-ID: <1092831067.1944.1.camel@emperor> A Ter, 2004-08-17 ?s 18:19, Michael Chermside escreveu: > > Why not go one step further and make all locking > > operations in python no-ops if PyEval_InitThreads has never been > > called? Currently, we risk python crashing if we call such functions > > without calling PyEval_InitThreads. > > +1. Seems like it can't possibly hurt, and as you say, it's trivial > to implement. Thanks for the feedback. http://www.python.org/sf?id=1011380 Regards. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic From s.percivall at chello.se Wed Aug 18 15:25:25 2004 From: s.percivall at chello.se (Simon Percivall) Date: Wed Aug 18 15:25:31 2004 Subject: [Python-Dev] tab in file tkFont.py Message-ID: <10AB5C0F-F11A-11D8-873D-0003934AD54A@chello.se> lib-tk/tkFont.py CVS HEAD has a tab character at line 94, breaking installation. //Simon From bac at OCF.Berkeley.EDU Wed Aug 18 19:07:09 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 18 19:07:19 2004 Subject: [Python-Dev] Python 3.0 list of goals In-Reply-To: <200408172321.i7HNL6L17005@guido.python.org> References: <20040817171622.GA10200@rogue.amk.ca> <41223F53.5060501@ocf.berkeley.edu> <200408172321.i7HNL6L17005@guido.python.org> Message-ID: <41238CBD.1050908@ocf.berkeley.edu> Guido van Rossum wrote: >>A.M. Kuchling wrote: >> >> >>>The thread about bytes() is about a Python 3.0 feature. Guido's >>>presentations have mentioned various changes he'd like to make in 3.0, >>>but there's no master list of things that would change. >>> >>>I think it would be useful to have such a list, because it would focus >>>our language development effort on ones that are steps to 3.0, and >>>maybe people can invent ways to smooth the transition. I've started a >>>list in the Wiki at http://www.python.org/moin/Python3.0 , but should >>>it be a PEP? (PEP 3000, perhaps?) > > > Good idea to start collecting all this. I'm currently too busy to > spend a lot of time thinking about all this, but eventually I will be > back and then it would be nice to find a list of things I said in the > past. :-) > =) Prepping your time machine for ya. > >>I say a PEP. While it can say that these features are just >>possibilities, I would rather have a controlled place to list this stuff >>so it is a little more official than having someone randomly come in and >>start throwing in everything they wished Python 3.0 will have. > > > Right, but the Wiki is a good start while you consider a PEP. > OK. I will start plugging in what I have on my personal collection of what you have said into the wiki after the next summary goes out (400 emails to go). > >>When it is decided where all of this info will end up I will go through >>the python-dev Summaries and see if I can find anything there (tried to >>make it a habit to doc stuff that Guido said would be a Python 3.0 feature. >> >>There is also another wiki page on Python 3.0 at >>http://www.python.org/cgi-bin/moinmoin/PythonThreeDotOh . > > > Hrm. That is mostly Mike McLay's wish list; best to ignore it. > OK. I will just go off what I have mentioned in the Summaries and what I can recollect from my own memory. -Brett From martin at v.loewis.de Wed Aug 18 19:28:04 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 19:28:00 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41231640.2020706@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122E2BE.1070101@v.loewis.de> <41231640.2020706@egenix.com> Message-ID: <412391A4.4080700@v.loewis.de> M.-A. Lemburg wrote: > You need to think about the important use-case of having > to convert Py2 applications to Py3 style. In many cases, > the application can be made to run under Py3 be adding the > small 'b' in front of the used string literals. That is hard to tell, because Py3 is not implemented, yet. It might be that in many cases, no change is necessary at all, because the system default encoding will convert the strings to bytes. > Since the code is already in place and the change is > minimal, I don't see any reason not to use it. I do. It would mean that we commit to the b"" notation, when there is no real need for that. Regards, Martin From martin at v.loewis.de Wed Aug 18 19:31:28 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 19:31:26 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4123177F.7050306@egenix.com> References: <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <200408172133.i7HLXsS16658@guido.python.org> <16674.47319.319502.989563@montanaro.dyndns.org> <4122E47E.1040806@v.loewis.de> <4123177F.7050306@egenix.com> Message-ID: <41239270.5090100@v.loewis.de> M.-A. Lemburg wrote: > If you write your string literal using just ASCII characters and > escapes, I don't see much of a problem with different source code > encodings. That is correct. I personally have no problem if byte fields and unicode strings are connected through some encoding; I personally think making it fixed at Latin-1 might be best. I was merely responding to Skip's question why an encoding comes into play at all, as byte fields inherently have no encoding, and might not even represent character data. I was responding that this is mostly true, except for source code. Regards, Martin From martin at v.loewis.de Wed Aug 18 19:44:18 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 19:44:17 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41231B30.2060307@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net> <41231B30.2060307@egenix.com> Message-ID: <41239572.8090808@v.loewis.de> M.-A. Lemburg wrote: > Wouldn't it be possible to extend the existing buffer type > to meet those standards ? Yes; you then need to change all codecs to return buffers from .encode. Regards, Martin From mal at egenix.com Wed Aug 18 19:55:52 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 19:55:55 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41239572.8090808@v.loewis.de> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net> <41231B30.2060307@egenix.com> <41239572.8090808@v.loewis.de> Message-ID: <41239828.9000301@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> Wouldn't it be possible to extend the existing buffer type >> to meet those standards ? > > Yes; you then need to change all codecs to return buffers > from .encode. I'm still not convinced that we can simply drop the existing immutable 8-bit string type and replace it with a mutable bytes or buffer type, e.g. would buffer.lower() work on the buffer itself or return a lowered copy ? However, if that's where Python will be heading, then you're right (for most of the codecs: some might want to return Unicode objects, e.g. unicode-escape). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Wed Aug 18 20:06:37 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 20:06:33 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41239828.9000301@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net> <41231B30.2060307@egenix.com> <41239572.8090808@v.loewis.de> <41239828.9000301@egenix.com> Message-ID: <41239AAD.2080604@v.loewis.de> M.-A. Lemburg wrote: > I'm still not convinced that we can simply drop the existing > immutable 8-bit string type and replace it with a mutable > bytes or buffer type, e.g. would buffer.lower() work on the > buffer itself or return a lowered copy ? The byte string type would not have a .lower method, as "lowering" is not meaningful for bytes, only for characters. Regards, Martin From fperez528 at yahoo.com Wed Aug 18 20:07:59 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Wed Aug 18 20:08:14 2004 Subject: [Python-Dev] Re: Re: Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> <1092802980.7810.29.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > On Wed, 2004-08-11 at 10:54, Fernando Perez wrote: >> It may be trivial to extend pep-292 for this, but not having to do it in >> every >> small utility you write/ship would be very useful, IMHO. If there are no >> technical reasons to prevent it, is the 'too complex' argument really that >> strong (especially since you state it's a trivial amount of code)? > > Well, it's about 3 lines of Python, but what would you call the class, > AttributeTemplate? I wouldn't be opposed to adding the class, since I > think it would be pretty useful. I'd like to argue that this form may be the most useful for common tasks, so you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar" without having to worry too much about which template class you are using. How about making a BaseTemplate class which does NOT allow $foo.bar, and having the default Template implement attributes? This would give us: - A default class covering what I think is a reasonable, common-case behavior. I'm a big fan of covering most _reasonable_ default behavior out-of-the-box, and I think python's stdlib achieves this very well in most cases. I'd argue this is one of the reasons for its success, and I have the feeling that in this case (PEP-292) the proposed default would be sub-par. - A clean base class for user-defined subclasses which want to be very exacting about what they want to implement or not. This sounds like a reasonable compromise to me, but ymmv. Best, and thanks for keeping an open mind about this issue (string interpolation is one of the very few areas where python's syntax bugs me, even after years of daily use). f From mal at egenix.com Wed Aug 18 20:31:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 20:31:21 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41239AAD.2080604@v.loewis.de> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net> <41231B30.2060307@egenix.com> <41239572.8090808@v.loewis.de> <41239828.9000301@egenix.com> <41239AAD.2080604@v.loewis.de> Message-ID: <4123A075.9010407@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> I'm still not convinced that we can simply drop the existing >> immutable 8-bit string type and replace it with a mutable >> bytes or buffer type, e.g. would buffer.lower() work on the >> buffer itself or return a lowered copy ? > > The byte string type would not have a .lower method, as > "lowering" is not meaningful for bytes, only for characters. Indeed... and the same is true for almost all other methods (except maybe .replace()). Sounds like a lot of code will break. OTOH, it will also enforce the notion of doing encoding and decoding only at IO boundaries and being explicit about character sets which is good in the long run. Auto-conversion using the default encoding will get us all ASCII character (Unicode) strings converted to buffers without problems (and without having the need for an extra b'something' modifier). This leaves the question of how to deal with the byte range 0x80 - 0xFF. The straight forward solution would be to switch to Latin-1 as default encoding and let the same magic take care of that byte range as well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Wed Aug 18 21:06:09 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 21:06:06 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4123A075.9010407@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net> <41231B30.2060307@egenix.com> <41239572.8090808@v.loewis.de> <41239828.9000301@egenix.com> <41239AAD.2080604@v.loewis.de> <4123A075.9010407@egenix.com> Message-ID: <4123A8A1.9060908@v.loewis.de> M.-A. Lemburg wrote: > Indeed... and the same is true for almost all other methods > (except maybe .replace()). > > Sounds like a lot of code will break. We will see. The default string type will be Unicode, so code using .lower will continue to work in many cases. The question is what open(path,"r").read() will return. It seems that Guido wants users to specify "rb" if they want that to be byte strings. Regards, Martin From bac at OCF.Berkeley.EDU Wed Aug 18 21:38:15 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 18 21:38:28 2004 Subject: [Python-Dev] tab in file tkFont.py In-Reply-To: <10AB5C0F-F11A-11D8-873D-0003934AD54A@chello.se> References: <10AB5C0F-F11A-11D8-873D-0003934AD54A@chello.se> Message-ID: <4123B027.7020405@ocf.berkeley.edu> Simon Percivall wrote: > lib-tk/tkFont.py CVS HEAD has a tab character at line 94, breaking > installation. > Looks like it has been fixed now. -Brett From Kevin.Smith at theMorgue.org Wed Aug 18 21:42:55 2004 From: Kevin.Smith at theMorgue.org (Kevin Smith) Date: Wed Aug 18 21:42:27 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? Message-ID: For what it's worth, I wrote the original PEP 318. I probably wasn't qualified, but I just wanted a nice simple way to declare class methods without having to repeat the function name. After submitting it to BDFL for approval, more work was needed and the discussion of PEP 318 on python-dev increased rapidly. It was evident that I was in over my head, so I asked more someone more experienced to take over. I guess others had bigger plans for my proposal that I had planned. It has turned into the "solution" to many problems: type checking (both arguments and returned values), metaclasses, metadata, interfaces, function attributes, etc.). Unfortunately, in the process, this simple request for syntactical sugar has turned into a monstrosity. In my opinion, none of the proposed syntaxes really seem Pythonic. This PEP just seems to be trying to solve too many problems. Bear with me, but I'd like to propose one more syntax that is simple, easy for newbies to understand, and nowhere near as powerful as the current PEP's syntax. However, it doesn't add incoherent, arbitrary syntax either. def classmethod foo(x, y, z): pass That's it. One "decorator" that is a callable object that takes a method as it's only argument. No expressions, lists, tuples, etc. Just one callable object. Ok, if you absolutely must have more than one. def classmethod synchronized foo(x, y, z): pass Once again, no expressions. I know that this isn't going to solve everyone's type-checking, metadata, and function attribute problems, but let's face it, using this PEP for all of those things just creates ugly syntax. There must be more Pythonic ways to do those things in their own PEPs. Kevin Smith Kevin.Smith@themorgue.org From walter at livinglogic.de Wed Aug 18 21:46:10 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Aug 18 21:46:26 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412285D1.7010907@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> Message-ID: <4123B202.5000200@livinglogic.de> M.-A. Lemburg wrote: > Walter D?rwald wrote: > >> M.-A. Lemburg wrote: >> >>> Overall, I don't like the idea of adding extra >>> APIs breaking the existing codec API. >> >> Adding a final argument that defaults to False doesn't >> break the API for the callers, only for the implementors. >> >> And if we drop the final argument, the API is completely >> backwards compatible both for users and implementors. >> The only thing that gets added is the feed() method, >> that implementors don't have to overwrite. >> >>> I believe that we can >>> extend stream codecs to also work in a feed mode without >>> breaking the API. >> >> Abandoning the final argument and adding a feed() method >> would IMHO be the simplest way to do this. >> >> But then there's no way to make sure that every byte from >> the input stream is really consumed. > > I've thought about this some more. Perhaps I'm still missing > something, but wouldn't it be possible to add a feeding > mode to the existing stream codecs by creating a new queue > data type (much like the queue you have in the test cases of > your patch) and using the stream codecs on these ? No, because when the decode method encounters an incomplete chunk (and so return a size that is smaller then size of the input) read() would have to push the remaining bytes back into the queue. This would be code similar in functionality to the feed() method from the patch, with the difference that the buffer lives in the queue not the StreamReader. So we won't gain any code simplification by going this route. > I think such a queue would be generally useful in other > contexts as well, e.g. for implementing fast character based > pipes between threads, non-Unicode feeding parsers, etc. > Using such a type you could potentially add a feeding > mode to stream or file-object API based algorithms very > easily. Yes, so we could put this Queue class into a module with string utilities. Maybe string.py? > We could then have a new class, e.g. FeedReader, which > wraps the above in a nice API, much like StreamReaderWriter > and StreamRecoder. But why should we, when decode() does most of what we need, and the rest has to be implemented in both versions? Bye, Walter D?rwald From bob at redivi.com Wed Aug 18 21:51:14 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Aug 18 21:51:49 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: References: Message-ID: On Aug 18, 2004, at 3:42 PM, Kevin Smith wrote: > For what it's worth, I wrote the original PEP 318. I probably wasn't > qualified, but I just wanted a nice simple way to declare class > methods without having to repeat the function name. After submitting > it to BDFL for approval, more work was needed and the discussion of > PEP 318 on python-dev increased rapidly. It was evident that I was in > over my head, so I asked more someone more experienced to take over. > > I guess others had bigger plans for my proposal that I had planned. > It has turned into the "solution" to many problems: type checking > (both arguments and returned values), metaclasses, metadata, > interfaces, function attributes, etc.). Unfortunately, in the > process, this simple request for syntactical sugar has turned into a > monstrosity. In my opinion, none of the proposed syntaxes really seem > Pythonic. This PEP just seems to be trying to solve too many > problems. > > Bear with me, but I'd like to propose one more syntax that is simple, > easy for newbies to understand, and nowhere near as powerful as the > current PEP's syntax. However, it doesn't add incoherent, arbitrary > syntax either. > > def classmethod foo(x, y, z): > pass > > That's it. One "decorator" that is a callable object that takes a > method as it's only argument. No expressions, lists, tuples, etc. > Just one callable object. Ok, if you absolutely must have more than > one. > > def classmethod synchronized foo(x, y, z): > pass > > Once again, no expressions. I know that this isn't going to solve > everyone's type-checking, metadata, and function attribute problems, > but let's face it, using this PEP for all of those things just creates > ugly syntax. There must be more Pythonic ways to do those things in > their own PEPs. The problem is that there is FAR FAR more use of type-checking, metadata, and function attributes in the wild (from what I've seen and written) than simple stuff like classmethod, staticmethod, and this hypothetical synchronized. I don't think proposals such as this do very much for anyone. It's too simple to be useful, and has too many side-effects on existing tools and people that try and read python source :) Is @syntax really SO horrible that you propose that we make decorators near useless to most but more "aesthetically pleasing" to some? Do you use a significant amount of decorators now? Do you plan to use them much in the future? It seems to me that most people who argue against the sensible and flexible syntaxes are the same people who will rarely if ever use decorators at all... -bob From mal at egenix.com Wed Aug 18 22:07:15 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 22:07:18 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123B202.5000200@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> Message-ID: <4123B6F3.6060708@egenix.com> Walter D?rwald wrote: >> I've thought about this some more. Perhaps I'm still missing >> something, but wouldn't it be possible to add a feeding >> mode to the existing stream codecs by creating a new queue >> data type (much like the queue you have in the test cases of >> your patch) and using the stream codecs on these ? > > No, because when the decode method encounters an incomplete > chunk (and so return a size that is smaller then size of the > input) read() would have to push the remaining bytes back into > the queue. This would be code similar in functionality > to the feed() method from the patch, with the difference that > the buffer lives in the queue not the StreamReader. So > we won't gain any code simplification by going this route. Maybe not code simplification, but the APIs will be well- separated. If we require the queue type for feeding mode operation we are free to define whatever APIs are needed to communicate between the codec and the queue type, e.g. we could define a method that pushes a few bytes back onto the queue end (much like ungetc() in C). >> I think such a queue would be generally useful in other >> contexts as well, e.g. for implementing fast character based >> pipes between threads, non-Unicode feeding parsers, etc. >> Using such a type you could potentially add a feeding >> mode to stream or file-object API based algorithms very >> easily. > > Yes, so we could put this Queue class into a module with > string utilities. Maybe string.py? Hmm, I think a separate module would be better since we could then recode the implementation in C at some point (and after the API has settled). We'd only need a new name for it, e.g. StreamQueue or something. >> We could then have a new class, e.g. FeedReader, which >> wraps the above in a nice API, much like StreamReaderWriter >> and StreamRecoder. > > But why should we, when decode() does most of what we need, > and the rest has to be implemented in both versions? To hide the details from the user. It should be possible to instantiate one of these StreamQueueReaders (named after the queue type) and simply use it in feeding mode without having to bother about the details behind the implementation. StreamReaderWriter and StreamRecoder exist for the same reason. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Wed Aug 18 22:07:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 22:07:42 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: References: Message-ID: <4123B712.6040502@v.loewis.de> Kevin Smith wrote: > def classmethod foo(x, y, z): > pass > > That's it. Indeed, that is form F (5.13) in http://www.python.org/moin/PythonDecorators So this proposal is not new, and has the advantages and disadvantages listed in the Wiki. The main reason to reject it (AFAICT) is the similarity to monstrosities such as "public final synchronized void foo()". Regards, Martin From mal at egenix.com Wed Aug 18 22:10:32 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 18 22:10:43 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <4123A8A1.9060908@v.loewis.de> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net> <41231B30.2060307@egenix.com> <41239572.8090808@v.loewis.de> <41239828.9000301@egenix.com> <41239AAD.2080604@v.loewis.de> <4123A075.9010407@egenix.com> <4123A8A1.9060908@v.loewis.de> Message-ID: <4123B7B8.2090003@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> Indeed... and the same is true for almost all other methods >> (except maybe .replace()). >> >> Sounds like a lot of code will break. > > > We will see. The default string type will be Unicode, so code > using .lower will continue to work in many cases. > > The question is what open(path,"r").read() will return. It seems > that Guido wants users to specify "rb" if they want that to be > byte strings. This would imply that we'd need to add an encoding parameter that becomes a required parameter in case 'r' (without 'b') is specified as mode. Perhaps we should drop 'b' altogether and make encoding a required parameter. We could have a 'binary' codec which then passes through the data as is (as buffer object instead of as Unicode object for most other codecs). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From walter at livinglogic.de Wed Aug 18 22:11:48 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Aug 18 22:12:00 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4122E19E.9050906@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> Message-ID: <4123B804.2000508@livinglogic.de> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> I've thought about this some more. Perhaps I'm still missing >> something, but wouldn't it be possible to add a feeding >> mode to the existing stream codecs by creating a new queue >> data type (much like the queue you have in the test cases of >> your patch) and using the stream codecs on these ? > > Here is the problem. In UTF-8, how does the actual algorithm > tell (the application) that the bytes it got on decoding provide > for three fully decodable characters, and that 2 bytes are left > undecoded, and that those bytes are not inherently ill-formed, > but lack a third byte to complete the multi-byte sequence? > > On top of that, you can implement whatever queuing or streaming > APIs you want, but you *need* an efficient way to communicate > incompleteness. We already have an efficient way to communicate incompleteness: the decode method returns the number of decoded bytes. The questions remaining are 1) communicate to whom? IMHO the info should only be used internally by the StreamReader. 2) When is incompleteness OK? Incompleteness is of course not OK in the stateless API. For the stateful API, incompleteness has to be OK even when the input stream is (temporarily) exhausted, because otherwise a feed mode wouldn't work anyway. But then incompleteness is always OK, because the StreamReader can't distinguish a temporarily exhausted input stream from a permanently exhausted one. The only fix for this I can think of is the final argument. Bye, Walter D?rwald From Kevin.Smith at theMorgue.org Wed Aug 18 22:13:35 2004 From: Kevin.Smith at theMorgue.org (Kevin Smith) Date: Wed Aug 18 22:13:06 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: References: Message-ID: <15E4012F-F153-11D8-BC29-000393829ED2@theMorgue.org> On Aug 18, 2004, at 3:51 PM, Bob Ippolito wrote: > > Is @syntax really SO horrible that you propose that we make decorators > near useless to most but more "aesthetically pleasing" to some? That's kind of a loaded question. I don't think that this simpler syntax is "near useless." It fulfills pretty much all of the problems that the original PEP aimed to fix. And, I don't mean to be rude, but yes the @syntax is that horrible. > Do you use a significant amount of decorators now? I used them enough to write the PEP in the first place, and I would use them even more if there were a better syntax for them. > Do you plan to use them much in the future? It seems to me that most > people who argue against the sensible and flexible syntaxes are the > same people who will rarely if ever use decorators at all... Kevin Smith Kevin.Smith@sas.com From walter at livinglogic.de Wed Aug 18 22:12:56 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Aug 18 22:13:11 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412314F6.5060502@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> Message-ID: <4123B848.2060105@livinglogic.de> M.-A. Lemburg wrote: > Martin v. L?wis wrote: > >> M.-A. Lemburg wrote: >> >>> I've thought about this some more. Perhaps I'm still missing >>> something, but wouldn't it be possible to add a feeding >>> mode to the existing stream codecs by creating a new queue >>> data type (much like the queue you have in the test cases of >>> your patch) and using the stream codecs on these ? >> >> Here is the problem. In UTF-8, how does the actual algorithm >> tell (the application) that the bytes it got on decoding provide >> for three fully decodable characters, and that 2 bytes are left >> undecoded, and that those bytes are not inherently ill-formed, >> but lack a third byte to complete the multi-byte sequence? > > This state can be stored in the stream codec instance, > e.g. by using a special state object that is stored in > the instance and passed to the encode/decode APIs of the > codec or by implementing the stream codec itself in C. That's exactly what my patch does. The state (the bytes that have already been read from the input stream, but couldn't be decoded and have to be used on the next call to read()) are stored in the bytebuffer attribute of the StreamReader. Most stateful decoder use this type of state, the only one I can think of that uses more than this is the UTF-7 decoder, where the decoder decodes partial +xxxx- sequences, but then has to keep the current shift state and the partially consumed bits and bytes. This decoder could be changed, so that it works with only a byte buffer too, but that would mean that the decoder doesn't enter incomplete +xxxx- sequences, but retains them in the byte buffer and only decodes them once the "-" is encountered. In fact a trivial implementation of any stateful decoder could put *everything* it reads into the bytebuffer when final==False and decode itin one go once read() is called with final==True. But IMHO each decoder should decode as much as possible. > We do need to extend the API between the stream codec > and the encode/decode functions, no doubt about that. > However, this is an extension that is well hidden from > the user of the codec and won't break code. Exactly: this shouldn't be anything officially documented, because what kind of data is passed around depends on the codec. And when the stream reader is implemented in C there isn't any API anyway. >> On top of that, you can implement whatever queuing or streaming >> APIs you want, but you *need* an efficient way to communicate >> incompleteness. > > Agreed. Bye, Walter D?rwald From Kevin.Smith at theMorgue.org Wed Aug 18 22:24:13 2004 From: Kevin.Smith at theMorgue.org (Kevin Smith) Date: Wed Aug 18 22:23:45 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <4123B712.6040502@v.loewis.de> References: <4123B712.6040502@v.loewis.de> Message-ID: <91FA4761-F154-11D8-BC29-000393829ED2@theMorgue.org> On Aug 18, 2004, at 4:07 PM, Martin v. L?wis wrote: > Kevin Smith wrote: >> def classmethod foo(x, y, z): >> pass >> That's it. > > Indeed, that is form F (5.13) in > > http://www.python.org/moin/PythonDecorators > > So this proposal is not new, and has the advantages > and disadvantages listed in the Wiki. The main reason > to reject it (AFAICT) is the similarity to monstrosities > such as "public final synchronized void foo()". Yes, I realize that this syntax isn't new (I believe that it was even used in an example in my original PEP). I hadn't actually read Wiki before, but the only minus that I really see for this syntax is that you can't have arguments to the decorators. I guess I just find the "public final synchronized void foo()" less monstrous than @public @final @synchronized @void def foo() Kevin Smith Kevin.Smith@sas.com From martin at v.loewis.de Wed Aug 18 22:30:14 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 22:30:33 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412314F6.5060502@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> Message-ID: <4123BC56.7030607@v.loewis.de> > We do need to extend the API between the stream codec > and the encode/decode functions, no doubt about that. > However, this is an extension that is well hidden from > the user of the codec and won't break code. So you agree to the part of Walter's change that introduces new C functions (PyUnicode_DecodeUTF7Stateful etc)? I think most of the patch can be discarded: there is no need for .encode and .decode to take an additional argument. It is only necessary that the StreamReader and StreamWriter are stateful, and that only for a selected subset of codecs. Marc-Andre, if the original patch (diff.txt) was applied: What *specific* change in that patch would break code? What *specific* code (C or Python) would break under that change? I believe the original patch can be applied as-is, and does not cause any breakage. It also introduces a change between the codec and the encode/decode functions that is well hidden from the user of the codec. Regards, Martin From Kevin.Smith at sas.com Wed Aug 18 21:38:24 2004 From: Kevin.Smith at sas.com (Kevin D.Smith) Date: Wed Aug 18 22:37:51 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? Message-ID: <2B21BFF4-F14E-11D8-BC29-000393829ED2@sas.com> For what it's worth, I wrote the original PEP 318. I probably wasn't qualified, but I just wanted a nice simple way to declare class methods without having to repeat the function name. After submitting it to BDFL for approval, more work was needed and the discussion of PEP 318 on python-dev increased rapidly. It was evident that I was in over my head, so I asked more someone more experienced to take over. I guess others had bigger plans for my proposal that I had planned. It has turned into the "solution" to many problems: type checking (both arguments and returned values), metaclasses, metadata, interfaces, function attributes, etc.). Unfortunately, in the process, this simple request for syntactical sugar has turned into a monstrosity. In my opinion, none of the proposed syntaxes really seem Pythonic. This PEP just seems to be trying to solve too many problems. Bear with me, but I'd like to propose one more syntax that is simple, easy for newbies to understand, and nowhere near as powerful as the current PEP's syntax. However, it doesn't add incoherent, arbitrary syntax either. def classmethod foo(x, y, z): pass That's it. One "decorator" that is a callable object that takes a method as it's only argument. No expressions, lists, tuples, etc. Just one callable object. Ok, if you absolutely must have more than one. def classmethod synchronized foo(x, y, z): pass Once again, no expressions. I know that this isn't going to solve everyone's type-checking, metadata, and function attribute problems, but let's face it, using this PEP for all of those things just creates ugly syntax. There must be more Pythonic ways to do those things in their own PEPs. Kevin Smith Kevin.Smith@sas.com From walter at livinglogic.de Wed Aug 18 22:35:31 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Aug 18 22:37:55 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123B6F3.6060708@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> Message-ID: <4123BD93.2040107@livinglogic.de> M.-A. Lemburg wrote: > Walter D?rwald wrote: > >>> I've thought about this some more. Perhaps I'm still missing >>> something, but wouldn't it be possible to add a feeding >>> mode to the existing stream codecs by creating a new queue >>> data type (much like the queue you have in the test cases of >>> your patch) and using the stream codecs on these ? >> >> >> No, because when the decode method encounters an incomplete >> chunk (and so return a size that is smaller then size of the >> input) read() would have to push the remaining bytes back into >> the queue. This would be code similar in functionality >> to the feed() method from the patch, with the difference that >> the buffer lives in the queue not the StreamReader. So >> we won't gain any code simplification by going this route. > > Maybe not code simplification, but the APIs will be well- > separated. They will not, because StreamReader.decode() already is a feed style API (but with state amnesia). Any stream decoder that I can think of can be (and most are) implemented by overwriting decode(). > If we require the queue type for feeding mode operation > we are free to define whatever APIs are needed to communicate > between the codec and the queue type, e.g. we could define > a method that pushes a few bytes back onto the queue end > (much like ungetc() in C). That would of course be a possibility. >>> I think such a queue would be generally useful in other >>> contexts as well, e.g. for implementing fast character based >>> pipes between threads, non-Unicode feeding parsers, etc. >>> Using such a type you could potentially add a feeding >>> mode to stream or file-object API based algorithms very >>> easily. >> >> Yes, so we could put this Queue class into a module with >> string utilities. Maybe string.py? > > Hmm, I think a separate module would be better since we > could then recode the implementation in C at some point > (and after the API has settled). > We'd only need a new name for it, e.g. StreamQueue or > something. Sounds reasonable. >>> We could then have a new class, e.g. FeedReader, which >>> wraps the above in a nice API, much like StreamReaderWriter >>> and StreamRecoder. >> >> But why should we, when decode() does most of what we need, >> and the rest has to be implemented in both versions? > > To hide the details from the user. It should be possible > to instantiate one of these StreamQueueReaders (named > after the queue type) and simply use it in feeding > mode without having to bother about the details behind > the implementation. > > StreamReaderWriter and StreamRecoder exist for the same > reason. Let's compare example uses: 1) Having feed() as part of the StreamReader API: --- s = u"???".encode("utf-8") r = codecs.getreader("utf-8")() for c in s: print r.feed(c) --- 2) Explicitely using a queue object: --- from whatever import StreamQueue s = u"???".encode("utf-8") q = StreamQueue() r = codecs.getreader("utf-8")(q) for c in s: q.write(c) print r.read() --- 3) Using a special wrapper that implicitely creates a queue: ---- from whatever import StreamQueueWrapper s = u"???".encode("utf-8") r = StreamQueueWrapper(codecs.getreader("utf-8")) for c in s: print r.feed(c) ---- I very much prefer option 1). "If the implementation is hard to explain, it's a bad idea." Bye, Walter D?rwald From martin at v.loewis.de Wed Aug 18 22:39:22 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 22:39:24 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123B804.2000508@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <4123B804.2000508@livinglogic.de> Message-ID: <4123BE7A.60305@v.loewis.de> Walter D?rwald wrote: >> But then incompleteness is always OK, > because the StreamReader can't distinguish a temporarily > exhausted input stream from a permanently exhausted one. > The only fix for this I can think of is the final argument. I don't think the final argument is needed. Methinks that the .encode/.decode should not be used by an application. Instead, applications should only use the file API on a reader/writer. If so, stateful readers/writers can safely implement encode/decode to take whatever state they have into account, creating new state as they see fit. Of course, stateful writers need to implement their own .close function, which flushes the remaining bytes, and need to make sure that .close is automatically invoked if the object goes away. Regards, Martin From martin at v.loewis.de Wed Aug 18 22:51:57 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 22:51:58 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123BD93.2040107@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de> Message-ID: <4123C16D.4060505@v.loewis.de> Walter D?rwald wrote: > They will not, because StreamReader.decode() already is a feed > style API (but with state amnesia). > > Any stream decoder that I can think of can be (and most are) > implemented by overwriting decode(). I consider that an unfortunate implementation artefact. You either use the stateless encode/decode that you get from codecs.get(encoder/decoder) or you use the file API on the streams. You never ever use encode/decode on streams. I would have preferred if the default .write implementation would have called self._internal_encode, and the Writer would *contain* a Codec, rather than inheriting from Codec. Alas, for (I guess) simplicity, a more direct (and more confusing) approach was taken. > 1) Having feed() as part of the StreamReader API: > --- > s = u"???".encode("utf-8") > r = codecs.getreader("utf-8")() > for c in s: > print r.feed(c) Isn't that a totally unrelated issue? Aren't we talking about short reads on sockets etc? I would very much prefer to solve one problem at a time. Regards, Martin From walter at livinglogic.de Wed Aug 18 23:17:31 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Aug 18 23:17:39 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123BC56.7030607@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> Message-ID: <4123C76B.3030100@livinglogic.de> Martin v. L?wis wrote: >> We do need to extend the API between the stream codec >> and the encode/decode functions, no doubt about that. >> However, this is an extension that is well hidden from >> the user of the codec and won't break code. > > So you agree to the part of Walter's change that introduces > new C functions (PyUnicode_DecodeUTF7Stateful etc)? > > I think most of the patch can be discarded: there is no > need for .encode and .decode to take an additional argument. But then a file that contains the two bytes 0x61, 0xc3 will never generate an error when read via an UTF-8 reader. The trailing 0xc3 will just be ignored. Another option we have would be to add a final() method to the StreamReader, that checks if all bytes have been consumed. Maybe this should be done by StreamReader.close()? > It is only necessary that the StreamReader and StreamWriter > are stateful, and that only for a selected subset of codecs. > > Marc-Andre, if the original patch (diff.txt) was applied: > What *specific* change in that patch would break code? > What *specific* code (C or Python) would break under that > change? > > I believe the original patch can be applied as-is, and > does not cause any breakage. The first version has a broken implementation of the UTF-7 decoder. When decoding the byte sequence "+-" in two calls to decode() (i.e. pass "+" in one call and "-" in the next), no character got generated, because inShift (as a flag) couldn't remember whether characters where encountered between the "+" and the "-". Now inShift counts the number of characters (and the shortcut for a "+-" sequence appearing together has been removed. > It also introduces a change > between the codec and the encode/decode functions that is > well hidden from the user of the codec. Would a version of the patch without a final argument but with a feed() method be accepted? I'm imagining implementing an XML parser that uses Python's unicode machinery and supports the xml.sax.xmlreader.IncrementalParser interface. With a feed() method in the stream reader this is rather simple: init() { PyObject *reader = PyCodec_StreamReader(encoding, Py_None, NULL); self.reader = PyObject_CallObject(reader, NULL); } int feed(char *bytes) { parse(PyObject_CallMethod(self.reader, "feed", "s", bytes); } The feed method itself is rather simple (see the second version of the patch). Without the feed method(), we need the following: 1) A StreamQueue class that a) supports writing at one end and reading at the other end b) has a method for pushing back unused bytes to be returned in the next call to read() 2) A StreamQueueWrapper class that a) gets passed the StreamReader factory in the constructor, creates a StreamQueue instance, puts it into an attribute and passes it to the StreamReader factory (which must also be put into an attribute). b) has a feed() method that calls write() on the stream queue and read() on the stream reader and returns the result Then the C implementation of the parser looks something like this: init() { PyObject *module = PyImport_ImportModule("whatever"); PyObject *wclass = PyObject_GetAttr(module, "StreamQueueWrapper"); PyObject *reader = PyCodec_StreamReader(encoding, Py_None, NULL); self.wrapper = PyObject_CallObject(wclass, reader); } int feed(char *bytes) { parse(PyObject_CallMethod(self.wrapper, "feed", "s", bytes); } I find this neither easier to implement nor easier to explain. Bye, Walter D?rwald From python-kbutler at sabaydi.com Wed Aug 18 23:33:14 2004 From: python-kbutler at sabaydi.com (Kevin J. Butler) Date: Wed Aug 18 23:35:13 2004 Subject: [Python-Dev] PEP 318: Suggest we drop it Message-ID: <4123CB1A.2060101@sabaydi.com> - There is no universally liked syntax (or even a universally _tolerable_ syntax) - decorators will incosistently retain or drop docstrings (this may even be the most appropriate thing to do) - a "decorated" function may have an arbitrarily distant relationship with the function as implemented in the def statement (this is a killer) - if you want to decorate a single function two separate ways, you're back to 2.3 syntax - say, one that logs, checks parameters, etc., and the "internal" optimal version. None of the decorator syntaxes facilitate this. Because of all these issues, I think we should drop PEP 318. The 2.3 form seems more reasonable all the time: def __func( self ): pass spanishInquisition = mydecorator( __func ) spanishInquisition.__doc__= """Something unexpected""" parrot = otherdecorator( __func ) parrot.__doc__ = """And now for something completely different""" Are any of the syntaxes enough better to justify the irritation or divisiveness of including it? Syntactic sugar causes cancer of the semicolon, indeed... kb From martin at v.loewis.de Wed Aug 18 23:57:22 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 18 23:57:28 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123C76B.3030100@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> Message-ID: <4123D0C2.4080505@v.loewis.de> Walter D?rwald wrote: > But then a file that contains the two bytes 0x61, 0xc3 > will never generate an error when read via an UTF-8 reader. > The trailing 0xc3 will just be ignored. > > Another option we have would be to add a final() method > to the StreamReader, that checks if all bytes have been > consumed. Alternatively, we could add a .buffer() method that returns any data that are still pending (either a Unicode string or a byte string). > Maybe this should be done by StreamReader.close()? No. There is nothing wrong with only reading a part of a file. > Now > inShift counts the number of characters (and the shortcut > for a "+-" sequence appearing together has been removed. Ok. I didn't actually check the correctness of the individual methods. OTOH, I think time spent on UTF-7 is wasted, anyway. > Would a version of the patch without a final argument but > with a feed() method be accepted? I don't see the need for a feed method. .read() should just block until data are available, and that's it. > I'm imagining implementing an XML parser that uses Python's > unicode machinery and supports the > xml.sax.xmlreader.IncrementalParser interface. I think this is out of scope of this patch. The incremental parser could implement a regular .read on a StringIO file that also supports .feed. > Without the feed method(), we need the following: > > 1) A StreamQueue class that Why is that? I thought we are talking about "Decoding incomplete unicode"? Regards, Martin From Jack.Jansen at cwi.nl Thu Aug 19 00:16:33 2004 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Thu Aug 19 00:16:36 2004 Subject: [Python-Dev] Byte string class hierarchy Message-ID: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> I may have missed a crucial bit of the discussion, having been away, so if this is completely besides the point let me know. But my feeling is that the crucial bit is the type inheritance graph of all the byte and string types. And I wonder whether the following graph would help us solve most problems (aside from introducing one new one, that may be a showstopper): genericbytes mutablebytes bytes genericstring string unicode The basic type for all bytes, buffers and strings is genericbytes. This abstract base type is neither mutable nor immutable, and has the interface that all of the types would share. Mutablebytes adds slice assignment and such. Bytes, on the other hand, adds hashing. genericstring is the magic stuff that's there already that makes unicode and string interoperable for hashing and dict keys and such. Casting to a basetype is always free and doesn't copy anything, i.e. the bits stay the same. 'foo' in sourcecode is a string, and if you cast it to bytes you'll just get the bits, which is pretty much the same as what you get now. If you really want to make sure you get an 8-bit ascii representation even if you run in an interpreter built with UCS4 as the default character set you must use bytes('foo'.encode('ascii')). Casting to a subtype may cause a copy, but does not modify the bits. Casting sideways copies, and may modify the bits too, the current unicode encode/decode stuff. These 2 rules mean that unicode('foo') is something different from unicode(bytes('foo')), and probably illegal to boot, but I don't think that's too much of a problem: you shouldn't explicitly cast to bytes() unless you really want uninterpreted bits. Operations like concatenation return the most specialised class. Mutablebytes is the only problem case here, we should probably forbid concatenating these with the others. The alternatives (return mutablebytes, return the other one, return the type of the first operand) all seem somewhat random. Read() is guaranteed only to return genericbytes, but if you open a file in textmode they'll returns strings, and we should add the ability to open files for unicode and probably mutablebytes too. I'm not sure about socket.recv() and such, but something similar probably holds. Readline() really shouldn't be allowed on files open in binary mode, but that may be a bit too much. Write and friends accept genericbytes, and binary files will just dump the bits. Files open in text mode may convert unicode and string objects between representations. The bad news (aside from any glaring holes I may have overseen in the above: shoot away!) is that I don't know what to do for hash on bytes objects. On the one hand I would like hash('foo') == hash(bytes('foo')). But that leads to also wanting hash(u'foo') == hash(bytes(u'foo')), and we can't really have that because hash('foo') == hash(u'foo') is needed to make string/unicode interoperability for dictionaries work. Note that for the value 'foo' this isn't a problem, but for 'f??' (thats F O-UMLAUT O-UMLAUT) it is. So it seems that making hash('foo') != hash(bytes('foo')) is the only reasonable solution (and probably also a good idea with the future in mind: explicit is better than implicit so just put a cast there if you want the binary bits to be interpreted as an ASCII or Unicode string!) it will probably break existing code. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From bac at OCF.Berkeley.EDU Thu Aug 19 00:26:12 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 19 00:26:17 2004 Subject: [Python-Dev] PEP 318: Suggest we drop it In-Reply-To: <4123CB1A.2060101@sabaydi.com> References: <4123CB1A.2060101@sabaydi.com> Message-ID: <4123D784.6060206@ocf.berkeley.edu> Kevin J. Butler wrote: > - There is no universally liked syntax (or even a universally > _tolerable_ syntax) Yeah, but this happens every time a new syntax is proposed. > - decorators will incosistently retain or drop docstrings (this may even > be the most appropriate thing to do) No one ever said decorators were for newbies. There are multiple things to take into consideration when designing them. > - a "decorated" function may have an arbitrarily distant relationship > with the function as implemented in the def statement (this is a killer) Why? The decorator is listed before you even reach the def statement so you know how its semantics will change. > - if you want to decorate a single function two separate ways, you're > back to 2.3 syntax - say, one that logs, checks parameters, etc., and > the "internal" optimal version. None of the decorator syntaxes > facilitate this. > Yeah, but hardly any of the other suggestions do either. And I don't think this 'one function, two separate decorators' will be common enough to warrant requiring it for the syntax. > Because of all these issues, I think we should drop PEP 318. > I disagree. I honestly think a bunch of people jumped the gun on this. Initially I wasn't crazy about the syntax either, but then I rewrote one of the test cases for test_decorators and I completely changed my mind. While I am not saying even the majority of people would change their mind if they used the syntax I do think a decent chunk of people would. And I think people need to realize that there is precedent for having this syntactically supported. Both Java and C#, while not my favorite languages, both have something like this. It would suggest that there is *something* to this if two separate language teams came up with supporting this all while we had our own non-syntactic support. > The 2.3 form seems more reasonable all the time: > Then use it and not the new syntax. This is not meant to come off as snippy, but part of the cool thing about all of this is you can *still* use the 2.2 style (I am going all the way back to the first version that supported descriptors and thus introduced this whole idea of decorators). After all of this, I think it is time to stop arguing over new syntax here or suggesting we drop it. Not enough people don't want the feature to convince Guido to drop it. Right now people should be hashing things out on c.l.py to get down to one alternative to propose to Guido (which I think is rather nice of him since he doesn't have to listen to anybody on any of this). Everything else has just turned into white noise at this point. And this is my final email on alternative decorators until that single alternative for Guido comes forth since that is the only acceptable next step. -Brett From nas at arctrix.com Thu Aug 19 00:36:29 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Thu Aug 19 00:36:32 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> Message-ID: <20040818223629.GA17550@mems-exchange.org> On Thu, Aug 19, 2004 at 12:16:33AM +0200, Jack Jansen wrote: > genericbytes > mutablebytes > bytes > genericstring > string > unicode I think characters (unicode or otherwise) should not be confused with bytes. Having 'unicode' as a subclass of 'bytes' is very confusing to me. Neil From martin at v.loewis.de Thu Aug 19 00:38:31 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 19 00:38:28 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> Message-ID: <4123DA67.7050902@v.loewis.de> Jack Jansen wrote: > genericbytes > mutablebytes > bytes > genericstring > string > unicode I think this hiearchy is wrong. unicode is not a specialization of genericybytes: a unicode strings is made out of characters, not out of bytes. > The basic type for all bytes, buffers and strings is genericbytes. This > abstract base type is neither mutable nor immutable, and has the > interface that all of the types would share. Mutablebytes adds slice > assignment and such. Bytes, on the other hand, adds hashing. There is a debate on whether such a type is really useful. Why do you need hashing on bytes? > genericstring is the magic stuff that's there already that makes unicode > and string interoperable for hashing and dict keys and such. Interoperability, in Python, does not necessarily involve a common base type. > Casting to a basetype is always free and doesn't copy anything And, of course, there is no casting at all in Python. > Operations like concatenation return the most specialised class. Assuming the hieararchy on the top of your message, what does that mean? Suppose I want to concatenate unicode and string: which of them is more specialized? > Read() is guaranteed only to return genericbytes, but if you open a file > in textmode they'll returns strings, and we should add the ability to > open files for unicode and probably mutablebytes too. I think Guido's proposal is that read(), in text mode, returns Unicode strings, and (probably) that there is no string type in Python anymore. read() on binary files would return a mutable byte array. Regards, Martin From barry at python.org Thu Aug 19 00:39:22 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 19 00:39:00 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <20040818223629.GA17550@mems-exchange.org> References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> <20040818223629.GA17550@mems-exchange.org> Message-ID: <1092868762.22400.97.camel@geddy.wooz.org> On Wed, 2004-08-18 at 18:36, Neil Schemenauer wrote: > On Thu, Aug 19, 2004 at 12:16:33AM +0200, Jack Jansen wrote: > > genericbytes > > mutablebytes > > bytes > > genericstring > > string > > unicode > > I think characters (unicode or otherwise) should not be confused > with bytes. Having 'unicode' as a subclass of 'bytes' is very > confusing to me. Agreed! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040818/cfad824b/attachment.pgp From guido at python.org Thu Aug 19 03:32:27 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 03:32:36 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 In-Reply-To: Your message of "Tue, 17 Aug 2004 22:22:09 PDT." References: Message-ID: <200408190132.i7J1WRO19056@guido.python.org> > Move the bytecode optimizer upstream so that its results are saved > in pyc files and not re-optimized upon import. Saves a bit of > startup time while still remaining decoupled from the rest of the > compiler. Hm, shouldn't the bytecode optimizer only be used when -O is used, and hence pyo files are being written? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 19 03:39:33 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 03:39:42 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Wed, 18 Aug 2004 10:49:52 +0200." <41231830.2090903@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <41231830.2090903@egenix.com> Message-ID: <200408190139.i7J1dX819100@guido.python.org> > Hmm, who ever said that we are going to drop the current 8-bit > string implementation ? I expect that in Python 3.0 aka Python 3000 we'll have only Unicode strings and byte arrays, so yes, the current 8-bit string implementation will eventually die. Jython and IronPython are ahead of us in this respect... --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at cosc.canterbury.ac.nz Thu Aug 19 03:42:36 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 19 03:43:34 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <41239828.9000301@egenix.com> Message-ID: <200408190142.i7J1gaZl025182@cosc353.cosc.canterbury.ac.nz> > I'm still not convinced that we can simply drop the existing > immutable 8-bit string type and replace it with a mutable bytes or > buffer type, e.g. would buffer.lower() work on the buffer itself or > return a lowered copy ? Byte strings probably shouldn't have a lower() method at all, or any other method that assumes the contents represent characters. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From python at rcn.com Thu Aug 19 04:02:04 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 19 04:02:26 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 In-Reply-To: <200408190132.i7J1WRO19056@guido.python.org> Message-ID: <000f01c48590$866705e0$e841fea9@oemcomputer> > > Move the bytecode optimizer upstream so that its results are saved > > in pyc files and not re-optimized upon import. Saves a bit of > > startup time while still remaining decoupled from the rest of the > > compiler. > > Hm, shouldn't the bytecode optimizer only be used when -O is used, and > hence pyo files are being written? Why? That would throw away most of the benefits to most of the users and gain nothing in return. The peepholer was in place in for Py2.3 and only benefits were seen. I would save the -O option for something where there is a trade-off (loss of docstrings, excessive compilation time, possibly risky optimizations, or somesuch). Here, the peepholer is superfast and costs almost nothing. Raymond From bac at OCF.Berkeley.EDU Thu Aug 19 05:40:09 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 19 05:40:10 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 In-Reply-To: <000f01c48590$866705e0$e841fea9@oemcomputer> References: <000f01c48590$866705e0$e841fea9@oemcomputer> Message-ID: <41242119.6060300@ocf.berkeley.edu> Raymond Hettinger wrote: >>>Move the bytecode optimizer upstream so that its results are saved >>>in pyc files and not re-optimized upon import. Saves a bit of >>>startup time while still remaining decoupled from the rest of the >>>compiler. >> >>Hm, shouldn't the bytecode optimizer only be used when -O is used, and >>hence pyo files are being written? > > > Why? That would throw away most of the benefits to most of the users > and gain nothing in return. The peepholer was in place in for Py2.3 and > only benefits were seen. I would save the -O option for something where > there is a trade-off (loss of docstrings, excessive compilation time, > possibly risky optimizations, or somesuch). Here, the peepholer is > superfast and costs almost nothing. > Seems we need a definition of philosophy for Python. Is a compiler optimization *anything* that changes the opcode initially emitted by the compiler, or only opcodes that can have some adverse affect, such as larger files or longer compile times. If you take a strict reading of the first philosophy the optimization of changing a constant having the unary negative op applied to it changed into a negative constant (that bit me in the ass hard for my thesis since it changes the CST directly; bet my next bug is something similar). That does change what is going to be emitted very far upstream. And if I remember from the last time I looked at the peephole optimizer it just changes some jump values to absolute and a few other minor things. Personally, I go with the latter philosophy. -O still has its place since it removes asserts. But if an optimization doesn't change the semantics (such as removing asserts), doesn't take a lot of time (such as a bunch of walks along the opcodes), or increase space (unrolling loops), then I don't see any harm. Just think of it as what the compiler would have emitted could it be inlined within its innards easily. -Brett From guido at python.org Thu Aug 19 06:01:47 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 06:01:54 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: Your message of "Wed, 18 Aug 2004 22:07:46 +0200." <4123B712.6040502@v.loewis.de> References: <4123B712.6040502@v.loewis.de> Message-ID: <200408190401.i7J41lt19485@guido.python.org> Is anybody seriously trying to come up with a single alternative decorator proposal that most folks "out there" can support, to be presented to me (with implementation, please!) in time for 2.4b1? The endless repetition of old proposals on the one hand, and the outrageous overgeneralizations being proposed on the other hand, make me wonder if @decorator is going to win by default -- because nobody can agree on a better proposal. Which is fine with me, but let it be clear that if that happens, the community has itself to blame (or to thank, depending on which syntax you favor :-). The ipython and Leo authors have already said that their tools will survive @decorators. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 19 06:07:10 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 06:07:17 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 In-Reply-To: Your message of "Wed, 18 Aug 2004 22:02:04 EDT." <000f01c48590$866705e0$e841fea9@oemcomputer> References: <000f01c48590$866705e0$e841fea9@oemcomputer> Message-ID: <200408190407.i7J47Ar19512@guido.python.org> > > Hm, shouldn't the bytecode optimizer only be used when -O is used, and > > hence pyo files are being written? > > Why? That would throw away most of the benefits to most of the > users and gain nothing in return. The peepholer was in place in for > Py2.3 and only benefits were seen. I would save the -O option for > something where there is a trade-off (loss of docstrings, excessive > compilation time, possibly risky optimizations, or somesuch). Here, > the peepholer is superfast and costs almost nothing. Maybe I'm out of tune, but I thought that optimizations should be turned off by default because most people don't need them and because of the risk that the optimizer might break something. Haven't there been situations in Python where one optimization or another was found to be unsafe after having been in use (in a release!) for a long time? I'd rather be safe than sorry. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Thu Aug 19 06:26:50 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 19 06:27:00 2004 Subject: [Python-Dev] Threading in the Standard Library Tour Part II In-Reply-To: <200408180458.i7I4wS2a023568@cosc353.cosc.canterbury.ac.nz> References: <200408180458.i7I4wS2a023568@cosc353.cosc.canterbury.ac.nz> Message-ID: <1f7befae04081821265795dd24@mail.gmail.com> >> done-right-it-only-takes-0.97c-to-give-the-appearance-of-6-inches-ly >> y'rs - tim [Greg Ewing] > Hmmm, so Tim is claiming to have an actual length of > > >>> 6 / sqrt(1 - (0.97 * 0.97)) > 24.680702093691806 > > inches... I *am* modest. Expandable flaccidware for bots is just a matter of purchasing and installing appropriate materials. But I've often noticed that insecure humans feel compelled to carry out the calculation, as if pitiable Nature could compete with modern hydraulic engineering. From rnd at onego.ru Thu Aug 19 06:27:16 2004 From: rnd at onego.ru (Roman Suzi) Date: Thu Aug 19 06:27:16 2004 Subject: [Python-Dev] PEP 318: Suggest we drop it In-Reply-To: <4123CB1A.2060101@sabaydi.com> References: <4123CB1A.2060101@sabaydi.com> Message-ID: On Wed, 18 Aug 2004, Kevin J. Butler wrote: >- decorators will incosistently retain or drop docstrings (this may even >be the most appropriate thing to do) Well, so does any function working on functions. Probably special Decorator class could help make decorators friendly when needed. >- a "decorated" function may have an arbitrarily distant relationship >with the function as implemented in the def statement (this is a killer) Is is the case in the DecoratorLibrary on the wiki? No. >- if you want to decorate a single function two separate ways, you're - any use cases for this? >back to 2.3 syntax - say, one that logs, checks parameters, etc., and >the "internal" optimal version. None of the decorator syntaxes >facilitate this. > >Because of all these issues, I think we should drop PEP 318. Still these issues do not outweight the usefulness of decorators and their support for "write once" concept, while in old syntax one need to write a function name 4 times to merely get it decorated and with __doc__ string. >The 2.3 form seems more reasonable all the time: > >def __func( self ): > pass >spanishInquisition = mydecorator( __func ) >spanishInquisition.__doc__= """Something unexpected""" > >parrot = otherdecorator( __func ) >parrot.__doc__ = """And now for something completely different""" > >Are any of the syntaxes enough better to justify the irritation or >divisiveness of including it? Theoretically it is an argument, but in practice I do not think the above construct will appear frequently enough. If you have the case, try class instead of function+decorator. Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3 From tdelaney at avaya.com Thu Aug 19 06:37:45 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Aug 19 06:37:54 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com> Guido van Rossum wrote: > Is anybody seriously trying to come up with a single alternative > decorator proposal that most folks "out there" can support, to be > presented to me (with implementation, please!) in time for 2.4b1? I'm afraid that absolutely zero consensus is emerging :( I've withdrawn myself entirely from the discussions as they've proven completely useless. Last time I saw behaviour like this was PEP 308. Hmm - PEP 308 - controversial syntax. PEP 318 - controversial syntax. PEP 328 - somewhat controversial syntax (I still don't overly like it). Anyone want to guess what PEP 338 will be like? ;) Tim Delaney From pje at telecommunity.com Thu Aug 19 06:43:40 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 19 06:43:31 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408190401.i7J41lt19485@guido.python.org> References: <4123B712.6040502@v.loewis.de> Message-ID: <5.1.1.6.0.20040819003254.02db88d0@mail.telecommunity.com> At 09:01 PM 8/18/04 -0700, Guido van Rossum wrote: >Is anybody seriously trying to come up with a single alternative >decorator proposal that most folks "out there" can support, to be >presented to me (with implementation, please!) in time for 2.4b1? I've been trying to drum up support here for option J4, a compromise between list-after-def and @decorators: def p_statement_expr(p) @staticmethod @grammarrule('statement : expression') @version("Added in 2.4") @returns(None) as: """Docstring could be here, or in the decorator part above""" # body goes here I think it combines the best of several proposals; @definitions are more visible than list-after-def, but are not part of the function body, and you get to see the signature first. It doesn't define a new keyword, "as" is an unambiguous pseudo-keyword here, because decorators start with "@". The principal downside is the presence of indentation without either a : or enclosing brackets. The syntax I proposed was something like: funcdef: "def" NAME arglist [decorators] ":" suite decorators: INDENT ("@" decorator_spec NEWLINE)* [docstring NEWLINE] DEDENT "as" There's been no response that I've noticed, perhaps because by the time I proposed it people were already sick of looking at options A through J3. :) Or maybe it just sucks for some reason invisible to me. From greg at cosc.canterbury.ac.nz Thu Aug 19 06:55:06 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Thu Aug 19 06:55:14 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408190401.i7J41lt19485@guido.python.org> Message-ID: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> Guido: > Is anybody seriously trying to come up with a single alternative > decorator proposal that most folks "out there" can support, to be > presented to me (with implementation, please!) in time for 2.4b1? >From the posts I've seen here, the only alternatives that have considerable popular support are ones that you've already rejected. So I suspect nobody really feels it's worth trying. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Thu Aug 19 07:05:50 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 19 07:05:55 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 In-Reply-To: <200408190407.i7J47Ar19512@guido.python.org> References: <000f01c48590$866705e0$e841fea9@oemcomputer> <200408190407.i7J47Ar19512@guido.python.org> Message-ID: <1f7befae0408182205231a0fe4@mail.gmail.com> [Guido] > Maybe I'm out of tune, but I thought that optimizations should be > turned off by default because most people don't need them and because > of the risk that the optimizer might break something. Haven't there > been situations in Python where one optimization or another was found > to be unsafe after having been in use (in a release!) for a long time? > I'd rather be safe than sorry. The "optimize unary minus" optimization Brett mentioned was the cause of at least 3 distinct bugs, from gross (threw away entire parse subtrees in some cases) to achingly subtle (excruciating consequences from losing the distinction between plus and minus *floating* zeroes after going thru marshal after the optimization), and which took more than a year to discover from first to last. Here's a bug opened this month against the peephole optimizer in 2.3 (and still in 2.4a2): http://www.python.org/sf/1004088 IME, optimizing has a much higher subtle-bug-per-line-of-code ratio,than any other kind of programming work excepting cowboy undisciplined approaches to threading. But since I made my living writing optimizing compilers (for threaded programs -- heh) at the time we first corresponded about Python, I may have influenced you toward being too jittery. Kinda like I've scared everyone to death about ZODB's BTrees in recent times, despite that they've been solid as a rock for at least a year <0.5 wink>. Python's optimizations are still so endearingly unambitious that I don't think they're any more dangerous than the rest of Python. For example, I've spent far more of my recent life tracking down horrid consequences of seemingly minor changes in asyncore.py. So while optimizations have been a source of bugs, they've been a very minor source. I also really appreciate that -O *doesn't* add "new bugs". I also really appreciate that bytecode optimizations are still so unambitious that the relationship between btye code and source code remains obvious. From kbk at shore.net Thu Aug 19 07:29:35 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Aug 19 07:29:41 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com> Patch / Bug Summary ___________________ Patches : 264 open (-20) / 2556 closed (+42) / 2820 total (+22) Bugs : 745 open ( +1) / 4384 closed (+23) / 5129 total (+24) RFE : 149 open ( +0) / 130 closed ( +0) / 279 total ( +0) New / Reopened Patches ______________________ baseinteger: abstract superclass for int and long (2004-08-11) http://python.org/sf/1007068 opened by Dmitry Vasiliev Return new string for single item joins (Bug #1001011) (2004-08-11) http://python.org/sf/1007087 opened by Nick Coghlan Multi-line imports implementation (2004-08-11) http://python.org/sf/1007189 opened by Dima Dorfman make inspect.getsource show @decorators (2004-08-09) http://python.org/sf/1006219 reopened by percivall Fix for bug #992078 (2004-08-11) CLOSED http://python.org/sf/1007539 opened by Mitchell Surface @decorators, including classes (2004-08-12) http://python.org/sf/1007991 opened by Jack Diederich patch for 767150 (2004-08-12) http://python.org/sf/1008086 opened by Mats Wichmann Let test__locale save locale (bug #992078) (2004-08-13) CLOSED http://python.org/sf/1008588 opened by Johannes Gijsbers (bug 952953) execve empty 2nd arg fix (2004-08-14) http://python.org/sf/1009075 opened by Dave Watson (bug 1005936) textunderscore python.sty fix (2004-08-14) http://python.org/sf/1009373 opened by Dave Watson Docstring fix in Modules/_csv.c (2004-08-15) CLOSED http://python.org/sf/1009384 opened by Cherniavsky Beni Non-ascii in non-unicode __credits__ in Lib/pydoc.py (2004-08-15) http://python.org/sf/1009389 opened by Cherniavsky Beni bottom-to-top decorator application order (2004-08-14) CLOSED http://python.org/sf/1009444 opened by Brett Cannon Fix @decorator evaluation order (2004-08-15) CLOSED http://python.org/sf/1009560 opened by Mark Russell Add missing types to __builtin__ (2004-08-16) http://python.org/sf/1009811 opened by James Y Knight PyErr_Clear() vs. AsynchronousException (2004-08-16) http://python.org/sf/1009929 opened by Armin Rigo thread Module Breaks PyGILState_Ensure() (2004-08-17) http://python.org/sf/1010677 opened by Phil Thompson Bad URL encoding in SimpleHTTPServer directory listing (2004-08-18) http://python.org/sf/1011123 opened by Niels Diepeveen regex nit in encoding declarations (2004-08-18) CLOSED http://python.org/sf/1011144 opened by George Yoshida SystemError generated by struct.pack('P', 'notanumber') (2004-08-18) http://python.org/sf/1011240 opened by Dima Dorfman Improve error reporting when Python opens source code (2004-08-18) http://python.org/sf/1011822 opened by Danny Yoo fix inspect.getsource breaking with line-continuation & more (2004-08-19) http://python.org/sf/1011890 opened by Simon Percivall Patches Closed ______________ Fix for bug #992078 (2004-08-11) http://python.org/sf/1007539 closed by jlgijsbers Match Solaris def'n of _XOPEN_SOURCE (2004-08-10) http://python.org/sf/1006629 closed by montanaro Remove ODBC library references from Windows build (2004-08-11) http://python.org/sf/1006916 closed by loewis support --with-tsc on PPC (2004-08-09) http://python.org/sf/1005891 closed by mwh Disambiguate "min() or max()" exception string (2004-08-08) http://python.org/sf/1005468 closed by loewis Make func_name writable (2004-08-06) http://python.org/sf/1004703 closed by mwh Patch to allow building of paper-*/dist.pdf (2004-08-09) http://python.org/sf/1005913 closed by nnorwitz Avoid seg fault if list object is modified during list.index (2004-08-09) http://python.org/sf/1005778 closed by nnorwitz Let test__locale save locale (bug #992078) (2004-08-13) http://python.org/sf/1008588 closed by jlgijsbers cgi.py and rfc822.py unquote fixes (2002-06-24) http://python.org/sf/573197 closed by jlgijsbers Adds cookie support to urllib2.py (2002-04-24) http://python.org/sf/548197 closed by jlgijsbers shutil.copyfile destroys hard links (Bug [ 851123 ]) (2003-12-05) http://python.org/sf/854853 closed by jlgijsbers realpath: resolve symlinks before normalizing (bug #990669) (2004-07-23) http://python.org/sf/996627 closed by jlgijsbers normpath symlinks doc warning (bug #990669) (2004-07-23) http://python.org/sf/996626 closed by jlgijsbers Docstring fix in Modules/_csv.c (2004-08-15) http://python.org/sf/1009384 closed by jlgijsbers bottom-to-top decorator application order (2004-08-14) http://python.org/sf/1009444 closed by bcannon Fixes for 'commands' module on win32 (2003-04-01) http://python.org/sf/713428 closed by jlgijsbers Make `commands' module work on Windows (2004-02-03) http://python.org/sf/889949 closed by jlgijsbers make commands.getstatusoutput work on windows (2002-12-31) http://python.org/sf/660505 closed by jlgijsbers Make commands.getstatusoutput work on Windows (2003-07-16) http://python.org/sf/772029 closed by jlgijsbers help() with readline support (2003-04-23) http://python.org/sf/726204 closed by jlgijsbers Fix @decorator evaluation order (2004-08-15) http://python.org/sf/1009560 closed by mwh extending readline functionality (2003-01-27) http://python.org/sf/675551 closed by montanaro fix inspect.getargs() crash on def foo((bar)) (bug #891637) (2004-08-08) http://python.org/sf/1005466 closed by doko email/Message.py: del_param fails when specifying a header (2004-01-08) http://python.org/sf/873418 closed by bwarsaw regex nit in encoding declarations (2004-08-18) http://python.org/sf/1011144 closed by loewis Fix for tkFont.Font(name=...) (2003-07-01) http://python.org/sf/764217 closed by loewis poor performance in xmlrpc response (2002-02-14) http://python.org/sf/517256 closed by loewis warn on assignment to None, True, False (2002-04-26) http://python.org/sf/549213 closed by loewis Oren Tirosh's fastnames patch (2002-08-20) http://python.org/sf/597907 closed by loewis PEP 269 Implementation (2002-08-23) http://python.org/sf/599331 closed by loewis factor out SMTPHandler.emit date handling (2003-08-20) http://python.org/sf/791776 closed by loewis add HTTPResponse.getheaders() (2003-09-04) http://python.org/sf/800236 closed by loewis Suggested change in how ntpath.expanduser works (2003-09-08) http://python.org/sf/802159 closed by loewis Lib/email/Encoders.py iso-2022 is 7bit (2003-09-12) http://python.org/sf/804885 closed by loewis socketmodule.c: fix for platforms w/o IPV6 (i.e.Solaris 5.7) (2003-11-19) http://python.org/sf/845306 closed by loewis inspect.getsource show @decorators (2004-08-09) http://python.org/sf/1006219 closed by jlgijsbers use just built python interp. to build the docs. (2003-10-15) http://python.org/sf/823775 closed by doko Missing INCREF in PyType_Ready (2004-06-26) http://python.org/sf/980082 closed by loewis Fix readline for utf-8 locales (2004-03-11) http://python.org/sf/914291 closed by loewis tarfile.py fix for bug #949052 (2004-07-21) http://python.org/sf/995126 closed by loewis fix for title case bug: #995422 (2004-07-22) http://python.org/sf/995740 closed by loewis New / Reopened Bugs ___________________ os.startfile() doesn't accept Unicode filenames (2004-08-11) http://python.org/sf/1007046 opened by Matthias Huening SGI (IRIX6.5.24) Problems building nismodule.c (2004-08-11) http://python.org/sf/1007223 opened by Maik Hertha SGI (IRIX 6.5.24) Error building _codecs_iso2022.c (2004-08-11) http://python.org/sf/1007249 opened by Maik Hertha os.getstatusoutput on win32 (2004-08-12) CLOSED http://python.org/sf/1008275 opened by Chmouel Boudjnah logging.getLevelName returns a number (2004-08-12) http://python.org/sf/1008295 opened by Paulo Eduardo Neves os.major() os.minor() example and description change (2004-08-12) http://python.org/sf/1008310 opened by Steven Incorrect href in Tutorial. (2004-08-13) CLOSED http://python.org/sf/1008690 opened by Johannes Gijsbers vertical bar typeset horizontal in docs (2004-08-13) http://python.org/sf/1008998 opened by Alan non-ascii readline input crashes python (2004-08-14) http://python.org/sf/1009263 opened by paul rubin Cannot doctest a SyntaxError (2004-08-15) CLOSED http://python.org/sf/1009729 opened by Raymond Hettinger xmlrpclib, PEP291, "yield" (2004-08-16) http://python.org/sf/1009803 opened by Anthony Baxter CPU usage shoots up with asyncore (2004-08-16) http://python.org/sf/1010098 opened by Thomas smtpd.py docstring says wrong default class (2004-08-16) http://python.org/sf/1010102 opened by Chris Green xml.dom documentation omits hasAttribute, hasAttributeNS (2004-08-16) http://python.org/sf/1010196 opened by Mike Brown sys.ps1 not protected in EditorWindow.py (2004-08-16) http://python.org/sf/1010370 opened by Dave Florek bsddb3 testsuite failure when running more than one time (2004-08-17) http://python.org/sf/1010645 opened by Matthias Klose test_queue fails occasionally (2004-08-17) http://python.org/sf/1010777 opened by Michael Hudson running test_codecmaps_* takes too much effort (2004-08-17) http://python.org/sf/1010952 opened by Ronald Oussoren distutils install with -b / --build-base (2004-08-18) http://python.org/sf/1011113 opened by daishi harada Make GIL acquire/release functions idempotent if PyEval_Init (2004-08-18) http://python.org/sf/1011380 opened by Gustavo J. A. M. Carneiro 2.4 asyncore breaks Zope (2004-08-18) http://python.org/sf/1011606 opened by Tim Peters distutils install with -b / --build-base (2004-08-18) CLOSED http://python.org/sf/1011687 opened by daishi harada GC assertion failure (2004-08-18) CLOSED http://python.org/sf/1011790 opened by Stephane Thiell Cannot import PKZIP 2.50 created ZIP file (2004-08-19) http://python.org/sf/1011893 opened by Dennis Chuah PyDoc does not support ZIP (2004-08-19) http://python.org/sf/1011894 opened by Dennis Chuah Bugs Closed ___________ IDLE and encodings (2003-09-18) http://python.org/sf/808719 closed by loewis Possible memory leak in output console (2004-08-10) http://python.org/sf/1006740 closed by akuchling FutureWarning when running regression tests (2004-08-09) http://python.org/sf/1006001 closed by akuchling _XOPEN_SOURCE issue on IRIX 5.3 (2004-08-08) http://python.org/sf/1005308 closed by loewis test_format fails 2.4a1 (2004-07-16) http://python.org/sf/992078 closed by jlgijsbers _SC_PAGE_SIZE unknown on IRIX 5.3 (2004-08-08) http://python.org/sf/1005568 closed by loewis HAVE_FDATASYNC incorrectly set for 2.3.4 under cygwin (2004-07-04) http://python.org/sf/985154 closed by nnorwitz new.code() not cleanly checking its arguments (2004-08-07) http://python.org/sf/1005248 closed by mwh os.getstatusoutput on win32 (2004-08-12) http://python.org/sf/1008275 closed by jlgijsbers Incorrect href in Tutorial. (2004-08-13) http://python.org/sf/1008690 closed by fdrake pdb doesn't find some source files (2003-05-15) http://python.org/sf/738154 closed by jlgijsbers inspect.getmodule symlink-related failur (2002-06-17) http://python.org/sf/570300 closed by bcannon inconsistent acceptance of floats for range() (2004-08-07) http://python.org/sf/1005325 closed by bcannon shutil.copy destroys hard links (2003-11-29) http://python.org/sf/851123 closed by jlgijsbers os.path.realpath() does not handle symlinks correctly (2004-07-14) http://python.org/sf/990669 closed by jlgijsbers misinforming help messages in pdb.py (2004-07-12) http://python.org/sf/989672 closed by jlgijsbers Carbon.CF.CFMutableArray hangs interpreter (2004-07-30) http://python.org/sf/1000914 closed by ronaldoussoren pydoc crashes on a class property (2004-02-06) http://python.org/sf/891637 closed by doko Cannot doctest a SyntaxError (2004-08-15) http://python.org/sf/1009729 closed by tim_one super instances don't support item assignment (2003-09-12) http://python.org/sf/805304 closed by rhettinger New module: doc versus action (2004-05-10) http://python.org/sf/951482 closed by rhettinger winzip cannot extrat tarfile created tars (2004-05-06) http://python.org/sf/949052 closed by loewis distutils install with -b / --build-base (2004-08-18) http://python.org/sf/1011687 closed by daishiharada GC assertion failure (2004-08-18) http://python.org/sf/1011790 closed by mbuna New / Reopened RFE __________________ translate Windows cr-lf when installing scripts on Linux (2004-08-06) http://python.org/sf/1004696 reopened by timcera RFE Closed __________ translate Windows cr-lf when installing scripts on Linux (2004-08-06) http://python.org/sf/1004696 closed by kbk From guido at python.org Thu Aug 19 07:58:25 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 07:58:33 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: Your message of "Thu, 19 Aug 2004 16:55:06 +1200." <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408190558.i7J5wPP19708@guido.python.org> > From the posts I've seen here, the only alternatives that have > considerable popular support are ones that you've already > rejected. So I suspect nobody really feels it's worth trying. Well, do people generally buy those rejections, or is their consensus that I'm mistaken? --Guido van Rossum (home page: http://www.python.org/~guido/) From tdelaney at avaya.com Thu Aug 19 08:37:24 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Aug 19 08:37:30 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D5906D@au3010avexu1.global.avaya.com> Guido van Rossum wrote: >> From the posts I've seen here, the only alternatives that have >> considerable popular support are ones that you've already >> rejected. So I suspect nobody really feels it's worth trying. > > Well, do people generally buy those rejections, or is their consensus > that I'm mistaken? I don't know about consensus, but there are a considerable number of people who think you're mistaken (it's gotten pretty heated over on c.l.py at times). Personally, I think you're mistaken about decorators being hidden in list-after-def, and problems with large, multi-line decorators - I think such problems would be few and far between. I also think that decorators are *not* the most important part of a function's signature - even classmethod - but should be *part* of a function's signature. Tim Delaney From jcarlson at uci.edu Thu Aug 19 08:53:16 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Aug 19 08:59:45 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408190401.i7J41lt19485@guido.python.org> References: <4123B712.6040502@v.loewis.de> <200408190401.i7J41lt19485@guido.python.org> Message-ID: <20040818234705.3A9F.JCARLSON@uci.edu> > Is anybody seriously trying to come up with a single alternative > decorator proposal that most folks "out there" can support, to be > presented to me (with implementation, please!) in time for 2.4b1? There was a poll linked from the decorators wiki, that now says (consider the poll now closed). If it were a democracy, the standings are: A. @classmethod def foo(): (107) 16% C1. def foo() [classmethod]: (269) 39% E1. def foo(): @classmethod (311) 45% Both C1 and E1, which are the clear favorites, have been previously rejected by Guido for various reasons. If I were to vote between C1 ad E1, I'd say E1, if only because its structure encourages reuse. - Josiah From fumanchu at amor.org Thu Aug 19 09:21:23 2004 From: fumanchu at amor.org (Robert Brewer) Date: Thu Aug 19 09:26:43 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E0A@exchange.hqamor.amorhq.net> Guido: > Is anybody seriously trying to come up with a single alternative > decorator proposal that most folks "out there" can support, to be > presented to me (with implementation, please!) in time for 2.4b1? Greg: > From the posts I've seen here, the only alternatives that have > considerable popular support are ones that you've already > rejected. So I suspect nobody really feels it's worth trying. Andrew (replying on c.l.p. for some odd reason): > To expand on that just a bit: the two alternatives that were most > generally favoured (in Doug Holton's estimate) were the > "list-after-def" (C1 in the wiki), and the "pie decorator at top of > function body" (E1 in the wiki), both of which Guido rejected. > In the poll that Doug ran, E1 had a slight margin over C1: > http://mail.python.org/pipermail/python-list/2004-August/233479.html > > However, C1 has an implementation, while E1 doesn't. (Aside: how > difficult would it be to create an implementation of E1?) Seems to me it's put up or shut up time, where "put up" means "implement". Of those proposals which have an implementation: A1: Currently in 2.4a. B: Guido prefers A1 over B. I haven't seen anyone advocating B, actually. C1: Guido rejected. J2: Guido rejected because of new keyword, but then "softened" his position. So if you can immediately reject J2, Guido, I'd say there are no candidates for an alternate. Otherwise, I'd be happy to drum up more support for J2 and get a complete proposal to you (something more than a pro/con list). Robert Brewer MIS Amor Ministries fumanchu@amor.org From mal at egenix.com Thu Aug 19 11:15:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 11:15:25 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408190139.i7J1dX819100@guido.python.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <41231830.2090903@egenix.com> <200408190139.i7J1dX819100@guido.python.org> Message-ID: <41246FA5.1010804@egenix.com> Guido van Rossum wrote: >>Hmm, who ever said that we are going to drop the current 8-bit >>string implementation ? > > > I expect that in Python 3.0 aka Python 3000 we'll have only Unicode > strings and byte arrays, so yes, the current 8-bit string > implementation will eventually die. Jython and IronPython are ahead > of us in this respect... Ok, so I suppose that we can learn from Jython and IronPython in this respect... How do they handle binary data and the interfacing between various I/O facilities, e.g. files, sockets, pipes, user input, etc. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 19 11:21:38 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 11:21:41 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <1092868762.22400.97.camel@geddy.wooz.org> References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> <20040818223629.GA17550@mems-exchange.org> <1092868762.22400.97.camel@geddy.wooz.org> Message-ID: <41247122.7030000@egenix.com> Barry Warsaw wrote: > On Wed, 2004-08-18 at 18:36, Neil Schemenauer wrote: > >>On Thu, Aug 19, 2004 at 12:16:33AM +0200, Jack Jansen wrote: >> >>>genericbytes >>> mutablebytes >>> bytes >>> genericstring >>> string >>> unicode >> >>I think characters (unicode or otherwise) should not be confused >>with bytes. Having 'unicode' as a subclass of 'bytes' is very >>confusing to me. > > > Agreed! I assume the picture will look more like this: basesequence mutable basebytes bytes array cStringIO mmap immutable unicode tuple basenumber integer float decimal complex etc. I've dropped basestring here since we'll probably only have one string type in Py3k. integer covers both int and long. Perhaps we can merge float and decimal by that time as well ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From g.biggs at auckland.ac.nz Thu Aug 19 11:23:12 2004 From: g.biggs at auckland.ac.nz (Geoffrey Biggs) Date: Thu Aug 19 11:23:26 2004 Subject: [Python-Dev] Weird errors when adding a new built-in type Message-ID: <41247180.5070304@auckland.ac.nz> Evening all, This is my first time posting to the dev list so, as the email instructed, I'll introduce myself. My name is Geoffrey Biggs and I'm a PhD student at the University of Auckland in New Zealand working on some specialised programming language concepts. I'm implementing my ideas in Python to prove they work (or don't work, as the case may be). Apologies in advance for posting something to the python-dev list that isn't directly about improving the standard Python, but I figured this is the place where the people who know what I'm asking about and might know the answer would best be found and it doesn't seem to fit in python-help's description. I'm trying to add a new built-in number data type to Python with its own syntax, so I'm working directly with the interpreter rather than creating my own extension module (side note: I've appended something extra to the version thing in the Makefile - I doubt this is relevant to the problem but it's probably best you have all the info). The complex data type is similar to what I'm trying to do so I've been following that as an example. I've successfully extended the tokenizer and the parsenumber() function in compile.c to do what I want and written the data type in my two new files Objects/mynewobject.c and Include/mynewobject.h. I've included mynewobject.h in Python.h and added the two files to the Makefile. However, when I tried to run the code by typing in the syntax to produce my data type, Python suffers a segmentation fault. I traced this to an attempt to get the hash of a null pointer when adding the new instance of my type after parsenumber() was called. For completeness, here's the backtrace: Program received signal SIGSEGV, Segmentation fault. 0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162 1162 if (tp->tp_hash != NULL) (gdb) bt #0 0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162 #1 0x0808e199 in tuplehash (v=0x40324574) at Objects/tupleobject.c:244 #2 0x08080eae in PyObject_Hash (v=0x40324574) at Objects/object.c:1163 #3 0x0807b46c in PyDict_GetItem (op=0x40319c14, key=0x40324574) at Objects/dictobject.c:492 #4 0x080c72c7 in com_add (c=0xbfffed60, list=0x40326114, dict=0x40319c14, v=0x40326290) at Python/compile.c:975 #5 0x080c74ac in com_addconst (c=0xbfffed60, v=0x40326290) at Python/compile.c:1001 #6 0x080c90e3 in com_atom (c=0xbfffed60, n=0x4028d500) at Python/compile.c:1689 The crash appears to be caused because while v exists, v's members are all 0 and so tp becomes 0 and you get a NULL pointer exception. As far as I can tell, this v is the second object in the tuple created in com_add() and is supposed to be the type of the object being added to the dictionary in a tuple. Not knowing why it was coming out as zero, I did some more poking around (been doing a lot of that over the past 5 days...) and found that there is a file called bltinmodule.c with a bunch of lines, one of which mentions the complex data type that I am using as a guide: SETBUILTIN("complex", &PyComplex_Type); So I made one of these for mynewobject and added it. I figured from this bit of code that the reason I was getting NULL pointer exceptions was because my type hadn't been initialised in some way and that this would do it. But here's the problem: despite trying for longer than I've slept in the past week, I can't get it to compile. It always produces the following error (after the linker step that produces the python executable completes): case $MAKEFLAGS in \ *-s*) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes -D__SAMMY_EXT__' ./python -E ./setup.py -q build;; \ *) CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall -Wstrict-prototypes -D__SAMMY_EXT__' ./python -E ./setup.py build;; \ esac make: *** [sharedmods] Error 139 Running the python executable doesn't work either, it starts and then segfaults immediatly with the backtrace indicating it dies trying to build the os module (which probably didn't get built). I haven't been able to figure out why this is happening. If I comment out the SETBUILTIN line I added it will compile fine but then I go back to the first problem. The only thing I found in google with this error was someone who was having trouble with a raw python source, and in that case it was a problem trying to build the ssl extension. I have structured my object identically to the complex object, which works. So the question (finally) is, what's broken? What have I missed? My best guess is only that I've broken the interpreter somehow by missing something with my object implementation and this is affecting that step of the make process. (This mess is all happening on Linux tillinshir 2.6.7-gentoo-r11-gb #1 Wed Aug 4 11:13:14 NZST 2004 i686 mobile AMD Athlon(tm) XP 2000+ AuthenticAMD GNU/Linux, if that matters.) Thanks in advance, Geoff Biggs From mal at egenix.com Thu Aug 19 11:34:19 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 11:34:25 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123B804.2000508@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <4123B804.2000508@livinglogic.de> Message-ID: <4124741B.3080605@egenix.com> Walter D?rwald wrote: > Martin v. L?wis wrote: > >> M.-A. Lemburg wrote: >> >>> I've thought about this some more. Perhaps I'm still missing >>> something, but wouldn't it be possible to add a feeding >>> mode to the existing stream codecs by creating a new queue >>> data type (much like the queue you have in the test cases of >>> your patch) and using the stream codecs on these ? >> >> >> Here is the problem. In UTF-8, how does the actual algorithm >> tell (the application) that the bytes it got on decoding provide >> for three fully decodable characters, and that 2 bytes are left >> undecoded, and that those bytes are not inherently ill-formed, >> but lack a third byte to complete the multi-byte sequence? >> >> On top of that, you can implement whatever queuing or streaming >> APIs you want, but you *need* an efficient way to communicate >> incompleteness. > > > We already have an efficient way to communicate incompleteness: > the decode method returns the number of decoded bytes. > > The questions remaining are > > 1) communicate to whom? IMHO the info should only be used > internally by the StreamReader. Handling incompleteness should be something for the codec to deal with. The queue doesn't have to know about it in an way. However, the queue should have interfaces allowing the codec to tell whether there are more bytes waiting to be processed. > 2) When is incompleteness OK? Incompleteness is of course > not OK in the stateless API. For the stateful API, > incompleteness has to be OK even when the input stream > is (temporarily) exhausted, because otherwise a feed mode > wouldn't work anyway. But then incompleteness is always OK, > because the StreamReader can't distinguish a temporarily > exhausted input stream from a permanently exhausted one. > The only fix for this I can think of is the final argument. A final argument may be the way to go. But it should be an argument for the .read() method (not only the .decode() method) since that's the method reading the data from the queue. I'd suggest that we extend the existing encode and decode codec APIs to take an extra state argument that holds the codec state in whatever format the codec needs (e.g. this could be a tuple or a special object): encode(data, errors='strict', state=None) decode(data, errors='strict', state=None) In the case of the .read() method, decode() would be called. If the returned length_consumed does not match the length of the data input, the remaining items would have to be placed back onto the queue in non-final mode. In final mode an exception would be raised to signal the problem. I think it's PEP time for this new extension. If time permits I'll craft an initial version over the weekend. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 19 11:41:51 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 11:42:05 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123BC56.7030607@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> Message-ID: <412475DF.9010308@egenix.com> Martin v. L?wis wrote: >> We do need to extend the API between the stream codec >> and the encode/decode functions, no doubt about that. >> However, this is an extension that is well hidden from >> the user of the codec and won't break code. > > > So you agree to the part of Walter's change that introduces > new C functions (PyUnicode_DecodeUTF7Stateful etc)? We'll have to see... all this is still very much brainstorming. > I think most of the patch can be discarded: there is no > need for .encode and .decode to take an additional argument. > It is only necessary that the StreamReader and StreamWriter > are stateful, and that only for a selected subset of codecs. > > Marc-Andre, if the original patch (diff.txt) was applied: > What *specific* change in that patch would break code? > What *specific* code (C or Python) would break under that > change? > > I believe the original patch can be applied as-is, and > does not cause any breakage. It also introduces a change > between the codec and the encode/decode functions that is > well hidden from the user of the codec. Let's flesh this out some more and get a better understanding of what is needed and how the separation between the stream queue, the stream codec and the underlying codec implementation can be put to good use. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 19 12:15:49 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 12:15:51 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123BD93.2040107@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de> Message-ID: <41247DD5.8020100@egenix.com> Walter D?rwald wrote: > Let's compare example uses: > > 1) Having feed() as part of the StreamReader API: > --- > s = u"???".encode("utf-8") > r = codecs.getreader("utf-8")() > for c in s: > print r.feed(c) > --- I consider adding a .feed() method to the stream codec bad design. .feed() is something you do on a stream, not a codec. > 2) Explicitely using a queue object: > --- > from whatever import StreamQueue > > s = u"???".encode("utf-8") > q = StreamQueue() > r = codecs.getreader("utf-8")(q) > for c in s: > q.write(c) > print r.read() > --- This is probably how an advanced codec writer would use the APIs to build new stream interfaces. > 3) Using a special wrapper that implicitely creates a queue: > ---- > from whatever import StreamQueueWrapper > s = u"???".encode("utf-8") > r = StreamQueueWrapper(codecs.getreader("utf-8")) > for c in s: > print r.feed(c) > ---- This could be turned into something more straight forward, e.g. from codecs import EncodedStream # Load data s = u"???".encode("utf-8") # Write to encoded stream (one byte at a time) and print # the read output q = EncodedStream(input_encoding="utf-8", output_encoding="unicode") for c in s: q.write(c) print q.read() # Make sure we have processed all data: if q.has_pending_data(): raise ValueError, 'data truncated' > I very much prefer option 1). I prefer the above example because it's easy to read and makes things explicit. > "If the implementation is hard to explain, it's a bad idea." The user usually doesn't care about the implementation, only it's interfaces. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 19 12:29:12 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 12:29:21 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123C76B.3030100@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> Message-ID: <412480F8.4060903@egenix.com> Walter D?rwald wrote: > Without the feed method(), we need the following: > > 1) A StreamQueue class that > a) supports writing at one end and reading at the other end > b) has a method for pushing back unused bytes to be returned > in the next call to read() Right. It also needs a method giving the number of pending bytes in the queue or just an API .has_pending_data() that returns True/False. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jim at zope.com Thu Aug 19 12:29:19 2004 From: jim at zope.com (Jim Fulton) Date: Thu Aug 19 12:30:01 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: References: Message-ID: <412480FF.2030203@zope.com> Bob Ippolito wrote: > > On Aug 18, 2004, at 3:42 PM, Kevin Smith wrote: > >> For what it's worth, I wrote the original PEP 318. I probably wasn't >> qualified, but I just wanted a nice simple way to declare class >> methods without having to repeat the function name. After submitting >> it to BDFL for approval, more work was needed and the discussion of >> PEP 318 on python-dev increased rapidly. It was evident that I was in >> over my head, so I asked more someone more experienced to take over. >> >> I guess others had bigger plans for my proposal that I had planned. >> It has turned into the "solution" to many problems: type checking >> (both arguments and returned values), metaclasses, metadata, >> interfaces, function attributes, etc.). Unfortunately, in the >> process, this simple request for syntactical sugar has turned into a >> monstrosity. In my opinion, none of the proposed syntaxes really seem >> Pythonic. This PEP just seems to be trying to solve too many problems. >> >> Bear with me, but I'd like to propose one more syntax that is simple, >> easy for newbies to understand, and nowhere near as powerful as the >> current PEP's syntax. However, it doesn't add incoherent, arbitrary >> syntax either. >> >> def classmethod foo(x, y, z): >> pass >> >> That's it. One "decorator" that is a callable object that takes a >> method as it's only argument. No expressions, lists, tuples, etc. >> Just one callable object. Ok, if you absolutely must have more than one. >> >> def classmethod synchronized foo(x, y, z): >> pass >> >> Once again, no expressions. I know that this isn't going to solve >> everyone's type-checking, metadata, and function attribute problems, >> but let's face it, using this PEP for all of those things just creates >> ugly syntax. There must be more Pythonic ways to do those things in >> their own PEPs. > > > The problem is that there is FAR FAR more use of type-checking, > metadata, and function attributes in the wild (from what I've seen and > written) than simple stuff like classmethod, staticmethod, and this > hypothetical synchronized. I've never seen anything but classmethod, staticmethod, property and my own "contextmethod" and "cachedproperty", both of which are better served by the simple syntax, in the "wild". The simple syntax serves these uses (yes, excluding write properties) very well. I certainly *will* use synchronized the the future (I've used something like it in the past.) > I don't think proposals such as this do very > much for anyone. Oh come on. It does a lot for me. IMO, it does a lot for future users of the language by keeping it simpler. Saying that a proposal you don't agree with doesn't do much for anyone is unfair IMO. I think it's pretty inappropriate to devalue arguments just because you don't agree with them. > It's too simple to be useful, and has too many > side-effects on existing tools and people that try and read python > source :) Please. Any syntax will have effects on existing source and users. IMO, this *obvious* syntax best serves the original purpose of the PEP, which is to solve the *simple* problem if making the syntax of class and static methods sane. > Is @syntax really SO horrible that you propose that we make decorators > near useless to most but more "aesthetically pleasing" to some? IMO, yes, although I'd happily accept both syntaxes. I can live with @foo to server the people who want it, but it bugs me that I have to write: @clasmethod def foo .... rather than the, to me, much more obvious and readable: def classmethod foo ... > Do you > use a significant amount of decorators now? I use class methods and static methods. I use properties a lot, including lots of read properties. I have my own kinds of decorators as well, as mentioned above. > Do you plan to use them > much in the future? It seems to me that most people who argue against > the sensible and flexible syntaxes are the same people who will rarely > if ever use decorators at all... Well, I use this sort of thing a lot. Most of my uses are served by the simpler syntax. For the uses that aren't served by the simpler syntax, the @foo syntax either doesn't help, as in properties and variations on properties, or doesn't add anything. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jim at zope.com Thu Aug 19 12:31:25 2004 From: jim at zope.com (Jim Fulton) Date: Thu Aug 19 12:32:02 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <91FA4761-F154-11D8-BC29-000393829ED2@theMorgue.org> References: <4123B712.6040502@v.loewis.de> <91FA4761-F154-11D8-BC29-000393829ED2@theMorgue.org> Message-ID: <4124817D.8030600@zope.com> Kevin Smith wrote: > On Aug 18, 2004, at 4:07 PM, Martin v. L?wis wrote: > ... > I guess I just find the "public > final synchronized void foo()" less monstrous than > > @public > @final > @synchronized > @void > def foo() Me too! Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From Michael.Sparks at rd.bbc.co.uk Thu Aug 19 12:32:57 2004 From: Michael.Sparks at rd.bbc.co.uk (Michael Sparks) Date: Thu Aug 19 12:39:34 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408190401.i7J41lt19485@guido.python.org> References: <4123B712.6040502@v.loewis.de> <200408190401.i7J41lt19485@guido.python.org> Message-ID: <200408191132.57095.Michael.Sparks@rd.bbc.co.uk> Guido, On Thursday 19 Aug 2004 05:01, Guido van Rossum wrote: > Is anybody seriously trying to come up with a single alternative > decorator proposal that most folks "out there" can support, to be > presented to me (with implementation, please!) in time for 2.4b1? Yes. The following syntax discussed on comp.lang.python struck me as having legs so I produced an implementation. Hopefully this email summarises the resulting discussion so far adequately, and if you have a moment, an answer to the question I pose at the end of this email would be very welcome. To me, this syntax appears to retain the advantages of @decorator whilst having some clear benefits for new users, or people who only occasionally look at python for maintenance. A number of old hands have said they too like this syntax, and couple of others who said they think it's too heavyweight have so far rated it -0. Example: # This is syntax J2 from http://www.python.org/moin/PythonDecorators decorate: staticmethod grammarrule('statement : expression') versioninfo("Added in 2.4") deprecated typeinfo(None) def p_statement_expr(self, p): print p[1] # Single line form, suggested by Paul Du Bois, Nick Coghlan decorate: staticmethod def p_statement_expr(self, p): pass This prompted me to look at the source for the first time and I've posted an initial implementation (for discussion purposes) here: * http://mail.python.org/pipermail/python-list/2004-August/233591.html And also referenced/discussed it here: * http://mail.python.org/pipermail/python-dev/2004-August/047807.html There is one issue outstanding with this implementation -- for some reason the change in indent level means that scoping errors occur unless you force the functions into scope thus: ========================= class Foo: staticmethod # This is cruft to force staticmethod into scope decorate: staticmethod def hello(who): print "woo?", who Foo.hello("HOO") ========================= I have held back on finishing this off because I'd like to hear whether it is an acceptable alternative to people. So far people who have expressed opinions on this syntax in since I posted the link to the implementation are: Bob Ippolito * Voted -0, too heavyweight for simple case Paul Du Bois, * Suggested single line for for simple case, and also stated "This is my favorite proposal so far." Barry Warsaw, * Voted -0, same reasons as Bob, _softened_ on thought of everything on single line for simple case with reservations on keyword. Nick Coghlan, * Liked the fact it simplifies folding for editors, also suggested single line for simple case Robert Brewer, * Prompted me to forward the implementation/suggestion to this list, and has offered to help drum up support for this, and put forward a complete proposal! (apologies if I've summarised things inaccurately...) Single line form suggested for the simple case was: decorate: staticmethod def p_statement_expr(self, p): pass The key reasons I prefer this slightly more verbose version are: 1 In the complex case, it makes it clearer to see where the function begins 2 The simple cases, such as staticmethod & classmethod, are very unusual definitions for general code. The fact that this is more heavyweight makes the staticmethod/classmethod decorator jump out at you more when skimming code. (especially unfamiliar code) To me this increases readability. Naturally the sacrifice there is an extra burden at writing time. 3 It gives someone something concrete to search for when looking for help: eg "python decorate" vs "python @" One clear possible downside: * Like @decorator these can only be declarations, not statements. This in itself might be confusing. This ambiguity does not exist with @decorator in my opinion. Regarding 1, compare the following two hypothetical pieces of code. The comment block before the function deliberately attaches to the function because some code documentation tools I've seen will take such blocks and include them in the docs as well as doc strings. (Meaning doc strings can be short/focussed) #Syntax J2 from http://www.python.org/moin/PythonDecorators #Some random comment #More random comment decorate: staticmethod grammarrule('statement : expression') versioninfo("Added in 2.4") deprecated typeinfo(None) def p_statement_expr(self, p): print p[1] Using current syntax: #Current syntax in 2.4a2 #Some random comment #More random comment @staticmethod @grammarrule('statement : expression') @versioninfo("Added in 2.4") @deprecated @typeinfo(None) def p_statement_expr(self, p): print p[1] I think other arguments I'd put forward are probably more aesthetic than practical. As noted above, I have held off resolving the (hopefully) minor scoping issues since I'd wanted to hear if this is an acceptable alternative. The responses so far make me think that it is acceptable to people if the simple case has the single line form available, but there are reservations due to the keyword requirement. If you have a moment, I would very much appreciate an indication as to whether you would reject this at this stage or not? Best Regards, Michael. -- Michael Sparks, Senior R&D Engineer, Digital Media Group Michael.Sparks@rd.bbc.co.uk, British Broadcasting Corporation, Research and Development Kingswood Warren, Surrey KT20 6NP This e-mail contains personal views which are not the views of the BBC. From gmccaughan at synaptics-uk.com Thu Aug 19 12:58:12 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu Aug 19 12:58:45 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408190558.i7J5wPP19708@guido.python.org> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> <200408190558.i7J5wPP19708@guido.python.org> Message-ID: <200408191158.12738.gmccaughan@synaptics-uk.com> On Thursday 2004-08-19 06:58, Guido van Rossum wrote: > > From the posts I've seen here, the only alternatives that have > > considerable popular support are ones that you've already > > rejected. So I suspect nobody really feels it's worth trying. > > Well, do people generally buy those rejections, or is their consensus > that I'm mistaken? I'm one of the people who doesn't really feel it's worth trying. *My* consensus :-) is that you're mistaken. I think that - the @-syntax is horrible and unpythonic as all-get-out, but usable; - your arguments against the []-before-colon form are utterly bogus; - I don't recall seeing a compelling argument against the decorate: classmethod memoize wibblify(3,"aardvark") def make_foo(cls, *args): do_something(args) proposal that was made recently in python-dev (but must surely have come up before). That syntax looks much better to me than the pie syntax. The "only" problem is that it's weird to have a suite of things that aren't statements to execute, but it doesn't seem any weirder to me than pie. Oh, one other problem: the order of application is non-obvious. Same as with pie. (I think it's obvious, by the way, that this syntax should include a without-line-break version for single decorators.) I should expand on that second point, though I expect it's all been said before. Here's what you said about the list-before-colon syntax, quoted from the Wiki page about decorators, with comments. | That's a total jumble of stuff ending with a smiley. (True story: I | left out the colon when typing up this example and only noticed in | proofing.) The jumbliness of your example is a consequence of the jumbly set of decorators it uses, and has precious little to do with the decorator syntax being used. It doesn't look much better, to me at least, using pie syntax. And if you're seriously using the fact that it ends with "]:" as an argument against it, then something has gone badly wrong. The fact that you made a minor error of the sort Python catches automatically while typing in an example in a syntax you have strong negative feelings about doesn't seem to me like much of an argument against that syntax, either. | Problems with this form: | | - it hides crucial information (e.g. that it is a static method) | after the signature, where it is easily missed I don't believe it's substantially more easily missed after the signature than before. | - it's easy to miss the transition between a long argument list and a | long decorator list Not much easier than to miss the transition between a long argument list and the function body. Anyway, what's so difficult about, say, def cumbersomely_named_method(arg1, arg2, arg3, arg4, arg5, arg91232, arg8724, arg8123721) \ [ staticmethod, accepts(int, int, str, dict, int, None, None, int), memoize((1, 2, 3)) ]: """Here's the docstring.""" do_something() do_something_else(arg1) # etc This sort of thing is the worst case for the list-before-colon syntax. It still doesn't seem much worse than pie. Whereas in the simplest case I think pie is a lot worse: def make_foo(cls) [classmethod]: return cls("foo") @classmethod def make_foo(cls): return cls("foo") | - it's cumbersome to cut and paste a decorator list for reuse, because | it starts and ends in the middle of a line If a decorator list is long enough to be worth cutting and pasting then (1) it's probably worth building a new decorator that encapsulates what it does, and (2) you can always use a layout that *does* put it on separate lines, like the cumbersomely_named_method example above. | Given that the whole point of adding decorator syntax is to move the | decorator from the end ("foo = staticmethod(foo)" after a 100-line | body) to the front, where it is more in-your-face, it should IMO be | moved all the way to the front. That isn't the whole point, though it's certainly a lot of the point. And list-before-colon seems pretty in-your-face to me, though admittedly it doesn't have that "Oi, you, listen up, there's something utterly unlike anything else in Python happening here" feel about it that pie does. * Now, having said all that: let's suppose for the sake of argument that your arguments against list-before-colon are completely without merit. There's still the *fact* that you dislike it, and your intuition is notoriously pretty good :-). So I suspect that maybe there's something really bad about list-before-colon that I haven't noticed, or that I've not seen the importance of. But, until such time as that's pointed out to me in a way I'm not too stupid to appreciate, I still like list-before-colon. * I'm not sure what you're thinking *should* happen here that would convince you (or anyone else) to adopt a different syntax. All the other Python developers choosing another syntax and agreeing to go with it? That's never going to happen spontaneously when there's more than one possible alternative. I think that either list-before-colon or indented-suite (options C1 and J2 on the Wiki page) are clearly preferable to pie, and for what little it's worth I hereby declare my support for any consensus or partial consensus that might emerge for either. -- g From hyeshik at gmail.com Thu Aug 19 14:21:50 2004 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Thu Aug 19 14:21:53 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412480F8.4060903@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <412480F8.4060903@egenix.com> Message-ID: <4f0b69dc040819052178b43de9@mail.gmail.com> On Thu, 19 Aug 2004 12:29:12 +0200, M.-A. Lemburg wrote: > Walter D?rwald wrote: > > Without the feed method(), we need the following: > > > > 1) A StreamQueue class that > > a) supports writing at one end and reading at the other end > > b) has a method for pushing back unused bytes to be returned > > in the next call to read() > > Right. > > It also needs a method giving the number of pending bytes in > the queue or just an API .has_pending_data() that returns > True/False. > +1 for adding .has_pending_data() stuff. But it'll need a way to flush pending data out for encodings where incomplete sequence not always invalid. This is true for JIS X 0213 encodings. >>> u'\u00e6'.encode('euc-jisx0213') '\xa9\xdc' >>> u'\u3000'.encode('euc-jisx0213') '\xa1\xa1' >>> u'\u00e6\u0300'.encode('euc-jisx0213') '\xab\xc4' Hye-Shik From mal at egenix.com Thu Aug 19 14:25:55 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 14:25:58 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4f0b69dc040819052178b43de9@mail.gmail.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <412480F8.4060903@egenix.com> <4f0b69dc040819052178b43de9@mail.gmail.com> Message-ID: <41249C53.8020902@egenix.com> Hye-Shik Chang wrote: > On Thu, 19 Aug 2004 12:29:12 +0200, M.-A. Lemburg wrote: > >>Walter D?rwald wrote: >> >>>Without the feed method(), we need the following: >>> >>>1) A StreamQueue class that >>> a) supports writing at one end and reading at the other end >>> b) has a method for pushing back unused bytes to be returned >>> in the next call to read() >> >>Right. >> >>It also needs a method giving the number of pending bytes in >>the queue or just an API .has_pending_data() that returns >>True/False. >> > > > +1 for adding .has_pending_data() stuff. But it'll need a way to > flush pending data out for encodings where incomplete sequence not > always invalid. This is true for JIS X 0213 encodings. > > >>>>u'\u00e6'.encode('euc-jisx0213') > > '\xa9\xdc' > >>>>u'\u3000'.encode('euc-jisx0213') > > '\xa1\xa1' > >>>>u'\u00e6\u0300'.encode('euc-jisx0213') > > '\xab\xc4' I'm not sure I understand. The queue will also have an .unread() method (or similiar) to write data back into the queue at the reading head position. Are you suggesting that we add a .truncate() method to truncate the read buffer at the current position ? Since the queue will be in memory, we can also add .writeseek() and .readseek() if that helps. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From gregory.lielens at fft.be Thu Aug 19 15:27:03 2004 From: gregory.lielens at fft.be (Gregory Lielens) Date: Thu Aug 19 15:27:07 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408191158.12738.gmccaughan@synaptics-uk.com> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> <200408190558.i7J5wPP19708@guido.python.org> <200408191158.12738.gmccaughan@synaptics-uk.com> Message-ID: <1092922023.6275.245.camel@gauss.fft> On Thu, 2004-08-19 at 12:58, Gareth McCaughan wrote: > On Thursday 2004-08-19 06:58, Guido van Rossum wrote: > > > From the posts I've seen here, the only alternatives that have > > > considerable popular support are ones that you've already > > > rejected. So I suspect nobody really feels it's worth trying. > > > > Well, do people generally buy those rejections, or is their consensus > > that I'm mistaken? > > I'm one of the people who doesn't really feel it's worth trying. For what it's worth, me too ;-) First, a warning: I have never used any decorator and I jumped in this discussion (poor me ;-) ) purely from an aesthetic/syntax-coherence point of view. It is just because @decorator feels so odd compared to the rest of python that I though I had to react... Why odd? Because: @ is a new symbol, and is slightly annoying for ipython that I use (while I have yet to encounter decorators :-) ) @ perform a "magic" operation, applying a callable object to something that will be defined (possibly multiple line) after (and in a certain order that we have to remember)...This is very unlike all other binary and unary symbol operation we have currently in python, and not common in programming languages imho That being said, I am not anti-punctuation, nor anti-symbol, nor anti-@?|!#%-cause-it-trigger-my-perl-allergy ;-) (I am behind a PEP for elementwise operators, if anybody need proof ;-) ), but in the decorator case I do not see it as a readability improvement And mostly, what I find odd is the application of decorators to next definition, this feels like an entirely new syntax concept to me, while the list after def seems more like a slight extension, so is the decorate: dec1 dec2 def f(): ... proposal (although I feel like the extension is more drastic in this case)... So my question/feeling is: are decorator so important that a whole-new syntax (instead of an extension of existing syntax) is needed? For now, I do not have seen any reason to believe it is, and worse, I do not personally find the list after def or variants (especially my own variant, def f(...) "pseudo-keyword" or "%" iterable-object :, of course ;-) ) any less clear or difficult to parse than the @ version, even with very complex decorators. I also think at this stage the only reason to change my feeling about this would be to have other cases beside decorator where the pie-syntax @stuff @more_stuff @and_some_extra_stuff applied_to_me would be used, so that this new syntax have more reason to exist... However, even if my feeling against @ pie is strong, I slowly got ready to bite the bullet (and to pray for seeing this @ as less as possible) giving that there is a huge multiplication of possible syntax, while comments on each possible syntax are seemingly decreasing...This looks like end-of-story too me, hence my "too bad, but let's move" feeling... One possible thing that may help make an alternative syntax emerge would be for Guido to parse the wiki and definitely reject some of the proposals, so that people can progressively gather behind the ones left...a sort of sorry-you're dismissed kind of game ;-) Else, I fear that only-one-alternative-syntax-or-it-will-be-@ is a polite way to say it-will-be-@ ;-P Regards, Greg. From skip at pobox.com Thu Aug 19 16:11:53 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 19 16:12:16 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com> Message-ID: <16676.46377.92891.139081@montanaro.dyndns.org> >> Is anybody seriously trying to come up with a single alternative >> decorator proposal that most folks "out there" can support, to be >> presented to me (with implementation, please!) in time for 2.4b1? Tim> I'm afraid that absolutely zero consensus is emerging :( I've Tim> withdrawn myself entirely from the discussions as they've proven Tim> completely useless. Ditto. I think the list-after-def proposal had some support early on but people got very carried away proposing all sorts of other alternatives. Skip From bac at OCF.Berkeley.EDU Thu Aug 19 09:02:07 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 19 16:16:58 2004 Subject: [Python-Dev] python-dev Summary for 2004-08-01 through 2004-08-15 [draft] Message-ID: <4124506F.6040503@ocf.berkeley.edu> Well, after reading 1,289 emails for this summary I finally finished it. As you will notice in the Summary Announcements I have my yearly stats on the list since this summary marks the end of year two for me doing this. As I did last year, the position is available to anyone who wants it and can do a comparable job. Also notice that in the Summary Announcements I am asking for people to vote on one of two ways I am going to cut back on the amount of time the Summaries take up of my time. If you actually read the summaries then do let me know since votes from regular python-dev'ers will count for more than from the general Python community. I don't plan to send this summary out any sooner than Sunday night so you have at least until then to send me corrections. I am burned out enough at the moment that I doubt I will hit that date, though. Also, I doubt I covered anything new in my decorators summary, but if I did let me know and I will make sure to plug it into the PEP or wiki if needed. --------- ===================== Summary Announcements ===================== Well, I have now been doing the summaries for two years. As has become a yearly tradition, I am offering to pass on the writing of the summaries to someone else. My only requirement is you do a comparable job. You do learn *tons* about Python's internals when you have to research a topic to be able to summarize it. I know it greatly accelerated my understanding of both Python as a language and of its implementation. With that said, it is also time for stats on the list to scare away anyone considering taking over this job =) . According to my probably buggy script that I wrote last year, I have read 10,190 emails over the past year (this month has already been the busiest month with 1,289 emails and it is only half over; could set the record for busiest month ever). The people with over 300 emails posted over the year are: 9. Michael Hudson (303) 8. Martin v. Lowis (307) 7. Barry Warsaw (341) 6. Phillip J. Eby (341) # not a typo; same number as Barry 5. Greg Ewing (354) 4. Raymond Hettinger (372) 3. Skip Montanaro (399) 2. Tim Peters (629) 1. Guido van Rossum (1031) These 9 people make up over 40% of all the emails from the past year. Longest threads were: 1. decorate-sort-undecorate (694 emails) 2. Call for defense of @decorators (195 emails) 3. PEP 318: Decorators last before colon (181 emails) 4. redefining is (162 emails) 5. Christmas Wishlist (162 emails) These stats along with the insane amount of email has showed me something; the Summaries have detracted from my participation on python-dev this past year. I have a bigger mouth and more opinions than the number of emails I sent to the list show. This means something has to change, and it isn't going to be my mouth. The Summaries need to change in one of two ways. Option one is that I focus in on certain areas of interest and skip other ones. I have labeled all of the summaries below with what their type is right under the titles. Please email me your top ares of interest. I realize that this month may not be typical but pretty much all areas are covered at least once so at least there is a good taste for the different areas. So, choose from: 1. improvement stuff Pretty much anything that is not a major change to the language/stdlib. Usually something you find out from the Misc/NEWS or the "What's New" doc. 2. how python-dev works Stuff like how python-dev handles things such as the PEP process, coding style, etc. 3. nuggets of info Cool tricks and ideas that come up on the list that the greater world probably does not know. 4. minor language change Stuff that deals with the language changing, but is not significant; such as discussions of existing PEPs. Language evolution stuff (such as decorators) will always be covered so you don't need to vote on that. If people like this first option then I will make sure I cover the area with most votes and everything else is just considered icing. Option two out of all of this is people just say, "summarize what you want, Brett." Then I just cover what I find interesting and just don't worry about covering a specific area. I obviously prefer this option but if people really care about a specific area I want to make sure to cover it. What will most likely happen is I will still cover almost everything but the thoroughness will be lower. I will go more off of memory for example. But something will change. Being on the sidelines for the decorators discussion because I dreaded having to start reading all of those emails in terms of summarizing is not acceptable (and no, I can't just ignore some of them since that is just not how I work). Having to read 1,289 emails for just the first two weeks of this month finally caused me to lose my sanity. ======== Summary ======== -------------------------------------------------------------------------------- Multiple interpreters at a time in a C program kind of broken -------------------------------------------------------------------------------- 3. nuggets of info Philip Eby thought he might have a fix for a known limitation of running multiple interpeters at once (using PyInterpreter_New() ) and having imports not being clearly separated between interpreters. But Martin v. L?wis popped Philip's bubble somewhat by saying that he and some other people viewed multiple interpreter support as inherently broken. Contributing threads: - `Safe to change a thread's interpreter? `__ -------------------------------------------- Another sucker becomes an official developer -------------------------------------------- 2. how python-dev works Sean Reifschneider took the bait to become another one of Guido's minions. Sean has been maintaining the RPM spec files for quite a while. Contributing threads: - `New developer `__ ---------------------------------------- Discovering leaks with a regrtest wrench ---------------------------------------- 1. minor language change Michael Hudson, who found and fixed a bunch of refcount leaks shortly after 2.3 was released, used his magical regrtest patch (which has subsequently been checked in) to hunt down leaks for 2.4 . A bunch of tests were cleaned up to make them not raise false-positives along with fixing some true leaks. Contributing threads: - `refleak hunting season `__ --------------------------------- Another Bug Day has come and gone --------------------------------- 2. how python-dev works The Bug Day held on August 7 led to 19 bugs and 12 patches being closed. You can see the results at http://www.python.org/cgi-bin/moinmoin/PythonBugDayStatus . Feel guilty for not being able to make it? Still want to help? Go to that page to see bugs that could use some squashing that shouldn't be too difficult to deal with. Contributing threads: - `Bug day coming up this Saturday `__ -------------------------------------------------- How to shut a compiler up about an unused variable -------------------------------------------------- 3. nuggets of info Tim Peters had come up with some optimizations for list.pop() that avoided unneeded test+branches. The problem is that it led to a variable possibly being unused. C compilers tend to complain about that and so an initial solution was used. Unfortunately gcc complained about it, and so yours truly came up with one. But that solution was labeled as "perverse" (and it was; total hack solution), so another solution was found thanks to Sjoerd Mullender by just taking the variable and casting it to void. Contributing threads: - `RE: [Python-checkins] python/dist/src/Objects listobject.c, ... `__ ------------------------------------------------------ Variable sequence unpacking assignment shot down again ------------------------------------------------------ language evolution David Cole suggested adding the ability to have sequence unpacking assignment just like *args for parameters; ``a, b, *c = (0, 1, 2, 3, 4, 5) # a == 0, b == 2, c == (3, 4, 5)``. This idea got a -1 from Guido as not really needed. That reaction makes Guido consistent; the idea was originally shot down back in Nov. 2002; http://www.python.org/dev/summary/2002-11-16_2002-11-30.html#half-baked-proposal-and-in-assignments . If you really like that idea the previous summary contains a function that helps you do this in a way. Contributing threads: - `Tuple/list assignment question `__ ------------------------------------------------------------------------------------- Changing the Big-O complexity for something in the language is now a language feature ------------------------------------------------------------------------------------- language evolution Armin Rigo came up with a way to have string concatenation in a loop (think ``for thing in iter_of_strings: concat_str += thing``) not be a quadratic algorithm thanks to some trickery for ``a = a + b`` and ``a += b`` conditions for strings. The hope was to remove the (commonly considered) wart of having ``"".join(iter_of_strings)`` be the suggested way to concatenate a bunch of strings. But Guido didn't like the patch. His reasoning was that changing something that led to a change in the Big-O complexity of certain algorithms would inherently hurt other implementations of Python when people would start to code specifically for that performance gain. For instance, having Jython be able to pull this trick off is, I believe, near impossible. So, in order to make sure changes like this are considered before applying them, Guido instated a new rule that "implementation features that affect not just the running speed but the O() rate for certain algorithms should be considered language features, because any implementation will be required to implement them in order to ensure code portability" between implementations of Python. In the end, though, this went in with a warning that the speed performance is not portable. It is not to be used in the stdlib ever. Contributing threads: - `Optimized string concatenation `__ - `PEP 0008 confusion - here it is, but don't use it? `__ ------------------------------------------------------------------------------- Bet you didn't think about string interning and how that affects .pyc, did you? ------------------------------------------------------------------------------- 1. minor language change Luckily Martin v. L?wis did. A patch was originally applied to not intern strings representing filenames. Problem is that every code object stores that string, so it increases .pyc files since it has to store multiple copies of that string instead of just one. Contributing threads: - `Re: [Python-checkins] python/dist/src/Python compile.c ... `__ ------------------------------------------------------------------------------ `PEP 324`_ (process - New POSIX process module) != process.py from ActiveState ------------------------------------------------------------------------------ 1. minor language change Guido asked if the APIs between the module proposed in `PEP 324`_ and the process.py module by Trent Mick of ActiveState were compatible. Turns out they are not. Then the discussion went into select(), broken pipes, and other stuff not relevant to the OP. =) .. _PEP 324: http://www.python.org/peps/pep-0324.html Contributing threads: - `PEP 324 (process module) `__ -------------------------------------------------------------------------------------------------------------------------------------------------- Getting it so string literals accept those nutty non-ASCII characters more easily (and getting files to be more upfront about their Unicode-ness) -------------------------------------------------------------------------------------------------------------------------------------------------- language evolution Fran?ois Pinard asked what people thought about two things related to Unicode. First, he thought having a __coding__ attribute for files that contained the encoding of the text would be nice. Martin v. L?wis said it would be possible. MA Lemburg added his vote of support. It has not been implemented to my knowledge yet, though. The second idea was to have a directive of some sort on a per-module basis so that all string literals could be considered in something other than ASCII. Once again Martin said it was doable with MA saying he liked the idea. But it was pointed out that you might as well just use the 'u' in front of strings now. This led to a slight discussion on good i18n practices. Both Martin and MA seemed to suggest that if you are going to be doing most of your work in a single encoding then just use that and declare everything Unicode. ANd if you are going to support a lot of different languages, use gettext and such. Martin also stated it is best to get text into Unicode ASAP and then only convert to the final encoding at the last moment. Contributing threads: - `Python in Unicode context `__ ---------------------------------------------------------------------- An exception is an exception, unless it doesn't inherit from Exception ---------------------------------------------------------------------- language evolution Coming up in a discussion on Unicode of all things, a discussion on moving exceptions over to new-style classes came up. Guido pointed out that moving over to new-style classes would seem to suddenly require that anything passed to 'raise' be a new-style class and that goes against the current acceptance. But in comes Paul Prescod with his argument that exceptions are inherently organized based on inheritance and thus requiring things being passed to 'raise' subclass Exception somewhere is not that big of a thing. Guido really liked that point. So if it goes this way and you don't like it blame Paul (who is a nice guy, so go easy on him =). And then along come Holger Krekel bringing up the point that using exceptions to do non-standard flow control is handy. But the question was asked as to why he couldn't still just subclass Exception? Was it that big of a deal? He admitted it wasn't and said when the name Raisable was suggested that would make it easier. Michael Hudson then came in and wrote up a hacked-up patch to turn exceptions into new-style classes. Most stuff seemed to still work. Python 3 was already going to switch to new-style classes for exceptions and string exceptions have already been deprecated. Now add to the mix the possible requirement that anything passed to 'raise' require a common base class. Contributing threads (note that the thread starts part way in a thread on Unicode and end before the end of the full thread): - `Python in Unicode context `__ ------------------------- Python 2.4a2 out the door ------------------------- language evolution Python 2.4a2 has been released. As usual, please download it and test it with your code along with the regression test suite. Contributing threads: - `trunk frozen for 2.4a2 `__ - `RELEASED Python 2.4, alpha 2 `__ ------------------------------------------------------- Amendment to how to compile with the free .NET compiler ------------------------------------------------------- 3. nuggets of info Nick Coghlan expanded upon my summary on how to compile with the free .NET compiler under Windows at http://www.python.org/dev/summary/2004-07-16_2004-07-31.html#how-to-get-python-to-compile-with-microsoft-s-free-compiler-that-should-just-come-with-the-os-standard . See his email at http://mail.python.org/pipermail/python-dev/2004-August/047215.html on the extra steps. Or just move to another OS. Contributing threads: - `python-dev Summary for 2004-07-16 through 2004-07-31 [draft] `__ -------------------------------------- Darned Solaris, g++, and _XOPEN_SOURCE -------------------------------------- 1. improvement stuff Skip Montanaro discovered that building under Solaris with g++ raised a warning about redefining _XOPEN_SOURCE. pyconfig.h defines it, but apparently so does g++ in order for Solaris' toolchain to expose some code that is only available if it is defined. Martin v. L?wis came up with a couple of suggestions on how to handle this. Skip ended up going with the idea of setting _XOPEN_SOURCE to the same value as it is defined by g++. Contributing threads: - `use of #pragma GCC system_header to suppress _XOPEN_SOURCE warnings `__ ------------------------------------------------------------------------------------- pre-PEP on a function for re that "captures matches for groups that match repeatedly" ------------------------------------------------------------------------------------- 1. minor language change Mike Coleman presented a pre-PEP on adding a function to re that would create a tree (in the form of a list) containing all group matches in a string. It got a little support, but the discussion quickly moved over to dreamings of a full-on scanner or parser package for the stdlib. But if the idea of the PEP works for you then speak up on c.l.py . Contributing threads: - `pre-PEP: Complete, Structured Regular Expression Group Matching `__ --------------------------------------- Making ourselves follow the PEP process --------------------------------------- 2. how python-dev works It was noticed early on that the PEP process had broken down for `PEP 318`__ (decorators). What should happen is a PEP gets written (and the author becomes its champion), there is a public discussion, the PEP is updated, that's repeated until it is deemed done, gets get BDFL pronouncement, if Guido okays it the code goes in. Unfortunately the part about updating the PEP didn't really happen. This led to two major things being stated and generally agreed upon. One is that PEPs should not be checked in if the PEP has not been updated. While a minor nit is not a reason to hold back code, not updating after a major public debate is not acceptable. This directly segued into the other point of a PEP needs to have a champion, period. Since the developers on python-dev do not have the time to keep PEPs updated it is up to the PEP champion to make sure it is kept current. If it isn't it is take to mean the champion no longer cares, which means python-dev no longer cares, which means the PEP gets rejected outright. This will also make sure that there is a focus to the design of what the PEP wants and does not lead to a design-by-committee problem. .. _PEP 318: http://www.python.org/peps/pep-0318.html Contributing threads: - `PEP 318, and the PEP process `__ ---------------------------------------------------------------- How to tell Windows from Linux without lifting up anyone's skirt ---------------------------------------------------------------- 3. nuggets of info The question came up on what the best way was to tell what platform you are running on. The answer came down to why you were cared. If it was whether or not you had a specific functionality (or lack thereof), then just test for the functionality. If you had other needs, though, using sys.platform seemed to be the most reliable way (at least for Windows since all of them use 'win32'). Contributing threads: - `Asyncore .set_reuse_addr() on Windows `__ ----------------------------------------- func_name can now have an identity crisis ----------------------------------------- 1. improvement stuff Thanks to the rampant decorator discussion, Skip Montanaro came up with the idea of wanting func_name to be writable. This would allow decorators to wrap a function with another function and yet reset func_name to its original value, thus not having the wrapper function become the prevailing name. Guido liked and Michael Hudson subsequently wrote up `patch #1004703`_. .. _patch #1004703: http://www.python.org/sf/1004703 Contributing threads: - `PEP 318 - writing to func_name attribute `__ ----------------------------------------- statistics.py ain't goin' into the stdlib ----------------------------------------- 1. improvement stuff Raymond Hettinger (at the last Python Bug Day) made the statement that he didn't think 'statistic's should be taken out of the sandbox in CVS and put into the stdlib. He felt that most of the important algorithms were available elsewhere (such as nsmallest and nlargest in the heapq module) and the remaining functions that were not found anywhere were not that critical. He did say, though, he would like to do a calculator module where those remaining functions could go. Contributing threads: - `status of statistics.py? `__ ------------------------------------ Making re.split care about emptiness ------------------------------------ 1. improvement stuff Mike Coleman wrote up a patch for re.split() that AM Kuchling presented to the list. It adds an argument to the function to allow an empty string to be put into the resulting list when a match occurs, even if the match consumes no characters. No decision on the exact semantics of the function, how to handle turning on the new functionality (some say an extra argument, some say adding another bit flag like traditional re arguments), or even if it will be accepted. Contributing threads: - `re.split on empty patterns `__ ---------------------------------------- Pickler's 'bin' argument on its last leg ---------------------------------------- 1. improvement stuff As per `PEP 307`_ (Extensions to the pickle protocol), the 'bin' argument is being upgraded from PendingDeprecationWarning to DeprecationWarning; so it's gone plaid and the only way to stop it is with some emergency change. Contributing threads: - `Pickler()'s bin argument `__ --------------------------------- Some modules are getting the boot --------------------------------- 1. improvement stuff TERMIOS, mpz, statcache, xreadlines and rotor are all being removed. mimify, MimeWriter, and whrandom will raise a DeprecationWarning. rfc822 and mimetools will not be raising DeprecationWarning as specified by `PEP 4`_ since some code in the stdlib still uses it. .. _PEP 4: http://www.python.org/peps/pep-0004.html Contributing threads: - `Removing stuff from 2.4 `__ ------------------------------------------------- Should decimal.Context.copy() be deep or shallow? ------------------------------------------------- 1. improvement stuff Raymond Hettinger brought up the question of whether decimal.Context.copy() should be a deep or shallow copy. While tradition dictates it be shallow based on name, it seems like the functionality should be deep. No one wants context to be shared between number naturally since numbers tend to be viewed as unique things. Staring in 2.4a3 it will be deep unless people come up with reasons to switch it to shallow. Contributing threads: - `decimal.Context.copy() shallow or deep? `__ ---------------------------------------------------------- The dark corners of Python allow you to skip return values ---------------------------------------------------------- 3. nuggets of info Christian Tismer discovered that you can actually stop a return statement from returning if you encase it in a 'finally' block and tweak it slightly (see the OP to get what I mean by this). Turns out this is a reliable feature of Python if you really want to use it. Contributing threads: - `capturing RETURN_VALUE `__ -------------------------------------- Is an int/long combined type worth it? -------------------------------------- language evolution Dmitry Vasiliev pointed out that `PEP 237`_ (Unifying Long Integers and Integers) mentioned that a a new type named 'integer' might be introduced that subclassed both int and long. The discussion waivered between whether it was at all needed, and if it was if it should be a true type or just a tuple containing both types for use with isinstance() . No conclusion was reached in the end. .. _PEP 237: http://www.python.org/peps/pep-0237.html Contributing threads: - `Unifying Long Integers and Integers: baseint `__ -------------------------------------- Should byte arrays be a built-in type? -------------------------------------- language evolution Through the discussion bout having a common type combining int and long a discussion on whether a byte array type should be introduced. The initial suggestion was for it to be practically synonymous with str since str itself stores everything as an array of 8-bit values. The use cases would be for streams and such that just want a stream of bytes with no care for any encoding. Syntactically, having a 'b' and 'B' cookie before literals was suggested. The issue with this, though, is that byte arrays should be mutable, which would make literals that can be mutated, and that is frowned upon by Guido. Semantically, aliasing bytes to the str type was suggested. That was not loved since that could create confusion. Returning an object from 'array' was suggested and seemed okay. In general it seemed this could be useful and could go in 2.5, but nothing for 2.4 . Contributing threads: - `Unifying Long Integers and Integers: baseint `__ ----------------------------- Thar the Windows stack blows! ----------------------------- 1. minor language change Running the new test_compiler test (if you run it with ``-uall`` for regrtest it will recompile the entire stdlib) was leading to an odd failure: the "process on Windows "just vanishes" without a trace, and without an error message of any kind, but with an exit code of 128". After a lot of work put in by a bunch of people (led by Tim Peters) the problem was tracked down to a blown C stack. Turned out that the stack-checking code was not being called frequently enough to pick up the error. The problem with this was that it was leading to odd errors that should have been MemoryError but were manifesting themselves as KeyError. This was because PyDict_GetItem() was failing and return NULL which is the same as signaling the key doesn't exist in the dict. Trick was trying to come up with a good way to deal with this. Adding more calls would be very expensive (reliable way of catching it was sticking a check in pymalloc code) and so that was ruled out. PyDict_GetItem() couldn't change its return value since that would break more code than could be imagined. So, in the end, the stack was increased to 2 MB on Windows. Contributing threads: - `Another test_compiler mystery `__ ------------------------------------------ Someone else falls for the dangling carrot ------------------------------------------ 2. how python-dev works Johannes Gijsbers now has checkin rights. May he close many bugs. Contributing threads: - `New Developer `__ ---------------------------------------------------------- Lying about being in __builtin__ is not always a bad thing ---------------------------------------------------------- 1. improvement stuff James Knight noticed that some objects (mostly C types such as iterators for the built-in types and such) claim in their __module__ attribute that they come from __builtin__ which is not technically correct since you can't access them from there. The desire to fix this came from allowing for proper introspection. The leading idea is to put these types in __builtin__ properly so that they are no longer lying about where they come from. Contributing threads: - `Classes that claim to be defined in __builtin__ but aren't `__ --------------------------------------------------------- Bringing order to the order of application for decorators --------------------------------------------------------- 1. minor language change Turns out that the order of application for decorators was implemented in the reverse order of what it was supposed to be in 2.4a2 . Luckily James Knight spotted this and let the list know. It has been fixed, though, in CVS and now follows `PEP 318`_ to apply in bottom-up order:: @first @second def blah(): pass is equivalent to:: blah = first(second(blah)) The arguments for this ordering beyond it just making more sense to Guido and others is that people will typically put decorators such as staticmethod and classmethod first. Going with the original order would have led to errors in most situations if people were not expecting to be receiving a class or static method to be passed to the other decorators. There was a short discussion on the order of instantiation for the decorators, but in the end the order chosen was the order listed; first would be first() instantiated followed by second(). Contributing threads: - `Decorator order implemented backwards? `__ ----------------------------------------------------------------------------------- PEP 292 (Simpler String Substitutions) getting gussied up for the Python 2.4a3 ball ----------------------------------------------------------------------------------- 1. minor language change `PEP 292`_'s implementation got fixed up. The names of the class names were changed to Template and SafeTemplate, the whole process was made lazy, and just cleaned up in general (solution is small, simple, and rather cool; all subclassable and works with Unicode to boot). But then questions over the merits of $-based string interpolation popped up. People wondered if going from ``%(blah)s`` to ``${blah}`` was worth it. The answer is "yes". And that is just worst case; when you just want ``$blah`` you get an even bigger win. The other question was over whether the string module should become a package. The original idea was to stick the original string module in as a submodule of the package and the Template code in another module. This would allow easy removal of the 'string' module code that has been deprecated for eons. Barry Warsaw (author of the PEP) asked Guido to make the call on this, but he hasn't had the time yet to get to this. .. _PEP 292: http://www.python.org./peps/pep-0292.html Contributing threads: - `PEP 292 `__ -------------------------------- Multi-line imports get clarified -------------------------------- 1. minor language change Multi-line imports will only accept parentheses around what is being explicitly imported and only if there is more than one item being imported. Trailing commas are also to be accepted. Contributing threads: - `Multi-line import implementation (first part) `__ - `Multi-line import implementation (second part) `__ - `Naming nit `__ ------------------------------------------------ For those of you who need Python to run on Win64 ------------------------------------------------ 3. nuggets of info Nick Bastin asked if anyone has gotten Python 2.3 to work under Win64. Martin v. L?wis said "yes" for Python 2.4, but not for 2.3 . He suggested to Nick that he run vsextcomp_ to generate the targets on the 2.4 VC 7 build files and then move that over to 2.3 . .. _vsextcomp: http://www.sf.net/projects/vsextcomp Contributing threads: - `2.3.4 on Win64? `__ ---------------------------------------------- Sometimes concealing the truth is a good thing ---------------------------------------------- 1. improvement stuff Nick Coghlan discovered that some of the function in the 'commands' module did actually work under Windows and he wanted to make sure it was okay to fix another module to work under Windows and to document the fact. But the whole idea was shot down by both Tim Peters and Guido in order to keep the module simple. Keeping the whole thing UNIX-only is much easier than having an API that is only partly compatible with Windows (and with only certain versions of Windows at that). Guido also said that the module would not even be accepted today since it doesn't add enough functionality. Contributing threads: - `'commands' module on Win32 `__ ---------------------------------------------- atexit module, good; sys.exitfunc, not so good ---------------------------------------------- 1. improvement stuff Raymond Hettinger pointed out that the docs for the atexit module state that sys.exitfunc was to be deprecated. Well, atexit has been in the stdlib since 2.0 so the deprecation is long overdue. Looks like it will get its deprecation finally. Contributing threads: - `Deprecate sys.exitfunc? `__ ---------------------------------------------------------------------------------------------- My Personal Hell: decorators and the people who suggest new syntax for them... next, on Oprah ---------------------------------------------------------------------------------------------- language evolution What led to a record-setting flurry of emails to python-dev was set up when Guido gave the green light to checking in code to implement decorators using the '@' syntax (now known at the pie syntax thanks to its checkin coming very shortly after the OSCON Pie-thon competition and '@' sort of looking like a pie). People didn't like it. People were screaming that there had to be a better syntax than just settling for the least offensive one. Others started to question whether decorators were really needed. Others wanted to extend them even more and what their role truly was. Either way this was all coming a little late. But then Guido decided to make my life difficult by saying that if the community could come up with an agreed-upon alternative syntax to propose to him he would consider ripping out the '@' syntax; decorators have always been experimental and '@' was checked in so people had *something* to play with. This meant everyone and their mother started to propose both new and old syntaxes for decorators. This led to a record amount of email on python-dev (at least compared to what we have archives for; back to April 1999). In order of my own personal sanity I am not going to cover any specific syntax. There are just too many and I actually like the '@' syntax (just *try* using it; it grows on you quickly). Instead this summary will cover any important design considerations for decorators (it should all be in the PEP but so much happened there is no way it all got in there) along with any other important considerations about them. I will discuss anything specific to the '@' syntax since it is the currently winning syntax. With that said, `PEP 318`_ and the PythonDecorators_ wiki page are the de-facto place for info on this whole situation. I am probably going to be more biased than normal in this summary just out of time restraints to get this done by not over-analyzing people's suggestions and just assuming what I summarize here is not covered elsewhere; checking if it is would take too long. First off, going with a syntax just so you could add backward-compatible support was not considered worth it. It's a new feature so there is no need to hobble it at the get-go just so people who don't want to upgrade can still use it. Plus the implementations to make this even possible were playing some tricks with the interpreter and not considered a good thing. Besides you can, always will be able to, still use the way to do it in 2.2 and beyond. Another design point that needed to be taken into consideration was ambiguity. There could not be a question about what is a decorator and what isn't (list-before-def is an example of a syntax that can be ambiguous). Syntactic support that allowed for possible future compiler optimizations was also considered important. The on-going hope is to eventually get a JIT compiler for Python and if the decorators are not placed before the def somehow you make it much harder to optimize since you learn about the decorators after the fact of the function starting to be defined. An argument against '@' was that people thought it got rather unwieldy quickly (and I am sure the new decision that each decorator must be on its own line is not making these objectors happy). But one thing that must be remembered is that chances are most functions will not need more than one decorator if the even need one. Looking through the stdlib sans the test suite you will notice there are not a lot of uses for any of the built-in decorators. Yes, usage will probably grow with syntactic support for decorators, but not exponentially in terms of applying multiple decorators to a single function. Allowing decorators to take arguments is a requirement. Not being able to pass arguments to a decorator at instantiation time would severely cripple their usefulness. Being prefix instead of postfix was considered important. Knowing about what is coming up was something Guido thought is better than finding out about some special tweak to the use of a function based on a decorator listed after the definition line. And in the method body had been completely ruled out. In terms of implementation details, decorators can only be functions directly (and not lambda). This means no ``@a or b`` tricks and such. It also means ``@foo().bar()`` is not allowed. This is all gut feeling from Guido for why the restriction should be there. It could change in the future since it is easier to loosen restrictions that make something more strict. A problem with '@' in terms of portability is that Leo_ and IPython_ both use '@' for special syntax. The authors of both tools seem willing to change their tools (not necessarily overly joyous, though =) . Guido wanted a syntax "that ... [is] easy to remember once explained". It did not need to be miraculously obvious what it did just by looking at the first time. Some people didn't like '@' because they keep wanting to say "at" when they saw it. To that, I give you Tim Peters' fix for this: "Pick the keyword you'd like. Then whenever you see "@", pronounce that word instead ". A single decorator that applied to multiple code blocks is completely out. Guido gave a couple of reasons why, but one that he had mentioned multiple times is indenting the code block again just for the decorator is out. Some people suggested that things be kept simple by just throwing out the entire idea of syntactic support. Could happen if everyone agreed to that, but I wouldn't count on it. A supposed knock against '@' was that it wasn't Pythonic. Well, it is Pythonic if Guido says it is so not really a valid argument. OK, tirade time. If you don't want to listen to me get up on my soapbox, then just skip the rest of this... "In the old days, Guido would Pronounce, and we'd all bite our tongues (although not necessarily each his own). The less time Guido can make for Python, the more important becomes graceful capitulation." Tim said this and it makes me wish for the old days. People had *months* to comment on decorators and no one spoke up until something went into the language. Procrastination is not a virtue when it comes to major language evolution discussions. What was worse was when the emails started repeating themselves (which was pretty much from the get-go when this exploded). Seemed like people decided to start talking without doing some research. Granted the PEP was outdated and the wiki page was not up yet, but this stuff was covered in the Summaries before and you could have just Googled for the previous threads. Personally, if I was Guido, I would have said that the community had their chance to speak up and they just didn't take it. But Guido is a nicer guy than that so people are getting a second chance with this. Personally this came off a great case of the tyranny of the majority in my eyes. There is a reason why Python is a dictatorship. At this point people should be hashing out which syntax alternative they want to present to Guido on comp.lang.python_. No more talking on python-dev, no more syntax proposals. The community should set a goal date (Sept. 1 seems good) and just choose a bloody alternative. Then when Guido makes his choice people accept it or just go to another language. No one had better voice their disappoint once Guido chooses his syntax or I will personally come beat you over with a stick for being a whiner. OK, that's out of my system. I feel better now. .. _PEP 318: http://www.python.org./peps/pep-0318.html .. _PythonDecorators: http://www.python.org/cgi-bin/moinmoin/PythonDecorators .. _Leo: http://leo.sourceforge.net/ .. _IPython: http://ipython.scipy.org/ Contributing threads (sans emails that were warnocked): - `2.4a2, and @decorators `__ - `Plea for simpler decorator syntax, in addition to pie-shaped syntax `__ - `Call for defense of @decorators `__ - `@decorators, the PEP and the "options" out there? `__ - `About pep 318--Intro `__ - `Questions about '@' in pep 318 `__ - `A usability argument for list-after-def `__ - `The impact of '@' on Leo `__ - `Similar syntax `__ - `def fn (args) [dec,dec]: `__ - `pep 318, Decorators for Functions, Methods and Classes `__ - `Density of pie-decorator syntax `__ - `elements of decorator syntax suggestions `__ - `318: Annotation background `__ - `IPython, @, and option E from the wiki `__ - `Decorators: vertical bar syntax `__ - `Suggesting '.' decorators (PEP318) `__ - `Attribute strings? `__ - `request: add keywords for static and class methods only `__ - `@decorator syntax is sugar, but for what exactly? `__ - `Semantics of decorators? `__ - `A decorator syntax not yet mentioned (I think!) `__ - `Another approach to decorators. `__ - `Decorators after 'def' should be reconsidered `__ - `def ... decorate `__ - `Decorator syntax J2 (decorate..def) `__ - `Decorators and docstrings `__ - `block-based decorators and other block issues `__ - `More concerns about decorators `__ From theller at python.net Thu Aug 19 16:18:54 2004 From: theller at python.net (Thomas Heller) Date: Thu Aug 19 16:18:59 2004 Subject: [Python-Dev] refcounts.dat Message-ID: Is the Doc/api/refcounts.dat file no longer maintained? The last commit was on 2002/10/04. How does the C api manual the refcount info? Thanks, Thomas From jim at zope.com Thu Aug 19 16:22:13 2004 From: jim at zope.com (Jim Fulton) Date: Thu Aug 19 16:22:16 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408190558.i7J5wPP19708@guido.python.org> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> <200408190558.i7J5wPP19708@guido.python.org> Message-ID: <4124B795.8000809@zope.com> Guido van Rossum wrote: >>From the posts I've seen here, the only alternatives that have >>considerable popular support are ones that you've already >>rejected. So I suspect nobody really feels it's worth trying. > > > Well, do people generally buy those rejections, or is their consensus > that I'm mistaken? As I've noted before (sorry for repeating myself), I think the controversy around this issue is evidence of a problem. If you know that the decorator idea is the right thing to do and you are comfortable with a syntax, then I think you should go forward, and we'll all (speaking for myself :) follow along. OTOH, if you are just tired of talking about it and want to move on, and don't know for sure that Python needs this, then I encourage you to move on without decorators for now. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From skip at pobox.com Thu Aug 19 16:35:59 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 19 16:36:23 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com> References: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com> Message-ID: <16676.47823.382278.737098@montanaro.dyndns.org> Kurt> Patch / Bug Summary Kurt> ___________________ Kurt> Patches : 264 open (-20) / 2556 closed (+42) / 2820 total (+22) Kurt> Bugs : 745 open ( +1) / 4384 closed (+23) / 5129 total (+24) Kurt> RFE : 149 open ( +0) / 130 closed ( +0) / 279 total ( +0) Kurt> New / Reopened Patches Kurt> ______________________ ... Kurt (et al), Any thoughts about posting this to c.l.py as well as python-dev? Maybe it would encourage some more participation... Skip From nas at arctrix.com Thu Aug 19 16:40:19 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Thu Aug 19 16:40:23 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <41247122.7030000@egenix.com> References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> <20040818223629.GA17550@mems-exchange.org> <1092868762.22400.97.camel@geddy.wooz.org> <41247122.7030000@egenix.com> Message-ID: <20040819144019.GA19851@mems-exchange.org> On Thu, Aug 19, 2004 at 11:21:38AM +0200, M.-A. Lemburg wrote: > I assume the picture will look more like this: > > basesequence > mutable > basebytes > bytes > array > cStringIO > mmap > immutable > unicode > tuple > basenumber > integer > float > decimal > complex Inheritance is overrated. :-) For example, I think mutability may be an orthogonal property. Also, I don't think cStringIO is sequence-like, it's more like a file. Neil From mal at egenix.com Thu Aug 19 16:49:27 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 16:49:31 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <20040819144019.GA19851@mems-exchange.org> References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> <20040818223629.GA17550@mems-exchange.org> <1092868762.22400.97.camel@geddy.wooz.org> <41247122.7030000@egenix.com> <20040819144019.GA19851@mems-exchange.org> Message-ID: <4124BDF7.8050508@egenix.com> Neil Schemenauer wrote: > On Thu, Aug 19, 2004 at 11:21:38AM +0200, M.-A. Lemburg wrote: > >>I assume the picture will look more like this: >> >>basesequence >> mutable >> basebytes >> bytes >> array >> cStringIO >> mmap >> immutable >> unicode >> tuple >>basenumber >> integer >> float >> decimal >> complex > > > Inheritance is overrated. :-) For example, I think mutability may > be an orthogonal property. Also, I don't think cStringIO is > sequence-like, it's more like a file. This was just a rough sketch. I agree that inheritance is overrated, but in absence of well-defined interfaces, it often helps :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mwh at python.net Thu Aug 19 16:52:31 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 19 16:52:33 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> (Jack Jansen's message of "Thu, 19 Aug 2004 00:16:33 +0200") References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> Message-ID: <2macwr9o40.fsf@starship.python.net> Jack Jansen writes: > I may have missed a crucial bit of the discussion, having been away, > so if this is completely besides the point let me know. But my feeling > is that the crucial bit is the type inheritance graph of all the byte > and string types. And I wonder whether the following graph would help > us solve most problems (aside from introducing one new one, that may > be a showstopper): > > genericbytes > mutablebytes > bytes > genericstring > string > unicode Um, I was thinking the hierarchy would look like this: object unicode bytes Cheers, mwh -- I saw `cout' being shifted "Hello world" times to the left and stopped right there. -- Steve Gonedes From mwh at python.net Thu Aug 19 17:14:35 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 19 17:14:37 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com> (Timothy C. Delaney's message of "Thu, 19 Aug 2004 14:37:45 +1000") References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com> Message-ID: <2mwtzv88is.fsf@starship.python.net> "Delaney, Timothy C (Timothy)" writes: > Guido van Rossum wrote: > >> Is anybody seriously trying to come up with a single alternative >> decorator proposal that most folks "out there" can support, to be >> presented to me (with implementation, please!) in time for 2.4b1? > > I'm afraid that absolutely zero consensus is emerging :( I've withdrawn > myself entirely from the discussions as they've proven completely > useless. Me too. Cheers, mwh -- The PROPER way to handle HTML postings is to cancel the article, then hire a hitman to kill the poster, his wife and kids, and fuck his dog and smash his computer into little bits. Anything more is just extremism. -- Paul Tomblin, asr From barry at python.org Thu Aug 19 17:15:47 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 19 17:15:48 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <412480FF.2030203@zope.com> References: <412480FF.2030203@zope.com> Message-ID: <1092928547.8719.70.camel@geddy.wooz.org> On Thu, 2004-08-19 at 06:29, Jim Fulton wrote: > IMO, yes, although I'd happily accept both syntaxes. I can live with > @foo to server the people who want it, but it bugs me that I have to > write: > > @clasmethod > def foo .... > > rather than the, to me, much more obvious and readable: > > def classmethod foo ... This one doesn't really bother me, in fact, I've actually come to like the decorator in the former position, rather than the latter. I've rewritten my decorator code to use almost all of the proposed syntaxes, just to see how they look or what the annoyance factor is. I'm pretty comfortable with pie-decorators before def syntax. I think they're a good compromise, and are quite readable /and/ writable. I don't like pie-decorator-in-first-line syntax as much for the same reason I don't like the former choice above: both make the decorator less prominent in the definition, which isn't something I think you want. The latter choice above is worse because it also hides the function name. I think it's important to understand that we're now down to aesthetic choices about elegance, readability, ease of use, etc. For those reasons, we'll never agree because everyone has different tastes. Python represents Guido's aesthetic in language design, and we all agree that he gets most such decisions right, so I trust him on this one too -- and I was /not/ a pie-decorator fan originally. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/53c8d5e4/attachment.pgp From barry at python.org Thu Aug 19 17:23:22 2004 From: barry at python.org (Barry Warsaw) Date: Thu Aug 19 17:22:56 2004 Subject: [Python-Dev] Byte string class hierarchy In-Reply-To: <2macwr9o40.fsf@starship.python.net> References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> <2macwr9o40.fsf@starship.python.net> Message-ID: <1092929002.8719.72.camel@geddy.wooz.org> On Thu, 2004-08-19 at 10:52, Michael Hudson wrote: > Um, I was thinking the hierarchy would look like this: > > object > unicode > bytes Flat is better than nested. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/fb1c32eb/attachment.pgp From amk at amk.ca Thu Aug 19 17:38:12 2004 From: amk at amk.ca (A.M. Kuchling) Date: Thu Aug 19 17:39:06 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <2mwtzv88is.fsf@starship.python.net> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com> <2mwtzv88is.fsf@starship.python.net> Message-ID: <20040819153812.GA21213@rogue.amk.ca> On Thu, Aug 19, 2004 at 04:14:35PM +0100, Michael Hudson wrote: > "Delaney, Timothy C (Timothy)" writes: > > I'm afraid that absolutely zero consensus is emerging :( I've withdrawn > > myself entirely from the discussions as they've proven completely > > useless. > Me too. Me three. (I don't think I'm going to become a heavy user of decorators -- little of my current code could use them -- but for what it's worth I'm happy with either @decorator or list-after-def forms. I don't hate the '@' character, but neither do I find list-after-def very confusing, especially if you format the code clearly.) --amk From aahz at pythoncraft.com Thu Aug 19 17:41:32 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 19 17:41:36 2004 Subject: [Python-Dev] Weird errors when adding a new built-in type In-Reply-To: <41247180.5070304@auckland.ac.nz> References: <41247180.5070304@auckland.ac.nz> Message-ID: <20040819154132.GC21234@panix.com> On Thu, Aug 19, 2004, Geoffrey Biggs wrote: > > I'm trying to add a new built-in number data type to Python with its own > syntax, so I'm working directly with the interpreter rather than > creating my own extension module (side note: I've appended something > extra to the version thing in the Makefile - I doubt this is relevant to > the problem but it's probably best you have all the info). The complex > data type is similar to what I'm trying to do so I've been following > that as an example. I've successfully extended the tokenizer and the > parsenumber() function in compile.c to do what I want and written the > data type in my two new files Objects/mynewobject.c and > Include/mynewobject.h. I've included mynewobject.h in Python.h and added > the two files to the Makefile. > However, when I tried to run the code by typing in the syntax to produce > my data type, Python suffers a segmentation fault. Unless you're planning to attempt pushing this into the core at some point in the future, you're best off asking on comp.lang.python; although some of the core developers no longer have time for c.l.py, they're also not likely to answer your questions here, either. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From walter at livinglogic.de Thu Aug 19 17:45:26 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 19 17:45:34 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123C16D.4060505@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de> <4123C16D.4060505@v.loewis.de> Message-ID: <4124CB16.90705@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> They will not, because StreamReader.decode() already is a feed >> style API (but with state amnesia). >> >> Any stream decoder that I can think of can be (and most are) >> implemented by overwriting decode(). > > I consider that an unfortunate implementation artefact. You > either use the stateless encode/decode that you get from > codecs.get(encoder/decoder) or you use the file API on > the streams. You never ever use encode/decode on streams. That is exactly the problem with the current API. StreamReader mixes two concepts: 1) The stateful API, which allows decoding a byte input in chunk and the state of the decoder is kept between calls. 2) A file API where the chunks to be decoded are read from a byte stream. > I would have preferred if the default .write implementation > would have called self._internal_encode, and the Writer > would *contain* a Codec, rather than inheriting from Codec. This would separate the two concepts from above. > Alas, for (I guess) simplicity, a more direct (and more > confusing) approach was taken. > >> 1) Having feed() as part of the StreamReader API: >> --- >> s = u"???".encode("utf-8") >> r = codecs.getreader("utf-8")() >> for c in s: >> print r.feed(c) > > > Isn't that a totally unrelated issue? Aren't we talking about > short reads on sockets etc? We're talking about two problems: 1) The current implementation does not really support the stateful API, because trailing incomplete byte sequences lead to errors. 2) The current file API is not really convenient for decoding when the input is not read for a stream. > I would very much prefer to solve one problem at a time. Bye, Walter D?rwald From anthony at interlink.com.au Thu Aug 19 17:50:49 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Aug 19 17:51:01 2004 Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords Message-ID: <4124CC59.80007@interlink.com.au> After consultations with the relevant folks, I want to get 2.4a3 out the door on September 2nd. I'll be following up to this email with a list of things that are still pending. This _should_ be the final alpha release. After that, the first beta. I'm wondering if there's a way I can mark bugs and patches that should be in before b1. It appears from SF's tracker that the only way I'm going to be able to do that is by putting keywords in the summary. I'm considering doing this. Rather than rip off the entirety of the mozilla's insane list of keywords, here's the ones I care about: - should be in before b1 (changes functionality) - should be in before final - regression since 2.3 Does anyone else have any others that they think are useful? I'd probably go with something like [beta], [final], [regr] in the summary. The other alternative is to make a bunch of new groups - this doesn't excite me at all, because a) I can't do it, and b) we can't get rid of them once they're in. -- Anthony Baxter It's never too late to have a happy childhood. From aahz at pythoncraft.com Thu Aug 19 17:55:07 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 19 17:55:16 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408191158.12738.gmccaughan@synaptics-uk.com> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> <200408190558.i7J5wPP19708@guido.python.org> <200408191158.12738.gmccaughan@synaptics-uk.com> Message-ID: <20040819155507.GE21234@panix.com> On Thu, Aug 19, 2004, Gareth McCaughan wrote: > On Thursday 2004-08-19 06:58, Guido van Rossum wrote: >> >>> From the posts I've seen here, the only alternatives that have >>> considerable popular support are ones that you've already >>> rejected. So I suspect nobody really feels it's worth trying. >> >> Well, do people generally buy those rejections, or is their consensus >> that I'm mistaken? > > I'm one of the people who doesn't really feel it's worth trying. > *My* consensus :-) is that you're mistaken. I think that > > - the @-syntax is horrible and unpythonic as all-get-out, > but usable; > > - your arguments against the []-before-colon form are utterly > bogus; Aside from changing "utterly bogus" to "excessive", I'm entirely +1 on everything Gareth wrote in his post. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From mal at egenix.com Thu Aug 19 18:06:46 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 18:06:49 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124CB16.90705@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de> <4123C16D.4060505@v.loewis.de> <4124CB16.90705@livinglogic.de> Message-ID: <4124D016.1050600@egenix.com> Walter D?rwald wrote: > Martin v. L?wis wrote: > >> Walter D?rwald wrote: >> >>> They will not, because StreamReader.decode() already is a feed >>> style API (but with state amnesia). >>> >>> Any stream decoder that I can think of can be (and most are) >>> implemented by overwriting decode(). >> >> >> I consider that an unfortunate implementation artefact. You >> either use the stateless encode/decode that you get from >> codecs.get(encoder/decoder) or you use the file API on >> the streams. You never ever use encode/decode on streams. > > > That is exactly the problem with the current API. > StreamReader mixes two concepts: > > 1) The stateful API, which allows decoding a byte input > in chunk and the state of the decoder is kept between > calls. > 2) A file API where the chunks to be decoded are read > from a byte stream. > >> I would have preferred if the default .write implementation >> would have called self._internal_encode, and the Writer >> would *contain* a Codec, rather than inheriting from Codec. > > This would separate the two concepts from above. Note that StreamCodec only inherits from Codec for convenience reasons (you can define a StreamCodec using the stateless .encode() and .decode() methods you get from Codec) and for logical reasons: a StreamCodec happens to be a Codec as well, so isinstance(obj, Codec) should be true for a StreamCodec as well. There's nothing preventing you from overriding .encode() and .decode() in a StreamReader or adding new methods that implement a different approach to encode and decode. Users should always use the file API of StreamReader et al., not the .encode() and .decode() methods. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mzarate at uoguelph.ca Thu Aug 19 18:13:11 2004 From: mzarate at uoguelph.ca (Martin Zarate) Date: Thu Aug 19 18:13:16 2004 Subject: [Python-Dev] Considering decorator syntax on an individual feature basis Message-ID: <1092931991.4124d1976b8ba@webmail.uoguelph.ca> Issues with decorator syntax. I think we can all agree that the decorator argument is now wholly a matter of aesthetic taste, with the brief exception of namespace-based stuff that got argued when I suggested to use a block instead. The problem is that we keep jumping from the beginning of the argument to the end - we keep saying "what syntax do we want" rather than "which syntactic features do we want". The features in question, which should be decided individually rather than as a whole: - decorator before/within/after def. Guido is obviously a fan of the "before" approach, and I agree. The "after" to me is just ugly, and the "within" (on the same line, I mean) is impractical for some of the lengthier uses people describe for decorators, although I think that it is the most "pythonic" in general appearance. - special character/keyword/no new symbol. Guido prefers the @ character (I dislike it - it looks meaningless, and it breaks existing tools - I find it unpythonic). The "no new symbol" approach (such as the "list before def", or decorator-after-def-before-name) is often difficult to parse and, while elegant looking, can make it visually difficult to distinguish the decorators (and hard to grep). As such, I prefer the new keyword statement approach - it is more verbose, but I like it. - decorator on one/many line(s) The approach described by the paired-block syntax, and the @ syntax both assume that users want to include only one decorator per line. Personally, I am inclined to follow something like the new import statment wherein the user can have a comma-delimited list of decorators and can span the list across lines if they so choose. - one symbol per decorator vs. one symbol for all decorators The @ approach has one symbol for each decorator. The block approach has one symbol that encompasses all decorators. If a single symbol is used, then one must consider whether that symbol occurs on the same line as the decorators or not. If not, then the symbol must be brief to avoid letting its repetition be clumsy. Personally, I prefer a single use of the symbol, but its really personal taste. - indent def/not indent def If the decorator is before the def, this must be considered. Indenting the def is very Pythonic, as it shows the heirarchial relationship visually... but it also seems an unnecessary indendation in this case, as there is no other case where an indentation occurs for the benefit of a single statement. I think if each of these issues could be resolved and agreed to individually, then a consensis would be far more apparent as to the ideal solution, or at least some of the myriad solutions could be filtered out. I think the Python community has mostly settled on a "before def" approach, and I think that most agree that the @ character is very ugly, but some are coming around (mostly out of exhaustion though, it sounds). So if we can agree on the latter considerations. Personally, my favourite would be a single-line but continuable statement (like the "import" statement) which causes the assignment that occurs on the following line be wrapped by its mentioned objects. Use a nice keyword like deco or remake (or even just "be"). Possibly indent the def one level to show the relationship of the import statement and to allow for def assignments to look good when used with the statement. The fact is that there is no Pythonic precedence here - we're talking about a statement that holds one other statement and has N parameters. Whenever Python has had to break into new ground with statements, its been a bit of a mess (take lambda for example). Oh, and in defense of "be" as a statement - how's this? be staticmethod, returning(int), taking(int, bool, bool) def foo: pass or be ( staticmethod, synchronised): foo = lambda: none but that's just my tastes. Oh well, I'm violating my own rule, jumping ahead with a full syntax when we can't even find consensus on the parts of it. From goodger at python.org Thu Aug 19 18:44:20 2004 From: goodger at python.org (David Goodger) Date: Thu Aug 19 18:44:25 2004 Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords In-Reply-To: <4124CC59.80007@interlink.com.au> References: <4124CC59.80007@interlink.com.au> Message-ID: <4124D8E4.8030705@python.org> [Anthony Baxter] > - regression since 2.3 What does that one mean? -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/965fcb4b/signature.pgp From neal at metaslash.com Thu Aug 19 18:47:34 2004 From: neal at metaslash.com (Neal Norwitz) Date: Thu Aug 19 18:47:52 2004 Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords In-Reply-To: <4124CC59.80007@interlink.com.au> References: <4124CC59.80007@interlink.com.au> Message-ID: <20040819164734.GY19366@epoch.metaslash.com> On Fri, Aug 20, 2004 at 01:50:49AM +1000, Anthony Baxter wrote: > > I'm wondering if there's a way I can mark bugs > and patches that should be in before b1. > > Does anyone else have any others that they think are > useful? I'd probably go with something like [beta], > [final], [regr] in the summary. The other alternative > is to make a bunch of new groups - this doesn't excite > me at all, because a) I can't do it, and b) we can't > get rid of them once they're in. There are two more possible hacks: 1) use priorities. This probably isn't good enough. 2) create SF users that aren't real people, but used for assigning levels. E.g., py-beta, py-final, py-regr. You would still have to get an admin to add these, but you could create the accounts yourself and they aren't necessarily permanent. I don't know how well this would work. It's definitely a hack. :-) Neal From tim.peters at gmail.com Thu Aug 19 18:48:08 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 19 18:48:11 2004 Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords In-Reply-To: <4124CC59.80007@interlink.com.au> References: <4124CC59.80007@interlink.com.au> Message-ID: <1f7befae040819094825bfacf0@mail.gmail.com> [Anthony Baxter] > After consultations with the relevant folks, I want to > get 2.4a3 out the door on September 2nd. +1 on both parts. In particular, changes since a2 have been too rapid and broad to be comfortable with calling the next one a beta. > ... > I'm wondering if there's a way I can mark bugs > and patches that should be in before b1. It appears > from SF's tracker that the only way I'm going to be > able to do that is by putting keywords in the summary. > I'm considering doing this. Changing priorities has worked well for this in the past. SF even colors them differently I'm going to add one for you: - must be resolved for a3 Priority 9. It gets addressed or a3 doesn't ship. > - should be in before b1 (changes functionality) Priority 8. > - should be in before final Priority 7. > - regression since 2.3 Depends on whether the specific regression must be resolved for a3, b1, or final -- or is going to be declared "a feature" and 2.3's behavior "a bug". > Does anyone else have any others that they think are > useful? I'd probably go with something like [beta], > [final], [regr] in the summary. Over time, those tags become misleading, because they're so easy to ignore. People won't allow high priorities to remain high, though. It irritates them. That's good . > The other alternative is to make a bunch of new groups - this > doesn't excite me at all, because a) I can't do it, and b) we can't > get rid of them once they're in. Yup, new groups suck. From walter at livinglogic.de Thu Aug 19 18:49:44 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 19 18:49:50 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4123D0C2.4080505@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> Message-ID: <4124DA28.50106@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> But then a file that contains the two bytes 0x61, 0xc3 >> will never generate an error when read via an UTF-8 reader. >> The trailing 0xc3 will just be ignored. >> >> Another option we have would be to add a final() method >> to the StreamReader, that checks if all bytes have been >> consumed. > > Alternatively, we could add a .buffer() method that returns > any data that are still pending (either a Unicode string or > a byte string). Both approaches have one problem: Error handling won't work for them. If the error handling is "replace", the decoder should return U+FFFD for the final trailing incomplete sequence in read(). This won't happen when read() never reads those bytes from the input stream. >> Maybe this should be done by StreamReader.close()? > > No. There is nothing wrong with only reading a part of a file. Yes, but if read() is called without arguments then everything from the input stream should be read and used. >> Now >> inShift counts the number of characters (and the shortcut >> for a "+-" sequence appearing together has been removed. > > Ok. I didn't actually check the correctness of the individual > methods. > > OTOH, I think time spent on UTF-7 is wasted, anyway. ;) But it's a good example of how complicated state management can get. >> Would a version of the patch without a final argument but >> with a feed() method be accepted? > > I don't see the need for a feed method. .read() should just > block until data are available, and that's it. There are situations where this can never work: Take a look at xml.sax.xmlreader.IncrementalParser. This interface has a feed() method which the user can call multiple times to pass byte string chunks to the XML parser. These chunks have to be decoded by the parser. Now if the parser wants to use Python's StreamReader it has to wrap the bytes passed to the feed() method into a stream interface. This looks something like the Queue class from the patch: class Queue(object): def __init__(self): self._buffer = "" def write(self, chars): self._buffer += chars def read(self, size=-1): if size<0: s = self._buffer self._buffer = "" return s else: s = self._buffer[:size] self._buffer = self._buffer[size:] return s The parser creates such an object and passes it to the StreamReader constructor. Now when feed() is called for the XML parser the parser calls queue.write(bytes) to put the bytes into the queue. Now the parser can call read() on the StreamReader (which in turn will read from the queue (on the other end)) to get decoded data. But this will fail when StreamReader.read() block until more data is available. This will never happen, because the data will be put in the queue explicitely by calls to the feed() method of the XML parser. Or take a look at sio.DecodingInputFilter. This should be an alternative implementation of reading a stream an decoding bytes to unicode. But the current implementation is broken because it uses the stateless API. But once we switch to the stateful API DecodingInputFilter becomes useless: DecodingInputFilter.read() just looks like this: def read(): return self.stream.read() (with stream being the stateful stream reader from codecs.getreader()), because DecodingInputFilter is forced to use the stream API of StreamReader. >> I'm imagining implementing an XML parser that uses Python's >> unicode machinery and supports the >> xml.sax.xmlreader.IncrementalParser interface. > > I think this is out of scope of this patch. The incremental > parser could implement a regular .read on a StringIO file > that also supports .feed. This adds to much infrastructure, when the alternative implementation is trivial. Take a look at the first version of the patch. Implementing a feed() method just mean factoring the lines: data = self.bytebuffer + newdata object, decodedbytes = self.decode(data, self.errors) self.bytebuffer = data[decodedbytes:] into a separate method named feed(): def feed(newdata): data = self.bytebuffer + newdata object, decodedbytes = self.decode(data, self.errors) self.bytebuffer = data[decodedbytes:] return object So the feed functionality does already exist. It's just not in a usable form. A using StringIO wouldn't work because we need both a read and a write position. >> Without the feed method(), we need the following: >> >> 1) A StreamQueue class that > > Why is that? I thought we are talking about "Decoding > incomplete unicode"? Well, I had to choose a subject. ;) Bye, Walter D?rwald From mwh at python.net Thu Aug 19 18:50:22 2004 From: mwh at python.net (Michael Hudson) Date: Thu Aug 19 18:50:39 2004 Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords In-Reply-To: <4124D8E4.8030705@python.org> (David Goodger's message of "Thu, 19 Aug 2004 12:44:20 -0400") References: <4124CC59.80007@interlink.com.au> <4124D8E4.8030705@python.org> Message-ID: <2mk6vv8435.fsf@starship.python.net> David Goodger writes: > [Anthony Baxter] >> - regression since 2.3 > > What does that one mean? I guess thing's like Tim's recent asyncore pain. Cheers, mwh -- The source passes lint without any complaint (if invoked with >/dev/null). -- Daniel Fischer, http://www.ioccc.org/1998/df.hint From pm_mon at yahoo.com Thu Aug 19 13:45:09 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Thu Aug 19 19:00:55 2004 Subject: [Python-Dev] Re: PEP 318: Can't we all just get along? In-Reply-To: <412480FF.2030203@zope.com> References: <412480FF.2030203@zope.com> Message-ID: Jim Fulton wrote: > > IMO, yes, although I'd happily accept both syntaxes. I can live with > @foo to server the people who want it, but it bugs me that I have to > write: > > @clasmethod > def foo .... > > rather than the, to me, much more obvious and readable: > > def classmethod foo ... > Probably the best-loved features of Python are the shortcuts. The fact that you don't have to explicitly declare very much and that the language supports clear, shorthand syntax for frequently used operations... x = [1,2,3] y = r[2:9] def __foo(): That's the good stuff of Python. I believe that identifying staticmethods and classmethods is an operation that is (or will be) frequently used as compared to the other decorators. So it would be great if there was an obvious, shorthand way for doing that as well. Even better (IMO) would be a solution that didn't really add any new keywords to the language, but instead simply formalized widely-used coding conventions, as was done to denote code block boundaries. The use of indentation to identify code blocks is a wonderful example of Python shorthand. It works (some even call it 'genius') because it leverages a coding convention shared by a majority of programmers. A week or so ago, Stefan Eischet on comp.lang.python suggested that the type (static|class|instance) of a method could be inferred through examination of its first parameter. I agree with his assessment. The vast majority of instance methods I've seen all use 'self' as the first parameter. Likewise, most class methods use 'cls' or 'klass' as their first parameter. If we exploit these conventions, we end up with a simple, clear, obvious mechanism for denoting (this aspect of) a method's type. class Foo(Object): def m1(self, a, b): # this is an instance method of Foo pass def m2(cls, a, b): # this is a class method of Foo pass def m3(a, b): # this is a static method of Foo pass A special Object (capital 'O') class could work this magic so that old code didn't break. I know that this is odd. But then so are most of the great things about Python. Paul From marktrussell at btopenworld.com Thu Aug 19 19:15:26 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Thu Aug 19 19:15:29 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <1092928547.8719.70.camel@geddy.wooz.org> References: <412480FF.2030203@zope.com> <1092928547.8719.70.camel@geddy.wooz.org> Message-ID: <1092935726.1542.23.camel@localhost> On Thu, 2004-08-19 at 16:15, Barry Warsaw wrote: > Python represents Guido's aesthetic in language design, and we all agree > that he gets most such decisions right, so I trust him on this one too > -- and I was /not/ a pie-decorator fan originally. I think it's a significant point in favour of pie syntax that it grows on people quite quickly if they use it for a bit. Has anyone has tried it out for a few days and still hated it just as much at the end? Mark From dave at boost-consulting.com Thu Aug 19 19:31:47 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 19 19:32:53 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 References: <000f01c48590$866705e0$e841fea9@oemcomputer> <200408190407.i7J47Ar19512@guido.python.org> Message-ID: Guido van Rossum writes: >> > Hm, shouldn't the bytecode optimizer only be used when -O is used, and >> > hence pyo files are being written? >> >> Why? That would throw away most of the benefits to most of the >> users and gain nothing in return. The peepholer was in place in for >> Py2.3 and only benefits were seen. I would save the -O option for >> something where there is a trade-off (loss of docstrings, excessive >> compilation time, possibly risky optimizations, or somesuch). Here, >> the peepholer is superfast and costs almost nothing. > > Maybe I'm out of tune, but I thought that optimizations should be > turned off by default because most people don't need them and because > of the risk that the optimizer might break something. Haven't there > been situations in Python where one optimization or another was found > to be unsafe after having been in use (in a release!) for a long > time? Isn't that a good argument for having them turned on all the time? The easiest way to ship code that suffers from an optimizer bug is to do all your development and most of your testing with unoptimized code. In C/C++, there's a good reason to develop code unoptimized: it's much easier to debug. I'm not sure that applies to Python. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From walter at livinglogic.de Thu Aug 19 19:34:33 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 19 19:34:38 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124741B.3080605@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <4123B804.2000508@livinglogic.de> <4124741B.3080605@egenix.com> Message-ID: <4124E4A9.7070204@livinglogic.de> M.-A. Lemburg wrote: > Walter D?rwald wrote: > >> Martin v. L?wis wrote: >> >> [...] >> We already have an efficient way to communicate incompleteness: >> the decode method returns the number of decoded bytes. >> >> The questions remaining are >> >> 1) communicate to whom? IMHO the info should only be used >> internally by the StreamReader. > > Handling incompleteness should be something for the codec > to deal with. Absolutely. This means that decode() should not be called by the user. (But the implementation of read() (and feed(), if we have it) calls it.) > The queue doesn't have to know about it in an > way. However, the queue should have interfaces allowing the > codec to tell whether there are more bytes waiting to be > processed. This won't work when the byte stream wrapped by the StreamReader is not a queue. (Or do you want the wrap the byte stream in a queue? This would be three wrapping layers.) And the information is not really useful, because it might change (e.g. when the user puts additional data into the queue/stream.) >> 2) When is incompleteness OK? Incompleteness is of course >> not OK in the stateless API. For the stateful API, >> incompleteness has to be OK even when the input stream >> is (temporarily) exhausted, because otherwise a feed mode >> wouldn't work anyway. But then incompleteness is always OK, >> because the StreamReader can't distinguish a temporarily >> exhausted input stream from a permanently exhausted one. >> The only fix for this I can think of is the final argument. > > A final argument may be the way to go. But it should be an > argument for the .read() method (not only the .decode() method) > since that's the method reading the data from the queue. Yes. E.g. the low level charmap decode function doesn't need the final argument, because there is zero state to be kept between calls. > I'd suggest that we extend the existing encode and decode > codec APIs to take an extra state argument that holds the > codec state in whatever format the codec needs (e.g. this > could be a tuple or a special object): > > encode(data, errors='strict', state=None) > decode(data, errors='strict', state=None) We don't need a specification for that. The stateless API doesn't need an explicit state (the state is just a bunch of variables at the C level) and for the stateful API the state can be put into StreamReader attributes. How this state looks is totally up to the StreamReader itself (see the UTF-7 reader in the patch for an example). If the stream reader passes on this state to a low level decoding function implemented in C, how this state info looks is again totally up to the codec. So I think we don't have to specify anything in this area. > In the case of the .read() method, decode() would be > called. If the returned length_consumed does not match > the length of the data input, the remaining items would > have to be placed back onto the queue in non-final mode. > In final mode an exception would be raised to signal > the problem. Yes, in non-final mode the bytes would have to be retained and in final mode an exception is raised (except when the error handling callback does something else). But I don't think we should put a queue between the byte stream and the StreamReader (at least not in the sense of a queue as another file like object). The remaining items can be kept in an attribute of the StreamReader instance, that's what --- data = self.bytebuffer + newdata object, decodedbytes = self.decode(data, self.errors) self.bytebuffer = data[decodedbytes:] --- does in the patch. The first line combines the items retained from the last call with those read from the stream (or passed to the feed method). The second line does semi-stateful decoding of those bytes. The third line puts the new remaining items back into the buffer. The decoding is "semi-stateful", because the info about the remaining bytes is not stored by decode itself, but by the caller of decode. feed() is the method that does fully stateful decoding of byte chunks. > I think it's PEP time for this new extension. If time > permits I'll craft an initial version over the weekend. I'm looking forward to the results. Bye, Walter D?rwald From bac at OCF.Berkeley.EDU Thu Aug 19 19:33:00 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 19 19:40:27 2004 Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords In-Reply-To: <1f7befae040819094825bfacf0@mail.gmail.com> References: <4124CC59.80007@interlink.com.au> <1f7befae040819094825bfacf0@mail.gmail.com> Message-ID: <4124E44C.8010805@ocf.berkeley.edu> Tim Peters wrote: > [Anthony Baxter] >>... >>I'm wondering if there's a way I can mark bugs >>and patches that should be in before b1. It appears >>from SF's tracker that the only way I'm going to be >>able to do that is by putting keywords in the summary. >>I'm considering doing this. > > > Changing priorities has worked well for this in the past. SF even > colors them differently > > I'm going to add one for you: > > - must be resolved for a3 > > Priority 9. It gets addressed or a3 doesn't ship. > > >> - should be in before b1 (changes functionality) > > > Priority 8. > > >> - should be in before final > > > Priority 7. > > >> - regression since 2.3 > > > Depends on whether the specific regression must be resolved for a3, > b1, or final -- or is going to be declared "a feature" and 2.3's > behavior "a bug". > Priority approach seems good. Can also easily sort by priority in the tracker. While you can sort by the summary as well would have to make sure that the names are in a proper alphabetical order to make that useful. > >>Does anyone else have any others that they think are >>useful? I'd probably go with something like [beta], >>[final], [regr] in the summary. > > > Over time, those tags become misleading, because they're so easy to > ignore. People won't allow high priorities to remain high, though. > It irritates them. That's good . > The other thing is that if something doesn't get done then the tag will become outdated. While it is true that this shouldn't happen if it does it would be a pain to go back through and remove those tags. -Brett From skip at pobox.com Thu Aug 19 19:40:49 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 19 19:41:04 2004 Subject: [Python-Dev] refcounts.dat In-Reply-To: References: Message-ID: <16676.58913.458682.732050@montanaro.dyndns.org> Thomas> Is the Doc/api/refcounts.dat file no longer maintained? The Thomas> last commit was on 2002/10/04. How does the C api manual the Thomas> refcount info? I suppose like any other piece of documentation refcounts.dat gets out-of-date without some periodic attention. If there are functions missing from that file they can be added (or a bug report filed). Take a look at Doc/tools/anno-api.py for details on how the file is used. Skip From jcarlson at uci.edu Thu Aug 19 19:59:13 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Aug 19 20:06:04 2004 Subject: [Python-Dev] Considering decorator syntax on an individual feature basis In-Reply-To: <1092931991.4124d1976b8ba@webmail.uoguelph.ca> References: <1092931991.4124d1976b8ba@webmail.uoguelph.ca> Message-ID: <20040819095717.3444.JCARLSON@uci.edu> > The features in question, which should be decided individually rather than as a > whole: Sounds good. I'll do something different here and remove your and my opinions from this post, and stick to "facts", or at least statements that can be reasonably derived from "facts.. > - decorator before/within/after def. While I understand that Python is not a democracy, voting does allow us to discover certain preferences within the community. According to the informal poll linked from the Python decorators wiki, that I posted about earlier, which seemed to have gotten no response, Guido's before approach is in the minority according to the community; "list within def" and "@ after def" favorites by a clear margin. A. @classmethod def foo(): (107) 16% C1. def foo() [classmethod]: (269) 39% E1. def foo(): @classmethod (311) 45% The fact that the two favored syntax families have been rejected by Guido suggests that the community has strong opinions either against @before, or in favor of [] within or @after. Based on people's statements against the @ symbol in the @before syntax, the voting suggests that location plays an important role, as @ before has slightly more than 34% as many votes as @after (or 40% as many as list within). It seems that those who voted for C1 disagree with the formatting argument presented against C1, and those who voted for E1 think that within the block is the proper location for decorators. > - special character/keyword/no new symbol. All of which are options in all three before, within and after def syntax families. If a symbol is necessary for the location that is chosen, perhaps an informal poll to choose the favorite within the community is in order. > - decorator on one/many line(s) > - one symbol per decorator vs. one symbol for all decorators Not really relevant. If we allow: @decorator(arg1, arg2) Then one/many lines and one/many symbols is a simple user-supplied decorator away, regardless of the underlying syntax. > - indent def/not indent def Mostly J1 vs every other option. Due to the lack of a reasonable number of people posting with "indentation is the right way" on python-dev during that flurry of hundreds of emails, I don't believe indenting the def is remotely in the running. > I think if each of these issues could be resolved and agreed to individually, > then a consensis would be far more apparent as to the ideal solution, or at > least some of the myriad solutions could be filtered out. Agreed. > I think the Python community has mostly settled on a "before def" approach, and The informal poll numbers say otherwise. - Josiah From walter at livinglogic.de Thu Aug 19 20:09:17 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 19 20:09:22 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <41247DD5.8020100@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de> <41247DD5.8020100@egenix.com> Message-ID: <4124ECCD.9080209@livinglogic.de> M.-A. Lemburg wrote: > Walter D?rwald wrote: > >> Let's compare example uses: >> >> 1) Having feed() as part of the StreamReader API: >> --- >> s = u"???".encode("utf-8") >> r = codecs.getreader("utf-8")() >> for c in s: >> print r.feed(c) >> --- > > I consider adding a .feed() method to the stream codec > bad design. .feed() is something you do on a stream, not > a codec. I don't care about the name, we can call it stateful_decode_byte_chunk() or whatever. (In fact I'd prefer to call it decode(), but that name is already taken by another method. Of course we could always rename decode() to _internal_decode() like Martin suggested.) >> 2) Explicitely using a queue object: >> --- >> from whatever import StreamQueue >> >> s = u"???".encode("utf-8") >> q = StreamQueue() >> r = codecs.getreader("utf-8")(q) >> for c in s: >> q.write(c) >> print r.read() >> --- > > This is probably how an advanced codec writer would use the APIs > to build new stream interfaces. > >> 3) Using a special wrapper that implicitely creates a queue: >> ---- >> from whatever import StreamQueueWrapper >> s = u"???".encode("utf-8") >> r = StreamQueueWrapper(codecs.getreader("utf-8")) >> for c in s: >> print r.feed(c) >> ---- > > > This could be turned into something more straight forward, > e.g. > > from codecs import EncodedStream > > # Load data > s = u"???".encode("utf-8") > > # Write to encoded stream (one byte at a time) and print > # the read output > q = EncodedStream(input_encoding="utf-8", output_encoding="unicode") This is confusing, because there is no encoding named "unicode". This should probably read: q = EncodedQueue(encoding="utf-8", errors="strict") > for c in s: > q.write(c) > print q.read() > > # Make sure we have processed all data: > if q.has_pending_data(): > raise ValueError, 'data truncated' This should be the job of the error callback, the last part should probably be: for c in s: q.write(c) print q.read() print q.read(final=True) >> I very much prefer option 1). > > I prefer the above example because it's easy to read and > makes things explicit. > >> "If the implementation is hard to explain, it's a bad idea." > > The user usually doesn't care about the implementation, only it's > interfaces. Bye, Walter D?rwald From walter at livinglogic.de Thu Aug 19 20:12:31 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 19 20:12:35 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412480F8.4060903@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <412480F8.4060903@egenix.com> Message-ID: <4124ED8F.1020105@livinglogic.de> M.-A. Lemburg wrote: > Walter D?rwald wrote: > >> Without the feed method(), we need the following: >> >> 1) A StreamQueue class that >> a) supports writing at one end and reading at the other end >> b) has a method for pushing back unused bytes to be returned >> in the next call to read() > > Right. As already stated in a previous post, I don't think StreamQueue needs a pushback() method. trailing bytes should all be stored in the StreamReader. > It also needs a method giving the number of pending bytes in > the queue or just an API .has_pending_data() that returns > True/False. What would this method be used for? Bye, Walter D?rwald From tim.peters at gmail.com Thu Aug 19 20:19:35 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 19 20:19:43 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 In-Reply-To: References: <000f01c48590$866705e0$e841fea9@oemcomputer> <200408190407.i7J47Ar19512@guido.python.org> Message-ID: <1f7befae04081911192336c3b8@mail.gmail.com> [Guido] >> Maybe I'm out of tune, but I thought that optimizations should be >> turned off by default because most people don't need them and because >> of the risk that the optimizer might break something. Haven't there >> been situations in Python where one optimization or another was found >> to be unsafe after having been in use (in a release!) for a long >> time? [David Abrahams] > Isn't that a good argument for having them turned on all the time? > The easiest way to ship code that suffers from an optimizer bug is to > do all your development and most of your testing with unoptimized > code. That's my belief in this case (as I tried to say earlier, perhaps unsuccessfully). The current peephole optimizer has been overwhelmingly successful (the bug report I referenced was about modules with more than 64KB of bytecode, and Python has always had troubles of one kind or another with those -- so it grew another one in that area, BFD), and I'm glad everyone runs it all the time. > In C/C++, there's a good reason to develop code unoptimized: it's much > easier to debug. I'm not sure that applies to Python. It *used* to apply, and for a similar reason, when -O also controlled whether SET_LINENO opcodes were generated, and the Python debugger was blind without them. That's no longer the case. The only reason to avoid -O now is to retain if __debug__: blocks and (same thing) assert statements. That can make a big speed difference in Zope and ZEO, for example. Life would definitely be worse if -O could introduce new bugs (other than programmer errors of putting essential code inside __debug__ blocks). From anthony at interlink.com.au Thu Aug 19 20:20:51 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Aug 19 20:21:02 2004 Subject: [Python-Dev] Considering decorator syntax on an individual feature basis In-Reply-To: <20040819095717.3444.JCARLSON@uci.edu> References: <1092931991.4124d1976b8ba@webmail.uoguelph.ca> <20040819095717.3444.JCARLSON@uci.edu> Message-ID: <4124EF83.40609@interlink.com.au> Please, please, please - could the discussion about decorators take place on comp.lang.python, unless there's something new to add to the thread? python-dev is rapidly becoming unusable. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido at python.org Thu Aug 19 20:38:48 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 20:38:55 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Your message of "Thu, 19 Aug 2004 11:15:17 +0200." <41246FA5.1010804@egenix.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <41231830.2090903@egenix.com> <200408190139.i7J1dX819100@guido.python.org> <41246FA5.1010804@egenix.com> Message-ID: <200408191838.i7JIcmm21008@guido.python.org> > Ok, so I suppose that we can learn from Jython and IronPython in > this respect... > > How do they handle binary data and the interfacing between various > I/O facilities, e.g. files, sockets, pipes, user input, etc. I'm not sure, but I expect that in most cases they use Unicode strings in order to be compatibly with Python's standard library. That's not the outcome I'd like to see though. I believe Jython at least also has a bytes-like type (probably a thin wrapper around Java's byte array) that's used for interfacing to java classes. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 19 20:52:38 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 20:52:45 2004 Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords In-Reply-To: Your message of "Thu, 19 Aug 2004 12:48:08 EDT." <1f7befae040819094825bfacf0@mail.gmail.com> References: <4124CC59.80007@interlink.com.au> <1f7befae040819094825bfacf0@mail.gmail.com> Message-ID: <200408191852.i7JIqcR21129@guido.python.org> +1 on Tim's suggestion of using priorities. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 19 20:53:02 2004 From: guido at python.org (Guido van Rossum) Date: Thu Aug 19 20:53:09 2004 Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords In-Reply-To: Your message of "Thu, 19 Aug 2004 17:50:22 BST." <2mk6vv8435.fsf@starship.python.net> References: <4124CC59.80007@interlink.com.au> <4124D8E4.8030705@python.org> <2mk6vv8435.fsf@starship.python.net> Message-ID: <200408191853.i7JIr2t21142@guido.python.org> > >> - regression since 2.3 > > > > What does that one mean? > > I guess thing's like Tim's recent asyncore pain. And time.strftime()? --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Aug 19 20:54:38 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 19 20:54:36 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412475DF.9010308@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <412475DF.9010308@egenix.com> Message-ID: <4124F76E.9040901@v.loewis.de> M.-A. Lemburg wrote: >> Marc-Andre, if the original patch (diff.txt) was applied: >> What *specific* change in that patch would break code? >> What *specific* code (C or Python) would break under that >> change? [...] > Let's flesh this out some more and get a better understanding > of what is needed and how the separation between the stream queue, > the stream codec and the underlying codec implementation can > be put to good use. That really didn't answer the question: What would be technically wrong with accepting Walter's patches? I smell over-designing: there is a specific problem at hand (incomplete reads resulting in partial decoding); any solution should attempt to *only* solve this problem, not any other problem. Regards, Martin From martin at v.loewis.de Thu Aug 19 20:58:07 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 19 20:58:04 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124CB16.90705@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de> <4123C16D.4060505@v.loewis.de> <4124CB16.90705@livinglogic.de> Message-ID: <4124F83F.1080707@v.loewis.de> Walter D?rwald wrote: > 1) The current implementation does not really support the > stateful API, because trailing incomplete byte sequences > lead to errors. Correct. > 2) The current file API is not really convenient for decoding > when the input is not read for a stream. I don't see this problem. It is straight-forward to come up with a file-like object that converts the data from whatever source into a stream. If it wasn't a byte stream/string eventually, there would be no way to meaningfully decode it. Regards, Martin From foom at fuhm.net Thu Aug 19 20:59:03 2004 From: foom at fuhm.net (James Y Knight) Date: Thu Aug 19 20:59:09 2004 Subject: [Python-Dev] Re: PEP 318: Can't we all just get along? In-Reply-To: References: <412480FF.2030203@zope.com> Message-ID: On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote: > The vast majority of instance methods I've seen all use 'self' as the > first parameter. Likewise, most class methods use 'cls' or 'klass' as > their first parameter. If we exploit these conventions, we end up > with a simple, clear, obvious mechanism for denoting (this aspect of) > a method's type. > > class Foo(Object): > def m1(self, a, b): # this is an instance method of Foo > pass > > def m2(cls, a, b): # this is a class method of Foo > pass > > def m3(a, b): # this is a static method of Foo > pass > > A special Object (capital 'O') class could work this magic so that old > code didn't break. > > I know that this is odd. But then so are most of the great things > about Python. You can do that today. See also http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py However, note that IMO it is quite rude to use a metaclass (or your capital O object -- same thing) to do this, as it will break any objects inheriting from your class that don't expect the strange automatic behavior. This auto-class/staticmethod-ification should be local to your code, and thus is really a candidate for a class decorator. @automethods class Foo(object): ... James From martin at v.loewis.de Thu Aug 19 21:05:43 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 19 21:05:38 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124DA28.50106@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> Message-ID: <4124FA07.1070701@v.loewis.de> Walter D?rwald wrote: > Both approaches have one problem: Error handling won't > work for them. If the error handling is "replace", the decoder > should return U+FFFD for the final trailing incomplete sequence > in read(). This won't happen when read() never reads those > bytes from the input stream. Ok. So it really looks like a final flag on read is necessary. > Well, I had to choose a subject. ;) I still would prefer if the two issues were discussed separately. Regards, Martin From martin at v.loewis.de Thu Aug 19 21:20:18 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 19 21:20:16 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <16676.47823.382278.737098@montanaro.dyndns.org> References: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com> <16676.47823.382278.737098@montanaro.dyndns.org> Message-ID: <4124FD72.4090901@v.loewis.de> Skip Montanaro wrote: > Any thoughts about posting this to c.l.py as well as python-dev? Maybe it > would encourage some more participation... +1. Martin From martin at v.loewis.de Thu Aug 19 21:21:59 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 19 21:21:57 2004 Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords In-Reply-To: <4124D8E4.8030705@python.org> References: <4124CC59.80007@interlink.com.au> <4124D8E4.8030705@python.org> Message-ID: <4124FDD7.6030305@v.loewis.de> David Goodger wrote: > [Anthony Baxter] > >> - regression since 2.3 > > > What does that one mean? Are you asking what a regression is? In that context: A feature/functionality that used to work in 2.3, but stopped working in CVS. Regards, Martin From pm_mon at yahoo.com Thu Aug 19 21:30:38 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Thu Aug 19 21:30:43 2004 Subject: [Python-Dev] Re: PEP 318: Can't we all just get along? In-Reply-To: References: <412480FF.2030203@zope.com> Message-ID: James Y Knight wrote: > On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote: > >> The vast majority of instance methods I've seen all use 'self' as the >> first parameter. Likewise, most class methods use 'cls' or 'klass' as >> their first parameter. If we exploit these conventions, we end up >> with a simple, clear, obvious mechanism for denoting (this aspect of) >> a method's type. >> >> class Foo(Object): >> def m1(self, a, b): # this is an instance method of Foo >> pass >> >> def m2(cls, a, b): # this is a class method of Foo >> pass >> >> def m3(a, b): # this is a static method of Foo >> pass >> >> A special Object (capital 'O') class could work this magic so that old >> code didn't break. >> >> I know that this is odd. But then so are most of the great things >> about Python. > > > You can do that today. See also > http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py > Aha! Thanks! > However, note that IMO it is quite rude to use a metaclass (or your > capital O object -- same thing) to do this, as it will break any objects > inheriting from your class that don't expect the strange automatic > behavior. This auto-class/staticmethod-ification should be local to your > code, and thus is really a candidate for a class decorator. > > @automethods > class Foo(object): > ... > But then wouldn't subclasses of Foo have the same rude behavior? In fact, isn't the use of any metaclass rude, by your definition, as it will change the default class behavior in some way? From bob at redivi.com Thu Aug 19 21:38:24 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Aug 19 21:39:01 2004 Subject: [Python-Dev] Re: PEP 318: Can't we all just get along? In-Reply-To: References: <412480FF.2030203@zope.com> Message-ID: <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com> On Aug 19, 2004, at 3:30 PM, Paul Morrow wrote: > James Y Knight wrote: > >> On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote: >>> The vast majority of instance methods I've seen all use 'self' as >>> the first parameter. Likewise, most class methods use 'cls' or >>> 'klass' as their first parameter. If we exploit these conventions, >>> we end up with a simple, clear, obvious mechanism for denoting (this >>> aspect of) a method's type. >>> >>> class Foo(Object): >>> def m1(self, a, b): # this is an instance method of Foo >>> pass >>> >>> def m2(cls, a, b): # this is a class method of Foo >>> pass >>> >>> def m3(a, b): # this is a static method of Foo >>> pass >>> >>> A special Object (capital 'O') class could work this magic so that >>> old code didn't break. >>> >>> I know that this is odd. But then so are most of the great things >>> about Python. >> You can do that today. See also >> http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py > > Aha! Thanks! Whatever happened to explicit is better than implicit? ;) >> However, note that IMO it is quite rude to use a metaclass (or your >> capital O object -- same thing) to do this, as it will break any >> objects inheriting from your class that don't expect the strange >> automatic behavior. This auto-class/staticmethod-ification should be >> local to your code, and thus is really a candidate for a class >> decorator. >> @automethods >> class Foo(object): >> ... > > But then wouldn't subclasses of Foo have the same rude behavior? In > fact, isn't the use of any metaclass rude, by your definition, as it > will change the default class behavior in some way? No, a decorator happens once. A metaclass can/does have permanent effects and becomes an integral and/or annoying part of the inheritance graph. class Foo(object): __metaclass__ = Bar ... is *much* different than class Foo(object): ... Foo = barEquivalentFunction(Foo) which is the same as @barEquivalentFunction class Foo(object): ... -bob From mal at egenix.com Thu Aug 19 21:59:06 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 21:59:12 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124F76E.9040901@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <412475DF.9010308@egenix.com> <4124F76E.9040901@v.loewis.de> Message-ID: <4125068A.2010509@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >>> Marc-Andre, if the original patch (diff.txt) was applied: >>> What *specific* change in that patch would break code? >>> What *specific* code (C or Python) would break under that >>> change? > > [...] > >> Let's flesh this out some more and get a better understanding >> of what is needed and how the separation between the stream queue, >> the stream codec and the underlying codec implementation can >> be put to good use. > > > That really didn't answer the question: What would be technically > wrong with accepting Walter's patches? > > I smell over-designing: there is a specific problem at hand > (incomplete reads resulting in partial decoding); any solution > should attempt to *only* solve this problem, not any other > problem. The specific problem is that of providing a codec that can run in feeding mode where you can feed in data and read it in a way that allows incomplete input data to fail over nicely. Since this requires two head positions (one for the writer, one for the reader), a queue implementation is the right thing to use. We are having this discussion to find a suitable design to provide this functionality in a nice and clean way. I don't see anything wrong with this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 19 22:03:41 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 22:03:45 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124ED8F.1020105@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <412480F8.4060903@egenix.com> <4124ED8F.1020105@livinglogic.de> Message-ID: <4125079D.4000609@egenix.com> Walter D?rwald wrote: > M.-A. Lemburg wrote: > >> Walter D?rwald wrote: >> >>> Without the feed method(), we need the following: >>> >>> 1) A StreamQueue class that >>> a) supports writing at one end and reading at the other end >>> b) has a method for pushing back unused bytes to be returned >>> in the next call to read() >> >> >> Right. > > > As already stated in a previous post, I don't think StreamQueue > needs a pushback() method. trailing bytes should all be stored > in the StreamReader. I'd leave that to the StreamReader implementor. I would always push the data back onto the queue, simply because it's unprocessed data. >> It also needs a method giving the number of pending bytes in >> the queue or just an API .has_pending_data() that returns >> True/False. > > > What would this method be used for? If you push the data back onto the queue, you will probably want to check whether there's pending data left. That's what this method is intended to tell you. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 19 22:09:09 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 19 22:09:12 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124ECCD.9080209@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de> <4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de> <41247DD5.8020100@egenix.com> <4124ECCD.9080209@livinglogic.de> Message-ID: <412508E5.7010307@egenix.com> Walter D?rwald wrote: > M.-A. Lemburg wrote: > >> Walter D?rwald wrote: >> >>> Let's compare example uses: >>> >>> 1) Having feed() as part of the StreamReader API: >>> --- >>> s = u"???".encode("utf-8") >>> r = codecs.getreader("utf-8")() >>> for c in s: >>> print r.feed(c) >>> --- >> >> >> I consider adding a .feed() method to the stream codec >> bad design. .feed() is something you do on a stream, not >> a codec. > > > I don't care about the name, we can call it > stateful_decode_byte_chunk() or whatever. (In fact I'd > prefer to call it decode(), but that name is already > taken by another method. Of course we could always > rename decode() to _internal_decode() like Martin > suggested.) It's not that name that doesn't fit, it's the fact that you are mixing a stream action into a codec which I'd rather see well separated. >>> 2) Explicitely using a queue object: >>> --- >>> from whatever import StreamQueue >>> >>> s = u"???".encode("utf-8") >>> q = StreamQueue() >>> r = codecs.getreader("utf-8")(q) >>> for c in s: >>> q.write(c) >>> print r.read() >>> --- >> >> >> This is probably how an advanced codec writer would use the APIs >> to build new stream interfaces. > > > > >>> 3) Using a special wrapper that implicitely creates a queue: >>> ---- >>> from whatever import StreamQueueWrapper >>> s = u"???".encode("utf-8") >>> r = StreamQueueWrapper(codecs.getreader("utf-8")) >>> for c in s: >>> print r.feed(c) >>> ---- >> >> >> >> This could be turned into something more straight forward, >> e.g. >> >> from codecs import EncodedStream >> >> # Load data >> s = u"???".encode("utf-8") >> >> # Write to encoded stream (one byte at a time) and print >> # the read output >> q = EncodedStream(input_encoding="utf-8", output_encoding="unicode") > > > This is confusing, because there is no encoding named "unicode". > This should probably read: > > q = EncodedQueue(encoding="utf-8", errors="strict") Fine. I was thinking of something similar to EncodedFile() which also has two separate encodings, one for the file side of things and one for the Python side. >> for c in s: >> q.write(c) >> print q.read() >> >> # Make sure we have processed all data: >> if q.has_pending_data(): >> raise ValueError, 'data truncated' > > > This should be the job of the error callback, the last part should > probably be: > > for c in s: > q.write(c) > print q.read() > print q.read(final=True) Ok; both methods have their use cases. (You seem to be obsessed with this final argument ;-) >>> I very much prefer option 1). >> >> >> I prefer the above example because it's easy to read and >> makes things explicit. >> >>> "If the implementation is hard to explain, it's a bad idea." >> >> >> The user usually doesn't care about the implementation, only it's >> interfaces. > > > Bye, > Walter D?rwald > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/mal%40egenix.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From exarkun at divmod.com Thu Aug 19 22:09:21 2004 From: exarkun at divmod.com (Jp Calderone) Date: Thu Aug 19 22:09:28 2004 Subject: [Python-Dev] Re: PEP 318: Can't we all just get along? In-Reply-To: <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com> References: <412480FF.2030203@zope.com> <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com> Message-ID: <412508F1.6010500@divmod.com> Bob Ippolito wrote: > > class Foo(object): > __metaclass__ = Bar > ... > > is *much* different than > > class Foo(object): > ... > Foo = barEquivalentFunction(Foo) > > which is the same as > > @barEquivalentFunction > class Foo(object): > ... > Only if you choose to write it such that it is different. Either can result in effects that persist down the inheritence tree, and either can result in effects that do not. Jp From skip at pobox.com Thu Aug 19 22:40:24 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 19 22:40:44 2004 Subject: [Python-Dev] decimal module portability to 2.3? Message-ID: <16677.4152.404288.108320@montanaro.dyndns.org> The docstring for the decimal module says in part: This is a Py2.3 implementation of decimal floating point arithmetic based on the General Decimal Arithmetic Specification: Is it intended to remain 2.3-compatible for the forseeable future? It's not mentioned in PEP 291, nor is there any discussion of portability to 2.3 in PEP 327. I ask because we're at 2.3.4 at work, but could use it today. I'd hate to have to track it to backport fixes though. It would be ideal if I could simply copy the 2.4 version as it matures. Thx, Skip From bac at OCF.Berkeley.EDU Thu Aug 19 22:42:06 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 19 22:42:26 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <16676.46377.92891.139081@montanaro.dyndns.org> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com> <16676.46377.92891.139081@montanaro.dyndns.org> Message-ID: <4125109E.6040706@ocf.berkeley.edu> Skip Montanaro wrote: > >> Is anybody seriously trying to come up with a single alternative > >> decorator proposal that most folks "out there" can support, to be > >> presented to me (with implementation, please!) in time for 2.4b1? > > Tim> I'm afraid that absolutely zero consensus is emerging :( I've > Tim> withdrawn myself entirely from the discussions as they've proven > Tim> completely useless. > > Ditto. I think the list-after-def proposal had some support early on but > people got very carried away proposing all sorts of other alternatives. > Having just read the bulk of the decorator emails in two nights, I can confidently say that three proposals had steam. One was definitely list-after-def. Another was using a pipe (E2, although there is none showing it above the function definition). People seemed to like that except for the font argument against it. The last one was using '!' instead of either '@' or '|'. If people want to get an alternative in I say vote on these four (the '!' has to choose where it goes relative to the 'def') on c.l.py and present it to Guido 5 days before a3 (so by the 27th; that's a week from tomorrow) to get it in since people will need time to play with it before we hit b1. Ignore if Guido rejected it at some point and just see if everyone can just agree on *something*. Then Guido decides and this thing can be over with and people can just move on to learning how to use decorators properly. -Brett From martin at v.loewis.de Thu Aug 19 22:42:48 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 19 22:42:43 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4125068A.2010509@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <412475DF.9010308@egenix.com> <4124F76E.9040901@v.loewis.de> <4125068A.2010509@egenix.com> Message-ID: <412510C8.5060508@v.loewis.de> M.-A. Lemburg wrote: > We are having this discussion to find a suitable design > to provide this functionality in a nice and clean way. > I don't see anything wrong with this. The problem is that this has nothing to do with decoding incomplete unicode. Regards, Martin From bob at redivi.com Thu Aug 19 22:55:04 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Aug 19 22:55:42 2004 Subject: [Python-Dev] Re: PEP 318: Can't we all just get along? In-Reply-To: <412508F1.6010500@divmod.com> References: <412480FF.2030203@zope.com> <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com> <412508F1.6010500@divmod.com> Message-ID: <0BD15540-F222-11D8-BC06-000A95686CD8@redivi.com> On Aug 19, 2004, at 4:09 PM, Jp Calderone wrote: > Bob Ippolito wrote: >> class Foo(object): >> __metaclass__ = Bar >> ... >> is *much* different than >> class Foo(object): >> ... >> Foo = barEquivalentFunction(Foo) >> which is the same as >> @barEquivalentFunction >> class Foo(object): >> ... > > Only if you choose to write it such that it is different. Either > can result in effects that persist down the inheritence tree, and > either can result in effects that do not. Of course, but my point is that it's default behavior for the "Bar" metaclass to become part of the inheritance graph, where default behavior for "barEquivalentFunction" wouldn't ever do such a thing. Actually, aside from returning a different class object altogether, the only thing I can imagine "barEquivalentFunction" doing that would affect the inheritance graph is to mutate __bases__, which is just crazy. -bob From pm_mon at yahoo.com Thu Aug 19 23:06:23 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Thu Aug 19 23:06:24 2004 Subject: [Python-Dev] Re: PEP 318: Can't we all just get along? In-Reply-To: <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com> References: <412480FF.2030203@zope.com> <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com> Message-ID: Bob Ippolito wrote: > > On Aug 19, 2004, at 3:30 PM, Paul Morrow wrote: > >> James Y Knight wrote: >> >>> On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote: >>> >>>> The vast majority of instance methods I've seen all use 'self' as >>>> the first parameter. Likewise, most class methods use 'cls' or >>>> 'klass' as their first parameter. If we exploit these conventions, >>>> we end up with a simple, clear, obvious mechanism for denoting (this >>>> aspect of) a method's type. >>>> >>>> class Foo(Object): >>>> def m1(self, a, b): # this is an instance method of Foo >>>> pass >>>> >>>> def m2(cls, a, b): # this is a class method of Foo >>>> pass >>>> >>>> def m3(a, b): # this is a static method of Foo >>>> pass >>>> >>>> A special Object (capital 'O') class could work this magic so that >>>> old code didn't break. >>>> >>>> I know that this is odd. But then so are most of the great things >>>> about Python. >>> >>> You can do that today. See also >>> http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py >> >> >> Aha! Thanks! > > > Whatever happened to explicit is better than implicit? ;) > I've often wondered about that principle. For example, wouldn't static typing (actual declarations, as in Java) be more explicit than dynamic typing? Either way, it seems that if dynamic method typing (static|class|instance) was enabled for a class, then def m2(cls, a, b): pass would be every bit as explicit (a means of specifying that m2 is a class method) as x = [] (is of specifying that x is (well, contains) a list). From FBatista at uniFON.com.ar Thu Aug 19 23:06:15 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Thu Aug 19 23:14:15 2004 Subject: [Python-Dev] decimal module portability to 2.3? Message-ID: [Skip Montanaro] #- Is it intended to remain 2.3-compatible for the forseeable #- future? It's not #- mentioned in PEP 291, nor is there any discussion of #- portability to 2.3 in #- PEP 327. I don't have plans to do some effort to keep it "2.3-runnable" when Py2.4 hits the street. Don't know if Raymond or somebody will preocupate about this. Could decimal be backported to 2.3 and be included in some 2.3.5? Don't know if this behaviour is common to Python. #- I ask because we're at 2.3.4 at work, but could use it #- today. I'd hate to #- have to track it to backport fixes though. It would be #- ideal if I could #- simply copy the 2.4 version as it matures. I have a similar issue. I copied the decimal.py code and included it on the SiGeFi package (http://sourceforge.net/projects/sigefi), so when somebody downloads the 0.2 version could use it with Py2.3 (it's the version required). My plan (right now) is to wait Py2.4, then remove decimal.py from the package and require Py2.4 (this is justified because SiGeFi will reach "user usage" next year). . Facundo From bac at OCF.Berkeley.EDU Thu Aug 19 23:35:58 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 19 23:36:01 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: References: Message-ID: <41251D3E.1090305@ocf.berkeley.edu> Batista, Facundo wrote: > Could decimal be backported to 2.3 and be included in some 2.3.5? Don't know > if this behaviour is common to Python. > That can't be done. That would introduce new features in a bug release which is a big no-no. -Brett From python at rcn.com Thu Aug 19 23:41:16 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 19 23:41:46 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: Message-ID: <000801c48635$46986e40$e841fea9@oemcomputer> > [Skip Montanaro] > > #- Is it intended to remain 2.3-compatible for the forseeable > #- future? It's not > #- mentioned in PEP 291, nor is there any discussion of > #- portability to 2.3 in > #- PEP 327. > > I don't have plans to do some effort to keep it "2.3-runnable" when Py2.4 > hits the street. Don't know if Raymond or somebody will preocupate about > this. Yes, I've been keeping it Py2.3 compatible and periodically test it on 2.3 just to make sure. Am hesitant to introduce this as a specific, on-going restriction to the code, but I don't want to throw-away 2.3 compatibility unless there is a darned good offsetting gain. Raymond From bac at OCF.Berkeley.EDU Thu Aug 19 23:42:00 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 19 23:42:05 2004 Subject: [Python-Dev] Unifying Long Integers and Integers: baseint In-Reply-To: <1092659867.4120aa9ba1239@mcherm.com> References: <1092659867.4120aa9ba1239@mcherm.com> Message-ID: <41251EA8.8050008@ocf.berkeley.edu> Michael Chermside wrote: >>>>>if the only reason for it is to use isinstance? >>>> >>>>So that an extension author *could* write an int-like type deriving >>>>from it? >>> >>>But didn't you just say that people shouldn't be >>>deriving their own int-like types from baseinteger? >> >>Indeed, in general they shouldn't. But for specialized purposes it >>might be needed (that's why I emphasized *could*). > > > I call YAGNI. We're talking about creating the class baseinteger > which might be useful ONLY for people creating new kinds of integers > in Python which will NOT extend int or long but WILL need to be > treated just like integers. Who is really likely to do that? And if > in the process we introduce a new class which won't be needed in > the long run (ie Python 3000 has just one type, called "int" and has > no need for baseinteger). So I maintain that it's not needed (and > is, in fact, confusing to users) unless someone has a real use case. > I'm with Michael on this. We have gone this long without having a need for a baseinteger type (when was long introduced?) so I don't see a need to add it now. Let's just live with the dichotomy until Python 3000 (moving over to 3000 as Guido suggested in the "PEP 3000" thread) comes out. -Brett From python at rcn.com Thu Aug 19 23:47:26 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 19 23:47:48 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: <41251D3E.1090305@ocf.berkeley.edu> Message-ID: <000901c48636$1eb2b240$e841fea9@oemcomputer> > Batista, Facundo wrote: > > > Could decimal be backported to 2.3 and be included in some 2.3.5? Don't > know > > if this behaviour is common to Python. [Brett] > That can't be done. That would introduce new features in a bug release > which is a big no-no. Right. Though keeping it 2.3 compatible means people can download it as a standalone module just like any other third-party module. This worked out well for sets.py which I've made 2.2 compatible. By keeping decimal 2.3 compatible, folks can make periodic downloads to take advantage of bug fixes and efforts to keep it in sync with the on-going evolution of the spec. Also, I'm expecting that at some point in the future, there will be a concerted effort to improve its performance -- it runs two orders of magnitude slower than we would like (see the telco benchmark in the sandbox). Raymond From tjreedy at udel.edu Thu Aug 19 23:49:25 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Thu Aug 19 23:49:30 2004 Subject: [Python-Dev] Re: PEP 318: Suggest we drop it References: <4123CB1A.2060101@sabaydi.com> Message-ID: "Roman Suzi" wrote in message news:Pine.LNX.4.58.0408190817070.16021@rnd.onego.ru... > and their support for "write once" concept, while in old syntax one need > to write a function name 4 times to merely get it decorated and with __doc__ > string. I believed this once also, but then realized that one only need write a long function name once (other than in the doc string) -- in the final assignment. Perhaps something like d = '''actual_long_name(ague) == return syncroed flooblejack ague == seq of doobies ''' def f(ague): pass f.__doc__ = d f.atto = 'yes' actual_long_name = syncro(staticmethod(deco(f))) Even without a doc string or comment at the top, finding the actual name at the bottem of a block might be as easy as finding it in the middle of a block with several @ lines. So I no longer count this as so strong an argument against the postfix form as I one did. Terry J. Reedy From bac at OCF.Berkeley.EDU Fri Aug 20 00:18:43 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 20 00:18:48 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: <000901c48636$1eb2b240$e841fea9@oemcomputer> References: <000901c48636$1eb2b240$e841fea9@oemcomputer> Message-ID: <41252743.7000301@ocf.berkeley.edu> Raymond Hettinger wrote: >>Batista, Facundo wrote: >> >> >>>Could decimal be backported to 2.3 and be included in some 2.3.5? > > Don't > >>know >> >>>if this behaviour is common to Python. > > > [Brett] > >>That can't be done. That would introduce new features in a bug > > release > >>which is a big no-no. > > > Right. Though keeping it 2.3 compatible means people can download it as > a standalone module just like any other third-party module. This worked > out well for sets.py which I've made 2.2 compatible. > > By keeping decimal 2.3 compatible, folks can make periodic downloads to > take advantage of bug fixes and efforts to keep it in sync with the > on-going evolution of the spec. > Right, but you stated in another email you don't want to make it explicit by declaring it in PEP 291 so that is only true at the moment and could go away. At which point people would need an easy way to get a hold of an older version before compatibility is broken unless the sandbox version is kept around and bugfixed on occasion like datetime. -Brett From barry at python.org Fri Aug 20 00:32:52 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 00:32:28 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1f7befae040811221916a955fd@mail.gmail.com> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> Message-ID: <1092954772.8712.147.camel@geddy.wooz.org> On Thu, 2004-08-12 at 01:19, Tim Peters wrote: > I do object to this part: > > If the $ character appears at the end of the line, or is followed > by any other character than those described above, it is treated > as if it had been escaped, appearing in the resulting string > unchanged. > > There's already a facility to escape $, and it's dead easy to use. $ > isn't a frequently needed character either in most apps. So escaping > $ "by magic" too is, I think, more likely to make typing errors harder > to find than to do anyone real good. What would you do about $'s at the end of the string? I think the implementation would be much more complex if you didn't have this rule, e.g. you'd have to match \$$ and the $-placeholder regexps would basically have to match everything. Then the .sub call would have to be more complex too, because it would have to check for the existence of those match groups and then raise an exception. Or something like that. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/8d26f4bc/attachment.pgp From barry at python.org Fri Aug 20 00:36:42 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 00:36:22 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <59e9fd3a0408112305fa1b667@mail.gmail.com> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <200408120539.i7C5dqL29226@guido.python.org> <59e9fd3a0408112305fa1b667@mail.gmail.com> Message-ID: <1092955002.8714.149.camel@geddy.wooz.org> On Thu, 2004-08-12 at 02:05, Andrew Durdin wrote: > One thing the PEP doesn't make clear: do these templates only > interpolate string variables, or will they handle anything with a > __str__ method? > > e.g., will the following raise an exception: > > print Template("I am ${height} centimetres tall.") % {height: 183} In the current implementation, this will raise an exception. I'm uncomfortable with automatic stringification (hey, create your own dict subclass for the mapping if you want ;) -- but I've included this as an open issue in the next rev of the PEP. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/5ac2e1f7/attachment.pgp From adurdin at gmail.com Fri Aug 20 00:41:47 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Fri Aug 20 00:41:57 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408191158.12738.gmccaughan@synaptics-uk.com> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> <200408190558.i7J5wPP19708@guido.python.org> <200408191158.12738.gmccaughan@synaptics-uk.com> Message-ID: <59e9fd3a040819154158b60833@mail.gmail.com> On Thu, 19 Aug 2004 11:58:12 +0100, Gareth McCaughan wrote: > On Thursday 2004-08-19 06:58, Guido van Rossum wrote: > > > > Well, do people generally buy those rejections, or is their consensus > > that I'm mistaken? > > I'm one of the people who doesn't really feel it's worth trying. > *My* consensus :-) is that you're mistaken. I think that > I agree with Gareth on both of his main points: I didn't buy Guido's rejection of list-after-def (and so argued against his points in another thread here); but the decorator suite before the def (J2) is quite nice also, and it doesn't hide the def when there are lots of decorators (due to the indentation). I would strongly support either proposal over the current @decorators. From python at rcn.com Fri Aug 20 00:53:20 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 20 00:53:42 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: <16677.4152.404288.108320@montanaro.dyndns.org> Message-ID: <001701c4863f$537bcb20$e841fea9@oemcomputer> > The docstring for the decimal module says in part: > > This is a Py2.3 implementation of decimal floating point arithmetic > based on > the General Decimal Arithmetic Specification: > > Is it intended to remain 2.3-compatible for the forseeable future? It's > not > mentioned in PEP 291, nor is there any discussion of portability to 2.3 in > PEP 327. I've added module comments to include an update policy and will update the PEP 291 to match. Nothing in Py2.4 warrants giving up 2.3 compatibility, so 2.3 will be kept at least until Py2.5. > I ask because we're at 2.3.4 at work, but could use it today. I'd hate to > have to track it to backport fixes though. It would be ideal if I could > simply copy the 2.4 version as it matures. Yes, that was the intention behind the 2.3 compatibility code I put in. Also, I will keep testing it under 2.3. Also, the policy states that spec updates will be treated as bugfixes and backported. There should be very few of these and they are likely to not be major. There were a couple of big changes on the table but Mike Cowlishaw advised me that these are now very unlikely. Raymond From bac at OCF.Berkeley.EDU Fri Aug 20 01:12:54 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 20 01:13:01 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1092954772.8712.147.camel@geddy.wooz.org> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> Message-ID: <412533F6.4000703@ocf.berkeley.edu> Barry Warsaw wrote: > On Thu, 2004-08-12 at 01:19, Tim Peters wrote: > > >>I do object to this part: >> >> If the $ character appears at the end of the line, or is followed >> by any other character than those described above, it is treated >> as if it had been escaped, appearing in the resulting string >> unchanged. >> >>There's already a facility to escape $, and it's dead easy to use. $ >>isn't a frequently needed character either in most apps. So escaping >>$ "by magic" too is, I think, more likely to make typing errors harder >>to find than to do anyone real good. > > > What would you do about $'s at the end of the string? I think the > implementation would be much more complex if you didn't have this rule, > e.g. you'd have to match \$$ and the $-placeholder regexps would > basically have to match everything. Then the .sub call would have to be > more complex too, because it would have to check for the existence of > those match groups and then raise an exception. Or something like that. > Got an implementation and it's simple. =) It only required one additional group at the end (r"(? References: <007301c480b1$0867bec0$5229c797@oemcomputer> Message-ID: <1092957426.8714.182.camel@geddy.wooz.org> On Thu, 2004-08-12 at 17:12, Raymond Hettinger wrote: > After more thought, I would like to voice a strong -1 on the packaging > of string. Since few people have commented either way, and taking myself out of the picture, it comes down to a battle royale between you and Brett. :) Seriously, I'm willing to drop this non-PEP'd suggestion, since it just makes my life easier, however see below. > At the outset, I felt that it introduced unnecessary complexity. Adding > a new directory, an __init__.py, and two modules just to add a few lines > of code is overkill. Package logic should be reserved for organizing > voluminous code like the email package. Arguably, the logging package > should have been just a single module -- putting it in a package just > made it more difficult to read and maintain. I think that logging would have been much more readable if more stuff were taken out of its __init__.py and put into sub-package modules. I happen to like the way email does it, and I've come to believe that package authors should strive for minimal __init__.py's, at most exposing the public interface for the package but keeping code to a minimum. That's besides the point though. > To test the existing sandbox code, I moved it into the library and found > a circular import issue: importing the string package, causes > template.py to be imported, which in turn imports re which uses > sre_parse.py, needs to import string. So, the whole thing fails. Something's wacky with either your, or my setup then: Python 2.4a2 (#1, Aug 18 2004, 00:08:12) [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from string import Template [25426 refs] I'd like to know if anybody else has tried to wedge the sandbox implementation into their Python tree, and whether it worked or not. If you do, don't forget to remove your old string.py and string.pyc files. > My recommendation: Create a stringtools module with the new template > stuff in it and with room for growth. Leave string.py alone and let it > die in peace. As I told Raymond in pvt email, I'm really not happy with creating another string module. I hate stringlib.py and stringtools.py seems too contrived. I'm putting this as an open issue in the PEP's next rev, but to me the most logical place to put the Template and SafeTemplate classes is in the string module. So if we /don't/ reorganize string, then I propose to just drop these two classes in string.py. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/59098ad9/attachment.pgp From python at rcn.com Fri Aug 20 01:22:09 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 20 01:22:31 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1092954772.8712.147.camel@geddy.wooz.org> Message-ID: <001f01c48643$5985faa0$e841fea9@oemcomputer> [Tim] > > I do object to this part: > > > > If the $ character appears at the end of the line, or is followed > > by any other character than those described above, it is treated > > as if it had been escaped, appearing in the resulting string > > unchanged. > > > > There's already a facility to escape $, and it's dead easy to use. $ > > isn't a frequently needed character either in most apps. So escaping > > $ "by magic" too is, I think, more likely to make typing errors harder > > to find than to do anyone real good. [Barry] > What would you do about $'s at the end of the string? I think the > implementation would be much more complex if you didn't have this rule, > e.g. you'd have to match \$$ and the $-placeholder regexps would > basically have to match everything. Then the .sub call would have to be > more complex too, because it would have to check for the existence of > those match groups and then raise an exception. Or something like that. FWIW, Cheetah implements the behavior as described by Barry. IDLE 1.1a2 >>> from Cheetah.Template import Template >>> t = Template('the $rank is $@ a $$ dead ${rank}! $') >>> t.rank='king' >>> print t the king is $@ a $$ dead king! $ I think "escaping by magic" is the operational description. A better way to think of it is that only well formed substitutions are made. Raymond From python at rcn.com Fri Aug 20 01:39:16 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 20 01:39:38 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1092957426.8714.182.camel@geddy.wooz.org> Message-ID: <002401c48645$bdf482c0$e841fea9@oemcomputer> > Since few people have commented either way, and taking myself out of the > picture, it comes down to a battle royale between you and Brett. :) Hmm. Brett is taller. I'm better looking. He's almost a master of computer science. I'm just old. He's needs a summer job to eat. I've got a retirement account. Perhaps, I can win him over through guile: 1. Call up IDLE, then press File, Open Module, string. Look at the source. Now, install your package and try again. No source. 2. Run "cvs ann string.py" or "cvs log string.py" and look at who did what and when. Now, install the package and try again. No history. 3. Look at the new string docs, decide that you want to subclass string.Template and want to look at the existing pattern. Time how long it takes you to find the source (change to the string subdirectory, look in __init__.py, learn the template.py is the real source, yada yada yada). > > To test the existing sandbox code, I moved it into the library and found > > a circular import issue: importing the string package, causes > > template.py to be imported, which in turn imports re which uses > > sre_parse.py, needs to import string. So, the whole thing fails. > > Something's wacky with either your, or my setup then: > > Python 2.4a2 (#1, Aug 18 2004, 00:08:12) > [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on > linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> from string import Template > [25426 refs] > > I'd like to know if anybody else has tried to wedge the sandbox > implementation into their Python tree, and whether it worked or not. If > you do, don't forget to remove your old string.py and string.pyc files. How could this work? Template has an re.compile that runs upon import. The sre_parse module has an "import string" that runs when imported. Raymond From xscottg at yahoo.com Fri Aug 20 02:02:04 2004 From: xscottg at yahoo.com (Scott Gilbert) Date: Fri Aug 20 02:02:07 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <59e9fd3a040819154158b60833@mail.gmail.com> Message-ID: <20040820000204.12671.qmail@web50201.mail.yahoo.com> Please count a humble vote for list after def. The following is simple: def foo(self, a, b) [classmethod]: pass And the pathological cases can be made more readable with a little formatting: def longMethodNameForEffect( longArgumentOne=None, longArgumentTwo=42 )[ staticmethod, funcattrs( grammar="'@' dotted_name [ '(' [arglist] ')' ]", status="experimental", author="BDFL" ) ]: """This method blah, blah. It supports the following arguments: - longArgumentOne -- a string giving ... - longArgumentTwo -- a number giving ... blah, blah. """ raise NotYetImplemented Back to lurking... From guido at python.org Fri Aug 20 02:07:36 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 20 02:07:44 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: Your message of "Fri, 20 Aug 2004 08:41:47 +1000." <59e9fd3a040819154158b60833@mail.gmail.com> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> <200408190558.i7J5wPP19708@guido.python.org> <200408191158.12738.gmccaughan@synaptics-uk.com> <59e9fd3a040819154158b60833@mail.gmail.com> Message-ID: <200408200007.i7K07av21619@guido.python.org> > I agree with Gareth on both of his main points: I didn't buy Guido's > rejection of list-after-def (and so argued against his points in > another thread here); but the decorator suite before the def (J2) is > quite nice also, and it doesn't hide the def when there are lots of > decorators (due to the indentation). I would strongly support either > proposal over the current @decorators. So pick a favorite already and argue with the folks on c.l.py about which one will be presented to me. I'm serious. I don't want a hundred people arguing their separate points with me. If that's all I get the current syntax stays. If the community can *agree* on something they can support, I will listen. But it has to be a unified voice. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 20 02:08:56 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 20 02:09:03 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: Your message of "Thu, 19 Aug 2004 17:47:26 EDT." <000901c48636$1eb2b240$e841fea9@oemcomputer> References: <000901c48636$1eb2b240$e841fea9@oemcomputer> Message-ID: <200408200008.i7K08uC21635@guido.python.org> > By keeping decimal 2.3 compatible, folks can make periodic downloads > to take advantage of bug fixes and efforts to keep it in sync with > the on-going evolution of the spec. +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 20 02:17:36 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 20 02:17:42 2004 Subject: [Python-Dev] PEP 318 has ReST errors Message-ID: <200408200017.i7K0HaT21703@guido.python.org> The cronjob on my home machine that formats the PEPs 4x daily complains about syntax errors in PEP 318. Last time that happened I fixed them all, but today I don't have time. Can somebody with checkin perms and ReST foo please fix these? pep-0318.txt:330: (ERROR/3) Unexpected indentation. pep-0318.txt:336: (ERROR/3) Unexpected indentation. pep-0318.txt:351: (WARNING/2) Inline substitution_reference start-string without end-string. pep-0318.txt:106: (ERROR/3) Unknown target name: "java-style". pep-0318.txt:132: (ERROR/3) Unknown target name: "gof book". pep-0318.txt:169: (ERROR/3) Unknown target name: "in your face". pep-0318.txt:218: (ERROR/3) Unknown target name: "a large number". pep-0318.txt:218: (ERROR/3) Unknown target name: "each possible syntax". pep-0318.txt:288: (ERROR/3) Unknown target name: "summarised the arguments". pep-0318.txt:507: (ERROR/3) Unknown target name: "annotations". --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Fri Aug 20 02:40:33 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 20 02:41:18 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: <000801c48635$46986e40$e841fea9@oemcomputer> References: <000801c48635$46986e40$e841fea9@oemcomputer> Message-ID: <16677.18561.746876.718204@montanaro.dyndns.org> Raymond> Yes, I've been keeping it Py2.3 compatible and periodically Raymond> test it on 2.3 just to make sure. Raymond> Am hesitant to introduce this as a specific, on-going Raymond> restriction to the code, but I don't want to throw-away 2.3 Raymond> compatibility unless there is a darned good offsetting gain. Raymond, Given that you've expended effort to keep decimal 2.3-compatible I sort of doubt at this point that something will be done to make it incompatible with 2.3 before 2.4 is released. Might I suggest an addition to PEP 291 like this: Package/Module Maintainer(s) Python Version Notes -------------- ------------- -------------- ----- ... decimal Raymond, et al 2.3 [2] ... with note 2 being something like: Compatibility with 2.3 maintained until at least 2.5 is released. The thought there is that after 2.4 is released no new language features or modules should be added to 2.4.n (n > 0) which you would want to use in the decimal module. If 2.4 is released with a decimal module that is 2.3-compatible, maintaining that compatibility until 2.5 is released shouldn't be too difficult. The above would satisfy me. Thx, Skip From tim.peters at gmail.com Fri Aug 20 03:10:56 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 20 03:11:04 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1092954772.8712.147.camel@geddy.wooz.org> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> Message-ID: <1f7befae0408191810474dc3de@mail.gmail.com> [Tim] >> I do object to this part: >> >> If the $ character appears at the end of the line, or is followed >> by any other character than those described above, it is treated >> as if it had been escaped, appearing in the resulting string >> unchanged. >> >> There's already a facility to escape $, and it's dead easy to use. $ >> isn't a frequently needed character either in most apps. So escaping >> $ "by magic" too is, I think, more likely to make typing errors harder >> to find than to do anyone real good. [Barry] > What would you do about $'s at the end of the string? Celebrate them, I suppose, if they're somehow better than $'s elsewhere in the string . > I think the implementation would be much more complex if you didn't have this > rule, e.g. you'd have to match \$$ and the $-placeholder regexps would > basically have to match everything. Then the .sub call would have to be > more complex too, because it would have to check for the existence of > those match groups and then raise an exception. Or something like that. Na. Add a 4th group to the regexp: |(\$) That will reliably match any $, anywhere, that one of the first three groups doesn't know what to do with. Then add ", illegal" to the end of the LHS of the line that unpacks the groups, and then, e.g., if illegal: raise ValueError("unsubstitutable $ at index %d" % mo.start(4)) In return for that new line, I'll never have to worry about typing "${barry)-rulz!" by mistake (depending on the font you're using, you may or may not have a hard time seeing the typo in that -- and assuming I didn't intend "${barry)-rulz!" to be the output too, which it as things are now). From andymac at bullseye.apana.org.au Fri Aug 20 00:17:38 2004 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Fri Aug 20 03:13:44 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320 In-Reply-To: <1f7befae04081911192336c3b8@mail.gmail.com> References: <000f01c48590$866705e0$e841fea9@oemcomputer> <200408190407.i7J47Ar19512@guido.python.org> <1f7befae04081911192336c3b8@mail.gmail.com> Message-ID: <20040820081535.X98652@bullseye.apana.org.au> On Thu, 19 Aug 2004, Tim Peters wrote: > It *used* to apply, and for a similar reason, when -O also controlled > whether SET_LINENO opcodes were generated, and the Python debugger was > blind without them. That's no longer the case. The only reason to > avoid -O now is to retain > > if __debug__: > > blocks and (same thing) assert statements. That can make a big speed > difference in Zope and ZEO, for example. Life would definitely be > worse if -O could introduce new bugs (other than programmer errors of > putting essential code inside __debug__ blocks). Seems to me that in this context -O would be more appropriately renamed to -D to reflect the debug relationship... -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From goodger at python.org Fri Aug 20 03:22:37 2004 From: goodger at python.org (David Goodger) Date: Fri Aug 20 03:22:39 2004 Subject: [Python-Dev] Re: PEP 318 has ReST errors In-Reply-To: <200408200017.i7K0HaT21703@guido.python.org> References: <200408200017.i7K0HaT21703@guido.python.org> Message-ID: <4125525D.5060902@python.org> [Guido van Rossum] > Can somebody with checkin perms and ReST foo please fix these? Done. -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/e8b9a386/signature.pgp From greg at cosc.canterbury.ac.nz Fri Aug 20 03:40:15 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 20 03:40:21 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408200007.i7K07av21619@guido.python.org> Message-ID: <200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz> Guido: > If the community can *agree* on > something they can support, I will listen. Even if it's []-after-args? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From neal at metaslash.com Fri Aug 20 04:09:19 2004 From: neal at metaslash.com (Neal Norwitz) Date: Fri Aug 20 04:09:27 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules socketmodule.c, 1.298, 1.299 In-Reply-To: References: Message-ID: <20040820020919.GG31470@epoch.metaslash.com> > Log Message: > Patch #1003700: Add socketpair function to socket module. > > Index: socketmodule.c > + #ifdef HAVE_SOCKETPAIR > + /* Create a pair of sockets using the socketpair() function. > + Arguments as for socket(). */ > + > + /*ARGSUSED*/ > + static PyObject * > + socket_socketpair(PyObject *self, PyObject *args) > + { > + PySocketSockObject *s0 = NULL, *s1 = NULL; > + SOCKET_T sv[2]; > + int family, type = SOCK_STREAM, proto = 0; > + PyObject *res = NULL; > + > + #if defined(AF_UNIX) > + family = AF_UNIX; > + #else > + family = AF_INET; > + #endif The docstring (below) states the arguments are the same as socket(). However, in sock_initobj() line 2496, the family is initialized to AF_INET. I think the #if defined(AF_UNIX) code above should be removed and family should be initialized to AF_INET. I didn't look to see if the documentation agrees with the docstring. > + if (!PyArg_ParseTuple(args, "|iii:socketpair", > + &family, &type, &proto)) > + return NULL; > + /* Create a pair of socket fds */ > + if (socketpair(family, type, proto, sv) < 0) > + return set_error(); > + #ifdef SIGPIPE > + (void) signal(SIGPIPE, SIG_IGN); > + #endif I don't think the #ifdef SIGPIPE code is correct. If the user installed a signal handler calling signal() will remove it. I think the call to signal() should be removed. Neal > + s0 = new_sockobject(sv[0], family, type, proto); > + if (s0 == NULL) > + goto finally; > + s1 = new_sockobject(sv[1], family, type, proto); > + if (s1 == NULL) > + goto finally; > + res = PyTuple_Pack(2, s0, s1); > + > + finally: > + if (res == NULL) { > + if (s0 == NULL) > + SOCKETCLOSE(sv[0]); > + if (s1 == NULL) > + SOCKETCLOSE(sv[1]); > + } > + Py_XDECREF(s0); > + Py_XDECREF(s1); > + return res; > + } > + > + PyDoc_STRVAR(socketpair_doc, > + "socketpair([family[, type[, proto]]]) -> (socket object, socket object)\n\ > + \n\ > + Create a pair of socket objects from the sockets returned by the platform\n\ > + socketpair() function.\n\ > + The arguments are the same as for socket()."); > + > + #endif /* HAVE_SOCKETPAIR */ From bac at OCF.Berkeley.EDU Fri Aug 20 04:10:44 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 20 04:10:45 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <002401c48645$bdf482c0$e841fea9@oemcomputer> References: <002401c48645$bdf482c0$e841fea9@oemcomputer> Message-ID: <41255DA4.1070002@ocf.berkeley.edu> Raymond Hettinger wrote: >>Since few people have commented either way, and taking myself out of > > the > >>picture, it comes down to a battle royale between you and Brett. :) > > > Hmm. Brett is taller. I'm better looking. Don't let the photos from PyCON fool you; I never have a good look on my face and I tend to look stoned. > He's almost a master of computer science. I'm just old. I hope you are not trying to use some "wiser" argument with that. =) > He's needs a summer job to eat. I've got a retirement account. > Got me there. > Perhaps, I can win him over through guile: > > 1. Call up IDLE, then press File, Open Module, string. Look at the > source. Now, install your package and try again. No source. > Yeah, but you will notice the directory and realize it is a package so look in there. > 2. Run "cvs ann string.py" or "cvs log string.py" and look at who did > what and when. Now, install the package and try again. No history. > Eh. That module has been only changed to keep up with the built-ins or tweak a docstring. Not exactly major stuff I feel bad about losing some commit messages over. > 3. Look at the new string docs, decide that you want to subclass > string.Template and want to look at the existing pattern. Time how long > it takes you to find the source (change to the string subdirectory, look > in __init__.py, learn the template.py is the real source, yada yada > yada). > Only reason that could be considered an issue is because the __init__.py file has the constants and everything else that is not deprecated in there. Basically I can live with having a single string module, but I would like to see some real deprecation happen. The nice thing about the pacakge separation is it made it clear in the code what would be going away. If we at least at PendingDeprecation to what is supposed to be taken out I will be happy. -Brett From anthony at interlink.com.au Fri Aug 20 05:25:27 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 20 05:25:31 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: References: Message-ID: <41256F27.8090602@interlink.com.au> Batista, Facundo wrote: > Could decimal be backported to 2.3 and be included in some 2.3.5? Don't know > if this behaviour is common to Python. Absolutely not. See PEP 0006. There's nothing stopping someone from making a seperate package of the decimal code (complete with setup.py) and distributing it for Python 2.3 users to download and add on. Barry does this with the email package. From python at rcn.com Fri Aug 20 05:33:25 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 20 05:33:52 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: <16677.18561.746876.718204@montanaro.dyndns.org> Message-ID: <003e01c48666$73afbe20$e841fea9@oemcomputer> > Raymond> Yes, I've been keeping it Py2.3 compatible and periodically > Raymond> test it on 2.3 just to make sure. > > Raymond> Am hesitant to introduce this as a specific, on-going > Raymond> restriction to the code, but I don't want to throw-away 2.3 > Raymond> compatibility unless there is a darned good offsetting gain. > > Raymond, > > Given that you've expended effort to keep decimal 2.3-compatible I sort of > doubt at this point that something will be done to make it incompatible > with > 2.3 before 2.4 is released. > > Might I suggest an addition to PEP 291 like this: > > Package/Module Maintainer(s) Python Version Notes > -------------- ------------- -------------- ----- > ... > decimal Raymond, et al 2.3 [2] > ... > > with note 2 being something like: > > Compatibility with 2.3 maintained until at least 2.5 is released. > > The thought there is that after 2.4 is released no new language features > or > modules should be added to 2.4.n (n > 0) which you would want to use in > the > decimal module. If 2.4 is released with a decimal module that is > 2.3-compatible, maintaining that compatibility until 2.5 is released > shouldn't be too difficult. > > The above would satisfy me. Done. Raymond From anthony at interlink.com.au Fri Aug 20 05:34:39 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 20 05:34:42 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408200007.i7K07av21619@guido.python.org> References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> <200408190558.i7J5wPP19708@guido.python.org> <200408191158.12738.gmccaughan@synaptics-uk.com> <59e9fd3a040819154158b60833@mail.gmail.com> <200408200007.i7K07av21619@guido.python.org> Message-ID: <4125714F.4000709@interlink.com.au> Guido van Rossum wrote: > So pick a favorite already and argue with the folks on c.l.py about > which one will be presented to me. I'm serious. I don't want a > hundred people arguing their separate points with me. If that's all I > get the current syntax stays. If the community can *agree* on > something they can support, I will listen. But it has to be a unified > voice. More importantly, _don't_ go either collecting or posting votes on this issue to python-dev. If all you have to add is either "+1 to this" or restating prior arguments, _use_ _comp.lang.python_. I'd suggest the thread "Alternative decorator syntax decision" in that group. Anthony From skip at pobox.com Fri Aug 20 06:14:13 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 20 06:14:25 2004 Subject: [Python-Dev] decimal module portability to 2.3? In-Reply-To: <003e01c48666$73afbe20$e841fea9@oemcomputer> References: <16677.18561.746876.718204@montanaro.dyndns.org> <003e01c48666$73afbe20$e841fea9@oemcomputer> Message-ID: <16677.31381.599608.540220@montanaro.dyndns.org> >> Might I suggest an addition to PEP 291 like this: ... >> >> Package/Module Maintainer(s) Python Version Notes >> -------------- ------------- -------------- ----- >> ... >> decimal Raymond, et al 2.3 [2] >> ... ... Raymond> Done. It would appear Raymond has his own time machine. I was responding to email as I read it. About five messages further into my python-dev mailbox I saw his checkin message... Skip From python at rcn.com Fri Aug 20 06:17:12 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 20 06:17:39 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1f7befae0408191810474dc3de@mail.gmail.com> Message-ID: <004501c4866c$919ea620$e841fea9@oemcomputer> > -----Original Message----- > From: python-dev-bounces+python=rcn.com@python.org [mailto:python-dev- > [Tim] > >> I do object to this part: > >> > >> If the $ character appears at the end of the line, or is followed > >> by any other character than those described above, it is treated > >> as if it had been escaped, appearing in the resulting string > >> unchanged. > >> > >> There's already a facility to escape $, and it's dead easy to use. $ > >> isn't a frequently needed character either in most apps. So escaping > >> $ "by magic" too is, I think, more likely to make typing errors harder > >> to find than to do anyone real good. . . . raise ValueError("unsubstitutable $ at index %d" % mo.start(4)) > > In return for that new line, I'll never have to worry about typing > "${barry)-rulz!" by mistake (depending on the font you're using, you > may or may not have a hard time seeing the typo in that -- and > assuming I didn't intend "${barry)-rulz!" to be the output too, which > it as things are now). That was a good example. However, I think we should follow Cheetah's example. That's makes it less of a pita when the template naturally contains dollar signs with numbers: Template("Dear ${donor}, send $100.00 to the PSF template fund.") Raymond From tim.peters at gmail.com Fri Aug 20 06:34:31 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 20 06:34:34 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <004501c4866c$919ea620$e841fea9@oemcomputer> References: <004501c4866c$919ea620$e841fea9@oemcomputer> Message-ID: <1f7befae0408192134734b6a38@mail.gmail.com> [Raymond Hettinger] > That was a good example. > > However, I think we should follow Cheetah's example. Sorry, I never heard of Cheetah before you mentioned it (& still don't know what it is), so feel no pressure to ape it. > That's makes it less of a pita when the template naturally contains > dollar signs with numbers: I don't care. The minority of Americans who want to use $ in a minority of templated strings in a minority of apps can learn to type $$, or use the ironically named SafeTemplate instead (as suggested earlier). > Template("Dear ${donor}, send $100.00 to the PSF template fund.") Which would trigger an exception, after which they type one character to repair it: Template("Dear ${donor}, send $$100.00 to the PSF template fund.") They'll live, but not at everyone else's expense. They're already confused, or they wouldn't have bothered with the unnecessary braces around "donor" . From andrewm at object-craft.com.au Fri Aug 20 06:44:36 2004 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri Aug 20 06:44:40 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules socketmodule.c, 1.298, 1.299 In-Reply-To: <20040820020919.GG31470@epoch.metaslash.com> References: <20040820020919.GG31470@epoch.metaslash.com> Message-ID: <20040820044436.0F8503C113@coffee.object-craft.com.au> >> Patch #1003700: Add socketpair function to socket module. > >The docstring (below) states the arguments are the same as socket(). >However, in sock_initobj() line 2496, the family is initialized to >AF_INET. I think the #if defined(AF_UNIX) code above should be >removed and family should be initialized to AF_INET. I talked Dave into this - AF_UNIX is typically the only address family that is valid for socketpair(). Using AF_INET under linux and OS X results in EOPNOTSUPP. >I don't think the #ifdef SIGPIPE code is correct. If the user >installed a signal handler calling signal() will remove it. I >think the call to signal() should be removed. I agree, but I think that was copied verbatim from elsewhere in socketmodule.c, so it was left with the aim of being bug for bug compatible with socket.socket(). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From djc at object-craft.com.au Fri Aug 20 07:06:21 2004 From: djc at object-craft.com.au (Dave Cole) Date: Fri Aug 20 07:06:34 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules socketmodule.c, 1.298, 1.299 In-Reply-To: <20040820044436.0F8503C113@coffee.object-craft.com.au> References: <20040820020919.GG31470@epoch.metaslash.com> <20040820044436.0F8503C113@coffee.object-craft.com.au> Message-ID: <412586CD.2000000@object-craft.com.au> Andrew McNamara wrote: >>>Patch #1003700: Add socketpair function to socket module. >> >>The docstring (below) states the arguments are the same as socket(). >>However, in sock_initobj() line 2496, the family is initialized to >>AF_INET. I think the #if defined(AF_UNIX) code above should be >>removed and family should be initialized to AF_INET. > > > I talked Dave into this - AF_UNIX is typically the only address family > that is valid for socketpair(). Using AF_INET under linux and OS X > results in EOPNOTSUPP. I wondered about the AF_UNIX/AF_INET thing. When looking through the code I noticed that all references to AF_UNIX were #ifdef enclosed. I decided to allow for the possibility of socketpair() support on a platform that supports AF_INET but not AF_UNIX (no idea if such a thing exists). The other alternative could have been to conditional include socketpair() only if both AF_UNIX and HAVE_SOCKETPAIR are defined. Another thing to consider is the prior art in the form of eunuchs. The implementation of socketpair() in eunuchs defaults to AF_UNIX. Code using eunuchs is probably assuming AF_UNIX - not a bad assumption really. Would a change to the documentation and docstring be sufficient it it explained that the default family is AF_UNIX if defined for the platform, AF_INET otherwise? >>I don't think the #ifdef SIGPIPE code is correct. If the user >>installed a signal handler calling signal() will remove it. I >>think the call to signal() should be removed. > > > I agree, but I think that was copied verbatim from elsewhere in > socketmodule.c, so it was left with the aim of being bug for bug > compatible with socket.socket(). Indeed. I was not going to change something as fundamental as the existing signal handling. The onyl sensible thing to do was to copy the behaviour already in place when you create a socket the "normal" way. -- http://www.object-craft.com.au From lists at hlabs.spb.ru Fri Aug 20 12:22:55 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Fri Aug 20 08:16:34 2004 Subject: [Python-Dev] python-dev Summary for 2004-08-01 through 2004-08-15 [draft] In-Reply-To: <4124506F.6040503@ocf.berkeley.edu> References: <4124506F.6040503@ocf.berkeley.edu> Message-ID: <4125D0FF.3080307@hlabs.spb.ru> Brett C. wrote: > -------------------------------------- > Is an int/long combined type worth it? > -------------------------------------- > language evolution > > Dmitry Vasiliev pointed out that `PEP 237`_ (Unifying Long Integers and > Integers) mentioned that a a new type named 'integer' might be > introduced that subclassed both int and long. The discussion waivered > between whether it was at all needed, and if it was if it should be a > true type or just a tuple containing both types for use with isinstance() . > > No conclusion was reached in the end. > > .. _PEP 237: http://www.python.org/peps/pep-0237.html > > Contributing threads: > - `Unifying Long Integers and Integers: baseint > `__ The discussion has been closed at: http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=1007068 -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From barry at python.org Fri Aug 20 16:16:03 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 16:15:34 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <41255DA4.1070002@ocf.berkeley.edu> References: <002401c48645$bdf482c0$e841fea9@oemcomputer> <41255DA4.1070002@ocf.berkeley.edu> Message-ID: <1093011363.8761.57.camel@geddy.wooz.org> On Thu, 2004-08-19 at 22:10, Brett C. wrote: > Don't let the photos from PyCON fool you; I never have a good look on my > face and I tend to look stoned. Just "look"? Dang, I really thought Tim, Fred, and I had a new recruit for our CrackPython project. The first one's free, y'know. > Basically I can live with having a single string module, but I would > like to see some real deprecation happen. The nice thing about the > pacakge separation is it made it clear in the code what would be going > away. If we at least at PendingDeprecation to what is supposed to be > taken out I will be happy. At the very least, my rewrite of libstring.tex will make it clear which inhabitants of the string module are going to be deprecated. Besides, since it's clear that Python 3000 will take a broader look at standard library packagization, I'll drop this if we can agree that the PEP 292 classes should go in the existing string module. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/34b36756/attachment.pgp From aahz at pythoncraft.com Fri Aug 20 16:40:48 2004 From: aahz at pythoncraft.com (Aahz) Date: Fri Aug 20 16:40:50 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1092957426.8714.182.camel@geddy.wooz.org> References: <007301c480b1$0867bec0$5229c797@oemcomputer> <1092957426.8714.182.camel@geddy.wooz.org> Message-ID: <20040820144048.GB4952@panix.com> On Thu, Aug 19, 2004, Barry Warsaw wrote: > On Thu, 2004-08-12 at 17:12, Raymond Hettinger wrote: >> >> After more thought, I would like to voice a strong -1 on the packaging >> of string. > > Since few people have commented either way, and taking myself out of the > picture, it comes down to a battle royale between you and Brett. :) I'm +1 on a package. I'm a strong +0 on naming the package ``string``. However, the reason I didn't speak up earlier was because I thought Raymond had a good technical point about his failure in getting your package to work, and I can't in good conscience support the package until that's resolved. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From mwh at python.net Fri Aug 20 17:03:14 2004 From: mwh at python.net (Michael Hudson) Date: Fri Aug 20 17:03:17 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1092955002.8714.149.camel@geddy.wooz.org> (Barry Warsaw's message of "Thu, 19 Aug 2004 18:36:42 -0400") References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <200408120539.i7C5dqL29226@guido.python.org> <59e9fd3a0408112305fa1b667@mail.gmail.com> <1092955002.8714.149.camel@geddy.wooz.org> Message-ID: <2mbrh597il.fsf@starship.python.net> Barry Warsaw writes: > On Thu, 2004-08-12 at 02:05, Andrew Durdin wrote: >> One thing the PEP doesn't make clear: do these templates only >> interpolate string variables, or will they handle anything with a >> __str__ method? >> >> e.g., will the following raise an exception: >> >> print Template("I am ${height} centimetres tall.") % {height: 183} > > In the current implementation, this will raise an exception. What, like a NameError for height? :-) Sorry, mwh -- I can't see a conspicuous evolutionary advantage in being good at higher mathematics. -- James Riden, asr From barry at python.org Fri Aug 20 17:05:27 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 17:04:59 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <001f01c48643$5985faa0$e841fea9@oemcomputer> References: <001f01c48643$5985faa0$e841fea9@oemcomputer> Message-ID: <1093014306.9254.3.camel@geddy.wooz.org> On Thu, 2004-08-19 at 19:22, Raymond Hettinger wrote: > IDLE 1.1a2 > >>> from Cheetah.Template import Template > >>> t = Template('the $rank is $@ a $$ dead ${rank}! $') > >>> t.rank='king' > >>> print t > the king is $@ a $$ dead king! $ There's a neat idea buried in there: Templates taking an interpolation mapping from their attributes. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/fe36fcea/attachment.pgp From barry at python.org Fri Aug 20 17:08:15 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 17:07:47 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <412533F6.4000703@ocf.berkeley.edu> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> <412533F6.4000703@ocf.berkeley.edu> Message-ID: <1093014495.9258.6.camel@geddy.wooz.org> On Thu, 2004-08-19 at 19:12, Brett C. wrote: > Got an implementation and it's simple. =) It only required one > additional group at the end (r"(? it matched. > > I tested against "$$" to be fine but for "blah $" and "$" to raise an > exception. Those all work and the tests in test_pep292 all pass. What do you think about this compared to Tim's pattern, which is more strict than just complaining about $'s at the end of strings? > If Barry is okay with this I can apply the patch and update the tests > and the PEP. Do need to know what exception to raise when this does > occur, though. ValueError? ValueError seems right to me. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/131a7810/attachment.pgp From barry at python.org Fri Aug 20 17:11:11 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 17:10:43 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <2mbrh597il.fsf@starship.python.net> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <200408120539.i7C5dqL29226@guido.python.org> <59e9fd3a0408112305fa1b667@mail.gmail.com> <1092955002.8714.149.camel@geddy.wooz.org> <2mbrh597il.fsf@starship.python.net> Message-ID: <1093014670.9249.8.camel@geddy.wooz.org> On Fri, 2004-08-20 at 11:03, Michael Hudson wrote: > What, like a NameError for height? :-) Heh. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/3c98476a/attachment.pgp From barry at python.org Fri Aug 20 17:21:22 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 17:20:55 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <20040820144048.GB4952@panix.com> References: <007301c480b1$0867bec0$5229c797@oemcomputer> <1092957426.8714.182.camel@geddy.wooz.org> <20040820144048.GB4952@panix.com> Message-ID: <1093015282.9255.11.camel@geddy.wooz.org> On Fri, 2004-08-20 at 10:40, Aahz wrote: > I'm +1 on a package. I'm a strong +0 on naming the package ``string``. > > However, the reason I didn't speak up earlier was because I thought > Raymond had a good technical point about his failure in getting your > package to work, and I can't in good conscience support the package > until that's resolved. Did you try it? I'm really interested in getting additional feedback, because AFAIK only Raymond and I have. It works for me, but doesn't for him. But if the package problem turns out to be real, we shouldn't support it, or expend too much effort fixing it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/73ea27df/attachment.pgp From fumanchu at amor.org Fri Aug 20 17:23:08 2004 From: fumanchu at amor.org (Robert Brewer) Date: Fri Aug 20 17:28:30 2004 Subject: [Python-Dev] Re: Update PEP 292 Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E13@exchange.hqamor.amorhq.net> Tim Peters wrote: > I don't care. The minority of Americans who want to use $ in a > minority of templated strings in a minority of apps can learn to type > $$, or use the ironically named SafeTemplate instead (as suggested > earlier). > > > Template("Dear ${donor}, send $100.00 to the PSF template fund.") > > Which would trigger an exception, after which they type one character > to repair it: > > Template("Dear ${donor}, send $$100.00 to the PSF template fund.") Yep. Just like they (I mean "I") have to remember to write: td width="10%%" FuManChu From bac at OCF.Berkeley.EDU Fri Aug 20 18:29:00 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 20 18:28:58 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1093014495.9258.6.camel@geddy.wooz.org> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> <412533F6.4000703@ocf.berkeley.edu> <1093014495.9258.6.camel@geddy.wooz.org> Message-ID: <412626CC.4050900@ocf.berkeley.edu> Barry Warsaw wrote: > On Thu, 2004-08-19 at 19:12, Brett C. wrote: > > >>Got an implementation and it's simple. =) It only required one >>additional group at the end (r"(?>it matched. >> >>I tested against "$$" to be fine but for "blah $" and "$" to raise an >>exception. Those all work and the tests in test_pep292 all pass. > > > What do you think about this compared to Tim's pattern, which is more > strict than just complaining about $'s at the end of strings? > That is coming down to a question of convenience compared to explicitness. Do we want "Give me $1,000,000 now!" to raise ValueError, or do we want to let it pass through? "Explicit is better than implicit" is good since it will cut down on errors (which is what Tim is arguing for along with "Special cases aren't special enough to break the rules"). But Raymond is pushing for "practicality beats purity" which is more or less true, but I am starting to agree with Tim that dollar amounts in an interpolated string probably won't hapeen that often. If anything the number will be what is interpolated in. So going for a more explicit version is seeming to be the better solution. Perk of the current implementation is that if someone wants a looser definition they can just change the regex themselves (you guys could even go as far as to break up the regex into individual class attributes and just have a property that does a '|'.join on them as a read method so it is even easier to just change the one group). So I guess it's your call, Mr PEP Champion, on whether you want all individual, non-interpolating $ signs to lead to an error or just random orphaned ones at the end of the line. -Brett From tim.peters at gmail.com Fri Aug 20 19:01:29 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 20 19:01:35 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <412626CC.4050900@ocf.berkeley.edu> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> <412533F6.4000703@ocf.berkeley.edu> <1093014495.9258.6.camel@geddy.wooz.org> <412626CC.4050900@ocf.berkeley.edu> Message-ID: <1f7befae04082010013adb0a85@mail.gmail.com> [Brett] > That is coming down to a question of convenience compared to > explicitness. Do we want "Give me $1,000,000 now!" to raise ValueError, > or do we want to let it pass through? > > "Explicit is better than implicit" is good since it will cut down on > errors (which is what Tim is arguing for along with "Special cases > aren't special enough to break the rules"). > > But Raymond is pushing for "practicality beats purity" which is more or > less true, It's far less true for non-Americans. I know it took me several decades to learn that "$ means currency" isn't an international standard enforced by the death penalty -- although it should be . > but I am starting to agree with Tim that dollar amounts in an > interpolated string probably won't hapeen that often. If anything the > number will be what is interpolated in. And that's where "handy shortcuts" bite too. "OK, I want to interpolate the value of variable 'dollars'. A $ sign on its own means a $ sign, and to interpolate the value of 'dollars' I do $dollars, so putting them those together I must need "Total: $$dollars" ". Which doesn't do what they want, although "Total: $ $dollars" might be close enough. A *consistent* (if you want $ in the output, you must have $$ in the input) rule may be surprising at first, but it doesn't remain surprising. > So going for a more explicit version is seeming to be the better solution. > Perk of the current implementation is that if someone wants a looser definition > they can just change the regex themselves (you guys could even go as far as to > break up the regex into individual class attributes and just have a > property that does a '|'.join on them as a read method so it is even > easier to just change the one group). We could indeed make this so complicated nobody will understand it . The use of named (as opposed to the current numbered) capturing groups could make it a lot less dependent on details of how the regexp happens to be written, though. > So I guess it's your call, Mr PEP Champion, on whether you want all > individual, non-interpolating $ signs to lead to an error or just random > orphaned ones at the end of the line. I still don't grasp why the end of the line is somehow special here. The real thrust of what I'm after is catching what will *certainly* be common typos, like $(barry) ${barry) $[barry] $(barry} ${barry{ ${barry The specific gimmick I suggested flags those as errors, and flags an odd number of trailing $;s too, but the latter isn't a kind of typo I expect occurs in real life. From python at rcn.com Fri Aug 20 19:04:46 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 20 19:05:09 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1093015282.9255.11.camel@geddy.wooz.org> Message-ID: <003e01c486d7$cbd5e5e0$e841fea9@oemcomputer> > > However, the reason I didn't speak up earlier was because I thought > > Raymond had a good technical point about his failure in getting your > > package to work, and I can't in good conscience support the package > > until that's resolved. > > Did you try it? I'm really interested in getting additional feedback, > because AFAIK only Raymond and I have. It works for me, but doesn't for > him. > > But if the package problem turns out to be real, we shouldn't support > it, or expend too much effort fixing it. That part is not a package problem. The circular import issue occurs whether you put Template in string.py or in a string package. Try a quick experiment to confirm on your machine: Take your existing string.py, add a couple of lines: import re pattern = re.compile('bingo') Now, clear out your pyc files, start python and import re. Here's what I get: C:\py24\Lib>python Python 2.4a2 (#46, Aug 19 2004, 18:01:28) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import re Traceback (most recent call last): File "", line 1, in ? File "C:\PY24\lib\re.py", line 5, in ? from sre import * File "C:\PY24\lib\sre.py", line 98, in ? import sre_parse File "C:\PY24\lib\sre_parse.py", line 16, in ? import string, sys File "string.py", line 43, in ? pattern = re.compile("bingo") AttributeError: 'module' object has no attribute 'compile' Raymond From python at rcn.com Fri Aug 20 19:19:14 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 20 19:19:39 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1f7befae0408192134734b6a38@mail.gmail.com> Message-ID: <003f01c486d9$d1298180$e841fea9@oemcomputer> > [Raymond Hettinger] > > However, I think we should follow Cheetah's example. > > Sorry, I never heard of Cheetah before you mentioned it (& still don't > know what it is), so feel no pressure to ape it. I'm surprised that we're introducing a new API into the standard library and no one is showing the slightest interest in prior art. Cheetah is well thought out, fast, and mature (it's been through several evolutions in the wild). Also, the Cheetah designers started out by studying the existing art from ASP, JSP, PHP, PSP, and such. Templating and string substituion is not a new, yet the discussion here has the flavor of being thought up from scratch. > > That's makes it less of a pita when the template naturally contains > > dollar signs with numbers: > > I don't care. The minority of Americans who want to use $ in a > minority of templated strings in a minority of apps can learn to type > $$, or use the ironically named SafeTemplate instead (as suggested > earlier). > > > Template("Dear ${donor}, send $100.00 to the PSF template fund.") > > Which would trigger an exception, after which they type one character > to repair it: > > Template("Dear ${donor}, send $$100.00 to the PSF template fund.") FWIW, I'm fine with that. It was a minor suggestion. Either way works. My thought was that simpler string substitutions were going to be exposed to the end-user (non-programmers) and that they should not be harassed unnecessarily. However, these are easy errors to trap and report back to the user -- the only question being how to assign a line number to the exception (if I have a ten page user supplied template with an error, it would be great to tell them where it is). Raymond > > They'll live, but not at everyone else's expense. They're already > confused, or they wouldn't have bothered with the unnecessary braces > around "donor" . From tim.peters at gmail.com Fri Aug 20 19:50:41 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 20 19:50:47 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <003f01c486d9$d1298180$e841fea9@oemcomputer> References: <003f01c486d9$d1298180$e841fea9@oemcomputer> Message-ID: <1f7befae04082010507d3c50d4@mail.gmail.com> [Raymond Hettinger] > I'm surprised that we're introducing a new API into the standard library > and no one is showing the slightest interest in prior art. Cheetah is > well thought out, fast, and mature (it's been through several evolutions > in the wild). Also, the Cheetah designers started out by studying the > existing art from ASP, JSP, PHP, PSP, and such. Templating and string > substituion is not a new, yet the discussion here has the flavor of > being thought up from scratch. The current discussion is also just the latest in years of discussion. I've had 25 years of using string-interpolation gimmicks in countless contexts myself, and my preference is a reaction against the practical problems I had with "too much magic" in them (e.g., I can't remember the difference between {} vs (), so I know I'll make typos related to that; I also know I'll have no trouble remembering "if you want $, say $$", and despite that I'll rarely want $). ... > FWIW, I'm fine with that. It was a minor suggestion. Either way works. The question is which works better. Let's make Guido decide . > My thought was that simpler string substitutions were going to be > exposed to the end-user (non-programmers) and that they should not be > harassed unnecessarily. However, these are easy errors to trap and > report back to the user -- the only question being how to assign a line > number to the exception (if I have a ten page user supplied template > with an error, it would be great to tell them where it is). Indeed! match_object.string contains the original template string, and match_object.start(4) gives the index at which the unsubstitutable $ appeared. A string-relative line number can be computed from that, and/or surrounding context can be inserted in the exception detail. From jimjjewett at yahoo.com Fri Aug 20 21:10:58 2004 From: jimjjewett at yahoo.com (Jim J Jewett) Date: Fri Aug 20 21:11:01 2004 Subject: [Python-Dev] Reserved Characters Message-ID: <20040820191058.51944.qmail@web50708.mail.yahoo.com> Python does not currently reserve any characters for user extensions. One of the objections to @decorators is that it will cause problems for applications that used '@', simply because of it was one of only three characters that *might* be available. Newer String interpolation will also use up '$', which leaves only '?'. Is it time to reserve a character to the user, or at least to reserve some in certain contexts? Examples: """Python will not use '!' except as part of '!=' """, or """The '?' character is reserved for the user.""" -jJ _______________________________ Do you Yahoo!? Win 1 of 4,000 free domain names from Yahoo! Enter now. http://promotions.yahoo.com/goldrush From barry at python.org Fri Aug 20 21:56:30 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 21:56:02 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1f7befae0408191810474dc3de@mail.gmail.com> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> <1f7befae0408191810474dc3de@mail.gmail.com> Message-ID: <1093031790.9257.56.camel@geddy.wooz.org> On Thu, 2004-08-19 at 21:10, Tim Peters wrote: > Na. Add a 4th group to the regexp: > > |(\$) > > That will reliably match any $, anywhere, that one of the first three > groups doesn't know what to do with. Works for me. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/af550877/attachment.pgp From barry at python.org Fri Aug 20 21:59:30 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 21:59:06 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1f7befae04082010013adb0a85@mail.gmail.com> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> <412533F6.4000703@ocf.berkeley.edu> <1093014495.9258.6.camel@geddy.wooz.org> <412626CC.4050900@ocf.berkeley.edu> <1f7befae04082010013adb0a85@mail.gmail.com> Message-ID: <1093031970.9257.61.camel@geddy.wooz.org> On Fri, 2004-08-20 at 13:01, Tim Peters wrote: > The use of named (as opposed to the current numbered) > capturing groups could make it a lot less dependent on details of how > the regexp happens to be written, though. Nice! I think this will make it easier to document how to do subclasses too. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/3a496dca/attachment.pgp From guido at python.org Fri Aug 20 22:07:01 2004 From: guido at python.org (Guido van Rossum) Date: Fri Aug 20 22:07:07 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: Your message of "Fri, 20 Aug 2004 13:40:15 +1200." <200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz> References: <200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz> Message-ID: <200408202007.i7KK71t23456@guido.python.org> > > If the community can *agree* on > > something they can support, I will listen. > > Even if it's []-after-args? If most people favor that over prefix @deco, sure, I'll give it another look. (It better come with an implementation though.) --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Fri Aug 20 22:12:10 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 20 22:12:02 2004 Subject: [Python-Dev] Reserved Characters In-Reply-To: <20040820191058.51944.qmail@web50708.mail.yahoo.com> References: <20040820191058.51944.qmail@web50708.mail.yahoo.com> Message-ID: <41265B1A.6010405@v.loewis.de> Jim J Jewett wrote: > One of the objections to @decorators is that it > will cause problems for applications that used > '@', simply because of it was one of only three > characters that *might* be available. I personally believe that the set of characters available to programming languages will extend beyond us-ascii sooner or later. So the number of available characters is several thousand. > Newer String interpolation will also use up '$', > which leaves only '?'. That is not true. $ will have a special meaning only inside string literals. It can be put into string literals already, so the lexical rules don't change at all with that change. > Is it time to reserve a character to the user, or > at least to reserve some in certain contexts? I don't think so. People who desire to put additional stuff into Python files should use Python comments. Indeed, #! already has a special meaning on some systems. Regards, Martin From fperez528 at yahoo.com Fri Aug 20 22:57:12 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Fri Aug 20 22:57:17 2004 Subject: [Python-Dev] Re: Re: Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> <1092802980.7810.29.camel@geddy.wooz.org> Message-ID: Note: I'm resending this, b/c I'd like to see this particular suggestion addressed as things seem to be finalizing on this PEP. My original message seems to have fallen below the radar, and I think it's a reasonable idea (or at least I would like to hear why it isn't :) Best to all, f ------ my original post: Barry Warsaw wrote: > On Wed, 2004-08-11 at 10:54, Fernando Perez wrote: >> It may be trivial to extend pep-292 for this, but not having to do it in >> every >> small utility you write/ship would be very useful, IMHO.??If?there?are?no >> technical reasons to prevent it, is the 'too complex' argument really that >> strong (especially since you state it's a trivial amount of code)? > > Well, it's about 3 lines of Python, but what would you call the class, > AttributeTemplate???I?wouldn't?be?opposed?to?adding?the?class,?since?I > think it would be pretty useful. I'd like to argue that this form may be the most useful for common tasks, so you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar" without having to worry too much about which template class you are using.?? How about making a BaseTemplate class which does NOT allow $foo.bar, and having the default Template implement attributes???This?would?give?us: -??A?default?class?covering?what?I?think?is?a?reasonable,?common-case?behavior.? I'm a big fan of covering most reasonable default behavior out-of-the-box, and I think python's stdlib achieves this very well in most cases.??I'd?argue this is one of the reasons for its success, and I have the feeling that in this case (PEP-292) the proposed default would be sub-par. - A clean base class for user-defined subclasses which want to be very exacting about what they want to implement or not. This sounds like a reasonable compromise to me, but ymmv. Best, and thanks for keeping an open mind about this issue (string interpolation is one of the very few areas where python's syntax bugs me, even after years of daily use). f From barry at python.org Fri Aug 20 23:17:21 2004 From: barry at python.org (Barry Warsaw) Date: Fri Aug 20 23:16:54 2004 Subject: [Python-Dev] Reserved Characters In-Reply-To: <20040820191058.51944.qmail@web50708.mail.yahoo.com> References: <20040820191058.51944.qmail@web50708.mail.yahoo.com> Message-ID: <1093036641.9258.64.camel@geddy.wooz.org> On Fri, 2004-08-20 at 15:10, Jim J Jewett wrote: > Newer String interpolation will also use up '$', > which leaves only '?'. Not true. PEP 292's $ characters are only special inside string literals, so there's really nothing new here. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/e4d695d4/attachment-0001.pgp From neal at metaslash.com Sat Aug 21 01:40:05 2004 From: neal at metaslash.com (Neal Norwitz) Date: Sat Aug 21 01:40:08 2004 Subject: [Python-Dev] Ignoring SIGPIPE (was: [Python-checkins] python/dist/src/Modules socketmodule.c, 1.298, 1.299) In-Reply-To: <412586CD.2000000@object-craft.com.au> References: <20040820020919.GG31470@epoch.metaslash.com> <20040820044436.0F8503C113@coffee.object-craft.com.au> <412586CD.2000000@object-craft.com.au> Message-ID: <20040820234005.GP31470@epoch.metaslash.com> On Fri, Aug 20, 2004 at 03:06:21PM +1000, Dave Cole wrote: > Andrew McNamara wrote: > >>>Patch #1003700: Add socketpair function to socket module. > >> > >>The docstring (below) states the arguments are the same as socket(). > >>However, in sock_initobj() line 2496, the family is initialized to > >>AF_INET. I think the #if defined(AF_UNIX) code above should be > >>removed and family should be initialized to AF_INET. > > > >I talked Dave into this - AF_UNIX is typically the only address family > >that is valid for socketpair(). Using AF_INET under linux and OS X > >results in EOPNOTSUPP. > > Would a change to the documentation and docstring be sufficient it it > explained that the default family is AF_UNIX if defined for the > platform, AF_INET otherwise? That would make me happy. I just don't want the doc/docstring to lie. I don't have a problem with it defaulting to AF_UNIX. > >>I don't think the #ifdef SIGPIPE code is correct. If the user > >>installed a signal handler calling signal() will remove it. I > >>think the call to signal() should be removed. > > > >I agree, but I think that was copied verbatim from elsewhere in > >socketmodule.c, so it was left with the aim of being bug for bug > >compatible with socket.socket(). > > Indeed. I was not going to change something as fundamental as the > existing signal handling. The onyl sensible thing to do was to copy the > behaviour already in place when you create a socket the "normal" way. You are correct. There are two places that call signal(SIGPIPE, SIG_IGN); I find this surprising. Guido, you added this code in version 1.4 from 1991. You do remember the reason why, don't you? It's only 13 years ago. :-) I'm hesitant to remove the old calls, but I'm not sure I want it added to new calls if it's not necessary. Any opinions? Neal From guido at python.org Sat Aug 21 03:11:37 2004 From: guido at python.org (Guido van Rossum) Date: Sat Aug 21 03:11:49 2004 Subject: [Python-Dev] Ignoring SIGPIPE (was: [Python-checkins] python/dist/src/Modules socketmodule.c, 1.298, 1.299) In-Reply-To: Your message of "Fri, 20 Aug 2004 19:40:05 EDT." <20040820234005.GP31470@epoch.metaslash.com> References: <20040820020919.GG31470@epoch.metaslash.com> <20040820044436.0F8503C113@coffee.object-craft.com.au> <412586CD.2000000@object-craft.com.au> <20040820234005.GP31470@epoch.metaslash.com> Message-ID: <200408210111.i7L1Bb623898@guido.python.org> > > >I agree, but I think that was copied verbatim from elsewhere in > > >socketmodule.c, so it was left with the aim of being bug for bug > > >compatible with socket.socket(). > > > > Indeed. I was not going to change something as fundamental as the > > existing signal handling. The onyl sensible thing to do was to copy the > > behaviour already in place when you create a socket the "normal" way. > > You are correct. There are two places that call > signal(SIGPIPE, SIG_IGN); I find this surprising. > > Guido, you added this code in version 1.4 from 1991. > You do remember the reason why, don't you? It's only > 13 years ago. :-) I'm hesitant to remove the old calls, > but I'm not sure I want it added to new calls if it's > not necessary. > > Any opinions? The only real requirement is that SIGPIPE is being ignored by default, and that is already taken care of by inisigs() in pythonrun.c. So I think both calls can be removed from socketmodule.c, heh heh. Originally, I was being careful, and not ignoring SIGPIPE until the first socket was created, hence the call there. The idea being that if you have a Python program that spits lots of stuff to stdout, and you pipe it into head (e.g.), it won't give you an exception -- the SIGPIPE will just kill it and the shell will ignore that; but if you're using sockets, it's important to ignore SIGPIPE so too bad for the other feature. But we've been ignoring SIGPIPE in the initialization forever, so the SIGPIPE call in socketmodule.c is really not needed any more. --Guido van Rossum (home page: http://www.python.org/~guido/) From fumanchu at amor.org Sat Aug 21 10:31:05 2004 From: fumanchu at amor.org (Robert Brewer) Date: Sat Aug 21 10:36:27 2004 Subject: [Python-Dev] python-dev Summary for 2004-08-01 through 2004-08-15[draft] Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E1D@exchange.hqamor.amorhq.net> Brett C. wrote: > Option two out of all of this is people just say, "summarize what you > want, Brett." Then I just cover what I find interesting and just don't > worry about covering a specific area. +1. Bring back "Skipped Threads" if you can, just for the sake of indexing completeness. Robert Brewer MIS Amor Ministries fumanchu@amor.org From pm_mon at yahoo.com Sat Aug 21 18:29:46 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sat Aug 21 18:29:53 2004 Subject: [Python-Dev] __metaclass__ and __author__ are already decorators Message-ID: [I posted this on comp.lang.python.general but I'm not sure how many of the folks here read that newsgroup, so my apologies if you're seeing this twice.] Thinking about decorators, and looking at what we are already doing in our Python code, it seems that __metaclass__, __author__, __version__, etc. are all examples of decorators. So we already have a decorator syntax. What is the compelling reason to invent a new one? And if we do, what's to become of the old one? Here's my take on this. We have two kinds of decorators: those that are informational only (like __author__ and __version__), and those which have side-effects, i.e. those that actually *do* something (like __metaclass__). class Foo: """ This describes the Foo class as normal. """ __metaclass__ = M __author__ = 'Paul Morrow' __version__ = '0.1' __automethods__ = True def baz(self, a, b): """ This describes the baz method. """ __synchronized__ = True __returns__ = None __author__ = 'Neville Shunt' # body of baz goes here... There, that looks pretty clear and pythonic. Now how to define the decorators. def metaclass(decoratedClass, subType): """ This describes the 'metaclass' decorator. """ __decorator__ = True __version__ = '1.9' # perform the metaclass operation on decoratedClass def synchronized(decoratedFunc, trueOrFalse): """ This describes the 'synchronized' decorator. """ __decorator__ = True __author__ = 'Martin Curry' __version__ = '0.5' # perform the synchronized operation on decoratedFunc def returns(decoratedFunc, *args): """Docstring for 'returns' decorator.""" __decorator__ = True # body of decorator goes here Each decorator function receives the class|function|method being decorated as the first parameter of the function (e.g. decoratedClass and decoratedFunc above). The remaining parameters are the values assigned to the __xxx__ variable in the definition of the decorated function. So, in the above example, when the 'returns' decorator function is called to decorate the 'baz' method, decoratedFunc would recieve the baz object and args[0] would be set to None (because of the statement "__returns__ = None" in the definition of baz). For a function to be used as a decorator function, it must be decorated as such, by setting its __decorator__ attribute to True. Does this handle enough of the decorator concerns? Paul From pje at telecommunity.com Sat Aug 21 18:54:37 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Aug 21 18:54:29 2004 Subject: [Python-Dev] __metaclass__ and __author__ are already decorators In-Reply-To: Message-ID: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> At 12:29 PM 8/21/04 -0400, Paul Morrow wrote: >[I posted this on comp.lang.python.general but I'm not sure how many of >the folks here read that newsgroup, so my apologies if you're seeing this >twice.] > >Thinking about decorators, and looking at what we are already doing in our >Python code, it seems that __metaclass__, __author__, __version__, etc. >are all examples of decorators. So we already have a decorator syntax. The items you describe are not decorators, they are attributes. The current syntax for decorators is 'x = decorator(x)'. > def baz(self, a, b): > """ This describes the baz method. """ > __synchronized__ = True > __returns__ = None > __author__ = 'Neville Shunt' > # body of baz goes here... This syntax already has a different and conflicting meaning in today's Python. A different such syntax: [synchronized(), returns(None), author("Neville Shunt")] def baz(self, a, b): """ This describes the baz method. """ # body of baz goes here... has already been rejected on the basis that this too has meaning in today's Python, and that the syntax produces "spooky action at a distance". That is, it's "magical" that the function calls affect the definition below. Following this logic, expecting assignments to magic names contained within a function's body to magically invoke decorator functions with different names *before and outside the block* where those names are assigned to in the code, seems far less likely to be considered acceptable. From pm_mon at yahoo.com Sat Aug 21 20:25:27 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sat Aug 21 20:25:43 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 12:29 PM 8/21/04 -0400, Paul Morrow wrote: > >> [I posted this on comp.lang.python.general but I'm not sure how many >> of the folks here read that newsgroup, so my apologies if you're >> seeing this twice.] >> >> Thinking about decorators, and looking at what we are already doing in >> our Python code, it seems that __metaclass__, __author__, __version__, >> etc. are all examples of decorators. So we already have a decorator >> syntax. > > > The items you describe are not decorators, they are attributes. The > current syntax for decorators is 'x = decorator(x)'. > In today's Python that's of course true, but these 'attributes' are used to supply meta info about the class. They *describe* the class. They are never intended to be inherited by instances of the class. So in this sense, they are very much more like decorators than (normal) class attributes. > >> def baz(self, a, b): >> """ This describes the baz method. """ >> __synchronized__ = True >> __returns__ = None >> __author__ = 'Neville Shunt' >> # body of baz goes here... > > > This syntax already has a different and conflicting meaning in today's > Python. Yes. There is a big conflict here. When I say def Foo: __author__ = 'me' __metaclass__ = M var1 = 10 var2 = 20 although I'm technically defining four class variables, my intention behind the first two is very different than my intention behind the second two. The first two supply meta information about Foo. It is *not* my intention that those variables be inherited by any instances of Foo, because they *don't apply* to instances of Foo --- they apply to Foo itself. They enhance the definition of Foo. They decorate Foo. This is true of all magic attributes (as far as I can tell), magic methods included. We define them right alongside instance methods. But we don't intend for the magic methods to be *inherited* by the instances so that they become part of the instances interface. We want to affect something much deeper than that (what happens when the instance is constructed, which operators may be used on the instance, etc.) So providing definitions for __xxx__ attributes is not done for the same purpose as the other attributes. They are special. Therefore, let's formally acknowledge that and call them 'decorators'. > > A different such syntax: > [synchronized(), returns(None), author("Neville Shunt")] > def baz(self, a, b): > """ This describes the baz method. """ > # body of baz goes here... > > has already been rejected on the basis that this too has meaning in > today's Python, and that the syntax produces "spooky action at a > distance". That is, it's "magical" that the function calls affect the > definition below. > The problem with this example isn't that there's magic involved, it's that 1) there's no precedent for idioms like the statement in your example to do anything special (to have magical side effects), and 2) its not intuitive that the declarations affect the upcoming def. I don't believe that we have an aversion to magic, where it's clear and intuitive what's going to happen. > Following this logic, expecting assignments to magic names contained > within a function's body to magically invoke decorator functions with > different names *before and outside the block* where those names are > assigned to in the code, seems far less likely to be considered acceptable. > 1. The decorator names aren't different than the decorator functions, they're the same (minus the leading and trailing double underscores). 2. All magic methods in today's Python are invoked 'magically' (are called indirectly; not called directly by any user code). 3. All of the interesting uses of decorators that I've seen proposed involve calling functions defined elsewhere (certainly outside of the current block of code). From chris.cavalaria at free.fr Sat Aug 21 22:14:07 2004 From: chris.cavalaria at free.fr (Christophe Cavalaria) Date: Sat Aug 21 22:14:12 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators References: Message-ID: Paul Morrow wrote: > class Foo: > """ This describes the Foo class as normal. """ > __metaclass__ = M > __author__ = 'Paul Morrow' > __version__ = '0.1' > __automethods__ = True > > > def baz(self, a, b): > """ This describes the baz method. """ > __synchronized__ = True > __returns__ = None > __author__ = 'Neville Shunt' > # body of baz goes here... It's a function call that masquerades as an attribute assignment. How worse can it be ? There's also the fact that it can't handle named parameters like a regular function call. You can't write that : def foo(): __decoration__ = (1,1,param=True) From pm_mon at yahoo.com Sat Aug 21 23:15:09 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sat Aug 21 23:15:15 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: Message-ID: Christophe Cavalaria wrote: > Paul Morrow wrote: > > >> class Foo: >> """ This describes the Foo class as normal. """ >> __metaclass__ = M >> __author__ = 'Paul Morrow' >> __version__ = '0.1' >> __automethods__ = True >> >> >> def baz(self, a, b): >> """ This describes the baz method. """ >> __synchronized__ = True >> __returns__ = None >> __author__ = 'Neville Shunt' >> # body of baz goes here... > > > It's a function call that masquerades as an attribute assignment. How worse > can it be ? How about we create new syntax that uses an @ and special words, where the words correspond with functions to be called. That's what is seriously being considered. And that would be worse (IMO). > can it be ? There's also the fact that it can't handle named parameters > like a regular function call. You can't write that : > > def foo(): > __decoration__ = (1,1,param=True) > As far as I know, we can't do that with the current decorator proposals either. [But that is something that I've often wanted to do (create a tuple that contains named arguments).] From kbk at shore.net Sat Aug 21 23:18:08 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Aug 21 23:18:11 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <4124FD72.4090901@v.loewis.de> (Martin v. =?iso-8859-1?q?L=F6wis's?= message of "Thu, 19 Aug 2004 21:20:18 +0200") References: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com> <16676.47823.382278.737098@montanaro.dyndns.org> <4124FD72.4090901@v.loewis.de> Message-ID: <87wtzs5gxb.fsf@hydra.localdomain> "Martin v. L?wis" writes: > Skip Montanaro wrote: >> Any thoughts about posting this to c.l.py as well as python-dev? Maybe it >> would encourage some more participation... > > +1. OK, if no one objects, I'll add c.l.p to the distribution. -- KBK From pje at telecommunity.com Sat Aug 21 23:21:28 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Aug 21 23:21:14 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: Message-ID: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >Christophe Cavalaria wrote: > >>can it be ? There's also the fact that it can't handle named parameters >>like a regular function call. You can't write that : >>def foo(): >> __decoration__ = (1,1,param=True) > >As far as I know, we can't do that with the current decorator proposals >either. @decoration(1,1,param=True) def foo(whatever): pass From pm_mon at yahoo.com Sat Aug 21 23:34:16 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sat Aug 21 23:34:21 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: > >> Christophe Cavalaria wrote: >> >>> can it be ? There's also the fact that it can't handle named parameters >>> like a regular function call. You can't write that : >>> def foo(): >>> __decoration__ = (1,1,param=True) >> >> >> As far as I know, we can't do that with the current decorator >> proposals either. > > > @decoration(1,1,param=True) > def foo(whatever): > pass > > Ok, then whatever changes you've made to the Python system to support that would allow the same syntax to be used in what I'm suggesting. def foo(whatever): __decoration__ = (1,1,param=True) def decoration(decoratedFunc, a, b, param=False): __decorator__ = True __version__ = '0.1' # body of 'decoration' decorator function goes here... From bob at redivi.com Sat Aug 21 23:47:18 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat Aug 21 23:48:01 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> Message-ID: On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote: > Phillip J. Eby wrote: > >> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >>> Christophe Cavalaria wrote: >>> >>>> can it be ? There's also the fact that it can't handle named >>>> parameters >>>> like a regular function call. You can't write that : >>>> def foo(): >>>> __decoration__ = (1,1,param=True) >>> >>> >>> As far as I know, we can't do that with the current decorator >>> proposals either. >> @decoration(1,1,param=True) >> def foo(whatever): >> pass > > Ok, then whatever changes you've made to the Python system to support > that would allow the same syntax to be used in what I'm suggesting. > > def foo(whatever): > __decoration__ = (1,1,param=True) > > def decoration(decoratedFunc, a, b, param=False): > __decorator__ = True > __version__ = '0.1' > # body of 'decoration' decorator function goes here... Congratulations, this by far the worst suggestion yet! I'm -Inf on this :) @decoration(1,1,param=True) makes no changes whatsoever to the Python system. Everything after the @ is really just an expression. The @ just implies that the result of the next class or def gets put through the result of the @expression before it is thrown into the namespace. -bob From pje at telecommunity.com Sat Aug 21 23:48:43 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Aug 21 23:48:27 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com> At 05:34 PM 8/21/04 -0400, Paul Morrow wrote: >Phillip J. Eby wrote: > >>At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >> >>>Christophe Cavalaria wrote: >>> >>>>can it be ? There's also the fact that it can't handle named parameters >>>>like a regular function call. You can't write that : >>>>def foo(): >>>> __decoration__ = (1,1,param=True) >>> >>> >>>As far as I know, we can't do that with the current decorator proposals >>>either. >> >>@decoration(1,1,param=True) >>def foo(whatever): >> pass > >Ok, then whatever changes you've made to the Python system to support that >would allow the same syntax to be used in what I'm suggesting. Huh? @decoration(1,1,param=True) is evaluated at the place where it appears. The *return value* of that expression is then called, passing in the foo function. In other words, the above is equivalent to today's: def foo(whatever): pass foo = decoration(1,1,param=True)(foo) except that the first assignment to 'foo' doesn't happen, only the second one. If the 'foo' function is a single expression, of course, today you can do the straightforward: foo = decoration(1,1,param=True)(lambda whatever: something) So, "@x func" is effectively a macro for "func = x(func)", where 'func' may be a function, or another decorator. That is: @x @y @z def foo(): ... is shorthand for 'foo = x(y(z(foo)))', no matter what the x/y/z expressions actually are. So there are *no* "changes to the Python system" here, just a very small amount of syntax sugar. By contrast, your attribute assignment approach isn't nearly so simple, as you will see if you attempt to write an implementation. From pm_mon at yahoo.com Sun Aug 22 00:15:43 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 00:15:50 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com> References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 05:34 PM 8/21/04 -0400, Paul Morrow wrote: > >> Phillip J. Eby wrote: >> >>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >>> >>>> Christophe Cavalaria wrote: >>>> >>>>> can it be ? There's also the fact that it can't handle named >>>>> parameters >>>>> like a regular function call. You can't write that : >>>>> def foo(): >>>>> __decoration__ = (1,1,param=True) >>>> >>>> >>>> >>>> As far as I know, we can't do that with the current decorator >>>> proposals either. >>> >>> >>> @decoration(1,1,param=True) >>> def foo(whatever): >>> pass >> >> >> Ok, then whatever changes you've made to the Python system to support >> that would allow the same syntax to be used in what I'm suggesting. > > > Huh? @decoration(1,1,param=True) is evaluated at the place where it > appears. The *return value* of that expression is then called, passing > in the foo function. In other words, the above is equivalent to today's: > > def foo(whatever): > pass > > foo = decoration(1,1,param=True)(foo) > > except that the first assignment to 'foo' doesn't happen, only the > second one. If the 'foo' function is a single expression, of course, > today you can do the straightforward: > > foo = decoration(1,1,param=True)(lambda whatever: something) > > So, "@x func" is effectively a macro for "func = x(func)", where 'func' > may be a function, or another decorator. That is: > > @x > @y > @z > def foo(): > ... > > is shorthand for 'foo = x(y(z(foo)))', Wouldn't that actually be shorthand for foo = x()(y()(z()(foo))) ? From bob at redivi.com Sun Aug 22 00:22:04 2004 From: bob at redivi.com (Bob Ippolito) Date: Sun Aug 22 00:22:51 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com> Message-ID: <87E9F4E0-F3C0-11D8-A072-000A95686CD8@redivi.com> On Aug 21, 2004, at 6:15 PM, Paul Morrow wrote: > Phillip J. Eby wrote: > >> At 05:34 PM 8/21/04 -0400, Paul Morrow wrote: >>> Phillip J. Eby wrote: >>> >>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >>>> >>>>> Christophe Cavalaria wrote: >>>>> >>>>>> can it be ? There's also the fact that it can't handle named >>>>>> parameters >>>>>> like a regular function call. You can't write that : >>>>>> def foo(): >>>>>> __decoration__ = (1,1,param=True) >>>>> >>>>> >>>>> >>>>> As far as I know, we can't do that with the current decorator >>>>> proposals either. >>>> >>>> >>>> @decoration(1,1,param=True) >>>> def foo(whatever): >>>> pass >>> >>> >>> Ok, then whatever changes you've made to the Python system to >>> support that would allow the same syntax to be used in what I'm >>> suggesting. >> Huh? @decoration(1,1,param=True) is evaluated at the place where it >> appears. The *return value* of that expression is then called, >> passing in the foo function. In other words, the above is equivalent >> to today's: >> def foo(whatever): >> pass >> foo = decoration(1,1,param=True)(foo) >> except that the first assignment to 'foo' doesn't happen, only the >> second one. If the 'foo' function is a single expression, of course, >> today you can do the straightforward: >> foo = decoration(1,1,param=True)(lambda whatever: something) >> So, "@x func" is effectively a macro for "func = x(func)", where >> 'func' may be a function, or another decorator. That is: >> @x >> @y >> @z >> def foo(): >> ... >> is shorthand for 'foo = x(y(z(foo)))', > > Wouldn't that actually be shorthand for foo = x()(y()(z()(foo))) ? No. -bob From pm_mon at yahoo.com Sun Aug 22 00:24:09 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 00:24:16 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> Message-ID: Bob Ippolito wrote: > > On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote: > >> Phillip J. Eby wrote: >> >>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >>> >>>> Christophe Cavalaria wrote: >>>> >>>>> can it be ? There's also the fact that it can't handle named >>>>> parameters >>>>> like a regular function call. You can't write that : >>>>> def foo(): >>>>> __decoration__ = (1,1,param=True) >>>> >>>> >>>> >>>> As far as I know, we can't do that with the current decorator >>>> proposals either. >>> >>> @decoration(1,1,param=True) >>> def foo(whatever): >>> pass >> >> >> Ok, then whatever changes you've made to the Python system to support >> that would allow the same syntax to be used in what I'm suggesting. >> >> def foo(whatever): >> __decoration__ = (1,1,param=True) >> >> def decoration(decoratedFunc, a, b, param=False): >> __decorator__ = True >> __version__ = '0.1' >> # body of 'decoration' decorator function goes here... > > > Congratulations, this by far the worst suggestion yet! I'm -Inf on this :) > Thanks. > @decoration(1,1,param=True) makes no changes whatsoever to the Python > system. Everything after the @ is really just an expression. The @ > just implies that the result of the next class or def gets put through > the result of the @expression before it is thrown into the namespace. > It seems that writing a decorator is going to be a bizarre experience. In the example, I would need to write a function named 'decoration' that returns a function that will recieve a function (foo) to be decorated and then return a function. Does that sound about right? What would functions like 'decoration' typically look like? Could you show a short code snippet? From bob at redivi.com Sun Aug 22 00:28:37 2004 From: bob at redivi.com (Bob Ippolito) Date: Sun Aug 22 00:29:13 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> Message-ID: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote: > Bob Ippolito wrote: > >> On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote: >>> Phillip J. Eby wrote: >>> >>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >>>> >>>>> Christophe Cavalaria wrote: >>>>> >>>>>> can it be ? There's also the fact that it can't handle named >>>>>> parameters >>>>>> like a regular function call. You can't write that : >>>>>> def foo(): >>>>>> __decoration__ = (1,1,param=True) >>>>> >>>>> >>>>> >>>>> As far as I know, we can't do that with the current decorator >>>>> proposals either. >>>> >>>> @decoration(1,1,param=True) >>>> def foo(whatever): >>>> pass >>> >>> >>> Ok, then whatever changes you've made to the Python system to >>> support that would allow the same syntax to be used in what I'm >>> suggesting. >>> >>> def foo(whatever): >>> __decoration__ = (1,1,param=True) >>> >>> def decoration(decoratedFunc, a, b, param=False): >>> __decorator__ = True >>> __version__ = '0.1' >>> # body of 'decoration' decorator function goes here... >> Congratulations, this by far the worst suggestion yet! I'm -Inf on >> this :) > > Thanks. > > >> @decoration(1,1,param=True) makes no changes whatsoever to the Python >> system. Everything after the @ is really just an expression. The @ >> just implies that the result of the next class or def gets put >> through the result of the @expression before it is thrown into the >> namespace. > > It seems that writing a decorator is going to be a bizarre experience. > In the example, I would need to write a function named 'decoration' > that returns a function that will recieve a function (foo) to be > decorated and then return a function. Does that sound about right? Yes that is correct. > What would functions like 'decoration' typically look like? Could you > show a short code snippet? http://python.org/peps/pep-0318.html -bob From pm_mon at yahoo.com Sun Aug 22 00:29:56 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 00:30:01 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> Message-ID: Paul Morrow wrote: > Bob Ippolito wrote: > >> >> On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote: >> >>> Phillip J. Eby wrote: >>> >>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >>>> >>>>> Christophe Cavalaria wrote: >>>>> >>>>>> can it be ? There's also the fact that it can't handle named >>>>>> parameters >>>>>> like a regular function call. You can't write that : >>>>>> def foo(): >>>>>> __decoration__ = (1,1,param=True) >>>>> >>>>> >>>>> >>>>> >>>>> As far as I know, we can't do that with the current decorator >>>>> proposals either. >>>> >>>> >>>> @decoration(1,1,param=True) >>>> def foo(whatever): >>>> pass >>> >>> >>> >>> Ok, then whatever changes you've made to the Python system to support >>> that would allow the same syntax to be used in what I'm suggesting. >>> >>> def foo(whatever): >>> __decoration__ = (1,1,param=True) >>> >>> def decoration(decoratedFunc, a, b, param=False): >>> __decorator__ = True >>> __version__ = '0.1' >>> # body of 'decoration' decorator function goes here... >> >> >> >> Congratulations, this by far the worst suggestion yet! I'm -Inf on >> this :) >> > > Thanks. > > >> @decoration(1,1,param=True) makes no changes whatsoever to the Python >> system. Everything after the @ is really just an expression. The @ >> just implies that the result of the next class or def gets put through >> the result of the @expression before it is thrown into the namespace. >> > > It seems that writing a decorator is going to be a bizarre experience. > In the example, I would need to write a function named 'decoration' that > returns a function that will recieve a function (foo) to be decorated > and then return a function. Does that sound about right? > > What would functions like 'decoration' typically look like? Could you > show a short code snippet? > Nevermind. I got wrapped around the axle. I get it now. Thanks. From pm_mon at yahoo.com Sun Aug 22 00:28:36 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 00:30:49 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <87E9F4E0-F3C0-11D8-A072-000A95686CD8@redivi.com> References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com> <87E9F4E0-F3C0-11D8-A072-000A95686CD8@redivi.com> Message-ID: Bob Ippolito wrote: > > On Aug 21, 2004, at 6:15 PM, Paul Morrow wrote: > >> Phillip J. Eby wrote: >> >>> At 05:34 PM 8/21/04 -0400, Paul Morrow wrote: >>> >>>> Phillip J. Eby wrote: >>>> >>>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote: >>>>> >>>>>> Christophe Cavalaria wrote: >>>>>> >>>>>>> can it be ? There's also the fact that it can't handle named >>>>>>> parameters >>>>>>> like a regular function call. You can't write that : >>>>>>> def foo(): >>>>>>> __decoration__ = (1,1,param=True) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> As far as I know, we can't do that with the current decorator >>>>>> proposals either. >>>>> >>>>> >>>>> >>>>> @decoration(1,1,param=True) >>>>> def foo(whatever): >>>>> pass >>>> >>>> >>>> >>>> Ok, then whatever changes you've made to the Python system to >>>> support that would allow the same syntax to be used in what I'm >>>> suggesting. >>> >>> Huh? @decoration(1,1,param=True) is evaluated at the place where it >>> appears. The *return value* of that expression is then called, >>> passing in the foo function. In other words, the above is equivalent >>> to today's: >>> def foo(whatever): >>> pass >>> foo = decoration(1,1,param=True)(foo) >>> except that the first assignment to 'foo' doesn't happen, only the >>> second one. If the 'foo' function is a single expression, of course, >>> today you can do the straightforward: >>> foo = decoration(1,1,param=True)(lambda whatever: something) >>> So, "@x func" is effectively a macro for "func = x(func)", where >>> 'func' may be a function, or another decorator. That is: >>> @x >>> @y >>> @z >>> def foo(): >>> ... >>> is shorthand for 'foo = x(y(z(foo)))', >> >> >> Wouldn't that actually be shorthand for foo = x()(y()(z()(foo))) ? > > > No. > Ok, I see, nevermind. Thanks. From pm_mon at yahoo.com Sun Aug 22 00:54:28 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 00:54:34 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> Message-ID: Bob Ippolito wrote: > > On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote: > >> >> It seems that writing a decorator is going to be a bizarre experience. >> In the example, I would need to write a function named 'decoration' >> that returns a function that will recieve a function (foo) to be >> decorated and then return a function. Does that sound about right? > > > Yes that is correct. > >> What would functions like 'decoration' typically look like? Could you >> show a short code snippet? > > > http://python.org/peps/pep-0318.html > Thanks. Of the 5 examples there, the first two are apparently not implemented correctly, as they expect that the function/class to be decorated is passed directly to them, rather than to the function they return. Would you agree? I pasted them here for your consideration... 1. Define a function to be executed at exit. Note that the function isn't actually "wrapped" in the usual sense. def onexit(f): import atexit atexit.register(f) return f @onexit def func(): ... 2. Define a class with a singleton instance. Note that once the class disappears enterprising programmers would have to be more creative to create more instances. (From Shane Hathaway on python-dev.) def singleton(cls): instances = {} def getinstance(): if cls not in instances: instances[cls] = cls() return instances[cls] return getinstance @singleton class MyClass: ... From bob at redivi.com Sun Aug 22 01:11:12 2004 From: bob at redivi.com (Bob Ippolito) Date: Sun Aug 22 01:11:50 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> Message-ID: <64E90FD9-F3C7-11D8-A072-000A95686CD8@redivi.com> On Aug 21, 2004, at 6:54 PM, Paul Morrow wrote: > Bob Ippolito wrote: > >> On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote: >>> >>> It seems that writing a decorator is going to be a bizarre >>> experience. In the example, I would need to write a function named >>> 'decoration' that returns a function that will recieve a function >>> (foo) to be decorated and then return a function. Does that sound >>> about right? >> Yes that is correct. >>> What would functions like 'decoration' typically look like? Could >>> you show a short code snippet? >> http://python.org/peps/pep-0318.html > > Thanks. Of the 5 examples there, the first two are apparently not > implemented correctly, as they expect that the function/class to be > decorated is passed directly to them, rather than to the function they > return. Would you agree? I pasted them here for your > consideration... No, they are correct. You are confused. What is expected is that the result of the expression after @ is callable and takes one parameter. If the expression after @ is just a name, then nothing particularly exciting happens at that time. @bar # NOTE THE LACK OF PARENTHESES def foo(): .... is equivalent to: _tmp = bar def foo(): .... foo = _tmp(foo) _tmp = bar clearly doesn't do anything special However, you're confusing that with examples that look more like: @bar() # NOTE THE PARENTHESES def foo(): .... which are equivalent to: _tmp = bar() def foo(): .... foo = _tmp(foo) -bob From pje at telecommunity.com Sun Aug 22 01:12:57 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun Aug 22 01:12:42 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> Message-ID: <5.1.1.6.0.20040821190916.03c6e610@mail.telecommunity.com> At 06:54 PM 8/21/04 -0400, Paul Morrow wrote: >Thanks. Of the 5 examples there, the first two are apparently not >implemented correctly, as they expect that the function/class to be >decorated is passed directly to them, rather than to the function they >return. Would you agree? I pasted them here for your consideration... They're correct. You're missing the fact that '@x' and '@x()' are not the same thing. '@x' means 'func=x(func)', while '@x()' means 'func = x()(func)'. There's no inconsistency here at all, it's just ordinary Python semantics. The only time a decorator needs to return a function is if it needs arguments other than the function being decorated. In which case, it might properly be termed a decorator factory, i.e. a function returning a decorator. Thus, in the '@' syntax, a decorator expression is always either '@decorator' or '@decorator_factory(arguments)'. From pm_mon at yahoo.com Sun Aug 22 01:25:30 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 01:25:39 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <5.1.1.6.0.20040821190916.03c6e610@mail.telecommunity.com> References: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> <5.1.1.6.0.20040821190916.03c6e610@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 06:54 PM 8/21/04 -0400, Paul Morrow wrote: > >> Thanks. Of the 5 examples there, the first two are apparently not >> implemented correctly, as they expect that the function/class to be >> decorated is passed directly to them, rather than to the function they >> return. Would you agree? I pasted them here for your consideration... > > > They're correct. You're missing the fact that '@x' and '@x()' are not > the same thing. '@x' means 'func=x(func)', while '@x()' means 'func = > x()(func)'. There's no inconsistency here at all, it's just ordinary > Python semantics. > > The only time a decorator needs to return a function is if it needs > arguments other than the function being decorated. In which case, it > might properly be termed a decorator factory, i.e. a function returning > a decorator. Thus, in the '@' syntax, a decorator expression is always > either '@decorator' or '@decorator_factory(arguments)'. > That's clear. Thank you. From pm_mon at yahoo.com Sun Aug 22 01:28:01 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 01:30:54 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <64E90FD9-F3C7-11D8-A072-000A95686CD8@redivi.com> References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com> <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com> <64E90FD9-F3C7-11D8-A072-000A95686CD8@redivi.com> Message-ID: Bob Ippolito wrote: > > On Aug 21, 2004, at 6:54 PM, Paul Morrow wrote: > >> Bob Ippolito wrote: >> >>> On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote: >>> >>>> >>>> It seems that writing a decorator is going to be a bizarre >>>> experience. In the example, I would need to write a function named >>>> 'decoration' that returns a function that will recieve a function >>>> (foo) to be decorated and then return a function. Does that sound >>>> about right? >>> >>> Yes that is correct. >>> >>>> What would functions like 'decoration' typically look like? Could >>>> you show a short code snippet? >>> >>> http://python.org/peps/pep-0318.html >> >> >> Thanks. Of the 5 examples there, the first two are apparently not >> implemented correctly, as they expect that the function/class to be >> decorated is passed directly to them, rather than to the function they >> return. Would you agree? I pasted them here for your consideration... > > > No, they are correct. You are confused. What is expected is that the > result of the expression after @ is callable and takes one parameter. > If the expression after @ is just a name, then nothing particularly > exciting happens at that time. > > @bar # NOTE THE LACK OF PARENTHESES > def foo(): > .... > > is equivalent to: > > _tmp = bar > def foo(): > .... > foo = _tmp(foo) > > _tmp = bar clearly doesn't do anything special > > However, you're confusing that with examples that look more like: > > @bar() # NOTE THE PARENTHESES > def foo(): > .... > > which are equivalent to: > > _tmp = bar() > def foo(): > .... > foo = _tmp(foo) > > -bob Yes, I was missing that. Thanks for the detailed explanation. From martin at v.loewis.de Sun Aug 22 09:21:39 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 22 09:21:43 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> Message-ID: <41284983.3000303@v.loewis.de> Paul Morrow wrote: >> The items you describe are not decorators, they are attributes. The >> current syntax for decorators is 'x = decorator(x)'. >> > > In today's Python that's of course true, but these 'attributes' are used > to supply meta info about the class. They *describe* the class. They > are never intended to be inherited by instances of the class. So in > this sense, they are very much more like decorators than (normal) class > attributes. No, that makes them completely unlike function decorators. Function decorators do not describe something, they modify something (namely, the thing that is bound to the function's name). Some attributes have a run-time meaning, like __metaclass__; others, like __author__, have not. Unfortunately, the meaning of __metaclass__ is conceptually different from decorators: the metaclass is evaluated *before* the class is constructed; the decorators are evaluated *after* the function is constructed. > So providing definitions for __xxx__ attributes is not done for the same > purpose as the other attributes. They are special. Therefore, let's > formally acknowledge that and call them 'decorators'. I acknowledge that they are special, and call them "special attributes". I won't call that decorators, because that means something else. > 2. All magic methods in today's Python are invoked 'magically' (are > called indirectly; not called directly by any user code). Correct. For each special attribute, the interpreter needs advance knowledge of the name of the attribute. Not so for decorators: the interpreter does not need advance knowledge - you can define your own function decorators, and Python will support them because of the leading @. Regards, Martin From jlgijsbers at planet.nl Sun Aug 22 13:08:35 2004 From: jlgijsbers at planet.nl (Johannes Gijsbers) Date: Sun Aug 22 13:07:10 2004 Subject: [Python-Dev] Adding 'lexists()' to os.path Message-ID: <20040822110835.GA7452@mail.planet.nl> I was just reviewing the patch at http://python.org/sf/941486, which proposes adding a new 'lexists()' function to os.path. This function would return True for dangling symbolic links, unlike os.path.exists() which returns False. One could use 'os.path.islink(path) or os.path.exists()' as well, but that's two stat calls instead of one. This function is useful and efficient for fixing a bug in glob.py (which the patch does as well) and it seems like it could be useful in tarfile.py and tempfile.py as well. Also, a high-quality patch is already available. So, any objections to adding this to os.path? Johannes From martin at v.loewis.de Sun Aug 22 14:00:20 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 22 14:00:24 2004 Subject: [Python-Dev] Adding 'lexists()' to os.path In-Reply-To: <20040822110835.GA7452@mail.planet.nl> References: <20040822110835.GA7452@mail.planet.nl> Message-ID: <41288AD4.7060407@v.loewis.de> Johannes Gijsbers wrote: > This function is useful and efficient for fixing a bug in glob.py (which the > patch does as well) and it seems like it could be useful in tarfile.py and > tempfile.py as well. Also, a high-quality patch is already available. So, any > objections to adding this to os.path? Go ahead. "Equivallent" contains a typo, though. Martin From pf_moore at yahoo.co.uk Sun Aug 22 15:17:00 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Sun Aug 22 15:17:02 2004 Subject: [Python-Dev] Re: Adding 'lexists()' to os.path References: <20040822110835.GA7452@mail.planet.nl> Message-ID: Johannes Gijsbers writes: > I was just reviewing the patch at http://python.org/sf/941486, which proposes > adding a new 'lexists()' function to os.path. This function would return True > for dangling symbolic links, unlike os.path.exists() which returns False. One > could use 'os.path.islink(path) or os.path.exists()' as well, but that's two > stat calls instead of one. As an alternative, would there be any value in cacheing the last stat result? In a sequence like this, with multiple calls for the same argument, many stat calls would be replaced by one. And it's likely that the checking overhead on cache misses would be minimal compared to the cost of the stat call, so it's an overall win. Of course, actual tests beat idle speculation like this... Paul. -- Any sufficiently advanced technology is indistinguishable from magic. -- Arthur C. Clarke From pm_mon at yahoo.com Sun Aug 22 16:15:11 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 16:15:17 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <41284983.3000303@v.loewis.de> References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Paul Morrow wrote: > >>> The items you describe are not decorators, they are attributes. The >>> current syntax for decorators is 'x = decorator(x)'. >>> >> >> In today's Python that's of course true, but these 'attributes' are >> used to supply meta info about the class. They *describe* the class. >> They are never intended to be inherited by instances of the class. So >> in this sense, they are very much more like decorators than (normal) >> class attributes. > > > No, that makes them completely unlike function decorators. Function > decorators do not describe something, they modify something (namely, > the thing that is bound to the function's name). > That's a terrible way to 'think' about this kind of programming --- programming thru imperative mutation. Ugh. It's much better to say "this function is synchronized and memoized" than to say "apply the synchronized mutator to this function, then apply the memoized mutator to the resulting function". So regardless of what is going on under the hood, it's better to think of this technique as a function annotation --- something additionally stated in its description --- than something that we *do* to the function. > Some attributes have a run-time meaning, like __metaclass__; others, > like __author__, have not. Unfortunately, the meaning of __metaclass__ > is conceptually different from decorators: the metaclass is evaluated > *before* the class is constructed; the decorators are evaluated > *after* the function is constructed. > *When* a feature gets added/applied to a class/function shouldn't matter to the programmer. We've screwed up if we make them think that hard. They should simply be able to state which features they want a class/function to have (as they can now with __metaclass__, __lt__, etc.), and the system makes sure they are there when the class/function is called. It's magical, but its easy to think about. >> So providing definitions for __xxx__ attributes is not done for the >> same purpose as the other attributes. They are special. Therefore, >> let's formally acknowledge that and call them 'decorators'. > > > I acknowledge that they are special, and call them "special attributes". > I won't call that decorators, because that means something else. > But it shouldn't, because they ("special attributes" and "decorators") fundamentally have the same purpose --- to make something magical happen to the function/class. You're getting hung up on what's going on under the covers and looking at the differences there. That should be irrelevant to the programmer. It's magic. All they need to know is what magic happens when they use an __xxx__ attribute and what they must do to use it properly --- and the later should be as easy as possible. >> 2. All magic methods in today's Python are invoked 'magically' (are >> called indirectly; not called directly by any user code). > > > Correct. For each special attribute, the interpreter needs advance > knowledge of the name of the attribute. Not so for decorators: > the interpreter does not need advance knowledge - you can define > your own function decorators, and Python will support them because > of the leading @. > Another way to look at that is that the interpretor has some built-in magic and you can create your own. But it's all still magic. And that makes it easier to think about. From jepler at unpythonic.net Sun Aug 22 16:18:19 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Sun Aug 22 16:18:31 2004 Subject: [Python-Dev] Re: Adding 'lexists()' to os.path In-Reply-To: References: <20040822110835.GA7452@mail.planet.nl> Message-ID: <20040822141819.GC2348@unpythonic.net> On Sun, Aug 22, 2004 at 02:17:00PM +0100, Paul Moore wrote: > As an alternative, would there be any value in cacheing the last stat > result? Well, code like this would break: def wait_for_change(filename, delay=.1): stamp = os.stat(filename).st_mtime while 1: time.sleep(delay) if os.stat(filename).st_mtime != stamp: break Hm ... >>> import statcache /usr/lib/python2.3/statcache.py:8: DeprecationWarning: The statcache module is obsolete. Use os.stat() instead. Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/c4236a3d/attachment.pgp From tino.lange at isg.de Sun Aug 22 16:25:24 2004 From: tino.lange at isg.de (Tino Lange) Date: Sun Aug 22 16:25:26 2004 Subject: [Python-Dev] Re: Adding 'lexists()' to os.path In-Reply-To: References: <20040822110835.GA7452@mail.planet.nl> Message-ID: <4128ACD4.1090609@isg.de> Paul Moore wrote: > >>I was just reviewing the patch at http://python.org/sf/941486, which proposes >>adding a new 'lexists()' function to os.path. This function would return True >>for dangling symbolic links, unlike os.path.exists() which returns False. One >>could use 'os.path.islink(path) or os.path.exists()' as well, but that's two >>stat calls instead of one. > > > As an alternative, would there be any value in cacheing the last stat > result? In a sequence like this, with multiple calls for the same > argument, many stat calls would be replaced by one. And it's likely > that the checking overhead on cache misses would be minimal compared > to the cost of the stat call, so it's an overall win. Hi Paul, you will never know how long the cached result is true. Files on disk can change outside your python script.... I doubt that we can find a good expire time for the caching? Someone will come up with an application in which this caching will give wrong results... If someone knows what he does and likes efficient programming he can cache the result of os.stat() himself and work with that. (Like this lexists() function is programmed internally...) My personal opinion: +0.5 for os.lexists() [it's useful, but we can also live without it...] -1 for a stat cache in os.*. Cheers, Tino From martin at v.loewis.de Sun Aug 22 16:58:39 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 22 16:58:42 2004 Subject: [Python-Dev] Re: Adding 'lexists()' to os.path In-Reply-To: References: <20040822110835.GA7452@mail.planet.nl> Message-ID: <4128B49F.3020505@v.loewis.de> Paul Moore wrote: > As an alternative, would there be any value in cacheing the last stat > result? That won't help (in addition to the problems it causes). One of the stat calls must be stat(2), the other one lstat(2). Regards, Martin From martin at v.loewis.de Sun Aug 22 17:10:48 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 22 17:10:52 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> Message-ID: <4128B778.1080503@v.loewis.de> Paul Morrow wrote: > That's a terrible way to 'think' about this kind of programming --- > programming thru imperative mutation. This is Pythonic. In Python, we don't have any declarations (well, perhaps except for global). Instead, everything in a module is executed, including class and def statements. This is an important property, as it allows: - conditional function definition: if some_condition: def foo():some_code else: def foo():other_code - computed inheritance: if some_condition: base = B1 else: base = B2 class D(base): body - and more Decorators extend this pattern. > So regardless of what is going on under > the hood, it's better to think of this technique as a function > annotation --- something additionally stated in its description --- than > something that we *do* to the function. No. It is best to simultaneously think of this as descriptive *and* imperative. I.e. for most purposes, it is sufficient to consider it descriptive, but one always needs to be aware when it is more correct to consider it imperative. For example, you have to understand that @foo def bar():self might cause bar *not* to be a function. This can only be understood if you don't think of decorators as kind of attribute. > *When* a feature gets added/applied to a class/function shouldn't matter > to the programmer. We've screwed up if we make them think that hard. Depends on the situation. In some cases, it does matter, and then the programmer needs to think hard. However, she can only succeed in that case if she is even *aware* that the declarative view on these things is a simplification. > They should simply be able to state which features they want a > class/function to have (as they can now with __metaclass__, __lt__, > etc.), and the system makes sure they are there when the class/function > is called. It's magical, but its easy to think about. Yes, it works most of the time. If this simplification is wrong, it gets so terribly wrong that the programmer needs somebody else who really understands it solves the problem. To avoid that, we should not try to hide the semantics, but make it obvious - and simultaneously explain that in most cases, the simplification will serve fine. > That should be > irrelevant to the programmer. It's magic. No, no, no. In a well-designed programming language, there is no place for magic, in the sense that what really happens is deliberately hidden for nobody to find. Instead, people should *know* the inner workings. Many people know the inner workings of the current language, and they can easily extend their mental model to decorators. Regards, Martin From pm_mon at yahoo.com Sun Aug 22 18:10:33 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 18:10:44 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <4128B778.1080503@v.loewis.de> References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> <4128B778.1080503@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Paul Morrow wrote: > >> That's a terrible way to 'think' about this kind of programming --- >> programming thru imperative mutation. > > > This is Pythonic. In Python, we don't have any declarations (well, > perhaps except for global). Instead, everything in a module is > executed, including class and def statements. This is an important > property, as it allows: > - conditional function definition: > if some_condition: > def foo():some_code > else: > def foo():other_code > - computed inheritance: > if some_condition: > base = B1 > else: > base = B2 > > class D(base): > body > - and more > > Decorators extend this pattern. > That pattern (conditional definitions) is a recipe for disaster, and you should not use it if it can be avoided. It's always best to use a declarative solution whenever possible and not disturb the natural resolution mechanism. What you show is possible in Python (gosh, what isn't), but it does not 'define' Python. It's not a salient feature of Python. So it is not Pythonic (IMO). >> So regardless of what is going on under the hood, it's better to think >> of this technique as a function annotation --- something additionally >> stated in its description --- than something that we *do* to the >> function. > > > No. It is best to simultaneously think of this as descriptive *and* > imperative. I.e. for most purposes, it is sufficient to consider it > descriptive, but one always needs to be aware when it is more correct > to consider it imperative. For example, you have to understand that > > @foo > def bar():self > > might cause bar *not* to be a function. This can only be understood > if you don't think of decorators as kind of attribute. > Another recipe for disaster. If foo can change bar into something that is no longer a function, then you can create defects that are unacceptably hard to find. Never use that pattern either, if there is a less 'fancy' alternative (and there almost always will be). >> *When* a feature gets added/applied to a class/function shouldn't >> matter to the programmer. We've screwed up if we make them think that >> hard. > > > Depends on the situation. In some cases, it does matter, and then the > programmer needs to think hard. However, she can only succeed in that > case if she is even *aware* that the declarative view on these things > is a simplification. > Using magic is a kind of declarative programming. But note to all programmers: magic isn't really 'magic', there's actually an underlying mechanism making the magic happen. You can alter the way magic works if you really, really, really need to. But we encourage you to first see if there isn't a way to solve your problem that doesn't change the magic (so that others have a prayer of understanding what you're doing). >> They should simply be able to state which features they want a >> class/function to have (as they can now with __metaclass__, __lt__, >> etc.), and the system makes sure they are there when the >> class/function is called. It's magical, but its easy to think about. > > > Yes, it works most of the time. If this simplification is wrong, it > gets so terribly wrong that the programmer needs somebody else who > really understands it solves the problem. To avoid that, we should not > try to hide the semantics, but make it obvious - and simultaneously > explain that in most cases, the simplification will serve fine. > No, 'how' magic works (details of the innerworkings of the interpreter) isn't necessary for most programmers to understand (unless of course you're working on the interpreter or in some other way trying to extend the Python system). They only need to know what each type of magic does and how to use it properly. That needs to be thoroughly documented. But the implementation should only be of interest to the low-level developers of the Python system (with rare exceptions). >> That should be irrelevant to the programmer. It's magic. > > > No, no, no. In a well-designed programming language, there is no place > for magic, in the sense that what really happens is deliberately hidden > for nobody to find. Instead, people should *know* the inner workings. > Many people know the inner workings of the current language, and they > can easily extend their mental model to decorators. > Well, define magic. If you mean automatic behavior, then I strongly disagree. You need the system to do things automatically for you, otherwise you're writing in nothing more than assembly language. What kind of magic are you referring to? From martin at v.loewis.de Sun Aug 22 18:53:47 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 22 18:53:51 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> <4128B778.1080503@v.loewis.de> Message-ID: <4128CF9B.4070107@v.loewis.de> Paul Morrow wrote: >> @foo >> def bar():self >> >> might cause bar *not* to be a function. This can only be understood >> if you don't think of decorators as kind of attribute. >> > > Another recipe for disaster. If foo can change bar into something that > is no longer a function, then you can create defects that are > unacceptably hard to find. Never use that pattern either, if there is a > less 'fancy' alternative (and there almost always will be). Yet, this is what happens for staticmethod and classmethod: the result of applying the decorator will cause bar not to be a function anymore (but a staticmethod or a classmethod, respectively). There is no alternative for this. > No, 'how' magic works (details of the innerworkings of the interpreter) > isn't necessary for most programmers to understand (unless of course > you're working on the interpreter or in some other way trying to extend > the Python system). They only need to know what each type of magic does > and how to use it properly. That needs to be thoroughly documented. But > the implementation should only be of interest to the low-level > developers of the Python system (with rare exceptions). In the case of decorators, atleast anybody designing one should know how this works. They are intentionally extensible, so many more than the Python developers proper will need to know how decorators work. > Well, define magic. If you mean automatic behavior, then I strongly > disagree. You need the system to do things automatically for you, > otherwise you're writing in nothing more than assembly language. What > kind of magic are you referring to? Your proposal specifically: use __foo__ for decorators, instead of @foo. You haven't actually specified how this works, but merely asserted that it will work, and (apparently) declared important details as irrelevant implementation details. Regards, Martin From guido at python.org Sun Aug 22 20:58:31 2004 From: guido at python.org (Guido van Rossum) Date: Sun Aug 22 20:58:37 2004 Subject: [Python-Dev] Adding 'lexists()' to os.path In-Reply-To: Your message of "Sun, 22 Aug 2004 13:08:35 +0200." <20040822110835.GA7452@mail.planet.nl> References: <20040822110835.GA7452@mail.planet.nl> Message-ID: <200408221858.i7MIwV502491@guido.python.org> > I was just reviewing the patch at http://python.org/sf/941486, which > proposes adding a new 'lexists()' function to os.path. This function > would return True for dangling symbolic links, unlike > os.path.exists() which returns False. One could use > 'os.path.islink(path) or os.path.exists()' as well, but that's two > stat calls instead of one. > > This function is useful and efficient for fixing a bug in glob.py > (which the patch does as well) and it seems like it could be useful > in tarfile.py and tempfile.py as well. Also, a high-quality patch is > already available. So, any objections to adding this to os.path? Looks like you already got the go-ahead. Just make sure with any API changes/additions to os.path, that there are many individual *path modules that need to be updated -- if a platform doesn't support symlinks, lexists() should probably just be an alias for exists(). posixpath.py, ntpath.py, macpath.py, os2emxpath.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Aug 22 21:02:54 2004 From: guido at python.org (Guido van Rossum) Date: Sun Aug 22 21:03:00 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: Your message of "Sun, 22 Aug 2004 12:10:33 EDT." References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> <4128B778.1080503@v.loewis.de> Message-ID: <200408221902.i7MJ2sd02547@guido.python.org> Who is Paul Morrow and why is he asserting what is Pythonic and isn't? --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Sun Aug 22 21:17:21 2004 From: barry at python.org (Barry Warsaw) Date: Sun Aug 22 21:17:04 2004 Subject: [Python-Dev] Re: Re: Re: Update PEP 292 In-Reply-To: References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> <1092802980.7810.29.camel@geddy.wooz.org> Message-ID: <1093202241.25618.19.camel@geddy.wooz.org> On Wed, 2004-08-18 at 14:07, Fernando Perez wrote: > I'd like to argue that this form may be the most useful for common tasks, so > you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar" > without having to worry too much about which template class you are using. It might be, but using attribute lookup syntax can be problematic. I can only relate my experiences with Mailman, where I've been using something similar for several years, albeit with traditional %-placeholders. With attribute placeholders, you typically want the mapping to be automatically created from the call site's locals and globals, otherwise you end up having to type many things three times. That's fine, but there are gotchas. First, the exact behavior of the mapping is somewhat application dependent (do you want locals only, globals and locals, only some specially named attributes in those namespaces, etc.). That makes it difficult to provide a generally useful mapping implementation in the standard library. And without a special mapping instance, I think attribute placeholders are less useful. Second, you want to be really careful about the context that attributes are looked up in. For example, in a CGI context, you can't simply expose the names of variables that hold data extracted from the form, otherwise you open yourself up to cross-site scripting attacks. Then, as is the case with Mailman, if the template string is really a key into a translation catalog, you have to be careful that your translators aren't accidentally or intentionally exposing unsafe objects through their translated message strings. So at the very least, I don't think we have enough application independent use cases to know what kinds of attribute path template support should go in the standard library. That may change a year and a half from now for Python 2.5, but for 2.4 I think it's enough to get the basic support and interface right, and see what use cases emerge and become general enough for the standard library. > How about making a BaseTemplate class which does NOT allow $foo.bar, and having > the default Template implement attributes? I'm leery of making this hierarchy too deep. Really, the only reason PEP 292's Template and SafeTemplate classes have an inheritance hierarchy is so that the pattern doesn't have to be written twice. Their __mod__()'s are quite different, and other than the pattern and that method, there ain't much to those classes. > Best, and thanks for keeping an open mind about this issue (string > interpolation is one of the very few areas where python's syntax bugs me, even > after years of daily use). And thank you for your suggestions. Hopefully by Python 2.5 time frame we'll have enough experience with this stuff to be able to extend them with other useful classes. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/6808d0ad/attachment.pgp From pm_mon at yahoo.com Sun Aug 22 22:56:19 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Sun Aug 22 22:56:27 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <200408221902.i7MJ2sd02547@guido.python.org> References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> <4128B778.1080503@v.loewis.de> <200408221902.i7MJ2sd02547@guido.python.org> Message-ID: Guido van Rossum wrote: > Who is Paul Morrow and why is he asserting what is Pythonic and isn't? > > --Guido van Rossum (home page: http://www.python.org/~guido/) I'm sorry Guido if I was out of line. I'm just a Python user (a strong proponent of the language), and have not seen the definition of 'pythonic' published anywhere, so I assumed that it was open to interpretation, as I've seen others make assertions that feature x or y "is pythonic". Is there a sanctioned definition for that term? Paul From guido at python.org Sun Aug 22 23:10:35 2004 From: guido at python.org (Guido van Rossum) Date: Sun Aug 22 23:10:41 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: Your message of "Sun, 22 Aug 2004 16:56:19 EDT." References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> <4128B778.1080503@v.loewis.de> <200408221902.i7MJ2sd02547@guido.python.org> Message-ID: <200408222110.i7MLAZJ16423@guido.python.org> > I'm sorry Guido if I was out of line. I'm just a Python user (a > strong proponent of the language), and have not seen the definition > of 'pythonic' published anywhere, so I assumed that it was open to > interpretation, as I've seen others make assertions that feature x > or y "is pythonic". > > Is there a sanctioned definition for that term? No, and that's intentionally so, because beauty is not absolute or quantifyable (even if there are extreme cases where we can all agree). All uses of pythonic should be explicitly qualified by "IMO" or something equivalent. And you can't assume that everybody automatically assumes this! Your recent posts seemed out of line to me only because you were being very argumentative and at the same time exposed serious misunderstanding (*). That's usually a clue to back off... High frequency posters are only okay if their posts have a high information content. (*) Which you acknowledged, but in messages showing more poor form by containing many dozens of lines of quoted material followed by a single "I see". --Guido van Rossum (home page: http://www.python.org/~guido/) From tdelaney at avaya.com Mon Aug 23 00:31:28 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Mon Aug 23 00:31:34 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D59429@au3010avexu1.global.avaya.com> Guido van Rossum wrote: >>> If the community can *agree* on >>> something they can support, I will listen. >> >> Even if it's []-after-args? > > If most people favor that over prefix @deco, sure, I'll give it > another look. (It better come with an implementation though.) Wasn't this the first implementation? Tim Delaney From pf_moore at yahoo.co.uk Mon Aug 23 00:37:56 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Mon Aug 23 00:37:54 2004 Subject: [Python-Dev] Re: Adding 'lexists()' to os.path References: <20040822110835.GA7452@mail.planet.nl> <4128ACD4.1090609@isg.de> Message-ID: Tino Lange writes: > Paul Moore wrote: >> As an alternative, would there be any value in cacheing the last stat >> result? In a sequence like this, with multiple calls for the same >> argument, many stat calls would be replaced by one. And it's likely >> that the checking overhead on cache misses would be minimal compared >> to the cost of the stat call, so it's an overall win. > you will never know how long the cached result is true. Files on disk > can change outside your python script.... Doh. Sorry about that - I wasn't thinking hard enough before posting. Back to sleep... Paul. -- Instant gratification takes too long -- Carrie Fisher From mkc at mathdogs.com Mon Aug 23 00:53:34 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Mon Aug 23 00:53:40 2004 Subject: [Python-Dev] Re: re.split on empty patterns References: <20040807145142.GB2529@rogue.amk.ca> <411C2B72.4090809@ocf.berkeley.edu> Message-ID: "Brett C." writes: > Mike Coleman wrote: > > [SNIP] > > # alternative 2: > > re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', 'bbczz'] > ^ > > re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz'] > > # alternative 3: > > re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', '', 'bbczz'] > ^ > > re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz'] > > > > I take it the first 'b' in both of the first examples for each alternative > were supposed to be 'a'? Yes, that's correct. Oops. > And as for which version, I actually like Mike's version more than the one AMK > and Tim like. The reason is that the '' in the middle of the example in > question in the OP tells you where the split would have occurred had split0 (I > like that or 'split_empty') not been turned on. That way there is no real loss > of data between the two, but a gain with the new feature being used. Is there something we can do to move this forward? It seems like a couple of people like one option and a couple the other, but I think at least we all agree that the general feature would be a good idea. So, should we take a vote? Or just go with the more conservative option, in order to get something in the tree for 2.4? Mike From fperez528 at yahoo.com Mon Aug 23 01:35:36 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Mon Aug 23 01:24:03 2004 Subject: [Python-Dev] Re: Re: Re: Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> <1092802980.7810.29.camel@geddy.wooz.org> <1093202241.25618.19.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > On Wed, 2004-08-18 at 14:07, Fernando Perez wrote: > >> I'd like to argue that this form may be the most useful for common tasks, so >> you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar" >> without having to worry too much about which template class you are using. > > It might be, but using attribute lookup syntax can be problematic. I > can only relate my experiences with Mailman, where I've been using > something similar for several years, albeit with traditional > %-placeholders. [...] I trust your judgment an openness on this, and indeed prudence has served python well in the past. I'd just like this to be available so I could handle a very common case like: 'a local var: $var, and some object params: $self.x, $self.y, $self.z' This can't easily be solved with the current syntax of %()s, since the namespaces for self (self.__dict__) and locals are different, and there's no attribute lookup. How about making template objects callable (or providing an eval() method)? It would allow the following: tpl = template('a var $foo') print tpl -> evaluates under the current rules, by calling __str__ but: tpl2 = template('local: $var and attr: $self.x') print tpl2() # or tpl2.eval(), or whatever other name where the call has _optional_ local/global arguments? This could satisfy the common case while allowing evaluation in restricted contexts: print tpl2(my_locals,my_globals) Anyway, just some thoughts on this... Best regards, Fernando From mkc at mathdogs.com Mon Aug 23 01:25:52 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Mon Aug 23 01:26:03 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp> <873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: "Fredrik Lundh" writes: > well, the examples in your PEP can be written as: > > data = [line[:-1].split(":") for line in open(filename)] Yes, in practice I would write this, too. The example was for pedagogical purposes, but perhaps the fact that it's not particularly useful in practice makes it a bad example. > and > > import ConfigParser > > c = ConfigParser.ConfigParser() > c.read(filename) > > data = [] > for section in c.sections(): > data.append((section, c.items(section))) > > both of which are shorter than your structparse examples. Hmm. In this case it doesn't seem fair to compare a call to ConfigParser, rather than the code in the ConfigParser module itself (or at least the subset of it that would provide the equivalent functionality). I used this as an example because I thought most people would be familiar with this file format, thus saving them having to figure out some new file format in order to follow the PEP. In practice, there are lots of other file formats of similar complexity that are not handled by any such special purpose module, and structmatch would make it easy to parse them. For example, this "SQT" format is output by a certain mass spec analysis program: http://fields.scripps.edu/sequest/SQTFormat.html There are a number of other bioinformatics programs whose output unfortunately must be scraped at present. The structmatch feature would also be useful for these cases. (This is what motivated the PEP.) > and most of the one-liners in your pre-PEP can be handled with a > combination of "match" and "finditer". I think "findall" and "finditer" are almost useless for this kind of thing, as they are essentially "searching" rather than "matching". That is, they'll happily, silently skip over garbage to get to something they like. Since finditer returns matches, you can always inspect the match to determine whether anything was skipped, but this seems kind of lame compared to just doing the right thing in the first place (i.e., matching). > here's a 16-line helper that > parses strings matching the "a(b)*c" pattern into a prefix/list/tail tuple. > > import re > > def parse(string, pat1, pat2): > """Parse a string having the form pat1(pat2)*""" > m = re.match(pat1, string) > i = m.end() > a = m.group(1) > b = [] > for m in re.compile(pat2 + "|.").finditer(string, i): > try: > token = m.group(m.lastindex) > except IndexError: > break > b.append(token) > i = m.end() > return a, b, string[i:] > > >>> parse("hello 1 2 3 4 # 5", "(\w+)", "\s*(\d+)") > ('hello', ['1', '2', '3', '4'], ' # 5') No offense, but this code makes me cringe. The "|." trick seems like a horrific hack, and I'd need to stare at this code for quite a while to convince myself that it doesn't have some subtle flaw. And even then it just handles the "a(b)*c" case. It seems like the code for more complex patterns parsed this way would just explode in size, and would have to be written custom for each pattern. Mike From mkc at mathdogs.com Mon Aug 23 01:47:43 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Mon Aug 23 01:47:52 2004 Subject: [Python-Dev] Re: python-dev Summary for 2004-08-01 through 2004-08-15[draft] References: <4124506F.6040503@ocf.berkeley.edu> Message-ID: > Mike Coleman presented a pre-PEP on adding a function to re that would create > ... > Contributing threads: > - `pre-PEP: Complete, Structured Regular Expression Group Matching > `__ The original PEP (which you link to here) is almost incomprehensible due to my typos, etc. You might want to link to the corrected version here http://mail.python.org/pipermail/python-dev/2004-August/047238.html in addition, or instead. > Mike Coleman wrote up a patch for re.split() that AM Kuchling presented to the > list. It adds an argument to the function to allow an empty string to be put > into the resulting list when a match occurs, even if the match consumes no > characters. On this, it might be clearer to add something like The point is that splitting on empty patterns lets you break up text without losing the delimiting parts. So, for example, splitting on r'(?=^S)' breaks a string into "paragraphs", where a paragraph begins with a line that starts with 'S'. BTW, FWIW, I read your python-dev summaries religiously. No way would I have time to read python-dev itself, so your summaries are the only thing that allows me to follow along. Thank you!!! Mike From pm_mon at yahoo.com Mon Aug 23 01:49:11 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Mon Aug 23 01:49:22 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <200408222110.i7MLAZJ16423@guido.python.org> References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> <4128B778.1080503@v.loewis.de> <200408221902.i7MJ2sd02547@guido.python.org> <200408222110.i7MLAZJ16423@guido.python.org> Message-ID: <412930F7.9080605@yahoo.com> Guido van Rossum wrote: > > Your recent posts seemed out of line to me only because you were being > very argumentative and at the same time exposed serious > misunderstanding (*). That's usually a clue to back off... High > frequency posters are only okay if their posts have a high information > content. > > (*) Which you acknowledged, but in messages showing more poor form by > containing many dozens of lines of quoted material followed by a > single "I see". > My apologies to all. I'll be more careful in the future. Paul From mkc at mathdogs.com Mon Aug 23 01:57:20 2004 From: mkc at mathdogs.com (Mike Coleman) Date: Mon Aug 23 01:57:30 2004 Subject: [Python-Dev] Re: Threading in the Standard Library Tour Part II References: <002001c48325$436d9460$e841fea9@oemcomputer> <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: "Stephen J. Turnbull" writes: > >>>>> "Raymond" == Raymond Hettinger writes: > Raymond> Also, multiple processors cannot boost performance > Raymond> because Python's Global Interpreter Lock (GIL) precludes > Raymond> more than one thread from running in the interpreter at > Raymond> the same time (this was done to simplify re-entrancy > Raymond> issues). > > This confused me because "running" is ambiguous. How about something > like: > > The Python interpreter itself is not fully reentrant, so threading > is accomplished by interleaving the execution of Python code from > different threads. Thus, use of multiple processors cannot boost > performance of threaded Python code, because a single instance of > the interpreter can only execute code on one processor at a time. > > (Season to taste, fix technical inaccuracies.) I see no reason to > mention the GIL, an implementation detail, at all. I somewhat prefer Raymond's version, myself. As soon as I read "execute code on one processor at a time", I immediately want to ask "Why?". The mention of the GIL more or less answers this, depending on how much you know about interpreters. Mike From djc at object-craft.com.au Mon Aug 23 02:01:02 2004 From: djc at object-craft.com.au (Dave Cole) Date: Mon Aug 23 02:01:21 2004 Subject: [Python-Dev] Ignoring SIGPIPE In-Reply-To: <200408210111.i7L1Bb623898@guido.python.org> References: <20040820020919.GG31470@epoch.metaslash.com> <20040820044436.0F8503C113@coffee.object-craft.com.au> <412586CD.2000000@object-craft.com.au> <20040820234005.GP31470@epoch.metaslash.com> <200408210111.i7L1Bb623898@guido.python.org> Message-ID: <412933BE.6020309@object-craft.com.au> Guido van Rossum wrote: >>>>I agree, but I think that was copied verbatim from elsewhere in >>>>socketmodule.c, so it was left with the aim of being bug for bug >>>>compatible with socket.socket(). >>> >>>Indeed. I was not going to change something as fundamental as the >>>existing signal handling. The onyl sensible thing to do was to copy the >>>behaviour already in place when you create a socket the "normal" way. >> >>You are correct. There are two places that call >>signal(SIGPIPE, SIG_IGN); I find this surprising. >> >>Guido, you added this code in version 1.4 from 1991. >>You do remember the reason why, don't you? It's only >>13 years ago. :-) I'm hesitant to remove the old calls, >>but I'm not sure I want it added to new calls if it's >>not necessary. >> >>Any opinions? > > > The only real requirement is that SIGPIPE is being ignored by default, > and that is already taken care of by inisigs() in pythonrun.c. So I > think both calls can be removed from socketmodule.c, heh heh. > > Originally, I was being careful, and not ignoring SIGPIPE until the > first socket was created, hence the call there. The idea being that > if you have a Python program that spits lots of stuff to stdout, and > you pipe it into head (e.g.), it won't give you an exception -- the > SIGPIPE will just kill it and the shell will ignore that; but if > you're using sockets, it's important to ignore SIGPIPE so too bad for > the other feature. But we've been ignoring SIGPIPE in the > initialization forever, so the SIGPIPE call in socketmodule.c is > really not needed any more. So should I remove both signal() calls and go offline for a few months? I should also fix the docstring and documentation to mention that the default socketpair() family is AF_UNIX. - Dave -- http://www.object-craft.com.au From barry at python.org Mon Aug 23 02:30:37 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 23 02:30:32 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <412626CC.4050900@ocf.berkeley.edu> References: <1092177771.21839.54.camel@localhost> <59e9fd3a040811052152548b6b@mail.gmail.com> <1092232683.9945.26.camel@localhost> <1f7befae040811221916a955fd@mail.gmail.com> <1092954772.8712.147.camel@geddy.wooz.org> <412533F6.4000703@ocf.berkeley.edu> <1093014495.9258.6.camel@geddy.wooz.org> <412626CC.4050900@ocf.berkeley.edu> Message-ID: <1093221037.25618.90.camel@geddy.wooz.org> On Fri, 2004-08-20 at 12:29, Brett C. wrote: > > What do you think about this compared to Tim's pattern, which is more > > strict than just complaining about $'s at the end of strings? > > > But Raymond is pushing for "practicality beats purity" which is more or > less true, but I am starting to agree with Tim that dollar amounts in an > interpolated string probably won't hapeen that often. If anything the > number will be what is interpolated in. So going for a more explicit > version is seeming to be the better solution. [...] I agree that explicit is better than implicit. I've added this as an open issue in the PEP, but the implementation (patch) I'm going to post will require throw the ValueError for all bad placeholders. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/15e88b1b/attachment.pgp From barry at python.org Mon Aug 23 02:32:56 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 23 02:32:28 2004 Subject: [Python-Dev] Re: Re: Re: Re: Update PEP 292 In-Reply-To: References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> <1092802980.7810.29.camel@geddy.wooz.org> <1093202241.25618.19.camel@geddy.wooz.org> Message-ID: <1093221175.25616.93.camel@geddy.wooz.org> On Sun, 2004-08-22 at 19:35, Fernando Perez wrote: > Anyway, just some thoughts on this... Good thoughts, and thanks for them, but for now I'm going to stick with the basics and put this PEP to bed. Feel free to write a follow on of PEP 292 for Python 2.5 . -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/69ac757b/attachment-0001.pgp From fperez528 at yahoo.com Mon Aug 23 03:00:27 2004 From: fperez528 at yahoo.com (Fernando Perez) Date: Mon Aug 23 02:47:00 2004 Subject: [Python-Dev] Re: Re: Re: Re: Re: Update PEP 292 References: <1092177771.21839.54.camel@localhost> <1092232536.9945.21.camel@localhost> <1092802980.7810.29.camel@geddy.wooz.org> <1093202241.25618.19.camel@geddy.wooz.org> <1093221175.25616.93.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > On Sun, 2004-08-22 at 19:35, Fernando Perez wrote: > >> Anyway, just some thoughts on this... > > Good thoughts, and thanks for them, but for now I'm going to stick with > the basics and put this PEP to bed. Feel free to write a follow on of > PEP 292 for Python 2.5 . No prob. Thanks for listening, but don't drop these emails from your pep-292 folder :) Best, f From python at rcn.com Mon Aug 23 02:47:14 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 23 02:47:55 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1093221037.25618.90.camel@geddy.wooz.org> Message-ID: <006d01c488aa$bbf91e60$e841fea9@oemcomputer> > ... the implementation (patch) I'm going to post > will require throw the ValueError for all bad placeholders. Since you're overriding the mod operator, it may be best to emulate its behavior which is to throw a KeyError: >>> '%(bingo)s' % {'bongo':'xxx'} Traceback (most recent call last): File "", line 1, in -toplevel- '%(bingo)s' % {'bongo':'xxx'} KeyError: 'bingo' Raymond From barry at python.org Mon Aug 23 02:50:32 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 23 02:50:04 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <006d01c488aa$bbf91e60$e841fea9@oemcomputer> References: <006d01c488aa$bbf91e60$e841fea9@oemcomputer> Message-ID: <1093222232.28570.1.camel@geddy.wooz.org> On Sun, 2004-08-22 at 20:47, Raymond Hettinger wrote: > > ... the implementation (patch) I'm going to post > > will require throw the ValueError for all bad placeholders. > > Since you're overriding the mod operator, it may be best to emulate its > behavior which is to throw a KeyError: > > > > >>> '%(bingo)s' % {'bongo':'xxx'} > > Traceback (most recent call last): > File "", line 1, in -toplevel- > '%(bingo)s' % {'bongo':'xxx'} > KeyError: 'bingo' That's a different error, and one that Template still throws, if a placeholder can't be found in the mapping. The ValueError is thrown when an invalid placeholder specification is found. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/372a6fde/attachment.pgp From fumanchu at amor.org Mon Aug 23 04:40:31 2004 From: fumanchu at amor.org (Robert Brewer) Date: Mon Aug 23 04:45:54 2004 Subject: [Python-Dev] Multiple decorators per line Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net> Guido, Are multiple decorators per line allowed in the current (pie) syntax? Should they be? They seem to be enabled in CVS, but nobody is verbally promoting them, and the wiki actually points out the "single deco per line" as a positive feature of the current syntax (A1). Robert Brewer MIS Amor Ministries fumanchu@amor.org From python at rcn.com Mon Aug 23 04:54:18 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 23 04:55:02 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <1093222232.28570.1.camel@geddy.wooz.org> Message-ID: <001a01c488bc$7bdc97a0$e841fea9@oemcomputer> > That's a different error, and one that Template still throws, if a > placeholder can't be found in the mapping. The ValueError is thrown > when an invalid placeholder specification is found. Got it! Raymond From guido at python.org Mon Aug 23 05:17:29 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 23 05:17:40 2004 Subject: [Python-Dev] Ignoring SIGPIPE In-Reply-To: Your message of "Mon, 23 Aug 2004 10:01:02 +1000." <412933BE.6020309@object-craft.com.au> References: <20040820020919.GG31470@epoch.metaslash.com> <20040820044436.0F8503C113@coffee.object-craft.com.au> <412586CD.2000000@object-craft.com.au> <20040820234005.GP31470@epoch.metaslash.com> <200408210111.i7L1Bb623898@guido.python.org> <412933BE.6020309@object-craft.com.au> Message-ID: <200408230317.i7N3HT716938@guido.python.org> > So should I remove both signal() calls and go offline for a few months? Yes to the former; to the latter, I don't know what that's referring to. > I should also fix the docstring and documentation to mention that the > default socketpair() family is AF_UNIX. Of course. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Aug 23 05:27:07 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 23 05:27:14 2004 Subject: [Python-Dev] Multiple decorators per line In-Reply-To: Your message of "Sun, 22 Aug 2004 19:40:31 PDT." <3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net> References: <3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net> Message-ID: <200408230327.i7N3R7717007@guido.python.org> > Are multiple decorators per line allowed in the current (pie) syntax? Yes in the 2.4a2 release, no in the PEP and current CVS. This was decided quickly and with full agreement of all around shortly after a2 was released. > Should they be? They seem to be enabled in CVS, but nobody is verbally > promoting them, and the wiki actually points out the "single deco per > line" as a positive feature of the current syntax (A1). I originally thought it would be useful to have a compact form for multiple short parameterless decorators, but in the end decided that it wasn't worth it. This is along the same lines of my current thinking that allowing if condition: statement instead of if condition: statement was a mistake (the statement is too hidden, especially if it's a break or return). --Guido van Rossum (home page: http://www.python.org/~guido/) From djc at object-craft.com.au Mon Aug 23 05:42:30 2004 From: djc at object-craft.com.au (Dave Cole) Date: Mon Aug 23 05:42:37 2004 Subject: [Python-Dev] Ignoring SIGPIPE In-Reply-To: <200408230317.i7N3HT716938@guido.python.org> References: <20040820020919.GG31470@epoch.metaslash.com> <20040820044436.0F8503C113@coffee.object-craft.com.au> <412586CD.2000000@object-craft.com.au> <20040820234005.GP31470@epoch.metaslash.com> <200408210111.i7L1Bb623898@guido.python.org> <412933BE.6020309@object-craft.com.au> <200408230317.i7N3HT716938@guido.python.org> Message-ID: <412967A6.7080002@object-craft.com.au> Guido van Rossum wrote: >>So should I remove both signal() calls and go offline for a few months? > > > Yes to the former; to the latter, I don't know what that's referring to. Go offline so I can't see the fallout from the damage I have caused :-). >> I should also fix the docstring and documentation to mention that the >>default socketpair() family is AF_UNIX. > > > Of course. Good. I will make the changes. - Dave -- http://www.object-craft.com.au From guido at python.org Mon Aug 23 07:45:21 2004 From: guido at python.org (Guido van Rossum) Date: Mon Aug 23 07:45:28 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0020.txt, NONE, 1.1 In-Reply-To: Your message of "Sun, 22 Aug 2004 20:41:24 PDT." References: Message-ID: <200408230545.i7N5jMC17212@guido.python.org> A good idea! What made you think of doing this? --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh at python.net Mon Aug 23 13:08:10 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 23 13:08:12 2004 Subject: [Python-Dev] PEP 318: Can't we all just get along? In-Reply-To: <200408202007.i7KK71t23456@guido.python.org> (Guido van Rossum's message of "Fri, 20 Aug 2004 13:07:01 -0700") References: <200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz> <200408202007.i7KK71t23456@guido.python.org> Message-ID: <2my8k66rj9.fsf@starship.python.net> Guido van Rossum writes: >> > If the community can *agree* on >> > something they can support, I will listen. >> >> Even if it's []-after-args? > > If most people favor that over prefix @deco, sure, I'll give it > another look. (It better come with an implementation though.) It has one, mine (well, at least the grammar and compiler changes). Anthony keeps accusing my posting of this patch two-and-a-half years ago (!) of starting this whole mudpie... Cheers, mwh -- About the use of language: it is impossible to sharpen a pencil with a blunt axe. It is equally vain to try to do it with ten blunt axes instead. -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time. From fredrik at pythonware.com Mon Aug 23 13:41:26 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 23 13:39:45 2004 Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular Expression Group Matching References: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp><873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: Mike Coleman wrote: >> >>> parse("hello 1 2 3 4 # 5", "(\w+)", "\s*(\d+)") >> ('hello', ['1', '2', '3', '4'], ' # 5') > > No offense, but this code makes me cringe. The "|." trick seems like a > horrific hack "or any other character" a horrific hack? that's RE 101, really. and you're supposed to *use* the code, not stare at the implementation. programming is all about abstractions, remember? From cce at clarkevans.com Mon Aug 23 14:58:48 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 14:58:50 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <001f01c48643$5985faa0$e841fea9@oemcomputer> References: <1092954772.8712.147.camel@geddy.wooz.org> <001f01c48643$5985faa0$e841fea9@oemcomputer> Message-ID: <20040823125848.GA47863@prometheusresearch.com> On Thu, Aug 19, 2004 at 07:22:09PM -0400, Raymond Hettinger wrote: | IDLE 1.1a2 | >>> from Cheetah.Template import Template | >>> t = Template('the $rank is $@ a $$ dead ${rank}! $') | >>> t.rank='king' | >>> print t | the king is $@ a $$ dead king! $ I second this behavior. Besides the missing 's', the second most common error is not escaping the %, for example, is painful, especially when you go through two or three stages of substitution. The behavior of Cheetah reflects real-world experience. Most of the time your templates are edited by HTML monkeys or people writing form letters or help screens. If a substitution fails, it will be obvious. This is one case where being explict causes more problems than it fixes. I've found Cheetah's implementation to be quite useful and would hope that any 'standard' template mechanism would be advised by the good work Chuck Esterbrook and Mike Orr. The other mechanism I like is the dotted notation to traverse objects. Also, I really like Cheetah's mechanism where the template object doubles as a dictionary of values. In particular, it would be nice to put default values, during substitution, it would first look in the dictionary applied via the % operator. If a key is not found in the applied dictionary, it would use the values given by the template itself. And, if cast to a string, it would only use the template's key values. If this feature was implemented, it'd be great to use keyword arguments as default mapping values. >>> t = Template("$foo $bar $bing",foo='wibbles',bar='womble') >>> print t wibbles womble $bing >>> print t % {'bar': 'bingles'} wibbles bingles $bing >>> t.bing = 'wicken' >>> t wibbles womble wicken Cheers, Clark From pinard at iro.umontreal.ca Mon Aug 23 15:20:43 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Aug 23 16:02:51 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1093232832.28573.16.camel@geddy.wooz.org> References: <1093232832.28573.16.camel@geddy.wooz.org> Message-ID: <20040823132043.GA4501@titan.progiciels-bpi.ca> [Barry Warsaw] > Attached is the latest version of PEP 292, which describes simpler > string substitutions. This PEP is slated for inclusion in Python 2.4 > and will likely be checked in before the next alpha release. Hi, people. The PEP was sent to python-list, but I'm replying to python-dev. As the PEP is "slated for inclusion in Python 2.4", I presume it was sent for information purposes, and not at all as a call for discussion or feedback. Nevertheless, here at least, I dare a few comments. :-) I like the avenue taken by this PEP, especially about _not_ altering Python syntax, and caring about internationalisation issues. Yet I'm rather -0.1 on the whole thing, considering that implementation might not fully meet its official goal, which is making string less error-prone, as per quote 1) below. However, consider quote 2) and 3): Quote 1) > This PEP is "simpler" in two respects: > 1. Python's current string substitution feature (i.e. > %-substitution) is complicated and error prone. Quote 2) > Open Issues > >>> from string import Template > >>> Template('The cost was $amount euros') % {'amount': 7} > Proposed resolution: no automatic stringification. Quote 3) > - Should the $-placeholder rules be more strict? > Proposed resolution: There seems to be consensus for strictness > on the grounds of explicit is better than implicit. These two quotes let me think that people will often forget to stringify numbers and various other value types, or to escape a few isolated `$'. This is a lost compared to the current `%(key)s' which is much more forgiving and helpful, in that it always stringify the value. So, the feature implemented by this PEP will _also_ be a bit complicated and error prone. We might not be gaining much overall, and yet, we loose the simplicity of having only one main interpolation mechanism. Of course, the PEP is accepted, and my opinion has no effect, so I'll stick on one or two details. I have the fuzzy feeling that one unstated, but important goal of the PEP is to find some way to push `$' forward for stringification in Python. The whole PEP might even be a stunt or justification for this unstated goal. However, as this goal is well reached, it might somehow have contributed to the PEP acceptation and counterweight the deficiency stated in the preceding paragraph. If my feeling is right, then the PEP should clearly explicit this goal, it will make the PEP stronger. > There seems little consensus around either suggestion, and > since the classes are just a few lines of Python, I propose no > string module re-organization, but to add these two classes to > string.py. I witnessed a serious effort in this developers' group to "empty" the `string' module in favour of string methods. That effort has been abandoned? I also personally think the word `string' should be left free, in the long run, for Python programmers to use, on the ground that modules should not be named after common words programmers would like to keep to themselves as variable names. So, I think _nothing_ should be added to the `string' module, with the effect of nailing it deeper in the library, even if there is a need to rush some solution in. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From ilya at bluefir.net Mon Aug 23 17:07:42 2004 From: ilya at bluefir.net (Ilya Sandler) Date: Mon Aug 23 17:07:45 2004 Subject: [Python-Dev] #ifdeffery Message-ID: On Fri, 13 Aug 2004, Michael Hudson wrote: > "Raymond Hettinger" writes: > > > P.S. Side rant: Also, in the last few months, the #ifdefs have > > multiplied. While I understand that some things like time stamping are > > worthwhile, the code is getting harder to read and maintain because of > > all this b.s. > > Is it really getting worse? > > I think there are two qualitatively different sorts of #ifdef mess: > feature ones, like the WITH_TSC I've recently been fiddling with > recently and portability ones. Actually, I have a suggestion regarding #ifdef WITH_TSC blocks in ceval.c Most of these blocks look like this: #ifdef WITH_TSC rdtscll(inst0); #endif where rdtscll() is defined in the beginning of ceval.c as follows #ifdef WITH_TSC ... #define rdtscll(var) ppc_getcounter(&var) ... #else /* this section is for linux/x86 */ ... #endif so why not to add a dummy rtdscll() macro to the else section of the above ifdef: #else /* this section is for linux/x86 */ #define rdtscll(var) #endif Then one can simply omit ifdef brackets for rdtscll invocations, removing about 20 ifdefs.... there are probably other places in the code where this trick could be used Does this make sense or am I missing something here? Ilya > The feature #ifdefs aren't (or shouldn't be) that much of a big deal. > Ideally, they are documented in Misc/SpecialBuilds.txt and depending > on whether they are defined or not, bits and pieces of code are or are > not included. > > From mwh at python.net Mon Aug 23 17:14:01 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 23 17:14:02 2004 Subject: [Python-Dev] #ifdeffery In-Reply-To: (Ilya Sandler's message of "Mon, 23 Aug 2004 08:07:42 -0700 (PDT)") References: Message-ID: <2mpt5h7upy.fsf@starship.python.net> Ilya Sandler writes: > On Fri, 13 Aug 2004, Michael Hudson wrote: > >> "Raymond Hettinger" writes: >> >> > P.S. Side rant: Also, in the last few months, the #ifdefs have >> > multiplied. While I understand that some things like time stamping are >> > worthwhile, the code is getting harder to read and maintain because of >> > all this b.s. >> >> Is it really getting worse? >> >> I think there are two qualitatively different sorts of #ifdef mess: >> feature ones, like the WITH_TSC I've recently been fiddling with >> recently and portability ones. > > Actually, I have a suggestion regarding > #ifdef WITH_TSC > > blocks in ceval.c > > Most of these blocks look like this: > > #ifdef WITH_TSC > rdtscll(inst0); > #endif > > where rdtscll() is defined in the beginning of ceval.c as follows > > #ifdef WITH_TSC > ... > #define rdtscll(var) ppc_getcounter(&var) > ... > #else /* this section is for linux/x86 */ > ... > #endif > > so why not to add a dummy rtdscll() macro to the else section of the > above ifdef: > > #else /* this section is for linux/x86 */ > #define rdtscll(var) > #endif > > Then one can simply omit ifdef brackets for rdtscll invocations, removing > about 20 ifdefs.... > > there are probably other places in the code where this trick could be used > > Does this make sense or am I missing something here? No, this definitely makes sense to me. I'd prefer to call the macro something more transparent, though. One of the other things I didn't do when porting this stuff to PPC was changing the names so that it's less Pentium specific. Patches welcome :-) Cheers, mwh -- Python enjoys making tradeoffs that drive *someone* crazy . -- Tim Peters, comp.lang.python From cce at clarkevans.com Mon Aug 23 17:39:23 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 17:39:25 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? Message-ID: <20040823153923.GA50408@prometheusresearch.com> I just read the thread 'Stackless Python' in June 2004 on python-dev and was wondering if you'd comment on a simpler cooperative mechanism, via a small hack to generators: 1. The PEP would introduce a new 'builtin' class called 'Cooperate' 2. Generator semantics would be altered so that 'yield X', where X is an instance of Cooperate, would automagically propigate to the outer-most non-generator. In for example, >>> def MyCooperate(magic.Cooperate): >>> __str__(self): return "cooperate" >>> >>> def top(): >>> yield "one" >>> yield MyCooperate() >>> yield "two" >>> >>> def middle(): >>> """ intermediate generator _only_ sees one and two """ >>> for x in top(): >>> print "middle", x >>> yield x >>> >>> def lower(): >>> """ this is not a generator, so it sees cooperate """ >>> for x in middle(): >>> print "lower", x >>> >>> lower() middle one lower one lower cooperate middle two lower two With these two changes, the "lower" function could be an async reactor like those found in Twisted, or async core. While the result isn't true coutines, it would be a huge improvement for those who would like to do async coding. I've done something similar with Twisted called Flow [1] and it works well, with the exception of being a painful syntax hack and being quite slow. If this was moved into Python's core, we'd get most of the advantages of coroutines without the full cost. Thoughts? Clark [1] http://twistedmatrix.com/documents/current/howto/flow -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/ - Research Exchange Database /\ - Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * From mcherm at mcherm.com Mon Aug 23 17:54:46 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Aug 23 17:51:30 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions Message-ID: <1093276486.412a1346e30e9@mcherm.com> Fran?ois Pinard writes: > [quotations from the PEP motivation] let me think that people will often > forget to stringify numbers and various other value types, I agree. I'd like to raise a small voice in favor of automatically calling str() on all arguments before interpolation. I'd rather type this: # ... code ... out.write( template("$foo\t$bar\t$baz\n") % locals() ) than this: # ... code ... out.write( template("$foo\t$bar\t$baz\n") % {'foo': str(foo), 'bar': str(bar), 'baz': str(baz)} ) -- Michael Chermside From pje at telecommunity.com Mon Aug 23 17:56:04 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 23 17:55:50 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <20040823153923.GA50408@prometheusresearch.com> Message-ID: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> At 11:39 AM 8/23/04 -0400, Clark C. Evans wrote: >I just read the thread 'Stackless Python' in June 2004 on python-dev and >was wondering if you'd comment on a simpler cooperative mechanism, via a >small hack to generators: > >1. The PEP would introduce a new 'builtin' class called 'Cooperate' > >2. Generator semantics would be altered so that 'yield X', where X > is an instance of Cooperate, would automagically propigate to > the outer-most non-generator. Perhaps you mean "inner-most"? >With these two changes, the "lower" function could be an async reactor >like those found in Twisted, or async core. While the result isn't true >coutines, it would be a huge improvement for those who would like to do >async coding. I've done something similar with Twisted called Flow [1] >and it works well, with the exception of being a painful syntax hack and >being quite slow. If this was moved into Python's core, we'd get most >of the advantages of coroutines without the full cost. > >Thoughts? It doesn't seem to me to actually help anything. You can already do this using a simple wrapper object that maintains a stack of active generators, as I do in 'peak.events'. I was hoping that you had actually come up with a solution for the more complex problem of suspending *non* generator functions, in a way that would work with CPython. :( From cce at clarkevans.com Mon Aug 23 18:12:33 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 18:12:36 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1093276486.412a1346e30e9@mcherm.com> References: <1093276486.412a1346e30e9@mcherm.com> Message-ID: <20040823161233.GB71531@prometheusresearch.com> +1 to automatic str() -- this is a common newbie mistake and I can't think of a case where you wouldn't want this to happen On Mon, Aug 23, 2004 at 08:54:46AM -0700, Michael Chermside wrote: | Fran?ois Pinard writes: | > [quotations from the PEP motivation] let me think that people will often | > forget to stringify numbers and various other value types, | | I agree. I'd like to raise a small voice in favor of automatically | calling str() on all arguments before interpolation. I'd rather | type this: | | # ... code ... | out.write( template("$foo\t$bar\t$baz\n") % locals() ) | | than this: | | # ... code ... | out.write( template("$foo\t$bar\t$baz\n") % | {'foo': str(foo), 'bar': str(bar), 'baz': str(baz)} ) From mcherm at mcherm.com Mon Aug 23 18:22:03 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Aug 23 18:18:47 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <1093278123.412a19ab99d0b@mcherm.com> Skip writes: > Any thoughts about posting this to c.l.py as well as python-dev? Maybe it > would encourage some more participation... I don't want to exercise a veto here or anything, but I don't think it would be helpful. The barrier to more participation isn't that people don't know there's work to be done. I suspect that anyone who was going to consider helping with development would either be scratching their own itch (ie, contributing a patch and wouldn't care about this report), or would already be reading python-dev (after all, it's not hard to qualify to receive it!). I worry that posting this to c.l.py would be unlikely to generate anything other than occasional bellyaching about how slow we are about addressing bug reports. What might be more helpful would be a call for volunteers. We could form a "patch reviewers" group that would review patches on SF. Or something. Actually, figuring out how to recruit volunteers is a difficult problem. But I'm not sure the weekly bug/patch summary is the solution. -- Michael Chermside From tim.peters at gmail.com Mon Aug 23 18:32:39 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 23 18:32:46 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <20040823161233.GB71531@prometheusresearch.com> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> Message-ID: <1f7befae040823093264275843@mail.gmail.com> [Clark C. Evans] > +1 to automatic str() -- this is a common newbie mistake and I can't > think of a case where you wouldn't want this to happen Apologies to Barry, but I'm +1 on auto-str() too. It's a string interpolation -- the user is explicitly asking for a string. If they made a mistake, it was in asking for a string to begin with, not in feeding it a non-string. The same applies to string.join(iterable), for that matter. From cce at clarkevans.com Mon Aug 23 18:34:20 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 18:34:23 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> References: <20040823153923.GA50408@prometheusresearch.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> Message-ID: <20040823163420.GA94018@prometheusresearch.com> On Mon, Aug 23, 2004 at 11:56:04AM -0400, Phillip J. Eby wrote: | >1. The PEP would introduce a new 'builtin' class called 'Cooperate' | > | >2. Generator semantics would be altered so that 'yield X', where X | > is an instance of Cooperate, would automagically propigate to | > the outer-most non-generator. | | Perhaps you mean "inner-most"? Yes. The top-most non-generator on the stack. | It doesn't seem to me to actually help anything. You can already do this | using a simple wrapper object that maintains a stack of active | generators, as I do in 'peak.events'. Could you provide an example? The problem this proposal solves is straight-foward -- it is tedious and slow to have intermediate generators do stuff like: def middle(): """ intermediate generator _only_ sees one and two """ for x in top(): ! if isinstance(x,X): ! yield x print "middle", x yield x This extra step is tedious and also slow; especially if one has lots of yield statements that cooperate. It could be standardized and made a bit snappier if it was built-in behavior. This is an 80/5 proposal. One gets 80% of the happiness, with 5% of the pain. | I was hoping that you had actually come up with a solution for the more | complex problem of suspending *non* generator functions, in a way that | would work with CPython. :( Yes, I know. I'm trying to avoid this much harder problem. Clark From nas at arctrix.com Mon Aug 23 18:38:29 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Mon Aug 23 18:38:33 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1f7befae040823093264275843@mail.gmail.com> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> Message-ID: <20040823163829.GA3411@mems-exchange.org> On Mon, Aug 23, 2004 at 12:32:39PM -0400, Tim Peters wrote: > The same applies to string.join(iterable), for that matter. Perhaps someone can add this as a wish for Python 3k. Neil From cce at clarkevans.com Mon Aug 23 18:41:09 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 18:41:12 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1f7befae040823093264275843@mail.gmail.com> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> Message-ID: <20040823164109.GB94018@prometheusresearch.com> On Mon, Aug 23, 2004 at 12:32:39PM -0400, Tim Peters wrote: | The same applies to string.join(iterable), for that matter. This code-snippet is littered everwhere in my applications: string.join([str(x) for x in iterable]) Its tedious and makes code hard to read. Do we need a PEP to fix this? Clark From python at rcn.com Mon Aug 23 18:54:33 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 23 18:55:02 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <20040823164109.GB94018@prometheusresearch.com> Message-ID: <002401c48931$dda69060$e841fea9@oemcomputer> > This code-snippet is littered everwhere in my applications: > > string.join([str(x) for x in iterable]) > > Its tedious and makes code hard to read. Do we need a PEP to fix this? A PEP would be overkill. Still, it would be helpful to do PEP-like things such as reference implementation, soliticing comments, keep an issue list, etc. A minor issue is that the implementation automatically shifts to Unicode upon encountering a Unicode string. So you would need to test for this before coercing to a string. Also, join works in multiple passes. The proposal should be specific about where stringizing occurs. IIRC, you need the object length on the first pass, but the error handling and switchover to Unicode occur on the second. Raymond From mcherm at mcherm.com Mon Aug 23 19:04:55 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Aug 23 19:01:38 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions Message-ID: <1093280695.412a23b7865d8@mcherm.com> Tim Peters writes: > The same applies to string.join(iterable), for that matter. Clark Evans writes: > This code-snippet is littered everwhere in my applications: > > string.join([str(x) for x in iterable]) > > Its tedious and makes code hard to read. Do we need a PEP to fix this? I'm convinced! Of course any code like the following would break: class Item: # ... def __eq__(self, other): "All Items are equal!" return isinstance(other, Item) def process_item(item): """Processes an item, returning a string.""" # ... def process_items(items): """This processes each Item in the list and returns a string containing the entire thing.""" while True: try: return ','.join(items) except TypeError: pos = items.index(Item()) # find first Item items[pos] = process_item( items[pos] ) continue Seriously, I can't think of a sane use for .join() raising a TypeError, so I'd say run it by Guido and if it doesn't offend his design intuition, let's fix it! -- Michael Chermside From pje at telecommunity.com Mon Aug 23 19:18:28 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 23 19:18:14 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <20040823163420.GA94018@prometheusresearch.com> References: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <20040823153923.GA50408@prometheusresearch.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> At 12:34 PM 8/23/04 -0400, Clark C. Evans wrote: >On Mon, Aug 23, 2004 at 11:56:04AM -0400, Phillip J. Eby wrote: >| It doesn't seem to me to actually help anything. You can already do this >| using a simple wrapper object that maintains a stack of active >| generators, as I do in 'peak.events'. > >Could you provide an example? The problem this proposal solves is >straight-foward -- it is tedious and slow to have intermediate >generators do stuff like: > > def middle(): > """ intermediate generator _only_ sees one and two """ > for x in top(): >! if isinstance(x,X): >! yield x > print "middle", x > yield x > >This extra step is tedious and also slow; especially if one has lots of >yield statements that cooperate. 'peak.events' uses "Task" objects that maintain a stack of active generators. The Task receives yields from the "innermost" generator directly, without them being passed through by intermediate generators. If the value yielded is *not* a control value, the Task object pops the generator stack, and resumes the previously suspended generator. A "magic" function, 'events.resume()' retrieves the value from the Task inside the stopped generator. Basically, this mechanism doesn't pass control values through multiple tests and generator frames: control values are consumed immediately by the Task. This makes it easy to suspend nested generators while waiting for some event, such as socket readability, a timeout, a Twisted "Deferred", etc. Yielding an "event" object like one of the aforementioned items causes the Task to return to its caller (the event loop) after requesting a callback for the appropriate event. When the callback re-invokes the thread, it saves the value associated with the event, if any, for 'events.resume()' to retrieve when the topmost generator is resumed. Also, 'events.resume()' supports passing errors from one generator to the next, so that it's "as if" the generators execute in a nested fashion. The drawback is that you must invoke events.resume() after each yield, but this is *much* less intrusive than requiring generators to pass through results from all nested generators. Take a look at: http://cvs.eby-sarna.com/PEAK/src/peak/events/ In particular, the 'interfaces' and 'event_threads' modules. Here's a usage example, a simple Task procedure: @events.taskFactory def monitorBusy(self): # get a "readable" event on this socket untilReadable = self.eventLoop.readable(self) while True: # Wait until we have stream activity yield untilReadable; events.resume() # Is everybody busy? if self.busyCount()==self.childCount(): self.supervisor.requestStart() # Wait until the child or busy count changes before proceeding yield events.AnyOf(self.busyCount,self.childCount); events.resume() This task waits until a listener socket is readable (i.e. an incoming connection is pending), and then asks the process supervisor to start more processes if all the child processes are busy. It then waits until either the busy count or the child process count changes, before it waits for another incoming connection. Basically, if you're invoking a sub-generator, you do: yield subGenerator(arguments); result=events.resume() This is if you're calling a sub-generator that only returns one "real" result. You needn't worry about passing through control values, because the current generator won't be resumed until the subgenerator yields a non-control value. If you're invoking a sub-generator that you intend to *iterate over*, however, and that generator can suspend on events, it's a bit more complex: iterator = subGenerator(arguments) while True: yield iterator; nextItem = events.resume() if nextItem is NOT_GIVEN: # sentinel value break # body of loop goes here, using 'nextItem' This is not very convenient, but I don't find it all that common to have data I'm iterating over in such a fashion, because 'peak.events' programs tends to have "infinite" streams that are organized as event sources in "pipe and filter" fashion. So, you tend to end up with Tasks that only have one generator running anyway, except for things that are more like "subroutines" than real generators, because you only expect one real return value from them, anyway. peak.events can work with Twisted, by the way, if you have it installed. For example, this: yield aDeferred; result = events.resume() suspends the generator until the Deferred fires, and then the result will be placed in 'result' upon resumption of the generator. If the Deferred triggers an "errback", the call to 'events.resume()' will reraise the error, inside the current generator. It would be nice if there were some way to "accept" data and exceptions within a generator that didn't require the 'events.resume' hack, e.g.: result = yield aDeferred would be really nice, especially if 'result' could cause an exception to be raised. I was hoping that this was something along the lines of what you were proposing. E.g. if generator-iterators could take arguments to 'next()' that would let you do this. I believe there's already a rejected PEP covering the issue of communicating "into" generators. Perhaps there should be a "simple coroutines" PEP, that doesn't try to extend generators into coroutines, but instead treats coroutines as a first-class animal that just happens to be implemented using some of the same techniques "under the hood". From amk at amk.ca Mon Aug 23 19:38:35 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 23 19:38:46 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <1093278123.412a19ab99d0b@mcherm.com> References: <1093278123.412a19ab99d0b@mcherm.com> Message-ID: <20040823173835.GA10408@rogue.amk.ca> On Mon, Aug 23, 2004 at 09:22:03AM -0700, Michael Chermside wrote: > I don't want to exercise a veto here or anything, but I don't think > it would be helpful. The barrier to more participation isn't that people > don't know there's work to be done. I suspect that anyone who was going I disagree. I think there's a pool of volunteer labour in the Python community that we aren't tapping yet. For example, at the first bug day the person who looked at the most bugs was someone who'd never done anything with the Python core before (and, AFAICT, hasn't done anything since). In a discussion on that first day, he said "... maybe ppl think that the tracker is only for core developers etc. I never would have even looked there if it wasn't for today". (That was in a discussion about automatically closing bugs once they reach a certain age; see the transcript at http://www.amk.ca/python/bugday/2004-06-05.html for the context.) Posting the bug summary may result in people getting an idea of the rate at which bugs and patches are processed. Readers may also see a new bug that's relevant to them, and offer debugging assistance or commentary. I can't see how posting the summaries to c.l.py would *hurt*. I say "do it". --amk From mwh at python.net Mon Aug 23 19:58:02 2004 From: mwh at python.net (Michael Hudson) Date: Mon Aug 23 19:58:04 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <20040823173835.GA10408@rogue.amk.ca> (A. M. Kuchling's message of "Mon, 23 Aug 2004 13:38:35 -0400") References: <1093278123.412a19ab99d0b@mcherm.com> <20040823173835.GA10408@rogue.amk.ca> Message-ID: <2meklx7n4l.fsf@starship.python.net> "A.M. Kuchling" writes: > On Mon, Aug 23, 2004 at 09:22:03AM -0700, Michael Chermside wrote: >> I don't want to exercise a veto here or anything, but I don't think >> it would be helpful. The barrier to more participation isn't that people >> don't know there's work to be done. I suspect that anyone who was going > > I disagree. I think there's a pool of volunteer labour in the Python > community that we aren't tapping yet. For example, at the first bug > day the person who looked at the most bugs was someone who'd never > done anything with the Python core before (and, AFAICT, hasn't done > anything since). In a discussion on that first day, he said > "... maybe ppl think that the tracker is only for core developers > etc. I never would have even looked there if it wasn't for today". > (That was in a discussion about automatically closing bugs once they > reach a certain age; see the transcript at > http://www.amk.ca/python/bugday/2004-06-05.html for the context.) > > Posting the bug summary may result in people getting an idea of the > rate at which bugs and patches are processed. Readers may also see a > new bug that's relevant to them, and offer debugging assistance or > commentary. I can't see how posting the summaries to c.l.py would > *hurt*. I say "do it". This mail expresses my opinions much better than the one I didn't get around to writing :-) Cheers, mwh -- Presumably pronging in the wrong place zogs it. -- Aldabra Stoddart, ucam.chat From cce at clarkevans.com Mon Aug 23 20:33:15 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 20:33:19 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> References: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <20040823153923.GA50408@prometheusresearch.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> Message-ID: <20040823183315.GA84465@prometheusresearch.com> On Mon, Aug 23, 2004 at 01:18:28PM -0400, Phillip J. Eby wrote: | It would be nice if there were some way to "accept" data and exceptions | within a generator that didn't require the 'events.resume' hack, e.g.: | | result = yield aDeferred | | would be really nice, especially if 'result' could cause an exception to | be raised. I was hoping that this was something along the lines of what | you were proposing. Perhaps it would be nice to add an alternative syntax to call a generator when you are expecting exactly one value. def generator(): yield 'one value' def consumer(): value = generator() This, when combined with the previous proposal would give: >>> def top(): >>> yield cooperate >>> yield "one value" >>> >>> def middle(): >>> """ intermediate generator _only_ sees 'one value' """ >>> bing = top() >>> # do something with bing >>> yield bing >>> >>> def lower(): >>> """ this is not a generator, so it sees cooperate """ >>> for x in middle(): >>> print x >>> >>> lower() cooperate one value | Perhaps there should be a "simple coroutines" PEP, that doesn't try to | extend generators into coroutines, but instead treats coroutines as a | first-class animal that just happens to be implemented using some of the | same techniques "under the hood". The problem is maintaining a 'stack' of generators requires that intermediate generators in the stack need a tedious hack (in both peek.event and twisted.flow) for them to work. If we can have python allow messages to be sent from the top-most generator to the inner-most non-generator (via yield cooperate, or some other magic), this difficulty is resolved -- intermediate generators can then be written in a operational style, without icky hacks for managing deferred execution. Full-blown corountines arn't necessary. A small tweak to generators will do. Best, Clark From pje at telecommunity.com Mon Aug 23 20:59:18 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 23 20:59:05 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <20040823183315.GA84465@prometheusresearch.com> References: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <20040823153923.GA50408@prometheusresearch.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> At 02:33 PM 8/23/04 -0400, Clark C. Evans wrote: >Perhaps it would be nice to add an alternative syntax to call a >generator when you are expecting exactly one value. > > def generator(): > yield 'one value' > def consumer(): > value = generator() We have that today: value, = generator() Or do I misunderstand you? >Full-blown corountines arn't necessary. A small tweak to generators >will do. I don't think this is true. Your hypothetical example can't resume 'top()' after it yields the "co-operate" control value, unless it either it *has* a stack of generators, or the Python core somehow maintains a stack of the executing generators. So, my point was that since this can already be done in user-level code with a stack of generators, I don't see the point to adding a facility to Python to create hidden stacks of generators. Instead, it would be more useful to get rid of the one piece that really is "magic", by providing a way to pass values or exceptions into a running generator. My comment about "coroutines" was more that Guido previously expressed distaste for adding such a communication mechanism to generators as abusing the concept of a generator just being a way to implement complex iterators. Therefore, I thought a coroutine proposal (backed by a suitable syntax and an implementation plan) might have more success. From mcherm at mcherm.com Mon Aug 23 21:04:09 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Aug 23 21:01:03 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <1093287849.412a3fa9ef447@mcherm.com> amk writes: > I disagree. I think there's a pool of volunteer labour in the Python > community that we aren't tapping yet. For example [...] Well, if there's ANY decent chance that you're right, then it's worth sending the email. So I'll enthustiastically back the idea of sending the bug/patch summary to c.l.py. -- Michael Chermside From bac at OCF.Berkeley.EDU Mon Aug 23 21:04:19 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 23 21:04:31 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <1093011363.8761.57.camel@geddy.wooz.org> References: <002401c48645$bdf482c0$e841fea9@oemcomputer> <41255DA4.1070002@ocf.berkeley.edu> <1093011363.8761.57.camel@geddy.wooz.org> Message-ID: <412A3FB3.4060900@ocf.berkeley.edu> Barry Warsaw wrote: > On Thu, 2004-08-19 at 22:10, Brett C. wrote: > > >>Don't let the photos from PyCON fool you; I never have a good look on my >>face and I tend to look stoned. > > > Just "look"? Dang, I really thought Tim, Fred, and I had a new recruit > for our CrackPython project. The first one's free, y'know. > Ooh. Tempting. But then I would have to give up my Opium addiction to go back and I don't know if I can do that. That communal feel of hanging out in the local Opium den at my cousin's place would be such a loss. =) > >>Basically I can live with having a single string module, but I would >>like to see some real deprecation happen. The nice thing about the >>pacakge separation is it made it clear in the code what would be going >>away. If we at least at PendingDeprecation to what is supposed to be >>taken out I will be happy. > > > At the very least, my rewrite of libstring.tex will make it clear which > inhabitants of the string module are going to be deprecated. Besides, > since it's clear that Python 3000 will take a broader look at standard > library packagization, I'll drop this if we can agree that the PEP 292 > classes should go in the existing string module. > OK, that works for me. -Brett From bac at OCF.Berkeley.EDU Mon Aug 23 21:08:54 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 23 21:09:04 2004 Subject: [Python-Dev] python-dev Summary for 2004-08-01 through 2004-08-15[draft] In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E3022E1D@exchange.hqamor.amorhq.net> References: <3A81C87DC164034AA4E2DDFE11D258E3022E1D@exchange.hqamor.amorhq.net> Message-ID: <412A40C6.7020405@ocf.berkeley.edu> Robert Brewer wrote: > Brett C. wrote: > >>Option two out of all of this is people just say, "summarize what you >>want, Brett." Then I just cover what I find interesting and just > > don't > >>worry about covering a specific area. > > > +1. Bring back "Skipped Threads" if you can, just for the sake of > indexing completeness. > Do people miss the "Skipped Threads" feature of the summaries? Part of the point of cutting down is so I can just delete the threads that won't be covered and cut down on my inbox. But if people do like the feature then I could stand to leave the emails in my box and have a list of threads I just didn't touch (sans linking, that is just not worth the hassle until I write a tool to generate the links automatically). -Brett From cce at clarkevans.com Mon Aug 23 21:09:53 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 21:09:56 2004 Subject: [Python-Dev] Multiple decorators per line In-Reply-To: <200408230327.i7N3R7717007@guido.python.org> References: <3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net> <200408230327.i7N3R7717007@guido.python.org> Message-ID: <20040823190953.GA62618@prometheusresearch.com> It seems that decorators solve two problems: - a way to move decoration close-to (or ideally on) the 'def' line - a way to specify N decorators together as a whole While J2 seems nice, I'd rather see someone build a 'composite' decorator as the function it really is, rather than introducing a new keyword: def composite(instance): return decorator_a(decorator_b(instance)) def @composite func(args): """ function body """ By splitting these problems, the definition of the decorator can be on the same line as the 'def' line, and the composite decorator can be applied to more than one function. Also, the operation of the decorator is very obvious ... far less "deep magic". One could even use function arguments... def makeDecorator(use_a = False, use_b = True): retval = lambda instance: instance if use_a: retval = lambda instance: retval(decorator_a(instance)) if use_b: retval = lambda instance: retval(decorator_b(instance)) return retval composite = makeDecorator(use_a = True) def composite func(args): """ function body """ I suppose if one wanted more magic... def @makeDecorator(use_a = True) func(args): """ function body """ In short, I don't see a reason why this solution needs to solve #2, as regular functions should suffice. Also, The problem is just #1 Cheers, Clark From cce at clarkevans.com Mon Aug 23 21:18:49 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 23 21:18:51 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> References: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <20040823153923.GA50408@prometheusresearch.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> Message-ID: <20040823191849.GA95532@prometheusresearch.com> On Mon, Aug 23, 2004 at 02:59:18PM -0400, Phillip J. Eby wrote: | > def generator(): | > yield 'one value' | value, = generator() Oh, cool. ;) | > Full-blown corountines arn't necessary. A small tweak to | > to generators will do. | | I don't think this is true. Your hypothetical example can't resume | 'top()' after it yields the "co-operate" control value, unless it either | it *has* a stack of generators, or the Python core somehow maintains a | stack of the executing generators. Yes. The idea is for Python to maintain a stack of generators, and when ever the top-most generator yields this yield skips the intermediate generators on the stack, and goes immediately to the calling function. | So, my point was that since this can already be done in user-level code | with a stack of generators, I don't see the point to adding a facility to | Python to create hidden stacks of generators. The problem is, if you maintain the stack of generators in userland, then each generator on the stack must do special stuff to handle 'cooperate' (in one form or another). You demonstrated an equivalent problem with your peek.event code presented earlier. While, if this is built into the core, intermediate generators can be happily oblivious to the cooperative magic. | Instead, it would be more useful to get rid of the one piece that really | is "magic", by providing a way to pass values or exceptions into a | running generator. I don't like this either. Clark From python at rcn.com Mon Aug 23 21:18:29 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 23 21:19:50 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <412A3FB3.4060900@ocf.berkeley.edu> Message-ID: <001101c48946$19591100$e841fea9@oemcomputer> [Barry] > > ... I'll drop this if we can agree that the PEP 292 > > classes should go in the existing string module. [Brett] > OK, that works for me. Does it? Apply the patch and try "import re". Raymond From python at rcn.com Mon Aug 23 21:19:24 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 23 21:19:58 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <412A3FB3.4060900@ocf.berkeley.edu> Message-ID: <001201c48946$19b49e80$e841fea9@oemcomputer> [Barry] > > ... I'll drop this if we can agree that the PEP 292 > > classes should go in the existing string module. [Brett] > OK, that works for me. Does it? Apply the patch and try "import re". Raymond From pje at telecommunity.com Mon Aug 23 21:34:13 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 23 21:33:57 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <20040823191849.GA95532@prometheusresearch.com> References: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <20040823153923.GA50408@prometheusresearch.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040823152302.036b88b0@mail.telecommunity.com> At 03:18 PM 8/23/04 -0400, Clark C. Evans wrote: >On Mon, Aug 23, 2004 at 02:59:18PM -0400, Phillip J. Eby wrote: >| > def generator(): >| > yield 'one value' >| value, = generator() >Oh, cool. ;) > >| > Full-blown corountines arn't necessary. A small tweak to >| > to generators will do. >| >| I don't think this is true. Your hypothetical example can't resume >| 'top()' after it yields the "co-operate" control value, unless it either >| it *has* a stack of generators, or the Python core somehow maintains a >| stack of the executing generators. > >Yes. The idea is for Python to maintain a stack of generators, and when >ever the top-most generator yields this yield skips the >intermediate generators on the stack, and goes immediately to the >calling function. That's not the part I mean. I'm talking about *after* that happens, when you *resume* the generators, and you'll need to reconstruct the stack as it stood when the generators were interrupted. (Your original proposal doesn't actually mention this part at all, but I assume it's necessary. After all, if all you needed was a way to stop a bunch of generators in their tracks, raising an exception would suffice.) >| So, my point was that since this can already be done in user-level code >| with a stack of generators, I don't see the point to adding a facility to >| Python to create hidden stacks of generators. > >The problem is, if you maintain the stack of generators in userland, >then each generator on the stack must do special stuff to handle >'cooperate' (in one form or another). You demonstrated an equivalent >problem with your peek.event code presented earlier. While, if this is >built into the core, intermediate generators can be happily oblivious to >the cooperative magic. Though the problems are "equivalent" in complexity, the difficulty of their respective solutions are not at all comparable, because your approach either requires a technique for dynamically reconstructing the stack, or some similar mechanism for generators to keep track of the state of *other* generators when the containing generator is interrupted. From skip at pobox.com Mon Aug 23 21:41:43 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 23 21:42:55 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <1093278123.412a19ab99d0b@mcherm.com> References: <1093278123.412a19ab99d0b@mcherm.com> Message-ID: <16682.18551.966353.315826@montanaro.dyndns.org> Michael> I worry that posting this to c.l.py would be unlikely to Michael> generate anything other than occasional bellyaching about how Michael> slow we are about addressing bug reports. Well: 1. They get what they pay for. 2. My guess is that most of the bellyaching will come from people who don't currently contribute. If they want things done more quickly, they know where to start. Michael> Actually, figuring out how to recruit volunteers is a difficult Michael> problem. But I'm not sure the weekly bug/patch summary is the Michael> solution. Until something better comes along it might pick up one or two extra pairs of eyeballs. Skip From pje at telecommunity.com Mon Aug 23 22:16:08 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Aug 23 22:15:55 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <20040823194155.GA19785@prometheusresearch.com> References: <5.1.1.6.0.20040823152302.036b88b0@mail.telecommunity.com> <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <20040823153923.GA50408@prometheusresearch.com> <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com> <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com> <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> <5.1.1.6.0.20040823152302.036b88b0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040823155203.022af320@mail.telecommunity.com> At 03:41 PM 8/23/04 -0400, Clark C. Evans wrote: >(off list) > >On Mon, Aug 23, 2004 at 03:34:13PM -0400, Phillip J. Eby wrote: >| >Yes. The idea is for Python to maintain a stack of generators, and when >| >ever the top-most generator yields this yield skips the >| >intermediate generators on the stack, and goes immediately to the >| >calling function. >| >| That's not the part I mean. I'm talking about *after* that happens, when >| you *resume* the generators, and you'll need to reconstruct the stack as >| it stood when the generators were interrupted. > >Yes. The entire generator stack would have to be put back into play. Right, but you haven't offered any proposal for where this stack will come from, how it will actually work, etc. I personally don't see any way to implement your approach in CPython that doesn't essentially require re-implementing a portion of 'peak.events' in C, while adding the overhead of the control value check to *every* generator .next() operation, even in programs that don't use the feature. From martin at v.loewis.de Mon Aug 23 22:15:54 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 23 22:16:01 2004 Subject: [Python-Dev] #ifdeffery In-Reply-To: <2mpt5h7upy.fsf@starship.python.net> References: <2mpt5h7upy.fsf@starship.python.net> Message-ID: <412A507A.7050203@v.loewis.de> Michael Hudson wrote: > No, this definitely makes sense to me. I'd prefer to call the macro > something more transparent, though. One of the other things I didn't > do when porting this stuff to PPC was changing the names so that it's > less Pentium specific. Patches welcome :-) I'd like to stress that last point. Ilya, please do consider contributing a patch. As you say, it is technically trivial, but probably too much effort for any by-standing volunteer. I only committed the original patch because I considered the feature "mostly harmless", and because it had been sitting on SF for several years. I certainly won't touch this unless a) somebody reports a REAL bug, and Jeremy doesn't fix that, or b) somebody contributes another patch. Actually writing a patch, and testing it (including the two recompilations that will take) is too much effort, so I rather write a longish email seeking volunteers... Regards, Martin From martin at v.loewis.de Mon Aug 23 22:23:14 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 23 22:23:26 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <1093278123.412a19ab99d0b@mcherm.com> References: <1093278123.412a19ab99d0b@mcherm.com> Message-ID: <412A5232.6060202@v.loewis.de> Michael Chermside wrote: > I don't want to exercise a veto here or anything, but I don't think > it would be helpful. The barrier to more participation isn't that people > don't know there's work to be done. I suspect that anyone who was going > to consider helping with development would either be scratching their > own itch (ie, contributing a patch and wouldn't care about this report), > or would already be reading python-dev (after all, it's not hard to > qualify to receive it!). I completely disagree (and I have to, because I was supporting more wide distribution :-). The posting lists bug reports and patches that some people might consider interesting, and just try out in order to find out whether it scratches their own itches. If it does, they might comment on the patch. If a patch scratches nobody's itches, it is best rejected. > I worry that posting this to c.l.py would be > unlikely to generate anything other than occasional bellyaching about > how slow we are about addressing bug reports. And that would be a good thing! People *need* to remember that this is free software, and that we are all volunteers. For some, this might cause them to go away, with so little support from the developers. Farewell to them! Others might consider taking over the project. Let them come! The rest will have an occasional bellyaching, which will remind them of how precisely free software works. They'll either get over it, or do something about it. > What might be more helpful would be a call for volunteers. We could > form a "patch reviewers" group that would review patches on SF. Or > something. Actually, figuring out how to recruit volunteers is a > difficult problem. But I'm not sure the weekly bug/patch summary is the > solution. Perhaps no. However, I do think I have personally exhausted all other possibilities, with regular postings on all available channels. It might not help, but it won't harm, and it is easy to implement. Regards, Martin From skip at pobox.com Mon Aug 23 22:38:28 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 23 22:38:41 2004 Subject: [Python-Dev] #ifdeffery In-Reply-To: <412A507A.7050203@v.loewis.de> References: <2mpt5h7upy.fsf@starship.python.net> <412A507A.7050203@v.loewis.de> Message-ID: <16682.21956.821020.924013@montanaro.dyndns.org> Martin> Actually writing a patch, and testing it (including the two Martin> recompilations that will take) is too much effort, so I rather Martin> write a longish email seeking volunteers... Teach them to fish... Skip From walter at livinglogic.de Mon Aug 23 22:41:55 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Mon Aug 23 22:42:01 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <4124FA07.1070701@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> Message-ID: <412A5693.4050008@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> Both approaches have one problem: Error handling won't >> work for them. If the error handling is "replace", the decoder >> should return U+FFFD for the final trailing incomplete sequence >> in read(). This won't happen when read() never reads those >> bytes from the input stream. > > Ok. So it really looks like a final flag on read is necessary. > >> Well, I had to choose a subject. ;) > > I still would prefer if the two issues were discussed separately. OK, let's come up with a patch that fixes the incomplete byte sequences problem and then discuss non-stream APIs. So, what should the next step be? Bye, Walter D?rwald From bac at OCF.Berkeley.EDU Mon Aug 23 22:41:53 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 23 22:42:17 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <001101c48946$19591100$e841fea9@oemcomputer> References: <001101c48946$19591100$e841fea9@oemcomputer> Message-ID: <412A5691.5080703@ocf.berkeley.edu> Raymond Hettinger wrote: > [Barry] > >>>... I'll drop this if we can agree that the PEP 292 >>>classes should go in the existing string module. > > > [Brett] > >>OK, that works for me. > > > Does it? Apply the patch and try "import re". > Sorry, should have been clearer. I meant that I didn't have an issue with not making 'string' a package anymore with Barry strongly wording what is deprecated. I have not looked at the import issue. -Brett From mal at egenix.com Mon Aug 23 23:56:37 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Aug 23 23:56:42 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <200408191838.i7JIcmm21008@guido.python.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com> <41227F83.4060403@egenix.com> <41231830.2090903@egenix.com> <200408190139.i7J1dX819100@guido.python.org> <41246FA5.1010804@egenix.com> <200408191838.i7JIcmm21008@guido.python.org> Message-ID: <412A6815.2050307@egenix.com> Guido van Rossum wrote: >>Ok, so I suppose that we can learn from Jython and IronPython in >>this respect... >> >>How do they handle binary data and the interfacing between various >>I/O facilities, e.g. files, sockets, pipes, user input, etc. > > > I'm not sure, but I expect that in most cases they use Unicode strings > in order to be compatibly with Python's standard library. That's not > the outcome I'd like to see though. I believe Jython at least also > has a bytes-like type (probably a thin wrapper around Java's byte > array) that's used for interfacing to java classes. I've had a discussion with Jack Janssen about using bytes as default return value for I/O operations where no encoding is specified (or unknown). He raised the issue of bytes not being usable as dictionary keys due to their mutability. He was also concerned about the increase in complexity when writing programs that work with non-text data or mixed text/data I/O. If we want to make the move from Python 2.x to 3.0 feasable for large code bases, then we have to do something about these issues. It seems that the simple solution of going with Unicode + bytes type is not going to be a suitable approach. Anyway, we still have 4-5 years to think about this :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 23 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From pm_mon at yahoo.com Mon Aug 23 01:49:11 2004 From: pm_mon at yahoo.com (Paul Morrow) Date: Tue Aug 24 00:12:58 2004 Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators In-Reply-To: <200408222110.i7MLAZJ16423@guido.python.org> References: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com> <41284983.3000303@v.loewis.de> <4128B778.1080503@v.loewis.de> <200408221902.i7MJ2sd02547@guido.python.org> <200408222110.i7MLAZJ16423@guido.python.org> Message-ID: <412930F7.9080605@yahoo.com> Guido van Rossum wrote: > > Your recent posts seemed out of line to me only because you were being > very argumentative and at the same time exposed serious > misunderstanding (*). That's usually a clue to back off... High > frequency posters are only okay if their posts have a high information > content. > > (*) Which you acknowledged, but in messages showing more poor form by > containing many dozens of lines of quoted material followed by a > single "I see". > My apologies to all. I'll be more careful in the future. Paul From revol at free.fr Mon Aug 23 14:26:07 2004 From: revol at free.fr (=?windows-1252?q?Fran=E7ois?= Revol) Date: Tue Aug 24 00:13:04 2004 Subject: [Python-Dev] problem with pymalloc on the BeOS port. Message-ID: <2246153618-BeMail@taz> Hi there, I'm updating the BeOS port of python to include the latest version in Zeta, the next incarnation of BeOS. I'm having some problem when enabling pymalloc: case $MAKEFLAGS in \ *-s*) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/ lib:/boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos libpython2.3.so' OPT='-O' ./python -E ./setup.py -q build;; \ *) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/lib:/ boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos libpython2.3.so' OPT='-O' ./python -E ./setup.py build;; \ esac running build running build_ext building 'struct' extension gcc -O -fno-strict-aliasing -I. -I/boot/home/Python-2.3.4/./Include -I/ boot/home/config/include -I/boot/home/Python-2.3.4/Include -I/boot/home /Python-2.3.4 -c /boot/home/Python-2.3.4/Modules/structmodule.c -o build/temp.beos-5.1-BePC-2.3/structmodule.o Debug memory block at address p=0x80010fb8: 0 bytes originally requested The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected. The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE (0xfb): at tail+0: 0xdb *** OUCH at tail+1: 0xdb *** OUCH at tail+2: 0xdb *** OUCH at tail+3: 0xdb *** OUCH The block was made by call #3688627195 to debug malloc/realloc. Fatal Python error: bad trailing pad byte indeed, there seem to be 4 deadblock marks between the forbidden ones, while the len is supposed to be null: python:dm 0x80010fb8-8 32 80010fb0 00000000 fbfbfbfb dbdbdbdb fbfbdbdb ................ 80010fc0 0100fbfb 507686ef 04000000 fbfbfbfb ......vP........ 80010fd0 8013cbc8 fbfbfbfb 44ee0100 ffed0100 ...........D.... Any clue ? Fran?ois. From janusfury at citlink.net Mon Aug 23 17:30:47 2004 From: janusfury at citlink.net (Kevin Gadd) Date: Tue Aug 24 00:13:10 2004 Subject: [Python-Dev] Python icons Message-ID: <412A0DA7.9070306@citlink.net> I was bored the other day so I played around with an image editor and redid most of the windows python icons. Image of most of the icons (I bet this is going to murder my bandwidth): http://luminance.org/py/screenshot.png I did the basic one in vector. Let me know what you guys think, I figure since they came out pretty decent that somebody might be able to make something of 'em! And, just to get in a bit of sucking up, thanks to all you devs for the great work you've done on the language. Keep it up! -- Kevin Gadd E-Mail | Website -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040823/37d30992/attachment.htm From ncoghlan at iinet.net.au Tue Aug 24 01:49:52 2004 From: ncoghlan at iinet.net.au (ncoghlan@iinet.net.au) Date: Tue Aug 24 01:49:57 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <002401c48931$dda69060$e841fea9@oemcomputer> References: <002401c48931$dda69060$e841fea9@oemcomputer> Message-ID: <1093304992.412a82a0a5a88@mail.iinet.net.au> Quoting Raymond Hettinger : > > This code-snippet is littered everwhere in my applications: > > > > string.join([str(x) for x in iterable]) > > > > Its tedious and makes code hard to read. Do we need a PEP to fix > this? > > A PEP would be overkill. > > Still, it would be helpful to do PEP-like things such as reference > implementation, soliticing comments, keep an issue list, etc. > > A minor issue is that the implementation automatically shifts to Unicode > upon encountering a Unicode string. So you would need to test for this > before coercing to a string. Perhaps have string join coerce to string, and Unicode join coerce to the separator's encoding. If we do that, the existing string->Unicode promotion code should handle the switch between the two join types. > Also, join works in multiple passes. The proposal should be specific > about where stringizing occurs. IIRC, you need the object length on the > first pass, but the error handling and switchover to Unicode occur on > the second. Having been digging in the guts of string join last week, I'm pretty sure the handover to the Unicode join happens on the first 'how much space do we need' pass (essentially, all of the work done so far is thrown away, and the Unicode join starts from scratch. If you know you have Unicode, you're better off using a Unicode separator to avoid this unnecessary work ). We then have special casing of zero length and single item sequences, before dropping into the 'build the new string' loop. By flagging the need for a 'stringisation' operation in the failed side of the 'PyUnicode_Check' that occurs during the first pass (to see if we should hand over to the Unicode join), we could avoid unnecessarily slowing the pure string cases. To keep the speed of the pure-string case, we would need to guarantee that the sequence consists of only strings when we run through the final pass to build the new string. So we would need an optional second pass that constructs a new sequence, containing any strings from the original sequence, plus 'stringised' versions of the non-strings. The final pass could remain as-is. The only possible difference is that it may be operating on the new 'stringised' sequence rather than the old one. The alternative implementation (checking each item's type as it is added to the new string in the final pass) has the significant downside of slowing down the existing case of joining only strings. Either implementation should still be a lot faster than ''.join([str(x) for x in seq]) though. Time to go knock out some code, I think. . . Cheers, Nick. P.S. "'\n'.join(locals().items())" sure would be pretty, though From tjreedy at udel.edu Tue Aug 24 02:03:38 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Tue Aug 24 02:03:42 2004 Subject: [Python-Dev] Re: Weekly Python Bug/Patch Summary References: <1093278123.412a19ab99d0b@mcherm.com> <412A5232.6060202@v.loewis.de> Message-ID: "Martin v. Löwis" wrote in message news:412A5232.6060202@v.loewis.de... > I completely disagree (and I have to, because I was supporting more > wide distribution :-). I also support clp posting because the little bit of help I have been able to give (on bug reports) has mostly been prompted by the weekly reminder. I can imagine that there are people even more competant to contribute who only read or skim c.l.p. If the consequences are net negative, it will be easy to cease the additional posting. Terry J. Reedy From tjreedy at udel.edu Tue Aug 24 02:33:09 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Tue Aug 24 02:33:14 2004 Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators? References: <20040823153923.GA50408@prometheusresearch.com> Message-ID: "Clark C. Evans" wrote in message news:20040823153923.GA50408@prometheusresearch.com... > I just read the thread 'Stackless Python' in June 2004 on python-dev and > was wondering if you'd comment Sure > on a simpler cooperative mechanism, via a small hack to generators: If it is possible, as proposed, maybe at least a medium hack? > 1. The PEP would introduce a new 'builtin' class called 'Cooperate' > > 2. Generator semantics would be altered so that 'yield X', where X > is an instance of Cooperate, would automagically propigate to > the outer-most non-generator. -1 Skipping over the generic objections of 'hard to teach to beginners' and 'opens to door to a hundred other such specialized features': 1. This alters and complexifies the current generator concept. At present, 'generator' (the return type of any generator function) is an instance of the iterator concept, as is 'iterator' (the return type of iter()) and any class with appropriate .__iter__ and .next methods. A generator .next method is a resumable function; yield is like return except not permanent. The local namespace is not destroyed and instruction pointer not reset for the next call. >From the outside, a generator .next method is, I believe, indistinguishable in Python (excluding possible implementation-specific voodoo) from an interator-type .next method and behaviorally indistingueshable from any other iterator-protocol parameterless .next method. As far as I know, the difference between a generator and other iterators is ease of writing (sometimes *much* easier) and speed of execution, not external behavior. 2. This imposes an isinstance(x, magic.Collaborate) function call cost on every yield in every generator. (I do not believe the alternative of imposing the same cost on every function call in every generator will work since I do not believe that callers can dependably know whether objects are returned or yielded. See comments on example.) 3. 'automagic' it is. The pertinent runtime call stack contains not generator objects but .next methods (or in CPython, method-wrappers). So I presume you are proposing that yielded Collaborates magically plunk themselves into the first frame up the stack not associated such objects. 4. If the returned to frame does not re-call the passed over suspended frames, they and anything they hold onto are leaked until gc'ed. > >>> def MyCooperate(magic.Cooperate): > >>> __str__(self): return "cooperate" > >>> > >>> def top(): > >>> yield "one" > >>> yield MyCooperate() > >>> yield "two" > >>> > >>> def middle(): > >>> """ intermediate generator _only_ sees one and two """ > >>> for x in top(): > >>> print "middle", x > >>> yield x Here, the effect you seek could be accomplished by an explicit 'if not isinstance(x, Collaborate):' before the print statement (or, in general, the block of code that uses x). But what if top is rewritten as def top(): return ("one", MyCooperate(), "two") Do you really want the behavior of middle to change, as with your proposal? > >>> def lower(): > >>> """ this is not a generator, so it sees cooperate """ > >>> for x in middle(): > >>> print "lower", x Now add 'if isinstance(x, Cooperate): return' and the suspended middle will be left in limbo. Not very tidy ;-) Terry J. Reedy From kbk at shore.net Tue Aug 24 03:21:40 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Aug 24 03:21:44 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <1093287849.412a3fa9ef447@mcherm.com> (Michael Chermside's message of "Mon, 23 Aug 2004 12:04:09 -0700") References: <1093287849.412a3fa9ef447@mcherm.com> Message-ID: <87pt5h49gb.fsf@hydra.localdomain> Michael Chermside writes: > Well, if there's ANY decent chance that you're right, then it's worth > sending the email. So I'll enthustiastically back the idea of sending > the bug/patch summary to c.l.py. OK, and we can always stop if there are unforseen consequences. Anyone else? -- KBK From cce at clarkevans.com Tue Aug 24 03:54:53 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Tue Aug 24 03:54:56 2004 Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators? In-Reply-To: References: <20040823153923.GA50408@prometheusresearch.com> Message-ID: <20040824015453.GA10615@prometheusresearch.com> Thanks for responding Terry. I'm not sure if my exposition was clear. On Mon, Aug 23, 2004 at 08:33:09PM -0400, Terry Reedy wrote: | > 1. The PEP would introduce a new 'builtin' class called 'Cooperate' | > | > 2. Generator semantics would be altered so that 'yield X', where X | > is an instance of Cooperate, would automagically propigate to | > the outer-most non-generator. Suppose I have three 'frames', two generators and one function, the call stack looks like: g2 g1 fn For this to occur, fn has called .next() on g1, which has called .next() on g2. The proposed behavior is that "yield magic.Collaborate" provides the value returned by g1.next() within fn. In particular, it would be equivalent as if every "val = g2.next()" in g1 was rewritten: val = g2.next() if isinstance(val, magic.Cooperate): yield val One could do this in three ways: (a) by a convention above, this is done in twisted.flow, (b) by a bytecode hack, or (c) by modifying eval.c to have equivalent behavior. | 1. This alters and complexifies the current generator concept. Correct; it introduces something similar to a corountine. | 2. This imposes an isinstance(x, magic.Collaborate) function call cost on | every yield in every generator. Ok. So, in addition to causing unexpected results (one would have to check to see what sort of object to debug), it would also cause a performance hit. | 3. 'automagic' it is. The pertinent runtime call stack contains not | generator objects but .next methods (or in CPython, method-wrappers). So I | presume you are proposing that yielded Collaborates magically plunk | themselves into the first frame up the stack not associated such objects. Right, the first frame that is not a generator, aka, not having a yield statement. And yes, it is different semantics. | 4. If the returned to frame does not re-call the passed over suspended | frames, they and anything they hold onto are leaked until gc'ed. No. See the above equivalence. It might take a bit of thinking to make the "C" implementation correct, but the semantics are farily straight-forward and should not create leaks. | Here, the effect you seek could be accomplished by an explicit 'if not | isinstance(x, Collaborate):' before the print statement (or, in general, | the block of code that uses x). But what if top is rewritten as | def top(): return ("one", MyCooperate(), "two") | Do you really want the behavior of middle to change, as with your proposal? Yes. | > >>> def lower(): | > >>> """ this is not a generator, so it sees cooperate """ | > >>> for x in middle(): | > >>> print "lower", x | | Now add 'if isinstance(x, Cooperate): return' and the suspended middle will | be left in limbo. Not very tidy ;-) No, it would be just like a generator had 'if 1: return'; the suspended middle would be cleaned up just as it is today. ... However, you've convinced me on the semantic differences and the performance hit. I guess the proposal would have to present a new keyword, say 'collaborate' and have to have its semantics clearly specified. Thank you _so_ much for your kind review, Clark From tim.peters at gmail.com Tue Aug 24 04:05:00 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 24 04:05:03 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <20040823164109.GB94018@prometheusresearch.com> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> <20040823164109.GB94018@prometheusresearch.com> Message-ID: <1f7befae040823190560339fff@mail.gmail.com> [Clark C. Evans] > This code-snippet is littered everwhere in my applications: > > string.join([str(x) for x in iterable]) > > Its tedious and makes code hard to read. Do we need a PEP to fix > this? You won't need a PEP to replace it with the similar code-snippet from my code: string.join(map(str, iterable)) Same thing in the end, but map reads quite well (perhaps even better than a listcomp) in applications as simple as this. From tim.peters at gmail.com Tue Aug 24 04:27:33 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 24 04:27:39 2004 Subject: [Python-Dev] problem with pymalloc on the BeOS port. In-Reply-To: <2246153618-BeMail@taz> References: <2246153618-BeMail@taz> Message-ID: <1f7befae04082319274df5de61@mail.gmail.com> [Fran?ois Revol] > I'm updating the BeOS port of python to include the latest version in > Zeta, the next incarnation of BeOS. > > I'm having some problem when enabling pymalloc: > > case $MAKEFLAGS in \ > *-s*) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/ > lib:/boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos > libpython2.3.so' OPT='-O' ./python -E ./setup.py -q build;; \ > *) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/lib:/ > boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos > libpython2.3.so' OPT='-O' ./python -E ./setup.py build;; \ > esac > running build > running build_ext > building 'struct' extension > gcc -O -fno-strict-aliasing -I. -I/boot/home/Python-2.3.4/./Include -I/ > boot/home/config/include -I/boot/home/Python-2.3.4/Include -I/boot/home > /Python-2.3.4 -c /boot/home/Python-2.3.4/Modules/structmodule.c -o > build/temp.beos-5.1-BePC-2.3/structmodule.o > Debug memory block at address p=0x80010fb8: > 0 bytes originally requested > The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected. > The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE > (0xfb): > at tail+0: 0xdb *** OUCH > at tail+1: 0xdb *** OUCH > at tail+2: 0xdb *** OUCH > at tail+3: 0xdb *** OUCH > The block was made by call #3688627195 to debug malloc/realloc. No it wasn't. The four bytes following the 4 trailing 0xfb hold the call number, and they're apparently corrupt too. > Fatal Python error: bad trailing pad byte > > indeed, there seem to be 4 deadblock marks between the forbidden ones, > while the len is supposed to be null: That's reliable. If there actually was a request for 0 bytes (that is, assuming this pointer isn't just pointing at a random memory address), the debug pymalloc allocates 16 bytes for it, filled with 00000000 fbfbfbfb fbfbfbfb serial where "serial" is a big-endian 4-byte "serial number" uniquely identifying which call to malloc (or realloc) this was. The address of the second block of fb's is returned. > python:dm 0x80010fb8-8 32 > 80010fb0 00000000 fbfbfbfb dbdbdbdb fbfbdbdb > ................. > 80010fc0 0100fbfb 507686ef 04000000 fbfbfbfb > .......vP........ > 80010fd0 8013cbc8 fbfbfbfb 44ee0100 ffed0100 > ............D.... > > Any clue ? Try gcc without -O. Nobody has reported anything like this before -- you're in for a lot of fun . From greg at cosc.canterbury.ac.nz Tue Aug 24 04:50:19 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 24 04:50:24 2004 Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators? In-Reply-To: <20040824015453.GA10615@prometheusresearch.com> Message-ID: <200408240250.i7O2oJCI002327@cosc353.cosc.canterbury.ac.nz> > it would be > equivalent as if every "val = g2.next()" in g1 was rewritten: > > val = g2.next() > if isinstance(val, magic.Cooperate): > yield val Something bothers me about that proposal, and I think it's this: it would mean that generators are no longer simply a particular technique for implementing iterators, but would have special powers that other iterators don't have. Unless recognising the magic Cooperate value and doing the right thing were made part of the iterator protocol, and all iterators were expected to do the right thing with it, which would be a big imposition. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From djc at object-craft.com.au Tue Aug 24 04:53:06 2004 From: djc at object-craft.com.au (Dave Cole) Date: Tue Aug 24 04:53:10 2004 Subject: [Python-Dev] Submitted patch #1014930 to exposes the current parse location in pyexpat Message-ID: <412AAD92.9010009@object-craft.com.au> I would like someone to review the patch I just submitted - it exposes more the the expat API to the xml.parsers.expat.XMLParser object. Expat supplies the XML_GetCurrentLineNumber, XML_GetCurrentColumnNumber, and XML_GetCurrentByteIndex functions to obtain the current parse location. The patch exposes these as the following respective members of the xml.parsers.expat.XMLParser object; CurrentLineNumber, CurrentColumnNumber, CurrentByteIndex. I have checkin permission, I just want someone to cast their eye over the patch before I commit. - Dave -- http://www.object-craft.com.au From tim.peters at gmail.com Tue Aug 24 05:02:34 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 24 05:02:41 2004 Subject: [Python-Dev] Missing peice of PEP 237 (int unification) Message-ID: <1f7befae040823200212c31262@mail.gmail.com> We're still generating OverflowWarnings in CVS: >>> import warnings >>> warnings.filterwarnings("error", "", OverflowWarning) >>> import sys >>> sys.maxint + 1 Traceback (most recent call last): File "", line 1, in ? OverflowError: integer addition >>> The PEP says OverflowWarning was for "transition phase A": Here are the rules that guide warnings generated in situations that currently raise OverflowError. This applies to transition phase A. Phase A was done in 2.2. Phase B0 was done in 2.3, and 2.4 is supposed to be phase B1. The OverflowWarnings are still there, but should have gone away in 2.3 already. It would be a lot of fun to rip out the bletcherous code raising OverflowWarning, and finding the infinite loop that will then occur in the test suite (you think I'm kidding -- heh). One question the PEP doesn't answer: does PyExc_OverflowWarning vanish from the C API at the same time? Does OverflowWarning vanish from Python too? Or do we need to add a PendingDeprecationWarning for OverflowWarning in 2.4, a DeprecationWarning for OverflowWarning in 2.5, and finally nuke the silly bastard tin 2.6? From guido at python.org Tue Aug 24 06:02:01 2004 From: guido at python.org (Guido van Rossum) Date: Tue Aug 24 06:02:15 2004 Subject: [Python-Dev] Missing piece of PEP 237 (int unification) In-Reply-To: Your message of "Mon, 23 Aug 2004 23:02:34 EDT." <1f7befae040823200212c31262@mail.gmail.com> References: <1f7befae040823200212c31262@mail.gmail.com> Message-ID: <200408240402.i7O421b19106@guido.python.org> > The PEP says OverflowWarning was for "transition phase A": > > Here are the rules that guide warnings generated in situations > that currently raise OverflowError. This applies to transition > phase A. > > Phase A was done in 2.2. Phase B0 was done in 2.3, and 2.4 is > supposed to be phase B1. The OverflowWarnings are still there, but > should have gone away in 2.3 already. > > It would be a lot of fun to rip out the bletcherous code raising > OverflowWarning, and finding the infinite loop that will then occur in > the test suite (you think I'm kidding -- heh). Sounds like a good idea. I totally forgot about this part, because it's so nearyly invisible. :-) > One question the PEP doesn't answer: does PyExc_OverflowWarning > vanish from the C API at the same time? Does OverflowWarning vanish > from Python too? Or do we need to add a PendingDeprecationWarning for > OverflowWarning in 2.4, a DeprecationWarning for OverflowWarning in > 2.5, and finally nuke the silly bastard tin 2.6? I would keep the APIs for one release and then nuke them, but not add any deprecation code. Just document them as on the way out. --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Tue Aug 24 06:56:13 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 24 06:56:21 2004 Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators? In-Reply-To: <200408240250.i7O2oJCI002327@cosc353.cosc.canterbury.ac.nz> References: <200408240250.i7O2oJCI002327@cosc353.cosc.canterbury.ac.nz> Message-ID: <412ACA6D.1090908@ocf.berkeley.edu> Greg Ewing wrote: >>it would be >>equivalent as if every "val = g2.next()" in g1 was rewritten: >> >> val = g2.next() >> if isinstance(val, magic.Cooperate): >> yield val > > > Something bothers me about that proposal, and I think it's this: it > would mean that generators are no longer simply a particular technique > for implementing iterators, but would have special powers that other > iterators don't have. Unless recognising the magic Cooperate value and > doing the right thing were made part of the iterator protocol, and all > iterators were expected to do the right thing with it, which would be > a big imposition. > I agree with Greg and the other comments on wanting to keep generators simple. Part of the beauty of generators is the are brain-dead simple to use while still being handy as hell. Playing with them to add the ability to play asynchronous stuff doesn't seem right. I would rather see full-blown coroutines for Python 3000 to deal with this somehow. -Brett From martin at v.loewis.de Tue Aug 24 07:13:29 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 24 07:13:34 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412A5693.4050008@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> Message-ID: <412ACE79.4060605@v.loewis.de> Walter D?rwald wrote: > OK, let's come up with a patch that fixes the incomplete byte > sequences problem and then discuss non-stream APIs. > > So, what should the next step be? I think your first patch should be taken as a basis for that. Add the state-supporting decoding C functions, and change the stream readers to use them. That still leaves the issue of the last read operation, which I'm tempted to leave unresolved for Python 2.4. No matter what the solution is, it would likely require changes to all codecs, which is not good. Regards, Martin From martin at v.loewis.de Tue Aug 24 07:50:16 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 24 07:50:17 2004 Subject: [Python-Dev] problem with pymalloc on the BeOS port. In-Reply-To: <1f7befae04082319274df5de61@mail.gmail.com> References: <2246153618-BeMail@taz> <1f7befae04082319274df5de61@mail.gmail.com> Message-ID: <412AD718.7020602@v.loewis.de> Tim Peters wrote: >>Any clue ? > > > Try gcc without -O. Nobody has reported anything like this before -- > you're in for a lot of fun . In addition, try to find out whether BeOS' malloc is somehow "strange". Does the system have the notion of a linear address space? Does it do pages? and so on. If you find a feature that you wouldn't expect in Windows NT, Linux, or HP-UX (*), that might give a clue. Regards, Martin (*) I did not expect the Spanish Inquisition. From greg at cosc.canterbury.ac.nz Tue Aug 24 07:57:18 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Aug 24 07:57:30 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> Message-ID: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz> > My comment about "coroutines" was more that Guido previously > expressed distaste for adding such a communication mechanism to > generators as abusing the concept of a generator just being a way to > implement complex iterators. This makes me think that maybe we want another kind of object, similar to a generator, but designed to be used for side effects rather than to produce values. For want of a better term, let's call it a "cooperator" (so it ends in "ator" like "generator" :-). Actually there will be two objects, a cooperator-function (analogous to a generator-function) and a cooperator-instance (analogous to a generator-iterator). Calling a cooperator-function will return a cooperator-instance, which will obey the cooperator protocol. This protocol consists of a single function run(), which causes the cooperator-instance to perform some amount of work and then stop. If there is more work remaining to be done, run() returns a true value, otherwise it returns a false value. There will be a new statement: suspend for use inside a cooperator-function. The presence of this statement is what distinguishes a cooperator-function from an ordinary function (just as the presence of 'yield' distinguishes a generator from an ordinary function). Execution of 'suspend' will cause the run() method to return with a true value. The next time run() is called, execution will resume after the 'suspend'. This is really all that's needed, but one further piece of syntax may be desirable to make it easier for one cooperator to invoke another: do another_coop() would be equivalent to co = another_coop() while co.run(): suspend (Because it implies a 'suspend', the presence of 'do' would also mark a function as a cooperator-function). Something to note is that there's no requirement for a cooperator- instance to be implemented by a cooperator-function, just as an iterator can be implemented in ways other than a generator. This sidesteps some of the difficulties of mixing Python and C calls, since a Python cooperator-function could invoke a cooperator implemented in C which in turn invokes another Python cooperator- function, etc. The only requirement is that all the objects on the way down obey the cooperator protocol. The drawback is that you always have to know when you're calling another cooperator and use 'do'. But that's no worse than the current situation with generators, where you always have to know you're using an iterator and use 'for... yield'. What think ye all? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From martin at v.loewis.de Tue Aug 24 07:57:45 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 24 07:57:46 2004 Subject: [Python-Dev] Python icons In-Reply-To: <412A0DA7.9070306@citlink.net> References: <412A0DA7.9070306@citlink.net> Message-ID: <412AD8D9.3050509@v.loewis.de> Kevin Gadd wrote: > I was bored the other day so I played around with an image editor and > redid most of the windows python icons. Here is my man! (sorry if this is not the proper English phrase) I have finally moved the Windows installer generator into the Python source tree proper (Tools/msi). In the process, I had to delete two icons taken from Microsoft sample code, for licensing issues, and had to replace them with plain text, because I have virtually no graphic skills. They are both 16x16 icons, one indicating "go up one directory", and the other "create a new folder". The original Microsoft bitmaps are at http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/msi/Attic/Up.bin?rev=1.1.1.1 http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/msi/Attic/New.bin?rev=1.1.1.1 Can you help creating a pair of such icons which are similar in appearance but not quite the same? Regards, Martin From martin at v.loewis.de Tue Aug 24 08:00:49 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 24 08:00:50 2004 Subject: [Python-Dev] Python icons In-Reply-To: <412A0DA7.9070306@citlink.net> References: <412A0DA7.9070306@citlink.net> Message-ID: <412AD991.3080404@v.loewis.de> Kevin Gadd wrote: > I did the basic one in vector. Let me know what you guys think, I figure > since they came out pretty decent that somebody might be able to make > something of 'em! Responding to your actual question: They look nice to me, indeed; assuming there are no down-sides, I'd be +1 for including them. Of course, as apparently always the case with artwork, we would need to know the licensing conditions that you want to associate with using them. Regards, Martin From fdrake at acm.org Tue Aug 24 09:07:38 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue Aug 24 09:07:50 2004 Subject: [Python-Dev] Submitted patch #1014930 to exposes the current parse location in pyexpat In-Reply-To: <412AAD92.9010009@object-craft.com.au> References: <412AAD92.9010009@object-craft.com.au> Message-ID: <200408240307.38486.fdrake@acm.org> On Monday 23 August 2004 10:53 pm, Dave Cole wrote: > Expat supplies the XML_GetCurrentLineNumber, XML_GetCurrentColumnNumber, > and XML_GetCurrentByteIndex functions to obtain the current parse > location. The patch exposes these as the following respective members > of the xml.parsers.expat.XMLParser object; CurrentLineNumber, > CurrentColumnNumber, CurrentByteIndex. This sounds like a good idea to me. Please post the patch on SF and assign to me. Thanks! -Fred -- Fred L. Drake, Jr. From mal at egenix.com Tue Aug 24 10:39:32 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 24 10:39:36 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1093304992.412a82a0a5a88@mail.iinet.net.au> References: <002401c48931$dda69060$e841fea9@oemcomputer> <1093304992.412a82a0a5a88@mail.iinet.net.au> Message-ID: <412AFEC4.6020007@egenix.com> ncoghlan@iinet.net.au wrote: > Quoting Raymond Hettinger : > > >>>This code-snippet is littered everwhere in my applications: >>> >>> string.join([str(x) for x in iterable]) >>> >>>Its tedious and makes code hard to read. Do we need a PEP to fix >> >>this? >> >>A PEP would be overkill. >> >>Still, it would be helpful to do PEP-like things such as reference >>implementation, soliticing comments, keep an issue list, etc. >> >>A minor issue is that the implementation automatically shifts to Unicode >>upon encountering a Unicode string. So you would need to test for this >>before coercing to a string. > > Perhaps have string join coerce to string, and Unicode join coerce to the > separator's encoding. If we do that, the existing string->Unicode promotion code > should handle the switch between the two join types. The general approach is always to coerce to Unicode if strings and Unicode meet; very much like coercion to floats is done when integers and floats meet. Your suggestion would break this logic and make coercion depend on an argument. >>Also, join works in multiple passes. The proposal should be specific >>about where stringizing occurs. IIRC, you need the object length on the >>first pass, but the error handling and switchover to Unicode occur on >>the second. > > > Having been digging in the guts of string join last week, I'm pretty sure the > handover to the Unicode join happens on the first 'how much space do we need' > pass (essentially, all of the work done so far is thrown away, and the Unicode > join starts from scratch. If you know you have Unicode, you're better off using > a Unicode separator to avoid this unnecessary work ). It's just a simple length querying loop; there's no storage allocation or anything expensive happening there, so the "throw-away" operation is not expensive. Aside: ''.join() currently only works for true sequences - not iterators. OTOH, the %-format operation is which is why PyString_Format goes through some extra hoops to make sure the work already is not dropped (indeed, it may not even be possible to reevaluate the arguments; think iterators here). We could probably add similar logic to ''.join() to have it also support iterators. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 24 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Tue Aug 24 11:36:45 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 24 11:36:47 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412ACE79.4060605@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> Message-ID: <412B0C2D.7030507@egenix.com> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> OK, let's come up with a patch that fixes the incomplete byte >> sequences problem and then discuss non-stream APIs. >> >> So, what should the next step be? > > I think your first patch should be taken as a basis for that. We do need a way to communicate state between the codec and Python. However, I don't like the way that the patch implements this state handling: I think we should use a generic "state" object here which is passed to the stateful codec and returned together with the standard return values on output: def decode_stateful(data, state=None): ... decode and modify state ... return (decoded_data, length_consumed, state) where the object type and contents of the state variable is defined per codec (e.g. could be a tuple, just a single integer or some other special object). Otherwise we'll end up having different interface signatures for all codecs and extending them to accomodate for future enhancement will become unfeasable without introducing yet another set of APIs. Let's discuss this some more and implement it for Python 2.5. For Python 2.4, I think we can get away with what we already have: If we leave out the UTF-7 codec changes in the patch, the only state that the UTF-8 and UTF-16 codecs create is the number of bytes consumed. We already have the required state parameter for this in the standard decode API, so no extra APIs are needed for these two codecs. So the patch boils down to adding a few new C APIs and using the consumed parameter in the standard _codecs module APIs instead of just defaulting to the input size (we don't need any new APIs in _codecs). > Add the state-supporting decoding C functions, and change > the stream readers to use them. The buffer logic should only be used for streams that do not support the interface to push back already read bytes (e.g. .unread()). From a design perspective, keeping read data inside the codec is the wrong thing to do, simply because it leaves the input stream in an undefined state in case of an error and there's no way to correlate the stream's read position to the location of the error. With a pushback method on the stream, all the stream data will be stored on the stream, not the codec, so the above would no longer be a problem. However, we can always add the .unread() support to the stream codecs at a later stage, so it's probably ok to default to the buffer logic for Python 2.4. > That still leaves the issue > of the last read operation, which I'm tempted to leave unresolved > for Python 2.4. No matter what the solution is, it would likely > require changes to all codecs, which is not good. We could have a method on the codec which checks whether the codec buffer or the stream still has pending data left. Using this method is an application scope consideration, not a codec issue. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 24 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From walter at livinglogic.de Tue Aug 24 11:38:12 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Aug 24 11:38:16 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: References: Message-ID: <412B0C84.30204@livinglogic.de> rhettinger@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Lib/test > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20193/Lib/test > > Modified Files: > test_string.py > Log Message: > SF Patch #1007087: Return new string for single subclass joins (Bug #1001011) > (Patch contributed by Nick Coghlan.) > > Now joining string subtypes will always return a string. > Formerly, if there were only one item, it was returned unchanged. test_string.py is supposed to test the string module. This new test should be moved to test_str.py (or even better to string_test.py, so that it can be reused for doing the same test with unicode objects). Bye, Walter D?rwald From jtk at yahoo.com Tue Aug 24 13:06:06 2004 From: jtk at yahoo.com (Jeff Kowalczyk) Date: Tue Aug 24 13:03:52 2004 Subject: [Python-Dev] Re: Python icons References: <412A0DA7.9070306@citlink.net> Message-ID: Kevin Gadd wrote: > I was bored the other day so I played around with an image editor and > redid most of the windows python icons... Let me know what you guys > think. Very nice to have a quality vector rendition. May I suggest a slight tweak to +smirk/grin and +top eyelid? He looks just a bit nervous in comparison to the original. http://www.python.org/wiki/python/img/odi.jpg http://luminance.org/py/screenshot.png From dbickett at gmail.com Tue Aug 24 13:30:50 2004 From: dbickett at gmail.com (Daniel Bickett) Date: Tue Aug 24 13:31:03 2004 Subject: [Python-Dev] Re: Python icons In-Reply-To: References: <412A0DA7.9070306@citlink.net> Message-ID: <1d6cdae304082404305fb5b553@mail.gmail.com> +1, those really look great :) Nice job. BTW, my name's Daniel Bickett, I've been lurking for a week or two with the intention of learning more. Finally something that isn't too far over my head to respond to! From pje at telecommunity.com Tue Aug 24 13:34:10 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 24 13:33:55 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz> References: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com> At 05:57 PM 8/24/04 +1200, Greg Ewing wrote: >What think ye all? I don't think it's in the right direction to achieve the effects desired by Clark or me. Here's a metaphor for what would be ideal: resumable exceptions. If you could throw a special kind of exception (or even a regular exception), and call traceback.resume() to pick up execution at the point where it was thrown, whether thrown by a generator or a regular function. If you had that capability, you could implement any sort of coroutine or task-switching facilities you wanted. Without such a facility, one needs a stack of *ators to emulate it, to provide the effects desired by Clark's Flow or by peak.events. However, co-operators' "suspend" doesn't provide a way to communicate between co-operators. At least 'yield' lets you yield a value. What's really needed (IMO) is to add a way to communicate *into* a co-operator, passing it a value to "accept" or a traceback to raise. E.g.: input = suspend output Where 'output' is returned from the current 'run()', and 'input' is a value passed to the next 'run()'. Or, if there's a type/value/traceback passed to a 'throw()' method, then the 'suspend' statement should raise an error. With that facility, 'peak.events' could drop the 'events.resume()' magic function. It'd still need a stack of co-operators, and there'd still be ugliness when iterating over a generator that needed to be suspendable. But at least it would be able to have clean syntax. (Though I don't think that 'input=suspend output' is actually clean syntax, it's just better than the yield/resume thing.) Anyway, as I said, what would be *most* useful for async programming is a way to resume a traceback, because then you wouldn't need for every intervening frame to have special suspension capability. From janusfury at citlink.net Tue Aug 24 12:54:14 2004 From: janusfury at citlink.net (Kevin Gadd) Date: Tue Aug 24 13:36:58 2004 Subject: [Python-Dev] Python icons In-Reply-To: <412AD991.3080404@v.loewis.de> References: <412A0DA7.9070306@citlink.net> <412AD991.3080404@v.loewis.de> Message-ID: <412B1E56.1060502@citlink.net> I'm not really worried about licensing... basically, I'd probably just want to keep my copyright on the actual vector art. I'm not terribly concerned about anything else, since they are based on the original python icons. I don't know much about how licensing artwork for open source projects works, so if you or someone else can let me know what steps would be necessary to make it possible for these icons to be used, I'd be glad to help. In response to your previous post, as well, I will try and put together replacements for those two icons. Martin v. L?wis wrote: > Kevin Gadd wrote: > >> I did the basic one in vector. Let me know what you guys think, I >> figure since they came out pretty decent that somebody might be able >> to make something of 'em! > > > Responding to your actual question: They look nice to me, indeed; > assuming there are no down-sides, I'd be +1 for including them. > Of course, as apparently always the case with artwork, we would > need to know the licensing conditions that you want to associate > with using them. > > Regards, > Martin > > -- Kevin Gadd E-Mail | Website -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040824/b2a7b731/attachment.htm From ajm at flonidan.dk Tue Aug 24 13:42:06 2004 From: ajm at flonidan.dk (Anders J. Munch) Date: Tue Aug 24 13:42:10 2004 Subject: [Python-Dev] Re: Update PEP 292 Message-ID: <6D9E824FA10BD411BE95000629EE2EC3C6DE1D@FLONIDAN-MAIL> >From Brett C. [mailto:bac@OCF.Berkeley.EDU] > > That is coming down to a question of convenience compared to > explicitness. Do we want "Give me $1,000,000 now!" to raise > ValueError, > or do we want to let it pass through? This particular problem has been considered and decided on years ago, in the context of %-interpolation: >>> "Give me %1,000,000 now!" % {} ValueError: unsupported format character ',' (0x2c) at index 10 String interpolation with named parameters given as a dictionary is a problem already solved in Python. I am puzzled at this urge to re-solve the same problem, only differently. If remembering to type the trailing s in "%(varname)s" is so much of a problem, why not extend the current interpolation syntax with a variation that is more convenient, yet still backwards-compatible? I would suggest a formatting-character-less version of "%(varname)s" that uses brackets instead: "%[varname]". dollar-signs?-we-don't-need-no-steenkin-dollar-signs-ly y'rs, Anders From dbickett at gmail.com Tue Aug 24 13:45:13 2004 From: dbickett at gmail.com (Daniel Bickett) Date: Tue Aug 24 13:45:21 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz> References: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz> Message-ID: <1d6cdae304082404452c15617b@mail.gmail.com> On Tue, 24 Aug 2004 17:57:18 +1200, Greg Ewing wrote: > > My comment about "coroutines" was more that Guido previously > > expressed distaste for adding such a communication mechanism to > > generators as abusing the concept of a generator just being a way to > > implement complex iterators. > > This makes me think that maybe we want another kind of object, similar > to a generator, but designed to be used for side effects rather than > to produce values. > > For want of a better term, let's call it a "cooperator" (so it ends in > "ator" like "generator" :-). Actually there will be two objects, a > cooperator-function (analogous to a generator-function) and a > cooperator-instance (analogous to a generator-iterator). > > Calling a cooperator-function will return a cooperator-instance, which > will obey the cooperator protocol. This protocol consists of a single > function run(), which causes the cooperator-instance to perform some > amount of work and then stop. If there is more work remaining to be > done, run() returns a true value, otherwise it returns a false value. Tell me if I'm missing something here, but with the exception of your two introductory keywords (a subject I thought was supposed to be sacred, with the intention of keeping the amount of keywords as small as possible,) how much difference is there to generators? Are you saying the run() method, after having been called initially, will return a value indicative of the cooperator's business when called again? It seems like a similar feat could be achieved with custom generators and boolean yield statements. Daniel Bickett From anthony at interlink.com.au Tue Aug 24 14:19:31 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 24 14:19:58 2004 Subject: [Python-Dev] SF Priorities, 2.4 release In-Reply-To: <1f7befae040819094825bfacf0@mail.gmail.com> References: <4124CC59.80007@interlink.com.au> <1f7befae040819094825bfacf0@mail.gmail.com> Message-ID: <412B3253.9080106@interlink.com.au> Working originally from Tim's list, I propose to start using the priorities in the SF tracker to try and manage the outstanding work for 2.4. Here's the proposed list: Priority 9: - MUST be done before the next release (whether it's a3, b1, b2, or whatever) Priority 8: - MUST be done before b1. It's a functionality change, and it's needed for 2.4. If it's not done before b1, it will have to wait until 2.5, whenever that might be. Priority 7: - SHOULD be done before 2.4 final. I'm going to go through and start updating various open bugs and patches with priorities. How can you help? I'd _really_ prefer that people not run around setting priorities themselves - instead, if you could let me know of bugs/patches that you think should be changed, let me know, either via email, or via IRC - #python-dev on irc.freenode.net. Once this is done, I plan to start actively nagging relevant people for feedback on patches/bugs that only they can provide. I'm happy to do the work of actually checking the patches in, if you're too busy, but in many cases, I'm not going to be able to provide the level of local knowledge about a particular patch or bug report. I'll also be updating the 2.4 release PEP, and I'll also create a 2.5 release PEP, and move stuff that misses 2.4 to it. Anyway, I'm open to feedback on this. I plan to start frobbing priorities shortly. Anthony From FBatista at uniFON.com.ar Tue Aug 24 15:27:21 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Aug 24 15:31:42 2004 Subject: [Python-Dev] SF Priorities, 2.4 release Message-ID: [Anthony Baxter] #- Priority 9: #- - MUST be done before the next release (whether it's a3, b1, b2, #- or whatever) #- Priority 8: #- - MUST be done before b1. It's a functionality change, and it's #- needed for 2.4. If it's not done before b1, it will have to wait #- until 2.5, whenever that might be. #- Priority 7: #- - SHOULD be done before 2.4 final. +1 . Facundo From Paul.Moore at atosorigin.com Tue Aug 24 15:58:42 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Tue Aug 24 15:58:44 2004 Subject: [Python-Dev] Simple coroutines? Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB3BE@UKDCX001.uk.int.atosorigin.com> From: Greg Ewing > There will be a new statement: > > suspend > [...] > > do another_coop() > > would be equivalent to > > co = another_coop() > while co.run(): > suspend I'm not sure I see the difference between suspend/do and yield True/for _ in co: pass, other than possibly that co-operators and generators are intended to be mixed (which strikes me as implausible). I'm likely to be missing something here, but I don't follow the semantics you are suggesting. If there was a simple, realistic use case for this, it might help clarify the semantics. (For extra credit, mix co-operators and generators in your use case so that the semantics of interaction are shown as well :-)) Paul. __________________________________________________________________________ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. __________________________________________________________________________ From kbk at shore.net Tue Aug 24 16:09:33 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Aug 24 16:09:35 2004 Subject: [Python-Dev] SF Priorities, 2.4 release In-Reply-To: <412B3253.9080106@interlink.com.au> (Anthony Baxter's message of "Tue, 24 Aug 2004 22:19:31 +1000") References: <4124CC59.80007@interlink.com.au> <1f7befae040819094825bfacf0@mail.gmail.com> <412B3253.9080106@interlink.com.au> Message-ID: <87fz6c4ogy.fsf@hydra.localdomain> Anthony Baxter writes: > Working originally from Tim's list, I propose to start using the > priorities in the SF tracker to try and manage the outstanding work > for 2.4. Here's the proposed list: > > Priority 9: > - MUST be done before the next release (whether it's a3, b1, b2, > or whatever) > Priority 8: > - MUST be done before b1. It's a functionality change, and it's > needed for 2.4. If it's not done before b1, it will have to wait > until 2.5, whenever that might be. > Priority 7: > - SHOULD be done before 2.4 final. > > I'm going to go through and start updating various open bugs and > patches with priorities. > > How can you help? I'd _really_ prefer that people not run around > setting priorities themselves - instead, if you could let me know > of bugs/patches that you think should be changed, let me know, > either via email, or via IRC - #python-dev on irc.freenode.net. OK, but how about a further tweak (I use priorities to manage IDLE): Priorities 7 - 9 are reserved and modifiable only by the Release Manager. Priorities 6 and below are not reserved and may be modified as desired within the range 1 - 6. -- KBK From revol at free.fr Tue Aug 24 14:37:47 2004 From: revol at free.fr (=?windows-1252?q?Fran=E7ois?= Revol) Date: Tue Aug 24 16:13:32 2004 Subject: [Python-Dev] problem with pymalloc on the BeOS port. In-Reply-To: <412AD718.7020602@v.loewis.de> Message-ID: <2456435963-BeMail@taz> > Tim Peters wrote: > > >>Any clue ? > > > > > > Try gcc without -O. Nobody has reported anything like this before > > -- > > you're in for a lot of fun . > > In addition, try to find out whether BeOS' malloc is somehow > "strange". > It's plain GNU malloc (with some locking for multithreading I think) libroot is actually a glibc +libm with some goodies. Though the glibc is getting quite old. > Does the system have the notion of a linear address space? Does it do > pages? and so on. If you find a feature that you wouldn't expect in > Windows NT, Linux, or HP-UX (*), that might give a clue. Sure, BeOS is a modern OS, mind you :D And it has a fairly complete (complete enough) POSIX subsystem. The only thing that I miss sometimes is mmap() but most of the time it can be worked around easily with areas. Fran?ois. From revol at free.fr Tue Aug 24 15:25:28 2004 From: revol at free.fr (=?windows-1252?q?Fran=E7ois?= Revol) Date: Tue Aug 24 16:13:38 2004 Subject: [Python-Dev] problem with pymalloc on the BeOS port. Message-ID: <5317443691-BeMail@taz> Tim Peters : > [Fran?ois Revol] > > I'm updating the BeOS port of python to include the latest version > > in > > Zeta, the next incarnation of BeOS. > > > > I'm having some problem when enabling pymalloc: > > [zapped] > > 0 bytes originally requested > > The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected. > > The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE > > (0xfb): > > at tail+0: 0xdb *** OUCH > > at tail+1: 0xdb *** OUCH > > at tail+2: 0xdb *** OUCH > > at tail+3: 0xdb *** OUCH > > The block was made by call #3688627195 to debug malloc/realloc. > > No it wasn't. The four bytes following the 4 trailing 0xfb hold the > call number, and they're apparently corrupt too. Eh... > > > Fatal Python error: bad trailing pad byte > > > > indeed, there seem to be 4 deadblock marks between the forbidden > > ones, > > while the len is supposed to be null: > > That's reliable. If there actually was a request for 0 bytes (that > is, assuming this pointer isn't just pointing at a random memory > address), the debug pymalloc allocates 16 bytes for it, filled with > > 00000000 fbfbfbfb fbfbfbfb serial > > where "serial" is a big-endian 4-byte "serial number" uniquely > identifying which call to malloc (or realloc) this was. The address > of the second block of fb's is returned. Yes that's what I deduced from the code of pymalloc. > > > python:dm 0x80010fb8-8 32 > > 80010fb0 00000000 fbfbfbfb dbdbdbdb fbfbdbdb > > ................. > > > > 80010fc0 0100fbfb 507686ef 04000000 fbfbfbfb > > .......vP........ > > 80010fd0 8013cbc8 fbfbfbfb 44ee0100 ffed0100 > > ............D.... > > > > Any clue ? > > Try gcc without -O. Nobody has reported anything like this before -- > you're in for a lot of fun . > OK, tried -O0 -g but same result. I suspect it might be a bad interaction with fork(), as it crashes in a child, quite badly, as no images are repported as loaded in the team (= no binary are mapped in the process), however the areas are there (= pages). Now, I don't see why malloc itself would give such a result, it's pyMalloc which places those marks, so the thing malloc does wouldn't place them 4 bytes of each other for no reason, or repport 0 bytes where 4 are allocated. Fran?ois. From mcherm at mcherm.com Tue Aug 24 16:57:22 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 24 16:53:54 2004 Subject: [Python-Dev] Python icons Message-ID: <1093359442.412b5752c3558@mcherm.com> Kevin Gadd writes: > I'm not really worried about licensing... basically, I'd probably just want > to keep my copyright on the actual vector art. I'm not terribly concerned > about anything else, since they are based on the original python icons. I > don't know much about how licensing artwork for open source projects works, > so if you or someone else can let me know what steps would be necessary to > make it possible for these icons to be used, I'd be glad to help. While I can't speak authoritatively, I have an idea of what folks would want. My guess is that the PSF (Python Software Foundation) would want the right to use the icons any way they deem necessary for promoting Python. I'm sure they wouldn't mind your keeping the copyright, just so long as the PSF can authorize people to use them (eg: in the installer). A simple note to the PSF stating that you own the copyright and that you're willing to allow the use MIGHT be sufficient, unfortunately it would take a real lawyer to say for sure what's sufficient. In the past other Icons have come with a request by the artist that they be credited... and I'm sure no one would begrudge you a mention in Python's credits (see [1]). And let me add my own thanks! It's nice to have good artwork -- I'm sure there are a few other places you could help also. -- Michael Chermside (speaking only for myself, of course) [1] http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Misc/ACKS?view=markup From mwh at python.net Tue Aug 24 16:57:25 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 24 16:57:27 2004 Subject: [Python-Dev] Python icons In-Reply-To: <1093359442.412b5752c3558@mcherm.com> (Michael Chermside's message of "Tue, 24 Aug 2004 07:57:22 -0700") References: <1093359442.412b5752c3558@mcherm.com> Message-ID: <2macwk7fe2.fsf@starship.python.net> Michael Chermside writes: > Kevin Gadd writes: >> I'm not really worried about licensing... basically, I'd probably just want >> to keep my copyright on the actual vector art. I'm not terribly concerned >> about anything else, since they are based on the original python icons. I >> don't know much about how licensing artwork for open source projects works, >> so if you or someone else can let me know what steps would be necessary to >> make it possible for these icons to be used, I'd be glad to help. > > While I can't speak authoritatively, I have an idea of what folks would > want. My guess is that the PSF (Python Software Foundation) would want > the right to use the icons any way they deem necessary for promoting > Python. I think the Python Powered License is inspirational in this respect (read the bottom of http://starship.python.net/~just/pythonpowered/). Well, maybe not. But it's always good for a chuckle :-) Cheers, mwh -- today's lesson don't strace X in an xterm -- from Twisted.Quotes From theller at python.net Tue Aug 24 17:07:09 2004 From: theller at python.net (Thomas Heller) Date: Tue Aug 24 17:07:13 2004 Subject: [Python-Dev] SF Priorities, 2.4 release In-Reply-To: <412B3253.9080106@interlink.com.au> (Anthony Baxter's message of "Tue, 24 Aug 2004 22:19:31 +1000") References: <4124CC59.80007@interlink.com.au> <1f7befae040819094825bfacf0@mail.gmail.com> <412B3253.9080106@interlink.com.au> Message-ID: Anthony Baxter writes: > Working originally from Tim's list, I propose to start using the > priorities in the SF tracker to try and manage the outstanding work > for 2.4. Here's the proposed list: > > Priority 9: > - MUST be done before the next release (whether it's a3, b1, b2, > or whatever) > Priority 8: > - MUST be done before b1. It's a functionality change, and it's > needed for 2.4. If it's not done before b1, it will have to wait > until 2.5, whenever that might be. > Priority 7: > - SHOULD be done before 2.4 final. What about bugs that MUST be fixed before 2.4 final, when I don't care about the exact release they are fixed? http://python.org/sf/1014215 is such an example, imo. Thomas From vinay_sajip at yahoo.co.uk Tue Aug 24 15:57:18 2004 From: vinay_sajip at yahoo.co.uk (=?iso-8859-1?q?Vinay=20Sajip?=) Date: Tue Aug 24 18:06:40 2004 Subject: [Python-Dev] Proposed change to logging Message-ID: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> An improvement to logging has been proposed by Miki Tebeka, which involves the creation of file-like wrapper objects for loggers. This makes it possible to use the print idiom for logging: import logging logging.basicConfig() logger = logging.getLogger("my.app") err = logger.getFileObject(logging.ERROR) dbg = logger.getFileObject(logging.DEBUG) ... print >> err, "My logging message with %s" % "arguments" print >> dbg, "A message at %s level" % "debug" Miki has helpfully sent in a patch (#1001864), and I propose to check it in by the 27th of August. I just wanted to get comments from python-dev first. There's one wart in the implementation - to avoid printing out blank lines caused by the "\n" appended by print, we ignore messages which are == "\n". This works only because print does a separate write for the "\n". Can anyone see a problem with this approach? class LoggerFileObject: """ Simulate a file object for a given logging level. """ def __init__(self, logger, level): """ Initialize file object with logger and level """ self.logger = logger self.level = level self.closed = 0 def write(self, msg): """ Write message to log. """ if self.closed: raise ValueError("I/O operation on closed file") # Avoid printing blank lines. if msg != "\n": self.logger.log(self.level, msg) def flush(self): """ Flush the file object. This flushes the logger's handlers. """ if self.closed: return for handler in self.logger.handlers: handler.flush() def close(self): """ Close file object. Calling "write" after "close" will raise a ValueError. """ self.closed = 1 Regards, Vinay ___________________________________________________________ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com From python at rcn.com Tue Aug 24 18:25:06 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 24 18:25:35 2004 Subject: [Python-Dev] SF Priorities, 2.4 release In-Reply-To: <412B3253.9080106@interlink.com.au> Message-ID: <003901c489f6$eb012f20$04f9cc97@oemcomputer> > I'd _really_ prefer that people not run around > setting priorities themselves - instead, if you could let me know > of bugs/patches that you think should be changed, let me know, > either via email, or via IRC - #python-dev on irc.freenode.net. Why are other developers suddenly no longer qualified to raise or lower the priority of a bug they've looked at? Raymond From anthony at interlink.com.au Tue Aug 24 18:29:31 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 24 18:29:40 2004 Subject: [Python-Dev] SF Priorities, 2.4 release In-Reply-To: <003901c489f6$eb012f20$04f9cc97@oemcomputer> References: <003901c489f6$eb012f20$04f9cc97@oemcomputer> Message-ID: <412B6CEB.30304@interlink.com.au> Raymond Hettinger wrote: > Why are other developers suddenly no longer qualified to raise or lower > the priority of a bug they've looked at? That's not the issue - the issue is that I want to use the top three priorities for tracking release management critical bugs. Perhaps we should then flag the other priorities for regular use. The SF tracker is too damn limited, but I'm trying to figure out the best approach to this. Anthony From tim.peters at gmail.com Tue Aug 24 18:31:08 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 24 18:31:21 2004 Subject: [Python-Dev] SF Priorities, 2.4 release In-Reply-To: <412B3253.9080106@interlink.com.au> References: <4124CC59.80007@interlink.com.au> <1f7befae040819094825bfacf0@mail.gmail.com> <412B3253.9080106@interlink.com.au> Message-ID: <1f7befae0408240931353fcda2@mail.gmail.com> BTW, priority 6 is a good way to identify issues that ought to be assigned a 7, 8 or 9. For example, I set the "asyncore breaks Zope" bug to 6. 6 is one above the default, so is the least intrusive way to say "I'm pretty sure this needs to be addressed one way or another 'soon'". From tim.peters at gmail.com Tue Aug 24 18:43:11 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Aug 24 18:43:17 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <412AFEC4.6020007@egenix.com> References: <002401c48931$dda69060$e841fea9@oemcomputer> <1093304992.412a82a0a5a88@mail.iinet.net.au> <412AFEC4.6020007@egenix.com> Message-ID: <1f7befae04082409436abe3ded@mail.gmail.com> [M.-A. Lemburg] > ... > Aside: ''.join() currently only works for true sequences - not iterators. >>> def gen(): ... for s in "sure", "it", "does": ... yield s ... >>> ' '.join(gen()) 'sure it does' >>> u' '.join(gen()) u'sure it does' >>> Every function implemented with PySequence_Fast() works with any iterable, although it's fastest if the input argument is a builtin list or tuple. For anything else (including list or tuple subclasses, and other "true sequences"), it materializes a temp tuple, via the iterator protocol. From barry at python.org Tue Aug 24 18:49:54 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 24 18:49:28 2004 Subject: [Python-Dev] Proposed change to logging In-Reply-To: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> Message-ID: <1093366081.18857.42.camel@geddy.wooz.org> On Tue, 2004-08-24 at 09:57, Vinay Sajip wrote: > An improvement to logging has been proposed by Miki > Tebeka, which involves the creation of file-like > wrapper objects for loggers. Yay! +1, especially if I can use them interchangeably. In my code, I usually like the log.debug() calls, but once in a while I have output that comes from something expecting a file-like API, and in that case, it would be nice to be able to pass in log.getFileObject(). Currently, I'm StringIO'ing it. > There's > one wart in the implementation - to avoid printing out > blank lines caused by the "\n" appended by print, we > ignore messages which are == "\n". This works only > because print does a separate write for the "\n". Can > anyone see a problem with this approach? What if the .log() method on loggers took an extra flag to suppress the newline or not, and the wrapper's .write() method passed the appropriate value? The default would be to include the extra newline so the .log() interface would be backward compatible. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/e08fbcaa/attachment-0001.pgp From barry at python.org Tue Aug 24 19:00:37 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 24 19:00:23 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1f7befae040823093264275843@mail.gmail.com> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> Message-ID: <1093366837.18858.51.camel@geddy.wooz.org> On Mon, 2004-08-23 at 12:32, Tim Peters wrote: > Apologies to Barry, but I'm +1 on auto-str() too. It's a string > interpolation -- the user is explicitly asking for a string. If they > made a mistake, it was in asking for a string to begin with, not in > feeding it a non-string. Should it be auto-unicode(), given that Template is derived from unicode? And if so, should we entertain the possibility of insanities like giving the user the ability to pass optional arguments to the unicode() call? If the answers to that are yes and no, that's fine with me. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/e2907160/attachment.pgp From barry at python.org Tue Aug 24 19:05:31 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 24 19:05:18 2004 Subject: [Python-Dev] Update PEP 292 In-Reply-To: <001201c48946$19b49e80$e841fea9@oemcomputer> References: <001201c48946$19b49e80$e841fea9@oemcomputer> Message-ID: <1093367131.18858.54.camel@geddy.wooz.org> On Mon, 2004-08-23 at 15:19, Raymond Hettinger wrote: > Does it? Apply the patch and try "import re". % cd Lib % find -name \*.pyc % cd .. % ./python Python 2.4a2 (#5, Aug 22 2004, 19:41:48) [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import re [25352 refs] >>> import string [25356 refs] >>> string.Template [25356 refs] >>> isn't-anybody-else-going-to-try-the-patch-or-should-i-just-check-it-in-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/3744a324/attachment.pgp From barry at python.org Tue Aug 24 19:14:07 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 24 19:13:41 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <20040823132043.GA4501@titan.progiciels-bpi.ca> References: <1093232832.28573.16.camel@geddy.wooz.org> <20040823132043.GA4501@titan.progiciels-bpi.ca> Message-ID: <1093367647.18858.62.camel@geddy.wooz.org> On Mon, 2004-08-23 at 09:20, Fran?ois Pinard wrote: > These two quotes let me think that people will often forget to stringify > numbers and various other value types, or to escape a few isolated `$'. I'm convinced that we should auto-convert the values. > I have the fuzzy feeling that one unstated, but important goal of the > PEP is to find some way to push `$' forward for stringification in > Python. The whole PEP might even be a stunt or justification for this > unstated goal. However, as this goal is well reached, it might somehow > have contributed to the PEP acceptation and counterweight the deficiency > stated in the preceding paragraph. If my feeling is right, then the PEP > should clearly explicit this goal, it will make the PEP stronger. I will neither confirm nor deny whether the PSU is bankrolling the PEP 292 initiative, nor the actual existence of any 527 organization claiming to be called the "PSU", nor whether if they did exist, they were or weren't acting in coordination with the campaign organizations of any 2004 US presidential nominee. > I witnessed a serious effort in this developers' group to "empty" the > `string' module in favour of string methods. That effort has been > abandoned? I also personally think the word `string' should be left > free, in the long run, for Python programmers to use, on the ground that > modules should not be named after common words programmers would like to > keep to themselves as variable names. So, I think _nothing_ should be > added to the `string' module, with the effect of nailing it deeper in > the library, even if there is a need to rush some solution in. My own personal opinion is that function that were traditionally available in the string module, but which have been available as string methods for a long time now, should be deprecated and eventually removed. But the ground for a string module is staked out, and to the extent that useful things like Template need a home, I'd rather see them end up in the existing string module (which everyone already deals with) than to contrive some name like 'stringtools' or 'stringlib'. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/04170b80/attachment.pgp From bac at OCF.Berkeley.EDU Tue Aug 24 19:39:25 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 24 19:39:41 2004 Subject: [Python-Dev] Re: Update PEP 292 In-Reply-To: <6D9E824FA10BD411BE95000629EE2EC3C6DE1D@FLONIDAN-MAIL> References: <6D9E824FA10BD411BE95000629EE2EC3C6DE1D@FLONIDAN-MAIL> Message-ID: <412B7D4D.6010702@ocf.berkeley.edu> Anders J. Munch wrote: >>From Brett C. [mailto:bac@OCF.Berkeley.EDU] > >>That is coming down to a question of convenience compared to >>explicitness. Do we want "Give me $1,000,000 now!" to raise >>ValueError, >>or do we want to let it pass through? > > > This particular problem has been considered and decided on years ago, > in the context of %-interpolation: > > >>> "Give me %1,000,000 now!" % {} > ValueError: unsupported format character ',' (0x2c) at index 10 > That is not a fair comparison. First off, to make it fair, you would need to make it ``%(1,000,000)s`` to compare to what $-interpolation is doing. Second, the whole point of this discussion is ease of use and thus %-interpolation is not even worth comparing to. If it was we wouldn't be bothering with this. > String interpolation with named parameters given as a dictionary is a > problem already solved in Python. I am puzzled at this urge to > re-solve the same problem, only differently. > It is not solving the idea of using a dictionary to key to values that are to interpolated. It is to use a simpler syntax to represent the text to be interpolated. > If remembering to type the trailing s in "%(varname)s" is so much of a > problem, why not extend the current interpolation syntax with a > variation that is more convenient, yet still backwards-compatible? I > would suggest a formatting-character-less version of "%(varname)s" > that uses brackets instead: "%[varname]". > This was all discussed and rejected. $ is used in so many other languages that there is the perk of familiarity to other languages. And in terms of backwards-compatibility, that is not an issue since this is all being put into a self-contained class and not into the str type. All of this has been discussed long ago and settled on. At this point the only thing left to do is to tell Barry his patch is fine and to just apply the sucker. =) -Brett From aahz at pythoncraft.com Tue Aug 24 20:45:28 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Aug 24 20:45:38 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1093366837.18858.51.camel@geddy.wooz.org> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> <1093366837.18858.51.camel@geddy.wooz.org> Message-ID: <20040824184528.GB2562@panix.com> On Tue, Aug 24, 2004, Barry Warsaw wrote: > On Mon, 2004-08-23 at 12:32, Tim Peters wrote: >> >> Apologies to Barry, but I'm +1 on auto-str() too. It's a string >> interpolation -- the user is explicitly asking for a string. If they >> made a mistake, it was in asking for a string to begin with, not in >> feeding it a non-string. > > Should it be auto-unicode(), given that Template is derived from > unicode? And if so, should we entertain the possibility of insanities > like giving the user the ability to pass optional arguments to the > unicode() call? If the answers to that are yes and no, that's fine with > me. Here you go: yes and no -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From cce at clarkevans.com Tue Aug 24 21:04:22 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Tue Aug 24 21:04:24 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz> References: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com> <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz> Message-ID: <20040824190422.GA48456@prometheusresearch.com> A few definitions: iterator: > An iterator is an object returned by an __iter__ with a next() method which produces values. An iterator's state is kept on the heap (not on the stack), thus one could consider it 'resumable'. An iterator ends when it raises an exception, the very special exception, 'StopIteration' is used to signal that the iterator is done producing values, but is not really an error condition. generator: > A syntax for making an iterator, it appears in python source code as a function or method with a 'yield' statement. An implementation is free to optimize this function, but it acts as if it had created an iterator. iterlink: > An iterlink is an association of one iterator, the consumer, to another iterator, the producer. The iterlink contains information about both iterators, and a marker to indicate the state of the consumer when it asked for the next value of the producer. iterstack: > A stack of iterlinks, where the producer of one iterlink is the consumer of another. The top iterator in the iterstack is an iterator which is behaving like a producer, but not a consumer. The bottom iterator in the iterstack is one which is a producer to a non-iterator. That is, the caller of the bottom iterator is a function or method which is not an iterator. Ok. With this in mind, I like your idea of a new keyword, 'suspend'. This keyword could be used by the top iterator in an iterstack. When 'suspend' happens, the entire iterstack, and its runstate, is setaside. An a signal is sent to the caller of the bottom iterator; the signal could be viewed as a 'resumable exception'. For example, it could be pictured as a 'SuspendIteration' exception. More concretely, using a very simple case. class TopIterator: """ def TopIterator(): yield "one" suspend yield "two" """ def _one(self): self.next = self._suspend return "one" def _suspend(self): self.next = self._two raise SuspendIteration() def _two(self): self.next = self._stop return "two" def _stop(self): raise StopIteration def __iter__(): self.next = self._one return self class BottomIterator(): """ def BottomIterator(): producer = iter(TopIterator()) saved_one = producer.next() saved_two = producer.next() yield (saved_one,saved_two) """ def _one(self): self.saved_one = self.producer.next() self.next = self._two return self.next() def _two(self): self.saved_two = self.producer.next() self.next = self._stop return (self.saved_one, self.saved_two) def _stop(self): raise StopIteration def __iter__(self): self.producer = iter(TopIterator()) self.next = self._one return self def caller(): it = iter(BottomIterator()) while True: try: print it.next() except StopIteration: break except SuspendIteration: sleep(10) continue Anyway, the behavior probably equivalent to 'resumable exceptions', however, I don't think it is _different_ item from an iterator, just a new set of behaviors triggered by the 'suspend' keyword. As long as the 'iterator construction' is automated, the 'caller()' function can be delegated to Twisted's reactor, or asynccore or any other cooperative multitasking base. The superclasses for SuspendIteration could be 'BlockForSocket', for example. On Tue, Aug 24, 2004 at 05:57:18PM +1200, Greg Ewing wrote: | This makes me think that maybe we want another kind of object, similar | to a generator, but designed to be used for side effects rather than | to produce values. (a) the generation of 'top-level' iterators using 'suspend' would be quite easy, see above (b) the generation logic for "pass-through" generators (aka BottomIterator in the example above) would have to happen for the mechanism to be truly useful; SuspendIteration would have to be a pass-through exception (c) updating itertools to allow SuspendIteration to 'passthrough' | For want of a better term, let's call it a "cooperator" (so it ends in | "ator" like "generator" :-). Actually there will be two objects, a | cooperator-function (analogous to a generator-function) and a | cooperator-instance (analogous to a generator-iterator). | | Calling a cooperator-function will return a cooperator-instance, which | will obey the cooperator protocol. This protocol consists of a single | function run(), which causes the cooperator-instance to perform some | amount of work and then stop. If there is more work remaining to be | done, run() returns a true value, otherwise it returns a false value. Well, run() need not be specified. It would be implemented by the lower-level 'reactor' code, aka the "caller" in the example | There will be a new statement: | | suspend | | for use inside a cooperator-function. The presence of this statement | is what distinguishes a cooperator-function from an ordinary function | (just as the presence of 'yield' distinguishes a generator from an | ordinary function). Execution of 'suspend' will cause the run() method | to return with a true value. The next time run() is called, execution | will resume after the 'suspend'. Right. Only that I'd allow both 'yield' and 'suspend' in the same function, or it really isn't that useful. | This is really all that's needed, but one further piece of syntax may | be desirable to make it easier for one cooperator to invoke another: | | do another_coop() | | would be equivalent to | | co = another_coop() | while co.run(): | suspend Exactly; this is where the tough part is, the 'intermediate' cooperators. Ideally, all generators (since they keep their state on the stack) would make good 'intermediate' cooperators. | Something to note is that there's no requirement for a cooperator- | instance to be implemented by a cooperator-function, just as an | iterator can be implemented in ways other than a generator. This | sidesteps some of the difficulties of mixing Python and C calls, since | a Python cooperator-function could invoke a cooperator implemented in | C which in turn invokes another Python cooperator- function, etc. The | only requirement is that all the objects on the way down obey the | cooperator protocol. Yes. This is the key, you _only_ attempt to suspend iterators, that is objects that already keep their state in the heap. | The drawback is that you always have to know when you're calling | another cooperator and use 'do'. But that's no worse than the current | situation with generators, where you always have to know you're using | an iterator and use 'for... yield'. Er, well, if the interface was an 'SuspendIteration' then it could be generalized to all iterators. Afterall, while an exception invalidates the stack-frame for the current call to next(), it doesn't prevent the caller from invoking next() once again. If someone calls next() manually, then SuspendIteration would act as a regular exception... and be uncaught. Clark From cce at clarkevans.com Tue Aug 24 21:12:08 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Tue Aug 24 21:12:11 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <1d6cdae304082404452c15617b@mail.gmail.com> References: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz> <1d6cdae304082404452c15617b@mail.gmail.com> Message-ID: <20040824191208.GB48456@prometheusresearch.com> On Tue, Aug 24, 2004 at 07:45:13AM -0400, Daniel Bickett wrote: | Tell me if I'm missing something here, but with the exception of your | two introductory keywords (a subject I thought was supposed to be | sacred, with the intention of keeping the amount of keywords as small | as possible,) how much difference is there to generators? Are you | saying the run() method, after having been called initially, will | return a value indicative of the cooperator's business when called | again? In that such a 'cooperator' would keep its state on the heap (in an iterator object to be more precice), it is very much like a generator. In fact, one could implement it using a SuspendIteration exception, in addition to a StopIteration. SuspendIteration would signal that the caller of the iterator _may_ call next() again. In other words, it says: "I didn't produce a value this time, but call me again, and I may have a value for you". Specializations of SuspendIteration could signal that the process is blocked on a socket read, or some other low-level process. So that the caller (which would be a generic run-loop in most cases) would only attempt next() once the block is resolved. | It seems like a similar feat could be achieved with custom generators | and boolean yield statements. I currently do this in twisted.flow, however, it requires that intermediate generators "check" for the special value. This intermediate work is slow (since it is not in C) and tedious since it is mostly boilerplate catch-check-yield kinda stuff On Tue, Aug 24, 2004 at 02:58:42PM +0100, Moore, Paul wrote: | From: Greg Ewing | > There will be a new statement: | > | > suspend | > | [...] | > | > do another_coop() | > | > would be equivalent to | > | > co = another_coop() | > while co.run(): | > suspend | | | I'm not sure I see the difference between suspend/do and | yield True/for _ in co: pass, other than possibly that | co-operators and generators are intended to be mixed (which | strikes me as implausible). You'd definately want to mix them. For example, a 'cooperator' would be reading from a socket, it could do one of three things: yield the next line from the sender raise StopIteration if the socket closed raise SuspendIteration if the read() is blocked The 'suspend' statement is analogous to raising SuspendIteration. | I'm likely to be missing something here, but I don't follow | the semantics you are suggesting. | | If there was a simple, realistic use case for this, it might | help clarify the semantics. (For extra credit, mix co-operators | and generators in your use case so that the semantics of | interaction are shown as well :-)) There ya go. ;) Clark From nas at arctrix.com Tue Aug 24 21:33:36 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Tue Aug 24 21:33:44 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: References: Message-ID: <20040824193336.GA8484@mems-exchange.org> On Tue, Aug 24, 2004 at 11:56:13AM -0700, montanaro@users.sourceforge.net wrote: > Modified Files: > pep-0318.txt > Log Message: > List some possible reasons why arriving at consensus about > decorators has been so hard (or impossible) to acheive. There are > certainly more. Perhaps you could add my reservation (objection is too strong a word). I think decorators are not powerful enough given their high syntactic profile. This could be rephrased as "if we are going the use the @ sign then it should be able to do really cool things". One idea is to have the compiler pass the AST for the function body to the decorator function. The decorator could create new nodes in the AST or modify existing ones. That would allow decorators to do things like adding a try/except without introducing another function call. The output of the decorator would be passed to the code generator. Neil From barry at python.org Tue Aug 24 21:42:58 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 24 21:42:33 2004 Subject: [Python-Dev] sre.py backward compatibility and PEP 291 Message-ID: <1093376578.18850.107.camel@geddy.wooz.org> PEP 291 says that sre.py needs to be backward compatible with Python 2.1. sre.py says: # this module works under 1.5.2 and later. don't use string methods import string Which is it? Raymond has seen some circular reference problems between my PEP 292 string.py patch and re.py (really, by way of sre.py and friends). Even though I haven't seen the same problems, I'd like to avoid any direct circular references by changing the few string module function calls in sre*.py to string methods. But the comment above stopped me, until I read PEP 291. The PEP should be the definitive answer, and I'm willing to string-meth-ify sre*.py, if there are no objections. out-posting-phillip-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/b392f5ba/attachment.pgp From bac at OCF.Berkeley.EDU Tue Aug 24 21:57:03 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 24 21:57:12 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: <20040824193336.GA8484@mems-exchange.org> References: <20040824193336.GA8484@mems-exchange.org> Message-ID: <412B9D8F.2060906@ocf.berkeley.edu> Neil Schemenauer wrote: > On Tue, Aug 24, 2004 at 11:56:13AM -0700, montanaro@users.sourceforge.net wrote: > >>Modified Files: >> pep-0318.txt >>Log Message: >>List some possible reasons why arriving at consensus about >>decorators has been so hard (or impossible) to acheive. There are >>certainly more. > > > Perhaps you could add my reservation (objection is too strong a > word). I think decorators are not powerful enough given their high > syntactic profile. This could be rephrased as "if we are going the > use the @ sign then it should be able to do really cool things". > > One idea is to have the compiler pass the AST for the function body > to the decorator function. The decorator could create new nodes in > the AST or modify existing ones. That would allow decorators to do > things like adding a try/except without introducing another function > call. The output of the decorator would be passed to the code > generator. > That kind of stuff is my dream use of the AST; modifying it before final compilation to a .pyc file. Although that could also just be set up in a list that gets called on *all* compilations. We could also just keep the AST around in the code object, although that would be space-consuming. -Brett From skip at pobox.com Tue Aug 24 21:59:33 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 24 21:59:48 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: <20040824193336.GA8484@mems-exchange.org> References: <20040824193336.GA8484@mems-exchange.org> Message-ID: <16683.40485.877453.285028@montanaro.dyndns.org> >> List some possible reasons why arriving at consensus about decorators >> has been so hard (or impossible) to acheive. There are certainly >> more. Neil> Perhaps you could add my reservation (objection is too strong a Neil> word). I think decorators are not powerful enough given their Neil> high syntactic profile. This could be rephrased as "if we are Neil> going the use the @ sign then it should be able to do really cool Neil> things". I was trying to elaborate general barriers to concluding this process, not issues related to selecting one proposal over another. Neil> One idea is to have the compiler pass the AST for the function Neil> body to the decorator function. The decorator could create new Neil> nodes in the AST or modify existing ones. That would allow Neil> decorators to do things like adding a try/except without Neil> introducing another function call. The output of the decorator Neil> would be passed to the code generator. I suppose you could do that, although if you did I suspect metaclasses would seem tame by comparison. Couldn't you do something like this? @ast_to_func @neils_cool_ast_transform @func_to_ast def f(a): pass After all, there's no restriction on what a decorator can return. How about we save true AST decorators for "$"? Skip From walter at livinglogic.de Tue Aug 24 22:04:25 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Aug 24 22:04:29 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412ACE79.4060605@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> Message-ID: <412B9F49.700@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> OK, let's come up with a patch that fixes the incomplete byte >> sequences problem and then discuss non-stream APIs. >> >> So, what should the next step be? > > I think your first patch should be taken as a basis for that. > Add the state-supporting decoding C functions, and change > the stream readers to use them. OK, I've updated the patch. > That still leaves the issue > of the last read operation, which I'm tempted to leave unresolved > for Python 2.4. Agreed! This shouldn't be done for Python 2.4. > No matter what the solution is, it would likely > require changes to all codecs, which is not good. True, but the changes should be rather trivial for most. Bye, Walter D?rwald From fumanchu at amor.org Tue Aug 24 22:08:21 2004 From: fumanchu at amor.org (Robert Brewer) Date: Tue Aug 24 22:13:48 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E54@exchange.hqamor.amorhq.net> Skip Montanaro (I think) wrote: > List some possible reasons why arriving at consensus about decorators > has been so hard (or impossible) to acheive. There are certainly more. First reaction: Each proposal, by its very syntax, suggested a range of non-decorator use cases and semantics. Unfortunately, we have dozens of syntax proposals with "ranges of implication" which do not overlap, and therefore cannot be evaluated fairly (very little "apples to apples" comparison). Most of the remaining discussions addressed Beauty, Fear, Magic, or Intuition, which take more effort and space to analyze properly than your typical Usenet post allows. Robert Brewer MIS Amor Ministries fumanchu@amor.org From walter at livinglogic.de Tue Aug 24 22:15:01 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Aug 24 22:16:04 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412B0C2D.7030507@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> Message-ID: <412BA1C5.5050808@livinglogic.de> M.-A. Lemburg wrote: > Martin v. L?wis wrote: > >> Walter D?rwald wrote: >> >>> OK, let's come up with a patch that fixes the incomplete byte >>> sequences problem and then discuss non-stream APIs. >>> >>> So, what should the next step be? >> >> I think your first patch should be taken as a basis for that. > > We do need a way to communicate state between the codec > and Python. > > However, I don't like the way that the patch > implements this state handling: I think we should use a > generic "state" object here which is passed to the stateful > codec and returned together with the standard return values > on output: > > def decode_stateful(data, state=None): > ... decode and modify state ... > return (decoded_data, length_consumed, state) Another option might be that the decode function changes the state object in place. > where the object type and contents of the state variable > is defined per codec (e.g. could be a tuple, just a single > integer or some other special object). If a tuple is passed and returned this makes it possible from Python code to mangle the state. IMHO this should be avoided if possible. > Otherwise we'll end up having different interface > signatures for all codecs and extending them to accomodate > for future enhancement will become unfeasable without > introducing yet another set of APIs. We already have slightly different decoding functions: utf_16_ex_decode() takes additional arguments. > Let's discuss this some more and implement it for Python 2.5. > For Python 2.4, I think we can get away with what we already > have: > [...] OK, I've updated the patch. > [...] > The buffer logic should only be used for streams > that do not support the interface to push back already > read bytes (e.g. .unread()). > > From a design perspective, keeping read data inside the > codec is the wrong thing to do, simply because it leaves > the input stream in an undefined state in case of an error > and there's no way to correlate the stream's read position > to the location of the error. > > With a pushback method on the stream, all the stream > data will be stored on the stream, not the codec, so > the above would no longer be a problem. On the other hand this requires special stream. Data already read is part of the codec state, so why not put it into the codec? > However, we can always add the .unread() support to the > stream codecs at a later stage, so it's probably ok > to default to the buffer logic for Python 2.4. OK. >> That still leaves the issue >> of the last read operation, which I'm tempted to leave unresolved >> for Python 2.4. No matter what the solution is, it would likely >> require changes to all codecs, which is not good. > > We could have a method on the codec which checks whether > the codec buffer or the stream still has pending data > left. Using this method is an application scope consideration, > not a codec issue. But this mean that the normal error handling can't be used for those trailing bytes. Bye, Walter D?rwald From martin at v.loewis.de Tue Aug 24 22:38:39 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 24 22:38:59 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412B0C2D.7030507@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> Message-ID: <412BA74F.6030701@v.loewis.de> M.-A. Lemburg wrote: > However, I don't like the way that the patch > implements this state handling: I think we should use a > generic "state" object here which is passed to the stateful > codec and returned together with the standard return values > on output: Why is that better? Practicality beats purity. This is useless over-generalization. > Otherwise we'll end up having different interface > signatures for all codecs and extending them to accomodate > for future enhancement will become unfeasable without > introducing yet another set of APIs. What is "a codec" here? A class implementing the StreamReader and/or Codec interface? Walter's patch does not change the API of any of these. It just adds a few functions to some module, which are not meant to be called directly. > If we leave out the UTF-7 codec changes in the > patch, the only state that the UTF-8 and UTF-16 > codecs create is the number of bytes consumed. We already > have the required state parameter for this in the > standard decode API, so no extra APIs are needed for > these two codecs. Where precisely is the number of decoded bytes in the API? Regards, Martin From nas at arctrix.com Tue Aug 24 22:58:58 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Tue Aug 24 22:59:03 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: <16683.40485.877453.285028@montanaro.dyndns.org> References: <20040824193336.GA8484@mems-exchange.org> <16683.40485.877453.285028@montanaro.dyndns.org> Message-ID: <20040824205858.GA8823@mems-exchange.org> On Tue, Aug 24, 2004 at 02:59:33PM -0500, Skip Montanaro wrote: > Couldn't you do something like this? > > @ast_to_func > @neils_cool_ast_transform > @func_to_ast > def f(a): > pass That doesn't work as there is, AFAIK, no way to write func_to_ast(). The necessary information has already been thrown away when it gets called. Neil From fumanchu at amor.org Tue Aug 24 22:56:04 2004 From: fumanchu at amor.org (Robert Brewer) Date: Tue Aug 24 23:01:31 2004 Subject: [Python-Dev] Important decorator proposal on c.l.p. Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E58@exchange.hqamor.amorhq.net> In case it wasn't made clear earlier, there's an effort being made to propose an alternative to the current @decorator syntax before 2.4 hits beta. For everyone's sanity, that discussion is going on wholly on comp.lang.python. I figured I'd mention it to those of you who don't read clp on a regular basis--I should have done so earlier. You can find the initial announcement at http://groups.google.com/groups?th=c62eba24231bc2fb if you are interested in contributing/commenting. But hurry! Deadlines loom. Robert Brewer MIS Amor Ministries fumanchu@amor.org From pje at telecommunity.com Tue Aug 24 23:02:43 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Aug 24 23:02:29 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: <20040824205858.GA8823@mems-exchange.org> References: <16683.40485.877453.285028@montanaro.dyndns.org> <20040824193336.GA8484@mems-exchange.org> <16683.40485.877453.285028@montanaro.dyndns.org> Message-ID: <5.1.1.6.0.20040824170201.023a33c0@mail.telecommunity.com> At 04:58 PM 8/24/04 -0400, Neil Schemenauer wrote: >On Tue, Aug 24, 2004 at 02:59:33PM -0500, Skip Montanaro wrote: > > Couldn't you do something like this? > > > > @ast_to_func > > @neils_cool_ast_transform > > @func_to_ast > > def f(a): > > pass > >That doesn't work as there is, AFAIK, no way to write func_to_ast(). There is one, but it's ugly and involves inspect.get_source. :) From fredrik at pythonware.com Tue Aug 24 23:05:24 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Aug 24 23:03:42 2004 Subject: [Python-Dev] Re: sre.py backward compatibility and PEP 291 References: <1093376578.18850.107.camel@geddy.wooz.org> Message-ID: Barry wrote: > PEP 291 says that sre.py needs to be backward compatible with Python > 2.1. sre.py says: > > # this module works under 1.5.2 and later. don't use string methods > import string > > Which is it? Martin changed the PEP last week (or so), but forgot to remove the comment. Note that you can remove some code from _sre.c as well (see join_list, and perhaps other places that checks for >= 1.6). cheers /F From barry at python.org Tue Aug 24 23:37:36 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 24 23:37:10 2004 Subject: [Python-Dev] Re: sre.py backward compatibility and PEP 291 In-Reply-To: References: <1093376578.18850.107.camel@geddy.wooz.org> Message-ID: <1093383456.18852.125.camel@geddy.wooz.org> On Tue, 2004-08-24 at 17:05, Fredrik Lundh wrote: > Martin changed the PEP last week (or so), but forgot to remove the > comment. > > Note that you can remove some code from _sre.c as well (see join_list, > and perhaps other places that checks for >= 1.6). Awesome, thanks. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/81c5ca69/attachment.pgp From martin at v.loewis.de Tue Aug 24 23:42:01 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 24 23:42:00 2004 Subject: [Python-Dev] Re: sre.py backward compatibility and PEP 291 In-Reply-To: References: <1093376578.18850.107.camel@geddy.wooz.org> Message-ID: <412BB629.2090104@v.loewis.de> Fredrik Lundh wrote: > Barry wrote: >>PEP 291 says that sre.py needs to be backward compatible with Python >>2.1. sre.py says: >> >># this module works under 1.5.2 and later. don't use string methods >>import string >> >>Which is it? > > > Martin changed the PEP last week (or so), but forgot to remove the > comment. I only changed it for xmlrpclib, and changed the comment in that module. Just made the change for sre in pep-0291.txt 1.5, on 02-Jul-03. The commit message there says Bumped sre's "baseline" to 2.1, per /F's recommendation in bug #764548 Regards, Martin From bac at OCF.Berkeley.EDU Tue Aug 24 23:53:26 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 24 23:53:34 2004 Subject: [Python-Dev] Need double-check on patch for bug #991962 (--disable-toolbox-glue compile on OS X) Message-ID: <412BB8D6.5020008@ocf.berkeley.edu> Finally got around to writing up a quick patch for setup.py to not compile OS X-specific modules if the build is configured for --disable-toolbox-glue (would always have the Mac modules fail to build thanks to link errors if the toolbox glue was not compiled). But since I hardly ever play with the build process and the last thing I want to do is break it I would appreciate someone taking a quick look at this (changed two 'if' statements so it's small). And this is an indirect request to Anthony to bump the priority. =) -Brett From ncoghlan at iinet.net.au Wed Aug 25 00:20:39 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Aug 25 00:22:29 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <412AFEC4.6020007@egenix.com> References: <002401c48931$dda69060$e841fea9@oemcomputer> <1093304992.412a82a0a5a88@mail.iinet.net.au> <412AFEC4.6020007@egenix.com> Message-ID: <412BBF37.4010402@iinet.net.au> M.-A. Lemburg wrote: > ncoghlan@iinet.net.au wrote: > >> pass (essentially, all of the work done so far is thrown away, and the >> Unicode >> join starts from scratch. If you know you have Unicode, you're better >> off using >> a Unicode separator to avoid this unnecessary work ). > > > It's just a simple length querying loop; there's no storage allocation > or anything expensive happening there, so the "throw-away" operation > is not expensive. Yes, my mistake. The tuple that string join creates when the argument is an iterable rather than a list or tuple isn't thrown away - it is given to PyUnicode_Join to work with (since the original iterator may not be able to be iterated a second time). The only work that is lost is that all of the type checks that have been done on prior elements will get repeated in the Unicode join code. So it's not as bad as I first thought - but it will still cost a few cycles. Regards, Nick. From python at rcn.com Wed Aug 25 00:24:49 2004 From: python at rcn.com (Raymond Hettinger) Date: Wed Aug 25 00:25:19 2004 Subject: [Python-Dev] FW: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 Message-ID: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer> > List some possible reasons why arriving at consensus about > decorators has been so hard (or impossible) to achieve Thanks. I think that was an important contribution to the discussion. At this point, looking at the meta-discussion is likely to best way to help cut through the current morass. +There is no one clear reason why this should be so, but a +few problems seem to be most problematic. A possible social issue is that decorators can be used in a tremendous variety of ways, each of which is only needed or appreciated by small, disjoint groups of users. For instance, applications to ctypes have unique needs that not understood or shared by others. Some users want to use decorators for metaclass magic that scares the socks off of the rest. > + Almost everyone agrees that decorating/transforming a function at > + the end of its definition is suboptimal. One other point of agreement is that no one like having to write the function name three times: def f(): ... f = deco(f) > +* Syntactic constraints. There is an existing (though clumsy) syntax. Most of the proposals are less flexible but more elegant. This trade-off has created much more consternation than if there were no existing ways to apply decorations. +* Overall unfamiliarity with the concept. For people who have a + passing acquaintance with algebra (or even basic arithmetic) or have + used at least one other programming language, much of Python is + intuitive. Very few people will have had any experience with the + decorator concept before encountering it in Python. There's just no + strong preexisting meme that captures the concept. My take on this one is that there are some existing memes from C# and Java and that in the future they will be more widely known. However, there are not good existing, thought-out concepts of what exactly decorators should do in terms of preserving docstrings, attributes, using parameters, etc. Most have only a single application in mind and the full implications (and possible applications) of a given syntax are shrouded in the mists of the future. Like metaclasses, everyone knew they were powerful when they were put in, but no one knew how they would be used or whether they were necessary. In fact, two versions later, we still don't know those answers. Raymond From pje at telecommunity.com Wed Aug 25 00:54:49 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 25 00:54:41 2004 Subject: [Python-Dev] FW: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer> Message-ID: <5.1.1.6.0.20040824184328.0297bd40@mail.telecommunity.com> At 06:24 PM 8/24/04 -0400, Raymond Hettinger wrote: >Like metaclasses, everyone knew they were powerful when they were put >in, but no one knew how they would be used or whether they were >necessary. In fact, two versions later, we still don't know those >answers. The people who've been using metaclasses since Python 1.5 (including e.g., the Zope developers) would probably disagree. (Metaclasses got an upgrade in capabilities due to type/class unification, plus a more convenient syntax in 2.2, but they were around long before that.) Similarly, there are plenty of "pyoneers" with use cases today for decorators, and they know how they're being used today, and that an improved syntax is necessary. I'd point to the fact that there has been *very* little argument among decorator proponents regarding the desired semantics, as an indication that we *do* know what capabilities are necessary and/or desirable. (By decorator proponents, I mean people who have developed and use their own decorators, who have spoken in favor of having PEP 318 implemented for 2.4. That is, people interested enough in the functionality to have used the current syntax to get them.) From trentm at ActiveState.com Wed Aug 25 02:16:34 2004 From: trentm at ActiveState.com (Trent Mick) Date: Wed Aug 25 02:17:15 2004 Subject: [Python-Dev] socketmodule build failures on Un*xs that don't HAVE_GETADDRINFO Message-ID: <20040824171634.A8031@ActiveState.com> Building the socket module with the current Python CVS (or at least with the Python 2.4a2 source) on Solaris 2.6, HP-UX 11, and AIX 5.1 fails in Modules/getaddrinfo.c with complaints about cc: ".../python/Modules/getaddrinfo.c", line 204: error 1588: "EAI_MAX" undefined. ".../python/Modules/getaddrinfo.c", line 282.25: 1506-045 (S) Undeclared identifier EAI_BADHINTS. ".../python/Modules/getaddrinfo.c", line 283.40: 1506-045 (S) Undeclared identifier AI_MASK. ".../python/Modules/getaddrinfo.c", line 373.41: 1506-045 (S) Undeclared identifier EAI_PROTOCOL. and the like. The problem seems to stem from this check-in which resulted in Modules/addrinfo.h *not* getting included for Unix flavours that do not HAVE_GETADDRINFO. http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.288&r2=1.289 Wed May 26 17:06:31 2004 UTC (2 months, 4 weeks ago) by mwh > Band-aid type fix for > > [ 728330 ] Don't define _SGAPI on IRIX > > The Right Thing would be nice, for now this'll do. At least it > isn't going to break anything *other* than IRIX... Relevant bug: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=728330&group_id=5470 Subseqent checkins have gone in to fix the lack of addrinfo.h import on some platforms: http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.292&r2=1.293 http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.297&r2=1.298 So the question: was that original change removing the #include "addrinfo.h" an accident? I.e. should it have been something like this instead? #if defined(__sgi) # if _COMPILER_VERSION>700 \ && !defined(_SS_ALIGNSIZE) /* defined in sys/socket.h */ /* by some newer versions of IRIX */ /* (e.g. not by 6.5.10 but by 6.5.21) */ # include "addrinfo.h" # endif #else # include "addrinfo.h" #endif Cheers, Trent -- Trent Mick TrentM@ActiveState.com From barry at python.org Wed Aug 25 03:54:22 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 25 03:53:58 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <20040824184528.GB2562@panix.com> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> <1093366837.18858.51.camel@geddy.wooz.org> <20040824184528.GB2562@panix.com> Message-ID: <1093398861.4452.19.camel@geddy.wooz.org> On Tue, 2004-08-24 at 14:45, Aahz wrote: > Here you go: yes and no Cool. Well Aahz, I'm taking that as the green light to check this stuff in. It doesn't seem like anybody else wants to test the patch before hand, so in it goes. I'll also be checking in a de-string-ified sre module, docs, test cases, and an updated PEP 292. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/4261adfe/attachment.pgp From greg at cosc.canterbury.ac.nz Wed Aug 25 04:51:22 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 25 04:51:27 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com> Message-ID: <200408250251.i7P2pMGI004207@cosc353.cosc.canterbury.ac.nz> "Phillip J. Eby" : > If you could throw a special kind of exception (or even a regular > exception), and call traceback.resume() to pick up execution at the > point where it was thrown, whether thrown by a generator or a > regular function. Actually, it probably wouldn't be too hard to make exceptions resumable -- provided all the frames in between are Python. If the exception gets thrown through any C calls, though, resumption becomes impossible. Some other structure is needed to hold the state of a resumable C call. I agree that maintaining a stack of *ators automatically somehow would be desirable, but I haven't figured out yet exactly where and how that stack would be maintained. > What's really needed (IMO) is to add a way to communicate *into* a > co-operator, passing it a value to "accept" or a traceback to raise. > E.g.: > > input = suspend output There have been many suggestions in the past for 'yield' to be extended to allow values to be passed in as well as out. They all suffer from a problem of asymmetry. However you arrange it, you always end up having to discard the first value passed out or pass a dummy value in the first time, or something like that. I deliberately left communication out of the suspend to avoid those problems. If you want communication, you have to arrange it some other way. > Anyway, as I said, what would be *most* useful for async programming is a > way to resume a traceback, because then you wouldn't need for every > intervening frame to have special suspension capability. Certainly, but that way, ultimately, lies Stackless... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Wed Aug 25 04:59:43 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 25 04:59:55 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <1d6cdae304082404452c15617b@mail.gmail.com> Message-ID: <200408250259.i7P2xhX6004230@cosc353.cosc.canterbury.ac.nz> > how much difference is there to generators? Very little! Probably too little, in fact, to justify having a different kind of entity at all. I only suggested it because there seems to be a reluctance to extend generators in any way that might suggest using them for purposes other than generation. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Wed Aug 25 05:00:50 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 25 05:00:52 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1093366837.18858.51.camel@geddy.wooz.org> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> <1093366837.18858.51.camel@geddy.wooz.org> Message-ID: <1f7befae04082420007ec812b@mail.gmail.com> [Barry Warsaw, on auto-str()] > Should it be auto-unicode(), given that Template is derived from > unicode? And if so, should we entertain the possibility of insanities > like giving the user the ability to pass optional arguments to the > unicode() call? If the answers to that are yes and no, that's fine with > me. unicode ... heh, I heard something about that in Java once. Did Python grow one of those too? I sure hope it doesn't get in the way of using God-given American strings! no-no-no-no-no-but-i'll-settle-for-yes-no-ly y'rs - tim From tim.peters at gmail.com Wed Aug 25 05:10:15 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 25 05:10:17 2004 Subject: [Python-Dev] problem with pymalloc on the BeOS port. In-Reply-To: <5317443691-BeMail@taz> References: <5317443691-BeMail@taz> Message-ID: <1f7befae04082420104f9158df@mail.gmail.com> [Fran?ois Revol] > Now, I don't see why malloc itself would give such a result, it's > pyMalloc which places those marks, so the thing malloc does wouldn't > place them 4 bytes of each other for no reason, or repport 0 bytes > where 4 are allocated. I think you're fooling yourself if you believe 4 *were* allocated. The memory dump shows nothing but gibberish, with 4 blocks of fbfbfbfb not a one of which makes sense in context (the numbers before and after them make no sense as "# of bytes allocated" or as "serial number" values, so these forbidden-byte blocks don't make sense as either end of an active pymalloc block). You should at least try to get a C traceback at this point, on the chance that the routine passing the pointer is a clue. We don't even know here yet whether the complaint came from a free() or realloc() call. This isn't going to be easy. From kbk at shore.net Wed Aug 25 05:23:15 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Aug 25 05:23:19 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200408250323.i7P3NFF7006771@h006008a7bda6.ne.client2.attbi.com> Patch / Bug Summary ___________________ Patches : 259 open ( -5) / 2573 closed (+17) / 2832 total (+12) Bugs : 745 open ( +0) / 4405 closed (+21) / 5150 total (+21) RFE : 150 open ( +1) / 130 closed ( +0) / 280 total ( +1) New / Reopened Patches ______________________ Patch for bug 933795. term.h and curses on Solaris (2004-08-19) http://python.org/sf/1012280 opened by Ariel Arjona PEP 292 implementation (2004-08-22) http://python.org/sf/1014055 opened by Barry A. Warsaw Rational.py various bugfixes (2004-08-19) http://python.org/sf/1012468 opened by Russell Blau cStringIO's truncate doesn't (2004-08-20) CLOSED http://python.org/sf/1012740 opened by Dima Dorfman Fix for duplicate attributes in generated HTML (2004-08-20) http://python.org/sf/1013055 opened by Richard Brodie Cleanup Peepholer Output (2004-08-22) CLOSED http://python.org/sf/1013667 opened by Raymond Hettinger Implementation for PEP 318 using syntax J2 (2004-08-22) http://python.org/sf/1013835 opened by Michael RobotFileParser.can_fetch return value (2004-08-23) CLOSED http://python.org/sf/1014237 opened by George Yoshida Expose current parse location to XMLParser (2004-08-24) http://python.org/sf/1014930 opened by Dave Cole bug in tarfile.ExFileObject.readline (2004-08-24) http://python.org/sf/1014992 opened by GaryD Improve markup and punctuation in libsocket.tex (2004-08-24) http://python.org/sf/1015012 opened by Dima Dorfman Docs claim that coerce can return None (2004-08-24) http://python.org/sf/1015021 opened by Dima Dorfman Patches Closed ______________ baseinteger: abstract superclass for int and long (2004-08-11) http://python.org/sf/1007068 closed by nnorwitz Improve error reporting when Python opens source code (2004-08-19) http://python.org/sf/1011822 closed by loewis Add Py_InitializeEx to allow embedding without signals. (2004-02-19) http://python.org/sf/900727 closed by loewis implementation of Text.dump method (2002-04-19) http://python.org/sf/546244 closed by loewis Return new string for single subclass joins (Bug #1001011) (2004-08-11) http://python.org/sf/1007087 closed by rhettinger cStringIO's truncate doesn't (2004-08-20) http://python.org/sf/1012740 closed by tim_one Bad URL encoding in SimpleHTTPServer directory listing (2004-08-18) http://python.org/sf/1011123 closed by jlgijsbers doctest: add a special (dedented) marker for blank lines (2004-04-11) http://python.org/sf/933238 closed by jlgijsbers incorrect end-of-message handling in mailbox.BabylMailbox (2004-01-20) http://python.org/sf/880621 closed by jlgijsbers (bug 1005936) textunderscore python.sty fix (2004-08-14) http://python.org/sf/1009373 closed by nnorwitz Cleanup Peepholer Output (2004-08-22) http://python.org/sf/1013667 closed by rhettinger Non-ascii in non-unicode __credits__ in Lib/pydoc.py (2004-08-15) http://python.org/sf/1009389 closed by loewis Allow pydoc to work with XP Themes (.manifest file) (2004-07-25) http://python.org/sf/997284 closed by loewis LC_CTYPE locale and strings (2004-07-26) http://python.org/sf/997768 closed by loewis Installation database patch (2002-05-29) http://python.org/sf/562100 closed by akuchling RobotFileParser.can_fetch return value (2004-08-23) http://python.org/sf/1014237 closed by loewis thread Module Breaks PyGILState_Ensure() (2004-08-17) http://python.org/sf/1010677 closed by mhammond New / Reopened Bugs ___________________ dictionary referencing error (2004-08-19) CLOSED http://python.org/sf/1012249 opened by AMIT Consulting LLC weakref.WeakValueDictionary should override .has_key() (2004-08-19) CLOSED http://python.org/sf/1012315 opened by Myers Carpenter ctrl-left/-right works incorectly with diacritics (2004-08-19) http://python.org/sf/1012435 opened by Krzysztof Wilkosz Can't pipe input to a python program (2004-08-20) CLOSED http://python.org/sf/1012692 opened by Ronald L. Rivest xmlrpclib, PEP291, "yield" (2004-08-16) CLOSED http://python.org/sf/1009803 reopened by anthonybaxter Python Tutorial 3.1.4 List example (2004-08-20) CLOSED http://python.org/sf/1013239 opened by Elizibeth Thompson Win XP DEP prevents Python call to 'C' DLL (2004-08-21) CLOSED http://python.org/sf/1013418 opened by ajhewitt xml.dom documentation omits createDocument, ...DocumentType (2004-08-21) http://python.org/sf/1013525 opened by Mike Brown Error in Chapter 4 of Tutorial (2004-08-22) CLOSED http://python.org/sf/1013760 opened by Alexey Peshehonov No documentation for PyFunction_* (C-Api) (2004-08-22) http://python.org/sf/1013800 opened by Henning G?nther tarfile.open with mode w|bz2 (2004-08-22) CLOSED http://python.org/sf/1013882 opened by Felix Wiemann Error in representation of complex numbers (2004-08-22) CLOSED http://python.org/sf/1013908 opened by Raymond Hettinger mimetypes add_type has bogus self parameter (2004-08-22) http://python.org/sf/1014022 opened by Cliff Wells Unspecific errors with metaclass (2004-08-23) http://python.org/sf/1014215 opened by Thomas Heller optparse: parser.remove_option("-h") inconsistency (2004-08-23) http://python.org/sf/1014230 opened by strop Missing urllib.urlretrieve docs (2004-08-23) http://python.org/sf/1014761 opened by David Abrahams Misc/NEWS no valid reStructuredText (2004-08-24) http://python.org/sf/1014770 opened by Felix Wiemann Misc/NEWS.help (2004-08-24) http://python.org/sf/1014775 opened by Felix Wiemann "article id" in description of NNTP objects (2004-08-24) http://python.org/sf/1015140 opened by Felix Wiemann cgi.FieldStorage.__len__ eventually throws TypeError (2004-08-24) http://python.org/sf/1015249 opened by Andreas Ames Create cgi.FieldStorage._get_new_instance method as factory (2004-08-24) http://python.org/sf/1015254 opened by Andreas Ames readline broken (2004-08-24) CLOSED http://python.org/sf/1015517 opened by Walter D?rwald Bugs Closed ___________ dictionary referencing error (2004-08-19) http://python.org/sf/1012249 closed by rhettinger weakref.WeakValueDictionary should override .has_key() (2004-08-19) http://python.org/sf/1012315 closed by rhettinger str.join([ str-subtype-instance ]) misbehaves (2004-07-30) http://python.org/sf/1001011 closed by rhettinger compile errors on _codecs_iso2022.c (2004-08-09) http://python.org/sf/1005737 closed by perky SGI (IRIX 6.5.24) Error building _codecs_iso2022.c (2004-08-11) http://python.org/sf/1007249 closed by perky Warning in cjkcodecs/iso2022common.h (2004-07-16) http://python.org/sf/991834 closed by perky test_queue fails occasionally (2004-08-17) http://python.org/sf/1010777 closed by mwh Can't pipe input to a python program (2004-08-20) http://python.org/sf/1012692 closed by tim_one xmlrpclib, PEP291, "yield" (2004-08-16) http://python.org/sf/1009803 closed by loewis xmlrpclib, PEP291, "yield" (2004-08-16) http://python.org/sf/1009803 closed by loewis logging.getLevelName returns a number (2004-08-12) http://python.org/sf/1008295 closed by vsajip Python Tutorial 3.1.4 List example (2004-08-21) http://python.org/sf/1013239 closed by perky optparse indents without respect to $COLUMNS (2004-01-16) http://python.org/sf/878453 closed by jlgijsbers Win XP DEP prevents Python call to 'C' DLL (2004-08-21) http://python.org/sf/1013418 closed by loewis Index entries with "_" are wrong in pdf file (2004-08-09) http://python.org/sf/1005936 closed by nnorwitz xmlrpclib and backward compatibility (2003-11-15) http://python.org/sf/842600 closed by loewis Error in Chapter 4 of Tutorial (2004-08-22) http://python.org/sf/1013760 closed by rhettinger tarfile.open with mode w|bz2 (2004-08-22) http://python.org/sf/1013882 closed by loewis Error in representation of complex numbers (2004-08-22) http://python.org/sf/1013908 closed by loewis Cmd in thread segfaults after Ctrl-C (2004-06-06) http://python.org/sf/967334 closed by mwh 2.3.4 fails build on solaris 10 - complexobject.c (2004-06-10) http://python.org/sf/970334 closed by mwh readline broken (2004-08-24) http://python.org/sf/1015517 closed by nnorwitz New / Reopened RFE __________________ Standard exceptions are hostile to Unicode (2004-08-20) http://python.org/sf/1012952 opened by Marius Gedminas From barry at python.org Wed Aug 25 05:30:57 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 25 05:30:31 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1f7befae04082420007ec812b@mail.gmail.com> References: <1093276486.412a1346e30e9@mcherm.com> <20040823161233.GB71531@prometheusresearch.com> <1f7befae040823093264275843@mail.gmail.com> <1093366837.18858.51.camel@geddy.wooz.org> <1f7befae04082420007ec812b@mail.gmail.com> Message-ID: <1093404657.4453.41.camel@geddy.wooz.org> On Tue, 2004-08-24 at 23:00, Tim Peters wrote: > unicode ... heh, I heard something about that in Java once. Did > Python grow one of those too? I sure hope it doesn't get in the way > of using God-given American strings! That's "Freedom Strings" to you, buddy. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/664082b5/attachment.pgp From greg at cosc.canterbury.ac.nz Wed Aug 25 05:32:28 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 25 05:32:34 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <20040824191208.GB48456@prometheusresearch.com> Message-ID: <200408250332.i7P3WSkG004275@cosc353.cosc.canterbury.ac.nz> "Clark C. Evans" : > In fact, one could implement it using a SuspendIteration exception, > in addition to a StopIteration. I don't think this can be done quite right by raising an exception in the normal way, because you don't want 'finally' clauses to be triggered. > You'd definately want to mix them. For example, a 'cooperator' > would be reading from a socket, it could do one of three things: > > yield the next line from the sender > raise StopIteration if the socket closed > raise SuspendIteration if the read() is blocked Hmmm. It seems that generation and cooperation are really orthogonal concepts -- you may want something that is both a generator *and* a cooperator. My proposal doesn't allow for this. I will have to do some more thinking... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Wed Aug 25 05:36:48 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Aug 25 05:36:58 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: <16683.40485.877453.285028@montanaro.dyndns.org> Message-ID: <200408250336.i7P3amJI004285@cosc353.cosc.canterbury.ac.nz> > How about we save true AST decorators for "$"? Nah, AST decorators should be spelled "@$~" (that's "AST" written in funny symbols). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido at python.org Wed Aug 25 05:47:04 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 25 05:47:16 2004 Subject: [Python-Dev] Proposed change to logging In-Reply-To: Your message of "Tue, 24 Aug 2004 14:57:18 BST." <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> Message-ID: <200408250347.i7P3l4O21288@guido.python.org> > import logging > > logging.basicConfig() > logger = logging.getLogger("my.app") > err = logger.getFileObject(logging.ERROR) > dbg = logger.getFileObject(logging.DEBUG) > ... > > print >> err, "My logging message with %s" % > "arguments" > print >> dbg, "A message at %s level" % "debug" Is this really a useful improvement? It seems to save a few keystrokes at most. TOOWTDI etc. --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond.hettinger at verizon.net Wed Aug 25 06:11:04 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed Aug 25 06:11:40 2004 Subject: [Python-Dev] Is something wrong with SF patch notifications? Message-ID: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> I don't think Barry got notes showing that I reviewed his patch. Also, I didn't get a note when Walter assigned a unittest patch to me. That may also explain why I had a hard time soliciting a reviewer for the peepholer patch. IIRC, we always used to get notes whenever we were assigned a patch or had participated in the patch discussions. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040825/692353c8/attachment.html From tim.peters at gmail.com Wed Aug 25 06:24:06 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Aug 25 06:24:09 2004 Subject: [Python-Dev] Is something wrong with SF patch notifications? In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> Message-ID: <1f7befae04082421246fa2a2ce@mail.gmail.com> [Raymond Hettinger] > I don't think Barry got notes showing that I reviewed his patch. > > so, I didn't get a note when Walter assigned a unittest patch to me. > > That may also explain why I had a hard time soliciting a reviewer for > the peepholer patch. I got SF email about all these things -- but I'm subscribed to the patches list, and it doesn't look like you are. > IIRC, we always used to get notes whenever we were assigned a > patch or had participated in the patch discussions. Never the latter. It should generate email to the submitter, and to the person currently assigned (if any). But since I'm subscribed to the patches list, I see all changes to all items regardless, and wouldn't know whether the to-submitter and to-assignee gimmicks stopped working. I'm pretty sure that at least the to-submitter part is working, since I've gotten quick replies from infrequent submitters after adding a comment to their report. From fumanchu at amor.org Wed Aug 25 07:30:44 2004 From: fumanchu at amor.org (Robert Brewer) Date: Wed Aug 25 07:36:12 2004 Subject: [Python-Dev] possible CO_FUTURE_DECORATORS Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E65@exchange.hqamor.amorhq.net> [Working on the J2 patch]... If we use a keyword, we need a __future__ and AFAICT a new CO_FUTURE_DECORATORS flag. Should it just be 0x4000? #define CO_NOFREE 0x0040 /* XXX Temporary hack. Until generators are a permanent part of the language, we need a way for a code object to record that generators were *possible* when it was compiled. This is so code dynamically compiled *by* a code object knows whether to allow yield stmts. In effect, this passes on the "from __future__ import generators" state in effect when the code block was compiled. */ #define CO_GENERATOR_ALLOWED 0x1000 /* no longer used in an essential way */ #define CO_FUTURE_DIVISION 0x2000 #define CO_FUTURE_DECORATORS 0x4000 I'm mostly wondering why there's a big gap between CO_NOFREE and CO_GENERATOR_ALLOWED. Or is it a non-issue, pick-what-you-want? Robert Brewer MIS Amor Ministries fumanchu@amor.org From martin at v.loewis.de Wed Aug 25 07:54:03 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 25 07:54:05 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412BA1C5.5050808@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> Message-ID: <412C297B.2040208@v.loewis.de> Walter D?rwald wrote: > If a tuple is passed and returned this makes it possible > from Python code to mangle the state. IMHO this should be > avoided if possible. Not necessarily. We are all consenting adults. So if the code checks whether the state is sane, from a typing point of view, i.e. if you can't crash the interpreter, then there is no need to hide the state from the user. Regards, Martin From martin at v.loewis.de Wed Aug 25 08:29:34 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 25 08:29:34 2004 Subject: [Python-Dev] socketmodule build failures on Un*xs that don't HAVE_GETADDRINFO In-Reply-To: <20040824171634.A8031@ActiveState.com> References: <20040824171634.A8031@ActiveState.com> Message-ID: <412C31CE.5060306@v.loewis.de> Trent Mick wrote: > The problem seems to stem from this check-in which resulted in > Modules/addrinfo.h *not* getting included for Unix flavours that do not > HAVE_GETADDRINFO. Thanks for discovering that. I have backed-out this change, and all subsequent changes that have tried to work around, and re-opened the patch. Regards, Martin From martin at v.loewis.de Wed Aug 25 08:33:06 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 25 08:33:04 2004 Subject: [Python-Dev] Is something wrong with SF patch notifications? In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> Message-ID: <412C32A2.6090602@v.loewis.de> Raymond Hettinger wrote: > IIRC, we always used to get notes whenever we were assigned a patch or > had participated in the patch discussions. I just got mail when I reopened a patch, and added a comment. Regards, Martin From theller at python.net Wed Aug 25 09:12:04 2004 From: theller at python.net (Thomas Heller) Date: Wed Aug 25 09:18:45 2004 Subject: [Python-Dev] FW: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26 In-Reply-To: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer> (Raymond Hettinger's message of "Tue, 24 Aug 2004 18:24:49 -0400") References: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer> Message-ID: "Raymond Hettinger" writes: [decorators...] > Like metaclasses, everyone knew they were powerful when they were put > in, but no one knew how they would be used or whether they were > necessary. In fact, two versions later, we still don't know those > answers. Sorry I have to say this, but I don't think you know what you're talking about in this paragraph. Thomas From anthony at interlink.com.au Wed Aug 25 09:48:20 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Aug 25 09:48:31 2004 Subject: [Python-Dev] Is something wrong with SF patch notifications? In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> Message-ID: <412C4444.7080001@interlink.com.au> Raymond Hettinger wrote: > IIRC, we always used to get notes whenever we were assigned a patch or > had participated in the patch discussions. I've found SF's behaviour to be inconsistent at best in this regard - the only way to get the messages reliably is to subscribe to both the python-bugs and python-patches lists. (As an aside: Why do we keep both these lists going? Why not simply point all of the trackers at python-bugs. I can see no reason why you'd want to know about only one of the trackers...) From mal at egenix.com Wed Aug 25 10:32:41 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 25 10:32:44 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412C297B.2040208@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de> Message-ID: <412C4EA9.8010003@egenix.com> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> If a tuple is passed and returned this makes it possible >> from Python code to mangle the state. IMHO this should be >> avoided if possible. > > Not necessarily. We are all consenting adults. So if the > code checks whether the state is sane, from a typing point > of view, i.e. if you can't crash the interpreter, then there > is no need to hide the state from the user. Martin, there are two reasons for hiding away these details: 1. we need to be able to change the codec state without breaking the APIs 2. we don't want the state to be altered by the user A single object serves this best and does not create a whole plethora of new APIs in the _codecs module. This is not over-design, but serves a reason. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 25 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 25 10:41:58 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 25 10:42:01 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412BA1C5.5050808@livinglogic.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> Message-ID: <412C50D6.6010801@egenix.com> Walter D?rwald wrote: > M.-A. Lemburg wrote: > >> Martin v. L?wis wrote: >> >>> Walter D?rwald wrote: >>> >>>> OK, let's come up with a patch that fixes the incomplete byte >>>> sequences problem and then discuss non-stream APIs. >>>> >>>> So, what should the next step be? >>> >>> >>> I think your first patch should be taken as a basis for that. >> >> >> We do need a way to communicate state between the codec >> and Python. >> >> However, I don't like the way that the patch >> implements this state handling: I think we should use a >> generic "state" object here which is passed to the stateful >> codec and returned together with the standard return values >> on output: >> >> def decode_stateful(data, state=None): >> ... decode and modify state ... >> return (decoded_data, length_consumed, state) > > Another option might be that the decode function changes > the state object in place. Good idea. >> where the object type and contents of the state variable >> is defined per codec (e.g. could be a tuple, just a single >> integer or some other special object). > > If a tuple is passed and returned this makes it possible > from Python code to mangle the state. IMHO this should be > avoided if possible. > >> Otherwise we'll end up having different interface >> signatures for all codecs and extending them to accomodate >> for future enhancement will become unfeasable without >> introducing yet another set of APIs. > > We already have slightly different decoding functions: > utf_16_ex_decode() takes additional arguments. Right - it was a step in the wrong direction. Let's not use a different path for the future. >> Let's discuss this some more and implement it for Python 2.5. >> For Python 2.4, I think we can get away with what we already >> have: > > > [...] > > OK, I've updated the patch. > >> [...] >> The buffer logic should only be used for streams >> that do not support the interface to push back already >> read bytes (e.g. .unread()). >> >> From a design perspective, keeping read data inside the >> codec is the wrong thing to do, simply because it leaves >> the input stream in an undefined state in case of an error >> and there's no way to correlate the stream's read position >> to the location of the error. >> >> With a pushback method on the stream, all the stream >> data will be stored on the stream, not the codec, so >> the above would no longer be a problem. > > On the other hand this requires special stream. Data > already read is part of the codec state, so why not > put it into the codec? Ideally, the codec should not store data, but only reference it. It's better to keep things well separated which is why I think we need the .unread() interface and eventually a queue interface to support the feeding operation. >> However, we can always add the .unread() support to the >> stream codecs at a later stage, so it's probably ok >> to default to the buffer logic for Python 2.4. > > OK. > >>> That still leaves the issue >>> of the last read operation, which I'm tempted to leave unresolved >>> for Python 2.4. No matter what the solution is, it would likely >>> require changes to all codecs, which is not good. >> >> >> We could have a method on the codec which checks whether >> the codec buffer or the stream still has pending data >> left. Using this method is an application scope consideration, >> not a codec issue. > > But this mean that the normal error handling can't be used > for those trailing bytes. Right, but then: missing data (which usually causes the trailing bytes) is really something for the application to deal with, e.g. by requesting more data from the user, another application or trying to work around the problem in some way. I don't think that the codec error handler can practically cover these possibilities. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 25 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From vinay_sajip at yahoo.co.uk Wed Aug 25 11:43:16 2004 From: vinay_sajip at yahoo.co.uk (=?iso-8859-1?q?Vinay=20Sajip?=) Date: Wed Aug 25 11:43:19 2004 Subject: [Python-Dev] Proposed change to logging In-Reply-To: <200408250347.i7P3l4O21288@guido.python.org> Message-ID: <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com> --- Guido van Rossum wrote: > > print >> err, "My logging message with %s" % > > "arguments" > > print >> dbg, "A message at %s level" % "debug" > > Is this really a useful improvement? It seems to > save a few > keystrokes at most. TOOWTDI etc. My personal vote would be +0, I posted the suggestion only because some people have raised concerns that the logging module is hard to use, and using "print" might seem easier for newbies or casual users. Before checking in, I want to see if I can remove that implementation wart I mentioned. I feel the solution Barry suggested is not as elegant as I'd like to have. Vinay Sajip ___________________________________________________________ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com From mwh at python.net Wed Aug 25 12:02:01 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 25 12:02:03 2004 Subject: [Python-Dev] Is something wrong with SF patch notifications? In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> (Raymond Hettinger's message of "Wed, 25 Aug 2004 00:11:04 -0400") References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> Message-ID: <2m65777cyu.fsf@starship.python.net> "Raymond Hettinger" writes: > Link: File-List > > I don't think Barry got notes showing that I reviewed his patch. > > Also, I didn't get a note when Walter assigned a unittest patch to me. I've failed to get the "personal" notifications (e.g. I don't think I got a notification when you unassigned the peepholer patch from me). OTOH, I'm not aware of the python.org lists missing anything. Filing a SF support request may or may not help (it did last time something went a bit wrong in this respect). Cheers, mwh -- All programs evolve until they can send email. -- Richard Letts Except Microsoft Exchange. -- Art -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From mwh at python.net Wed Aug 25 12:44:48 2004 From: mwh at python.net (Michael Hudson) Date: Wed Aug 25 12:44:50 2004 Subject: [Python-Dev] socketmodule build failures on Un*xs that don't HAVE_GETADDRINFO In-Reply-To: <20040824171634.A8031@ActiveState.com> (Trent Mick's message of "Tue, 24 Aug 2004 17:16:34 -0700") References: <20040824171634.A8031@ActiveState.com> Message-ID: <2m1xhv7azj.fsf@starship.python.net> Trent Mick writes: > http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.288&r2=1.289 > Wed May 26 17:06:31 2004 UTC (2 months, 4 weeks ago) by mwh > > Band-aid type fix for > > > > [ 728330 ] Don't define _SGAPI on IRIX > > > > The Right Thing would be nice, for now this'll do. At least it > > isn't going to break anything *other* than IRIX... > > Relevant bug: > https://sourceforge.net/tracker/?func=detail&atid=105470&aid=728330&group_id=5470 > > Subseqent checkins have gone in to fix the lack of addrinfo.h import on > some platforms: > > http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.292&r2=1.293 > http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.297&r2=1.298 > > So the question: was that original change removing the > #include "addrinfo.h" > an accident? I.e. should it have been something like this instead? > > #if defined(__sgi) > # if _COMPILER_VERSION>700 \ > && !defined(_SS_ALIGNSIZE) /* defined in sys/socket.h */ > /* by some newer versions of IRIX */ > /* (e.g. not by 6.5.10 but by 6.5.21) */ > # include "addrinfo.h" > # endif > #else > # include "addrinfo.h" > #endif Well, it certainly seems I goofed with that checkin: as the message implies, I thought the code I was touching only affected Irix. I seem to have mentally entirely paged out the details, though. Sorry for all the pain. This hilights what I said in the #ifdeffery thread, though: it's difficult to check in code that affects systems you have no access to. Thanks to Trent for taking the time to actually try to work out what's going on! Cheers, mwh -- About the use of language: it is impossible to sharpen a pencil with a blunt axe. It is equally vain to try to do it with ten blunt axes instead. -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time. From skip at pobox.com Wed Aug 25 13:38:48 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Aug 25 13:38:58 2004 Subject: [Python-Dev] Proposed change to logging In-Reply-To: <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com> References: <200408250347.i7P3l4O21288@guido.python.org> <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com> Message-ID: <16684.31304.985278.482389@montanaro.dyndns.org> Vinay> I posted the suggestion only because some people have raised Vinay> concerns that the logging module is hard to use, and using Vinay> "print" might seem easier for newbies or casual users. My personal head scratching about the logging package has more to do with initialization of the system than with emission of log info. I wouldn't worry about the function call vs. print thing, though if it was there I'd probably use it. Skip From barry at python.org Wed Aug 25 14:17:43 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 25 14:17:17 2004 Subject: [Python-Dev] Proposed change to logging In-Reply-To: <200408250347.i7P3l4O21288@guido.python.org> References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> <200408250347.i7P3l4O21288@guido.python.org> Message-ID: <1093436263.4453.55.camel@geddy.wooz.org> On Tue, 2004-08-24 at 23:47, Guido van Rossum wrote: > Is this really a useful improvement? It seems to save a few > keystrokes at most. TOOWTDI etc. I'm really more interested in providing a compatible file-like (i.e. write()) interface to logging objects. There are a lot of APIs in the world that are written to accept file-like objects and it would be nice to be able to pass logging objects directly to them, without having to first make a detour with StringIO. I don't really care too much about the actual implementation of it all. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040825/5a23affe/attachment.pgp From barry at python.org Wed Aug 25 14:33:06 2004 From: barry at python.org (Barry Warsaw) Date: Wed Aug 25 14:32:40 2004 Subject: [Python-Dev] Is something wrong with SF patch notifications? In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> Message-ID: <1093437186.4453.63.camel@geddy.wooz.org> On Wed, 2004-08-25 at 00:11, Raymond Hettinger wrote: > I don?t think Barry got notes showing that I reviewed his patch. I didn't, but I'm not sure it's SF's fault. It's possible that such a message got caught by my Spambayes filters, however I recently built out a new mail server and my filters were pointing at imap folders on the old server. So any Unsures or Spam I got in the last week or so were bit bucketed. I definitely didn't see them in my regular inbox. I also get patches in a digest and tend to ignore them when swamped. But I will review your comments on the PEP 292 patch later today. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040825/ed2b5a96/attachment.pgp From guido at python.org Wed Aug 25 17:32:42 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 25 17:32:53 2004 Subject: [Python-Dev] Proposed change to logging In-Reply-To: Your message of "Wed, 25 Aug 2004 10:43:16 BST." <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com> References: <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com> Message-ID: <200408251532.i7PFWgX22320@guido.python.org> > > > print >> err, "My logging message with %s" % "arguments" > > > print >> dbg, "A message at %s level" % "debug" > > > > Is this really a useful improvement? It seems to > > save a few keystrokes at most. TOOWTDI etc. > > My personal vote would be +0, I posted the suggestion > only because some people have raised concerns that the > logging module is hard to use, and using "print" might > seem easier for newbies or casual users. Before > checking in, I want to see if I can remove that > implementation wart I mentioned. I feel the solution > Barry suggested is not as elegant as I'd like to have. But the complexity of the logging module doesn't stem from the message logging interface, which is actually very simple. It comes from the configuration interface, and from the complexity of having separate concepts of loggers, handlers, and formatters, and of the unintuitive filter rules (often setting the logger's level doesn't work because the handler's level is still wrong, or the other way around). All of that complexity was inherited from Java, BTW (I've run in exactly those same problems at work with the Java logging package). I'm -0.5 on the file interface -- passing loggers around actually becomes more complicated because you have to pass around separate info and debug streams (etc.). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Aug 25 17:34:40 2004 From: guido at python.org (Guido van Rossum) Date: Wed Aug 25 17:34:44 2004 Subject: [Python-Dev] Proposed change to logging In-Reply-To: Your message of "Wed, 25 Aug 2004 08:17:43 EDT." <1093436263.4453.55.camel@geddy.wooz.org> References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> <200408250347.i7P3l4O21288@guido.python.org> <1093436263.4453.55.camel@geddy.wooz.org> Message-ID: <200408251534.i7PFYeI22352@guido.python.org> > I'm really more interested in providing a compatible file-like (i.e. > write()) interface to logging objects. There are a lot of APIs in the > world that are written to accept file-like objects and it would be nice > to be able to pass logging objects directly to them, without having to > first make a detour with StringIO. I don't really care too much about > the actual implementation of it all. Hm. So would you turn each separate line into a log message? When you write a partial line, it needs to be buffered somewhere until you see the newline. --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Aug 25 18:13:46 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 25 18:13:46 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <200408250251.i7P2pMGI004207@cosc353.cosc.canterbury.ac.nz> References: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com> At 02:51 PM 8/25/04 +1200, Greg Ewing wrote: >"Phillip J. Eby" : > > > If you could throw a special kind of exception (or even a regular > > exception), and call traceback.resume() to pick up execution at the > > point where it was thrown, whether thrown by a generator or a > > regular function. > >Actually, it probably wouldn't be too hard to make exceptions >resumable -- provided all the frames in between are Python. If the >exception gets thrown through any C calls, though, resumption becomes >impossible. Some other structure is needed to hold the state of a >resumable C call. Unfortunately, as was discussed on the previous Stackless thread, nearly *all* Python bytecodes pass through C on their way to other Python code. The trick is to be able to figure out whether the return value of a given frame is being put onto the value stack of its parent frame. In principle, you could tell this from the operation being performed at the current opcode pointer in the parent frame, and the prior contents of the value stack. In practice, the eval loop sometimes messes with the value stack prior to the call (e.g. to optimize method calls), and in the case of e.g try/finally and try/except, it's going to execute other stuff in the frame before the exception is passed up to the next frame. So, to make it work,the interpreter loop would need to be particular about putting the stack back to the way it was before the current instruction began, when an exception occurs. Also, it would need to *not* invoke except/finally clauses for one of these special exceptions, or else have a way of restoring the frame to its pre-exception state, which seems arbitrarily complex. I guess it would have to treat the resumable exception as if a YIELD operation took place. I'm going to have to think about this one some more, but it seems to me that it would be much easier to add bidirectional communication to generators to create a coroutine type, than to try to implement resumable exceptions. From cce at clarkevans.com Wed Aug 25 19:16:54 2004 From: cce at clarkevans.com (Clark C. Evans) Date: Wed Aug 25 19:16:59 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com> References: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com> <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com> Message-ID: <20040825171653.GA90924@prometheusresearch.com> I'm not tied to my proposal, I'm suggesting that if the scope of continuations is limited to only work with iterators, that a solution may emerge. This limitation, in pratice with generator syntax, is not all that bothersome to an application developer. On Wed, Aug 25, 2004 at 12:13:46PM -0400, Phillip J. Eby wrote: | At 02:51 PM 8/25/04 +1200, Greg Ewing wrote: | >> If you could throw a special kind of exception (or even a regular | >> exception), and call traceback.resume() to pick up execution at the | >> point where it was thrown, whether thrown by a generator or a | >> regular function. | > | >Actually, it probably wouldn't be too hard to make exceptions | >resumable -- provided all the frames in between are Python. If the | >exception gets thrown through any C calls, though, resumption becomes | >impossible. Some other structure is needed to hold the state of a | >resumable C call. | | Unfortunately, as was discussed on the previous Stackless thread, nearly | *all* Python bytecodes pass through C on their way to other Python code. However, the C code in the distribution could be scrubbed to make sure that it works with the mechanism. And the C API could be detailed so that this case is managable; else, it is an "uncaught" exception. | I'm going to have to think about this one some more, but it seems to me | that it would be much easier to add bidirectional communication to | generators to create a coroutine type, than to try to implement resumable | exceptions. Sure; but resumable exceptions would only need to operate with generators, and there are some simple rewrite tricks one could do. If an iterator is calling a generator it could be re-written in a form that maintains state and _doesn't_ catch the exception. Already for generators 'finally' blocks are troublesome, so this doesn't change things for the worse. StopIteration has many of these same issues. On Wed, Aug 25, 2004 at 03:32:28PM +1200, Greg Ewing wrote: | > You'd definately want to mix them. For example, a 'cooperator' | > would be reading from a socket, it could do one of three things: | > | > yield the next line from the sender | > raise StopIteration if the socket closed | > raise SuspendIteration if the read() is blocked | | Hmmm. It seems that generation and cooperation are really orthogonal | concepts -- you may want something that is both a generator *and* a | cooperator. My proposal doesn't allow for this. I will have to do some | more thinking... Not only othogonal, but quite complementary. Kind Regards, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/ - Research Exchange Database /\ - Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * From martin at v.loewis.de Wed Aug 25 21:56:25 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Aug 25 21:56:26 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412C4EA9.8010003@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de> <412C4EA9.8010003@egenix.com> Message-ID: <412CEEE9.60207@v.loewis.de> M.-A. Lemburg wrote: > Martin, there are two reasons for hiding away these details: > > 1. we need to be able to change the codec state without > breaking the APIs That will be possible with the currently-proposed patch. The _codecs methods are not public API, so changing them would not be an API change. > 2. we don't want the state to be altered by the user We are all consenting adults, and we can't *really* prevent it, anyway. For example, the user may pass an old state, or a state originating from a different codec (instance). We need to support this gracefully (i.e. with a proper Python exception). > A single object serves this best and does not create > a whole plethora of new APIs in the _codecs module. > This is not over-design, but serves a reason. It does put a burden on codec developers, which need to match the "official" state representation policy. Of course, if they are allowed to return a tuple representing their state, that would be fine with me. Regards, Martin From martin at v.loewis.de Wed Aug 25 22:02:56 2004 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed Aug 25 22:02:53 2004 Subject: [Python-Dev] socketmodule build failures on Un*xs that don't HAVE_GETADDRINFO In-Reply-To: <2m1xhv7azj.fsf@starship.python.net> References: <20040824171634.A8031@ActiveState.com> <2m1xhv7azj.fsf@starship.python.net> Message-ID: <412CF070.6040502@v.loewis.de> Michael Hudson wrote: > Well, it certainly seems I goofed with that checkin: as the message > implies, I thought the code I was touching only affected Irix. I seem > to have mentally entirely paged out the details, though. > > Sorry for all the pain. No need to worry. This happens, and CVS is there for tracing things back, and undoing/redoing them. Of course, anybody involved could have done the analysis that Trent did (and yes, that includes yours truly). Regards, Martin From ncoghlan at iinet.net.au Wed Aug 25 08:58:42 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Aug 26 00:20:24 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <412B0C84.30204@livinglogic.de> References: <412B0C84.30204@livinglogic.de> Message-ID: <412C38A2.9000909@iinet.net.au> Walter D?rwald wrote: > test_string.py is supposed to test the string module. This new test > should be moved to test_str.py (or even better to string_test.py, > so that it can be reused for doing the same test with unicode objects). Ah, I think I see where the tests should have gone (MixinStrUnicodeUserStringTest.test_join in string_test.py). I'll work up a patch to bring those tests in sync with the rest of the string tests (the structure of which I find harder to follow than the string code itself. . .). However, I probably won't get to it for a week or so. Cheers, Nick. From djc at object-craft.com.au Thu Aug 26 02:44:07 2004 From: djc at object-craft.com.au (Dave Cole) Date: Thu Aug 26 02:44:13 2004 Subject: [Python-Dev] Submitted patch #1014930 to exposes the current parse location in pyexpat In-Reply-To: <200408240307.38486.fdrake@acm.org> References: <412AAD92.9010009@object-craft.com.au> <200408240307.38486.fdrake@acm.org> Message-ID: <412D3257.4060707@object-craft.com.au> Fred L. Drake, Jr. wrote: > On Monday 23 August 2004 10:53 pm, Dave Cole wrote: > > Expat supplies the XML_GetCurrentLineNumber, XML_GetCurrentColumnNumber, > > and XML_GetCurrentByteIndex functions to obtain the current parse > > location. The patch exposes these as the following respective members > > of the xml.parsers.expat.XMLParser object; CurrentLineNumber, > > CurrentColumnNumber, CurrentByteIndex. > > This sounds like a good idea to me. Please post the patch on SF and assign to > me. I have committed the patch after adding the suggestions from the reviewer. - Dave -- http://www.object-craft.com.au From pje at telecommunity.com Thu Aug 26 03:01:47 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Aug 26 03:01:53 2004 Subject: [Python-Dev] Simple coroutines? In-Reply-To: <20040825171653.GA90924@prometheusresearch.com> References: <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com> <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com> <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20040825205040.02a6cb10@mail.telecommunity.com> At 01:16 PM 8/25/04 -0400, Clark C. Evans wrote: >On Wed, Aug 25, 2004 at 12:13:46PM -0400, Phillip J. Eby wrote: >| At 02:51 PM 8/25/04 +1200, Greg Ewing wrote: >| >> If you could throw a special kind of exception (or even a regular >| >> exception), and call traceback.resume() to pick up execution at the >| >> point where it was thrown, whether thrown by a generator or a >| >> regular function. >| > >| >Actually, it probably wouldn't be too hard to make exceptions >| >resumable -- provided all the frames in between are Python. If the >| >exception gets thrown through any C calls, though, resumption becomes >| >impossible. Some other structure is needed to hold the state of a >| >resumable C call. >| >| Unfortunately, as was discussed on the previous Stackless thread, nearly >| *all* Python bytecodes pass through C on their way to other Python code. > >However, the C code in the distribution could be scrubbed to make sure >that it works with the mechanism. And the C API could be detailed so >that this case is managable; else, it is an "uncaught" exception. Quick question: does Stackless' "greenlet" extension module do what you need? I just built it and played around with it a bit. One of the demos is a pseudo-generator implementation using a 'Yield' function that uses greenlet task switching to simulate the behavior of a normal generator function. It looks to me that your "yield Cooperate" scenarios could perhaps be met by simply switching to a reactor/mainloop greenlet, when it's necessary to wait for an event, and by having your event callback switch back to the task's greenlet, returning a result if needed. Interestingly, this greenlet thing looks like it would be perfect for peak.events, allowing me to get rid of all the generators and yield/resume stuff. (In fairness, Bob Ippolito told me that before I ever implemented peak.events, but at the time greenlets didn't exist as an extension module that could be built outside of Stackless.) From tim.peters at gmail.com Thu Aug 26 04:07:27 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 26 04:07:29 2004 Subject: [Python-Dev] test_distutils failing in 3 ways today Message-ID: <1f7befae0408251907b6d1a2f@mail.gmail.com> This is on Windows. I've heard that it fails on Linux too. Who isn't running the test suite? A reminder: with lots of people trying to make changes at once, it's essential to run all tests before checking in. C:\Code\python\PCbuild>python ../lib/test/test_distutils.py test_package_data (distutils.tests.test_build_py.BuildPyTestCase) ... ok test_build (distutils.tests.test_build_scripts.BuildScriptsTestCase) ... ERROR test_default_settings (distutils.tests.test_build_scripts.BuildScriptsTestCase) ... ERROR test_command_packages_cmdline (distutils.tests.test_dist.DistributionTestCase) ... ok test_command_packages_configfile (distutils.tests.test_dist.DistributionTestCase) ... ok test_command_packages_unspecified (distutils.tests.test_dist.DistributionTestCase) ... ok test_home_installation_scheme (distutils.tests.test_install.InstallTestCase) ... ok test_default_settings (distutils.tests.test_install_scripts.InstallScriptsTestCase) ... ok test_installation (distutils.tests.test_install_scripts.InstallScriptsTestCase) ... ok ====================================================================== ERROR: test_build (distutils.tests.test_build_scripts.BuildScriptsTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\distutils\tests\test_build_scripts.py", line 34, in test_build cmd.finalize_options() File "C:\Code\python\lib\distutils\command\build_scripts.py", line 44, in finalize_options ('executable', 'executable')) File "C:\Code\python\lib\distutils\cmd.py", line 309, in set_undefined_options getattr(src_cmd_obj, src_option)) AttributeError: DummyCommand instance has no attribute 'executable' ====================================================================== ERROR: test_default_settings (distutils.tests.test_build_scripts.BuildScriptsTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\distutils\tests\test_build_scripts.py", line 21, in test_default_settings cmd.finalize_options() File "C:\Code\python\lib\distutils\command\build_scripts.py", line 44, in finalize_options ('executable', 'executable')) File "C:\Code\python\lib\distutils\cmd.py", line 309, in set_undefined_options getattr(src_cmd_obj, src_option)) AttributeError: DummyCommand instance has no attribute 'executable' ---------------------------------------------------------------------- Ran 9 tests in 0.156s FAILED (errors=2) Traceback (most recent call last): File "../lib/test/test_distutils.py", line 17, in ? test_main() File "../lib/test/test_distutils.py", line 13, in test_main test.test_support.run_unittest(distutils.tests.test_suite()) File "C:\Code\python\lib\test\test_support.py", line 290, in run_unittest run_suite(suite, testclass) File "C:\Code\python\lib\test\test_support.py", line 272, in run_suite msg = "errors occurred in %s.%s" \ AttributeError: 'TestSuite' object has no attribute '__name__' C:\Code\python\PCbuild> From alex_nanou at cs.msu.su Thu Aug 26 12:58:30 2004 From: alex_nanou at cs.msu.su (alex_nanou@cs.msu.su) Date: Thu Aug 26 12:58:33 2004 Subject: [Python-Dev] decorators: If you go for it, go all the way!!! :) Message-ID: <3405.212.192.249.122.1093517910.squirrel@212.192.249.122> the inability to decorate anything but a function, is quite a limitation, though I must agree that decorating classes is not such a common task, but using the syntax to *decorate* attributes would indeed prove useful (in both the readability department and the uniformity of code).. here is an example we use quite often: ---cut--- import time import sys def intonly(obj=0, doc=None): ''' intonly(int) -> descriptor the resulting descriptor will prevent both the deletion of the attribute and assignments of any value other than ints. ''' if type(obj) is not int: raise TypeError, 'value must be int (got: %s)' % obj f_locals = sys._getframe(1).f_locals # name the attr (I know this is ugly ^_^ ) while True: name = '_priv_' + str(int(time.time()*10**8)) if name not in f_locals: f_locals[name] = obj break # the descriptor reader function... def read(self): return getattr(self, name) # the descriptor writer... def write(self, val): if type(val) is not int: raise TypeError, 'value must be int (got: %s)' % val setattr(self, name, val) # now create the descriptor and return it... return property(read, write, doc=doc) --uncut-- and here are a couple of usage examples: the desired decorator syntax: ---cut--- class A(object): @intonly i = 123 --uncut-- the current usage syntax: ---cut--- class A(object): i = intonly(123) --uncut-- // sorry if I bring this up agen! :) thanks! --- Best Regards... Alex. From s.percivall at chello.se Thu Aug 26 13:50:22 2004 From: s.percivall at chello.se (Simon Percivall) Date: Thu Aug 26 13:50:27 2004 Subject: [Python-Dev] decorators: If you go for it, go all the way!!! :) In-Reply-To: <3405.212.192.249.122.1093517910.squirrel@212.192.249.122> References: <3405.212.192.249.122.1093517910.squirrel@212.192.249.122> Message-ID: <1CBF291C-F756-11D8-A1B4-0003934AD54A@chello.se> On 2004-08-26, at 12.58, alex_nanou@cs.msu.su wrote: > the inability to decorate anything but a function, is quite a > limitation, > though I must agree that decorating classes is not such a common task, > but > using the syntax to *decorate* attributes would indeed prove useful (in > both the readability department and the uniformity of code).. > [...] > > and here are a couple of usage examples: > the desired decorator syntax: > ---cut--- > class A(object): > @intonly > i = 123 > > --uncut-- > > the current usage syntax: > ---cut--- > class A(object): > i = intonly(123) > > --uncut-- IMHO, this would be _bad_. As I see it, the need for a special syntax for decoration, in this case a prefix syntax, comes mainly from the fact that method/function and class declaration is just that, declarations. You can only pass the entity to a decorating function after it has been bound. Hence the irritating need to type the entity name three times, once declaring, once re-binding and once passing to the decorating function. This just isn't an issue here. If you create an instance of some type, such as the integer literal in your example, you can, just as you show, pass it to a function directly. This is truly the best way, and the decoration is unnecessary. Also, I don't agree it increases readability. The @ character marks something special, something you might really need to see to understand what is defined with a function/method or class. That's why re-binding that entity afterwards isn't good enough: you might miss it. Again, this isn't an issue here. //Simon From oliver.walczak at momatec.de Thu Aug 26 14:00:40 2004 From: oliver.walczak at momatec.de (Oliver Walczak) Date: Thu Aug 26 14:00:43 2004 Subject: [Python-Dev] exit in __builtins__ Message-ID: Dear list, as I am already quite experienced in python programming I recently started to dive into the C-depths of the language. So it was a logical implication to register for this newsgroup as there is no one else I can talk about this. So here's one thing I tried to evolve out of the sources but couldn't find out how it works internally: If you call dir(__builtins__) after starting a python (2.3.4) session you get a list all well known builtin functions. I noticed, that there is already an exit function available as builtin (NOT sys.exit!!). But when you call exit(0) it says TypeError: 'str' object is not callable So I called print __builtins__.exit.__doc__ And surprisingly I got the doc string from str. So I started browsing the sources, and in ./Python/bltinmodule.c there get most of the builtin functions attached to __builtin__, but not the "exit" call. Is it a bug or a feature? exit as builtin is not documented anywhere and I cannot find out where it gets included or attached to the str object in the sources. Any idea? Best regards Oliver From rodrigobamboo at gmail.com Thu Aug 26 14:07:02 2004 From: rodrigobamboo at gmail.com (Rodrigo B. de Oliveira) Date: Thu Aug 26 14:07:16 2004 Subject: [Python-Dev] exit in __builtins__ In-Reply-To: References: Message-ID: <5917478b0408260507707bfc56@mail.gmail.com> Hi! Try to type 'exit' or 'quit' in the interactive shell and you see its purpose. Pretty clever. Cheers, Rodrigo On Thu, 26 Aug 2004 14:00:40 +0200, Oliver Walczak wrote: > Dear list, > ... > > > Is it a bug or a feature? exit as builtin is not documented anywhere and I > cannot find out where it gets included or attached to the str object in the > sources. Any idea? > > Best regards > Oliver From adurdin at gmail.com Thu Aug 26 14:23:09 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Thu Aug 26 14:23:17 2004 Subject: [Python-Dev] exit in __builtins__ In-Reply-To: <5917478b0408260507707bfc56@mail.gmail.com> References: <5917478b0408260507707bfc56@mail.gmail.com> Message-ID: <59e9fd3a04082605237676146b@mail.gmail.com> On Thu, 26 Aug 2004 09:07:02 -0300, Rodrigo B. de Oliveira wrote: > Hi! > > Try to type 'exit' or 'quit' in the interactive shell and you see its > purpose. Pretty clever. Yes, it's the string that tells you the proper way to exit the interpreter. But does this really belong in __builtins__? Wouldn't it be better for the interactive interpreter to register it as a global when it sets up, as it only makes sense in the context of the interactive interpreter? Not that it's a big deal :) From oliver.walczak at momatec.de Thu Aug 26 14:30:50 2004 From: oliver.walczak at momatec.de (Oliver Walczak) Date: Thu Aug 26 14:30:52 2004 Subject: AW: [Python-Dev] exit in __builtins__ In-Reply-To: Message-ID: Yes, it's "only" in site.py: __builtin__.quit = __builtin__.exit = exit Allright, I think it makes sense like it is :-) Thanx Oliver -----Urspr?ngliche Nachricht----- Von: Rodrigo B. de Oliveira [mailto:rodrigobamboo@gmail.com] Gesendet: Donnerstag, 26. August 2004 14:07 An: Oliver Walczak Cc: python-dev@python.org Betreff: Re: [Python-Dev] exit in __builtins__ Hi! Try to type 'exit' or 'quit' in the interactive shell and you see its purpose. Pretty clever. Cheers, Rodrigo On Thu, 26 Aug 2004 14:00:40 +0200, Oliver Walczak wrote: > Dear list, > ... > > > Is it a bug or a feature? exit as builtin is not documented anywhere and I > cannot find out where it gets included or attached to the str object in the > sources. Any idea? > > Best regards > Oliver From mcherm at mcherm.com Thu Aug 26 14:38:35 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Thu Aug 26 14:34:48 2004 Subject: [Python-Dev] exit in __builtins__ Message-ID: <1093523915.412dd9cb5a57f@mcherm.com> Andrew Durdin writes: > But does this really belong in __builtins__? Wouldn't it be better for > the interactive interpreter to register it as a global when it sets > up, as it only makes sense in the context of the interactive > interpreter? It's a basic principle that the behavior of the interactive interpreter and the behavior of Python run non-interactively should be as close to the same as is possible. The requirement for blank lines after an indented suite is unavoidable because you can't "look ahead" in the interpreter. There are also the prompts. While I'm sure there are a few others, we try to keep the differences minimal. -- Michael Chermside From walter at livinglogic.de Thu Aug 26 14:47:36 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 26 14:47:42 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <412C38A2.9000909@iinet.net.au> References: <412B0C84.30204@livinglogic.de> <412C38A2.9000909@iinet.net.au> Message-ID: <412DDBE8.2070206@livinglogic.de> Nick Coghlan wrote: > Walter D?rwald wrote: > >> test_string.py is supposed to test the string module. This new test >> should be moved to test_str.py (or even better to string_test.py, >> so that it can be reused for doing the same test with unicode objects). > > > Ah, I think I see where the tests should have gone > (MixinStrUnicodeUserStringTest.test_join in string_test.py). > > I'll work up a patch to bring those tests in sync with the rest of the > string tests (the structure of which I find harder to follow than the > string code itself. . .). However, I probably won't get to it for a week > or so. I'm working on it, however I discovered that unicode.join() doesn't optimize this special case: s = "foo" assert "".join([s]) is s u = u"foo" assert u"".join([s]) is s The second assertion fails. I'd say that this test (joining a one item sequence returns the item itself) should be removed because it tests an implementation detail. I'm not sure, whether the optimization should be added to unicode.find(). Bye, Walter D?rwald From pinard at iro.umontreal.ca Thu Aug 26 15:54:01 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Thu Aug 26 15:54:36 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <1093367647.18858.62.camel@geddy.wooz.org> References: <1093232832.28573.16.camel@geddy.wooz.org> <20040823132043.GA4501@titan.progiciels-bpi.ca> <1093367647.18858.62.camel@geddy.wooz.org> Message-ID: <20040826135401.GA31295@alcyon.progiciels-bpi.ca> [Barry Warsaw] > > If my feeling is right, then the PEP should clearly explicit this > > goal [of pushing `$' forward], it will make the PEP stronger. > I will neither confirm nor deny whether the PSU is bankrolling the PEP > 292 initiative, nor the actual existence of any 527 organization > claiming to be called the "PSU", nor whether if they did exist, they > were or weren't acting in coordination with the campaign organizations > of any 2004 US presidential nominee. Nice joking. Still, yet, and nevertheless, I think making the PEP closer to the truth might make it stronger. It might also repair a bit the feeling that the PEP process is sometimes mildly lacking (the @-PEP having been ratehr publicised as an example of this, recently). -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From dave at boost-consulting.com Thu Aug 26 16:18:39 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 26 16:18:57 2004 Subject: [Python-Dev] list += string?? Message-ID: I just discovered the following behavior: C:\Documents and Settings\dave>python Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> prefix = ['foo', 'bar', 'baz'] >>> prefix += 'bomb' # meant to write prefix += [ 'bomb' ] >>> prefix ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b'] >>> Is it new and/or intentional? I would have expected an error from the +=. I was very surprised when my error "passed silently". -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From aahz at pythoncraft.com Thu Aug 26 16:23:25 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 26 16:23:43 2004 Subject: [Python-Dev] list += string?? In-Reply-To: References: Message-ID: <20040826142325.GA2855@panix.com> On Thu, Aug 26, 2004, David Abrahams wrote: > > I just discovered the following behavior: > > C:\Documents and Settings\dave>python > Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> prefix = ['foo', 'bar', 'baz'] > >>> prefix += 'bomb' # meant to write prefix += [ 'bomb' ] > >>> prefix > ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b'] > >>> > > Is it new and/or intentional? I would have expected an error from the > +=. I was very surprised when my error "passed silently". >>> l = ['foo'] >>> l += ('bar', 'baz') >>> l ['foo', 'bar', 'baz'] Augmented assignment for lists works with any sequence type (and probably iterators, too, but I didn't check). Strings are a sequence type, which occasionally has unintended consequences. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From tim.peters at gmail.com Thu Aug 26 16:30:53 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 26 16:30:56 2004 Subject: [Python-Dev] list += string?? In-Reply-To: References: Message-ID: <1f7befae04082607303b7bc26e@mail.gmail.com> [David Abrahams] > I just discovered the following behavior: > > C:\Documents and Settings\dave>python > Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> prefix = ['foo', 'bar', 'baz'] > >>> prefix += 'bomb' # meant to write prefix += [ 'bomb' ] > >>> prefix > ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b'] > >>> > > Is it new and/or intentional? I would have expected an error from the > +=. I was very surprised when my error "passed silently". It was new when the iterator protocol was introduced. It wasn't explicitly intended that you get surprises for strings specifically, but it was explicitly intended that list += whatever work like list.extend(whatever) and that whatever can be any iterable object. >>> prefix = ['foo', 'bar', 'baz'] >>> def g(): ... for ch in "bomb": ... yield ch ... >>> prefix += g() >>> prefix ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b'] >>> Since strings are iterable objects, it's no more or less surprising than that 'for ch in "bomb"' works too . From dave at boost-consulting.com Thu Aug 26 16:58:26 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 26 16:58:42 2004 Subject: [Python-Dev] list += string?? In-Reply-To: <1f7befae04082607303b7bc26e@mail.gmail.com> (Tim Peters's message of "Thu, 26 Aug 2004 10:30:53 -0400") References: <1f7befae04082607303b7bc26e@mail.gmail.com> Message-ID: Tim Peters writes: > [David Abrahams] >> I just discovered the following behavior: >> >> C:\Documents and Settings\dave>python >> Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> prefix = ['foo', 'bar', 'baz'] >> >>> prefix += 'bomb' # meant to write prefix += [ 'bomb' ] >> >>> prefix >> ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b'] >> >>> >> >> Is it new and/or intentional? I would have expected an error from the >> +=. I was very surprised when my error "passed silently". > > It was new when the iterator protocol was introduced. It wasn't > explicitly intended that you get surprises for strings specifically, > but it was explicitly intended that > > list += whatever > > work like > > list.extend(whatever) > > and that whatever can be any iterable object. I figured as much. > Since strings are iterable objects, it's no more or less surprising > than that 'for ch in "bomb"' works too . I want my ,= operator! -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From tismer at stackless.com Thu Aug 26 17:12:46 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Aug 26 17:12:58 2004 Subject: [Python-Dev] Minimal 'stackless' PEP using generators? In-Reply-To: <20040823153923.GA50408@prometheusresearch.com> References: <20040823153923.GA50408@prometheusresearch.com> Message-ID: <412DFDEE.5000108@stackless.com> Clark C. Evans wrote: ... > With these two changes, the "lower" function could be an async reactor > like those found in Twisted, or async core. While the result isn't true > coutines, it would be a huge improvement for those who would like to do > async coding. I've done something similar with Twisted called Flow [1] > and it works well, with the exception of being a painful syntax hack and > being quite slow. If this was moved into Python's core, we'd get most > of the advantages of coroutines without the full cost. > > Thoughts? Well, I just think "no". Generators, as limited as they are in Python, make some sense. Coroutines for me have the advantage to make a context switch. While generators are very often called in a context where they even could be inlined, coroutines should be really independent. But they are not independent if you just cannot switch, because one of them just happens to call a different function. A typical use of coroutines is the situation where it cannot be deduced who is caller or callee. What I mean, those situations which can be solved with a stack are the trivial cases, and that is exactly *not* what Stackless is about. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Carmerstr. 2 : *Starship* http://starship.python.net/ 10623 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 31 86 04 18 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mal at egenix.com Thu Aug 26 17:28:43 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 26 17:28:59 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412CEEE9.60207@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de> <412C4EA9.8010003@egenix.com> <412CEEE9.60207@v.loewis.de> Message-ID: <412E01AB.8030808@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> Martin, there are two reasons for hiding away these details: >> >> 1. we need to be able to change the codec state without >> breaking the APIs > > > That will be possible with the currently-proposed patch. > The _codecs methods are not public API, so changing them > would not be an API change. Uhm, I wasn't talking about the builtin codecs only (of course, we can change those to our liking). I'm after a generic interface for stateful codecs. >> 2. we don't want the state to be altered by the user > > > We are all consenting adults, and we can't *really* > prevent it, anyway. For example, the user may pass an > old state, or a state originating from a different codec > (instance). We need to support this gracefully (i.e. with > a proper Python exception). True, but the codec writer should be in control of the state object, its format and what the user can or cannot change. >> A single object serves this best and does not create >> a whole plethora of new APIs in the _codecs module. >> This is not over-design, but serves a reason. > > It does put a burden on codec developers, which need > to match the "official" state representation policy. > Of course, if they are allowed to return a tuple > representing their state, that would be fine with > me. They can use any object they like to keep the state in whatever format they choose. I think this makes it easier on the codec writer, rather than harder. Furthermore, they can change the way they store state e.g. to accomodate for new features they may want to add to the codec, without breaking the interface. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 26 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From tim.peters at gmail.com Thu Aug 26 20:21:25 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 26 20:21:32 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <412DDBE8.2070206@livinglogic.de> References: <412B0C84.30204@livinglogic.de> <412C38A2.9000909@iinet.net.au> <412DDBE8.2070206@livinglogic.de> Message-ID: <1f7befae040826112168b99b08@mail.gmail.com> [Walter D?rwald] > I'm working on it, however I discovered that unicode.join() > doesn't optimize this special case: > > s = "foo" > assert "".join([s]) is s > > u = u"foo" > assert u"".join([s]) is s > > The second assertion fails. Well, in that example it *has* to fail, because the input (s) wasn't a unicode string to begin with, but u"".join() must return a unicode string. Maybe you intended to say that assert u"".join([u]) is u fails (which is also true today, but doesn't need to be true tomorrow). > I'd say that this test (joining a one item sequence returns > the item itself) should be removed because it tests an > implementation detail. Neverthess, it's an important pragmatic detail. We should never throw away a test just because rearrangement makes a test less convenient. > I'm not sure, whether the optimization should be added to > unicode.find(). Believing you mean join(), yes. Doing common endcases efficiently in C code is an important quality-of-implementation concern, lest people need to add reams of optimization test-&-branch guesses in their own Python code. For example, the SpamBayes tokenizer has many passes that split input strings on magical separators of one kind or another, pasting the remaining pieces together again via string.join(). It's explicitly noted in the code that special-casing the snot out of "separator wasn't found" in Python is a lot slower than letting string.join(single_element_list) just return the list element, so that simple, uniform Python code works well in all cases. It's expected that *most* of these SB passes won't find the separator they're looking for, and it's important not to make endless copies of unboundedly large strings in the expected case. The more heavily used unicode strings become, the more important that they treat users kindly in such cases too. From mal at egenix.com Thu Aug 26 20:29:13 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 26 20:29:16 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <1f7befae040826112168b99b08@mail.gmail.com> References: <412B0C84.30204@livinglogic.de> <412C38A2.9000909@iinet.net.au> <412DDBE8.2070206@livinglogic.de> <1f7befae040826112168b99b08@mail.gmail.com> Message-ID: <412E2BF9.7030601@egenix.com> Tim Peters wrote: > [Walter D?rwald] >>I'm not sure, whether the optimization should be added to >>unicode.find(). > > Believing you mean join(), yes. +1 Unicode objects should behave as much as string objects as possible. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 26 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bac at OCF.Berkeley.EDU Thu Aug 26 21:02:14 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 26 21:02:22 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 Message-ID: <412E33B6.8080701@ocf.berkeley.edu> test__locale has been failing on OS X for quite a while now (I bet ever since we started compiling against the CoreFoundation framework since that is where the problem exists). I would like to deal with it before a3 goes out in case we need to change something semantically. Bug http://www.python.org./sf/1005113 has most of the details. Basically it looks like Apple locks down the locale so that it can't be change through locale.h . This means that all attempts to change the locale just fails silently. Obviously this sucks. I do have a contact at Apple who said it looks like it the function that caused this issue has been removed from the tree they are working with for Tiger, but that can obviously change and they could also have just moved the functionality to another function. There are a few options on how to deal with this. One is to just turn off test__locale for OS X. We do this for test_locale already so skipping locale tests on OS X already has a precedent. We can raise locale.Error on OS X for all attempts to set the locale. Problem I have with this is I don't know if this problem exists in other versions of OS X beyond 10.3 (test__locale passing for anyone on other OS X versions?). Last option is to try to fix it so that we set the locale through CoreFoundation. I have no clue how doable this is (quick searching turned up http://developer.apple.com/documentation/CoreFoundation/Conceptual/CFLocales/index.html so it might be possible). Problem is I know I can't get it in before a3. Possibly before b1, but no guarantee. And of course this is assuming this works the way we need it to which it might not. Personally I would like to turn off the test on OS X for now and then try to see if there is a proper way to solve this. -Brett From walter at livinglogic.de Thu Aug 26 21:54:34 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 26 21:54:52 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <1f7befae040826112168b99b08@mail.gmail.com> References: <412B0C84.30204@livinglogic.de> <412C38A2.9000909@iinet.net.au> <412DDBE8.2070206@livinglogic.de> <1f7befae040826112168b99b08@mail.gmail.com> Message-ID: <412E3FFA.5030002@livinglogic.de> Tim Peters wrote: > [Walter D?rwald] > >>I'm working on it, however I discovered that unicode.join() >>doesn't optimize this special case: >> >>s = "foo" >>assert "".join([s]) is s >> >>u = u"foo" >>assert u"".join([s]) is s >> >>The second assertion fails. > > Well, in that example it *has* to fail, because the input (s) wasn't a > unicode string to begin with, but u"".join() must return a unicode > string. Maybe you intended to say that > > assert u"".join([u]) is u > > fails Argl, you're right. > (which is also true today, but doesn't need to be true tomorrow). I've removed the test today, so it won't fail tomorrow. ;) >>I'd say that this test (joining a one item sequence returns >>the item itself) should be removed because it tests an >>implementation detail. > > Neverthess, it's an important pragmatic detail. We should never throw > away a test just because rearrangement makes a test less convenient. So, should I put the test back in (in test_str.py)? >>I'm not sure, whether the optimization should be added to >>unicode.find(). > > Believing you mean join(), yes. Unfortunately the implementations of str.join and unicode.join look completely different. str.join does a PySequence_Fast() and then tests whether the sequence length is 0 or 1, unicode.join iterates through the argument via PyObject_GetIter/PyIter_Next. Adding the optimization might result in a complete rewrite of PyUnicode_Join(). > Doing common endcases efficiently in > C code is an important quality-of-implementation concern, lest people > need to add reams of optimization test-&-branch guesses in their own > Python code. For example, the SpamBayes tokenizer has many passes > that split input strings on magical separators of one kind or another, > pasting the remaining pieces together again via string.join(). It's > explicitly noted in the code that special-casing the snot out of > "separator wasn't found" in Python is a lot slower than letting > string.join(single_element_list) just return the list element, so that > simple, uniform Python code works well in all cases. It's expected > that *most* of these SB passes won't find the separator they're > looking for, and it's important not to make endless copies of > unboundedly large strings in the expected case. The more heavily used > unicode strings become, the more important that they treat users > kindly in such cases too. Seems like we have to rewrite PyUnicode_Join(). Bye, Walter D?rwald From walter at livinglogic.de Thu Aug 26 22:06:18 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 26 22:06:28 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412C297B.2040208@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de> Message-ID: <412E42B9.4090508@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> If a tuple is passed and returned this makes it possible >> from Python code to mangle the state. IMHO this should be >> avoided if possible. > > Not necessarily. We are all consenting adults. So if the > code checks whether the state is sane, from a typing point > of view, i.e. if you can't crash the interpreter, then there > is no need to hide the state from the user. OK, that's true. Unfortunately I'm not really that familiar with the bit twiddling in the UTF7 decoder to know it this could break anything or not. Bye, Walter D?rwald From walter at livinglogic.de Thu Aug 26 22:10:40 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 26 22:10:52 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412C50D6.6010801@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C50D6.6010801@egenix.com> Message-ID: <412E43C0.5010006@livinglogic.de> M.-A. Lemburg wrote: >>> [...] >>> def decode_stateful(data, state=None): >>> ... decode and modify state ... >>> return (decoded_data, length_consumed, state) >> >> Another option might be that the decode function changes >> the state object in place. > > Good idea. But that's totally up to the implementor. >>> where the object type and contents of the state variable >>> is defined per codec (e.g. could be a tuple, just a single >>> integer or some other special object). >> >> If a tuple is passed and returned this makes it possible >> from Python code to mangle the state. IMHO this should be >> avoided if possible. > >>> Otherwise we'll end up having different interface >>> signatures for all codecs and extending them to accomodate >>> for future enhancement will become unfeasable without >>> introducing yet another set of APIs. >> >> We already have slightly different decoding functions: >> utf_16_ex_decode() takes additional arguments. > > Right - it was a step in the wrong direction. Let's not > use a different path for the future. utf_16_ex_decode() serves a purpose: it help implement the UTF16 decoder, which has to switch to UTF-16-BE or UTF-16-LE according to the BOM, so utf_16_ex_decode() needs a way to comunicate that back to the caller. >>> Let's discuss this some more and implement it for Python 2.5. >>> For Python 2.4, I think we can get away with what we already >>> have: >> >> > [...] >> >> OK, I've updated the patch. >> >>> [...] >>> The buffer logic should only be used for streams >>> that do not support the interface to push back already >>> read bytes (e.g. .unread()). >>> >>> From a design perspective, keeping read data inside the >>> codec is the wrong thing to do, simply because it leaves >>> the input stream in an undefined state in case of an error >>> and there's no way to correlate the stream's read position >>> to the location of the error. >>> >>> With a pushback method on the stream, all the stream >>> data will be stored on the stream, not the codec, so >>> the above would no longer be a problem. >> >> On the other hand this requires special stream. Data >> already read is part of the codec state, so why not >> put it into the codec? > > Ideally, the codec should not store data, I consider the remaining undecoded bytes to be part of the codec state once the have been read from the stream. > but only > reference it. It's better to keep things well > separated which is why I think we need the .unread() > interface and eventually a queue interface to support > the feeding operation. > >>> However, we can always add the .unread() support to the >>> stream codecs at a later stage, so it's probably ok >>> to default to the buffer logic for Python 2.4. >> >> OK. >> >>>> That still leaves the issue >>>> of the last read operation, which I'm tempted to leave unresolved >>>> for Python 2.4. No matter what the solution is, it would likely >>>> require changes to all codecs, which is not good. >>> >>> We could have a method on the codec which checks whether >>> the codec buffer or the stream still has pending data >>> left. Using this method is an application scope consideration, >>> not a codec issue. >> >> But this mean that the normal error handling can't be used >> for those trailing bytes. > > Right, but then: missing data (which usually causes the trailing > bytes) is really something for the application to deal with, > e.g. by requesting more data from the user, another application > or trying to work around the problem in some way. I don't think > that the codec error handler can practically cover these > possibilities. But in many cases the user might want to use "ignore" or "replace" error handling. Bye, Walter D?rwald From walter at livinglogic.de Thu Aug 26 22:13:17 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 26 22:13:21 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412CEEE9.60207@v.loewis.de> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de> <412C4EA9.8010003@egenix.com> <412CEEE9.60207@v.loewis.de> Message-ID: <412E445D.2050806@livinglogic.de> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >> Martin, there are two reasons for hiding away these details: >> >> 1. we need to be able to change the codec state without >> breaking the APIs > > That will be possible with the currently-proposed patch. > The _codecs methods are not public API, so changing them > would not be an API change. Exactly. >> 2. we don't want the state to be altered by the user > > We are all consenting adults, and we can't *really* > prevent it, anyway. For example, the user may pass an > old state, or a state originating from a different codec > (instance). We need to support this gracefully (i.e. with > a proper Python exception). The state communicated in the UTF-7 decoder is just a bunch of values. Checking the type is done via PyArg_ParseTuple(). >> A single object serves this best and does not create >> a whole plethora of new APIs in the _codecs module. >> This is not over-design, but serves a reason. > > It does put a burden on codec developers, which need > to match the "official" state representation policy. > Of course, if they are allowed to return a tuple > representing their state, that would be fine with > me. Looking at the UTF-7 decoder this seems to be the simplest option. Bye, Walter D?rwald From walter at livinglogic.de Thu Aug 26 22:19:48 2004 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 26 22:19:57 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412E01AB.8030808@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de> <412C4EA9.8010003@egenix.com> <412CEEE9.60207@v.loewis.de> <412E01AB.8030808@egenix.com> Message-ID: <412E45E4.2050504@livinglogic.de> M.-A. Lemburg wrote: > Martin v. L?wis wrote: > >> M.-A. Lemburg wrote: >> >>> Martin, there are two reasons for hiding away these details: >>> >>> 1. we need to be able to change the codec state without >>> breaking the APIs >> >> That will be possible with the currently-proposed patch. >> The _codecs methods are not public API, so changing them >> would not be an API change. > > Uhm, I wasn't talking about the builtin codecs only (of course, > we can change those to our liking). I'm after a generic > interface for stateful codecs. But that interface is only between the StreamReader and any helper function that the codec implementer might want to use. If there ise no helper function there is no interface. >>> 2. we don't want the state to be altered by the user >> >> We are all consenting adults, and we can't *really* >> prevent it, anyway. For example, the user may pass an >> old state, or a state originating from a different codec >> (instance). We need to support this gracefully (i.e. with >> a proper Python exception). > > True, but the codec writer should be in control of the > state object, its format and what the user can or cannot > change. Yes, we should not dictate, how, why or if the codec implementer has to pass around any state. The only thing we have to dictate is that StreamReaders have to keep their state between calls to read(). >>> A single object serves this best and does not create >>> a whole plethora of new APIs in the _codecs module. >>> This is not over-design, but serves a reason. >> >> It does put a burden on codec developers, which need >> to match the "official" state representation policy. >> Of course, if they are allowed to return a tuple >> representing their state, that would be fine with >> me. > > They can use any object they like to keep the state > in whatever format they choose. I think this makes it > easier on the codec writer, rather than harder. > > Furthermore, they can change the way they store state > e.g. to accomodate for new features they may want to > add to the codec, without breaking the interface. That's basically the current state of the codec machinery, so we don't have to change anything in the specification. BTW, I hope that I can add updated documentation to the patch tomorrow (for PyUnicode_DecodeUTF8Stateful() and friends and for the additional arguments to read()), because I'll be on vacation the next week. Bye, Walter D?rwald From dave at boost-consulting.com Thu Aug 26 22:34:35 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Aug 26 22:34:51 2004 Subject: [Python-Dev] urllib.urlretrieve Message-ID: Is there a known reliability problem with urllib.urlretrieve? I'm getting silent truncation of downloaded files. http://sourceforge.net/tracker/?func=detail&aid=1016880&group_id=5470&atid=105470 has the details. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From martin at v.loewis.de Thu Aug 26 23:00:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 26 23:01:06 2004 Subject: [Python-Dev] Decoding incomplete unicode In-Reply-To: <412E01AB.8030808@egenix.com> References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com> <4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de> <4107751C.2020207@egenix.com> <4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de> <411B6D79.2040503@egenix.com> <411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de> <412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de> <4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de> <4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de> <412C4EA9.8010003@egenix.com> <412CEEE9.60207@v.loewis.de> <412E01AB.8030808@egenix.com> Message-ID: <412E4F7C.80201@v.loewis.de> M.-A. Lemburg wrote: > Uhm, I wasn't talking about the builtin codecs only (of course, > we can change those to our liking). I'm after a generic > interface for stateful codecs. But we already have that! The StreamReader/StreamWriter interface is perfectly suited for stateful codecs, and we have been supporting it for several years now. The only problem is that a few builtin codecs failed to implement that correctly, and Walter's patch fixes that bug. There is no API change necessary whatsoever. > True, but the codec writer should be in control of the > state object, its format and what the user can or cannot > change. And indeed, with the current API, he is. Regards, Martin From martin at v.loewis.de Thu Aug 26 23:10:21 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Aug 26 23:10:54 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <412E33B6.8080701@ocf.berkeley.edu> References: <412E33B6.8080701@ocf.berkeley.edu> Message-ID: <412E51BD.6090401@v.loewis.de> Brett C. wrote: > Personally I would like to turn off the test on OS X for now and then > try to see if there is a proper way to solve this. In what way does that help? The test shows that the locale module is broken on OS X. So the only sensible options are to remove the module (or atleast the relevant functions), or to fix them. I fail to see how skipping the test does any good. The test fails, and life goes on. Regards, Martin From tim.peters at gmail.com Thu Aug 26 23:20:28 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Aug 26 23:20:36 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <412E3FFA.5030002@livinglogic.de> References: <412B0C84.30204@livinglogic.de> <412C38A2.9000909@iinet.net.au> <412DDBE8.2070206@livinglogic.de> <1f7befae040826112168b99b08@mail.gmail.com> <412E3FFA.5030002@livinglogic.de> Message-ID: <1f7befae04082614206fc7722@mail.gmail.com> [Walter D?rwald] ... > Seems like we have to rewrite PyUnicode_Join(). We probably should anyway. By eyeball just now, I noticed it slings expressions like "reslen + itemlen + seplen" and "sz*2" without worrying about integer overflow. string_join() is careful about such stuff; sooner or later, shortcuts of this nature turn into bug reports From fredrik at pythonware.com Thu Aug 26 23:28:28 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu Aug 26 23:26:45 2004 Subject: [Python-Dev] Re: urllib.urlretrieve References: Message-ID: David Abrahams wrote: > Is there a known reliability problem with urllib.urlretrieve? I'm > getting silent truncation of downloaded files. > > http://sourceforge.net/tracker/?func=detail&aid=1016880&group_id=5470&atid=105470 > > has the details. sourceforge is known to have reliability problems when downloading large files. here's a wget session on my machine (okay, it's not a silent truncation, but SF is clearly not well): 18450K -> .......... .......... .......... .......... .......... [ 54%] 18500K -> ..... [ 54%] Closing fd 1944 23:11:52 (62.94 KB/s) - Connection closed at byte 18949128. Retrying. --23:11:52-- http://cvs.sourceforge.net:80/cvstarballs/boost-cvsroot.tar.bz2 (try: 2) => `boost-cvsroot.tar.bz2.3' Connecting to cvs.sourceforge.net:80... Created fd 1944. connected! ---request begin--- GET /cvstarballs/boost-cvsroot.tar.bz2 HTTP/1.0 User-Agent: Wget/1.5.3.1 Host: cvs.sourceforge.net:80 Accept: */* Range: bytes=18949128- ---request end--- HTTP request sent, awaiting response... HTTP/1.1 502 Proxy Error Date: Thu, 26 Aug 2004 21:10:04 GMT Server: Apache/2.0.40 (Red Hat Linux) Content-Length: 441 Connection: close Content-Type: text/html; charset=iso-8859-1 Closing fd 1944 23:11:57 ERROR 502: Proxy Error. From raymond.hettinger at verizon.net Thu Aug 26 23:38:56 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu Aug 26 23:39:29 2004 Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String Substitutions Message-ID: <002301c48bb5$17bfbe40$e841fea9@oemcomputer> Before it's too late and the API gets frozen, I would like to propose an alternate implementation for PEP292 that exposes two functions instead of two classes. Current way: print Template('Turn $direction') % dict(direction='right') Proposed: print dollarsub('Turn $direction', dict(direction='right')) or: print dollarsub('Turn $direction', direction='right') My main issue with the current implementation is that we get no leverage from using a class instead of a function. Though the API is fairly simple either way, it is easier to learn and document functions than classes. We gain nothing from instantiation -- the underlying data remains immutable and no additional state is attached. The only new behavior is the ability to apply the mod operator. Why not do it in one step. I had thought a possible advantage of classes was that they could be usefully subclassed. However, a little dickering around showed that to do anything remotely interesting (beyond changing the pattern alphabet) you have to rewrite everything by overriding both the method and the pattern. Subclassing gained you nothing, but added a little bit of complexity. A couple of simple exercises show this clearly: write a subclass using a different escape character or one using dotted identifiers for attribute lookup in the local namespace -- either way subclasses provide no help and only get in the way. One negative effect of the class implementation is that it inherits from unicode and always returns a unicode result even if all of the inputs (mapping values and template) are regular strings. With a function implementation, that can be avoided (returning unicode only if one of the inputs is unicode). The function approach also makes it possible to have keyword arguments (see the example above) as well as a mapping. This isn't a big win, but it is nice to have and reads well in code that is looping over multiple substitutions (mailmerge style): for girl in littleblackbook: print dollarsub(loveletter, name=girl[0].title(), favoritesong=girl[3]) Another minor advantage for a function is that it is easier to lookup in the reference. If a reader sees the % operator being applied and looks it up in the reference, it is going to steer them in the wrong direction. This is doubly true if the Template instantiation is remote from the operator application. Summary for functions: * is more appropriate when there is no state * no unnecessary instantiation * can be applied in a single step * a little easier to learn/use/document * doesn't force result to unicode * allows keyword arguments * easy to find in the docs Raymond ----------- Sample Implementation ------------- def dollarsub(template, mapping=None, **kwds): """A function for supporting $-substitutions.""" typ = type(template) if mapping is None: mapping = kwds def convert(mo): escaped, named, braced, bogus = mo.groups() if escaped is not None: return '$' if bogus is not None: raise ValueError('Invalid placeholder at index %d' % mo.start('bogus')) val = mapping[named or braced] return typ(val) return _pattern.sub(convert, template) def safedollarsub(template, mapping=None, **kwds): """A function for $-substitutions. This function is 'safe' in the sense that you will never get KeyErrors if there are placeholders missing from the interpolation dictionary. In that case, you will get the original placeholder in the value string. """ typ = type(template) if mapping is None: mapping = kwds def convert(mo): escaped, named, braced, bogus = mo.groups() if escaped is not None: return '$' if bogus is not None: raise ValueError('Invalid placeholder at index %d' % mo.start('bogus')) if named is not None: try: return typ(mapping[named]) except KeyError: return '$' + named try: return typ(mapping[braced]) except KeyError: return '${' + braced + '}' return _pattern.sub(convert, template) From python at rcn.com Thu Aug 26 23:41:33 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 26 23:42:09 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/testtest_string.py, 1.25, 1.26 In-Reply-To: <1f7befae04082614206fc7722@mail.gmail.com> Message-ID: <002401c48bb5$765463c0$e841fea9@oemcomputer> > [Walter D?rwald] > ... > > Seems like we have to rewrite PyUnicode_Join(). > [Uncle Tim] > We probably should anyway. By eyeball just now, I noticed it slings > expressions like "reslen + itemlen + seplen" and "sz*2" without > worrying about integer overflow. string_join() is careful about such > stuff; sooner or later, shortcuts of this nature turn into bug reports Sounds like an excellent project for Nick. Too bad I had just convinced him it was a can of worms ;-) Raymond From tim.peters at gmail.com Fri Aug 27 00:20:50 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 27 00:20:59 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/testtest_string.py, 1.25, 1.26 In-Reply-To: <002401c48bb5$765463c0$e841fea9@oemcomputer> References: <002401c48bb5$765463c0$e841fea9@oemcomputer> Message-ID: <1f7befae04082615206760c806@mail.gmail.com> [Walter D?rwald] >>> ... >>> Seems like we have to rewrite PyUnicode_Join(). [Uncle Timmy] >> We probably should anyway. By eyeball just now, I noticed it slings >> expressions like "reslen + itemlen + seplen" and "sz*2" without >> worrying about integer overflow. string_join() is careful about such >> stuff; sooner or later, shortcuts of this nature turn into bug reports [Nephew Raymond] > Sounds like an excellent project for Nick. > Too bad I had just convinced him it was a can of worms ;-) I needed a break from intractable database problems, and am almost done with PyUnicode_Join(). I'm not doing auto-unicode(), though, so there will still be plenty of fun left for Nick! From dave at boost-consulting.com Fri Aug 27 00:45:47 2004 From: dave at boost-consulting.com (David Abrahams) Date: Fri Aug 27 00:46:02 2004 Subject: [Python-Dev] Re: urllib.urlretrieve References: Message-ID: "Fredrik Lundh" writes: >> Is there a known reliability problem with urllib.urlretrieve? I'm >> getting silent truncation of downloaded files. >> >> http://sourceforge.net/tracker/?func=detail&aid=1016880&group_id=5470&atid=105470 >> >> has the details. > > sourceforge is known to have reliability problems when downloading > large files. here's a wget session on my machine (okay, it's not a silent > truncation, but SF is clearly not well): Okay, that tells me something... but shouldn't urllib throw an exception in case of a problem... or shouldn't it do something to retry? -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From bac at OCF.Berkeley.EDU Fri Aug 27 00:56:23 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 27 00:56:34 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <412E51BD.6090401@v.loewis.de> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> Message-ID: <412E6A97.5030509@ocf.berkeley.edu> Martin v. L?wis wrote: > Brett C. wrote: > >> Personally I would like to turn off the test on OS X for now and then >> try to see if there is a proper way to solve this. > > > In what way does that help? The test shows that the locale module is > broken on OS X. So the only sensible options are to remove the module > (or atleast the relevant functions), or to fix them. I fail to see how > skipping the test does any good. The test fails, and life goes on. > It prevents repetitive bug reports for an issue we already know about. And as I said in the email, test_locale is skipped for OS X so I thought same reasoning would apply here. -Brett From dave at boost-consulting.com Fri Aug 27 01:12:30 2004 From: dave at boost-consulting.com (David Abrahams) Date: Fri Aug 27 01:12:47 2004 Subject: [Python-Dev] Re: list += string?? References: <1f7befae04082607303b7bc26e@mail.gmail.com> Message-ID: David Abrahams writes: >> It was new when the iterator protocol was introduced. It wasn't >> explicitly intended that you get surprises for strings >> specifically, but it was explicitly intended that >> >> list += whatever >> >> work like >> >> list.extend(whatever) >> >> and that whatever can be any iterable object. > > > I figured as much. Report from the field: I just discovered a bug that's been hiding in my code for a few months. The code was being exercised, but the result was the same as if the code had been correct until I threw some new examples at it. The problem was: self.body[e[0]:e[1]] = s self.body is a list of strings, and s is a string. Later ''.join(self.body) gets called, so for many cases the bug was hidden. No offense intended (okay, maybe a tiny little offense), but the experience here is a bit Perl-like. >> Since strings are iterable objects, it's no more or less surprising >> than that 'for ch in "bomb"' works too . > > I want my ,= operator! I still do, but it wouldn't have solved _this_ problem. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com From ncoghlan at iinet.net.au Fri Aug 27 01:20:15 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Aug 27 01:21:58 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <412DDBE8.2070206@livinglogic.de> References: <412B0C84.30204@livinglogic.de> <412C38A2.9000909@iinet.net.au> <412DDBE8.2070206@livinglogic.de> Message-ID: <412E702F.6030808@iinet.net.au> Walter D?rwald wrote: > I'm working on it, however I discovered that unicode.join() > doesn't optimize this special case: I noticed that during the implementation of the patch. I think it's because unicode join doesn't use a multi-pass approach, so there's no opportunity to notice that the optimisation is possible. > s = "foo" > assert "".join([s]) is s > > u = u"foo" > assert u"".join([s]) is s > > The second assertion fails. > > I'd say that this test (joining a one item sequence returns > the item itself) should be removed because it tests an > implementation detail. Fine by me - I was uncertain about that when I posted the patch to SF (there's a question about it in the patch tracker item). The main reason I had that test there was to make sure that the second version of the patch didn't break this optimisation the way the first version did. Regards, Nick. From bac at OCF.Berkeley.EDU Fri Aug 27 01:41:10 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 27 01:41:23 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <412E51BD.6090401@v.loewis.de> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> Message-ID: <412E7516.5080805@ocf.berkeley.edu> Martin v. L?wis wrote: > Brett C. wrote: > >> Personally I would like to turn off the test on OS X for now and then >> try to see if there is a proper way to solve this. > > > In what way does that help? The test shows that the locale module is > broken on OS X. So the only sensible options are to remove the module > (or atleast the relevant functions), or to fix them. I have an idea for detecting a broken setlocale. When configure.in does its compiling does it compile with the same flags that the Makefile uses for compiling? If it does we can test for something we know will be different (fr_FR has a different decimal point, for instance) and compare them and then have a value in pyconfig.h to signal that a busted setlocale exists. setlocale can then raise locale.Error automatically for platforms that have that macro defined. But the test has to be compiled with the ``-framework CoreFoundation`` option or else this won't work. The reason I don't want to do a blanket ``#ifdef __APPLE__`` solution is because I don't know if this was an issue in OS X 10.2 or earlier (http://www.hmug.org/man/3/setlocale.html seems to suggest setlocale was deprecated which means it may have still worked). -Brett From ncoghlan at iinet.net.au Fri Aug 27 01:53:19 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Aug 27 01:55:01 2004 Subject: [Python-Dev] Auto-str and auto-unicode in join In-Reply-To: <1f7befae04082615206760c806@mail.gmail.com> References: <002401c48bb5$765463c0$e841fea9@oemcomputer> <1f7befae04082615206760c806@mail.gmail.com> Message-ID: <412E77EF.80007@iinet.net.au> Tim Peters wrote: > I needed a break from intractable database problems, and am almost > done with PyUnicode_Join(). I'm not doing auto-unicode(), though, so > there will still be plenty of fun left for Nick! I actually got that mostly working (off slightly out-of-date CVS though). Joining a sequence of 10 integers with auto-str seems to take about 60% of the time of a str(x) list comprehension on that same sequence (and the PySequence_Fast call means that a generator is slightly slower than a list comp!). For a sequence which mixed strings and non-strings, the gains could only increase. However, there is one somewhat curly problem I'm not sure what to do about. To avoid slowing down the common case of string join (a list of only strings) it is necessary to do the promotion to string in the type-check & size-calculation pass. That's fine in the case of a list that consists of only strings and non-basestrings, or the case of a unicode separator - every non-basestring is converted using either PyObject_Str or PyObject_Unicode. Where it gets weird is something like this: ''.join([an_int, a_unicode_str]) u''.join([an_int, a_unicode_str]) In the first case, the int will first be converted to a string via PyObject_Str, and then that string representation is what will get converted to Unicode after the detection of the unicode string causes the join to be handed over to Unicode join. In the latter case, the int is converted directly to Unicode. So my question would be, is it reasonable to expect that PyObject_Unicode(PyObject_Str(some_object)) give the same answer as PyObject_Unicode(some_object)? If not, then the string join would have to do something whereby it kept a 'pristine' version of the sequence around to hand over to the Unicode join. My first attempt at implementing this feature had that property, but also had the effect of introducing about a 1% slowdown of the standard sequence-of-strings case (it introduced an extra if statement to see if a 'stringisation' pass was required after the initial type checking and sizing pass). For longer sequences than 10 strings, I imagine the relative slowdown would be much less. Hmm. . . I think I see a way to implement this, while still avoiding adding any code to the standard path through the function. It'd be slower for the case where an iterator is passed in, and we automatically invoke PyObject_Str but don't end up delegating to Unicode join, though, as it involves making a copy of the sequence that only gets used if the Unicode join is invoked. (If the original object is a real sequence, rather than an iterator, there is no extra overhead - we have to make the copy anyway, to avoid mutating the user's sequence). If people are definitely interested in this feature, I could probably put a patch together next week. Regards, Nick. From nbastin at opnet.com Fri Aug 27 02:49:53 2004 From: nbastin at opnet.com (Nick Bastin) Date: Fri Aug 27 02:50:15 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <412E7516.5080805@ocf.berkeley.edu> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> <412E7516.5080805@ocf.berkeley.edu> Message-ID: <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> On Aug 26, 2004, at 7:41 PM, Brett C. wrote: > Martin v. L?wis wrote: > >> Brett C. wrote: >>> Personally I would like to turn off the test on OS X for now and >>> then try to see if there is a proper way to solve this. >> In what way does that help? The test shows that the locale module is >> broken on OS X. So the only sensible options are to remove the module >> (or atleast the relevant functions), or to fix them. > > I have an idea for detecting a broken setlocale. When configure.in > does its compiling does it compile with the same flags that the > Makefile uses for compiling? If it does we can test for something we > know will be different (fr_FR has a different decimal point, for > instance) and compare them and then have a value in pyconfig.h to > signal that a busted setlocale exists. setlocale can then raise > locale.Error automatically for platforms that have that macro defined. > But the test has to be compiled with the ``-framework > CoreFoundation`` option or else this won't work. I think we should actually just fix this, rather than removing locale support for MacOS X builds. The trick is that we need to call __setonlyClocaleconv() again, to remove the protection against calling setlocale() in framework builds (why they did this in the first place is beyond me). Seems like it should be an easy gate against darwin, and then call that on startup. -- Nick From greg at cosc.canterbury.ac.nz Fri Aug 27 03:32:44 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Fri Aug 27 03:32:50 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <1f7befae040826112168b99b08@mail.gmail.com> Message-ID: <200408270132.i7R1Wilx007968@cosc353.cosc.canterbury.ac.nz> Tim Peters : > > u = u"foo" > > assert u"".join([s]) is s > > Well, in that example it *has* to fail, because the input (s) wasn't a > unicode string to begin with, but u"".join() must return a unicode > string. It could be argued that joining a single-element list doesn't involve using the separator at all, so its type shouldn't matter. It could also be argued that joining with an empty string, of whatever flavour, is equivalent to just concatenating the list elements with no separator, so again the type of the separator shouldn't matter. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Fri Aug 27 03:46:24 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 27 03:46:26 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_string.py, 1.25, 1.26 In-Reply-To: <200408270132.i7R1Wilx007968@cosc353.cosc.canterbury.ac.nz> References: <200408270132.i7R1Wilx007968@cosc353.cosc.canterbury.ac.nz> Message-ID: <1f7befae04082618465600f3aa@mail.gmail.com> [Greg Ewing] > It could be argued that joining a single-element list doesn't > involve using the separator at all, so its type shouldn't matter. > > It could also be argued that joining with an empty string, of > whatever flavour, is equivalent to just concatenating the list > elements with no separator, so again the type of the separator > shouldn't matter. Since such arguments wouldn't prevail, I'd rather not bother. From bac at OCF.Berkeley.EDU Fri Aug 27 03:54:06 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 27 03:54:17 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> <412E7516.5080805@ocf.berkeley.edu> <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> Message-ID: <412E943E.4060908@ocf.berkeley.edu> Nick Bastin wrote: > > I think we should actually just fix this, rather than removing locale > support for MacOS X builds. The trick is that we need to call > __setonlyClocaleconv() again, to remove the protection against calling > setlocale() in framework builds (why they did this in the first place is > beyond me). Seems like it should be an easy gate against darwin, and > then call that on startup. > I don't like that solution since that function is obviously not part of any public API. -Brett From bob at redivi.com Fri Aug 27 04:32:12 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 27 04:32:48 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <412E943E.4060908@ocf.berkeley.edu> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> <412E7516.5080805@ocf.berkeley.edu> <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> <412E943E.4060908@ocf.berkeley.edu> Message-ID: <4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com> On Aug 26, 2004, at 9:54 PM, Brett C. wrote: > Nick Bastin wrote: > >> I think we should actually just fix this, rather than removing locale >> support for MacOS X builds. The trick is that we need to call >> __setonlyClocaleconv() again, to remove the protection against >> calling setlocale() in framework builds (why they did this in the >> first place is beyond me). Seems like it should be an easy gate >> against darwin, and then call that on startup. > > I don't like that solution since that function is obviously not part > of any public API. Well there are two ways to fix it: 1) Call __setonlyClocaleconv() if it's there via weak linking or whatnot around any setlocale or the like. This will at least affect OS X 10.3, I'm not sure about 10.2, and the rumor says it's fixed "now" in 10.4. 2) Write a whole new module that uses Apple API for localization. Obviously 2 is the "best" solution, but requires the most time. 1 is easy-ish and will work reliably on all the machines that need it (assuming the rumor is correct) unless Apple does something totally strange and changes the behavior of a previous-release OS for reasons other than security flaws :) -bob From bac at OCF.Berkeley.EDU Fri Aug 27 05:33:54 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 27 05:33:59 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> <412E7516.5080805@ocf.berkeley.edu> <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> <412E943E.4060908@ocf.berkeley.edu> <4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com> Message-ID: <412EABA2.2000502@ocf.berkeley.edu> Bob Ippolito wrote: > > Well there are two ways to fix it: > > 1) Call __setonlyClocaleconv() if it's there via weak linking or whatnot > around any setlocale or the like. This will at least affect OS X 10.3, > I'm not sure about 10.2, and the rumor says it's fixed "now" in 10.4. > > 2) Write a whole new module that uses Apple API for localization. > > Obviously 2 is the "best" solution, but requires the most time. 1 is > easy-ish and will work reliably on all the machines that need it > (assuming the rumor is correct) unless Apple does something totally > strange and changes the behavior of a previous-release OS for reasons > other than security flaws :) > OK, starting to sound like detecting __setonlyClocaleconv() in configure.in and using that info to deal with it is winning with other people, at least as an initial solution. Everyone else agree with this? I just checked and it looks like calling the function with a non-zero argument will force the locale back to "C" even if you just set it to a specific locale; so the function seems to force the locale to "C" and lock it down. So it will most likely need to be called right before the first setlocale call made by Python (I think it is in PyInitialize() ) and then not call it again. -Brett From bob at redivi.com Fri Aug 27 06:59:06 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Aug 27 06:59:44 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <412EABA2.2000502@ocf.berkeley.edu> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> <412E7516.5080805@ocf.berkeley.edu> <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> <412E943E.4060908@ocf.berkeley.edu> <4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com> <412EABA2.2000502@ocf.berkeley.edu> Message-ID: On Aug 26, 2004, at 11:33 PM, Brett C. wrote: > Bob Ippolito wrote: > >> Well there are two ways to fix it: >> 1) Call __setonlyClocaleconv() if it's there via weak linking or >> whatnot around any setlocale or the like. This will at least affect >> OS X 10.3, I'm not sure about 10.2, and the rumor says it's fixed >> "now" in 10.4. >> 2) Write a whole new module that uses Apple API for localization. >> Obviously 2 is the "best" solution, but requires the most time. 1 is >> easy-ish and will work reliably on all the machines that need it >> (assuming the rumor is correct) unless Apple does something totally >> strange and changes the behavior of a previous-release OS for reasons >> other than security flaws :) > > OK, starting to sound like detecting __setonlyClocaleconv() in > configure.in and using that info to deal with it is winning with other > people, at least as an initial solution. Everyone else agree with > this? No. Don't use configure. The machine you compile Python with is not necessarily the machine you will use Python with (for example, compiling a 10.2 compatible Python and running on 10.4). Use dyld or CFBundle API to look up the function dynamically, or use weak linking. > I just checked and it looks like calling the function with a non-zero > argument will force the locale back to "C" even if you just set it to > a specific locale; so the function seems to force the locale to "C" > and lock it down. So it will most likely need to be called right > before the first setlocale call made by Python (I think it is in > PyInitialize() ) and then not call it again. Calling the function with a non-zero argument forces the locale to stay at "C", and it returns the last value that was passed to it. I would say that any Python function that depends on locale should do: x = __setonlyClocaleconv(0) # do python stuff carefully __setonlyClocaleconv(x) Otherwise you will probably break CoreFoundation (which could break just about anything, since nearly all Apple APIs depend on it). Maybe this is just about as much work as writing a CoreFoundation based locale module? I'm not sure which parts of Python are affected by setlocale(). -bob From nbastin at opnet.com Fri Aug 27 07:10:57 2004 From: nbastin at opnet.com (Nick Bastin) Date: Fri Aug 27 07:11:17 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> <412E7516.5080805@ocf.berkeley.edu> <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> <412E943E.4060908@ocf.berkeley.edu> <4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com> <412EABA2.2000502@ocf.berkeley.edu> Message-ID: <7A882E04-F7E7-11D8-8D32-000D932927FE@opnet.com> On Aug 27, 2004, at 12:59 AM, Bob Ippolito wrote: > Maybe this is just about as much work as writing a CoreFoundation > based locale module? I'm not sure which parts of Python are affected > by setlocale(). As a side note, I'm currently working on wrapping ICU as a Python module, which would give you reliable cross-platform locale support, independent of the system locale. Incidentally, ICU is what Apple uses in MacOS X anyhow... This module will not be ready before 2.4... :-( -- Nick From ronaldoussoren at mac.com Fri Aug 27 08:28:58 2004 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri Aug 27 08:29:16 2004 Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3 In-Reply-To: <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de> <412E7516.5080805@ocf.berkeley.edu> <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com> Message-ID: <60F976B0-F7F2-11D8-8627-000D93AD379E@mac.com> On 27-aug-04, at 2:49, Nick Bastin wrote: > > On Aug 26, 2004, at 7:41 PM, Brett C. wrote: > >> Martin v. L?wis wrote: >> >>> Brett C. wrote: >>>> Personally I would like to turn off the test on OS X for now and >>>> then try to see if there is a proper way to solve this. >>> In what way does that help? The test shows that the locale module is >>> broken on OS X. So the only sensible options are to remove the module >>> (or atleast the relevant functions), or to fix them. >> >> I have an idea for detecting a broken setlocale. When configure.in >> does its compiling does it compile with the same flags that the >> Makefile uses for compiling? If it does we can test for something we >> know will be different (fr_FR has a different decimal point, for >> instance) and compare them and then have a value in pyconfig.h to >> signal that a busted setlocale exists. setlocale can then raise >> locale.Error automatically for platforms that have that macro >> defined. But the test has to be compiled with the ``-framework >> CoreFoundation`` option or else this won't work. > > I think we should actually just fix this, rather than removing locale > support for MacOS X builds. The trick is that we need to call > __setonlyClocaleconv() again, to remove the protection against calling > setlocale() in framework builds (why they did this in the first place > is beyond me). Seems like it should be an easy gate against darwin, > and then call that on startup. Why do you think it is save to re-enable setlocale? There must be some reason for disabling setlocale. If you re-enable setlocale some random CoreFoundation function might suddenly fail. Ronald From fredrik at pythonware.com Fri Aug 27 09:45:02 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri Aug 27 09:43:24 2004 Subject: [Python-Dev] Re: urllib.urlretrieve References: Message-ID: David Abrahams wrote: >> sourceforge is known to have reliability problems when downloading >> large files. here's a wget session on my machine (okay, it's not a silent >> truncation, but SF is clearly not well): > > Okay, that tells me something... but shouldn't urllib throw an > exception in case of a problem... or shouldn't it do something to > retry? from the HTTP speification: HTTP/1.1 user agents MUST notify the user when an invalid length is received and detected. is urllib a user agent? or is that better left to your application? file, headers = urllib.urlretrieve(...) if os.path.getsize(file) != int(headers.get("content-length", 0)): print "oops!" From mal at egenix.com Fri Aug 27 11:02:05 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 27 11:02:08 2004 Subject: [Python-Dev] Auto-str and auto-unicode in join In-Reply-To: <412E77EF.80007@iinet.net.au> References: <002401c48bb5$765463c0$e841fea9@oemcomputer> <1f7befae04082615206760c806@mail.gmail.com> <412E77EF.80007@iinet.net.au> Message-ID: <412EF88D.2050908@egenix.com> Nick Coghlan wrote: > Tim Peters wrote: > >> I needed a break from intractable database problems, and am almost >> done with PyUnicode_Join(). I'm not doing auto-unicode(), though, so >> there will still be plenty of fun left for Nick! > > > I actually got that mostly working (off slightly out-of-date CVS though). > > Joining a sequence of 10 integers with auto-str seems to take about 60% > of the time of a str(x) list comprehension on that same sequence (and > the PySequence_Fast call means that a generator is slightly slower than > a list comp!). For a sequence which mixed strings and non-strings, the > gains could only increase. > > However, there is one somewhat curly problem I'm not sure what to do about. > > To avoid slowing down the common case of string join (a list of only > strings) it is necessary to do the promotion to string in the type-check > & size-calculation pass. > > That's fine in the case of a list that consists of only strings and > non-basestrings, or the case of a unicode separator - every > non-basestring is converted using either PyObject_Str or PyObject_Unicode. > > Where it gets weird is something like this: > ''.join([an_int, a_unicode_str]) > u''.join([an_int, a_unicode_str]) This gives you a TypeError, so it's a non-issue (.join() does not do an implicit call to str(obj) on the list elements). The real issue is the case where you have [a_str, a_unicode_obj] and for that the current implementation already does the right thing, namely to look for Unicode objects in the length checking pass. > In the first case, the int will first be converted to a string via > PyObject_Str, and then that string representation is what will get > converted to Unicode after the detection of the unicode string causes > the join to be handed over to Unicode join. > > In the latter case, the int is converted directly to Unicode. > > So my question would be, is it reasonable to expect that > PyObject_Unicode(PyObject_Str(some_object)) give the same answer as > PyObject_Unicode(some_object)? > > If not, then the string join would have to do something whereby it kept > a 'pristine' version of the sequence around to hand over to the Unicode > join. > > My first attempt at implementing this feature had that property, but > also had the effect of introducing about a 1% slowdown of the standard > sequence-of-strings case (it introduced an extra if statement to see if > a 'stringisation' pass was required after the initial type checking and > sizing pass). For longer sequences than 10 strings, I imagine the > relative slowdown would be much less. > > Hmm. . . I think I see a way to implement this, while still avoiding > adding any code to the standard path through the function. It'd be > slower for the case where an iterator is passed in, and we automatically > invoke PyObject_Str but don't end up delegating to Unicode join, though, > as it involves making a copy of the sequence that only gets used if the > Unicode join is invoked. (If the original object is a real sequence, > rather than an iterator, there is no extra overhead - we have to make > the copy anyway, to avoid mutating the user's sequence). > > If people are definitely interested in this feature, I could probably > put a patch together next week. > > Regards, > Nick. > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/mal%40egenix.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 27 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Aug 27 11:20:50 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 27 11:20:52 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects unicodeobject.c, 2.219, 2.220 In-Reply-To: References: Message-ID: <412EFCF2.4030608@egenix.com> tim_one@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Objects > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15103/Objects > > Modified Files: > unicodeobject.c > Log Message: > PyUnicode_Join(): Two primary aims: > > 1. u1.join([u2]) is u2 > 2. Be more careful about C-level int overflow. > > Since PySequence_Fast() isn't needed to achieve #1, it's not used -- but > the code could sure be simpler if it were. Hmm, you've now made PyUnicode_Join() to work with iterators whereas PyString_Join() only works for sequences. What are the performance implications of this for PyUnicode_Join() ? Since the string and Unicode implementations have to be in sync, we'd also need to convert PyString_Join() to work on iterators. Which brings up the second question: What are the performance implications of this for PyString_Join() ? The join operation is a widely used method, so both implementations need to be as fast as possible. It may be worthwhile making the PySequence_Fast() approach a special case in both routines and using the iterator approach as fallback if no sequence is found. Note that PyString_Join() with iterator support will also have to be careful about not trying to iterate twice, so it will have to use a similiar logic to the one applied in PyString_Format() where the work already done up to the point where it finds a Unicode string is reused when calling PyUnicode_Format(). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 27 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From aahz at pythoncraft.com Fri Aug 27 15:56:50 2004 From: aahz at pythoncraft.com (Aahz) Date: Fri Aug 27 15:56:53 2004 Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String Substitutions In-Reply-To: <002301c48bb5$17bfbe40$e841fea9@oemcomputer> References: <002301c48bb5$17bfbe40$e841fea9@oemcomputer> Message-ID: <20040827135650.GB25839@panix.com> On Thu, Aug 26, 2004, Raymond Hettinger wrote: > > * doesn't force result to unicode This is the main reason I'm +0, pending further arguments. OTOH, I also like using %, so you'd have to come up with more points to move me beyond +0. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From fumanchu at amor.org Fri Aug 27 16:14:23 2004 From: fumanchu at amor.org (Robert Brewer) Date: Fri Aug 27 16:19:55 2004 Subject: [Python-Dev] J2 proposal final Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E91@exchange.hqamor.amorhq.net> Hello, Guido, The J2 patch and proposal are finished (http://www.aminus.org/rbre/python/pydec.html) and ready for your consideration. In the interest of time, I'm submitting the arguments to you now. The "call for signatories" is still open, but I think you can get a feel for the level of support based upon the current endorsements, and modify that as more names come in. I'll be in Ensenada (at a friend's bachelor party, if you must know :) until Sunday evening, at which time I will add in any new signatories. If you have any questions until that time, I'm sure Michael Sparks (among others) would be happy to answer or delegate them. Your decision schedule is of course your own. Thanks for your consideration and support throughout this process! For the Python community, Robert Brewer MIS Amor Ministries fumanchu@amor.org P.S. For the record, I had selected and am now submitting the J2 syntax based on a community vote on c.l.p., wherein each voter cast 3 votes, for a total of 70 voters with 209 votes. The top contenders as of today (a good time to officially close that vote): 1st 2nd 3rd Total % A1 4 2 3 9 C1 9 9 12 30 0.14 C2 7 7 2 16 J2 33 34 30 97 0.46 The total number of votes for J2 beat the total number of votes for C1 (the next contender) by over three to one. It was unclear to many in that poll whether A1 was a valid vote; it's not recommended to draw any conclusions regarding the preference of voters for A1 from that vote--that may be more correctly drawn from the list of signatories of the proposal. From tim.peters at gmail.com Fri Aug 27 16:30:27 2004 From: tim.peters at gmail.com (Tim Peters) Date: Fri Aug 27 16:30:30 2004 Subject: [Python-Dev] Re: Re: [Python-checkins] python/dist/src/Objects unicodeobject.c, 2.219, 2.220 In-Reply-To: <412EFCF2.4030608@egenix.com> References: <412EFCF2.4030608@egenix.com> Message-ID: <1f7befae040827073056affd4d@mail.gmail.com> [M.-A. Lemburg] > Hmm, you've now made PyUnicode_Join() to work with iterators > whereas PyString_Join() only works for sequences. They have both worked with iterators since the release in which iterators were introduced. Nothing changed now in this respect. > What are the performance implications of this for PyUnicode_Join() ? None. > Since the string and Unicode implementations have to be in sync, > we'd also need to convert PyString_Join() to work on iterators. It already does. I replied earlier this week on the same topic -- maybe you didn't see that, or maybe you misunderstand what PySequence_Fast does. > Which brings up the second question: > What are the performance implications of this for PyString_Join() ? None. > The join operation is a widely used method, so both implementations > need to be as fast as possible. It may be worthwhile making the > PySequence_Fast() approach a special case in both routines and > using the iterator approach as fallback if no sequence is found. string_join uses PySequence_Fast already; the Unicode join didn't, and still doesn't. In the cases of exact list or tuple arguments, PySequence_Fast would be quicker in Unicode join. But in any cases other than those, PySequence_Fast materializes a concrete tuple containing the full materialized iteration, so could be more memory-consuming. That's probably a good tradeoff, though. > Note that PyString_Join() with iterator support will also > have to be careful about not trying to iterate twice, It already is. Indeed, the primary reason it uses PySequence_Fast is to guarantee that it never iterates over an iterator argument more than once. The Unicode join doesn't have that potential problem. > so it will have to use a similiar logic to the one applied > in PyString_Format() where the work already done up to the > point where it finds a Unicode string is reused when calling > PyUnicode_Format(). >>> def g(): ... for piece in 'a', 'b', u'c', 'd': # force Unicode promotion on 3rd yield ... yield piece ... >>> ' '.join(g()) u'a b c d' >>> From mal at egenix.com Fri Aug 27 16:47:49 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 27 16:47:54 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects unicodeobject.c, 2.219, 2.220 In-Reply-To: <1f7befae040827073056affd4d@mail.gmail.com> References: <412EFCF2.4030608@egenix.com> <1f7befae040827073056affd4d@mail.gmail.com> Message-ID: <412F4995.6080606@egenix.com> Tim Peters wrote: > [M.-A. Lemburg] > >>Hmm, you've now made PyUnicode_Join() to work with iterators >>whereas PyString_Join() only works for sequences. > > > They have both worked with iterators since the release in which > iterators were introduced. Nothing changed now in this respect. > > >>What are the performance implications of this for PyUnicode_Join() ? > > > None. > > >>Since the string and Unicode implementations have to be in sync, >>we'd also need to convert PyString_Join() to work on iterators. > > > It already does. I replied earlier this week on the same topic -- > maybe you didn't see that, or maybe you misunderstand what > PySequence_Fast does. Indeed. At the time Fredrik added this API, it was optimized for lists and tuples and had a fallback mechanism for arbitrary sequences. Didn't know that it now also works for iterators. Nice ! >>Which brings up the second question: >>What are the performance implications of this for PyString_Join() ? > > > None. > > >>The join operation is a widely used method, so both implementations >>need to be as fast as possible. It may be worthwhile making the >>PySequence_Fast() approach a special case in both routines and >>using the iterator approach as fallback if no sequence is found. > > > string_join uses PySequence_Fast already; the Unicode join didn't, and > still doesn't. In the cases of exact list or tuple arguments, > PySequence_Fast would be quicker in Unicode join. But in any cases > other than those, PySequence_Fast materializes a concrete tuple > containing the full materialized iteration, so could be more > memory-consuming. That's probably a good tradeoff, though. Indeed. I'd opt for going the PySequence_Fast() way for Unicode as well. >>Note that PyString_Join() with iterator support will also >>have to be careful about not trying to iterate twice, > > > It already is. Indeed, the primary reason it uses PySequence_Fast is > to guarantee that it never iterates over an iterator argument more > than once. The Unicode join doesn't have that potential problem. > > >>so it will have to use a similiar logic to the one applied >>in PyString_Format() where the work already done up to the >>point where it finds a Unicode string is reused when calling >>PyUnicode_Format(). > > >>>>def g(): > > ... for piece in 'a', 'b', u'c', 'd': # force Unicode promotion on 3rd yield > ... yield piece > ... > >>>>' '.join(g()) > > u'a b c d' Nice :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 27 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From anthony at interlink.com.au Fri Aug 27 17:28:04 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 27 17:28:24 2004 Subject: [Python-Dev] J2 proposal final In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E3022E91@exchange.hqamor.amorhq.net> References: <3A81C87DC164034AA4E2DDFE11D258E3022E91@exchange.hqamor.amorhq.net> Message-ID: <412F5304.1040105@interlink.com.au> Robert Brewer wrote: > Hello, Guido, > > The J2 patch and proposal are finished > (http://www.aminus.org/rbre/python/pydec.html) and ready for your > consideration. In the interest of time, I'm submitting the arguments to > you now. The "call for signatories" is still open, but I think you can > get a feel for the level of support based upon the current endorsements, > and modify that as more names come in. The patch from Michael Sparks is at http://www.python.org/sf/1013835 Anthony From fredrik at pythonware.com Fri Aug 27 17:30:31 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri Aug 27 17:28:49 2004 Subject: [Python-Dev] Re: Re: def fn (args) [dec,dec]: References: <41136453.1040908@interlink.com.au><200408061445.i76EjqT07026@guido.python.org> Message-ID: Jeremy Hylton wrote: >> (If I could design a language from scratch, I might want to move >> docstrings to before the 'def' as well; I've come to appreciate this >> in Java code. Fredrik Lundh's PythonDoc also uses comments prefixing >> the method definition; he also uses @ so he should be happy with this >> syntax, as he seemed to be when I mentioned it to him at EuroPython.) > > It's not at all clear that Fredrik likes it, but he doesn't participate > in Python development any more so we may never know :-(. you can ask. I do read mail, eventually. ;-) From jhylton at gmail.com Fri Aug 27 18:30:17 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Aug 27 18:30:55 2004 Subject: [Python-Dev] Re: list += string?? In-Reply-To: References: <1f7befae04082607303b7bc26e@mail.gmail.com> Message-ID: On Thu, 26 Aug 2004 19:12:30 -0400, David Abrahams wrote: > Report from the field: I just discovered a bug that's been hiding in > my code for a few months. The code was being exercised, but the > result was the same as if the code had been correct until I threw > some new examples at it. The problem was: > > self.body[e[0]:e[1]] = s > > self.body is a list of strings, and s is a string. Later > ''.join(self.body) gets called, so for many cases the bug was hidden. > No offense intended (okay, maybe a tiny little offense), but the > experience here is a bit Perl-like. It's a fair complaint. I've made the mistake myself, and it usually takes a few head scratching and investigation before I realize that I'm passing a string instead of some other sequence. On the other hand, I like slicing lists and occasionally I iterate over their characters. What's the alternative? A typecheck seems like it would be handy, but I'm not sure what type I would want "all sequences except strings"? Jeremy From python at rcn.com Fri Aug 27 19:37:11 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Aug 27 19:37:48 2004 Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String Substitutions Message-ID: <000601c48c5c$7e88f040$e841fea9@oemcomputer> > > * doesn't force result to unicode [Aahz] > This is the main reason I'm +0, pending further arguments. Employing class logic for a function problem is my main reason. Any function can be wrapped in class logic and use an overloaded operator (string.template for example) -- it's invariably the wrong thing to do for reasons of complexity, separation of instantiation and application, and the issues that always arise when an operator is overloaded (a good technique that should only be applied sparingly). [Aahz] > OTOH, I also > like using %, so you'd have to come up with more points to move me > beyond +0. The reasons center around the remoteness of instantiation from application, the differences from current uses of %, and an issue with the % operator itself. When application is remote from instantiation (for instance, when a template argument is supplied to a function), we get several problems. Given a line like "r = t % m", it is hard to verify that the code is correct. Should there be KeyError trapping? Can m be a tuple? Is t a %template, a safetemplate, or dollartemplate? Is the result a str or Unicode object? Should the decision of safe vs regular substitution be made at the point of instantiation or application? The temptation will be to reuse existing code that uses the % operator which unfortunately is not the same (especially with respect to applying tuples, return types, and auto-stringizing). The % operator is hard to search for in the docs and has precedence issues arising from its primary use for modulo arithemetic. Also, using a function makes it possible to use both % formatting and $ formatting (I do have use a case for this). Further, the % operator mnemonically only makes sense with % identifier tags -- it makes less sense with $ tags. Whatever answer is chosen, it should be forward looking and consider that others will want to add functionality (local namespace lookups for example) and that some poor bastards are going to waste time figuring out how to subclass the current implementation. In time, there will likely be more than two of these -- do you want more classes or more functions? Raymond From anthony at interlink.com.au Fri Aug 27 19:49:06 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Aug 27 19:49:29 2004 Subject: [Python-Dev] PEP-0273 (zip imports) needs to be finished off Message-ID: <412F7412.4050001@interlink.com.au> I just noticed that PEP-0273 is not yet up to date. As the code's been in there since 2.3, it'd be a shame if we had to rip it out of 2.4a3 for lack of a current PEP And yes, I've not had time to finish PEP-0318. This _will_ be done before 2.4a3. Hopefully _this_ weekend I'll get time to finish it... -- Anthony Baxter It's never too late to have a happy childhood. From dan.gass at gmail.com Fri Aug 27 20:06:16 2004 From: dan.gass at gmail.com (Dan Gass) Date: Fri Aug 27 20:06:20 2004 Subject: [Python-Dev] HTML Side By Side Differencing -- Alpha3 Message-ID: I'm looking for a developer to apply this patch for inclusion in Alpha3. The patch provides methods to generate side by side differences in HTML format (highlights and hyperlinks changed areas). The patch adds functionality to the difflib.py module without changing any of the existing code (low risk of breaking anything). There has been a good deal of participation in improving and reviewing the patch. What I need is someone who has check in privileges and feels comfortable implementing the patch. I've had good support (see patch comments) from a number of people with the exception of some hesitation from Martin v. L?wis which was countered by arguments from Tim Peters and Jim Jewett. In addition to having a wider audience to solicit help from, this is being posted on Python-Dev to provide an opportunity for others to comment on the patch as well since it is being considered for inclusion in Alpha3. For that person who steps forward I can promise that I will provide any needed support in a timely manner. Thanks, Dan Gass From dan.gass at gmail.com Fri Aug 27 20:18:55 2004 From: dan.gass at gmail.com (Dan Gass) Date: Fri Aug 27 20:19:01 2004 Subject: [Python-Dev] HTML Side By Side Differencing -- Alpha3 In-Reply-To: References: Message-ID: Sorry, I forgot to include the link: Python Patch #914575 (https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470) From pythondev at bitfurnace.com Fri Aug 27 20:24:34 2004 From: pythondev at bitfurnace.com (damien morton) Date: Fri Aug 27 20:29:08 2004 Subject: [Python-Dev] Re: list += string?? In-Reply-To: References: <1f7befae04082607303b7bc26e@mail.gmail.com> Message-ID: <412F7C62.8090508@bitfurnace.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040827/7e493a75/attachment.html From martin at v.loewis.de Fri Aug 27 21:14:50 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Aug 27 21:14:52 2004 Subject: [Python-Dev] Re: list += string?? In-Reply-To: <412F7C62.8090508@bitfurnace.com> References: <1f7befae04082607303b7bc26e@mail.gmail.com> <412F7C62.8090508@bitfurnace.com> Message-ID: <412F882A.4010604@v.loewis.de> damien morton wrote: > Maybe this is the time to thow out the idea that strings arent > sequences. Not backwards comaptable, but strings might grow a > string.chars property that _is_ a sequence, but strings themselves arent > sequences. I very much doubt users will accept such a change - especially if the rationale is that it is made to prevent a certain class of errors; the change itself is likely to *introduce* new errors, both for changing the existing code, and for the feature itself. We would need to come up with a type for string.chars, and people would pass *that* around, recreating the original problem. I would hope that pychecker could diagnose this kind of problem in many cases. Regards, Martin From pf_moore at yahoo.co.uk Fri Aug 27 22:12:04 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Fri Aug 27 22:11:53 2004 Subject: [Python-Dev] Re: PEP-0273 (zip imports) needs to be finished off References: <412F7412.4050001@interlink.com.au> Message-ID: Anthony Baxter writes: > I just noticed that PEP-0273 is not yet up to date. As the > code's been in there since 2.3, it'd be a shame if we had to > rip it out of 2.4a3 for lack of a current PEP I'm not sure how (directly) relevant PEP 273 is any more. It documented Jim Ahlstrom's approach, which was superseded by PEP 302's import hooks, and the zipimport module that uses them. I believe that PEP 302 is more or less up to date (there is discussion of a "Phase 2" rationalisation in the PEP, which hasn't been implemented, but IIRC that is mainly an internal detail and doesn't affect the user-visible interface). I'll do a quick check of the PEP against reality, but if there are mismatches, I can't offer to fix them (Just was the coding wizard, I just wrote some of the blurb). There needs to be some documentation of the zipimport module, certainly. I'm not sure if that should be in PEP 273 (zipimport is acknowledged in that PEP, but as a competitor rather than an implementation), in a separate PEP, or as a part of PEP 203 (with PEP 273 marked as rejected in the latter 2 cases). Personally, I wouldn't feel comfortable modifying PEP 273 (it would be more or less a complete rewrite) without Jim's agreement. In practice, my preference would be to write a section for the documentation covering all of this, and just put a prominent pointer at the top of PEP 273 - something like "A variation on this proposal was implemented in Python 2.3. For details see section XXX of the documentation". But where should this go? Even packages aren't too well documented (see the "XXX Can't be bothered to spell this out" comment at the end of section 6.12 of the language reference). Would a new section 6.12.2 "Import Semantics" be appropriate? Hmm, that would be my preference. I'll see if I can work up a documentation patch... Always-promising-more-than-I-can-deliver-ly y'rs Paul. -- The brain is a wonderful organ. It starts working the moment you get up in the morning and does not stop until you get into the office. -- Robert Frost From zathras at thwackety.com Fri Aug 27 23:53:38 2004 From: zathras at thwackety.com (Michael Sparks) Date: Fri Aug 27 23:31:34 2004 Subject: [Python-Dev] J2 proposal final In-Reply-To: <412F5304.1040105@interlink.com.au> Message-ID: On Sat, 28 Aug 2004, Anthony Baxter wrote: > Robert Brewer wrote: > > Hello, Guido, > > > > The J2 patch and proposal are finished > > (http://www.aminus.org/rbre/python/pydec.html) and ready for your > > consideration. In the interest of time, I'm submitting the arguments to > > you now. The "call for signatories" is still open, but I think you can > > get a feel for the level of support based upon the current endorsements, > > and modify that as more names come in. > > > The patch from Michael Sparks is at http://www.python.org/sf/1013835 Hopefully it goes without saying, but obviously I'll be willing to make any necessary changes if/as required. The implementation is deliberately simplistic - hopefully it covers all the areas that would be expected. Best Regards, Michael. From pje at telecommunity.com Fri Aug 27 23:37:56 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Aug 27 23:37:36 2004 Subject: [Python-Dev] PEP 333: Python Web Server Gateway Interface v1.0 Message-ID: <5.1.1.6.0.20040827173709.025f5c50@mail.telecommunity.com> (Separately posted to python-list to avoid cross-traffic.) PEP: 333 Title: Python Web Server Gateway Interface v1.0 Version: $Revision: 1.1 $ Last-Modified: $Date: 2004/08/27 17:30:09 $ Author: Phillip J. Eby Discussions-To: Python Web-SIG Status: Draft Type: Informational Content-Type: text/x-rst Created: 07-Dec-2003 Post-History: 07-Dec-2003, 08-Aug-2004, 20-Aug-2004, 27-Aug-2004 Abstract ======== This document specifies a proposed standard interface between web servers and Python web applications or frameworks, to promote web application portability across a variety of web servers. Rationale and Goals =================== Python currently boasts a wide variety of web application frameworks, such as Zope, Quixote, Webware, SkunkWeb, PSO, and Twisted Web -- to name just a few [1]_. This wide variety of choices can be a problem for new Python users, because generally speaking, their choice of web framework will limit their choice of usable web servers, and vice versa. By contrast, although Java has just as many web application frameworks available, Java's "servlet" API makes it possible for applications written with any Java web application framework to run in any web server that supports the servlet API. The availability and widespread use of such an API in web servers for Python -- whether those servers are written in Python (e.g. Medusa), embed Python (e.g. mod_python), or invoke Python via a gateway protocol (e.g. CGI, FastCGI, etc.) -- would separate choice of framework from choice of web server, freeing users to choose a pairing that suits them, while freeing framework and server developers to focus on their preferred area of specialization. This PEP, therefore, proposes a simple and universal interface between web servers and web applications or frameworks: the Python Web Server Gateway Interface (WSGI). But the mere existence of a WSGI spec does nothing to address the existing state of servers and frameworks for Python web applications. Server and framework authors and maintainers must actually implement WSGI for there to be any effect. However, since no existing servers or frameworks support WSGI, there is little immediate reward for an author who implements WSGI support. Thus, WSGI *must* be easy to implement, so that an author's initial investment in the interface can be reasonably low. Thus, simplicity of implementation on *both* the server and framework sides of the interface is absolutely critical to the utility of the WSGI interface, and is therefore the principal criterion for any design decisions. Note, however, that simplicity of implementation for a framework author is not the same thing as ease of use for a web application author. WSGI presents an absolutely "no frills" interface to the framework author, because bells and whistles like response objects and cookie handling would just get in the way of existing frameworks' handling of these issues. Again, the goal of WSGI is to facilitate easy interconnection of existing servers and applications or frameworks, not to create a new web framework. Note also that this goal precludes WSGI from requiring anything that is not already available in deployed versions of Python. Therefore, new standard library modules are not proposed or required by this specification, and nothing in WSGI requires a Python version greater than 2.2.2. (It would be a good idea, however, for future versions of Python to include support for this interface in web servers provided by the standard library.) In addition to ease of implementation for existing and future frameworks and servers, it should also be easy to create request preprocessors, response postprocessors, and other WSGI-based "middleware" components that look like an application to their containing server, while acting as a server for their contained applications. If middleware can be both simple and robust, and WSGI is widely available in servers and frameworks, it allows for the possibility of an entirely new kind of Python web application framework: one consisting of loosely-coupled WSGI middleware components. Indeed, existing framework authors may even choose to refactor their frameworks' existing services to be provided in this way, becoming more like libraries used with WSGI, and less like monolithic frameworks. This would then allow application developers to choose "best-of-breed" components for specific functionality, rather than having to commit to all the pros and cons of a single framework. Of course, as of this writing, that day is doubtless quite far off. In the meantime, it is a sufficient short-term goal for WSGI to enable the use of any framework with any server. Finally, it should be mentioned that the current version of WSGI does not prescribe any particular mechanism for "deploying" an application for use with a web server or server gateway. At the present time, this is necessarily implementation-defined by the server or gateway. After a sufficient number of servers and frameworks have implemented WSGI to provide field experience with varying deployment requirements, it may make sense to create another PEP, describing a deployment standard for WSGI servers and application frameworks. [Remainder truncated due to python-dev posting size limit; please see: http://python.org/peps/pep-0333.html for remaining text.] From aahz at pythoncraft.com Sat Aug 28 01:39:58 2004 From: aahz at pythoncraft.com (Aahz) Date: Sat Aug 28 01:40:01 2004 Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String Substitutions In-Reply-To: <000601c48c5c$7e88f040$e841fea9@oemcomputer> References: <000601c48c5c$7e88f040$e841fea9@oemcomputer> Message-ID: <20040827233958.GA5560@panix.com> On Fri, Aug 27, 2004, Raymond Hettinger wrote: > > When application is remote from instantiation (for instance, when a > template argument is supplied to a function), we get several problems. > Given a line like "r = t % m", it is hard to verify that the code is > correct. Should there be KeyError trapping? Can m be a tuple? Is t a > %template, a safetemplate, or dollartemplate? Is the result a str or > Unicode object? Should the decision of safe vs regular substitution be > made at the point of instantiation or application? All right, this moves me to +1, in the absence of any good arguments in favor of keeping the class-based system. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From anthony at interlink.com.au Sat Aug 28 09:08:55 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat Aug 28 09:09:10 2004 Subject: [Python-Dev] __mangled in stdlib considered poor form Message-ID: <41302F87.1000805@interlink.com.au> I'd like to see all occurrances of __mangledName in the stdlib removed and replaced with just _mangledName. Rationale: The double-under "private" names are a hack. They don't really protect variables, except against accidental stomping by subclasses. Looking at the uses in the std lib, they're not being used for that - they're being used for internal details. A single underscore is sufficient for that. The mangled names cause problems when someone _needs_ to override the implementation details in a subclass - I can recall a case with ConfigParser in SpamBayes, where the end result was the subclass with code like __ConfigParser_foo (a subsequent release of Python fixed that instance of double-under madness). Yes, you shouldn't be fiddling with internals of a class in a base class, and any time you do this, you run the risk of being refactored into oblivion, but sometimes it's necessary. Double-under mangled names are the worst of both worlds - they don't actually protect the variables, but they force someone who needs to poke into the class to use a mangled name. Given that part of the purpose of the stdlib is to act as a sort of "best practice" example of Python code, I'd like to see all instances of it in the std library removed. I'm not sure if it's feasible to attack this before 2.4b1, or if it should wait until 2.5, but I _would_ like to see it happen. What sayeth the rest of python-dev? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mwh at python.net Sat Aug 28 13:36:44 2004 From: mwh at python.net (Michael Hudson) Date: Sat Aug 28 13:36:48 2004 Subject: [Python-Dev] __mangled in stdlib considered poor form In-Reply-To: <41302F87.1000805@interlink.com.au> (Anthony Baxter's message of "Sat, 28 Aug 2004 17:08:55 +1000") References: <41302F87.1000805@interlink.com.au> Message-ID: <2md61b5wab.fsf@starship.python.net> Anthony Baxter writes: > I'd like to see all occurrances of __mangledName in the stdlib > removed and replaced with just _mangledName. [...] > What sayeth the rest of python-dev? +1. Make sure you get unittest.py too. Cheers, mwh -- want to write my requirements for me? Sure! "show a dancing monkey in the about box" -- from Twisted.Quotes From fredrik at pythonware.com Sat Aug 28 11:19:36 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat Aug 28 19:58:21 2004 Subject: [Python-Dev] Re: python/dist/src/Lib/lib-tk tkFont.py,1.6,1.7 Message-ID: <200408280917.i7S9Hqk10867@pythonware.com> loewis@users.sourceforge.net wrote: > Expand tabs. > > Index: tkFont.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/lib-tk/tkFont.py,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -C2 -d -r1.6 -r1.7 > *** tkFont.py 18 Aug 2004 11:06:44 -0000 1.6 > --- tkFont.py 18 Aug 2004 17:47:40 -0000 1.7 > *************** > *** 92,96 **** > # if font config info supplied, apply it > if font: > ! print "font=%r" % font > root.tk.call("font", "configure", self.name, *font) > else: > --- 92,96 ---- > # if font config info supplied, apply it > if font: > ! print "font=%r" % font > root.tk.call("font", "configure", self.name, *font) > else: ahem. tab or not, that sure looks like a debugging statement, right? From gvanrossum at gmail.com Sat Aug 28 20:50:45 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat Aug 28 20:50:54 2004 Subject: [Python-Dev] __mangled in stdlib considered poor form In-Reply-To: <2md61b5wab.fsf@starship.python.net> References: <41302F87.1000805@interlink.com.au> <2md61b5wab.fsf@starship.python.net> Message-ID: > > I'd like to see all occurrances of __mangledName in the stdlib > > removed and replaced with just _mangledName. > > [...] > > > What sayeth the rest of python-dev? > > +1. +0.5 here. Let's first get the terminology straight: __foo is a private name, _Class__foo is the mangled form of that. Whenever the stdlib uses a mangled name, clearly what ought to be done is change the private name to a single underscore. (I don't want to call that protected, because that has a different meaning in Java etc.) There's one potential use case where I'd like to keep the private name even if there's a mangled use of it nearby: when the private name really is intended to be private, and the mangled use is a "friend" in the C++ sense. Sometimes (though rarely) the stdlib has to indulge in practices that aren't good example code, because it's the stdlib, and it should implement functionality in the most bulletproof way possible. Using private names for internal details is the right thing sometimes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sat Aug 28 21:20:37 2004 From: python at rcn.com (Raymond Hettinger) Date: Sat Aug 28 21:21:26 2004 Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String Substitutions In-Reply-To: <20040827233958.GA5560@panix.com> Message-ID: <000501c48d34$19ce9000$e841fea9@oemcomputer> > > When application is remote from instantiation (for instance, when a > > template argument is supplied to a function), we get several problems. > > Given a line like "r = t % m", it is hard to verify that the code is > > correct. Should there be KeyError trapping? Can m be a tuple? Is t a > > %template, a safetemplate, or dollartemplate? Is the result a str or > > Unicode object? Should the decision of safe vs regular substitution be > > made at the point of instantiation or application? [Aahz] > All right, this moves me to +1, in the absence of any good arguments in > favor of keeping the class-based system. FWIW, I've loaded the proposed improvements to the sandbox: \nondist\sandbox\string\alt292.py It includes a doctest section that illustrates all of the improvements in action (appropriate return type, error handling, commenting, code simplicity, keyword arguments, function API, etc). I'm not sure what to do next. Aside from Aahz, no one on python-dev seems to be interested and Barry appears to have been off-line for a few days. Ideally, I would like to get this in a few days ahead of the alpha release. Raymond From skip at pobox.com Sat Aug 28 22:58:49 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Aug 28 22:59:12 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> Message-ID: <16688.61961.182410.162379@montanaro.dyndns.org> >> > Is there any consensus forming on whether bytes() instances are >> > mutable or not? >> >> Mutable! Phillip> So, how will it be different from: Phillip> from array import array Phillip> def bytes(*initializer): Phillip> return array('B',*initializer) That might well be a decent trial implementation, though it seems that if we allow strings as initializers we should also allow strings in assignment: >>> b = bytes("abc\xff") >>> b array('B', [97, 98, 99, 255]) >>> print buffer(b) abc? >>> b[3] = '\xfe' Traceback (most recent call last): File "", line 1, in ? TypeError: an integer is required >>> b += "\xfe" Traceback (most recent call last): File "", line 1, in ? TypeError: can only extend array with array (not "str") I must admit I'm having a bit of trouble getting past this point with a more traditional subclass (can array objects not be subclassed?) in part I think because I don't understand new-style classes very well. In particular, I couldn't find a description of __new__, and once I fumbled past that, I didn't seem to be able to override append() or extend(). Skip From gvanrossum at gmail.com Sat Aug 28 23:17:01 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat Aug 28 23:17:08 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16688.61961.182410.162379@montanaro.dyndns.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <16688.61961.182410.162379@montanaro.dyndns.org> Message-ID: > I must admit I'm having a bit of trouble getting past this point with a more > traditional subclass (can array objects not be subclassed?) in part I think > because I don't understand new-style classes very well. In particular, I > couldn't find a description of __new__, and once I fumbled past that, I > didn't seem to be able to override append() or extend(). Alas, the array type doesn't seem completely fit for subclassing; I tried similar things and couldn't gt it to work! Even more mysterious is that the array implementation appears to support the buffer API and yet it can't be used as an argument to write(). What's going on? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sun Aug 29 00:10:14 2004 From: python at rcn.com (Raymond Hettinger) Date: Sun Aug 29 00:11:05 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: Message-ID: <001901c48d4b$cb86f6e0$e841fea9@oemcomputer> [GvR] > Alas, the array type doesn't seem completely fit for subclassing; Famous last words: It doesn't look like it would be hard to make array's subclassable. I'll work on that this weekend. Reminder: there is an outstanding array feature request awaiting your adjudication: www.python.org/sf/992967 Raymond From martin at v.loewis.de Sun Aug 29 01:57:50 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 29 01:57:51 2004 Subject: [Python-Dev] Re: python/dist/src/Lib/lib-tk tkFont.py,1.6,1.7 In-Reply-To: <200408280917.i7S9Hqk10867@pythonware.com> References: <200408280917.i7S9Hqk10867@pythonware.com> Message-ID: <41311BFE.4050405@v.loewis.de> > ahem. tab or not, that sure looks like a debugging statement, right? Right, and it is also gone a while ago. Martin From martin at v.loewis.de Sun Aug 29 02:06:54 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 29 02:06:56 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <16688.61961.182410.162379@montanaro.dyndns.org> References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <16688.61961.182410.162379@montanaro.dyndns.org> Message-ID: <41311E1E.2040801@v.loewis.de> Skip Montanaro wrote: > >>> b[3] = '\xfe' I think you meant to write b[3] = 0xfe # or byte(0xfe) here. Assigning to an index always takes the element type in Python. It's only strings where reading an index returns the container type. > >>> b += "\xfe" And here, you probably meant to write b += bytes("\xfe") Regards, Martin From dima at trit.org Sun Aug 29 02:42:22 2004 From: dima at trit.org (Dima Dorfman) Date: Sun Aug 29 02:44:20 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: References: <2mbrhaaq0e.fsf@starship.python.net> <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <16688.61961.182410.162379@montanaro.dyndns.org> Message-ID: <20040829004221.GB1593@trit.org> Guido van Rossum wrote: > Even more mysterious > is that the array implementation appears to support the buffer API and > yet it can't be used as an argument to write(). What's going on? It supports the "read" buffer API but not the "character" buffer API, so the file has to be opened in binary mode for it to work: >>> a = array.array('c', 'fish') >>> open('/dev/null', 'w').write(a) Traceback (most recent call last): File "", line 1, in ? TypeError: argument 1 must be string or read-only character buffer, not array.array >>> open('/dev/null', 'wb').write(a) >>> That restriction(?) comes from this in file_write: PyArg_ParseTuple(args, f->f_binary ? "s#" : "t#", ... where s# requires a read buffer and t# requires a character buffer. array.array is the only type in the core that's a read buffer but not a character buffer, and I can't find any semantic differences between read and character buffers. If someone can explain the differences or confirm that there aren't any, I'll make this work. The easiest thing to do would be to make array support the character buffer API (but maybe only for [cbBu] types?). Dima. From tim.peters at gmail.com Sun Aug 29 03:51:48 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 29 03:51:51 2004 Subject: [Python-Dev] Auto-str and auto-unicode in join In-Reply-To: <412E77EF.80007@iinet.net.au> References: <002401c48bb5$765463c0$e841fea9@oemcomputer> <1f7befae04082615206760c806@mail.gmail.com> <412E77EF.80007@iinet.net.au> Message-ID: <1f7befae040828185146444b6d@mail.gmail.com> If we were to do auto-str, it would be better to rewrite str.join() as a 1-pass algorithm, using the kind of "double allocated space as needed" gimmick unicode.join uses. It would be less efficient if auto-promotion to Unicode turns out to be required, but it's hard to measure how little I care about that; it might be faster if auto-str and Unicode promotion aren't needed (as only 1 pass would be needed). auto-str couldn't really *mean* string.join(map(str, seq)) either. The problem with the latter is that if a seq element x is a unicode instance, str(x) will convert it into an encoded (8-bit) str, which would not be backward compatible. So the logic would be more (in outline): class string: def join(self, seq): seq = PySequence_Fast(seq) if seq is NULL: return NULL if len(seq) == 0: return "" elif len(seq) == 1 and type(seq[0]) is str: return seq[0] allocate a string object with (say) 100 bytes of space let p point to the first free byte for x in seq: if type(x) is str: copy x's guts into p, getting more space if needed elif isinstance(x, unicode): return unicode,join(self, seq) else: x = PyObject_Str(x) if x is NULL: return NULL copy x's guts into p, etc if not the last element: copy the separator's guts into p, etc cut p back to the space actually used return p's string object Note a peculiarity: if x is neither str nor unicode, but has a __str__ or __repr__ method that returns a unicode object, PyObject_Str() will convert that into an 8-bit str. That may be surprising. It would be ugly to duplicate most of the logic from PyObject_Unicode() to try to guess whether there's "a natural" Unicode spelling of x. I think I'd rather say "tough luck -- use unicode.join if that's what you want". From gvanrossum at gmail.com Sun Aug 29 03:53:47 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun Aug 29 03:53:50 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <20040829004221.GB1593@trit.org> References: <2mbrhaaq0e.fsf@starship.python.net> <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> <200408170134.i7H1YDR14560@guido.python.org> <4121ABAB.9060605@interlink.com.au> <200408171045.i7HAjQZ15411@guido.python.org> <4121E859.8000908@interlink.com.au> <2mbrhaaq0e.fsf@starship.python.net> <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> <16688.61961.182410.162379@montanaro.dyndns.org> <20040829004221.GB1593@trit.org> Message-ID: > array.array is the only type in the core that's a read buffer but not > a character buffer, and I can't find any semantic differences between > read and character buffers. If someone can explain the differences or > confirm that there aren't any, I'll make this work. The easiest thing > to do would be to make array support the character buffer API (but > maybe only for [cbBu] types?). Ah, that makes sense. And no please, keep it that way. It's compatible with the idea that bytes arrays should be read from / written to binary files. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Sun Aug 29 13:21:14 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Aug 29 13:21:22 2004 Subject: [Python-Dev] Auto-str and auto-unicode in join In-Reply-To: <1f7befae040828185146444b6d@mail.gmail.com> References: <002401c48bb5$765463c0$e841fea9@oemcomputer> <1f7befae04082615206760c806@mail.gmail.com> <412E77EF.80007@iinet.net.au> <1f7befae040828185146444b6d@mail.gmail.com> Message-ID: <4131BC2A.9060206@egenix.com> Tim Peters wrote: > If we were to do auto-str, it would be better to rewrite str.join() as > a 1-pass algorithm, using the kind of "double allocated space as > needed" gimmick unicode.join uses. It would be less efficient if > auto-promotion to Unicode turns out to be required, but it's hard to > measure how little I care about that; it might be faster if auto-str > and Unicode promotion aren't needed (as only 1 pass would be needed). FWIW, I'm -1 on doing auto-conversion using str()/unicode() in .join(). It's easy enough to make the stringification explicit in the application. Making this automatic (with all the problems that go with it), would only hide programming errors related to mismatching types and create a whole new set of weird error situations. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From pf_moore at yahoo.co.uk Sun Aug 29 15:48:19 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Sun Aug 29 15:48:22 2004 Subject: [Python-Dev] Re: Alternative Implementation for PEP 292: Simple String Substitutions References: <20040827233958.GA5560@panix.com> <000501c48d34$19ce9000$e841fea9@oemcomputer> Message-ID: "Raymond Hettinger" writes: > I'm not sure what to do next. Aside from Aahz, no one on python-dev > seems to be interested and Barry appears to have been off-line for a few > days. Ideally, I would like to get this in a few days ahead of the > alpha release. I'm not too worried either way, as I don't think that this will be something I use a lot (I may be wrong, though..) Having said that, I do feel that your points should be addressed. I haven't looked at either implementation, but I'd assume that the key benefit of a class-based implementation would be the ability to subclass to modify behaviour. If that isn't happening in practice, then I agree that a function is probably a better option. As a test case, how would I implement the case that has come up here recently, of supporting $obj.attr substitutions, in each of the two implementations? It would be interesting to see the relative simplicity. I'm assuming that the function-based case would require effectively a cut & paste reimplementation. [FX: Dives into source code...] The Template class is pretty trivial, but it doesn't seem to be designed for extension. Thus, the above test case would *still* need pretty much a rewrite. However, if the Template class was rewritten with overriding in mind, it probably could make things easier: 1. Split the match pattern into pieces, such that adding new syntax like $obj.attr is just a case of adding a new pattern to a list. 2. Make the __mod__ code more case-like, having separate actions based on group name. Then overriding code could just add actions for the groups it added, and call the base class for the rest. This isn't fully thought through, and may well not be worth it in practice, but it gives the sort of benefits I'd expect from a class-based approach. Paul. -- Home computers are being called upon to perform many new functions, including the consumption of homework formerly eaten by the dog -- Doug Larson From gvanrossum at gmail.com Sun Aug 29 19:37:30 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun Aug 29 19:37:46 2004 Subject: [Python-Dev] Re: Alternative Implementation for PEP 292: Simple String Substitutions In-Reply-To: References: <20040827233958.GA5560@panix.com> <000501c48d34$19ce9000$e841fea9@oemcomputer> Message-ID: > As a test case, how would I implement the case that has come up here > recently, of supporting $obj.attr substitutions, in each of the two > implementations? It would be interesting to see the relative > simplicity. I'm assuming that the function-based case would require > effectively a cut & paste reimplementation. > > [FX: Dives into source code...] > The Template class is pretty trivial, but it doesn't seem to be > designed for extension. Thus, the above test case would *still* need > pretty much a rewrite. However, if the Template class was rewritten > with overriding in mind, it probably could make things easier: Subclassing is only one possibility. With the templating version, you could also write another class that implements the same interface -- very Pythonic. If Raymond's function approach were chosen, you're stuck with what that function can and can't do. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond.hettinger at verizon.net Sun Aug 29 22:37:17 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun Aug 29 22:37:57 2004 Subject: [Python-Dev] os.urandom API Message-ID: <000701c48e07$f9cea040$e841fea9@oemcomputer> I would like to change the API for the new os.urandom(n) function to return a long integer instead of a string. The former better serves more use cases and fits better with existing modules. In favor of a long integer: 1) The call random.seed(os.random(100)) is a likely use case. If the intermediate value is a string, then random.seed() will hash it and only use 32 bits. If the intermediate value is a long integer, all bits are used. In the given example, the latter is clearly what the user expects (otherwise, they would only request 4 bytes). 2) Another likely use case is accessing all the tools in the random module with a subclass that overrides random() and getrandbits(). Both can be done easier and faster if os.random() returns long integers. If the starting point is a string, the code gets ugly and slow. 3) Most use cases for random values involve numeric manipulation. Simple tasks like finding a random integer in the range [0,100000) become unnecessarily more complicated when starting from a string. 4) The decimal module supports instantiation directly from long integers but not from binary strings. In favor of a string of bytes: 1) This form is handy for cyptoweenies to xor with other byte strings (perhaps for a one-time pad). Raymond From martin at v.loewis.de Sun Aug 29 23:26:20 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Aug 29 23:26:20 2004 Subject: [Python-Dev] os.urandom API In-Reply-To: <000701c48e07$f9cea040$e841fea9@oemcomputer> References: <000701c48e07$f9cea040$e841fea9@oemcomputer> Message-ID: <413249FC.10206@v.loewis.de> Raymond Hettinger wrote: > I would like to change the API for the new os.urandom(n) function to > return a long integer instead of a string. The former better serves > more use cases and fits better with existing modules. -1. Bytes is what the underlying system returns, and it is also conceptually the right thing. We are really talking about a stream of random bytes here (where u signals unlimitedness). > 1) The call random.seed(os.random(100)) is a likely use case. If the > intermediate value is a string, then random.seed() will hash it and only > use 32 bits. If the intermediate value is a long integer, all bits are > used. In the given example, the latter is clearly what the user expects > (otherwise, they would only request 4 bytes). Then add an os.randint if you think this is important. Given the easiness of using the struct module, I don't think it is important to provide this out of the box. > 2) Another likely use case is accessing all the tools in the random > module with a subclass that overrides random() and getrandbits(). Both > can be done easier and faster if os.random() returns long integers. If > the starting point is a string, the code gets ugly and slow. Don't try guessing use cases too much. I don't think either the original submitter, nor the original reviewer, had sequences of pseudo-random numbers as their use case. Instead, the typical application will be a one-time token for some crypto algorithm, in which case sequences of pseudo-randomness are evil. What kind of data structure these things will need is hard to guess, but "sequence of bytes" is a good bet. > 3) Most use cases for random values involve numeric manipulation. > Simple tasks like finding a random integer in the range [0,100000) > become unnecessarily more complicated when starting from a string. That is not true. Most use cases of random numbers involve bit manipulation. > 1) This form is handy for cyptoweenies to xor with other byte strings > (perhaps for a one-time pad). And indeed, cryptoweenies have contributed that code. He who writes the code choses the interface. Regards, Martin From tim.peters at gmail.com Sun Aug 29 23:55:57 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sun Aug 29 23:56:02 2004 Subject: [Python-Dev] os.urandom API In-Reply-To: <413249FC.10206@v.loewis.de> References: <000701c48e07$f9cea040$e841fea9@oemcomputer> <413249FC.10206@v.loewis.de> Message-ID: <1f7befae040829145555b219cb@mail.gmail.com> I agree with Martin that the patch submitter and reviewers really want strings to come back -- there are many use cases for that in cryptoweenie land, as simple as generating a random session key. Long ints might be cool too, but if so that deserves a distinct API. Something like "urandom(n) -- return a uniform random int from 0 through 256**n-1" would be a pretty bizarre spec. Note that it's easy (abeit obscure) to generate a long from a string s: long(binascii.hexlify(s), 16) It's harder to go in the other direction. First you do hex(n). Then you chop off the leading "0x". Then there may or may not be a trailing "L", and iff there is you have to lose that too. Then binascii.unhexlify() gets the string. Then you may have to go on to pad with one or more leading NUL bytes. Regardless of whether there's another API added, it might be good to change random.seed() under the covers to use (say) 4 urandom bytes for default initialization (when os.urandom is available). Initializing based on time.time() is weak, and especially on Windows (where there are typically only 18.2 distinct time.time() values per second). From barry at python.org Mon Aug 30 01:37:40 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 30 01:37:47 2004 Subject: [Python-Dev] __mangled in stdlib considered poor form In-Reply-To: References: <41302F87.1000805@interlink.com.au> <2md61b5wab.fsf@starship.python.net> Message-ID: <1093822659.18459.1.camel@geddy.wooz.org> On Sat, 2004-08-28 at 14:50, Guido van Rossum wrote: > There's one potential use case where I'd like to keep the private name > even if there's a mangled use of it nearby: when the private name > really is intended to be private, and the mangled use is a "friend" in > the C++ sense. Sometimes (though rarely) the stdlib has to indulge in > practices that aren't good example code, because it's the stdlib, and > it should implement functionality in the most bulletproof way > possible. Using private names for internal details is the right thing > sometimes. Please don't just mindlessly change all leading double underscores to singles. +1, when following Guido's guidelines above though. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040829/59eee9a3/attachment.pgp From greg at cosc.canterbury.ac.nz Mon Aug 30 03:05:39 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 30 03:05:47 2004 Subject: [Python-Dev] Re: adding a bytes sequence type to Python In-Reply-To: <20040829004221.GB1593@trit.org> Message-ID: <200408300105.i7U15dJb013777@cosc353.cosc.canterbury.ac.nz> Dima Dorfman : > It supports the "read" buffer API but not the "character" buffer API, > so the file has to be opened in binary mode for it to work: If we're serious about bytes not being characters, isn't this the *right* behaviour for a byte array? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From barry at python.org Mon Aug 30 05:29:09 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 30 05:29:11 2004 Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions In-Reply-To: <20040826135401.GA31295@alcyon.progiciels-bpi.ca> References: <1093232832.28573.16.camel@geddy.wooz.org> <20040823132043.GA4501@titan.progiciels-bpi.ca> <1093367647.18858.62.camel@geddy.wooz.org> <20040826135401.GA31295@alcyon.progiciels-bpi.ca> Message-ID: <1093836549.18459.85.camel@geddy.wooz.org> On Thu, 2004-08-26 at 09:54, Fran?ois Pinard wrote: > [Barry Warsaw] > > > > If my feeling is right, then the PEP should clearly explicit this > > > goal [of pushing `$' forward], it will make the PEP stronger. > > > I will neither confirm nor deny whether the PSU is bankrolling the PEP > > 292 initiative, nor the actual existence of any 527 organization > > claiming to be called the "PSU", nor whether if they did exist, they > > were or weren't acting in coordination with the campaign organizations > > of any 2004 US presidential nominee. > > Nice joking. Still, yet, and nevertheless, I think making the PEP > closer to the truth might make it stronger. It might also repair a bit > the feeling that the PEP process is sometimes mildly lacking (the @-PEP > having been ratehr publicised as an example of this, recently). Um, PEP 318 has nothing to do with this, and I think the PEP 292 process has been fairly accurate, disagreements about interface notwithstanding. Maybe $'s make sense for Python 3000, but that was never my goal, explicit or subversive, for PEP 292. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040829/984c2d9d/attachment-0001.pgp From barry at python.org Mon Aug 30 05:37:25 2004 From: barry at python.org (Barry Warsaw) Date: Mon Aug 30 05:37:29 2004 Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String Substitutions In-Reply-To: <002301c48bb5$17bfbe40$e841fea9@oemcomputer> References: <002301c48bb5$17bfbe40$e841fea9@oemcomputer> Message-ID: <1093837045.18457.94.camel@geddy.wooz.org> On Thu, 2004-08-26 at 17:38, Raymond Hettinger wrote: > My main issue with the current implementation is that we get no leverage > from using a class instead of a function. Though the API is fairly > simple either way, it is easier to learn and document functions than > classes. We gain nothing from instantiation -- the underlying data > remains immutable and no additional state is attached. The only new > behavior is the ability to apply the mod operator. Why not do it in one > step. Weren't you the one who gave the Cheetah example? What was interesting about that was that the instance's attributes formed the substitution namespace. That's a use case I instantly liked. So there you have state attached to an instance. Another case for that would be in i18n applications where you might want to attach information such as the gettext domain to the instance. You might also want to build up the namespace in several locations, and delay performing the substitution until the last possible moment. In all those cases you have state attached to an instance (and would immediately invent such an instance for those use cases if you didn't have one). > One negative effect of the class implementation is that it inherits from > unicode and always returns a unicode result even if all of the inputs > (mapping values and template) are regular strings. With a function > implementation, that can be avoided (returning unicode only if one of > the inputs is unicode). To me that's not a disadvantage. For i18n applications, unicode is the only reasonable thing for human readable text. str's are only useful for byte arrays . It's not a disadvantage for Jython or IronPython either. :) > Another minor advantage for a function is that it is easier to lookup in > the reference. If a reader sees the % operator being applied and looks > it up in the reference, it is going to steer them in the wrong > direction. The mod operator was chosen because that's what people are familiar with, but it would probably be okay to pick a different method name. I think Guido's suggested using __call__() -- which I want to think more about. > This is doubly true if the Template instantiation is remote > from the operator application. Which, in some use cases, it most definitely will be. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040829/6f22ce3d/attachment.pgp From jcarlson at uci.edu Mon Aug 30 05:48:49 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon Aug 30 05:49:14 2004 Subject: [Python-Dev] os.urandom API In-Reply-To: <1f7befae040829145555b219cb@mail.gmail.com> References: <000701c48e07$f9cea040$e841fea9@oemcomputer> <413249FC.10206@v.loewis.de> <1f7befae040829145555b219cb@mail.gmail.com> Message-ID: <4132A3A1.5040305@uci.edu> (Quick apology to Tim for accidentally sending this to him only, I recently switched email clients and am still getting used to it) > Note that it's easy (abeit obscure) to generate a long from a string s: > > long(binascii.hexlify(s), 16) or even: long(s.encode('hex'), 16) > It's harder to go in the other direction. First you do hex(n). Then Perhaps what is needed is a method to easily convert between large integers and strings. It seems as though a new struct conversion code is in order, something that works similarly to the way the 's' code works: #pack the integer bigint as a signed big-endian packed binary string, #null-filling as necessary, for 64 bytes of precision a = struct.pack('>64g', bigint) #unpack an unsigned little-endian packed binary string of 24 bytes to a #python long b = struct.unpack('<24G', a) With such a change, I think many of the string/integer translation complaints would disappear. - Josiah From python at rcn.com Mon Aug 30 07:48:46 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Aug 30 07:49:22 2004 Subject: [Python-Dev] Alternative Implementation for PEP 292: SimpleString Substitutions In-Reply-To: <1093837045.18457.94.camel@geddy.wooz.org> Message-ID: <003301c48e55$04942ac0$e841fea9@oemcomputer> [Raymond] > > One negative effect of the class implementation is that it inherits from > > unicode and always returns a unicode result even if all of the inputs > > (mapping values and template) are regular strings [Barry] > To me that's not a disadvantage. By not inheriting from unicode, the bug can be fixed while retaining a class implementation (see sandbox\curry292.py for an example). But, be clear, it *is* a bug. If all the inputs are strings, Unicode should not magically appear. See all the other string methods as an example. Someday, all will be Unicode, until then, some apps choose to remain Unicode free. Also, there is a build option to not even compile Unicode support -- it would be a bummer to have the $ templates fail as a result. Raymond P.S. Here's the doctest from the sandbox code. What is at issue is the result of the first test: >>> Template('the $xxx and')(xxx='10') 'the 10 and' >>> Template(u'the $xxx and')(xxx='10') u'the 10 and' >>> Template('the $xxx and')(xxx=u'10') u'the 10 and' >>> Template(u'the $xxx and')(xxx=u'10') u'the 10 and' From adurdin at gmail.com Mon Aug 30 08:11:34 2004 From: adurdin at gmail.com (Andrew Durdin) Date: Mon Aug 30 08:11:42 2004 Subject: [Python-Dev] Alternative placeholder delimiters for PEP 292 Message-ID: <59e9fd3a040829231141cd3fe4@mail.gmail.com> A Yet Simpler Proposal, modifying that of PEP 292 I propose that the Template module not use $ to set off placeholders; instead, placeholders are delimited by braces {}. The following rules for {}-placeholders apply: 1. {{ and }} are escapes; they are replaced with a single { or } respectively. 2. {identifier} names a substitution placeholder matching a mapping key of "identifier". By default, "identifier" must spell a Python identifier as defined in Identifiers and Keywords[1]. No other characters have special meaning. If the left-brace { is unmatched, appears at the end of the string, or is followed by any non-identifier character, a ValueError will be raised at interpolation time[2]. If a single, unmatched right-brace } occurs in the string, a ValueError will be raised at interpolation time. This avoids ambiguity: did the user want a single right-brace, or did they inadvertently omit the left-brace? This will also cause a probably erroneous "{foo}}" or "{{foo}" to raise a ValueError. Rationale There are several reasons for preferring the paired delimiters {} to a single prefixed $: 1. The placeholder name stands out more clearly from its surroundings, due to the presence of a closing delimiter, and also to the fact that the braces bear less resemblance to any alphabetic characters than the dollar sign: "Hello, {name}, how are you?" vs "Hello, $name, how are you?" 2. Only two characters have special meanings in the string, as opposed to three. Additionally, dollar signs are expected to be more often used in templated strings (e.g. for currency values) than braces: "The {item} costs ${price}." vs "The $item costs $$$price." 3. The placeholder syntax is consistent, and does not change even when valid identifier characters follow the placeholder but are not part of the placeholder: "Look at the {plural}s!" vs "Look at the ${plural}s!" 4. The template substitution could be changed in future to support dotted names without breaking existing code. The example below would break if the $-template were changed to allow dotted names: "Evaluate {obj}.__doc__" vs "Evaluate $obj.__doc__" There are two drawbacks to the proposal when compared with $-templates: 1. An extra character must be typed for each placeholder in the common case: "{name}, welcome to {site}!" vs "$name, welcome to $site!" 2. Templated strings containing braces become more complicated: "dict = {{'{key}': '{value}'}}" vs "dict = {'$key': '$value'}" The first is not a real issue; the extra closing braces needed for the placeholder when compared with the number of other characters in the templated string will usually be insignificant. Furthermore, the {}-placeholders require fewer characters to be typed in the less common case when valid identifier characters follow the placeholder but are not part of it. The need for braces in a templated string is not expected to occur frequently. Because of this, the second drawback is considered of minor importance. Reference Implementation If the general nature of feedback on this proposal is positive, or expressive of interest in an implementation, then a reference implementation will be created forthwith. References and Notes [1] Identifiers and Keywords http://www.python.org/doc/current/ref/identifiers.html [2] The requirement for interpolation-time error raising is the same as in PEP 292. Although not a part of this proposal, I suggest that it would be better if the error occured when the Template instance is created. It is worth noting that the PEP 292 reference implementation (in python/nondist/sandbox/string/string/template.py) does not fully conform to the PEP as regards raising errors for invalid template strings: >>> t = Template("This should cause an error: $*") >>> t % {} u'This should cause an error: $*' >>> t = Template("This also should cause an error: $") >>> t % {} u'This also should cause an error: $' From jlgijsbers at planet.nl Mon Aug 30 12:23:53 2004 From: jlgijsbers at planet.nl (Johannes Gijsbers) Date: Mon Aug 30 12:22:23 2004 Subject: [Python-Dev] Adding 'lexists()' to os.path In-Reply-To: <200408221858.i7MIwV502491@guido.python.org> References: <20040822110835.GA7452@mail.planet.nl> <200408221858.i7MIwV502491@guido.python.org> Message-ID: <20040830102353.GA14784@mail.planet.nl> > Looks like you already got the go-ahead. Just make sure with any API > changes/additions to os.path, that there are many individual *path > modules that need to be updated -- if a platform doesn't support > symlinks, lexists() should probably just be an alias for exists(). > > posixpath.py, ntpath.py, macpath.py, os2emxpath.py. Yes, cben's patch takes cares of that. I've checked it in just now. Johannes From anthony at interlink.com.au Mon Aug 30 15:22:10 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 30 15:40:47 2004 Subject: [Python-Dev] (my) revisions to PEP318 finally done. Message-ID: <41332A02.1040902@interlink.com.au> Here's the version of PEP 318 I just checked in. If you have any additions or changes, I prefer patches mailed to me in unified diff format. Comments like "You should cover point X, Y and Z", without the accompanying words, I won't be acting on (others with checkin privs are more than welcome to do so). Sorry about that, but I _cannot_ spend any more time on this document now - this whole decorators thing has consumed far more of my time than I can afford right now. -------------- next part -------------- PEP: 318 Title: Decorators for Functions and Methods Version: $Revision: 1.28 $ Last-Modified: $Date: 2004/08/30 13:16:56 $ Author: Kevin D. Smith, Jim Jewett, Skip Montanaro, Anthony Baxter Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 05-Jun-2003 Python-Version: 2.4 Post-History: 09-Jun-2003, 10-Jun-2003, 27-Feb-2004, 23-Mar-2004, 30-Aug-2004 WarningWarningWarning ===================== The final decision on the syntax for 2.4a3 is not yet made. This will be done before 2.4a3, and this document will be updated to match. Note also that this document does not attempt to cover the huge number of potential alternative syntaxes, nor is it an attempt to exhaustively list all the positives and negatives of each form. Abstract ======== The current method for transforming functions and methods (for instance, declaring them as a class or static method) is awkward and can lead to code that is difficult to understand. Ideally, these transformations should be made at the same point in the code where the declaration itself is made. This PEP introduces new syntax for transformations of a function or method declaration. Motivation ========== The current method of applying a transformation to a function or method places the actual translation after the function body. For large functions this separates a key component of the function's behavior from the definition of the rest of the function's external interface. For example:: def foo(self): perform method operation foo = classmethod(foo) This becomes less readable with longer methods. It also seems less than pythonic to name the function three times for what is conceptually a single declaration. A solution to this problem is to move the transformation of the method closer to the method's own declaration. While the new syntax is not yet final, the intent is to replace:: def foo(cls): pass foo = synchronized(lock)(foo) foo = classmethod(foo) with an alternative that places the decoration in the function's declaration:: @classmethod @synchronized(lock) def foo(cls): pass Modifying classes in this fashion is also possible, though the benefits are not as immediately apparent. Almost certainly, anything which could be done with class decorators could be done using metaclasses, but using metaclasses is sufficiently obscure that there is some attraction to having an easier way to make simple modifications to classes. For Python 2.4, only function/method decorators are being added. Why Is This So Hard? -------------------- Two decorators (``classmethod()`` and ``staticmethod()``) have been available in Python since version 2.2. It's been assumed since approximately that time that some syntactic support for them would eventually be added to the language. Given this assumption, one might wonder why it's been so difficult to arrive at a consensus. Discussions have raged off-and-on at times in both comp.lang.python and the python-dev mailing list about how best to implement function decorators. There is no one clear reason why this should be so, but a few problems seem to be most problematic. * Disagreement about where the "declaration of intent" belongs. Almost everyone agrees that decorating/transforming a function at the end of its definition is suboptimal. Beyond that there seems to be no clear consensus where to place this information. * Syntactic constraints. Python is a syntactically simple language with fairly strong constraints on what can and can't be done without "messing things up" (both visually and with regards to the language parser). There's no obvious way to structure this information so that people new to the concept will think, "Oh yeah, I know what you're doing." The best that seems possible is to keep new users from creating a wildly incorrect mental model of what the syntax means. * Overall unfamiliarity with the concept. For people who have a passing acquaintance with algebra (or even basic arithmetic) or have used at least one other programming language, much of Python is intuitive. Very few people will have had any experience with the decorator concept before encountering it in Python. There's just no strong preexisting meme that captures the concept. * Syntax discussions in general appear to cause more contention than almost anything else. Readers are pointed to the ternary operator discussions that were associated with PEP 308 for another example of this. Background ========== There is general agreement that syntactic support is desirable to the current state of affairs. Guido mentioned `syntactic support for decorators`_ in his DevDay keynote presentation at the `10th Python Conference`_, though `he later said`_ it was only one of several extensions he proposed there "semi-jokingly". `Michael Hudson raised the topic`_ on ``python-dev`` shortly after the conference, attributing the initial bracketed syntax to an earlier proposal on ``comp.lang.python`` by `Gareth McCaughan`_. .. _syntactic support for decorators: http://www.python.org/doc/essays/ppt/python10/py10keynote.pdf .. _10th python conference: http://www.python.org/workshops/2002-02/ .. _michael hudson raised the topic: http://mail.python.org/pipermail/python-dev/2002-February/020005.html .. _he later said: http://mail.python.org/pipermail/python-dev/2002-February/020017.html .. _gareth mccaughan: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=slrna40k88.2h9o.Gareth.McCaughan%40g.local Class decorations seem like an obvious next step because class definition and function definition are syntactically similar. The discussion continued on and off on python-dev from February 2002 through July 2004. Hundreds and hundreds of posts were made, with people proposing many possible syntax variations. Guido took a list of proposals to `EuroPython 2004`_, where a discussion took place. Subsequent to this, he decided that for 2.4a2 we'd have the `Java-style`_ @decorator syntax. Barry Warsaw named this the 'pie-decorator' syntax, in honor of the Pie-thon Parrot shootout which was announced about the same time as the decorator syntax, and because the @ looks a little like a pie. Guido `outlined his case`_ on Python-dev, including `this piece`_ on the various rejected forms. .. _EuroPython 2004: http://www.python.org/doc/essays/ppt/euro2004/euro2004.pdf .. _outlined his case: http://mail.python.org/pipermail/python-dev/2004-August/author.html .. _this piece: http://mail.python.org/pipermail/python-dev/2004-August/046672.html .. _Java-style: http://java.sun.com/j2se/1.5.0/docs/guide/language/annotations.html On the name 'Decorator' ======================= There's been a number of complaints about the choice of the name 'decorator' for this feature. The major one is that the name is not consistent with its use in the `GoF book`_. The name 'decorator' probably owes more to its use in the compiler area -- a syntax tree is walked and annotated. It's quite possible that a better name may turn up. .. _GoF book: http://patterndigest.com/patterns/Decorator.html Design Goals ============ The new syntax should * work for arbitrary wrappers, including user-defined callables and the existing builtins ``classmethod()`` and ``staticmethod()``. This requirement also means that a decorator syntax must support passing arguments to the wrapper constructor * work with multiple wrappers per definition * make it obvious what is happening; at the very least it should be obvious that new users can safely ignore it when writing their own code * be a syntax "that ... [is] easy to remember once explained" * not make future extensions more difficult * be easy to type; programs that use it are expected to use it very frequently * not make it more difficult to scan through code quickly. It should still be easy to search for all definitions, a particular definition, or the arguments that a function accepts * not needlessly complicate secondary support tools such as language-sensitive editors and other "`toy parser tools out there`_" * allow future compilers to optimize for decorators. With the hope of a JIT compiler for Python coming into existence at some point this tends to require the syntax for decorators to come before the function definition * move from the end of the function, where it's currently hidden, to the front where it is more `in your face`_ Andrew Kuchling has links to a bunch of the discussions about motivations and use cases `in his blog`_. Particularly notable is `Jim Huginin's list of use cases`_. .. _toy parser tools out there: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=mailman.1010809396.32158.python-list%40python.org .. _in your face: http://mail.python.org/pipermail/python-dev/2004-August/047112.html .. _in his blog: http://www.amk.ca/diary/archives/cat_python.html#003255 .. _Jim Huginin's list of use cases: http://mail.python.org/pipermail/python-dev/2004-April/044132.html Current Syntax ============== The current syntax for function decorators as implemented in Python 2.4a2 is:: @dec2 @dec1 def func(arg1, arg2, ...): pass This is equivalent to:: def func(arg1, arg2, ...): pass func = dec2(dec1(func)) without the intermediate assignment to the variable ``func``. The decorators are near the function declaration. The @ sign makes it clear that something new is going on here. The decorator statement is limited in what it can accept -- arbitrary expressions will not work. Guido preferred this because of a `gut feeling`_. .. _gut feeling: http://mail.python.org/pipermail/python-dev/2004-August/046711.html Syntax Alternatives =================== There have been `a large number`_ of different syntaxes proposed -- rather than attempting to work through these individual syntaxes, it's worthwhile to break the syntax discussion down into a number of areas. Attempting to discuss `each possible syntax`_ individually would be an act of madness, and produce a completely unwieldy PEP. .. _a large number: http://www.python.org/moin/PythonDecorators .. _each possible syntax: http://ucsu.colorado.edu/~bethard/py/decorators-output.py Decorator Location ------------------ The first syntax point is the location of the decorators. For the following examples, we use the @syntax used in 2.4a2. Decorators before the def statement are the first alternative, and the syntax used in 2.4a2:: @classmethod def foo(arg1,arg2): pass @accepts(int,int) @returns(float) def bar(low,high): pass There have been a number of objections raised to this location -- the primary one is that it's the first real Python case where a line of code has a result on a following line. The syntax that will be in 2.4a3 will also require one decorator per line (in a2, multiple decorators can be specified on the same line). People also complained that the syntax got unworldly quickly when multiple decorators were used. The point was made, though, that the chances of a large number of decorators being used on a single function were small and thus this was not a large worry. Some of the advantages of this form are that the decorators live outside the method body -- they are obviously executed at the time the function is defined. Another advantage is that being prefix to the function definition fit the idea of knowing about a change to the semantics of the code before the code itself, thus knowing how to interpret the code's semantics properly without having to go back and change your initial perceptions if the syntax did not come before the function definition. Guido decided `he preferred`_ having the decorators on the line before the 'def', because it was felt that a long argument list would mean that the decorators would be 'hidden' .. _he preferred: http://mail.python.org/pipermail/python-dev/2004-March/043756.html The second form is the decorators between the def and the function name, or the function name and the argument list:: def @classmethod foo(arg1,arg2): pass def @accepts(int,int),@returns(float) bar(low,high): pass def foo @classmethod (arg1,arg2): pass def bar @accepts(int,int),@returns(float) (low,high): pass There are a couple of objections to this form. The first is that it breaks easily 'greppability' of the source -- you can no longer search for 'def foo(' and find the definition of the function. The second, more serious, objection is that in the case of multiple decorators, the syntax would be extremely unwieldy. The next form, which has had a number of strong proponents, is to have the decorators between the argument list and the trailing ``:`` in the 'def' line:: def foo(arg1,arg2) @classmethod: pass def bar(low,high) @accepts(int,int),@returns(float): pass Guido `summarized the arguments`_ against this form (many of which also apply to the previous form) as: - it hides crucial information (e.g. that it is a static method) after the signature, where it is easily missed - it's easy to miss the transition between a long argument list and a long decorator list - it's cumbersome to cut and paste a decorator list for reuse, because it starts and ends in the middle of a line .. _summarized the arguments: http://mail.python.org/pipermail/python-dev/2004-August/047112.html The next form is that the decorator syntax go inside the method body at the start, in the same place that docstrings currently live: def foo(arg1,arg2): @classmethod pass def bar(low,high): @accepts(int,int) @returns(float) pass The primary objection to this form is that it requires "peeking inside" the method body to determine the decorators. In addition, even though the code is inside the method body, it is not executed when the method is run. Guido felt that docstrings were not a good counter-example, and that it was quite possible that a 'docstring' decorator could help move the docstring to outside the function body. The final form is a new block that encloses the method's code. For this example, we'll use a 'decorate' keyword, as it makes no sense with the @syntax. :: decorate: classmethod def foo(arg1,arg2): pass decorate: accepts(int,int) returns(float) def bar(low,high): pass This form would result in inconsistent indentation for decorated and undecorated methods. In addition, a decorated method's body would start three indent levels in. Syntax forms ------------ * ``@decorator``:: @classmethod def foo(arg1,arg2): pass @accepts(int,int) @returns(float) def bar(low,high): pass The major objections against this syntax are that the @ symbol is not currently used in Python (and is used in both IPython and Leo), and that the @ symbol is not meaningful. Another objection is that this "wastes" a currently unused character (from a limited set) on something that is not perceived as a major use. * ``|decorator``:: |classmethod def foo(arg1,arg2): pass |accepts(int,int) |returns(float) def bar(low,high): pass This is a variant on the @decorator syntax -- it has the advantage that it does not break IPython and Leo. Its major disadvantage compared to the @syntax is that the | symbol looks like both a capital I and a lowercase l. * list syntax:: [classmethod] def foo(arg1,arg2): pass [accepts(int,int), returns(float)] def bar(low,high): pass The major objection to the list syntax is that it's currently meaningful (when used in the form before the method). It's also lacking any indication that the expression is a decorator. * list syntax using other brackets (``<...>``, ``[[...]]``, ...):: def foo(arg1,arg2): pass def bar(low,high): pass None of these alternatives gained much traction. The alternatives which involve square brackets only serve to make it obvious that the decorator construct is not a list. They do nothing to make parsing any easier. The '<...>' alternative presents parsing problems because '<' and '>' already parse as un-paired. They present a further parsing ambiguity because a right angle bracket might be a greater than symbol instead of a closer for the decorators. * ``decorate()`` The ``decorate()`` proposal was that no new syntax be implemented -- instead a magic function that used introspection to manipulate the following function. Both Jp Calderone and Philip Eby produced implementations of functions that did this. Guido was pretty firmly against this -- with no new syntax, the magicness of a function like this is extremely high: Using functions with "action-at-a-distance" through sys.settraceback may be okay for an obscure feature that can't be had any other way yet doesn't merit changes to the language, but that's not the situation for decorators. The widely held view here is that decorators need to be added as a syntactic feature to avoid the problems with the postfix notation used in 2.2 and 2.3. Decorators are slated to be an important new language feature and their design needs to be forward-looking, not constrained by what can be implemented in 2.3. * new keyword (and block) This idea was the consensus alternate from comp.lang.python. Robert Brewer wrote up a detailed `J2 proposal`_ document outlining the arguments in favor of this. The issues with this form are that it requires a new keyword (and therefore a ``from __future__ import decorators`` statement), it seems like there is no obvious choice for the keyword (``using`` is the choice of the proposal and implementation), and that the form produces something that looks like a normal code block, but isn't. Attempts to use statements in this block will cause a syntax error. It's also going to potentially confuse users who don't expect a block to contain bare expressions. .. _J2 proposal: http://www.aminus.org/rbre/python/pydec.html There are plenty of other variants and proposals on `the wiki page`_. .. _the wiki page: http://www.python.org/moin/PythonDecorators Why @? ------ There is some history in Java using @ initially as a marker in `Javadoc comments`_ and later in Java 1.5 for `annotations`_, which are similar to Python decorators. The fact that @ was previously unused as a token in Python also means it's clear there is no possibility of such code being parsed by an earlier version of Python, leading to possibly subtle semantic bugs. It also means that ambiguity of what is a decorator and what isn't is removed. of That said, @ is still a fairly arbitrary choice. Some have suggested using | instead. For syntax options which use a list-like syntax (no matter where it appears) to specify the decorators a few alternatives were proposed: ``[|...|]``, ``*[...]*``, and ``<...>``. .. _Javadoc comments: http://java.sun.com/j2se/javadoc/writingdoccomments/ .. _annotations: http://java.sun.com/j2se/1.5.0/docs/guide/language/annotations.html Current Implementation, History =============================== Guido asked for a volunteer to implement his preferred syntax, and Mark Russell stepped up and posted a `patch`_ to SF. The syntax accepted for 2.4a2 is:: @dec2 @dec1 def func(arg1, arg2, ...): pass This is equivalent to:: def func(arg1, arg2, ...): pass func = dec2(dec1(func)) though without the intermediate creation of a variable named ``func``. A `previous patch`_ from Michael Hudson which implements the list-after-def syntax is also still kicking around. .. _patch: http://www.python.org/sf/979728 .. _previous patch: http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff After 2.4a2 was released, in response to community reaction, Guido stated that he'd re-examine a community proposal, if the community could come up with a community consensus, a decent proposal, and an implementation. After an amazing number of posts, collecting a vast number of alternatives in the `Python wiki`_, the proposed J2 syntax (the new keyword ``using`` in a block before the def). Robert Brewer wrote a `detailed proposal`_ for this form, and Michael Sparks produced `a patch`_. As at time of writing, we're waiting for Guido's decision. .. _Python wiki: http://www.python.org/moin/PythonDecorators .. _detailed proposal: http://www.aminus.org/rbre/python/pydec.html .. _a patch: http://www.python.org/sf/1013835 Examples ======== Much of the discussion on ``comp.lang.python`` and the ``python-dev`` mailing list focuses on the use of decorators as a cleaner way to use the ``staticmethod()`` and ``classmethod()`` builtins. This capability is much more powerful than that. This section presents some examples of use. 1. Define a function to be executed at exit. Note that the function isn't actually "wrapped" in the usual sense. :: def onexit(f): import atexit atexit.register(f) return f @onexit def func(): ... Note that this example is probably not suitable for real usage, but is for example purposes only. 2. Define a class with a singleton instance. Note that once the class disappears enterprising programmers would have to be more creative to create more instances. (From Shane Hathaway on ``python-dev``.) :: def singleton(cls): instances = {} def getinstance(): if cls not in instances: instances[cls] = cls() return instances[cls] return getinstance @singleton class MyClass: ... 3. Add attributes to a function. (Based on an example posted by Anders Munch on ``python-dev``.) :: def attrs(**kwds): def decorate(f): for k in kwds: setattr(f, k, kwds[k]) return f return decorate @attrs(versionadded="2.2", author="Guido van Rossum") def mymethod(f): ... 4. Enforce function argument and return types. Note that this copies the func_name attribute from the old to the new function. func_name was made writable in Python 2.4a3:: def accepts(*types): def check_accepts(f): assert len(types) == f.func_code.co_argcount def new_f(*args, **kwds): for (a, t) in zip(args, types): assert isinstance(a, t), \ "arg %r does not match %s" % (a,t) return f(*args, **kwds) new_f.func_name = f.func_name return new_f return check_accepts def returns(rtype): def check_returns(f): def new_f(*args, **kwds): result = f(*args, **kwds) assert isinstance(result, rtype), \ "return value %r does not match %s" % (result,rtype) return result new_f.func_name = f.func_name return new_f return check_returns @accepts(int, (int,float)) @returns((int,float)) def func(arg1, arg2): return arg1 * arg2 5. Declare that a class implements a particular (set of) interface(s). This is from a posting by Bob Ippolito on ``python-dev`` based on experience with `PyProtocols`_. :: def provides(*interfaces): """ An actual, working, implementation of provides for the current implementation of PyProtocols. Not particularly important for the PEP text. """ def provides(typ): declareImplementation(typ, instancesProvide=interfaces) return typ return provides class IBar(Interface): """Declare something about IBar here""" @provides(IBar) class Foo(object): """Implement something here...""" .. _PyProtocols: http://peak.telecommunity.com/PyProtocols.html Of course, all these examples are possible today, though without syntactic support. Open Issues =========== 1. It's not yet certain that class decorators will be incorporated into the language at this point. Guido expressed skepticism about the concept, but various people have made some `strong arguments`_ (search for ``PEP 318 -- posting draft``) on their behalf in ``python-dev``. .. _strong arguments: http://mail.python.org/pipermail/python-dev/2004-March/thread.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From anthony at interlink.com.au Mon Aug 30 16:22:35 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 30 16:23:02 2004 Subject: [Python-Dev] (my) revisions to PEP318 finally done. In-Reply-To: <41332A02.1040902@interlink.com.au> References: <41332A02.1040902@interlink.com.au> Message-ID: <4133382B.7070308@interlink.com.au> Anthony Baxter wrote: > > Here's the version of PEP 318 I just checked in. Forgot to note - as the block at the start says, there will be changes made once Guido makes a decision on the J2-vs-pie thing. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From goodger at python.org Mon Aug 30 17:04:38 2004 From: goodger at python.org (David Goodger) Date: Mon Aug 30 17:04:56 2004 Subject: [Python-Dev] PEP 318 & Community Consensus In-Reply-To: <41332A02.1040902@interlink.com.au> References: <41332A02.1040902@interlink.com.au> Message-ID: I just made some modifications: Made J2 proposal more prominent, and removed some perceived bias. Pushed to web: . See the "Community Consensus" section. -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040830/90345397/signature.pgp From anthony at interlink.com.au Mon Aug 30 17:13:21 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 30 17:13:45 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.28, 1.29 In-Reply-To: References: Message-ID: <41334411.1040906@interlink.com.au> goodger@users.sourceforge.net wrote: > Update of /cvsroot/python/python/nondist/peps > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16653 > > Modified Files: > pep-0318.txt > Log Message: > Made J2 proposal more prominent, and removed some perceived bias. Minor cleanup. It's quite possible there was some bias in there. I considered going through it to make sure it was scrupulously even and balanced, but I was a bit too burned out on the topic to be sure of it. Figured that any grotesque slanders would get feedback Thanks for the additions! Anthony From fredrik at pythonware.com Mon Aug 30 17:35:55 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 30 17:34:09 2004 Subject: [Python-Dev] Re: PEP 318 & Community Consensus References: <41332A02.1040902@interlink.com.au> Message-ID: David Goodger wrote: > It may confuse users who don't expect a block to contain bare > expressions. blocks can contain bare expressions today, and such expressions are evaluated, but the results are discarded. plain function calls belong to this category, for example. > On the other hand, the ``@`` syntax is bound to cause at least as much > confusion. and that qualifies as an unbiased observation in exactly in what way? From goodger at python.org Mon Aug 30 18:05:42 2004 From: goodger at python.org (David Goodger) Date: Mon Aug 30 18:05:46 2004 Subject: [Python-Dev] Re: PEP 318 & Community Consensus In-Reply-To: References: <41332A02.1040902@interlink.com.au> Message-ID: <41335056.8090800@python.org> [Fredrik Lundh] > David Goodger wrote: >> It may confuse users who don't expect a block to contain bare >> expressions. > > blocks can contain bare expressions today, and such > expressions are evaluated, but the results are discarded. > plain function calls belong to this category, for example. Corrected in CVS & on the web. >> On the other hand, the ``@`` syntax is bound to cause at least as >> much confusion. > > and that qualifies as an unbiased observation in exactly in what > way? Perhaps in trying to balance the text I overshot. Removed. -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20040830/97d3134a/signature.pgp From ndbecker2 at verizon.net Mon Aug 30 19:08:33 2004 From: ndbecker2 at verizon.net (Neal D. Becker) Date: Mon Aug 30 19:08:38 2004 Subject: [Python-Dev] python vs. valgrind Message-ID: I just tried running valgrind-2.1.2 on python-2.3.3. [nbecker@rpppc1 ~]$ valgrind --tool=memcheck /usr/bin/python ./Hello.py ==12284== Memcheck, a memory error detector for x86-linux. ==12284== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al. ==12284== Using valgrind-2.1.2, a program supervision framework for x86-linux. ==12284== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al. ==12284== For more details, rerun with: -v ==12284== ==12284== Conditional jump or move depends on uninitialised value(s) ==12284== at 0x807AFA4: PyObject_Free (in /usr/bin/python) ==12284== by 0x8074A4F: (within /usr/bin/python) ==12284== by 0x8082AF0: PyString_InternInPlace (in /usr/bin/python) ==12284== by 0x8082BA2: PyString_InternFromString (in /usr/bin/python) ==12284== ==12284== Use of uninitialised value of size 4 ==12284== at 0x807AFB2: PyObject_Free (in /usr/bin/python) ==12284== by 0x8074A4F: (within /usr/bin/python) ==12284== by 0x8082AF0: PyString_InternInPlace (in /usr/bin/python) ==12284== by 0x8082BA2: PyString_InternFromString (in /usr/bin/python) [many, many more complaints...] Does this indicate a problem? From fredrik at pythonware.com Mon Aug 30 19:25:58 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 30 19:24:12 2004 Subject: [Python-Dev] Re: python vs. valgrind References: Message-ID: Neal D. Becker wrote: >I just tried running valgrind-2.1.2 on python-2.3.3. > [many, many more complaints...] compare and contrast: $ valgrind python2.1 Python 2.1.3 ... >>> ==12393== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 17 from 2) ==12393== malloc/free: in use at exit: 78911 bytes in 220 blocks. ==12393== malloc/free: 6367 allocs, 6147 frees, 764773 bytes allocated. $ valgrind python2.3 Python 2.3.3 ==12397== ERROR SUMMARY: 893 errors from 71 contexts (suppressed: 19 from 2) ==12397== malloc/free: in use at exit: 704466 bytes in 227 blocks. ==12397== malloc/free: 2033 allocs, 1806 frees, 1540716 bytes allocated. (using a slightly older version of valgrind 2.x) most (all?) errors occur in PyObject_Free and PyObject_Realloc, which indicates that the problem is either in pymalloc or valgrind itself (and in my experience, while vladimir and tim almost never mess up, valgrind is *never* wrong...) http://www.python.org/doc/2.3.3/whatsnew/section-pymalloc.html From tim.peters at gmail.com Mon Aug 30 19:32:51 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 30 19:32:54 2004 Subject: [Python-Dev] python vs. valgrind In-Reply-To: References: Message-ID: <1f7befae0408301032773b7ba9@mail.gmail.com> [Neal D. Becker] > I just tried running valgrind-2.1.2 on python-2.3.3. See Misc/README.valgrind in 2.4a2, or from current CVS. From fredrik at pythonware.com Mon Aug 30 19:36:27 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Aug 30 19:34:42 2004 Subject: [Python-Dev] Re: python vs. valgrind References: Message-ID: > most (all?) errors occur in PyObject_Free and PyObject_Realloc, which indicates > that the problem is either in pymalloc or valgrind itself (and in my experience, while > vladimir and tim almost never mess up, valgrind is *never* wrong...) on the other hand: http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Objects/obmalloc.c?r1=2.51&r2=2.52 (but I must be tired; can anyone explain how "making the function last" prevents the code from looking at uninitialized data?) From tim.peters at gmail.com Mon Aug 30 19:45:33 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 30 19:45:36 2004 Subject: [Python-Dev] Re: python vs. valgrind In-Reply-To: References: Message-ID: <1f7befae04083010459fb6b7e@mail.gmail.com> [Fredrik Lundh] > (but I must be tired; can anyone explain how "making the function last" > prevents the code from looking at uninitialized data?) It doesn't. obmalloc can read uninitialized memory, and by design (see REAMDE.valgrind). Putting the function last was, as Neal said in his comment there. to prevent gcc from inlining it. That in turn is required so that the Valgrind suppression file can name the function. From amk at amk.ca Mon Aug 30 21:23:55 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 30 21:24:27 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040809211517.GB11199@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> Message-ID: <20040830192355.GA11836@rogue.amk.ca> The following list of suggestions has been sitting around since the beginning of August? If not, I'll proceed with the actions described below. --amk On Mon, Aug 09, 2004 at 05:15:17PM -0400, A.M. Kuchling wrote: > Anyway, here's a revised action list: > > * Remove TERMIOS, mpz, statcache, xreadlines, rotor. > (Unchanged from my earlier proposal.) > > These modules have raised DeprecationWarning for a while, but only > rotor is listed in PEP 4. I'll add them to the PEP in any event; > is not being listed sufficient grounds for delaying their removal to > 2.5? > > (I would say we don't need to wait until 2.5; they've been raising > DeprecationWarnings for a long time, so users should be well aware > the modules' days are numbered.) > > * Leave rfc822, mimetools alone; the stdlib will have to avoid their use > first, and I'm not going to do that right now. > > * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the > stdlib uses them. From amk at amk.ca Mon Aug 30 21:30:30 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 30 21:31:01 2004 Subject: [Python-Dev] Volunteer for next bug day? Message-ID: <20040830193030.GB11836@rogue.amk.ca> Currently 2.4a3 is scheduled for Thursday Sept. 2nd. It's been an age-old tradition for three months now to have a bug day the weekend after a release. I won't be organizing this one because I can't guarantee being around all day on Saturday. Anyone want to volunteer to run it? (Tasks: update the Wiki page; send out announcements to python-dev, c.l.py.a.; be on IRC during the bug day to answer questions and help people.) --amk From tim.peters at gmail.com Mon Aug 30 21:45:17 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 30 21:45:55 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib random.py, 1.62, 1.63 In-Reply-To: References: Message-ID: <1f7befae04083012456c67acc@mail.gmail.com> [rhettinger@users.sourceforge.net] > Modified Files: > random.py > Log Message: > Teach the random module about os.urandom(). > ... > * Provide an alternate generator based on it. ... > + _tofloat = 2.0 ** (-7*8) # converts 7 byte integers to floats ... > +class HardwareRandom(Random): ... > + def random(self): ... > + return long(_hexlify(_urandom(7)), 16) * _tofloat Feeding in more bits than actually fit in a float leads to bias due to rounding. Here: """ import random import math import sys def main(n, useHR): from math import ldexp if useHR: get = random.HardwareRandom().random else: get = random.random counts = [0, 0] for i in xrange(n): x = long(ldexp(get(), 53)) & 1 counts[x] += 1 print counts expected = n / 2.0 chisq = (counts[0] - expected)**2 / expected + \ (counts[1] - expected)**2 / expected print "chi square statistic, 1 df, =", chisq n, useNR = map(int, sys.argv[1:]) main(n, useNR) """ Running with the Mersenne random gives comfortable chi-squared values for the distribution of bit 2**-53: C:\Code\python\PCbuild>python temp.py 100000 0 [50082, 49918] chi square statistic, 1 df, = 0.26896 C:\Code\python\PCbuild>python temp.py 100000 0 [49913, 50087] chi square statistic, 1 df, = 0.30276 C:\Code\python\PCbuild>python temp.py 100000 0 [50254, 49746] chi square statistic, 1 df, = 2.58064 Running with HardwareRandom instead gives astronomically unlikely values: C:\Code\python\PCbuild>python temp.py 100000 1 [52994, 47006] chi square statistic, 1 df, = 358.56144 C:\Code\python\PCbuild>python temp.py 100000 1 [53097, 46903] chi square statistic, 1 df, = 383.65636 C:\Code\python\PCbuild>python temp.py 100000 1 [53118, 46882] chi square statistic, 1 df, = 388.87696 One way to repair that is to replace the computation with return _ldexp(long(_hexlify(_urandom(7)), 16) >> 3, -BPF) where _ldexp is math.ldexp (and BPF is already a module constant). Of course that would also be biased on a box where C double had fewer than BPF (53) bits of precision (but the Twister implementation would show the same bias then). From gvanrossum at gmail.com Mon Aug 30 21:57:37 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon Aug 30 21:57:48 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040830192355.GA11836@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> Message-ID: > > * Remove TERMIOS, mpz, statcache, xreadlines, rotor. > > (Unchanged from my earlier proposal.) > > > > These modules have raised DeprecationWarning for a while, but only > > rotor is listed in PEP 4. I'll add them to the PEP in any event; > > is not being listed sufficient grounds for delaying their removal to > > 2.5? > > > > (I would say we don't need to wait until 2.5; they've been raising > > DeprecationWarnings for a long time, so users should be well aware > > the modules' days are numbered.) > > > > * Leave rfc822, mimetools alone; the stdlib will have to avoid their use > > first, and I'm not going to do that right now. > > > > * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the > > stdlib uses them. All +1 from me. (I still haven't started using the new email package, so I'm reluctant to see my beloved rfc822.py disappearing in the future; but I trust its fate is sealed, and I'm happy it gets an extension. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) Ask me about gmail. From nas at arctrix.com Mon Aug 30 22:18:16 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Mon Aug 30 22:18:22 2004 Subject: [Python-Dev] unicode and __str__ Message-ID: <20040830201816.GA32565@mems-exchange.org> With Python 2.4: >>> u = u'\N{WHITE SMILING FACE}' >>> class A: ... def __str__(self): ... return u ... >>> class B: ... def __unicode__(self): ... return u ... >>> u'%s' % A() u'\u263a' >>> u'%s' % B() u'\u263a' With Python 2.3: >>> u'%s' % A() Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode character u'\u263a' in position 0: ordinal not in range(128) >>> u'%s' % B() u'<__main__.B instance at 0x401f910c>' The only thing I found in the NEWS file that seemed relevant is this note: u'%s' % obj will now try obj.__unicode__() first and fallback to obj.__str__() if no __unicode__ method can be found. I don't think that describes the behavior difference. Allowing __str__ return unicode strings seems like a pretty noteworthy change (assuming that's what actually happened). Also, I'm a little unclear on the purpose of the __unicode__ method. If you can return unicode from __str__ then why would I want to provide a __unicode__ method? Perhaps it is meant for objects that can either return a unicode or a string representation depending on what the caller prefers. I have a hard time imagining a use for that. Neil From tim.peters at gmail.com Mon Aug 30 22:41:10 2004 From: tim.peters at gmail.com (Tim Peters) Date: Mon Aug 30 22:41:13 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <20040830201816.GA32565@mems-exchange.org> References: <20040830201816.GA32565@mems-exchange.org> Message-ID: <1f7befae040830134170e54697@mail.gmail.com> [Neil Schemenauer] > ... > The only thing I found in the NEWS file that seemed relevant is > this note: > > u'%s' % obj will now try obj.__unicode__() first and fallback to > obj.__str__() if no __unicode__ method can be found. > > I don't think that describes the behavior difference. Allowing > __str__ return unicode strings seems like a pretty noteworthy > change (assuming that's what actually happened). It's confusing. A __str__ method or tp_str type slot can return unicode, but what happens after that depends on the caller. PyObject_Str() and PyObject_Repr() try to encode it as an 8-bit string then. But unicode.__mod__ says "oh, cool -- I'm done". > Also, I'm a little unclear on the purpose of the __unicode__ method. > If you can return unicode from __str__ then why would I want to > provide a __unicode__ method? Is the purpose clearer if you purge your mind of the belief that str() (as opposed to __str__!) can return unicode? Here w/ current CVS: >>> class A: ... def __str__(self): return u'a' >>> print A() a >>> type(str(A())) >>> >>> class A: ... def __str__(self): return u'\u1234' >>> print A() Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode character u'\u1234' in position 0: ordinal not in range(128) >>> >>> '%s' % A() Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode character u'\u1234' in position 0: ordinal not in range(128) >>> u'%s' % A() u'\u1234' >>> So unicode.__mod__ is what's special here, But not sure that helps . From martin at v.loewis.de Mon Aug 30 22:57:36 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 30 22:57:35 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <20040830201816.GA32565@mems-exchange.org> References: <20040830201816.GA32565@mems-exchange.org> Message-ID: <413394C0.3090005@v.loewis.de> Neil Schemenauer wrote: > Also, I'm a little unclear on the purpose of the __unicode__ method. > If you can return unicode from __str__ then why would I want to > provide a __unicode__ method? Perhaps it is meant for objects that > can either return a unicode or a string representation depending on > what the caller prefers. I have a hard time imagining a use for > that. It's what unicode() returns: >>> class A: ... def __str__(self):return "1" ... def __unicode__(self): return u"2" ... >>> a=A() >>> str(a) '1' >>> unicode(a) u'2' str() is guaranteed to return a byte string; unicode() is guaranteed to return a unicode string. Regards, Martin From nas at arctrix.com Mon Aug 30 23:23:18 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Mon Aug 30 23:23:20 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <413394C0.3090005@v.loewis.de> References: <20040830201816.GA32565@mems-exchange.org> <413394C0.3090005@v.loewis.de> Message-ID: <20040830212317.GA419@mems-exchange.org> On Mon, Aug 30, 2004 at 10:57:36PM +0200, "Martin v. L?wis" wrote: > It's what unicode() returns: But unicode() will also return __str__, eg. >>> class A: ... def __str__(self): ... return u'\u1234' ... >>> unicode(A()) u'\u1234' Why would I want to use __unicode__? Shouldn't we be heading to a world where __str__ always returns unicode objects? Now, say your class stores unicode character data. You could have __unicode__ return a unicode object and have __str__ encode it somehow and return a str. However, that seems like a horrible design to me. The class can't know what kind of encoding the caller expects. Neil From martin at v.loewis.de Mon Aug 30 23:35:17 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Aug 30 23:35:41 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <20040830212317.GA419@mems-exchange.org> References: <20040830201816.GA32565@mems-exchange.org> <413394C0.3090005@v.loewis.de> <20040830212317.GA419@mems-exchange.org> Message-ID: <41339D95.7050507@v.loewis.de> Neil Schemenauer wrote: > But unicode() will also return __str__, eg. > > >>> class A: > ... def __str__(self): > ... return u'\u1234' > ... > >>> unicode(A()) > u'\u1234' > > Why would I want to use __unicode__? This class is incorrect: it does not support str(). > Shouldn't we be heading to a > world where __str__ always returns unicode objects? No. In some cases, str() needs to compromise, where unicode() doesn't. > Now, say your class stores unicode character data. You could have > __unicode__ return a unicode object and have __str__ encode it > somehow and return a str. However, that seems like a horrible > design to me. Perhaps. What are you proposing to do about this? Ban, from the face of the earth, what seems like a horrible design to you? Regards, Martin From skip at pobox.com Mon Aug 30 23:36:18 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 30 23:36:36 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <20040830212317.GA419@mems-exchange.org> References: <20040830201816.GA32565@mems-exchange.org> <413394C0.3090005@v.loewis.de> <20040830212317.GA419@mems-exchange.org> Message-ID: <16691.40402.872902.297882@montanaro.dyndns.org> Neil> Shouldn't we be heading to a world where __str__ always returns Neil> unicode objects? Which reminds me... Didn't someone (MAL?) add an experimental configure flag or something similar to allow building Python with PyString* == PyUnicode* (or something like that)? I think the idea was to see what needs to be fixed before Python can become 100% Unicode. Last I recall it was lots. Skip From nas at arctrix.com Tue Aug 31 00:38:52 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Tue Aug 31 00:38:55 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <41339D95.7050507@v.loewis.de> References: <20040830201816.GA32565@mems-exchange.org> <413394C0.3090005@v.loewis.de> <20040830212317.GA419@mems-exchange.org> <41339D95.7050507@v.loewis.de> Message-ID: <20040830223852.GA662@mems-exchange.org> On Mon, Aug 30, 2004 at 11:35:17PM +0200, "Martin v. L?wis" wrote: > Neil Schemenauer wrote: > >But unicode() will also return __str__, eg. > > > > >>> class A: > > ... def __str__(self): > > ... return u'\u1234' > > ... > > >>> unicode(A()) > > u'\u1234' > > > >Why would I want to use __unicode__? > > This class is incorrect: it does not support str(). Forgive me if I'm being obtuse, but I'm trying to understand the overall Python unicode design. This works: >>> sys.getdefaultencoding() 'utf-8' >>> str(A()) '\xe1\x88\xb4' Can you be more specific about what is incorrect with the above class? > >Shouldn't we be heading to a > >world where __str__ always returns unicode objects? > > No. In some cases, str() needs to compromise, where unicode() > doesn't. Sorry, I don't understand that statement. Are you saying that we will eventually get rid of __str__ and only have __unicode__? [on having __str__ assume some character encoding] > Perhaps. What are you proposing to do about this? Ban, from the face > of the earth, what seems like a horrible design to you? If only we could. :-) Seriously though, I'm trying to understand the point of __unicode__. To me it seems to make the transition to unicode string needlessly more complicated. Neil From mwh at python.net Tue Aug 31 00:43:20 2004 From: mwh at python.net (Michael Hudson) Date: Tue Aug 31 00:43:23 2004 Subject: [Python-Dev] Volunteer for next bug day? In-Reply-To: <20040830193030.GB11836@rogue.amk.ca> (A. M. Kuchling's message of "Mon, 30 Aug 2004 15:30:30 -0400") References: <20040830193030.GB11836@rogue.amk.ca> Message-ID: <2mwtzg4587.fsf@starship.python.net> "A.M. Kuchling" writes: > Currently 2.4a3 is scheduled for Thursday Sept. 2nd. It's been an > age-old tradition for three months now to have a bug day the weekend > after a release. I won't be organizing this one because I can't > guarantee being around all day on Saturday. Anyone want to volunteer > to run it? (Tasks: update the Wiki page; send out announcements to > python-dev, c.l.py.a.; be on IRC during the bug day to answer > questions and help people.) Um, er, maybe. I need to think about my diary :-) And no offense, but if I'm running it, I'll run it my own way... Cheers, mwh -- "Well, the old ones go Mmmmmbbbbzzzzttteeeeeep as they start up and the new ones go whupwhupwhupwhooopwhooooopwhooooooommmmmmmmmm." -- Graham Reed explains subway engines on asr From scav at blueyonder.co.uk Tue Aug 31 00:59:39 2004 From: scav at blueyonder.co.uk (Peter Harris) Date: Tue Aug 31 00:44:23 2004 Subject: [Python-Dev] PEP 309 updated slightly In-Reply-To: <20040830192447.4AB841E400A@bag.python.org> References: <20040830192447.4AB841E400A@bag.python.org> Message-ID: <4133B15B.702@blueyonder.co.uk> I have updated PEP309 a little, mostly to add the SF patch numbers for the implementation (still waiting for someone to check it in), and to note that the '@' character has been claimed for decorators, although nobody really wanted it for PEP309 anyway. I hope there will still be time to squeeze it in before 2.4 beta 1, and if there is any reason why not (apart from there has to be a cut-off point somewhere, which I accept), please someone let me know what work is still needed to make it ready to go in. Thanks, Peter Harris scav at blueyonder co (uk) From martin at v.loewis.de Tue Aug 31 06:56:26 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 31 06:56:35 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <16691.40402.872902.297882@montanaro.dyndns.org> References: <20040830201816.GA32565@mems-exchange.org> <413394C0.3090005@v.loewis.de> <20040830212317.GA419@mems-exchange.org> <16691.40402.872902.297882@montanaro.dyndns.org> Message-ID: <413404FA.2080102@v.loewis.de> Skip Montanaro wrote: > Neil> Shouldn't we be heading to a world where __str__ always returns > Neil> unicode objects? > > Which reminds me... Didn't someone (MAL?) add an experimental configure > flag or something similar to allow building Python with PyString* == > PyUnicode* (or something like that)? I think the idea was to see what needs > to be fixed before Python can become 100% Unicode. Last I recall it was > lots. The option is -U, and it causes string literals to be treated as Unicode objects. Currently, it gets so far as to start the interactive interpreter. Regards, Martin From martin at v.loewis.de Tue Aug 31 07:09:40 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 31 07:09:40 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <20040830223852.GA662@mems-exchange.org> References: <20040830201816.GA32565@mems-exchange.org> <413394C0.3090005@v.loewis.de> <20040830212317.GA419@mems-exchange.org> <41339D95.7050507@v.loewis.de> <20040830223852.GA662@mems-exchange.org> Message-ID: <41340814.4010508@v.loewis.de> Neil Schemenauer wrote: > Forgive me if I'm being obtuse, but I'm trying to understand the > overall Python unicode design. This works: > > >>> sys.getdefaultencoding() > 'utf-8' > >>> str(A()) > '\xe1\x88\xb4' Ah, ok, so you have changed sys.getdefaultencoding on your installation. Doing so means that some programs will only run on your installation, but not on others (e.g. mine). One shouldn't change the default encoding away from ASCII except to work around buggy applications which would fail because of their unicode-unawareness. > Can you be more specific about what is incorrect with the above > class? In the default installation, it gives a UnicodeEncodeError. >>No. In some cases, str() needs to compromise, where unicode() >>doesn't. > > > Sorry, I don't understand that statement. Are you saying that we > will eventually get rid of __str__ and only have __unicode__? No. Eventually, when strings are Unicode objects, the string conversion function will return such a thing. Whether this will be called __str__, __unicode__, or __string__, I don't know. However, this won't happen until Python 3, and it is not clear to me how it will look. We may also need a conversion routine into byte strings. > If only we could. :-) Seriously though, I'm trying to understand > the point of __unicode__. To me it seems to make the transition to > unicode string needlessly more complicated. Why do you say that? You don't *have* to implement __unicode__ if you don't need it - just like as you don't have to implement __len__ or __nonzero__: If your class is fine with the standard "non-None is false", implement neither. If your conceptually have a sequence type, implement __len__ for "empty is false". If you have a more different class, implement __nonzero__ for "I decide what false is". Likewise, if you are happy with the standard '', implement neither __str__ nor __unicode__. If your class has a canonical byte string representation, implement __str__. If this byte string representation is not meaningful ASCII, and if a more meaningful string representation using other Unicode characters would be possible, also implement __unicode__. Never rely on the default encoding being something other than ASCII, though. Eventually, when strings are Unicode objects, you won't be able to change it. Regards, Martin From martin at v.loewis.de Tue Aug 31 08:27:00 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 31 08:26:58 2004 Subject: [Python-Dev] Rename tcl84 directory Message-ID: <41341A34.5070606@v.loewis.de> In order to reduce the number of places that depend on the Tcl version, I have changed the INSTALLDIR directory of Tcl on the Windows build to tcltk. If you have a VC 7.1 build, please rename the tcl84 directory to tcltk. Unfortunately, I could not eliminate all version dependencies: the library is still tcl84.lib. Regards, Martin From mal at egenix.com Tue Aug 31 10:23:33 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 31 10:23:44 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <20040830201816.GA32565@mems-exchange.org> References: <20040830201816.GA32565@mems-exchange.org> Message-ID: <41343585.6070808@egenix.com> Neil Schemenauer wrote: > With Python 2.4: > > >>> u = u'\N{WHITE SMILING FACE}' > >>> class A: > ... def __str__(self): > ... return u > ... > >>> class B: > ... def __unicode__(self): > ... return u > ... > >>> u'%s' % A() > u'\u263a' > >>> u'%s' % B() > u'\u263a' > > With Python 2.3: > > >>> u'%s' % A() > Traceback (most recent call last): > File "", line 1, in ? > UnicodeEncodeError: 'ascii' codec can't encode character u'\u263a' in > position 0: ordinal not in range(128) > >>> u'%s' % B() > u'<__main__.B instance at 0x401f910c>' > > The only thing I found in the NEWS file that seemed relevant is > this note: > > u'%s' % obj will now try obj.__unicode__() first and fallback to > obj.__str__() if no __unicode__ method can be found. > > I don't think that describes the behavior difference. Allowing > __str__ return unicode strings seems like a pretty noteworthy > change (assuming that's what actually happened). __str__ is indeed allowed to return Unicode objects (and has been for quite a while). The reason we added __unicode__ was to provide a hook for PyObject_Unicode() to try before reverting to __str__. It is needed because even though returning Unicode objects from __str__ is allowed, in most cases PyObject_Str() gets to talk to it and this API always converts Unicode to a string using the default encoding which can easily fail. > Also, I'm a little unclear on the purpose of the __unicode__ method. > If you can return unicode from __str__ then why would I want to > provide a __unicode__ method? Perhaps it is meant for objects that > can either return a unicode or a string representation depending on > what the caller prefers. I have a hard time imagining a use for > that. That's indeed the use case. An object might want to return an approximate string representation in some form if ask for a string, but a true content representation when asked for Unicode. Because of the default encoding problems you might run into with __str__, we need two slots to provide this kind of functionality. In Py3k we will probably see __str__ and __unicode__ reunite. Now back to your original question: the change you see in %-formatting was actually a bug fix. Python 2.3 should have exposed the same behavior as 2.4 does now. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 31 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From amk at amk.ca Tue Aug 31 15:16:16 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 31 15:16:50 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> Message-ID: <20040831131616.GA15320@rogue.amk.ca> On Mon, Aug 30, 2004 at 12:57:37PM -0700, Guido van Rossum wrote: > (I still haven't started using the new email package, > so I'm reluctant to see my beloved rfc822.py disappearing in the > future; but I trust its fate is sealed, and I'm happy it gets an > extension. :-) I don't know if you followed the subsequent thread, but removing rfc822 is going to take quite a while. A 2.5 problem... Removing statcache.py breaks two modules in Lib/lib-old, cmpcache.py and dircmp.py. What should be done: rewrite them to use os.stat instead of statcache, or just delete them? The files in lib-old were last touched six weeks ago when Tim cleaned up the whitespace; before that they hadn't been touched since 2001 at the latest.. Wow, before Tim's whitespace n11n, lib-old/newdir.py was last touched in August 1994. 10 years ago! --amk From skip at pobox.com Tue Aug 31 15:29:38 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 31 15:30:00 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040831131616.GA15320@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> <20040831131616.GA15320@rogue.amk.ca> Message-ID: <16692.32066.6049.493875@montanaro.dyndns.org> amk> Removing statcache.py breaks two modules in Lib/lib-old, amk> cmpcache.py and dircmp.py. What should be done: rewrite them to amk> use os.stat instead of statcache, or just delete them? Why not migrate statcache.py to lib-old? Skip From amk at amk.ca Tue Aug 31 15:38:53 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 31 15:39:28 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <16692.32066.6049.493875@montanaro.dyndns.org> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> <20040831131616.GA15320@rogue.amk.ca> <16692.32066.6049.493875@montanaro.dyndns.org> Message-ID: <20040831133853.GB15320@rogue.amk.ca> On Tue, Aug 31, 2004 at 08:29:38AM -0500, Skip Montanaro wrote: > Why not migrate statcache.py to lib-old? I don't really see the point of lib-old these days; we now have a better mechanism for marking modules as outdated. If people prefer moving statcache to lib-old, fine; personally, I'd just delete all of lib-old, too. --amk From aahz at pythoncraft.com Tue Aug 31 15:41:04 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Aug 31 15:41:15 2004 Subject: [Python-Dev] PEP 309 updated slightly In-Reply-To: <4133B15B.702@blueyonder.co.uk> References: <20040830192447.4AB841E400A@bag.python.org> <4133B15B.702@blueyonder.co.uk> Message-ID: <20040831134104.GA25907@panix.com> On Mon, Aug 30, 2004, Peter Harris wrote: > > I have updated PEP309 a little, mostly to add the SF patch numbers for > the implementation (still waiting for someone to check it in), and to > note that the '@' character has been claimed for decorators, although > nobody really wanted it for PEP309 anyway. > > I hope there will still be time to squeeze it in before 2.4 beta 1, and > if there is any reason why not (apart from there has to be a cut-off > point somewhere, which I accept), please someone let me know what work > is still needed to make it ready to go in. What's PEP309? (People reading python-dev do not all have all PEP numbers memorized, and it's not reasonable to expect everyone to go look them up when it would take you only a few moments to save everyone else time.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy@lion.austin.ibm.com From skip at pobox.com Tue Aug 31 15:53:34 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Aug 31 15:54:03 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040831133853.GB15320@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> <20040831131616.GA15320@rogue.amk.ca> <16692.32066.6049.493875@montanaro.dyndns.org> <20040831133853.GB15320@rogue.amk.ca> Message-ID: <16692.33502.937551.7930@montanaro.dyndns.org> >> Why not migrate statcache.py to lib-old? amk> I don't really see the point of lib-old these days; I thought lib-old was just a convenience so people using defunct modules could resurrect them more easily. (I realize people can also grab them from the CVS attic.) Skip From amk at amk.ca Tue Aug 31 16:03:52 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 31 16:04:30 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS, 1.1125, 1.1126 In-Reply-To: References: Message-ID: <20040831140352.GC15320@rogue.amk.ca> On Tue, Aug 31, 2004 at 06:51:03AM -0700, akuchling@users.sourceforge.net wrote: > Add news item. > +- The mpz, rotor, and xreadlines modules, all deprecated in earlier > + versions of Python, have now been removed. > + Well, *that* was messier than I was expecting... Done now. I haven't touched the Makefiles for the PC and OS2 ports to remove these modules; if the maintainers want me to do that, please let me know. I did modify the RISC-OS port, because I don't know how responsive the maintainer is and thought it best to ensure the Makefile up to date. --amk From Paul.Moore at atosorigin.com Tue Aug 31 16:31:23 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Tue Aug 31 16:31:31 2004 Subject: [Python-Dev] PEP 309 updated slightly Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F72@UKDCX001.uk.int.atosorigin.com> From: Aahz >On Mon, Aug 30, 2004, Peter Harris wrote: >> >> I have updated PEP309 a little, [...] > >> I hope there will still be time to squeeze it in before 2.4 beta 1, and >> if there is any reason why not (apart from there has to be a cut-off >> point somewhere, which I accept), please someone let me know what work >> is still needed to make it ready to go in. > What's PEP309? PEP 309 is the "Partial Function Application" class. (Used to be referred to as "Currying", but that was agreed not to be an accurate name). Paul __________________________________________________________________________ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. __________________________________________________________________________ From alex.nanou at gmail.com Tue Aug 31 16:51:01 2004 From: alex.nanou at gmail.com (Alex Naanou) Date: Tue Aug 31 16:51:13 2004 Subject: [Python-Dev] PEP 309 updated slightly In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060F72@UKDCX001.uk.int.atosorigin.com> References: <16E1010E4581B049ABC51D4975CEDB8803060F72@UKDCX001.uk.int.atosorigin.com> Message-ID: <36f88922040831075133a98188@mail.gmail.com> though this might be a bit late, I would like to suggest something a bit functionally different: ---cut--- class LCurry(object): ''' this is the left curry class. ''' def __new__(cls, func, *args, **kw): obj = object.__new__(cls) if isinstance(func, LCurry) or isinstance(func, RCurry): obj._curry_func = func._curry_func obj._curry_args = (func._curry_args[0] + args, func._curry_args[1]) obj._curry_kw = kw = kw.copy() kw.update(func._curry_kw) else: obj._curry_func = func obj._curry_args = (args, ()) obj._curry_kw = kw.copy() return obj def __call__(self, *args, **kw): self._curry_func(*self._curry_args[0] + args + self._curry_args[1], **dict(self._curry_kw.items() + kw.items())) --uncut-- this mainly has one thing different from the reference implementation in the pep: 1) it is recursive that is we can curry/partial something more than one and yet avoid the extra function call per curry level... IMO this is a worthwhile optimisation.... taken from: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/222061 From barry at python.org Tue Aug 31 16:59:51 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 31 17:00:00 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <20040830192355.GA11836@rogue.amk.ca> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> Message-ID: <1093964391.27204.0.camel@geddy.wooz.org> On Mon, 2004-08-30 at 15:23, A.M. Kuchling wrote: > > * Leave rfc822, mimetools alone; the stdlib will have to avoid their use > > first, and I'm not going to do that right now. > > > > * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the > > stdlib uses them. +1 on these from me, as the only realistic option. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040831/ade12f57/attachment.pgp From anthony at interlink.com.au Tue Aug 31 17:23:52 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Aug 31 17:27:13 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <1093964391.27204.0.camel@geddy.wooz.org> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> <1093964391.27204.0.camel@geddy.wooz.org> Message-ID: <41349808.9050704@interlink.com.au> Barry Warsaw wrote: > On Mon, 2004-08-30 at 15:23, A.M. Kuchling wrote: > >>>* Leave rfc822, mimetools alone; the stdlib will have to avoid their use >>> first, and I'm not going to do that right now. >>> >>>* Make mimify, MimeWriter raise DeprecationWarnings; nothing in the >>> stdlib uses them. > > > +1 on these from me, as the only realistic option. I'd like to have a really serious effort to make sure we can DW the first group for 2.5. In particular, the mimetools.Message class is used for a whole pile of different protocols (anything with rfc2822 style Header: Value lines). To me, it doesn't make much sense that this lives in a module called 'mimetools'. But maybe that's just me. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From barry at python.org Tue Aug 31 18:42:00 2004 From: barry at python.org (Barry Warsaw) Date: Tue Aug 31 18:42:05 2004 Subject: [Python-Dev] Removing stuff from 2.4 In-Reply-To: <41349808.9050704@interlink.com.au> References: <20040807205703.GA3501@rogue.amk.ca> <1091914025.1064.48.camel@anthem.wooz.org> <4115DD5A.2070108@interlink.com.au> <20040809211517.GB11199@rogue.amk.ca> <20040830192355.GA11836@rogue.amk.ca> <1093964391.27204.0.camel@geddy.wooz.org> <41349808.9050704@interlink.com.au> Message-ID: <1093970520.27211.9.camel@geddy.wooz.org> On Tue, 2004-08-31 at 11:23, Anthony Baxter wrote: > I'd like to have a really serious effort to make sure we can DW the > first group for 2.5. In particular, the mimetools.Message class is > used for a whole pile of different protocols (anything with rfc2822 > style Header: Value lines). To me, it doesn't make much sense that > this lives in a module called 'mimetools'. > > But maybe that's just me. Naw, I'm with you. I'd hoped to get farther along for all this stuff for 2.4, but I didn't expect to get so distracted <292 winks>. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20040831/35f9c9a0/attachment-0001.pgp From nas at arctrix.com Tue Aug 31 20:41:16 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Tue Aug 31 20:41:19 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <41343585.6070808@egenix.com> References: <20040830201816.GA32565@mems-exchange.org> <41343585.6070808@egenix.com> Message-ID: <20040831184116.GA3994@mems-exchange.org> On Tue, Aug 31, 2004 at 10:23:33AM +0200, M.-A. Lemburg wrote: > __str__ is indeed allowed to return Unicode objects > (and has been for quite a while). [...] > Now back to your original question: the change you see > in %-formatting was actually a bug fix. Python 2.3 should > have exposed the same behavior as 2.4 does now. I think the note in NEWS is not quite accurate. It says: Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). The change in PyUnicode_Join replaces a PyObject_Str() call with PyObject_Unicode(). That means that this works in 2.4: class A: def __str__(self): return u'\u1234' u'%s' % A() Perhaps that is unintentional but it seems cleaner to me than adding a __unicode__ method and not providing a __str__ method. Neil From mal at egenix.com Tue Aug 31 20:58:31 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 31 20:58:35 2004 Subject: [Python-Dev] unicode and __str__ In-Reply-To: <20040831184116.GA3994@mems-exchange.org> References: <20040830201816.GA32565@mems-exchange.org> <41343585.6070808@egenix.com> <20040831184116.GA3994@mems-exchange.org> Message-ID: <4134CA57.8050100@egenix.com> Neil Schemenauer wrote: > On Tue, Aug 31, 2004 at 10:23:33AM +0200, M.-A. Lemburg wrote: > >>__str__ is indeed allowed to return Unicode objects >>(and has been for quite a while). > > [...] > >>Now back to your original question: the change you see >>in %-formatting was actually a bug fix. Python 2.3 should >>have exposed the same behavior as 2.4 does now. > > > I think the note in NEWS is not quite accurate. It says: > > Let u'%s' % obj try obj.__unicode__() first and fallback to > obj.__str__(). > > The change in PyUnicode_Join replaces a PyObject_Str() call with I think you meant PyUnicode_Format(). > PyObject_Unicode(). That means that this works in 2.4: > > class A: > def __str__(self): > return u'\u1234' > > u'%s' % A() > > Perhaps that is unintentional but it seems cleaner to me than adding > a __unicode__ method and not providing a __str__ method. This is intended. However, you'll find that calling str(A()) will not give you a Unicode object, but an exception instead. The reason is that even though the __str__ method may pass back a Unicode object, PyObject_Str() still assures that its return value is a string (and that API is used by str()). Basically, __unicode__ is to unicode() what __str__ is to str(). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 31 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From raymond.hettinger at verizon.net Tue Aug 31 21:25:40 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue Aug 31 21:27:12 2004 Subject: [Python-Dev] PEP 309 updated slightly In-Reply-To: <4133B15B.702@blueyonder.co.uk> Message-ID: <000401c48f90$6c7fe480$e841fea9@oemcomputer> > I hope there will still be time to squeeze it in before 2.4 beta 1 Is it the case that partial() cannot perform as well as an equivalent lambda expression? Raymond From scav at blueyonder.co.uk Tue Aug 31 21:38:46 2004 From: scav at blueyonder.co.uk (Peter Harris) Date: Tue Aug 31 21:38:51 2004 Subject: [Python-Dev] Re: What is PEP309? In-Reply-To: <20040831164226.96D151E4008@bag.python.org> References: <20040831164226.96D151E4008@bag.python.org> Message-ID: <4134D3C6.8090001@blueyonder.co.uk> Aahz wrote: >What's PEP309? > >(People reading python-dev do not all have all PEP numbers memorized, >and it's not reasonable to expect everyone to go look them up when it >would take you only a few moments to save everyone else time.) > > OK, sorry about that. PEP309 is for a class that emulates partial function application (a bit like currying in languages that do it that way, but not exactly). It lets you make callable wrappers round a callable object, with some arguments 'frozen in'. #e.g. classes and functions can both be used: win = Tkinter.Tk() # my window blueButn = partial(Tkinter.Button, win, fg='blue') # factory for blue buttons in my window def callback(name): print name,"pressed" b1 = blueButn(text="Hello", command=partial(callback,"hello")) b2 = blueButn(text="Goodbye", command=partial(callback,"goodbye")) The PEP proposes a module 'functional' for stuff (like partial) that operates on or returns other functions. I expect there are a few things that might belong there, but only 'partial' is in the proposal. Peter Harris From scav at blueyonder.co.uk Tue Aug 31 22:24:29 2004 From: scav at blueyonder.co.uk (Peter Harris) Date: Tue Aug 31 22:24:33 2004 Subject: [Python-Dev] Right curry considered harmful In-Reply-To: <20040831164226.96D151E4008@bag.python.org> References: <20040831164226.96D151E4008@bag.python.org> Message-ID: <4134DE7D.9020204@blueyonder.co.uk> Alex Nanouu wrote: >though this might be a bit late, I would like to suggest something a >bit functionally different: >---cut--- >class LCurry(object): > ''' > this is the left curry class. > ''' > def __new__(cls, func, *args, **kw): > obj = object.__new__(cls) > if isinstance(func, LCurry) or isinstance(func, RCurry): > obj._curry_func = func._curry_func > obj._curry_args = (func._curry_args[0] + args, func._curry_args[1]) > obj._curry_kw = kw = kw.copy() > kw.update(func._curry_kw) > else: > obj._curry_func = func > obj._curry_args = (args, ()) > obj._curry_kw = kw.copy() > return obj > def __call__(self, *args, **kw): > self._curry_func(*self._curry_args[0] + args + >self._curry_args[1], **dict(self._curry_kw.items() + kw.items())) > >--uncut-- > >this mainly has one thing different from the reference implementation >in the pep: >1) it is recursive >that is we can curry/partial something more than one and yet avoid the >extra function call per curry level... > >IMO this is a worthwhile optimisation.... > I think we'll see if partial() is a useful enough feature to be worth optimising once it actually makes it into a build and gets used. Mostly I think nesting many layers of partial(), or optimising the implementation with that in mind will not be a win for clarity, therefore a net loss however fast you can make it. By the way, I think 'right curry' is just a broken idea from the point of view of code readability. (I know the n'th argument is '2', but 'n' will depend on how many arguments are passed at run time!) The PEP doesn't propose to implement such a monster, or call partial() a curry at all, let alone a 'left' one. Peter Harris From alex.nanou at gmail.com Tue Aug 31 22:59:10 2004 From: alex.nanou at gmail.com (Alex Naanou) Date: Tue Aug 31 22:59:14 2004 Subject: [Python-Dev] Right curry considered harmful In-Reply-To: <4134DE7D.9020204@blueyonder.co.uk> References: <20040831164226.96D151E4008@bag.python.org> <4134DE7D.9020204@blueyonder.co.uk> Message-ID: <36f88922040831135974892728@mail.gmail.com> On Tue, 31 Aug 2004 21:24:29 +0100, Peter Harris wrote: > I think we'll see if partial() is a useful enough feature to be worth optimising once it > actually makes it into a build and gets used. I for one use the beast quite allot! :) > Mostly I think nesting many layers of partial(), or optimising the implementation with > that in mind will not be a win for clarity, therefore a net loss however fast you can > make it. well, about clarity, the implementation I posted above is actual code form the library I am writing (pli.sf.net), I could write a clearer and *prettier* version with the same functionality if desired... ...why intentionally make an under-designed version when adding some minor optimisations would not impact either clarity nor efficiency (IMHO). > By the way, I think 'right curry' is just a broken idea from the point of view of > code readability. (I know the n'th argument is '2', but 'n' will depend on how > many arguments are passed at run time!) The PEP doesn't propose to implement > such a monster, or call partial() a curry at all, let alone a 'left' one. in part I do agree, but I had to implement both as going the other way would have been a big (and I do mean BIG) loss in both clarity an flexibility. --- Alex. From ncoghlan at iinet.net.au Tue Aug 31 23:00:43 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Aug 31 23:02:10 2004 Subject: [Python-Dev] Auto-str and auto-unicode in join In-Reply-To: <4131BC2A.9060206@egenix.com> References: <002401c48bb5$765463c0$e841fea9@oemcomputer> <1f7befae04082615206760c806@mail.gmail.com> <412E77EF.80007@iinet.net.au> <1f7befae040828185146444b6d@mail.gmail.com> <4131BC2A.9060206@egenix.com> Message-ID: <4134E6FB.2070104@iinet.net.au> M.-A. Lemburg wrote: > Tim Peters wrote: > >> If we were to do auto-str, it would be better to rewrite str.join() as >> a 1-pass algorithm, using the kind of "double allocated space as >> needed" gimmick unicode.join uses. It would be less efficient if >> auto-promotion to Unicode turns out to be required, but it's hard to >> measure how little I care about that; it might be faster if auto-str >> and Unicode promotion aren't needed (as only 1 pass would be needed). > > > FWIW, I'm -1 on doing auto-conversion using str()/unicode() in > .join(). I'm leaning that way myself. The 'auto-str' behaviour gives significant speed increases over the ''.join([str(x) for x in seq]) idiom, but the semantics become tricky when unicode, strings and things which are neither get mixed in the sequence. It also increases the complexity of the C code for the join methods. (Note that the version of auto-str I tried out *doesn't* actually mimic the list comprehension idiom - it tries to detect unicode and switch join implementations as appropriate, etc, etc.) Cheers, Nick. From nickm at alum.mit.edu Mon Aug 30 18:50:27 2004 From: nickm at alum.mit.edu (Nick Mathewson) Date: Wed Sep 1 12:23:40 2004 Subject: [Python-Dev] os.urandom API In-Reply-To: <000701c48e07$f9cea040$e841fea9@oemcomputer> Message-ID: <20040830165027.GE29743@totoro.wangafu.net> On Sun Aug 29 22:37:57 2004, Raymond Hettinger wrote: > I would like to change the API for the new os.urandom(n) function to > return a long integer instead of a string. The former better serves > more use cases and fits better with existing modules. With all respect, I disagree. As a potential user, and as lead developer of a particularly crypto-heavy Python app (http://mixminion.net/), I'd have to say an interface that returned a long integer would not serve my purposes very well at all. For most crypto apps, you never use the output of your strong entropy source directly. Instead, you use your strong entropy source to generate seeds for (cryptographically strong) PRNGs, to generate keys for your block ciphers, and so on. Nearly all of these modules expect their keys as a sequence of bits, which in Python corresponds more closely to a character-string than to an arbitrary long. > In favor of a long integer: > > 1) The call random.seed(os.random(100)) is a likely use case. If the > intermediate value is a string, then random.seed() will hash it and > only use 32 bits. If the intermediate value is a long integer, all > bits are used. In the given example, the latter is clearly what the > user expects (otherwise, they would only request 4 bytes). Plausible, but as others indicate, it isn't hard to write: random.seed(long(hexlify(os.urandom(100)), 16) And if you think it's really important, you could change random.seed(None) to get the default seed in this way, instead of looking at time.time. But this isn't the primary use case of cryptographically strong entropy: The Mersenne Twister algorithm isn't cryptographically secure. If the developer wants a cryptographically strong PRNG, she shouldn't be using random.random(). If she doesn't want a cryptographically strong PRNG, it's overkill for her to use os.urandom(), and overkill for her to want more than 32 bits of entropy anyway. > 2) Another likely use case is accessing all the tools in the random > module with a subclass that overrides random() and getrandbits(). > Both can be done easier and faster if os.random() returns long > integers. If the starting point is a string, the code gets ugly and > slow. As above, this isn't the way people use strong entropy in well-designed crypto applications. You use your strong entropy to seed a strong PRNG, and you plug your strong PRNG into a subclass overriding random() and getrandbits(). I agree that somebody might decide to just use os.urandom directly as a shortcut, but such a person isn't likely to care about whether her code is slow -- a good PRNG should outperform calls to os.urandom by an order of magnitude or two. [....] > > In favor of a string of bytes: > > 1) This form is handy for cyptoweenies to xor with other byte strings > (perhaps for a one-time pad). 2) By returning the result of the OS's random function directly, we make it easier for cryptoweenies to assure themselves that their entropy is good. If it gets massaged into a long, then skeptical cryptoweenies will have that much more code to audit. 3) Most crypto libraries don't currently support keying from longs, and (as noted in other posts) the long->string conversion isn't as easy or clean as the string->long conversion. yrs, -- Nick Mathewson (PGP key changed on 15Aug2004; see http://wangafu.net/key.txt)